Understanding Large Language Models — how they actually work, what they can and cannot do, their failure modes, and when to use Claude, ChatGPT, or Gemini.
01 · Foundation
The AI family — nested like Russian dolls
These terms get used interchangeably. They shouldn't. Each one lives inside the one above it — like Matryoshka dolls.
🌍 Level 1
Artificial Intelligence (AI)
Any computer system doing something we'd call "intelligent" — chess, face recognition, translation.
📚 Level 2
Machine Learning (ML)
AI that learns from examples instead of hard-coded rules. Show it 10,000 spam emails; it learns the pattern itself.
🧠 Level 3
Deep Learning
ML using stacked layers of calculations. Layer 1 sees edges. Layer 5 sees "cat." Depth = abstraction.
💬 Level 4
Large Language Models (LLMs)
Deep learning trained on massive text. Powers Claude, ChatGPT, Gemini. Predicts likely next words.
🚗 The transport analogy
AI"vehicles"
ML"motor vehicles"
Deep Learning"cars"
LLMs"electric cars"
Every electric car is a car — but not every vehicle is electric. Same logic.
Key insight: nobody programmed grammar or facts into LLMs. They emerged from billions of corrections during training — just like a child learning to speak by hearing corrections over and over. GPT-4 alone has ~1.8 trillion parameters and cost hundreds of millions of dollars to train. Size ≈ capability.
02 · Core mechanism
How LLMs actually work
One simple trick repeated billions of times: predict the next word. Done at scale, it produces something that looks like intelligence.
🎹 Training — the piano practice analogy
1
Feed it text
Trillions of words — books, websites, code, articles. Everything readable.
2
Guess the next word
"The cat sat on the ___" — the model has to predict what comes next.
3
Get corrected
Real answer was "mat." Adjust all internal parameters slightly toward that answer.
4
Repeat billions of times
Grammar, facts, and reasoning emerge on their own — without ever being explicitly programmed.
🧱 Tokens — the Lego bricks of language
LLMs don't read word by word. They read tokens — chunks of ~4 characters. "unbelievable" becomes:
unbelievable
A short sentence like "The cat sat on the mat" splits like this:
Thecatsatonthemat
📋 The context window — the whiteboard analogy
What it is
An LLM can only "see" a fixed amount of text at once (e.g., 4K to 200K tokens). Anything outside is invisible — as if it was never said.
🖼️ Like a whiteboard: a smart consultant who can only see what fits on one whiteboard. Keep adding, and older notes get erased.
Why it matters
When your conversation gets too long, the earliest instructions or facts fall out of view. The model "forgets" — not because it's broken, but because those tokens no longer exist in its working memory.
⚠️ Long conversations = risk of the model losing your original instructions.
🎻 Parameters — the harp strings analogy
Billions of tiny numbers inside the model. Each gets nudged slightly when the model is wrong during training. After training they're frozen — together, they encode everything the model "knows."
The technical stack: LLMs use the Transformer architecture — attention mechanisms that let the model understand relationships between words across long distances. At inference, they generate text by sampling from a probability distribution over possible next tokens. Modern models are further refined with RLHF (Reinforcement Learning from Human Feedback) to align outputs with human preferences.
03 · Capabilities
What LLMs can & can't do
LLMs are extraordinarily capable at language tasks — and surprisingly brittle at things that feel simple.
✅ What they CAN do
Generate fluent, coherent text in many styles
Summarize, translate, and paraphrase
Answer questions based on training data
Write and explain code
Draft emails, reports, and content
Brainstorm ideas & assist creative tasks
Follow complex multi-step instructions
Classify, extract, and structure information
Explain complex topics simply
Engage in multi-turn conversation
❌ What they CANNOT do
Access real-time or live information (no tools)
Guarantee factual accuracy
Truly "reason" or perform reliable logic
Remember past chats (no memory tools)
Perform precise arithmetic reliably
Know what they don't know (metacognition)
Replace domain experts for high-stakes calls
Act in the world without tools
See past their context window
🧮 Why math is hard: asking an LLM to do arithmetic is like asking a literature professor to mentally calculate your taxes. They might get it right — but they're pattern-matching words about numbers, not actually computing. A calculator is always better.
📚 RAG fixes the knowledge gap: RAG gives the model a librarian who fetches relevant documents before answering — turning a closed-book exam into an open-book one. This is how AI assistants access your company's internal documents. (More on RAG in chapter 5.)
04 · Failure modes
How and why LLMs fail
These aren't bugs waiting to be fixed. They're fundamental properties of how these models work — understanding them makes you a better AI user.
🎭 Hallucination
The model generates a fluent, confident answer that is simply false. It doesn't know it's wrong — it's completing a pattern, not checking facts.
"The party guest who never says 'I don't know' — sounds credible, but the facts are invented."
📋 Context limit
Once the conversation exceeds the context window, earlier instructions or facts become invisible — as if they were never said.
"The whiteboard consultant: when it fills up, older notes get erased to make room."
🙋 Sycophancy — the yes-man problem
Models learn to tell you what you want to hear, because agreeable answers scored higher in human training ratings. Not lies — but unhelpful agreement.
"The overeager intern: you suggest a bad idea, they say 'great idea!' — agreement gets rewarded."
📅 Knowledge cutoff
Training has an end date. The model knows nothing after that date unless given tools like web search — like someone who's been off-grid for two years.
"Asking about current events returns confident but outdated answers."
📐 Reasoning errors
LLMs can fail at multi-step logical reasoning, math, and spatial tasks. They may produce plausible-looking but logically flawed chains of thought.
"A confident chain of reasoning that arrives at the wrong conclusion — hard to spot without checking each step."
🎯 Prompt sensitivity
Small changes in phrasing can produce dramatically different outputs. The model's behavior is highly sensitive to how questions are framed.
"Two questions that feel identical to you can produce two completely different answers."
🔀 Inconsistency
The same question asked multiple times may yield different answers. LLMs are non-deterministic by default — a factor in critical applications.
"Ask twice, get two answers. Neither is necessarily wrong — they're both plausible completions."
🧩 Instruction gaps
Complex or long prompts with many constraints may result in the model partially ignoring some instructions — especially those buried in the middle.
"Give it ten rules and it might quietly drop the third and seventh."
🎯 How to reduce these risks
Against hallucination
Ask the model to cite sources. Verify specific facts. Use RAG for factual tasks. Never trust confident tone alone.
Against sycophancy
Ask "what's wrong with this idea?" Push back on answers. Request the counterargument explicitly.
Against context limits
Summarise long context periodically. Start fresh conversations for new topics. Repeat key instructions.
🛡️ Best practice: always verify LLM outputs for factual claims, especially in high-stakes contexts. Treat LLMs as a powerful first-draft tool, not a source of truth.
05 · Extensions
RAG & MCP — giving AI eyes and hands
By default, an LLM is isolated — no internet, no files, no memory. RAG and MCP are the two main ways to fix that.
📚 RAG — Retrieval-Augmented Generation
RAG lets an LLM search a knowledge base before answering. Instead of relying only on training, it fetches the most relevant documents and reads them first — then answers based on what it found.
1
You ask: "What's our refund policy?"
2
System searches your documents for relevant pages
3
Those pages are handed to the LLM along with your question
4
LLM reads and answers — now grounded in your actual documents
📖 The open-book exam analogy: without RAG = closed-book exam — answer from memory only. With RAG = open-book — a librarian sprints to get the right pages before you answer.
🔌 MCP — Model Context Protocol
MCP is an open standard (created by Anthropic) that lets an AI connect to external tools — calendars, databases, browsers, files — in a consistent, plug-and-play way. Think of it as USB-C for AI: one universal connector that works with everything.
❌ Without MCP
AI ←custom→ Slack
AI ←custom→ Gmail
AI ←custom→ Database
AI ←custom→ Calendar
Every tool needs its own wiring.
✅ With MCP
Slack → MCP → AI
Gmail → MCP → AI
Database → MCP → AI
Calendar → MCP → AI
Any tool. Same plug. One standard.
What MCP lets AI do
📁 Read & write files
🌐 Search the web
📅 Check calendar
⚙️ Run code
🗄️ Query databases
💬 Post to Slack
RAG vs MCP in one line: RAG gives AI a library to read from. MCP gives AI hands to act with. RAG is about fetching knowledge — MCP is about doing things.
06 · Platform comparison
Claude vs ChatGPT vs Gemini — busting myths
Common belief: ChatGPT = empathy & personality, Claude = cold data analysis. Reality? It's the opposite — and understanding why matters.
⚠️ Common misconception: "ChatGPT feels warmer and more personal, so it must be better at empathy." — Warmth of tone is not the same as depth of understanding. ChatGPT's agreeable, casual tone can feel empathetic — but it's closely linked to sycophancy (telling you what you want to hear). Claude was explicitly designed to understand human context carefully, which is the real foundation of empathy.
Claude
by Anthropic · Honest & context-aware
Actually best for
Genuine empathy & sensitive writing
Long documents & deep analysis (200k context)
Honest feedback — won't just agree with you
Ethical, nuanced, multi-perspective topics
Coding with careful explanations
ChatGPT
by OpenAI · Versatile ecosystem
Actually best for
Broadest plugin & tool ecosystem
Image generation (DALL-E built-in)
Voice mode conversations
Fast first drafts & general tasks
Custom GPTs & workflow automation
Gemini
by Google · Real-time & multimodal
Actually best for
Current news & live web info
Google Workspace integration
Multimodal — text, image, video
Research with live citations
When data freshness is critical
Side-by-side comparison
Dimension
Claude
ChatGPT
Gemini
Empathy & nuance
✅ Genuinely strong — by design
⚠️ Warm tone, but sycophantic
➖ More clinical & factual
Data & long-doc analysis
✅ 200k context — advantage here
✅ Strong, 128k context
✅ Strong + live data access
Honest pushback
✅ Will challenge bad ideas
⚠️ Tends to agree to please
✅ Fairly balanced
Tool ecosystem
Growing (MCP-based)
✅ Largest — plugins, DALL-E, voice
✅ Native Google suite
Real example — same prompt, different results
Prompt A — needs real empathy
"I need to write a sensitive email to a colleague who made a mistake in front of the team. Help me be honest but kind."
✅ Claude
Nimbly weighs emotional stakes. Offers multiple tones with reasoning. Flags what might sting. Asks clarifying questions.
⚠️ ChatGPT
Polished, friendly-sounding draft. Warm — but may not probe deeper. Validates your framing without questioning it.
➖ Gemini
Solid, structured draft. More clinical — may reference communication frameworks. Less attuned to emotional subtext.
Prompt B — research + current facts
"What are the latest AI regulations in the EU and how do they affect businesses?"
✅ Claude
Deep, nuanced analysis up to training cutoff. 200k context digests full legal docs. Needs web search for truly live updates.
✅ ChatGPT
Strong analysis with browsing enabled. Plugin ecosystem useful. Can be overconfident on specifics — worth double-checking.
🏆 Gemini
Strongest here — native Google Search delivers real-time results with source links. Gemini's home turf.
The corrected rule of thumb: use Claude when you need genuine depth — empathy, nuance, or careful analysis. Use ChatGPT when you need speed and the widest tool ecosystem. Use Gemini when you need what happened this week. And remember: how well you prompt matters more than which platform you pick.
07 · Practice
Knowledge check
8 questions covering everything in this session. Pick an answer — you'll get instant feedback and an explanation.
0 / 8 answered
Q1. Which is the correct nesting order from largest to smallest category?
Q2. What is an LLM's fundamental task at inference time?
Q3. A model gives a confident, fluent answer that turns out to be completely made up. What's this called?
Q4. You suggest a bad idea to the model. It responds "Great idea!" without pushback. This is an example of:
Q5. When would you use RAG (Retrieval-Augmented Generation)?
Q6. What's the biggest misconception about ChatGPT vs Claude for empathetic tasks?
Q7. You need real-time news about a regulation passed this week. Best choice?
Q8. MCP is best described as:
🎯 Key takeaways
The essentials to remember: LLMs are probabilistic text predictors, not reasoning engines or knowledge databases. Hallucination is inherent — not a bug that will simply be "fixed." Context windows are the model's working memory — anything outside is invisible. Prompt engineering matters as much as platform choice. And LLMs are tools that augment human capability, not replace human judgment.