In 2023, a New York lawyer submitted a court filing citing six past cases. Judges checked. None of the cases existed. ChatGPT had invented them — complete with realistic case numbers, judges' names and convincing quotes — and the lawyer had trusted it. He faced sanctions; the story went global. That failure mode has a name, it's the single most important limitation you must understand before building anything, and interviewers at every company ask about it: hallucination.
What hallucination is (and what makes it dangerous)
A hallucination is output that is fluent, confident and wrong — invented facts, fake citations, non-existent API functions, wrong dates delivered in perfect prose. The danger is not the wrongness; humans are wrong all the time. The danger is that a hallucinated answer looks exactly like a correct answer — same tone, same fluency, same confidence. There's no stammer, no "I think maybe...". Your eyes cannot tell the difference; only verification can.
Why it happens — it's not a bug
Back to the master key from lesson 2: the model generates the most plausible continuation, computed from patterns in its weights. There is no fact database inside, no lookup step, no truth-checker. Usually, plausible and true coincide — that's why the model is useful at all. But ask for something the patterns don't reliably cover — a rare case citation, a niche statistic, anything after its training cutoff — and the machinery does the only thing it can: produce the most plausible-looking answer anyway. A realistic-sounding case number is more plausible as text than "I don't know", unless the model was specifically trained to admit uncertainty.
So the honest framing — and the sentence that wins interviews — is: hallucination is not a malfunction; it's the flip side of how generation works. Plausibility is the objective; truth is a frequent side effect.
When the risk is highest — the grounding ladder
Not all tasks are equally risky. The pattern: the further the answer lives from text you supplied, the higher the risk.
| Task | Answer comes from | Risk |
|---|---|---|
| "Summarize this paragraph" | text in the prompt | low |
| "Rewrite this email politely" | text in the prompt | low |
| "Explain how HTTP works" | heavily repeated training patterns | moderate |
| "Cite the exact 2019 judgment on X" | rare, specific memory | high |
| "What happened last week?" | after training cutoff — nothing | extreme |
The most-hallucinated details are specific identifiers — names, numbers, dates, citations, URLs, function signatures — precisely because specific details are what make text look authoritative. A wrong vibe is rare; a wrong case number is common.
How to reduce it — the production toolkit
You cannot eliminate hallucination. You can engineer it down to acceptable levels. The toolkit, in the order teams reach for it:
1 · Ground the model. Put the true facts in the prompt and instruct "answer only from this context" — the most plausible continuation of true text is usually the truth. Doing this at scale with retrieval is RAG, lesson 9, and it's the #1 hallucination defence in industry. 2 · Give an escape hatch. "If the context doesn't contain the answer, say so" — permission to admit ignorance measurably reduces inventions. 3 · Lower the temperature for factual tasks — fewer adventurous tokens (but see the myth below). 4 · Demand citations the user (or your code) can check — uncheckable claims are where inventions hide. 5 · Validate programmatically — if the output contains an order id, look it up; if it names a function, check it exists. 6 · Human in the loop for high-stakes outputs: legal, medical, financial, anything customer-promising.
Selection-round radar: "What is hallucination and how do you prevent it?" — asked at Infosys, Cognizant, Accenture, Google, everywhere. Structure your answer: definition (confident + wrong) → root cause (plausibility, not truth) → say "reduce, not prevent" → then RAG, escape hatch, low temperature, citations, validation. The word "grounding" belongs in your answer.
Wait — I set temperature to 0. Fixed, right?
The most common wrong belief in this topic. Temperature controls randomness, not truth (lesson 5). At temperature 0 the model reliably picks its top-probability guess — and if that guess is a fabricated case citation, you now get the same fabrication every single time. Consistency is not correctness; a deterministic liar is still a liar. Temperature helps at the margins (it stops rare bad tokens from starting a bad path), but grounding is the real medicine.
The rest of the limitation family
Hallucination headlines the list, but interviews expect the full picture. Knowledge cutoff: training data ends at a date; later events simply don't exist for the model — fresh facts need RAG or search tools. Arithmetic: the model predicts digit-tokens, it doesn't calculate — for reliable math, give it a calculator/code tool (lesson 11). Character-level blindness: the strawberry problem from lesson 2 — tokens hide letters. Bias: trained on human text, it absorbs human skews — full treatment in lesson 12. Context limits: outside the window, nothing exists (lesson 5). One framing ties them together: the model is a brilliant pattern-continuation engine — every limitation is a place where pattern-continuation and the actual requirement part ways.
Common mistakes
- Shipping LLM output containing names, numbers, citations or links without verification — the most-invented details.
- Believing temperature 0 or a stern "be accurate!" instruction fixes hallucination.
- Forcing an answer — no escape hatch means the model fills silence with fiction.
- Saying "prevent hallucinations" in interviews — the credible verb is reduce.
- Treating fluency as evidence — confidence is a writing style, not a truth signal.
Quick recap
| Concept | One-liner |
|---|---|
| Hallucination | fluent, confident, wrong — indistinguishable by reading |
| Root cause | the objective is plausibility; truth is a side effect |
| Highest risk | specific identifiers from rare memory; anything post-cutoff |
| Best defence | grounding (RAG) + escape hatch + citations + validation |
| Temperature 0 | repeatable ≠ correct — a deterministic liar is still a liar |
| Honest verb | reduce, never prevent |
Practice Zone — PYQs from real selection rounds
Six MCQs and three tasks — including hunting fabrications in a legal-bot answer, the way a reviewer would.
An LLM hallucination is:
Asked in
The root cause of hallucination is that an LLM:
Asked in
Which set of techniques genuinely reduces hallucinations in production?
Asked in
A model's knowledge cutoff means:
Asked in
Hallucination risk is HIGHEST in which of these requests?
Asked in
Setting temperature to 0 — does it stop hallucinations?
Asked in
Hands-on tasks:
A legal-research bot answered: "As held in *Sharma v. Union of India (2016) 4 SCC 225*, Section 66A of the IT Act was struck down. The judgment, delivered by a five-judge bench on 12 March 2016, cited the earlier *Desai Committee Report of 2011*." You know one fact: Section 66A was actually struck down. List everything in this answer you should treat as potentially hallucinated, and say how you'd verify.
Asked in
You're shipping an FAQ bot for a mutual-fund company. A wrong answer about lock-in periods or tax could cause real harm. Design a 4-point hallucination-defence plan for this bot.
Asked in
In one short paragraph with one concrete example, explain what grounding means for LLMs — as you would to an interviewer.
Asked in
FAQ
Will bigger, newer models stop hallucinating?
Rates keep dropping with better models and training — but the mechanism (plausibility-driven generation) remains, so careful systems still ground and verify. Design for "less often", never for "never".
Why doesn't the model just say 'I don't know'?
Base training rewards continuing text, and confident answers are the dominant pattern in training data. Alignment training and explicit prompt permission ("say so if unsure") both raise the odds of honest uncertainty — which is why the escape hatch works.
Is hallucination ever useful?
Reframed as imagination, yes — brainstorming, fiction and design ideation deliberately want plausible inventions. The problem is never generation of new things; it's presenting inventions as facts.
Next lesson: from chatting to building — Lesson 8: LLM APIs & Function Calling →