AI EncyclopediaCore concepts
AI Hallucination
When a language model generates false or fabricated information - a made-up citation, a wrong fact, a nonexistent feature - stated with the same fluent confidence as a correct answer.
In plain English
The model isn't lying on purpose and it isn't aware it's wrong. It is doing exactly what it was built to do - predict plausible next words - and sometimes the most statistically plausible continuation is simply untrue.
Technical explanation
LLMs do not query a verified fact store; they predict tokens from patterns learned in training. Sparse or inconsistent training data on a topic leads the model to 'fill gaps' with something plausible but false. OpenAI's own 2025 research on the subject additionally attributes hallucination to standard training and evaluation incentives that reward confident guessing over an honest 'I don't know' - a model scored only on accuracy has no incentive to abstain.
Why it matters
Hallucination is the central reliability problem in deploying LLMs for anything factual - legal, medical, financial or regulatory use in particular - and is the primary reason RAG, citations and human review exist as standard practice around production AI systems.
Real-world example
Google's Bard chatbot incorrectly claimed the James Webb Space Telescope had captured the first images of a planet outside our solar system - a widely cited public example of a fluent, confident, and factually wrong AI output.
Common misunderstanding
That hallucination is a bug that will simply be patched out as models improve. Research from OpenAI itself frames it as partly structural - an artefact of how models are trained and evaluated - which is why mitigation (grounding via RAG, citations, calibrated uncertainty) rather than elimination is the realistic current goal.
Something wrong here?
Every entry is hand-researched and hand-written by Nakoda. If a fact is stale, a source has changed or a definition needs sharpening, tell us and we will check it.