A hallucination occurs when a generative AI model produces a statement that sounds plausible and is delivered with confidence, yet is factually wrong or invented. The fitting analogy: a bright student who, rather than admitting ignorance during an exam, improvises a perfectly worded — and entirely fictional — answer.
Why does it happen?
A large language model (LLM) does not "know" facts: it predicts the most probable next word, token after token. Its training objective is not truth but the statistical likelihood of the text. Formally, the model maximizes:
$$P(w_t \mid w_1, w_2, \dots, w_{t-1})$$
Nothing in this objective guarantees accuracy: a false sentence can be more "fluent" than a true one. The model therefore fills gaps in its memory with plausible interpolations.
Why it is a risk
- Misleading confidence: a wrong answer is delivered just as assertively as a correct one.
- Critical domains: health, law, finance — a fabricated legal citation or scientific reference can have serious consequences.
- Hard to detect: without a source, the user cannot tell the genuine from the fabricated.
| Type | Example |
|---|---|
| Invented fact | A scientific paper that does not exist |
| Wrong attribution | A quote credited to the wrong author |
| Fabricated detail | A precise but false date or figure |
How to reduce the risk
The most effective approaches ground the model in verifiable sources: RAG (retrieval-augmented generation), automated fact-checking, and the systematic citation of sources.
An eloquent AI is not a reliable AI: always demand its sources.