Hallucination

What Hallucination means

Hallucination is a language model producing text that reads as fact but was never checked against any fact: the citation, the date, the quote are the most plausible next words, and plausible is all the model was ever asked for. Owning this changes our habits with the tools. We stop asking "is it reliable?" as if it were a colleague with good and bad days, and start asking "would this answer be easy to fake?" because the easier a sentence is to make up, the more fluently it will be made up. We start checking the specifics, names, numbers, references, and trusting the general shape. Before: it usually knows, so we trust it. After: it always sounds like it knows, so we verify anything we would have to defend.

Not to be confused with

AI hallucination is not a hallucination in the medical sense. A person who hallucinates perceives something that is not there. A model that hallucinates has not perceived anything; it has produced the most plausible words, and plausible was all it was ever asked for.

Hallucination examples: where it shows up

• At work: the report a tool drafted contained a figure nobody in the company had ever recorded, and it passed two readers because it looked like all the other figures.
• With money: a chatbot's confident answer about a tax rule or a fee is exactly as confident when the rule changed last year.
• In the news: lawyers filing cases that cited judgments which never existed were not careless readers; they trusted a voice that never wavers.

Where it sits

Concept 2 of 5 in Working with AI, a journey in the course Understand AI.

See where Hallucination sits on the sky →

Nearest on the sky

• The Generation Loop
• Prompting
• Capabilities and Limits
• Tokenisation
• The Training Pipeline
• The Context Window

A filled dot marks a star that is already open to walk.