In 2025, OpenAI explained why language models make things up. The short answer: they are rewarded for guessing.
This is called hallucination. The word suggests a malfunction. In fact, it follows directly from how language models work and how they are scored.
A language model knows probabilities, not truth
A language model does not look things up. Put simply, it calculates word by word which word is most likely to come next. Because it was trained on enormous amounts of text, the most likely word is very often the right one.
The most likely word is not always the right one: Sydney instead of Canberra.
When the model does not know something, the process stays the same. It still picks the most likely next word. What comes out is a sentence that sounds just as confident as a correct answer. Hesitation or a “maybe” does not appear on its own.
Guessing earns points, silence doesn’t
OpenAI’s explanation came with a research paper. Language models are scored much like students on a multiple-choice test. If you guess, you sometimes get points. If you leave it blank, you certainly get none.
If scoring rewards guessing and penalises restraint, a model learns that guessing pays off. Hallucination is therefore also a consequence of how models are measured and improved.
A confident tone says nothing about accuracy
That is the most important insight for learners. An AI answer is a draft, not a reference work.
Three questions help in everyday use:
- Can I find this in a second, independent source? Not in a second AI, but in a source that is itself backed up.
- Can names, numbers and quotes be verified? These details are especially vulnerable because they have to sound plausible but should be exact.
- What happens if I ask the same question differently? If the answer changes with a slightly different wording, that is a warning sign.
An exercise for your next lesson
Have learners work in groups and ask an AI about a topic they already know well. Each group uses the three questions to find a statement that is wrong and backs up the error with a source.
In the debrief, it usually becomes clear quickly how convincing wrong answers can be. That turns distrust into a technique that can be practised.
AI is not an encyclopaedia, it’s a very good guesser.
Frequently asked questions
Why does AI hallucinate?
A language model calculates, word by word, which word is most likely to come next. When it doesn't know something, it still picks the most likely word, and the result sounds just as confident as a correct answer.
Why doesn't AI just say "I don't know"?
According to an explanation OpenAI published in 2025, language models are scored much like students on a multiple-choice test: guessing sometimes earns points, leaving it blank certainly earns none. So a model learns that guessing pays off.
How can you spot invented AI answers?
Three questions help: can I find this in a second, independent source? Can names, numbers and quotes be verified? What happens if I ask the same question differently?
