Anyone who has used an AI-based assistant has sooner or later come across an answer that is confident, well written and completely wrong: a book attributed to the wrong author, a legal provision that does not exist, a bibliographic reference with a perfectly plausible title, journal and year that were all invented. This phenomenon is called hallucination. It is not an occasional glitch but a direct consequence of how language models work.
Why a model "hallucinates"
A language model generates text by choosing, word after word, the most plausible continuation based on what it has learned. Its goal during training is to produce realistic text, not true text. In the vast majority of cases the two coincide, because the texts it learned from mostly contain correct information. But when the requested information is rare, ambiguous or missing, the model has no built-in mechanism for saying "I don't know": it keeps producing what sounds right.
Some factors make hallucinations more likely:
- niche topics: little-known people, local businesses, sector-specific regulations;
- precise details: numbers, dates, proper names, codes, verbatim quotations;
- questions that assume a false fact: if you ask "why did that minister resign in 2019?" and they never resigned, the model may construct an explanation;
- requests for sources: article titles and URLs are sequences where small deviations produce references that do not exist;
- very long conversations, where early information is handled less precisely.
Typical examples
Recognising a hallucination is easier when you know the forms it usually takes.
| Type | Example | Why it is tricky |
|---|---|---|
| Invented source | A study with real authors and journal but a non-existent title | The credible setting masks the falsehood |
| Plausible number | A "round" percentage with no origin | Numbers give an air of precision |
| Non-existent function | A software command that looks official | The code seems right until you run it |
| Blended biography | Facts about two namesakes merged into one | Each fragment on its own is true |
In my digital forensics work I sometimes see AI-produced texts presented as technical documentation: the most recurrent warning sign is precisely the presence of specific references that, checked one by one, lead nowhere.
How to reduce the risk when writing your request
You cannot eliminate hallucinations entirely, but you can make them much less frequent.
- Provide the material yourself. If you paste the text of a document and ask for answers based only on it, the model has something concrete to work from and invents far less.
- Explicitly allow "I don't know". Phrases such as "if the information is not in the text, say so clearly" reduce the tendency to fill gaps.
- Ask it to separate facts from assumptions. A request like "distinguish what is certain from what is inferred" makes the weak points visible.
- Avoid questions built on unverified premises. Better to ask "did that minister resign?" than "why did they resign?".
- Use tools with search or linked documents when you need up-to-date or highly specific information, and check the sources they cite.
How to check an answer
The golden rule is simple: the more a piece of information matters, the more it needs checking at the source. In practice:
- check names, dates and figures against primary sources, such as the official text of a regulation or the institution's own website;
- open every bibliographic reference and confirm that it really exists and says what the model claims;
- for code, run the tests and consult the library's official documentation;
- where possible, ask the same question in a different way: answers that contradict each other are a warning sign;
- be wary of answers that seem too perfect on obscure subjects.
When hallucinations become a serious problem
In a draft social media post an error is easy to fix. In other contexts the consequences are heavy: a legal brief citing case law that does not exist, a technical diagnosis based on a command that does not do what you think, a business report with invented market data. In these cases AI can remain a valuable help for drafting and organising, but every factual claim must go through human checking.
Will hallucinations disappear?
Recent models hallucinate less than early ones, thanks to better data, targeted training and connection to external sources. However, as long as the underlying mechanism is the generation of plausible text, the risk will never be zero. It is therefore wise to treat AI as a brilliant but not always accurate collaborator: extremely useful for getting started, to be checked before you sign off.
In short
- Hallucinations arise because the model produces realistic text, not necessarily true text.
- They are more frequent with niche topics, precise details, sources and false premises.
- Providing documents, allowing "I don't know" and separating facts from assumptions reduces the risk.
- Important information should always be checked against primary sources.
