itsbrill.ai

How it works · 2 of 4

Why it makes things up

What you get from this page

3 min read

01Plausible and wrong

People call it hallucinating, and OpenAI defines it as "plausible but false statements generated by language models". It isn't lying, because there's no intent. It's the model producing text that sounds right when it doesn't have the right answer.

The researchers give a good example. They asked a widely used chatbot for the title of one of the authors' PhD dissertations, and it "confidently produced three different answers", none of which was right. Source: OpenAI, Why language models hallucinate, 5 September 2025, checked 3 November 2026.

02Why it happens

1. It predicts what sounds right, and nothing checks it against the truth. As What's inside an AI model explains, the model learned by predicting the next piece of text. In that training it "sees only positive examples of fluent language", with nothing labelled true or false. Patterns that repeat, like spelling, get learned well. Rare one-off facts don't. In OpenAI's words, "arbitrary low-frequency facts, like a pet's birthday, cannot be predicted from patterns alone and hence lead to hallucinations".

2. Guessing gets rewarded. OpenAI's research argues that models hallucinate "because standard training and evaluation procedures reward guessing over acknowledging uncertainty". Think of a multiple-choice test. A guess might score a point, and a blank always scores zero, so a model graded only on right answers learns that guessing pays.

A multiple-choice test where a right guess scores +1, a wrong guess scores 0 and leaving it blank always scores 0, so the agent picks an answer instead of skipping.
A guess can score, a blank never does, so guessing gets learned.

3. Gaps get filled. When your request leaves something out, it fills the gap with the most likely-sounding detail. Ask for a reminder email with no names and dates, and it'll invent them unless you tell it not to.

4. Long chats drop details. As a conversation grows, older details slip out of view, and it fills those gaps too. Sessions, context and memory explains why.

03Where it happens most

  • Exact numbers, dates, prices and statistics.
  • Names, job titles and who said what.
  • Quotes, references, citations and links.
  • Anything recent, niche or local.
  • Maths done "in its head" instead of with a calculator or code.
  • Anything it was never shown, such as your files, your inbox or your company's policies.

04What helps

  • Give it the source. Paste the document, the email or the page, and ask it to answer only from that and to quote the line it used.
  • Make "I don't know" a good answer. OpenAI's finding is that made-up answers aren't inevitable, "because language models can abstain when uncertain". Tell it you'd rather have a gap than a guess.
  • Use a thinking mode for anything that matters. OpenAI says its newer models make fewer of these mistakes, especially when reasoning, and that they still happen. See Models and reasoning levels.
  • Let it calculate. For numbers, ask it to use a calculator, code or a spreadsheet formula instead of working it out in its head.
  • Check what matters with something that isn't the model. Check the facts shows how, including a check that runs the same way every time.

None of these stop it making things up. They make it rarer and easier to catch, which is why the checking matters.

05Paste this

Add this to your standing instructions, or paste it at the start of any chat where facts matter:

Gaps instead of guesses
If you don't know something or can't check it, say so. "I don't know" is a good answer.
Never make up a name, number, date, quote, reference or link.
When you state a fact, say where it came from: what I gave you, a source you can name, or your general knowledge.
Mark anything you're not sure of, and leave a [gap] instead of filling it.

Download this page as a text file (.md)

Enter to openEsc to close