# What's inside an AI model

*How it works · 1 of 4*

- What actually happens between your question and its answer.
- Why it has no store of facts to look things up in.
- The difference between the model and the app you use it in.

4 min read

You don't need any of this to use AI. It helps, though, because nearly every odd thing an AI tool does makes sense once you know roughly what's inside.

## It predicts the next piece

When you ask a question, the model doesn't look the answer up. It predicts the next small piece of text, adds it, then predicts the next piece, and keeps going until the reply is finished.

It's the same idea as the word suggestions above your phone's keyboard, on a vastly bigger scale. Your phone has learned which word usually comes next in your messages. An AI model has learned which piece usually comes next across a huge amount of writing, and it's good enough at it that the pieces add up to an answer, an email or a plan.

![The words "The cat sat on the" are split into pieces, pass through a web of connections, and one next piece, "mat", comes out.](/playbook/illustrations/v2/next-word.png)

*The model predicts the next piece of text, one piece at a time.*

## Pieces called tokens

The pieces are called tokens. A token can be a whole common word, part of a longer word or a bit of punctuation. Each token has a number, and that list of numbers is what the model actually works with.

Tokens matter to you for one practical reason. Plans, limits and costs are counted in them, and so is how much the model can keep in view at once, which comes up again in Sessions, context and memory (/guide/sessions-and-memory.html).

## A network of numbers

Inside the model is a neural network, which is a huge set of numbers arranged in layers. The numbers coming in from your message pass through each layer, and each layer turns them into new numbers, until the last layer gives a score for every possible next token. The highest-scoring ones are the likely next pieces.

No single number in there means "pizza" or "invoice". What the model knows is spread across billions of numbers, a little in each. Picture a mixing desk with billions of knobs, each turned slightly to one side, where the sound you hear comes from all of them together.

## How it learned

Those knobs got set in training. The model read an enormous amount of text and, over and over, tried to predict the next piece. Each time it was wrong, every knob was nudged a tiny bit towards the right answer. After enough of that, it had picked up grammar, facts, styles of writing and patterns of reasoning, without anyone writing any of them in as rules.

After that first stage, models get more training to follow instructions and be helpful. Part of it is people rating answers, and the model learning to give the kind of answer people rate highly. That's useful, and it also explains why a model can lean towards telling you what you want to hear, which Straight answers (/guide/straight-answers.html) deals with.

## What it doesn't have

- No list of facts to check. It produces what sounds right given its training, and it can't tell you where a particular "fact" came from.
- No memory of you between chats, unless the app it runs in adds one.
- Nothing after its training, unless the app lets it search the web or read your files. Even then it can misread or misreport what it finds, so a searched answer still needs checking.
- Only what's in front of it. It sees the current conversation and whatever's been given to it, and nothing else.

## The model and the app

You never use a model on its own. You use it inside an app, such as a chat website, a phone app, a coding tool or a bot. A handy way to picture it is that the model is the brain and the app is the body. The app decides what the model can do, like search the web, read your files, remember things about you, run on a schedule or send an email.

That's why the same model can feel quite different in two apps, and why most of this guide is about setting up the app around the model so it works the way you need.

## Why this matters

Each part of how it works leads to one of the setups:

- It predicts what sounds right, so important facts need checking. See Why it makes things up (/guide/why-it-makes-things-up.html) and Check the facts (/guide/check-the-facts.html).
- It only sees the current chat, so it needs a memory you control and fresh sessions. See A memory that works (/guide/04-make-it-yours.html) and Fresh sessions (/guide/fresh-sessions.html).
- It was trained on what people rate highly, so it can flatter. See Straight answers (/guide/straight-answers.html).
- The app decides what it can do, so set the rules first. See Ground rules (/guide/ground-rules.html).
