The Prediction Machine
Every large language model, at its core, does one thing: it predicts what comes next.
Not "searches for answers." Not "looks things up." Not "thinks." It predicts the next token — a word, part of a word, or punctuation mark — based on everything that came before it. Then it does it again. And again. Until it decides to stop.
That's the entire mechanism behind a 10,000-word essay, a working Python script, or a convincing legal brief.
One word at a time
Take the simplest possible example. Given the prompt "The capital of France is", the model assigns a probability to every possible next token in its vocabulary and picks from the top candidates:
The model doesn't look up the answer — it predicts which token most plausibly follows this sequence.
The model picks "Paris" — not because it retrieved a fact from a database, but because in its training data, the sequence "The capital of France is" was overwhelmingly followed by "Paris." Pattern recognition at an enormous scale.
That token gets appended to the context, and the process repeats for the next word, and the next, until the response is complete. A 500-word answer is 500 individual predictions chained together.
Not all predictions are this certain
The nursery rhyme question below has one very obvious completion — but what happens when you turn up the temperature? Drag the slider to see how the probability distribution shifts, and what that means for the model's output:
Nearly always picks the highest-probability token. Responses are consistent — sometimes to the point of being repetitive.
Temperature controls how the model samples from that probability distribution. At low temperature it almost always takes the safest bet. As you raise it, lower-probability tokens get more of a chance — which is why higher temperature feels more creative, and also why it produces more errors.
Most AI products run somewhere around 0.7–1.0 by default. Writing tools lean higher; factual assistants lean lower. It's a dial, not a switch — and understanding it helps you recognize when a model is hedging versus when it's genuinely uncertain.
How a model learns to predict
No one wrote rules for understanding language. The model was trained on an enormous corpus of text — books, research papers, source code, legal documents, Wikipedia, news archives, and much of the public internet — and learned purely by doing.
For every sequence in that corpus, the model was asked: "What comes next?" It guessed, compared its guess to the actual text, and adjusted billions of internal numerical weights — called parameters — based on the error. Repeat across trillions of examples, and you get a model that's very good at predicting language.
The key insight: to predict human language accurately, you have to learn something about the world humans are describing. A model can't reliably predict what follows "The boiling point of water is" without having learned something about physics. It can't predict the next line of a poem without developing a sense of meter and rhyme.
Capable-seeming behavior isn't a separate feature built on top of the prediction engine. It emerges directly from it.
What it isn't
The "prediction machine" framing also explains what these models are not — which matters as much as what they are.
Not a search engine. A search engine retrieves documents that already exist. A language model generates new text by prediction. It doesn't "look up" answers — it composes them. This matters because if a fact is obscure, contested, or post-training-cutoff, the model will still generate something plausible-sounding regardless of accuracy.
Not a database. Databases return stored records with precision. Language models encode knowledge as distributed numerical weights — patterns, not records. They can't retrieve a specific fact the way a database returns a row. Ask for a phone number and you might get a plausible-looking one that was never real.
Not a reasoning engine. Prediction is not reasoning. When a model walks through a math problem step by step, it's generating tokens that resemble a correct math walkthrough — not running arithmetic. Sometimes those tokens are right. Sometimes they look right and aren't. The model has no internal calculator or logic verifier.
Why models make things up
Once you understand prediction, hallucinations stop being mysterious.
The model generates what is statistically likely to follow — not what is true. It has no separate "fact-check" step and cannot distinguish between "I reliably know this" and "this pattern sounds right." So it produces confident-sounding text in both cases.
This isn't a bug that will be patched in the next release. It's a structural feature of how these systems work. Newer models are more calibrated — they admit uncertainty more often and use internal reasoning steps to catch errors — but the tendency doesn't disappear entirely. It's baked in.
What to take from this
The prediction model gives you a practical heuristic for when to trust these tools:
- Style, tone, format, structure — very reliable. The model has seen millions of examples and has strong patterns.
- Well-documented facts — usually reliable. The signal was dense in training data.
- Niche facts, recent events, specific numbers — verify independently. The patterns are sparse or absent.
- Math, logic, step-by-step reasoning — use with caution. Check the output rather than assuming the steps are sound.
The most common mistake new users make isn't being too skeptical — it's the opposite. The model's fluency and confidence feel like accuracy. They aren't. Fluency is a property of prediction; accuracy requires verification.
The next module looks at tokens more closely — the actual units the model works in — which explains how cost is calculated, why context windows fill up, and some of the stranger patterns you'll notice in model behavior.