What an LLM actually is

One thing, ten thousand times

There’s a moment in every conversation with a language model where you forget it’s a program. It writes like a thoughtful person. It anticipates what you mean. It explains things patiently, and pivots when you push back.

Underneath all of that, it is doing exactly one thing, once for every word it writes: given everything that has come before, figure out which piece of text should come next. Write it. Then do the same thing again.

This is called next-token prediction, and it is the entire training objective of a modern language model. Not “understand language.” Not “be helpful.” Just: given this sequence, which element belongs next? Do it accurately enough, across enough text, and the resulting system acquires capabilities no one explicitly programmed in. Everything you find impressive — the range of topics, the apparent reasoning, the appropriate tone — follows from this one mechanism applied at enormous scale.

The output at each step is not simply the next word. It is a probability distribution over every possible next token — roughly 100,000 of them. The model reads the input, runs it through billions of learned parameters, and produces 100,000 numbers that sum to one. One token is then sampled from that distribution and appended. The whole process repeats until the response is complete.

The sampling step is why the model’s outputs are not deterministic. Ask the same question twice and you will often get a different answer. That is not a defect — it reflects the fact that most prompts have more than one reasonable continuation. The “temperature” setting you may have seen in API parameters controls how broadly the model samples: a higher temperature means more willingness to pick lower-probability tokens, producing more varied or surprising output; a lower temperature keeps it near the peak of the distribution, making it more consistent and predictable.

What a token is

The word “token” is doing real work here, so it is worth being precise. A language model does not operate on words — it operates on subword pieces. The word “unhelpful” might become three tokens: un, help, ful. The word “cat” is one. A space before a word often fuses into the token that follows it. Common short words are single tokens; rare or technical words fragment into several.

Modern models have vocabularies of around 100,000 tokentypes. Each forward pass through the network produces a probability for every one of them. The probability assigned to “cat” might be 0.12, to “the” 0.08, to “dog” 0.06, and so on down to vanishingly small values for tokens that are implausible in context. The model then samples — picking one proportional to those probabilities, not always the highest one.

This is why generation feels natural rather than mechanical. The model is not finding a single correct answer; it is sampling from a learned distribution over plausible continuations. The same underlying mechanism produces both predictable output and occasional surprises, depending on where the probability mass lands.

Where the knowledge lives — weights, not a database

Here is the part that surprises most people when they first encounter it: the model is not looking anything up.

There is no database being queried at generation time. No index being searched. No Wikipedia article being retrieved. Unless you have separately wired up a retrieval system — which is a distinct tool layered on top of the model, not the model itself — the model has no access to anything outside its own parameters during inference.

Those parameters are called weights. They are the billions of numbers that make up the neural network, and they encode everything the model knows. During training, the model was shown an enormous quantity of text — web pages, books, code, conversations — and after each prediction, its weights were nudged slightly in the direction that would reduce prediction error. After enough adjustments, across enough text, the weights come to reflect the statistical structure of the training data: what words tend to follow what, which facts cluster with which, what patterns of reasoning tend to appear in explanations.

Think of the weights as a compressed imprint of the training corpus — not a lossless recording, but a statistical fingerprint. The model has absorbed patterns at every scale: the spelling of words, the grammar of sentences, the structure of arguments, the relationships between concepts. All of it is encoded, lossy, in those numbers.

When training ends, the weights freeze. They do not change when you talk to the model. Every conversation runs through the same fixed function. The model is not learning from you in the moment; it is applying what it already learned from everything it was trained on. At inference time, the model does a single forward pass through those frozen weights for each token it generates. No retrieval, no lookup, no external memory. Just: input in, probability distribution out, sample, repeat.

Fluency is not accuracy

Now we can explain something that trips nearly everyone up at first.

How can the same model write beautifully structured prose and also confidently assert that a historical event happened on the wrong date, or that a law works differently from how it actually does?

The model’s fluency comes from the fact that grammatical, coherent, well-structured text is what appeared most often in its training data. The weights have absorbed the patterns of good writing — sentences have subjects and verbs, explanations follow certain shapes, arguments have premises and conclusions. That structure is baked into the probability distributions the model produces, so the output naturally inherits it.

But factual accuracy is a separate property entirely. The model is not consulting a fact-checked database — it is sampling from a distribution learned from text produced by humans, which includes false claims, misconceptions, outdated information, and outright errors. If a wrong claim appeared many times in training data, the model has learned to reproduce it. If the correct version was rare, the model may never have strongly learned it.

Worse: there is no internal signal in the generation process that distinguishes confident knowledge from a confident guess. The model produces plausible-sounding text whether or not the underlying claim is solid. Fluency is achieved either way, because fluency is what was optimized.

This is what the field calls hallucination: not random garbled output, but coherent, confidently delivered text that is factually wrong. It is not an implementation bug. It is a direct consequence of training a system to produce probable continuations of text, rather than a system that verifies claims before making them.

Knowing this changes how you read model output. Fluency is evidence that the model has learned the patterns of the domain you are working in. It is not evidence that any particular claim is true. The two things feel identical from the outside, which is why they are worth keeping explicitly separate in your head.

How to read this primer

That is the one idea: a model assigns probabilities to tokens, samples one, and repeats — and its behavior reflects what was in its training data because the weights are fit to that data, not to a fact-checked oracle. The rest of this primer unpacks the layers underneath that idea, from how models are built to why they behave the way they do in practice.

Here is the arc ahead of you:

Life of a Model traces the journey from blank weights to a useful assistant — pretraining, where knowledge is absorbed; instruction tuning, where behavior is shaped; and reinforcement learning, where judgment is refined.

Pretraining goes deep on what training on a trillion tokens actually involves: what the network is optimizing, why scale changes what emerges, and what a base model can and cannot do on its own.

Instruction Tuning covers the step that transforms a text-completer into a conversational assistant — and why a surprisingly small number of carefully chosen examples does most of the work.

RL and Alignment explains how feedback from humans and other models steers behavior without replacing the knowledge already in the weights.

RL for Reasoning covers models that generate a chain of thought before answering — and what gets optimized when the reasoning itself is the training signal.

From Input to Output goes into the machinery itself: attention, layers, and how context flows through the network to produce each probability distribution.

The Evolution of Scale closes with the ideas that explain why language models exist at their current capability: the scaling laws, the architectural choices, and the empirical surprises that shaped the field.

Read them in order for the full arc. Or jump to the one that answers whatever you are trying to understand today — each essay is written to stand on its own.