How Large Language Models Actually Work, Step by Step
Large language models can feel like magic: you type a question and a thoughtful paragraph appears. Under the surface the mechanism is simpler than it looks, and knowing it explains both what these models do well and where they fall down. At the center is one repeated act: predicting the next piece of text.
Step one: text becomes tokens
The model never sees your words as words. Your text is first split into tokens, small chunks that are usually a word or part of a word. Each token has a number. "The hotel pool" might become three tokens, each represented by an ID the model recognizes. From here on, the model works with those numbers.
Step two: tokens become meaning
Each token ID is converted into an embedding, a long list of numbers that captures what the token tends to mean and how it relates to other tokens. The model then runs these through many layers of a structure called a transformer. The key move inside a transformer is attention: for each token, the model weighs how much every other token in the input matters to it. In "the pool is closed because it is being cleaned", attention is what lets "it" connect back to "pool".
Layer by layer, the representation of each token is refined by its context, until the model has a rich sense of what the whole input is about.
Step three: predict the next token
Now the core act. Given everything so far, the model produces a score for every token in its vocabulary, estimating how likely each one is to come next. From "The spa is open from ten until" the model might rate "six" and "eight" highly and "banana" very low. One token is chosen, usually one of the most likely, with a little randomness so the output is not always identical.
Step four: repeat
That chosen token is added to the text, and the entire process runs again on the slightly longer input. Predict, append, predict, append. A paragraph is produced one token at a time, each prediction conditioned on everything before it. This is why answers appear word by word on screen: you are watching the loop run.
Where the knowledge comes from
Training is what makes the predictions good. The model was shown an enormous amount of text and, for each position, asked to predict the next token. Every wrong guess nudged its internal numbers toward a better one. Repeated across trillions of examples, this produces a system that has absorbed grammar, facts, reasoning patterns, and styles, not as stored sentences but as learned tendencies about what follows what.
That has a direct consequence. The model does not look anything up when it answers. It regenerates text that is statistically consistent with what it learned. When the pattern it learned is accurate, so is the answer. When the relevant fact was never in its training, or was rare, it will still produce a confident continuation, which is why models hallucinate.
Why this explains so much
The next token loop accounts for most of the behavior people find surprising.
It explains fluency: the model is optimized for text that reads well. It explains hallucination: plausible comes before true. It explains the context window: the loop can only condition on what fits in the input. It explains why facts in the prompt beat facts in memory: tokens sitting in the input directly shape the prediction, far more strongly than a faint pattern from training.
What this means for being understood
If you want a model to say accurate things about your business, you are not trying to teach it. You are trying to put the right tokens in front of it at the moment it predicts. Clear, specific facts that a retrieval system can find and place in the input become part of what the model conditions on. Everything else is left to the pattern, and the pattern is a guess.