How a large language model actually works

1. Text becomes tokens

A model doesn't see words or letters — it sees tokens, the chunks text is split into. A token is often a word piece: "payroll" might be one token, "unbelievable" might be three (un · believ · able). Every token maps to a number, so a sentence becomes a list of numbers the model can do math on.

2. Tokens become vectors (embeddings)

Each token is turned into a long list of numbers called an embedding — a position in a high-dimensional "meaning space." Tokens used in similar ways end up near each other. This is why a model can tell that "Toronto" and "Vancouver" are the same kind of thing, without anyone writing a rule that says so.

3. Attention: every token looks at the others

The core of the transformer (the architecture behind every modern LLM) is attention. For each token, the model weighs how much every other token in the context matters to it, and mixes their meaning together. That's how "it" gets connected to the right noun, and how a fact mentioned 2,000 words earlier can influence the next word.

4. The one thing it predicts: the next token

After all that processing, the model outputs a probability for every possible next token. It picks one, appends it to the text, and runs the whole thing again — token by token — until the answer is complete. "Writing a paragraph" is really thousands of next-token predictions in a row.

The whole pipeline, visualized

5. Training vs. using the model

• Pre-training: the model reads a vast amount of text and adjusts billions of internal weights to get better at predicting the next token. This is the slow, expensive part — and where it absorbs grammar, facts, and patterns.
• Fine-tuning & alignment: a second phase (often with human feedback) shapes it to be helpful, follow instructions, and stay safe.
• Inference: when you actually use it. The weights are frozen; it's just running predictions over your prompt.

6. The context window is its working memory

A model can only "see" a limited amount of text at once — its context window. Anything outside it is invisible unless you put it back in. The model has no memory between conversations on its own; techniques like retrieval (RAG) work by fetching the right text and placing it into the context so the model can attend to it.

7. Why it sometimes makes things up

Because it predicts plausible text, not true text. When the model is unsure, the most probable-sounding continuation can still be wrong — a hallucination. It isn't lying or guessing randomly; it's completing a pattern. That's exactly why grounding matters: give it the real facts in context, or have it call a tool that computes the real answer, instead of trusting it to recall.