← Tutorials6 / 22
What is an LLM?
How does an LLM write a whole answer?Tap to reveal
prompt ─▶ model ─▶ probabilities ▲ │ sample └── append token ◀────┘ repeat until <end>
Autoregressively: one token at a time, each new token added to the input for the next step, until it produces a stop token or hits a length limit.
That's why output streams word by word, and why long answers take longer: each token costs one full pass through the model.