← Tutorials10 / 22

What is an LLM?

Why can a GPT-style model only attend to earlier tokens?Tap to reveal

It's trained to predict the next token. If a position could see the future, it would just copy the answer and learn nothing.

A causal mask blocks attention to later positions. That lets one training pass learn from every position of a text at once.

← PreviousNext card →