← Tutorials7 / 22
What is an LLM?
What is a transformer?Tap to reveal
tokens ─▶ embeddings
│
┌────────▼────────┐
│ self-attention │ ×N blocks
│ feed-forward │
└────────┬────────┘
▼
logits ─▶ next-token probsThe neural network architecture behind modern LLMs. It's a stack of identical blocks (dozens of them). Each block has:
1. Self-attention, where tokens gather information from earlier tokens. 2. A feed-forward network (MLP), which processes each token's vector on its own.
Residual connections and normalization keep the deep stack trainable. The final layer turns the last vector into logits over the vocabulary.