← Tutorials9 / 22
What is an LLM?
How is attention computed (queries, keys, values)?Tap to reveal
Each token produces three vectors: a query (what am I looking for?), a key (what do I contain?) and a value (what do I pass on?).
score = q · k / √d for every earlier token, softmax turns scores into weights, and the output is the weighted sum of values. All of this is matrix multiplication, which GPUs do very fast.
// for one token t (simplified):
scores[j] = dot(q[t], k[j]) / sqrt(d) // j <= t
weights = softmax(scores)
out[t] = sum_j weights[j] * v[j]