← Tutorials9 / 22

What is an LLM?

How is attention computed (queries, keys, values)?Tap to reveal

Each token produces three vectors: a query (what am I looking for?), a key (what do I contain?) and a value (what do I pass on?).

score = q · k / √d for every earlier token, softmax turns scores into weights, and the output is the weighted sum of values. All of this is matrix multiplication, which GPUs do very fast.

// for one token t (simplified):
scores[j] = dot(q[t], k[j]) / sqrt(d)   // j <= t
weights   = softmax(scores)
out[t]    = sum_j weights[j] * v[j]
← PreviousNext card →