← Tutorials11 / 22
What is an LLM?
Why multi-head attention?Tap to reveal
The model runs several attention 'heads' in parallel, each with its own query/key/value projections, then combines them.
Different heads learn different relationships: one tracks syntax, another which noun a pronoun refers to, another nearby words.