← Tutorials11 / 22

What is an LLM?

Why multi-head attention?Tap to reveal

The model runs several attention 'heads' in parallel, each with its own query/key/value projections, then combines them.

Different heads learn different relationships: one tracks syntax, another which noun a pronoun refers to, another nearby words.

← PreviousNext card →