← Tutorials8 / 22

What is an LLM?

What does self-attention do?Tap to reveal

It lets each token look at the tokens before it and pull in information from the relevant ones.

In 'The cat didn't eat because it was full', the vector for 'it' attends strongly to 'cat', so the model knows what 'it' refers to.

← PreviousNext card →