Practice
Learn
← Quit
0 / 0 correct
LLMs & AI
Hard
1 of 409
In scaled dot-product attention, why divide the scores by √d (d = key dimension)?
A
To reduce memory usage
B
Dot products grow with d; scaling keeps softmax from saturating, which would cause vanishing gradients
C
To normalize vectors to unit length
D
To make attention causal