Practice
Learn
← Quit
0 / 0 correct
LLMs & AI
Hard
1 of 475
Why is single-request decoding on a GPU usually memory-bandwidth-bound?
A
Each new token must read all model weights from GPU memory while doing relatively little math per byte
B
Tokenization is slow
C
The KV cache is stored on disk
D
GPUs have too little compute