Practice
Learn
← Quit
0 / 0 correct
LLMs & AI
Hard
1 of 563
Why is single-request decoding on a GPU usually memory-bandwidth-bound?
A
The KV cache is stored on disk
B
Tokenization is slow
C
Each new token must read all model weights from GPU memory while doing relatively little math per byte
D
GPUs have too little compute