Practice
Learn
← Quit
0 / 0 correct
LLMs & AI
Hard
1 of 20
Why is single-request decoding on a GPU usually memory-bandwidth-bound?
A
Tokenization is slow
B
GPUs have too little compute
C
The KV cache is stored on disk
D
Each new token must read all model weights from GPU memory while doing relatively little math per byte