Practice
Learn
← Quit
0 / 0 correct
LLMs & AI
Hard
1 of 331
Why is single-request decoding on a GPU usually memory-bandwidth-bound?
A
Tokenization is slow
B
GPUs have too little compute
C
Each new token must read all model weights from GPU memory while doing relatively little math per byte
D
The KV cache is stored on disk