← Quit0 / 0 correct
LLMs & AIHard1 of 615

Why is single-request decoding on a GPU usually memory-bandwidth-bound?