Practice
Learn
← Quit
0 / 0 correct
LLMs & AI
Hard
1 of 20
Design: you serve an open-weights 70B model and need lower cost per token. Which levers help most?
A
Quantization, continuous batching, prefix caching, speculative decoding, and routing easy traffic to a smaller model
B
Disabling streaming
C
Lowering the temperature
D
Increasing max_tokens