Practice
Learn
← Quit
0 / 0 correct
LLMs & AI
Hard
1 of 40
What is the main trade-off of quantizing an LLM to 4-bit weights?
A
It makes training impossible to run afterwards
B
About 4× less memory than fp16 and faster memory-bound decoding, at the cost of some quality loss (method-dependent)
C
No memory savings but better quality
D
It increases the context window