Practice
Learn
← Quit
0 / 0 correct
LLMs & AI
Hard
1 of 10
What does ZeRO / FSDP do in distributed training?
A
Compresses the training data
B
Runs training on CPUs
C
Removes the need for gradients
D
Shards optimizer states, gradients and weights across GPUs, instead of replicating them on each