Practice
Learn
← Quit
0 / 0 correct
LLMs & AI
Hard
1 of 20
What does ZeRO / FSDP do in distributed training?
A
Compresses the training data
B
Removes the need for gradients
C
Runs training on CPUs
D
Shards optimizer states, gradients and weights across GPUs, instead of replicating them on each