Practice
Learn
← Quit
0 / 0 correct
LLMs & AI
Hard
1 of 40
What does ZeRO / FSDP do in distributed training?
A
Runs training on CPUs
B
Shards optimizer states, gradients and weights across GPUs, instead of replicating them on each
C
Removes the need for gradients
D
Compresses the training data