Practice
Learn
← Quit
0 / 0 correct
LLMs & AI
Hard
1 of 430
Data, tensor and pipeline parallelism: which is accurate?
A
They all split the batch
B
Data parallel splits batches; tensor parallel splits individual matrix operations across GPUs; pipeline parallel splits layers into stages
C
Tensor parallelism splits the dataset
D
Pipeline parallelism replicates the model