Practice
Learn
← Quit
0 / 0 correct
LLMs & AI
Hard
1 of 10
What is a Mixture-of-Experts (MoE) transformer?
A
Each MLP layer has many 'experts' and a router sends each token to a few, so total parameters are large but compute per token is small
B
A model that uses several tokenizers
C
A model trained by many teams
D
An ensemble of separate models voting