Practice
Learn
← Quit
0 / 0 correct
LLMs & AI
Hard
1 of 20
Why are learning rate warmup and decay schedules used in LLM pretraining?
A
Warmup avoids unstable early updates with large gradients; decay lets the model settle into a good minimum
B
To reduce dataset size
C
To make training deterministic
D
To speed up tokenization