← Tutorials15 / 17
Build a tiny LLM on your CPU
Is our model actually that good, or just memorizing?Tap to reveal
Mostly memorizing. We measured loss on the same text it trained on. With 632 characters and ~4,000 parameters, it can nearly learn the text by heart.
Real training holds out validation text the model never trains on, and watches that loss. When training loss keeps falling but validation loss rises, the model is overfitting.
Exercise: split the text 90/10 and report both losses.