← Tutorials17 / 17

Build a tiny LLM on your CPU

What should you try next?Tap to reveal

Experiments in examples/tiny_llm.rs: 1. Change CONTEXT from 3 to 5. Does loss drop? Does it memorize more? 2. Sample with temperature 0.3 and 1.5 and compare. 3. Train on your own text: cargo run --release --example tiny_llm -- my.txt 4. Add a validation split and print both losses. 5. Replace the fixed context with a single attention head.

For a full transformer in readable code, study Andrej Karpathy's nanoGPT and his 'Neural Networks: Zero to Hero' series.

← PreviousPractice questions →