← Tutorials1 / 17

Build a tiny LLM on your CPU

What are we going to build?Tap to reveal

A character-level language model, written from scratch in plain Rust with no dependencies. It reads text, learns which characters tend to follow which, and then writes new text.

We build it in three steps, each a better model: 1. Count pairs of characters. 2. Learn the same thing with gradient descent. 3. A small neural network that reads 3 characters of context.

The whole program is examples/tiny_llm.rs in this project.

cargo run --release --example tiny_llm
Next card →