← Tutorials6 / 17

Build a tiny LLM on your CPU

How does a model generate text?Tap to reveal

Start from a newline, ask the model for next-character probabilities, sample one, append it, and repeat. This is exactly how ChatGPT writes, just with a much better model.

Temperature divides the logits before softmax. We use 0.8, slightly more conservative than as-trained.

for _ in 0..len {
    let logits: Vec<f32> = next(&out)
        .iter().map(|l| l / temperature).collect();
    out.push(rng.sample(&softmax(&logits)));
}
← PreviousNext card →