← Tutorials6 / 17
Build a tiny LLM on your CPU
How does a model generate text?Tap to reveal
Start from a newline, ask the model for next-character probabilities, sample one, append it, and repeat. This is exactly how ChatGPT writes, just with a much better model.
Temperature divides the logits before softmax. We use 0.8, slightly more conservative than as-trained.
for _ in 0..len {
let logits: Vec<f32> = next(&out)
.iter().map(|l| l / temperature).collect();
out.push(rng.sample(&softmax(&logits)));
}