← Tutorials12 / 17

Build a tiny LLM on your CPU

What are the embeddings in this model?Tap to reveal

A 24 × 10 table: each character gets 10 learned numbers. They start random; training moves them so characters that behave alike (like vowels) end up with similar vectors.

That's exactly what an LLM's token embeddings are, just 10 numbers here instead of thousands.

let x: Vec<f32> = ctx.iter()
    .flat_map(|&t| self.emb[t * d..(t + 1) * d].iter().copied())
    .collect();
← PreviousNext card →