← Tutorials7 / 17

Build a tiny LLM on your CPU

What does the counting model produce?Tap to reveal

Loss 1.757, much better than 3.178, but the text is mostly gibberish with real fragments ('the', 'sat').

It only ever sees one previous character, so after 't' it can't tell whether it's in 'the', 'cat' or 'sat'.

1) counting bigram   loss 1.757
   sample: t ca t tfog. / the obkw shbhecungongca
   ck ofebil arwad d theg cwg dhwatopm.krm te at.
← PreviousNext card →