← Tutorials7 / 17
Build a tiny LLM on your CPU
What does the counting model produce?Tap to reveal
Loss 1.757, much better than 3.178, but the text is mostly gibberish with real fragments ('the', 'sat').
It only ever sees one previous character, so after 't' it can't tell whether it's in 'the', 'cat' or 'sat'.
1) counting bigram loss 1.757
sample: t ca t tfog. / the obkw shbhecungongca
ck ofebil arwad d theg cwg dhwatopm.krm te at.