← Tutorials13 / 22

What is an LLM?

What are 'parameters', and why do they decide hardware needs?Tap to reveal

Parameters (weights) are all the learned numbers: embedding tables and the matrices inside every block.

A 7-billion-parameter model at 2 bytes each (16-bit) needs ~14 GB just to hold the weights. Quantizing to 4 bits cuts that to ~4 GB, which is how large models run on laptops.

← PreviousNext card →