← Tutorials18 / 22

What is an LLM?

What do temperature and top-p do?Tap to reveal

They control how the next token is picked from the probabilities.

- Temperature divides logits before softmax. Near 0 is almost greedy (most likely token, deterministic). Higher values flatten the distribution and give more variety. - Top-p (nucleus) samples only from the smallest set of tokens whose probabilities add up to p (e.g. 0.9), cutting off the unlikely tail.

probs = softmax(logits / temperature)
// T = 0.2: confident, repetitive
// T = 1.0: as trained
// T = 1.5: creative, error-prone
← PreviousNext card →