Turning the dials

Logits, softmax, temperature, top-p, streaming and stop conditions — the whole control panel, without the maths anxiety.

10 concepts, about 60 minutes of reading at roughly six minutes each. Free to read, with no account.

What is in this track

1. Logits — A model never picks a word. It scores every word it knows, and something else picks. (free)
2. Softmax — The step that turns a pile of raw scores into percentages that add up to 100. (free)
3. Sampling and decoding — Same prompt, same model, different answer. The model did not change its mind; the draw did. (free)
4. Temperature — One number decides how far the sampler is allowed to stray from the model’s favourite token. (free)
5. Top-p and top-k — Temperature reshapes the whole list. These two just delete the bottom of it. (free)
6. Determinism — Temperature 0 removes the dice. It does not make the system reproducible. (free)
7. Stop sequences and max_tokens — A model has no sense of how long your answer should be. Two settings decide when it ends. (free)
8. Streaming and time to first token — Streaming does not make generation faster. It makes the waiting shorter, which is a different product. (free)
9. Throughput versus latency — A serving system can be fast for one person or fast for everyone, and the two fight each other. (free)
10. Perplexity — One number for how surprised a model is by a piece of text. It travels much less well than people assume. (free)

Before this: What is this thing?

After this: Getting good answers

Every track · Pricing · Claims we checked and could not stand behind