Logits

A model never picks a word. It scores every word it knows, and something else picks.

Part of the Turning the dials track on lAItest.

A language model has never chosen a word in its life.

It scores them. All of them. Every step.

One raw score for every token in the vocabulary.

At each step the model reads everything written so far and emits a long list of numbers — one for each token it could produce next. Tens of thousands of them. Nothing has been selected yet. These raw scores are called logits. They are not percentages, they do not add up to anything in particular, and they can be negative. Higher simply means the model favours that token more.

A common misconception

Commonly believed: The model writes a word, then thinks about the next one, the way a person drafts a sentence.

Actually: The model produces a full scoreboard over its entire vocabulary at every single step. Turning that scoreboard into one token is a separate act, performed by ordinary code outside the model. That split is the reason you get any knobs at all — every dial in this track lives on the code side, not the model side.

What does a language model actually output at each step?

Answer: A score for every token in its vocabulary. The model emits the whole scoreboard — one raw score per token, called logits. Choosing a single entry happens afterwards, in code you control. Nothing in that scoreboard is a claim about being right; it is a claim about what tends to come next.

In one sentence

A language model does not choose words. It scores them, and something else does the choosing.