Determinism

Temperature 0 removes the dice. It does not make the system reproducible.

Part of the Turning the dials track on lAItest.

Set temperature to zero, send the same prompt twice, and you can still get two different answers.

This one catches people who have already shipped.

Zero removes the dice. Other sources of variation survive it.

At temperature 0 the sampler stops rolling and always takes the top-scoring token, which kills one source of variation. Others remain. Adding floating-point numbers in a different order gives slightly different results, and a server batches your request with whatever other traffic happens to arrive at that moment. When two tokens are nearly tied, a difference in the last decimal place flips the winner — and one flipped token changes everything after it.

A common misconception

Commonly believed: Temperature 0 makes a model deterministic, so I can assert on the exact output string in a test.

Actually: Anthropic states plainly that temperature 0 does not guarantee fully deterministic output. Low temperature makes runs much more alike, and it is the right setting when you want stability. It is not a reproducibility guarantee, and a test suite that asserts on exact wording will be flaky for reasons you cannot fix.

Test the shape of the answer instead.

Assert on properties, not on strings. Check that the JSON parses, that required fields are present, that the number falls in range, that the label is one of the five you allow. When you genuinely need the identical bytes twice, store the output and reuse it rather than generating again and hoping.

In one sentence

Low temperature buys consistency, not reproducibility — so test the shape of an answer, never its wording.