Structured output

Constrained decoding makes invalid JSON impossible to generate, which retires the retry loop everyone still writes.

Part of the Getting good answers track on lAItest.

Plenty of production code still retries a model until the JSON parses. On current models that loop cannot fire.

The output is incapable of being malformed.

The model is not asked to be careful. It is prevented from being wrong.

A model normally picks each token from the whole vocabulary. Constrained decoding compiles your schema into a grammar and, at every step, blocks every token that could not legally come next. If the only valid continuation is a closing brace, nothing else is available to sample. Malformed output is not repaired afterwards. It is never generated.

A common misconception

Commonly believed: A loose JSON mode and schema-enforced output are the same feature under two names.

Actually: A JSON mode asks the model to emit JSON and usually gets it. Constrained decoding enforces the shape at the token level, so it holds even on the awkward inputs: long lists, unusual characters, a model that wants to add an apology before the opening brace. Those inputs are precisely where the old parser broke.

Two separate switches.

One governs the format of the response itself. The other makes tool arguments conform strictly to the tool schema. They are independent features and you can want either without the other. The grammar compiled from a schema is cached, so reusing one schema costs less than sending a fresh one on every call.

Your service still retries until the JSON parses. Why might that now be pure cost?

Answer: Because a schema-constrained model cannot emit output that violates the schema. With constrained decoding the illegal tokens are never sampled, so a parse failure is not bad luck to retry through. It means the feature is off, the model does not support it, or the real failure is somewhere else. A retry loop hides which of the three is true.

In one sentence

If your model supports schema-enforced output, a parse error is a bug in your setup rather than bad luck to retry through.