Definition
Constrained decoding restricts token choices during generation. At each step, the decoding process uses the generated prefix and a set of constraints, such as a grammar or schema, to find token continuations that could still produce a valid response. It rules out invalid candidates before the next token is selected, then updates the allowed set as generation continues.
Post-processing checks or repairs text after the model has generated it. Constrained decoding acts before each token is selected, so a disallowed continuation never becomes part of the response.
Simple example
Suppose an application needs JSON with a status field whose value must be open or closed. After the model has produced {"status":"o, the decoder allows only token continuations that can complete open. A continuation toward closed is no longer valid at that point. The exact allowed tokens depend on the tokenizer. One token may contain several characters.
Without a decoding constraint, the model could produce {"status":"pending"} even if the prompt asks it to use only the two permitted values.
Why it matters
When application code needs a specific output shape, constrained decoding can avoid formatting failures and the retries they cause. Structured output APIs often use it to enforce a supported schema. The caller can work with a defined response shape instead of searching free-form text for fields.
One important nuance
Constrained decoding enforces only constraints that the runtime supports and applies during generation. An open value can still be wrong for the actual ticket. Application code must check facts and business rules separately. A refusal or interrupted generation can leave the application without a complete object, so callers must handle those outcomes too.