Inference & performance · Concept

Decode

Generate subsequent output tokens using the model and available state.

Related terms

TTFT · ITL · TPOT

Primary references

Reference definition