Inference & performance · Concept

TPOT

Also called: time per output token

Time per output token: an average token-generation latency under a specified measurement method.

Related terms

TTFT · ITL · TPS

Primary references

Reference definition