Inference & performance · Concept
TPOT
Also called: time per output token
Time per output token: an average token-generation latency under a specified measurement method.
Related terms
Primary references
Reference definition
Inference & performance · Concept
Also called: time per output token
Time per output token: an average token-generation latency under a specified measurement method.
Reference definition