Inference & performance · Concept

TTFT

Also called: time to first token

Time to first token: delay from a defined request start until the first generated token arrives.

Example

Track first-token delay separately from the time required to complete the user task.

The distinction that matters

A quick first token can coexist with poor end-to-end latency.

Related terms

TPS · End-to-end latency · Tail latency

Primary references

Reference definition