Inference & performance · Concept

End-to-end latency

Total request or task duration, including queues, model calls, tools, retrieval, and retries.