Inference & performance · Concept

ITL

Also called: inter-token latency

Inter-token latency: elapsed time between consecutive emitted tokens.