Inference & performance · Concept

Prefix caching

Reuse computation for matching input prefixes.

Related terms

TTFT · ITL · TPOT

Primary references

Reference definition