Inference & performance · Concept

Semantic cache

Reuse a previous result for a sufficiently similar request, subject to validity and permission checks.

Related terms

TTFT · ITL · TPOT

Primary references

Reference definition