Inference & performance · Concept

Prompt caching

A provider's mechanism for reusing eligible prompt processing; exact behavior and billing are platform-specific.

Example

Keep stable shared instructions in a cache-eligible prefix.

The distinction that matters

Cached prompt processing is not cached answers or persistent user memory.

Related terms

KV cache · Semantic cache · Long-term memory

Primary references

Reference definition