Inference & performance · Concept

Paged attention

Manage attention KV-cache memory in blocks to improve allocation and sharing efficiency.