Inference & performance · Concept
Prefill-decode disaggregation
Manage input processing and token generation on separately allocated serving resources.
Related terms
Primary references
Reference definition
Inference & performance · Concept
Manage input processing and token generation on separately allocated serving resources.
Reference definition