Inference & performance · Concept
Speculative decoding
Use a cheaper draft mechanism to propose tokens that a target model verifies.
Related terms
Primary references
Reference definition
Inference & performance · Concept
Use a cheaper draft mechanism to propose tokens that a target model verifies.
Reference definition