Inference & performance · Concept

Speculative decoding

Use a cheaper draft mechanism to propose tokens that a target model verifies.