← Quit0 / 0 correct
How does speculative decoding speed up generation without changing the output distribution?
draft model: proposes t1 t2 t3 t4 target model: verifies all 4 in ONE pass accept t1 t2 ✓, reject t3 ✗ resample t3 from target, continue
How does speculative decoding speed up generation without changing the output distribution?
draft model: proposes t1 t2 t3 t4 target model: verifies all 4 in ONE pass accept t1 t2 ✓, reject t3 ✗ resample t3 from target, continue