Training
Draft-model speculative decoding
Accelerate decodingSpeculative decodingReferenceA small, fast draft model proposes tokens; the big model verifies a whole chunk in one pass — the original speculative decoding.
- Objective
- Accelerate decoding
- Relieves
- Bandwidth
- Format
- Small draft model + target verify
- Granularity
- Token-level draft-then-verify
- Lifecycle
- PTQ · no gradients
- Calibration
- No calibration
- Compression
- ~1.5–3× decode throughput
- Quality
- 100% — verification keeps only tokens the target model would emit
- Hardware
- Runs a second (small) model alongside the target
- Runtimes
- vLLMTensorRT-LLMSGLangllama.cpp
A worked Spark recipe for this method hasn't been written yet — it lives here as a reference point in the ontology.