Training
Medusa heads
Accelerate decodingSpeculative decodingReferenceBolt several extra prediction heads onto the model to guess the next few tokens in parallel — no separate draft model.
- Objective
- Accelerate decoding
- Relieves
- Bandwidth
- Format
- Extra decoding heads on the base model
- Granularity
- Parallel token heads + tree attention
- Lifecycle
- QAT / QAD · needs gradients
- Calibration
- Small calibration set
- Compression
- ~2–3× decode throughput
- Quality
- 100% — verification keeps only tokens the target model would emit
- Hardware
- Trained Medusa heads; tree-attention verify kernel
- Runtimes
- vLLMTensorRT-LLM
A worked Spark recipe for this method hasn't been written yet — it lives here as a reference point in the ontology.