Training
Multi-Token Prediction (MTP)
Accelerate decodingSpeculative decodingReferenceThe model is pretrained to emit several tokens at once (DeepSeek-style MTP), so it self-speculates — no draft model or extra head needed.
On the Spark
Ships inside the checkpoint (e.g. GestaltLabs' MTP-capable GGUF); free decode speedup wherever the engine supports MTP verify.
- Objective
- Accelerate decoding
- Relieves
- Bandwidth
- Format
- Built-in multi-token heads (self-speculation)
- Granularity
- Model predicts several next tokens itself
- Lifecycle
- Training-time · needs gradients
- Calibration
- No calibration
- Compression
- ~1.5–2× decode throughput
- Quality
- 100% — verification keeps only tokens the target model would emit
- Hardware
- MTP tensors baked in at pretraining; MTP-aware engine
- Runtimes
- vLLMSGLangllama.cpp
In the wild
DeepSeek-V3 MTP module
Ships a multi-token-prediction layer that runtimes use for self-speculative decode — no separate draft model needed.
vllm-Moet MTP decode
2.3–2.8 token acceptance with bit-deterministic greedy output under TP/PP — the speed layer on top of the 2-bit expert scheme.
A worked Spark recipe for this method hasn't been written yet — it lives here as a reference point in the ontology.