* segment sizes marked with an asterisk are estimates pending a measured run
Estimated per-node footprint, weights + KV sharded across the Spark (tensor parallel). Overhead ~8 GB/node; KV at FP16.
one MoE layer · 384 experts, 6+1 fire per token · showing 256
each expert ≈ 12M params
resident: 40 MoE layers × 384 experts × 12M ≈ 190.7B
active/token: 40 × 6 × 12M ≈ 8.5B experts
+ 5.9B always-on dense (attention, embeddings, shared stack) = 14.4B active
Optimize
deepseek-ai/DeepSeek-V4.1-Flash-1017
Analyze the original repo for the full architecture picture.
197B params, ~197 GB at already fp8-quantized — doesn't fit natively, but ~98 GB at 4-bit does.
Make it fast
Once it fits: fewer bytes per token and fewer decode steps against 273 GB/s.
Use its MTP heads
Ships 3 multi-token-prediction layer(s) — self-speculative decode with no separate draft model; check vLLM/SGLang support for this arch.
Prune experts for speed
Fewer experts ⇒ smaller working set ⇒ better cache/page behavior, even when it already fits.
Make it yours
Change what the model does — adapt, edit, or steer it — independent of size.
Fine-tune it
LoRA/QLoRA adapts behavior on one Spark without touching the base weights.
Edit or steer it
One-shot weight math — no gradients, no corpus.