* segment sizes marked with an asterisk are estimates pending a measured run
Estimated per-node footprint, weights + KV sharded across the Spark (tensor parallel). Overhead ~8 GB/node; KV at FP16.
one MoE layer · 256 experts, 8 fire per token
each expert ≈ 3M params
resident: 40 MoE layers × 256 experts × 3M ≈ 32.2B
active/token: 40 × 8 × 3M ≈ 1.0B experts
+ 3.7B always-on dense (attention, embeddings, shared stack) = 4.7B active
Eval scores (compare all)
Reported
Recipes (all recipes)
Worked deployments that run this model on DGX Spark — each shows how it fills each node's unified memory.
Qwen3.6 35B-A3B NVFP4 (Unsloth Fast)
Serve the full 262,144-token context of Qwen3.6 35B-A3B on ONE DGX Spark at 106 tok/s single-stream, using the checkpoint's own MTP head for speculative decode at k=3 — a 57% gain over the same config with no draft. Nothing needs patching and nothing is tight: the whole working set is 36 of 114 usable GiB.
Qwen3.6 35B-A3B — FP8
Qwen's own FP8 build of Qwen3.6 35B-A3B serves the full 262,144-token context on one DGX Spark in about 41 GiB and, unlike the NVFP4A16 build, it gets a real FP8 kernel path — vLLM picks the TRITON FP8 MoE backend, not the MARLIN fallback. It still decodes slower: 38.3 tok/s against 42.6 for NVFP4A16 and 106.5 for Unsloth's NVFP4-Fast. The reason is bytes, not kernels — the FP8 export carries 30.1 GiB of expert planes against roughly 17.5 — and like every official Qwen3.6 export it ships no MTP head.
Qwen3.6 35B-A3B — NVIDIA NVFP4
NVIDIA's official NVFP4 build of Qwen3.6 35B-A3B serves the full 262,144-token context on one DGX Spark — but it decodes at 42.6 tok/s against 106.5 for Unsloth's NVFP4-Fast build of the same base model. Two structural reasons, both measured here: it is weight-only NVFP4, so the FP4 tensor-core MoE kernels refuse it and vLLM falls back to MARLIN; and the export ships without the MTP head, so it cannot speculative-decode at all.
Optimize
36B params, ~72 GB at bfloat16 — fits on one Spark as-is.
Make it fit
Get the weights (and headroom for KV) inside the 128 GB pool — the hard gate.
Lossless compression
BF16 weights entropy-code ~30% smaller bit-exact — smaller downloads and less bandwidth per token, no quality question at all.
Make it fast
Once it fits: fewer bytes per token and fewer decode steps against 273 GB/s.
Add a speculative decoder
Decode is bandwidth-bound at ~28.8 tok/s ceiling; a trained EAGLE-style draft multiplies tokens per weight-read.
Prune experts for speed
Fewer experts ⇒ smaller working set ⇒ better cache/page behavior, even when it already fits.
Tame the KV cache
At 262144 tokens the KV cache alone is ~21 GB at FP16; quantize or evict.
Make it yours
Change what the model does — adapt, edit, or steer it — independent of size.
Fine-tune it
LoRA/QLoRA adapts behavior on one Spark without touching the base weights.
Edit or steer it
One-shot weight math — no gradients, no corpus.