benchmarks / qwen3-6-35b-a3b--dgx-spark-x1--vllm--fp8--p2048-o256-c1--2026-07-06

Qwen3.6 35B-A3B on NVIDIA DGX Spark

compare

52.3tok/s

389msttft

5.2kprefill

vLLM 0.24.0FP8Jul 6, 2026

Run configuration

nodes

batch size

—

concurrency

context

10240

prompt / output

2041 / 256

weights Qwen/Qwen3.6-35B-A3B-FP8 ↗

Metrics

decode tok/s

52.3

prefill tok/s

5.2k

ttft

389ms

total tok/s

48.7

inter-token

19ms

e2e latency

5.26s

req/s

0.19

peak memory

69.1 GB

avg power

28.1 W

energy/req

147.8 J

Reproduce

PATH="$HOME/venvs/vllm/bin:$PATH" VLLM_USE_DEEP_GEMM=0 TORCHINDUCTOR_COMPILE_THREADS=2 MAX_JOBS=4 vllm serve ~/models/hf/Qwen3.6-35B-A3B-FP8 --served-model-name Qwen/Qwen3.6-35B-A3B-FP8 --max-model-len 10240 --gpu-memory-utilization 0.5 --max-num-seqs 8 --limit-mm-per-prompt '{"image": 0, "video": 0}' --port 8000
python3 bench/harness.py bench/configs/qwen3-6-35b-a3b-fp8--vllm.json --rows-out data/benchmarks/qwen3-6-35b-a3b.json

Official Qwen FP8 checkpoint, vLLM on GB10. Completes the Qwen3.6 precision trio vs BF16 and NVFP4.