benchmarks / qwen3-5-35b-a3b-fp8--dgx-spark-x1--vllm--fp8--p2048-o256-c1--2026-07-05

Qwen3.5 35B-A3B on NVIDIA DGX Spark

compare

53.9tok/s

388msttft

5.3kprefill

vLLM 0.24.0FP8Jul 5, 2026

Run configuration

nodes

batch size

—

concurrency

context

10240

prompt / output

2042 / 256

weights Qwen/Qwen3.5-35B-A3B-FP8 ↗

Metrics

decode tok/s

53.9

prefill tok/s

5.3k

ttft

388ms

total tok/s

50.0

inter-token

19ms

e2e latency

5.12s

req/s

0.195

peak memory

70.2 GB

avg power

27.8 W

energy/req

142.3 J

Reproduce

PATH="$HOME/venvs/vllm/bin:$PATH" VLLM_USE_DEEP_GEMM=0 vllm serve ~/models/hf/Qwen3.5-35B-A3B-FP8 --served-model-name Qwen/Qwen3.5-35B-A3B-FP8 --max-model-len 10240 --gpu-memory-utilization 0.5 --port 8000
python3 bench/harness.py bench/configs/qwen3-5-35b-a3b-fp8--vllm.json --rows-out data/benchmarks/qwen3-5-35b-a3b-fp8.json

Official Qwen FP8 checkpoint, vLLM on GB10 (native Blackwell FP8)