benchmarks / qwen3-5-35b-a3b--dgx-spark-x1--llama-cpp--mxfp4-moe--p2048-o256-c1--2026-07-06

Qwen3.5 35B-A3B on NVIDIA DGX Spark

compare

65.4tok/s

983msttft

2.2kprefill

llama.cpp b1-2da6686MXFP4_MOEJul 6, 2026

Run configuration

nodes

batch size

—

concurrency

context

10240

prompt / output

2040 / 256

weights unsloth/Qwen3.5-35B-A3B-GGUF ↗

Metrics

decode tok/s

65.4

prefill tok/s

2.2k

ttft

983ms

total tok/s

52.3

inter-token

15ms

e2e latency

4.90s

req/s

0.204

peak memory

28.9 GB

avg power

41.8 W

energy/req

204.7 J

Reproduce

~/Dev/llama.cpp/build/bin/llama-server -m ~/models/gguf/Qwen3.5-35B-A3B-MXFP4_MOE.gguf -c 40960 --parallel 4 -ngl 99 -fa on --host 127.0.0.1 --port 8080
python3 bench/harness.py bench/configs/qwen3-5-35b-a3b--llama-cpp--mxfp4-moe.json --rows-out data/benchmarks/qwen3-5-35b-a3b.json

unsloth GGUF, llama.cpp CUDA build for GB10 (sm121), flash attention on, 4 slots