benchmarks / qwen3-5-35b-a3b--dgx-spark-x1--llama-cpp--q4-k-m--p2048-o256-c1--2026-07-05
Qwen3.5 35B-A3B on NVIDIA DGX Spark
compare68.3tok/s
1.03sttft
2.1kprefill
llama.cpp b1-2da6686Q4_K_MJul 5, 2026
Run configuration
nodes
1
batch size
—
concurrency
1
context
10240
prompt / output
2041 / 256
weights unsloth/Qwen3.5-35B-A3B-GGUF ↗
Metrics
decode tok/s
68.3
prefill tok/s
2.1k
ttft
1.03s
total tok/s
53.5
inter-token
15ms
e2e latency
4.79s
req/s
0.209
peak memory
28.5 GB
avg power
41.8 W
energy/req
199.9 J
Reproduce
~/Dev/llama.cpp/build/bin/llama-server -m ~/models/gguf/Qwen3.5-35B-A3B-Q4_K_M.gguf -c 40960 --parallel 4 -ngl 99 -fa on --host 127.0.0.1 --port 8080 python3 bench/harness.py bench/configs/qwen3-5-35b-a3b--llama-cpp--q4-k-m.json --rows-out data/benchmarks/qwen3-5-35b-a3b.json
unsloth GGUF, llama.cpp CUDA build for GB10 (sm121), flash attention on, 4 slots