benchmarks / qwen3-5-35b-a3b--dgx-spark-x1--llama-cpp--q8-0--p2048-o256-c1--2026-07-06

Qwen3.5 35B-A3B on NVIDIA DGX Spark

compare

56.1tok/s

1.26sttft

1.7kprefill

llama.cpp b1-2da6686Q8_0Jul 6, 2026

Run configuration

nodes

batch size

—

concurrency

context

10240

prompt / output

2041 / 256

weights unsloth/Qwen3.5-35B-A3B-GGUF ↗

Metrics

decode tok/s

56.1

prefill tok/s

1.7k

ttft

1.26s

total tok/s

44.2

inter-token

18ms

e2e latency

5.82s

req/s

0.172

peak memory

43.3 GB

avg power

37.3 W

energy/req

216.2 J

Reproduce

~/Dev/llama.cpp/build/bin/llama-server -m ~/models/gguf/Qwen3.5-35B-A3B-Q8_0.gguf -c 40960 --parallel 4 -ngl 99 -fa on --host 127.0.0.1 --port 8080
python3 bench/harness.py bench/configs/qwen3-5-35b-a3b--llama-cpp--q8-0.json --rows-out data/benchmarks/qwen3-5-35b-a3b.json

unsloth GGUF, llama.cpp CUDA build for GB10 (sm121), flash attention on, 4 slots