benchmarks / qwen3-5-35b-a3b--dgx-spark-x1--llama-cpp--bf16--p2048-o256-c1--2026-07-06
Qwen3.5 35B-A3B on NVIDIA DGX Spark
compare31.5tok/s
2.68sttft
782prefill
llama.cpp b1-2da6686BF16Jul 6, 2026
Run configuration
nodes
1
batch size
—
concurrency
1
context
10240
prompt / output
2041 / 256
weights Qwen/Qwen3.5-35B-A3B ↗
Metrics
decode tok/s
31.5
prefill tok/s
782
ttft
2.68s
total tok/s
23.7
inter-token
32ms
e2e latency
10.82s
req/s
0.092
peak memory
75.2 GB
avg power
32.6 W
energy/req
352.5 J
Reproduce
~/Dev/llama.cpp/build/bin/llama-server -m ~/models/gguf/Qwen3.5-35B-A3B-BF16.gguf -c 40960 --parallel 4 -ngl 99 -fa on --host 127.0.0.1 --port 8080 python3 bench/harness.py bench/configs/qwen3-5-35b-a3b--llama-cpp--bf16.json --rows-out data/benchmarks/qwen3-5-35b-a3b.json
BF16 GGUF converted locally from official safetensors via convert_hf_to_gguf.py; same-weights engine comparison vs vLLM BF16 row. llama.cpp CUDA build for GB10, flash attention on, 4 slots.