benchmarks / gemma-4-26b-a4b-it--dgx-spark-x1--llama-cpp--q4-k-m--p2048-o256-c1--2026-07-06

Gemma 4 26B-A4B on NVIDIA DGX Spark

compare

51.1tok/s

916msttft

2.7kprefill

llama.cpp b1-2da6686Q4_K_MJul 6, 2026

Run configuration

nodes

batch size

—

concurrency

context

10240

prompt / output

2036 / 256

weights unsloth/gemma-4-26B-A4B-it-GGUF ↗

Metrics

decode tok/s

51.1

prefill tok/s

2.7k

ttft

916ms

total tok/s

43.7

inter-token

20ms

e2e latency

5.86s

req/s

0.171

peak memory

26.5 GB

avg power

36.9 W

energy/req

216.2 J

Reproduce

~/Dev/llama.cpp/build/bin/llama-server -m ~/models/gguf/gemma-4-26B-A4B-it-UD-Q4_K_M.gguf -c 40960 --parallel 4 -ngl 99 -fa on --host 127.0.0.1 --port 8080
python3 bench/harness.py bench/configs/gemma-4-26b-a4b-it--llama-cpp--q4-k-m.json --rows-out data/benchmarks/gemma-4-26b-a4b-it.json

unsloth UD-Q4_K_M GGUF, llama.cpp CUDA build for GB10 (sm121), flash attention on, 4 slots. Google MoE rival to Qwen3.5-35B-A3B.