benchmarks / gemma-4-26b-a4b-it--dgx-spark-x1--llama-cpp--q4-k-m--draft-f16--p2048-o256-c1--2026-07-06

Gemma 4 26B-A4B on NVIDIA DGX Spark

compare

33.9tok/s

1.08sttft

2.6kprefill

llama.cpp b15-ee445f9Q4_K_Mspec-decodingJul 6, 2026

Run configuration

nodes

batch size

—

concurrency

context

10240

prompt / output

2036 / 256

weights unsloth/gemma-4-26B-A4B-it-GGUF ↗draft model google/gemma-4-26B-A4B-it-assistant ↗

Metrics

decode tok/s

33.9

prefill tok/s

2.6k

ttft

1.08s

total tok/s

31.2

inter-token

30ms

e2e latency

8.41s

req/s

0.122

peak memory

26.1 GB

avg power

41.5 W

energy/req

340.8 J

temperature

draftAcceptanceRate

0.179

Reproduce

~/Dev/llama.cpp/build/bin/llama-server -m ~/models/gguf/gemma-4-26B-A4B-it-UD-Q4_K_M.gguf -c 10240 --parallel 1 -ngl 99 -fa on --host 127.0.0.1 --port 8080 -md ~/models/gguf/gemma-4-assistant-F16.gguf --spec-type draft-mtp --spec-draft-n-max 16 --spec-draft-n-min 1
python3 bench/harness.py bench/configs/gemma-4-26b-a4b-it--spec-draft.json --rows-out data/benchmarks/gemma-4-26b-a4b-it.json

spec-decoding profile: greedy (temp 0) with google/gemma-4-26B-A4B-it-assistant (0.42B, F16, converted locally from google safetensors) as draft, draft-max 16. Speedup is workload-dependent (acceptance rate in extraMetrics).