benchmarks / devstral-small-2-24b--dgx-spark-x1--llama-cpp--q4-k-m--p2048-o256-c1--2026-07-06

Devstral Small 2 on NVIDIA DGX Spark

compare

13.5tok/s

1.56sttft

1.1kprefill

llama.cpp b15-ee445f9Q4_K_MJul 6, 2026

Run configuration

nodes

batch size

—

concurrency

context

10240

prompt / output

2079 / 256

weights unsloth/Devstral-Small-2-24B-Instruct-2512-GGUF ↗

Metrics

decode tok/s

13.5

prefill tok/s

1.1k

ttft

1.56s

total tok/s

12.5

inter-token

74ms

e2e latency

20.52s

req/s

0.049

peak memory

31.1 GB

avg power

42.6 W

energy/req

873.2 J

Reproduce

~/Dev/llama.cpp/build/bin/llama-server -m ~/models/gguf/Devstral-Small-2-24B-Instruct-2512-Q4_K_M.gguf -c 40960 --parallel 4 -ngl 99 -fa on --host 127.0.0.1 --port 8080
python3 bench/harness.py bench/configs/devstral-small-2-24b--llama-cpp--q4-k-m.json --rows-out data/benchmarks/devstral-small-2-24b.json

unsloth GGUF, llama.cpp CUDA build for GB10 (sm121), flash attention on, 4 slots. Dense 24B coding model (mistral3 arch).