benchmarks / nemotron-3-nano-30b-a3b--dgx-spark-x1--vllm--bf16--p2048-o256-c1--2026-07-07

Nemotron 3 Nano 30B-A3B on NVIDIA DGX Spark

compare

28.7tok/s

461msttft

4.4kprefill

vLLM 0.24.0BF16Jul 7, 2026

Run configuration

nodes

batch size

—

concurrency

context

10240

prompt / output

2042 / 256

weights nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B-BF16 ↗

Metrics

decode tok/s

28.7

prefill tok/s

4.4k

ttft

461ms

total tok/s

27.4

inter-token

35ms

e2e latency

9.34s

req/s

0.107

peak memory

83.5 GB

avg power

28.6 W

energy/req

267 J

Reproduce

PATH="$HOME/venvs/vllm/bin:$PATH" VLLM_USE_DEEP_GEMM=0 TORCHINDUCTOR_COMPILE_THREADS=2 MAX_JOBS=4 vllm serve ~/models/hf/Nemotron-3-Nano-30B-A3B-BF16 --served-model-name Nemotron-3-Nano-30B-A3B-BF16 --max-model-len 10240 --gpu-memory-utilization 0.62 --max-num-seqs 8 --limit-mm-per-prompt '{"image": 0, "video": 0}' --port 8000
python3 bench/harness.py bench/configs/nemotron-3-nano-30b-a3b-bf16--vllm.json --rows-out data/benchmarks/nemotron-3-nano-30b-a3b.json

Unquantized BF16 baseline of Nemotron-3-Nano (nemotron_h hybrid Mamba-Transformer MoE, ~3B active), vLLM on GB10. Precision baseline vs the NVFP4 row.