benchmarks / kimi-linear-48b-a3b--dgx-spark-x1--vllm--bf16--p2048-o256-c1--2026-07-09

Kimi Linear 48B-A3B on NVIDIA DGX Spark

compare

29.8tok/s

978msttft

2.1kprefill

vLLM 0.24.0BF16Jul 9, 2026

Run configuration

nodes

batch size

—

concurrency

context

10240

prompt / output

2043 / 256

weights moonshotai/Kimi-Linear-48B-A3B-Instruct ↗

Metrics

decode tok/s

29.8

prefill tok/s

2.1k

ttft

978ms

total tok/s

26.8

inter-token

34ms

e2e latency

9.55s

req/s

0.105

peak memory

47.4 GB

avg power

63 W

energy/req

601.9 J

Reproduce

PATH="$HOME/venvs/vllm/bin:$PATH" VLLM_USE_DEEP_GEMM=0 TORCHINDUCTOR_COMPILE_THREADS=2 MAX_JOBS=4 vllm serve ~/models/hf/Kimi-Linear-48B-A3B-Instruct --served-model-name Kimi-Linear-48B-A3B-Instruct --max-model-len 10240 --gpu-memory-utilization 0.85 --max-num-seqs 8 --trust-remote-code --host 0.0.0.0 --port 8000
python3 bench/harness.py bench/configs/kimi-linear-48b-a3b-bf16--vllm.json --rows-out data/benchmarks/kimi-linear-48b-a3b.json

Kimi Linear (KDA hybrid linear attention, 3:1 KDA:MLA, ~3B active of 48B). BF16 weights ~92 GiB — near the GB10 unified-memory ceiling; viable because the linear-attention KV state is tiny. Served on the second Spark (spark-c84b), harness run over the 200G fabric.