benchmarks / qwen3-5-27b--dgx-spark-x1--vllm--bf16--p2048-o256-c1--2026-07-06

Qwen3.5 27B on NVIDIA DGX Spark

compare

4.5tok/s

1.53sttft

1.3kprefill

vLLM 0.24.0BF16Jul 6, 2026

Run configuration

nodes

batch size

—

concurrency

context

10240

prompt / output

2040 / 256

weights Qwen/Qwen3.5-27B ↗

Metrics

decode tok/s

4.5

prefill tok/s

1.3k

ttft

1.53s

total tok/s

4.4

inter-token

222ms

e2e latency

58.13s

req/s

0.017

peak memory

87.7 GB

avg power

34 W

energy/req

1976.4 J

Reproduce

PATH="$HOME/venvs/vllm/bin:$PATH" VLLM_USE_DEEP_GEMM=0 vllm serve ~/models/hf/Qwen3.5-27B --served-model-name Qwen/Qwen3.5-27B --max-model-len 10240 --gpu-memory-utilization 0.65 --port 8000
python3 bench/harness.py bench/configs/qwen3-5-27b-bf16--vllm.json --rows-out data/benchmarks/qwen3-5-27b.json

Unquantized baseline (original safetensors), vLLM on GB10