benchmarks / qwen3-5-27b--dgx-spark-x1--vllm--fp8--p2048-o256-c1--2026-07-06

Qwen3.5 27B on NVIDIA DGX Spark

compare

8.2tok/s

1.03sttft

2.0kprefill

vLLM 0.24.0FP8Jul 6, 2026

Run configuration

nodes

batch size

—

concurrency

context

10240

prompt / output

2040 / 256

weights Qwen/Qwen3.5-27B-FP8 ↗

Metrics

decode tok/s

8.2

prefill tok/s

2.0k

ttft

1.03s

total tok/s

7.8

inter-token

122ms

e2e latency

32.12s

req/s

0.031

peak memory

72.4 GB

avg power

30.3 W

energy/req

993 J

Reproduce

PATH="$HOME/venvs/vllm/bin:$PATH" VLLM_USE_DEEP_GEMM=0 vllm serve ~/models/hf/Qwen3.5-27B-FP8 --served-model-name Qwen/Qwen3.5-27B-FP8 --max-model-len 10240 --gpu-memory-utilization 0.5 --port 8000
python3 bench/harness.py bench/configs/qwen3-5-27b-fp8--vllm.json --rows-out data/benchmarks/qwen3-5-27b.json

Official Qwen FP8 checkpoint, vLLM on GB10 (native Blackwell FP8)