benchmarks / hy3-reap-48e--dgx-spark-x2--vllm--bf16--p2048-o256-c1--2026-07-09
Hy3 REAP-48e (Sapid Labs) on NVIDIA DGX Spark ×2
compare4.8tok/s
3.79sttft
539prefill
vLLM 0.24.0BF16Jul 9, 2026
Run configuration
nodes
2
batch size
—
concurrency
1
context
10240
prompt / output
2042 / 256
weights sapidlabs/Hy3-REAP-48e ↗
Metrics
decode tok/s
4.8
prefill tok/s
539
ttft
3.79s
total tok/s
4.5
inter-token
206ms
e2e latency
56.38s
req/s
0.018
peak memory
109.1 GB
avg power
23.1 W
energy/req
1302.7 J
Reproduce
PATH="$HOME/venvs/vllm/bin:$PATH" RAY_memory_monitor_refresh_ms=0 NCCL_IB_DISABLE=1 NCCL_SOCKET_IFNAME=enp1s0f1np1 GLOO_SOCKET_IFNAME=enp1s0f1np1 VLLM_HOST_IP=192.168.100.1 vllm serve ~/models/hf/Hy3-REAP-48e-v2 --served-model-name hy3-reap-48e-v2 --dtype bfloat16 --max-model-len 10240 --enforce-eager --distributed-executor-backend ray --pipeline-parallel-size 2 --gpu-memory-utilization 0.8 --max-num-seqs 4 --host 0.0.0.0 --port 8000 python3 bench/harness.py bench/configs/hy3-reap-48e-bf16--vllm-pp2.json --rows-out data/benchmarks/hy3-reap-48e.json
Our REAP75 prune of Hunyuan Hy3 (295B -> 157 GB BF16, 48/192 experts kept, calibrated on 2048 code samples across both Sparks). Too big for one GB10 in BF16 -- served pipeline-parallel across two Sparks over the 200G fabric (Ray, enforce-eager). First two-node row on the site.