benchmarks / qwen3-6-35b-a3b-heretic--dgx-spark-x1--vllm--nvfp4--p2048-o256-c1--2026-07-06

Qwen3.6 35B-A3B Heretic (uncensored) on NVIDIA DGX Spark

compare

41.5tok/s

315msttft

6.5kprefill

vLLM 0.24.0NVFP4Jul 6, 2026

Run configuration

nodes

batch size

—

concurrency

context

10240

prompt / output

2041 / 256

weights AEON-7/Qwen3.6-35B-A3B-heretic-NVFP4 ↗

Metrics

decode tok/s

41.5

prefill tok/s

6.5k

ttft

315ms

total tok/s

39.6

inter-token

24ms

e2e latency

6.46s

req/s

0.155

peak memory

67.5 GB

avg power

28.9 W

energy/req

186.8 J

Reproduce

PATH="$HOME/venvs/vllm/bin:$PATH" VLLM_USE_DEEP_GEMM=0 TORCHINDUCTOR_COMPILE_THREADS=2 MAX_JOBS=4 vllm serve ~/models/hf/Qwen3.6-35B-A3B-heretic-NVFP4 --served-model-name Qwen3.6-35B-A3B-heretic-NVFP4 --max-model-len 10240 --gpu-memory-utilization 0.5 --max-num-seqs 8 --limit-mm-per-prompt '{"image": 0, "video": 0}' --port 8000
python3 bench/harness.py bench/configs/qwen3-6-35b-a3b-heretic-nvfp4--vllm.json --rows-out data/benchmarks/qwen3-6-35b-a3b-heretic.json

Community NVFP4 quant (AEON-7) of an abliterated Qwen3.6-35B-A3B fine-tune. Same format/arch as the official NVIDIA NVFP4; community-vs-official recipe comparison.