Training
NVFP4 quantization
FP4 microscalingPlannedNVIDIA FP4, block 16 with two-level scaling — best perf/accuracy on Blackwell.
On the Spark
The on-device target for Spark/5090: ~1.6× throughput over BF16, ~41% less energy. Our next quant pass for the pruned Hy3.
- Relieves
- CapacityBandwidthCompute
- Targets
- Weights · Activations
- Format
- NVFP4 (E2M1, block 16, two-level scale)
- Granularity
- Microscaling — block 16 + per-tensor FP32
- Lifecycle
- PTQ · no gradients
- Calibration
- Small calibration set
- Compression
- ~4× vs BF16
- Quality
- 98–99% of FP16 with MR-GPTQ calibration
- Hardware
- Blackwell tensor cores (RTX 5090, B200, GB10) for FP4 throughput; emulated elsewhere
- Runtimes
- vLLMTensorRT-LLM
A worked Spark recipe for this method hasn't been written yet — it lives here as a reference point in the ontology. It's on the list.