Training
AWQ
QuantizationReferenceProtects the ~1% salient weights by activation scale; best-in-class INT4.
On the Spark
A solid default for W4A16 serving on vLLM.
- Relieves
- CapacityBandwidth
- Targets
- Weights
- Format
- INT4 (W4A16)
- Granularity
- Per-group (e.g. 128)
- Lifecycle
- PTQ · no gradients
- Calibration
- Small calibration set
- Compression
- ~4×
- Quality
- ≈FP16; beats GPTQ at the same bit-width
- Hardware
- Activation-aware INT4 kernels; broad GPU support
- Runtimes
- vLLMTensorRT-LLMSGLang
A worked Spark recipe for this method hasn't been written yet — it lives here as a reference point in the ontology.