Training
AWQ
Reduce model sizeQuantizationReferenceProtects the ~1% salient weights by activation scale; best-in-class INT4.
On the Spark
A solid default for W4A16 serving on vLLM.
- Objective
- Reduce model size · Lossy
- Relieves
- CapacityBandwidth
- Targets
- Weights
- Format
- INT4 (W4A16)
- Granularity
- Per-group (e.g. 128)
- Lifecycle
- PTQ · no gradients
- Calibration
- Small calibration set
- Compression
- ~4×
- Quality
- ≈FP16; beats GPTQ at the same bit-width
- Hardware
- Activation-aware INT4 kernels; broad GPU support
- Runtimes
- vLLMTensorRT-LLMSGLang
In the wild
AngelSlim AWQ pipeline (Tencent)
Turnkey W4A16 AWQ for the major open families, exporting directly to vLLM and SGLang.
A worked Spark recipe for this method hasn't been written yet — it lives here as a reference point in the ontology.