Training
GPTQ
QuantizationReferenceHessian-based layerwise error compensation; the classic 4-bit weight quant.
- Relieves
- CapacityBandwidth
- Targets
- Weights
- Format
- INT4 / INT3 (W4A16)
- Granularity
- Per-group
- Lifecycle
- PTQ · no gradients
- Calibration
- Small calibration set
- Compression
- ~4×
- Quality
- Near-FP16 at 4-bit
- Hardware
- INT4 dequant kernels; broad support
- Runtimes
- vLLMTensorRT-LLMSGLangTransformers
A worked Spark recipe for this method hasn't been written yet — it lives here as a reference point in the ontology.