Training
bitsandbytes NF4 / LLM.int8
QuantizationReferenceZero-calibration load-time quant; NF4 is the base for QLoRA fine-tuning.
- Relieves
- Capacity
- Targets
- Weights
- Format
- NF4 / INT8
- Granularity
- Per-block (NF4 block 64)
- Lifecycle
- PTQ · no gradients
- Calibration
- No calibration
- Compression
- 2–4×
- Quality
- Good; NF4 fitted to a normal weight distribution
- Hardware
- CUDA; bitsandbytes kernels
- Runtimes
- Transformers
A worked Spark recipe for this method hasn't been written yet — it lives here as a reference point in the ontology.