Training
Low-rank factorization
Low-rankReferenceSVD of weight matrices; LoRA adapters, or a low-rank error term paired with quant (SVDQuant, EoRA).
- Relieves
- Capacity
- Targets
- Weights
- Format
- Low-rank factors (rank r)
- Granularity
- SVD of weight matrices
- Lifecycle
- PTQ · no gradients
- Calibration
- Small calibration set
- Compression
- Modest standalone; a hybrid enabler
- Quality
- Often a low-rank term that absorbs quant error
- Hardware
- None special
- Runtimes
- vLLMTransformers
A worked Spark recipe for this method hasn't been written yet — it lives here as a reference point in the ontology.