Training
Dynamic quantization (importance-aware)
Reduce model sizeQuantizationPlannedSpend bits where they hurt — nominal 1-bit quants that are really ~2.3 bpw in the right places.
On the Spark
How a 753B model becomes a 217 GB artifact: UD-IQ1_S keeps attention at 4–6 bit and crushes the cold experts.
- Objective
- Reduce model size · Lossy
- Relieves
- CapacityBandwidth
- Targets
- Weights
- Format
- Mixed 1–8 bit per tensor (UD-IQ1_S … UD-Q8_K_XL)
- Granularity
- Per-tensor bit allocation by measured importance
- Lifecycle
- PTQ · no gradients
- Calibration
- Small calibration set
- Compression
- 4–7×
- Quality
- Far above uniform quants at equal size; cliff below ~2 effective bits/weight
- Hardware
- None — CPU/GPU split, runs on any silicon
- Runtimes
- llama.cpp
In the wild
Unsloth dynamic GGUFs
Heterogeneous per-layer GGUF quants of frontier open models — aggressive bits where it's safe, more where it hurts.
Unsloth GLM-5.2 UD-IQ1_S
753B → 216.7 GB (2.30 effective bits/weight) — the measured basis for the optimizer's 1-bit GGUF row.
A worked Spark recipe for this method hasn't been written yet — it lives here as a reference point in the ontology. It's on the list.