Training
Token eviction (H2O, StreamingLLM)
KV compressionReferenceDrop low-impact tokens from the KV cache to cap long-context memory.
- Relieves
- KV cache
- Targets
- KV cache
- Format
- Sparse KV (evict tokens)
- Granularity
- Per-token
- Lifecycle
- PTQ · no gradients
- Calibration
- No calibration
- Compression
- Bounded KV regardless of length
- Quality
- Task-dependent
- Hardware
- Runtime eviction support
- Runtimes
- SGLangvLLM
A worked Spark recipe for this method hasn't been written yet — it lives here as a reference point in the ontology.