* segment sizes marked with an asterisk are estimates pending a measured run
Estimated per-node footprint, weights + KV sharded across the Spark (tensor parallel). Overhead ~8 GB/node; KV at FP16.
Optimize
7B params, precision varies by file — ~7 GB at 8-bit fits on one Spark.
Make it fit
Get the weights (and headroom for KV) inside the 128 GB pool — the hard gate.
Pick the right GGUF quant
The repo already ships multiple GGUF quant levels — grab the largest that fits under 110 GB instead of quantizing yourself.
~7 GB target
Make it fast
Once it fits: fewer bytes per token and fewer decode steps against 273 GB/s.
Add a speculative decoder
Decode is bandwidth-bound at ~38.4 tok/s ceiling; a trained EAGLE-style draft multiplies tokens per weight-read.
Make it yours
Change what the model does — adapt, edit, or steer it — independent of size.
Fine-tune it
LoRA/QLoRA adapts behavior on one Spark without touching the base weights.
Edit or steer it
One-shot weight math — no gradients, no corpus.