howtospark
models / higurashi-ai/Haru-4B
?

higurashi-ai/Haru-4B
5.3B paramsbfloat16apache-2.0
Sparks
~19 GB
~16 GB
~13 GB
~11 GB
~10 GB
110 usable
Spec decode
On the Sparks — 1× · 1-bit GGUF (dynamic)
spark
101 GB free
110 usable
10* / 110 GB
9% of usable
Dense weightsActivations + graphsOS reserve

* segment sizes marked with an asterisk are estimates pending a measured run

Estimated per-node footprint, weights + KV sharded across the Spark (tensor parallel). Overhead ~8 GB/node; KV at FP16.

Optimize
Verdict

5B params, ~11 GB at bfloat16 — fits on one Spark as-is.

Make it fit

Get the weights (and headroom for KV) inside the 128 GB pool — the hard gate.

Lossless compression

BF16 weights entropy-code ~30% smaller bit-exact — smaller downloads and less bandwidth per token, no quality question at all.

Make it fast

Once it fits: fewer bytes per token and fewer decode steps against 273 GB/s.

Add a speculative decoder

Decode is bandwidth-bound at ~25.9 tok/s ceiling; a trained EAGLE-style draft multiplies tokens per weight-read.

Make it yours

Change what the model does — adapt, edit, or steer it — independent of size.

Fine-tune it

LoRA/QLoRA adapts behavior on one Spark without touching the base weights.