howtospark
models / unsloth/Qwen3.8-27B-GGUF

27B paramsbfloat16GQA 24:4262K ctxmultimodalapache-2.0GGUF
Sparks
Users
~71 GB
~48 GB
~46 GB
~45 GB
~42 GB
~41 GB
~40 GB
~39 GB
~18 GB
110 usable
Spec decode
Context33K / 262KKV
On the Sparks — 1× · MTP · 33K ctx · FP16 KV~384K ctx fits usable · ~11 users fit
spark
92 GB free
110 usable
18* / 110 GB
16% of usable
Dense weightsKV cacheActivations + graphsOS reserve

* segment sizes marked with an asterisk are estimates pending a measured run

Estimated per-node footprint, weights + KV sharded across the Spark (tensor parallel). Overhead ~8 GB/node; KV at FP16.

layer map
64 layerslinear attention×48full attention×16
in
out
all 64 layers fire per token — 27.3B active
Optimize
Quantized from

Qwen/Qwen3.8-27B

Analyze the original repo for the full architecture picture.

Verdict

27B params, ~55 GB at bfloat16 — fits on one Spark as-is.

Make it fit

Get the weights (and headroom for KV) inside the 128 GB pool — the hard gate.

Lossless compression

BF16 weights entropy-code ~30% smaller bit-exact — smaller downloads and less bandwidth per token, no quality question at all.

Make it fast

Once it fits: fewer bytes per token and fewer decode steps against 273 GB/s.

Add a speculative decoder

Decode is bandwidth-bound at ~5 tok/s ceiling; a trained EAGLE-style draft multiplies tokens per weight-read.

Tame the KV cache

At 262144 tokens the KV cache alone is ~69 GB at FP16; quantize or evict.

Make it yours

Change what the model does — adapt, edit, or steer it — independent of size.

Fine-tune it

LoRA/QLoRA adapts behavior on one Spark without touching the base weights.