Training
Partial layer offload
Offload / distributeOffloadingReferencePin the first N layers on the GPU and run the rest from host RAM (llama.cpp --n-gpu-layers, accelerate device_map).
On the Spark
The commodity path when a model just barely overflows GPU memory on a discrete-GPU box; on unified Spark memory it's moot.
- Objective
- Offload / distribute
- Relieves
- Capacity
- Targets
- Weights
- Format
- Static GPU/CPU layer split
- Granularity
- Per-layer (N layers on GPU, rest on host)
- Lifecycle
- PTQ · no gradients
- Calibration
- No calibration
- Compression
- None — relocates, doesn't shrink
- Quality
- 100% — same weights, bit-exact
- Hardware
- Host RAM for the offloaded layers; PCIe bandwidth bounds decode
- Runtimes
- llama.cppTransformers
A worked Spark recipe for this method hasn't been written yet — it lives here as a reference point in the ontology.