[Pfannkuchensack]

Stacked on #10192. Until that merges, the diff here also shows its commit; the change of this PR is the last commit. Draft until #10192 is in.

Summary

Single-file loaders that widen float8 weights to bf16 kept the whole checkpoint file in memory for as long as the model stayed cached. safetensors.torch.load_file maps the file and every tensor is a slice of that mapping. The widened fp8 weights get new storage, but tensors already in the target dtype (norms, biases) stay views because .to(dtype) is a no-op for them, and those few views keep the entire mapping alive. The RAM cache does not count it.

Checkpoints whose .weight tensors include float8 are now read with safetensors' pread backend, where each tensor owns its storage. Files without float8 weights (bf16, fp16, int8, pure nvfp4) keep the mmap path unchanged.

• One rule in fp8_scaled.py: safetensors_backend(path) and load_safetensors_checkpoint(path).
• Used by the single-file loaders of FLUX.1, FLUX.2, FLUX.3, Krea-2, Qwen-Image, Qwen-Image-2.1, Z-Image, Anima, MOSS-Image, Ideogram 4 and ERNIE-Image. Also by the T5, CLIP, Mistral, Qwen2.5-VL, Qwen3, Qwen3.5 and Qwen3-VL encoder reads, including the Qwen3-VL vision tower and the key-filtered safe_open reads.
• FLUX.2 and FLUX.3 also drop the source dict once it is converted, so the load peak falls too.

Working set after load (GB), real loaders, Windows, CPU only:

Checkpoint	Model	Before	After
Qwen-Image Edit 2511 fp8	38.1	65.3	46.2
Mistral Small 3 (FLUX.2 dev encoder) fp8	32.3	50.5	35.1
Krea-2 Turbo fp8 scaled	23.9	36.8	25.2
FLUX.2 Klein 9B fp8	16.9	27.0	19.2
Qwen3-VL 8B fp8 scaled	14.1	23.7	18.3
Qwen2.5-VL 7B fp8 scaled	15.5	22.5	15.8
FLUX.2 Klein 4B fp8	7.2	11.4	8.0
Qwen3.5 4B fp8 (Anima-3.8B encoder)	7.7	11.0	7.7

FLUX.2 Klein 9B load peak: 26.4 -> 19.3 GB.

Related Issues / Discussions

Follow-up to #10192, which fixed the same pin for Wan 2.2.

QA Instructions

• Measurement: the real ModelLoader._load_model on CPU, reservations stubbed. Each run touches every parameter after the load and reads the process working set, then lists the mapped model-file regions that no parameter still uses.

• Before: main. After: this branch.

• Weights: a hash over every state-dict tensor (name, dtype, shape, bytes) is identical before and after for all 9 fp8 files and both mixed nvfp4 files (Qwen3-VL 4B and 8B).

• No-float8 controls are unchanged and still fully mapped: Krea-2 int8, FLUX.3 bf16, Anima-3.8B bf16.

• Mixed nvfp4 Qwen3-VL 8B: 5.5 -> 4.9 GB, with 2.2 GB of file no longer held.

• Kept fp8, where weights are not widened: same end state and bit-identical, but the file is read up front. Krea-2 load went from 0.1 s to 5 s.

• Load time with the file already in the OS cache, interleaved, one warm-up plus 3 runs, median:

  Checkpoint	Before	After
  Krea-2 fp8	16.4 s	21.7 s
  Qwen-Image Edit fp8	28.5 s	35.8 s
  Qwen3-VL 8B fp8	16.0 s	25.4 s

  pread copies what mmap only maps. A cold load from disk was not measured for this change. For fix(wan): keep system RAM within the model cache budget #10192, pread matched mmap on a cold 19 GiB read (462 against 431 MiB/s).

• Tests: 3645 passed, 170 skipped. The run covered tests/backend/model_manager/load, tests/backend/model_manager/configs, tests/test_model_manager, tests/test_calibrate_wan_vae_working_memory.py, and tests/backend/ quantization, flux3, anima, qwen3_5, moss_image. uv lock --check and Ruff are clean.

• New tests:

  • Real tiny float8 + bf16 files show that the surviving bf16 tensors no longer sit inside a file mapping, both for whole-file reads and for the Qwen3-VL tower, Qwen3.5 shards and MOSS reads.
  • Files without float8 weights stay on mmap.
  • A platform_independent AST guard fails if a loader that folds or casts fp8 goes back to raw load_file.

Not run: Linux, macOS, ROCm, GPU generation, family E2E. No local single-file Ideogram 4 or ERNIE fp8 file, so those two sites are covered by unit tests only. A pure nvfp4 file could not be measured, because the local FLUX.3 nvfp4 file is not identified on main either.

Review

Fixed during review:

• Missed read paths: Ideogram 4 single file, ERNIE raw fp8, the Qwen3-VL vision tower used by Krea-2 edit, Qwen3.5 shard reads, MOSS shard reads.
• FLUX.2 load peak: the source dict survived the conversion, so the peak did not fall.
• Trigger: it now ignores float8 block scales, so pure nvfp4 files keep mmap.
• Test seams stub the helper narrowly and keep the loud failure for a stale seam.

Known limits:

• Loading an fp8 checkpoint that is already in the OS file cache is slower, by the figures above.
• With FP8 Storage or fp8 compute on, nothing is widened, yet the file is now read up front. RAM is the same and loading is slower.
• safetensors holds the GIL while pread reads one tensor. On a slow disk other Python threads stall for that tensor's read: 1.2 GB tensors exist in Mistral and Qwen3-VL. Multiple GPUs feel this most.
• Components a loader drops, such as the vision towers of the Mistral and Qwen3-VL encoders or the bundled parts of an all-in-one FLUX.1 fp8 file, are read and then freed. The same goes for a file refused after the read. This stays within the reservation load_default.py makes for the file size before the load.
• On Linux the pinned pages were reclaimable file pages, so the gain there is likely smaller than on Windows. Not measured.
• Unchanged here:
  • the Ideogram 4 fp8 folder load, a separate cause: it peaks around 46 GB for a 17 GB model;
  • raw-fp8 ControlNet and adapter reads, for which no fp8 builds are published;
  • Wan, which already reads with pread since fix(wan): keep system RAM within the model cache budget #10192.

Checklist

• The PR has a short but descriptive title, suitable for a changelog
• Meaningful regression coverage added / updated where needed; obsolete tests/code removed
• Persisted-state and API changes include required migrations / compatibility validation
• Relevant performance/efficiency opportunities considered; material claims have evidence
• Material review findings resolved and relevant checks rerun
• Documentation added / updated (if applicable)
• Updated What's New copy (if doing a release after this PR)

---

^{Open in [code]smith · Need help on this PR? Tag @codesmith-bot with what you need.}