Load local ModularPipeline snapshots when modular_model_index.json still names Hub ids

[22elix3r]

What does this PR do?

Fixes #14640.

Motivation

ModularPipeline.from_pretrained(local_dir) / blocks.init_pipeline(local_dir) followed by load_components() still resolved components against the Hub id stored in modular_model_index.json. A complete local snapshot of MiniMax-H3 therefore re-downloaded ~134 GB (or left components as None offline), even though the files were already on disk.

This is the model_index.json vs modular_model_index.json asymmetry: the standard-repo fallback already binds repo to the local path; the modular index is taken verbatim, including published Hub ids.

Root cause

ModularPipeline.__init__ copies each from_pretrained spec from modular_model_index.json as-is. Published checkpoints such as MiniMax-H3 write "pretrained_model_name_or_path": "MiniMaxAI/MiniMax-H3" into every entry. When the caller passed a local directory, those Hub ids still won, so ComponentSpec.load() contacted the Hub.

The model_index.json fallback already does the right thing:

component_spec_dict = {
    "repo": pretrained_model_name_or_path,  # the local directory
    "subfolder": name,
    ...
}

The invariant that was missing: if the pipeline path is a local directory and <local_dir>/<subfolder> exists, load that component from the directory; otherwise keep the Hub id.

That second clause is required. Modular repos are allowed to be pointer repos (index only, weights elsewhere). Blindly rewriting every spec to the local directory would break those.

Solution

After specs are applied from either index, if pretrained_model_name_or_path is a local directory, rewrite a from_pretrained spec onto that directory only when:

• the spec is not already a local path (explicit local overrides stay put, including stale-path recovery when the recorded path no longer exists)
• the spec has a non-empty subfolder
• <local_dir>/<subfolder> exists on disk

Missing subfolders keep their Hub id (pointer repos / pruned snapshots). Hub from_pretrained("org/model") is unchanged because os.path.isdir("org/model") is false.

Not done: wrapping pin_memory/from_pretrained in try/except, MiniMax-specific branches, or changing how published indexes are written. save_pretrained(..., overwrite_modular_index=True) remains the way to persist local references.

Testing

Synthetic snapshot (no MiniMax weights, no GPU):

PYTHONPATH=src python -m pytest tests/modular_pipelines/test_modular_pipeline_loading.py::TestLocalModularSnapshotLoading -q
5 passed in 1.40s

Without the __init__ change the three remapping tests fail; the two "leave Hub id / leave existing local path" tests already passed on main.

PYTHONPATH=src python -m pytest tests/modular_pipelines/test_modular_pipeline_loading.py -q
17 passed in 111.45s

PYTHONPATH=src python -m pytest tests/modular_pipelines/test_modular_pipelines_custom_blocks.py -q -k "not slow and not nightly"
18 passed, 2 skipped in 9.63s

ruff check src/diffusers/modular_pipelines/modular_pipeline.py tests/modular_pipelines/test_modular_pipeline_loading.py
ruff format --check src/diffusers/modular_pipelines/modular_pipeline.py tests/modular_pipelines/test_modular_pipeline_loading.py
All checks passed; 2 files already formatted

New coverage:

• init_pipeline(local_dir) with Hub ids in the index and a present scheduler/ subfolder loads offline
• constructor path, same contract
• missing subfolder keeps the Hub id
• mixed snapshot rewrites only present components
• an existing local spec path is not overwritten by the snapshot directory

Performance

No extra copies and no extra Hub traffic on the common path. The new branch is an os.path.isdir scan of already-loaded specs, only when the pipeline path is a local directory. Hub loads are unchanged. Local snapshots avoid a second download.

Self-review

• Blocking: none.
• Left for review: English docs only (docs/source/en/modular_diffusers/modular_pipeline.md); the zh page was not updated. Specs remapped at init also flow into register_components / saved config — same as passing pretrained_model_name_or_path into load_components() today. overwrite_modular_index=True is still the way to retarget a save.
• Verdict: READY.

Related issue

Fixes #14640

Coordination: #14640 (open, unassigned, no linked PR at the time of this change). Investigation comment: #14640 (comment)

Before submitting

• Did you use an AI agent (Claude Code, Codex, Cursor, etc.) to help with this PR? If so:
  • Did you read the Coding with AI agents guide?
  • Did you run the self-review skill on the diff?
  • Did you share the final self-review notes in the PR description or a comment?
• Did you read the contributor guideline?
• Did you read our philosophy doc? (important for complex PRs)
• Was this discussed/approved via a GitHub issue or the forum? Please add a link to it if that's the case. ModularPipeline: loading from a local directory still resolves components against the hub id stored in modular_model_index.json (MiniMax-H3 re-downloads 134 GB) #14640 (investigation comment left; no maintainer ack yet — the issue was unassigned with no linked work)
• Did you make sure to update the documentation with your changes?
• Did you write any new necessary tests?

Who can review?

cc @yiyixuxu @apolinario

[Copilot]

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.