[LeslieWylie]

Closes #15132.

Scope and coordination are being discussed in #15132; maintainer acknowledgment is still pending.

MiniMax-H3 keeps the same packed sequence across transformer blocks, but Ulysses Anything gathers shard lengths again for the Q, K, V and output exchanges in each attention call.

With attention_kwargs={"reuse_ulysses_metadata": True}, this change builds one shard-length plan per inference forward and passes it across those blocks. The plan uses a per-call copy of ParallelConfig, so it does not persist between requests. The token refiner keeps its existing path, and gradient-enabled use is rejected. All ranks must use the same setting and receive the same full packed layout.

CPU/Gloo checks were run on base d42ebf9c6e0b46103b74283f5324357a67acaa04. A real two-process tiny H3 model matched default outputs bitwise for lengths 18 → 20 → 18 while making fewer all-gathers. An uneven attention check covered 9 → 10 → 9; mismatched full layouts raised on both ranks, and the original configuration stayed unchanged. Affected-file style and consistency checks passed.

DIFFUSERS_TEST_DEVICE=cpu PYTEST_DISABLE_PLUGIN_AUTOLOAD=1 OMP_NUM_THREADS=1 \
  python -m pytest tests/models/transformers/test_models_transformer_minimax_h3.py::TestMiniMaxH3SequencePlan -q

Result: the focused CPU/Gloo checks passed at the base above.

CUDA/NCCL, torch.compile, full-checkpoint generation, isolated latency and output quality remain unverified. No speedup is claimed. The collective-order guarantee is limited to matching rank settings and H3's standard split plan.

#14834 shares lengths within one attention call and supports backward. This change shares a plan across H3 transformer layers; its lower-level helper changes will need reconciliation if that work lands first.

Self-review found no remaining blocking correctness issue or dead code. The opt-in is exercised through the real model, every forward starts with the same length collective on all ranks, and no rank-local cache can skip it.

AI assistance: Codex helped with implementation, CPU validation and this coordination update.

[LeslieWylie]

@codex review

In 19dd14b5, I reuse Ulysses shard lengths across transformer blocks within a single forward. I want the default results and token-refiner path to stay unchanged, so please check collective ordering and that the shard plan cannot carry over into another request.

The CPU/Gloo checks matched the default path for repeated 18/20/18 model inputs and uneven 9/10/9 attention inputs; mismatched full layouts raised on both ranks. CUDA/NCCL, compile and full-checkpoint generation remain unvalidated.

[github-actions]

Hi @LeslieWylie, thanks for the PR! It does not appear to link an issue it fixes. If this PR addresses an existing issue, please add a closing keyword (e.g. Fixes #1234) to the PR description so the issue is linked. See the contribution guide for more details. If this PR intentionally does not fix a tracked issue, a maintainer can add the no-issue-needed label to silence this reminder.

Please note that PRs without a linked issue are likely to be automatically closed 10 days after this notice.

Once the PR links an issue (or gets the no-issue-needed label), you can ignore this message — it stays here as a comment, but it no longer applies.