Repository navigation
Conversation
* 'main' of https://github.com/huggingface/diffusers: (327 commits) downcast MiniMax-H3 position_ids at the device transfer on fp64-less backends (huggingface#14767) Fix TaylorSeer cache crash when the feature shape changes between steps (huggingface#14831) [docs] Adapters (huggingface#14842) [LoRA] add LoRA training for Qwen Image 2.1 (huggingface#14808) Pin GitHub Actions to commit SHAs (huggingface#14853) Scope GITHUB_TOKEN permissions per job (huggingface#14852) test and docs for xet dedup based file sharing. (huggingface#14835) Add single file support for Qwen 2.1 (huggingface#14836) Fix Wuerstchen LoRA scheduler steps for distributed epochs (huggingface#14748) [docs] Fix legacy hf CLI references (diffusers) (huggingface#14830) [docs] Using diffusion pipelines (huggingface#14805) [docs] Fix docstring arguments that don't match the signatures (huggingface#14778) [docs] typo with hyperlink (huggingface#14826) Fix Ascend NPU RMSNorm and fused-attention mask shapes (huggingface#14288) [core] Fix component device placement for split-device pipelines (huggingface#14739) Fix flash/sage varlen prep under torch.compile with dynamic shapes (huggingface#14568) Synchronize the compute stream before offloading to disk (huggingface#14657) extend tests/others/test_cli_commands.py::TestRunCommand::test_group_… (huggingface#14697) Add SeaCache support for Cosmos3 pipelines (huggingface#14663) Add Qwen-Image 2.1 (huggingface#14804) ...
* 'main' of https://github.com/mingyi456/diffusers: Fix bug with VAE slicing in autoencoder_dc.py
* 'main' of https://github.com/huggingface/diffusers: [tests] remove `test_pipelines_common.py` (huggingface#14669) [chore] remove force_upcast from autoencoder flux2 (huggingface#14946) Configure fixed sampling sigmas on QwenImage21Pipeline (huggingface#14950) [tests] refactor the pipeline-level attention processor tests (huggingface#14701) [modular] Echo team joy future academy jd echo by @Gelercatty (huggingface#14941) Fix: Deprecating Siglip2ImageProcessorFast (huggingface#14781)
github-actions
Bot
added
size/S
Needed if we want to preserve diffusers format checkpoints.
|
/diffusers-bot style |
|
The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update. |
…/mingyi456/diffusers into animatextconditioner-single-file * 'animatextconditioner-single-file' of https://github.com/mingyi456/diffusers: (50 commits) Remove expired non_ema_revision deprecation blocks from training examples (huggingface#15118) Remove duplicated converter and fix import order Update src/diffusers/loaders/single_file_utils.py add fibo-2 LoRA training and loading (huggingface#15084) [ci] fix compilation CI. (huggingface#15092) [LoRA] apply .alpha to Qwen-Image lora_A/lora_B LoRAs (huggingface#15016) Use HF Trusted Publisher token in update_metadata workflow (huggingface#15096) [ci] skip bf16 precision test on T4. (huggingface#15091) [LTX-2] Add `motion_speed` to condition the positional time base apart from the playback frame rate (huggingface#14967) [ci] fix misnamed test steps in push_tests.yml (huggingface#15094) [ci] use python 3.11 when using metadata. (huggingface#15093) Fix model imports when torchao is older than 0.15 (huggingface#15095) [tests] xfail some kandinsky6 tests (huggingface#14977) Add RFD4-Proteina (huggingface#15088) Add Bria fibo-2 (huggingface#14991) Fix PyArrow compatibility in CPU CI (huggingface#15082) [feat] add KVAE-Audio autoencoder (huggingface#14384) move unpack out of the flux, krea2 and ltx decoders into the core denoise groups (huggingface#14737) post-release 0.41.0 (huggingface#15062) FLUX.3: match the released BFL reference image sizing (huggingface#15073) ...
There was a problem hiding this comment.
I think the way to fix this is to check if any of the tensor keys contain "llm_adapter", then we only take those keys (with the relevant prefixes stripped) and discard the rest.
If none of the keys contain "llm_adapter", we assume it is a diffusers-format file containing only the text conditioner weights and nothing else, and we just take all the tensors wholesale.
| for k in list(checkpoint.keys()): | ||
| if "llm_adapter." in k: | ||
| converted_state_dict[k.split("llm_adapter.", 1)[1]] = checkpoint.pop(k) | ||
| elif not k.startswith(("net.", "model.diffusion_model.")): |
There was a problem hiding this comment.
I think there may be an issue here. I understand the motivation is for a safetensors file that only contains the text conditioner tensors, without any llm_adapter prefix.
But I am very sure that an Anima single file model in comfyui format can come with no prefixes at all, and this code will not work in such a case. For example, if I take this file and remove the model.diffusion_model prefix, the llm_adapter keys will correspond to the actual text conditioner weights while those without any prefixes will be the diffusion model weights.
There was a problem hiding this comment.
I just took this file, stripped all the model.diffusion_model prefixes, and loaded it into ComfyUI, and I can confirm it does load correctly and produce identical outputs to the original file.
|
@DN6 Thanks for taking the time to look through my PR, and addressing the case that I neglected (diffusers style files). But honestly, I would prefer if you could bring it up as feedback for me to address by myself, because I would really like the opportunity to learn and correct my own mistakes. |
|
Wait a minute, according to my original PR code, since I neglected the case where I am trying to use And yet, the code over here was based on this erroneous implementation of the function, back when I was doing my own testing, and yet it was able to somehow load the weights? Is there some other bug elsewhere? |
Ok, I ran the code through a debugger, and it appears that this line of code checks to see if the conversion function is even necessary in the first place, and skips the conversion functions otherwise. So the mystery is solved for me. |
What does this PR do?
After checking with the help of a debugger, I have concluded that the issue I mentioned about the single file loading method producing different results should be due to a bug in the
FromOriginalModelMixin.from_single_file()code, not my own code.Fixes #14930
Before submitting
self-reviewskill on the diff?documentation guidelines, and
here are tips on formatting docstrings.
Who can review?
Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors who may be interested in your PR.
@DN6 @yiyixuxu