Skip to content

fix(lora): identify SimpleTuner LyCORIS flat-key LoRAs - #10239

Open
Shanmukh-l33t wants to merge 1 commit into
invoke-ai:mainfrom
Shanmukh-l33t:fix/lycoris-simpletuner-lora-identification
Open

Shanmukh-l33t wants to merge 1 commit into
invoke-ai:mainfrom
Shanmukh-l33t:fix/lycoris-simpletuner-lora-identification

Conversation

@Shanmukh-l33t

@Shanmukh-l33t Shanmukh-l33t commented Nov 11, 2026 •

Copy link
Copy Markdown

Summary

SimpleTuner's LyCORIS training writes LoRA/LoKR/LoHA layers named by flattening the model's own diffusers module
path with underscores behind a lycoris_ prefix (lycoris_single_transformer_blocks_0_attn_to_k.lokr_w1), or
behind lycoris__. No probe recognized the layout, so the files installed as unknown on every base and forcing
the base failed in the converters (#10174's table). They now identify and convert on FLUX.1 (both spellings),
FLUX.2 Klein, SDXL, Z-Image and Wan, the way SD 3's own flat spelling already did.

  • A new lycoris_flat_lora_constants resolves each flattened name against the family's module list (module names
    contain underscores of their own, so only the whole list can unflatten them). A family is claimed on names only
    it holds — single_transformer_blocks.N.attn.to_q for FLUX.1, to_qkv_mlp_proj / ff.linear_in for FLUX.2
    Klein, the UNet's deep transformer_blocks for SDXL, layers.N.attention for Z-Image, blocks.N.attn1 for
    Wan — with the width check that keeps Z-Image off Ideogram 4 and SDXL's block depth that keeps it off SD 1/2.
    Names several families share without one only-this-family name identify nothing: SD 3's flat files stay SD 3
    (locked in by test_the_sd3_flat_layout_is_still_sd3), files of no family stay unknown.
  • The conversion maps the names back to diffusers paths and reuses each family's existing route: FLUX.1 through
    the diffusers-to-BFL mapping (fused QKV, swapped final modulation) unchanged, Wan through the kohya converter's
    lora_unet_blocks_<idx>_<tail> names, SDXL through the SD converter's flattened lora_unet_ keys. A key that
    names no module of the claimed family raises instead of being dropped.
  • Wan's variant detection reads the inner dimension from LoKR factors the way it reads a low-rank pair, so the
    issue's 5120-wide Wan file records its variant.

Before → after on the issue's files (key layouts captured from their safetensors headers):

Base File before after
FLUX.1 toxicwind/glassobject-lokr-flux unknown FLUX.1, converts
FLUX.1 (lycoris__) jimmycarter/flux-training-losercity-next-lycoris5 unknown FLUX.1, converts
FLUX.2 Klein gattaplayer/besch-flux2-klein-9b-lokr-lion-5e-6-bs2-ga2-v01 unknown FLUX.2 Klein, converts
SDXL davidrd123/Hiroshige-SDXL-LoKr unknown (Unrecognized SDXL LoRA key prefix when forced) SDXL, converts
Z-Image noodlecake/Retro_Anime_Painting_lokr-R8-LR2-HIGHNOISEBIAS unknown Z-Image, converts
Wan uxoah/wan2.1_lokr_simpletuner unknown Wan (variant A14B), converts

Related Issues / Discussions

Fixes #10174.

The issue's Qwen-Image row is covered by #10189 (its probe takes Qwen-Image's own module names as evidence), and
the SD 3 rows already worked. #10184 and #10189 touch adjacent converter and test files; this PR touches only
configs/lora.py in common with them, in different functions.

QA Instructions

Regression first: tests/backend/model_manager/configs/test_lycoris_flat_lora_config.py fails on unmodified main
showing exactly the misdetection — assert [] == ['LoRA_LyCORIS_FLUX_Config'] (and the same for Flux2, SDXL,
Z-Image, Wan) in test_a_flat_lycoris_lora_is_claimed_by_its_family_alone[*] and
test_a_flat_lycoris_lora_records_its_base[*]: 12 failed, 2 passed (the two negative controls pass on main too).

Checks run (CPU, .venv/bin/python -m pytest in a fresh uv sync --extra test environment):

  • pytest -n 4 tests/backend/patches: 616 passed, 24 skipped.
  • pytest -n 4 tests/backend/model_manager/configs: 1578 passed, 1 skipped, 13 failed — the 13 are all in
    test_model_path_validation.py and fail identically on unmodified main in this environment (verified by
    stashing this branch's changes and re-running the file: the same 13 failures), unrelated to LoRA code.
  • pytest -n 4 tests/model_identification: 129 passed.
  • pytest -n 4 tests/backend/model_manager/load tests/test_model_manager: 1301 passed, 127 skipped (the loader
    gained the new dispatch).
  • ruff check and ruff format on every changed path: clean.

Not run: the full test suite (large) and the frontend checks (no frontend changes).

The real-file evidence is at the key-layout level: the new fixtures are the six issue files' safetensors headers,
and every converted layer is resolved against the model it patches — Invoke's BFL FLUX class for FLUX.1, and
diffusers' own models (Flux2Transformer2DModel, ZImageTransformer2DModel, WanTransformer3DModel, and an SDXL UNet
with the released block structure) for the rest — with the FLUX.1 layers' weight deltas checked against the real
modules' shapes. Applying one through a live generation was not done (no GPU here).

Review

No material review findings; this is the initial version. Known limitations, in the same spirit as the existing
probes' documented ones:

  • A FLUX.1/FLUX.2 file that trains only module names SD 3 shares (transformer_blocks.N.attn.to_q, ...) carries
    no only-this-family name and stays unidentified rather than being guessed at.
  • An SDXL file confined to transformer_blocks indices 0–1 is not distinguished from SD 1/2 — the limitation
    _state_dict_looks_like_sdxl_unet_lora already documents.
  • FLUX.3's flattened lycoris_… names are a different layout and remain unrecognized, as its family page says.

Compatibility / Rollout

No migrations and no API or persisted-state changes. Identification runs at install, so a file already installed
as unknown keeps that record until it is reinstalled (Re-identify model in the Model Manager).

Checklist

  • The PR has a short but descriptive title, suitable for a changelog
  • Meaningful regression coverage added / updated where needed; obsolete tests/code removed
  • Persisted-state and API changes include required migrations / compatibility validation
  • Relevant performance/efficiency opportunities considered; material claims have evidence
  • Material review findings resolved and relevant checks rerun
  • Documentation added / updated (if applicable)
  • Updated What's New copy (if doing a release after this PR)

Autofix with [code]smith
Open in [code]smith · Need help on this PR? Tag @codesmith-bot with what you need.

SimpleTuner's LyCORIS output flattens each module name behind a lycoris_ prefix; no probe
recognized it, so the files installed as unknown on every base. The names now resolve against
each family's module list and convert through the existing per-family routes on FLUX.1,
FLUX.2, SDXL, Z-Image and Wan; Wan's variant detection reads LoKR factor widths.
@github-actions github-actions Bot added python PRs that change python files backend PRs that change backend files python-tests PRs that change python tests docs PRs that change docs labels Nov 11, 2026

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

backend PRs that change backend files docs PRs that change docs python PRs that change python files python-tests PRs that change python tests

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[bug]: SimpleTuner LyCORIS LoRAs (lycoris_ key prefix) are not identified on any base

1 participant