Skip to content

Training examples crash on --revision in expired non-EMA deprecation blocks #15115

Closed

Description

@SeverinVisionary

Describe the bug

An ordinary --revision=main makes the text-to-image and instruct-pix2pix training examples raise before model loading, even without --non_ema_revision:

Traceback (most recent call last):
  File "/content/issue_fix_validation_20261023/checkouts/diffusers/examples/text_to_image/train_text_to_image.py", line 1169, in <module>
    main()
  File "/content/issue_fix_validation_20261023/checkouts/diffusers/examples/text_to_image/train_text_to_image.py", line 538, in main
    deprecate(
TypeError: deprecate() got an unexpected keyword argument 'message'

This is an excerpt from the actual CPU test traceback with the original training source at 48cdbaef5ee3d718aaee392b44f2099ae511c4a5. The argument parser defaults non_ema_revision to revision, so ordinary model pinning reaches the old warning block. The deprecation utility rejects message=; making it positional alone would encounter the expired removal version 0.15.0.

Reproduction

The CPU test invoked this text-to-image command through Accelerate. Below, the temporary config/output paths are replaced by local paths; this portable spelling has not been run separately. Run from the pinned checkout with the text-to-image requirements installed:

python -c 'from accelerate.utils import write_basic_config; write_basic_config(save_location="cpu_config.yml")'
export PYTHONPATH="$PWD/src"
accelerate launch --config_file cpu_config.yml \
  examples/text_to_image/train_text_to_image.py \
  --pretrained_model_name_or_path hf-internal-testing/tiny-stable-diffusion-torch \
  --dataset_name hf-internal-testing/dummy_image_text_data \
  --resolution 64 --center_crop --random_flip --train_batch_size 1 \
  --gradient_accumulation_steps 1 --max_train_steps 2 --learning_rate 5.0e-04 \
  --scale_lr --lr_scheduler constant --lr_warmup_steps 0 \
  --output_dir revision-repro-output --revision=main

Two new regression cases failed on the original training scripts with that TypeError: text-to-image and instruct-pix2pix, each using --revision=main. After removing the obsolete warnings, all six cases passed: default arguments, --revision=main, and --non_ema_revision=main for each example (6 passed in 224.65s, no skips). These are the existing tiny-model training/save and checkpoint tests, not mocked startup tests.

Scope and proposed fix

The same expired call occurs in six scripts at the pinned commit:

  • examples/text_to_image/train_text_to_image.py
  • examples/instruct_pix2pix/train_instruct_pix2pix.py
  • examples/instruct_pix2pix/train_instruct_pix2pix_sdxl.py
  • examples/research_projects/instructpix2pix_lora/train_instruct_pix2pix_lora.py
  • examples/research_projects/onnxruntime/text_to_image/train_text_to_image.py
  • examples/research_projects/scheduled_huber_loss_training/text_to_image/train_text_to_image.py

Would an AI-assisted patch removing these six obsolete warning blocks and their unused imports be appropriate? The patch keeps every parser and loading call unchanged. The four additional SDXL/research scripts have source/static coverage only; no training result or correctness of their revision-loading behavior is claimed.

#2421 addressed the unsupported keyword elsewhere. This is separate from the skipped-update EMA behavior in #15112. The older fp16/EMA experiment stopped at this startup exception and remains inconclusive; the CPU tests do not complete that experiment.

System Info

Colab standard CPU, Linux x86_64, pinned Diffusers source above. The CPU environment probe recorded Python 3.13.16, PyTorch 2.11.0+cpu, torchvision 0.26.0+cpu and no CUDA. Accelerate launched the training children through /usr/local/bin/accelerate and /usr/bin/python3; their dependency versions were not separately captured. PYTHONPATH selected the exact pinned Diffusers checkout source, but the venv package inventory must not be read as the children's dependency inventory.

Who can help?

Training-example maintainers.

Prepared with Codex assistance.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions