diff --git a/.ai/references/modular.md b/.ai/references/modular.md index 892d59342d7d..8ab88e65cc98 100644 --- a/.ai/references/modular.md +++ b/.ai/references/modular.md @@ -324,20 +324,22 @@ ComponentSpec( 5. **Using `InputParam.template()` / `OutputParam.template()` when semantics don't match.** Templates carry predefined descriptions — e.g. the `"latents"` output template means "Denoised latents". Don't use it for initial noisy latents from a prepare-latents step. Use a plain `InputParam(...)` / `OutputParam(...)` with an accurate description instead. -6. **Test model paths pointing to contributor repos.** Tiny test models ultimately live under `hf-internal-testing/`, not personal repos like `username/tiny-model`. Developing against a personal repo is fine and not merge-blocking — a maintainer moves the model (before or after merge) and updates the path. +6. **`required=True` on an input that carries a default.** An input cannot be both required and have a default. When that happens, `InputParam` keeps the default, treats the input as optional and logs a warning. This is likely to happen when you reuse a template for its type hint and description but make it required: `InputParam.template("num_inference_steps", required=True)` stays an optional input that defaults to 50, plus a warning. Decide which one you mean: drop `required=True` to keep the template default without the warning, or pass `default=None` explicitly to make the input required. -7. **Respect the declared IO system.** Components in `expected_components`, fields in `inputs` / `intermediate_outputs` — once declared, the modular framework guarantees them. So: +7. **Test model paths pointing to contributor repos.** Tiny test models ultimately live under `hf-internal-testing/`, not personal repos like `username/tiny-model`. Developing against a personal repo is fine and not merge-blocking — a maintainer moves the model (before or after merge) and updates the path. + +8. **Respect the declared IO system.** Components in `expected_components`, fields in `inputs` / `intermediate_outputs` — once declared, the modular framework guarantees them. So: - **Don't read defensively.** Declared components are always set as attributes (possibly `None`); declared upstream outputs are always populated in `block_state` after the upstream block runs. `getattr(components, "vae", None)`, `hasattr(self, "vae")`, `getattr(block_state, "prompt_embeds", None)` are dead code that hides typos. Use `components.vae` / `block_state.prompt_embeds` directly. Check `is not None` only when nullability is meaningful (a component the user might not have loaded). - **Don't write undeclared.** If a block sets `block_state.foo = ...`, declare `OutputParam("foo", ...)` in `intermediate_outputs`. The declarations are the public contract — undeclared writes can't be wired to downstream blocks. - **Don't call `state.set()` directly inside a block.** Write to state only through declared `intermediate_outputs` via `self.get_block_state(state)` / `self.set_block_state(state, block_state)`. A direct `state.set("foo", value)` bypasses the block's interface entirely — the field never appears as a declared output, so downstream blocks can't see it through the normal wiring and the framework can't generate docs / validate types for it. -8. **No-op skip logic inside an optional block.** If a step is conditional (e.g. an optional prompt enhancer), don't have the block check a flag at the top of `__call__` and `return` early. Wrap it in an `AutoPipelineBlocks` with `block_trigger_inputs = ["use_xxx"]` so the block is only assembled into the pipeline when the trigger input is provided. The block's own `__call__` should always assume its components and inputs are present. +9. **No-op skip logic inside an optional block.** If a step is conditional (e.g. an optional prompt enhancer), don't have the block check a flag at the top of `__call__` and `return` early. Wrap it in an `AutoPipelineBlocks` with `block_trigger_inputs = ["use_xxx"]` so the block is only assembled into the pipeline when the trigger input is provided. The block's own `__call__` should always assume its components and inputs are present. -9. **Serving a checkpoint variant through a config flag in a shared block.** `ConfigSpec(name="is_distilled")` plus `if components.config.is_distilled:` bundles two checkpoints' behavior into one blockset — and it can't change the input surface at all (the distilled variant would still accept `negative_prompt`). Suggest a separate blockset for the variant instead (see Key pattern: Checkpoint variants). +10. **Serving a checkpoint variant through a config flag in a shared block.** `ConfigSpec(name="is_distilled")` plus `if components.config.is_distilled:` bundles two checkpoints' behavior into one blockset — and it can't change the input surface at all (the distilled variant would still accept `negative_prompt`). Suggest a separate blockset for the variant instead (see Key pattern: Checkpoint variants). -10. **Raw `torch.randn(device=...)` for noise.** Use `randn_tensor(...)` from `utils/torch_utils`: it draws on the generator's device and moves the result, so CPU generators (what the test mixins pass) work, and the CUDA-generator path is bit-identical to `torch.randn`. +11. **Raw `torch.randn(device=...)` for noise.** Use `randn_tensor(...)` from `utils/torch_utils`: it draws on the generator's device and moves the result, so CPU generators (what the test mixins pass) work, and the CUDA-generator path is bit-identical to `torch.randn`. -11. **Helpers taking `components`.** Prefer explicit args, for example, `get_xx_prompt_embeds(text_encoder, tokenizer, prompt)` instead of `get_xx_prompt_embeds(components, prompt)`. Helpers and block methods should take only what they use so dependencies are visible and they can run without an assembled pipeline. Leave the full `components` object to `__call__`, which unpacks what its helpers need. See `get_qwen_prompt_embeds` in `qwenimage/encoders.py`. +12. **Helpers taking `components`.** Prefer explicit args, for example, `get_xx_prompt_embeds(text_encoder, tokenizer, prompt)` instead of `get_xx_prompt_embeds(components, prompt)`. Helpers and block methods should take only what they use so dependencies are visible and they can run without an assembled pipeline. Leave the full `components` object to `__call__`, which unpacks what its helpers need. See `get_qwen_prompt_embeds` in `qwenimage/encoders.py`. ## Conversion checklist diff --git a/.ai/references/testing.md b/.ai/references/testing.md index 640ef6f0bd5a..2ccbc406a9b8 100644 --- a/.ai/references/testing.md +++ b/.ai/references/testing.md @@ -61,6 +61,7 @@ Since [#14268](https://github.com/huggingface/diffusers/pull/14268), a standard - Location: `tests/modular_pipelines//test_modular_pipeline_.py` (one config class + set of test classes per blockset / pipeline variant). - **Define one config class**, `ModularPipelineTesterConfig`, subclassing `BaseModularPipelineTesterConfig` (from `..testing_utils`). Set `pipeline_class`, `pipeline_blocks_class`, `pretrained_model_name_or_path`, `params` / `batch_params`, and implement `get_dummy_inputs(seed=0)`. Set `expected_workflow_blocks` to pin the block name → class ordering per workflow. The config holds the whole testing contract and performs no assertions. +- **Set `expected_workflow_defaults`** to what each workflow is expected to have: every component with its class, every config with its default, and every input — the optional ones with their defaults under `inputs`, the required ones by name under `required_inputs`. The test fails when the code declares anything else, so this dict is the pipeline's contract written out in one place, and filling it in is also the moment to check that contract. Once it is filled in, read it against the same dict in the other families under `tests/modular_pipelines/`: for example, `prompt` is normally required, `num_inference_steps` is normally optional with a default, `latents` is normally accepted so a run can be reproduced, and so on. An entry that breaks the common pattern needs a reason. If there isn't one, something is wrong: find out what and fix it, rather than writing the odd entry into the dict. - **Then one test class per concern**, each composing the config with a tester mixin from `..testing_utils`. Keep them separate — pytest reads class-level markers off the whole MRO, so folding a marked mixin (`@is_memory`, ...) into the same class as the others would tag every test in it: - `ModularPipelineTesterMixin` — call signature, batch consistency, float16, device placement, NaN-free output. Put pipeline-specific tests as methods on this class. - `ModularLoadingTesterMixin` — `save_pretrained`/`from_pretrained` round-trips, `modular_model_index.json` contents, `load_components`/`unload_components`. diff --git a/src/diffusers/modular_pipelines/cosmos/before_denoise.py b/src/diffusers/modular_pipelines/cosmos/before_denoise.py index e6f9dbf05abb..689657d0878a 100644 --- a/src/diffusers/modular_pipelines/cosmos/before_denoise.py +++ b/src/diffusers/modular_pipelines/cosmos/before_denoise.py @@ -979,7 +979,7 @@ def expected_configs(self) -> list[ConfigSpec]: @property def inputs(self) -> list[InputParam]: return [ - InputParam.template("num_inference_steps", required=True), + InputParam.template("num_inference_steps"), ] @property @@ -1258,7 +1258,7 @@ def expected_components(self) -> list[ComponentSpec]: @property def inputs(self) -> list[InputParam]: - return [InputParam.template("num_inference_steps", required=True)] + return [InputParam.template("num_inference_steps")] @property def intermediate_outputs(self) -> list[OutputParam]: diff --git a/src/diffusers/modular_pipelines/cosmos/denoise.py b/src/diffusers/modular_pipelines/cosmos/denoise.py index 294213f48f4b..2422defe4d46 100644 --- a/src/diffusers/modular_pipelines/cosmos/denoise.py +++ b/src/diffusers/modular_pipelines/cosmos/denoise.py @@ -500,7 +500,7 @@ def loop_expected_components(self) -> list[ComponentSpec]: def loop_inputs(self) -> list[InputParam]: return [ InputParam.template("timesteps", required=True), - InputParam.template("num_inference_steps", required=True), + InputParam.template("num_inference_steps", required=True, default=None), InputParam( name="num_warmup_steps", type_hint=int, required=True, description="Number of scheduler warmup steps." ), diff --git a/src/diffusers/modular_pipelines/cosmos/modular_blocks_cosmos3.py b/src/diffusers/modular_pipelines/cosmos/modular_blocks_cosmos3.py index ab6ebac2c8cd..864e687adad2 100644 --- a/src/diffusers/modular_pipelines/cosmos/modular_blocks_cosmos3.py +++ b/src/diffusers/modular_pipelines/cosmos/modular_blocks_cosmos3.py @@ -394,7 +394,7 @@ class Cosmos3VisionCoreDenoiseStep(SequentialPipelineBlocks): Pre-generated noisy vision latents. generator (`Generator`, *optional*): Torch generator for deterministic generation. - num_inference_steps (`int`): + num_inference_steps (`int`, *optional*, defaults to 50): The number of denoising steps. mixed_precision_format (`str`, *optional*): None follows the ModelOpt FP8 checkpoint schedule; 'none' keeps the native quantized forward; 'fp8' is @@ -474,7 +474,7 @@ class Cosmos3VisionSoundCoreDenoiseStep(SequentialPipelineBlocks): Pre-generated noisy vision latents. generator (`Generator`, *optional*): Torch generator for deterministic generation. - num_inference_steps (`int`): + num_inference_steps (`int`, *optional*, defaults to 50): The number of denoising steps. sound_latents (`Tensor`, *optional*): Pre-generated noisy sound latents. @@ -567,7 +567,7 @@ class Cosmos3VisionActionCoreDenoiseStep(SequentialPipelineBlocks): Pre-generated noisy vision latents. generator (`Generator`, *optional*): Torch generator for deterministic generation. - num_inference_steps (`int`): + num_inference_steps (`int`, *optional*, defaults to 50): The number of denoising steps. action (`CosmosActionCondition`): Action-conditioning metadata. @@ -664,7 +664,7 @@ class Cosmos3VisionSoundActionCoreDenoiseStep(SequentialPipelineBlocks): Pre-generated noisy vision latents. generator (`Generator`, *optional*): Torch generator for deterministic generation. - num_inference_steps (`int`): + num_inference_steps (`int`, *optional*, defaults to 50): The number of denoising steps. sound_latents (`Tensor`, *optional*): Pre-generated noisy sound latents. @@ -782,7 +782,7 @@ class Cosmos3TransferChunkDenoiseStep(SequentialPipelineBlocks): Unconditional text segment. fps (`float`, *optional*, defaults to 24.0): Frame rate of the generated video. - num_inference_steps (`int`): + num_inference_steps (`int`, *optional*, defaults to 50): The number of denoising steps. mixed_precision_format (`str`, *optional*): None follows the ModelOpt FP8 checkpoint schedule; 'none' keeps the native quantized forward; 'fp8' is @@ -937,7 +937,7 @@ class Cosmos3TransferCoreDenoiseStep(SequentialPipelineBlocks): Torch generator for deterministic generation. fps (`float`, *optional*, defaults to 24.0): Frame rate of the generated video. - num_inference_steps (`int`): + num_inference_steps (`int`, *optional*, defaults to 50): The number of denoising steps. mixed_precision_format (`str`, *optional*): None follows the ModelOpt FP8 checkpoint schedule; 'none' keeps the native quantized forward; 'fp8' is diff --git a/src/diffusers/modular_pipelines/krea2/denoise.py b/src/diffusers/modular_pipelines/krea2/denoise.py index ccb972ca74c1..8fc019052a54 100644 --- a/src/diffusers/modular_pipelines/krea2/denoise.py +++ b/src/diffusers/modular_pipelines/krea2/denoise.py @@ -91,7 +91,7 @@ def expected_components(self) -> list[ComponentSpec]: def inputs(self) -> list[InputParam]: return [ InputParam(name="latents", required=True, type_hint=torch.Tensor, description="Packed image latents."), - InputParam.template("num_inference_steps", required=True), + InputParam.template("num_inference_steps", required=True, default=None), InputParam( name="prompt_embeds", required=True, @@ -264,7 +264,7 @@ def loop_inputs(self) -> list[InputParam]: type_hint=torch.Tensor, description="Denoising timesteps from set_timesteps.", ), - InputParam.template("num_inference_steps", required=True), + InputParam.template("num_inference_steps", required=True, default=None), InputParam.template("attention_kwargs"), ] diff --git a/src/diffusers/modular_pipelines/ltx/before_denoise.py b/src/diffusers/modular_pipelines/ltx/before_denoise.py index 5543742a4d46..470ea7325ec7 100644 --- a/src/diffusers/modular_pipelines/ltx/before_denoise.py +++ b/src/diffusers/modular_pipelines/ltx/before_denoise.py @@ -282,7 +282,7 @@ def inputs(self) -> list[InputParam]: InputParam.template("latents"), InputParam.template("num_images_per_prompt", name="num_videos_per_prompt"), InputParam.template("generator"), - InputParam.template("batch_size", required=True), + InputParam.template("batch_size", required=True, default=None), ] @property @@ -348,7 +348,7 @@ def inputs(self) -> list[InputParam]: InputParam.template("width", default=704), InputParam("num_frames", type_hint=int, default=161), InputParam.template("num_images_per_prompt", name="num_videos_per_prompt"), - InputParam.template("batch_size", required=True), + InputParam.template("batch_size", required=True, default=None), ] @property diff --git a/src/diffusers/modular_pipelines/ltx/decoders.py b/src/diffusers/modular_pipelines/ltx/decoders.py index 2917eca125d0..cbde2effd4cb 100644 --- a/src/diffusers/modular_pipelines/ltx/decoders.py +++ b/src/diffusers/modular_pipelines/ltx/decoders.py @@ -131,7 +131,7 @@ def inputs(self) -> list[tuple[str, Any]]: InputParam("decode_noise_scale", default=None), InputParam.template("generator"), InputParam.template("batch_size"), - InputParam.template("dtype", required=True), + InputParam.template("dtype", required=True, default=None), # Only read on the deprecated path that still accepts packed `[B, S, C]` latents. InputParam.template("height", default=512), InputParam.template("width", default=704), diff --git a/src/diffusers/modular_pipelines/ltx/denoise.py b/src/diffusers/modular_pipelines/ltx/denoise.py index 44a3c91d471b..d04c0c7f18d5 100644 --- a/src/diffusers/modular_pipelines/ltx/denoise.py +++ b/src/diffusers/modular_pipelines/ltx/denoise.py @@ -45,7 +45,7 @@ def description(self) -> str: def inputs(self) -> list[InputParam]: return [ InputParam.template("latents", required=True), - InputParam.template("dtype", required=True), + InputParam.template("dtype", required=True, default=None), ] @torch.no_grad() @@ -97,7 +97,7 @@ def description(self) -> str: def inputs(self) -> list[tuple[str, Any]]: inputs = [ InputParam.template("attention_kwargs"), - InputParam.template("num_inference_steps", required=True), + InputParam.template("num_inference_steps", required=True, default=None), InputParam("rope_interpolation_scale", type_hint=tuple), InputParam.template("height"), InputParam.template("width"), @@ -211,7 +211,7 @@ def loop_expected_components(self) -> list[ComponentSpec]: def loop_inputs(self) -> list[InputParam]: return [ InputParam.template("timesteps", required=True), - InputParam.template("num_inference_steps", required=True), + InputParam.template("num_inference_steps", required=True, default=None), ] @torch.no_grad() @@ -277,7 +277,7 @@ def inputs(self) -> list[InputParam]: return [ InputParam.template("latents", required=True), InputParam("conditioning_mask", required=True, type_hint=torch.Tensor), - InputParam.template("dtype", required=True), + InputParam.template("dtype", required=True, default=None), ] @torch.no_grad() @@ -331,7 +331,7 @@ def description(self) -> str: def inputs(self) -> list[tuple[str, Any]]: inputs = [ InputParam.template("attention_kwargs"), - InputParam.template("num_inference_steps", required=True), + InputParam.template("num_inference_steps", required=True, default=None), InputParam("rope_interpolation_scale", type_hint=tuple), InputParam.template("height"), InputParam.template("width"), diff --git a/src/diffusers/modular_pipelines/ltx2/decoders.py b/src/diffusers/modular_pipelines/ltx2/decoders.py index cbeb1a66826d..34a2e9b6a74f 100644 --- a/src/diffusers/modular_pipelines/ltx2/decoders.py +++ b/src/diffusers/modular_pipelines/ltx2/decoders.py @@ -161,7 +161,7 @@ def inputs(self) -> list[tuple[str, Any]]: "num_frames", type_hint=int, default=121, description="The number of frames in the generated video." ), InputParam.template("generator"), - InputParam.template("dtype", required=True), + InputParam.template("dtype", required=True, default=None), ] @property @@ -251,7 +251,7 @@ def inputs(self) -> list[tuple[str, Any]]: ), InputParam.template("generator"), InputParam.template("batch_size"), - InputParam.template("dtype", required=True), + InputParam.template("dtype", required=True, default=None), ] @property diff --git a/src/diffusers/modular_pipelines/ltx2/denoise.py b/src/diffusers/modular_pipelines/ltx2/denoise.py index f83b86a656f6..c396d01141be 100644 --- a/src/diffusers/modular_pipelines/ltx2/denoise.py +++ b/src/diffusers/modular_pipelines/ltx2/denoise.py @@ -305,7 +305,7 @@ def inputs(self) -> list[InputParam]: # `audio_num_frames`, `video_coords`, `audio_coords` arrive tagged `denoiser_input_fields` upstream and # are collected from the tagged dict (filtered against the transformer signature) in `__call__`. InputParam.template("denoiser_input_fields"), - InputParam.template("num_inference_steps", required=True), + InputParam.template("num_inference_steps", required=True, default=None), InputParam.template("height", default=512), InputParam.template("width", default=704), InputParam( @@ -654,7 +654,7 @@ def loop_expected_components(self) -> list[ComponentSpec]: def loop_inputs(self) -> list[InputParam]: return [ InputParam("timesteps", type_hint=torch.Tensor, required=True), - InputParam.template("num_inference_steps", required=True), + InputParam.template("num_inference_steps", required=True, default=None), ] @torch.no_grad() diff --git a/src/diffusers/modular_pipelines/minimax_h3/before_denoise.py b/src/diffusers/modular_pipelines/minimax_h3/before_denoise.py index 94395a7c3ebe..2f95a1a2576e 100644 --- a/src/diffusers/modular_pipelines/minimax_h3/before_denoise.py +++ b/src/diffusers/modular_pipelines/minimax_h3/before_denoise.py @@ -1146,7 +1146,7 @@ def expected_components(self) -> list[ComponentSpec]: @property def inputs(self) -> list[InputParam]: return [ - InputParam.template("num_inference_steps", required=True), + InputParam.template("num_inference_steps"), InputParam( name="video_indices", type_hint=torch.Tensor, diff --git a/src/diffusers/modular_pipelines/minimax_h3/modular_blocks_minimax_h3.py b/src/diffusers/modular_pipelines/minimax_h3/modular_blocks_minimax_h3.py index 23e7f1b83bba..e96454a44fa4 100644 --- a/src/diffusers/modular_pipelines/minimax_h3/modular_blocks_minimax_h3.py +++ b/src/diffusers/modular_pipelines/minimax_h3/modular_blocks_minimax_h3.py @@ -261,7 +261,7 @@ class MiniMaxH3CoreDenoiseStep(SequentialPipelineBlocks): instead of the draw. audio_latents (`Tensor`, *optional*): Pre-generated audio noise of shape `(2, 32, num_audio_latents)`. - num_inference_steps (`int`): + num_inference_steps (`int`, *optional*, defaults to 50): The number of denoising steps. prompt_embeds (`Tensor`): text embeddings used to guide the image generation. Can be generated from text_encoder step. @@ -360,7 +360,7 @@ class MiniMaxH3FL2VACoreDenoiseStep(SequentialPipelineBlocks): instead of the draw. audio_latents (`Tensor`, *optional*): Pre-generated audio noise of shape `(2, 32, num_audio_latents)`. - num_inference_steps (`int`): + num_inference_steps (`int`, *optional*, defaults to 50): The number of denoising steps. prompt_embeds (`Tensor`): text embeddings used to guide the image generation. Can be generated from text_encoder step. @@ -459,7 +459,7 @@ class MiniMaxH3Ref2VACoreDenoiseStep(SequentialPipelineBlocks): instead of the draw. audio_latents (`Tensor`, *optional*): Pre-generated audio noise of shape `(2, 32, num_audio_latents)`. - num_inference_steps (`int`): + num_inference_steps (`int`, *optional*, defaults to 50): The number of denoising steps. prompt_embeds (`Tensor`): text embeddings used to guide the image generation. Can be generated from text_encoder step. @@ -564,7 +564,7 @@ class MiniMaxH3AutoDenoiseStep(ConditionalPipelineBlocks): instead of the draw. audio_latents (`Tensor`): Pre-generated audio noise of shape `(2, 32, num_audio_latents)`. - num_inference_steps (`int`): + num_inference_steps (`int`, *optional*, defaults to 50): The number of denoising steps. prompt_embeds (`Tensor`): text embeddings used to guide the image generation. Can be generated from text_encoder step. @@ -721,7 +721,7 @@ class MiniMaxH3Blocks(SequentialPipelineBlocks): instead of the draw. audio_latents (`Tensor`): Pre-generated audio noise of shape `(2, 32, num_audio_latents)`. - num_inference_steps (`int`): + num_inference_steps (`int`, *optional*, defaults to 50): The number of denoising steps. **denoiser_input_fields (`None`, *optional*): The structural description of the packed sequence the transformer reads by name: `token_tags`, diff --git a/src/diffusers/modular_pipelines/modular_pipeline_utils.py b/src/diffusers/modular_pipelines/modular_pipeline_utils.py index e4a81e398e8c..07dd94c5693e 100644 --- a/src/diffusers/modular_pipelines/modular_pipeline_utils.py +++ b/src/diffusers/modular_pipelines/modular_pipeline_utils.py @@ -571,6 +571,15 @@ class InputParam: # resolves its own default at runtime in `get_block_state` defaults_by_block: dict[str, Any] = None + def __post_init__(self): + if self.required and self.default is not None: + logger.warning( + f"InputParam '{self.name}' is required and declares a default ({self.default!r}), which cannot both " + f"hold: a required input is never missing, so its default is unreachable. Keeping the default and " + f"treating the input as optional. Pass `default=None` explicitly if '{self.name}' is meant to be required." + ) + self.required = False + def __repr__(self): return f"<{self.name}: {'required' if self.required else 'optional'}, default={self.default}>" diff --git a/src/diffusers/modular_pipelines/qwenimage/denoise.py b/src/diffusers/modular_pipelines/qwenimage/denoise.py index 7f271782f82b..73fbef722b10 100644 --- a/src/diffusers/modular_pipelines/qwenimage/denoise.py +++ b/src/diffusers/modular_pipelines/qwenimage/denoise.py @@ -476,7 +476,7 @@ def loop_inputs(self) -> list[InputParam]: type_hint=torch.Tensor, description="The timesteps to use for the denoising process. Can be generated in set_timesteps step.", ), - InputParam.template("num_inference_steps", required=True), + InputParam.template("num_inference_steps", required=True, default=None), ] @torch.no_grad() diff --git a/src/diffusers/modular_pipelines/qwenimage21/denoise.py b/src/diffusers/modular_pipelines/qwenimage21/denoise.py index 67e41ed56966..be44396e5042 100644 --- a/src/diffusers/modular_pipelines/qwenimage21/denoise.py +++ b/src/diffusers/modular_pipelines/qwenimage21/denoise.py @@ -386,7 +386,7 @@ def loop_inputs(self) -> list[InputParam]: type_hint=torch.Tensor, description="The timesteps to use for the denoising process. Can be generated in set_timesteps step.", ), - InputParam.template("num_inference_steps", required=True), + InputParam.template("num_inference_steps", required=True, default=None), InputParam( name="use_kv_cache", type_hint=bool, diff --git a/tests/modular_pipelines/minimax_h3/test_modular_pipeline_minimax_h3.py b/tests/modular_pipelines/minimax_h3/test_modular_pipeline_minimax_h3.py index 03239ef82e3d..dd7f04f1c7f1 100644 --- a/tests/modular_pipelines/minimax_h3/test_modular_pipeline_minimax_h3.py +++ b/tests/modular_pipelines/minimax_h3/test_modular_pipeline_minimax_h3.py @@ -119,6 +119,7 @@ # reference or from the bare 16:9 default — so all three declare the rule as pipeline config. _SHARED_CONFIGS = {"canvas_short_edge": 768, "canvas_max_pixels": 768 * 1344} _SHARED_INPUTS = { + "num_inference_steps": 50, "height": None, "width": None, "generator": None, @@ -131,7 +132,7 @@ "t2va": { "components": {**_SHARED_COMPONENTS, "transformer": "MiniMaxH3Transformer3DModel"}, "configs": _SHARED_CONFIGS, - "required_inputs": ["prompt", "num_inference_steps"], + "required_inputs": ["prompt"], "inputs": {**_SHARED_INPUTS, "num_frames": 124}, }, "fl2va": { @@ -141,7 +142,7 @@ "image_processor": "VaeImageProcessor", }, "configs": _SHARED_CONFIGS, - "required_inputs": ["prompt", "num_inference_steps"], + "required_inputs": ["prompt"], "inputs": {**_SHARED_INPUTS, "num_frames": 124, "image": None, "last_image": None}, }, } @@ -157,7 +158,7 @@ "configs": {**_SHARED_CONFIGS, "reference_image_short_edge": 2048}, # `num_frames` cannot default on ref2va: reference soundtracks are truncated to the generated duration, so a # silent 124-frame default would cut them short. It is required instead. - "required_inputs": ["prompt", "references", "num_frames", "num_inference_steps"], + "required_inputs": ["prompt", "references", "num_frames"], "inputs": _SHARED_INPUTS, }, } @@ -174,8 +175,8 @@ class MiniMaxH3ModularPipelineTesterConfig(BaseModularPipelineTesterConfig): # MiniMax-H3 packs one request into one sequence and rejects a list of prompts, so nothing is batched. batch_params = frozenset() # The checkpoint is guidance-distilled and generates a video, so neither `guidance_scale` nor - # `num_images_per_prompt` exists. `num_inference_steps` is declared required, so it is not listed here either. - optional_params = frozenset(["latents", "output_type"]) + # `num_images_per_prompt` exists. + optional_params = frozenset(["num_inference_steps", "latents", "output_type"]) output_name = "videos" expected_workflow_blocks = MINIMAX_H3_WORKFLOWS expected_workflow_defaults = MINIMAX_H3_WORKFLOW_DEFAULTS @@ -614,8 +615,8 @@ class MiniMaxH3Ref2VAModularPipelineTesterConfig(BaseModularPipelineTesterConfig # MiniMax-H3 packs one request into one sequence and rejects a list of prompts, so nothing is batched. batch_params = frozenset() # The checkpoint is guidance-distilled and generates a video, so neither `guidance_scale` nor - # `num_images_per_prompt` exists. `num_inference_steps` is declared required, so it is not listed here either. - optional_params = frozenset(["latents", "output_type"]) + # `num_images_per_prompt` exists. + optional_params = frozenset(["num_inference_steps", "latents", "output_type"]) output_name = "videos" expected_workflow_blocks = MINIMAX_H3_REF2VA_WORKFLOWS expected_workflow_defaults = MINIMAX_H3_REF2VA_WORKFLOW_DEFAULTS diff --git a/tests/modular_pipelines/testing_utils/common.py b/tests/modular_pipelines/testing_utils/common.py index 545ca0f31ef0..3ea912a6db5c 100644 --- a/tests/modular_pipelines/testing_utils/common.py +++ b/tests/modular_pipelines/testing_utils/common.py @@ -24,6 +24,7 @@ from diffusers.utils import logging from ...testing_utils import ( + CaptureLogger, backend_empty_cache, numpy_cosine_similarity_distance, require_accelerator, @@ -223,6 +224,16 @@ def _check_for_parameters(parameters, expected_parameters, param_type): f"Parameters declared in `not_params` unexpectedly present in the pipeline inputs: {unsupported_parameters}" ) + def test_no_input_is_both_required_and_has_a_default(self): + # `InputParam` warns and drops `required` when a declaration is both required and has a default (see the + # pitfalls list in `.ai/references/modular.md`). Building the blocks and reading their `inputs` constructs + # every `InputParam` of every sub-block, so a clean log means no block in this pipeline declares one + logger = logging.get_logger("diffusers.modular_pipelines.modular_pipeline_utils") + with CaptureLogger(logger) as cap_logger: + blocks = self.pipeline_blocks_class() + blocks.inputs + assert "is required and declares a default" not in cap_logger.out, cap_logger.out + def test_inference_batch_consistent(self, batch_sizes=[2], batch_generator=True): pipe = self.get_pipeline().to(torch_device)