Skip to content

fixed_large_log variance sampling returns NaNs #14569

Description

@Robertboy18

Describe the finding

DDPMScheduler.step() and DDPMParallelScheduler.step() treat fixed_large_log like an ordinary variance and take its square root. _get_variance() returns log(current_beta_t) for this mode, though, so the square root is applied to a negative value and the sample becomes NaN.

The intended scale is exp(0.5 * log_variance), which is equivalent to the sqrt(variance) used by fixed_large.

I have a small fix ready for both scheduler implementations, along with CPU regression tests comparing fixed_large_log against the equivalent fixed_large sampling result. Happy to open the PR if this approach sounds right : )

Reproduction

import torch
from diffusers import DDPMScheduler, DDPMParallelScheduler

for scheduler_class in (DDPMScheduler, DDPMParallelScheduler):
    scheduler = scheduler_class(variance_type="fixed_large_log")
    sample = torch.zeros((1, 2, 2, 2))
    model_output = torch.zeros_like(sample)
    output = scheduler.step(
        model_output,
        500,
        sample,
        generator=torch.Generator().manual_seed(0),
    ).prev_sample
    print(scheduler_class.__name__, torch.isnan(output).sum().item())

Current output:

DDPMScheduler 8
DDPMParallelScheduler 8

Expected: both outputs are finite and match fixed_large when using the same generator.

System info

  • Diffusers version: 0.40.0.dev0 (main at 58eb52c)
  • Python: 3.12.13
  • PyTorch: 2.13.0+cu130
  • Platform: Linux
  • GPU used by reproduction: No
  • Distributed setup: No

@yiyixuxu

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions