from_single_file forces pre_quantized=True, breaking bnb/TorchAO/Quanto single-file loading #15123

Describe the bug

DiffusersAutoQuantizer.from_config() / the quantizer base class default pre_quantized to True when it isn't explicitly passed. src/diffusers/loaders/single_file_model.py calls DiffusersAutoQuantizer.from_config(quantization_config) without passing pre_quantized, so every single-file load is treated as already-quantized.

GGUF and BFL are unaffected because their own quantizer __init__s set self.pre_quantized themselves from the config. Every backend that needs to quantize on load instead of assuming pre-quantized weights breaks or silently no-ops:

• BitsAndBytes 4-bit: raises "does not contain bitsandbytes__*".
• BitsAndBytes 8-bit: raises "Missing quantization component SCB".
• TorchAO: silently does nothing — weights load unquantized, but the model reports itself as quantized.
• Quanto: the docs advertise single-file support, but the loader's freeze() call renames the expected keys to weight.<parts>, and the checkpoint's plain *.weight keys then don't match and are skipped.

None of these paths has a test; only GGUF and BFL single-file loading are tested.

Reproduction

This reproduces the root cause directly — DiffusersAutoQuantizer.from_config doing exactly what single_file_model.py does, without passing pre_quantized:

from diffusers import BitsAndBytesConfig
from diffusers.quantizers.auto import DiffusersAutoQuantizer

config = BitsAndBytesConfig(load_in_8bit=True)

# This is the exact call single_file_model.py makes when quantization_config is passed
# to from_single_file — pre_quantized is never forwarded.
quantizer = DiffusersAutoQuantizer.from_config(config)

print(quantizer.pre_quantized)  # True, even though this config quantizes on load

System Info

• diffusers @ main, commit aaf57d0
• Python 3.12, Linux

Who can help?

(leaving blank — no specific quantization reviewer listed in the template's tag guide)