Skip to content

NVIDIAModelOptConfig.to_dict() crashes from_pretrained/save_pretrained when forward_loop is set #15122

Open

Description

@OnePunchMonk

Describe the bug

forward_loop is a documented NVIDIAModelOptConfig constructor parameter. NVIDIAModelOptConfig has no to_dict() override, so it inherits QuantizationConfigMixin.to_dict(), which returns copy.deepcopy(self.__dict__) — including the raw forward_loop callable.

src/diffusers/models/modeling_utils.py calls json.dumps(hf_quantizer.quantization_config.to_dict(), sort_keys=True) for telemetry during from_pretrained, with no error handling around it. Any model loaded (or saved) with a NVIDIAModelOptConfig(forward_loop=...) crashes with TypeError: Object of type function is not JSON serializable.

Reproduction

import json
from diffusers.quantizers.quantization_config import NVIDIAModelOptConfig

config = NVIDIAModelOptConfig(
    quant_type="FP8",
    forward_loop=lambda model: model,
    modelopt_config={"quant_cfg": {}},
)

json.dumps(config.to_dict())

Logs

Traceback (most recent call last):
  File "<string>", line 1, in <module>
TypeError: Object of type function is not JSON serializable

This is the same call modeling_utils.py makes internally during from_pretrained:

  File ".../diffusers/src/diffusers/models/modeling_utils.py", line 1206, in from_pretrained
    serialized_quant_config = json.dumps(hf_quantizer.quantization_config.to_dict(), sort_keys=True)
TypeError: Object of type function is not JSON serializable

System Info

  • diffusers @ main, commit aaf57d0
  • Python 3.12, Linux

Who can help?

(leaving blank — no specific quantization reviewer listed in the template's tag guide)

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions