Skip to content

Nunchaku AWQ config accepts any group_size, but the kernel only supports 64 #15125

Open

Description

@OnePunchMonk

Describe the bug

NunchakuLiteQuantizationConfig's validation for the AWQ path (src/diffusers/quantizers/quantization_config.py) only checks precision == "int4". It never checks group_size. The actual AWQ kernel call site (AWQW4A16Linear) always calls awq_gemm_w4a16_g64_int32 / passes a literal 64 to gemv, so it only ever supports group_size=64.

A checkpoint quantized with group_size=128 loads without any error and silently produces wrong output. The SVDQ path in the same config class does validate its group size against the expected value per precision, so this looks like an oversight in the AWQ branch rather than intentional.

Reproduction

from diffusers.quantizers.quantization_config import NunchakuLiteQuantizationConfig

config = NunchakuLiteQuantizationConfig(
    awq_w4a16={"precision": "int4", "group_size": 128, "targets": ["final_linear"]}
)
# No error raised here, even though the AWQ kernel only supports group_size=64.
print(config.awq_w4a16["group_size"])  # 128

System Info

  • diffusers @ main, commit aaf57d0
  • Python 3.12, Linux

Who can help?

(leaving blank — no specific quantization reviewer listed in the template's tag guide)

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions