From 24a4aff97b47d91ed84308321de07177ac727833 Mon Sep 17 00:00:00 2001 From: Daniel Han Date: Fri, 7 Aug 2026 08:21:33 +0000 Subject: [PATCH] Spare 1-D norm weights from a blanket --type tensor_should_be_converted has no rule for 1-D weights, so a per-channel norm scale survives a blanket --type only by accident: when its length does not divide the quant block size (FLUX q_norm/k_norm are [128] and 128 % 256 != 0), or when one of the FLUX-era name rules happens to match it. MiniMax-H3's per-block norms are [5376], and 5376 % 256 == 0, so a blanket --type q4_K quantizes 105 of them. The result loads and renders a plausible video, so a "does it run" check passes it, but scored against a bf16 render of the same prompt and seed it is destroyed. A 1-D weight is a per-channel gain, never a matmul weight. Every channel of a block would share one scale and one min, and a gain vector has no reason to be locally smooth, so quantizing it buys almost nothing and costs a lot: on this model holding all 211 1-D tensors adds 0.77 MiB to a 10.60 GiB file, 0.007%. Measured on minimax_h3_fl2va_pruned, converted from the same bf16 checkpoint with a blanket --type q4_K and no --tensor-type-rules, rendered at 640x384, 25 frames, 4 steps, cfg 1.0, seed 1234, --rng cpu, and scored against a bf16 render of that same prompt and seed: 1-D tensor types PSNR SSIM LPIPS before F32 5, F16 51, BF16 106, Q4_K 105 9.87 0.074 0.981 after F32 5, F16 51, BF16 211 22.22 0.841 0.292 SSIM 0.074 means the output is essentially uncorrelated with the reference. The 1-D rule alone is sufficient. A build made with --tensor-type-rules 'norm[0-9]*\.weight$=bf16,condition_proj\.weight$=bf16' produces the same 1-D layout and scores 21.52 / 0.838 / 0.299, so nothing here depends on also sparing condition_proj.weight, which is 2-D and a separate quality choice. Peak VRAM is unchanged at 16.83 GiB. --- src/model_loader.cpp | 12 ++++++++++++ 1 file changed, 12 insertions(+) diff --git a/src/model_loader.cpp b/src/model_loader.cpp index 047c9d996..b286f2a5f 100644 --- a/src/model_loader.cpp +++ b/src/model_loader.cpp @@ -1511,6 +1511,18 @@ bool ModelLoader::tensor_should_be_converted(const TensorStorage& tensor_storage if (type != GGML_TYPE_COUNT) { if (ggml_is_quantized(type) && tensor_storage.ne[0] % ggml_blck_size(type) != 0) { // Pass, do not convert + } else if (ggml_is_quantized(type) && tensor_storage.n_dims <= 1) { + // Pass, do not convert. A 1-D weight is a per-channel scale (LayerNorm/RMSNorm gain), + // never a matmul weight, so quantizing it buys almost nothing and costs a lot: every + // channel of the block shares one scale and one min, and a gain vector has no reason + // to be locally smooth. Until now these survived only by accident, when their length + // did not divide the block size (FLUX q_norm/k_norm are [128] and 128 % 256 != 0) or + // when a name rule above happened to match. A model whose norms DO divide the block + // size, such as MiniMax-H3 with [5376] and 5376 % 256 == 0, had 106 norm scales + // crushed to 4 bits by a blanket --type. The result still loads and still renders a + // plausible image, so a "does it run" check passes it, while measured against a bf16 + // render of the same prompt and seed it is destroyed: PSNR 9.87 / SSIM 0.074 / + // LPIPS 0.981, against 22.22 / 0.841 / 0.292 with this rule in place. } else if (ends_with(name, ".bias")) { // Pass, do not convert } else if (ends_with(name, ".scale")) {