feat(wan): load ComfyUI int8 convrot checkpoints

docs/src/content/docs/Users Guide/03.Models/02.model-families.mdx

@@ -47,7 +47,7 @@ InvokeAI recognizes the format from the file's contents, so you install them lik
 - **GGUF** (`.gguf`, llama.cpp quantization such as Q4_K or Q8_0): weights stay packed in memory and are unpacked layer by layer during generation. Supported for the FLUX.1, FLUX.2, FLUX.3, Z-Image, Krea-2, Qwen-Image, Qwen-Image-2.1, Ideogram 4, ERNIE-Image and Wan transformers, and for the T5, Qwen3, Qwen3-VL, Qwen2.5-VL (text only), Mistral and Gemma 2 text encoders. ComfyUI-GGUF's `Q8_CR` and `Q4_CR` files, which store int8 or int4 convrot layers inside a `.gguf`, load for the Krea-2 transformer; for other models installation is refused with a message saying so.
 - **NF4** (bitsandbytes): 4-bit. InvokeAI installs bitsandbytes on Windows and Linux, with ROCm binaries for AMD GPUs, so NF4 models load on NVIDIA and AMD GPUs; macOS installs have no bitsandbytes, and NF4 models don't load there.
 - **SDNQ**: see [SDNQ Quantization](/configuration/optimization/sdnq-quantization/).
-- **int8** (`int8_convrot`, from ComfyUI repackages): the weights stay int8 in memory on every device, including Apple Silicon, and need no setting to turn on. Supported for FLUX.1, FLUX.2 Klein, FLUX.3, Z-Image, Anima, Krea-2, Qwen-Image, Qwen-Image-2.1, Ideogram 4, MiniMax H3, LTX-2 and the PiD decoders, and for the Qwen3 4B and 8B, Qwen3-VL, Qwen2.5-VL and LTX-2 Gemma encoders.
+- **int8** (`int8_convrot`, from ComfyUI repackages): the weights stay int8 in memory on every device, including Apple Silicon, and need no setting to turn on. Supported for FLUX.1, FLUX.2 Klein, FLUX.3, Z-Image, Anima, Krea-2, Qwen-Image, Qwen-Image-2.1, Ideogram 4, MiniMax H3, LTX-2, Wan 2.2 and the PiD decoders, and for the Qwen3 4B and 8B, Qwen3-VL, Qwen2.5-VL and LTX-2 Gemma encoders.
 - **int4** (`convrot_w4a4` or `int4_tensorwise`, ComfyUI repackages that usually keep some layers int8): the weights stay packed in memory at half a byte each and are unpacked per layer, on every device. This saves memory, not time: InvokeAI unpacks the weights for every step rather than running 4-bit matrix kernels, and 4-bit weights keep less detail than int8. Supported for the Krea-2 transformer.
 - **nvfp4** (4-bit, in ComfyUI's checkpoint format, such as Comfy-Org's builds; other nvfp4 exports such as AWQ are refused): the weights stay packed in memory and are unpacked per layer, so no special GPU is needed. Supported for the FLUX.3, Z-Image, Krea-2, Qwen-Image and LTX-2 transformers and for the Qwen3, Qwen3-VL 4B/8B, Qwen2.5-VL and FLUX.2 Mistral encoders; for FLUX.3 also Black Forest Labs' own nvfp4 build, in NVIDIA ModelOpt's layout. The Ideogram 4 and Qwen-Image-2.1 nvfp4 transformer files are refused at install.
 - **fp8** (plain or `fp8_scaled`): InvokeAI keeps the weights in fp8 when [FP8 Storage](/configuration/optimization/fp8-storage/) is on, which installation does for you, or when `fp8_compute` is enabled.

docs/src/content/docs/Users Guide/03.Models/03.model-format-support.mdx

@@ -47,7 +47,7 @@ reader's error; if the file is incomplete or corrupt, download it again.
 | [Ideogram 4](/users-guide/models/local-models/ideogram4/) | ✓ | ✓ | ✓ | ✓ | R | R | ✓ | ✓ | ✗ | inside the Diffusers folder |
 | [ERNIE-Image](/users-guide/models/local-models/ernie-image/) | ✓ | ✓ | L | L | L | L | ✗ | ✓ | ✗ | ✗ |
 | [MOSS-Image](/users-guide/models/local-models/moss-image/) | ✓ | ✓ | R | ✓ | R | R | R | ✓ | ✗ | ✗ |
-| [Wan 2.2](/users-guide/video-generation/wan-22/) | ✓ | ✓ | ✓ | L | L | L | L | ✓ | ✗ | ✗ |
+| [Wan 2.2](/users-guide/video-generation/wan-22/) | ✓ | ✓ | ✓ | ✓ | R | L | L | ✓ | ✗ | ✗ |
 | [LTX-2](/users-guide/video-generation/ltx-2/) (2.5 only) | official layout ✓, Diffusers layout ✗ | ✓ | L | ✓ | L | ✓ | ✗ | ✓ | ✗ | ✗ |
 | [MiniMax H3](/users-guide/video-generation/minimax-h3/) | ✓ | ✓ | L | ✓ | L | L | ✗ | ✗ | ✗ | ✗ |
 
@@ -59,8 +59,8 @@ reader's error; if the file is incomplete or corrupt, download it again.
   AWQ and other producers' nvfp4 at install. For Qwen-Image-2.1 and FLUX.3, torchao exports such as unsloth's FP8
   builds are refused at install. MOSS-Image loads Comfy-Org's int8 build and GGUFs; its fp8, nvfp4 and w4a8
   transformers, and unsloth's torchao builds, are refused at install, unscaled fp8 included. ComfyUI's other low-bit
-  layouts (`w4a8`, `w6a8`, SVDQuant) are not supported; Krea-2 and
-  Qwen-Image refuse them at install.
+  layouts (`w4a8`, `w6a8`, SVDQuant) are not supported; Krea-2,
+  Qwen-Image and Wan 2.2 refuse them at install.
 - **GGUF** covers the llama.cpp types (Q4_0 through Q8_0, the K-quants, BF16). ComfyUI-GGUF's `Q8_CR` (int8) and
   `Q4_CR` (int4) files load for Krea-2 only; for every other model with GGUF support the install fails, as it does
   for `Q4_CR` files of ComfyUI-GGUF's retired W4A16 kind.

docs/src/content/docs/Users Guide/05.Video Generation/wan-2.2.mdx

@@ -59,3 +59,8 @@ Higher-quality Q8_0 quantizations and full Diffusers builds are available a-la-c
 **On a 12 GB VRAM card**: use TI2V-5B — it covers both text-to-video and image-to-video and fits comfortably. The bundle includes it; to skip the A14B downloads, install the **Wan 2.2 TI2V-5B (Q4_K_M)** starter model on its own, which brings its VAE and the UMT5-XXL encoder. The A14B variants will technically run via aggressive offloading but are slow and prone to OOM.
 :::
 
+#### ComfyUI int8 builds
+
+Single-file A14B and TI2V-5B transformers in ComfyUI's `int8_tensorwise` (convrot) format load like any other single file, with the same two-expert setup for A14B. The attention and feed-forward weights stay int8 in memory, which saves memory. By default they are dequantized per layer on each forward pass; [int8 Compute](/configuration/optimization/int8-compute/) can instead run them on NVIDIA int8 tensor cores. LoRAs, including Lightning, are applied alongside the int8 weights rather than merged into them, except that DoRA layers do not run on an int8 transformer, so a DoRA LoRA fails there when it is applied. Files marked only `{"convrot": true}`, without a `format` entry, and the int4 builds are refused when you install them.
+
+On one Windows machine with an RTX 4090 (T2V A14B, 832x480, 17 frames), an int8 expert file is about 14.5 GB against about 28.6 GB for a bf16 expert. With both experts and the UMT5 encoder kept in the RAM cache, the int8 build peaked at about 41 GB of RAM. With `max_cache_ram_gb: 16` it still runs but reloads the experts for each clip, and peaked at about 30 GB of RAM, because both experts are held in memory during the swap. Windows also counts VRAM toward a process's committed memory, so that figure reads higher than the RAM in use. int8 mainly saves memory; how much speed it changes depends on whether the model otherwise fits in VRAM. Other systems will differ.

invokeai/app/invocations/wan/wan_denoise.py

@@ -44,11 +44,11 @@
 from invokeai.app.invocations.model import LoRAField, WanTransformerField
 from invokeai.app.invocations.primitives import LatentsOutput
 from invokeai.app.services.shared.invocation_context import InvocationContext
-from invokeai.backend.model_manager.taxonomy import BaseModelType, ModelFormat, WanVariantType
+from invokeai.backend.model_manager.taxonomy import BaseModelType, WanVariantType
 from invokeai.backend.patches.layer_patcher import LayerPatcher, PatchSpec
 from invokeai.backend.patches.lora_conversions.wan_lora_constants import WAN_LORA_TRANSFORMER_PREFIX
 from invokeai.backend.patches.model_patch_raw import ModelPatchRaw
-from invokeai.backend.quantization.dequantizing_linear import peak_dequant_transient_bytes
+from invokeai.backend.quantization.dequantizing_linear import peak_dequant_transient_bytes, requires_sidecar_patching
 from invokeai.backend.rectified_flow.rectified_flow_inpaint_extension import RectifiedFlowInpaintExtension
 from invokeai.backend.stable_diffusion.diffusers_pipeline import PipelineIntermediateState
 from invokeai.backend.stable_diffusion.diffusion.conditioning_data import WanConditioningInfo
@@ -189,8 +189,6 @@ def __init__(
         inference_dtype: torch.dtype,
         high_lora_factory: LoRAIteratorFactory | None = None,
         low_lora_factory: LoRAIteratorFactory | None = None,
-        high_is_quantized: bool = False,
-        low_is_quantized: bool = False,
         working_mem_bytes: int | None = None,
         max_resident_model_bytes: int | None = None,
     ) -> None:
@@ -200,8 +198,6 @@ def __init__(
         self._inference_dtype = inference_dtype
         self._high_lora_factory = high_lora_factory
         self._low_lora_factory = low_lora_factory
-        self._high_is_quantized = high_is_quantized
-        self._low_is_quantized = low_is_quantized
         self._working_mem_bytes = working_mem_bytes
         self._max_resident_model_bytes = max_resident_model_bytes
         self._active_label: str | None = None
@@ -257,7 +253,7 @@ def get(self, label: str) -> Any:
         info = self._context.models.load(model_id)
         supports_partial_loading = getattr(info, "supports_partial_loading", None)
         if self._working_mem_bytes is None or supports_partial_loading is False:
-            # No estimate to add it to, so a GGUF expert's per-forward dequantization transient is passed
+            # No estimate to add it to, so a quantized expert's per-forward dequantization transient is passed
             # alone; the cache floors it at `device_working_mem_gb`, so zero asks for the default.
             device_ctx = info.model_on_device(
                 working_mem_bytes=peak_dequant_transient_bytes(info.model, self._inference_dtype)
@@ -270,7 +266,7 @@ def get(self, label: str) -> Any:
         # likely the LoRA patcher), the surrounding ExitStack will eventually call
         # ``swapper.close() -> _release()`` to clean up, and ``_release`` needs to
         # see the device context to exit it. Without this early stash the expert's
-        # weights stay GPU-resident (8-9 GB for GGUF experts) until the cache's LRU
+        # weights stay GPU-resident (8-9 GB for quantized experts) until the cache's LRU
         # finally evicts them, masquerading as a memory leak.
         self._active_label = label
         self._active_info = info
@@ -291,10 +287,10 @@ def get(self, label: str) -> Any:
                     info.unload_from_vram(vram_bytes_to_free, keep_required_weights_in_vram=True)
                     TorchDevice.empty_cache()
 
-        # Apply LoRA patches for this expert. GGUF transformers need sidecar
-        # patching since direct patching of GGMLTensors isn't supported.
+        # Apply LoRA patches for this expert. Quantized transformers (GGUF, or an int8 checkpoint whose Linears
+        # hold buffers instead of parameters) need sidecar patching since direct patching cannot write into
+        # their weights. Decided on the loaded module tree, per expert: a pair may mix a quantized and a dense one.
         lora_factory = self._high_lora_factory if label == self.HIGH else self._low_lora_factory
-        is_quantized = self._high_is_quantized if label == self.HIGH else self._low_is_quantized
         lora_ctx: Any | None = None
         if lora_factory is not None:
             lora_ctx = LayerPatcher.apply_smart_model_patches(
@@ -303,7 +299,7 @@ def get(self, label: str) -> Any:
                 prefix=WAN_LORA_TRANSFORMER_PREFIX,
                 dtype=self._inference_dtype,
                 cached_weights=cached_weights,
-                force_sidecar_patching=is_quantized,
+                force_sidecar_patching=requires_sidecar_patching(model, info.config.format),
             )
             lora_ctx.__enter__()
 
@@ -608,7 +604,6 @@ def _run_diffusion(self, context: InvocationContext) -> torch.Tensor:
         # ``get_config`` doesn't touch the weights cache.
         high_model = self.transformer.transformer
         low_model = self.transformer.transformer_low_noise
-        low_config = context.models.get_config(low_model) if low_model is not None else None
         # FlowMatchEulerDiscreteScheduler stores num_train_timesteps in its config
         # (default 1000). Diffusers' WanPipeline computes:
         #   boundary_timestep = boundary_ratio * num_train_timesteps
@@ -622,13 +617,9 @@ def _run_diffusion(self, context: InvocationContext) -> torch.Tensor:
         # "no LoRAs on the low expert" — reusing the primary list here would
         # silently apply high-expert-only LoRAs (e.g. Lightning high-noise
         # distills) to the wrong expert.
-        # Quantized (GGUF) experts force sidecar patching so GGMLTensor weights
-        # aren't touched directly.
+        # Quantized experts (GGUF, int8) get sidecar patching; the swapper decides per expert once loaded.
         high_loras = self.transformer.loras
         low_loras = self.transformer.loras_low_noise
-        high_config = context.models.get_config(high_model)
-        high_is_quantized = high_config.format == ModelFormat.GGUFQuantized
-        low_is_quantized = low_config.format == ModelFormat.GGUFQuantized if low_config is not None else False
 
         def high_lora_factory() -> Iterable[PatchSpec]:
             return self._lora_iterator(context, high_loras)
@@ -651,8 +642,6 @@ def low_lora_factory() -> Iterable[PatchSpec]:
                 inference_dtype=inference_dtype,
                 high_lora_factory=high_lora_factory if high_loras else None,
                 low_lora_factory=low_lora_factory if low_loras else None,
-                high_is_quantized=high_is_quantized,
-                low_is_quantized=low_is_quantized,
                 working_mem_bytes=working_mem_bytes,
                 max_resident_model_bytes=(
                     WAN_MAX_RESIDENT_TRANSFORMER_BYTES if working_mem_bytes is not None else None

invokeai/app/invocations/wan/wan_video_denoise.py

@@ -37,7 +37,7 @@
     _validate_spatial_dimensions,
 )
 from invokeai.app.services.shared.invocation_context import InvocationContext
-from invokeai.backend.model_manager.taxonomy import BaseModelType, ModelFormat, WanVariantType
+from invokeai.backend.model_manager.taxonomy import BaseModelType, WanVariantType
 from invokeai.backend.patches.layer_patcher import PatchSpec
 from invokeai.backend.stable_diffusion.diffusers_pipeline import PipelineIntermediateState
 from invokeai.backend.stable_diffusion.diffusion.conditioning_data import WanConditioningInfo
@@ -290,17 +290,13 @@ def _run_diffusion(self, context: InvocationContext) -> torch.Tensor:
 
         high_model = self.transformer.transformer
         low_model = self.transformer.transformer_low_noise
-        low_config = context.models.get_config(low_model) if low_model is not None else None
         num_train_timesteps = int(scheduler.config.num_train_timesteps)
         boundary_timestep = self.transformer.boundary_ratio * num_train_timesteps if low_model is not None else None
 
         # No fallback between the LoRA lists — see the matching comment in
         # wan_denoise: an empty low list means "no LoRAs on the low expert".
         high_loras = self.transformer.loras
         low_loras = self.transformer.loras_low_noise
-        high_config = context.models.get_config(high_model)
-        high_is_quantized = high_config.format == ModelFormat.GGUFQuantized
-        low_is_quantized = low_config.format == ModelFormat.GGUFQuantized if low_config is not None else False
 
         def high_lora_factory() -> Iterable[PatchSpec]:
             return proxy._lora_iterator(context, high_loras)
@@ -320,8 +316,6 @@ def low_lora_factory() -> Iterable[PatchSpec]:
                 inference_dtype=inference_dtype,
                 high_lora_factory=high_lora_factory if high_loras else None,
                 low_lora_factory=low_lora_factory if low_loras else None,
-                high_is_quantized=high_is_quantized,
-                low_is_quantized=low_is_quantized,
                 working_mem_bytes=working_mem_bytes,
                 max_resident_model_bytes=(
                     WAN_MAX_RESIDENT_TRANSFORMER_BYTES if working_mem_bytes is not None else None