[Feature] Support for MiniMax-H3 distilled 4B student text encoder (MLP adapter projection)

[wxorx]

Feature Summary

Support for MiniMax-H3 distilled 4B student text encoder (MLP adapter projection)

Detailed Description

Background

I am trying to run the MiniMax-H3 model using the distilled 4B student text encoder qwen3vl-4b-h3student-Q4_K_M.gguf provided in the woodfireind/MiniMax-H3-GGUF-MiniStack
repository.
When attempting to generate a video, sd-cli crashes immediately during the conditioning phase with a dimension mismatch assertion failure.

The Problem

The crash occurs here:

GGML_ASSERT(context->ne[0] == config.text_dim) failed

Root Cause:

The MiniMax-H3 DiT expects text embeddings with a hidden dimension of 5120. However, the distilled 4B student model natively outputs a hidden dimension of 2560.
To bridge this gap in ComfyUI, the repository author provides an MLP projection adapter (te_adapter_v1.safetensors) which maps the student's hidden states into the 5120-dimensional slots expected by the DiT (2560 -> 4096 -> 5120). Currently, stable-diffusion.cpp passes the raw 2560-dimensional output directly to the DiT, triggering the assertion failure.

Reproduction Steps

Using the latest master branch of stable-diffusion.cpp with the MiniMax-H3 files from woodfireind/MiniMax-H3-GGUF-MiniStack:

./sd-cli \
    -M vid_gen \
    --diffusion-model ./Minimax-H3-GGUF-MiniStack/diffusion_models/MiniMax-H3-FL2VA-pruned-Q4_K_M.gguf \
    --vae ./Minimax-H3-GGUF-MiniStack/vae/minimax_h3_video_vae_fp16.safetensors \
    --llm ./Minimax-H3-GGUF-MiniStack/text_encoders/qwen3vl-4b-h3student-Q4_K_M.gguf \
    --fps 24 \
    --video-frames 56 \
    --cfg-scale 1.0 \
    --steps 20 \
    --diffusion-fa \
    --vae-on-cpu \
    --backend te=cpu \
    --offload-to-cpu \
    --rng cpu \
    -H 640 -W 480 \
    -v \
    -p "A person in a space suit walking up hill on dusty moon mountain."

Resulting Log snippet:

[INFO ] stable-diffusion.cpp:1846 - running in FLOW mode
[INFO ] denoiser.hpp:1051 - get_sigmas with discrete scheduler
[INFO ] stable-diffusion.cpp:4356 - sampling using Euler method
[DEBUG] conditioner.hpp:2882 - computing condition graph completed, taking 2144 ms
[INFO ] stable-diffusion.cpp:6516 - get_learned_condition completed, taking 2.14s
[INFO ] stable-diffusion.cpp:6878 - generate_video 480x640x56
/home/user/src/stable-diffusion.cpp/src/model/diffusion/minimax_h3.hpp:533: GGML_ASSERT(context->ne[0] == config.text_dim) failed
Aborted (core dumped)

Proposed Solution / Feature Request

Could we add support for loading and applying this external MLP adapter?
Ideally, this could be handled via a new CLI flag, for example: --llm-adapter (or integrated into --embeddings-connectors). The C++ engine would need to:

1. Load the te_adapter_v1.safetensors file (which contains the weights/biases for the 2560 -> 4096 -> 5120 linear layers).

2. Inject the MLP projection into the text conditioning compute graph right after the LLM forward pass and before feeding it into the DiT cross-attention/AdaLN layers.

Technical Notes & Reference Implementation for Maintainers

For whoever picks this up, the exact Python implementation (including the activation function and norm logic) is available in the ComfyUI custom node provided by the model author:

• Repository: woodfireind/MiniMax-H3-GGUF-MiniStack

• Python Source Code Location: custom_nodes/h3_small_te/nodes.py

Important Quirk (Gotcha):

The model card for this stack explicitly notes that the student TE uses an "Identity final-norm". This means the C++ implementation must likely skip the standard final RMSNorm that is normally applied to Qwen LLM hidden states before passing the raw output tensor into the adapter's first linear layer.

Thank you

for your incredible work on stable-diffusion.cpp and the recent Day-1 support for MiniMax-H3! This adapter support would make the 4B student model fully usable on VRAM-constrained local hardware.

Alternatives you considered

No response

Additional context

No response