Question: Minimum GPU Memory Required for Training MiniMax H3 Video LoRA

[aoliao12138]

Hi,

Could you please let me know the minimum GPU memory required to train a LoRA for MiniMax H3 Video without using NF4 quantization?

Thanks!

[mi804]

The DiT model alone is over 60 GB in size. In our tests, training on 124-frame videos at 480p requires approximately 78 GB of VRAM using the MiniMax-H3-FL2VA training script.

If you prefer not to train with a quantized model, you can use the pruned version instead. Its precision is close to that of the original DiT, while the training VRAM requirement is reduced to approximately 53 GB. Please refer to the MiniMax-H3-Pruned-FL2VA training script.

If you still run out of VRAM during training, you can try multi-GPU training with DeepSpeed, use quantization, or reduce the video resolution or number of frames.

[aoliao12138]

Thank you very much!