Long-RL: Scaling RL to Long Sequences (NeurIPS 2025)
InternEvo is an open-sourced lightweight training framework aims to support model pre-training without the need for extensive dependencies.
Large scale 4D parallelism pre-training for 🤗 transformers in Mixture of Experts *(still work in progress)*
Ulysses sequence-parallel all-to-all as a torch custom op, moved by the GPU copy engines into torch symmetric memory. Zero SM usage; 1.66-2.17x over torch.distributed on NVLink.
Fast and easy distributed model training examples.
Multi-GPU acceleration for MiniMax H3 video generation on NVIDIA V100 (sm_70). Ulysses sequence parallelism as a drop-in ComfyUI custom node — ~19 min to ~7 min on 8x V100.
MiniMax-H3 multi-GPU parallel inference for ComfyUI | 多卡并行加速节点:2-8 GPU Ulysses sequence parallel, bit-identical video+audio generation
Minimal tensor, sequence, and pipeline parallelism for LLaMA.
Implements sequence parallelism for Transformer layers using MPI4Python and multi-GPU acceleration, exploring scalability trade-offs and communication bottlenecks in distributed attention.
Add a description, image, and links to the sequence-parallelism topic page so that developers can more easily learn about it.
To associate your repository with the sequence-parallelism topic, visit your repo's landing page and select "manage topics."