Skip to content

Fetch from nvidia Megatron-LM#5

Open
RaymondLi0 wants to merge 7640 commits into
ElementAI:load-iterfrom
NVIDIA:main
Open

Fetch from nvidia Megatron-LM#5
RaymondLi0 wants to merge 7640 commits into
ElementAI:load-iterfrom
NVIDIA:main

Conversation

@RaymondLi0

@RaymondLi0 RaymondLi0 commented Aug 3, 2022

Copy link
Copy Markdown

No description provided.

YangFei1990 and others added 29 commits May 30, 2026 08:02
@YangFei1990 @claude
Fix test failures for new inference APIs (#5068)
791a45f 
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@ko3n1g
fix(ci): set PYTHONUNBUFFERED=1 in JET workload env (#5072)
f33a51f 
Signed-off-by: oliver könig <okoenig@nvidia.com>
@wujingyue @claude
Preserve non-FSDP-unit buckets across AllGatherPipeline reset (#4717)
378d81f 
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@gdengk @Phlip79
Add opt-in MXFP8 LM-head output projection (#4825)
3f6a2ed 
Co-authored-by: a <a>
Co-authored-by: Philip Petrakian <ppetrakian@nvidia.com>
@github-actions
chore(beep boop 🤖): Bump (main) (2026-06-01)
24cc2a8
@ko3n1g @claude
fix(ci): bound JET pipeline polling with a watchdog to prevent indefi…
0a1726a 
…nite hangs (#5076)

Signed-off-by: oliver könig <okoenig@nvidia.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@ko3n1g
ci: prune old artifacts on cluster lustre during weekly/release runs (#…
595f697 
…5084)

Signed-off-by: oliver könig <okoenig@nvidia.com>
@ko3n1g @claude
ci(test): isolate ckpt-resume tensorboard per phase (#5074)
46f1af7 
Signed-off-by: oliver könig <okoenig@nvidia.com>
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@lhb8125
test: unmark EP A2A activation offload test flaky (#5009)
33da12c
@Phlip79
Change ownership groups (#5021)
e656940
@wujingyue @claude
test: skip mfsdp_fully_shard cases when world_size < mesh size (#4487)
496e1ff 
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@liding-nv
fix mimo optimizer checkpoint metadata restore (#4791)
80cf756 
Signed-off-by: Li Ding <liding@nvidia.com>
@liding-nv
[mimo] Support bridge fan-out for variable modality tokens (#5062)
de030fc 
Signed-off-by: Li Ding <liding@nvidia.com>
@yfw @claude
Add separate mtp_grad_scale_func for MTP loss scaling (#3459)
b8dcaab 
Signed-off-by: Yi-Fu Wu <yifu.wu@gmail.com>
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
@maanug-nv
[training migration] Migrate GPT builder (#4741)
5f88438 
Signed-off-by: Maanu Grover <maanug@nvidia.com>
@wujingyue
Make Mamba conv params direct mixer params (#4899)
35992ba 
Signed-off-by: Jingyue Wu <wujingyue@gmail.com>
@github-actions
Update copy-pr-bot.yaml [skip ci]
907ebcd
@Phlip79
Update oncall reviewer assignment (#5093)
f5da5ea
@yashaswikarnati
Pass explicit process groups to hybrid logging (#4781)
bb192ff
@Phlip79
Clean up top-level repository files (#5097)
7d9f259
@yuzhongw-nvidia
[main] fix(moe): Fix several bugs for DSA rope and spec. (#3026)
b0eb914 
Co-authored-by: Yuzhong Wang <yuzhongw@computelab-frontend-3.nvidia.com>
@yashaswikarnati
Move MIMO unit tests into models/mimo (#5063)
c69697d
@wujingyue
test: update DeepSeek FSDP2 GB200 memory golden (#5094)
54e1996
@janEbert
Remove DeepEP hardware limit check (#4846)
a1ac2f6
@balasaajay
Update transformer-engine dependency to revision 4220403 (#5112)
591605e
@ko3n1g @claude
ci: make CI resilient to pip/uv network timeouts (#5118)
44f254a 
Signed-off-by: oliver könig <okoenig@nvidia.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@ko3n1g @claude
ci: treat docker container-removal conflict as flaky (#5120)
d94604e 
Signed-off-by: oliver könig <okoenig@nvidia.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@conver334
Fix GDN DTensor splitting for FSDP checkpointing (#4843)
655a064 
Signed-off-by: conver334 <conver334@gmail.com>
@deepakn94 @claude
Fix MoE aux_loss / z_loss gradient scaling with TP > 1 (#5047)
9ef8a2a 
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
balasaajay and others added 30 commits July 2, 2026 13:43
@balasaajay
Fix smoke BERT/T5 test failures (#5629)
6608a05 
Signed-off-by: Ajay Balasa <abalasa@nvidia.com>
@wujingyue @claude
Add NCCL symmetric-memory staging to experimental FSDP (#5440)
25f6117 
Signed-off-by: Jingyue Wu <jingyuew@nvidia.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@balasaajay
Add smoke test notification functionality and update notify script (#…
d89aae5 
…5631)

Signed-off-by: Ajay Balasa <abalasa@nvidia.com>
@balasaajay
Update golden value files for GPT-3 weekly (#5459)
4828d65 
Signed-off-by: Ajay Balasa <abalasa@nvidia.com>
@github-actions
Update copy-pr-bot.yaml [skip ci]
06b07a1
@Phlip79
Ignore contributor DCO failures in Claude fix (#5625)
0823c73 
Signed-off-by: Philip Petrakian <ppetrakian@nvidia.com>
@github-actions
chore(beep boop 🤖): Bump (main) (2026-07-06)
6856424
@santhnm2
Pre-size the all-gather buffer for inference to max capacity (#5546)
959f3f1 
Signed-off-by: Keshav Santhanam <ksanthanam@nvidia.com>
@kevalmorabia97 @claude
Scatter embeddings for sequence parallelism in standalone LM forwards (…
7ee524e 
…#5628)

Signed-off-by: Keval Morabia <28916987+kevalmorabia97@users.noreply.github.com>
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
@deepakn94 @claude
Fix inter-document masking crash and NaNs with TP > 1 and micro_batch…
bf32f44 
…_size > 1 (#5635)

Signed-off-by: Deepak Narayanan <dnarayanan@nvidia.com>
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
@dimapihtar
add safe version of numpy.load (#5500)
1bb5ff5 
Signed-off-by: dimapihtar <dpykhtar@nvidia.com>
@sraman-rgb
Fuse shared expert MLP with grouped GEMM (#5604)
115ce7f 
Signed-off-by: Siddhartha Raman Sundara Raman <sraman@nvidia.com>
@mathemakitten
MoE routing analysis and metrics capture (#5220)
c1560d9 
Signed-off-by: Helen Ngo <helenn@nvidia.com>
@Phlip79
Add cspades to oncall rotation (#5695)
c797a5e 
Signed-off-by: Philip Petrakian <ppetrakian@nvidia.com>
@wujingyue
Add microbatch context helper (#5652)
5dbea46 
Signed-off-by: Jingyue Wu <wujingyue@gmail.com>
Signed-off-by: Jingyue Wu <jingyuew@nvidia.com>
@balasaajay
ci: Update test configurations to unify legacy scope names (#5316)
68f4c62 
Signed-off-by: Ajay Balasa <abalasa@nvidia.com>
@factnn @cspades
Fix Torch FSDP2 crash: add force_all_reduce kwarg to base finish_grad…
6ee356a 
…_sync (#4953)

Signed-off-by: factnn <166481866+factnn@users.noreply.github.com>
Co-authored-by: Cory Ye <44509866+cspades@users.noreply.github.com>
@wujingyue
Separate mFSDP v2 unit tests (#5640)
72a78d6 
Signed-off-by: Jingyue Wu <jingyuew@nvidia.com>
@github-actions
chore: rotate oncall schedule
465264b
@ZhiyuLi-Nvidia @claude
test(determinism): add determinism tests (#5041)
3ab71ee 
Signed-off-by: Zhiyu Li <zhiyul@NVIDIA.com>
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@dimapihtar
deprecate common strategy (#5160)
40b1fd3 
Signed-off-by: dimapihtar <dpykhtar@nvidia.com>
@ko3n1g @claude
ci: revert unify legacy scope names (#5316) (#5709)
509efe6 
Signed-off-by: oliver könig <okoenig@nvidia.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@Phlip79
Normalize CRLF in Claude fix commands (#5712)
65d6c23 
Signed-off-by: Philip Petrakian <ppetrakian@nvidia.com>
@shurkat-nvidia @deepakn94
Remove some barriers in save_checkpoint_and_time (#5557)
2a21b8e 
Signed-off-by: Skand Hurkat <shurkat@nvidia.com>
Co-authored-by: Deepak Narayanan <dnarayanan@nvidia.com>
@wujingyue
Add FSDP NVTX annotations (#5704)
91fcdfe 
Signed-off-by: Jingyue Wu <jingyuew@nvidia.com>
@YangFei1990
NCCL EP support (#5129)
2d7060f 
Signed-off-by: YangFei1990 <feiw@nvidia.com>
@balasaajay
Update base image to nvcr.io/nvidia/pytorch:26.06-py3 (#5632)
64bee49 
Signed-off-by: Ajay Balasa <abalasa@nvidia.com>
@lauradang @claude
Refactor RL rollout pipeline (#5491)
027fa4a 
Signed-off-by: Laura Dang <laurad@nvidia.com>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
@HollowMan6
[2/2] Wiring cuDNN fused DSA kernels support with THD, CP and IndexSh…
d1cab16 
…are (GLM5.2) (#5099)

Signed-off-by: Hollow Man <hollowman@opensuse.org>
@sidsingh-nvidia
Triton kernels - avoid recompilation and autotuning in prod (#5608)
f29c747 
Signed-off-by: Siddharth Singh <sidsingh@nvidia.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.