[DO NOT MERGE] Upstream feb 26 by guapisolo · Pull Request #15 · radixark/Megatron-LM

guapisolo · 2026-03-02T02:04:39Z

What does this PR do ?

⚠️ For major changes (either in lines of code or in its impact), please make sure to first share a design doc with the team. If you're unsure what's the best way to do so, contact the @mcore-oncall.

Contribution process

flowchart LR
    A[Pre-checks] --> B[PR Tests]
    subgraph Code Review/Approval
        C1[Expert Review] --> C2[Final Review]
    end
    B --> C1
    C2 --> D[Merge]

Pre-checks

I want this PR in a versioned release and have added the appropriate Milestone (e.g., Core 0.8)
I have added relevant unit tests
I have added relevant functional tests
I have added proper typing to my code Typing guidelines
I have added relevant documentation
I have run the autoformatter.sh on my PR

Code review

The following process is enforced via the CODEOWNERS file for changes into megatron/core. For changes outside of megatron/core, it is up to the PR author whether or not to tag the Final Reviewer team.

For MRs into `main` branch

Feel free to message or comment the @mcore-oncall to help accelerate your merge into main. The less complex your PR is, the faster it will be approved and merged!

(Step 1): Add PR label `Expert Review`

(Step 2): Collect the expert reviewers reviews

Attach the Expert Review label when your PR is ready for review.
GitHub auto-assigns expert reviewers based on your changes. They will get notified and pick up your PR soon.

⚠️ Only proceed to the next step once all reviewers have approved, merge-conflict are resolved and the CI is passing.
Final Review might get declined if these requirements are not fulfilled.

(Step 3): Final Review

Add Final Review label
GitHub auto-assigns final reviewers based on your changes. They will get notified and pick up your PR soon.

(Optional Step 4): Cherry-pick into release branch

If this PR also needs to be merged into core_r* release branches, after this PR has been merged, select Cherry-pick to open a new PR into the release branch.

For MRs into `dev` branch

The proposed review process for `dev` branch is under active discussion.

MRs are mergable after one approval by either eharper@nvidia.com or zijiey@nvidia.com.

Merging your PR

Any member of core-adlr and core-nemo will be able to merge your PR.

Co-authored-by: Philip Petrakian <ppetrakian@nvidia.com> Co-authored-by: oliver könig <okoenig@nvidia.com>

Signed-off-by: oliver könig <okoenig@nvidia.com>

Signed-off-by: Charlie Truong <chtruong@nvidia.com>

Signed-off-by: oliver könig <okoenig@nvidia.com>

Co-authored-by: root <root@gpu-h100-0348.cm.cluster> Co-authored-by: root <root@gpu-h100-0193.cm.cluster> Co-authored-by: root <root@gpu-h100-0082.cm.cluster> Co-authored-by: root <root@gpu-h100-0495.cm.cluster> Co-authored-by: William Dykas <wdykas@cw-pdx-cs-001-vscode-02.cm.cluster> Co-authored-by: root <root@gpu-h100-0213.cm.cluster> Co-authored-by: root <root@gpu-h100-0435.cm.cluster> Co-authored-by: root <root@gpu-h100-0188.cm.cluster> Co-authored-by: root <root@gpu-h100-0032.cm.cluster> Co-authored-by: root <root@gpu-h100-0023.cm.cluster> Co-authored-by: root <root@gpu-h100-0368.cm.cluster> Co-authored-by: root <root@gpu-h100-0203.cm.cluster> Co-authored-by: root <root@gpu-h100-0229.cm.cluster> Co-authored-by: root <root@gpu-h100-0123.cm.cluster> Co-authored-by: root <root@gpu-h100-0217.cm.cluster> Co-authored-by: root <root@gpu-h100-0496.cm.cluster> Co-authored-by: root <root@gpu-h100-0261.cm.cluster>

…ernorm. (NVIDIA#2434) Co-authored-by: Yuzhong Wang <yuzhongw@nvidia.com>

Signed-off-by: Hongbin Liu <hongbinl@nvidia.com>

Signed-off-by: Youngeun Kwon <youngeunk@nvidia.com> Co-authored-by: Youngeun Kwon <youngeunk@nvidia.com>

Signed-off-by: Keshav Santhanam <ksanthanam@nvidia.com>

This reverts commit ffbc43f.

Signed-off-by: oliver könig <okoenig@nvidia.com>

Signed-off-by: Jimmy Zhang <jiemingz@nvidia.com>

Co-authored-by: Dmytro Pykhtar <37850217+dimapihtar@users.noreply.github.com> Co-authored-by: Xin Yao <xiny@nvidia.com>

Co-authored-by: Rabeeh Mahabadi <rkarimimahab@nb-hel-cs-001-vscode-02.cm.cluster> Co-authored-by: Sanjeev Satheesh <sasatheesh@nvidia.com> Co-authored-by: Deepak Narayanan <dnarayanan@nvidia.com>

Signed-off-by: Santosh Bhavani <santosh.bhavani@live.com> Co-authored-by: Xin Yao <xiny@nvidia.com>

…offload` (NVIDIA#2874)

Co-authored-by: Maanu Grover <109391026+maanug-nv@users.noreply.github.com>

Signed-off-by: Deepak Narayanan <dnarayanan@nvidia.com>

Co-authored-by: Dmytro Pykhtar <37850217+dimapihtar@users.noreply.github.com>

…VIDIA#3194)" This reverts commit e836e62.

This reverts commit 300d1b6.

Co-authored-by: Antoni-Joan Solergibert <asolergibert@nvidia.com>

Signed-off-by: Hollow Man <hollowman@opensuse.org> Co-authored-by: Dmytro Pykhtar <37850217+dimapihtar@users.noreply.github.com> Co-authored-by: Xin Yao <xiny@nvidia.com>

Co-authored-by: Dmytro Pykhtar <37850217+dimapihtar@users.noreply.github.com>

Signed-off-by: dimapihtar <dpihtar@gmail.com>

… load-exchange-algo (NVIDIA#2161)

Signed-off-by: Faradawn Yang <73060648+faradawn@users.noreply.github.com>

Signed-off-by: Charlie Truong <chtruong@nvidia.com>

…IA#3425)

…#3419)

Signed-off-by: Keshav Santhanam <ksanthanam@nvidia.com>

…VIDIA#3424)

Signed-off-by: mikail <mkhona@nvidia.com>

Signed-off-by: Charlie Truong <chtruong@nvidia.com>

Signed-off-by: oliver könig <okoenig@nvidia.com> Signed-off-by: Charlie Truong <chtruong@nvidia.com> Signed-off-by: Hongbin Liu <hongbinl@nvidia.com> Signed-off-by: Youngeun Kwon <youngeunk@nvidia.com> Signed-off-by: Keshav Santhanam <ksanthanam@nvidia.com> Signed-off-by: Jimmy Zhang <jiemingz@nvidia.com> Signed-off-by: Santosh Bhavani <santosh.bhavani@live.com> Signed-off-by: Deepak Narayanan <dnarayanan@nvidia.com> Signed-off-by: Hollow Man <hollowman@opensuse.org> Signed-off-by: Robin Zhang <robinz@nvidia.com> Signed-off-by: jinliangl <jinliangl@nvidia.com> Signed-off-by: Maanu Grover <maanug@nvidia.com> Signed-off-by: dimapihtar <dpihtar@gmail.com> Signed-off-by: xiaoxi-wangfj <690912414@qq.com> Signed-off-by: skydoorkai <htsantaclara@163.com> Signed-off-by: Asha Anoosheh <aanoosheh@nvidia.com> Signed-off-by: meg miranda <mmiranda@nvidia.com> Signed-off-by: Keval Morabia <28916987+kevalmorabia97@users.noreply.github.com> Signed-off-by: sajadn <snorouzi@nvidia.com> Signed-off-by: lit <lit@nvidia.com> Signed-off-by: Faradawn Yang <73060648+faradawn@users.noreply.github.com> Signed-off-by: Cory Ye <cye@nvidia.com> Signed-off-by: adithyare <adithyare@nvidia.com> Signed-off-by: Soumye Singhal <soumyes@cw-dfw-cs-001-dc-01.cm.cluster> Signed-off-by: Ahmad Kiswani <kiswani.ahmad@gmail.com> Signed-off-by: mikail <mkhona@nvidia.com> Co-authored-by: HaochenYuan <106647990+HaochenYuan@users.noreply.github.com> Co-authored-by: Philip Petrakian <ppetrakian@nvidia.com> Co-authored-by: oliver könig <okoenig@nvidia.com> Co-authored-by: Duncan Riach <33532941+duncanriach@users.noreply.github.com> Co-authored-by: yobi byte <yobibyte@users.noreply.github.com> Co-authored-by: Charlie Truong <chtruong@nvidia.com> Co-authored-by: wdykas <73254672+wdykas@users.noreply.github.com> Co-authored-by: root <root@gpu-h100-0348.cm.cluster> Co-authored-by: root <root@gpu-h100-0193.cm.cluster> Co-authored-by: root <root@gpu-h100-0082.cm.cluster> Co-authored-by: root <root@gpu-h100-0495.cm.cluster> Co-authored-by: William Dykas <wdykas@cw-pdx-cs-001-vscode-02.cm.cluster> Co-authored-by: root <root@gpu-h100-0213.cm.cluster> Co-authored-by: root <root@gpu-h100-0435.cm.cluster> Co-authored-by: root <root@gpu-h100-0188.cm.cluster> Co-authored-by: root <root@gpu-h100-0032.cm.cluster> Co-authored-by: root <root@gpu-h100-0023.cm.cluster> Co-authored-by: root <root@gpu-h100-0368.cm.cluster> Co-authored-by: root <root@gpu-h100-0203.cm.cluster> Co-authored-by: root <root@gpu-h100-0229.cm.cluster> Co-authored-by: root <root@gpu-h100-0123.cm.cluster> Co-authored-by: root <root@gpu-h100-0217.cm.cluster> Co-authored-by: root <root@gpu-h100-0496.cm.cluster> Co-authored-by: root <root@gpu-h100-0261.cm.cluster> Co-authored-by: GitHub Actions <github-actions[bot]@users.noreply.github.com> Co-authored-by: Jiayi Yan <66017932+1195343015@users.noreply.github.com> Co-authored-by: Yuzhong Wang <yuzhongw@nvidia.com> Co-authored-by: Hongbin Liu <lhb8125@users.noreply.github.com> Co-authored-by: Youngeun Kwon <youngeunk@nvidia.com> Co-authored-by: Keshav Santhanam <ksanthanam@nvidia.com> Co-authored-by: Jimmy Zhang <133159885+jiemingz@users.noreply.github.com> Co-authored-by: tgkyrie <74066353+tgkyrie@users.noreply.github.com> Co-authored-by: Dmytro Pykhtar <37850217+dimapihtar@users.noreply.github.com> Co-authored-by: Xin Yao <xiny@nvidia.com> Co-authored-by: rkarimimahab <rkarimimahab@nvidia.com> Co-authored-by: Rabeeh Mahabadi <rkarimimahab@nb-hel-cs-001-vscode-02.cm.cluster> Co-authored-by: Sanjeev Satheesh <sasatheesh@nvidia.com> Co-authored-by: Deepak Narayanan <dnarayanan@nvidia.com> Co-authored-by: Santosh Bhavani <santosh.bhavani@live.com> Co-authored-by: Ahmad Kiswani <kiswani.ahmad@gmail.com> Co-authored-by: Li Tao <lit@nvidia.com> Co-authored-by: Maanu Grover <109391026+maanug-nv@users.noreply.github.com> Co-authored-by: mvirts <mvirts@gmail.com> Co-authored-by: Antoni-Joan Solergibert <asolergibert@nvidia.com> Co-authored-by: ℍ𝕠𝕝𝕝𝕠𝕨 𝕄𝕒𝕟 <hollowman@opensuse.org> Co-authored-by: Robin Zhang <robinz@nvidia.com> Co-authored-by: Sheng Fu <shengf@nvidia.com> Co-authored-by: Venmugil Elango <498703+venmugil@users.noreply.github.com> Co-authored-by: mathemakitten <helenn@nvidia.com> Co-authored-by: Jared Casper <155158+jaredcasper@users.noreply.github.com> Co-authored-by: Parth Mannan <38387286+parthmannan@users.noreply.github.com> Co-authored-by: Teodor-Dumitru Ene <34819528+tdene@users.noreply.github.com> Co-authored-by: Tong Liu <tongliu@nvidia.com> Co-authored-by: Li Jinliang <jinliangl@nvidia.com> Co-authored-by: Jinliang Li <jinliangl@pool0-01676.cm.cluster> Co-authored-by: Jinliang Li <jinliangl@cw-dfw-cs-001-vscode-01.cm.cluster> Co-authored-by: Yashaswi Karnati <144376261+yashaswikarnati@users.noreply.github.com> Co-authored-by: Nick Schank <nick@reflection.ai> Co-authored-by: Jeffrey Chen <jeffrey@reflection.ai> Co-authored-by: janEbert <janpabloe@nvidia.com> Co-authored-by: rj42 <lbkzman@gmail.com> Co-authored-by: Juntao Wang <juntaow@nvidia.com> Co-authored-by: Pingtian Li <158665726+Wohox@users.noreply.github.com> Co-authored-by: Chris Grimm <chris@reflection.ai> Co-authored-by: Chenhan D. Yu <5185878+ChenhanYu@users.noreply.github.com> Co-authored-by: Eric Harper <eharper@nvidia.com> Co-authored-by: xiaoxi-wangfj <690912414@qq.com> Co-authored-by: Jianbin Chang <shjwudp@gmail.com> Co-authored-by: c1lovez1 <141424951+c1lovez1@users.noreply.github.com> Co-authored-by: Zhang Haitao <htsantaclara@163.com> Co-authored-by: yeyu-nvidia <yeyu@nvidia.com> Co-authored-by: kwyss-nvidia <kwyss@nvidia.com> Co-authored-by: Jon Barker <jbarker@nvidia.com> Co-authored-by: Asha Anoosheh <aanoosheh@nvidia.com> Co-authored-by: Siddharth Singh <136645615+sidsingh-nvidia@users.noreply.github.com> Co-authored-by: megnvidia <mmiranda@nvidia.com> Co-authored-by: thecaptain789 <257642323+thecaptain789@users.noreply.github.com> Co-authored-by: thecaptain789 <thecaptain789@users.noreply.github.com> Co-authored-by: litianjian <litianjian@bytedance.com> Co-authored-by: Yan Bai <baiyan1996@icloud.com> Co-authored-by: xuwchen <xuwenc@nvidia.com> Co-authored-by: John St. John <jstjohn@users.noreply.github.com> Co-authored-by: Lawrence McAfee <85179052+lmcafee-nvidia@users.noreply.github.com> Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com> Co-authored-by: Robert Kirby <ArEsKay3@users.noreply.github.com> Co-authored-by: Siddharth Singh <sidsingh@nvidia.com> Co-authored-by: Robert Kirby <rkirby@cw-dfw-cs-001-vscode-01.cm.cluster> Co-authored-by: Teodor-Dumitru Ene <teodord.ene@gmail.com> Co-authored-by: Dennis(Zhenhuan) Liu <denliu@nvidia.com> Co-authored-by: Cursor <cursoragent@cursor.com> Co-authored-by: Keval Morabia <28916987+kevalmorabia97@users.noreply.github.com> Co-authored-by: Shanmugam Ramasamy <111910568+shanmugamr1992@users.noreply.github.com> Co-authored-by: vasunvidia <108759426+vasunvidia@users.noreply.github.com> Co-authored-by: Philip Petrakian <pgpetrak@gmail.com> Co-authored-by: Sajad Norouzi <sajad.n@gmail.com> Co-authored-by: Kunlun Li <94586211+kunlunl@users.noreply.github.com> Co-authored-by: xielaixin <xielx@shanghaitech.edu.cn> Co-authored-by: Robert Kirby <rkirby@nvidia.com> Co-authored-by: Ming <93323717+dndnda@users.noreply.github.com> Co-authored-by: liming127 <liming127@meituan.com> Co-authored-by: Jon Barker <jbarker@oci-hsg-cs-001-vscode-01.cm.cluster> Co-authored-by: helen ngo <helen.ngo14@gmail.com> Co-authored-by: Jenny Chen <jennifchen@nvidia.com> Co-authored-by: yueshen2016 <39203804+yueshen2016@users.noreply.github.com> Co-authored-by: Faradawn Yang <73060648+faradawn@users.noreply.github.com> Co-authored-by: Cory Ye <44509866+cspades@users.noreply.github.com> Co-authored-by: Adi Renduchintala <adithya.r@gmail.com> Co-authored-by: Soumye Singhal <soumyes@cw-dfw-cs-001-dc-01.cm.cluster> Co-authored-by: Seonjin Na <sna@nvidia.com> Co-authored-by: Seonmyeong Bak <sbak@nvidia.com> Co-authored-by: Mikail Khona (NVIDIA) <mkhona@nvidia.com>

…tp_size. (NVIDIA#3529) Co-authored-by: xiaotaoliu <xiaotaoliu@tencent.com> Co-authored-by: Yuzhong Wang <yuzhongw@nvidia.com> Co-authored-by: Zijie Yan <zijiey@nvidia.com>

Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>

Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com> Co-authored-by: Yueming Yuan <yym022502@gmail.com>

…VIDIA#5) Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>

- Detach output layer params to prevent MTP gradient flowing to output layer - Add mtp_kwargs interface for flexible MTP label/loss_mask passing - Roll mtp_labels and loss_mask for RL training compatibility Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com> Co-authored-by: Yueming Yuan <yym022502@gmail.com>

…yers (NVIDIA#10) - Add is_mtp flag to MoE layers and multi_token_prediction module - Bypass routing replay for MTP layers (MTP uses fresh routing) - Replace rdxa/dev's built-in RouterReplay with miles.utils.routing_replay: - moe_utils.py: use get_routing_replay_compute_topk() wrapper - router.py: use register_routing_replay() for initialization Co-authored-by: Yueming Yuan <yym022502@gmail.com>

Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com> Co-authored-by: Yueming Yuan <yym022502@gmail.com>

After bumping Megatron (rdxa/dev), colocated IPC weight update fails with torch.AcceleratorError: CUDA error: invalid argument during torch.multiprocessing serialization of CUDA tensors. Root cause: Megatron's new TMS hook (PR NVIDIA#3048) alters allocator behavior in training flow, causing allocations via cuMemCreate/cuMemMap which are incompatible with CUDA IPC (_share_cuda_() fails). Fix: resolve mapping.py and dynamic_context.py conflicts to isolate hook side effects so TMS/allocator state remains IPC-compatible during the weight update phase. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

- merge(): truncate dp_reshardable padding on optimizer/param_state path - load_parameter_state_from_dp_reshardable: tolerate missing 'padding' key - ShardedTensor: relax flattened_range to deprecation warning Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

HaochenYuan and others added 30 commits January 30, 2026 09:54

Fix for PR-2142 (NVIDIA#3165)

4cd9563

Co-authored-by: Philip Petrakian <ppetrakian@nvidia.com> Co-authored-by: oliver könig <okoenig@nvidia.com>

ci: Onboard more GB200 tests (NVIDIA#3145)

6de6362

Signed-off-by: oliver könig <okoenig@nvidia.com>

ci(hotfix): Alert for GB200 (NVIDIA#3168)

de15117

Signed-off-by: oliver könig <okoenig@nvidia.com>

Fix SFTDataset truncation bug (NVIDIA#3158)

7952d7e

Vitalyk/multiturn v2 (NVIDIA#3167)

b9ee19e

ci: Disable the api check for now (NVIDIA#3157)

b168849

Signed-off-by: Charlie Truong <chtruong@nvidia.com>

ci: Add DSv3 proxy (NVIDIA#3169)

a205538

Signed-off-by: oliver könig <okoenig@nvidia.com>

Update copy-pr-bot.yaml [skip ci]

fdc04f6

[Community][Main] fix(moe): Fix theoretical memory calculation of lay…

9ad5906

…ernorm. (NVIDIA#2434) Co-authored-by: Yuzhong Wang <yuzhongw@nvidia.com>

fix: Set --refit-method default to gloo (NVIDIA#3172)

5415e1d

[fix] Bug fix for offloading in evaluate() (NVIDIA#3043)

a976754

Signed-off-by: Hongbin Liu <hongbinl@nvidia.com>

Update copy-pr-bot.yaml [skip ci]

991c38f

cp: Fix: nccl-ub in ddp path (3181) into main (NVIDIA#3182)

5d0a7fd

Signed-off-by: Youngeun Kwon <youngeunk@nvidia.com> Co-authored-by: Youngeun Kwon <youngeunk@nvidia.com>

Miscellaneous inference cleanup (NVIDIA#2955)

ffbc43f

Signed-off-by: Keshav Santhanam <ksanthanam@nvidia.com>

Revert "Miscellaneous inference cleanup (NVIDIA#2955)"

0fe3232

This reverts commit ffbc43f.

ci: Fix DSv3 (NVIDIA#3188)

69a5c63

Signed-off-by: oliver könig <okoenig@nvidia.com>

Fix missing argument in MoELayer.forward() (NVIDIA#3133)

2fadde8

Signed-off-by: Jimmy Zhang <jiemingz@nvidia.com>

Fix H2D stream synchronization in optimizer offload (NVIDIA#3140)

ae67076

Co-authored-by: Dmytro Pykhtar <37850217+dimapihtar@users.noreply.github.com> Co-authored-by: Xin Yao <xiny@nvidia.com>

Add MTP support for hybrid models (NVIDIA#2363)

300d1b6

Co-authored-by: Rabeeh Mahabadi <rkarimimahab@nb-hel-cs-001-vscode-02.cm.cluster> Co-authored-by: Sanjeev Satheesh <sasatheesh@nvidia.com> Co-authored-by: Deepak Narayanan <dnarayanan@nvidia.com>

docs: improve Megatron-LM and Megatron Core descriptions (NVIDIA#3115)

dceb1fb

Signed-off-by: Santosh Bhavani <santosh.bhavani@live.com> Co-authored-by: Xin Yao <xiny@nvidia.com>

Handle step key correctly in checkpoint save with `--optimizer-cpu-…

f4502eb

…offload` (NVIDIA#2874)

mRoPE for MTP (NVIDIA#3114)

70719cd

Co-authored-by: Maanu Grover <109391026+maanug-nv@users.noreply.github.com>

Fix two minor bugs in MTP implementation for hybrid models (NVIDIA#3194)

e836e62

Signed-off-by: Deepak Narayanan <dnarayanan@nvidia.com>

Update README.md (NVIDIA#2111)

1362e4a

Co-authored-by: Dmytro Pykhtar <37850217+dimapihtar@users.noreply.github.com>

Revert "Fix two minor bugs in MTP implementation for hybrid models (N…

31d0c87

…VIDIA#3194)" This reverts commit e836e62.

Revert "Add MTP support for hybrid models (NVIDIA#2363)"

a0cc8ca

This reverts commit 300d1b6.

Fix bug in SFTDataset (NVIDIA#3185)

50546da

Co-authored-by: Antoni-Joan Solergibert <asolergibert@nvidia.com>

Fix several syntax error (NVIDIA#3004)

dff4189

Signed-off-by: Hollow Man <hollowman@opensuse.org> Co-authored-by: Dmytro Pykhtar <37850217+dimapihtar@users.noreply.github.com> Co-authored-by: Xin Yao <xiny@nvidia.com>

Fix for RL Test (NVIDIA#3148)

c4bea0a

Co-authored-by: Dmytro Pykhtar <37850217+dimapihtar@users.noreply.github.com>

dimapihtar and others added 27 commits February 24, 2026 12:41

remove encoder_and_decoder from enums (NVIDIA#3406)

7c7c9e1

Signed-off-by: dimapihtar <dpihtar@gmail.com>

chore(beep boop 🤖): Bump (main) (2026-02-24)

dd39eb5

Add knobs to choose process groups for fully-parallel-save / load and…

cb24802

… load-exchange-algo (NVIDIA#2161)

Fix off-by-2 error in RL sequence packing (NVIDIA#3551)

5f5f465

Skip unnecessary flattening for Save / Load Planner (NVIDIA#3263)

a8efd34

Multimodal: fix model provider (NVIDIA#3508)

5dc98a6

Signed-off-by: Faradawn Yang <73060648+faradawn@users.noreply.github.com>

docs: Enable nightly docs publish (NVIDIA#3546)

76b200c

Signed-off-by: Charlie Truong <chtruong@nvidia.com>

Ensure type-checker understands use of Submodules in unit tests (NVID…

f721069

…IA#3425)

Use copy_signature to preserve typing of pass-through methods (NVIDIA…

782e54b

…#3419)

Ensure type-checker understands use of Submodules in MTP (NVIDIA#3308)

3597312

Add mxfp8 quantization for inference linear layers (NVIDIA#3447)

44e27d0

Signed-off-by: Keshav Santhanam <ksanthanam@nvidia.com>

Add single-process checkpoint save to avoid forked multiprocessing (N…

2cac78b

…VIDIA#3424)

Fixed fp32 residuals (NVIDIA#3504)

08857d9

Signed-off-by: mikail <mkhona@nvidia.com>

[Dev] Fix MoE aux loss tracker hang with MTP enabled (NVIDIA#3400)

aa86018

ci: Remove multi-approval action from dev branch (NVIDIA#3576)

2b4b9c4

Signed-off-by: Charlie Truong <chtruong@nvidia.com>

Merge branch 'main' into dev

0ab47fa

[dev] fix(moe): fix the bug where gate was not sliced when kv_head < …

2e4a5d4

…tp_size. (NVIDIA#3529) Co-authored-by: xiaotaoliu <xiaotaoliu@tencent.com> Co-authored-by: Yuzhong Wang <yuzhongw@nvidia.com> Co-authored-by: Zijie Yan <zijiey@nvidia.com>

[1/8] fix: misc compatibility fixes for PyTorch and TE (NVIDIA#2)

9ca7af6

Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>

[2/8] feat: support partial checkpoint loading (NVIDIA#3)

307390e

Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>

[3/8] feat: add post-attention and post-MLP layernorm support (NVIDIA#4)

4cef384

Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com> Co-authored-by: Yueming Yuan <yym022502@gmail.com>

[4/8] fix: MLA RoPE triton kernel head indexing and v_dim=0 support (N…

470b592

…VIDIA#5) Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>

[7/8] feat: add INT4 fake QAT for MoE grouped linear (NVIDIA#9)

bd0b03b

Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com> Co-authored-by: Yueming Yuan <yym022502@gmail.com>

guapisolo force-pushed the upstream_feb_26 branch from 28ada11 to d08ead7 Compare March 2, 2026 02:15

guapisolo mentioned this pull request Mar 2, 2026

[Upstream] Update megatron version to dev branch (Feb 13) and rebase modifications #13

Merged

guapisolo force-pushed the miles-main branch from 992c0a2 to 038e8e5 Compare March 4, 2026 22:59

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[DO NOT MERGE] Upstream feb 26#15

[DO NOT MERGE] Upstream feb 26#15
guapisolo wants to merge 233 commits into
radixark:miles-mainfrom
guapisolo:upstream_feb_26

guapisolo commented Mar 2, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

20 participants

Conversation

guapisolo commented Mar 2, 2026

What does this PR do ?

Contribution process

Pre-checks

Code review

(Step 1): Add PR label Expert Review

(Step 2): Collect the expert reviewers reviews

(Step 3): Final Review

(Optional Step 4): Cherry-pick into release branch

Merging your PR

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

20 participants

(Step 1): Add PR label `Expert Review`