Skip to content

fix save_weights_fast_with_mtp#137

Merged
ISEEKYAN merged 1 commit into
ISEEKYAN:mainfrom
LiuXTao:xiaotaoliu/fix_mtp_save_fast_and_pp_first_layer_bug
May 14, 2026
Merged

fix save_weights_fast_with_mtp#137
ISEEKYAN merged 1 commit into
ISEEKYAN:mainfrom
LiuXTao:xiaotaoliu/fix_mtp_save_fast_and_pp_first_layer_bug

Conversation

@LiuXTao
Copy link
Copy Markdown
Contributor

@LiuXTao LiuXTao commented May 14, 2026

This PR includes two fixes:

  1. The first fix addresses an issue where, when MTP is enabled, save_weights_fast did not correctly handle the duplicated MTP embedding. As a result, the layer with this name was saved twice, which triggered the shard count check failure.;

  2. the other fixes an MTP construction error triggered when pp > 0 and num_layers_in_first_pipeline_stage is set.

image

@ISEEKYAN ISEEKYAN merged commit 5f31621 into ISEEKYAN:main May 14, 2026
0 of 2 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants