Conversation
doctorMcy
pushed a commit
that referenced
this pull request
Aug 24, 2026
…1d/seed bugs (modelscope#249) * feat: integrate Liger Kernel with fused-CE for NPU + fix conv1d/seed bugs Liger Kernel integration: - Add liger_builtin() bundle with _prefer_cann_on_npu() that swaps Liger Triton-on-Ascend kernels for CANN equivalents (faster on NPU) - Add LigerFusedLinearCrossEntropyLoss + TransformersFusedCEPatch that skips lm_head GEMM and [B,T,V] logits materialisation - Wire fused-CE in fsdp2.py/sp_fsdp_dense.py/ep_fsdp2_lora_qwen3_5_moe.py (task='fused_lm_ce' + set_loss + pre-loop barrier) - Add --enable-liger / --no-enable-fused-ce CLI flags Bug fixes: - causal_conv1d.py: fix npu_causal_conv1d_fn axis confusion when conv_dim == seq_len (e.g. 35B-A3B with max_length=8192); add x.shape[1] != D check to disambiguate [B,D,T] vs [B,T,D] layouts - framework.py: fix torch.manual_seed gated behind is_gpu_available() on NPU-only hosts; move to unconditional + independent NPU branch Tests: - test_qwen35_fla_bwd_precision.py: FLA on/off backward precision - test_liger_builtin.py, test_liger_fused_linear_ce.py, test_fused_ce_patch.py Benchmark (35B-A3B, 4x910B3, max_len=8192): - fused-CE saves 12-20 GB/NPU (20-30%) across all 3 scripts - Enables EP batch=8 (was OOM without fused-CE) - SP path 1.6x speedup from skipping logits SP all-gather * fix: address gemini code review feedback - Replace unsafe try/except ImportError with Torch.is_npu_available() / Torch.is_gpu_available() in fsdp2.py, sp_fsdp_dense.py, ep_fsdp2_lora_qwen3_5_moe.py pre-loop barrier (AttributeError not caught by except ImportError) - Gather bias via full_tensor() in LigerFusedLinearCrossEntropyLoss fallback (bias could be FSDP-sharded DTensor) - Convert _no_split_modules from set to list for HF Transformers compatibility - Gemini #1 (conv1d missing else) is a false positive: else branch exists at line 1114 * style: apply yapf formatting to sequence_parallel --------- Co-authored-by: ys2025-AI <ys2025-AI@users.noreply.github.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
PR type
PR information
Write the detail information belongs to this PR.
Experiment results
Paste your experiment result here(if needed).
loss指标采集：
[2026-07-07 11:57:45][INFO:twinkle] [Step 2/10] {'loss': '16.9013', 'accuracy': '0.67', 'correct_tokens': 2800, 'total_tokens': 4209, 'learning rate(param group 1)': '1.000000e-05', 'learning rate(param group 2)': '1.000000e-05', 'iters': 0, 'total time elapse': '3.2 minutes', 'speed': '0.00 iters/s'}
[2026-07-07 11:58:21][INFO:twinkle] [Step 4/10] {'loss': '16.8169', 'grad_norm': '0.001015', 'accuracy': '0.66', 'correct_tokens': 1545, 'total_tokens': 2345, 'learning rate(param group 1)': '9.779754e-06', 'learning rate(param group 2)': '9.779754e-06', 'iters': 1, 'total time elapse': '229 seconds', 'speed': '0.01 iters/s'}
[2026-07-07 11:59:09][INFO:twinkle] [Step 6/10] {'loss': '16.8721', 'grad_norm': '0.001015', 'accuracy': '0.68', 'correct_tokens': 2013, 'total_tokens': 2971, 'learning rate(param group 1)': '9.779754e-06', 'learning rate(param group 2)': '9.779754e-06', 'iters': 1, 'total time elapse': '277 seconds', 'speed': '0.01 iters/s'}
[2026-07-07 11:59:52][INFO:twinkle] [Step 8/10] {'loss': '16.8050', 'grad_norm': '0.001244', 'accuracy': '0.72', 'correct_tokens': 1882, 'total_tokens': 2632, 'learning rate(param group 1)': '9.140576e-06', 'learning rate(param group 2)': '9.140576e-06', 'iters': 2, 'total time elapse': '320 seconds', 'speed': '0.01 iters/s'}
单步loss计算debug日志：
�[36m(TransformersModel( pid=501187)�[0m [CTKD DEBUG] _compute_pkl_loss: student_logits shape: torch.Size([8, 482, 151936])
�[36m(TransformersModel( pid=501187)�[0m [CTKD DEBUG] _compute_pkl_loss: teacher_probs shape: torch.Size([8, 482, 151669])
�[36m(TransformersModel( pid=501187)�[0m [CTKD DEBUG] _compute_pkl_loss: labels shape: torch.Size([8, 482])
�[36m(TransformersModel( pid=501187)�[0m [CTKD DEBUG] _compute_pkl_loss: shift_student_logits shape: torch.Size([8, 481, 151936])
�[36m(TransformersModel( pid=501187)�[0m [CTKD DEBUG] _compute_pkl_loss: shift_teacher_probs shape: torch.Size([8, 481, 151669])
�[36m(TransformersModel( pid=501187)�[0m [CTKD DEBUG] _compute_pkl_loss: shift_labels shape: torch.Size([8, 481])
�[36m(TransformersModel( pid=501187)�[0m [CTKD DEBUG] _compute_pkl_loss: loss_mask sum: 1857.0, total: 3848
�[36m(TransformersModel( pid=501187)�[0m [CTKD DEBUG] _compute_pkl_loss: student_logits range: [-25.8750, 35.2500]
�[36m(TransformersModel( pid=501187)�[0m [CTKD DEBUG] _compute_pkl_loss: student_probs range: [0.000000, 1.000000]
�[36m(TransformersModel( pid=501187)�[0m [CTKD DEBUG] _compute_pkl_loss: student_probs sum per position: [1.0000, 1.0000]
�[36m(TransformersModel( pid=501187)�[0m [CTKD DEBUG] _compute_pkl_loss: teacher_probs range: [0.000000, 1.000000]
�[36m(TransformersModel( pid=501187)�[0m [CTKD DEBUG] _compute_pkl_loss: teacher_probs sum per position: [0.141023, 1.000000]
�[36m(TransformersModel( pid=501187)�[0m [CTKD DEBUG] _compute_pkl_loss: teacher_nonzero_count per position: [1, 512]
�[36m(TransformersModel( pid=501187)�[0m [CTKD DEBUG] _compute_pkl_loss: temperature: 0.8
�[36m(TransformersModel( pid=501187)�[0m [CTKD DEBUG] _compute_pkl_loss: projected_student_probs range: [0.000000, 1.000000]
�[36m(TransformersModel( pid=501187)�[0m [CTKD DEBUG] _compute_pkl_loss: projected_student_probs sum per position: [0.999996, 1.000001]
�[36m(TransformersModel( pid=501187)�[0m [CTKD DEBUG] _compute_pkl_loss: log_projected_student range: [-18.4207, 0.0000]
�[36m(TransformersModel( pid=501187)�[0m [CTKD DEBUG] _compute_pkl_loss: kl_contrib range: [-0.366220, 18.420681]
�[36m(TransformersModel( pid=501187)�[0m [CTKD DEBUG] _compute_pkl_loss: kl_div range (per position): [0.0380, 18.4207]
�[36m(TransformersModel( pid=501187)�[0m [CTKD DEBUG] _compute_pkl_loss: kl_div mean (unmasked): 17.6663
�[36m(TransformersModel( pid=501187)�[0m [CTKD DEBUG] _compute_pkl_loss: masked_kl range: [0.0000, 18.4207]
�[36m(TransformersModel( pid=501187)�[0m [CTKD DEBUG] _compute_pkl_loss: final loss: 16.861702