[model] feat: migrate seed_oss to transformers v5#662
Conversation
Migrate seed_oss via Pattern A (v4<->v5 coexist) + NPU sibling at 5.2.0 gate.
- Add seed_oss_{gpu,npu}_patch_gen_config.py and generated/patched_modeling_seed_oss_{gpu,npu}.py
- GPU: LigerRMSNorm + LigerSwiGLUMLP + liger rotary + fused-CE ForCausalLM.forward
- NPU: npu_fused_operator rotary + RMSNorm.forward + fused-CE ForCausalLM.forward
- __init__.py: branch on is_transformers_version_greater_or_equal_to("5.2.0");
v4 path keeps apply_veomni_seed_oss_patch; v5 path selects GPU/NPU generated
module via IS_NPU_AVAILABLE
- Tests: add seed_oss to _TEST_CASES_TRANSFORMERS_V5 and seed_oss_v5 e2e param
There was a problem hiding this comment.
Code Review
This pull request introduces support for the seed_oss model within the transformers v5 framework, providing optimized implementations for both NPU and GPU environments. The changes include auto-generated patched modeling files that utilize LigerKernel for GPU and fused operators for NPU, along with updated registration logic and new E2E tests. Review feedback identifies a version mismatch between the model registration gate and the tests, as well as a decorator conflict in the NPU modeling file that could overwrite the intended performance optimizations.
| SeedOssForTokenClassification, | ||
| SeedOssModel, | ||
| ) | ||
| if is_transformers_version_greater_or_equal_to("5.2.0"): |
There was a problem hiding this comment.
The version gate for the transformers v5 path is set to 5.2.0, but the associated tests in tests/e2e/test_e2e_parallel.py and tests/models/test_models_patch.py use a 5.0.0 threshold. This inconsistency means that for transformers versions between 5.0.0 and 5.2.0, the code will fall back to the v4 monkey-patching path (lines 39-49), which is likely incompatible with the v5 internal model structure and will cause test failures or runtime crashes. Please align the gate with the actual minimum supported transformers v5 version (likely 5.0.0).
| if is_transformers_version_greater_or_equal_to("5.2.0"): | |
| if is_transformers_version_greater_or_equal_to("5.0.0"): |
| # ====================================================================== | ||
|
|
||
|
|
||
| @use_kernel_forward_from_hub("RMSNorm") |
There was a problem hiding this comment.
The @use_kernel_forward_from_hub("RMSNorm") decorator is retained on the SeedOssRMSNorm class. In transformers, this decorator replaces the forward method of the class at import time if a compatible kernel is found. This will overwrite the NPU-fused forward implementation defined at line 67, negating the intended optimization or potentially causing crashes if the hub kernel (typically Triton/CUDA-based) is incompatible with the NPU execution environment. This decorator should be removed in the NPU-specific generated file.
| @use_kernel_forward_from_hub("RMSNorm") | |
| class SeedOssRMSNorm(nn.Module): |
What does this PR do?
Checklist Before Starting
tingyang/transformer_v5/STATUS.md[{modules}] {type}: {description}formatTest
API and Usage Example
Design & Code Changes
Checklist Before Submitting