Skip to content

[model] feat: migrate seed_oss to transformers v5#662

Draft
TimYangst wants to merge 1 commit intomainfrom
tingyang/migrate/seed_oss_v5
Draft

[model] feat: migrate seed_oss to transformers v5#662
TimYangst wants to merge 1 commit intomainfrom
tingyang/migrate/seed_oss_v5

Conversation

@TimYangst
Copy link
Copy Markdown
Collaborator

What does this PR do?

Migrate seed_oss from the v4 runtime monkey-patch path to the transformers v5 patchgen + self-contained generated modeling path, following Pattern A (v4↔v5 coexist) with an NPU sibling at the 5.2.0 gate. Mirrors the qwen3 (GPU coexist) and glm_moe_dsa (NPU patchgen) references.

Checklist Before Starting

  • Search for relative PRs/issues and link here: part of the transformers v5 migration queue in tingyang/transformer_v5/STATUS.md
  • PR title follows [{modules}] {type}: {description} format

Test

  • python -m veomni.patchgen.check_patchgen — green for both patched_modeling_seed_oss_{gpu,npu}.py
  • pytest tests/models/test_models_patch.py -k seed_oss -v — passed
  • make quality — clean
  • E2E (tests/e2e/test_e2e_parallel.py -k seed_oss_v5) added under _v5_only mark; requires multi-GPU runner — will be exercised in CI

API and Usage Example

N/A — loader registry output is unchanged; v4 runtime callers keep working against transformers==4.57.3.

Design & Code Changes

  • New seed_oss_gpu_patch_gen_config.py: LigerRMSNorm + LigerSwiGLUMLP via create_patch_from_external, liger apply_rotary_pos_emb via @replace_function, SeedOssForCausalLM.forward fused-CE via @override_method.
  • New seed_oss_npu_patch_gen_config.py: npu_fused_operator rotary + SeedOssRMSNorm.forward + fused-CE ForCausalLM.forward.
  • Regenerated generated/patched_modeling_seed_oss_{gpu,npu}.{py,diff} via python -m veomni.patchgen.run_codegen.
  • __init__.py: gate on is_transformers_version_greater_or_equal_to("5.2.0"); v5 branch selects GPU/NPU generated module via IS_NPU_AVAILABLE; v4 branch preserves apply_veomni_seed_oss_patch for back-compat.
  • Tests: tests/models/test_models_patch.py — add seed_oss entry to _TEST_CASES_TRANSFORMERS_V5; tests/e2e/test_e2e_parallel.py — add seed_oss_v5 pytest.param with _v5_only marker.
  • No edits to modeling_seed_oss.py, gpu_patch.py, npu_patch.py — the v4 surface is untouched.

Checklist Before Submitting

  • Read the Contribute Guide
  • Applied pre-commit checks
  • Added/updated documentation
  • Added tests to CI workflow (or explained why not feasible)

Migrate seed_oss via Pattern A (v4<->v5 coexist) + NPU sibling at 5.2.0 gate.

- Add seed_oss_{gpu,npu}_patch_gen_config.py and generated/patched_modeling_seed_oss_{gpu,npu}.py
- GPU: LigerRMSNorm + LigerSwiGLUMLP + liger rotary + fused-CE ForCausalLM.forward
- NPU: npu_fused_operator rotary + RMSNorm.forward + fused-CE ForCausalLM.forward
- __init__.py: branch on is_transformers_version_greater_or_equal_to("5.2.0");
  v4 path keeps apply_veomni_seed_oss_patch; v5 path selects GPU/NPU generated
  module via IS_NPU_AVAILABLE
- Tests: add seed_oss to _TEST_CASES_TRANSFORMERS_V5 and seed_oss_v5 e2e param
@github-actions github-actions Bot added the hf_v5 Related for transformers v5 label Apr 15, 2026
@TimYangst TimYangst marked this pull request as draft April 15, 2026 22:56
Copy link
Copy Markdown
Contributor

@gemini-code-assist gemini-code-assist Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request introduces support for the seed_oss model within the transformers v5 framework, providing optimized implementations for both NPU and GPU environments. The changes include auto-generated patched modeling files that utilize LigerKernel for GPU and fused operators for NPU, along with updated registration logic and new E2E tests. Review feedback identifies a version mismatch between the model registration gate and the tests, as well as a decorator conflict in the NPU modeling file that could overwrite the intended performance optimizations.

SeedOssForTokenClassification,
SeedOssModel,
)
if is_transformers_version_greater_or_equal_to("5.2.0"):
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

high

The version gate for the transformers v5 path is set to 5.2.0, but the associated tests in tests/e2e/test_e2e_parallel.py and tests/models/test_models_patch.py use a 5.0.0 threshold. This inconsistency means that for transformers versions between 5.0.0 and 5.2.0, the code will fall back to the v4 monkey-patching path (lines 39-49), which is likely incompatible with the v5 internal model structure and will cause test failures or runtime crashes. Please align the gate with the actual minimum supported transformers v5 version (likely 5.0.0).

Suggested change
if is_transformers_version_greater_or_equal_to("5.2.0"):
if is_transformers_version_greater_or_equal_to("5.0.0"):

# ======================================================================


@use_kernel_forward_from_hub("RMSNorm")
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

high

The @use_kernel_forward_from_hub("RMSNorm") decorator is retained on the SeedOssRMSNorm class. In transformers, this decorator replaces the forward method of the class at import time if a compatible kernel is found. This will overwrite the NPU-fused forward implementation defined at line 67, negating the intended optimization or potentially causing crashes if the hub kernel (typically Triton/CUDA-based) is incompatible with the NPU execution environment. This decorator should be removed in the NPU-specific generated file.

Suggested change
@use_kernel_forward_from_hub("RMSNorm")
class SeedOssRMSNorm(nn.Module):

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

hf_v5 Related for transformers v5

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant