Skip to content

[Roadmap] VeOmni 2026 Q2 roadmap #669

@FoolPlayer

Description

@FoolPlayer

📅 [Roadmap] VeOmni Q2 2026 Roadmap

💡 Overview

Following the successful restructuring and optimization of the VeOmni architecture in Q1 #268, the focus for Q2 shifts towards the official 0.2.0 release, broadening the open-source model ecosystem, deepening the integration with the verl training framework, and advancing Agent capabilities.


🎯 1. Primary Objective: Release v0.2.0

The main goal is to provide a stable, production-ready version for the community.

  • Dependency Alignment: Fully upgrade to transformers > 5.3.0 to support the latest features.
  • Kernel & Ops Optimization: Refine the registration mechanism for Kernels/Ops to improve extensibility and performance.

🤖 2. Open-Source Model Ecosystem

Expand compatibility to support the most influential recent model releases by codegen pattern:

  • Qwen Series: Full support for the latest Qwen models.
  • DeepSeek Series: Support for DeepSeek architectures and MoE specific optimizations.
  • GLM Series: Compatibility with the Zhipu GLM generations.
  • Kimi Series: Optimization for Moonshot's architecture.

🚀 3. Deep Integration: VeOmni + verl Training Pipeline

Collaborate closely with verl (Roadmap #5836) to build a more user-friendly and efficient training ecosystem.

🛠️ Ecosystem Construction

  • Rapid Model Support: Enable fast integration of new architectures.
  • Advanced Parallelism: Implement and optimize SP and EP support.
  • Initialization & Checkpointing: Support Meta Init for memory-efficient model loading and DCP (Distributed Checkpoint) for high-performance saving/loading.
  • Unified Kernel Registry: Standardize kernel registration across the training pipeline.

🔍 Scenario Exploration (VeOmni x verl)

  • Multimodal Training: Explore and adapt multimodal training workflows within the verl ecosystem.
  • Generation: Support for generative scenarios.

🧠 4. Agent & Workflow Evolution

Enhance the framework's intelligence by introducing more sophisticated Agent capabilities:

  • New Skillsets: Develop and integrate specialized skills for Agents.
  • Agent Workflow Overhaul: Design and implement advanced workflow management for complex multi-step tasks.

⚠️ 5. Deprecation & Sunsetting Plan

To maintain a clean and efficient codebase, we are phasing out several legacy components:

  • FSDP1: Will be gradually deprecated. Throughout the v0.2.x lifecycle, we will maintain its core usability, but no new features will be added.
  • Deprecated Tasks: v0.1.9 will serve as the final version supporting these legacy tasks. Try to use trainer.
  • Transformers v4 Models: Will be completely unsupported starting from v0.2.0.

🤝 6. Call for Community Contribution

We actively invite and encourage the community to join us in co-building VeOmni! Your contributions are highly welcome, including but not limited to the initiatives mentioned above. Whether you want to tackle an item on this roadmap, propose a new feature, optimize existing code, or improve documentation, we look forward to collaborating with you.


Note: This roadmap is a living document. Tasks and priorities may be adjusted based on community feedback and project progress.

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions