diff --git a/knowledge/repos/vllm-omni/ci/rules-amd.md b/knowledge/repos/vllm-omni/ci/rules-amd.md index 7422b541..09ef71f6 100644 --- a/knowledge/repos/vllm-omni/ci/rules-amd.md +++ b/knowledge/repos/vllm-omni/ci/rules-amd.md @@ -1,7 +1,7 @@ --- title: "AMD/ROCm CI 规则" created: 2026-09-05 -updated: 2026-09-10 +updated: 2026-09-17 type: rule tags: [vllm-omni, ci] sources: ["PR #6704", "PR #6830", "PR #6884", .buildkite/amd/, tests/helpers/clean.py, tests/helpers/stage_config.py, tests/buildkite/test_amd_pipeline.py, tests/e2e/offline_inference/test_qwen3_omni_colocate_async.py, "PR #7234"] @@ -42,3 +42,10 @@ confidence: high - 强制：`mi300_1` 一类单卡 model job 的 pytest marker 必须 `not (cards_2 or … or cards_8)`，同时保留无 `cards_1` 的 legacy 单卡用例；真正需要双卡的用例（如 LTX2 Ulysses parity）改到已有双卡 lane，并声明 `rocm` 资源与 `device_count >= world_size` 早失败。 - 禁止：让 `cards_2+` 测试在单卡 worker 上 spawn rank1→GPU1 导致 `invalid device ordinal`；用邻近 green shard 宣称 multi-GPU routing 已修好。 - 验收：pipeline argv/collection 断言单卡 job 排除 multi-card markers、双卡 job 收集目标文件；硬件 marker helper 覆盖 ROCm 声明。^[PR #7234] + +## OMNI-CI-2i — AMD entrypoints GPU ready job 必须单次 pytest、有界超时并检测进程泄漏 + +- 触发：修改 `.buildkite/amd/test-amd-ready.yml` 的 entrypoints/R2-02 GPU coverage、artifact 根路径，或 teardown 进程快照比较。 +- 强制：`mi300_1` NonBlocking job 设 `VLLM_WORKER_MULTIPROC_METHOD=spawn`，marker 排除 `cards_2`–`cards_8`，只跑一次 verbose pytest（禁止第二趟 `--collect-only`）。内层 `timeout` 短于 Buildkite step，为 teardown/artifact 留窗口。artifact 根在 Buildkite checkout；前后 `ps` 快照用 PID+start-time 身份比较，泄漏则 fail closed。零 collection 不得被允许通过。 +- 禁止：依赖 PID-only 比较掩盖 reuse；把 CUDA pipeline 定义当作 ROCm ready 覆盖；用邻近 green shard 宣称本 job 合同已满足。 +- 验收：结构断言单次 pytest、超时信封、marker、六类 artifact 与 `process-cleanup` PASS；真实 MI300 跑通 selected node。^[PR #7398] diff --git a/knowledge/repos/vllm-omni/components/diffusion/rules-system-runtime.md b/knowledge/repos/vllm-omni/components/diffusion/rules-system-runtime.md index 6bec90c7..100810b1 100644 --- a/knowledge/repos/vllm-omni/components/diffusion/rules-system-runtime.md +++ b/knowledge/repos/vllm-omni/components/diffusion/rules-system-runtime.md @@ -1,7 +1,7 @@ --- title: "Diffusion paged cache 与系统运行时规则" created: 2026-09-03 -updated: 2026-09-09 +updated: 2026-09-17 type: rule tags: [vllm-omni, components, diffusion] sources: ["PR #5255", "PR #5344", "PR #5543", "PR #5838", "PR #6094", "PR #6102", "PR #6385", "PR #6340", "PR #6714", "PR #6814", "PR #6563", "PR #5716", "PR #6786", vllm_omni/diffusion/attention/, vllm_omni/diffusion/attention/parallel/ulysses.py, vllm_omni/diffusion/attention/parallel/ring_kernels.py, vllm_omni/diffusion/diffusion_kv/, vllm_omni/diffusion/distributed/cfg_parallel.py, vllm_omni/diffusion/distributed/parallel_state.py, vllm_omni/diffusion/worker/diffusion_model_runner.py, vllm_omni/platforms/interface.py, vllm_omni/platforms/npu/platform.py, tests/diffusion/diffusion_kv/, tests/diffusion/distributed/test_cfg_parallel.py, tests/diffusion/attention/test_piecewise_attn.py, tests/diffusion/attention/test_ulysses_uaa.py, "PR #5491", "PR #5194", "vllm_omni/diffusion/data.py", "vllm_omni/diffusion/utils/hf_utils.py", "PR #7041", "PR #6463", "PR #6844"] @@ -226,3 +226,10 @@ confidence: high - 强制：一次 `generate()` 驱动整段 rollout：`prepare_encode` 只跑一次，随后多轮 `denoise_step`/`step_scheduler`，由 `post_decode` 产出每个 AR chunk。request-mode 与 stepwise 必须共用同一份 block/DMD math，禁止维护两套可漂移实现。runner 仅为实现该合同的 pipeline 打开 `step_execution`；绑定 runner-owned KV 时 `session_id == request_id`；错误路径 fail-closed 释放 session，完成或 scheduler abort 时退役。持有 AR-Diffusion paged KV 的 stage 必须保持 `max_num_seqs=1`。 - 禁止：用多次 `generate()`/tick 冒充同一 session；在 denoise step 内提交本应属于 `post_decode` 的 clean-x0 KV commit；或把逐步 camera/prompt mid-request interaction 写成已由本合同覆盖。 - 验收：覆盖 stepwise 与 request-mode 的共享 math 边界、session bind/release、abort/completion 退役，以及 `max_num_seqs=1` 拓扑；request-scoped camera script 等模型字段另由模型 owner 验收，不得外推为通用 mid-request interaction。^[PR #6844] + +## DIFF-4ad — AR 条件编码历史必须会话持有、双份计入并在终止边界释放 + +- 触发：修改 AR-Diffusion realtime/stepwise 条件编码、Wan VAE encoder cache、session admission 字节预算，或 temporal RoPE 超出预计算表。 +- 强制：跨 block 推进时保留 causal encoder history（committed + in-flight），每 session 只驻留当前 condition block 与有界 cache；admission 必须计入两份 encoder history 与 streaming-decode 字节。reset/close 必须释放 encoder cache。超出预计算 RoPE 表的 temporal 位置按绝对位置即时算 cos/sin，不得扩张常驻表。条件编码要求 unpatched、非 tiled Wan encoder。 +- 禁止：用固定像素/latent 帧上限冒充无界 realtime；只预算 self-KV 而漏算 encoder cache；在失败 block 后提交 pending history；把本变更写成已解决共享 paged-KV 长度上限。 +- 验收：因果条件递进、session 隔离/清理、失败不提交、内存会计、tick/stepwise 一致，以及 RoPE 越界且 cache 尺寸固定。^[PR #6838] diff --git a/knowledge/repos/vllm-omni/components/model-executor/rules-output-contract.md b/knowledge/repos/vllm-omni/components/model-executor/rules-output-contract.md index ac58b57d..535a7993 100644 --- a/knowledge/repos/vllm-omni/components/model-executor/rules-output-contract.md +++ b/knowledge/repos/vllm-omni/components/model-executor/rules-output-contract.md @@ -1,7 +1,7 @@ --- title: "Omni 输出类型合同" created: 2026-09-04 -updated: 2026-09-04 +updated: 2026-09-17 type: rule tags: [vllm-omni, components, model-executor] sources: ["PR #5146", "PR #6152", vllm_omni/outputs/] @@ -27,3 +27,17 @@ confidence: high - 验收：用真实 `RequestOutput` 与 `OmniRequestOutput` 做属性 parity 测试；覆盖新字段默认值为 `None`，以及 `ec_transfer_params`、`num_cache_creation_tokens` 等非空值经 `from_stage_output()` 后保持不变。 ^[PR #6152] 相关执行流见 [model-executor architecture](architecture.md)；跨 stage 合同见 [bridge/batch 规则](rules-bridge-batch.md)。 + +## EXEC-7c — 已知 sample-rate 键必须按最新快照合并，不得当生成内容累积 + +- 触发：修改 `MultimodalPayload.merged_with`、DELTA/CUMULATIVE 音频输出合并，或 `sr`/`sample_rate`/`audio_sample_rate` 的分区与巩固。 +- 强制：这三个键是 metadata 快照，不是 waveform 内容。合并前先从 incoming 捕获快照，再写入 metadata 分区一次；缺失键保留既有值。tensor/scalar 表示切换时必须从 tensors 分区移除同名残留。audio/latent 内容继续按既有策略累积。 +- 禁止：把 sample-rate 追加成 list 直到 consolidate；让非终态 DELTA 输出保留增长的标量历史；在两边分区同时留下陈旧值。 +- 验收：覆盖整数/标量/向量/metadata tensor、表示切换、缺键保留、self-merge、长 DELTA/CUMULATIVE 流与 abort 刷新；断言快照始终非 list 且 `payload[key] is payload.to_dict()[key]`。^[PR #7448] + +## EXEC-7d — 张量累积策略必须按 (modality, key) 解析，codec 键不得套用波形默认 + +- 触发：修改 `get_accumulation_strategy`、`MultimodalPayload.consolidate_tensors`、`_consolidate_tensor_list`，或 AUDIO 模态下非波形键（如 `codes.audio`/`codes.ref`）的巩固。 +- 强制：`consolidate_tensors` 接收 modality，并对每个 key 调用 `get_accumulation_strategy(modality, key)`。流水线通过 `register_key_accumulation_strategy` 注册覆盖；Qwen3-TTS 必须将 `codes.audio`→`CONCAT_DIM0`、`codes.ref`→`REPLACE`。`CONCAT_LAST` 失败仍可 flatten 后拼接；其他策略失败必须带 key 名 raise，不得静默 keep-last。 +- 禁止：整 payload 共用单一 modality 默认；把 codec-frame 矩阵当 `CONCAT_LAST` 波形；把非 `audio` 键的 concat 失败吞成 keep-last。 +- 验收：合成张量证明 `codes.audio` 沿 dim0 拼满、`codes.ref` 只保留一份，以及错误策略 raise；真实波形键仍走 `CONCAT_LAST`。^[PR #7608] diff --git a/knowledge/repos/vllm-omni/components/serving/rules-fault-isolation.md b/knowledge/repos/vllm-omni/components/serving/rules-fault-isolation.md index 425813d5..92777060 100644 --- a/knowledge/repos/vllm-omni/components/serving/rules-fault-isolation.md +++ b/knowledge/repos/vllm-omni/components/serving/rules-fault-isolation.md @@ -1,7 +1,7 @@ --- title: "Serving replica fault-isolation 规则" created: 2026-09-02 -updated: 2026-09-05 +updated: 2026-09-17 type: rule tags: [vllm-omni, components, serving] sources: ["PR #4583", "PR #6170", vllm_omni/engine/orchestrator.py, vllm_omni/engine/stage_pool.py, vllm_omni/entrypoints/async_omni.py, vllm_omni/entrypoints/omni_base.py, tests/dfx/reliability/test_reliability_qwen3_omni.py, tests/engine/test_orchestrator_error_handling.py, tests/entrypoints/test_omni_entrypoints.py] @@ -59,3 +59,10 @@ confidence: high 发送 error 后以 abort/close-duplex-session cleanup 回收该 request；普通非-duplex 路径继续抛出， 不能扩大 catch 范围而吞没共享 orchestrator 的编程错误。验收要证明坏 duplex handoff 不终止 orchestrator，且同一 engine 的后续 request 仍可完成。^[PR #6170] + +## SERV-5u — 异步 video job 必须等 scheduler 准入才标 in_progress，DELETE 须先有界 abort engine + +- 触发：修改 `POST /v1/videos` 异步 job 状态机、`DELETE /v1/videos/{id}`，或 diffusion `emit_request_lifecycle` / request-started 控制面事件。 +- 强制：job 保持 `queued`，直到 scheduler 首次准入并经 opt-in lifecycle marker 通知；不得在 background task 启动时就标 `in_progress`。DELETE 先对 engine request 做有界 abort（`VLLM_OMNI_ABORT_TIMEOUT`，默认 2s），再取消 frontend task，并再等一小段 cleanup 以免 orchestrator 卡住；abort 后重读 job，使并发完成的 artifact 仍被删除。 +- 禁止：只取消 asyncio task 而留下 engine 侧扩散；把 task 启动当作已开始推理；让 DELETE 在 abort 上无限挂起。 +- 验收：queued-until-admission、in-progress DELETE abort、abort timeout、完成后 DELETE 清 artifact，以及 `generate()` cancel 的有界 cleanup。^[PR #6759] diff --git a/knowledge/repos/vllm-omni/components/serving/rules-speech-output.md b/knowledge/repos/vllm-omni/components/serving/rules-speech-output.md index 78bed551..745433a6 100644 --- a/knowledge/repos/vllm-omni/components/serving/rules-speech-output.md +++ b/knowledge/repos/vllm-omni/components/serving/rules-speech-output.md @@ -1,7 +1,7 @@ --- title: "Speech 输出采样率规则" created: 2026-09-05 -updated: 2026-09-05 +updated: 2026-09-17 type: rule tags: [vllm-omni, components, serving] sources: ["PR #6553", docs/serving/speech_api.md, vllm_omni/entrypoints/openai/audio_utils_mixin.py, vllm_omni/entrypoints/openai/protocol/audio.py, vllm_omni/entrypoints/openai/serving_speech.py, vllm_omni/entrypoints/openai/tts_adapters/base.py, vllm_omni/entrypoints/openai/tts_adapters/qwen3_tts.py, tests/entrypoints/openai_api/test_audio_format.py, tests/entrypoints/openai_api/test_serving_speech.py, tests/e2e/online_serving/test_qwen3_tts_customvoice_expansion.py] @@ -20,3 +20,10 @@ confidence: high - 禁止：upsample、16k/stereo/general adapter support，或据 server-side resampling 声称降低模型生成 latency/cost。 - 验收：Qwen native/8k、other adapter reject、batch override、streaming chunk continuity/tail/source mismatch/ first-audio source-sr failure。CPU/DSP 与单一 expansion fixture 不证明跨硬件音质、TTFB 或带宽收益。^[PR #6553] + +## SERV-9c — 稀疏音频非流式累积必须走 adapter OutputPolicy，不得按模型名分支 + +- 触发：新增/修改 TTS adapter 的非流式输出、`OutputPolicy.accumulate_nonstreaming`，或 `serving_speech._generate_audio_bytes` 对 per-step delta 的拼接。 +- 强制：MOSS-TTS-Nano、Gepard 等 `async_chunk=false` 的稀疏音频模型由 adapter 在 `PreparedRequest.output_policy` 声明 `accumulate_nonstreaming`。serving 用 `request_id` 键存 policy，仅非流式路径在准备成功后写入，并在累积器 `pop` 消费；优先使用 FINAL_ONLY 已拼接波形，否则 `torch.cat` delta。streaming 永不读写该表。 +- 禁止：在 `serving_speech.py` 增加 Gepard/MOSS 等模型名分支；在可失败步骤前写入 policy 导致泄漏；让并发请求共享可变 policy 槽位。 +- 验收：adapter 设/不设 flag 的非流式拼接与 sentinel-only final；并发非流式隔离；streaming 路径无 policy 条目。^[PR #7499] diff --git a/knowledge/repos/vllm-omni/models/minimax-music3/rules.md b/knowledge/repos/vllm-omni/models/minimax-music3/rules.md index 83b216ef..81c49fe9 100644 --- a/knowledge/repos/vllm-omni/models/minimax-music3/rules.md +++ b/knowledge/repos/vllm-omni/models/minimax-music3/rules.md @@ -1,7 +1,7 @@ --- title: "MiniMax Music3 规则" created: 2026-09-05 -updated: 2026-09-05 +updated: 2026-09-17 type: rule tags: [vllm-omni, models, model-executor] sources: ["PR #6640", vllm_omni/model_executor/models/minimax_music3/acoustic.py, vllm_omni/model_executor/models/minimax_music3/weights.py, tests/model_executor/models/test_minimax_music3_repo_root.py] @@ -34,3 +34,10 @@ confidence: high 缺 `condition_encoder`、缺一个 numeric indexed shard、snapshot revision recovery、custom loadable shard naming 与 strict-load failure；真实 checkpoint/audio E2E、音质、吞吐和跨平台仍需独立证据。 ^[PR #6640] + +## MM3-1b — float32 声学注意力在自动选中不兼容 backend 时必须改走 SDPA + +- 触发：修改 MiniMax Music 3 DiT `Attention`、共享 diffusion attention 的自动 backend 选择，或该 stage 的 float32 decode 路径。 +- 强制：该 `LLM_GENERATION` stage 常以 float32 运行。当自动选中的 backend 属于 float32 不兼容集合（`FLASH_ATTN`/`FLASH_ATTN_HUB`/`FLASH_ATTN_3_HUB`/`CUDNN_ATTN`/`FLASHINFER_ATTN`）时，float32 Q/K/V 必须走 torch SDPA；低精度输入仍用选定 native backend。显式 backend 选择保持 fail-fast，不得在此静默替换。`backend_name` 须同时报告两条运行时路由。 +- 禁止：把自动 FA/cuDNN/FlashInfer 路径直接喂 float32；把自动回退逻辑套用到显式 backend；改变共享 diffusion attention 全局 dispatch。 +- 验收：参数化覆盖上述 backend 的 float32→SDPA、fp16/bf16 与显式 float32 仍走 native、兼容自动 backend 保留、native 不可用时 SDPA。^[PR #7354] diff --git a/knowledge/repos/vllm-omni/models/qwen-omni/rules.md b/knowledge/repos/vllm-omni/models/qwen-omni/rules.md index db1db3c9..94e40538 100644 --- a/knowledge/repos/vllm-omni/models/qwen-omni/rules.md +++ b/knowledge/repos/vllm-omni/models/qwen-omni/rules.md @@ -1,7 +1,7 @@ --- title: "Qwen-Omni 规则" created: 2026-09-04 -updated: 2026-09-09 +updated: 2026-09-17 type: rule tags: [vllm-omni, models, qwen-omni] sources: ["PR #5687", "PR #6284", "PR #6449", "PR #4322", "PR #6748", "PR #6886", "PR #7019", vllm_omni/config/pipeline_registry.py, vllm_omni/deploy/qwen3_omni_moe.yaml, vllm_omni/deploy/qwen3_omni_moe_thinking.yaml, vllm_omni/engine/stage_init_utils.py, vllm_omni/model_executor/models/qwen2_5_omni/qwen2_5_omni.py, vllm_omni/model_executor/models/qwen3_omni/quantization.py, vllm_omni/model_executor/models/qwen3_omni/qwen3_omni.py, vllm_omni/model_executor/models/qwen3_omni/qwen3_omni_moe_thinker.py, vllm_omni/quantization/component_config.py, tests/config/test_config_factory.py, tests/diffusion/quantization/test_component_routing.py, tests/engine/test_stage_engine_args.py, tests/model_executor/models/qwen3_omni/test_qwen3_omni_quantization.py, "PR #7228"] @@ -86,3 +86,10 @@ confidence: high - 强制：构造 `inv_freq` 时优先 `config.rope_parameters["rope_theta"]`（若 mapping 存在），否则回退 top-level `rope_theta`，再默认 `10000.0`。Qwen3-Omni（Transformers 5.10+ 常把 checkpoint 的 `1_000_000` 只放在 nested 字段）与仍带 top-level 的 Qwen3-TTS/legacy 必须共用该顺序。 - 禁止：只读 top-level 导致静默回退 `10000`；把 nested 优先写成破坏 TTS top-level 兼容的唯一来源。 - 验收：参数化覆盖 nested-only、nested 优先于冲突 top-level、默认 `10000`，以及 TTS/legacy top-level（含删除 `rope_parameters`）重建正确 `inv_freq`。^[PR #7228] + +## QOMNI-1h — 混合长度 streaming batch 的 crop 必须用整窗 trim，不得把 padding 当有效未来帧 + +- 触发：修改 `Qwen3OmniMoeCode2Wav.chunked_decode_streaming` 的 start/end 切片、`seq_token_counts`、left context，或混长 batch 的 decoder trim 计算。 +- 强制：decoder 对整批共享同一 right-edge trim。`tail` 必须由 `codes.shape[-1]*total_upsample - batch_wav.shape[-1]` 相对整窗测量，再对每行用 `start = max(0, left_context*up - tail)` 与 `end = max(0, code_seq_len*up - tail)`。padding 不提供有效未来 codec 上下文。 +- 禁止：按单行 `code_seq_len` 相对整窗波形长度算 per-row tail（短请求会 clamp 到 0 并保留 padding 样本）；只用时长/长度相等证明 crop 正确。 +- 验收：独立单请求 oracle 对比混长 batch 的值与长度；覆盖 eager/cudagraph、多种短帧/left-context、batch 顺序，以及 singleton→mixed→singleton 连续流无 skip/repeat。^[PR #7340]