Update qwen3.5-bf16-b300-sglang and -mtp SGLang image to v0.5.12-cu130#1422
Conversation
…ng image to v0.5.12-cu130 Ref #1154 Co-authored-by: Klaud Cold <Klaud-Cold@users.noreply.github.com>
|
Thanks for the contribution! For vLLM & SGLang, please ensure that your recipes is similar to the official vLLM recipes and/or the SGLang cookbook If it is not, please create a PR first before we can merge your single node PR into the master branch. Let's ensure that the documentation is first class such that the entire ML community can benefit from your hard work! Thank you PR authors are responsible for ensuring that after merging, all GitHub Action jobs fully pass. A lot of the time, failures are just flakes and simply re-running the failed jobs will fix it. If re-running failed jobs is attempted, PR authors are responsible for ensuring it passes. See GitHub's docs on re-running failed jobs: https://docs.github.com/en/actions/how-tos/manage-workflow-runs/re-run-workflows-and-jobs#re-running-failed-jobs-in-a-workflow As a rule of thumb, generally, PR authors should request a review & get a PR approval from the respective companies' CODEOWNERS before requesting a review from core maintainers. If additional help is needed, PR authors can reach out to core maintainers over Slack. |
![claude[bot]](https://avatars.githubusercontent.com/in/1236702?s=60&v=4){kind=small}
| - qwen3.5-bf16-b300-sglang-mtp | ||
| description: | ||
| - "Update SGLang image from v0.5.11-cu130 to v0.5.12-cu130" | ||
| pr-link: XXX |
There was a problem hiding this comment.
🔴 The new perf-changelog entry uses the literal placeholder pr-link: XXX instead of the actual PR URL. All other entries in this file follow the canonical format pr-link: https://github.com/SemiAnalysisAI/InferenceX/pull/<num>, so this should be https://github.com/SemiAnalysisAI/InferenceX/pull/1422 before merging.
Extended reasoning...
What the bug is
The diff adds a new entry to perf-changelog.yaml at line 2557 with pr-link: XXX. The string XXX is clearly an unfilled template placeholder that was supposed to be replaced with the actual PR URL before opening this PR.
How it manifests
Every prior entry in perf-changelog.yaml follows the same convention. Looking at the four most recent entries that come immediately before this one (lines 2532, 2538, 2544, 2550), each uses the canonical format:
pr-link: https://github.com/SemiAnalysisAI/InferenceX/pull/<num>The new entry breaks this pattern by retaining a template placeholder.
Step-by-step proof
- Open
perf-changelog.yamland look at the four most recent entries that come immediately before the newly added block:- Line 2532:
pr-link: https://github.com/SemiAnalysisAI/InferenceX/pull/... - Line 2538:
pr-link: https://github.com/SemiAnalysisAI/InferenceX/pull/... - Line 2544:
pr-link: https://github.com/SemiAnalysisAI/InferenceX/pull/... - Line 2550:
pr-link: https://github.com/SemiAnalysisAI/InferenceX/pull/1402
- Line 2532:
- The new entry added by this PR ends at line 2557 with:
pr-link: XXX XXXis not a URL and does not match the established pattern.- This PR is Update qwen3.5-bf16-b300-sglang and -mtp SGLang image to v0.5.12-cu130 #1422, so the value should be
https://github.com/SemiAnalysisAI/InferenceX/pull/1422.
Why existing code doesn't prevent it
The changelog convention is enforced by author discipline rather than by a schema check — YAML happily parses XXX as a string, so neither YAML validation nor any current CI step rejects the placeholder. The PR description even references #1154 as context, but the changelog placeholder was never filled in.
Impact
Any link-following automation, release-notes generator, or human reader navigating the changelog will hit a dead string (XXX) instead of a real PR link for this entry. While the file still parses, the structured pr-link field becomes useless for this row and breaks the documented convention going forward.
How to fix
Replace line 2557 with:
pr-link: https://github.com/SemiAnalysisAI/InferenceX/pull/1422
|
see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=25980018232 |
Upstream issue filedFiled sgl-project/sglang#25564 for the B300 (sm_103) flash-attn assertion that's blocking this PR's full sweep. Includes the full traceback (vision encoder → Why it fails despite sm_103 nominally being in rangeThe assertion is Upstream flash-attn fix already existsDao-AILab/flash-attention#2572 by ```diff
Once that merges and Tri Dao tags a new flash-attn-4 release, sglang needs to bump the |
Workaround for the flash_attn v4 cute kernel's sm_103 assertion failure in the Qwen3.5-VL vision encoder (filed as sgl-project/sglang#25564, upstream fix in Dao-AILab/flash-attention#2572). The text decoder still uses --attention-backend trtllm_mha; this only swaps the multi-modal (vision encoder) attention path to triton_attn, bypassing the broken flash_attn cute dispatch on B300. Suggested by upstream sglang reviewer.
Trying suggested workaround:
|
# Conflicts: # perf-changelog.yaml
Upstream commentary: the fix is actually a
|
Same workaround as PR #1422 — bypass the broken flash-attn cute kernel sm_103 assertion in the Qwen-3.5-VL vision encoder by switching only the multi-modal attention path to triton_attn. Text decoder still uses --attention-backend trtllm_mha. See sgl-project/sglang#25564 + Dao-AILab/flash-attention#2572 for the upstream root cause and the in-flight fix.
Same workaround as #1422 (bf16) and #1451 (fp8) — bypass the broken flash-attn cute kernel sm_103 assertion in the Qwen-3.5-VL vision encoder by switching only the multi-modal attention path to triton_attn. Text decoder still uses --attention-backend trtllm_mha. See sgl-project/sglang#25564 (root cause: cutedsl Arch enum aliasing on non-cu13 path collapses sm_100..sm_110f range to exclude sm_103) and Dao-AILab/flash-attention#2572 for the upstream fix in flight.
|
see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=26015573402 |
|
/reuse-sweep-run |
|
see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=26052019013 |
Same workaround as #1422 (bf16) and #1451 (fp8) — bypass the broken flash-attn cute kernel sm_103 assertion in the Qwen-3.5-VL vision encoder by switching only the multi-modal attention path to triton_attn. Text decoder still uses --attention-backend trtllm_mha. See sgl-project/sglang#25564 (root cause: cutedsl Arch enum aliasing on non-cu13 path collapses sm_100..sm_110f range to exclude sm_103) and Dao-AILab/flash-attention#2572 for the upstream fix in flight.
Same workaround as PR #1422 — bypass the broken flash-attn cute kernel sm_103 assertion in the Qwen-3.5-VL vision encoder by switching only the multi-modal attention path to triton_attn. Text decoder still uses --attention-backend trtllm_mha. See sgl-project/sglang#25564 + Dao-AILab/flash-attention#2572 for the upstream root cause and the in-flight fix.
….5.12-cu130 (#1475) * Update qwen3.5-fp4-b300-sglang (+mtp) SGLang image to v0.5.12-cu130 Update SGLang image from v0.5.11-cu130 (5d old) to v0.5.12-cu130 Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix(qwen3.5_fp4_b300): use --mm-attention-backend triton_attn Same workaround as #1422 (bf16) and #1451 (fp8) — bypass the broken flash-attn cute kernel sm_103 assertion in the Qwen-3.5-VL vision encoder by switching only the multi-modal attention path to triton_attn. Text decoder still uses --attention-backend trtllm_mha. See sgl-project/sglang#25564 (root cause: cutedsl Arch enum aliasing on non-cu13 path collapses sm_100..sm_110f range to exclude sm_103) and Dao-AILab/flash-attention#2572 for the upstream fix in flight. * Re-trigger sweep (previous Run Sweep run stuck pending with 0 jobs) --------- Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com> Co-authored-by: claude-fix-bot <claude-fix-bot@local>
2 participants
Updates SGLang image for
qwen3.5-bf16-b300-sglangandqwen3.5-bf16-b300-sglang-mtpfrom v0.5.11-cu130 to v0.5.12-cu130.\nRef #1154
Generated with Claude Code