[Handoff to @Oseltamivir Claude /loop] [Klaud Cold] Update qwen3.5-fp8-b300-sglang (+mtp) SGLang image to v0.5.12-cu130#1451
Conversation
|
Thanks for the contribution! For vLLM & SGLang, please ensure that your recipes is similar to the official vLLM recipes and/or the SGLang cookbook If it is not, please create a PR first before we can merge your single node PR into the master branch. Let's ensure that the documentation is first class such that the entire ML community can benefit from your hard work! Thank you PR authors are responsible for ensuring that after merging, all GitHub Action jobs fully pass. A lot of the time, failures are just flakes and simply re-running the failed jobs will fix it. If re-running failed jobs is attempted, PR authors are responsible for ensuring it passes. See GitHub's docs on re-running failed jobs: https://docs.github.com/en/actions/how-tos/manage-workflow-runs/re-run-workflows-and-jobs#re-running-failed-jobs-in-a-workflow As a rule of thumb, generally, PR authors should request a review & get a PR approval from the respective companies' CODEOWNERS before requesting a review from core maintainers. If additional help is needed, PR authors can reach out to core maintainers over Slack. |
1 similar comment
|
Thanks for the contribution! For vLLM & SGLang, please ensure that your recipes is similar to the official vLLM recipes and/or the SGLang cookbook If it is not, please create a PR first before we can merge your single node PR into the master branch. Let's ensure that the documentation is first class such that the entire ML community can benefit from your hard work! Thank you PR authors are responsible for ensuring that after merging, all GitHub Action jobs fully pass. A lot of the time, failures are just flakes and simply re-running the failed jobs will fix it. If re-running failed jobs is attempted, PR authors are responsible for ensuring it passes. See GitHub's docs on re-running failed jobs: https://docs.github.com/en/actions/how-tos/manage-workflow-runs/re-run-workflows-and-jobs#re-running-failed-jobs-in-a-workflow As a rule of thumb, generally, PR authors should request a review & get a PR approval from the respective companies' CODEOWNERS before requesting a review from core maintainers. If additional help is needed, PR authors can reach out to core maintainers over Slack. |
|
see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=26006160501 |
![claude[bot]](https://avatars.githubusercontent.com/in/1236702?s=60&v=4){kind=small}
There was a problem hiding this comment.
LGTM, straightforward SGLang image version bump with matching changelog entry.
Extended reasoning...
Overview
This PR bumps the SGLang image for two B300 configurations (qwen3.5-fp8-b300-sglang and qwen3.5-fp8-b300-sglang-mtp) to v0.5.12-cu130 in .github/configs/nvidia-master.yaml, plus a corresponding entry in perf-changelog.yaml. Total diff is 4 lines of config and 7 lines of changelog.
Security risks
None. This is a configuration change to container image tags used by the perf sweep, not production-serving code, and no auth/crypto/permissions surface is touched.
Level of scrutiny
Low. Image version bumps with a matching changelog entry are mechanical and follow the established pattern in recent commits (e.g., #1404, #1394, #1416). The full-sweep-enabled label means the full sweep will validate the new image works for these recipes.
Other factors
The bug hunting system found no issues, there are no outstanding human reviewer comments, and the two automated recipe-reminder bot comments are boilerplate that don't require action. No prior reviews from me on this PR.
functionstackx
changed the title
Trying the same workaround as PR #1422:
|
|
see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=26006162286 |
Same workaround as #1422 (bf16) and #1451 (fp8) — bypass the broken flash-attn cute kernel sm_103 assertion in the Qwen-3.5-VL vision encoder by switching only the multi-modal attention path to triton_attn. Text decoder still uses --attention-backend trtllm_mha. See sgl-project/sglang#25564 (root cause: cutedsl Arch enum aliasing on non-cu13 path collapses sm_100..sm_110f range to exclude sm_103) and Dao-AILab/flash-attention#2572 for the upstream fix in flight.
|
see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=26018510955 |
functionstackx
changed the title
|
Handing off to @Oseltamivir — tracked alongside 7 other stuck Klaud-Cold PRs in #1511. /loop will stop auto-retrying this one. AI-generated via Claude Code /loop. |
Same workaround as #1422 (bf16) and #1451 (fp8) — bypass the broken flash-attn cute kernel sm_103 assertion in the Qwen-3.5-VL vision encoder by switching only the multi-modal attention path to triton_attn. Text decoder still uses --attention-backend trtllm_mha. See sgl-project/sglang#25564 (root cause: cutedsl Arch enum aliasing on non-cu13 path collapses sm_100..sm_110f range to exclude sm_103) and Dao-AILab/flash-attention#2572 for the upstream fix in flight.
Same workaround as PR #1422 — bypass the broken flash-attn cute kernel sm_103 assertion in the Qwen-3.5-VL vision encoder by switching only the multi-modal attention path to triton_attn. Text decoder still uses --attention-backend trtllm_mha. See sgl-project/sglang#25564 + Dao-AILab/flash-attention#2572 for the upstream root cause and the in-flight fix.
functionstackx
force-pushed
the
update-qwen3.5-fp8-b300-sglang-v0.5.12
branch
from
e9ea09d to
e1d3a18
Compare
functionstackx
changed the title
….5.12-cu130 (#1475) * Update qwen3.5-fp4-b300-sglang (+mtp) SGLang image to v0.5.12-cu130 Update SGLang image from v0.5.11-cu130 (5d old) to v0.5.12-cu130 Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix(qwen3.5_fp4_b300): use --mm-attention-backend triton_attn Same workaround as #1422 (bf16) and #1451 (fp8) — bypass the broken flash-attn cute kernel sm_103 assertion in the Qwen-3.5-VL vision encoder by switching only the multi-modal attention path to triton_attn. Text decoder still uses --attention-backend trtllm_mha. See sgl-project/sglang#25564 (root cause: cutedsl Arch enum aliasing on non-cu13 path collapses sm_100..sm_110f range to exclude sm_103) and Dao-AILab/flash-attention#2572 for the upstream fix in flight. * Re-trigger sweep (previous Run Sweep run stuck pending with 0 jobs) --------- Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com> Co-authored-by: claude-fix-bot <claude-fix-bot@local>
|
see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=26144042784 |
|
Post-rebase sweep on this PR's eval-only matrix ( Server starts cleanly (cuda-graph capture completes, requests succeed) — this is a silent quality failure, not a crash. The earlier Filed upstream: sgl-project/sglang#25863 Failing run: https://github.com/SemiAnalysisAI/InferenceX/actions/runs/26144042784/job/76895376231 Re-adding to handoff #1511 until sglang triages. |
functionstackx
changed the title
…00-sglang-v0.5.12 # Conflicts: # perf-changelog.yaml
|
see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=26250771571 |
|
see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=26252167464 |
1 similar comment
|
see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=26252167464 |
|
/reuse-sweep-run |
2 participants
Summary
qwen3.5-fp8-b300-sglangfromlmsysorg/sglang:v0.5.10.post1-cu130(30d old) tolmsysorg/sglang:v0.5.12-cu130.qwen3.5-fp8-b300-sglang-mtpfromlmsysorg/sglang:v0.5.11-cu130tolmsysorg/sglang:v0.5.12-cu130.Test plan
full-sweep-enabledlabel.🤖 Generated with Claude Code