[Klaud Cold] Update qwen3.5-fp4-b300-sglang (+mtp) SGLang image to v0.5.12-cu130#1475
Conversation
|
Thanks for the contribution! For vLLM & SGLang, please ensure that your recipes is similar to the official vLLM recipes and/or the SGLang cookbook If it is not, please create a PR first before we can merge your single node PR into the master branch. Let's ensure that the documentation is first class such that the entire ML community can benefit from your hard work! Thank you PR authors are responsible for ensuring that after merging, all GitHub Action jobs fully pass. A lot of the time, failures are just flakes and simply re-running the failed jobs will fix it. If re-running failed jobs is attempted, PR authors are responsible for ensuring it passes. See GitHub's docs on re-running failed jobs: https://docs.github.com/en/actions/how-tos/manage-workflow-runs/re-run-workflows-and-jobs#re-running-failed-jobs-in-a-workflow As a rule of thumb, generally, PR authors should request a review & get a PR approval from the respective companies' CODEOWNERS before requesting a review from core maintainers. If additional help is needed, PR authors can reach out to core maintainers over Slack. |
2 similar comments
|
Thanks for the contribution! For vLLM & SGLang, please ensure that your recipes is similar to the official vLLM recipes and/or the SGLang cookbook If it is not, please create a PR first before we can merge your single node PR into the master branch. Let's ensure that the documentation is first class such that the entire ML community can benefit from your hard work! Thank you PR authors are responsible for ensuring that after merging, all GitHub Action jobs fully pass. A lot of the time, failures are just flakes and simply re-running the failed jobs will fix it. If re-running failed jobs is attempted, PR authors are responsible for ensuring it passes. See GitHub's docs on re-running failed jobs: https://docs.github.com/en/actions/how-tos/manage-workflow-runs/re-run-workflows-and-jobs#re-running-failed-jobs-in-a-workflow As a rule of thumb, generally, PR authors should request a review & get a PR approval from the respective companies' CODEOWNERS before requesting a review from core maintainers. If additional help is needed, PR authors can reach out to core maintainers over Slack. |
|
Thanks for the contribution! For vLLM & SGLang, please ensure that your recipes is similar to the official vLLM recipes and/or the SGLang cookbook If it is not, please create a PR first before we can merge your single node PR into the master branch. Let's ensure that the documentation is first class such that the entire ML community can benefit from your hard work! Thank you PR authors are responsible for ensuring that after merging, all GitHub Action jobs fully pass. A lot of the time, failures are just flakes and simply re-running the failed jobs will fix it. If re-running failed jobs is attempted, PR authors are responsible for ensuring it passes. See GitHub's docs on re-running failed jobs: https://docs.github.com/en/actions/how-tos/manage-workflow-runs/re-run-workflows-and-jobs#re-running-failed-jobs-in-a-workflow As a rule of thumb, generally, PR authors should request a review & get a PR approval from the respective companies' CODEOWNERS before requesting a review from core maintainers. If additional help is needed, PR authors can reach out to core maintainers over Slack. |
|
see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=26009982423 |
There was a problem hiding this comment.
LGTM, straightforward SGLang image version bump (v0.5.11-cu130 → v0.5.12-cu130) with matching changelog entry.
Extended reasoning...
Overview
This PR bumps the SGLang container image for two recipes (qwen3.5-fp4-b300-sglang and qwen3.5-fp4-b300-sglang-mtp) from v0.5.11-cu130 to v0.5.12-cu130 in .github/configs/nvidia-master.yaml, and appends a corresponding entry to perf-changelog.yaml linking back to this PR.
Security risks
None. This is a pinned upstream image version tag bump in CI config. The image source (lmsysorg/sglang) is unchanged, only the version tag advances by a patch release.
Level of scrutiny
Low. This is a mechanical, config-only change with no logic, no auth/crypto, and no production code paths. It follows the exact same pattern as numerous prior image-bump PRs in this repo (e.g. the recent #1458 entry directly above it in the changelog).
Other factors
The PR carries the full-sweep-enabled label so CI sweeps will validate the new image. No bugs were found by the bug hunting system, and there are no outstanding human reviewer comments — only the standard recipe-reminder bot messages.
Same vision-encoder workaround as PRs #1422 / #1451Confirmed same Added Upstream / root cause cross-links
Once either upstream lands, this workaround can come back out. |
|
see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=26009984417 |
|
see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=26047575440 |
1 similar comment
|
see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=26047575440 |
|
Handing off to @Oseltamivir — tracked alongside 7 other stuck Klaud-Cold PRs in #1511. /loop will stop auto-retrying this one. AI-generated via Claude Code /loop. |
Update SGLang image from v0.5.11-cu130 (5d old) to v0.5.12-cu130 Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Same workaround as #1422 (bf16) and #1451 (fp8) — bypass the broken flash-attn cute kernel sm_103 assertion in the Qwen-3.5-VL vision encoder by switching only the multi-modal attention path to triton_attn. Text decoder still uses --attention-backend trtllm_mha. See sgl-project/sglang#25564 (root cause: cutedsl Arch enum aliasing on non-cu13 path collapses sm_100..sm_110f range to exclude sm_103) and Dao-AILab/flash-attention#2572 for the upstream fix in flight.
bfac96d to
b8f0cd5
Compare
|
see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=26144009563 |
|
/reuse-sweep-run |
|
see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=26146822534 |
Summary
Update SGLang image from v0.5.11-cu130 (5d old) to v0.5.12-cu130
Recipes touched: `qwen3.5-fp4-b300-sglang`,
qwen3.5-fp4-b300-sglang-mtpTest plan
🤖 Generated with Claude Code