Patch 3 by hhhh1252023 · Pull Request #1222 · Ascend/sglang

hhhh1252023 · 2026-09-03T13:25:16Z

Motivation

Modifications

Accuracy Tests

Speed Tests and Profiling

Checklist

Format your code according to the Format code with pre-commit.
Add unit tests according to the Run and add unit tests.
Update documentation according to Write documentations.
Provide accuracy and speed benchmark results according to Test the accuracy and Benchmark the speed.
Follow the SGLang code style guidance.

Review and Merge Process

Ping Merge Oncalls to start the process. See the PR Merge Process.
Get approvals from CODEOWNERS and other reviewers.
Trigger CI tests with comments or contact authorized users to do so.
- Common commands include /tag-and-rerun-ci, /tag-run-ci-label, /rerun-failed-ci
After green CI and required approvals, ask Merge Oncalls or people with Write permission to merge the PR.

CI States

Latest PR Test (Base): ❌ Run #34815030001
Latest PR Test (Extra): ❌ Run #34815029868
Latest PR Test (AMD ROCm 7.2): ❌ Run #34815029964

…k grids on 1cf2b8c (sgl-project#35825) Co-authored-by: Claude Fable 5 <noreply@anthropic.com>

…gl-project#35933)

…reprocessing, and MTP decoding (sgl-project#33829) Co-authored-by: miraclezqc <dysania@pku.edu.cn>

Co-authored-by: Yichao Fu <yichaofu@meta.com>

…l-project#35463)

…gl-project#35945)

…gl-project#35948)

…/receiver (sgl-project#35950)

Co-authored-by: liusy58 <liusy58@smail.nju.edu.cn>

…read (sgl-project#35882) Co-authored-by: Claude Opus 5 <noreply@anthropic.com>

…ject#35962)

…project#35946) Co-authored-by: Claude Opus 5 <noreply@anthropic.com>

…35950 (sgl-project#35974)

…ype from load (sgl-project#35967) Co-authored-by: Claude Fable 5 <noreply@anthropic.com>

…offload (sgl-project#35812) Co-authored-by: Claude Opus 5 <noreply@anthropic.com>

…dary dispatch (sgl-project#32577) Co-authored-by: 1am9trash <1am9trash@gmail.com> Co-authored-by: HAI <hixiao@gmail.com>

…t#35989)

…-project#35979)

… passthrough (sgl-project#35352)

… restore 2-NPU CI testcase (sgl-project#34855) Co-authored-by: Elizaveta Martirosian <elizabet3000@mail.ru> Co-authored-by: Arseniy Mironov <98156294+Napkin-AI@users.noreply.github.com> Co-authored-by: Alexandr <117110413+Allor-maker@users.noreply.github.com> Co-authored-by: P_Alex_Tr <aleksandr.smyshlaev@yandex.ru>

…project#35816) Co-authored-by: Claude Opus 5 <noreply@anthropic.com>

…oops (sgl-project#35980)

…tention (verify & draft_extend) (sgl-project#30105)

…gl-project#35305) Co-authored-by: Mohammad Miadh Angkad <176301910+mmangkad@users.noreply.github.com>

…ject#35994)

…store (sgl-project#35986) Co-authored-by: Claude Fable 5 <noreply@anthropic.com>

…ct#35997)

…e encoders (sgl-project#36023)

…etween uses (sgl-project#35734)

ascend-robot · 2026-09-08T16:10:00Z

CLA Signature Guide

@cherryblo , thanks for your pull request.

The following commit(s) are not associated with a signed Contributor License Agreement (CLA).

Commit	Reason
[`0ee3749` [Fix] Read the granite sinks dt...](`0ee3749`)	the email used in the commit is not linked to a signed CLA! please verify that it matches the email you used when signing the CLA.

To sign CLA, click here.

To check if your email is configured correctly, refer to the FAQs.

Once you've signed the CLA or updating your email, please comment /check-cla to revalidate CLA status.

ascend-robot · 2026-09-09T02:19:36Z

CLA Signature Guide

@cherryblo , thanks for your pull request.

The following commit(s) are not associated with a signed Contributor License Agreement (CLA).

Commit	Reason
[`22dafbc` [diffusion] feat: keep a cpu-st...](`22dafbc`)	the email used in the commit is not linked to a signed CLA! please verify that it matches the email you used when signing the CLA.

To sign CLA, click here.

To check if your email is configured correctly, refer to the FAQs.

Once you've signed the CLA or updating your email, please comment /check-cla to revalidate CLA status.

ascend-robot · 2026-09-09T09:55:09Z

CLA Signature Guide

@hhhh1252023 , thanks for your pull request.

The following commit(s) are not associated with a signed Contributor License Agreement (CLA).

Commit	Reason
[`b26695a` [diffusion] feat: reject unsupp...](`b26695a`)	the email used in the commit is not linked to a signed CLA! please verify that it matches the email you used when signing the CLA.

To sign CLA, click here.

To check if your email is configured correctly, refer to the FAQs.

Once you've signed the CLA or updating your email, please comment /check-cla to revalidate CLA status.

ascend-robot · 2026-09-09T10:50:44Z

CLA Signature Guide

@hhhh1252023 , thanks for your pull request.

The following commit(s) are not associated with a signed Contributor License Agreement (CLA).

Commit	Reason
[`0be2a20` [diffusion] refactor: hand out ...](`0be2a20`)	the email used in the commit is not linked to a signed CLA! please verify that it matches the email you used when signing the CLA.

To sign CLA, click here.

To check if your email is configured correctly, refer to the FAQs.

Once you've signed the CLA or updating your email, please comment /check-cla to revalidate CLA status.

ascend-robot · 2026-09-09T12:28:11Z

CLA Signature Guide

@hhhh1252023 , thanks for your pull request.

The following commit(s) are not associated with a signed Contributor License Agreement (CLA).

Commit	Reason
[`3b5909d` [DeepSeek V4] Add W4A4 MegaMoE ...](`3b5909d`)	the email used in the commit is not linked to a signed CLA! please verify that it matches the email you used when signing the CLA.

To sign CLA, click here.

To check if your email is configured correctly, refer to the FAQs.

Once you've signed the CLA or updating your email, please comment /check-cla to revalidate CLA status.

ascend-robot · 2026-09-09T12:30:08Z

CLA Signature Guide

@hhhh1252023 , thanks for your pull request.

The following commit(s) are not associated with a signed Contributor License Agreement (CLA).

Commit	Reason
[`d90318b` [MLX] Upgrade to Torch 2.13/MLX...](`d90318b`)	the email used in the commit is not linked to a signed CLA! please verify that it matches the email you used when signing the CLA.

To sign CLA, click here.

To check if your email is configured correctly, refer to the FAQs.

Once you've signed the CLA or updating your email, please comment /check-cla to revalidate CLA status.

ascend-robot · 2026-09-09T12:32:22Z

CLA Signature Guide

@hhhh1252023 , thanks for your pull request.

The following commit(s) are not associated with a signed Contributor License Agreement (CLA).

Commit	Reason
5662c033 Support CPU offload for mxfp8 KV...	the email used in the commit is not linked to a signed CLA! please verify that it matches the email you used when signing the CLA.

To sign CLA, click here.

To check if your email is configured correctly, refer to the FAQs.

Once you've signed the CLA or updating your email, please comment /check-cla to revalidate CLA status.

ascend-robot · 2026-09-10T06:47:57Z

CLA Signature Guide

@cherryblo , thanks for your pull request.

The following commit(s) are not associated with a signed Contributor License Agreement (CLA).

Commit	Reason
fbafd1b1 Add sampling observer auxiliary ...	the email used in the commit is not linked to a signed CLA! please verify that it matches the email you used when signing the CLA.

To sign CLA, click here.

To check if your email is configured correctly, refer to the FAQs.

Once you've signed the CLA or updating your email, please comment /check-cla to revalidate CLA status.

ascend-robot · 2026-09-10T08:18:58Z

CLA Signature Guide

@cherryblo , thanks for your pull request.

The following commit(s) are not associated with a signed Contributor License Agreement (CLA).

Commit	Reason
0db2bdfe Fix buffer-mode HiCache load-bac...	the email used in the commit is not linked to a signed CLA! please verify that it matches the email you used when signing the CLA.

To sign CLA, click here.

To check if your email is configured correctly, refer to the FAQs.

Once you've signed the CLA or updating your email, please comment /check-cla to revalidate CLA status.

ascend-robot · 2026-09-10T09:29:27Z

CLA Signature Guide

@cherryblo , thanks for your pull request.

The following commit(s) are not associated with a signed Contributor License Agreement (CLA).

Commit	Reason
ac179eec fix(disagg): PD transfer-failure...	the email used in the commit is not linked to a signed CLA! please verify that it matches the email you used when signing the CLA.

To sign CLA, click here.

To check if your email is configured correctly, refer to the FAQs.

Once you've signed the CLA or updating your email, please comment /check-cla to revalidate CLA status.

ascend-robot · 2026-09-14T06:46:45Z

CLA Signature Guide

@hhhh1252023 , thanks for your pull request.

The following commit(s) are not associated with a signed Contributor License Agreement (CLA).

Commit	Reason
[`5290327` [diffusion] feat: resolve hub c...](`5290327`)	the email used in the commit is not linked to a signed CLA! please verify that it matches the email you used when signing the CLA.

To sign CLA, click here.

To check if your email is configured correctly, refer to the FAQs.

Once you've signed the CLA or updating your email, please comment /check-cla to revalidate CLA status.

ascend-robot · 2026-09-14T06:48:19Z

CLA Signature Guide

@hhhh1252023 , thanks for your pull request.

The following commit(s) are not associated with a signed Contributor License Agreement (CLA).

Commit	Reason
[`3e09662` [CI] Re-enable B300 jobs (sgl-project#3560...](`3e09662`)	the email used in the commit is not linked to a signed CLA! please verify that it matches the email you used when signing the CLA.

To sign CLA, click here.

To check if your email is configured correctly, refer to the FAQs.

Once you've signed the CLA or updating your email, please comment /check-cla to revalidate CLA status.

ascend-robot · 2026-09-14T06:48:35Z

CLA Signature Guide

@hhhh1252023 , thanks for your pull request.

The following commit(s) are not associated with a signed Contributor License Agreement (CLA).

Commit	Reason
[`83e9ece` [diffusion] Fuse SANA-Video int...](`83e9ece`)	the email used in the commit is not linked to a signed CLA! please verify that it matches the email you used when signing the CLA.

To sign CLA, click here.

To check if your email is configured correctly, refer to the FAQs.

Once you've signed the CLA or updating your email, please comment /check-cla to revalidate CLA status.

ascend-robot · 2026-09-14T06:49:07Z

CLA Signature Guide

@hhhh1252023 , thanks for your pull request.

The following commit(s) are not associated with a signed Contributor License Agreement (CLA).

Commit	Reason
[`96bfd24` [diffusion] Enable SANA-Video b...](`96bfd24`)	the email used in the commit is not linked to a signed CLA! please verify that it matches the email you used when signing the CLA.

To sign CLA, click here.

To check if your email is configured correctly, refer to the FAQs.

Once you've signed the CLA or updating your email, please comment /check-cla to revalidate CLA status.

…ases - HCCL_BUFFSIZE 0 -> 2200 - Comment out ZBAL envs - Add DEEPEP_NORMAL_COMBINE_ENABLE_LONG_SEQ for in16k/in3k5 cases - in16k 50ms: max_concurrency / num_prompts 132 -> 120 Co-Authored-By: Claude Code <noreply@anthropic.com>

BBuf and others added 30 commits August 22, 2026 12:57

[diffusion] Enable SANA-Video breakable CUDA graphs (sgl-project#35729)

96bfd24

[docs] Re-measure the Qwen3.8-27B RTX 5090, RTX PRO 6000 and DGX Spar…

4cb5aeb

…k grids on 1cf2b8c (sgl-project#35825) Co-authored-by: Claude Fable 5 <noreply@anthropic.com>

[HiCache] Clamp tombstoned SWA locs in UnifiedSWAKVPool translation (s…

c35683f

…gl-project#35933)

[Model] Complete dots.note.omni support with native encoders, video p…

af39ad9

…reprocessing, and MTP decoding (sgl-project#33829) Co-authored-by: miraclezqc <dysania@pku.edu.cn>

Make draft attention backends extensible (sgl-project#35932)

6fd0384

Co-authored-by: Yichao Fu <yichaofu@meta.com>

[VLM] Split Pixtral multi-image features before the CUDA IPC wrap (sg…

9035432

…l-project#35463)

[diffusion] feat: load serialized bnb4 components with transformers (s…

b391ef1

…gl-project#35945)

refactor(disagg): hoist duplicated _handle_staging_req into a mixin (s…

15a4398

…gl-project#35948)

refactor(disagg): drop dead placeholder overrides in Common KV sender…

5c03069

…/receiver (sgl-project#35950)

[FEAT] Weight Daemon abstraction (sgl-project#33279)

cb10ca1

Co-authored-by: liusy58 <liusy58@smail.nju.edu.cn>

[diffusion] optimization: transfer mapped layers through a courier th…

382343f

…read (sgl-project#35882) Co-authored-by: Claude Opus 5 <noreply@anthropic.com>

[diffusion] feat: admit compatible quantized native encoders (sgl-pro…

489e605

…ject#35962)

[diffusion] feature: use the directory for the vae mapping gate (sgl-…

a8c16b2

…project#35946) Co-authored-by: Claude Opus 5 <noreply@anthropic.com>

fix(test): unbreak test_kv_transfer_replica_metric after sgl-project#…

3c69a4c

…35950 (sgl-project#35974)

[diffusion] optimization: keep vae decoder weights in their decode dt…

61981e1

…ype from load (sgl-project#35967) Co-authored-by: Claude Fable 5 <noreply@anthropic.com>

[diffusion] chore: let the auto policy select h3's dit for layerwise …

db570fe

…offload (sgl-project#35812) Co-authored-by: Claude Opus 5 <noreply@anthropic.com>

[AMD] DeepSeek-V4: add aiter fused mHC post+pre with cross-layer boun…

d315eb7

…dary dispatch (sgl-project#32577) Co-authored-by: 1am9trash <1am9trash@gmail.com> Co-authored-by: HAI <hixiao@gmail.com>

[diffusion] fix: fix hunyuan3d stale extension lock hangs (sgl-projec…

46cb12a

…t#35989)

[diffusion] feat: support single-file component weight overrides (sgl…

453b98c

…-project#35979)

[diffusion] comfyui: add a minimax-h3 node and a generic extra-fields…

0b064e3

… passthrough (sgl-project#35352)

[diffusion] docs: add tuning guide for h3 on consumer-level gpu (sgl-…

7d22b7a

…project#35816) Co-authored-by: Claude Opus 5 <noreply@anthropic.com>

refactor(disagg): hoist staging helper imports out of the bootstrap l…

cce0a12

…oops (sgl-project#35980)

[AMD][Spec] Fix aiter GQA packing + split-KV routing in NEXTN spec at…

eec794b

…tention (verify & draft_extend) (sgl-project#30105)

[Kimi-K3] Fix "wrong grids" crash in DP-sharded vision preprocessing (s…

7f30d66

…gl-project#35305) Co-authored-by: Mohammad Miadh Angkad <176301910+mmangkad@users.noreply.github.com>

[diffusion] feat: support serialized comfy convrot int8 dits (sgl-pro…

a36c074

…ject#35994)

[diffusion] chore: re-home decode-dtype vae weights to a file-backed …

59cdc9d

…store (sgl-project#35986) Co-authored-by: Claude Fable 5 <noreply@anthropic.com>

[diffusion] fix: stabilize ltx-2.3 two-stage cold requests (sgl-proje…

bbbcbf9

…ct#35997)

[diffusion] feat: support loading serialized comfy convrot int8 nativ…

70319a0

…e encoders (sgl-project#36023)

[diffusion] feat: release a layerwise component's non-layer weights b…

97ae278

…etween uses (sgl-project#35734)

Update launch_dsv4_flash_w8a8_server.sh

db33da0

Update launch_dsv4_flash_w8a8_server.sh

0067298

Update nightly-test-npu-e2e-single-node.yml

5c63b40

Update full-test-npu.yml

9a787ab

Update nightly-test-npu-e2e-single-node.yml

a9fa24d

Update test_npu_deepseek_v4_flash_w8a8_8p_in8k_out1k_50ms.py

5c2ce55

Update nightly-test-npu-e2e-single-node.yml

89b56be

Update test_npu_deepseek_v4_flash_w8a8_8p_in8k_out1k_50ms.py

c6fcba5

Update test_npu_deepseek_v4_flash_w8a8_8p_in8k_out1k_50ms.py

4b6d51c

pllimax mentioned this pull request Sep 10, 2026

feat(npu): run dsv4-flash w8a8 8p perf case standalone in single-node e2e workflow #1236

Open

Update test_npu_deepseek_v4_flash_w8a8_8p_in8k_out1k_50ms.py

4283b69

Delete scripts/ci/npu/launch_dsv4_flash_w8a8_server.sh

dd858e4

hhhh1252023 added 2 commits September 14, 2026 14:47

Update full-test-npu.yml

c69c4fc

Delete .github/workflows/nightly-test-npu-e2e-single-node.yml

b86ae55

Update test_npu_deepseek_v4_flash_w8a8_8p_in8k_out1k_50ms.py

91f5f8d

Conversation

hhhh1252023 commented Sep 3, 2026 • edited by github-actions Bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Motivation

Modifications

Accuracy Tests

Speed Tests and Profiling

Checklist

Review and Merge Process

CI States

Uh oh!

ascend-robot commented Sep 8, 2026

CLA Signature Guide

Uh oh!

ascend-robot commented Sep 9, 2026

CLA Signature Guide

Uh oh!

ascend-robot commented Sep 9, 2026

CLA Signature Guide

Uh oh!

ascend-robot commented Sep 9, 2026

CLA Signature Guide

Uh oh!

ascend-robot commented Sep 9, 2026

CLA Signature Guide

Uh oh!

ascend-robot commented Sep 9, 2026

CLA Signature Guide

Uh oh!

ascend-robot commented Sep 9, 2026

CLA Signature Guide

Uh oh!

ascend-robot commented Sep 10, 2026

CLA Signature Guide

Uh oh!

ascend-robot commented Sep 10, 2026

CLA Signature Guide

Uh oh!

ascend-robot commented Sep 10, 2026

CLA Signature Guide

Uh oh!

ascend-robot commented Sep 14, 2026

CLA Signature Guide

Uh oh!

ascend-robot commented Sep 14, 2026

CLA Signature Guide

Uh oh!

ascend-robot commented Sep 14, 2026

CLA Signature Guide

Uh oh!

ascend-robot commented Sep 14, 2026

CLA Signature Guide

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

20 participants

hhhh1252023 commented Sep 3, 2026 •

edited by github-actions Bot

Loading