Conversation
Functional NPU model test that keeps the unified radix cache enabled and forces LRU eviction followed by re-insertion of the evicted prefix, covering the sgl-project#37091 contract change (recover_after_unevict / update_component_on_insert_overlap) on the DeepSeek-V4 sidecar component. Also point single-test-npu.yml at the new case and run it against the cann9.1.0-a3 and cann9.0.0-a3 20260828 images.
CLA Signature Guide@EdwardXuy , thanks for your pull request. The following commit(s) are not associated with a signed Contributor License Agreement (CLA).
To sign CLA, click here. To check if your email is configured correctly, refer to the FAQs. Once you've signed the CLA or updating your email, please comment |
DeepSeek-V4 needs --tp-size 16/--dp-size 16, so the 2-card linux-aarch64-a3-2 runner cannot host it; switch to linux-aarch64-a3-16. Replace the two 20260828 images with main-cann9.0.0-a3 (daily build, contains sgl-project#37091 -> expected RED) plus the 20260828-cann9.0.0-a3 snapshot (control -> expected GREEN).
CLA Signature Guide@EdwardXuy , thanks for your pull request. The following commit(s) are not associated with a signed Contributor License Agreement (CLA).
To sign CLA, click here. To check if your email is configured correctly, refer to the FAQs. Once you've signed the CLA or updating your email, please comment |
With fail-fast defaulting to true, the main-cann9.0.0-a3 job failure cancels the 20260828 control job before it finishes, so no red/green comparison is produced. Disable fail-fast so both images complete independently.
CLA Signature Guide@EdwardXuy , thanks for your pull request. The following commit(s) are not associated with a signed Contributor License Agreement (CLA).
To sign CLA, click here. To check if your email is configured correctly, refer to the FAQs. Once you've signed the CLA or updating your email, please comment |
The bundled image sglang cannot reproduce sgl-project#37091: the main image crashes at cuda-graph capture (aclnnHcPre missing in libopapi.so) and the 8/28 image predates the PR. Checkout upstream sglang main and override via PYTHONPATH so the recover_after_unevict result-param change is exercised, while overlaying the ascend test utils from this PR's checkout.
CLA Signature Guide@EdwardXuy , thanks for your pull request. The following commit(s) are not associated with a signed Contributor License Agreement (CLA).
To sign CLA, click here. To check if your email is configured correctly, refer to the FAQs. Once you've signed the CLA or updating your email, please comment |
Both cann9.0.0 images fail at capture_decode_graph with 'aclnnHcPre not in libopapi.so': the DSV4 mHC pre-norm path calls torch.ops.custom.npu_hc_pre, which cann9.0.0 does not ship. Use the 20260828 cann9.1.0 snapshot (has HcPre) and keep the PYTHONPATH override to upstream main (contains sgl-project#37091).
CLA Signature Guide@EdwardXuy , thanks for your pull request. The following commit(s) are not associated with a signed Contributor License Agreement (CLA).
To sign CLA, click here. To check if your email is configured correctly, refer to the FAQs. Once you've signed the CLA or updating your email, please comment |
aclnnHcPre still missing on cann9.1.0 with only cann + nnal/atb sourced. The DSV4 mHC fused pre-norm op (torch.ops.custom.npu_hc_pre) ships in the custom_transformer/customize operator packages; source their set_env.bash before launch (per sgl-project#29794), and dump the opp/vendors dir for diagnosis.
CLA Signature Guide@EdwardXuy , thanks for your pull request. The following commit(s) are not associated with a signed Contributor License Agreement (CLA).
To sign CLA, click here. To check if your email is configured correctly, refer to the FAQs. Once you've signed the CLA or updating your email, please comment |
The evict->unevict path (recover_after_unevict) only runs when a node is soft-evicted (FULL.device value=None, node kept). With a pure device cache + per-request distinct prefixes, eviction hard-deletes leaves and the re-sent anchor hit 0 cached tokens (scenario silently skipped). Enable hierarchical cache so filled-up prefixes offload to host and get unevict-ed on re-hit, and assert cached_tokens>0 so the scenario cannot pass vacuously.
CLA Signature Guide@EdwardXuy , thanks for your pull request. The following commit(s) are not associated with a signed Contributor License Agreement (CLA).
To sign CLA, click here. To check if your email is configured correctly, refer to the FAQs. Once you've signed the CLA or updating your email, please comment |
--hicache-storage-backend has no 'memory' choice (file/sim/shm/...). The evict->unevict path only needs the device->host tier via --enable-hierarchical-cache/--hicache-ratio, so remove the storage backend instead of picking one.
CLA Signature Guide@EdwardXuy , thanks for your pull request. The following commit(s) are not associated with a signed Contributor License Agreement (CLA).
To sign CLA, click here. To check if your email is configured correctly, refer to the FAQs. Once you've signed the CLA or updating your email, please comment |
2 participants
Summary
Add a functional NPU model test covering the unified radix cache evict -> unevict path for DeepSeek-V4-Flash (W8A8, 8p PD-mix), to reproduce the contract-change regression from sgl-project/sglang#37091.
Background
sgl-project#37091 ("Unified Cache [1/N]") added a required
resultparameter toTreeComponent.recover_after_unevict/update_component_on_insert_overlap. The NPU DeepSeek-V4 sidecar component overrides the old signature, so the evict -> unevict path raisesTypeError.Changes
test/registered/npu/performance/deepseek_v4_flash/test_npu_deepseek_v4_flash_w8a8_radix_cache_evict_unevict.py--disable-radix-cache; this one omits it).TypeError)..github/workflows/single-test-npu.ymltest_casesat the new case.Note on the images
Both images are built 2026-08-28, while sgl-project#37091 merged 2026-08-30, so neither image contains the regression code. This case is expected to be green now; it will turn red once the branch/images catch up with sgl-project#37091.
CI States
Latest PR Test (Base): ❌ Run #33599390890
Latest PR Test (Extra): ❌ Run #33599390686
Latest PR Test (AMD ROCm 7.2): ❌ Run #33599391035