Skip to content

[WIP][feat](kt-kernel): Add Ascend NPU CPU-MoE backend + llama.cpp b3173 patch support - #2153

Draft
Nikonz wants to merge 3 commits into
kvcache-ai:mainfrom
Nikonz:pr/ascend-npu-cpu-moe
Draft

[WIP][feat](kt-kernel): Add Ascend NPU CPU-MoE backend + llama.cpp b3173 patch support#2153
Nikonz wants to merge 3 commits into
kvcache-ai:mainfrom
Nikonz:pr/ascend-npu-cpu-moe

Conversation

@Nikonz

@Nikonz Nikonz commented Aug 13, 2026

Copy link
Copy Markdown

What does this PR do?

Adds an Ascend NPU (CANN) backend to kt-kernel's CPU-expert-offload engine (CPUINFER_USE_ASCEND_NPU), including a CPU-side MXFP4 MoE kernel path, together with the llama.cpp b3173 patches it needs to build (gguf-py NumPy-2 reader fix, GGML_TYPE_MXFP4), auto-applied at CMake configure time.

This is part of a DeepSeek-V4-Flash CPU-offload port to Ascend 910B. Companion sglang-side PR: sgl-project/sglang#TBD (KT wiring for the AscendTP dispatcher).

Box-verified: full kt-kernel build from these exact commits succeeds — Ascend backend selected, both llama.cpp patches auto-applied by the new CMake step against a pristine checkout, .so builds and import kt_kernel_ext succeeds.

Opening as Draft/WIP for early visibility while the companion sglang PR is finalized — not requesting review yet.

Fixes # (issue)

N/A — new hardware backend, not a bug report.

Before submitting

  • Did you read the contributor guideline?
  • Did you write any new necessary tests? — exercised via a full on-hardware build (Ascend 910B) rather than a new unit test, since this backend requires NPU hardware to build/run; see verification notes above.

@greptile-apps greptile-apps Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Your trial has ended. Reactivate Greptile to resume code reviews.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant