Skip to content

[Feature]: Support W4A8 for DeepSeek-V4-Flash on NPU Async AFD #330

Description

@jiangkuaixue123

Feature, motivation, and pitch

Support W4A8 quantization for DeepSeek-V4-Flash (DSV4-Flash) on the Ascend NPU Async AFD path, enabling this model's quantized deployment with asynchronous Attention/FFN execution.

Proposed change

  • Enable W4A8 weight loading and quantized execution for DSV4-Flash in the NPU Async path.
  • Integrate the native Ascend quantization operators and required quantization metadata with Async FFN execution and connector communication.
  • Document supported checkpoint formats, runtime requirements, and launch configuration.

vLLM compatibility and extension points

Target the repository-pinned vLLM/vLLM-Ascend runtime pair. Implement through plugin-owned NPU model, runner, and connector extension points, reusing native operators where possible. Any necessary compatibility patch must be minimal, version-pinned, and documented with an upstream/removal plan; do not modify upstream source trees.

Alternatives considered

Continue using the currently supported quantization configurations; this does not provide W4A8 support for DSV4-Flash Async deployment.

Validation plan

  • Verify quantized weight loading, scales, tensor shapes, and communication metadata.
  • Run DSV4-Flash W4A8 Async E2E on Ascend hardware, including repeated requests and supported prefill/decode scenarios.
  • Compare correctness and accuracy against a native W4A8 reference with the same checkpoint; record throughput, latency, and memory use.
  • Confirm existing NPU Async configurations remain functional.

Before submitting

  • I searched existing issues and RFCs.
  • I identified whether this belongs in plugin-owned code, compat helpers, or compat patches.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions