Feature, motivation, and pitch
Support W4A8 quantization for DeepSeek-V4-Flash (DSV4-Flash) on the Ascend NPU Async AFD path, enabling this model's quantized deployment with asynchronous Attention/FFN execution.
Proposed change
- Enable W4A8 weight loading and quantized execution for DSV4-Flash in the NPU Async path.
- Integrate the native Ascend quantization operators and required quantization metadata with Async FFN execution and connector communication.
- Document supported checkpoint formats, runtime requirements, and launch configuration.
vLLM compatibility and extension points
Target the repository-pinned vLLM/vLLM-Ascend runtime pair. Implement through plugin-owned NPU model, runner, and connector extension points, reusing native operators where possible. Any necessary compatibility patch must be minimal, version-pinned, and documented with an upstream/removal plan; do not modify upstream source trees.
Alternatives considered
Continue using the currently supported quantization configurations; this does not provide W4A8 support for DSV4-Flash Async deployment.
Validation plan
- Verify quantized weight loading, scales, tensor shapes, and communication metadata.
- Run DSV4-Flash W4A8 Async E2E on Ascend hardware, including repeated requests and supported prefill/decode scenarios.
- Compare correctness and accuracy against a native W4A8 reference with the same checkpoint; record throughput, latency, and memory use.
- Confirm existing NPU Async configurations remain functional.
Before submitting
Feature, motivation, and pitch
Support W4A8 quantization for DeepSeek-V4-Flash (DSV4-Flash) on the Ascend NPU Async AFD path, enabling this model's quantized deployment with asynchronous Attention/FFN execution.
Proposed change
vLLM compatibility and extension points
Target the repository-pinned vLLM/vLLM-Ascend runtime pair. Implement through plugin-owned NPU model, runner, and connector extension points, reusing native operators where possible. Any necessary compatibility patch must be minimal, version-pinned, and documented with an upstream/removal plan; do not modify upstream source trees.
Alternatives considered
Continue using the currently supported quantization configurations; this does not provide W4A8 support for DSV4-Flash Async deployment.
Validation plan
Before submitting