[NPU] Update commandlist for dynamic pipeline by XinWangIntel · Pull Request #35626 · openvinotoolkit/openvino

XinWangIntel · 2026-04-30T10:45:05Z

Details:

Added the CommandListMode enum with possible values DEFAULT, FORCE_UPDATE_MUTABLE_COMMANDLIST and FORCE_COMMANDLIST_RECORDING_ONLY, and integrated it as a property (NPU_COMMANDLIST_MODE) in the plugin's configuration system. This allows users to select the command list update strategy for the NPU plugin
Refactored the logic in executeGraph to check for input/output tensor changes (pointer, shape, stride), and to decide whether to record a new command list, update an existing one, or reuse it, based on the selected mode and device capabilities.

Tickets:

C182267

AI Assistance:

AI assistance used: no / yes
If yes, summarize how AI was used and what human validation was performed (build/tests/manual checks).

insoow · 2026-05-12T02:41:49Z

+        : _engineProperties{},
+          _logger("DynamicGraphImpl", Logger::global().level()) {
+        if (config.has<COMMANDLIST_MODE>()) {
+            _bindingCommandListMode = config.get<COMMANDLIST_MODE>();


can you add a check to see if optimized_dynamic_strides is supported? If it is not, set _bindingCommandListMode = DEFAULT.

At this stage, we may just do compilation without device and not able to get value of optimized_dynamic_supported. By the way, We do not have DEFAULT in value now, Does default means recording commandlist and only update if just ptr change

Is safe to just get the value and don't need to check if it exists.

Done, thanks

insoow · 2026-05-12T02:48:09Z

    std::shared_ptr<DynamicGraph::GraphArgumentsImpl> argsImpl =
        args._impl ? std::static_pointer_cast<DynamicGraph::GraphArgumentsImpl>(args._impl)
                   : std::make_shared<DynamicGraph::GraphArgumentsImpl>();

-    bool noTensorChange = true;
+    std::vector<uint64_t> commandListIndexArray;
+    bool noShapeChange = true;


can you rename it with cmdListRecordingRequired?

insoow · 2026-05-12T02:52:03Z

+
+            // according to spec, CloseCommandList should be called after
+            // UpdateMutableCommandList is called.
+            for (auto& cmdList : commandLists) {


please remove this close, we will command list submission will be done in npuVMRuntimeUpdateMutableCommandList.

insoow · 2026-05-12T02:54:07Z

+                zeCommandListClose(cmdList);
+            }
+        } else {
+            _logger.debug("Reuse command list without update since no tensor change detected");


move command list submission here.
auto result = zeCommandQueueExecuteCommandLists(commandQueue,
static_cast<uint32_t>(commandLists.size()),
commandLists.data(),
fence);
if (result != ZE_RESULT_SUCCESS) {
OPENVINO_THROW("Failed to submit command lists");
}

Done, but this impact current test which only update tensor ptr and then we update commandlist too

insoow · 2026-05-12T02:54:26Z

+            }
+        } else {
+            _logger.debug("Reuse command list without update since no tensor change detected");
+        }

        auto result = zeCommandQueueExecuteCommandLists(commandQueue,


Remove command list submission here.

Copilot

Pull request overview

This PR updates the Intel NPU plugin’s dynamic host pipeline behavior to support updating (rather than always resetting) command lists when tensor pointers/strides change, and wires a new internal configuration option (commandlist_mode) through plugin properties/options. It also expands functional coverage around command-list reuse/update scenarios.

Changes:

Introduce ov::intel_npu::CommandListMode / NPU_COMMANDLIST_MODE and register it across options + plugin/compiled-model properties.
Update dynamic graph execution logic to decide between command-list reset, reuse, or mutable-command-list update based on tensor changes and driver capability.
Extend functional tests to validate command-list update/reuse logging in additional scenarios (reference comparison, aligned external memory, etc.).

Reviewed changes

Copilot reviewed 10 out of 10 changed files in this pull request and generated 5 comments.

Show a summary per file

File	Description
src/plugins/intel_npu/tests/functional/internal/plugin/test_properties.hpp	Registers `COMMANDLIST_MODE` in test property manager.
src/plugins/intel_npu/tests/functional/behavior/dynamic_host_pipeline/infer_with_host_compile.hpp	Expands dynamic host pipeline tests and updates expected log messages for command-list update behavior.
src/plugins/intel_npu/src/plugin/src/properties.cpp	Registers `commandlist_mode` as a plugin + compiled-model property and treats it as a “special both” property.
src/plugins/intel_npu/src/plugin/src/plugin.cpp	Adds `COMMANDLIST_MODE` to plugin option registration.
src/plugins/intel_npu/src/plugin/include/properties.hpp	Adds `commandlist_mode` to supported property names list.
src/plugins/intel_npu/src/compiler_adapter/src/ze_graph_ext_wrappers.cpp	Queries driver support for `OPTIMIZED_DYNAMIC_STRIDE`.
src/plugins/intel_npu/src/compiler_adapter/src/dynamic_graph.cpp	Implements command-list reuse/update/reset decision logic using tensor change detection and capability checks; reads `COMMANDLIST_MODE`.
src/plugins/intel_npu/src/compiler_adapter/include/ze_graph_ext_wrappers.hpp	Exposes `isOptimizedDynamicStridesSupported()` and stores capability result.
src/plugins/intel_npu/src/al/include/intel_npu/npu_private_properties.hpp	Defines `CommandListMode` and `commandlist_mode` property.
src/plugins/intel_npu/src/al/include/intel_npu/config/options.hpp	Adds `COMMANDLIST_MODE` option parsing and default value.

pereanub · 2026-05-18T13:01:40Z

+        : _engineProperties{},
+          _logger("DynamicGraphImpl", Logger::global().level()) {
+        if (config.has<COMMANDLIST_MODE>()) {
+            _bindingCommandListMode = config.get<COMMANDLIST_MODE>();


Is safe to just get the value and don't need to check if it exists.

pereanub · 2026-05-18T13:06:05Z

+    }
+
+    static OptionMode mode() {
+        return OptionMode::Both;


For Both options, we may still need to consider removing it from serialization if set. I don't think it's the case for the host pipeline, but can we consider removing it or throwing an error in case this property is set but the host compile isn't used?

Only need for RunTime, changed

pereanub · 2026-05-18T13:11:43Z

@@ -536,6 +536,7 @@ void Properties::registerPluginProperties() {
    TRY_REGISTER_SIMPLE_PROPERTY(ov::intel_npu::profiling_type, PROFILING_TYPE);
    TRY_REGISTER_SIMPLE_PROPERTY(ov::intel_npu::backend_compilation_params, BACKEND_COMPILATION_PARAMS);
    TRY_REGISTER_SIMPLE_PROPERTY(ov::intel_npu::batch_mode, BATCH_MODE);
+    TRY_REGISTER_SIMPLE_PROPERTY(ov::intel_npu::commandlist_mode, COMMANDLIST_MODE);


Suggested change

TRY_REGISTER_SIMPLE_PROPERTY(ov::intel_npu::commandlist_mode, COMMANDLIST_MODE);

pereanub · 2026-05-18T13:12:33Z

@@ -61,6 +75,9 @@ class DynamicGraphImpl : public DynamicGraph::Impl {
    npu_vm_runtime_handle_t _engine = nullptr;
    npu_vm_runtime_properties_t _engineProperties;
    DynamicGraph::GraphArguments _binding;
+    ov::intel_npu::CommandListMode _bindingCommandListMode =
+        ov::intel_npu::CommandListMode::FORCE_UPDATE_MUTABLE_COMMANDLIST;


Not needed this if using get<> without has

Copilot

Pull request overview

Copilot reviewed 11 out of 11 changed files in this pull request and generated 5 comments.

Copilot

Pull request overview

Copilot reviewed 11 out of 11 changed files in this pull request and generated 2 comments.

Copilot

Pull request overview

Copilot reviewed 11 out of 11 changed files in this pull request and generated 1 comment.

Copilot

Pull request overview

Copilot reviewed 12 out of 12 changed files in this pull request and generated 2 comments.

Copilot

Pull request overview

Copilot reviewed 12 out of 12 changed files in this pull request and generated no new comments.

Copilot

Pull request overview

Copilot reviewed 11 out of 11 changed files in this pull request and generated 3 comments.

Copilot

Pull request overview

Copilot reviewed 11 out of 11 changed files in this pull request and generated 3 comments.

Copilot

Pull request overview

Copilot reviewed 11 out of 11 changed files in this pull request and generated no new comments.

*Add commandlist_mode option *Update executeGraph Signed-off-by: Xin Wang <xin1.wang@intel.com>

Signed-off-by: Xin Wang <xin1.wang@intel.com> Signed-off-by: Xin Wang <xin1.wang@intel.com>

Signed-off-by: Xin Wang <xin1.wang@intel.com>

Signed-off-by: Xin Wang <xin1.wang@intel.com> Signed-off-by: Xin Wang <xin1.wang@intel.com>

Signed-off-by: Xin Wang <xin1.wang@intel.com>

github-actions Bot added the category: NPU OpenVINO NPU plugin label Apr 30, 2026

XinWangIntel force-pushed the update_commandlist branch from f4a2f15 to 951552c Compare May 7, 2026 01:57

XinWangIntel marked this pull request as ready for review May 7, 2026 02:26

XinWangIntel requested review from a team as code owners May 7, 2026 02:26

XinWangIntel force-pushed the update_commandlist branch 2 times, most recently from 77b23b5 to b8df713 Compare May 11, 2026 09:21

insoow reviewed May 12, 2026

View reviewed changes

XinWangIntel requested a review from Copilot May 14, 2026 02:56

Copilot started reviewing on behalf of XinWangIntel May 14, 2026 02:57 View session

Copilot AI reviewed May 14, 2026

View reviewed changes

XinWangIntel force-pushed the update_commandlist branch 2 times, most recently from f829c93 to 94e513d Compare May 14, 2026 06:23

pereanub reviewed May 18, 2026

View reviewed changes

XinWangIntel force-pushed the update_commandlist branch from 8d0d1f5 to aa37077 Compare May 25, 2026 06:43

XinWangIntel requested a review from Copilot May 27, 2026 07:14

Copilot started reviewing on behalf of XinWangIntel May 27, 2026 07:15 View session

Copilot AI reviewed May 27, 2026

View reviewed changes

XinWangIntel force-pushed the update_commandlist branch 2 times, most recently from 61f6d06 to b3809c6 Compare May 28, 2026 06:33

XinWangIntel requested a review from Copilot May 28, 2026 06:50

Copilot started reviewing on behalf of XinWangIntel May 28, 2026 06:50 View session