[GPU] tensor from nthandle by michal-miotk · Pull Request #35543 · openvinotoolkit/openvino

michal-miotk · 2026-04-27T10:45:15Z

Details:

Tickets:

CVS-179404

…prefix to dx12 tests

maxnick · 2026-05-27T10:58:38Z

+                                 const Shape& shape,
+                                 void* shared_buffer,
+                                 const MemType memory_type) {
+        OPENVINO_ASSERT(shared_buffer != nullptr, "shared_buffer must not be nullptr for SHARED_BUF memory type");


Suggested change

OPENVINO_ASSERT(shared_buffer != nullptr, "shared_buffer must not be nullptr for SHARED_BUF memory type");

OPENVINO_ASSERT(shared_buffer != nullptr, "shared_buffer must not be nullptr for SHARED_BUF memory type");

OPENVINO_ASSERT(memory_type == MemType::SHARED_BUF, "Only SHARED_BUF memory type is supported for raw buffer pointer or NT handle")

At first glance this check is useless as MemType contains only one record, but MemType may be extended independently, and in that case this check becomes meaningful.

maxnick · 2026-05-27T15:00:25Z

Did you evaluate how much the Vulkan bootstrap affects the overall configure/build time? Not it looks like a hard dependency for the functional tests, so Vulkan now must be always there whenever we build tests and only for one test. Should we consider extracting this test into a separate binary or converting it into a demo?

got 27.83s(this PR + setting set(Vulkan_FOUND OFF), not including one time when got hang) vs 9.52s (this PR without vulkan changes in cmake - similar to master cmake) for command cmake .. -DENABLE_INTEL_CPU=OFF -DENABLE_INTEL_NPU=OFF -DENABLE_TESTS=ON -DBUILD_TYPE=Release . To measure it was done 20times.

this file vulkan_handle.json takes about 5s to compile (from clang report), full build (gpuplugin + tests takes 33minutes)
{
"pid": 240035,
"tid": 240036,
"ph": "X",
"ts": 0,
"dur": 4730359,
"name": "Total ExecuteCompiler",
"args": {
"count": 1,
"avg ms": 4730
}
},

Good to know! Thanks!

…M_HANDLE

isanghao · 2026-06-02T12:05:02Z

+                                 const Shape& shape,
+                                 void* shared_buffer,
+                                 const MemType memory_type) {
+        OPENVINO_ASSERT(shared_buffer != nullptr, "shared_buffer must not be nullptr for SHARED_BUF memory type");


could you update documentation as well? docs/articles_en/openvino-workflow/running-inference/inference-devices-and-modes/gpu-device/remote-tensor-api-gpu-plugin.rst

isanghao · 2026-06-02T12:11:59Z

+     * @param type Tensor element type
+     * @param shape Tensor shape
+     * @param shared_buffer External memory handle from another API (DX12 shared NT handle on Windows,
+     *                     DMA-BUF fd on Linux), passed as void*


fd on Linux is handled as int from NPU plugin. I think it should be separated.

isanghao · 2026-06-02T12:12:37Z

+    ClBufferTensor create_tensor(const element::Type type,
+                                 const Shape& shape,
+                                 void* shared_buffer,
+                                 const MemType memory_type) {


I think this should support importing usm as well. It is identical with usm-version for first three arguments.

I think that at least at windows creating handles supports only allocation on gpu

isanghao · 2026-06-02T12:16:00Z

    BT_EMPTY,
    BT_BUF_INTERNAL,
    BT_BUF_SHARED,
+    BT_BUF_SHARED_IMPORTED,


could you leave some comment about this? it is difficult to distinguish against BT_BUF_SHARED. Maybe SHARED_FROM_HANDLE?

isanghao · 2026-06-02T12:16:42Z

    /// Create shared memory object using user-supplied memory buffer @p buf using specified @p layout
    memory_ptr share_buffer(const layout& layout, shared_handle buf);

+    virtual memory_ptr import_external_buffer(const layout& layout, shared_handle external_handle);


nit: import implies it is coming from outside. so I suggest to rename it to import_buffer

isanghao · 2026-06-02T12:20:55Z

+#ifdef _WIN32
+    constexpr auto handle_type_token = CL_EXTERNAL_MEMORY_HANDLE_OPAQUE_WIN32_KHR;
+#elif defined(__linux__)
+    constexpr auto handle_type_token = CL_EXTERNAL_MEMORY_HANDLE_DMA_BUF_KHR;


does this PR support dmabuf as well? if not, I think we need to add an assert, instead.

supports but tests from vulkan was deleted

isanghao · 2026-06-02T12:32:37Z

    cl::Buffer _buffer;
 };

+struct gpu_external_buffer : public gpu_buffer {


It is SHARED_IMPORTED and BUFFER_FROM_HANDLE from enum, but external here. Could you unify the term?

maxnick · 2026-06-03T14:15:42Z

    /// Create shared memory object using user-supplied memory buffer @p buf using specified @p layout
    memory_ptr share_buffer(const layout& layout, shared_handle buf);

+    virtual memory_ptr import_buffer(const layout& layout, shared_handle external_handle);


It makes sense to convert it to a pure method virtual memory_ptr import_buffer(const layout& layout, shared_handle external_handle) = 0; and implement a stub in the ze engine (ocl implementation is already a part of the PR)
Also please add a comment with this method description.

maxnick · 2026-06-03T14:26:26Z

+ * @ingroup ov_runtime_ocl_gpu_cpp_api
+ */
+#ifdef __linux__
+using handle_param = int;


NIT. One line above there is gpu_hanlde_param, here we see handle_param. This is rather confusing. Can a more appropriate type name be thought of?

michal-miotk added 16 commits March 30, 2026 16:33

wip

68d68c1

wip

3a92b0c

wip

17b5f13

wip

603c4fa

wip

2ebfdd3

wip

58ad113

works dx11

70604fb

pass dx12

e3025dc

x

fb20b2c

dx12 works

526e80e

works dx11

c962e4b

better dx12 test, closing handles in dx11 test

903fc7d

added vulkan test

68e500b

fix vulkan

ad3e5f6

compilation only on windows

434293f

delete unnecesssary things

1b3dec9

michal-miotk requested review from a team as code owners April 27, 2026 10:45

michal-miotk requested review from whitneyfoster and removed request for a team April 27, 2026 10:45

github-actions Bot added category: inference OpenVINO Runtime library - Inference category: GPU OpenVINO GPU plugin category: build OpenVINO cmake script / infra category: CPP API OpenVINO CPP API bindings category: docs_snippets OpenVINO docs snippets (docs/snippets) labels Apr 27, 2026

michal-miotk changed the title ~~[GPU] Remote tensor from nthandle~~ [GPU] tensor from nthandle Apr 27, 2026

michal-miotk added 2 commits April 27, 2026 13:01

delete unnecessary things

d6af4b9

delete unneccessary things v2

7ca34c8

Lyamin-Roman reviewed May 20, 2026

View reviewed changes

Comment thread src/plugins/intel_gpu/src/runtime/ocl/ocl_engine.cpp Outdated

Comment thread src/plugins/intel_gpu/src/plugin/remote_tensor.cpp Outdated

michal-miotk added 3 commits May 21, 2026 09:24

apply linux part of review

0046453

add windows prt of review

bfcfa60

added missing changes, renamed vulkan test file

db6d80c

Lyamin-Roman reviewed May 21, 2026

View reviewed changes

Comment thread src/plugins/intel_gpu/tests/functional/CMakeLists.txt Outdated

update version of minimal working driver on vulkan test, added smoke …

39aa251

…prefix to dx12 tests

p-durandin assigned maxnick May 21, 2026

michal-miotk added 2 commits May 21, 2026 14:28

symlinks

7447945

Merge branch 'master' into remoteNT

41c91f9

p-durandin assigned isanghao May 22, 2026

michal-miotk added 2 commits May 22, 2026 08:57

minor cleaning

d37555d

delete misleading dx context

0e8fff7

maxnick reviewed May 27, 2026

View reviewed changes

michal-miotk added 4 commits May 28, 2026 12:38

applied review comments, renamed OCL_BUFFER_FROM_HANDLE to BUFFER_FRO…

9685a7d

…M_HANDLE

fix format

568e042

fix format

825024d

delete vulkan test

e1df435

github-actions Bot removed the category: dependency_changes Pull requests that update a dependency file label Jun 1, 2026

isanghao reviewed Jun 2, 2026

View reviewed changes

apply review comments

99b0578

github-actions Bot added category: docs OpenVINO documentation category: docs_snippets OpenVINO docs snippets (docs/snippets) labels Jun 3, 2026

michal-miotk added 4 commits June 3, 2026 11:50

fix bad macro and code style

4f5cc5f

fix format

4fe5523

fix copyright

e541e85

fix linux

23edd7c

maxnick reviewed Jun 3, 2026

View reviewed changes

michal-miotk added 2 commits June 3, 2026 15:50

apply review comments

6e37a77

changed name

7a1c41c

	OPENVINO_ASSERT(shared_buffer != nullptr, "shared_buffer must not be nullptr for SHARED_BUF memory type");
	OPENVINO_ASSERT(shared_buffer != nullptr, "shared_buffer must not be nullptr for SHARED_BUF memory type");
	OPENVINO_ASSERT(memory_type == MemType::SHARED_BUF, "Only SHARED_BUF memory type is supported for raw buffer pointer or NT handle")

Conversation

michal-miotk commented Apr 27, 2026

Details:

Tickets:

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

7 participants