Reduce IR Constant Memory Retention#35457
Conversation
github-actions
Bot
added
category: Core
barnasm1
added
do_not_merge
and removed
category: Core
github-actions
Bot
added
category: Core
|
|
||
| const auto separate_const_weights_loading = use_separate_const_weights_loading() && !weights; | ||
| if (separate_const_weights_loading) { | ||
| weights_provider = std::make_shared<ov::util::FileWeightsProvider>(weights_path); |
There was a problem hiding this comment.
The same use should be if input is stream not only for path
| input_type.size()); | ||
| } | ||
| if (const_offset + const_size > weights->size()) { | ||
| OPENVINO_ASSERT(weights_provider, "Empty weights data in bin file or bin file cannot be found!"); |
There was a problem hiding this comment.
This assertion should be moved level up - it's local function which should operate in valid context.
I suppose InputModelIRImpl ctor should assert valid m_weights_provider, or even it's creator (since it's Impl)
|
|
||
| namespace { | ||
|
|
||
| class FileRegionBuffer : public ov::AlignedBuffer { |
There was a problem hiding this comment.
Why this buffer is required what is different from shared buffer?
| public: | ||
| virtual ~WeightsProvider() = default; | ||
|
|
||
| virtual std::shared_ptr<ov::AlignedBuffer> load_region(size_t offset, size_t size) const = 0; |
There was a problem hiding this comment.
| virtual std::shared_ptr<ov::AlignedBuffer> load_region(size_t offset, size_t size) const = 0; | |
| virtual void load(size_t offset, size_t size) = 0; |
It suggest to load data and modify the instance.
There was a problem hiding this comment.
why should it return void?
| std::shared_ptr<ov::util::WeightsProvider> make_buffer_weights_provider( | ||
| const std::shared_ptr<ov::AlignedBuffer>& weights) { | ||
| if (!weights) { | ||
| return nullptr; |
There was a problem hiding this comment.
Why null should be provided if there is not weights. Is possible to read such model?
| weights_stream.seekg(static_cast<std::streamoff>(offset), std::ios::beg); | ||
| OPENVINO_ASSERT(weights_stream.good(), "Failed to seek weights file ", m_weights_path, " to offset ", offset); | ||
|
|
||
| auto buffer = std::make_shared<FileRegionBuffer>(size, m_weights_source_id, offset, m_weights_source_handle); |
There was a problem hiding this comment.
emplacement done at:
https://github.com/openvinotoolkit/openvino/pull/35457/changes/BASE..f1bbb4e34ec70fd5d38ce2a3cfae638716a2bc74#diff-945f4375dd8e7f5fd62e0b5ceded4d858193d6262935389b98d92af073a02b6aR105
Or you mean some another approach?
| return found->second; | ||
| } | ||
|
|
||
| std::ifstream weights_stream(m_weights_path, std::ios::binary); |
There was a problem hiding this comment.
Is it required to open each time?
Can be opened during the while de-serialization process and closed when de-serializer removed with weight provider.
Consider bloc write during read to ensure file do not mutate on read?
| std::filesystem::path m_weights_path; | ||
| size_t m_weights_size = 0; | ||
| size_t m_weights_source_id = 0; | ||
| std::shared_ptr<ov::AlignedBuffer> m_weights_source_handle; |
There was a problem hiding this comment.
It keeps the data source buffer + buffer in the map.
Is there no data duplication?
Will this provider will not live too long that data buffers set in Constant node will be not blocked to be removed if Constant node removed?
barnasm1
changed the title
…ethods, use m_weights_provider directly
3 participants
Details:
Tickets:
AI Assistance: