[Bugfix][Store] Report the mounted segment's protocol in buffer descriptors - #4045
Open
NUABO wants to merge 1 commit into
Open
[Bugfix][Store] Report the mounted segment's protocol in buffer descriptors#4045NUABO wants to merge 1 commit into
NUABO wants to merge 1 commit into
Conversation
…iptors /query_key reported protocol_: "tcp" for replicas on an RDMA-mounted segment. The descriptor's protocol comes from AllocatedBuffer::protocol, which defaults to "tcp" and was only ever written by change_to_cxl(), so it never carried the protocol of the segment the buffer was allocated from. Carry the protocol on the allocator and stamp it when a buffer is constructed: - BufferAllocatorBase::getTransferProtocol() (defaults to "tcp", deliberately not pure so DummyBufferAllocator and out-of-tree subclasses keep compiling). - CachelibBufferAllocator / OffsetBufferAllocator keep the protocol they were created with; Create / CreateBufferAllocator / Import* take it as a trailing defaulted parameter so existing callers (tests, benchmarks) are unchanged. - The AllocatedBuffer ctor moves out of line (BufferAllocatorBase is incomplete in the header) and stamps the protocol from the owning allocator. - Production call sites pass it through: the segment mount path, the region driver open/adopt paths, and the HA remount / standby-restore imports, which now carry it in RegionResourceSpec.protocol. "cxl" is never stamped from an allocator. Unlike a transport it marks an address already offset-encoded by change_to_cxl(), which rewrites the pointer in the same call, so it must stay a per-buffer invariant: the CXL allocators are created without a protocol. Taking it from the allocator would let a buffer that has not been offset-encoded advertise "cxl", and deallocate() would then free ptr + DEFAULT_CXL_BASE — reachable by mounting a segment as "cxl" on a master with CXL disabled, which falls through to the regular allocator path. NoF segments keep the transfer default: NoFSegment has no protocol field (NoF replicas are addressed by endpoint alone). The HA snapshot format carries no protocol either, so allocators rebuilt from it keep the default. Tests: AllocatedBufferCarriesSegmentProtocol, CxlSegmentProtocolNotStampedOnFreshBuffers, ImportedBuffersCarrySegmentProtocol. Signed-off-by: tan changzhi <544463199@qq.com>
NUABO
requested review from
XucSh,
YiXR,
stmatengss and
ykwd
as code owners
September 11, 2026 10:09
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
/query_key reported protocol_: "tcp" for replicas on an RDMA-mounted segment. The descriptor's protocol comes from AllocatedBuffer::protocol, which defaults to "tcp" and was only ever written by change_to_cxl(), so it never carried the protocol of the segment the buffer was allocated from.
Carry the protocol on the allocator and stamp it when a buffer is constructed:
"cxl" is never stamped from an allocator. Unlike a transport it marks an address already offset-encoded by change_to_cxl(), which rewrites the pointer in the same call, so it must stay a per-buffer invariant: the CXL allocators are created without a protocol. Taking it from the allocator would let a buffer that has not been offset-encoded advertise "cxl", and deallocate() would then free ptr + DEFAULT_CXL_BASE — reachable by mounting a segment as "cxl" on a master with CXL disabled, which falls through to the regular allocator path.
NoF segments keep the transfer default: NoFSegment has no protocol field (NoF replicas are addressed by endpoint alone). The HA snapshot format carries no protocol either, so allocators rebuilt from it keep the default.
Tests: AllocatedBufferCarriesSegmentProtocol,
CxlSegmentProtocolNotStampedOnFreshBuffers,
ImportedBuffersCarrySegmentProtocol.
Description
Module
mooncake-transfer-engine)mooncake-store)mooncake-reshard)mooncake-ep)mooncake-pg)mooncake-integration)mooncake-p2p-store)mooncake-wheel)mooncake-common)mooncake-rl)Type of Change
How Has This Been Tested?
Test commands:
# Example: bash scripts/run_ci_test.shTest results:
Checklist
./scripts/code_format.shAI Assistance Disclosure