[Conductor] conductor: tiered prefix indexer - #4041
Open
Misak2333 wants to merge 1 commit into
Open
Conversation
Third slice of the Mooncake Conductor upstreaming series, on top of the prefix index types and hash strategy. Adds PrefixCacheTable, the in-memory index that answers cache-hit queries across GPU, CPU and disk tiers Co-authored-by: Misak2333 <167268798+Misak2333@users.noreply.github.com> Co-authored-by: Chase-R <lolopop.rong@gmail.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
Third slice of the Mooncake Conductor upstreaming series, on top of the
prefix index types and hash strategy. Adds PrefixCacheTable, the
in-memory index that answers cache-hit queries across GPU, CPU and disk
tiers:
binding validation, the GPU and shared-tier store/remove/clear
mutations, the Query prefix walk, and GetGlobalView. Blocks are keyed
by ProjectedPrefix and carry per-tier owner sets, so one walk reports
the longest match per instance plus per-dp-rank and per-tier hit
counts, with optional cache_salt and instance filtering.
lookup, and each ContextState has its own mutex. Contexts are held by
shared_ptr so a query drops the map lock before touching state, and
the lock order is fixed one way (map, then state) to keep it
deadlock-free. Query does not reorder entries, so read-only traffic
keeps a shared lock.
disables it) with insertion-order eviction that drops the oldest
entries in batches down to 90% occupancy, and a cumulative eviction
counter that surfaces stored/removed events going out of sync.
(presence snapshots, order-metadata sizes, state locks) without
widening the public surface, through a single friend declaration.
Module
mooncake-transfer-engine)mooncake-store)mooncake-reshard)mooncake-ep)mooncake-pg)mooncake-integration)mooncake-p2p-store)mooncake-wheel)mooncake-common)mooncake-rl)Type of Change
How Has This Been Tested?
Test commands:
# Example: bash scripts/run_ci_test.shTest results:
Checklist
./scripts/code_format.shAI Assistance Disclosure