refactor(agent): isolate cuinterpose host-carrier storage - #292
Closed
galletas1712 wants to merge 1 commit into
Closed
galletas1712 wants to merge 1 commit into
galletas1712 wants to merge 1 commit into
Conversation
|
Important Draft PR not reviewedDraft PRs are not automatically reviewed by default.
To automatically review draft PRs, update your CodeRabbit configuration: reviews:
auto_review:
drafts: trueComment |
galletas1712
added this pull request to stack #293
September 11, 2026 06:11
This was referenced Sep 11, 2026
galletas1712
force-pushed
the
feat/cuinterpose-host-carrier
branch
from
September 11, 2026 06:16
df595d0 to
a6751ce
Compare
galletas1712
force-pushed
the
feat/cuinterpose-host-carrier
branch
from
September 11, 2026 06:21
a6751ce to
bbe2299
Compare
galletas1712
force-pushed
the
feat/cuinterpose-host-carrier
branch
2 times, most recently
from
September 11, 2026 08:31
34ac128 to
fe05a59
Compare
galletas1712
force-pushed
the
feat/cuinterpose-host-carrier
branch
from
September 11, 2026 09:41
fe05a59 to
f6b13d4
Compare
Move pinned host arena management, staging mappings, context batching, and D2H/H2D copies behind a small concrete module. The lifecycle remains responsible for selecting allocations and publishing restored topology, so a future PageBroker content path can replace this implementation without changing handle tracking or phase ordering. Signed-off-by: Schwinn Saereesitthipitak <schwinns@nvidia.com>
galletas1712
force-pushed
the
feat/cuinterpose-host-carrier
branch
from
September 11, 2026 10:08
f6b13d4 to
8a2ab87
Compare
This was referenced Sep 12, 2026
Contributor
Author
|
Superseded by the replacement 13-PR C-frontend/Rust-backend cuinterpose draft stack: https://github.com/ai-dynamo/snapshot/stack/339 (#326–#338), tracking approved proposal #295. The new stack preserves one reviewable stage per PR, with tests collected in #338 and the design in #337. PageBroker transfer changes remain a separate dependency. Closing this older C implementation; retaining its branch and discussion for reference. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Layer 8 of the eleven-PR cuinterpose stack #293. Closed PR #214 is intentionally not in the active stack.
This PR isolates the current allocation-content transport in
host_carrier.c/.h, with CUDA context scoping incontext.c/.h. It does not add lifecycle operations, change allocation eligibility, introduce a backend registry, or add a save-all flag. #218 calls this concrete module from the genericSAVE_ALLOCATIONSandLOAD_ALLOCATIONSoperations.The public surface is deliberately small: save a batch, load a batch, release restored storage, and discard inherited CPU-side state after
fork. The lifecycle caller supplies each allocation's context, properties, size, device-handle pointer, and host-address pointer; it remains responsible for eligibility, mappings, access grants, export caches, and importer topology.On save, the module creates one anonymous host arena per shim process and registers it as portable pinned memory. It groups allocations by CUDA context, maps each group into a contiguous temporary device VA range, and copies D2H on one nonblocking stream with one synchronization per group. CRIU captures the arena as ordinary process memory.
On load, the module verifies or recreates host registration, creates fresh physical allocations with the original properties, stages each context group, copies H2D, and publishes fresh handles only after the complete batch succeeds. Save and load each use an explicit operation-state object with scoped cleanup and direct returns, so partial failures release fresh handles and partial host state without
goto. The adjacent context helper restores the caller's prior context and can retain a device primary context when tracking never observed one.PageBroker boundary
This is a replacement seam, not a speculative pluggable framework. PageBroker currently implements only
TransferEngineType::POSIX_COPYand registers onlyPosixCopyEngine; its protobuf reservesDirectRestoreRequestfor a future GPU engine. There is no GPU engine to integrate with today.When PageBroker defines a GPU/direct engine and a transaction contract for allocation IDs, content, and completion barriers, the two host-carrier lifecycle calls are the intended replacement point. Tickets, participant identity, topology records, and coordinator ordering should not need to become PageBroker-aware.
Stack boundary
Based on #217. #218 owns which supported exportable allocations preserve content and composes this module with unicast teardown, creator publication, and importer reconstruction.
Validation
The pinned CUDA 13.1 builder compiles
host_carrier.cinto the production shim with-Werror. The #218 lifecycle suite exercises successful save/load, context grouping, missing registration recovery, failure cleanup, and fork behavior through the fake CUDA driver; the final stack passes all 4 lifecycle tests under ASan/UBSan.make testpasses in all Go modules. The final published stack head0eae9c4f9b66093332f00a332afaf56048e1e64apassed the fullmake checkgate, the CUDA 13.1 production build, and all 73 sanitizer-backed native cuinterpose tests. Its physical-GPU suite passed all 3 tests with no skips on two DRA-assigned NVIDIA B200 GPUs. The unicast test kept an explicit allocation-ID ticket-backed peer mapping per worker live across capture and restore; the multicast test required a nonzero multicast VA and shim-logical handle, exercisedBindAddr, and passed its collective and captured-graph replay. Detailed hardware evidence and measurements are in #220.