feat(agent): checkpoint CUDA multicast state

[galletas1712]

Summary

Layer 10 of the eleven-PR cuinterpose stack #293, based on #218. Closed PR #214 is not in the active stack.

This PR extends logical handles, descriptor tickets, export-cache ownership, inspection, and lifecycle state to CUDA multicast objects used by NCCL NVLS and PyTorch symmetric memory.

Tracked objects use exactly the POSIX FD handle type. Private handle-type-0 multicast objects remain native. Unsupported nonzero handle types run normally but cause preflight refusal before teardown.

Private versus shared VMM ownership

The updated #218 leaves private VMM allocations to native CUDA checkpoint, even if those allocations are POSIX-export-capable. This layer retains the important exception: successful binding of a tracked allocation into multicast marks that allocation shared. Such members still use the interposer's host-carrier and reconstruction path even when the application never exported a unicast ticket.

Private allocations never exported and never bound into tracked multicast remain mapped with their native handles intact during interposer preparation. Native CUDA owns their checkpoint and restore. Legacy cuIpc* ownership is unchanged.

Multicast lifecycle

The shim records requested properties, observed extent, attached devices, BindMem/BindAddr bindings, maps, and access grants. Unsupported untracked handle bindings remain refused; address bindings retain their existing native-address contract.

PREPARE_MULTICAST → SAVE_ALLOCATIONS → PREPARE_UNICAST

Multicast preparation runs concurrently across participants, closing exports, unmapping ranges, unbinding members, and releasing object handles before shared unicast teardown.

After creator and importer unicast reconstruction, multicast restore uses:

RESTORE_MULTICAST_CREATORS
  → RESTORE_MULTICAST_IMPORTERS
  → RESTORE_MULTICAST_DEVICES
  → RESTORE_MULTICAST_BINDINGS

The barriers prevent descriptor-fetch and collective bind/add-device deadlocks. Collective driver calls execute without the main state lock, then revalidate object identity and lifecycle state before committing topology.

Stack transport

The private-VMM ownership fix is introduced in #218 and merged forward here without rewriting the existing branch history. This update adds no NIXL transfer code, CustomStorage implementation, driver patch, or phase-batching experiment to the interposer stack.

Validation

Source review confirms that successful tracked multicast binding still sets allocation->shared; the inherited shared-only predicate therefore preserves member content. Preparation checks passed: git diff --check and strict C syntax checking against CUDA 13.1 headers.

Candidate a554da95b54d6f375a0f938f45be209b6ba3a986 passes all 74 native fake-driver tests, including five lifecycle and six multicast tests, with the configured ASan/UBSan instrumentation. Tests used CUDA 13.1 headers, the system GCC/G++ toolchain, cached GoogleTest, and a short control-socket directory. Logs: stack-private-vmm/pr219-final-tests.log.

The full stack at exact #220 head 21008b50b93a9879a805665e331e777bb93abf49, including this update, passed all 3 physical-GPU tests with 0 skips and 0 failures in 19.896 s on two B200 GPUs on nscale-dev, 2026-09-12 UTC. This is full-stack GPU validation, not a standalone GPU run of #219. Historical results for 73 native tests refer to old stack head 0eae9c4f9b66093332f00a332afaf56048e1e64a, not this update.

Contribution notes

This updates the existing stack rather than creating duplicate work. AI assistance was used; human review and actual test results remain required.

GPU compilation used the documented ALLOW_OLD_CUDA_HEADERS=1 override with CUDA 13.0 headers. No CustomStorage, NIXL, or driver-experiment changes are included in this stack update.

[coderabbitai]

Important

Draft PR not reviewed

Draft PRs are not automatically reviewed by default.

• Trigger a manual review

To automatically review draft PRs, update your CodeRabbit configuration:

reviews:
  auto_review:
    drafts: true

---

_{Comment @coderabbitai help to get the list of available commands.}

[copy-pr-bot]

This pull request requires additional validation before any workflows can run on NVIDIA's runners.

Pull request vetters can view their responsibilities here.

Contributors can view more details about this message here.

[galletas1712]

Superseded by the replacement 13-PR C-frontend/Rust-backend cuinterpose draft stack: https://github.com/ai-dynamo/snapshot/stack/339 (#326–#338), tracking approved proposal #295. The new stack preserves one reviewable stage per PR, with tests collected in #338 and the design in #337. PageBroker transfer changes remain a separate dependency. Closing this older C implementation; retaining its branch and discussion for reference.