feat(agent): orchestrate the cuinterpose coordinator around checkpoint and restore

[galletas1712]

Summary

Layer 4 of the eleven-PR cuinterpose stack #293. This PR replaces the closed #214 in the active stack; #214 remains closed and unstacked.

This PR connects the Snapshot agent to the short-lived C coordinator. The Go agent never implements the shim wire protocol. It maps host-observed CUDA PIDs to namespace PIDs, verifies the shim's per-process Unix sockets, invokes coordinator prepare immediately before native CUDA capture, and invokes coordinator restore after CRIU/native CUDA restore while the workload remains parked.

Detection is fail closed. An opted-in workload must expose one real socket for every CUDA process; an unannotated workload must expose none. Partial coverage, a non-socket path, a missing shim after opt-in, or an unexpected shim without opt-in prevents capture. Procfs environment is not accepted as evidence because workloads can overwrite it.

The manifest keeps three separate facts: cudaTools.delivered recreates file-backed tool paths, cuinterpose.requested records source opt-in and triggers stale-socket cleanup, and cuinterpose.prepared means destructive prepare completed and requires cuinterpose.state on restore.

For capture, the agent opens the trusted coordinator executable, checkpoint directory, and target mount, UTS, IPC, network, and PID namespace descriptors. It enters those pinned namespaces and executes the coordinator through /proc/self/fd/<n>; inside that context the coordinator addresses /snapshot-control directly and writes its state through the already-open checkpoint-directory descriptor. Restore uses the same namespace set through nsrestore, with the coordinator binary opened before CRIU and later executed through its descriptor. User and cgroup context deliberately remain with the agent.

The control directory is a Pod-local emptyDir, mounted with subPath=<container-name> so Snapshot-managed target containers have isolated views, and shim socket paths are mode 0600. Source and restore shaping reject reserved-volume collisions unless they match that exact private contract, so a workload cannot substitute a PVC or hostPath. A filesystem UDS is not exposed over the Pod network, so another ordinary Pod cannot see or connect to it. The workload Pod remains one trust domain: a sidecar deliberately given the reserved volume can reach it. Node root and equivalently privileged workloads remain inside the trusted node boundary; the Snapshot agent itself must have that privilege to checkpoint containers.

There is no rollback after successful destructive prepare: a later native checkpoint or CRIU failure is fail-stop. During restore, native CUDA is unlocked so shim CUDA calls can run, but application threads remain behind Snapshot's restore-complete sentinel until coordinator reconstruction succeeds.

Stack boundary

Based on #222. #215 adds transparent CUDA forwarding, #216 implements coordinator logic, and #217 is the first layer that starts participant sockets and tracks state.

Validation

On the final stack, make test passes in all three Go modules. The agent tests cover detection, manifest state, stale-socket cleanup, progress parsing, all five pinned namespace descriptors, and the executable/checkpoint descriptor and coordinator argv contract. The pinned CUDA 13.1 builder passes all C/C++ fake-driver suites. The final published stack head 0eae9c4f9b66093332f00a332afaf56048e1e64a passed the full make check gate, the CUDA 13.1 production build, and all 73 sanitizer-backed native cuinterpose tests. Its standalone physical-GPU suite passed all 3 data-plane tests with no skips on two DRA-assigned NVIDIA B200 GPUs. The unicast test kept an explicit allocation-ID ticket-backed peer mapping per worker live across capture and restore; the multicast test required a nonzero multicast VA and shim-logical handle, exercised BindAddr, and passed its collective and captured-graph replay. Detailed hardware evidence and measurements are in #220.

[coderabbitai]

Important

Draft PR not reviewed

Draft PRs are not automatically reviewed by default.

• Trigger a manual review

To automatically review draft PRs, update your CodeRabbit configuration:

reviews:
  auto_review:
    drafts: true

---

_{Comment @coderabbitai help to get the list of available commands.}

[copy-pr-bot]

This pull request requires additional validation before any workflows can run on NVIDIA's runners.

Pull request vetters can view their responsibilities here.

Contributors can view more details about this message here.

[galletas1712]

Superseded by the replacement 13-PR C-frontend/Rust-backend cuinterpose draft stack: https://github.com/ai-dynamo/snapshot/stack/339 (#326–#338), tracking approved proposal #295. The new stack preserves one reviewable stage per PR, with tests collected in #338 and the design in #337. PageBroker transfer changes remain a separate dependency. Closing this older C implementation; retaining its branch and discussion for reference.