feat: enable cuinterpose for annotated Pods

[galletas1712]

Summary

Layer 3 of the eleven-PR cuinterpose stack #293. Closed PR #214 is intentionally not in the active stack.

This PR introduces the explicit workload opt-in and makes the shim part of each target process from startup:

metadata:
  annotations:
    nvidia.com/cuinterpose: enabled

An absent annotation disables interposition; any other present value is rejected. CUDA tool delivery and multi-GPU launch wrapping are independent contracts owned by #213. For an enabled template, ShapeCuinterposeCapture only places /tmp/snapshot-cuda/libcuinterpose.so first in each target's LD_PRELOAD. Existing preload entries are retained, duplicates are removed, and a valueFrom-backed LD_PRELOAD is rejected because it cannot be rewritten safely. The annotation does not cause a one-GPU target to use cuda-checkpoint --launch-job.

The shape is idempotent and mutates only a deep copy until every check succeeds. The operator verifies the immutable contract before adopting an existing source Job and records the opt-in annotation on the generated PodSnapshot.

At this layer the packaged shim is still a placeholder. The durable contribution is the process-start contract: later layers can intercept each target process and CUDA-using fork child from its first CUDA operation.

Stack boundary

Based on #213. #223 adds agent-side detection and coordinator invocation. This PR does not start shim sockets, assign participant identities, or alter CUDA behavior.

Validation

On the final stack, make test passes in api, agent, and operator. The pinned CUDA 13.1 builder compiles the production binaries and passes all cuinterpose fake-driver suites. The final published stack head 0eae9c4f9b66093332f00a332afaf56048e1e64a passed the full make check gate, the CUDA 13.1 production build, and all 73 sanitizer-backed native cuinterpose tests. Its physical-GPU suite passed all 3 tests with no skips on two DRA-assigned NVIDIA B200 GPUs. The unicast test kept an explicit allocation-ID ticket-backed peer mapping per worker live across capture and restore; the multicast test required a nonzero multicast VA and shim-logical handle, exercised BindAddr, and passed its collective and captured-graph replay. Detailed hardware evidence and measurements are in #220.

[coderabbitai]

Important

Draft PR not reviewed

Draft PRs are not automatically reviewed by default.

• Trigger a manual review

To automatically review draft PRs, update your CodeRabbit configuration:

reviews:
  auto_review:
    drafts: true

---

_{Comment @coderabbitai help to get the list of available commands.}

[copy-pr-bot]

This pull request requires additional validation before any workflows can run on NVIDIA's runners.

Pull request vetters can view their responsibilities here.

Contributors can view more details about this message here.

[galletas1712]

Superseded by the replacement 13-PR C-frontend/Rust-backend cuinterpose draft stack: https://github.com/ai-dynamo/snapshot/stack/339 (#326–#338), tracking approved proposal #295. The new stack preserves one reviewable stage per PR, with tests collected in #338 and the design in #337. PageBroker transfer changes remain a separate dependency. Closing this older C implementation; retaining its branch and discussion for reference.