Skip to content

feat(cdi/generate): add --no-ipc-sockets flag to exclude IPC sockets - #2086

Open
Pcmhacker-piro wants to merge 2 commits into
NVIDIA:mainfrom
Pcmhacker-piro:fix/cdi-generate-no-ipc-sockets
Open

Pcmhacker-piro wants to merge 2 commits into
NVIDIA:mainfrom
Pcmhacker-piro:fix/cdi-generate-no-ipc-sockets

Conversation

@Pcmhacker-piro

Copy link
Copy Markdown

What does this PR do?

This PR adds a dedicated --no-ipc-sockets flag (with aliases --disable-ipc-sockets and --disable-ipc-discoverer) and a corresponding environment variable NVIDIA_CTK_CDI_GENERATE_NO_IPC_SOCKETS to nvidia-ctk cdi generate. This allows excluding NVIDIA IPC domain sockets (/run/nvidia-persistenced/socket, /run/nvidia-fabricmanager/socket, and /tmp/nvidia-mps) from generated CDI specifications.

Changes proposed in this pull request:

CLI & Spec Generation (cmd/nvidia-ctk/cdi/generate/)

  • Expose CLI Flag & Environment Variable: Added --no-ipc-sockets (aliased to --disable-ipc-sockets and --disable-ipc-discoverer) with source env var NVIDIA_CTK_CDI_GENERATE_NO_IPC_SOCKETS.
  • Feature Flag Plumbing: In validateFlags, when noIPCSockets is enabled, nvcdi.FeatureDisableIPCDiscoverer is automatically appended to opts.featureFlags.
  • Unit & Regression Testing: Added TestGenerateSpec/no-ipc-sockets in generate_test.go verifying flag validation and CDI specification output without IPC mounts.

Systemd Configuration (deployments/systemd/)

  • Documented Environment Variable: Updated nvidia-cdi-refresh.env documenting NVIDIA_CTK_CDI_GENERATE_NO_IPC_SOCKETS=true for sandboxed container runtimes (such as gVisor runsc or Kata Containers).

NVCDI Library & Discoverer (pkg/nvcdi/ & internal/discover/)

  • Discoverer Unit Tests: Added pkg/nvcdi/driver-nvml_test.go asserting that newIPCDiscoverer returns the discoverer by default, but returns nil, nil when FeatureDisableIPCDiscoverer is toggled.
  • Typo Fix: Fixed typo in internal/discover/ipc.go doc comment (creats -> creates).

Checklist

  • No secrets, sensitive information, or unrelated changes
  • Unit tests passing (make test)
  • Lint checks passing (go vet)
  • Test cases are added for new code paths
  • Commits are signed-off and cryptographically signed

How to test this:

  1. Verify help output includes the new flag and environment variable:
    go run ./cmd/nvidia-ctk cdi generate --help | grep -C 1 no-ipc-sockets
  2. Run unit tests for CDI spec generation:
    go test -v ./cmd/nvidia-ctk/cdi/generate/...
  3. Run unit tests for the IPC discoverer feature flag:
    go test -v -run TestNewIPCDiscoverer ./pkg/nvcdi/...

Demonstration & Proof:

Terminal Recording (Video Demo)

CDI No IPC Sockets Video Demonstration

Verification Screenshot

Verification Test Results

@copy-pr-bot

copy-pr-bot Bot commented Sep 13, 2026

Copy link
Copy Markdown

This pull request requires additional validation before any workflows can run on NVIDIA's runners.

Pull request vetters can view their responsibilities here.

Contributors can view more details about this message here.

When running containers under sandboxed runtimes like gVisor (runsc) or
Kata Containers, bind mounting host IPC domain sockets (such as
/run/nvidia-persistenced/socket, /run/nvidia-fabricmanager/socket, or
/tmp/nvidia-mps) causes container creation to fail with ENXIO because
sandboxed gofers open existing mount points with O_CREAT.

Furthermore, multi-tenant and secure container environments often prohibit
sharing host IPC sockets into untrusted containers.

While nvcdi already implemented FeatureDisableIPCDiscoverer, it was never
exposed as a flag or environment variable in nvidia-ctk cdi generate or
systemd refresh services.

This commit exposes:
- CLI flag: --no-ipc-sockets (aliases: --disable-ipc-sockets, --disable-ipc-discoverer)
- Environment variable: NVIDIA_CTK_CDI_GENERATE_NO_IPC_SOCKETS
- Configuration option documented in nvidia-cdi-refresh.env
- Comprehensive unit tests covering flag validation, spec generation, and
  nvcdilib discoverer toggling.

Fixes NVIDIA#2034

Signed-off-by: Prakash Meena <pcmhacker511@gmail.com>
@Pcmhacker-piro
Pcmhacker-piro force-pushed the fix/cdi-generate-no-ipc-sockets branch from 97638d1 to 491d214 Compare September 13, 2026 08:17
@rajatchopra

Copy link
Copy Markdown
Contributor

The PR looks good, but the argument to fix gvisor to properly handle IPC files holds true. What about the use case where one needs GPU utility capabilities?

@Pcmhacker-piro

Copy link
Copy Markdown
Author

Hi @rajatchopra,

Thanks for the review!

Regarding the utility capability:

  1. Standard GPU utility capabilities (nvidia-smi, NVML queries, device telemetry) rely on the driver libraries (libnvidia-ml.so), binaries (nvidia-smi), and kernel device nodes (/dev/nvidiactl, /dev/nvidia*), which are discovered and mounted via NewDriverLibraryDiscoverer and newDriverBinariesDiscoverer. These remain fully functional with --no-ipc-sockets.
  2. The IPC discoverer (NewIPCDiscoverer) specifically targets host daemon sockets: /run/nvidia-persistenced/socket, /run/nvidia-fabricmanager/socket, and /tmp/nvidia-mps. In sandboxed container runtimes (such as gVisor/runsc or Kata Containers), bind mounting host IPC domain sockets fails due to gofer/VFS filesystem semantics (e.g., open(..., O_CREAT) yielding ENXIO), and multi-tenant sandboxes typically disallow host IPC sharing by design.
  3. This flag is strictly opt-in (default behavior retains IPC socket discovery), allowing users running sandboxed or multi-tenant workloads to generate clean CDI specs without breaking standard runtime environments where host IPC sockets are desired.

Also, I have merged the latest main branch and resolved the conflict in generate_test.go. All unit tests are passing!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants