From e2ff8c32df6d7d0fa9bd92c5b6346a5fafa2c504 Mon Sep 17 00:00:00 2001 From: Felix Stubner Date: Wed, 7 Oct 2026 23:46:14 +0100 Subject: [PATCH 1/2] Run the Linux sandbox's command as the user, with no capabilities The contained process ran as root in a user namespace that maps only the user's own id. A tool that unpacks an archive as root gives each file the owner the archive records, and any owner the namespace does not hold is EINVAL. Measured on Linux 6.18 in a Debian 13 container, a contained `npm install sqlite3@6.0.1` stopped at "prebuild-install warn install EINVAL: invalid argument, lchown", because the archive of its prebuilt binary is owned by uid 1001, and the fallback build from source failed as well. The supervisor's user namespace now maps the user to itself, so the command runs with the user's own uid and gid. A process that is not root keeps no capability across exec, so the supervisor carries its own across its exec as ambient capabilities. On the thread that forks the command it then clears its inheritable set, which clears the ambient set with it, and empties the bounding set. The command now holds no capability at all, where it held every one but the four the supervisor dropped. A user who runs nvx as root still runs the command as root, and it holds none there either. The supervisor sets the namespace's unprivileged port floor to 0, so a contained server can still listen below 1024, as it could as root. The same install now exits 0 and a contained node loads the module. The new test fails on the old code on its uid and capability sets, both as an ordinary user and as root. The privileged CI set passes as root, and as an ordinary user apart from the test that needs root to lower the inotify limit. network.mode loopback, --expose to port 80 inside, and bun and pnpm installs were also run contained in the same container. --- CHANGELOG.md | 15 +++ docs/enforcement-matrix.md | 7 +- internal/nvx/sandbox_dotenv_linux.go | 14 ++- internal/nvx/sandbox_git_readonly_linux.go | 11 +- internal/nvx/sandbox_landlock_linux.go | 10 ++ internal/nvx/sandbox_native_linux.go | 5 +- internal/nvx/sandbox_network_linux.go | 15 +++ .../nvx/sandbox_proc_private_linux_test.go | 4 +- internal/nvx/sandbox_reap_linux.go | 17 ++- internal/nvx/sandbox_userns_linux.go | 80 +++++++++++++ internal/nvx/sandbox_userns_linux_test.go | 105 ++++++++++++++++++ site/src/content/docs/docs/limitations.md | 6 + 12 files changed, 268 insertions(+), 21 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index ab189d06..f3c70bd8 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -726,6 +726,21 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 junction is compared by where the junction leads, and a path nvx cannot resolve counts as outside the home. +* **On Linux a contained install can unpack a prebuilt binary that another + user owns in its archive.** The contained process ran as root in a user + namespace that holds only your own user id. A tool that unpacks an archive + as root gives each file the owner the archive records, and an owner the + namespace does not hold failed with `EINVAL`. Measured 2026-10-07 in a + Debian 13 container, a contained `npm install sqlite3@6.0.1` stopped at + `prebuild-install warn install EINVAL: invalid argument, lchown`, because the + archive of its prebuilt binary is owned by uid 1001, and its fallback build + from source failed too. The contained process now runs as you, with your own + user and group ids, and starts with no capabilities. The same install exits + 0, and a contained `node` loads the module. A contained server can still + listen on a port below 1024 in its own network namespace. If you run nvx as + root, a contained process still runs as root, so the archive still fails to + unpack. + ### Security * **A contained process can no longer type into your terminal.** nvx gives the diff --git a/docs/enforcement-matrix.md b/docs/enforcement-matrix.md index 1da9f5dd..0a8846f7 100644 --- a/docs/enforcement-matrix.md +++ b/docs/enforcement-matrix.md @@ -988,9 +988,10 @@ provider mounts the project as it is. file with mode 0000 over each one in its private mount namespace, before Landlock applies. A read or a write fails with `EACCES`, a `chmod` with `EROFS`, a rename with `EBUSY` and a hard link with `EXDEV`. The target runs - as root in its user namespace, so the supervisor drops `CAP_DAC_OVERRIDE` and - `CAP_DAC_READ_SEARCH` from it, which would read past the mode, and - `CAP_SYS_ADMIN`, which would unmount the mask. The target could otherwise make + as you in its user namespace and starts with no capabilities, so it holds + neither `CAP_DAC_OVERRIDE` nor `CAP_DAC_READ_SEARCH`, which would read past the + mode, nor `CAP_SYS_ADMIN`, which would unmount the mask. The supervisor also + drops all three from its bounding set. The target could otherwise make a user namespace of its own, where it holds `CAP_SYS_ADMIN` again and can build a mount namespace the run-time masks never reach. The supervisor stops that by writing 0 to `/proc/sys/user/max_user_namespaces` before Landlock keeps `/proc` diff --git a/internal/nvx/sandbox_dotenv_linux.go b/internal/nvx/sandbox_dotenv_linux.go index 5eb78d9d..ded35292 100644 --- a/internal/nvx/sandbox_dotenv_linux.go +++ b/internal/nvx/sandbox_dotenv_linux.go @@ -26,12 +26,14 @@ import ( // EXDEV, so the real file cannot be moved to a name nothing covers. // // Mode 0000 holds only against a process without CAP_DAC_OVERRIDE and -// CAP_DAC_READ_SEARCH. The target runs as root in the supervisor's user -// namespace, where it holds both over every file the user owns, the mask -// included. So they leave the bounding set with CAP_SYS_ADMIN, which would let -// the target unmount the mask. Without them the target reads and writes the -// user's files as the user does outside the sandbox. They go even when the -// launch finds no dotenv file, because watchDotenvFiles may cover one later. +// CAP_DAC_READ_SEARCH. Root in the supervisor's user namespace holds both over +// every file the user owns, the mask included, and the target ran as that root +// until 2026-10-07. So they leave the bounding set with CAP_SYS_ADMIN, which +// would let the target unmount the mask. The target runs as the user now and +// dropTargetCapabilities empties the bounding set, so these drops are a second +// layer. Without the capabilities the target reads and writes the user's files +// as the user does outside the sandbox. They go even when the launch finds no +// dotenv file, because watchDotenvFiles may cover one later. // // This covers the files present at launch. watchDotenvFiles covers those // created or replaced during the run. The walk stops after dotenvScanLimit diff --git a/internal/nvx/sandbox_git_readonly_linux.go b/internal/nvx/sandbox_git_readonly_linux.go index 1a9a76d6..8d52f12e 100644 --- a/internal/nvx/sandbox_git_readonly_linux.go +++ b/internal/nvx/sandbox_git_readonly_linux.go @@ -84,11 +84,12 @@ const ( // dropSysAdminFromBoundingSet removes CAP_SYS_ADMIN from what the target can // hold after exec. // -// The supervisor is root in its own user namespace, and the target runs as that -// root, so without this the target holds every capability there, CAP_SYS_ADMIN -// over the mount namespace that carries the read-only .git mount included. -// Landlock refuses mount, umount and remount to a restricted process. This does -// not rely on that covering every interface that can change a mount's +// The target ran as root in the supervisor's user namespace until 2026-10-07, +// and held every capability there, CAP_SYS_ADMIN over the mount namespace that +// carries the read-only .git mount included. It runs as the user now and +// dropTargetCapabilities empties the bounding set, so this drop is a second +// layer. Landlock refuses mount, umount and remount to a restricted process. +// This does not rely on that covering every interface that can change a mount's // attributes: without CAP_SYS_ADMIN the target cannot change any of them. // // The bounding set limits what exec grants and leaves this thread's own diff --git a/internal/nvx/sandbox_landlock_linux.go b/internal/nvx/sandbox_landlock_linux.go index 7b6e1a82..6dbde1be 100644 --- a/internal/nvx/sandbox_landlock_linux.go +++ b/internal/nvx/sandbox_landlock_linux.go @@ -451,6 +451,9 @@ func runLandlockExecChild(a supervisorExecArgs) int { return 1 } LogInfo("Linux loopback-only network namespace active") + if err := allowLowPortsInNamespace(); err != nil { + LogWarn("A contained server cannot listen on a port below 1024 in this run (%v).", err) + } } // The egress proxy runs in the parent, outside this namespace, because a @@ -584,6 +587,13 @@ func runLandlockExecChild(a supervisorExecArgs) int { LogError("Could not stop the sandbox typing into the terminal (fail-closed): %v", err) return 1 } + // Last, because `ip` and `iptables` above got their capabilities from this + // thread. Fail closed, or the target starts with this thread's ambient + // capabilities. + if err := dropTargetCapabilities(); err != nil { + LogError("Could not take the sandbox's capabilities away from the command (fail-closed): %v", err) + return 1 + } cmd := exec.Command(cmdPath, args...) cmd.Stdin = os.Stdin diff --git a/internal/nvx/sandbox_native_linux.go b/internal/nvx/sandbox_native_linux.go index 004ce195..9a6db3f7 100644 --- a/internal/nvx/sandbox_native_linux.go +++ b/internal/nvx/sandbox_native_linux.go @@ -175,8 +175,9 @@ func platformLaunchNative(config SandboxConfig, guestHome, workDir, cmdPath stri // still in the old namespace and one reached the public internet. Supplying the // flag at clone time puts the whole child process in the new namespace from // birth, which is deterministic and needs no thread pinning. - // CLONE_NEWUSER, with this user mapped to root inside it, is what makes the - // rest of these flags possible without privileges. + // CLONE_NEWUSER, which grants the capabilities the new namespace needs (see + // supervisorSysProcAttr for how they last past exec), is what makes the rest + // of these flags possible without privileges. // // CLONE_NEWPID and CLONE_NEWNET both require CAP_SYS_ADMIN in the current // user namespace. Without CLONE_NEWUSER an ordinary user has none, so the diff --git a/internal/nvx/sandbox_network_linux.go b/internal/nvx/sandbox_network_linux.go index 1e58d112..0e008c11 100644 --- a/internal/nvx/sandbox_network_linux.go +++ b/internal/nvx/sandbox_network_linux.go @@ -4,6 +4,7 @@ package nvx import ( "fmt" + "os" "os/exec" "strings" ) @@ -34,6 +35,20 @@ func bringUpLoopback() error { return nil } +// unprivilegedPortStartPath is the lowest port a process without +// CAP_NET_BIND_SERVICE may bind in the caller's network namespace. It covers +// IPv6 as well, and a new namespace starts at 1024. +const unprivilegedPortStartPath = "/proc/sys/net/ipv4/ip_unprivileged_port_start" + +// allowLowPortsInNamespace lets the target listen on any port in its own network +// namespace. As root in its user namespace it could, and running as the user +// it could not listen below 1024. The namespace holds only its own loopback, +// so a low port there reaches nothing a high one does not. Docker sets the same +// value for its containers, 0 as read from one on 2026-10-07. +func allowLowPortsInNamespace() error { + return os.WriteFile(unprivilegedPortStartPath, []byte("0"), 0) +} + // networkModeRequiresNamespace reports whether mode needs a loopback-only // network namespace. // diff --git a/internal/nvx/sandbox_proc_private_linux_test.go b/internal/nvx/sandbox_proc_private_linux_test.go index 0600688b..148c6483 100644 --- a/internal/nvx/sandbox_proc_private_linux_test.go +++ b/internal/nvx/sandbox_proc_private_linux_test.go @@ -84,8 +84,8 @@ func TestContainedProcessReadsItsOwnProcAndNotTheHosts(t *testing.T) { // Creating the namespaces and being allowed to USE them are different // questions, and requireNamespaceSupport above only answers the first. // - // supervisorSysProcAttr maps this user to root inside a new user namespace, so - // the child should hold CAP_SYS_ADMIN there and unshare(CLONE_NEWNS) should + // supervisorSysProcAttr gives the child every capability in a new user + // namespace, so it should hold CAP_SYS_ADMIN and unshare(CLONE_NEWNS) should // succeed. Ubuntu 24.04 hardens unprivileged user namespaces through AppArmor // and strips it: the clone succeeds, the unshare comes back EPERM, and every // assertion below then reads as a product failure caused by a distribution diff --git a/internal/nvx/sandbox_reap_linux.go b/internal/nvx/sandbox_reap_linux.go index 4d9b9c77..7c4999c3 100644 --- a/internal/nvx/sandbox_reap_linux.go +++ b/internal/nvx/sandbox_reap_linux.go @@ -40,7 +40,16 @@ func supervisorCloneFlags(networkMode string) uintptr { // supervisorSysProcAttr is how the supervisor is actually launched: the clone // flags above plus the user namespace that makes them possible for an ordinary -// user, with this user mapped to root inside it. +// user, with this user mapped to itself inside it. +// +// Not to root. The target runs as the supervisor's user, and tar libraries +// restore an archive's owners only when getuid() is 0. With one id mapped, every +// other owner is EINVAL. Measured 2026-10-07 on Linux 6.18 with root mapped, a +// contained `npm install sqlite3` failed on `lchown ... EINVAL`, because its +// prebuilt binary's archive is owned by 1001. A process that is not root in +// the namespace keeps no capability across exec, so the supervisor carries +// its own across as ambient capabilities (supervisorCapabilities) and takes +// them away again before the target starts (dropTargetCapabilities). // // Pdeathsig SIGKILL is what ties the supervisor's life to nvx's. SIGKILL reaches // PID 1 of a namespace from outside it, and PID 1 dying takes the namespace's @@ -56,15 +65,17 @@ func supervisorCloneFlags(networkMode string) uintptr { // namespace unaided, got EPERM, and skipped. The behaviour it guards was running // fine by then; nothing said so. func supervisorSysProcAttr(networkMode string) *syscall.SysProcAttr { + uid, gid := syscall.Getuid(), syscall.Getgid() return &syscall.SysProcAttr{ Pdeathsig: syscall.SIGKILL, Cloneflags: syscall.CLONE_NEWUSER | supervisorCloneFlags(networkMode), UidMappings: []syscall.SysProcIDMap{ - {ContainerID: 0, HostID: syscall.Getuid(), Size: 1}, + {ContainerID: uid, HostID: uid, Size: 1}, }, GidMappings: []syscall.SysProcIDMap{ - {ContainerID: 0, HostID: syscall.Getgid(), Size: 1}, + {ContainerID: gid, HostID: gid, Size: 1}, }, + AmbientCaps: supervisorCapabilities(), } } diff --git a/internal/nvx/sandbox_userns_linux.go b/internal/nvx/sandbox_userns_linux.go index 7e2274d1..cdb018af 100644 --- a/internal/nvx/sandbox_userns_linux.go +++ b/internal/nvx/sandbox_userns_linux.go @@ -5,8 +5,88 @@ package nvx import ( "fmt" "os" + "syscall" + "unsafe" ) +// prCapBSetRead is PR_CAPBSET_READ from linux/prctl.h. +const prCapBSetRead = 23 + +// supervisorCapabilities lists every capability the running kernel knows. The +// supervisor carries them across its own exec as ambient capabilities, because +// it is not root in its user namespace and exec would otherwise clear them (see +// supervisorSysProcAttr). Root held all of them, and the setup uses several: +// CAP_SYS_ADMIN for the mounts, CAP_SYS_CHROOT with it for the .env watcher's +// setns, CAP_SYS_RESOURCE for the user-namespace limit, CAP_SETPCAP for the +// bounding-set drops and CAP_NET_ADMIN for `ip` and `iptables`, which get them +// as ambient capabilities too. +// +// PR_CAPBSET_READ fails with EINVAL past the kernel's last capability. Raising +// one the kernel does not know would fail the launch. +func supervisorCapabilities() []uintptr { + var caps []uintptr + for c := uintptr(0); c < 64; c++ { + if _, _, errno := syscall.RawSyscall6(prctlSyscall(), prCapBSetRead, c, 0, 0, 0, 0); errno == syscall.EINVAL { + break + } + caps = append(caps, c) + } + return caps +} + +// linuxCapabilityVersion3 is _LINUX_CAPABILITY_VERSION_3, which takes two data +// structs, one per 32 capabilities. +const linuxCapabilityVersion3 = 0x20080522 + +// capUserHeader and capUserData are struct __user_cap_header_struct and struct +// __user_cap_data_struct from linux/capability.h. +type capUserHeader struct { + version uint32 + pid int32 +} + +type capUserData struct { + effective uint32 + permitted uint32 + inheritable uint32 +} + +// dropTargetCapabilities leaves the target no capability to gain at exec. Call +// it on the thread that forks the target, after the last step that needs a +// capability to pass to a child. +// +// The target runs as the user, so exec grants it nothing by itself. Two ways +// remain, and this closes both. The ambient set crosses exec to a process that +// is not root, so the inheritable set is cleared, and the kernel lowers the +// ambient set with it. The bounding set limits what exec grants to root and to +// a file with capabilities, so it is emptied. That covers a user who runs nvx +// as root, who is root in the namespace too, and a target that is itself a file +// with capabilities, which no_new_privs would let keep what this thread holds. +// Measured 2026-10-07 without this, the target held all 41 capabilities, the +// four the bounding set had dropped included. +// +// Per thread, like Landlock. The .env watcher's thread keeps what it needs for +// its setns and mounts. +func dropTargetCapabilities() error { + hdr := capUserHeader{version: linuxCapabilityVersion3} + var data [2]capUserData + if _, _, errno := syscall.RawSyscall(syscall.SYS_CAPGET, + uintptr(unsafe.Pointer(&hdr)), uintptr(unsafe.Pointer(&data[0])), 0); errno != 0 { + return fmt.Errorf("capget: %w", errno) + } + data[0].inheritable, data[1].inheritable = 0, 0 + if _, _, errno := syscall.RawSyscall(syscall.SYS_CAPSET, + uintptr(unsafe.Pointer(&hdr)), uintptr(unsafe.Pointer(&data[0])), 0); errno != 0 { + return fmt.Errorf("capset: %w", errno) + } + for _, c := range supervisorCapabilities() { + if err := dropFromBoundingSet(c); err != nil { + return fmt.Errorf("drop capability %d from the bounding set: %w", c, err) + } + } + return nil +} + // maxUserNamespacesPath is the per-user-namespace ceiling on how many user // namespaces may be created beneath this one (the UCOUNT_USER_NAMESPACES limit, // linux/user_namespace.h). Writing 0 stops the contained process creating one. diff --git a/internal/nvx/sandbox_userns_linux_test.go b/internal/nvx/sandbox_userns_linux_test.go index 4abddf71..122ba569 100644 --- a/internal/nvx/sandbox_userns_linux_test.go +++ b/internal/nvx/sandbox_userns_linux_test.go @@ -7,15 +7,120 @@ import ( "errors" "fmt" "io/fs" + "net" "os" "os/exec" "path/filepath" + "strconv" "strings" "syscall" "testing" "time" ) +// The contained process runs as the user who started nvx and holds no +// capabilities, and it can still listen on a port below 1024 in its own network +// namespace, as it could when it ran as root there. +// +// Until 2026-10-07 the supervisor mapped the user to root and the target ran as +// that root, holding every capability but the four the supervisor dropped. A +// tar library restores an archive's owners when it runs as root, and with one +// id mapped every other owner is EINVAL. That is how a contained `npm install +// sqlite3` failed, on its prebuilt binary's archive owned by 1001. +// +// The supervisor is the real one, with this test binary as its target. CI runs +// it again as root in the privileged step, where the user is root in the +// namespace too, and the capability sets still have to be empty. +func TestContainedProcessRunsAsTheUserWithNoCapabilities(t *testing.T) { + const name = "TestContainedProcessRunsAsTheUserWithNoCapabilities" + switch { + case os.Getenv("NVX_TEST_IDENTITY_TARGET") == "1": + runIdentityProbe() + return + case os.Getenv("NVX_TEST_IDENTITY_SUPERVISOR") == "1": + exe, err := os.Executable() + if err != nil { + fmt.Printf("setup_failed=%v\n", err) + os.Exit(1) + } + // Inherited by the target, which is how it knows to probe. + _ = os.Setenv("NVX_TEST_IDENTITY_TARGET", "1") + os.Exit(runLandlockExecChild(supervisorExecArgs{ + GuestHome: os.Getenv("NVX_TEST_GUEST"), + WorkDir: os.Getenv("NVX_TEST_WORK"), + NvxHome: os.Getenv("NVX_TEST_NVXHOME"), + NetworkMode: "proxy", + ReadExecRoots: []string{filepath.Dir(exe)}, + CmdPath: exe, + CmdArgs: []string{"-test.run=^" + name + "$"}, + })) + } + + if _, err := exec.LookPath("ip"); err != nil { + t.Skip("iproute2 not installed; bringUpLoopback needs `ip`") + } + if fd, err := landlockCreateRuleset(landlockHandledAccess()); err != nil { + t.Skipf("landlock unavailable on this kernel: %v", err) + } else { + _ = syscall.Close(fd) + } + attr := supervisorSysProcAttr("proxy") + requireLoopbackControl(t, attr) + + cmd := exec.Command(os.Args[0], "-test.run=^"+name+"$") + cmd.Env = append(os.Environ(), + "NVX_TEST_IDENTITY_SUPERVISOR=1", + "NVX_TEST_GUEST="+tempDir(t), + "NVX_TEST_WORK="+tempDir(t), + "NVX_TEST_NVXHOME="+tempDir(t), + ) + cmd.SysProcAttr = attr + out, err := cmd.CombinedOutput() + skipWithoutMountNamespace(t, out) + if err != nil { + t.Fatalf("supervisor failed: %v\noutput:\n%s", err, out) + } + got := parseProbeResults(string(out)) + + const none = "0000000000000000" + for _, want := range []struct{ key, val string }{ + {"uid", strconv.Itoa(os.Getuid())}, + {"gid", strconv.Itoa(os.Getgid())}, + {"CapInh", none}, + {"CapPrm", none}, + {"CapEff", none}, + {"CapBnd", none}, + {"CapAmb", none}, + {"low_port", "ok"}, + } { + if got[want.key] != want.val { + t.Errorf("%s = %q, want %q\noutput:\n%s", want.key, got[want.key], want.val, out) + } + } +} + +// runIdentityProbe runs as the contained target. It reports its ids, its +// capability sets and whether it can listen on port 80. +func runIdentityProbe() { + fmt.Printf("uid=%d\ngid=%d\n", os.Getuid(), os.Getgid()) + status, err := os.ReadFile("/proc/self/status") + if err != nil { + fmt.Printf("status_error=%v\n", err) + } + for _, line := range strings.Split(string(status), "\n") { + if k, v, ok := strings.Cut(line, ":"); ok && strings.HasPrefix(k, "Cap") { + fmt.Printf("%s=%s\n", k, strings.TrimSpace(v)) + } + } + ln, err := net.Listen("tcp", "127.0.0.1:80") + if err != nil { + fmt.Printf("low_port=%v\n", err) + return + } + _ = ln.Close() + fmt.Println("low_port=ok") +} + // A contained process cannot create a user namespace of its own, so it cannot // use a nested user+mount namespace to escape the run-time .env watcher. // diff --git a/site/src/content/docs/docs/limitations.md b/site/src/content/docs/docs/limitations.md index 0d0aa4dd..76ccbf0c 100644 --- a/site/src/content/docs/docs/limitations.md +++ b/site/src/content/docs/docs/limitations.md @@ -149,6 +149,12 @@ and the evidence and measurements for each platform are in the only the directories it is granted, so host sockets such as Docker's are absent. A socket placed in the project directory, or in a directory added with `allow_read_exec`, can still be connected to. +- **Run as root, nvx cannot install every prebuilt binary contained.** A + contained process runs as the user who started nvx, so for root it is root, + in a user namespace that holds no other user. A tool that unpacks an archive + as root gives each file the owner the archive records, and any other owner + fails with `EINVAL`. sqlite3's prebuilt binary is one such archive. Run nvx as + an ordinary user. - **On Ubuntu 23.10 and later, the sandbox may refuse to start.** Ubuntu restricts the user namespaces it is built on, through `kernel.apparmor_restrict_unprivileged_userns`. Contained commands then fail From 6f2d124bf4415ab2e7a02b0c36b50755397f61dc Mon Sep 17 00:00:00 2001 From: Felix Stubner Date: Wed, 7 Oct 2026 23:46:24 +0100 Subject: [PATCH 2/2] Say when a browser a contained install downloaded is deleted with the sandbox puppeteer's postinstall and `playwright install` keep their browsers under the home directory. Inside the sandbox that is the guest home, which nvx deletes when the command ends, so the browser went with it and the install still exited 0. Measured in a Debian 13 container with storage.googleapis.com allowed, the puppeteer cache in the guest home reached 856 MB during a contained `npm install puppeteer`, nvx said nothing, and no browser was left. Before it deletes an ephemeral guest home, nvx now looks in the folders puppeteer and Playwright use. When one holds 1 MiB or more it names the tool and prints the command that installs the browser where that tool looks for it, such as `nvx --no-sandbox npx puppeteer browsers install chrome`. A blocked download leaves only small records there, 103 bytes for Playwright, and says nothing. On Windows Playwright's folder is under the AppContainer package's own LOCALAPPDATA, which a contained node reported as AppData\Local\Packages\nvx.sandbox.\AC under the guest home. PUPPETEER_CACHE_DIR and PLAYWRIGHT_BROWSERS_PATH join the variables nvx names when it removes them from a contained command's environment. Checked with real downloads on Linux, puppeteer 25.12.0 and Playwright 1.64.0, and by writing a browser-sized file into the guest home on Linux and on Windows. Following the printed puppeteer command installed Chrome where puppeteer's own code then found it. --- CHANGELOG.md | 13 ++ internal/nvx/sandbox_browser_downloads.go | 111 ++++++++++++++++++ .../nvx/sandbox_browser_downloads_test.go | 63 ++++++++++ internal/nvx/sandbox_env_scrub.go | 4 + internal/nvx/sandbox_env_scrub_test.go | 7 +- internal/nvx/sandbox_native.go | 2 + site/src/content/docs/docs/limitations.md | 7 ++ 7 files changed, 205 insertions(+), 2 deletions(-) create mode 100644 internal/nvx/sandbox_browser_downloads.go create mode 100644 internal/nvx/sandbox_browser_downloads_test.go diff --git a/CHANGELOG.md b/CHANGELOG.md index f3c70bd8..28d0ba0c 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -741,6 +741,19 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 root, a contained process still runs as root, so the archive still fails to unpack. +* **nvx says when a browser that a contained install downloaded is deleted + with the sandbox.** puppeteer's postinstall and `playwright install` keep + their browsers under the home directory, and a contained command's home is + the sandbox's own, which nvx deletes when the command ends. The install + exited 0 and nothing said the browser was gone. Measured 2026-10-07 in a + Debian 13 container with `storage.googleapis.com` allowed, the puppeteer + cache in the sandbox's home reached 856 MB during a contained `npm install + puppeteer`, and no browser was left afterwards. nvx now warns when this + happens and prints the command that installs the browser where the tool + looks for it, such as `nvx --no-sandbox npx puppeteer browsers install + chrome`. It also names `PUPPETEER_CACHE_DIR` and `PLAYWRIGHT_BROWSERS_PATH` + when it removes them from a contained command's environment. + ### Security * **A contained process can no longer type into your terminal.** nvx gives the diff --git a/internal/nvx/sandbox_browser_downloads.go b/internal/nvx/sandbox_browser_downloads.go new file mode 100644 index 00000000..b9489744 --- /dev/null +++ b/internal/nvx/sandbox_browser_downloads.go @@ -0,0 +1,111 @@ +package nvx + +import ( + "errors" + "io/fs" + "os" + "path/filepath" + "runtime" + "strings" +) + +// A browser a contained install downloads goes with the guest home. +// +// puppeteer's postinstall and `playwright install` keep their browsers in a +// cache under the home directory, and inside the sandbox that is the guest +// home, which nvx deletes when the command ends. The install still exits 0, so +// the first sign was puppeteer or Playwright failing to find the browser later. +// Measured 2026-10-07 in a Debian 13 container with storage.googleapis.com +// allowed, the cache in the guest home reached 856 MB during a contained `npm +// install puppeteer`. npm exited 0, nvx said nothing, and no browser was left +// afterwards. + +// browserCache is a folder a browser installer writes under the home, and the +// command that installs the browser where the tool looks for it. +type browserCache struct { + tool string + // dir is relative to the home and slash-separated. One element may be *, + // for every folder at that level. + dir string + install string +} + +// browserCaches are the folders for goos. puppeteer uses ~/.cache/puppeteer +// everywhere. Playwright uses the platform's cache folder, which the sandbox +// points into the guest home through XDG_CACHE_HOME on Linux. Windows points an +// AppContainer's LOCALAPPDATA at the package's own folder. Measured 2026-10-07, +// a contained node saw it as AppData\Local\Packages\nvx.sandbox.\AC under +// the guest home. +func browserCaches(goos string) []browserCache { + playwright := ".cache/ms-playwright" + switch goos { + case "windows": + playwright = "AppData/Local/Packages/*/AC/ms-playwright" + case "darwin": + playwright = "Library/Caches/ms-playwright" + } + return []browserCache{ + {tool: "puppeteer", dir: ".cache/puppeteer", install: "npx puppeteer browsers install chrome"}, + {tool: "Playwright", dir: playwright, install: "npx playwright install"}, + } +} + +// dirsUnder returns the folders under home that c.dir names. Not +// filepath.Glob, which would read a * or [ in home itself as a pattern. +func (c browserCache) dirsUnder(home string) []string { + parent, rest, wild := strings.Cut(c.dir, "/*/") + if !wild { + return []string{filepath.Join(home, filepath.FromSlash(c.dir))} + } + parentDir := filepath.Join(home, filepath.FromSlash(parent)) + entries, _ := os.ReadDir(parentDir) + var dirs []string + for _, e := range entries { + dirs = append(dirs, filepath.Join(parentDir, e.Name(), filepath.FromSlash(rest))) + } + return dirs +} + +// browserDownloadFloor is how much a cache folder must hold to count as a +// download. A blocked download leaves folders and small records behind, such as +// Playwright's note of the project in .links. A browser is hundreds of MB. +const browserDownloadFloor = 1 << 20 + +// warnLostBrowserDownloads says when a browser landed in a guest home that nvx +// is about to delete, and how to install it where the tool looks for it. +func warnLostBrowserDownloads(guestHome string) { + for _, c := range browserCaches(runtime.GOOS) { + for _, dir := range c.dirsUnder(guestHome) { + if !holdsAtLeast(dir, browserDownloadFloor) { + continue + } + LogWarn("%s downloaded a browser into the sandbox's home, which nvx deletes when the command ends.", c.tool) + LogInfo("To install it where %s looks for it, run: %s --no-sandbox %s", c.tool, installedNvxHint(), c.install) + break + } + } +} + +// errEnough stops the walk in holdsAtLeast. +var errEnough = errors.New("enough") + +// holdsAtLeast reports whether the regular files under dir add up to at least +// n bytes. It stops as soon as they do, and does not follow links. +func holdsAtLeast(dir string, n int64) bool { + var total int64 + err := filepath.WalkDir(dir, func(_ string, d fs.DirEntry, err error) error { + if err != nil || !d.Type().IsRegular() { + return nil + } + info, err := d.Info() + if err != nil { + return nil + } + total += info.Size() + if total >= n { + return errEnough + } + return nil + }) + return errors.Is(err, errEnough) +} diff --git a/internal/nvx/sandbox_browser_downloads_test.go b/internal/nvx/sandbox_browser_downloads_test.go new file mode 100644 index 00000000..b6e4058b --- /dev/null +++ b/internal/nvx/sandbox_browser_downloads_test.go @@ -0,0 +1,63 @@ +package nvx + +import ( + "os" + "path/filepath" + "runtime" + "strings" + "testing" +) + +// A browser in a guest home that nvx is about to delete is named, with the +// command that installs it where the tool looks for it. A blocked download +// leaves only small records behind, and those say nothing. +func TestBrowserDownloadInTheGuestHomeIsNamed(t *testing.T) { + var playwright string + for _, c := range browserCaches(runtime.GOOS) { + if c.tool == "Playwright" { + // On Windows the folder is under the AppContainer package's own. + playwright = filepath.FromSlash(strings.Replace(c.dir, "*", "nvx.sandbox.0123456789abcdef", 1)) + } + } + writeSized := func(path string, size int64) { + t.Helper() + if err := os.MkdirAll(filepath.Dir(path), 0o755); err != nil { + t.Fatal(err) + } + if err := os.WriteFile(path, nil, 0o644); err != nil { + t.Fatal(err) + } + if err := os.Truncate(path, size); err != nil { + t.Fatal(err) + } + } + + // A puppeteer download, and what Playwright leaves after a blocked one. + home := tempDir(t) + writeSized(filepath.Join(home, ".cache", "puppeteer", "chrome", "linux-154.0.8037.57", "chrome"), 2<<20) + writeSized(filepath.Join(home, playwright, ".links", "0123abcd"), 64) + out := captureStderrHere(t, func() { warnLostBrowserDownloads(home) }) + for _, want := range []string{ + "puppeteer downloaded a browser into the sandbox's home, which nvx deletes when the command ends.", + "--no-sandbox npx puppeteer browsers install chrome", + } { + if !strings.Contains(out, want) { + t.Errorf("the warning does not say %q:\n%s", want, out) + } + } + if strings.Contains(out, "Playwright") { + t.Errorf("a blocked Playwright download was reported as a browser:\n%s", out) + } + + // A Playwright download, where this platform's sandbox puts it. + home = tempDir(t) + writeSized(filepath.Join(home, playwright, "chromium-1243", "chrome"), 2<<20) + out = captureStderrHere(t, func() { warnLostBrowserDownloads(home) }) + if !strings.Contains(out, "Playwright downloaded a browser") || !strings.Contains(out, "--no-sandbox npx playwright install") { + t.Errorf("a Playwright download was not named:\n%s", out) + } + + if out := captureStderrHere(t, func() { warnLostBrowserDownloads(tempDir(t)) }); out != "" { + t.Errorf("a guest home with no browser in it printed:\n%s", out) + } +} diff --git a/internal/nvx/sandbox_env_scrub.go b/internal/nvx/sandbox_env_scrub.go index eb87c9ba..9aa6220e 100644 --- a/internal/nvx/sandbox_env_scrub.go +++ b/internal/nvx/sandbox_env_scrub.go @@ -61,6 +61,10 @@ var notableEnvKeys = map[string]bool{ "DEBUG": true, "FORCE_COLOR": true, "NO_COLOR": true, + // Where puppeteer and Playwright put a browser. Without them it goes into + // the guest home and is deleted with it (see warnLostBrowserDownloads). + "PUPPETEER_CACHE_DIR": true, + "PLAYWRIGHT_BROWSERS_PATH": true, } // notableEnvPrefixes catch families rather than single names. diff --git a/internal/nvx/sandbox_env_scrub_test.go b/internal/nvx/sandbox_env_scrub_test.go index 9602cb01..7f9e3b39 100644 --- a/internal/nvx/sandbox_env_scrub_test.go +++ b/internal/nvx/sandbox_env_scrub_test.go @@ -149,10 +149,13 @@ func TestACredentialNamedBySuffixIsRefused(t *testing.T) { // furniture (COMPUTERNAME, PROCESSOR_LEVEL). Naming them all on every run would // be noise nobody reads, which is its own way of saying nothing. func TestOnlyBehaviourChangingVariablesAreWorthPrinting(t *testing.T) { - dropped := []string{"COMPUTERNAME", "PROCESSOR_LEVEL", "CI", "ONEDRIVE", "NODE_ENV", "npm_config_yes"} + dropped := []string{"COMPUTERNAME", "PROCESSOR_LEVEL", "CI", "ONEDRIVE", "NODE_ENV", "npm_config_yes", + "PUPPETEER_CACHE_DIR", "PLAYWRIGHT_BROWSERS_PATH"} notable := notableDropped(dropped) - for _, want := range []string{"CI", "NODE_ENV"} { + // Without the last two a browser download lands in the guest home and is + // deleted with it. + for _, want := range []string{"CI", "NODE_ENV", "PUPPETEER_CACHE_DIR", "PLAYWRIGHT_BROWSERS_PATH"} { if !containsString(notable, want) { t.Errorf("%q changes how tools behave and would not be reported: %v", want, notable) } diff --git a/internal/nvx/sandbox_native.go b/internal/nvx/sandbox_native.go index 0a60c91f..490c80ee 100644 --- a/internal/nvx/sandbox_native.go +++ b/internal/nvx/sandbox_native.go @@ -100,6 +100,8 @@ func runNativeSandbox(config SandboxConfig, policy Policy, egress *EgressProxy, LogInfo("The sandbox's debug logs were copied out before its home was removed: %s", dest) } }() + // A browser an installer put in the home goes with it, on success too. + defer warnLostBrowserDownloads(guestHome) } // Platforms that cut the sandboxed process off from the parent's loopback diff --git a/site/src/content/docs/docs/limitations.md b/site/src/content/docs/docs/limitations.md index 76ccbf0c..d0f8c946 100644 --- a/site/src/content/docs/docs/limitations.md +++ b/site/src/content/docs/docs/limitations.md @@ -21,6 +21,13 @@ and the evidence and measurements for each platform are in the - **A `.env` inside the project is readable by a contained install.** The project directory has to be readable for the install to work, and `.env` lives in it. Scrubbing covers environment *variables* only. +- **A browser that a contained install downloads is deleted with the + sandbox.** puppeteer and Playwright keep their browsers in a cache under the + home directory, and a contained command gets a home of its own, which nvx + deletes when the command ends. nvx warns when this happens. Install the + browser outside the sandbox with + `nvx --no-sandbox npx puppeteer browsers install chrome` or + `nvx --no-sandbox npx playwright install`. - **Only an `http://` upstream proxy is used.** An `https://` or `socks5://` value in `HTTPS_PROXY` is ignored with a warning, and contained connections are then made directly. Behind a proxy, a host nvx's own resolver cannot look