Install from the lockfile nvx checked, and take the sweep and the hang-up watch off the start and end of a command

CHANGELOG.md

@@ -188,6 +188,61 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
   and Administrators full control (and you, for your profile). Otherwise it says
   why and changes nothing. `nvx doctor` points at `nvx setup` for this.
 
+* **An npm install now installs the lockfile nvx checked.** An install that
+  brings in new packages runs npm twice. The first run resolves the tree and
+  nvx checks what it wrote. The second run used to resolve the tree again from
+  the registry, so a version published between the two runs was installed
+  without a check, and it repeated the first run's work. It now starts from the
+  lockfile the first run wrote, and npm asks the registry for nothing but the
+  tarballs. An install that names packages first makes each name exact from
+  that lockfile, so `foo` and `foo@latest` become `foo@1.2.3`, because npm
+  asks the registry again about any package it is told to install. Your
+  `package.json` and `package-lock.json` come out as npm leaves them. Against a
+  fake registry that publishes new versions between the two runs, a bare
+  install and an install of a name, a tag and `-D` left both files byte for byte
+  as a plain npm run does, with npm 7.24.2, 8.19.4, 9.9.4, 10.9.3, 10.9.9 and
+  11.11.0. If npm does not write the lockfile itself, as with
+  `package-lock=false` in `.npmrc` or `--no-save`, nvx takes the one it put
+  there out again. `npm update`, `npm dedupe` and an install that names a
+  version range, an alias, a URL or a git source resolve again in the second
+  run, as before, and `--verbose` says why. pnpm, Yarn and Bun have no
+  resolving run and are unchanged. Measured 2026-10-08 as the median of 5
+  interleaved runs through the contained shim, on machines busy with other
+  work. The 321-package project with no lockfile took 88.2 s on Windows and
+  now takes 64.8 s. In a Linux container it took 89.8 s and now takes 61.4 s.
+  An install of one package did not change, 8.7 s and 8.8 s on Windows and
+  2.47 s and 2.39 s on Linux.
+
+* **A command no longer waits for nvx's housekeeping.** After a shimmed
+  command, nvx deletes what killed runs left behind, and the command did not
+  exit until it had finished. On a Windows machine with 638 package profiles,
+  the sweep of those alone took 83 to 91 ms a call, and `node -e 0` took 3.50 s
+  with one guest home of 12,000 files left behind, because it waited for the
+  home to be deleted (medians of 5 interleaved runs). Package profiles, rescued
+  logs and staged command copies only count once they are a week or two old,
+  so they are now looked at when the last full look is an hour old, and sooner
+  while one of them is working through a backlog. The sweep runs in the
+  background, and the command waits for it for 100 ms at most and leaves the
+  rest to the next command. One sweep runs at a time, under a lock that
+  Windows or Linux releases if the sweep dies. Profile deletions stay on the
+  command's own thread, so leaving never stops one halfway. On Windows
+  `node --version` through the shim took a median of 0.221 s and now takes
+  0.101 s, against 0.052 s for node alone (21 interleaved runs, stdin from
+  NUL), and `node -e 0` with the leftover home took 3.50 s and now takes
+  0.25 s. On Linux the same home took 0.46 s and now takes 0.18 s, and
+  `node --version` through the shim stayed at 0.018 s.
+
+* **On Windows, a command whose stdin is a pipe no longer lists every process
+  as it starts.** An agent harness or an MCP client leaves stdin as a pipe,
+  and nvx then watches for the program that started it to go away. Finding that
+  program meant a snapshot of every process on the machine, taken twice. One
+  snapshot took 19 to 27 ms (three runs of 40 calls). nvx now asks Windows for
+  the parent's process id, which took about 0.4 microseconds a call (three runs
+  of 2000), and falls back to the snapshot only if that fails. Measured as the
+  median of 21 interleaved runs of `node --version` through the shim, it took
+  0.300 s with stdin a pipe and 0.221 s with stdin from NUL before, and takes
+  0.103 s and 0.101 s now.
+
 ### Security
 
 * **Windows: in 0.7.0 a contained launch could, rarely, start the command

SECURITY.md

@@ -66,7 +66,12 @@ install and run scripts. Its defenses are layered:
    `--package-lock-only --ignore-scripts`, contained like the install, on a
    scratch copy of `package.json` and the lockfile. Every package in the tree
    it writes is checked, dependencies included, apart from those already
-   installed at the same version, which npm leaves alone. For `npm ci`, `npm rebuild` and
+   installed at the same version, which npm leaves alone. An `npm install`
+   then starts from the lockfile that resolver wrote, so it installs the tree
+   that was checked even if the registry changes in between. `npm update`,
+   `npm dedupe` and an install nvx cannot pin to that lockfile (one that names
+   a version range, an alias or a git source) run npm's resolver a second time
+   instead. For `npm ci`, `npm rebuild` and
    an `npm install` whose lockfile matches `package.json`, every lockfile entry
    for this platform is checked. A lockfile entry's `resolved` URL and `integrity` hash
    must match the registry's record for its name and version, and an entry that

internal/nvx/env.go

@@ -889,8 +889,9 @@ func runShim(cmdName string, args []string, nvxHome string) int {
 	// Housekeeping after the command, never before it: leftovers from processes
 	// that were killed cannot clean up after themselves, and waiting for someone
 	// to type `nvx cleanup` meant 91 guest homes on the machine this was found
-	// on. Bounded and silent -- see reclaimStaleSandboxes.
-	reclaimStaleSandboxes(nvxHome)
+	// on. Bounded and silent -- see reclaimStaleSandboxes. The command does not
+	// wait for it for long -- see reclaim.go.
+	reclaimAfterCommand(nvxHome)
 	return code
 }
 
@@ -967,13 +968,16 @@ func runShimTraced(trace *runTrace, cmdName string, args []string, nvxHome strin
 		toolName = tool
 	}
 
+	var resolved *resolvedInstall
 	switch strings.ToLower(pmCmd) {
 	case "npm", "yarn", "pnpm", "npx", "bun", "bunx":
+		resolved = &resolvedInstall{}
 		// Returning rather than exiting here is what lets runShim record the
 		// abort. A blocked or refused install is a run, and the one a later
 		// review most wants to find.
 		if code, reason, label := verifyBeforeRun(verifyRequest{
 			pmCmd: pmCmd, pmArgs: pmArgs, nvxHome: nvxHome, contain: contain,
+			resolved: resolved,
 			launch: SandboxConfig{
 				NvxHome:            nvxHome,
 				FilesystemProvider: opts.filesystemProvider,
@@ -990,6 +994,12 @@ func runShimTraced(trace *runTrace, cmdName string, args []string, nvxHome strin
 			return verifyExitCode(code, reason)
 		}
 	}
+	// An install that was resolved for the checks starts from the lockfile they
+	// ran on, and not from a second resolution that could differ. See adopt.
+	if run, finish := resolved.adopt(cmdName, args, pmCmd, pmArgs); finish != nil {
+		args = run
+		defer finish()
+	}
 	if cmdName == "npm" || cmdName == "yarn" || cmdName == "pnpm" {
 		if cwd, err := os.Getwd(); err == nil {
 			if root := findProjectRoot(cwd); root != "" {

internal/nvx/file_lock_unix.go

@@ -16,3 +16,16 @@ func lockFileExclusive(f *os.File) error {
 func unlockFile(f *os.File) error {
 	return syscall.Flock(int(f.Fd()), syscall.LOCK_UN)
 }
+
+// tryLockFileExclusive takes the lock lockFileExclusive does without waiting for
+// it. It reports false and no error when another file holds the lock.
+func tryLockFileExclusive(f *os.File) (bool, error) {
+	err := syscall.Flock(int(f.Fd()), syscall.LOCK_EX|syscall.LOCK_NB)
+	if err == nil {
+		return true, nil
+	}
+	if err == syscall.EWOULDBLOCK {
+		return false, nil
+	}
+	return false, err
+}

internal/nvx/file_lock_windows.go

@@ -34,3 +34,22 @@ func unlockFile(f *os.File) error {
 	}
 	return nil
 }
+
+const (
+	lockfileFailImmediately = 0x00000001
+	errorLockViolation      = 33
+)
+
+// tryLockFileExclusive takes the lock lockFileExclusive does without waiting for
+// it. It reports false and no error when another handle holds the lock.
+func tryLockFileExclusive(f *os.File) (bool, error) {
+	var ov syscall.Overlapped
+	r, _, err := procLockFileEx.Call(f.Fd(), lockfileExclusiveLock|lockfileFailImmediately, 0, 1, 0, uintptr(unsafe.Pointer(&ov)))
+	if r != 0 {
+		return true, nil
+	}
+	if errno, ok := err.(syscall.Errno); ok && errno == errorLockViolation {
+		return false, nil
+	}
+	return false, err
+}

internal/nvx/main_test.go

@@ -62,6 +62,10 @@ func TestMain(m *testing.M) {
 	// would otherwise print whatever an old setup left on the host running the
 	// suite.
 	reportSetupLeftoversFn = func(string) {}
+	// A command waits only briefly for the housekeeping sweep and leaves the rest
+	// to the next one. A test that ends while it is still running would see its
+	// temporary directory deleted under it, so the suite waits for all of it.
+	reclaimExitGrace = 0
 
 	code := m.Run()
 	cleanupProbeChildBinary()

internal/nvx/npm_fake_registry_test.go

@@ -0,0 +1,183 @@
+package nvx
+
+import (
+	"archive/tar"
+	"bytes"
+	"compress/gzip"
+	"crypto/sha1" // #nosec G505 -- the registry's shasum field is SHA-1
+	"crypto/sha512"
+	"encoding/base64"
+	"encoding/hex"
+	"encoding/json"
+	"fmt"
+	"net/http"
+	"net/http/httptest"
+	"strings"
+	"sync"
+	"testing"
+	"time"
+)
+
+// fakeNpmRegistry is a registry real npm can install from. It serves packuments
+// and tarballs for what a test publishes, and logs what was asked, so a test can
+// change what the registry holds between two runs of npm and see what each asked.
+type fakeNpmRegistry struct {
+	t   *testing.T
+	srv *httptest.Server
+
+	mu   sync.Mutex
+	pkgs map[string]*fakeNpmPackage
+	log  []fakeNpmRequest
+}
+
+type fakeNpmPackage struct {
+	versions map[string]fakeNpmVersion
+	tags     map[string]string
+}
+
+type fakeNpmVersion struct {
+	deps    map[string]string
+	scripts map[string]string
+	tgz     []byte
+}
+
+// fakeNpmRequest is one request, with the program that made it.
+type fakeNpmRequest struct {
+	path, agent string
+}
+
+// byNpm reports a request that npm itself made, as opposed to nvx's own checks.
+func (r fakeNpmRequest) byNpm() bool { return strings.HasPrefix(r.agent, "npm/") }
+
+// forPackument reports a request for a package's metadata rather than a tarball.
+func (r fakeNpmRequest) forPackument() bool { return !strings.Contains(r.path, "/-/") }
+
+func newFakeNpmRegistry(t *testing.T) *fakeNpmRegistry {
+	t.Helper()
+	r := &fakeNpmRegistry{t: t, pkgs: map[string]*fakeNpmPackage{}}
+	r.srv = httptest.NewServer(http.HandlerFunc(r.serve))
+	t.Cleanup(r.srv.Close)
+	return r
+}
+
+func (r *fakeNpmRegistry) url() string { return r.srv.URL + "/" }
+
+// publish adds a version and makes it the latest.
+func (r *fakeNpmRegistry) publish(name, version string, deps map[string]string) {
+	r.t.Helper()
+	r.publishWith(name, version, deps, nil)
+}
+
+func (r *fakeNpmRegistry) publishWith(name, version string, deps, scripts map[string]string) {
+	r.t.Helper()
+	manifest := map[string]any{"name": name, "version": version}
+	if len(deps) > 0 {
+		manifest["dependencies"] = deps
+	}
+	if len(scripts) > 0 {
+		manifest["scripts"] = scripts
+	}
+	body, err := json.Marshal(manifest)
+	if err != nil {
+		r.t.Fatal(err)
+	}
+	var tgz bytes.Buffer
+	gz := gzip.NewWriter(&tgz)
+	tw := tar.NewWriter(gz)
+	for _, f := range []struct{ name, body string }{
+		{"package/package.json", string(body)},
+		{"package/index.js", "module.exports = " + fmt.Sprintf("%q", version) + ";\n"},
+	} {
+		if err := tw.WriteHeader(&tar.Header{Name: f.name, Mode: 0o644, Size: int64(len(f.body)), ModTime: time.Unix(0, 0)}); err != nil {
+			r.t.Fatal(err)
+		}
+		if _, err := tw.Write([]byte(f.body)); err != nil {
+			r.t.Fatal(err)
+		}
+	}
+	if err := tw.Close(); err != nil {
+		r.t.Fatal(err)
+	}
+	if err := gz.Close(); err != nil {
+		r.t.Fatal(err)
+	}
+
+	r.mu.Lock()
+	defer r.mu.Unlock()
+	p := r.pkgs[name]
+	if p == nil {
+		p = &fakeNpmPackage{versions: map[string]fakeNpmVersion{}, tags: map[string]string{}}
+		r.pkgs[name] = p
+	}
+	p.versions[version] = fakeNpmVersion{deps: deps, scripts: scripts, tgz: tgz.Bytes()}
+	p.tags["latest"] = version
+}
+
+// requests returns what has been asked so far.
+func (r *fakeNpmRegistry) requests() []fakeNpmRequest {
+	r.mu.Lock()
+	defer r.mu.Unlock()
+	return append([]fakeNpmRequest(nil), r.log...)
+}
+
+func (r *fakeNpmRegistry) serve(w http.ResponseWriter, req *http.Request) {
+	path := req.URL.EscapedPath()
+	r.mu.Lock()
+	r.log = append(r.log, fakeNpmRequest{path: path, agent: req.UserAgent()})
+	r.mu.Unlock()
+
+	name, tarball, isTarball := strings.Cut(strings.TrimPrefix(req.URL.Path, "/"), "/-/")
+	r.mu.Lock()
+	defer r.mu.Unlock()
+	p := r.pkgs[name]
+	if p == nil {
+		http.NotFound(w, req)
+		return
+	}
+	if isTarball {
+		for version, v := range p.versions {
+			if tarball == name+"-"+version+".tgz" {
+				w.Header().Set("Content-Type", "application/octet-stream")
+				_, _ = w.Write(v.tgz)
+				return
+			}
+		}
+		http.NotFound(w, req)
+		return
+	}
+
+	versions := map[string]any{}
+	times := map[string]string{"created": "2020-01-01T00:00:00.000Z", "modified": "2020-01-01T00:00:00.000Z"}
+	for version, v := range p.versions {
+		sum := sha512.Sum512(v.tgz)
+		sha := sha1.Sum(v.tgz) // #nosec G401 -- the registry's shasum field is SHA-1
+		manifest := map[string]any{
+			"name": name, "version": version,
+			"dist": map[string]string{
+				"tarball":   r.srv.URL + "/" + name + "/-/" + name + "-" + version + ".tgz",
+				"integrity": "sha512-" + base64.StdEncoding.EncodeToString(sum[:]),
+				"shasum":    hex.EncodeToString(sha[:]),
+			},
+		}
+		if len(v.deps) > 0 {
+			manifest["dependencies"] = v.deps
+		}
+		if len(v.scripts) > 0 {
+			manifest["scripts"] = v.scripts
+		}
+		versions[version] = manifest
+		times[version] = "2020-01-02T03:04:05.000Z"
+	}
+	w.Header().Set("Content-Type", "application/json")
+	_ = json.NewEncoder(w).Encode(map[string]any{
+		"name": name, "dist-tags": p.tags, "versions": versions, "time": times,
+	})
+}
+
+// reset forgets every package and every request.
+func (r *fakeNpmRegistry) reset() {
+	r.mu.Lock()
+	defer r.mu.Unlock()
+	r.pkgs = map[string]*fakeNpmPackage{}
+	r.log = nil
+}