Skip to content

fix(cli): detect orphaned containers on build, flag unusable healthchecks (G-014) - #409

Merged
acamarata merged 2 commits into
mainfrom
p6/g014-orphan-detection
Sep 12, 2026
Merged

acamarata merged 2 commits into
mainfrom
p6/g014-orphan-detection

Conversation

@acamarata

Copy link
Copy Markdown
Collaborator

Summary

Closes CLI gap G-014.

nself build only ever generated files — it never looked at the live Docker daemon, so a service dropped from the generated compose (removed from nself.yaml, an uninstalled plugin, a rename) left its container running forever: no service definition, no nginx vhost, no traffic, and nothing reported it. Measured live on prod: four containers (nself-claw, nself-notify, nself-mux, nself-cron) were in this state; two had been dead a full month (a database DNS failure) before anyone noticed — because a second, compounding defect made their healthchecks permanently meaningless too (curl was never installed in their image, so the healthcheck always errored and the container reported "unhealthy" whether or not it actually worked).

1. Orphan detection + removal

  • internal/docker/orphans.go (new): DetectOrphans lists containers filtered strictly by this project's com.docker.compose.project label (the exact label Docker Compose stamps from the generated compose file's name: field, i.e. cfg.ProjectName) and flags any whose com.docker.compose.service label isn't in the union of services from the freshly generated compose files (base + plugin fragments, read via build.ReadComposeManifest). RemoveOrphans force-removes what's found via the same docker rm -f primitive cleanup.go already uses for init/zombie containers.
  • nself build now runs this detection by default and reports orphans prominently; a new --remove-orphans flag makes removal opt-in. Detection is best-effort — any docker-level failure (daemon unreachable, docker not installed) is logged at debug level and swallowed, so a build in a Docker-less CI image still succeeds.
  • nself status gets the same read-only detection (a container can drift orphaned between builds, not only during one) — it never removes anything itself.

Scoping guarantee: every container is created by docker compose with com.docker.compose.project set to the compose file's top-level name:, which nSelf sets to cfg.ProjectName (internal/compose/generator.go). DetectOrphans filters on that exact label server-side in the docker ps call itself — a container from a different project (different label value) or one Docker didn't create via compose (no label) can never be selected. This mirrors the label filter cleanup.go's cleanupZombieContainers already uses for the same reason.

2. Healthcheck validity

  • internal/doctor/deep_docker_healthcheck.go (new): given a container doctor --deep already found "unhealthy", diagnoseUnhealthyContainer inspects its healthcheck Test command, extracts the binary it would invoke, and confirms via docker exec ... command -v <binary> whether that binary actually exists in the image. If it's confirmed absent, the check reports a distinct, honest message ("healthcheck command %q is not installed... status carries no information") with a real fix (install the binary or change the test), instead of the pre-existing generic docker restart suggestion that can never help this case. Any ambiguity (healthcheck unset, can't verify) falls back to the original generic message — this never downgrades a real failure.
  • Home: nself doctor --deep, not nself build. internal/build is a pure generator — nothing in it touches the Docker daemon — and confirming a binary is on a container's PATH requires a live container to exec into, which only exists once something is running. doctor --deep's DockerDeepChecks already inspects live container health and is exactly where the false "unhealthy"/"docker restart" signal was produced.

Other

  • Regenerated .github/command-inventory.json for the new --remove-orphans flag (verified via internal/repoqa's TestCommandInventoryIsCurrent); refreshed SPORT F02-COMMAND-INVENTORY.md to match (unaffected content-wise — that table lists commands/subcommands, not flags — but timestamp/regeneration kept current).

Test plan

  • go build ./... — clean
  • go vet ./... — clean
  • golangci-lint run ./... — 0 issues
  • go test ./... — 5154 passed, 97 packages, no regressions
  • New unit tests (20 total) for the pure parsing/classification logic — no live docker daemon or real project containers required:
    • internal/docker/orphans_test.go: buildOrphanPsArgs scoping, parseOrphanPsOutput orphan/non-orphan/blank-label/malformed-line cases, ComposeServiceNames union/missing-file/invalid-YAML cases
    • internal/doctor/deep_docker_healthcheck_test.go: extractHealthcheckBinary across CMD/CMD-SHELL/NONE/malformed shapes, classifyBinaryProbe found/confirmed-missing/inconclusive cases

…ecks

G-014: `nself build` only ever generated files — it never looked at the
live daemon, so a service dropped from the generated compose left its
container running forever with no service definition, no nginx vhost, and
no traffic. Measured live: nself-claw/notify/mux/cron survived this way,
two of them dead a full month before anyone noticed.

- internal/docker: DetectOrphans lists containers scoped strictly to this
  project's compose-project label (never touches unrelated workloads on
  the host) and compares their service label against the union of
  services in the freshly generated compose files (base + plugin
  fragments). RemoveOrphans force-removes what's found.
- cmd/commands: `nself build` now reports orphans by default and adds
  `--remove-orphans` to remove them (opt-in). `nself status` gets the same
  read-only detection, since a container can drift orphaned between builds.
- internal/doctor: a second, compounding defect made these containers
  undetectable — their healthchecks ran a binary (curl) not installed in
  the image, so they reported "unhealthy" permanently regardless of
  whether the service worked, and `doctor --deep` suggested a useless
  `docker restart`. diagnoseUnhealthyContainer now confirms whether the
  healthcheck command actually exists in the image before trusting the
  status, and gives an honest fix (install the binary or change the test)
  instead of a restart that can never help.

Regenerated .github/command-inventory.json for the new --remove-orphans
flag; SPORT F02 refreshed to match.
@acamarata
acamarata merged commit ba588b4 into main Sep 12, 2026
32 checks passed
@acamarata
acamarata deleted the p6/g014-orphan-detection branch September 12, 2026 14:09
@acamarata acamarata mentioned this pull request Sep 12, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant