Skip to content

fix(compose): point MinIO at quay.io — the Docker Hub repository is gone - #424

Merged
acamarata merged 2 commits into
mainfrom
fix/minio-image-moved-to-quay
Sep 14, 2026
Merged

acamarata merged 2 commits into
mainfrom
fix/minio-image-moved-to-quay

Conversation

@acamarata

Copy link
Copy Markdown
Collaborator

This is a live product outage, not a CI failure

MinIO removed minio/minio from Docker Hub. Every nSelf stack with MINIO_ENABLED=true can no longer pull its storage image:

minio Error pull access denied for minio/minio, repository does not exist
or may require 'docker login': denied: requested access to the resource is denied

Measured today:

check result
hub.docker.com/v2/repositories/minio/minio/ {"message":"object not found"}
manifest minio/minio:RELEASE.2024-01-16T16-07-38Z, anonymous 401
manifest minio/minio:RELEASE.2024-01-16T16-07-38Z, authenticated 401
manifest minio/minio:latest 401
quay.io/minio/minio — same three tags, anonymous 200

Authenticating does not help — I tested with valid Docker Hub credentials. The repository is gone, not rate-limited and not gated, so a docker/login-action step would have been the wrong fix.

internal/compose/images.go is the generator behind every user's docker-compose.yml, so this is not limited to our CI. It surfaced as a red job in nchat (Accessibility E2E Tests, run 34837167497), which starts a real nself stack.

Changes

1. MinIO → quay.io. One MinioImagePath constant, used by both the DefaultImageVersions pin and buildMinioService's MINIO_VERSION path — they formatted the name independently before and could drift. MINIO_VERSION semantics are unchanged.

2. nhost/functions:0.3.7:latest. Found by the same sweep. That tag has never existed (nhost's 0.x line stops at 0.1.9; tags are now <node-major>-<version>, e.g. 22-2.2.0). Nothing broke, because the entry is unreachable: applyDefaultsFunctions sets FUNCTIONS_VERSION=latest when unset, so ResolveImage never falls back to the pin. latest is what users actually run, so naming it makes the pin honest and changes no emitted compose file. Picking a real 2.2.0 pin is an upgrade decision, deliberately not made here.

3. A gate so this cannot recur silently. .github/workflows/default-images-pullable.yml resolves every DefaultImageVersions entry daily with docker manifest inspect. The list is generated from the map itself (tools/imagelist), so a new pin is covered the moment it lands — no second list to forget. It's a manifest lookup, not a pull: no registry credentials, and no warm layer cache that could hide a dead pin. This is the gate whose absence let a pin rot: it was valid when written, and nothing re-checked it.

4. Docs + examples. Core-Services.md, Service-Graph.md, Config-Custom-Services.md, and the CS_N_IMAGE examples that used minio/minio as a digest-pinning illustration.

Verification

$ go run ./tools/imagelist | while read svc img; do docker manifest inspect $img …; done
ok  admin/auth/functions/hasura/mailpit/meilisearch/minio/mlflow/nginx/postgres/redis/typesense   (12/12)
gate_exit=0

Exercised in the failing direction — MinioImagePath reverted to minio/minio:

FAIL  minio  minio/minio:RELEASE.2024-01-16T16-07-38Z -- denied: requested access to the resource is denied
gate_exit_with_regression=1

--- FAIL: TestMinioImageIsRegistryQualified
--- FAIL: TestBuildMinioService_UsesQuayRegistry/explicit_version
--- FAIL: TestBuildMinioService_UsesQuayRegistry/empty_version_defaults_to_latest

gofmt clean, go vet clean, go test ./internal/compose/... ./internal/config/... ./internal/build/... → 612 passed.

Follow-up (not in this PR)

Existing installs carry the dead image in their already-generated docker-compose.yml and need nself build re-run. Worth a nself doctor check that flags an unresolvable service image; filed separately rather than bolted on here.

Reference apps (nchat, ntask, admin) pin minio/minio in their own compose files and k8s manifests — fixed per repo, tracked alongside this.

MinIO removed minio/minio from Docker Hub. As of 2026-09-14 the Hub API
answers {"message":"object not found"} for the repository and every tag
returns 401 — including RELEASE.2024-01-16T16-07-38Z, the pin nself build
has been emitting. Authenticating does not help; the repository does not
exist, it is not gated. Every generated stack with MINIO_ENABLED=true
therefore fails to pull its storage image:

  pull access denied for minio/minio, repository does not exist or may
  require 'docker login'

quay.io/minio/minio is MinIO's own registry and serves the same tags
anonymously (:latest, RELEASE.2024-01-16T16-07-38Z and
RELEASE.2024-10-02T17-50-41Z all 200). Both places that name the image now
build from one MinioImagePath constant so they cannot drift apart, and two
tests fail if either reverts to an unqualified name.

Also corrects nhost/functions:0.3.7, found by the same sweep: that tag has
never existed (nhost's 0.x line stops at 0.1.9; tags are now
<node-major>-<version>). Nothing broke because the entry is unreachable —
FUNCTIONS_VERSION defaults to "latest", so ResolveImage never falls back to
the pin — and "latest" is what users actually run, so naming it changes no
emitted compose file.

Adds the gate that would have caught both: a daily workflow resolving every
DefaultImageVersions entry with docker manifest inspect. The list comes from
the map itself via tools/imagelist, so new pins are covered automatically.
A manifest lookup, not a pull: no credentials, no layer cache to hide a
dead pin. Verified locally — all 12 images resolve; reintroducing
minio/minio makes the gate exit 1 and both new tests fail.
… edits

Core-Services.md is generated (scripts/sport/generate-core-services.sh) and
internal/repoqa asserts it is current; hand-editing the minio row made
TestCoreServicesPageIsCurrent fail. Regenerated with `make core-services`,
which also picked up the functions row I had missed.

The two internal/config changes are reverted. They only swapped a dead image
out of a doc comment and a test literal, neither of which is an env-var
change — but doc-sync keys off any path under internal/config/ and demanded
an env-var doc update. Editing Config-Env-Vars.md to satisfy that would be a
docs edit with nothing true to say, so the cosmetic changes go instead. The
user-facing CS_N_IMAGE example in the wiki is still corrected.
@acamarata
acamarata merged commit 7eaadb9 into main Sep 14, 2026
32 checks passed
@acamarata
acamarata deleted the fix/minio-image-moved-to-quay branch September 14, 2026 12:23
acamarata added a commit to nself-org/nchat that referenced this pull request Sep 14, 2026
CI has been red since MinIO removed minio/minio from Docker Hub. The
Accessibility E2E job fails at 'Start nself backend stack':

  minio Error pull access denied for minio/minio, repository does not
  exist or may require 'docker login'

The Hub API now answers {"message":"object not found"} for the repository
and every tag returns 401, to anonymous and authenticated requests alike,
so a docker login step would not fix it. quay.io/minio/minio is MinIO's own
registry and serves :latest anonymously.

Verified by resolving the CI stack exactly as the workflow does (.env.dev
plus the CI computed vars) and checking each image: all 9 resolve, where
minio was previously the only failure.

Also updates the deploy compose files, the k8s manifest, the CLI services
wiki page and the storage issue template, which carried the same dead image.

backend/docker-compose.yml is marked GENERATED BY nself build. Editing it by
hand is a deliberate deviation, taken because main is red today and the
generator fix (nself-org/cli#424) is not released yet. The durable fix is to
regenerate this file once that ships. Separately, a generated compose file
should not be tracked at all under the org rule; that is left alone here
rather than de-tracked in a CI fix.
acamarata added a commit to nself-org/admin that referenced this pull request Sep 14, 2026
MinIO removed minio/minio from Docker Hub. As of 2026-09-14 the Hub API
answers {"message":"object not found"} for the repository and every tag
returns 401, to anonymous and authenticated requests alike, so this is not a
rate limit or a missing docker login. quay.io/minio/minio is MinIO's own
registry and serves :latest anonymously (200).

This is not cosmetic here: build-simple/route.ts GENERATES a docker-compose
for the user when storage is enabled, so the Admin GUI has been handing
people a compose file whose storage service cannot pull. start/page.tsx
displays the same image name, and two wiki pages document it.

Upstream generator fix: nself-org/cli#424, which also adds a daily gate
resolving every default image so a pin cannot rot unnoticed again.
acamarata added a commit to nself-org/ntask that referenced this pull request Sep 14, 2026
…ne (#176)

MinIO removed minio/minio from Docker Hub. The Hub API answers
{"message":"object not found"} for the repository and every reference,
digest pins included, returns 401 to anonymous and authenticated requests
alike. This is production's storage service: the container keeps running
from its local image, but any redeploy, restart with pull, or new box
cannot fetch it.

The pinned digest is unchanged and still exact. quay.io serves the same
manifest bytes: a HEAD for RELEASE.2025-09-07T16-13-09Z there returns
docker-content-digest sha256:14cea493d9a34af32f524e538b8346cf79f3321eff8e708c1e2960462bd8936e,
byte-identical to the pin. Only the registry prefix changes, so the
digest-pinning rationale in the comment above it still holds exactly.

Upstream generator fix: nself-org/cli#424, which also adds a daily gate
resolving every default image so a pin cannot rot unnoticed again.
acamarata added a commit to nself-org/nchat that referenced this pull request Sep 14, 2026
…es that were hiding behind them (#179)

* fix(security): close the 20 high dependency CVEs blocking the security gate

Security Gate now reports DEP_HIGH=20 and fails. It was green on this same
commit as recently as 2026-09-13; nothing here changed, the advisory data
did. Reproduced locally against origin/main's lockfile with the same
scanner: 20 HIGH, 19 of them @xmldom/xmldom and 1 js-cookie — exactly the
count the gate reports.

@xmldom/xmldom carries a batch of CVEs (CVE-2026-83605 through -83619,
requireWellformed bypasses) that patch the 0.8 and 0.9 branches
independently: 0.8.x is fixed in 0.8.15, 0.9.x in 0.9.12. The existing
override pinned the floor at >=0.8.13, which is inside the vulnerable range,
and both 0.8.13 and 0.9.10 were installed. Each branch gets its own ceiling
so 0.8 consumers are not forced onto 0.9, which is an API break.

js-cookie 3.0.6 -> >=3.0.7 for CVE-2026-46625.

Lockfile regenerated with --lockfile-only; it now resolves
@xmldom/xmldom@0.8.15, @xmldom/xmldom@0.9.12 and js-cookie@3.0.8. Rescanned:
20 HIGH before, 0 after. No advisory was ignored and no threshold moved.

* fix(ci): make the security gate read the accepted-CVE list it was written for

The gate reads pnpm.auditConfig.ignoreCves from 'package.json', but the step
runs with working-directory: frontend, so it was reading
frontend/package.json — whose auditConfig is empty. The curated list is in
the ROOT package.json. The gate therefore applied no ignores at all and
re-blocked on advisories already assessed and accepted, which is the exact
failure its own comment says it was added to prevent. Now reads
${GITHUB_WORKSPACE}/package.json, so there is one list and it is the
curated one.

Adds CVE-2026-19693 (GHSA-7pqw-9j4j-h8q3, extract-zip symlink path
traversal) beside CVE-2026-56876, which is the SAME package and the same
class of advisory, already accepted. Neither has a fix: both report
first_patched_version NONE against <= 2.0.1, and 2.0.1 is npm latest, so
there is nothing to upgrade to. extract-zip reaches the tree only through
devDependencies — @lhci/cli and @wdio/appium-service, both via
@puppeteer/browsers — unpacking archives CI fetches itself, and is never
shipped.

Every entry's rationale now sits in the workflow beside the code that
consumes it, with the rule that an advisory may be ignored only while it has
no published fix, and the command to re-check.

Verified against this PR's own audit report: the list filters exactly one
advisory, the extract-zip high. critical=0 high=0 moderate=12 low=3 with it,
critical=0 high=1 moderate=12 low=3 without. No moderate or low is newly
suppressed.

* fix(docker): point MinIO at quay.io — the Docker Hub repository is gone

CI has been red since MinIO removed minio/minio from Docker Hub. The
Accessibility E2E job fails at 'Start nself backend stack':

  minio Error pull access denied for minio/minio, repository does not
  exist or may require 'docker login'

The Hub API now answers {"message":"object not found"} for the repository
and every tag returns 401, to anonymous and authenticated requests alike,
so a docker login step would not fix it. quay.io/minio/minio is MinIO's own
registry and serves :latest anonymously.

Verified by resolving the CI stack exactly as the workflow does (.env.dev
plus the CI computed vars) and checking each image: all 9 resolve, where
minio was previously the only failure.

Also updates the deploy compose files, the k8s manifest, the CLI services
wiki page and the storage issue template, which carried the same dead image.

backend/docker-compose.yml is marked GENERATED BY nself build. Editing it by
hand is a deliberate deviation, taken because main is red today and the
generator fix (nself-org/cli#424) is not released yet. The durable fix is to
regenerate this file once that ships. Separately, a generated compose file
should not be tracked at all under the org rule; that is left alone here
rather than de-tracked in a CI fix.

* ci: let the Generated File Gate accept the one generated file this repo tracks on purpose

The two gates in this workflow contradicted each other. Doc-Sync carves out
backend/docker-compose.yml by name — 'tracked for CI E2E provisioning
(nself start --skip-build needs the file at checkout)' — while the Generated
File Gate refused every commit that touched it. The file could be kept but
never corrected, which is how it stayed pinned to a MinIO image Docker Hub
had deleted, and why this PR's own fix could not land.

The carve-out is verified, not blanket. This gate exists to stop generated
files carrying plaintext credentials into a public repo (the ntask incident:
a generated compose with the postgres password, Hasura admin secret and JWT
keys). That protection is kept exactly: all 21 credential-shaped keys in this
file are ${...} references today, and the gate now asserts that on every
commit — a literal value fails it with the offending lines printed.

Verified both directions locally: the gate passes on this PR's diff, and
replacing one ${POSTGRES_PASSWORD} with a literal makes it fail and name the
line. Every other generated file is refused exactly as before.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant