Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .github/wiki/Config-Custom-Services.md
Original file line number Diff line number Diff line change
Expand Up @@ -84,7 +84,7 @@ All variables use the pattern `CS_N_*` where `N` is the slot number (1–10). Va
| `CS_N_ENV_PASSTHROUGH` | string | *(empty)* | Comma-separated allowlist of project `.env` var names to forward into this container in addition to the fixed core set. `CS_N_ENV` still wins on a name conflict. |
| `CS_N_ENV_FILE` | string | *(empty)* | Project-relative path to a dotenv-format file whose `KEY=VALUE` lines are injected into this container. Applied after `CS_N_ENV_PASSTHROUGH`, before `CS_N_ENV`. Use this instead of `CS_N_ENV` when a value itself contains a comma (e.g. some SMTP passwords) or when there are too many vars for one line. A missing file fails `nself build` rather than silently starting the service without those vars. |
| `CS_N_ENV` | string | *(empty)* | Additional env vars to inject, in `KEY=VALUE,KEY=VALUE` format. Always applied last — overrides the fixed core set, `CS_N_ENV_PASSTHROUGH`, and `CS_N_ENV_FILE`. |
| `CS_N_IMAGE` | string | *(empty)* | Run a pre-built image instead of building from a Dockerfile — e.g. `minio/minio:RELEASE.2024-01-16T16-07-38Z@sha256:...` to pin an exact digest. Mutually exclusive with `CS_N_PATH`; when set, no `build:` block is emitted at all. |
| `CS_N_IMAGE` | string | *(empty)* | Run a pre-built image instead of building from a Dockerfile — e.g. `quay.io/minio/minio:RELEASE.2024-01-16T16-07-38Z@sha256:...` to pin an exact digest. Mutually exclusive with `CS_N_PATH`; when set, no `build:` block is emitted at all. |
| `CS_N_VOLUMES` | string | *(empty)* | Comma-separated extra bind mounts in `host:container[:mode]` form, e.g. `./email-templates:/app/templates:ro`. Appended to the service's generated volume list. |

All `CS_*` variables are automatically exempt from "unknown env var" warnings.
Expand Down
4 changes: 2 additions & 2 deletions .github/wiki/Core-Services.md
Original file line number Diff line number Diff line change
Expand Up @@ -25,10 +25,10 @@ Added to the stack only when their enabling variable is `true`.
| Service | Purpose | Enable with | Version override | Default image |
|---|---|---|---|---|
| `nself-admin` | nSelf Admin web UI (localhost only; never deployed) | `NSELF_ADMIN_ENABLED=true` | `NSELF_ADMIN_VERSION` | `nself/nself-admin:latest` |
| `functions` | Serverless function runtime | `FUNCTIONS_ENABLED=true` | `FUNCTIONS_VERSION` | `nhost/functions:0.3.7` |
| `functions` | Serverless function runtime | `FUNCTIONS_ENABLED=true` | `FUNCTIONS_VERSION` | `nhost/functions:latest` |
| `mailpit` | Development SMTP catcher — captures outbound mail instead of sending it | `MAILPIT_ENABLED=true` | `MAILPIT_VERSION` | `axllent/mailpit:v1.15` |
| `meilisearch` | Full-text search index (SEARCH_ENGINE=meilisearch, the default) | `SEARCH_ENABLED=true` | `MEILISEARCH_VERSION` | `getmeili/meilisearch:v1.6` |
| `minio` | S3-compatible object storage | `MINIO_ENABLED=true` | `MINIO_VERSION` | `minio/minio:RELEASE.2024-01-16T16-07-38Z` |
| `minio` | S3-compatible object storage | `MINIO_ENABLED=true` | `MINIO_VERSION` | `quay.io/minio/minio:RELEASE.2024-01-16T16-07-38Z` |
| `typesense` | Full-text search index (SEARCH_ENGINE=typesense alternative) | `SEARCH_ENABLED=true` | `TYPESENSE_VERSION` | `typesense/typesense:0.25.2` |
| `redis` | Cache and queue backend | `REDIS_ENABLED=true` | `REDIS_VERSION` | `redis:7.2-alpine` |

Expand Down
2 changes: 1 addition & 1 deletion .github/wiki/Service-Graph.md
Original file line number Diff line number Diff line change
Expand Up @@ -51,7 +51,7 @@ Optional services are enabled by setting the corresponding environment variable
| Service | Toggle | Image | Internal Port | Purpose |
|---------|--------|-------|---------------|---------|
| Redis | `REDIS_ENABLED=true` | `redis:7-alpine` | 6379 | Caching, session storage, job queues |
| Storage (MinIO) | `MINIO_ENABLED=true` | `minio/minio:latest` | 9000 (API), 9001 (console) | S3-compatible object storage |
| Storage (MinIO) | `MINIO_ENABLED=true` | `quay.io/minio/minio:latest` | 9000 (API), 9001 (console) | S3-compatible object storage |
| Email | `MAILPIT_ENABLED=true` | `axllent/mailpit:latest` | 1025 (SMTP), 8025 (UI) | Email testing in development , not for production use |
| Search | `SEARCH_ENABLED=true` | varies | 7700 (MeiliSearch) or 8108 (Typesense) | Full-text search indexing and querying |
| Functions | `FUNCTIONS_ENABLED=true` | `nhost/functions:latest` | 3008 | Serverless function runtime |
Expand Down
82 changes: 82 additions & 0 deletions .github/workflows/default-images-pullable.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,82 @@
name: Default images pullable

# Every image `nself build` can emit must still exist in its registry.
#
# Added 2026-09-14 after MinIO deleted the `minio/minio` repository from Docker
# Hub. Every generated stack with MINIO_ENABLED=true stopped being able to pull
# its storage image, and nothing in this repo noticed: the pin was valid the day
# it was written, and no test or gate re-checks a pin after it lands. The same
# sweep found nhost/functions:0.3.7, a tag that never existed at all.
#
# This gate closes that class of failure. It is deliberately a manifest lookup,
# not a pull: it costs a few HTTP requests, needs no registry credentials, and
# cannot be defeated by a warm layer cache on the runner.
#
# It runs on a schedule because the failure is external — a pin that passed
# yesterday can break today with no commit in this repo — and on pushes that
# touch the image list so a bad pin is caught before it ships.

on:
schedule:
- cron: '0 5 * * *'
push:
branches: [main]
paths:
- 'internal/compose/images.go'
- 'tools/imagelist/**'
- '.github/workflows/default-images-pullable.yml'
pull_request:
paths:
- 'internal/compose/images.go'
- 'tools/imagelist/**'
- '.github/workflows/default-images-pullable.yml'
workflow_dispatch:

permissions:
contents: read

jobs:
check:
name: Every default image resolves
runs-on: ubuntu-latest
timeout-minutes: 10
steps:
- uses: actions/checkout@v7

- uses: actions/setup-go@v7
with:
go-version-file: go.mod
cache: false

- name: Check every default image manifest resolves
run: |
set -uo pipefail

# The list comes from compose.DefaultImageVersions itself, so a new
# pin is covered the moment it lands and there is no second list.
go run ./tools/imagelist > /tmp/default-images.tsv
if [ ! -s /tmp/default-images.tsv ]; then
echo "::error::imagelist produced no images — the generator or the map is broken"
exit 1
fi

failed=0
while IFS="$(printf '\t')" read -r service image; do
[ -n "${image}" ] || continue
if err=$(docker manifest inspect "${image}" 2>&1 >/dev/null); then
echo "ok ${service} ${image}"
else
failed=1
echo "FAIL ${service} ${image}"
echo "::error title=${service} image is not pullable::${image} — ${err}"
fi
done < /tmp/default-images.tsv

echo
if [ "${failed}" -ne 0 ]; then
echo "One or more images in internal/compose/images.go no longer resolve."
echo "Fix the pin (a registry move needs the new registry-qualified path,"
echo "e.g. quay.io/minio/minio), do NOT delete the entry to make this pass."
exit 1
fi
echo "All default images resolve."
2 changes: 1 addition & 1 deletion internal/compose/custom_service_extras_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -24,7 +24,7 @@ func TestBuildCustomService_ImageSkipsBuild(t *testing.T) {
cfg := minimalConfigWithCS()
g := NewGenerator(cfg)
cs := testCS()
cs.Image = "minio/minio:RELEASE.2024-01-16T16-07-38Z@sha256:abc123"
cs.Image = "quay.io/minio/minio:RELEASE.2024-01-16T16-07-38Z@sha256:abc123"

svc, err := g.buildCustomService(cs)
if err != nil {
Expand Down
45 changes: 38 additions & 7 deletions internal/compose/images.go
Original file line number Diff line number Diff line change
Expand Up @@ -16,16 +16,47 @@ import (
// Intentionally referencing nself/nself-admin (Docker Hub) — never github.com/nself-org/ paths.
const AdminImagePath = "nself/nself-admin"

// MinioImagePath is the registry path for the MinIO object-storage image.
//
// It is REGISTRY-QUALIFIED and must stay that way. MinIO removed the
// `minio/minio` repository from Docker Hub: as of 2026-09-14 the Hub API
// returns `{"message":"object not found"}` for it and every tag — including
// long-published pins such as RELEASE.2024-01-16T16-07-38Z — answers 401 to
// both anonymous and authenticated manifest requests. A `docker pull` of it
// fails with "pull access denied for minio/minio, repository does not exist
// or may require 'docker login'", so every generated stack with
// MINIO_ENABLED=true could no longer start. Authenticating does NOT help;
// the repository is gone, not gated.
//
// quay.io/minio/minio is MinIO's own registry and serves the same tags
// anonymously (verified 2026-09-14: :latest, RELEASE.2024-01-16T16-07-38Z
// and RELEASE.2024-10-02T17-50-41Z all return 200).
//
// Both the DefaultImageVersions pin below and buildMinioService's
// MINIO_VERSION path must build from this constant so the two cannot drift
// back to an unqualified Docker Hub name.
const MinioImagePath = "quay.io/minio/minio"

// DefaultImageVersions maps service name to pinned image:tag.
// Update with each nSelf release.
var DefaultImageVersions = map[string]string{
"postgres": "pgvector/pgvector:pg16",
"hasura": "hasura/graphql-engine:v2.44.0",
"auth": "nhost/hasura-auth:0.36.0",
"nginx": "nginx:1.25-alpine",
"redis": "redis:7.2-alpine",
"minio": "minio/minio:RELEASE.2024-01-16T16-07-38Z",
"functions": "nhost/functions:0.3.7",
"postgres": "pgvector/pgvector:pg16",
"hasura": "hasura/graphql-engine:v2.44.0",
"auth": "nhost/hasura-auth:0.36.0",
"nginx": "nginx:1.25-alpine",
"redis": "redis:7.2-alpine",
"minio": MinioImagePath + ":RELEASE.2024-01-16T16-07-38Z",
// nhost/functions:0.3.7 never existed. nhost's 0.x line stops at 0.1.9 and
// the repository now tags as <node-major>-<version> (22-2.2.0, 26-2.2.0);
// `docker manifest inspect nhost/functions:0.3.7` answers "no such
// manifest". Nothing broke because this entry is unreachable in practice:
// applyDefaultsFunctions sets FUNCTIONS_VERSION to "latest" when unset, so
// buildFunctionsService always passes a non-empty image and ResolveImage
// never falls back to this pin. "latest" is therefore what users actually
// run, and naming it here makes the pin honest without changing any
// emitted compose file. Choosing a real pinned tag is an upgrade decision
// (latest is not any of the current 2.2.0 tags), not a drive-by edit.
"functions": "nhost/functions:latest",
"mailpit": "axllent/mailpit:v1.15",
"meilisearch": "getmeili/meilisearch:v1.6",
"typesense": "typesense/typesense:0.25.2",
Expand Down
54 changes: 54 additions & 0 deletions internal/compose/images_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -140,3 +140,57 @@ func TestDefaultImageVersions_NoGoModulePaths(t *testing.T) {
}
}
}

// TestMinioImageIsRegistryQualified guards the fix for the 2026-09-14 storage
// outage: MinIO deleted the `minio/minio` repository from Docker Hub, so an
// unqualified reference resolves to a repository that no longer exists and
// every generated stack with MINIO_ENABLED=true failed to pull. The Hub API
// returns "object not found" and every tag answers 401, to anonymous and
// authenticated requests alike, so this is not something a docker login fixes.
//
// Both places that name the image must stay pointed at MinIO's own registry:
// the DefaultImageVersions pin (used when no MINIO_VERSION is set) and
// buildMinioService's MINIO_VERSION path. A bare "minio/minio:..." in either
// is the regression this test exists to catch.
func TestMinioImageIsRegistryQualified(t *testing.T) {
const wantPrefix = "quay.io/minio/minio:"

if !strings.HasPrefix(MinioImagePath+":", wantPrefix) {
t.Fatalf("MinioImagePath = %q, want %q without the tag", MinioImagePath, "quay.io/minio/minio")
}

pinned, ok := DefaultImageVersions["minio"]
if !ok {
t.Fatal(`DefaultImageVersions has no "minio" entry`)
}
if !strings.HasPrefix(pinned, wantPrefix) {
t.Errorf("DefaultImageVersions[\"minio\"] = %q, want prefix %q", pinned, wantPrefix)
}
}

// TestBuildMinioService_UsesQuayRegistry covers the MINIO_VERSION path, which
// formats its own image string and so can drift away from the pin above
// independently. Both an explicit version and the empty-version default are
// checked, because the empty case builds "…:latest" rather than falling
// through to DefaultImageVersions.
func TestBuildMinioService_UsesQuayRegistry(t *testing.T) {
for _, tc := range []struct {
name string
version string
want string
}{
{"explicit version", "RELEASE.2024-10-02T17-50-41Z", "quay.io/minio/minio:RELEASE.2024-10-02T17-50-41Z"},
{"empty version defaults to latest", "", "quay.io/minio/minio:latest"},
} {
t.Run(tc.name, func(t *testing.T) {
g := &Generator{cfg: &config.Config{
ProjectName: "testproject",
DockerNetwork: "testproject_network",
Minio: config.MinioConfig{Enabled: true, Version: tc.version},
}}
if got := g.buildMinioService().Image; got != tc.want {
t.Errorf("buildMinioService().Image = %q, want %q", got, tc.want)
}
})
}
}
4 changes: 3 additions & 1 deletion internal/compose/optional_services.go
Original file line number Diff line number Diff line change
Expand Up @@ -91,7 +91,9 @@ func (g *Generator) buildMinioService() ServiceConfig {
}

return ServiceConfig{
Image: ResolveImage("minio", fmt.Sprintf("minio/minio:%s", version)),
// MinioImagePath, never a bare "minio/minio" — that Docker Hub
// repository no longer exists. See the constant's doc comment.
Image: ResolveImage("minio", fmt.Sprintf("%s:%s", MinioImagePath, version)),
ContainerName: fmt.Sprintf("%s_minio", g.cfg.ProjectName),
Restart: "unless-stopped",
Networks: []string{g.cfg.DockerNetwork},
Expand Down
43 changes: 43 additions & 0 deletions tools/imagelist/main.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,43 @@
// Command imagelist prints every default service image `nself build` can emit.
//
// Purpose: give CI a single, authoritative list to check for pullability
// without restating the images anywhere. It reads them straight from
// compose.DefaultImageVersions, so a new or changed pin is covered the moment
// it lands — there is no second list to forget to update.
//
// Inputs: none.
//
// Outputs: one "<service>\t<image:tag>" line per service on stdout, sorted by
// service name so the output is stable and diffable.
//
// Constraints: prints only the DEFAULT pins. Images a user selects through env
// (POSTGRES_IMAGE, a custom CS_N_IMAGE) are that operator's choice and are
// deliberately out of scope.
//
// SPORT: consumed by .github/workflows/default-images-pullable.yml
package main

import (
"fmt"
"os"
"sort"

"github.com/nself-org/cli/internal/compose"
)

func main() {
services := make([]string, 0, len(compose.DefaultImageVersions))
for service := range compose.DefaultImageVersions {
services = append(services, service)
}
sort.Strings(services)

for _, service := range services {
fmt.Printf("%s\t%s\n", service, compose.DefaultImageVersions[service])
}

if len(services) == 0 {
fmt.Fprintln(os.Stderr, "imagelist: DefaultImageVersions is empty")
os.Exit(1)
}
}
Loading