Skip to content

feat(compose): extend CS_N custom services with image, env-file, and volume mounts - #408

Merged
acamarata merged 6 commits into
mainfrom
p6/g013-service-digest-env-mounts
Sep 12, 2026
Merged

acamarata merged 6 commits into
mainfrom
p6/g013-service-digest-env-mounts

Conversation

@acamarata

Copy link
Copy Markdown
Collaborator

Summary

Closes CLI gap G-013: nself build could not express a pinned image
digest, injected SMTP env vars, or an extra volume mount for a custom
service. The ntask production stack needed all three, so it was
hand-authored into docker-compose.override.yml — invisible to nself build and silently lost on any CLI-only rebuild.

Extends the existing CS_N custom-service model (no new mechanism) with
three new env vars, all declarable in .env:

  • CS_N_IMAGE — run a pre-built image (optionally digest-pinned via
    @sha256:...) instead of building from a Dockerfile. Mutually
    exclusive with CS_N_PATH.
  • CS_N_ENV_FILE — inject KEY=VALUE pairs from a dotenv-format file,
    applied after CS_N_ENV_PASSTHROUGH and before CS_N_ENV. Handles
    values CS_N_ENV's comma-joined format can't represent safely (e.g.
    a password containing a comma) and many-var cases like SMTP config.
  • CS_N_VOLUMES — comma-separated host:container[:mode] bind mounts,
    appended to the generated service.

Example expressing the three ntask requirements:

CS_1=email-storage:go
CS_1_IMAGE=minio/minio:RELEASE.2024-01-16T16-07-38Z@sha256:<digest>
CS_1_VOLUMES=./email-templates:/app/templates:ro

CS_2=ntask-api:go:8010
CS_2_ENV_FILE=./secrets/smtp.env

Generator gains WithWorkDir to anchor CS_N_ENV_FILE reads to the
project root; the build orchestrator now threads st.workdir through
so it resolves correctly regardless of invocation directory.
buildCustomService/Generate() now return an error on a missing or
unreadable CS_N_ENV_FILE rather than silently starting the service
without the vars it needs.

coreEnvVars was refactored into smaller named steps
(fixedCoreEnvVars/addOptionalStoreEnvVars/applyEnvPassthrough/
applyExtraEnv) while adding this — it was already over the repo's
50-line function cap before this change.

Docs: .github/wiki/Config-Custom-Services.md reference table + notes
updated (this also fixes a pre-existing doc/code drift: the page
already forward-referenced CS_N_IMAGE as "advanced usage" before it
existed in code).

Test plan

  • go build ./... clean
  • gofmt -l . clean, make vet clean, golangci-lint run clean
  • go test ./... — 5139 tests pass across 98 packages (no regressions)
  • New unit tests: parse-time validation (config package — image/path
    mutual exclusion, path traversal, volume entry shape) and
    compose-time behavior (image skips build, volumes appended,
    env-file precedence order, missing env-file fails loudly)
  • internal/compose coverage 90.1%, internal/config coverage 80.5%
    (both above the 70% gate)
  • make cmd-inventory re-run: no diff (no new commands added, as
    expected for an env-var-only extension)

…ownloaded

A plugin with a binaryName has downloadPluginPackageForTier try its
per-platform release asset first, falling back to the source tarball only
when that download fails. But the registry carried exactly one checksum per
plugin (the source tarball's), and installLocked always verified against it
regardless of which artifact was actually on disk — so any binaryName plugin
whose per-platform asset downloaded successfully had its platform tarball's
bytes hashed and compared against the source tarball's checksum, which can
never match.

downloadPluginPackageForTier now reports which artifact it fetched
(ArtifactKindSource, or a platform string from PlatformArch()) alongside the
path. installLocked resolves the checksum that matches via the new
resolveArtifactChecksum: the source artifact uses manifest.Checksum exactly
as before (including its existing FIX-CLI-6 warn-and-proceed leniency when
empty); a platform artifact uses the matching entry in the new
PluginManifest.PlatformChecksums map, parsed from the registry's
checksums.platforms object.

A platform artifact with no matching registry checksum is refused
unconditionally — it never falls through to the source-checksum leniency,
and no env var (NSELF_PLUGIN_REQUIRE_CHECKSUM included) changes that. That
leniency exists for the documented, tracked source-checksum coverage gap; it
was never a license to install a downloaded executable with zero
verification. A release that predates PlatformChecksums, or one platform
whose checksum was never backfilled, is a registry data gap to fix upstream,
not a flag to bypass here.

PlatformChecksums round-trips through the registry cache (Registry.MarshalJSON)
the same way every other field in this package must, per
TestRegistryRoundTripLosesNoField.
… policy

resolveArtifactChecksum: the source artifact always uses manifest.Checksum
(including the empty-string case, left for verifyChecksum's own leniency to
handle); a platform artifact resolves to its matching PlatformChecksums
entry, never the source checksum or a different platform's; a platform
artifact with no matching entry is refused regardless of
NSELF_PLUGIN_REQUIRE_CHECKSUM, a present source checksum, or whether the map
is nil versus just missing that one key.

Three end-to-end tests exercise the same policy through verifyChecksum: a
correct platform checksum passes, a mismatched one is rejected (mirroring
the existing source-checksum mismatch test), and a missing one never reaches
verifyChecksum's lenient empty-string branch at all.
…ile cap

The per-artifact checksum change pushed this file to 309 lines, past the
engineering-standard cap enforced by internal/repoqa's
TestFileSizeBudgetNotExceeded (budget: 0 files allowed over). Tightened the
new Step 5 comments and removed a pre-existing duplicated sentence in the
Step 4 comment — no behavior change, same logic, back to 300 lines exactly.
…volume mounts

nself build could not express a pinned image digest, injected SMTP env
vars, or an extra volume mount for a custom service, so any stack
needing them had to be hand-authored into a docker-compose.override.yml
that nself build would silently drop on a rebuild (G-013).

Adds three env vars to the CS_N model, all declarable in .env:

- CS_N_IMAGE: run a pre-built image (optionally digest-pinned via
  @sha256:...) instead of building from a Dockerfile. Mutually
  exclusive with CS_N_PATH.
- CS_N_ENV_FILE: inject KEY=VALUE pairs from a dotenv-format file,
  applied after CS_N_ENV_PASSTHROUGH and before CS_N_ENV. Handles
  values CS_N_ENV's comma-joined format cannot represent safely (e.g.
  a password containing a comma), and many-var cases like SMTP config.
- CS_N_VOLUMES: comma-separated host:container[:mode] bind mounts,
  appended to the generated service.

Generator gains WithWorkDir to anchor CS_N_ENV_FILE reads to the
project root; the build orchestrator now threads st.workdir through so
CS_N_ENV_FILE resolves correctly regardless of invocation directory.
buildCustomService and Generate() now return an error on a missing/
unreadable CS_N_ENV_FILE rather than silently omitting the vars.
@acamarata
acamarata merged commit 20c018b into main Sep 12, 2026
31 checks passed
@acamarata
acamarata deleted the p6/g013-service-digest-env-mounts branch September 12, 2026 14:09
@acamarata acamarata mentioned this pull request Sep 12, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant