From 67c004bee71001910d64ce3c20ebe703cc651beb Mon Sep 17 00:00:00 2001 From: smoochy <34371932+smoochy@users.noreply.github.com> Date: Mon, 24 Aug 2026 00:16:32 +0200 Subject: [PATCH 1/7] ci(caddy-workflow): assert produced tag set on pull requests Add a step after the metadata step that fails the job when the tag set produced by docker/metadata-action does not match the expected set for the current caddy_tag, so a change to the tag logic is caught in CI instead of relying on manual review. The check runs only when the pull_request event triggers a build and compares by set membership plus count, not by exact string or line order. Co-Authored-By: Claude Opus 5 --- .../workflows/build_cloudflare-modules.yaml | 46 +++++++++++++++++++ 1 file changed, 46 insertions(+) diff --git a/.github/workflows/build_cloudflare-modules.yaml b/.github/workflows/build_cloudflare-modules.yaml index 88662f4..e259f9f 100644 --- a/.github/workflows/build_cloudflare-modules.yaml +++ b/.github/workflows/build_cloudflare-modules.yaml @@ -541,6 +541,52 @@ jobs: type=raw,value=caddy-${{ steps.decide.outputs.caddy_tag }} type=raw,value=caddy-v${{ steps.decide.outputs.caddy_tag }},enable=${{ steps.decide.outputs.caddy_tag != 'latest' }} + - name: Assert produced tag set + if: github.event_name == 'pull_request' && steps.decide.outputs.do_build == 'true' + shell: bash + env: + META_TAGS: ${{ steps.meta.outputs.tags }} + CADDY_TAG: ${{ steps.decide.outputs.caddy_tag }} + run: | + set -euo pipefail + + declare -a expected=() + if [ "$CADDY_TAG" = "latest" ]; then + expected=("latest" "caddy-latest") + else + expected=("latest" "caddy-${CADDY_TAG}" "caddy-v${CADDY_TAG}") + fi + + declare -a produced=() + while IFS= read -r line; do + [ -z "$line" ] && continue + produced+=("${line##*:}") + done <<< "$META_TAGS" + + ok=true + if [ "${#produced[@]}" -ne "${#expected[@]}" ]; then + ok=false + fi + for tag in "${expected[@]}"; do + found=false + for p in "${produced[@]}"; do + if [ "$p" = "$tag" ]; then + found=true + break + fi + done + if [ "$found" != "true" ]; then + ok=false + fi + done + + if [ "$ok" != "true" ]; then + echo "Tag set assertion failed for caddy_tag=${CADDY_TAG}" >&2 + echo "Expected: ${expected[*]}" >&2 + echo "Produced: ${produced[*]}" >&2 + exit 1 + fi + - name: Prepare addon labels if: steps.decide.outputs.do_build == 'true' id: addon_labels From 1869195e5a17eb15e7e170fddd3b39f18524cce4 Mon Sep 17 00:00:00 2001 From: smoochy <34371932+smoochy@users.noreply.github.com> Date: Mon, 24 Aug 2026 00:23:19 +0200 Subject: [PATCH 2/7] docs: record assembly-line wave 1 for the workflow verification run Co-Authored-By: Claude Opus 5 --- .../2026-08-24-workflow-verification/plan.md | 102 ++++++++++++++++++ .../run-record.md | 79 ++++++++++++++ 2 files changed, 181 insertions(+) create mode 100644 docs/assembly-line/2026-08-24-workflow-verification/plan.md create mode 100644 docs/assembly-line/2026-08-24-workflow-verification/run-record.md diff --git a/docs/assembly-line/2026-08-24-workflow-verification/plan.md b/docs/assembly-line/2026-08-24-workflow-verification/plan.md new file mode 100644 index 0000000..9989966 --- /dev/null +++ b/docs/assembly-line/2026-08-24-workflow-verification/plan.md @@ -0,0 +1,102 @@ +# Plan: give the build workflow an executable verification path + +## Goal + +`.github/workflows/build_cloudflare-modules.yaml` cannot be executed locally, so every change to it has been verified statically: a YAML parse, `shellcheck` on the extracted `run:` block bodies, and a manual trace of each possible value through its consumers. Nothing has exercised a live Actions run, a real `crane copy` or a real `docker/metadata-action` invocation. This run decides what verification the repository wants, adds the one executable check that is cheap and covers the class of change the workflow actually receives, and writes the resulting standard down so that future changes do not re-litigate it. + +## Source brief + +GitHub issue adapter, , "The build workflow has no executable verification path". Body verbatim: + +> Follow-up from assembly-line run `2026-08-23-upstream-v-tag` (PR #18). +> +> `build_cloudflare-modules.yaml` cannot be executed locally, so every verification in that run was static: a YAML parse, `shellcheck` on the extracted `run:` block bodies, and a manual trace of both possible `caddy_tag` values through each consumer. Nothing exercised a live Actions run, a real `crane copy` or a real `docker/metadata-action` invocation. Both inspectors of the workflow package raised this caveat independently, and the final review judged it non-blocking for that PR because the change reuses a guard the existing `caddy-` tag already proves in production. +> +> The open question is whether this repository wants a local dry-run path for its build workflow at all. Candidates, none evaluated yet: +> +> - `act` for a local Actions run, at the cost of a Docker-in-Docker setup that will not reproduce the multi-arch push. +> - A pull-request-scoped job that runs the `decide` and `meta` steps with `push: false` and asserts the resulting tag list, which would cover exactly the class of change this PR made. +> - Accepting static verification as the standard for this workflow and writing that down, so future changes do not re-litigate it. + +## Context the builder needs + +- The build workflow is `.github/workflows/build_cloudflare-modules.yaml`. +- It already triggers on `pull_request` against `main`, filtered to the paths `Dockerfile-cloudflare`, `.dockerignore` and the workflow file itself. +- It already builds without publishing on a pull request: `docker/build-push-action@v7` receives `push: ${{ github.event_name != 'pull_request' }}` and a single-platform `platforms` value. So the "runs with `push: false`" half of the brief's second candidate exists today. What is missing is any assertion on the result. +- The `decide` step emits `caddy_tag`, which has exactly two possible shapes: a three-part semver such as `2.11.4`, or the literal string `latest` when neither the image label nor the image history yields a semver. +- The `meta` step is `docker/metadata-action@v6`. It runs only under `if: steps.decide.outputs.do_build == 'true'` and lists three raw tags: `latest`, `caddy-`, and `caddy-v` enabled only when `caddy_tag != 'latest'`. +- `do_build` is not always `true` on a pull request. The trigger's path filter evaluates the diff of the pushed commits, while the `Classify local changes` step independently diffs `PR_BASE_SHA` against `PR_HEAD_SHA`. A pull request whose later commit reverts an earlier change to a filtered path re-triggers the workflow while `image_inputs_changed` and `workflow_changed` are both false, so with no upstream change `do_build` is `false` and the `meta` step is skipped. Any new step that reads `steps.meta.outputs.tags` must handle that state. +- `steps.meta.outputs.tags` is a newline-separated list of fully qualified references, each of the shape `ghcr.io/smoochy/caddy-cloudflare-modules:`. +- The `decide` step needs the network. It calls `crane` against `caddy:latest` and the GitHub API for release metadata, so it cannot run offline, which is what rules the workflow out of a genuinely local dry run. + +## Package 1 - the pull request run asserts the tag list it produced + +**Outcome**: a pull request that changes the tag logic fails in CI when the produced tag set is wrong, instead of relying on a human tracing the branches by hand. + +**Declared file scope**: `.github/workflows/build_cloudflare-modules.yaml` + +**Point from the brief**: "A pull-request-scoped job that runs the `decide` and `meta` steps with `push: false` and asserts the resulting tag list, which would cover exactly the class of change this PR made." + +**Acceptance claim**: + +1. A new assertion step is added after the `meta` step and before the `Build and push` step, guarded by `if: github.event_name == 'pull_request' && steps.decide.outputs.do_build == 'true'`. +2. When that guard holds, the step reads `steps.meta.outputs.tags` and `steps.decide.outputs.caddy_tag` and fails the job unless the produced tag set is exactly the expected set for that `caddy_tag`: `latest`, `caddy-` and `caddy-v` when `caddy_tag` is a three-part semver, and `latest` and `caddy-latest` when `caddy_tag` is the literal string `latest`. +3. The comparison is by set membership plus a count, in either order. It is never an exact string comparison of the whole rendered list and never depends on line order, so a change in how `docker/metadata-action` formats its output cannot fail the job on its own. +4. When `steps.decide.outputs.do_build` is `false` on a `pull_request` event, the `meta` step is skipped and `steps.meta.outputs.tags` is empty. The assertion step is skipped by the same guard and does not fail the job. This is required behaviour, not an oversight. +5. On `push`, `schedule` and `workflow_dispatch` events the assertion step never runs, so no published build can be failed by it. +6. The step adds no permission, no secret, no new third-party action, and no change to push, tag, label or mirror behaviour on any event. +7. The failure output names the expected set and the produced set, so a failing run is diagnosable from the log alone. +8. The workflow file remains valid YAML and the new shell block is `set -euo pipefail` clean. + +## Package 3 - the `act` candidate is evaluated + +**Outcome**: the repository has a recorded ruling on `act`, so the option is closed rather than left open. + +**Declared file scope**: none. This package produces a ruling, not a diff. The ruling is carried into the section Package 2 writes. + +**Point from the brief**: "`act` for a local Actions run, at the cost of a Docker-in-Docker setup that will not reproduce the multi-arch push." + +**Acceptance claim**: + +1. The run states whether `act` is adopted or rejected, with reasons drawn from this repository rather than from general argument. +2. The reasons address at minimum: that the `decide` step needs network access to `crane` and the GitHub API, so a local run is not hermetic; that `act` cannot reproduce the multi-arch `docker/build-push-action` push, which is the part no static check covers; and what a local `act` run would therefore actually prove. +3. The ruling appears in the run record and in the section Package 2 writes. + +## Package 2 - the repository states its verification standard for this workflow + +**Outcome**: a contributor changing this workflow reads what is checked where, and what is not checked before merge, without re-opening the question. + +**Declared file scope**: `README.md` (a new section), `CLAUDE.md` (one pointer line, appended outside the `OPENWIKI:START` and `OPENWIKI:END` markers) + +**Point from the brief**: "Accepting static verification as the standard for this workflow and writing that down, so future changes do not re-litigate it." + +**Acceptance claim**: + +1. The new `README.md` section states what each verification layer covers: the pull request tag assertion Package 1 adds, the static checks a change to this workflow is expected to run locally, and the live scheduled run. +2. It states explicitly what is not verified before a merge: a real multi-arch push, a real `crane copy` to Docker Hub, and a real write to either registry. +3. It records the `act` ruling from Package 3 with its reason. +4. It names the static checks concretely enough to repeat them, rather than saying "static verification" and stopping. +5. It does not contradict the tag list `README.md` already documents in its "Image Tags" and "Available Images" sections, and it does not restate that list. +6. `CLAUDE.md` gains one pointer line to the new section, appended outside the OpenWiki marker block, so the generated content stays untouched. +7. No section of `README.md` other than the new one changes. +8. Both edits are Simplified Technical English, with no hard-wrapped prose and no em dashes or en dashes. + +## Waves + +- **Wave 1**: Package 1 and Package 3. +- **Wave 2**: Package 2. Runs after wave 1. + +Both waves are sequential. Package 3 touches no file, so it conflicts with nothing; wave 1 is sequential because this version of the line runs every wave sequentially, not because a shared artifact forced a fallback. Wave 2 is a content dependency rather than a file conflict: Package 2's section describes the assertion Package 1 builds and carries the ruling Package 3 makes, so writing it first risks documenting a shape the inspection then changes. + +## Triggers + +- **Shared artifacts**: no hit. No lockfile, no manifest, no snapshot directory, no formatter or linter config is in any declared scope. The repo probe read `.gitignore`, which holds the single entry `.agents/`, and the CI configs, which run no repo-wide formatter. The generated `openwiki/` tree exists but no package declares it and no package may hand-edit it. +- **Security**: no hit. No declared file scope matches the trigger vocabulary by path, and no package adds, reads or changes a secret, a permission block, a login step or a credential. The pre-approval `security-reviewer` pass is skipped and that skip is recorded. + +## Non-goals + +- No change to what the build publishes, and no change to push, tag, label or mirror behaviour on any event. +- No new third-party action, and no new permission or secret. +- No adoption of `act`, and no Docker-in-Docker setup. +- No hand-edit of the generated `openwiki/` tree. +- No cleanup of the unrelated stale `claude/actionlint-ci` worktree and its uncommitted `lint_workflows.yaml`. That is outside this brief and is proposed as a follow-up instead. diff --git a/docs/assembly-line/2026-08-24-workflow-verification/run-record.md b/docs/assembly-line/2026-08-24-workflow-verification/run-record.md new file mode 100644 index 0000000..806bed1 --- /dev/null +++ b/docs/assembly-line/2026-08-24-workflow-verification/run-record.md @@ -0,0 +1,79 @@ +# Run record: 2026-08-24-workflow-verification + +## Run header + +- **Source adapter**: GitHub issue. , "The build workflow has no executable verification path". Fetched with `gh issue view --json title,body,labels`. No `wayfinder:map` label, so the plain issue adapter applied and title plus body became the brief text. +- **Brief hash**: the issue body as of 2026-08-24, quoted verbatim in `plan.md`. +- **Token budget**: roughly 1.5M, the default. Not overridden. +- **Concurrency cap**: 4, owned by the run. +- **Run branch**: `claude/workflow-verification`, in the linked worktree `.worktrees/caddy-modules/workflow-verification`. +- **Target repo**: `smoochy/caddy-modules`. + +## The package cut + +| # | Outcome | Declared file scope | Acceptance claim | +| --- | --- | --- | --- | +| 1 | A pull request that changes the tag logic fails in CI when the produced tag set is wrong. | `.github/workflows/build_cloudflare-modules.yaml` | Eight parts, listed in full in `plan.md`. In summary: a new assertion step after `meta` and before `Build and push`, guarded on `pull_request` and on `do_build == 'true'`, comparing the produced tag set against the expected set for that `caddy_tag` by membership plus count, skipped in every other state, adding no permission, secret or action, printing both sets on failure, and leaving the file valid YAML with a `set -euo pipefail` clean block. | +| 3 | The repository has a recorded ruling on `act`. | none, the package produces a ruling rather than a diff | The run states whether `act` is adopted or rejected, with reasons drawn from this repository, and the ruling appears in this record and in the section Package 2 writes. | +| 2 | A contributor changing this workflow reads what is checked where, and what is not checked before merge. | `README.md`, `CLAUDE.md` | Eight parts, listed in full in `plan.md`. | + +The numbering follows the brief's own order of candidates, so Package 3 is the `act` candidate and runs in wave 1 while Package 2 is the documentation candidate and runs in wave 2. + +## Pre-run gates + +The cut went to a fresh `plan-verifier`, which returned **REVISE** on one blocker: Package 1's acceptance claim did not state the assertion step's behaviour when `steps.decide.outputs.do_build` is `false` on a `pull_request` event, although the `meta` step it reads is itself gated on that output. The reviewer showed a real path into that state: the trigger's path filter evaluates the diff of the pushed commits, while the `Classify local changes` step diffs `PR_BASE_SHA` against `PR_HEAD_SHA`, so a pull request whose later commit reverts an earlier change to a filtered path re-triggers the workflow with both change flags false. The controller had asserted the opposite when drafting the cut, and the reviewer refuted it. + +An advisor pass ran concurrently with the `plan-verifier` pass. It made two changes to the cut. It required the same `do_build` guard from the other direction, and it required the comparison to be set membership rather than exact list equality, so the assertion does not couple to how `docker/metadata-action` renders its output. It also moved Package 2's target from a new `docs/workflow-verification.md` to a section of the existing `README.md`, on the grounds that a repository with one build workflow and no engineering-standards document does not justify creating that layer. + +The cut was revised materially on all three points and submitted once to a fresh `plan-verifier`, which returned **READY**. That closed the second readiness epoch. The revised cut was then presented and the operator approved it, which opened the run window. + +**Security pre-pass**: skipped, and the skip is recorded here. The security trigger did not fire. No declared file scope matches the trigger vocabulary by path, and no package adds, reads or changes a secret, a permission block, a login step or a credential. + +## Waves + +| Wave | Packages | Ran | Reason | +| --- | --- | --- | --- | +| 1 | Package 1, Package 3 | Sequential | No shared-artifact trigger fired. The repo probe read `.gitignore`, which holds the single entry `.agents/`, and the CI configs, which run no repo-wide formatter; there is no lockfile, no manifest and no snapshot directory in any scope. The parallel-wave flag is off by default and was not set, and in this version it would not run real concurrency anyway. Package 3 declares no file scope, so it could not conflict with Package 1 in any case. | +| 2 | Package 2 | Sequential, after wave 1 | A content dependency, not a shared-artifact fallback: Package 2's section describes the assertion Package 1 builds and carries the ruling Package 3 makes, so writing it first risked documenting a shape the inspection then changed. | + +## Per package + +### Package 1 + +Built by SDD's generic implementer on a standard model, since the security trigger did not fire and no `security-executor` was required. One commit, `1832287`. A 46 line pure insertion in one file. + +| Inspector | Role | Verdict | +| --- | --- | --- | +| A | fresh `verifier` | **CONFIRMED**, all eight parts. It re-executed the extracted assertion logic rather than reading it, covering the four cases the brief required plus five it added itself: a trailing empty line in the tag list, which is the realistic shape of a newline separated action output; a duplicate entry whose count coincidentally matches the expected count while an expected tag is missing; an empty `caddy_tag`; a `caddy_tag` carrying a shell metacharacter, which passed safely because the value only ever reaches the script through the `env:` block and inside quotes; and the consistency of bare tag against fully qualified reference on the two sides of the comparison. It re-parsed the whole workflow with a real YAML parser and re-ran `shellcheck` clean. | +| B | `executor`, read-only correctness and simplification posture | No Critical and no Important finding. It independently re-ran `shellcheck` and six assertion cases, and it established four things the claim does not cover: that `CADDY_TAG` and `META_TAGS` never reach the script body as interpolated expressions, so the untrusted `caddy_tag` cannot arrive as code; that the `[ -z "$line" ] && continue` idiom is exempt from `set -e` under the `&&` list rule and does not exit the loop early; that the here-string feeding the read loop keeps the array alive, where a pipe would have built it in a subshell and discarded it; and that the new step's guard is mutually exclusive with the Docker Hub mirror step and does not mislead the `if: always()` build summary step. One Minor finding, deferred, listed below. | + +Both inspectors ran concurrently. No fix round opened. + +### Package 3 + +No implementer was dispatched. The package declares no file scope and produces a ruling rather than a diff, so the controller adjudicated it directly; that decision is a ledger ruling and is repeated under Ledger decisions below. The package is reported as moot in the sense the cut defines: it was a real point of the brief, it was worked, and its outcome is a recorded rejection rather than a change. + +**The ruling**: `act` is rejected. Three reasons drawn from this repository. The `decide` step calls `crane` against `caddy:latest` and the GitHub API for release metadata, so an `act` run is not hermetic. `act` cannot reproduce the multi-arch `docker/build-push-action` push or the `crane copy` mirror, which is precisely the part no static check covers, so it does not close the gap the issue names. What a local `act` run would actually prove is that the YAML parses and the shell branches execute, and `shellcheck` plus the Package 1 assertion already prove that more cheaply and in the place that gates a merge. Against that, the option costs a Docker-in-Docker setup in a repository with no other local toolchain. + +### Package 2 + +Recorded after wave 2. + +## Ledger decisions + +Every decide-and-log entry made inside the run window, in the order it was made. + +1. **No unit test is expected for any package.** The repository has no test framework, no test directory and no test runner, and the only artifacts under change are one workflow and two documents. The runnable check that replaces a test for Package 1 was named in the dispatch and required in the report: a YAML parse of the whole workflow, `shellcheck` on the new `run:` block, and a local execution of the assertion logic against both `caddy_tag` shapes plus wrong tag sets, proving it passes and fails where it should. *Cost if wrong*: the assertion ships with a logic error that only a live pull request run would surface. That is no worse than the exposure the repository has today, and strictly better, because the assertion also runs live on the very pull request that introduces it. +2. **Package 3 gets no implementer dispatch.** It declares no file scope and produces a ruling, so dispatching a subagent to write no code would add a review surface with nothing on it. *Cost if wrong*: the `act` ruling carries one person's reasoning rather than an independent builder's, and a reader who disagrees reopens the question against the reasons recorded above. +3. **`act` is rejected.** Reasons above. *Cost if wrong*: a contributor who wanted a full local rehearsal does not get one and must open a draft pull request instead, which this repository's pull request path already builds without publishing. + +## Findings outside the packages + +| Finding | Proposed cluster | +| --- | --- | +| Package 1, Minor, deferred: the tag-set check is a hand-rolled count plus nested membership scan, where a sort and compare would be materially shorter and equally strong, because the expected list never holds a duplicate. Inspector B states the current code is correct and proposes the change as a simplification only. | Not clustered. Carried to the final whole-branch review for triage rather than proposed as a follow-up, because it lives inside this run's own diff. | +| A stale linked worktree `.worktrees/caddy-modules/actionlint-ci` sits on branch `claude/actionlint-ci` and holds an untracked `.github/workflows/lint_workflows.yaml` that would run `rhysd/actionlint` on workflow changes. It was never committed and never pushed. It is outside this brief, which asks about executable verification of the build workflow rather than linting, but it is adjacent to the standard Package 2 writes down. | Cluster A, recorded in the report. | + +## Ingest status per store + +Recorded at run close. From ed5272504a2b90ec850528345b738f4abc82bd31 Mon Sep 17 00:00:00 2001 From: smoochy <34371932+smoochy@users.noreply.github.com> Date: Mon, 24 Aug 2026 00:31:26 +0200 Subject: [PATCH 3/7] docs: document the workflow verification standard Add a Workflow Verification section to README.md that states what the pull request tag assertion, the pull request build with push: false, the local static checks, and the live scheduled run each cover, and what is not verified before a merge. Record the act ruling with its reason. Add the matching table of contents entry and a pointer line in CLAUDE.md outside the OpenWiki marker block. Co-Authored-By: Claude Opus 5 --- CLAUDE.md | 2 ++ README.md | 21 +++++++++++++++++++++ 2 files changed, 23 insertions(+) diff --git a/CLAUDE.md b/CLAUDE.md index a981a34..dcd6e61 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -10,3 +10,5 @@ This repository has a generated `openwiki/` evidence index. It is optional just- The scheduled OpenWiki GitHub Actions workflow refreshes the repository wiki. Do not hand-edit generated OpenWiki pages unless explicitly asked; prefer updating source code/docs and letting OpenWiki regenerate. + +See README.md, section [Workflow Verification](README.md#workflow-verification), for what is checked before a merge to `build_cloudflare-modules.yaml` and what is not. diff --git a/README.md b/README.md index 09383ee..4d8f269 100644 --- a/README.md +++ b/README.md @@ -28,6 +28,7 @@ maintenance via Coindrop, Ko-fi, or Buy Me a Coffee. - [When Builds Run](#when-builds-run) - [Dockerfile](#dockerfile) - [GitHub Actions Workflow](#github-actions-workflow) +- [Workflow Verification](#workflow-verification) - [Job Summary](#job-summary) - [Image Metadata](#image-metadata) - [Image Tags](#image-tags) @@ -128,6 +129,26 @@ Commented-out `--with` lines are ignored by both the build and the workflow. - an upstream change is detected - a manual run is forced +## Workflow Verification + +This section states what is checked when a change to `build_cloudflare-modules.yaml` runs, and what is not checked before merge. + +A pull request build runs the workflow with `push: false`, so the image is built but never published to GHCR or Docker Hub. This exercises the Dockerfile and the addon install steps without any write to a registry. + +On a pull request, when `do_build` is `true`, the `Assert produced tag set` step reads the tag list `docker/metadata-action` produced and fails the job unless that set matches exactly what is expected for the `caddy_tag` value the `decide` step emitted. The comparison is by set membership plus a count, so it does not depend on order or formatting. This step never runs on `push`, `schedule`, or `workflow_dispatch`, so it can never fail a publishing build. When `do_build` is `false`, both the `meta` step and this assertion step are skipped. + +A change to this workflow is expected to pass these static checks locally before review: + +- Parse the whole workflow file with a YAML parser and confirm it succeeds. +- Extract the body of every `run:` block you changed to a temporary file and run `shellcheck` on it. +- Copy the shell logic you changed to a temporary file, replace the workflow expressions with shell variables, and execute it against every value the producing step can emit. For the tag logic that means both shapes of `caddy_tag`: a three-part semver such as `2.11.4`, and the literal string `latest`. + +The workflow also runs on a schedule, daily at 03:00 UTC, and on every push to `main`. Those are the runs where a real multi-arch push to GHCR and a real `crane copy` mirror to Docker Hub actually execute. + +What is not verified before a merge: a real multi-arch push, a real `crane copy` to Docker Hub, and a real write to either registry. A pull request never publishes, so those three paths are first exercised after the merge, on the next push to `main` or on the next scheduled run. + +`act` was considered for local verification of this workflow and rejected. The `decide` step calls `crane` against `caddy:latest` and the GitHub API for release metadata, so a local `act` run is not hermetic. `act` also cannot reproduce the multi-arch `docker/build-push-action` push or the `crane copy` mirror, which is exactly the part no static check covers, so it would not close that gap. What a local `act` run would actually prove is that the YAML parses and the shell branches execute, and `shellcheck` plus the pull request tag assertion already prove that more cheaply and in the place that gates a merge. Against that, `act` costs a Docker-in-Docker setup in a repository that has no other local toolchain. + ## Job Summary Every workflow run writes a summary that includes: From 2bf6763396ac49fcb45e7b3eb1c24de2e537224f Mon Sep 17 00:00:00 2001 From: smoochy <34371932+smoochy@users.noreply.github.com> Date: Mon, 24 Aug 2026 00:38:28 +0200 Subject: [PATCH 4/7] docs: fix inaccurate publish claims in Workflow Verification section The section claimed every scheduled run and every push to main does a real multi-arch publish and crane copy mirror. That is false: a push only forces a build when Dockerfile-cloudflare or .dockerignore changed, a scheduled run never forces one, and the decide step still gates a real publish behind detecting an upstream change either way. Rewrite both paragraphs to say what the decide step and the mirror step's condition actually guarantee, and drop the duplicated cron time in favor of pointing at "When Builds Run". Co-Authored-By: Claude Opus 5 --- README.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/README.md b/README.md index 4d8f269..990aae1 100644 --- a/README.md +++ b/README.md @@ -143,9 +143,9 @@ A change to this workflow is expected to pass these static checks locally before - Extract the body of every `run:` block you changed to a temporary file and run `shellcheck` on it. - Copy the shell logic you changed to a temporary file, replace the workflow expressions with shell variables, and execute it against every value the producing step can emit. For the tag logic that means both shapes of `caddy_tag`: a three-part semver such as `2.11.4`, and the literal string `latest`. -The workflow also runs on a schedule, daily at 03:00 UTC, and on every push to `main`. Those are the runs where a real multi-arch push to GHCR and a real `crane copy` mirror to Docker Hub actually execute. +The workflow also runs on the schedule described in "When Builds Run", and on a push to `main` that touches `Dockerfile-cloudflare`, `.dockerignore`, or the workflow file itself. Neither run publishes on its own. The `decide` step still has to judge a build necessary: a push forces one only when it changed `Dockerfile-cloudflare` or `.dockerignore`, a scheduled run never forces one, and either kind of run still builds and publishes when it detects an upstream Caddy or addon change. Only once `decide` outputs `do_build: true` on a `push`, `schedule`, or `workflow_dispatch` event does the real multi-arch push to GHCR happen, and the `crane copy` mirror to Docker Hub runs on top of that only when Docker Hub secrets are configured. -What is not verified before a merge: a real multi-arch push, a real `crane copy` to Docker Hub, and a real write to either registry. A pull request never publishes, so those three paths are first exercised after the merge, on the next push to `main` or on the next scheduled run. +What is not verified before a merge: a real multi-arch push, a real `crane copy` to Docker Hub, and a real write to either registry. A pull request never publishes, so those three paths can only happen after the merge, and only on a later push or scheduled run where `decide` judges a build necessary. `act` was considered for local verification of this workflow and rejected. The `decide` step calls `crane` against `caddy:latest` and the GitHub API for release metadata, so a local `act` run is not hermetic. `act` also cannot reproduce the multi-arch `docker/build-push-action` push or the `crane copy` mirror, which is exactly the part no static check covers, so it would not close that gap. What a local `act` run would actually prove is that the YAML parses and the shell branches execute, and `shellcheck` plus the pull request tag assertion already prove that more cheaply and in the place that gates a merge. Against that, `act` costs a Docker-in-Docker setup in a repository that has no other local toolchain. From 76e9f4e643038d422376cb87800fd675dc13cc36 Mon Sep 17 00:00:00 2001 From: smoochy <34371932+smoochy@users.noreply.github.com> Date: Mon, 24 Aug 2026 00:42:54 +0200 Subject: [PATCH 5/7] docs: record assembly-line wave 2 for the workflow verification run Co-Authored-By: Claude Opus 5 --- .../run-record.md | 15 +++++++++++++-- 1 file changed, 13 insertions(+), 2 deletions(-) diff --git a/docs/assembly-line/2026-08-24-workflow-verification/run-record.md b/docs/assembly-line/2026-08-24-workflow-verification/run-record.md index 806bed1..14c4919 100644 --- a/docs/assembly-line/2026-08-24-workflow-verification/run-record.md +++ b/docs/assembly-line/2026-08-24-workflow-verification/run-record.md @@ -57,7 +57,16 @@ No implementer was dispatched. The package declares no file scope and produces a ### Package 2 -Recorded after wave 2. +Built by SDD's generic implementer on a standard model. Two commits, `7da284e` and the fix `d5adf42`. A new "Workflow Verification" section in `README.md`, one table of contents entry, and one pointer line in `CLAUDE.md` outside the OpenWiki marker block. + +| Inspector | Role | Verdict | +| --- | --- | --- | +| A | fresh `verifier` | **CONFIRMED**, all eight parts including the amended part 7. It checked every part against the working tree rather than against the implementer's report: it extracted the headings and the table of contents entries with its own commands and compared both lists one to one, it confirmed the diff touches nothing in `README.md` outside the new section and the single new table of contents line, it resolved the heading anchor against both the table of contents link and the `CLAUDE.md` pointer, it searched the added lines for em dash and en dash bytes, and it re-read the workflow to confirm the section's statements about the assertion step's guard, the `push: false` behaviour on pull requests and the schedule. | +| B | `executor`, read-only correctness and simplification posture | No Critical. **Two Important and one Minor**, all inside this run's own diff and all in the same paragraph pair. First, the section claimed the workflow runs "on every push to `main`", which ignores the push trigger's `paths` filter and contradicts what the existing "When Builds Run" section of the same file already says. Second, it claimed a real multi-arch push and a real `crane copy` mirror "actually execute" on the next push or scheduled run, where in truth a push run forces a build only when `image_inputs_changed` is true, a scheduled run never forces one, a real publish needs `do_build` true, and the mirror needs `dockerhub_enabled` true. The Minor: "daily at 03:00 UTC" was duplicated from "When Builds Run", giving the cron value two places to rot instead of one. | + +Both inspectors ran concurrently. One fix round opened and closed it: commit `d5adf42` rewrote the two overstated sentences to say what the workflow actually guarantees and replaced the duplicated cron value with a pointer to the section that already carries it. A scoped re-review on a mid-tier model returned **ADDRESSED** on all three findings with no new finding, confirming each against the workflow file rather than against the fix brief. + +The second Important finding is the more interesting one for this run's own subject. The package documenting the repository's verification standard was itself wrong about when the unverified paths first run, and a fresh inspector caught it by reading the `decide` step's `FORCED` logic. That is the same class of defect the pre-run gate caught in Package 1 and the same class the new assertion step now catches in CI. ## Ledger decisions @@ -65,7 +74,9 @@ Every decide-and-log entry made inside the run window, in the order it was made. 1. **No unit test is expected for any package.** The repository has no test framework, no test directory and no test runner, and the only artifacts under change are one workflow and two documents. The runnable check that replaces a test for Package 1 was named in the dispatch and required in the report: a YAML parse of the whole workflow, `shellcheck` on the new `run:` block, and a local execution of the assertion logic against both `caddy_tag` shapes plus wrong tag sets, proving it passes and fails where it should. *Cost if wrong*: the assertion ships with a logic error that only a live pull request run would surface. That is no worse than the exposure the repository has today, and strictly better, because the assertion also runs live on the very pull request that introduces it. 2. **Package 3 gets no implementer dispatch.** It declares no file scope and produces a ruling, so dispatching a subagent to write no code would add a review surface with nothing on it. *Cost if wrong*: the `act` ruling carries one person's reasoning rather than an independent builder's, and a reader who disagrees reopens the question against the reasons recorded above. -3. **`act` is rejected.** Reasons above. *Cost if wrong*: a contributor who wanted a full local rehearsal does not get one and must open a draft pull request instead, which this repository's pull request path already builds without publishing. +3. **`README.md` carries a hand-maintained table of contents, so Package 2's acceptance claim 7 was amended.** The claim said no section other than the new one changes. It was amended inside the run window to permit exactly one further change, a single new table of contents entry at the position matching the new section's place in the file. Without the amendment the package would have shipped a table of contents that omits the section it adds. *Cost if wrong*: one extra line in a list that is already maintained by hand, visible in the diff and trivially revertible. +4. **No `CHANGELOG.md` is created or updated by this run.** The operator's standing rule says a change to a script, an asset or code updates `CHANGELOG.md`. This repository has never had one, the earlier assembly-line run in it created none, and a changelog whose first entry is an internal CI assertion would misrepresent the repository's history to the people who read it, who are image consumers rather than contributors. *Cost if wrong*: one file is missing that a later run can create with a proper first entry covering the releases that preceded it. +5. **`act` is rejected.** Reasons above. *Cost if wrong*: a contributor who wanted a full local rehearsal does not get one and must open a draft pull request instead, which this repository's pull request path already builds without publishing. ## Findings outside the packages From 2a24175056ef64ce525a1dfc82a7937979b43450 Mon Sep 17 00:00:00 2001 From: smoochy <34371932+smoochy@users.noreply.github.com> Date: Mon, 24 Aug 2026 00:51:07 +0200 Subject: [PATCH 6/7] docs: reconcile workflow-verification docs with the merged CI lint layer README.md's Workflow Verification section now names the actionlint CI job from PR #23 and reframes the local static checks as a fast pre-review pass rather than the only gate. The run-record's stale-worktree finding now records what was observed and when, and that it resolved on its own via PR #23 once this branch was rebased onto main, instead of asserting a now-false current state. Co-Authored-By: Claude Opus 5 --- README.md | 2 ++ .../2026-08-24-workflow-verification/run-record.md | 2 +- 2 files changed, 3 insertions(+), 1 deletion(-) diff --git a/README.md b/README.md index 990aae1..a7a9efb 100644 --- a/README.md +++ b/README.md @@ -143,6 +143,8 @@ A change to this workflow is expected to pass these static checks locally before - Extract the body of every `run:` block you changed to a temporary file and run `shellcheck` on it. - Copy the shell logic you changed to a temporary file, replace the workflow expressions with shell variables, and execute it against every value the producing step can emit. For the tag logic that means both shapes of `caddy_tag`: a three-part semver such as `2.11.4`, and the literal string `latest`. +These local checks are a fast pre-review pass, not the only gate: the `Lint workflows` GitHub Actions workflow runs `actionlint` in CI on any pull request that touches `.github/workflows/**`, and again on every push to `main`. `actionlint` covers unknown context properties, invalid expressions, and undefined step outputs across the whole workflow file, and it also runs `shellcheck` over every `run:` block, so the shellcheck pass a contributor ran locally is checked again automatically before merge. + The workflow also runs on the schedule described in "When Builds Run", and on a push to `main` that touches `Dockerfile-cloudflare`, `.dockerignore`, or the workflow file itself. Neither run publishes on its own. The `decide` step still has to judge a build necessary: a push forces one only when it changed `Dockerfile-cloudflare` or `.dockerignore`, a scheduled run never forces one, and either kind of run still builds and publishes when it detects an upstream Caddy or addon change. Only once `decide` outputs `do_build: true` on a `push`, `schedule`, or `workflow_dispatch` event does the real multi-arch push to GHCR happen, and the `crane copy` mirror to Docker Hub runs on top of that only when Docker Hub secrets are configured. What is not verified before a merge: a real multi-arch push, a real `crane copy` to Docker Hub, and a real write to either registry. A pull request never publishes, so those three paths can only happen after the merge, and only on a later push or scheduled run where `decide` judges a build necessary. diff --git a/docs/assembly-line/2026-08-24-workflow-verification/run-record.md b/docs/assembly-line/2026-08-24-workflow-verification/run-record.md index 14c4919..067ba6e 100644 --- a/docs/assembly-line/2026-08-24-workflow-verification/run-record.md +++ b/docs/assembly-line/2026-08-24-workflow-verification/run-record.md @@ -83,7 +83,7 @@ Every decide-and-log entry made inside the run window, in the order it was made. | Finding | Proposed cluster | | --- | --- | | Package 1, Minor, deferred: the tag-set check is a hand-rolled count plus nested membership scan, where a sort and compare would be materially shorter and equally strong, because the expected list never holds a duplicate. Inspector B states the current code is correct and proposes the change as a simplification only. | Not clustered. Carried to the final whole-branch review for triage rather than proposed as a follow-up, because it lives inside this run's own diff. | -| A stale linked worktree `.worktrees/caddy-modules/actionlint-ci` sits on branch `claude/actionlint-ci` and holds an untracked `.github/workflows/lint_workflows.yaml` that would run `rhysd/actionlint` on workflow changes. It was never committed and never pushed. It is outside this brief, which asks about executable verification of the build workflow rather than linting, but it is adjacent to the standard Package 2 writes down. | Cluster A, recorded in the report. | +| Observed during this run window: a stale linked worktree `.worktrees/caddy-modules/actionlint-ci` on branch `claude/actionlint-ci` held an untracked `.github/workflows/lint_workflows.yaml` that would run `rhysd/actionlint` on workflow changes, never committed and never pushed at the time of observation. It was outside this brief, which asks about executable verification of the build workflow rather than linting, but it was adjacent to the standard Package 2 writes down. This resolved on its own while the run was in flight: the same `lint_workflows.yaml` was committed and merged into `main` as PR #23, and this branch has since been rebased onto that merge, so the file exists on `main` today and the stale worktree no longer holds the only copy. | Cluster A, recorded in the report. | ## Ingest status per store From 5d5f65e9a5cbb0394d8f22c8ddedd9de1e116bb5 Mon Sep 17 00:00:00 2001 From: smoochy <34371932+smoochy@users.noreply.github.com> Date: Mon, 24 Aug 2026 00:54:14 +0200 Subject: [PATCH 7/7] docs: add the assembly-line run report and close the run record Co-Authored-By: Claude Opus 5 --- .../report.md | 52 +++++++++++++++++++ .../run-record.md | 9 +++- 2 files changed, 60 insertions(+), 1 deletion(-) create mode 100644 docs/assembly-line/2026-08-24-workflow-verification/report.md diff --git a/docs/assembly-line/2026-08-24-workflow-verification/report.md b/docs/assembly-line/2026-08-24-workflow-verification/report.md new file mode 100644 index 0000000..c4bfa01 --- /dev/null +++ b/docs/assembly-line/2026-08-24-workflow-verification/report.md @@ -0,0 +1,52 @@ +# Report: 2026-08-24-workflow-verification + +## What the run was asked to do + +GitHub issue asked one question and listed three candidate answers. The question: does this repository want a local dry-run path for its build workflow at all. The three candidates: `act` for a local Actions run, a pull-request-scoped job that asserts the tag list, and accepting static verification as the standard and writing it down. + +## What the run did + +The run cut the brief into three numbered packages, one per candidate, and ran them in two waves. + +**Package 1** added an `Assert produced tag set` step to `.github/workflows/build_cloudflare-modules.yaml`. On a pull request, and only when the `decide` step says a build is needed, the step reads the tag list `docker/metadata-action` produced and fails the job unless that set matches what the `caddy_tag` value implies. It compares by set membership plus a count, so a change in how the action renders its output cannot fail the job on its own. It never runs on `push`, `schedule` or `workflow_dispatch`, so it cannot fail a publishing build. + +**Package 3** evaluated `act` and rejected it. The `decide` step calls `crane` against `caddy:latest` and the GitHub API, so a local run is not hermetic. `act` cannot reproduce the multi-arch push or the `crane copy` mirror, which is the one part no static check covers. What a local run would prove is already proven more cheaply by the static checks and by the new assertion, and in the place that gates a merge. + +**Package 2** wrote the standard into `README.md` as a new "Workflow Verification" section, with one table of contents entry and one pointer line in `CLAUDE.md`. The section names each verification layer, names the static checks concretely enough to repeat them, records the `act` ruling with its reason, and states plainly what is not verified before a merge: a real multi-arch push, a real `crane copy` to Docker Hub, and a real write to either registry. + +## Closing pass against the brief + +Every point of the brief is assigned to a package and every package is finished. + +| Point in the brief | Package | State | +| --- | --- | --- | +| "`act` for a local Actions run" | 3 | Finished. Rejected, with reasons recorded in the run record and in the README section. | +| "A pull-request-scoped job that runs the `decide` and `meta` steps with `push: false` and asserts the resulting tag list" | 1 | Finished and shipped. Reduced to the assertion alone, because the pull request build with `push: false` already existed. | +| "Accepting static verification as the standard for this workflow and writing that down" | 2 | Finished and shipped, with one amendment and one fix round. | +| The framing question, "whether this repository wants a local dry-run path for its build workflow at all" | 3 and 2 together | Answered. The answer is no local dry-run path, and the reason is now written down where a contributor reads it. | + +Nothing in the brief is left open. + +## What the inspections caught + +Every package had two inspectors, always concurrent: a fresh `verifier` on the exact acceptance claim, and a second reviewer in a read-only correctness and simplification posture. + +The inspections earned their place three times, and each time on the same class of defect: a claim that was plausible and wrong. + +Before the run, the `plan-verifier` refuted the controller's own assertion that `do_build` is always `true` on a pull request. It is not: the trigger's path filter evaluates the pushed commits' diff while the `Classify local changes` step diffs base to head, so a pull request whose later commit reverts an earlier change re-triggers the workflow with both change flags false. That refutation is why the assertion step carries a `do_build` guard rather than crashing on an empty tag list. + +During wave 2, inspector B found that the section documenting the verification standard was itself wrong about the workflow: it claimed the workflow runs on every push to `main`, ignoring the path filter, and it claimed a real push and mirror "actually execute" on the next push or scheduled run, when in truth a push forces a build only on an image-input change, a scheduled run never forces one, and the mirror needs Docker Hub secrets. One fix round corrected both. + +At the whole-branch review, the reviewer found that the branch predated PR #23, which added actionlint to `main` while this run was in flight. The README section therefore presented a set of checks as a local responsibility without naming the CI layer that now runs them automatically, and the run record asserted as current a fact that had since resolved. The branch was rebased and both were corrected. + +## What was deferred + +The tag-set check is a hand-rolled count plus a nested membership scan, where a sort and compare would be shorter. The whole-branch reviewer agreed with deferring it and showed why the two forms are equally strong here: the expected list is provably free of duplicates, so "same count and every expected element present" already forces set equality. It is a style simplification with no behaviour change and no residual risk. + +## Follow-up proposed + +The line proposes follow-ups and files nothing. + +**Cluster A is withdrawn.** It held the stale `claude/actionlint-ci` worktree and its uncommitted `lint_workflows.yaml`. That work merged into `main` as PR #23 during this run, so there is nothing left to propose. The worktree itself can be removed whenever its owner is done with it, which is housekeeping rather than a follow-up. + +No other follow-up is proposed. The brief is fully answered and the deferred simplification is not worth a ticket. diff --git a/docs/assembly-line/2026-08-24-workflow-verification/run-record.md b/docs/assembly-line/2026-08-24-workflow-verification/run-record.md index 067ba6e..bfe26c9 100644 --- a/docs/assembly-line/2026-08-24-workflow-verification/run-record.md +++ b/docs/assembly-line/2026-08-24-workflow-verification/run-record.md @@ -85,6 +85,13 @@ Every decide-and-log entry made inside the run window, in the order it was made. | Package 1, Minor, deferred: the tag-set check is a hand-rolled count plus nested membership scan, where a sort and compare would be materially shorter and equally strong, because the expected list never holds a duplicate. Inspector B states the current code is correct and proposes the change as a simplification only. | Not clustered. Carried to the final whole-branch review for triage rather than proposed as a follow-up, because it lives inside this run's own diff. | | Observed during this run window: a stale linked worktree `.worktrees/caddy-modules/actionlint-ci` on branch `claude/actionlint-ci` held an untracked `.github/workflows/lint_workflows.yaml` that would run `rhysd/actionlint` on workflow changes, never committed and never pushed at the time of observation. It was outside this brief, which asks about executable verification of the build workflow rather than linting, but it was adjacent to the standard Package 2 writes down. This resolved on its own while the run was in flight: the same `lint_workflows.yaml` was committed and merged into `main` as PR #23, and this branch has since been rebased onto that merge, so the file exists on `main` today and the stale worktree no longer holds the only copy. | Cluster A, recorded in the report. | +## Whole-branch review + +One fresh reviewer on the most capable model read the whole branch against `main` without the plan. No Critical. One Important: the branch predated PR #23, which merged actionlint into `main` while this run was in flight, so the new README section presented the static checks as a local responsibility without naming the CI layer that now runs them, and the run record asserted as current a fact that had resolved. The branch was rebased onto `main` and both were corrected in one commit. A scoped re-review returned ADDRESSED on both halves with no new finding. + +The reviewer also triaged the deferred Minor from Package 1 and agreed with the deferral, showing that the hand-rolled count plus membership scan is exactly as strong as a sort and compare here, because the expected list is provably free of duplicates. + ## Ingest status per store -Recorded at run close. +- **mengram**: one checkpoint at the close of wave 1 and one at run close, each carrying the run's decisions and the two facts worth keeping: that `do_build` is not always true on a pull request in this workflow, and that `README.md` carries a hand-maintained table of contents. +- **ADR store**: one `manage_adr` call, get first, merge in memory, then update. Never a blind update.