ci: assert the produced tag set on pull requests and write down the verification standard

.github/workflows/build_cloudflare-modules.yaml

@@ -541,6 +541,52 @@ jobs:
             type=raw,value=caddy-${{ steps.decide.outputs.caddy_tag }}
             type=raw,value=caddy-v${{ steps.decide.outputs.caddy_tag }},enable=${{ steps.decide.outputs.caddy_tag != 'latest' }}
 
+      - name: Assert produced tag set
+        if: github.event_name == 'pull_request' && steps.decide.outputs.do_build == 'true'
+        shell: bash
+        env:
+          META_TAGS: ${{ steps.meta.outputs.tags }}
+          CADDY_TAG: ${{ steps.decide.outputs.caddy_tag }}
+        run: |
+          set -euo pipefail
+
+          declare -a expected=()
+          if [ "$CADDY_TAG" = "latest" ]; then
+            expected=("latest" "caddy-latest")
+          else
+            expected=("latest" "caddy-${CADDY_TAG}" "caddy-v${CADDY_TAG}")
+          fi
+
+          declare -a produced=()
+          while IFS= read -r line; do
+            [ -z "$line" ] && continue
+            produced+=("${line##*:}")
+          done <<< "$META_TAGS"
+
+          ok=true
+          if [ "${#produced[@]}" -ne "${#expected[@]}" ]; then
+            ok=false
+          fi
+          for tag in "${expected[@]}"; do
+            found=false
+            for p in "${produced[@]}"; do
+              if [ "$p" = "$tag" ]; then
+                found=true
+                break
+              fi
+            done
+            if [ "$found" != "true" ]; then
+              ok=false
+            fi
+          done
+
+          if [ "$ok" != "true" ]; then
+            echo "Tag set assertion failed for caddy_tag=${CADDY_TAG}" >&2
+            echo "Expected: ${expected[*]}" >&2
+            echo "Produced: ${produced[*]}" >&2
+            exit 1
+          fi
+
       - name: Prepare addon labels
         if: steps.decide.outputs.do_build == 'true'
         id: addon_labels

CLAUDE.md

@@ -10,3 +10,5 @@ This repository has a generated `openwiki/` evidence index. It is optional just-
 The scheduled OpenWiki GitHub Actions workflow refreshes the repository wiki. Do not hand-edit generated OpenWiki pages unless explicitly asked; prefer updating source code/docs and letting OpenWiki regenerate.
 
 <!-- OPENWIKI:END -->
+
+See README.md, section [Workflow Verification](README.md#workflow-verification), for what is checked before a merge to `build_cloudflare-modules.yaml` and what is not.

README.md

@@ -28,6 +28,7 @@ maintenance via Coindrop, Ko-fi, or Buy Me a Coffee.
 - [When Builds Run](#when-builds-run)
 - [Dockerfile](#dockerfile)
 - [GitHub Actions Workflow](#github-actions-workflow)
+- [Workflow Verification](#workflow-verification)
 - [Job Summary](#job-summary)
 - [Image Metadata](#image-metadata)
 - [Image Tags](#image-tags)
@@ -128,6 +129,28 @@ Commented-out `--with` lines are ignored by both the build and the workflow.
    - an upstream change is detected
    - a manual run is forced
 
+## Workflow Verification
+
+This section states what is checked when a change to `build_cloudflare-modules.yaml` runs, and what is not checked before merge.
+
+A pull request build runs the workflow with `push: false`, so the image is built but never published to GHCR or Docker Hub. This exercises the Dockerfile and the addon install steps without any write to a registry.
+
+On a pull request, when `do_build` is `true`, the `Assert produced tag set` step reads the tag list `docker/metadata-action` produced and fails the job unless that set matches exactly what is expected for the `caddy_tag` value the `decide` step emitted. The comparison is by set membership plus a count, so it does not depend on order or formatting. This step never runs on `push`, `schedule`, or `workflow_dispatch`, so it can never fail a publishing build. When `do_build` is `false`, both the `meta` step and this assertion step are skipped.
+
+A change to this workflow is expected to pass these static checks locally before review:
+
+- Parse the whole workflow file with a YAML parser and confirm it succeeds.
+- Extract the body of every `run:` block you changed to a temporary file and run `shellcheck` on it.
+- Copy the shell logic you changed to a temporary file, replace the workflow expressions with shell variables, and execute it against every value the producing step can emit. For the tag logic that means both shapes of `caddy_tag`: a three-part semver such as `2.11.4`, and the literal string `latest`.
+
+These local checks are a fast pre-review pass, not the only gate: the `Lint workflows` GitHub Actions workflow runs `actionlint` in CI on any pull request that touches `.github/workflows/**`, and again on every push to `main`. `actionlint` covers unknown context properties, invalid expressions, and undefined step outputs across the whole workflow file, and it also runs `shellcheck` over every `run:` block, so the shellcheck pass a contributor ran locally is checked again automatically before merge.
+
+The workflow also runs on the schedule described in "When Builds Run", and on a push to `main` that touches `Dockerfile-cloudflare`, `.dockerignore`, or the workflow file itself. Neither run publishes on its own. The `decide` step still has to judge a build necessary: a push forces one only when it changed `Dockerfile-cloudflare` or `.dockerignore`, a scheduled run never forces one, and either kind of run still builds and publishes when it detects an upstream Caddy or addon change. Only once `decide` outputs `do_build: true` on a `push`, `schedule`, or `workflow_dispatch` event does the real multi-arch push to GHCR happen, and the `crane copy` mirror to Docker Hub runs on top of that only when Docker Hub secrets are configured.
+
+What is not verified before a merge: a real multi-arch push, a real `crane copy` to Docker Hub, and a real write to either registry. A pull request never publishes, so those three paths can only happen after the merge, and only on a later push or scheduled run where `decide` judges a build necessary.
+
+`act` was considered for local verification of this workflow and rejected. The `decide` step calls `crane` against `caddy:latest` and the GitHub API for release metadata, so a local `act` run is not hermetic. `act` also cannot reproduce the multi-arch `docker/build-push-action` push or the `crane copy` mirror, which is exactly the part no static check covers, so it would not close that gap. What a local `act` run would actually prove is that the YAML parses and the shell branches execute, and `shellcheck` plus the pull request tag assertion already prove that more cheaply and in the place that gates a merge. Against that, `act` costs a Docker-in-Docker setup in a repository that has no other local toolchain.
+
 ## Job Summary
 
 Every workflow run writes a summary that includes:

docs/assembly-line/2026-08-24-workflow-verification/plan.md

@@ -0,0 +1,102 @@
+# Plan: give the build workflow an executable verification path
+
+## Goal
+
+`.github/workflows/build_cloudflare-modules.yaml` cannot be executed locally, so every change to it has been verified statically: a YAML parse, `shellcheck` on the extracted `run:` block bodies, and a manual trace of each possible value through its consumers. Nothing has exercised a live Actions run, a real `crane copy` or a real `docker/metadata-action` invocation. This run decides what verification the repository wants, adds the one executable check that is cheap and covers the class of change the workflow actually receives, and writes the resulting standard down so that future changes do not re-litigate it.
+
+## Source brief
+
+GitHub issue adapter, <https://github.com/smoochy/caddy-modules/issues/20>, "The build workflow has no executable verification path". Body verbatim:
+
+> Follow-up from assembly-line run `2026-08-23-upstream-v-tag` (PR #18).
+>
+> `build_cloudflare-modules.yaml` cannot be executed locally, so every verification in that run was static: a YAML parse, `shellcheck` on the extracted `run:` block bodies, and a manual trace of both possible `caddy_tag` values through each consumer. Nothing exercised a live Actions run, a real `crane copy` or a real `docker/metadata-action` invocation. Both inspectors of the workflow package raised this caveat independently, and the final review judged it non-blocking for that PR because the change reuses a guard the existing `caddy-<x.y.z>` tag already proves in production.
+>
+> The open question is whether this repository wants a local dry-run path for its build workflow at all. Candidates, none evaluated yet:
+>
+> - `act` for a local Actions run, at the cost of a Docker-in-Docker setup that will not reproduce the multi-arch push.
+> - A pull-request-scoped job that runs the `decide` and `meta` steps with `push: false` and asserts the resulting tag list, which would cover exactly the class of change this PR made.
+> - Accepting static verification as the standard for this workflow and writing that down, so future changes do not re-litigate it.
+
+## Context the builder needs
+
+- The build workflow is `.github/workflows/build_cloudflare-modules.yaml`.
+- It already triggers on `pull_request` against `main`, filtered to the paths `Dockerfile-cloudflare`, `.dockerignore` and the workflow file itself.
+- It already builds without publishing on a pull request: `docker/build-push-action@v7` receives `push: ${{ github.event_name != 'pull_request' }}` and a single-platform `platforms` value. So the "runs with `push: false`" half of the brief's second candidate exists today. What is missing is any assertion on the result.
+- The `decide` step emits `caddy_tag`, which has exactly two possible shapes: a three-part semver such as `2.11.4`, or the literal string `latest` when neither the image label nor the image history yields a semver.
+- The `meta` step is `docker/metadata-action@v6`. It runs only under `if: steps.decide.outputs.do_build == 'true'` and lists three raw tags: `latest`, `caddy-<caddy_tag>`, and `caddy-v<caddy_tag>` enabled only when `caddy_tag != 'latest'`.
+- `do_build` is not always `true` on a pull request. The trigger's path filter evaluates the diff of the pushed commits, while the `Classify local changes` step independently diffs `PR_BASE_SHA` against `PR_HEAD_SHA`. A pull request whose later commit reverts an earlier change to a filtered path re-triggers the workflow while `image_inputs_changed` and `workflow_changed` are both false, so with no upstream change `do_build` is `false` and the `meta` step is skipped. Any new step that reads `steps.meta.outputs.tags` must handle that state.
+- `steps.meta.outputs.tags` is a newline-separated list of fully qualified references, each of the shape `ghcr.io/smoochy/caddy-cloudflare-modules:<tag>`.
+- The `decide` step needs the network. It calls `crane` against `caddy:latest` and the GitHub API for release metadata, so it cannot run offline, which is what rules the workflow out of a genuinely local dry run.
+
+## Package 1 - the pull request run asserts the tag list it produced
+
+**Outcome**: a pull request that changes the tag logic fails in CI when the produced tag set is wrong, instead of relying on a human tracing the branches by hand.
+
+**Declared file scope**: `.github/workflows/build_cloudflare-modules.yaml`
+
+**Point from the brief**: "A pull-request-scoped job that runs the `decide` and `meta` steps with `push: false` and asserts the resulting tag list, which would cover exactly the class of change this PR made."
+
+**Acceptance claim**:
+
+1. A new assertion step is added after the `meta` step and before the `Build and push` step, guarded by `if: github.event_name == 'pull_request' && steps.decide.outputs.do_build == 'true'`.
+2. When that guard holds, the step reads `steps.meta.outputs.tags` and `steps.decide.outputs.caddy_tag` and fails the job unless the produced tag set is exactly the expected set for that `caddy_tag`: `latest`, `caddy-<tag>` and `caddy-v<tag>` when `caddy_tag` is a three-part semver, and `latest` and `caddy-latest` when `caddy_tag` is the literal string `latest`.
+3. The comparison is by set membership plus a count, in either order. It is never an exact string comparison of the whole rendered list and never depends on line order, so a change in how `docker/metadata-action` formats its output cannot fail the job on its own.
+4. When `steps.decide.outputs.do_build` is `false` on a `pull_request` event, the `meta` step is skipped and `steps.meta.outputs.tags` is empty. The assertion step is skipped by the same guard and does not fail the job. This is required behaviour, not an oversight.
+5. On `push`, `schedule` and `workflow_dispatch` events the assertion step never runs, so no published build can be failed by it.
+6. The step adds no permission, no secret, no new third-party action, and no change to push, tag, label or mirror behaviour on any event.
+7. The failure output names the expected set and the produced set, so a failing run is diagnosable from the log alone.
+8. The workflow file remains valid YAML and the new shell block is `set -euo pipefail` clean.
+
+## Package 3 - the `act` candidate is evaluated
+
+**Outcome**: the repository has a recorded ruling on `act`, so the option is closed rather than left open.
+
+**Declared file scope**: none. This package produces a ruling, not a diff. The ruling is carried into the section Package 2 writes.
+
+**Point from the brief**: "`act` for a local Actions run, at the cost of a Docker-in-Docker setup that will not reproduce the multi-arch push."
+
+**Acceptance claim**:
+
+1. The run states whether `act` is adopted or rejected, with reasons drawn from this repository rather than from general argument.
+2. The reasons address at minimum: that the `decide` step needs network access to `crane` and the GitHub API, so a local run is not hermetic; that `act` cannot reproduce the multi-arch `docker/build-push-action` push, which is the part no static check covers; and what a local `act` run would therefore actually prove.
+3. The ruling appears in the run record and in the section Package 2 writes.
+
+## Package 2 - the repository states its verification standard for this workflow
+
+**Outcome**: a contributor changing this workflow reads what is checked where, and what is not checked before merge, without re-opening the question.
+
+**Declared file scope**: `README.md` (a new section), `CLAUDE.md` (one pointer line, appended outside the `OPENWIKI:START` and `OPENWIKI:END` markers)
+
+**Point from the brief**: "Accepting static verification as the standard for this workflow and writing that down, so future changes do not re-litigate it."
+
+**Acceptance claim**:
+
+1. The new `README.md` section states what each verification layer covers: the pull request tag assertion Package 1 adds, the static checks a change to this workflow is expected to run locally, and the live scheduled run.
+2. It states explicitly what is not verified before a merge: a real multi-arch push, a real `crane copy` to Docker Hub, and a real write to either registry.
+3. It records the `act` ruling from Package 3 with its reason.
+4. It names the static checks concretely enough to repeat them, rather than saying "static verification" and stopping.
+5. It does not contradict the tag list `README.md` already documents in its "Image Tags" and "Available Images" sections, and it does not restate that list.
+6. `CLAUDE.md` gains one pointer line to the new section, appended outside the OpenWiki marker block, so the generated content stays untouched.
+7. No section of `README.md` other than the new one changes.
+8. Both edits are Simplified Technical English, with no hard-wrapped prose and no em dashes or en dashes.
+
+## Waves
+
+- **Wave 1**: Package 1 and Package 3.
+- **Wave 2**: Package 2. Runs after wave 1.
+
+Both waves are sequential. Package 3 touches no file, so it conflicts with nothing; wave 1 is sequential because this version of the line runs every wave sequentially, not because a shared artifact forced a fallback. Wave 2 is a content dependency rather than a file conflict: Package 2's section describes the assertion Package 1 builds and carries the ruling Package 3 makes, so writing it first risks documenting a shape the inspection then changes.
+
+## Triggers
+
+- **Shared artifacts**: no hit. No lockfile, no manifest, no snapshot directory, no formatter or linter config is in any declared scope. The repo probe read `.gitignore`, which holds the single entry `.agents/`, and the CI configs, which run no repo-wide formatter. The generated `openwiki/` tree exists but no package declares it and no package may hand-edit it.
+- **Security**: no hit. No declared file scope matches the trigger vocabulary by path, and no package adds, reads or changes a secret, a permission block, a login step or a credential. The pre-approval `security-reviewer` pass is skipped and that skip is recorded.
+
+## Non-goals
+
+- No change to what the build publishes, and no change to push, tag, label or mirror behaviour on any event.
+- No new third-party action, and no new permission or secret.
+- No adoption of `act`, and no Docker-in-Docker setup.
+- No hand-edit of the generated `openwiki/` tree.
+- No cleanup of the unrelated stale `claude/actionlint-ci` worktree and its uncommitted `lint_workflows.yaml`. That is outside this brief and is proposed as a follow-up instead.