Skip to content

Commit 3222c57

Browse files
fix(release): the image backfill skips a version whose publishing run has not finished building its image (#21634)
Fixes #21606 Clause-②: no ## What this does `release-integrity`'s audit step now skips the image backfill for a version whose publishing run has not finished its own image build. It still skips the Releases backfill for a version whose publish is in flight, as before. Both answers come from one read of PR #21605's `in-flight` reader. While the publishing run's `docker` job is not yet created, queued or running, the step leaves `image-missing` unset and prints a notice naming that run. When the read is unreadable, or cannot be asked at all, it leaves `image-missing` unset and prints a warning. Otherwise it takes the existing path. The guard skips a write and refuses nothing: the step stays green, and no gate is added. - `scripts/release-pending-publish.mjs`: `judgePublishInFlight` returns a `docker` verdict beside the publish job's top-level one (`in-flight` with reason `docker-in-flight` or `unreadable`, or `clear`). `publishInFlight` answers `unreadable` for both whenever the read fails. `collectPublishRuns` is unchanged, because it already reads the jobs of every candidate run that is not completed. This adds one self-test battery (12 cases), and the roster floor rises from 21 to 22. - `.github/workflows/release.yml`: only the audit step changes. It asks the reader once, before either backfill branch, and gates `image-missing` on `.docker.state` the same way it gates `releases-missing` on `.state`. No other job, step or permission changes; `actions: read` came with PR #21605. - `scripts/release-verify-npm.mjs`: only the self-test harness changes. Battery 12 executes the audit step's text. Its Actions stub gains the publish job's status and the docker job's status, and the battery gains 10 cases (floor 24 to 34). The battery name and the success line now mention the image build. ## The premise, measured (read-only `GET`s on the runs and jobs) | Run | Job | `created_at` | started | completed | |:--|:--|:--|:--|:--| | `36955885276` (push at `dcc5ef4c`, the publishing run) | `Release integrity` | 02:29:47Z | 02:29:50Z | 02:30:33Z | | | `Publish 17.6.0 to npm (awaiting approval)` `110678824190` | **02:30:34Z** | 02:31:09Z | **03:05:45Z** | | | `Docker image / Build & push ghcr.io/objectstack-ai/objectstack` `110687097666` | **03:05:45Z** | 03:05:49Z | 03:10:02Z | | `36958423332` (push at `4e530568`, the landing) | `Release integrity` `110686494582` | 03:03:12Z | 03:03:14Z | 03:05:46Z | | | `Docker image / Build & push ...` `110687103770` (the backfill) | 03:05:47Z | 03:05:48Z | 03:09:18Z | - **Mechanism assumption 1 holds.** GitHub creates a job when its `needs` settle. The publish job was created one second after `release-integrity` completed. The docker job was created the same second the publish job completed. So from 02:30:34Z to 03:05:45Z the publishing run held no docker job at all. The landing's audit read the image as missing at about 03:04Z, and at that instant the run's jobs list held only the publish job, `in_progress`. PR #21605's answer is `clear` as soon as the publish job completes, and from that second until the docker job completes the image is still being built. So "not yet created" is a real state, and the reader counts it as in flight. - **The job's names.** The jobs API names the docker job `Docker image / Build & push ...` while its reusable workflow runs, and `Docker image` when it is skipped (measured on run `37148267152`, created 19:33:23Z with the skipped publish job). The reader matches both names (`DOCKER_JOB_NAME`). - **Assumption 2 holds.** This is an extension of `judgePublishInFlight` and `publishInFlight`, not a second reader. The same candidate runs and the same reads produce both answers. - **Assumption 3, measured.** Batteries 12 and 13 do execute the audit text. With the workflow change in place and the harness not yet updated, `node scripts/release-verify-npm.mjs --self-test` printed `OK ... 103 cases pass`, with no red at all (PR #21605 got 2 reds at that point). The stubs already answered the Actions read, and no existing pin asserted `image-missing` while a publish was in flight. That is why battery 12 gains the pins below: ablation 1 shows that they, and nothing else, catch the new branch being disabled. Battery 13 needed no change. Its audit reads the image as present, so the image branch never requests anything there. - **Assumption 4 holds.** The producer is the audit step plus the reader, as expected. ## The guard's rule A run is the version's **publishing run** when it holds `Publish V to npm (awaiting approval)` in any status but `waiting` (approved, running or done). The calling run is excluded, as for the Releases guard. In such a run that is not completed, the image is **in flight** until every docker job of the run is `completed`, including while it has none yet. Pinned per triage's pins (`5971281558`): | Pin | Reader (`release-pending-publish`, the new battery) | Audit text (`release-verify-npm` battery 12) | |:--|:--|:--| | A publishing run whose docker job is in flight means no image backfill | publish running with no docker job; publish completed with no docker job yet; docker `queued` / `in_progress` / `waiting` / unknown under either name; the whole read through the push run at the version commit; the repair lane through the `in_progress` filter | publish `in_progress`: no `image-missing`, plus a notice naming run 4242 and "not yet created"; publish completed with no docker job (Releases present): no `image-missing`; docker `in_progress`: no `image-missing` while the Releases are backfilled | | A completed docker job with the image present means no backfill | completed docker job, in a run still finishing or in a completed run: `clear` | docker completed, image present: no `image-missing`, "image ... is present" | | A completed docker job with the image absent means a backfill | the same `clear`, so the image probe alone decides | docker completed, image absent: `image-missing=true` | | An unreadable run means skip and warn | jobs never read; a filter answering non-200; jobs answering non-200; a shallow clone: `unreadable` | API answering 503: no `image-missing`, and a warning; the reader unable to run at all: no backfill, and a warning for both | Both columns also pin the boundary. A publish held at the `release` environment builds nothing before a human approves it, so it is `clear` for the image too; this matches the Releases guard and keeps a stale prompt from blocking the backfill. Another version's publish, the calling run, and a run whose skipped publish job names no version are all `clear`. ## Tests (at `4d31b13409`, the final commit) - `node scripts/release-pending-publish.mjs --self-test`: `✓ release-pending-publish self-test: 92 cases across 22 batteries pass.` That is 80 before this PR, plus the new battery's 12. - `node scripts/release-verify-npm.mjs --self-test`: `OK release-verify-npm self-test: 113 cases pass across 13 batteries`. That is 103 before, plus 10. - **Ablations.** Each mutation went through `scripts/ablation-replace.mjs` in wrap mode. Each anchor hit once and went from 1 to 0, and each restore was proven by blob equal to HEAD and an empty `git diff HEAD`: 1. The audit's docker branch was disabled (`elif [ "$(jq -r '.docker.state' ...)" != 'clear' ]; then` became `elif false; then`). `release-verify-npm --self-test` went 5 of 113 red: unreadable, then a backfill; the publish in flight, then a backfill; the notice missing; the publish done with no docker job yet, then a backfill; docker `in_progress`, then a backfill. The blob went `451300d30c34` to `69bb917dd3b4` and was restored to `451300d30c34`. 2. A run with no docker job yet was read as settled (`docker.length > 0 &&` deleted). `release-pending-publish --self-test` went 3 red (the 03:04Z instant, the not-yet-created instant, the whole read), and `release-verify-npm --self-test` went 3 of 113 red through the same reader. The blob went `b8fc0ce80442` to `3efaba0b79ce` and was restored. 3. The unreadable answer's `docker` was made `clear`. `release-pending-publish --self-test` went 1 red (the three unreadable reads), and `release-verify-npm --self-test` went 1 of 113 red (the 503 case). The blob went `b8fc0ce80442` to `523a9ea699c4` and was restored. - **Live read-only smoke against this repository.** `publishInFlight` was called for 17.6.0 with a GET-only client. It made 5 GETs: the `in_progress` and `queued` filters (both empty), `head_sha` probes at `617f25f8a4` (none) and `dcc5ef4c5a` (run `36955885276`), and a direct read of that run, which is `completed`. The answer was `clear` / `clear`. Replaying the real job list of run `36955885276` cut at 03:04:15Z gave `in-flight` for both; at 03:05:45Z, with the docker job just created, it gave `clear` for the publish and `in-flight` for the image. - **Gates.** `node scripts/pm/dispatch-gates.mjs --commands --repo objectstack-ai/objectstack` at `4d31b13409` derived 51 commands. All 51 were run, each exited 0, recorded per command before reading. `--ran` printed: `Run reconciliation — 51 derived, 51 run, 0 NOT-MEASURED, 0 UNRUN.` - **ESLint, as a narrowed run with its proof.** - `eslint --no-inline-config --format json` over the two changed `.mjs` files: 2 files, 0 errors, 0 warnings. - ESLint's own `isPathIgnored` and `calculateConfigForFile` put both files in the linted population. `release.yml` is ignored, so it has no config. - Their config carries no `parserOptions.project` and no `projectService`. Type-aware linting is off, so this diff cannot move any verdict on an untouched file. - **Declared to CI.** These were not run locally: the type-check lanes (no TypeScript package is touched), `ci.yml` Test Core's shard steps, and the families `dispatch-gates` lists as wide-population or workflow-valued (the latter printed as NOT MEASURED there). ## Changeset None. `skip-changeset` holds because nothing here is published. The diff is `.github/workflows/` plus two repo-root `scripts/` files, and the root `package.json` is private. None of the 69 public packages' `files[]` names `scripts/` or `.github/`. As a positive control, all 69 of them name `dist`. ## Acceptance notes - **Not in this PR: two backfills racing each other.** A landing whose audit requests the image starts a docker job of its own. A later landing's audit runs once the first `release-integrity` finishes, and it can read the image as missing while that docker job still builds. The guard reads only the publishing run, as triage directed. It cannot attribute a backfill run's docker job to a version, because the job name carries none. Nothing is measured: no run has been named. Carrier: none. - `release-integrity`'s `outputs.published` comment ("set only when npm ALREADY has this version's WHOLE fixed group and its runtime image is missing") is left as it is. It names necessary conditions and stays true, and the claim limits `release.yml` to the audit step. The new condition is stated in the step. - A rename of the `docker` job that `DOCKER_JOB_NAME` misses reads as "not yet created". The image then counts as in flight until the publishing run completes: late, never early. The constant's docblock says so. - Releases present with the image missing now pays the in-flight read (5 GETs in the smoke above), where before it paid none. The common path, with both present, still pays none, and battery 12 pins that. - `lint.yml`'s comment above "Release version-commit selection self-test" says the script "holds all three decisions". PR #21605 already noted it. `lint.yml` is outside this card's surface. Carrier: the next PR that edits that step's comment. - No ADR governs this. ADR-0125 decides which push publishes and who approves it; it is silent on the image backfill, and this changes neither. --- _Generated by [Claude Code](https://claude.ai/code/session_01T9u38rswFp5Rw8DswRUReJ)_ --------- Co-authored-by: Claude <noreply@anthropic.com>
1 parent 6946f2f commit 3222c57

3 files changed

Lines changed: 298 additions & 49 deletions

File tree

‎.github/workflows/release.yml‎

Lines changed: 33 additions & 16 deletions
Original file line numberDiff line numberDiff line change
@@ -1070,38 +1070,55 @@ jobs:
10701070
;;
10711071
esac
10721072
1073-
# ── the publish in flight, before any Releases backfill ───────────
1073+
# ── the publish in flight, before any backfill ────────────────────
10741074
# npm settles BEFORE the publish job reaches its own "Create GitHub
1075-
# Releases" step, so a landing audited in that window found the group
1076-
# complete and wrote the same Releases beside it: on 17.6.0 the
1077-
# publish of run 36955885276 wrote them 03:03:47Z -> 03:05:42Z, this
1078-
# backfill in run 36958423332 started at 03:04:37Z, and five tags got
1079-
# two Release objects each. The two writers are in different runs,
1080-
# so no `needs:` can order them; the publish's own run creates its
1081-
# Releases and D4 asset, and a landing after it finishes backfills
1082-
# whatever it did not. scripts/release-pending-publish.mjs `in-flight`
1083-
# (its --self-test, run by lint.yml, holds the rule) answers
1084-
# `in-flight` on anything it cannot read, so nothing is backfilled off
1085-
# a guess. It only ever leaves `releases-missing` unset: this step
1086-
# stays green and the image request below is not its business.
1075+
# Releases" step, and before that run's `docker` job even exists (it
1076+
# needs `publish`, so GitHub creates it when the publish job
1077+
# completes). A landing audited in that window found the group
1078+
# complete and wrote beside the publishing run. On 17.6.0 the publish
1079+
# of run 36955885276 wrote its Releases 03:03:47Z -> 03:05:42Z, this
1080+
# backfill in run 36958423332 started writing them at 03:04:37Z, and
1081+
# five tags got two Release objects each; and both runs built and
1082+
# pushed the 17.6.0 image, 03:05:49Z -> 03:10:02Z beside 03:05:48Z ->
1083+
# 03:09:18Z. The writers are in different runs, so no `needs:` can
1084+
# order them; the publishing run creates its Releases, D4 asset and
1085+
# image, and a landing after it finishes backfills whatever it did
1086+
# not. scripts/release-pending-publish.mjs `in-flight` (its
1087+
# --self-test, run by lint.yml, holds the rule) answers both in one
1088+
# read: `.state` for the publish job, `.docker.state` for that run's
1089+
# image build, in flight from the approval until its docker job
1090+
# completes. Anything it cannot read answers `in-flight`, so nothing
1091+
# is backfilled off a guess. It only ever leaves `releases-missing`
1092+
# and `image-missing` unset: this step stays green.
1093+
flight_asked=true
1094+
flight=$(GITHUB_TOKEN="$GH_TOKEN" node scripts/release-pending-publish.mjs in-flight --version "$version" --version-commit "$version_commit" --head "$SHA" --workflow release.yml) || flight_asked=false
1095+
if [ "$flight_asked" = true ]; then
1096+
jq -c . <<<"$flight"
1097+
fi
10871098
if [ "$releases_ok" = true ]; then
10881099
echo "GitHub Releases + ADR-0087 D4 asset are present for ${version}."
1089-
elif ! flight=$(GITHUB_TOKEN="$GH_TOKEN" node scripts/release-pending-publish.mjs in-flight --version "$version" --version-commit "$version_commit" --head "$SHA" --workflow release.yml); then
1100+
elif [ "$flight_asked" != true ]; then
10901101
echo "::warning::${version}'s GitHub Releases or ADR-0087 D4 asset are incomplete, and whether its publish is still in flight could not be asked (reason above). Nothing is backfilled off a guess; the next landing reads again."
10911102
elif [ "$(jq -r '.state' <<<"$flight")" != 'clear' ]; then
1092-
jq -c . <<<"$flight"
10931103
if [ "$(jq -r '.reason' <<<"$flight")" = 'publish-in-flight' ]; then
10941104
echo "::notice::${version}'s GitHub Releases or ADR-0087 D4 asset are incomplete, but its publish is still in flight ($(jq -r '.detail' <<<"$flight")). That run creates them; nothing is backfilled beside it. A landing after it finishes backfills whatever it did not."
10951105
else
10961106
echo "::warning::${version}'s GitHub Releases or ADR-0087 D4 asset are incomplete, and whether its publish is still in flight could not be read ($(jq -r '.detail' <<<"$flight")). Nothing is backfilled off a guess; the next landing reads again."
10971107
fi
10981108
else
1099-
jq -c . <<<"$flight"
11001109
echo "::warning::${version} is on npm but its GitHub Releases or the ADR-0087 D4 asset are incomplete (#4900) — backfilling."
11011110
echo "releases-missing=true" >> "$GITHUB_OUTPUT"
11021111
fi
11031112
if [ "$image_ok" = true ]; then
11041113
echo "ghcr: image for $version is present."
1114+
elif [ "$flight_asked" != true ]; then
1115+
echo "::warning::No ghcr image for ${version} (or the registry could not be probed), and whether its publishing run is still building it could not be asked (reason above). No Docker job is requested off a guess; the next landing reads again."
1116+
elif [ "$(jq -r '.docker.state' <<<"$flight")" != 'clear' ]; then
1117+
if [ "$(jq -r '.docker.reason' <<<"$flight")" = 'docker-in-flight' ]; then
1118+
echo "::notice::No ghcr image for ${version} yet, but its publishing run has not finished building it ($(jq -r '.docker.detail' <<<"$flight")). That run pushes the image; no Docker job is requested beside it. A landing after it finishes requests one if the image is still missing."
1119+
else
1120+
echo "::warning::No ghcr image for ${version} (or the registry could not be probed), and whether its publishing run is still building it could not be read ($(jq -r '.docker.detail' <<<"$flight")). No Docker job is requested off a guess; the next landing reads again."
1121+
fi
11051122
else
11061123
echo "::warning::No ghcr image for $version (or the registry could not be probed) — requesting the Docker job."
11071124
echo "image-missing=true" >> "$GITHUB_OUTPUT"

0 commit comments

Comments
 (0)