Skip to content

fix(release): the image backfill skips a version whose publishing run has not finished building its image - #21634

Merged
objectstack-fleet[bot] merged 2 commits into
mainfrom
claude/issue-21606-image-backfill-guard
Oct 3, 2026
Merged

objectstack-fleet[bot] merged 2 commits into
mainfrom
claude/issue-21606-image-backfill-guard

Conversation

@objectstack-fleet

Copy link
Copy Markdown
Contributor

Fixes #21606

Clause-②: no

What this does

release-integrity's audit step now skips the image backfill for a version whose publishing run has not finished its own image build. It still skips the Releases backfill for a version whose publish is in flight, as before. Both answers come from one read of PR #21605's in-flight reader. While the publishing run's docker job is not yet created, queued or running, the step leaves image-missing unset and prints a notice naming that run. When the read is unreadable, or cannot be asked at all, it leaves image-missing unset and prints a warning. Otherwise it takes the existing path. The guard skips a write and refuses nothing: the step stays green, and no gate is added.

  • scripts/release-pending-publish.mjs: judgePublishInFlight returns a docker verdict beside the publish job's top-level one (in-flight with reason docker-in-flight or unreadable, or clear). publishInFlight answers unreadable for both whenever the read fails. collectPublishRuns is unchanged, because it already reads the jobs of every candidate run that is not completed. This adds one self-test battery (12 cases), and the roster floor rises from 21 to 22.
  • .github/workflows/release.yml: only the audit step changes. It asks the reader once, before either backfill branch, and gates image-missing on .docker.state the same way it gates releases-missing on .state. No other job, step or permission changes; actions: read came with PR fix(release): the Releases backfill skips a version whose publish is in flight in another run #21605.
  • scripts/release-verify-npm.mjs: only the self-test harness changes. Battery 12 executes the audit step's text. Its Actions stub gains the publish job's status and the docker job's status, and the battery gains 10 cases (floor 24 to 34). The battery name and the success line now mention the image build.

The premise, measured (read-only GETs on the runs and jobs)

Run Job created_at started completed
36955885276 (push at dcc5ef4c, the publishing run) Release integrity 02:29:47Z 02:29:50Z 02:30:33Z
Publish 17.6.0 to npm (awaiting approval) 110678824190 02:30:34Z 02:31:09Z 03:05:45Z
Docker image / Build & push ghcr.io/objectstack-ai/objectstack 110687097666 03:05:45Z 03:05:49Z 03:10:02Z
36958423332 (push at 4e530568, the landing) Release integrity 110686494582 03:03:12Z 03:03:14Z 03:05:46Z
Docker image / Build & push ... 110687103770 (the backfill) 03:05:47Z 03:05:48Z 03:09:18Z
  • Mechanism assumption 1 holds. GitHub creates a job when its needs settle. The publish job was created one second after release-integrity completed. The docker job was created the same second the publish job completed. So from 02:30:34Z to 03:05:45Z the publishing run held no docker job at all. The landing's audit read the image as missing at about 03:04Z, and at that instant the run's jobs list held only the publish job, in_progress. PR fix(release): the Releases backfill skips a version whose publish is in flight in another run #21605's answer is clear as soon as the publish job completes, and from that second until the docker job completes the image is still being built. So "not yet created" is a real state, and the reader counts it as in flight.
  • The job's names. The jobs API names the docker job Docker image / Build & push ... while its reusable workflow runs, and Docker image when it is skipped (measured on run 37148267152, created 19:33:23Z with the skipped publish job). The reader matches both names (DOCKER_JOB_NAME).
  • Assumption 2 holds. This is an extension of judgePublishInFlight and publishInFlight, not a second reader. The same candidate runs and the same reads produce both answers.
  • Assumption 3, measured. Batteries 12 and 13 do execute the audit text. With the workflow change in place and the harness not yet updated, node scripts/release-verify-npm.mjs --self-test printed OK ... 103 cases pass, with no red at all (PR fix(release): the Releases backfill skips a version whose publish is in flight in another run #21605 got 2 reds at that point). The stubs already answered the Actions read, and no existing pin asserted image-missing while a publish was in flight. That is why battery 12 gains the pins below: ablation 1 shows that they, and nothing else, catch the new branch being disabled. Battery 13 needed no change. Its audit reads the image as present, so the image branch never requests anything there.
  • Assumption 4 holds. The producer is the audit step plus the reader, as expected.

The guard's rule

A run is the version's publishing run when it holds Publish V to npm (awaiting approval) in any status but waiting (approved, running or done). The calling run is excluded, as for the Releases guard. In such a run that is not completed, the image is in flight until every docker job of the run is completed, including while it has none yet. Pinned per triage's pins (5971281558):

Pin Reader (release-pending-publish, the new battery) Audit text (release-verify-npm battery 12)
A publishing run whose docker job is in flight means no image backfill publish running with no docker job; publish completed with no docker job yet; docker queued / in_progress / waiting / unknown under either name; the whole read through the push run at the version commit; the repair lane through the in_progress filter publish in_progress: no image-missing, plus a notice naming run 4242 and "not yet created"; publish completed with no docker job (Releases present): no image-missing; docker in_progress: no image-missing while the Releases are backfilled
A completed docker job with the image present means no backfill completed docker job, in a run still finishing or in a completed run: clear docker completed, image present: no image-missing, "image ... is present"
A completed docker job with the image absent means a backfill the same clear, so the image probe alone decides docker completed, image absent: image-missing=true
An unreadable run means skip and warn jobs never read; a filter answering non-200; jobs answering non-200; a shallow clone: unreadable API answering 503: no image-missing, and a warning; the reader unable to run at all: no backfill, and a warning for both

Both columns also pin the boundary. A publish held at the release environment builds nothing before a human approves it, so it is clear for the image too; this matches the Releases guard and keeps a stale prompt from blocking the backfill. Another version's publish, the calling run, and a run whose skipped publish job names no version are all clear.

Tests (at 4d31b13409, the final commit)

  • node scripts/release-pending-publish.mjs --self-test: ✓ release-pending-publish self-test: 92 cases across 22 batteries pass. That is 80 before this PR, plus the new battery's 12.
  • node scripts/release-verify-npm.mjs --self-test: OK release-verify-npm self-test: 113 cases pass across 13 batteries. That is 103 before, plus 10.
  • Ablations. Each mutation went through scripts/ablation-replace.mjs in wrap mode. Each anchor hit once and went from 1 to 0, and each restore was proven by blob equal to HEAD and an empty git diff HEAD:
    1. The audit's docker branch was disabled (elif [ "$(jq -r '.docker.state' ...)" != 'clear' ]; then became elif false; then). release-verify-npm --self-test went 5 of 113 red: unreadable, then a backfill; the publish in flight, then a backfill; the notice missing; the publish done with no docker job yet, then a backfill; docker in_progress, then a backfill. The blob went 451300d30c34 to 69bb917dd3b4 and was restored to 451300d30c34.
    2. A run with no docker job yet was read as settled (docker.length > 0 && deleted). release-pending-publish --self-test went 3 red (the 03:04Z instant, the not-yet-created instant, the whole read), and release-verify-npm --self-test went 3 of 113 red through the same reader. The blob went b8fc0ce80442 to 3efaba0b79ce and was restored.
    3. The unreadable answer's docker was made clear. release-pending-publish --self-test went 1 red (the three unreadable reads), and release-verify-npm --self-test went 1 of 113 red (the 503 case). The blob went b8fc0ce80442 to 523a9ea699c4 and was restored.
  • Live read-only smoke against this repository. publishInFlight was called for 17.6.0 with a GET-only client. It made 5 GETs: the in_progress and queued filters (both empty), head_sha probes at 617f25f8a4 (none) and dcc5ef4c5a (run 36955885276), and a direct read of that run, which is completed. The answer was clear / clear. Replaying the real job list of run 36955885276 cut at 03:04:15Z gave in-flight for both; at 03:05:45Z, with the docker job just created, it gave clear for the publish and in-flight for the image.
  • Gates. node scripts/pm/dispatch-gates.mjs --commands --repo objectstack-ai/objectstack at 4d31b13409 derived 51 commands. All 51 were run, each exited 0, recorded per command before reading. --ran printed: Run reconciliation — 51 derived, 51 run, 0 NOT-MEASURED, 0 UNRUN.
  • ESLint, as a narrowed run with its proof.
    • eslint --no-inline-config --format json over the two changed .mjs files: 2 files, 0 errors, 0 warnings.
    • ESLint's own isPathIgnored and calculateConfigForFile put both files in the linted population. release.yml is ignored, so it has no config.
    • Their config carries no parserOptions.project and no projectService. Type-aware linting is off, so this diff cannot move any verdict on an untouched file.
  • Declared to CI. These were not run locally: the type-check lanes (no TypeScript package is touched), ci.yml Test Core's shard steps, and the families dispatch-gates lists as wide-population or workflow-valued (the latter printed as NOT MEASURED there).

Changeset

None. skip-changeset holds because nothing here is published. The diff is .github/workflows/ plus two repo-root scripts/ files, and the root package.json is private. None of the 69 public packages' files[] names scripts/ or .github/. As a positive control, all 69 of them name dist.

Acceptance notes

  • Not in this PR: two backfills racing each other. A landing whose audit requests the image starts a docker job of its own. A later landing's audit runs once the first release-integrity finishes, and it can read the image as missing while that docker job still builds. The guard reads only the publishing run, as triage directed. It cannot attribute a backfill run's docker job to a version, because the job name carries none. Nothing is measured: no run has been named. Carrier: none.
  • release-integrity's outputs.published comment ("set only when npm ALREADY has this version's WHOLE fixed group and its runtime image is missing") is left as it is. It names necessary conditions and stays true, and the claim limits release.yml to the audit step. The new condition is stated in the step.
  • A rename of the docker job that DOCKER_JOB_NAME misses reads as "not yet created". The image then counts as in flight until the publishing run completes: late, never early. The constant's docblock says so.
  • Releases present with the image missing now pays the in-flight read (5 GETs in the smoke above), where before it paid none. The common path, with both present, still pays none, and battery 12 pins that.
  • lint.yml's comment above "Release version-commit selection self-test" says the script "holds all three decisions". PR fix(release): the Releases backfill skips a version whose publish is in flight in another run #21605 already noted it. lint.yml is outside this card's surface. Carrier: the next PR that edits that step's comment.
  • No ADR governs this. ADR-0125 decides which push publishes and who approves it; it is silent on the image backfill, and this changes neither.

Generated by Claude Code

claude added 2 commits October 3, 2026 20:27
…n's image build is finished

`judgePublishInFlight` now carries a `docker` verdict beside the publish
job's: a run holding the version's publish job past the `release`
environment builds the image until every docker job of that run is
`completed`, including before GitHub creates it (it needs the publish
job). An unreadable read answers `in-flight` / `unreadable` for both.
New self-test battery (12 cases); the roster floor rises to 22.

Claude-Session: https://claude.ai/code/session_01T9u38rswFp5Rw8DswRUReJ
Co-authored-by: Claude <noreply@anthropic.com>
… has not finished building its image

The audit step asks the in-flight reader once, before either backfill,
and gates `image-missing` on its `docker` verdict the way it gates
`releases-missing` on the publish job's: a notice while the publishing
run's docker job is not yet created, queued or running; a warning, and
no request, when the read is unreadable or cannot be asked; the existing
request otherwise. Battery 12 of release-verify-npm's self-test pins the
four triage pins plus the not-yet-created case at the audit's own text.

Claude-Session: https://claude.ai/code/session_01T9u38rswFp5Rw8DswRUReJ
Co-authored-by: Claude <noreply@anthropic.com>
@github-actions github-actions Bot added the size/m label Oct 3, 2026
@objectstack-fleet objectstack-fleet Bot added the skip-changeset PR has no user-facing published change; bypasses the changeset gate label Oct 3, 2026
@github-actions github-actions Bot added the ci/cd label Oct 3, 2026
@objectstack-fleet
objectstack-fleet Bot marked this pull request as ready for review October 3, 2026 21:22
@objectstack-fleet
objectstack-fleet Bot enabled auto-merge October 3, 2026 21:22
@objectstack-fleet
objectstack-fleet Bot added this pull request to the merge queue Oct 3, 2026
Merged via the queue into main with commit 3222c57 Oct 3, 2026
37 checks passed
@objectstack-fleet
objectstack-fleet Bot deleted the claude/issue-21606-image-backfill-guard branch October 3, 2026 21:56
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ci/cd size/m skip-changeset PR has no user-facing published change; bypasses the changeset gate

Projects

None yet

2 participants