Skip to content

fix(pm): ci-failure resolves a red lane's Lane verdict to its red gates' own commands - #23246

Merged
objectstack-fleet[bot] merged 2 commits into
mainfrom
claude/issue-23223-ci-failure-lane-verdict
Oct 13, 2026
Merged

objectstack-fleet[bot] merged 2 commits into
mainfrom
claude/issue-23223-ci-failure-lane-verdict

Conversation

@objectstack-fleet

Copy link
Copy Markdown
Contributor

Fixes #23223
Clause-②: no

What this changes

Since #23191, every gate step of a Lint · * / Type Check · * lane runs under continue-on-error: true. A red gate's jobs-API conclusion is therefore success, so the only failed step of a red lane is its last one, Lane verdict. scripts/pm/ci-failure.mjs resolved that step name through stepBodies. The name exists once per lane, so the substitute block printed node scripts/ci/lane-verdict.mjs 12 times. Run locally, that command exits 3 and reproduces nothing.

Now, when the failed step is Lane verdict, the tool reads the red gate names from the verdict's own sentences (gate step "NAME" (id ID) failed). It resolves each name through stepBodies to that gate's own run: block. The substitute block prints those commands. The verdict's own command is never offered.

Only one file changes: scripts/pm/ci-failure.mjs and its --self-test. Nothing under .github/workflows/**, scripts/ci/lane-verdict.mjs or .claude/** is touched.

  • classifyAnnotation gains one kind, lane-verdict. It recognises the five sentences that lane-verdict.mjs's renderVerdict writes as error annotations: red, unread, did-not-run, fail-fast and refusal. For red, it keeps the gate name and id. Everything else still classifies as before. assertionStatus and verdictOf read the new kind as content, the same as the old other, so the exit table does not move.
  • readLaneVerdict takes the union of those sentences from two sources, deduplicated: the check-run's annotations and the job log's error lines. The log is already fetched for this family, so it costs no extra request. GitHub's documentation says one step can create only a limited number of error annotations. A lane with more red gates than that limit would name only some of them in the annotations; the log names them all. That limit is NOT measured here, and the union makes the result independent of it.
  • resolveFailedStep: every step except Lane verdict resolves by its own name, unchanged. For Lane verdict, repro holds the red gates' hits, each carrying gate: { name, id }, and laneVerdict holds the full reading. Both appear in --json.
  • renderSubstitute is the old inline sayRepro, now a pure function so the self-test can pin the printed lines. When the verdict names no runnable red gate, it says why, in one of three ways:
    • Names no red gate (an unread gate, a gate that did not run, a fail-fast step, or a refusal): it lists what the verdict did name and says that no gate command reproduces this red.
    • NOT READ: the annotations call failed and no log was read.
    • UNRESOLVED, loudly: no sentence matches a known shape. The wording may have changed, or the verdict died before printing. It never falls back to printing the verdict command.
  • stepBodies now filters a new workflowSteps scan. The output is identical, and the existing pins still hold.

Proof on the measured sample

Run 38373082677 (PR head 5301220aa), check-run / job 115214618941, node scripts/pm/ci-failure.mjs --run 38373082677. The tool has no --job flag, so --run was used.

Before, on main at 197fbe2c04:

    failed step  #68  Lane verdict
    ...
    substitute   run the failing step locally — it prints the same assertion:
      from .github/workflows/lint.yml, step "Lane verdict" (run:)
        node scripts/ci/lane-verdict.mjs
      (the same pair, 12 times in all)

After, on this branch:

    lane verdict #68 "Lane verdict" judges this lane's gates; it is not one, and its own
                 command reproduces nothing (run locally it exits 3). What it named:
                   RED       "Docs anchors resolve to real headings" (id docs-anchors-resolve-to-real-headings)
                 1 red gate(s); each one's OWN command is the substitute below.
    substitute   run the failing step locally — it prints the same assertion:
      from .github/workflows/lint.yml, red gate "Docs anchors resolve to real headings" (id docs-anchors-resolve-to-real-headings), named by step "Lane verdict" (run:)
        pnpm check:doc-anchors
        node scripts/check-system-context-census.mjs --self-test
        node scripts/check-system-context-census.mjs

diff before after shows only three changes: the two other heads for that sentence become lane-verdict, and the substitute block above changes. The other three failing checks on that sha render byte-identical. The walk still exits 2 (UNDETERMINED) both times, because none of the four failing checks carries a file-anchored assertion. The exit table is unchanged on purpose.

Self-test, and ablation that proves it can fail

A new battery, Lane verdict: the red gates, not the verdict (#23223), has 22 cases. The stepBodies / resolveStep battery goes from 8 to 9 cases, and the battery floor from 14 to 15.

The battery imports scripts/ci/lane-verdict.mjs and drives its real judgeLane and renderVerdict. The parser is therefore pinned to what the producer actually prints, so a wording change there turns this battery red. No rendered line is printed: they are workflow commands, and this self-test runs on a runner.

The cases pin the card's three requirements:

The cases also cover the measured specimen, all five sentence shapes, the UNRESOLVED and NOT READ cases, the log/annotation union, a red gate that is missing from the local tree, and a gate name containing quotes and parentheses. A live pin requires that every step in the real workflow tree that runs the verdict is named Lane verdict (12 of them today). A lane that renames its verdict step therefore turns the self-test red instead of silently bringing the defect back.

Ablations were run on committed 77480b13ce through scripts/ablation-replace.mjs. In each leg the anchor hit 1 to 0, the blob changed, and the restore was proven blob == HEAD with an empty git diff HEAD:

leg mutation self-test cases red
1 resolveFailedStep stops treating Lane verdict specially exit 1 8 (N→N, substitute, roster mismatch, UNRESOLVED, NOT READ, missing gate)
2 the red sentence pattern no longer matches exit 1 8 (specimen, real-renderer parse, quoted name, N→N, union, missing gate)
3 every other step resolves to no command exit 1 2 (the two pre-A1 pins)

My first leg-1 run crashed the harness on r.gate.name instead of reporting the failure. Two accessors are now null-safe, so a regression is reported rather than thrown (77480b13ce).

Local gates, at 77480b13ce

  • node scripts/pm/dispatch-gates.mjs --repo objectstack-ai/objectstack --ran FILE reports 30 derived, 30 run, 0 NOT-MEASURED, 0 UNRUN, with an exit code recorded for every one (all 0). That count includes pnpm check:pm-dispatch-gates (EXIT=0, 397 s), node scripts/pm/ci-failure.mjs --self-test, node scripts/check-self-test-workflow-commands.mjs (its end-to-end case runs this self-test and finds no runner-parsed line), pnpm check:nul-bytes and pnpm check:entry-guard.
  • The rule that an edited tool's own test suites are owed: node scripts/pm/check-governed-merges.mjs --self-test (EXIT=0, 500 assertions; it reads this file's unchanged PROXY_REARM_GUARD line).
  • eslint, narrowed to this one file:
    • pnpm lint is eslint . --no-inline-config, and eslint --print-config scripts/pm/ci-failure.mjs resolves a config for it, so the file is inside the lint population.
    • --no-inline-config --format json scripts/pm/ci-failure.mjs reports 1 file, 0 errors, 0 warnings.
    • The printed config has no parserOptions.project (eslint.config.mjs never enables type-aware linting), and the two rules that apply are per-file syntactic rules. So this diff cannot change the verdict for any file it does not touch.
  • No changeset: scripts/pm/** publishes nothing (skip-changeset).

Acceptance notes

  • A gate name that appears in more than one workflow step still resolves to every hit. This is the existing documented resolveStep behaviour ("picking one would be a silent guess"). One example: Sweep the built closure for vanished declarations is a step in ci.yml build-core and in two lint.yml lanes, and the two lint steps share one id:. So the annotation's id cannot narrow it either. Narrowing by job would need a mapping from job display names to job ids, which is outside this card.
  • The card's optional references/platform-readings.md half is not in this PR: it is a governed .claude/** path, and the claim leaves it to skills(pm-dispatch): a text-only DELIBERATE CORRECTION is confirmed by the seat's ACCEPT quoting before/after; the second same-head subagent review is dropped for that class (#23177 ruling B, item 4) #23201 or a skills card. The platform reading it would record is quoted in this file's header instead: under continue-on-error, a red gate's API conclusion is success and only the steps-context outcome is failure.
  • The Lint & Repo Gates aggregator row (Verify every lint lane succeeded) still offers its own aggregator script as the substitute. It names the red lane in an annotation (lint lane lint-repo-tooling concluded failure), and the lane's own row, fixed here, carries the gate commands. Noted only.

Generated by Claude Code

…es' own commands

Since every gate step of a lint/typecheck lane runs under continue-on-error,
a red gate's API conclusion is success and the lane's one failed step is
Lane verdict, whose name resolved once per lane to the verdict command
(exit 3 locally). The red gate names are read from the verdict's sentences
in the annotations and the job log's error lines, and each resolves to the
gate's own run: block. A red verdict that names no red gate says so; an
unparseable one is reported as UNRESOLVED, never as the verdict command.

Co-authored-by: Claude <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012zwsLMxQBwHUYJgPyrpEvx
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

size/l skip-changeset PR has no user-facing published change; bypasses the changeset gate

Projects

None yet

2 participants