Repository navigation
ci(lint): report how many gates never ran when Lint & Repo Gates aborts - #17678
Conversation
`Lint & Repo Gates` runs ~161 gates as separate Actions steps and stops at the first non-zero exit. That abort is deliberate and adjudicated (#13690); what was missing is the number it leaves behind. One red is a LOWER BOUND on the problems in the tree, and a gate that never executed produces the same "no output" a passing gate does. Measured on run 34468505490 before writing this: the jobs API and the web UI do mark the un-run steps `skipped` (112 of them), but never count them, and use that same word for a step its own `if:` turned off. The check-run `output` was null on every field and the job log said nothing at all. Adds `scripts/report-unmeasured-gate-tail.mjs` plus one `if: failure()` step that prints the count to the job log, the job summary and one `notice` annotation. Fail-fast is untouched, no gate moved, no `continue-on-error`. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_012GKcPZbMoGq7WPzKLfRBTU
Caught by the live double-failure probe (run 34583803573), not by reasoning: the reporter runs while the job is still in progress, and the runner stamps `skipped` on the steps behind the failure progressively. The completed job ended with 161 skipped declared steps; at the instant the reporter read the API only 12 carried that word, so it printed `never_ran=12` — a lower bound on a lower bound, wearing the look of a precise number. Behind the failure the tail is now counted by POSITION, and `conclusion` is used only to subtract the steps that demonstrably did run. A mid-run fixture pins it. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_012GKcPZbMoGq7WPzKLfRBTU
…ring Run 34584689309 ended its own report with `- Report how many gates never ran`, while that step was the one printing it: the jobs API had not yet marked the step `in_progress`, so the in-flight rule did not catch it. The workflow now hands the step its own name through OS_TAIL_REPORT_STEP and the reporter subtracts it. The two lines agreeing is not left to a comment — `wiringVerdict()` reads lint.yml as text and the self-test fails on a stale value, a missing one, or a rename. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_012GKcPZbMoGq7WPzKLfRBTU
…only too Acceptance clause 2 of the card is that an all-green head's output does not change at all. An unconditional self-test step would have printed one line on every green run, and one line is still new noise. Both new steps now carry `if: failure()`, so a green run of this job produces exactly the bytes it did before: no log output, no summary, no annotation. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_012GKcPZbMoGq7WPzKLfRBTU
|
PM 复核:收下,已 undraft + 武装。两个 open question 都判了,第一个是本席的错。 本席自己读过的(⛔ 不是核对你的报告)
⭐⭐ 你自己抓到了邻居注释被改成假的
本班有一个 PR(#17353)正是因为没做这件事被退回的。你不用被提醒就做了,而且把「改的是措辞、不是裁决」这一点写在了原地。 ⭐ 前提被纠正了两次 —— 这正是派单要的
⭐ 而两个「推理推不出、只有真跑才暴露」的缺陷,你是用真 CI 跑出来的并且都钉住了:runner 是渐进打 四条验收各有一次真实 run 做证,含逐 run 的 job id —— 特别是消融那条把 1.
|
Closes #17382
Clause-②: no
Lint & Repo Gatesruns its gates as a sequence of Actions steps and stops atthe first non-zero exit. That abort is deliberate and adjudicated — the
GATE ORDER IS DELIBERATEnote abovejobs:inlint.ymlcarries amaintainer ruling (#13690, 2026-09-03) and even names the quantity the ordering
is tuned to shrink: "the unmeasured tail". Nothing anywhere computed it.
This PR computes it. Fail-fast is untouched, no gate moved, no
continue-on-erroranywhere.The card's premise, re-measured — and narrowed
The card says the gates are "
bash -e下的顺序步骤" and that the output contains"没有任何东西说它们没跑" (nothing that says they did not run). Measured on
origin/mainatd46deba195, both halves need correcting, and the second onenarrows the defect. The narrowed claim is what this PR rests on.
The gates are not lines in one
bash -escript: thelintjob declares 168separate
- name:steps (6 setup + 1 gate-family selector + 161 gates). Theabort mechanism is Actions step sequencing. That difference matters, because it
changes what a reader receives — measured on three surfaces separately, against
run 34468505490 (job 102842713265,
Lint & Repo Gates, failed at step #55):steps[]conclusion: "skipped""skipped"is also the word for a step its ownif:turned off, so the one surface that says it cannot tell the two apartoutputtitle,summary,textallnull; onlyannotations_count: 1, the failing gate's own.GITHUB_STEP_SUMMARYappears 0 times inlint.yml, so there is no writer for that surfaceProcess completed with exit code 1,Post job cleanup. Not one line about the 112 behind it⇒ The honest defect is not "nothing says it". It is: no surface anywhere
carries a COUNT, and the two surfaces read by the API and by this repo's own
agent seats carry nothing at all. That is still the card's acceptance #1.
Two more readings taken at the same time:
continue-on-errorappears 0 timesas a key in
lint.yml(one occurrence, in prose inside a comment), andGITHUB_STEP_SUMMARY0 times.The card's second measurement: are there real inter-gate dependencies?
Probed statically over the 168 steps, in five edge kinds, with a positive
control — the probe had to find a dependency known to exist before its zero
could mean anything:
steps.<id>.outputs.*consumptionSelect the gate families this run pays for(id: gate-families) into theif:of five later gates. This is the positive control: a real, known dependency the probe reaches.$GITHUB_ENV/$GITHUB_PATH/$GITHUB_OUTPUTwritesGet pnpm store directory)envrun:bodies redirecting output to a repo pathrun:could mutate the tree (gen:/regen/--write/--fix/sed -i/mv/cp/tee)os-regen-merge self-test, matched on the word "regen" in a script name, not on an observed writeBetween the 161 gate steps themselves: zero edges. Every dependency found
runs from the seven preamble steps into the gates, never gate-to-gate.
⛔ Boundary, stated so the zero is not over-read: this probe is static and
structural. It cannot see a dependency mediated by an unnamed side effect on the
working tree. Kind ④/⑤ are proxies for that, not proofs. The reading is
therefore "no declared inter-gate dependency", not "no dependency can exist" —
and it is reported for the record, because this PR does not need it: it keeps
fail-fast, so nothing here can produce the cascading false reds that making all
gates run would risk.
What this adds
scripts/report-unmeasured-gate-tail.mjs— asks the jobs API for this run'sown job and counts three populations apart: what ran, what its own
if:turned off before the failure (a deliberate absence), and what is behind
the failure (the unmeasured tail). Prints the count and every un-run gate
by name to the job log, to
$GITHUB_STEP_SUMMARY, and as one machine-readableline, plus a single
noticeannotation carrying that line.lintjob, bothif: failure()— an offlineself-test and the reporter — and
actions: readon the job (read-only, andthe only permission change).
Design points worth a reviewer's eye:
job is already failed and always exits 0; a non-zero exit there would only add
a second annotation pointing at the reporter instead of at the gate. It emits a
notice, never awarningorerror— on an already-failed check a notice canbe read neither as a pass nor as a downgrade. There is no
continue-on-error.than printing
0. That distinction is the whole subject of the card.ends in "… and 94 more" has reproduced the defect one level up.
name:. It finds its own job byrunner_name(=
RUNNER_NAME), so the required-status-check context literal thatlint.ymlpins is not copied anywhere and a rename cannot make the reporter silently stop
matching.
Acceptance, measured on real runs
The probe branch
claude/issue-17382-tail-probe(deleted) carried twoindependent planted failures: a broken docs anchor (reds gate step #9, the
first gate) and an unquoted workflow step name (reds a gate ~160 steps behind
it). Its
lint.ymlblob was byte-identical toorigin/main's in the ablatedstate — verified by
git hash-object.1. Positive — run 34585740119 (job 103219414669). Failed at step #9. The job log now ends:
The reader learns from one job run that 161 gates are unmeasured — including the
one carrying the second planted bug, named in the list.
2. Negative control — run 34586827251 (job 103222868848), this branch, all green.
Lint & Repo Gates= success, 173 steps success, exactly 2 skipped: both newsteps.
started_at == completed_aton both. check-runannotations_count: 0;output.title/summary/textallnull; the log goes straight from thelast gate to
Post job cleanup.⇒ zero new bytes on a green run — no log output, no summary, no annotation.
Literal byte-identity between two different runs is not achievable (timestamps,
shas, cache keys differ), so the claim made and measured here is the one that
matters: this change contributes nothing to a green run's output. That is also
why the self-test step carries
if: failure()— running it unconditionallywould have printed one line on every green run, and one line is still new noise.
3. No downgrade.
continue-on-errorcount inlint.yml: 0 before, 0 after.Every gate's exit code still decides the job.
4. Ablation — run 34589155445 (job 103230201133). Same
double-failure head, change removed (
lint.ymlrestored to theorigin/mainblob, script deleted). Result: failed at step #9, 162 steps
skipped,annotations_count: 1(the failing gate's own),output.summary: null, andnothing anywhere about the 162 gates that never ran. Acceptance #1 reverts
exactly.
Two defects the live runs caught that reasoning did not
Recorded because both were invisible offline, and both are now pinned:
never_ran=12for a tail of 161. The reporter runswhile the job is still in progress, and the runner stamps
skippedon thesteps behind the failure progressively — at that instant only 12 carried the
word. Counting
conclusion === 'skipped'produced a lower bound on a lowerbound, wearing the look of a precise number. Behind the failure the tail is now
counted by position, and
conclusionmay only subtract steps thatdemonstrably ran. A mid-run fixture pins it.
- Report how many gates never ran— the step that was printing it, because the API had not yet marked itin_progress. The workflow now hands the step its own name viaOS_TAIL_REPORT_STEP, and that the two lines agree is not left to a comment:wiringVerdict()readslint.ymlas text and the self-test fails on a stalevalue, a missing one, or a rename. Reverse-verified — mutated the env value on
disk (blob
135f9caa→5f57f8e7), self-test went red naming the drift,restored via
git checkout HEAD --, blob back to135f9caa,git diff HEADempty, self-test green again.
Local verification
node scripts/report-unmeasured-gate-tail.mjs --self-test— 32 assertions,exit 0. Driven over recorded jobs-API shapes, and separately validated against
five real recorded jobs of this workflow (tails of 112, 109, 14, 0 and 161).
node scripts/pm/dispatch-gates.mjs --commandsderived 56 families; all 56 runwith exit codes recorded and reconciled via
--ran: 55 green, 1 NOTMEASURED —
pnpm check:type-check-debtexits 3 (PREREQUISITE NOT MET: theworkspace type closure is not built locally;
lint.ymlbuilds it before thatstep). Not a finding, and unrelated to this diff.
pnpm exec eslint . --no-inline-configover the whole repo — 6624 files, 0errors, 0 warnings, exit 0. Not narrowed; the full population ran.
Changeset
skip-changeset, measured rather than assumed: walked 77 workspace manifests(70 non-private) and no
files[]entry can carry rootscripts/or.github/; the root manifest isprivate: true. Nothing published moves. Thelabel is applied on the PR.
Deliberately not touched
lintjob'sname:— it is the required-status-check context.lint.yml:135's "~70 sequential gate steps" prose (the real count is 168).Unenforced stale prose, ruled not a finding for this PR; left alone.
duration-unit-keys step's "Placed LAST in this job" now reads "Placed LAST
AMONG THE GATES", since two non-gate steps now follow it. The ruling it
describes — nothing behind it is masked by its red — is untouched and still
true: neither new step is a gate, and neither can be masked.
🤖 Generated with Claude Code
https://claude.ai/code/session_012GKcPZbMoGq7WPzKLfRBTU
Generated by Claude Code