Skip to content

ci(lint): report how many gates never ran when Lint & Repo Gates aborts - #17678

Merged
claude[bot] merged 4 commits into
mainfrom
claude/issue-17382-lint-gates-unmeasured-tail
Sep 11, 2026
Merged

claude[bot] merged 4 commits into
mainfrom
claude/issue-17382-lint-gates-unmeasured-tail

Conversation

@claude

@claude claude Bot commented Sep 11, 2026

Copy link
Copy Markdown
Contributor

Closes #17382

Clause-②: no

Lint & Repo Gates runs its gates as a sequence of Actions steps and stops at
the first non-zero exit. That abort is deliberate and adjudicated — the
GATE ORDER IS DELIBERATE note above jobs: in lint.yml carries a
maintainer ruling (#13690, 2026-09-03) and even names the quantity the ordering
is tuned to shrink: "the unmeasured tail". Nothing anywhere computed it.

This PR computes it. Fail-fast is untouched, no gate moved, no
continue-on-error anywhere.

The card's premise, re-measured — and narrowed

The card says the gates are "bash -e 下的顺序步骤" and that the output contains
"没有任何东西说它们没跑" (nothing that says they did not run). Measured on
origin/main at d46deba195, both halves need correcting, and the second one
narrows the defect. The narrowed claim is what this PR rests on.

The gates are not lines in one bash -e script: the lint job declares 168
separate - name: steps
(6 setup + 1 gate-family selector + 161 gates). The
abort mechanism is Actions step sequencing. That difference matters, because it
changes what a reader receives — measured on three surfaces separately, against
run 34468505490 (job 102842713265, Lint & Repo Gates, failed at step #55):

surface does it say the rest did not run? does it give a count?
Actions Web UI / jobs API steps[] yes — 112 declared steps came back conclusion: "skipped" no, and "skipped" is also the word for a step its own if: turned off, so the one surface that says it cannot tell the two apart
check-run output no — title, summary, text all null; only annotations_count: 1, the failing gate's own. GITHUB_STEP_SUMMARY appears 0 times in lint.yml, so there is no writer for that surface no
job log (what an API/agent reader consumes) no — the log ends with the failing gate's output, Process completed with exit code 1, Post job cleanup. Not one line about the 112 behind it no

⇒ The honest defect is not "nothing says it". It is: no surface anywhere
carries a COUNT, and the two surfaces read by the API and by this repo's own
agent seats carry nothing at all.
That is still the card's acceptance #1.

Two more readings taken at the same time: continue-on-error appears 0 times
as a key in lint.yml (one occurrence, in prose inside a comment), and
GITHUB_STEP_SUMMARY 0 times.

The card's second measurement: are there real inter-gate dependencies?

Probed statically over the 168 steps, in five edge kinds, with a positive
control
— the probe had to find a dependency known to exist before its zero
could mean anything:

edge kind found
① steps.<id>.outputs.* consumption 5 — all from step 7 Select the gate families this run pays for (id: gate-families) into the if: of five later gates. This is the positive control: a real, known dependency the probe reaches.
② $GITHUB_ENV / $GITHUB_PATH / $GITHUB_OUTPUT writes 1, in setup step 4 (Get pnpm store directory)
③ job-level env none declared
④ run: bodies redirecting output to a repo path 0
⑤ gate steps whose run: could mutate the tree (gen:/regen/--write/--fix/sed -i/mv/cp/tee) 1 candidate — os-regen-merge self-test, matched on the word "regen" in a script name, not on an observed write

Between the 161 gate steps themselves: zero edges. Every dependency found
runs from the seven preamble steps into the gates, never gate-to-gate.

⛔ Boundary, stated so the zero is not over-read: this probe is static and
structural. It cannot see a dependency mediated by an unnamed side effect on the
working tree. Kind ④/⑤ are proxies for that, not proofs. The reading is
therefore "no declared inter-gate dependency", not "no dependency can exist" —
and it is reported for the record, because this PR does not need it: it keeps
fail-fast, so nothing here can produce the cascading false reds that making all
gates run would risk.

What this adds

  • scripts/report-unmeasured-gate-tail.mjs — asks the jobs API for this run's
    own job and counts three populations apart: what ran, what its own if:
    turned off before the failure (a deliberate absence), and what is behind
    the failure (the unmeasured tail). Prints the count and every un-run gate
    by name to the job log, to $GITHUB_STEP_SUMMARY, and as one machine-readable
    line, plus a single notice annotation carrying that line.
  • Two steps at the end of the lint job, both if: failure() — an offline
    self-test and the reporter — and actions: read on the job (read-only, and
    the only permission change).

Design points worth a reviewer's eye:

  • It never fails the job and never downgrades anything. It runs only when the
    job is already failed and always exits 0; a non-zero exit there would only add
    a second annotation pointing at the reporter instead of at the gate. It emits a
    notice, never a warning or error — on an already-failed check a notice can
    be read neither as a pass nor as a downgrade. There is no continue-on-error.
  • When it cannot measure, it says NOT MEASURED and names the reason rather
    than printing 0. That distinction is the whole subject of the card.
  • It never truncates the list it is about. A report on an unmeasured tail that
    ends in "… and 94 more" has reproduced the defect one level up.
  • It hardcodes no job name:. It finds its own job by runner_name
    (= RUNNER_NAME), so the required-status-check context literal that lint.yml
    pins is not copied anywhere and a rename cannot make the reporter silently stop
    matching.

Acceptance, measured on real runs

The probe branch claude/issue-17382-tail-probe (deleted) carried two
independent planted failures
: a broken docs anchor (reds gate step #9, the
first gate) and an unquoted workflow step name (reds a gate ~160 steps behind
it). Its lint.yml blob was byte-identical to origin/main's in the ablated
state — verified by git hash-object.

1. Positive — run 34585740119 (job 103219414669). Failed at step #9. The job log now ends:

  failed at step #9: Docs anchors resolve to real headings
  declared steps in this job:            170
  ran before the failure:                7
  turned off by their own condition:     0
  NEVER RAN (the unmeasured tail):       161
  ...
    - Unquoted workflow step names do not silently truncate at " #" (#15149)
    - Duration-shaped spec keys carry their unit in the key name
    - Unmeasured-gate-tail reporter self-test

  unmeasured-gate-tail: measured=yes never_ran=161 failed=1 ran=7 skipped_by_condition=0 declared=170 failed_at_step=9

The reader learns from one job run that 161 gates are unmeasured — including the
one carrying the second planted bug, named in the list.

2. Negative control — run 34586827251 (job 103222868848), this branch, all green.
Lint & Repo Gates = success, 173 steps success, exactly 2 skipped: both new
steps. started_at == completed_at on both. check-run annotations_count: 0;
output.title / summary / text all null; the log goes straight from the
last gate to Post job cleanup.

⇒ zero new bytes on a green run — no log output, no summary, no annotation.
Literal byte-identity between two different runs is not achievable (timestamps,
shas, cache keys differ), so the claim made and measured here is the one that
matters: this change contributes nothing to a green run's output. That is also
why the self-test step carries if: failure() — running it unconditionally
would have printed one line on every green run, and one line is still new noise.

3. No downgrade. continue-on-error count in lint.yml: 0 before, 0 after.
Every gate's exit code still decides the job.

4. Ablation — run 34589155445 (job 103230201133). Same
double-failure head, change removed (lint.yml restored to the origin/main
blob, script deleted). Result: failed at step #9, 162 steps skipped,
annotations_count: 1 (the failing gate's own), output.summary: null, and
nothing anywhere about the 162 gates that never ran. Acceptance #1 reverts
exactly.

Two defects the live runs caught that reasoning did not

Recorded because both were invisible offline, and both are now pinned:

  1. Run 34583803573 printed never_ran=12 for a tail of 161. The reporter runs
    while the job is still in progress, and the runner stamps skipped on the
    steps behind the failure progressively — at that instant only 12 carried the
    word. Counting conclusion === 'skipped' produced a lower bound on a lower
    bound, wearing the look of a precise number. Behind the failure the tail is now
    counted by position, and conclusion may only subtract steps that
    demonstrably ran. A mid-run fixture pins it.
  2. Run 34584689309 ended its own report with - Report how many gates never ran — the step that was printing it, because the API had not yet marked it
    in_progress. The workflow now hands the step its own name via
    OS_TAIL_REPORT_STEP, and that the two lines agree is not left to a comment:
    wiringVerdict() reads lint.yml as text and the self-test fails on a stale
    value, a missing one, or a rename. Reverse-verified — mutated the env value on
    disk (blob 135f9caa → 5f57f8e7), self-test went red naming the drift,
    restored via git checkout HEAD --, blob back to 135f9caa, git diff HEAD
    empty, self-test green again.

Local verification

  • node scripts/report-unmeasured-gate-tail.mjs --self-test — 32 assertions,
    exit 0. Driven over recorded jobs-API shapes, and separately validated against
    five real recorded jobs of this workflow (tails of 112, 109, 14, 0 and 161).
  • node scripts/pm/dispatch-gates.mjs --commands derived 56 families; all 56 run
    with exit codes recorded and reconciled via --ran: 55 green, 1 NOT
    MEASURED
    — pnpm check:type-check-debt exits 3 (PREREQUISITE NOT MET: the
    workspace type closure is not built locally; lint.yml builds it before that
    step). Not a finding, and unrelated to this diff.
  • pnpm exec eslint . --no-inline-config over the whole repo — 6624 files, 0
    errors, 0 warnings, exit 0
    . Not narrowed; the full population ran.
  • Gate list re-derived at the final head: unchanged.

Changeset

skip-changeset, measured rather than assumed: walked 77 workspace manifests
(70 non-private) and no files[] entry can carry root scripts/ or
.github/; the root manifest is private: true. Nothing published moves. The
label is applied on the PR.

Deliberately not touched

🤖 Generated with Claude Code

https://claude.ai/code/session_012GKcPZbMoGq7WPzKLfRBTU


Generated by Claude Code

`Lint & Repo Gates` runs ~161 gates as separate Actions steps and stops at
the first non-zero exit. That abort is deliberate and adjudicated (#13690);
what was missing is the number it leaves behind. One red is a LOWER BOUND on
the problems in the tree, and a gate that never executed produces the same
"no output" a passing gate does.

Measured on run 34468505490 before writing this: the jobs API and the web UI
do mark the un-run steps `skipped` (112 of them), but never count them, and
use that same word for a step its own `if:` turned off. The check-run
`output` was null on every field and the job log said nothing at all.

Adds `scripts/report-unmeasured-gate-tail.mjs` plus one `if: failure()` step
that prints the count to the job log, the job summary and one `notice`
annotation. Fail-fast is untouched, no gate moved, no `continue-on-error`.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012GKcPZbMoGq7WPzKLfRBTU
Caught by the live double-failure probe (run 34583803573), not by reasoning:
the reporter runs while the job is still in progress, and the runner stamps
`skipped` on the steps behind the failure progressively. The completed job
ended with 161 skipped declared steps; at the instant the reporter read the
API only 12 carried that word, so it printed `never_ran=12` — a lower bound
on a lower bound, wearing the look of a precise number.

Behind the failure the tail is now counted by POSITION, and `conclusion` is
used only to subtract the steps that demonstrably did run. A mid-run fixture
pins it.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012GKcPZbMoGq7WPzKLfRBTU
…ring

Run 34584689309 ended its own report with `- Report how many gates never
ran`, while that step was the one printing it: the jobs API had not yet
marked the step `in_progress`, so the in-flight rule did not catch it.

The workflow now hands the step its own name through OS_TAIL_REPORT_STEP and
the reporter subtracts it. The two lines agreeing is not left to a comment —
`wiringVerdict()` reads lint.yml as text and the self-test fails on a stale
value, a missing one, or a rename.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012GKcPZbMoGq7WPzKLfRBTU
…only too

Acceptance clause 2 of the card is that an all-green head's output does not
change at all. An unconditional self-test step would have printed one line on
every green run, and one line is still new noise. Both new steps now carry
`if: failure()`, so a green run of this job produces exactly the bytes it did
before: no log output, no summary, no annotation.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012GKcPZbMoGq7WPzKLfRBTU
@claude claude Bot added the skip-changeset PR has no user-facing published change; bypasses the changeset gate label Sep 11, 2026
@github-actions github-actions Bot added the ci/cd label Sep 11, 2026
@claude
claude Bot marked this pull request as ready for review September 11, 2026 10:45
@claude
claude Bot enabled auto-merge September 11, 2026 10:45

Copy link
Copy Markdown
Contributor

PM 复核:收下,已 undraft + 武装。两个 open question 都判了,第一个是本席的错。

本席自己读过的(⛔ 不是核对你的报告)

  • permissions: 只加了 actions: read,附了理由;⛔ 没有写权限。
  • 两个新步骤都带 if: failure() ⇒ 绿色 head 上零字节。⭐ 你给 self-test 也加上这个条件、并在注释里写明「一行也是新噪音,这个改动不许花任何代价」—— 这一步很多人会漏,无条件跑的 self-test 会在每一次绿跑上留一行。
  • 报告步骤 run: node … 不带 set -e 之外的花样,且你写明它总是 exit 0,理由是 job 此刻已经失败,非零只会多出一条指向报告器而不是指向门禁的注解。⛔ 这条很重要:验收第 3 条禁止的是「把失败降级」,而一个会自己变红的报告器等于在红上再加一条假红。
  • ⛔ continue-on-error 作为 key 的出现次数:改动前后都是 0。

⭐⭐ 你自己抓到了邻居注释被改成假的

Placed LAST in this job on purpose … nothing behind it would be masked by its red —— 在它后面追加两个步骤,这句话就不再成立。你把它改成 Placed LAST **AMONG THE GATES**,并写明「那两个后置步骤不是门禁、遮不住任何东西;限定词是本 PR 加的,它描述的那条裁决未动,没有任何门禁的相对顺序移动过」。

本班有一个 PR(#17353)正是因为没做这件事被退回的。你不用被提醒就做了,而且把「改的是措辞、不是裁决」这一点写在了原地。

⭐ 前提被纠正了两次 —— 这正是派单要的

premise_still_valid: false,而你给出了收窄后为真的版本:门禁是 168 个独立 Actions 步骤(不是 bash -e 脚本里的行),而且 jobs API 与 Web UI 确实把未跑的步骤标成 skipped ⇒ 卡面「没有任何东西这么说」过强。收窄后的真缺陷是三句:没有任何一个面带计数;唯一会说话的那个面对「被自己 if: 关掉」和「根本没轮到」用同一个词;而 API / agent 读的那两个面(job log、check-run output)什么都没有。

⭐ 而两个「推理推不出、只有真跑才暴露」的缺陷,你是用真 CI 跑出来的并且都钉住了:runner 是渐进打 skipped 的(mid-run 只有 12 个带这个词,尾巴其实 161),以及报告器把自己列进了没跑的名单。第二个的修法(把步骤名经 OS_TAIL_REPORT_STEP 交给它,并让 wiringVerdict() 读 lint.yml 文本,漂移即红)⭐ 是把一个「会安静出错」的东西变成「不可能安静出错」。

四条验收各有一次真实 run 做证,含逐 run 的 job id —— 特别是消融那条把 lint.yml 还原成 origin/main 的 blob 并用 git hash-object 验过逐字节相同,再跑同一个双失败 head,162 skipped / output.summary null / 没有任何地方提到那 162 个。验收第 1 条准确地回到了今天的样子。

1. Clause-② 载体 —— 采纳 A + C,A 已做

⭐ 这是本席的错,而且是同一族的第三次。 简报只告诉你写 PR 正文那个载体;但这条款有两个载体,卡面那个由认领席写,check-clause2-carriers.mjs:946 明文禁止 dev 代填。你两边都没做错:正文那条你写了并自验过,卡面那条你不能碰。

已把 #17382 的 claim 评论(5631792339)第二行补成 Clause-②: no,并用从 checker 源码里读出的那条正则验过:恰好 1 行命中,值 no。判据是本席自量的交付 diff —— 仓库根 manifest private: true,没有任何包的 files[] 能携带根 scripts/ 或 .github/ ⇒ 没有已发布的东西移动。

C 也采纳:简报模板的措辞把你导向了一个谓词不读的载体,下一个 dev 会同样被导错。已写进本席的常备清单:两个载体,各自的归属要在简报里点名。

2. 探针分支 —— 本席也删不掉

claude/issue-17382-tail-probe 我用容器的 REST 通道试了 DELETE /git/refs/heads/...,同样 403。⇒ 两个席位都没有删 ref 的权限,这不是你的疏忽。它带着两个刻意种下的缺陷和一个被消融过的 lint.yml,⇒ 记在这里给有权限的人:该分支应当删除,它没有 PR、未合并,merged-branch reaper 不会回收它。

一处不返工的记号

lint.yml 那段被你改写的注释里,…and no gate's relative order moved.) Reads src/ through tsx (no build); self-tests first, wired into 这一行重排后长度跑出了该注释块的折行宽度。本席查过:仓里没有任何门禁管 workflow 注释的折行 ⇒ 纯观感,⛔ 不值一个 CI 往返。若你因别的原因还要再推一次,顺手折一下即可;否则就这样。


Generated by Claude Code

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ci/cd size/l skip-changeset PR has no user-facing published change; bypasses the changeset gate

Projects

None yet

2 participants