Skip to content

ci(timing gates): the two timing steps report-only on every event, schedule included (#22949 ruling E, item 1) - #23063

Merged
os-zhuang merged 3 commits into
mainfrom
claude/issue-23058-timing-gates-report-only
Oct 12, 2026
Merged

os-zhuang merged 3 commits into
mainfrom
claude/issue-23058-timing-gates-report-only

Conversation

@objectstack-fleet

Copy link
Copy Markdown
Contributor

Fixes #23058

Clause-②: no

Both Test Core timing steps, "Check this shard's timing drift" and "Check the per-package suite-duration ceilings", are now report-only on every event, schedule included. A package over its ceiling or a shard past its drift bound prints the same table, raises ::warning:: annotations and exits 0. Ruling: #22949 letter E, item 1, the maintainer's word 「22949 同意」 (#22949 (comment)). A refusal (a missing --event, a malformed ceiling table or raise record, a dataset or run summary that does not parse) stays red on every event, as ACCEPT 6119677604 decided.

The diagnosis the ruling rests on, in two sentences: a wall-clock reading on a shared runner measures the run's load as much as the code (the same package reads 82 s on a quiet hour and 209 s on a busy one, 2.56x, while the ceiling formula allows 2.0x of weight); the baseline window was mostly quiet-density and every event, schedule included, meets both densities, so busy hours red the same near-ceiling packages whatever their code does: density, not code.

What changed

  • scripts/partition-test-shards.mjs: TIMING_GRADED_EVENT is removed. timingGateMode(event) returns report for every event and still throws on a missing or empty one. renderDriftVerdict has one DRIFT path: one ::warning:: annotation, exit 0. timingModeNote says the mode once. Header carries the ruling URL and the two-sentence diagnosis. The formula, the 1.3x and 1.5x constants and the dataset are untouched.
  • scripts/check-test-suite-ceilings.mjs: imports lose TIMING_GRADED_EVENT; renderCeilingVerdict has one OVER path (one ::warning:: per package over, exit 0; REFUSED still exits 1). Header carries the ruling URL and the diagnosis, the exit-code section is rewritten. CEILING_FLOOR_SECONDS, RULED_CEILING_RAISES and the dataset reader are untouched. RED_STATUSES and the local red are renamed OVER_STATUSES and overs, because nothing is red any more.
  • .github/workflows/ci.yml: comments only. The step names, --event "$GITHUB_EVENT_NAME", if: and continue-on-error of both steps are byte-identical (checked: no non-comment line of the file changed). The drift step and the ceilings step say why no event grades; the aggregator note that called the ceilings step "a gate that DOES red the job" now says it reds only on a refusal.
  • Self-tests: every event is a report case. The graded-mode cases became report-mode cases, the refusal cases are unchanged, and each file gained a sweep case that fails if any tier on any event exits non-zero or raises an ::error::. Both batteries keep their pinned case floor (8); the ceilings self-test still reads 58 cases across 9 batteries.

The spelling of the mode

TIMING_GRADED_EVENT is removed rather than set to a value no event matches. With a never-matching constant the exit-1 branch and its ::error:: annotation would stay in the file with nothing able to reach them, and a dead branch reads as a switch someone may flip. With the constant gone there is no graded mode to select, the self-test says it as one line (every event reads report), and grading a timing again is a new ruling and a new edit, not a flag. timingGateMode keeps its name and its refusal, which is what the ruling and ACCEPT 6119677604 name.

Verification

At head 8197058 (after merging origin/main):

  • node scripts/check-test-suite-ceilings.mjs --self-test: "self-test OK (58 cases across 9 batteries)", exit 0.
  • node scripts/partition-test-shards.mjs --self-test: "self-test OK (72 measured packages, 74 shard items, 6 shards, ...)", exit 0.
  • CLI on fixtures, ceilings (live dataset, @objectstack/client ceiling 179.74 s, fixture run 209 s): --event schedule, merge_group, pull_request, push and workflow_dispatch each exit 0 with exactly one ::warning:: and no ::error::; no --event exits 1 with the refusal; a fixture within its ceiling exits 0 with no annotation.
  • CLI on fixtures, drift (@objectstack/client predicted 111.17 s): measured 180 s (DRIFT, 1.62x), 150 s (WARN, 1.35x) and 100 s (OK) each exit 0 on all five events, with one ::warning:: for DRIFT and WARN and none for OK; no --event exits 1.
  • node scripts/pm/dispatch-gates.mjs --commands --repo objectstack-ai/objectstack derived 59 commands from this diff, plus pnpm check:doc-authoring which I added: 56 exit 0, 4 exit 3 (below). --ran reconciles: 59 derived, 55 run, 4 NOT-MEASURED, 0 UNRUN. The parity checkers named in ACCEPT 6119677604 are all in the 56 green: self-test-workflow-commands, step-collectors, workflow-status-functions, required-contexts, aggregator-roster, ratchet-remedy-authority, stall-guard-budget, stall-guard-headroom, workflow-step-name-quoting, shard-attestation. So are check:nul-bytes, check:pm-dispatch-gates and check:entry-guard.
  • NOT MEASURED, exit 3 PREREQUISITE NOT MET (no dist/ in a fresh worktree): check:dts-closure, check:dual-build-cjs-loads, check:lean-entry-closure, check:sourcemap-no-sources-content. They are Build Core's family, derived because ci.yml is on that job's path filter. This diff touches no package, so no dist/ content can differ; Build Core builds the closure and runs them.
  • Reverse verification (fix committed first, each leg through scripts/ablation-replace.mjs in wrap mode, restore proved by blob hash equal to HEAD and an empty git diff HEAD): re-grading ceilings on schedule (exitCode: event === 'schedule' ? 1 : 0) turned the ceilings self-test red ("over: exit 1, verdict OVER"); the same in the DRIFT return turned the partition self-test red ("a DRIFT rendered DRIFT exit 1"); timingGateMode returning grade for schedule turned it red ("schedule read a mode other than report"). Direction: red in all three, the common case. These are one-time proofs; no permanent test file was added for them.
  • Not applicable: package pnpm test and pnpm typecheck (no package changed), changeset (workflow and tooling only, skip-changeset).

Acceptance notes

  • Declared file surface: the claim named the comments of the two timing steps. One more comment changed, the aggregator note near ci.yml:1470, because it called the ceilings step "a gate that DOES red the job" and this PR makes that false. No other edit is outside the claim.
  • The comments in partition-test-shards.mjs above the new header block still say "the red" for the DRIFT bound (they carry the 1.5x measurement). One sentence in the new block says the DRIFT bound is the upper warning tier since this ruling. The self-test battery key "drift warning tier under the red" is unchanged so the pinned roster does not move.
  • A missing --event still surfaces as an uncaught throw with a stack trace, the cosmetic ACCEPT 6119677604 already recorded as non-blocking. Only the message text changed.
  • Regression detection for the wall moves to E2 (ci(refresh lane): each test-timing refresh diffs the window's per-package weights against the previous dataset — a package up 25 % or more, or a file over the 10 s budget, is a filed card for its lane, never a red build (#22949 ruling E, items 2–3) #23059), the dataset refresh diff and the per-file budget; until it lands a real slowdown is visible as the warning annotation only. hourly full run: red on main (CI) #23007 is settled by ruling item 4 through the seat, not by this PR's merge.
  • Acceptance on main after landing: the first hourly schedule run shows the two steps green with their ::warning:: annotations where a package is over. That reading is the seat's, after merge.
  • .github/workflows/ci.yml is under the round's Tier H posture, so this PR stays a draft until a GOVERNED_APPROVERS member approves it.

维护者速读(草稿)

改了什么

主干上每小时一次的定时全量跑(schedule)里,Test Core 的两个计时检查——"单个包的测试耗时上限"和"分片耗时漂移"——不再因为超限而变红,改成只在日志里给一条黄色警告,步骤照样通过。定时、合并队列、PR、手动四类事件一视同仁。耗时公式、20 秒下限、上限提升登记表、数据集和每小时刷新的流程都没有动;ci.yml 里只有注释变了,步骤名、if:、continue-on-error 一个字节没动。

为什么改

这是 #22949 裁决 E 的第 1 项(您的原话「22949 同意」)。同一个包在安静的小时跑 82 秒,在繁忙的小时跑 209 秒(2.56 倍),而上限公式只容许 2.0 倍;所以主干定时跑 4 次里有 3 次变红,红的是机器负载,不是代码,每次都要有人按"主干红"规则去修一个并不存在的回归。

风险与代价(含回滚)

代价:计时变慢不再让构建变红,真正的变慢要等后续的 E2(#23059,数据集刷新时对比差分,外加单文件 10 秒预算)才有系统化的发现手段;在它落地前只有黄色警告可看。风险低:只改计时检查的退出码和注释,测试本身和其它门禁不变;缺少 --event、表格或登记记录格式错误这类"拒收"仍然是红灯,不会悄悄放行。回滚:revert 本 PR 即回到只有 schedule 会红的状态。

席位意见

你要做的

本 PR 改了 .github/workflows/ci.yml,按本轮的受管面姿态保持草稿,请有批准权的维护者审阅后批准,批准后由归口席位入队。合并后看第一次整点的定时跑:两个计时步骤应为绿色,超限的包带黄色警告;随后 #23007 按裁决收口。


Generated by Claude Code

…ncluded (WIP: scripts)

The grading mode goes: TIMING_GRADED_EVENT is removed, timingGateMode reads
report for every event and still refuses a missing one, and a package over its
ceiling or a shard past its drift bound is one ::warning:: annotation and exit 0
on every event. Headers carry the ruling URL and the two-sentence diagnosis;
the graded-mode self-test cases became report-mode cases, the refusal cases are
unchanged. The ci.yml step comments follow in the next commit.

Claude-Session: https://claude.ai/code/session_01LqNgdPZB2Jnr5HpGbaqLeB
Co-authored-by: Claude <noreply@anthropic.com>
Only comments change: the drift step, the ceilings step and the aggregator note
that called the ceilings step a gate. The step names, `--event
"$GITHUB_EVENT_NAME"`, `if:` and `continue-on-error` are byte-identical.

Claude-Session: https://claude.ai/code/session_01LqNgdPZB2Jnr5HpGbaqLeB
Co-authored-by: Claude <noreply@anthropic.com>
@objectstack-fleet

Copy link
Copy Markdown
Contributor Author

维护者速读(终稿)· PR #23063 · 席位已对照 diff 逐行校正,2026-10-12T15:06Z

改了什么:Test Core 的两个计时检查——「单个包的测试耗时上限」和「分片耗时漂移」——在所有事件上都只报告、不判红:超限或漂移时照样打印同一张表、给一条黄色 ::warning:: 注解,步骤退出码 0;此前只有主干每小时的 schedule 运行会判红。「拒收」(缺 --event、上限表或提升登记格式错、数据集或汇总解析失败)仍然是红,不变。ci.yml 只改注释(席位核过:非注释行零改动),步骤名、--event、if:、continue-on-error 原样。耗时公式、20 秒下限、提升登记表、数据集、每小时刷新流程都没动。TIMING_GRADED_EVENT 直接删除而不是改成永不匹配的值,理由是留一条谁也到不了的 exit-1 分支会被当成开关。

为什么改:#22949 裁决 E 第 1 项(「22949 同意」)。同一个包安静时 82 秒、繁忙时 209 秒(2.56 倍),公式只容 2.0 倍;主干整点跑 4 次红 3 次,红的是负载不是代码,每次都要按「主干红」去修一个不存在的回归。

风险与代价(含回滚):代价是变慢不再把构建判红,在 E2(#23059:刷新时对比数据集差分 + 单文件 10 秒预算)落地前只能看黄色注解。风险低:只改退出码和注释,三条「拒收」仍红,不会悄悄放行。本 PR 自己的 run:六个 Test Core 分片、Build Core、Lint 与 Type Check 各车道至发稿时全绿或在跑、无红。回滚:revert 本 PR 回到「只有 schedule 判红」。

席位意见:建议批准。这是裁决 E 的机械落实,diff 与裁决逐条对得上;真正的回归发现手段由 #23059 接棒(已在开发)。

你要做的:在 PR 上 Approve;席位等全部检查绿后翻正式、入队,合并后读第一次整点 schedule 运行回填。

@os-zhuang
os-zhuang marked this pull request as ready for review October 12, 2026 21:37
@os-zhuang
os-zhuang added this pull request to the merge queue Oct 12, 2026
Merged via the queue into main with commit 56986e9 Oct 12, 2026
44 checks passed
@os-zhuang
os-zhuang deleted the claude/issue-23058-timing-gates-report-only branch October 12, 2026 21:55
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ci/cd size/l skip-changeset PR has no user-facing published change; bypasses the changeset gate

Projects

None yet

2 participants