Skip to content

Commit c53d8c2

Browse files
baozhoutaoclaude
andauthored
test(devx): pin the merge-queue triage comment's operative guidance prose (#14550)
check-merge-queue-triage-outcome.mjs asserted only the fenced excerpt and the cross-PR aggregation; the timeout-vs-assertion rule, its named exception, and the boundary sentence that lives in the same posted body were readable by nothing in the suite, so deleting them left every gate green (#13967). Adds a whole-body accessor (guidanceProse, built on postedBody, deliberately not excerptLines -- see its docblock) plus one assertion per operative claim in scenario E1, and three matching MUTATIONS rows (M17-M19) so --self-test proves each new assertion can go red. Co-authored-by: Claude <noreply@anthropic.com>
1 parent 1ce8930 commit c53d8c2

2 files changed

Lines changed: 65 additions & 8 deletions

File tree

‎.github/workflows/lint.yml‎

Lines changed: 12 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -2870,24 +2870,28 @@ jobs:
28702870
# line, driven against REAL captured vitest logs of the incident's own
28712871
# test file — one timeout, one assertion, byte-identical FAIL lines and
28722872
# opposite diagnoses, which is the confusion that produced a confidently
2873-
# wrong card (#10112) and cost a whole dispatch; and (2) the cross-PR
2873+
# wrong card (#10112) and cost a whole dispatch; (2) the cross-PR
28742874
# aggregation files exactly ONE anchor issue at >= 2 DISTINCT PRs,
28752875
# REFRESHES it on the next ejection instead of filing a second, and files
28762876
# none for two different keys or for one PR ejecting twice. The refresh
28772877
# leg is driven as a PAIR — run 2's world is built from run 1's output —
28782878
# because idempotency is a property of the pair and a hand-written marker
2879-
# would prove each half while leaving the two runs free to disagree.
2879+
# would prove each half while leaving the two runs free to disagree; and
2880+
# (3) the operative guidance prose (the timeout-vs-assertion rule, its
2881+
# named exception, and the boundary sentence that the re-queue allowance
2882+
# is governed elsewhere) is present in the posted body, one assertion per
2883+
# claim over the WHOLE body, not the excerpt.
28802884
#
28812885
# Its --self-test runs first and is the half that stops the battery
2882-
# rotting into decoration: it mutates the shipped script fourteen ways —
2886+
# rotting into decoration: it mutates the shipped script twenty ways —
28832887
# drop the reason lookahead, test the FAIL pattern before the reason
28842888
# pattern (which silently loses every assertion), count runs instead of
28852889
# distinct PRs, always create instead of refreshing, drop the sighting
2886-
# markers, downgrade each anti-no-op announcement to an info line, and
2887-
# more — and requires the battery to go RED for each, naming the scenario
2888-
# that catches it. A mutation whose anchor no longer exists is a failure
2889-
# too, so a rewrite of the workflow cannot leave the mutations silently
2890-
# matching nothing.
2890+
# markers, downgrade each anti-no-op announcement to an info line, drop
2891+
# each half of the guidance prose, and more — and requires the battery to
2892+
# go RED for each, naming the scenario that catches it. A mutation whose
2893+
# anchor no longer exists is a failure too, so a rewrite of the workflow
2894+
# cannot leave the mutations silently matching nothing.
28912895
#
28922896
# Invoked as `node` rather than through a `pnpm check:*` alias, same
28932897
# reason as the step above: see the GATE INVOCATION IDIOM note at the top

‎scripts/check-merge-queue-triage-outcome.mjs‎

Lines changed: 53 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -505,6 +505,24 @@ const excerptLines = (r) => {
505505
return out;
506506
};
507507

508+
/**
509+
* The whole posted body, for assertions about the operative GUIDANCE PROSE
510+
* (the timeout-vs-assertion rule, its named exception, and the boundary
511+
* sentence) rather than the extracted excerpt.
512+
*
513+
* This deliberately does NOT go through `excerptLines`: that helper is
514+
* scoped to code-fence content precisely so the comment's own explanatory
515+
* prose -- which names both `FAIL` and `AssertionError` -- cannot satisfy an
516+
* assertion about the EXTRACTION (see its docblock). The guidance prose
517+
* lives outside every fence, so `excerptLines` cannot see it at all; going
518+
* through it here would silently pass on a body that shipped `excerptLines`
519+
* still finds the FAIL/reason pair, but with the guidance paragraph deleted.
520+
* The question this accessor answers is different and does not have that
521+
* failure mode: is the guidance text the workflow composes into every
522+
* comment actually present in what it posted?
523+
*/
524+
const guidanceProse = (r) => postedBody(r);
525+
508526
// ── Scenarios ───────────────────────────────────────────────────────────────
509527

510528
const REFUSED = httpError(503, 'No server is currently available to service your request.');
@@ -547,11 +565,19 @@ function scenarios(root) {
547565
world: () => base(),
548566
check(r, t) {
549567
const lines = excerptLines(r);
568+
const prose = guidanceProse(r);
550569
return [
551570
t(lines.some((l) => l.includes(`FAIL ${KEY_A}`)), 'the FAIL line is excerpted'),
552571
t(lines.some((l) => l.startsWith('↳ 失败原因:') && l.includes('Error: Test timed out in 1ms.')),
553572
`the reason line is excerpted and labelled, got: ${JSON.stringify(lines)}`),
554573
t(!lines.some((l) => l.includes('AssertionError')), 'no assertion is reported for a timeout log'),
574+
// ── operative guidance prose, read over the WHOLE posted body ──
575+
t(prose.includes('负载/时序') && prose.includes('真实的行为改变'),
576+
'the timeout-vs-assertion rule is present in the posted body: timeout -> load/timing, assertion -> real behaviour change'),
577+
t(prose.includes('实验自身的有效性前提') && prose.includes('时长、一个时间戳、一个耗时计数'),
578+
'the named exception is present in the posted body: an assertion about the experiment\'s own validity premise (a duration, a timestamp, an elapsed count) reads like a timeout'),
579+
t(prose.includes('后者由别处管'),
580+
'the boundary sentence is present in the posted body: this only changes how a red is read, not which tests may be re-queued'),
555581
];
556582
},
557583
},
@@ -1213,6 +1239,33 @@ const MUTATIONS = [
12131239
to: " core.info('triage comment for this run already exists \u2014 skipping.');",
12141240
expect: ['A5'],
12151241
},
1242+
{
1243+
id: 'M17',
1244+
what: 'the timeout-vs-assertion rule silently loses its "load/timing" half',
1245+
from: '\u8d1f\u8f7d/\u65f6\u5e8f',
1246+
to: '',
1247+
expect: ['E1'],
1248+
// A1 files an anchor from the same triage comment but never reads the
1249+
// guidance prose -- the control that proves this mutation is caught by
1250+
// the new prose assertion, not by every scenario reacting to any change.
1251+
keepGreen: ['A1'],
1252+
},
1253+
{
1254+
id: 'M18',
1255+
what: 'the named exception silently loses the phrase identifying its own object -- the experiment\'s own validity premise',
1256+
from: '\u5b9e\u9a8c\u81ea\u8eab\u7684\u6709\u6548\u6027\u524d\u63d0',
1257+
to: '',
1258+
expect: ['E1'],
1259+
keepGreen: ['A1'],
1260+
},
1261+
{
1262+
id: 'M19',
1263+
what: 'the boundary sentence -- this only changes how a red is read, not which tests may be re-queued -- silently drops',
1264+
from: '\u540e\u8005\u7531\u522b\u5904\u7ba1',
1265+
to: '',
1266+
expect: ['E1'],
1267+
keepGreen: ['A1'],
1268+
},
12161269
];
12171270

12181271
// Returned by `selfTest()` only after its verdict is printed. The dispatch

0 commit comments

Comments
 (0)