Repository navigation
Commit 535c019
ci(suite-ceilings): a new package is not red before its first ceiling (#22498)
Fixes #16468
Clause-②: no
Implements ruling Q2 B on the suite-duration ratchet
(#16468 (comment)):
a package absent from the ceiling dataset is a new package. It prints
"no ceiling: new package" and is not red. The first refresh after it
lands enters it in the dataset. A package the dataset knows but that
lost its row stays red. Q1 A needs no code (the hold is as built). Part
1 of the ratchet landed as PR #22480 (`6abd265320`).
## What changed
`scripts/check-test-suite-ceilings.mjs`:
- **Grading.** A package absent from the dataset with no ruled entry now
gets row status `new` (it was `absent`). `RED_STATUSES` is `over` alone,
so the exit code follows only an over-ceiling package or a refusal.
- **Rendering.** The row prints `-- no ceiling: new package (not in
scripts/test-shard-timings.json; the first refresh after it lands enters
it)`, in the same column as the `provisional` and `carried` rows. The
red header, the red detail and the error annotation lose their
absent-package branch, because no red row can lack a ceiling now.
- **"Lost its row" stays red** (see the finding below).
`readCeilingTable` already refused a weight with neither a ceiling nor
an `uncapped` reason. It now also refuses a ceiling, an `uncapped`
reason or a `provisional` mark that names a package with no weight.
Without this, that shape would have changed from red (as built, it
graded `absent`) to "no ceiling: new package".
- **Prose the ruling made false**, prose only. The header bullet now
states the ruled rule and cites the ruling URL. Also updated: the NOT
MEASURED paragraph, the exit-code table, the `gradeCeilings` status
table and the `RULED_CEILING_RAISES` comment ("the only way a package
that has never landed can get one").
`.github/workflows/ci.yml`: the comment above `Check the per-package
suite-duration ceilings` said "a package absent from the dataset is red
(the new-package shape)". It now states the ruled rule. Comment only, no
step change.
Sweep for other sentences stating the old rule: `git grep -n -i -E
"absent from the dataset|new-package shape|new package|ruled raise|no
ceiling"` over the tree found only ci.yml lines 1462-1463, and they are
fixed here. The ceiling prose in
`.github/workflows/shard-timings-refresh.yml` and
`scripts/measure-test-shard-timings.mjs` describes the hold and the
formula, not the new-package rule, so it is untouched. Neither file is
in this diff, and neither is `scripts/test-shard-timings.json` or
`scripts/partition-test-shards.mjs`.
## The inverted pin
The battery `'a package absent from the dataset is red; an uncapped one
is not'` (floor 4) is now `'a new package prints "no ceiling: new
package" and is not red; a known one that lost its row is refused'`
(floor 10):
1. A new package (9999 s) beside one within its ceiling: row status
`new`, exit 0, verdict `OK`.
2. The printed text is `brand-new: 5.00s -- no ceiling: new package`.
3. Beside a package over its ceiling: exit 1, the header counts 1
package, and the single error annotation does not name the new package.
4. A new package alone: `NOT MEASURED`, exit 0.
5. and 6. The `uncapped` half is kept: a `carried` package is not red
and its reason is printed.
7. to 10. A known package that lost its row is refused. Covered: a
weight with no ceiling or reason; a ceiling with no weight; an
`uncapped` reason with no weight; a `provisional` mark with no weight.
Each case asserts the refusal text names the package.
The ruled-raise battery gains one case (floor 7 to 8), which measures
dispatch assumption 6. A ruled entry for a new package still gives it a
ceiling that it can run over: 31 s against a ruled 30 s reads `over`,
exit 1. The existing case (5 s against a ruled 30 s reads `ok`) is
reworded to the new meaning.
`node scripts/check-test-suite-ceilings.mjs --self-test`: `self-test OK
(42 cases across 7 batteries)`, up from 35. Exit 0 at `791aaf55`.
## Ablation
The fix was committed first (`791aaf55`). Each mutation went through
`scripts/ablation-replace.mjs`: the anchor had to hit once, the blob had
to change on disk, and the restore was proven as blob == `HEAD`
(`570a0ceaa046`) with `git diff HEAD` empty. These are root scripts with
no `dist/`, so no build leg applies.
| leg | mutation | self-test |
|:--|:--|:--|
| 1 | `RED_STATUSES = new Set(['over', 'new'])` (the PM's ablation) |
exit 1. Red, but via a `TypeError` in the red renderer (`r.ceiling` is
null), thrown inside the inverted battery's first case. |
| 1b | leg 1, plus the red detail and annotation filtered to rows with a
ceiling: the as-built semantic restored without the crash | exit 1, by
the pin's own assertion: `new: exit 1, verdict OVER, new` |
| 2 | the weightless refusal disabled (its `if` short-circuited with
`false &&`) | exit 1: `lost row: a ceiling with no weight was graded as
new (null)` |
| 2b | `provisional` dropped from the weightless set | exit 1: `lost
row: a provisional mark with no weight was graded as new (null)` |
| 2c | `uncapped` dropped from the weightless set | exit 1: `lost row:
an uncapped reason with no weight was graded as new (null)` |
| 3 | the printed text reverted to `no ceiling: not in ...` | exit 1:
`new: the line is wrong` |
Leg 1's direction is red, as expected, but the crash fires before the
pin's assertions run. Leg 1b is there so that the inverted assertion
itself is what fails.
## Finding: where a package the dataset knows could be graded as new
(dispatch assumption 2)
Searched: every path in `gradeCeilings` / `readCeilingTable` that yields
"not in the dataset", and every path in the generator (`suiteCeilings`,
`buildDataset`, `provisionalPackages` in
`scripts/measure-test-shard-timings.mjs`) and the refresh lane
(`shard-timings-refresh.yml`, `coverageReport` in
`scripts/ci/select-shard-timings-run.mjs`) that can remove a package's
row.
- **A weight with no ceiling and no `uncapped` reason.** Already
refused, as the PM expected. It is now pinned in the inverted battery as
well.
- **A ceiling, an `uncapped` reason or a `provisional` mark with no
weight.** This was NOT refused. The grader keys "in the dataset" on
`packages` alone, so before this change it graded `absent` (red). A bare
Q2 change would have graded it `new` (not red) and ignored the ceiling
the table holds. The generator never writes this shape: `suiteCeilings`
iterates `packages`, and `provisional` is a subset of executed plus
carried, which is `packages`. So only a hand edit reaches it, and it is
now refused like its mirror. Probe: a first table built by the
generator's own `suiteCeilings` over the live dataset (72 weights, 71
ceilings, 1 `uncapped: provisional`) passes the reader. A new package
graded against it prints the new line, exit 0.
- **The generator dropping a held ceiling.** `suiteCeilings` drops a
prior ceiling whose package left `packages`, and the next executing
refresh would set a fresh one. The refresh lane closes this for a
package that still exists. `coverageReport` names any package the
committed dataset weighs, which the workspace still contains, but which
the refresh no longer covers. The workflow then exits 1 with "NOTHING
was regenerated and no PR was opened". What can still drop is a package
that left the workspace: deleted, or renamed, and a rename's new name is
exactly the "new or renamed" case Q2 B rules not red.
`@objectstack/dogfood` is excluded from coverage, but ci.yml excludes it
from Test Core, so it is never graded. The residual is in Acceptance
notes.
## Q1 A, as built (no code, per the ruling)
The effective ceiling is `max(generated, ruled entry)`. A
`RULED_CEILING_RAISES` entry below the generated ceiling therefore
lowers nothing: the verdict lists it under "changing nothing (delete
them)". So the ruling's parenthetical "a lowering is a ruled entry too"
has no as-built mechanism. Nothing about lowering was changed here.
## Verification at `791aaf55`
- `node scripts/check-test-suite-ceilings.mjs --self-test`: exit 0, 42
cases across 7 batteries. On the base (`35ef501e13`) it read 35 across
7.
- `node scripts/measure-test-shard-timings.mjs --self-test`: exit 0,
unchanged.
- `node scripts/pm/dispatch-gates.mjs --commands --repo
objectstack-ai/objectstack`, with no paths (it derives the change set
from the merge base, 2 paths): 58 commands. 53 exit 0. Four exit 3
(PREREQUISITE NOT MET, they read built `dist/`): `check:dts-closure`,
`check:dual-build-cjs-loads`, `check:lean-entry-closure` and
`check:sourcemap-no-sources-content`. These four are declared NOT
MEASURED. None of them reads `ci.yml` (only doc comments name it), the
diff touches no package's `files[]`, and the shared turbo cache is cold.
CI runs them after Build Core.
- `pnpm check:pm-dispatch-gates` was run detached, as its header
prescribes, and read from its log: `✓ dispatch-gates self-test: 2011
cases pass.` It took 1377.4 s on this box, with no lifecycle failure
line.
- `dispatch-gates --ran`: 58 derived families, 54 run, 4 NOT-MEASURED
(derived from the recorded exit 3), 0 UNRUN.
- Lint, a proven narrowing rather than a repo-wide run. (1) Population:
`eslint --print-config` for the touched script resolves 2 rules and no
`parserOptions.project` or `projectService`, and the `files` glob does
not cover `.yml`. (2) Count: `eslint --no-inline-config --format json`
read 1 file, with 0 errors and 0 warnings. (3) Invariance: type-aware
linting is not enabled, so this diff cannot move any other file's
verdict.
## Acceptance notes
- Noted, not filed. A refresh run by hand with the generator's
documented command (`provenance.refresh`) has no coverage check. It can
therefore drop a still-existing package's held ceiling with its weight.
After that, the package would grade as new and the next refresh would
set a fresh ceiling. The lane workflow refuses this, and no hand-run
refresh has happened. Carrier: whoever next touches `suiteCeilings` or
the refresh lane.
---
_Generated by [Claude
Code](https://claude.ai/code/session_01LYXc6ckoWuZyVZpWYizdMh)_
Co-authored-by: Claude <noreply@anthropic.com>1 parent 2e10c9a commit 535c019
2 files changed
Lines changed: 124 additions & 41 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
1460 | 1460 | | |
1461 | 1461 | | |
1462 | 1462 | | |
1463 | | - | |
| 1463 | + | |
| 1464 | + | |
| 1465 | + | |
1464 | 1466 | | |
1465 | 1467 | | |
1466 | 1468 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
28 | 28 | | |
29 | 29 | | |
30 | 30 | | |
31 | | - | |
32 | | - | |
33 | | - | |
34 | | - | |
| 31 | + | |
| 32 | + | |
| 33 | + | |
| 34 | + | |
| 35 | + | |
| 36 | + | |
| 37 | + | |
| 38 | + | |
35 | 39 | | |
36 | 40 | | |
37 | 41 | | |
| |||
44 | 48 | | |
45 | 49 | | |
46 | 50 | | |
47 | | - | |
48 | | - | |
49 | | - | |
50 | | - | |
| 51 | + | |
| 52 | + | |
| 53 | + | |
| 54 | + | |
51 | 55 | | |
52 | 56 | | |
53 | 57 | | |
| |||
89 | 93 | | |
90 | 94 | | |
91 | 95 | | |
92 | | - | |
93 | | - | |
94 | | - | |
| 96 | + | |
| 97 | + | |
| 98 | + | |
| 99 | + | |
| 100 | + | |
95 | 101 | | |
96 | 102 | | |
97 | 103 | | |
| |||
120 | 126 | | |
121 | 127 | | |
122 | 128 | | |
123 | | - | |
124 | | - | |
125 | | - | |
| 129 | + | |
| 130 | + | |
| 131 | + | |
126 | 132 | | |
127 | 133 | | |
128 | 134 | | |
| |||
198 | 204 | | |
199 | 205 | | |
200 | 206 | | |
| 207 | + | |
| 208 | + | |
| 209 | + | |
| 210 | + | |
| 211 | + | |
| 212 | + | |
| 213 | + | |
| 214 | + | |
| 215 | + | |
| 216 | + | |
| 217 | + | |
| 218 | + | |
| 219 | + | |
| 220 | + | |
| 221 | + | |
201 | 222 | | |
202 | 223 | | |
203 | 224 | | |
| |||
221 | 242 | | |
222 | 243 | | |
223 | 244 | | |
224 | | - | |
225 | 245 | | |
| 246 | + | |
226 | 247 | | |
227 | 248 | | |
228 | 249 | | |
| |||
268 | 289 | | |
269 | 290 | | |
270 | 291 | | |
271 | | - | |
| 292 | + | |
272 | 293 | | |
273 | 294 | | |
274 | 295 | | |
| |||
285 | 306 | | |
286 | 307 | | |
287 | 308 | | |
288 | | - | |
| 309 | + | |
| 310 | + | |
| 311 | + | |
289 | 312 | | |
290 | 313 | | |
291 | 314 | | |
| |||
357 | 380 | | |
358 | 381 | | |
359 | 382 | | |
360 | | - | |
361 | | - | |
| 383 | + | |
| 384 | + | |
| 385 | + | |
| 386 | + | |
| 387 | + | |
362 | 388 | | |
363 | 389 | | |
364 | 390 | | |
| |||
385 | 411 | | |
386 | 412 | | |
387 | 413 | | |
388 | | - | |
389 | | - | |
390 | | - | |
391 | | - | |
392 | | - | |
393 | | - | |
394 | | - | |
| 414 | + | |
| 415 | + | |
| 416 | + | |
| 417 | + | |
395 | 418 | | |
396 | 419 | | |
397 | 420 | | |
| |||
400 | 423 | | |
401 | 424 | | |
402 | 425 | | |
403 | | - | |
404 | | - | |
405 | | - | |
406 | | - | |
| 426 | + | |
| 427 | + | |
407 | 428 | | |
408 | 429 | | |
409 | 430 | | |
410 | | - | |
| 431 | + | |
411 | 432 | | |
412 | 433 | | |
413 | 434 | | |
| |||
515 | 536 | | |
516 | 537 | | |
517 | 538 | | |
518 | | - | |
| 539 | + | |
519 | 540 | | |
520 | 541 | | |
521 | | - | |
| 542 | + | |
522 | 543 | | |
523 | 544 | | |
524 | 545 | | |
| |||
630 | 651 | | |
631 | 652 | | |
632 | 653 | | |
633 | | - | |
634 | | - | |
| 654 | + | |
635 | 655 | | |
636 | | - | |
| 656 | + | |
| 657 | + | |
| 658 | + | |
| 659 | + | |
| 660 | + | |
637 | 661 | | |
638 | 662 | | |
639 | | - | |
640 | | - | |
| 663 | + | |
| 664 | + | |
| 665 | + | |
| 666 | + | |
| 667 | + | |
| 668 | + | |
| 669 | + | |
| 670 | + | |
| 671 | + | |
| 672 | + | |
| 673 | + | |
| 674 | + | |
| 675 | + | |
| 676 | + | |
| 677 | + | |
| 678 | + | |
| 679 | + | |
| 680 | + | |
| 681 | + | |
| 682 | + | |
641 | 683 | | |
642 | 684 | | |
643 | 685 | | |
| |||
648 | 690 | | |
649 | 691 | | |
650 | 692 | | |
| 693 | + | |
| 694 | + | |
| 695 | + | |
| 696 | + | |
| 697 | + | |
| 698 | + | |
| 699 | + | |
| 700 | + | |
| 701 | + | |
| 702 | + | |
| 703 | + | |
| 704 | + | |
| 705 | + | |
| 706 | + | |
| 707 | + | |
| 708 | + | |
| 709 | + | |
| 710 | + | |
| 711 | + | |
| 712 | + | |
| 713 | + | |
| 714 | + | |
| 715 | + | |
| 716 | + | |
| 717 | + | |
| 718 | + | |
| 719 | + | |
| 720 | + | |
| 721 | + | |
| 722 | + | |
| 723 | + | |
| 724 | + | |
651 | 725 | | |
652 | 726 | | |
653 | 727 | | |
| |||
727 | 801 | | |
728 | 802 | | |
729 | 803 | | |
730 | | - | |
| 804 | + | |
731 | 805 | | |
732 | | - | |
| 806 | + | |
| 807 | + | |
| 808 | + | |
| 809 | + | |
| 810 | + | |
| 811 | + | |
| 812 | + | |
| 813 | + | |
733 | 814 | | |
734 | 815 | | |
735 | 816 | | |
| |||
0 commit comments