Skip to content

ci(suite-ceilings): a new package is not red before its first ceiling - #22498

Merged
objectstack-fleet[bot] merged 1 commit into
mainfrom
claude/issue-16468-new-package-not-red
Oct 9, 2026
Merged

objectstack-fleet[bot] merged 1 commit into
mainfrom
claude/issue-16468-new-package-not-red

Conversation

@objectstack-fleet

Copy link
Copy Markdown
Contributor

Fixes #16468
Clause-②: no

Implements ruling Q2 B on the suite-duration ratchet (#16468 (comment)): a package absent from the ceiling dataset is a new package. It prints "no ceiling: new package" and is not red. The first refresh after it lands enters it in the dataset. A package the dataset knows but that lost its row stays red. Q1 A needs no code (the hold is as built). Part 1 of the ratchet landed as PR #22480 (6abd265320).

What changed

scripts/check-test-suite-ceilings.mjs:

  • Grading. A package absent from the dataset with no ruled entry now gets row status new (it was absent). RED_STATUSES is over alone, so the exit code follows only an over-ceiling package or a refusal.
  • Rendering. The row prints -- no ceiling: new package (not in scripts/test-shard-timings.json; the first refresh after it lands enters it), in the same column as the provisional and carried rows. The red header, the red detail and the error annotation lose their absent-package branch, because no red row can lack a ceiling now.
  • "Lost its row" stays red (see the finding below). readCeilingTable already refused a weight with neither a ceiling nor an uncapped reason. It now also refuses a ceiling, an uncapped reason or a provisional mark that names a package with no weight. Without this, that shape would have changed from red (as built, it graded absent) to "no ceiling: new package".
  • Prose the ruling made false, prose only. The header bullet now states the ruled rule and cites the ruling URL. Also updated: the NOT MEASURED paragraph, the exit-code table, the gradeCeilings status table and the RULED_CEILING_RAISES comment ("the only way a package that has never landed can get one").

.github/workflows/ci.yml: the comment above Check the per-package suite-duration ceilings said "a package absent from the dataset is red (the new-package shape)". It now states the ruled rule. Comment only, no step change.

Sweep for other sentences stating the old rule: git grep -n -i -E "absent from the dataset|new-package shape|new package|ruled raise|no ceiling" over the tree found only ci.yml lines 1462-1463, and they are fixed here. The ceiling prose in .github/workflows/shard-timings-refresh.yml and scripts/measure-test-shard-timings.mjs describes the hold and the formula, not the new-package rule, so it is untouched. Neither file is in this diff, and neither is scripts/test-shard-timings.json or scripts/partition-test-shards.mjs.

The inverted pin

The battery 'a package absent from the dataset is red; an uncapped one is not' (floor 4) is now 'a new package prints "no ceiling: new package" and is not red; a known one that lost its row is refused' (floor 10):

  1. A new package (9999 s) beside one within its ceiling: row status new, exit 0, verdict OK.
  2. The printed text is brand-new: 5.00s -- no ceiling: new package.
  3. Beside a package over its ceiling: exit 1, the header counts 1 package, and the single error annotation does not name the new package.
  4. A new package alone: NOT MEASURED, exit 0.
  5. and 6. The uncapped half is kept: a carried package is not red and its reason is printed.
  6. to 10. A known package that lost its row is refused. Covered: a weight with no ceiling or reason; a ceiling with no weight; an uncapped reason with no weight; a provisional mark with no weight. Each case asserts the refusal text names the package.

The ruled-raise battery gains one case (floor 7 to 8), which measures dispatch assumption 6. A ruled entry for a new package still gives it a ceiling that it can run over: 31 s against a ruled 30 s reads over, exit 1. The existing case (5 s against a ruled 30 s reads ok) is reworded to the new meaning.

node scripts/check-test-suite-ceilings.mjs --self-test: self-test OK (42 cases across 7 batteries), up from 35. Exit 0 at 791aaf55.

Ablation

The fix was committed first (791aaf55). Each mutation went through scripts/ablation-replace.mjs: the anchor had to hit once, the blob had to change on disk, and the restore was proven as blob == HEAD (570a0ceaa046) with git diff HEAD empty. These are root scripts with no dist/, so no build leg applies.

leg mutation self-test
1 RED_STATUSES = new Set(['over', 'new']) (the PM's ablation) exit 1. Red, but via a TypeError in the red renderer (r.ceiling is null), thrown inside the inverted battery's first case.
1b leg 1, plus the red detail and annotation filtered to rows with a ceiling: the as-built semantic restored without the crash exit 1, by the pin's own assertion: new: exit 1, verdict OVER, new
2 the weightless refusal disabled (its if short-circuited with false &&) exit 1: lost row: a ceiling with no weight was graded as new (null)
2b provisional dropped from the weightless set exit 1: lost row: a provisional mark with no weight was graded as new (null)
2c uncapped dropped from the weightless set exit 1: lost row: an uncapped reason with no weight was graded as new (null)
3 the printed text reverted to no ceiling: not in ... exit 1: new: the line is wrong

Leg 1's direction is red, as expected, but the crash fires before the pin's assertions run. Leg 1b is there so that the inverted assertion itself is what fails.

Finding: where a package the dataset knows could be graded as new (dispatch assumption 2)

Searched: every path in gradeCeilings / readCeilingTable that yields "not in the dataset", and every path in the generator (suiteCeilings, buildDataset, provisionalPackages in scripts/measure-test-shard-timings.mjs) and the refresh lane (shard-timings-refresh.yml, coverageReport in scripts/ci/select-shard-timings-run.mjs) that can remove a package's row.

  • A weight with no ceiling and no uncapped reason. Already refused, as the PM expected. It is now pinned in the inverted battery as well.
  • A ceiling, an uncapped reason or a provisional mark with no weight. This was NOT refused. The grader keys "in the dataset" on packages alone, so before this change it graded absent (red). A bare Q2 change would have graded it new (not red) and ignored the ceiling the table holds. The generator never writes this shape: suiteCeilings iterates packages, and provisional is a subset of executed plus carried, which is packages. So only a hand edit reaches it, and it is now refused like its mirror. Probe: a first table built by the generator's own suiteCeilings over the live dataset (72 weights, 71 ceilings, 1 uncapped: provisional) passes the reader. A new package graded against it prints the new line, exit 0.
  • The generator dropping a held ceiling. suiteCeilings drops a prior ceiling whose package left packages, and the next executing refresh would set a fresh one. The refresh lane closes this for a package that still exists. coverageReport names any package the committed dataset weighs, which the workspace still contains, but which the refresh no longer covers. The workflow then exits 1 with "NOTHING was regenerated and no PR was opened". What can still drop is a package that left the workspace: deleted, or renamed, and a rename's new name is exactly the "new or renamed" case Q2 B rules not red. @objectstack/dogfood is excluded from coverage, but ci.yml excludes it from Test Core, so it is never graded. The residual is in Acceptance notes.

Q1 A, as built (no code, per the ruling)

The effective ceiling is max(generated, ruled entry). A RULED_CEILING_RAISES entry below the generated ceiling therefore lowers nothing: the verdict lists it under "changing nothing (delete them)". So the ruling's parenthetical "a lowering is a ruled entry too" has no as-built mechanism. Nothing about lowering was changed here.

Verification at 791aaf55

  • node scripts/check-test-suite-ceilings.mjs --self-test: exit 0, 42 cases across 7 batteries. On the base (35ef501e13) it read 35 across 7.
  • node scripts/measure-test-shard-timings.mjs --self-test: exit 0, unchanged.
  • node scripts/pm/dispatch-gates.mjs --commands --repo objectstack-ai/objectstack, with no paths (it derives the change set from the merge base, 2 paths): 58 commands. 53 exit 0. Four exit 3 (PREREQUISITE NOT MET, they read built dist/): check:dts-closure, check:dual-build-cjs-loads, check:lean-entry-closure and check:sourcemap-no-sources-content. These four are declared NOT MEASURED. None of them reads ci.yml (only doc comments name it), the diff touches no package's files[], and the shared turbo cache is cold. CI runs them after Build Core.
  • pnpm check:pm-dispatch-gates was run detached, as its header prescribes, and read from its log: ✓ dispatch-gates self-test: 2011 cases pass. It took 1377.4 s on this box, with no lifecycle failure line.
  • dispatch-gates --ran: 58 derived families, 54 run, 4 NOT-MEASURED (derived from the recorded exit 3), 0 UNRUN.
  • Lint, a proven narrowing rather than a repo-wide run. (1) Population: eslint --print-config for the touched script resolves 2 rules and no parserOptions.project or projectService, and the files glob does not cover .yml. (2) Count: eslint --no-inline-config --format json read 1 file, with 0 errors and 0 warnings. (3) Invariance: type-aware linting is not enabled, so this diff cannot move any other file's verdict.

Acceptance notes

  • Noted, not filed. A refresh run by hand with the generator's documented command (provenance.refresh) has no coverage check. It can therefore drop a still-existing package's held ceiling with its weight. After that, the package would grade as new and the next refresh would set a fresh ceiling. The lane workflow refuses this, and no hand-run refresh has happened. Carrier: whoever next touches suiteCeilings or the refresh lane.

Generated by Claude Code

A package absent from the ceiling dataset now prints "no ceiling: new
package" and is not red; the first refresh after it lands enters it in
the dataset. A package the dataset knows but that lost its row stays
red: besides the existing refusal of a weight with no ceiling or reason,
the table reader now refuses a ceiling, `uncapped` reason or
`provisional` mark with no weight, which would otherwise have graded as
a new package. The self-test battery for the absent case is inverted to
the new meaning, and a ruled ceiling for a new package is shown to grade
it and to go red when it runs over.

Claude-Session: https://claude.ai/code/session_01LYXc6ckoWuZyVZpWYizdMh
Co-authored-by: Claude <noreply@anthropic.com>
@objectstack-fleet objectstack-fleet Bot added the skip-changeset PR has no user-facing published change; bypasses the changeset gate label Oct 9, 2026
@objectstack-fleet
objectstack-fleet Bot marked this pull request as ready for review October 9, 2026 15:33
@objectstack-fleet
objectstack-fleet Bot enabled auto-merge October 9, 2026 15:34
@objectstack-fleet
objectstack-fleet Bot added this pull request to the merge queue Oct 9, 2026
Merged via the queue into main with commit 535c019 Oct 9, 2026
39 checks passed
@objectstack-fleet
objectstack-fleet Bot deleted the claude/issue-16468-new-package-not-red branch October 9, 2026 16:13
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ci/cd size/m skip-changeset PR has no user-facing published change; bypasses the changeset gate

Projects

None yet

2 participants