Skip to content

test(cli): pay the oclif cold load at module scope, not in a 10s hook nobody chose - #18782

Merged
os-support-ai merged 1 commit into
mainfrom
claude/issue-18748-envelope-unwrap-hook-timeout
Sep 17, 2026
Merged

os-support-ai merged 1 commit into
mainfrom
claude/issue-18748-envelope-unwrap-hook-timeout

Conversation

@os-support-ai

@os-support-ai os-support-ai commented Sep 17, 2026 •

Copy link
Copy Markdown
Collaborator

Fixes #18748

packages/cli/src/commands/datasource/envelope-unwrap.test.ts paid oclif's
Config.load({ root: CLI_ROOT }) inside a beforeAll, where vitest's default
hookTimeout of 10000ms judged it. This moves that cold load out of every clocked
window and pins the placement. No budget was invented, raised, skipped or quarantined.

The budget nobody chose, and the cost it was judging

packages/cli/vitest.config.ts sets no timeout key at all — deliberately, by the
declared design in its own header ("the test block only carries keys with a recorded
warrant"). So the 10000ms is vitest's own default.

Measured on this 4-vCPU container, n=5 per row — the Config.load call itself, timed
inside a vitest worker in the same directory as the subject:

condition legs (ms) worst leg vs the 10000ms budget
idle 3934 / 3962 / 4272 / 4451 / 4469 45% consumed
4 spinners on 4 vCPU 7843 / 7981 / 8585 / 8782 / 9011 90% consumed — 989 ms of margin

An idle box already spends 39–45% of the budget, and a box that cannot even reach the
load a merge-queue shard applies leaves under one second. A budget a real cost approaches
to within a second is not a budget, it is a load sensor — and what it senses is how
busy the runner is, reported as "this file failed".

Why not a bigger number, and why this number was not invented

Widening the window around the cost relocates the cliff to the next heavier shard. The
merge queue runs the full suite where PR-side CI runs only the affected subset, so
the queue shard is heavier than anything a PR check measures — which is where this class
has already ejected other people's green PRs.

The card asked whether an already-written adjacent bound could be reused. Measured, and
the honest answer is no, and none was needed:

So this PR takes the repo's own stated convention instead, which needs no number at all:

Clocked windows measure behaviour, never loading — a test that boots a real plugin chain
pays its first load at module top.
— AGENTS.md, § Build & Test

The load is now a module-scope await, paid during collection. Verified against the
runner this tree installs rather than recalled — @vitest/runner@**4.1.11** (the prior
art verified 4.1.10): withTimeout(...) wraps exactly the hooks (beforeAll, afterAll,
beforeEach, afterEach, onTestFailed, onTestFinished) and the test bodies, while
collectTests() awaits runner.importFile(filepath, 'collect') bare; and
vitest --help on 4.1.11 offers exactly three timeout knobs (testTimeout, hookTimeout,
teardownTimeout), none of which covers module loading.

Before → after, same file, same box

before   Test Files 1 passed (1)  Tests 11 passed (11)
         Duration 4.59s (transform  97ms, import  481ms, tests 3.95s)
after    Test Files 1 passed (1)  Tests 12 passed (12)
         Duration 4.40s (transform 170ms, import 4.20s, tests   36ms)

The cost did not shrink and was never meant to. It left the clocked region: tests
3.95s → 36ms, import 481ms → 4.20s, wall clock unchanged.

The pin — fails before, passes after, both readings shown

Reverse verification, run from the committed fix. The implementation half was mutated back
to the pre-fix shape, proved on disk before the run, and restored from HEAD:

ON-DISK PROOF: module-scope-load lines 1 -> 0;  'beforeAll(async' lines 0 -> 1
ON-DISK PROOF: blob 25a8d67302eb… -> 88575306222b…

mutated    exit 1   Tests 1 failed | 11 passed (12)
                    FAIL … > pays `Config.load` at module scope, leaving no hook to clock it
                    AssertionError: expected '   …' to match /^const\s+config\s*=\s*await\s+Config…/m
                    Duration 4.91s (import  587ms, tests 4.15s)   <- cost back INSIDE the clock

restored   blob 25a8d67302eb… == HEAD, `git diff HEAD` empty
           exit 0   Tests 12 passed (12)
                    Duration 5.09s (import 4.84s, tests   39ms)   <- cost back OUTSIDE it

Both halves move together, in both directions.

⚠️ Why the pin is a SOURCE assertion — the behavioural instrument is inert here

The prior art one package over (plugins/plugin-dev/src/dev-plugin-security-enforcement-warning.test.ts) witnesses this property behaviourally: "stays green even under
--hookTimeout=1 — there is no hook time left to clock."
That instrument does not work
in packages/cli
, measured rather than assumed:

probe (beforeAll sleeping 500ms) command result
@objectstack/plugin-dev (no test.projects) vitest run PROBE_FILE --hookTimeout=1 exit 1 — Error: Hook timed out in 1ms.
@objectstack/cli (test.projects) vitest run --project unit PROBE_FILE --hookTimeout=1 exit 0 — 1 passed
@objectstack/cli vitest run PROBE_FILE --hookTimeout=1 (no --project) exit 0 — 1 passed

A CLI timeout override does not reach a project-level config on vitest 4.1.11. The lit
control is the plugin-dev row: the flag works, so the two zeros are readings and not a
dead instrument. That leaves the structural fact as the only thing assertable here, and
the pin asserts it — using maskCommentsAndLiterals because this file's prose and the
pin's own regex bodies contain the very spellings being searched for.

Tier, measured in BOTH directions

packages/cli/vitest-tiers.ts treats new ObjectQL( and friends as KERNEL signals, so a
pin can move a whole file across tiers. It did not:

unit entries for this file integration entries unit total integration total
before 11 0 3034 413
after 12 (the pin) 0 3035 413

Read from vitest list, the config's own answer, not from the predicate by hand. Control
for the zero column: the integration list is non-empty at 413 entries (e.g.
test/authoring-rule-command-parity.test.ts), so 0 is a reading.

Changeset: skip-changeset, measured not assumed

Clause-②: no

Declared from the measured diff, not inherited: the dispatching seat's claim comment
deliberately carried no value, and a fabricated declaration passes where a missing one
reddens. The change set is exactly one test file, and it moves no authorable surface in
either direction — neither widening nor narrowing anything an author can write.

AGENTS.md (§ Post-Task Checklist 3): "⛔ never skip-changeset: that label is for a
diff that publishes nothing from any released package."
This diff is exactly that case,
and it was measured after a full build by grepping the paths @objectstack/cli's
files[] actually ships (dist, README.md, CHANGELOG.md):

symbol origin hits under files[]
envelope-unwrap file-local 0
SILENT_PASS file-local 0
DRIFT_RESULT file-local 0
readEnvelopeFrom non-test src (control) 5
DatasourceValidate non-test src (control) 2
No federated objects to validate non-test src (control) 3

tsconfig.build.json excludes src/**/*.test.ts from dist, so the file cannot ship;
the controls are lit, so the zeros are readings. Nothing published moves.

Verification

what result
pnpm --filter @objectstack/cli exec vitest run --project unit 212 files / 3035 tests passed, exit 0
pnpm --filter @objectstack/cli typecheck exit 0 (test layer compiles under tsconfig.test.json)
dependency-closure build (turbo … --filter=@objectstack/cli^...) 56/56 successful
scripts/pm/dispatch-gates.mjs --ran 53 derived, 53 run, 0 NOT-MEASURED, 0 UNRUN, every family recorded with its exit code, all 0
eslint . --no-inline-config (the whole repo-wide population, NOT narrowed) 6850 files, 0 errors, 0 warnings, exit 0

The four gates that first exited 3 (check:dual-build-cjs-loads, check:i18n,
check:i18n-coverage, check:i18n-walk-parity) were PREREQUISITE NOT MET — they read
built output, and only the cli closure was built. Re-run after a full turbo run build:
all four exit 0. ⛔ Those threes are not recorded as failures; they were not
measurements.

packages/cli's integration tier is declared to CI: this diff touches no
integration-tier file, no bin/ entry and no spawn helper.

Acceptance notes

  • Not built here, by the card's own fence — the class-level remedy. It is written up
    with readings in the report handed back to the dispatching seat: what a shared cold-load
    budget would and would not buy, and why the gate-shaped option (extending
    check:test-source-alias's clocked-window rule past dynamic import() to any cold
    load in a clocked window) is the one with a measured warrant. That is a decision, not an
    execution.
  • --hookTimeout / --testTimeout are inert in packages/cli (table above). This is
    wider than this card: it means no seat can lower or raise a timeout from the CLI in this
    package, and a merge-queue triage that reaches for that flag will read a false green.
    Reported for filing with dedupe words, ⛔ not filed from here and ⛔ not fixed here.
  • Noted, not filed: plugins/plugin-dev/src/dev-plugin-security-enforcement-warning.test.ts
    guards the identical invariant with prose only (⛔ Do not add one back) — no assertion.
    The pin added here is the shape that would close it. Carrier: whoever next touches that
    file; none queued.

🤖 Generated with Claude Code

https://claude.ai/code/session_01DvvamiacK328idtBYJBxV3


Generated by Claude Code


Generated by Claude Code

… nobody chose

`envelope-unwrap.test.ts` loaded oclif's `Config.load({ root: CLI_ROOT })`
inside a `beforeAll`, where vitest's DEFAULT `hookTimeout` of 10000ms judged
it. `packages/cli/vitest.config.ts` sets no timeout key at all — by the
declared design in its own header — so the budget is vitest's default and
nobody in this package chose it.

Measured on a 4-vCPU container, n=5 per row, the call itself:

    idle                    3934 / 3962 / 4272 / 4451 / 4469 ms
    4 spinners on 4 vCPU    7843 / 7981 / 8585 / 8782 / 9011 ms

39-45% of the budget on an IDLE box, and as little as 989 ms of margin on a
box that cannot even reach the load a merge-queue shard applies. A budget a
real cost approaches to within a second is not a budget — it is a load
sensor, and what it senses is reported as "this file failed".

The remedy is not a bigger number: widening the window around the cost
relocates the cliff to the next heavier shard. It is the repo's own stated
convention — "clocked windows measure behaviour, never loading" (AGENTS.md,
Build & Test) — so the load moves OUT of every clocked window and is paid by
a module-scope `await`, during COLLECTION. Verified against the runner this
tree installs, not recalled: in `@vitest/runner@4.1.11` `withTimeout(...)`
wraps exactly the hooks and the test bodies, while `collectTests()` awaits
`runner.importFile(filepath, 'collect')` bare.

Same file, same box, before -> after:

    before   Duration 4.59s (import 481ms, tests 3.95s)   11 tests
    after    Duration 4.40s (import 4.20s, tests  36ms)   12 tests

The cost did not shrink and was never meant to; it left the clocked region.

A pin holds the placement, because the identical prose warning one package
over ("do not add one back") is a comment nobody asserts. It is a SOURCE
assertion on purpose: the behavioural instrument the prior art used —
`vitest run --hookTimeout=1`, green iff no hook time is left to clock — is
INERT in this package. Measured: a probe `beforeAll` sleeping 500ms passes
under `--hookTimeout=1` in `packages/cli` with and without `--project`, while
the identical probe under `@objectstack/plugin-dev` (no `test.projects`)
fails with `Hook timed out in 1ms`.

Tier unchanged, measured both ways: `unit` 11 -> 12 entries for this file,
`integration` 0 -> 0, against a lit integration list of 413 entries.

Co-Authored-By: Claude <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01DvvamiacK328idtBYJBxV3
@github-actions

Copy link
Copy Markdown
Contributor

📓 Docs Drift Check

Nothing in this diff resolved to a documentable surface (no symbol, route or SDK anchor derived from 0 changed package(s)), so this run has no opinion about the docs.

What this run could not see
  • a page that states a rule by its inputs shares no identifier with the emitter that implements the rule, so an emitter-only diff cannot list it — not on this run and not on any run. Measured on fix(driver-sql): emit varchar(maxLength) for a text field a declared index keys on #11430: content/docs/protocol/objectql/types.mdx documents the text-family column mapping by the ObjectQL type names it maps FROM (text / textarea / html) while the diff changed createColumn; it went unlisted, and it was the page that diff falsified, in four places. No shared token exists to detect this on, so a rule your change carries has to be re-read by hand in the pages that restate it.

Coarse fallback — 0 page(s) merely mention a changed package (the pre-#9192 predicate, kept for the deliberately-wide backstop): node scripts/docs-audit/affected-docs.mjs --json 2085be2b2d8769c6227167bb3525f4fa72b9a486 → packageMentionDocs.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

size/s skip-changeset PR has no user-facing published change; bypasses the changeset gate tests

Projects

None yet

2 participants