Skip to content

Commit 65767d2

Browse files
claude[bot]claude
andauthored
fix(pm): measure the runtime stderr noise floor in the dispatch-gates self-test (#17977)
Fixes #17765 ## What was wrong The self-test battery in `scripts/pm/dispatch-gates.mjs` described a spawned child's stderr **stream** in two places while meaning what **this module** prints. Every such child inherits the run's `NODE_OPTIONS`, and under `--use-env-proxy` node writes two lines before any user code runs: ``` (node:NNN) [UNDICI-EHPA] Warning: EnvHttpProxyAgent is experimental, expect them to change at any time. (Use `node --trace-warnings ...` to show where the warning was created) ``` That flag is not exotic: it is what `check-required-contexts.mjs --verify-required-set` prints as its own remedy, citing #9642. So a seat that follows one gate's advice is punished by another — and the punishing gate is one nobody runs casually, because this self-test runs past the container's ten-minute foreground cap. Both assertions were true statements about a real property, and both reds were entirely spurious: | case | assertion before | |:--|:--| | entry guard (the card's, at `:22373` on the merge base) | `(imported.stderr ?? '').trim() === ''` | | end-to-end CLI banner (found by running the full self-test under the flag) | `(plainRun.stderr ?? '').split('\n')[0].includes('gate list derived from the tree of')` | The second one is a **deviation from the dispatched file face**, declared here: it is the same defect class, in the same file, in the same gate family, and the card's own acceptance ("the self-test reaches its verdict with exit 0 under `--use-env-proxy`") is unreachable while it stands. No other file is touched. ## The fix — one measured floor, not a pattern For each probe an identical child that does **nothing** runs with the same argv shape, cwd and inherited env. Whatever **it** prints is what this runtime prints unprompted; anything the probe prints beyond that came from the module. The single normalisation is node's pid, which differs between any two children by construction. - `stderrBeyondRuntime(probe, baseline)` — what the probe wrote **beyond** the floor (the entry-guard case). - `moduleStderrLines(probe, baseline)` — the probe's lines with an exact leading match of the floor removed, so a case can still speak about **which** line the module wrote first (the banner case). Both live once, above the end-to-end CLI block, and both instances call them — one explanation and one subtraction in the file rather than two copies. ⛔ **Not a shape filter.** The `(Use ...)` hint line carries no `[CODE] Warning:` at all, so a filter written to node's warning shape leaves it behind — the case stays red — while swallowing a line the module wrote in that same shape. Triage measured this on the card: `grep -c "Warning:"` over that stderr answers 1, not 2. ⛔ **No shared helper across files, and `scripts/check-cross-package-test-inputs.mjs` is untouched** — that half is another domain's. ## Controls Without controls, "prints nothing of its own" and "prints nothing that is ever looked at" are the same green. Four were added: - NEGATIVE: importing a module that writes nothing is clean under THIS runtime. - POSITIVE: one line the imported module writes to stderr still reds. - POSITIVE: a line **disguised** as a node warning still reds — this is what separates a measured baseline from a shape filter, which would swallow it. - CONTROL: the floor child writes no line of its own, so the banner subtraction cannot be answering an empty list. ## Acceptance readings Direct-drive probe of each case's own consumer shape, flag as the only variable. `OLD` is the assertion on the merge base `400167ab`, `NEW` is the assertion on this head. **Entry-guard case** — a child importing this module, argv `--self-test --tier packages/spec/src/index.ts`: ``` NODE_OPTIONS=--max-old-space-size=4096 status 0, stdout CONSUMER-REACHED, probe stderr "" OLD true NEW true NODE_OPTIONS=--use-env-proxy --max-old-space-size=4096 status 0, stdout CONSUMER-REACHED probe stderr: (node:30022) [UNDICI-EHPA] Warning: ... + hint line baseline stderr: (node:30030) [UNDICI-EHPA] Warning: ... + hint line beyond-runtime text: "" OLD false NEW true ``` **Banner case** — the real CLI child, argv `--tier packages/spec/src/index.ts`: ``` NODE_OPTIONS=--max-old-space-size=4096 baseline stderr lines: [] OLD true NEW true NODE_OPTIONS=--use-env-proxy --max-old-space-size=4096 baseline stderr lines: 2 (the UNDICI-EHPA warning + the hint line) module first line: "dispatch-gates: gate list derived from the tree of ..." control: floor-vs-floor 0 lines, probe 7 lines OLD false NEW true ``` **Full self-test, detached** — `pnpm check:pm-dispatch-gates` runs past the ten-minute foreground cap, so it is run detached and polled to its verdict line. No full run was taken on the merge base (the merge-base readings above are the direct-drive probes). The run taken with the entry-guard repair alone, at `e53c83c3` under `--use-env-proxy`, ended: ``` ✗ and it opens with the banner, on the FIRST line of stderr ✗ dispatch-gates self-test: 1 of 1681 case(s) failed. ``` which is how the second instance was found. On this head under the same flag: ``` ✓ and it opens with the banner, on the FIRST stderr line THIS module writes ✓ CONTROL: the floor child writes no stderr line of its own — the subtraction above cannot be vacuous ✓ imported, this module prints nothing of its own on either stream ✓ NEGATIVE CONTROL: importing a module that writes nothing is clean under THIS runtime ✓ POSITIVE CONTROL: one line the imported module writes to stderr still reds ✓ POSITIVE CONTROL: an imported line DISGUISED as a node warning still reds ✓ dispatch-gates self-test: 1682 cases pass. ``` `pnpm check:pm-dispatch-gates` exit **0** at `12a89168` (merged with `origin/main` `45b90b6a`), under `NODE_OPTIONS=--use-env-proxy --max-old-space-size=4096`. Case count 1682 = the 1678 the card measured, plus the four controls this PR adds. ## Gates Derived with `node scripts/pm/dispatch-gates.mjs --commands --repo objectstack-ai/objectstack` (no paths; change set taken from git, 1 path, against merge base `45b90b6a9`). The first derivation was taken on a stale tree and warned so; `origin/main` was merged in and the list re-derived on `12a89168` — identical, 31 commands. All 31 run, every exit code recorded, reconciled: ``` ✓ dispatch-gates --ran: 31 derived famil(ies) accounted for — 31 run, 0 NOT-MEASURED (a DERIVED zero — all 31 recorded an exit code and none of them is 3). ``` Outside that link, and not claimed here: the 52 artifact-roster families, the 11 wide-population families, the 14 pending-changeset families (none apply — this PR carries no changeset by design), the path-scheduled CI job and the always-runs tail. Repo-wide `pnpm lint` is CI's run, not this one. ## Acceptance notes - **Should the two files share one helper?** The card asks this and triage adopted it as a must-answer. Measured answer: **not yet.** The population of "literally empty child stderr" assertions in the repo is exactly two — this file and the sibling — and a `git grep` over `scripts/**` finds no third: every other child-stderr assertion there is a targeted `.includes()` on content the child itself wrote, which runtime noise cannot falsify. The part that could be shared is four lines of pure text subtraction; the part that must **not** be shared is the spawn, whose argv shape has to match each file's real probe by construction. A cross-file import would also add an edge into this tool's own derivation, whose module-body literals are inherited as watch hints by anything importing it. The trigger that changes the answer is a third copy; filing that is for whoever meets one. - Noted, not filed: nothing else. Successor for the shared-helper question if a third instance appears: the `domain:devx` seat that owns `scripts/check-cross-package-test-inputs.mjs`. - `skip-changeset`: `scripts/pm/**` ships in no package's `files[]`, so this diff publishes nothing. - Hold #14290 declares `Restart-touch: scripts/pm/dispatch-gates.mjs`; its re-measurement after this lands is the devx seat's, not this PR's. #17797 names this file on the docblock-prose face and is untouched here. --- _Generated by [Claude Code](https://claude.ai/code/session_01DAcomhvR9kKizeYgg89Vo8)_ --- _Generated by [Claude Code](https://claude.ai/code)_ --------- Co-authored-by: claude <noreply@anthropic.com>
1 parent 4dab2bc commit 65767d2

1 file changed

Lines changed: 132 additions & 6 deletions

File tree

‎scripts/pm/dispatch-gates.mjs‎

Lines changed: 132 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -22190,9 +22190,84 @@ function selfTest() {
2219022190
// test is the guard around the answer, not the answer.
2219122191
const CLI = fileURLToPath(import.meta.url);
2219222192
const runCli = (args) => spawnSync(process.execPath, [CLI, ...args], { encoding: 'utf8', cwd: ROOT });
22193+
22194+
// ── Why the runtime's noise floor is MEASURED and not pattern-matched ─────
22195+
//
22196+
// Every child spawned by this self-test inherits the run's `NODE_OPTIONS`,
22197+
// and under some values the RUNTIME writes to the child's stderr before any
22198+
// user code runs. The measured case: `check-required-contexts.mjs
22199+
// --verify-required-set` exits 2 = NOT VERIFIED without `--use-env-proxy`
22200+
// and prescribes exactly that flag, citing #9642 for why the inference it
22201+
// prevents matters — so a seat that follows one gate's own remedy opens
22202+
// EVERY child here with two lines:
22203+
//
22204+
// (node:NNN) [UNDICI-EHPA] Warning: EnvHttpProxyAgent is experimental, ...
22205+
// (Use `node --trace-warnings ...` to show where the warning was created)
22206+
//
22207+
// Two cases below used to describe the child's stderr STREAM while meaning
22208+
// this MODULE's output — one asserting the stream is literally empty, one
22209+
// asserting the banner sits on its first line. Both are true statements
22210+
// about the module and both went red for obeying the other gate's advice:
22211+
// a reported regression that does not exist, in a self-test that runs past
22212+
// the container's foreground cap and so is expensive to re-read.
22213+
//
22214+
// ⛔ The remedy is NOT to stop looking at these children's stderr — that
22215+
// trades a true assertion for silence. ⛔ Nor is it to match node's warning
22216+
// SHAPE: the hint line above carries no `[CODE] Warning:` at all, so a
22217+
// filter written to that description leaves it behind and the cases stay
22218+
// red, while swallowing a line the MODULE wrote in that same shape.
22219+
//
22220+
// So the floor is MEASURED. For each probe an identical child that does
22221+
// NOTHING runs with the same argv shape, cwd and inherited env, and
22222+
// whatever IT prints is what this runtime prints unprompted; anything the
22223+
// probe prints BEYOND that came from the module. No pattern describes the
22224+
// noise, so this cannot rot when node changes its warning text, and any
22225+
// line the module writes still reds — including one disguised as a node
22226+
// warning, which the controls in the entry-guard battery drive. The single
22227+
// normalisation is node's pid, which differs between any two children by
22228+
// construction; nothing else about the text is touched.
22229+
const withoutPid = (stream) => (stream ?? '').replace(/^\(node:\d+\)/gm, '(node:PID)').trim();
22230+
/** What `probe` wrote on stderr BEYOND what this runtime writes unprompted. */
22231+
const stderrBeyondRuntime = (probe, baseline) => (
22232+
withoutPid(probe.stderr) === withoutPid(baseline.stderr) ? '' : withoutPid(probe.stderr)
22233+
);
22234+
/**
22235+
* `probe`'s stderr lines with this runtime's own opening noise removed, so a
22236+
* case can speak about WHICH line the module wrote first. Only an exact
22237+
* leading match of the measured floor is removed; anything else is kept.
22238+
*/
22239+
const moduleStderrLines = (probe, baseline) => {
22240+
const noise = withoutPid(baseline.stderr);
22241+
const text = withoutPid(probe.stderr);
22242+
const own = noise && text.startsWith(noise) ? text.slice(noise.length) : text;
22243+
return own.split('\n').map((line) => line.trim()).filter(Boolean);
22244+
};
22245+
22246+
// The floor for the CLI children below: same argv shape, same cwd, same
22247+
// inherited env, and an entry that does nothing at all.
22248+
const cliNoiseTmp = mkdtempSync(nodePath.join(tmpdir(), 'dispatch-gates-noise-'));
22249+
let cliBaseline;
22250+
try {
22251+
const quietEntry = nodePath.join(cliNoiseTmp, 'runtime-baseline.mjs');
22252+
writeFileSync(quietEntry, 'export const nothing = 1;\n');
22253+
cliBaseline = spawnSync(process.execPath, [quietEntry, '--tier', 'packages/spec/src/index.ts'], { encoding: 'utf8', cwd: ROOT });
22254+
} finally {
22255+
rmSync(cliNoiseTmp, { recursive: true, force: true });
22256+
}
22257+
2219322258
const plainRun = runCli(['--tier', 'packages/spec/src/index.ts']);
2219422259
t('an unasserted explicit-path run still answers', plainRun.status === 0 && (plainRun.stdout ?? '').trim().length > 0);
22195-
t('and it opens with the banner, on the FIRST line of stderr', (plainRun.stderr ?? '').split('\n')[0].includes('gate list derived from the tree of'));
22260+
t(
22261+
'and it opens with the banner, on the FIRST stderr line THIS module writes',
22262+
(moduleStderrLines(plainRun, cliBaseline)[0] ?? '').includes('gate list derived from the tree of'),
22263+
);
22264+
// The control that keeps the case above from passing on an empty list: the
22265+
// floor child writes nothing of its own, so subtracting it from ITSELF
22266+
// leaves no line to mistake for a banner.
22267+
t(
22268+
'CONTROL: the floor child writes no stderr line of its own — the subtraction above cannot be vacuous',
22269+
moduleStderrLines(cliBaseline, cliBaseline).length === 0 && moduleStderrLines(plainRun, cliBaseline).length > 0,
22270+
);
2219622271
t('the banner stays OFF stdout, which is pasted verbatim into claim comments', !(plainRun.stdout ?? '').includes('gate list derived from the tree of'));
2219722272
const liveSlug = repoIdentity().slug;
2219822273
const assertedRun = runCli(['--tier', 'packages/spec/src/index.ts', REPO_FLAG, liveSlug ?? 'an-owner/a-repo']);
@@ -22362,15 +22437,66 @@ function selfTest() {
2236222437
`const m = await import(${JSON.stringify(pathToFileURL(SELF).href)});\n` +
2236322438
`console.log('CONSUMER-REACHED', typeof m.maskComments, typeof m.isExtractConfigPath, typeof m.deriveTier);\n`,
2236422439
);
22365-
const imported = spawnSync(process.execPath, [consumer, '--self-test', '--tier', 'packages/spec/src/index.ts'], {
22366-
encoding: 'utf8',
22367-
cwd: entryTmp,
22368-
});
22440+
// ONE spawn helper for both children below, so the noise floor cannot
22441+
// drift away from the probe it is the floor for: same argv shape, same
22442+
// cwd, same inherited env, by construction rather than by two copies.
22443+
const spawnEntryChild = (entry) => spawnSync(
22444+
process.execPath,
22445+
[entry, '--self-test', '--tier', 'packages/spec/src/index.ts'],
22446+
{ encoding: 'utf8', cwd: entryTmp },
22447+
);
22448+
const imported = spawnEntryChild(consumer);
2236922449
t(
2237022450
'imported, the importer reaches its own first statement and the re-exports are there',
2237122451
imported.status === 0 && (imported.stdout ?? '').trim() === REACHED,
2237222452
);
22373-
t('imported, this module prints nothing of its own on either stream', (imported.stderr ?? '').trim() === '');
22453+
22454+
// The measured noise floor again — see `stderrBeyondRuntime` above for why
22455+
// this is subtracted rather than pattern-matched. The floor child here is
22456+
// spawned through the SAME helper as the probe, so the two cannot drift
22457+
// apart in argv shape, cwd or inherited env.
22458+
const baselineEntry = nodePath.join(entryTmp, 'runtime-baseline.mjs');
22459+
writeFileSync(baselineEntry, "console.log('BASELINE-REACHED');\n");
22460+
const runtimeBaseline = spawnEntryChild(baselineEntry);
22461+
t('imported, this module prints nothing of its own on either stream', stderrBeyondRuntime(imported, runtimeBaseline) === '');
22462+
22463+
// The controls that keep the case above from passing by ignoring
22464+
// everything. ⚠️ Without them, "prints nothing of its own" and "prints
22465+
// nothing that is ever looked at" are the same green — and this battery
22466+
// would have no way to tell a working subtraction from one that answers
22467+
// '' for every input.
22468+
const probeImporting = (name, source) => {
22469+
const target = nodePath.join(entryTmp, name);
22470+
writeFileSync(target, source);
22471+
const importer = nodePath.join(entryTmp, `import-${name}`);
22472+
writeFileSync(importer, `await import(${JSON.stringify(pathToFileURL(target).href)});\n`);
22473+
return spawnEntryChild(importer);
22474+
};
22475+
t(
22476+
'NEGATIVE CONTROL: importing a module that writes nothing is clean under THIS runtime',
22477+
stderrBeyondRuntime(probeImporting('quiet.mjs', 'export const nothing = 1;\n'), runtimeBaseline) === '',
22478+
);
22479+
t(
22480+
'POSITIVE CONTROL: one line the imported module writes to stderr still reds',
22481+
stderrBeyondRuntime(
22482+
probeImporting('noisy.mjs', "console.error('a line this module wrote itself');\n"),
22483+
runtimeBaseline,
22484+
).includes('a line this module wrote itself'),
22485+
);
22486+
// The case that separates a MEASURED baseline from a shape filter: this
22487+
// line is shaped exactly like one of node's own warnings, so a filter
22488+
// written to that shape swallows it. Subtracting a measured baseline
22489+
// cannot — the baseline child never wrote it.
22490+
t(
22491+
'POSITIVE CONTROL: an imported line DISGUISED as a node warning still reds',
22492+
stderrBeyondRuntime(
22493+
probeImporting(
22494+
'disguised.mjs',
22495+
"console.error('(node:4242) [FAKE-CODE] Warning: written by the MODULE, not the runtime');\n",
22496+
),
22497+
runtimeBaseline,
22498+
).includes('written by the MODULE, not the runtime'),
22499+
);
2237422500
t(
2237522501
"imported by a consumer whose own argv says --self-test, THIS file's self-test does not fire",
2237622502
!(imported.stdout ?? '').includes('dispatch-gates self-test:'),

0 commit comments

Comments
 (0)