Skip to content

test(mcp-probe): order the settle-fence test on the probe's connect instead of a 20 ms sleep - #449

Merged
ScriptedAlchemy merged 2 commits into
mainfrom
fix/mcp-probe-settle-fence-event-order
Sep 3, 2026
Merged

ScriptedAlchemy merged 2 commits into
mainfrom
fix/mcp-probe-settle-fence-event-order

Conversation

@ScriptedAlchemy

@ScriptedAlchemy ScriptedAlchemy commented Sep 3, 2026 •

Copy link
Copy Markdown
Owner

Summary

Two changes:

  1. Test determinism (packages/agent-bundle/tests/mcp-probe-service.test.ts) — see below.
  2. AGENTS.md "Pull requests" — replaces the @codex review re-request / head-SHA-confirmation / quota-fallback rules with the maintainer's new process: a local reviewer subagent self-review before merge, findings fixed or dismissed in a "Self-review" PR section, organic review comments still addressed, external reviews never solicited.

Test fix

pnpm check:local-ci on origin/main @ 10a98a0fb (final sweep, load average 50–75) failed exactly one test, on the Node 22.19 verify leg:

packages/agent-bundle/tests/mcp-probe-service.test.ts
  settle() fences in-flight probes, not only already-registered teardowns (#397 review)
  AssertionError: promise rejected "Error: ENOENT: no such file or directory, … proof.txt" instead of resolving   (line 737)

Root cause: the test slept 20 ms "to let the probe reach its (blocked) connect" and then read the plugin-data file that createPluginData writes before connect. Under load the probe had not reached createPluginData inside 20 ms, so pluginData was still the previous value and the read hit ENOENT. A second 50 ms sleep guarded the "settle must not resolve early" assertion the same way.

Fix at source (test only, no product change): the stubbed connect resolves a connectStarted deferred the moment it is entered, and the test awaits that instead of the sleep — plugin data provably exists at that point and the teardown is provably not registered yet. The early-resolution check drains one macrotask (setImmediate) instead of sleeping 50 ms: a fence that only covered registered teardowns resolves within the current macrotask, so the regression is still observed, with no wall-clock budget left in the test.

Evidence

  • Failure: .worktrees/local-ci/logs/verify-node22/07-test-unit.log from the sweep run (3015/3021, one failed test; Node 24 and 26 verify legs and the gates leg were green on the same commit).
  • After the fix: the file run 20× in a loop at load average 66–77 → 20/20 green (/tmp/final-sweep/probe-loop-*.log).
  • Pre-fix reproduction on unmodified main under the same load is recorded in the sweep report (10-run loop).

Test plan

  • pnpm exec rstest --config rstest.unit.config.ts packages/agent-bundle/tests/mcp-probe-service.test.ts ×20 under load
  • CI Verify

Self-review

  • Reviewer: change-risk-reviewer subagent, model gpt-5.6-sol-high, on head f3dfd3d4b vs origin/main, asked for concrete merge risks only.
  • Findings (all verified against source, none actionable):
    1. mcp-probe-service.ts:428,590 — createPluginData and the proof.txt write complete before connect() is called on the same probe, so the awaited read cannot ENOENT. Disposition: confirms the fix; no change.
    2. mcp-probe-service.test.ts:745-750 — a teardown-only settle() resolves through microtasks before setImmediate fires, so the regression is still caught. Disposition: confirms the guard; no change.
    3. mcp-probe-service.ts:558-590 — one probe calls connect() exactly once; never reaching it fails/times out rather than false-greening. Disposition: no change.
    4. AGENTS.md:66-80 — repo-wide search found no remaining instruction to post @codex review or record unreviewed SHAs. Disposition: no change.
    5. .changeset/README.md:24-27, .changeset/config.json:6 — tests/** and root guidance are exempt; no changeset required (the "Changeset present" check passed without a label). Disposition: no change.
  • Verdict: no blockers; no fixes were needed, so no second pass was required.
  • No organic review comments have arrived on this PR.

@ScriptedAlchemy

Copy link
Copy Markdown
Owner Author

@codex review

@changeset-bot

changeset-bot Bot commented Sep 3, 2026 •

Copy link
Copy Markdown

⚠️ No Changeset found

Latest commit: f3dfd3d

Merging this PR will not cause a version bump for any packages. If these changes should not result in a new version, you're good to go. If these changes should result in a version bump, you need to add a changeset.

This PR includes no changesets

When changesets are added to this PR, you'll see the packages that this PR includes changesets for and the associated semver types

Click here to learn what changesets are, and how to add one.

Click here if you're a maintainer who wants to add a changeset to this PR

@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Sep 3, 2026 •

Copy link
Copy Markdown

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ✅ Completed 2026-09-03T20:48:18.300045Z df99f14 Manual request
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@pkg-pr-new

pkg-pr-new Bot commented Sep 3, 2026 •

Copy link
Copy Markdown
npm i https://pkg.pr.new/ScriptedAlchemy/agent-bundle@449
npm i https://pkg.pr.new/ScriptedAlchemy/agent-bundle/create-agent-bundle@449
npm i https://pkg.pr.new/ScriptedAlchemy/agent-bundle/@agent-bundle/runtime@449

commit: f3dfd3d

@chatgpt-codex-connector

Copy link
Copy Markdown

Codex Review: Didn't find any major issues. Keep them coming!

Reviewed commit: df99f14bbe

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant