fix(test): stop awaiting the bad-credentials login through expect().rejects - #75
Conversation
…ejects "a persona whose credentials are wrong fails the run instead of opening it signed out" has been failing on `main` since 2026-09-02 (runs 33969855127, 33647731476), timing out at exactly 30,000ms. The cause is the assertion, not the behaviour it asserts. Awaiting a long, multi-round-trip browser flow THROUGH the `.rejects` matcher starves the loop driving it, so every CDP call inside the pending promise costs ~1s. The login types 20 characters and presses 6 keys, so it never finished inside the test's 30s ceiling. Measured, same `performLogin` / adapter / page, only the await differing: awaited directly 1.4s awaited via .rejects 31.5s `login.ts`'s `NAVIGATION_SETTLE_MS` note already chased this timeout once and fixed a real contributor (a 30s idle wait, now 5s). It was not the whole cost — the settle wait measures 3ms here, while the typing measured 32s. Catching the rejection with try/catch and asserting on the caught error takes the flow out of the matcher: 30s timeout -> 1.84s, well inside the existing ceiling, so no timeout needed raising. Left `.rejects` alone everywhere else in the suite: every other use wraps a call that fails on one round trip or none, where a single stall costs nothing. The rule is about how much work is in flight, not about the matcher. Verified: `bun test src/services` 127/127, lint clean, typecheck clean. The 29 failures in browser-adapter.integration.test.ts are pre-existing and environmental (the dummy/web fixture CI builds is not built locally) — they fail identically on stock main. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_015KQipy9s9tJFkGBeGWnZCX
|
No actionable comments were generated in the recent review. 🎉 ℹ️ Recent review info⚙️ Run configurationConfiguration used: Organization UI Review profile: CHILL Plan: Essentials Run ID: 📒 Files selected for processing (1)
Included review availability: 0 reviews are currently available. Your included PR review attempts over the past 7 days set your current allowance at 1 review per hour. 📜 Recent review details⏰ Context from checks skipped due to timeout. (1)
🧰 Additional context used📓 Path-based instructions (4)Never expose authentication secrets to model context or logs.📄 CodeRabbit inference engine (CLAUDE.md) Files:
Use strict TypeScript and never use `any`.📄 CodeRabbit inference engine (CLAUDE.md) Files:
Keep MCP handlers thin and place business logic in services.📄 CodeRabbit inference engine (CLAUDE.md) Files:
Maintain fast unit tests, with the documented test suite completing in under 10 seconds, and test session isolation and other correctness invariants.📄 CodeRabbit inference engine (CLAUDE.md) Files:
🔇 Additional comments (1)
📝 WalkthroughWalkthroughThe authentication failure test now awaits the browser login flow directly, captures the thrown error, and verifies that it is an ChangesAuthentication Test
Priority: ⬇️ Low Estimated code review effort: 2 (Simple) | ~10 minutes Merge Risk: ⚪ Minimal · up to The bad-credentials browser test now captures and verifies the authentication error directly, avoiding the prior timeout while preserving the expected failure behavior. The change is ready to merge. 🚥 Pre-merge checks | ✅ 5✅ Passed checks (5 passed)
✨ Finishing Touches📝 Generate docstrings
🧪 Generate unit tests (beta)
A rabbit awaits the login gate Comment |
a persona whose credentials are wrong fails the run instead of opening it signed outhas been red onmainsince 2026-09-02 (runs 33969855127, 33647731476), timing out at exactly 30,000ms. This is the failure that was already red when #74 merged.The cause is the assertion, not the behaviour
Awaiting a long, multi-round-trip browser flow through the
.rejectsmatcher starves the loop driving it: every CDP call inside the pending promise then costs ~1s. This login types 20 characters and presses 6 keys, so it could never finish inside the test's 30s ceiling.Isolated with an A/B in the same process, same test file — same
performLogin, same adapter, same page, same config, same profile dir, same log sink. Only the await differs:await performLogin(...)in try/catchawait expect(performLogin(...)).rejects.toThrow(...)Everything else was ruled out first by elimination: browser binary (system Chrome vs Playwright's), the log sink (real
FindingsStorevs no-op — it receives 6 lines total), the profile dir, launch options (byte-identical), single vs double navigation, and launch order.Why the previous fix wasn't enough
login.ts'sNAVIGATION_SETTLE_MScomment already chased this exact timeout and fixed a genuine contributor — a settle wait that could run the full 30s budget, now capped at 5s. That fix is real and stays. It just wasn't the whole cost: instrumented here, the settle wait measures 3ms while the typing measured 32s.The fix
Catch the rejection and assert on the caught error, taking the flow out of the matcher. 30s timeout → 1.84s, well inside the existing ceiling — so no timeout needed raising, which is the outcome worth having: the test is fast because the work is fast, not because it was given more room.
.rejectsis left alone everywhere else in the suite. Every other use wraps a call that fails on one round trip or none, where a single stall costs nothing. The rule is about how much work is in flight, not about the matcher — the comment in the test says so, so the next person doesn't "tidy" it back.Verified
bun test src/services— 127/127bun run lint,bun run typecheck— cleanThe 29 failures in
browser-adapter.integration.test.tsare pre-existing and environmental (they need thedummy/webfixture CI builds, which isn't built locally); they fail identically on stockmain, before this change.🤖 Generated with Claude Code
https://claude.ai/code/session_015KQipy9s9tJFkGBeGWnZCX
Need help on this PR? Tag
@codesmith-botwith what you need. Autofix is disabled.Summary by CodeRabbit