Skip to content

test: real Puppeteer/Playwright connect smoke test (xfail until the answerer is complete) - #3

Merged
kmjones1979 merged 1 commit into
1clawAI:mainfrom
redbotster:test/framework-connect-smoke
Sep 2, 2026
Merged

kmjones1979 merged 1 commit into
1clawAI:mainfrom
redbotster:test/framework-connect-smoke

Conversation

@redbotster

Copy link
Copy Markdown
Contributor

Follow-up to #2. That issue was closed as fixed by the handshake answerer (be3d888), but a real-client check shows stock Puppeteer and Playwright still can't open a page (details in the issue thread). This adds the test that would have caught it, so the "point your framework at the bridge" claim stays honest as the answerer grows.

Why this test

handshake.test.ts asserts a synthetic Browser.getVersion reply. It's green, but it drives a hand-built client, not a real one — so it can't see that a stock client gets past getVersion and then blocks. This drives the actual clients (puppeteer-core, playwright-core): connect → newPage → goto, which is the thing the README invites people to do.

Why it.fails

As of the answerer landing, a stock client:

  • with the gate enforced, is refused on the next method it sends after getVersion — Target.getBrowserContexts (Puppeteer), Browser.setDownloadBehavior (Playwright);
  • even with every method allowed, never resolves newPage(), because the proxy doesn't synthesise the target-lifecycle events (targetCreated / attachedToTarget / targetInfoChanged) the client waits for to build its Page.

So the body is the real acceptance test, marked it.fails because it can't pass yet. When the answerer presents a coherent target lifecycle for a client's own pages, the body succeeds, it.fails turns the test red, and that's the signal to drop the .fails. It's a live tripwire, not a skipped aspiration.

No new dependencies

A smoke test shouldn't pull Playwright's install into everyone's CI, so puppeteer-core / playwright-core are not added to package.json. The test imports them by a name the type checker can't resolve and skips when they're absent (like the real-Chromium tests skip without Chrome). To run it:

pnpm add -D -w puppeteer-core playwright-core
pnpm test framework-connect

Verified locally both ways: skips clean without the frameworks (full suite stays 23 files green), and passes as xfail with them installed. Typechecks under the test tsconfig.

Pairs with the compat discussion in #2 — merge whenever, or hold until the answerer work is scheduled; either way the tripwire is ready.

🤖 Generated with Claude Code

… now)

handshake.test.ts drives a synthetic client, so it stays green while a stock
client still cannot open a page. This drives the real ones (puppeteer-core,
playwright-core): connect -> newPage -> goto. It is the acceptance test for the
handshake answerer (issue 1clawAI#2).

Marked it.fails on purpose. As of the answerer landing, a stock client gets
past Browser.getVersion and then blocks: with the gate enforced it is refused
on the next method it sends (Target.getBrowserContexts for Puppeteer,
Browser.setDownloadBehavior for Playwright); even with every method allowed,
newPage() never resolves, because the proxy does not synthesise the
target-lifecycle events the client waits for to build its Page. When the
answerer presents a coherent target lifecycle for a client's own pages, the
body passes, it.fails turns red, and that is the signal to drop the .fails.

The framework clients are not added as dependencies — a smoke test should not
pull Playwright's install into everyone's CI. The test skips unless they are
present; run it with `pnpm add -D -w puppeteer-core playwright-core` then
`pnpm test framework-connect`. Verified both paths locally: skips clean without
them, passes as xfail with them.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WqPU5z6K3maS9J6LrcsEic
kmjones1979 added a commit that referenced this pull request Sep 2, 2026
The test arrived as an xfail tripwire for a gap that was real, and it was
right on every detail. It has fired: stock puppeteer-core and playwright-core
now connect, open a page and navigate through the gate, so the `.fails` is
gone and these are ordinary tests that run in CI.

Co-authored-by: redbotster <redbotster@users.noreply.github.com>
Claude-Session: https://claude.ai/code/session_019d5ks2kCa4eXD1ftdkECPu
@kmjones1979
kmjones1979 merged commit 402782f into 1clawAI:main Sep 2, 2026
3 of 4 checks passed
@kmjones1979

Copy link
Copy Markdown
Member

Merged as 9d765cc, with your commit and authorship intact.

You were right, and I was wrong to close #2 the way I did. I checked that the proxy answered the handshake methods and wrote "Puppeteer and Playwright connect" — I had never run either one. Your test is what showed that, and both of your predictions were exact: Puppeteer stops on Target.getBrowserContexts, Playwright on Browser.setDownloadBehavior, and past those a page still would not open.

The .fails is gone. Both bodies pass now, against a launched Chromium, in CI.

What it took, past the two methods you named — I stopped guessing and diffed the wire against a raw Chromium:

  • Auto-attach performed for real, on the client's behalf. A client that asks for auto-attach does not attach to its own pages; it waits to be told, and builds its Page from the sessionId in that event. The attach has to be real — a fabricated id gives you a client that believes it has a page and is refused on every command it sends to it.
  • The two Network.*ExtraInfo events, forwarded with their cookie fields emptied. This was the one I would not have found by reading. They were refused outright, correctly: they carry raw Cookie and Set-Cookie. But no framework will settle a navigation without them, so page.goto() hung while page.content() returned the new document — the navigation had completed. Diffing the event stream against a raw Chromium showed those two as the only difference. What a client needs is that the event happened; what it must not have is the headers. Emptied, not deleted: a client does Object.keys(headers) without checking the field is there, so deleting it crashed the client instead of protecting anything.
  • Locally-answered replies echoing their sessionId. A client routes a reply by session, so one that arrives on the root session carrying an id the root never sent is a protocol violation. Playwright asserts on it. Puppeteer does not — which is exactly the argument for your test driving both.

Two things I changed from your version, both worth saying out loud:

puppeteer-core and playwright-core are devDependencies now. Your reasoning was sound for playwright, which downloads browsers — but the -core packages are 21MB together and download none, since they drive a Chromium you already have. Left optional, the one test that checks this repo's headline claim would skip in CI, and a skipped test is indistinguishable from a passing one. The optional import stays, so a checkout without them still runs the rest of the suite.

I also wrote the strip list for those cookie fields against a live browser rather than the protocol docs, after the first version missed blockedCookies — which holds whole cookie objects — and included rawHeaders, which the docs list and Chromium never sends. There is a real-Chromium test that sets a real Set-Cookie, navigates twice so the cookie is sent back as well as set, and requires the value to be absent from every frame crossing the socket.

Thanks for filing this the way you did. The xfail was the right instrument: it made the gap impossible to close by assertion, which is precisely how I closed it the first time.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants