Skip to content

Commit faf9359

Browse files
committed
docs: update AGENTS.md; add install-sim test and landscape doc
- AGENTS.md: sync durable facts (extension backend, IBS utilities, login-flow, preparePage hook, rescue-flights source migration, competitor landscape note) - docs/landscape-session-auth-tools.md: new competitor landscape doc - tests/e2e/install-simulation.test.ts: install simulation E2E test - vitest.install-sim.config.ts: vitest config for install-sim suite - package.json: add test:install-sim script Made-with: Cursor
1 parent a897c97 commit faf9359

5 files changed

Lines changed: 559 additions & 2 deletions

File tree

AGENTS.md

Lines changed: 20 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -42,7 +42,12 @@ Durable facts and correction patterns for this workspace. Updated by continual-l
4242
- `warm_up_browser()` (visiting google/wiki/github before login) was evaluated from stickerdaniel's code — decided as "nice to have" for first-time login, not adopted yet
4343
- `browserkit login <site>` is blocked by the `CI=1` env var that Cursor sets — must run as `CI="" node packages/core/dist/cli.js login <site>` to open a headed browser from within Cursor terminal
4444
- `browser` tool `snapshot` action is planned — returns incremental aria-snapshot diff, more token-efficient than screenshots; `page-snapshot` MCP resource already exists, the action adds diff support. Inspired by `SawyerHood/dev-browser`.
45+
- **Chrome Extension browser backend (planned)**: `authStrategy: "extension"` as a new option alongside `persistent`, `storage-state`, `cdp-attach`. The extension uses `chrome.debugger` API as a CDP relay — Patchright connects via `connectOverCDP()` through a WebSocket relay in the daemon, giving full Playwright API with zero reimplementation. Prior art: Playwright MCP (`microsoft/playwright-mcp`) and Playwriter (`remorses/playwriter`). Extension is ~200 lines of CDP relay + tab group management. Key UX: per-adapter tab groups (named, color-coded) and no-focus-stealing navigation (`chrome.tabs.create` with `active: false`). Accepted tradeoffs: Chrome-only, Web Store review friction, `chrome.debugger` shows yellow "automated" banner on attached tabs, no headless fallback. Best for sites with aggressive bot detection (Google, LinkedIn) — mix-and-match with Patchright per adapter in `browserkit.config.ts`.
46+
- **Chrome Extension backend implemented** using **Playwriter** (`remorses/playwriter`, MIT) as optional peer dependency — no custom extension required. Playwriter was chosen over `microsoft/playwright-mcp` extension because it already has tab group management (named group, green color, `tabGroups` manifest permission) AND no-focus navigation (`active: false` on all tab creates). User installs Playwriter extension from Chrome Web Store once; daemon calls `connectOverCDP()` to get a full Patchright `Page` — adapter code is entirely unchanged. Key implementation rules: `closeSite` must NOT close the user's Chrome context for extension strategy; `setMode` and `injectStorageState` are no-ops; `getCurrentMode` returns `"watch"`; `handleAuthFailure` skips human-handoff and returns early for extension mode.
4547
- CloakBrowser is an optional npm package integrated in the Booking.com adapter for DataDome/headless bot-detection bypass — opt-in via adapter config, no hard dependency on core; other stealth patches (Patchright) remain in place alongside it.
48+
- `preparePage?(page)` optional hook added to `SiteAdapter` interface — called automatically by `SessionManager.getPage()` before returning the page to any tool; adapters use it for per-tool setup (accept cookies, dismiss dialogs, etc.)
49+
- `getLoginOptions?()` optional hook on `SiteAdapter` — returns `LoginOptions` for automated form-fill login; adapters without this hook fall back to existing human-handoff behavior unchanged
50+
- `withLoginFlow(page, opts)` implemented in `login-flow.ts` — executes form-fill login sequence, matches post-submit URL against `possibleResults`, returns void on success, throws `LoginError` on definitive failure (wrong password, account blocked), throws plain `Error` for transient/unexpected failures so callers fall back to human-handoff
4651

4752
## Design Process Preferences
4853

@@ -67,20 +72,33 @@ Durable facts and correction patterns for this workspace. Updated by continual-l
6772
- The main browserkit README doubles as the project's public-facing "blogpost" — user refers to it interchangeably; keep it polished and up-to-date with available + planned adapters
6873
- Personal adapters (outside the browserkit org) live in `jonzarecki/` GitHub repos — e.g., the rescue-flights adapter (Israir + El Al) is at `jonzarecki/` and must not appear in `browserkit-dev/` repos or CI
6974
- Verification harness convention: `make agent-check` runs browser-snapshot-based checks and loops until they pass — add to `CLAUDE.md` of each adapter and run after every change
75+
- Competitor landscape for session persistence / auth-handoff tools is documented in `docs/landscape-session-auth-tools.md`. **Chromectl** is the closest architectural competitor to watch (local-first design ethos, similar to browserkit); others in the landscape: AgentAuth, BrowserState, web-ctl, Playwrightess.
7076

7177
## Rescue-Flights Adapter (Personal)
7278

73-
- Source `.ts` files are NOT in the local workspace — only compiled `dist/` JS exists at `packages/adapter-rescue-flights/dist/`; the source lives in the `jonzarecki/` GitHub repo
79+
- Source `.ts` files ARE in the local workspace at `packages/adapter-rescue-flights/src/` (elal.ts, index.ts, israir.ts, run-check.ts, types.ts); the adapter was migrated from dist-only to full source in the monorepo
80+
- The adapter is NOT in the pnpm workspace (`pnpm-workspace.yaml` excludes it) — build with `cd packages/adapter-rescue-flights && pnpm build` directly
81+
- `package.json` uses `@browserkit-dev/core: ">=0.1.0"` as peerDep and `github:browserkit-dev/browserkit#main` as devDep (not `workspace:*`); has standalone `tsconfig.json` (not extending monorepo base)
82+
- Has `.github/workflows/ci.yml`: dual-checkout (adapter + browserkit core), core build, dep path patch, patchright install, adapter build, unit tests (`timeout-minutes: 10`)
7483
- Runs locally at port 52746; registered in `.cursor/mcp.json` as `"rescue-flights"` (local config only, not committed to monorepo)
7584
- **Israir tool**: `detailUrl`, `flightNumber`, and `departureTime` are only populated when `availableSeats > 0`; sold-out flights return empty strings for those fields
76-
- **Israir `buildDetail()` bug**: guard `if (!available || !price)` is overly strict — flights with seats but no price get no booking link; fix is to change to `if (!available)` (1-line change, zero risk)
85+
- **Israir `buildDetail()` bug**: guard `if (!available || !price)` is overly strict — flights with seats but no price get no booking link; fix implemented as `if (finalSeats === null || finalSeats === 0)` (gates on actual seat count from the segment, not priceBar flag — also correctly suppresses links when segment overrides priceBar count to 0)
7786
- **El Al tool**: always returns `flightNumber` and `departureTime` for all flights (including sold-out); `detailUrl` links to the seat-availability page (`?d=0` from Israel / `?d=1` to Israel) — El Al booking pages all return 403 (session tokens required), so the availability page is the best accessible link
7887
- **El Al virtual scroll bug**: Angular virtual scroll recycles DOM nodes on scroll-back — must collect flight data incrementally *during* each scroll step (not after); single-pass post-scroll extraction returns only the currently-visible rows (~7 flights vs 168+ total)
7988
- **Coverage difference**: El Al covers the next 8 days only; Israir covers 30+ days ahead
8089
- El Al scraper returns `ERR_ABORTED` when called concurrently with Israir — run the two scrapers sequentially to avoid
8190
- **Israir booking URL** format: `https://www.israir.co.il/he-IL/reservation/deal/searchFlight/abroadFlight?destCode=TLV&departDate=...&fNumbers=...&sessionId=...` — the `sessionId` is live-session-scoped and expires; cannot be reused outside the active browser session
8291
- **Verification preference**: use the adapter's own headless Patchright browser (not the cursor-ide-browser MCP) for rescue-flights verification — user stated strong preference ("I prefer it immensely")
8392

93+
## Core Utility Modules (IBS-Inspired)
94+
95+
Adopted from `eshaham/israeli-bank-scrapers` patterns — all additive, no existing adapters broken:
96+
97+
- **`waiting.ts`**`TypedError`, `SECOND`, `waitUntil(asyncTest, description, timeout, interval)`, `raceTimeout(promise, ms)`, `runSerial(tasks)`, `sleep(ms)`; adapter-facing polling primitive — prefer over ad-hoc `waitForTimeout()` loops
98+
- **`fetch-utils.ts`** — Node-side: `fetchGet(url, headers?)`, `fetchPost(url, body, headers?)`, `fetchGraphql(url, query, variables, headers?)`; in-page (inherits browser cookies/session): `fetchGetWithinPage(page, url)`, `fetchPostWithinPage(page, url, body)` — critical for adapters that call authenticated XHR/JSON APIs without navigating
99+
- **`adapter-utils.ts` additions**`fillInput`, `clickButton`, `waitUntilElementFound`, `waitUntilElementDisappear`, `waitUntilIframeFound`, `waitForUrl`, `navigateWithRetry` — all re-exported from `index.ts`
100+
- **`login-flow.ts`**`AuthErrorType` enum (`INVALID_PASSWORD`, `CHANGE_PASSWORD`, `ACCOUNT_BLOCKED`, `TIMEOUT`, `GENERIC`), `LoginError` class, `LoginOptions`/`PossibleLoginResults` types; `withLoginFlow` executes form-fill and matches result URL — all opt-in
101+
84102
## Versioning
85103

86104
### @browserkit-dev/core
Lines changed: 88 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,88 @@
1+
# Session Persistence & Auth-Handoff Tools — Landscape Notes
2+
3+
> Researched April 2026. These are the closest tools to browserkit's auth/session layer.
4+
> None of them have a site-specific adapter model or `tools()` interface on top — that's the gap.
5+
6+
---
7+
8+
## Chromectl
9+
10+
- **Repo:** [BartlomiejLewandowski/chromectl](https://github.com/BartlomiejLewandowski/chromectl)
11+
- **HN:** [Show HN #47207790](https://news.ycombinator.com/item?id=47207790) (April 2026)
12+
- **Language:** TypeScript (CLI)
13+
- **Approach:** CLI that gives an AI agent a named, isolated Chrome session. You start a session, navigate to a site, log in manually, then hand control back to the agent. Each session gets its own `--user-data-dir` so cookies, auth, and localStorage survive restarts.
14+
- **Key features:**
15+
- Named sessions (`chromectl session start mysite`)
16+
- Human-handoff: agent hands browser to human for login, then takes it back
17+
- Commands: `navigate`, `screenshot`, `eval`, `scrape`, `pick` (click any element → get selector/HTML/styles as JSON)
18+
- Lightweight CLI, not MCP tool definitions (avoids context bloat)
19+
- **Gap vs browserkit:** No adapter layer, no `tools()`, no LockManager, no per-site MCP server, no ecosystem. A primitive that proves demand for the concept.
20+
- **Watch:** Closest soul to browserkit's login flow; HN thread worth monitoring for community signal.
21+
22+
---
23+
24+
## AgentAuth
25+
26+
- **PyPI:** [agentauth-py](https://pypi.org/project/agentauth-py/)
27+
- **Language:** Python
28+
- **Approach:** SDK that extracts encrypted cookies from a live Chrome session for a target domain, then exports/imports them for remote agents.
29+
- **Key features:**
30+
- `agent-auth grab <domain>` — extracts cookies without extensions
31+
- `agent-auth export / import` — portable session state for remote servers
32+
- AES encryption for stored sessions
33+
- **Gap vs browserkit:** Python only; grabs cookies but doesn't expose them as typed AI tools; no adapter model; no headed browser management.
34+
35+
---
36+
37+
## BrowserState
38+
39+
- **PyPI:** [browserstate](https://pypi.org/project/browserstate/0.0.4/)
40+
- **Language:** Python
41+
- **Approach:** Captures and restores full browser context (cookies, localStorage, IndexedDB, service workers, fingerprints) across environments.
42+
- **Key features:**
43+
- Works with Playwright, Selenium, Pyppeteer
44+
- Pluggable storage backends: local filesystem, Redis, S3, GCS
45+
- Enables portable session state across machines and CI pipelines
46+
- **Gap vs browserkit:** Purely a state snapshot/restore utility; no live browser management, no MCP, no tool layer.
47+
48+
---
49+
50+
## web-ctl
51+
52+
- **Repo:** [agent-sh/web-ctl](https://github.com/agent-sh/web-ctl)
53+
- **Language:** TypeScript
54+
- **Approach:** Browser automation for AI agents with persistent session-based control. Headless actions (goto, click, type, read, snapshot) backed by Chrome `userDataDir` for persistence.
55+
- **Key features:**
56+
- AES-256-GCM encrypted session storage
57+
- Human-in-the-loop authentication support
58+
- Works with Claude Code and shell-capable clients
59+
- **Gap vs browserkit:** Generic action runner; no site-specific tooling, no adapter model, no MCP server.
60+
61+
---
62+
63+
## Playwrightess MCP
64+
65+
- **Link:** [scriptbyai.com/playwrightess](https://www.scriptbyai.com/playwrightess/)
66+
- **Approach:** Single `playwright_eval` MCP interface that preserves Playwright browser context between API calls. No session resets between commands.
67+
- **Key features:**
68+
- Persistent context across multi-step flows
69+
- Handles auth flows and form filling without re-login
70+
- **Gap vs browserkit:** Generic eval surface; no typed tools, no adapter ecosystem, no headed/headless switching, no lock manager.
71+
72+
---
73+
74+
## Summary: What they all lack
75+
76+
| Capability | Chromectl | AgentAuth | BrowserState | web-ctl | Playwrightess |
77+
|---|:---:|:---:|:---:|:---:|:---:|
78+
| Site-specific typed tool library ||||||
79+
| `tools()` adapter interface ||||||
80+
| MCP server per site ||||| ✓ (generic) |
81+
| Multi-site daemon ||||||
82+
| Human-handoff fallback ||||||
83+
| Headed/headless switching ||||||
84+
| LockManager (FIFO serialization) ||||||
85+
| npm-publishable adapter packages ||||||
86+
| Testing harness (`/testing` subpath) ||||||
87+
88+
browserkit operates one layer above all of these — it can use any of them as a backend primitive while adding the adapter/tool/ecosystem layer they all lack.

package.json

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -7,6 +7,7 @@
77
"build": "pnpm -r --workspace-concurrency=1 build",
88
"test": "pnpm -r test",
99
"test:e2e": "vitest run --config vitest.e2e.config.ts",
10+
"test:install-sim": "vitest run --config vitest.install-sim.config.ts",
1011
"test:integration": "pnpm --filter @browserkit-dev/adapter-hackernews test:integration",
1112
"lint": "pnpm -r lint"
1213
},

0 commit comments

Comments
 (0)