feat(panel): fully interactive CDP-screencast viewport — full switchover (v0.6.0)

CHANGELOG.md

@@ -1,5 +1,31 @@
 # Changelog
 
+## v0.6.0
+- **The Browser panel is now a fully interactive, drivable viewport — and it is the ONLY mode.**
+  A live **CDP screencast** — event-driven JPEG frames (not a screenshot poll) painted on a
+  `<canvas>` — forwards your **mouse, keyboard, and scroll** back into the page via
+  `Input.dispatch*`. You can click, type, and scroll the real browser from the console, right
+  alongside the agent. Because every byte is a **gated same-origin WebSocket**, it works on the
+  host **and** a remote fleet member.
+- **How it works.** A second CDP client attaches to the same Chrome agent-browser drives (via
+  `agent-browser get cdp-url`); `browser_stream.py` bridges `Page.startScreencast` ⇆ the panel.
+  The nav toolbar (url / back / forward / reload) reuses the gated HTTP `/nav` route.
+- **Start button + configurable homepage.** When no page is open, the panel shows a **Start
+  button** instead of a dead end. A new `home_url` config sets the page it opens to: when set,
+  the panel **auto-opens** it (and the button reads “Open <host>”); blank → the button opens
+  `about:blank`. Editable in Settings ▸ Plugins.
+- **WebSocket auth.** The host's operator-bearer gate is HTTP-only and does **not** cover WS
+  handshakes, so the stream self-gates: the panel mints a **single-use ticket** from the gated
+  `POST /stream-ticket` (bearer-checked) and presents it on the WS URL; the handler validates and
+  burns it. Safe in gated deployments, transparent in open ones.
+- **Full switchover — the old dashboard-embed approach is removed** (no backward compatibility):
+  - `panel_mode` is gone (there is one panel now); so is the screenshot `minimal` mode and its
+    `/shot` route, and the `full` dashboard-embed page.
+  - The **`browser_dashboard` tool is removed** (16 tools now) along with the boot/shutdown
+    **dashboard lifecycle** (`lifecycle.py`) and the panel's `/dashboard` control routes.
+  - Removed config: `panel_mode`, `dashboard_port`, `manage_dashboard`. The interactive panel
+    talks CDP directly and never needs agent-browser's separate dashboard daemon.
+
 ## v0.5.1
 - **Full mode: an "Open ↗" button** to pop the dashboard out into a full browser tab, alongside the
   inline embed. Shown whenever the dashboard is on this machine (loopback host) — including over an

README.md

@@ -33,25 +33,24 @@ point.
   `browser_snapshot`, `browser_click`, `browser_fill`, `browser_type`,
   `browser_get_text`/`get_html`/`get_value`, `browser_press`, `browser_hover`,
   `browser_eval`, `browser_screenshot`, `browser_back`/`forward`/`reload`,
-  `browser_close`, `browser_dashboard`.
+  `browser_close`.
 - **Skill** — a discovery skill that defers to the CLI's always-current workflow
   content (`agent-browser skills get core`), so instructions never go stale.
 - **Workflows** — declarative browser recipes (browse-and-extract, fill-a-form, …).
-- **Browser panel** — a console view (ADR 0026) for watching/driving the browser. Two
-  modes, set by `panel_mode`:
-  - **`full` (default)** — **embeds agent-browser's dashboard inline** (viewport + activity/
-    console/network feeds) at its own **local origin** (`http://<host>:<port>/`). Best for a
-    **local setup** (console + agent-browser on one machine). Because the dashboard is a
-    Next.js app with root-absolute assets (no base-path), it only loads at its own origin —
-    so when the console is opened **remotely** (a fleet member, a non-loopback host, or over
-    https) its `localhost` isn't reachable from your browser, and the panel shows a **clear
-    error** (pointing you at `minimal`) rather than a blank frame.
-  - **`minimal`** — a **live screenshot** of the viewport + a nav toolbar, all through the
-    **gated same-origin routes**. Works everywhere (host and member), no dashboard daemon
-    needed — use it for a remote/member agent.
-
-  Either mode can **start the dashboard from the panel** (no terminal) — the Start/Stop
-  control hits the gated `POST /api/plugins/agent_browser/dashboard`.
+- **Browser panel** — a console view (ADR 0026) that is a **fully drivable viewport**. A live
+  **CDP screencast** (event-driven JPEG frames, not a screenshot poll) is painted on a
+  `<canvas>`, and your **mouse / keyboard / scroll** are forwarded back into the page via
+  `Input.dispatch*` — so you can click, type, and scroll the real browser from the console,
+  alongside the agent. Everything rides a **gated same-origin WebSocket**, so it works on the
+  **host and a remote fleet member** alike. A second CDP client attaches to the same Chrome
+  agent-browser drives (`agent-browser get cdp-url`); `browser_stream.py` does the bridging.
+
+  When no page is open the panel shows a **Start button** (not a dead end). Set `home_url` to a
+  page and the panel **auto-opens** it — a homepage — otherwise Start opens `about:blank`.
+
+  **WebSocket auth:** the host's operator-bearer gate is HTTP-only and doesn't cover WS
+  handshakes, so the stream self-gates — the panel mints a **single-use ticket** from the gated
+  `POST /api/plugins/agent_browser/stream-ticket` and presents it on the WS URL.
 
 ## Requirements
 
@@ -75,20 +74,19 @@ plugins:
 
 agent_browser:
   binary: agent-browser
-  dashboard_port: 4848
 ```
 
 ## Layout
 
 | File | What |
 |---|---|
 | `tools.py` | the browser tools — subprocess wrappers over the `agent-browser` CLI |
-| `browser_panel.py` | the Browser panel — `minimal` (live viewport + nav + dashboard control) and `full` (a launcher for the dashboard) |
-| `lifecycle.py` | the dashboard daemon surface — start on boot, stop on shutdown (ADR 0018) |
+| `browser_panel.py` | the Browser panel page + routes — the interactive canvas + the gated nav / stream-ticket / WS-stream routes |
+| `browser_stream.py` | the CDP bridge — screencast frames out, input in; the WS ticket auth |
 | `skills/` | the discovery skill (defers to `agent-browser skills get core`) |
 | `workflows/` | declarative browser recipes |
 | `tests/` | the host-free pytest suite (subprocess mocked — no binary needed) |
-| `__init__.py` | `register()` — wires tools + panel + lifecycle; skills/workflows auto-discovered |
+| `__init__.py` | `register()` — wires tools + the interactive panel; skills/workflows auto-discovered |
 
 The operator knobs (panel mode, headed, allowed domains, profile, device, …) are editable in
 **Settings ▸ Plugins ▸ Agent Browser**, or under `agent_browser:` in `langgraph-config.yaml`.

__init__.py

@@ -2,9 +2,10 @@
 
 Composition over construction: this plugin is a thin shell over the `agent-browser`
 CLI/daemon. It contributes the browser **tools** (subprocess wrappers), a discovery
-**skill** + browser **workflows** (auto-discovered from skills/ and workflows/), and a
-**Browser panel** console view that embeds agent-browser's own live dashboard — it
-does NOT reimplement browser automation or a renderer.
+**skill** + browser **workflows** (auto-discovered from skills/ and workflows/), and an
+interactive **Browser panel** console view — a live, drivable CDP-screencast viewport
+(browser_stream bridges Chrome's CDP to a canvas over a gated WebSocket). It does NOT
+reimplement browser automation or a renderer.
 
 Ships DISABLED. Enable with `plugins: { enabled: [agent_browser] }` and put the
 `agent-browser` binary on PATH (`npm i -g agent-browser && agent-browser install`).
@@ -28,12 +29,13 @@ def register(registry) -> None:
     except Exception:  # noqa: BLE001 — tools are the foundation; log loudly if they fail
         log.exception("[agent_browser] registering browser tools failed")
 
-    # Browser panel console view (embeds agent-browser's dashboard). Built out by the
-    # board; register it best-effort so the foundation works before the view lands.
-    # TWO routers at DISTINCT prefixes: the PAGE (+ the iframe-loaded /panel/dash
-    # proxy, which can't carry a bearer) stays on the public /plugins/agent_browser;
-    # the shot/nav DATA routes mount under /api/plugins/agent_browser so they
-    # inherit the operator bearer gate (plugin-view rule 2).
+    # Interactive Browser panel console view. Register it best-effort so the tools still
+    # serve if the panel can't import. TWO routers at DISTINCT prefixes: the PAGE stays on
+    # the public /plugins/agent_browser (an iframe page-load can't carry a bearer); the
+    # DATA routes (the nav toolbar, the stream ticket + the /stream WS) mount under
+    # /api/plugins/agent_browser so the HTTP ones inherit the operator bearer gate
+    # (plugin-view rule 2). The WS gates itself with a single-use ticket — the host's auth
+    # middleware doesn't cover WS handshakes.
     try:
         from .browser_panel import build_panel_data_router, build_panel_router
         registry.register_router(build_panel_router(cfg))
@@ -43,16 +45,5 @@ def register(registry) -> None:
     except Exception:  # noqa: BLE001
         log.exception("[agent_browser] mounting browser panel failed")
 
-    # Lifecycle (ADR 0018): on shutdown, stop the dashboard daemon we manage so it
-    # doesn't outlive the server (dashboard-only; the session is left alone).
-    try:
-        from .lifecycle import make_dashboard_surface
-        start, stop = make_dashboard_surface(cfg)
-        registry.register_surface(start, stop=stop, name="agent-browser-dashboard")
-    except Exception:  # noqa: BLE001 — lifecycle is best-effort; tools/panel still serve
-        log.exception("[agent_browser] registering lifecycle surface failed")
-
     # skills/ and workflows/ are auto-discovered (ADR 0027) — no register call.
-    log.info("[agent_browser] registered browser tools (binary=%s, dashboard:%s, manage=%s)",
-             cfg.get("binary", "agent-browser"), cfg.get("dashboard_port", 4848),
-             cfg.get("manage_dashboard", True))
+    log.info("[agent_browser] registered browser tools (binary=%s)", cfg.get("binary", "agent-browser"))