diff --git a/CHANGELOG.md b/CHANGELOG.md index 89a3344..c594a0c 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -1,5 +1,18 @@ # Changelog +## v0.4.0 +- **The Browser panel works out of the box now.** The default `panel_mode` is **`minimal`** — a + live screenshot + nav toolbar driven through the gated same-origin routes; it works everywhere + (host + member), no dashboard daemon needed. +- **`full` mode is now a launcher, not an embed.** The dashboard is a Next.js app whose assets are + **root-absolute** (`/_next/...`) with no base-path, so it can't render under a sub-path panel — + that was the long-standing blank panel. Full mode now opens the dashboard at its **own origin** + ("Open dashboard ↗", works on a local/host setup) and the **dead sub-path reverse proxy + (HTTP + WebSocket) is removed**. +- **Start the dashboard from the UI** — a live status dot + **Start / Stop** control in the panel + (gated `GET`/`POST /api/plugins/agent_browser/dashboard`), so you never need a terminal. Verified + end-to-end against the real `agent-browser` binary (status → start → running → stop). + ## v0.3.0 - **Host-free test suite** (`tests/`) — the tool subprocess wrappers (arg-building + graceful error degradation), the panel routers (page / four-rules / gating / dashboard diff --git a/README.md b/README.md index 0a4bfa3..6ae2c47 100644 --- a/README.md +++ b/README.md @@ -37,15 +37,20 @@ point. - **Skill** — a discovery skill that defers to the CLI's always-current workflow content (`agent-browser skills get core`), so instructions never go stale. - **Workflows** — declarative browser recipes (browse-and-extract, fill-a-form, …). -- **Browser panel** — a console view (ADR 0026) that **embeds agent-browser's own - live dashboard** (`agent-browser dashboard start`, port 4848): the live viewport - (CDP screencast) + the command activity / console / network feeds. We hijack their - renderer rather than building one. The dashboard is served **same-origin through - the plugin's own reverse-proxy route** (`/plugins/agent_browser/panel/dash`, - HTTP + WebSocket), so the embed rides the fleet proxy (ADR 0042) on the host and - on a member alike — it never points the operator's browser at `localhost:PORT` - (issue #6). `minimal` mode (`panel_mode: minimal`) renders a viewport-only page - (live screenshot + nav toolbar) with no WS dependency. +- **Browser panel** — a console view (ADR 0026) for watching/driving the browser. Two + modes, set by `panel_mode`: + - **`minimal` (default)** — a **live screenshot** of the viewport + a nav toolbar + a + **Dashboard control** (start/stop/status), all through the **gated same-origin routes**. + Works everywhere (host and member), no dashboard daemon needed. The reliable mode. + - **`full`** — a **launcher** for agent-browser's own dashboard (viewport + activity/ + console/network feeds). It's not embedded: that dashboard is a Next.js app with + **root-absolute asset paths** (no base-path), so it can't render under a sub-path + panel — it only loads at its **own origin**. Full mode opens it there + ("Open dashboard ↗"), which works on a local/host setup. For a remote member, use + `minimal`. + + Either mode can **start the dashboard from the panel** (no terminal) — the Start/Stop + control hits the gated `POST /api/plugins/agent_browser/dashboard`. ## Requirements @@ -77,7 +82,7 @@ agent_browser: | File | What | |---|---| | `tools.py` | the browser tools — subprocess wrappers over the `agent-browser` CLI | -| `browser_panel.py` | the console view that embeds the agent-browser dashboard (same-origin reverse proxy) | +| `browser_panel.py` | the Browser panel — `minimal` (live viewport + nav + dashboard control) and `full` (a launcher for the dashboard) | | `lifecycle.py` | the dashboard daemon surface — start on boot, stop on shutdown (ADR 0018) | | `skills/` | the discovery skill (defers to `agent-browser skills get core`) | | `workflows/` | declarative browser recipes | diff --git a/browser_panel.py b/browser_panel.py index d799010..a22ce94 100644 --- a/browser_panel.py +++ b/browser_panel.py @@ -1,22 +1,21 @@ """Browser panel console view (ADR 0026) — two modes, set by ``panel_mode``. -- ``full`` (default): iframes agent-browser's own live dashboard (viewport + - activity/console/network/… feeds), reusing their renderer wholesale. The - dashboard is served **through this plugin's router** (a same-origin reverse - proxy under ``/plugins/agent_browser/panel/dash``) so the iframe rides the - fleet proxy (ADR 0042) on the host AND on a member — never a hardcoded - ``http://localhost:PORT`` (issue #6: that URL resolved against the operator's - own machine, not the member box → "refused to connect"). -- ``minimal``: a viewport-only page we render ourselves — a live screenshot (polled - from the CLI, same-origin, no WS-protocol dependency) plus a slim nav toolbar - (url / back / forward / reload). "Expose less": just the page, nothing else. - -Self-contained vanilla JS (no build step). Every fetch / iframe-src / asset is -derived from ``base = location.pathname.split("/plugins/")[0]`` (="" on the host, -``/agents/`` when proxied) so the page is same-origin + slug-aware — the rule -that keeps the postMessage token/theme handshake working through the fleet proxy. -The page reads/drives the browser only through the CLI behind same-origin routes; -the agent's own browser_* tools are the primary driver — this is an operator viewport. +- ``minimal`` (default): a viewport-only page we render ourselves — a live screenshot + (polled from the CLI through the gated /shot route, same-origin, no WS/proxy + dependency) + a nav toolbar (url / back / forward / reload) + a Dashboard control + (start/stop/status). Works everywhere — host and member alike. +- ``full``: a launcher for agent-browser's own dashboard (viewport + activity/console/ + network feeds). We deliberately do NOT embed it: that dashboard is a prebuilt Next.js + app whose assets are ROOT-ABSOLUTE (``/_next/…``) with no base-path option, so it can't + render under a sub-path reverse proxy (the long-standing blank panel) — it only renders + at its OWN origin. Full mode opens it there ("Open dashboard ↗"), which works on a + local/host setup; for a remote member, use minimal. + +Self-contained vanilla JS (no build step). Every fetch / link is derived from +``base = location.pathname.split("/plugins/")[0]`` (="" on the host, ``/agents/`` +when proxied) so the page is same-origin + slug-aware (the token/theme handshake). The +browser is driven only through the CLI behind same-origin routes; the agent's own +browser_* tools are the primary driver — this is an operator viewport. """ from __future__ import annotations @@ -28,20 +27,10 @@ import tempfile import time -# Imported at MODULE scope (not inside the factory) so FastAPI can resolve the -# string annotations these routes carry — with `from __future__ import annotations` -# every annotation is a string and FastAPI's get_type_hints() looks them up in the -# module globals, so `request: Request` / `websocket: WebSocket` must live here. -# Safe: the host always provides fastapi, and __init__.py catches ImportError -# (tools still serve if the panel can't import). -from fastapi import APIRouter, Body, Request, WebSocket -from fastapi.responses import ( - FileResponse, - HTMLResponse, - JSONResponse, - Response, - StreamingResponse, -) +# Module-scope fastapi imports (the host always provides fastapi; __init__.py catches +# ImportError so the tools still serve if the panel can't import). +from fastapi import APIRouter, Body +from fastapi.responses import FileResponse, HTMLResponse, JSONResponse, Response log = logging.getLogger("protoagent.plugins.agent_browser") @@ -53,132 +42,14 @@ def build_panel_router(cfg: dict | None): cfg = cfg or {} port = int(cfg.get("dashboard_port", 4848)) - mode = str(cfg.get("panel_mode", "full")).strip().lower() - # NB: the page + dash-proxy routes don't shell out (binary/timeout) — they only - # reverse-proxy the dashboard daemon on `port`. The shot/nav DATA routes that DO - # run the CLI live in build_panel_data_router (gated under /api). + mode = str(cfg.get("panel_mode", "minimal")).strip().lower() router = APIRouter() @router.get("/panel") async def _panel(): - return HTMLResponse(_MINIMAL_PAGE if mode == "minimal" else _FULL_PAGE) - - # The minimal-mode shot/nav DATA routes moved to build_panel_data_router — - # gated under /api/plugins/agent_browser (plugin-view rule 2). The /panel/dash - # reverse proxy below stays on the public prefix OF NECESSITY: it's loaded by - # an + +
Browser
+
+
agent-browser dashboard
+
The full dashboard — live viewport plus activity, console, and network + feeds — runs in its own window. It can't embed in this panel (its assets load from the + page root, which a sub-path panel can't serve), so it opens in a new tab.
+ Open dashboard ↗ +
Want the browser inside the console? Set + panel_mode: minimal — a live viewport you can drive right here.
+
""" @@ -308,6 +228,9 @@ async def _nav(body: dict = Body(...)): overflow:auto;background:var(--pl-color-bg-inset)} img{max-width:100%;display:block} .stage .pl-empty{margin:auto} + .dash{display:flex;align-items:center;gap:4px;font-size:11px;color:var(--pl-color-fg-muted);margin-left:4px} + .dot{width:7px;height:7px;border-radius:50%;display:inline-block;flex:none} + .dot.ok{background:#22c55e}.dot.off{background:var(--pl-color-fg-muted,#9aa0aa)}
@@ -315,6 +238,7 @@ async def _nav(body: dict = Body(...)): +
@@ -356,11 +280,36 @@ async def _nav(body: dict = Body(...)): }catch(_){} busy=false; } +// Dashboard control — start/stop the agent-browser dashboard daemon from the UI (no +// terminal), show its status, and link out to it. The rich dashboard can't be EMBEDDED +// through our sub-path proxy (its Next.js assets are root-absolute), so "Dashboard ↗" +// opens it at its own origin in a new tab — which works on a local/host setup. +var dashPort=__DASH_PORT__; +function dashOpenUrl(){ return "http://"+location.hostname+":"+dashPort+"/"; } +async function dashStatus(){ + try{ var r=await kit.apiFetch("/api/plugins/agent_browser/dashboard"); var d=await r.json(); + dashPort=d.port||dashPort; renderDash(!!d.running); }catch(_){ renderDash(null); } +} +function renderDash(running){ + var el=$("dash"); if(running===null){ el.innerHTML=""; return; } + var open='Dashboard ↗'; + el.innerHTML = running + ? ''+open+'' + : ''; +} +async function dashAct(action){ + var el=$("dash"); el.innerHTML='…'; + try{ await kit.apiFetch("/api/plugins/agent_browser/dashboard",{method:"POST", + headers:{"Content-Type":"application/json"},body:JSON.stringify({action})}); }catch(_){} + setTimeout(dashStatus,800); +} +window.dashAct=dashAct; + // Boot ONCE, on whichever fires first: the handshake (the bearer arrives with // protoagent:init, so the gated shot/nav calls authenticate) or a short timer // for the no-handshake case (standalone page / older host). let booted=false; -function boot(){ if(booted)return; booted=true; refresh(); setInterval(refresh,1200); } +function boot(){ if(booted)return; booted=true; refresh(); setInterval(refresh,1200); dashStatus(); setInterval(dashStatus,6000); } kit.initPluginView(boot); setTimeout(boot, 800); """ diff --git a/protoagent.plugin.yaml b/protoagent.plugin.yaml index 1955ae7..aa64ff6 100644 --- a/protoagent.plugin.yaml +++ b/protoagent.plugin.yaml @@ -1,6 +1,6 @@ id: agent_browser name: Agent Browser -version: 0.3.0 +version: 0.4.0 description: >- Browser automation for protoAgent, backed by **agent-browser** (vercel-labs) — a fast native-Rust CLI/daemon that drives Chrome over CDP with accessibility-tree @@ -23,8 +23,14 @@ config: binary: agent-browser # the agent-browser CLI on PATH (override for a pinned path) dashboard_port: 4848 # agent-browser dashboard start --port; full-mode panel iframes this timeout_s: 60 # per-command subprocess timeout - panel_mode: full # Browser panel layout: `full` (iframe the dashboard — viewport + - # feeds) or `minimal` (viewport-only: live screenshot + a nav toolbar) + panel_mode: minimal # Browser panel layout. `minimal` (default): viewport-only — a live + # screenshot + a nav toolbar + a Dashboard control, driven through the + # gated same-origin routes. Works everywhere (host + member), no daemon + # needed. `full`: iframe agent-browser's own dashboard (viewport + feeds) + # — only renders when the dashboard is reachable at its OWN origin from + # your browser (a local/host setup); it can't be embedded through a + # sub-path reverse proxy because its Next.js assets are root-absolute. + # Use the panel's "Open ↗" to view the dashboard directly. # Runtime / launch options — a curated set passed to `agent-browser open` so you can # shape and lock down the browser the agent spins up. Blank/0/false = CLI default. headed: false # show a real browser window instead of headless @@ -38,7 +44,7 @@ config: # Editable in Settings ▸ Plugins (ADR 0019) — the operator knobs above as UI fields. settings: - - { key: panel_mode, label: "Browser panel mode", type: select, options: [full, minimal], description: "full = iframe agent-browser's live dashboard (viewport + activity/console/network feeds); minimal = viewport-only (a polled screenshot + a nav toolbar)." } + - { key: panel_mode, label: "Browser panel mode", type: select, options: [minimal, full], description: "minimal (recommended): a live screenshot + nav toolbar + Dashboard control, via gated same-origin routes — works everywhere. full: iframe agent-browser's dashboard (viewport + feeds) — only renders when the dashboard is reachable at its own origin from your browser (local/host); it can't embed through a sub-path proxy. Use the panel's Open ↗ either way." } - { key: headed, label: "Headed browser", type: bool, description: "Show a real browser window instead of running headless." } - { key: allowed_domains, label: "Allowed domains", type: string, description: "Comma-separated navigable-domain allowlist (e.g. example.com,*.foo.com). Blank = unrestricted." } - { key: confirm_actions, label: "Confirm actions", type: string, description: "Comma-separated action categories that require confirmation before the agent runs them." } diff --git a/pyproject.toml b/pyproject.toml index dd1cb6d..d2f5ecc 100644 --- a/pyproject.toml +++ b/pyproject.toml @@ -1,6 +1,6 @@ [project] name = "agent-browser-plugin" -version = "0.3.0" +version = "0.4.0" description = "Browser-automation plugin for protoAgent, backed by vercel-labs/agent-browser (tools + skill + workflows + a live browser panel)." requires-python = ">=3.11" diff --git a/tests/test_agent_browser.py b/tests/test_agent_browser.py index 5fb807e..bfd4ecb 100644 --- a/tests/test_agent_browser.py +++ b/tests/test_agent_browser.py @@ -177,27 +177,62 @@ def _app(cfg=None): return app -def test_panel_page_full_mode_is_four_rules_compliant(): +def test_panel_page_full_mode_is_a_launcher(): from fastapi.testclient import TestClient - c = TestClient(_app({"panel_mode": "full"})) + c = TestClient(_app({"panel_mode": "full", "dashboard_port": 4955})) r = c.get("/plugins/agent_browser/panel") assert r.status_code == 200 html = r.text assert "/_ds/plugin-kit.css" in html # DS kit assert 'location.pathname.split("/plugins/")[0]' in html # slug-aware base - assert "/plugins/agent_browser/panel/dash/" in html # same-origin proxied dashboard - # never a hardcoded http origin (issue #6) — the iframe src is BASE-derived same-origin. + assert "Open dashboard" in html # a launcher to the dashboard's own origin (not an embed) + assert "/api/plugins/agent_browser/dashboard" in html # the start/stop control + assert "panel_mode: minimal" in html # points users at the mode that embeds inline + # full mode no longer embeds a sub-path proxy; the port placeholder is interpolated. + assert "/panel/dash" not in html + assert "__DASH_PORT__" not in html and "4955" in html + # the "open" link is built from location.hostname, never a hardcoded origin. assert "http://localhost" not in html and "http://127.0.0.1" not in html def test_panel_page_minimal_mode_uses_gated_data_routes(): from fastapi.testclient import TestClient - html = TestClient(_app({"panel_mode": "minimal"})).get("/plugins/agent_browser/panel").text + html = TestClient(_app({"panel_mode": "minimal", "dashboard_port": 4933})).get( + "/plugins/agent_browser/panel" + ).text assert "/api/plugins/agent_browser/shot" in html # gated screenshot assert "/api/plugins/agent_browser/nav" in html # gated nav assert "kit.apiFetch" in html # authed fetch via the kit + # the dashboard control (start it from the UI) + the port placeholder is interpolated + assert "/api/plugins/agent_browser/dashboard" in html and "Start dashboard" in html + assert "__DASH_PORT__" not in html and "4933" in html + + +def test_default_panel_mode_is_minimal(): + import yaml + + m = yaml.safe_load((ROOT / "protoagent.plugin.yaml").read_text()) + assert m["config"]["panel_mode"] == "minimal" # the reliable mode is the default now + by_key = {f["key"]: f for f in m["settings"]} + assert by_key["panel_mode"]["options"][0] == "minimal" # recommended first + + +def test_dashboard_control_endpoint(monkeypatch): + from fastapi.testclient import TestClient + + c = TestClient(_app({"dashboard_port": 4934})) # nothing listens → status "stopped" + st = c.get("/api/plugins/agent_browser/dashboard").json() + assert st["running"] is False and st["port"] == 4934 + # start/stop run the CLI (mocked); bad action is rejected. + rec = [] + monkeypatch.setattr(bp.subprocess, "run", fake_run(record=rec)) + assert c.post("/api/plugins/agent_browser/dashboard", json={"action": "x"}).json()["ok"] is False + assert c.post("/api/plugins/agent_browser/dashboard", json={"action": "start"}).json()["ok"] is True + assert ["agent-browser", "dashboard", "start", "--port", "4934"] in rec + assert c.post("/api/plugins/agent_browser/dashboard", json={"action": "stop"}).json()["ok"] is True + assert ["agent-browser", "dashboard", "stop"] in rec def test_shot_route_503s_without_a_frame(monkeypatch): @@ -220,9 +255,3 @@ def test_nav_route_validates(monkeypatch): assert c.post("/api/plugins/agent_browser/nav", json={"action": "reload"}).json()["ok"] is True -def test_dash_proxy_502s_when_daemon_down(): - from fastapi.testclient import TestClient - - # nothing listens on the dashboard port in CI → the reverse proxy returns 502, not a crash. - r = TestClient(_app({"dashboard_port": 4999})).get("/plugins/agent_browser/panel/dash/") - assert r.status_code == 502