feat(panel): fully interactive CDP-screencast viewport — full switchover (v0.6.0)#14
Merged
Conversation
…ver (v0.6.0) Replace the entire dashboard-embed / screenshot panel approach with a single fully drivable browser viewport. A live CDP screencast (event-driven JPEG frames) is painted on a <canvas> and operator mouse/keyboard/scroll are forwarded back via Input.dispatch*, all over a gated same-origin WebSocket — so it works on the host AND a remote fleet member, and you can drive the page alongside the agent. New: - browser_stream.py — a second-CDP-client bridge (Page.startScreencast out, input in) + single-use ticket auth. resolve_page_target/pick_page_target/input_to_cdp are pure + host-free-tested. - browser_panel.py — the interactive canvas page + gated POST /stream-ticket and the self-gated WS /stream. WS auth: the host's operator-bearer middleware is HTTP-only and skips WS handshakes, so the panel mints a ticket over the gated HTTP route and presents it on the WS URL; the handler validates + burns it. Full switchover (no backward compatibility): - Removed panel_mode (interactive is the only panel), the screenshot `minimal` mode and /shot, and the `full` dashboard-embed page. - Removed the browser_dashboard tool (16 tools now), lifecycle.py (dashboard daemon boot/shutdown surface), and the /dashboard control routes. - Removed config: panel_mode, dashboard_port, manage_dashboard. Validated end-to-end against the real agent-browser binary through the production routes (bad ticket refused, ticket minted, real frame streamed, keys + click land). 34 host-free tests pass; ruff clean. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
|
👀 Quinn is reviewing — verdict (PASS / WARN / FAIL) + findings to follow. |
![protoquinn[bot]](https://avatars.githubusercontent.com/in/3282126?s=60&v=4){kind=small}
There was a problem hiding this comment.
QA Audit — PR #14 | feat(panel): fully interactive CDP-screencast viewport — full switchover (v0.6.0)
VERDICT: WARN
CI Status
- test: queued ⏳
Diff Review
- Clean architectural switchover:
panel_modegone,lifecycle.pyremoved,browser_dashboardtool removed,dashboard_port/manage_dashboardconfig removed — consistent across__init__.py, README, CHANGELOG. - Single-use ticket pattern for WS auth mirrors the host's SSE-token approach — sound by design. Ticket is minted via bearer-gated
POST /stream-ticket, validated + burned on WS connect. - 34 host-free tests added, ruff clean per PR description.
Observations
- GAP: clawpatch structural review unavailable for this repo (HTTP 500) — cross-file patterns in
browser_stream.py+browser_panel.pynot structurally verified. - GAP: diff truncated at 200/1522 lines — the bulk of
browser_panel.pyrewrite and the newbrowser_stream.pyare not visible to this reviewer. Ticket validation logic, CDP input dispatch, and WS handler implementation could not be verified directly. - GAP:
path_existsnot available in tool schema — cannot confirmbrowser_stream.pyandtests/test_browser_stream.pyexist at the PR head. - LOW: No unresolved CodeRabbit threads — prior automated review is clean.
- LOW: CI still queued. Formal PASS/FAIL deferred until
testcompletes.
Summary: The architectural approach (single-use ticket, CDP screencast, Input.dispatch*) is sound and the removals are clean. However, the core implementation in browser_stream.py could not be reviewed due to tooling limitations. Recommend the author ensure the ticket-burn is atomic (validate-then-burn, not the reverse) and that CDP input dispatch sanitizes forwarded coordinates/keys. Re-review once CI is terminal-green.
— Quinn, QA Engineer
|
Submitted COMMENT review on #14. |
The empty viewport was a dead end. Now, when no page is open, the panel shows a Start button, and a new `home_url` config sets the page it opens to: - home_url set → the panel auto-opens it once; the button reads "Open <host>" - home_url blank → the button opens about:blank (no auto-open) home_url is injected as a JS string literal via json.dumps + `<`→< so a value containing `</script>` can't break out of the inline script. Editable in Settings ▸ Plugins. 35 tests pass; ruff clean. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
There was a problem hiding this comment.
QA Audit — PR #14 | feat(panel): fully interactive CDP-screencast viewport — full switchover (v0.6.0)
VERDICT: WARN (non-terminal — CI still queued, re-review after CI settles)
CI Status
- test: queued
Diff Review
- Clean architectural switchover: old dashboard-embed (
lifecycle.py,panel_mode,/shot,browser_dashboardtool) removed entirely — no backward-compat shims left cruft behind. - New
browser_stream.pybridges CDP screencast ↔ WS; ticket auth pattern (bearer-gatedPOST /stream-ticket→ single-use ticket → validated + burned on WS connect) is well-reasoned and mirrors existing SSE-token pattern. browser_panel.pyrewritten as interactive canvas + nav/stream routes;__init__.pysimplified (lifecycle surface removed, log line trimmed).- 34 tests + ruff clean per PR description; new
tests/test_browser_stream.pypresent.
Observations
- LOW: clawpatch structural review unavailable for this repo (502 — checkout cache couldn't resolve
mainref). Manual review of visible diff + PR description used instead. - LOW: Second CDP client attaches to Chrome for the screencast bridge — inherent to the architecture; no obvious alternative. Risk is bounded: CDP URL is gated behind
agent-browser get cdp-url. - NOTE: CI test suite still queued. Will re-assess on terminal completion. The PR body claims 34 host-free tests pass — awaiting CI confirmation.
— Quinn, QA Engineer
|
Submitted COMMENT review on #14. CI is still queued — I'll re-assess for terminal PASS/FAIL when the test suite completes and the re-dispatch fires. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
Replaces the entire dashboard-embed / screenshot panel approach with one fully drivable browser viewport. A live CDP screencast (event-driven JPEG frames, not a screenshot poll) is painted on a
<canvas>, and operator mouse / keyboard / scroll are forwarded back into the page viaInput.dispatch*— all over a gated same-origin WebSocket. You can click, type, and scroll the real browser from the console, alongside the agent, and it works on the host and a remote fleet member.How
agent-browser get cdp-url).browser_stream.pybridgesPage.startScreencast⇆ the panel; the nav toolbar reuses the gated HTTP/navroute.POST /stream-ticket(bearer-checked) and presents it on the WS URL; the handler validates + burns it. Mirrors the host's own SSE-token pattern. Safe when gated, transparent when open.Full switchover (no backward compatibility)
panel_modegone — interactive is the only panel. Removed the screenshotminimalmode +/shot, and thefulldashboard-embed page.browser_dashboardtool (16 tools now),lifecycle.py(the dashboard daemon boot/shutdown surface), and the panel's/dashboardcontrol routes.panel_mode,dashboard_port,manage_dashboard. The panel talks CDP directly and never needs agent-browser's separate dashboard daemon.Validation
Exercised the production routes against the real
agent-browserbinary + live browser: bad ticket refused → ticket minted → real JPEG frame + metadata streamed over the WS → typed text landed in a real input → a mapped click fired the button. 34 host-free tests pass, ruff clean (CDP parsing, page-target selection, full input→CDP translation, ticket single-use/expiry, the panel page, WS reject/accept).Files
browser_stream.py(CDP bridge + ticket auth),tests/test_browser_stream.pybrowser_panel.py(interactive page + WS route),tools.py,__init__.py, manifest/pyproject/README/CHANGELOGlifecycle.py🤖 Generated with Claude Code