Electron shell: host the browser panel as a real view, not a video stream - #47
Conversation
…ream The BROWSER tab works, but it never stopped feeling remote — because it is. Tauri hands us the OS webview (WKWebView on macOS, WebView2 on Windows), which cannot host a second browser, so the panel is a CDP screencast: Chrome paints, core re-encodes to JPEG, the client draws it to a canvas and maps clicks back through the letterbox. Every complaint traced to that pipeline rather than to a tunable in it. Raising the cast width and quality fixed the blur twice and it came back both times; the click and typing regressions were coordinate-mapping bugs that only exist because coordinates need mapping at all. Electron bundles Chromium, so the panel can be a `WebContentsView` the window composites directly: real pixels, real input, real scrolling, no encode step. This adds that shell **alongside** Tauri rather than replacing it. `src-tauri/` and `pnpm desktop:build` are untouched and still the shipping path; retiring either is a later decision, and the streaming pipeline stays until then because the Tauri build is its only consumer. What is here: - `electron/main.ts` spawns or attaches to core, opens the window on the port core serves, and hands the panel's CDP target id to `browser_attach` so the agent's `browser_*` tools drive the same view the user sees. One browser, two drivers — not one each. - `electron/browserPanel.ts` creates the view on a persistent partition, hides rather than destroys it (a destroyed view loses the page, which is what made tab toggles look like a refresh), and captures downloads to `~/.cali/downloads`. - `electron/preload.ts` exposes a narrow `window.cali` over `contextBridge` under `sandbox: true` — no generic invoke, one named method per operation. It is bundled with esbuild because a sandboxed preload cannot `require` a sibling module, and the failure is silent. - `BrowserTab.tsx` detects the bridge and, under Electron, skips the cast entirely: it becomes a geometry reporter that tells the shell where to put the view, and hides it while a popover or dialog is open so native pixels never paint over the editor's own overlays. `scripts/compare-shells.mjs` drives both shells over CDP and diffs the editor region, which is how the UI was checked rather than asserted: the two agree inside the antialiasing floor measured at 2.6-3.0k pixels, so the threshold sits at 6k. Nothing in the design system changed — the tokens, the no-focus-ring rule and the drag regions all render identically, which was the thing most at risk in a shell swap. Scoped to the client on purpose. The core-side half (`browser_attach`, `browser_downloads`, the attach-aware `close`) is held back because those hunks live in `rpc.rs`, `tools.rs` and `agent.rs`, which are currently ~95% another session's in-flight work; shipping them here would drag three unfinished modules into this diff. Until that lands, the Electron shell runs against a core without `browser_attach` and the panel simply does not receive agent input — the editor, the viewport and every other tab are fully functional. Not done: signing and notarisation (entitlements are written, the identity is not), and no LLM turn has yet been observed choosing these tools against the native panel. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01DAV2UBU1PQ8mJFVWMhKyHx
|
The red E2E check is pre-existing on
This branch touches no attachment-routing code; the client diff is 174 lines across Worth fixing, but it belongs in its own change rather than being folded into this one. |
…hrome The client half of this branch made the BROWSER tab a real `WebContentsView`, but core knew nothing about it. `Browsers::ensure` launches a Chrome on first use, so the agent's `browser_*` tools drove an invisible browser of their own while the user looked at a different one. Two browsers, and the agent's work never appeared in the tab — which is worse than the screencast it replaced, because at least the screencast was the same page. `browser_attach` closes that gap. The shell reads its panel's devtools target id and hands it to core, which adopts that target rather than starting anything. One browser, two drivers. The target id is passed, never discovered. Guessing by url or title would eventually match the editor's own window, and core would start driving the app instead of the page inside it. Ownership is now explicit: `child` and `profile` are `Option`, `None` meaning attached. `close` returns early in that case rather than sending `Browser.close`, which would quit the application hosting the view — the editor the user is working in. `live_attach_drives_a_browser_it_did_not_launch` asserts both halves against a real Chrome, and reads the resulting url back over its *own* devtools connection so it cannot pass on core merely believing it navigated. Removing the ownership guard fails it on the second assertion, which is the one with teeth. Also lands the three browser tools the client half already assumes: `browser_mouse_move` (a delta, because a camera turns by motion rather than destination), `browser_play` (keys held *while* looking — the other two are strictly sequential, so strafing or aiming while advancing was impossible), and `browser_downloads`. The registered-surface invariant moves 12 -> 15 and AGENTS.md documents all three, which is what that assertion exists to force. Scoped to the browser subsystem: `rpc.rs` and `tools.rs` get only their browser hunks, and the `spawn_ledger` registration another session added to `browser.rs` is left out because that module is not on main yet. It should be restored when their branch lands. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01DAV2UBU1PQ8mJFVWMhKyHx
|
Follow-up: the core half is now in, because without it the branch was still running a second browser. Caught in review: the client commit alone made the panel native, but core knew nothing about it.
Proven rather than asserted:
|
…on our line
Tauri's webview is a different engine per platform and none of them can host a
second browser, so the BROWSER tab could only ever be a video stream of a Chrome
running somewhere else. That was the reason for the Electron shell, and keeping
both meant every fix had to be made twice into a shell that could not have the
feature. So `client/src-tauri/` is gone, along with `@tauri-apps/*`, the `tauri`
script, and `compare-shells.mjs`, which needed two shells to compare.
Three things the removal had to carry rather than drop:
- **The folder picker.** `chooseNativeWorkspace` imported
`@tauri-apps/plugin-dialog`. It goes through the shell's `chooseFolder` now,
and `defaultPath` is threaded through the bridge because one caller passes the
folder it expects and silently losing that is a regression nobody would notice
until the panel opened in the wrong place.
- **The icons.** `icon.{icns,ico,png}` and the source SVG move to
`build-electron/`, electron-builder's `buildResources`. Same artwork and the
same `com.calicode.desktop` identifier, so this replaces the installed app
rather than sitting beside it with a second `~/.cali`.
- **`scripts/desktop.sh`.** Rewritten, not deleted: it still owns
`build|dev|install`, and it still builds with a *stable* local signing
identity, because an ad-hoc signature is keyed to the binary hash and macOS
drops the app's Desktop/Documents grants on every rebuild. Signing moved from a
post-hoc `codesign --deep` to electron-builder's own — an Electron bundle's
helpers and framework must be signed inside-out, and `--deep` does it in the
wrong order.
**The traffic lights.** `trafficLightPosition` was copied from `tauri.conf.json`,
and the two shells do not measure it the same way: `y: 23` sat the lights ~10pt
below the sidebar's window-controls row, on a line of their own. Measured against
a real window, Electron places the group ~6.75pt above the buttons' visual
centre, so `y: 13` centres them at 20pt — the same line as the `h-10` row. On
screen the lights, the sidebar toggle, the history arrows and the tab strip now
share one centre to within 0.5pt, which is glyph shape rather than layout.
**The name.** `app.setName` gives the userData directory and `app.getName()`, and
the packaged bundle reports `CFBundleName`/`CFBundleDisplayName` of CaliCode with
the brand icon. It does *not* rename the menu bar in an unpackaged run — that
title comes from the running bundle's plist, which is Electron's own — so
`pnpm desktop:electron` still says "Electron". Documented rather than chased.
`SettingsPage.test.tsx` faked the desktop shell by defining `__TAURI_INTERNALS__`.
That global no longer exists, so the test was changed to install the Electron
bridge instead — a test edit, said out loud: it asserts the same thing about the
same titlebar, only through the detection that is actually left.
Verified by driving both builds and measuring the pixels: unpackaged and packaged
windows agree on the alignment, the packaged app carries the icon and the name,
and core drives the panel with no Chrome of its own — proven by pointing the
panel at a distinct url over CDP and watching `browser_status` report it back.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01DAV2UBU1PQ8mJFVWMhKyHx
|
Tauri is gone; Electron is the only shell ( Keeping both meant every fix had to be made twice, into a shell that structurally could not have the browser panel. Removed: Three things the removal carried rather than dropped:
Traffic lights. Name. Packaged bundle reports One test changed, said out loud: Verified by driving both builds and measuring pixels rather than asserting: unpackaged and packaged agree on alignment, the packaged app carries icon and name, and core drives the panel with no Chrome of its own (proven by pointing the panel at a distinct url over CDP and watching Client and Rust core pass. E2E is still the pre-existing |
main retired Tauri (#47) while this branch was building starters, memory, hooks and file-defined commands. Eleven files conflicted and the two sides were newer in opposite places, so the resolution is not "take one side". main wins on the shell, because retiring Tauri is the deliberate newer decision: `client/electron/*` (which both sides had added independently — main's carries `app.setName`, the brand icon, and a `trafficLightPosition` measured against a real window rather than inherited from tauri.conf.json), `desktop.ts`, `workspace.ts` (the `@tauri-apps/plugin-dialog` path is gone, `chooseFolder` takes `defaultPath`), `package.json`, `index.css` (`data-tauri-drag-region` becomes `data-drag-region`), and the plan doc. This branch wins on the agent browser, because its `browser_play` is the newer one: `recordFrames` and `act_while_held` capture frames *during* the hold, and `nativeVirtualKeyCode` is deliberately absent — it was being handed the Windows VK code, which on macOS is a different keycode space, so every held key arrived as Numpad5 and auto-repeated ~20k times in 600ms. Kept from both: main's `browser_attach` and its `live_attach` test, which is the Electron panel's whole contract, alongside this branch's frame recording and keycode assertions. AGENTS.md took main's shell narrative and the `browser_attach` bullet, minus the older duplicate "Playing a game is a different verb" bullet it would have reintroduced over this branch's newer one. Verified on the merge: cargo fmt/clippy clean, 1075 Rust tests passed, tsc clean, 941 client tests passed. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_013CUFpThK67PNcJz16UvH8C
Why
The BROWSER tab works, but it never stopped feeling remote — because it is.
Tauri hands us the OS webview (WKWebView on macOS, WebView2 on Windows), which cannot host a second browser. So the panel is a CDP screencast: Chrome paints, core re-encodes to JPEG, the client draws it to a canvas and maps clicks back through the letterbox. Every complaint about it traced to that pipeline existing rather than to a tunable inside it:
Electron bundles Chromium, so the panel can be a
WebContentsViewthe window composites directly. Real pixels, real input, real scrolling, no encode step.What this does not do
It adds the Electron shell alongside Tauri.
src-tauri/andpnpm desktop:buildare untouched and still the shipping path. Retiring either is a later decision, and the streaming pipeline stays until then because the Tauri build is its only consumer.What's here
electron/main.ts— spawns or attaches to core, opens the window on the port core serves, and hands the panel's CDP target id tobrowser_attach, so the agent'sbrowser_*tools drive the same view the user sees. One browser with two drivers, not one each.electron/browserPanel.ts— creates the view on a persistent partition, hides rather than destroys it (a destroyed view loses the page — that was the "refresh on tab toggle"), and captures downloads to~/.cali/downloads.electron/preload.ts— a narrowwindow.caliovercontextBridgeundersandbox: true. No generic invoke; one named method per operation. Bundled with esbuild because a sandboxed preload cannotrequirea sibling module and the failure is silent (window.cali === undefinedat render time, no error anywhere).BrowserTab.tsx— detects the bridge and, under Electron, skips the cast entirely. It becomes a geometry reporter that tells the shell where to put the view, and hides the view while a popover or dialog is open so native pixels never paint over the editor's own overlays.How the UI was checked
scripts/compare-shells.mjsdrives both shells over CDP and diffs the editor region — the concern here was a shell swap quietly wrecking the look, so it's measured rather than asserted.The two agree inside the antialiasing floor, measured at 2,666–3,023 px; the threshold sits at 6,000. Tokens, the no-focus-ring rule, and the drag regions all render identically.
node scripts/compare-shells.mjs # from client/Verification
npx tsc -b --noEmit— cleantsc -p tsconfig.electron.json --noEmit— cleanpnpm test— 904 passed / 74 files, green on two consecutive runs (one earlier run had a single timing flake in an unrelated retry test, which did not reproduce)pnpm desktop:electron:build— packages unsigned torelease-electron/Scope
Two commits: the client shell, then the core half that makes it mean anything.
The core commit is scoped to the browser subsystem —
rpc.rsandtools.rstakeonly their browser hunks, so another session's in-flight
computer.rs,guardian.rsandspawn_ledger.rswork stays out. The one casualty: thespawn_ledgerregistration they added tobrowser.rsis omitted, because thatmodule is not on main yet. It should be restored when their branch lands.
Not done
Plan and status:
docs/plans/electron-shell.md.🤖 Generated with Claude Code
https://claude.ai/code/session_01DAV2UBU1PQ8mJFVWMhKyHx