Merged
Conversation
Electron main process now probes if the preferred CDP port is available before starting. If busy (zombie process, etc.), it auto-finds a free port in the 9335-9399 range. Also added timeouts to lsof/kill calls to prevent hangs on zombie processes.
- Replaced hardcoded alice/bob/narrator palette with rotating 8-color auto-palette (coral, sky, lime, violet, amber, teal, rose, indigo) - First actor gets classic white cursor, subsequent actors get colored - Cursors start hidden, appear on first moveCursor call - Non-default cursors show a colored label with the actor name - Each actor gets cursorId from pane title (e.g. 'boss', 'worker') - Cursor overlay injected once for all actors (lazy creation) - Cursors fully cleaned up on scenario switch
Stop button: - Added _aborted flag to Executor, checked between steps in runTo() - Wrapped runAll loop in try/catch to handle abort cleanly - Cancel now sends 'cancelled' to client on abort Cursor improvements: - Removed cursor labels (colors-only differentiation) - First cursor appearance teleports without transition (no corner slide) - Cursors start hidden until actor's first interaction UI: - Cache button text: 'Clear cache (2.5 MB)' format - CDP port auto-fallback when default 9334 is busy
jabterm v0.1.4 adds: - accessibilitySupport prop (populates .xterm-rows text natively) - Resize deduplication (skips WS resize when cols/rows unchanged) Cleanup: - Removed data-b2v-output polling hack from TerminalPane - Removed jabtermRef (no longer need readAll() capture buffer) - Removed data-b2v-output fallback from waitForText/waitForPrompt - Added accessibilitySupport="on" to JabTerm component
TUI timeout fix: - xterm v6 puts accessible text in .xterm-accessibility-tree, not .xterm-rows - Updated waitForText, waitForPrompt, isBusy, and grid prompt wait to check .xterm-accessibility-tree first, .xterm-rows as fallback Cursor positioning: - Added _cursorInitialized flag to Actor class - First cursor movement teleports to target instead of windMouse from (0,0) - Applied to both moveCursorTo() and private moveTo() (used by click/hover/type) - Prevents cursors appearing at edge of player window
Chat scenario: - Split intro into 'Introduction' (narration) + 'Meet the actors' (circleAround) - Alice and Bob type concurrently via Promise.all - Added Web Audio API notification beep when Bob sees Alice's message - Removed emoji chars from typed messages (caused rendering artifacts) E2E tests: - Added XTERM_TEXT_SELECTOR constant for xterm v6 compatibility - Updated all .xterm-rows selectors to check .xterm-accessibility-tree first CSS buttons scenario: works with xterm-accessibility-tree fix in terminal-actor
All actors share one page.keyboard, so Promise.all interleaves chars. Fix: Bob types a slow-output bash script first (sequential keyboard), then Alice types while Bob's script runs in the background producing progressive 'compiling module N...' output. Visually concurrent without keyboard conflict.
Command panes (type=terminal with cmd) show their own TUI, not a shell prompt. Previously the grid creation wait checked for $/#/% chars in xterm content — this would timeout for mc/htop. Now: command panes wait for any non-empty xterm content, shell panes (no cmd) still wait for prompt characters.
User reverted the split between command pane (mc, htop) and shell pane grid creation wait. All terminal panes now wait for prompt chars ($/#/%) regardless of command type. All E2E tests verified passing: - electron: all-in-one ✓ (17.8s) - electron: collab ✓ (18.3s) - electron: tui-terminals ✓ (1.4s)
New package: packages/browser2video-test/ - test.extend() with auto step-wrapping: each test(title) → beginStep(title) / endStep() - Fixtures: session (worker-scoped), actor, grid - Helpers: setActor(), setGrid(), getSession() Session API additions: - beginStep(caption) — emit stepStart - endStep() — emit stepEnd with breathing pause Sample test: tests/scenarios/notes-demo.b2v.test.ts
getSession() is now async and lazily creates the session on first call. This makes it safe to use from test.beforeAll (which doesn't have access to Playwright fixtures). Worker auto-fixture handles session cleanup. Fixed server null check in sample test.
New test visits alexn.pro portfolio, finds Three Charts project, navigates to the demo page (holiber.github.io/three-charts/demo/), and interacts with the chart: switches line/bars view, changes timeframes (5m, 30m, 1h), and toggles trend overlays. Deleted: github-mobile.scenario.ts, github-mobile.test.ts Added: external-website.scenario.ts, external-website.test.ts
collab and all-in-one tests require Electron CDP for createTerminalGrid(). Marked as test.skip in headless Playwright runner. They pass via apps/player E2E tests. Full test results: - 7 passed, 4 skipped, 0 failed (headless) - 3 passed (Electron E2E: all-in-one, collab, tui-terminals)
The cursor overlay is only visible after its first moveCursorTo() call. scroll() alone doesn't trigger cursor visibility. Added explicit moveCursorTo() before scrolls and after page transitions so the cursor is visible throughout the scenario.
createGrid was waiting for shell prompt chars ($/#/%) in ALL terminal panes, including mc and htop which are TUI apps that never show prompts. Now: command panes (pc.cmd set) wait for any non-empty xterm content, shell panes still wait for prompts. TUI E2E test now passes in 18.8s instead of hanging.
30s was too tight after running heavy scenarios (collab, tui). All 7 scenario tests pass in Electron E2E.
startTerminalWsServer now accepts optional cwd param. Session passes this.artifactDir so terminal shells start in the scenario's output directory instead of process.cwd().
…tion cursor The cursor overlay was only injected via page.evaluate() which gets wiped on navigation. The framenavigated listener tried to re-inject but raced with page load (catch swallowed errors). Now CURSOR_OVERLAY_SCRIPT is registered as addInitScript so it persists across all navigations automatically. framenavigated listener kept as belt-and-suspenders fallback.
CURSOR_OVERLAY_SCRIPT now guards all document.body/head operations: - getCursorEl() returns null if body not ready - Ripple container lazy-created via ensureRippleContainer() - Animation style deferred to DOMContentLoaded if head not ready - moveCursor/clickEffect gracefully no-op when body unavailable Eliminates 'Cannot read properties of null (appendChild)' errors when script runs as addInitScript before DOM is ready.
Added Session.abort() that immediately closes all browser pages and contexts, interrupting any running Playwright operations. Unlike finish(), it skips video composition entirely. Added Executor.abort() that calls session.abort() and resets state. Server cancel handler now uses executor.abort() instead of executor.reset() which previously tried to gracefully finish. The running step's Playwright operation (goto, waitForSelector, etc.) throws when the page is force-closed, which propagates up through runTo() → runAll catch handler → sends 'cancelled' to UI. Added E2E test: loads basic-ui, clicks Play All, waits for Stop button, clicks Stop, verifies Play All button returns.
…test - New InjectedActor class (packages/browser2video/injected-actor.ts) - Injects visible cursor overlay + typing into any page via page.evaluate() - Reuses CURSOR_OVERLAY_SCRIPT, WindMouse cursor paths, real mouse.click() - API: click, type, pressKey, waitFor, scroll, goto, breathe - Player self-test E2E (apps/player/tests/player-self-test.e2e.test.ts) - InjectedActor drives the player's own studio UI - Tests: cursor injection, + placeholder, Browser popup, URL dialog, iframe - Human-mode demo runner (apps/player/tests/human-mode-demo.ts) - Captures screenshots at each step for visual verification - Export InjectedActor from browser2video package
- New tests/scenarios/player-self-test.scenario.ts - Setup spawns inner Electron player on port 9581 - session.openPage() connects to inner player's web UI - InjectedActor drives the studio UI with visible cursor - 5 steps: verify ready, open picker, click +, Browser, URL confirm - Fixed electron binary resolution via createRequire from player pkg - All 5 steps pass in ~6.6s
- Phase 1: Split screen horizontally, add terminal
- Phase 2: Launch demo vite server in terminal, open todo app
- Phase 3: Todo CRUD — add 8 todos, reorder, scroll, delete
- Phase 4: Close terminal, verify todo app stops working
- Phase 5: Load basic-ui scenario, play/stop, step through slides
- Phase 6: Assert no unexpected console errors
Also adds data-testid attributes to controls (ctrl-play-all, ctrl-stop,
ctrl-next, ctrl-prev, ctrl-reset), scenario-picker (picker-select,
picker-switch), and step-graph (step-card-{i}).
Three fixes for running the self-test inside the Player's Executor:
1. Viewport tracking: add page.setViewportSize({width:1280,height:720})
after setup. Electron's WebContentsView starts at 0x0 and is later
resized via IPC, but Playwright's CDP-side viewport tracking keeps
the initial 0x0 value, causing all elements to be reported as
'outside of the viewport'.
2. Electron binary resolution: require('electron') returns the module
object (not the binary path) inside Electron's runtime. Fixed by
reading electron/path.txt or falling back to process.execPath.
3. Port conflict: changed inner player ports from 9581/9385 to
9591/9395 to avoid collision when the root player is running.
When the player-self-test scenario spawns an inner player, set B2V_EMBEDDED=1 so the inner player's Electron window is hidden. The inner player's UI is rendered inside the root player's scenario WebContentsView, so showing a second window was incorrect.
- Inner player window: off-screen (-10000), 1x1 size, show:false, skipTaskbar:true via B2V_EMBEDDED=1 env var. - New step 'Inner player shuts down cleanly': sends SIGTERM, waits for exit (10s timeout), verifies exit code, probes port 9591 is freed. - addCleanup is now a safety-net that only kills if process is still alive. - 16/16 steps pass both standalone and inside the Player app. All steps produce screenshots for the step graph.
When running in Electron mode (CDP endpoint), Playwright's recordVideo is unavailable because the session connects to existing pages rather than creating new browser contexts. Added CdpScreencastRecorder class that: 1. Starts Page.startScreencast via CDP to capture JPEG frames 2. Pipes frames to ffmpeg (image2pipe -vcodec mjpeg) → raw webm 3. Stops screencast and waits for ffmpeg to finish in finish() The raw webm is then processed by the existing composeVideos pipeline. Also fixed libx264 'height not divisible by 2' error by adding pad=ceil(iw/2)*2:ceil(ih/2)*2 to reencodeToMp4 and the fallback path. Verified: 5203 frames captured, 1.4MB MP4 output, 16/16 steps pass.
Single-file runner that: 1. Launches the Player Electron app 2. Connects via WS and loads the player-self-test scenario 3. Runs all 16 steps with progress reporting 4. Reports pass/fail, video path, and duration 5. Cleanly shuts down the Player Run: node --experimental-strip-types --no-warnings apps/player/tests/run-self-test.ts Removes the old test-player-ws.ts helper.
Electron embedded window hiding: - type: 'toolbar', focusable: false, hasShadow: false - mainWindow.hide() + minimize() after creation - Re-hide after each loadURL() call (macOS can show windows) Self-test: new 'Inner player window is hidden' step uses osascript to enumerate all Electron windows and flag any large windows near origin that could overlap the parent player. 17/17 steps pass, 123.5s, video produced.
- Disable electronView when B2V_EMBEDDED=1 to fix black background - Enable CDP screencasting in executor for embedded mode - Add __b2v_setCursorColor() for custom per-actor cursor colors - Add cursorColor option to InjectedActor (coral for self-test tester) - Add port cleanup before spawning inner player (kills stale processes) - Reduce post-waitForPort delay from 3s to 1s - Add 'Verify scenario screenshots are not blank' assertion step - Remove broken React CursorOverlay from screenshot mode
- Remove fixed 1s post-waitForPort delay - Switch from networkidle to domcontentloaded (faster) - Reduce studio-react retry attempts from 5 to 3 (with 15s timeout) - Add timing instrumentation to setup phases - Total time: 2.6m → 2.4m
- Add cursorColor to SessionOptionsSchema, Session, and Actor - Session reads from opts or B2V_CURSOR_COLOR env (format: fill,stroke) - Actor.injectCursor() applies color via __b2v_setCursorColor - Session adds cursor color init script for navigation persistence - Self-test: pink tester cursor, orange scenario Actor cursor - Both cursors visually distinct during self-test playback
- Increase cursor from 20x20 to 32x32px for visibility in screencast JPEGs - Add drop shadow filter for contrast against any background - Tester cursor: hot pink (#ff69b4) — unmistakably distinct - Scenario cursor: orange (#fb923c) via B2V_CURSOR_COLOR env
Root cause: addInitScript calls were placed AFTER page.goto() in
session.ts openPage(). Playwright addInitScript only fires on
*subsequent* navigations — so the cursor overlay script ran (via
framenavigated fallback), but the cursor color init script never
executed on the initial page load.
Diagnostic confirmed: inner player showed cursors=0 colors={} before
fix, and cursors=1 colors={default:{fill:#fb923c}} after fix.
Also removed temporary diagnostic code from executor.ts and
player-self-test.scenario.ts.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Test plan
pnpm -r buildpnpm b2v run --scenario basic-ui --mode human --record screencast --headed