Skip to content

Demo-session tooling for live user-testing sessions - #1834

Closed
michielbdejong wants to merge 4 commits into
developfrom
claude/demo-session-tooling
Closed

michielbdejong wants to merge 4 commits into
developfrom
claude/demo-session-tooling

Conversation

@michielbdejong

@michielbdejong michielbdejong commented Sep 25, 2026 •

Copy link
Copy Markdown
Contributor

What this adds

scripts/demo-session/ is a set of tools for running live user-testing sessions. One command starts an isolated atomic-server and data-browser for a branch or commit, then seeds demo data, and a developer-only interaction log lets someone watch what the tester does as they do it.

scripts/demo-session/demo-session.sh develop
# then open http://localhost:6757/app/dev-drive?demo-seed=calendar
  • Isolated stack. Each run uses its own worktree and a fresh data, config and cache dir under ~/.cache/atomic-demo/sessions/<stamp>-<branch>/. The server runs on :9893 and Vite on :6757; --port/--vite-port change them. A commit SHA gets a detached worktree. --build-only and --no-build are there too.

  • Builds only what's stale:

    • pnpm install
    • the workspace packages the data-browser imports from dist/
    • the wasm pair, cached by the git trees of wasm/ and lib/
    • cargo build, with SKIP_WASM_BUILD=1 and ATOMICSERVER_SKIP_JS_BUILD=true (the server embeds a placeholder page, because the app comes from Vite)
  • No signup. /app/dev-drive creates an agent and a drive in the browser. seed.js then builds the demo data through the app's own buildTableFromSpec/createPropertyOnClass, so nobody types a secret.

  • Interaction log. uxLogPlugin.ts is a Vite plugin with apply: 'serve', added only when VITE_UX_LOG=true by the config the script generates, so it can't end up in a production build. It appends events to ux.jsonl:

    • clicks, with text, label, role, test id and table row/column
    • committed input values, with password and secret fields redacted
    • routes, dialogs, toasts, console errors, failed or slow fetches, and unclean WebSocket closes
  • The calendar seed has recurring, moved, all-day, multi-day and cross-timezone events, plus a table with a date column and no calendar view yet.

  • Drive apps (--plugin NAME). Uses an atomic-plugins checkout (--plugins-dir, with browser/ linked).

    • It builds integrations/NAME/app and starts that repo's mock integration proxy on --mock-port, by default --port + 10000. The scenario is MOCK_SCENARIO=user-testing by default (ontola/atomic-plugins#195); --mock-scenario changes it.
    • It points the server (ATOMIC_INTEGRATION_PROXY_URL) and Vite (VITE_INTEGRATION_PROXY_URL) at that mock.
    • The issue-tracker seed installs the built app with createApp + handOverAppKey, the same path "New app" uses, and reads its source from /__demo-plugin/app.js.
    • The README records two limits. The desktop app's built-in browser pane blocks sandboxed iframes, so drive apps need Chrome. And connecting in the dev build needs a local patch for App connect return fails in dev builds: ProxyConnectReturn redeems twice under StrictMode #1883.

No app code changes: the logger wraps the data-browser's own vite.config.ts from outside, so it also works on branches that predate it.

Context

Used for user-testing session 1 (table calendar view), 25 Sep 2026. The findings are filed as #1792–#1808.

Test plan

  • demo-session.sh claude/demo-session-tooling: full build path, stack up, seed in about 3 s, calendar renders
  • --build-only on a pinned commit (bc39dac4b)
  • A live 20-minute session: the log recorded every click, route, dialog, toast and console error
  • --plugin issue-tracker on bc39dac4b with the App connect return fails in dev builds: ProxyConnectReturn redeems twice under StrictMode #1883 patch (2026-09-25). A headless run went through seed, connect, the repository picker and import (16 issues, 7 comments, about 33 s). No live session yet: the issue-tracker session moved to the usertest droplet.
  • Nothing here runs in CI. A review of the scripts is welcome.

🤖 Generated with Claude Code

Michiel de Jong and others added 3 commits September 25, 2026 10:31
scripts/demo-session/demo-session.sh starts an isolated atomic-server and
data-browser for a branch, with a fresh data dir, a dev-only interaction log
and an in-page seed. See scripts/demo-session/README.md.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…n checklist

A seeded note answered a task in the first session, so explanations move to
code comments. Clicks and inputs in the table editor now log their row and
column. The README gets the observer checklist: keep the browser pane
visible, let the tester set phone width by dragging the pane.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A drive-app session has to run against atomic-plugins' pinned host commit,
not a moving branch head, so a SHA gets a detached worktree. --build-only
builds without starting anything, for when another session holds the ports.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@michielbdejong

Copy link
Copy Markdown
Contributor Author

Merge-risk triage (2026-09-28). This is a read-only review of this PR's own diff. The rule it applies: nothing that works today may break for existing users or clients, and new behaviour is added alongside the old. The stack is being rebased onto develop as one chain before merging, so line references are against the current stacked base.

Verdict: SAFE-WITH-FIX (the draft status is a formality; the work is finished)

All new files are under scripts/demo-session/. No app code changes. The UX logger is apply: 'serve' and only added when VITE_UX_LOG=true, so it can't reach a production build.

Fix before merging: scripts/demo-session/demo-session.sh:142 calls cargo bin wasm-pack, which #1830 removes. Use cargo run -q --package cargo-bin -- wasm-pack build …, which works both before and after #1830.

🤖 Generated with Claude Code

…ion proxy

--plugin NAME builds atomic-plugins' integrations/NAME/app, starts the mock
proxy (MOCK_SCENARIO=user-testing by default) and points the server and Vite
at it. The issue-tracker seed installs the built app with createApp and
handOverAppKey, as "New app" does. The README records two limits found on
2026-09-25: the desktop browser pane blocks sandboxed frames, and the host's
ProxyConnectReturn fails under StrictMode in the dev build.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@joepio

joepio commented Sep 30, 2026

Copy link
Copy Markdown
Member

Trekmeester: included in batch #1947 (run 36781515104; #1915 had been merged without it). It lands when that run is green; if this PR breaks something there, I take it out and say why here.

joepio added a commit that referenced this pull request Oct 1, 2026
… e2e traces, Android menus, demo tooling, docs (#1947)

Includes #1884, #1930, #1940, #1888, #1933, #1882, #1834, #1631, #1932, #1945 and the ForkBar part of #1530.

joepio commented Oct 1, 2026

Copy link
Copy Markdown
Member

Merged into develop through batch #1947 (squash 1f892aa); this head was verified as part of it. Closing.


Generated by Claude Code

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

devex More PRs per minute

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants