From 984f031f218bb5cd42ad1fb9b8eda620955f490c Mon Sep 17 00:00:00 2001 From: Michiel de Jong Date: Mon, 28 Sep 2026 12:42:24 +0200 Subject: [PATCH 1/5] usertest: the user-testing droplet's setup, as files The instance at plugins.178-62-223-35.sslip.io (set up 2026-09-28) was configured by hand. This folder holds what rebuilds it: the Caddyfile and its systemd drop-in, server.sh (the pinned e2e image behind Caddy), catalog.mjs (a test catalog with every drive app installable and shown without the experimental toggle, unpublished apps built from this checkout) and deploy.sh. The README lists the hosts, the limits testers hit, how to update an app, and where testers' data ends up. Co-Authored-By: Claude Opus 5.5 --- .gitignore | 1 + usertest/Caddyfile | 24 ++++++++ usertest/README.md | 100 ++++++++++++++++++++++++++++++ usertest/caddy-usertest.conf | 5 ++ usertest/catalog.mjs | 114 +++++++++++++++++++++++++++++++++++ usertest/deploy.sh | 33 ++++++++++ usertest/server.sh | 28 +++++++++ 7 files changed, 305 insertions(+) create mode 100644 usertest/Caddyfile create mode 100644 usertest/README.md create mode 100644 usertest/caddy-usertest.conf create mode 100755 usertest/catalog.mjs create mode 100755 usertest/deploy.sh create mode 100755 usertest/server.sh diff --git a/.gitignore b/.gitignore index 08d6bac5..2cf18199 100644 --- a/.gitignore +++ b/.gitignore @@ -7,3 +7,4 @@ node_modules /integrations/tooling/test-results playwright-report/ /integrations/*/app/dist +usertest/out/ diff --git a/usertest/Caddyfile b/usertest/Caddyfile new file mode 100644 index 00000000..c0fbd318 --- /dev/null +++ b/usertest/Caddyfile @@ -0,0 +1,24 @@ +# Caddy config for the user-testing droplet. Installed as +# /etc/caddy/Caddyfile by deploy.sh; Caddy fetches the Let's Encrypt +# certificates itself. BASE_DOMAIN and ACME_EMAIL come from +# /etc/caddy/usertest.env, which the systemd drop-in caddy-usertest.conf +# loads (see README.md). +{ + email {$ACME_EMAIL} +} + +# atomic-server (server.sh), plain HTTP on loopback. It builds its links from +# ATOMIC_DOMAIN and reads the scheme from X-Forwarded-Proto, which Caddy sets. +plugins.{$BASE_DOMAIN} { + reverse_proxy 127.0.0.1:8080 +} + +# The test catalog and the drive app modules it points at (catalog.mjs). +# The data-browser fetches the catalog cross-origin; app modules are +# downloaded by the host and checked against their SRI hash. +catalog.{$BASE_DOMAIN} { + root * /srv/catalog + header Access-Control-Allow-Origin * + header Cache-Control "no-cache" + file_server +} diff --git a/usertest/README.md b/usertest/README.md new file mode 100644 index 00000000..abdcd4a1 --- /dev/null +++ b/usertest/README.md @@ -0,0 +1,100 @@ +# User-testing instance + +An atomic-server test instance, where people try the drive apps on their own +laptop and with their own provider accounts. It replaces sessions on one +prepared laptop. It runs on one DigitalOcean droplet, set up on 2026-09-28 +(Ubuntu 24.04, 2 vCPU, 4 GB, AMS3). This folder holds everything needed to +rebuild it. + +| Host | What | +| --- | --- | +| `https://plugins.` | atomic-server, the published e2e image of the pinned commit (`server.sh`) | +| `https://catalog./catalog.json` | the test catalog and the app modules it points at (`catalog.mjs`) | +| `https://localthought.io` | the integration proxy, shared with everyone else (not on the droplet) | + +The base domain is currently `178-62-223-35.sslip.io`. sslip.io resolves +`..sslip.io` to the IP address `a.b.c.d`, so no DNS setup +is needed. Drives are tied to the host name. Moving to a real domain later +means fresh drives. + +## What differs from the published catalog + +`catalog.mjs` starts from `integrations/catalog.json` and makes every drive +app installable: Google Calendar, GitHub issues, Bank statements, Clockify, +Notion and Pets. It also sets `experimental: false`, so testers need no +toggle. The published catalog keeps these disabled until launch. Apps not yet +in `apps/` are built from this checkout and served from the droplet. Pets +uses its published module. + +Known limits, as of 2026-09-28: + +- Pets cannot connect: localthought.io has no `pets` platform (#174). +- The calendar app does not import recurring events. +- The e2e image exposes test-only routes (`/app/prunetests`, + `/app/sandbox`). +- `ATOMIC_HOST_MODE=open`: anyone with the URL can create a drive. Stop the + container between session days. + +## Setting it up from scratch + +On a fresh Ubuntu 24.04 droplet, as root: + +```sh +apt-get update && apt-get install -y docker.io caddy +ufw allow 22/tcp && ufw allow 80/tcp && ufw allow 443/tcp && ufw --force enable +printf 'BASE_DOMAIN=%s\nACME_EMAIL=%s\n' 178-62-223-35.sslip.io you@example.org > /etc/caddy/usertest.env +``` + +Docker publishes the server on `127.0.0.1` only, so ufw's rules are not +bypassed. + +From a checkout of this repo, with the layout from `AGENTS.md` +(`node integrations/tooling/link-atomic-server.mjs`) and each app's +dependencies installed: + +```sh +for a in calendar money notion timesheets; do (cd integrations/$a && pnpm install); done +(cd integrations/issue-tracker/app && pnpm install --frozen-lockfile) +node usertest/catalog.mjs +sh usertest/deploy.sh root@178.62.223.35 +ssh root@178.62.223.35 sh /opt/usertest/server.sh 178-62-223-35.sslip.io +``` + +## Updating an app + +1. Change the app, then bump its entry in `VERSIONS` in `catalog.mjs`. +2. Run `node usertest/catalog.mjs && sh usertest/deploy.sh root@178.62.223.35`. +3. On Integrations, testers who installed the old version see "Update to + ". + +Never rebuild into an existing version: the host refuses a module whose bytes +no longer match the hash in the catalog. + +## For testers + +1. Open `https://plugins./app/dev-drive`. It creates an agent + and a drive in the browser, without signup. +2. In Settings → Integration, set the plugin catalog URL to + `https://catalog./catalog.json`. +3. On Integrations, install an app and connect it with your own account. + +Step 2 goes away once the planned `/usertest` page sets it. + +## Logs + +```sh +ssh root@178.62.223.35 docker logs --since 1h atomic-plugins +heroku logs -a integration-proxy -n 500 # needs access to the Heroku app +``` + +A plugin's errors happen inside its sandboxed frame and do not reach either +log yet. Collecting them is the next step (a Sentry-compatible collector on +the droplet). + +## Privacy + +Testers connect real accounts. Their data is on the droplet, in the Docker +volume `atomic-plugins-store`, and their provider tokens are in +localthought.io's database. Tell testers this before they start. Reset the +store with `docker rm -f atomic-plugins && docker volume rm atomic-plugins-store`, +then run `server.sh` again. diff --git a/usertest/caddy-usertest.conf b/usertest/caddy-usertest.conf new file mode 100644 index 00000000..48aed83f --- /dev/null +++ b/usertest/caddy-usertest.conf @@ -0,0 +1,5 @@ +# systemd drop-in, installed by deploy.sh as +# /etc/systemd/system/caddy.service.d/usertest.conf: gives Caddy the +# variables the Caddyfile reads. +[Service] +EnvironmentFile=/etc/caddy/usertest.env diff --git a/usertest/catalog.mjs b/usertest/catalog.mjs new file mode 100755 index 00000000..e6525782 --- /dev/null +++ b/usertest/catalog.mjs @@ -0,0 +1,114 @@ +#!/usr/bin/env node +/** + * Builds the user-testing catalog into a folder that deploy.sh copies to the + * droplet's /srv/catalog: + * + * node usertest/catalog.mjs [out] # default out: usertest/out + * + * It starts from this checkout's integrations/catalog.json and makes every + * drive app installable and visible without the "Show experimental plugins" + * toggle, including the ones the published catalog keeps disabled until + * launch. Apps not yet published to apps/ are built from this checkout with + * their own app/build.mjs and served next to the catalog, at + * apps///ui.js, with the SRI hash the host checks on install. + * Pets keeps its published module. + * + * VERSIONS below is the only thing to edit. Bump an app's version whenever + * its build changes: the host offers "Update to " only for a new + * version string, and a changed file under an old version fails the + * integrity check for anyone who installs it afterwards. + * + * Needs the layout AGENTS.md describes (browser/ from the pinned + * atomic-server) and each app's dependencies installed (see README.md). + */ +import { createHash } from 'node:crypto'; +import { mkdirSync, readFileSync, writeFileSync } from 'node:fs'; +import { dirname, resolve } from 'node:path'; +import { fileURLToPath, pathToFileURL } from 'node:url'; + +const here = dirname(fileURLToPath(import.meta.url)); +const repo = resolve(here, '..'); +const out = resolve(process.argv[2] ?? resolve(here, 'out')); + +const A = 'https://atomicdata.dev/properties/'; +const I = 'https://atomicdata.dev/integrations/properties/'; + +/** Drive apps built here. `base` is the catalog entry whose copy they reuse. */ +const VERSIONS = { + calendar: 'usertest-2', + 'issue-tracker': 'usertest', + money: 'usertest', + notion: 'usertest', + timesheets: 'usertest', +}; +const APPS = { + calendar: { + base: 'devonian-google-calendar', + name: 'Google Calendar', + emoji: '📅', + row: ['Event', 'Events'], + }, + 'issue-tracker': { + base: 'devonian-todoist', + name: 'GitHub issues', + emoji: '🐙', + description: 'Import the issues of a GitHub repository into a table.', + row: ['Issue', 'Issues'], + }, + money: { base: 'money', row: ['Transaction', 'Transactions'] }, + notion: { base: 'notion', row: ['Page', 'Pages'] }, + timesheets: { base: 'timesheets', row: ['Time entry', 'Time entries'] }, +}; + +const catalog = JSON.parse( + readFileSync(resolve(repo, 'integrations/catalog.json'), 'utf8'), +); +const byShortname = name => catalog.find(r => r[A + 'shortname'] === name); + +for (const [id, app] of Object.entries(APPS)) { + const version = VERSIONS[id]; + const { build } = await import( + pathToFileURL(resolve(repo, 'integrations', id, 'app/build.mjs')).href + ); + const file = resolve(out, 'apps', id, version, 'ui.js'); + mkdirSync(dirname(file), { recursive: true }); + await build({ outfile: file }); + const bytes = readFileSync(file); + + const source = byShortname(app.base); + if (!source) throw new Error(`catalog.json has no entry ${app.base}`); + const entry = { ...source }; + entry[A + 'localId'] = id; + entry[A + 'shortname'] = id; + if (app.name) entry[A + 'name'] = app.name; + if (app.emoji) entry[A + 'emoji'] = app.emoji; + if (app.description) entry[A + 'description'] = app.description; + // A drive app, not a sandbox plugin: it needs no API-plugins host. + delete entry[I + 'requires-api-plugins']; + entry[I + 'version'] = version; + entry[I + 'app-module'] = `apps/${id}/${version}/ui.js`; + entry[I + 'app-module-integrity'] = + 'sha384-' + createHash('sha384').update(bytes).digest('base64'); + entry[I + 'app-row-name'] = app.row[0]; + entry[I + 'app-row-name-plural'] = app.row[1]; + + const at = catalog.indexOf(source); + if (app.base === id) catalog[at] = entry; + else catalog.splice(at + 1, 0, entry); +} + +// Every drive app, Pets included: enabled, and shown without the toggle. +for (const entry of catalog) { + if (!entry[I + 'app-module']) continue; + entry[I + 'enabled'] = true; + entry[I + 'experimental'] = false; +} + +mkdirSync(out, { recursive: true }); +writeFileSync( + resolve(out, 'catalog.json'), + JSON.stringify(catalog, null, 2) + '\n', +); +for (const entry of catalog) + if (entry[I + 'app-module']) + console.log(`${entry[A + 'shortname']} ${entry[I + 'version']}`); diff --git a/usertest/deploy.sh b/usertest/deploy.sh new file mode 100755 index 00000000..d0227fdb --- /dev/null +++ b/usertest/deploy.sh @@ -0,0 +1,33 @@ +#!/bin/sh +# Copies the built catalog (catalog.mjs), the Caddyfile and server.sh to the +# droplet, then reloads Caddy. Existing app module versions on the droplet are +# kept, so testers who installed an older version can still update from it. +# +# sh usertest/deploy.sh root@ [ssh options…] +# +# It does not restart atomic-server; run server.sh on the droplet for that. +set -eu + +TARGET=${1:?usage: deploy.sh [ssh options…]} +shift +HERE=$(cd "$(dirname "$0")" && pwd) + +[ -f "$HERE/out/catalog.json" ] || + { echo "no usertest/out/catalog.json; run node usertest/catalog.mjs first" >&2; exit 1; } + +# COPYFILE_DISABLE keeps macOS tar from adding ._ metadata files. +(cd "$HERE/out" && COPYFILE_DISABLE=1 tar czf - catalog.json apps) | + ssh "$@" "$TARGET" 'mkdir -p /srv/catalog && tar xzf - -C /srv/catalog' +(cd "$HERE" && COPYFILE_DISABLE=1 tar czf - Caddyfile caddy-usertest.conf server.sh) | + ssh "$@" "$TARGET" 'set -e + test -f /etc/caddy/usertest.env || + { echo "create /etc/caddy/usertest.env first (README.md)" >&2; exit 1; } + mkdir -p /opt/usertest /etc/systemd/system/caddy.service.d + tar xzf - -C /opt/usertest + cp /opt/usertest/caddy-usertest.conf /etc/systemd/system/caddy.service.d/usertest.conf + cp /opt/usertest/Caddyfile /etc/caddy/Caddyfile + set -a; . /etc/caddy/usertest.env; set +a + caddy validate --config /etc/caddy/Caddyfile --adapter caddyfile >/dev/null + systemctl daemon-reload + systemctl restart caddy' +echo "deployed to $TARGET" diff --git a/usertest/server.sh b/usertest/server.sh new file mode 100755 index 00000000..6492e298 --- /dev/null +++ b/usertest/server.sh @@ -0,0 +1,28 @@ +#!/bin/sh +# (Re)starts the test instance's atomic-server on the droplet. Run as root on +# the droplet: `sh server.sh []`. +# +# It runs the published e2e image of the pinned commit, which is built with +# VITE_E2E: that keeps /app/dev-drive (a drive without signup) and also +# exposes test-only routes such as /app/prunetests. The image has no built-in +# HTTPS, so Caddy terminates TLS in front of it (Caddyfile). +# +# The store lives in the named volume atomic-plugins-store and survives +# restarts. `docker volume rm atomic-plugins-store` (with the container +# stopped) resets every tester's drive. +set -eu + +BASE_DOMAIN=${1:?usage: server.sh []} +SHA=${2:-2567fc30ba2da19124bbfb600e6a82c642b45819} +IMAGE=ghcr.io/ontola/atomic-server-e2e:$SHA + +docker pull -q "$IMAGE" >/dev/null +docker rm -f atomic-plugins >/dev/null 2>&1 || true +# ATOMIC_HOST_MODE=open: anyone who reaches the URL can create a drive, which +# is what testers need. Stop the container between session days. +docker run -d --name atomic-plugins --restart unless-stopped --init \ + -p 127.0.0.1:8080:80 -v atomic-plugins-store:/data \ + -e ATOMIC_DOMAIN="plugins.$BASE_DOMAIN" -e ATOMIC_PORT=80 \ + -e ATOMIC_HOST_MODE=open \ + -e ATOMIC_INTEGRATION_PROXY_URL=https://localthought.io \ + "$IMAGE" From 78334a42c3efae7eeef296adb79b946a2ee56474 Mon Sep 17 00:00:00 2001 From: Michiel de Jong Date: Mon, 28 Sep 2026 12:44:11 +0200 Subject: [PATCH 2/5] usertest: a log collector for Sentry envelopes and drive app posts Co-Authored-By: Claude Opus 5.5 --- usertest/Caddyfile | 6 + usertest/collector/run.sh | 12 ++ usertest/collector/server.mjs | 204 ++++++++++++++++++++++++++++++++++ usertest/deploy.sh | 2 +- usertest/server.sh | 7 ++ 5 files changed, 230 insertions(+), 1 deletion(-) create mode 100755 usertest/collector/run.sh create mode 100644 usertest/collector/server.mjs diff --git a/usertest/Caddyfile b/usertest/Caddyfile index c0fbd318..1fa688ee 100644 --- a/usertest/Caddyfile +++ b/usertest/Caddyfile @@ -22,3 +22,9 @@ catalog.{$BASE_DOMAIN} { header Cache-Control "no-cache" file_server } + +# The log collector (collector/server.mjs): Sentry envelopes from +# atomic-server and the data-browser, and /log posts from drive apps. +logs.{$BASE_DOMAIN} { + reverse_proxy 127.0.0.1:8081 +} diff --git a/usertest/collector/run.sh b/usertest/collector/run.sh new file mode 100755 index 00000000..3c192cb9 --- /dev/null +++ b/usertest/collector/run.sh @@ -0,0 +1,12 @@ +#!/bin/sh +# (Re)starts the log collector (server.mjs) on the droplet, as root: +# `sh run.sh`. Logs go to /var/lib/usertest-logs/.jsonl; follow a +# session with `tail -f /var/lib/usertest-logs/$(date -u +%F).jsonl`. +set -eu +HERE=$(cd "$(dirname "$0")" && pwd) +docker rm -f usertest-collector >/dev/null 2>&1 || true +docker run -d --name usertest-collector --restart unless-stopped --init \ + -p 127.0.0.1:8081:8081 \ + -v "$HERE/server.mjs:/app/server.mjs:ro" \ + -v /var/lib/usertest-logs:/logs \ + node:22-alpine node /app/server.mjs diff --git a/usertest/collector/server.mjs b/usertest/collector/server.mjs new file mode 100644 index 00000000..2e76e8a2 --- /dev/null +++ b/usertest/collector/server.mjs @@ -0,0 +1,204 @@ +#!/usr/bin/env node +/** + * The user-testing log collector: one JSON line per event, in a file per UTC + * day under LOG_DIR, so a session can be followed with `tail -f` and read + * back afterwards. No dependencies; see ../README.md, "Logs". + * + * Two ways in: + * + * - POST /api//envelope/ — Sentry's envelope endpoint. atomic-server + * (SENTRY_DSN, project 1) and the data-browser it serves + * (SENTRY_DSN_BROWSER, project 2) report here once server.sh sets those. + * Each item becomes one line: events are summarised (level, message, + * exception, top frames, URL), feedback keeps its message and contact. + * - POST /log — a JSON object, or an array of them, from anything else: a + * drive app in its sandboxed (null-origin) frame, or the /usertest page. + * Stored as sent, under `data`. + * + * Every line has `t` (receive time, ISO), `via` and `source`. Nothing checks + * who sends: like a Sentry DSN, the endpoint is public. Bodies over + * MAX_BODY bytes are refused, and a stored line is cut at MAX_LINE bytes. + */ +import { appendFileSync, mkdirSync } from 'node:fs'; +import { createServer } from 'node:http'; +import { join } from 'node:path'; +import { gunzipSync, inflateSync } from 'node:zlib'; + +const PORT = Number(process.env.PORT ?? 8081); +const LOG_DIR = process.env.LOG_DIR ?? '/logs'; +const MAX_BODY = 1024 * 1024; +const MAX_LINE = 64 * 1024; +/** Sentry project ids, as in the DSNs server.sh passes. */ +const PROJECTS = { 1: 'atomic-server', 2: 'data-browser' }; + +mkdirSync(LOG_DIR, { recursive: true }); + +function write(entry) { + const t = new Date().toISOString(); + let line = JSON.stringify({ t, ...entry }); + if (Buffer.byteLength(line) > MAX_LINE) + line = JSON.stringify({ + t, + via: entry.via, + source: entry.source, + truncated: line.slice(0, MAX_LINE), + }); + appendFileSync(join(LOG_DIR, `${t.slice(0, 10)}.jsonl`), line + '\n'); +} + +const frames = exception => + (exception?.stacktrace?.frames ?? []) + .slice(-5) + .reverse() + .map(f => `${f.function ?? '?'} (${f.filename ?? '?'}:${f.lineno ?? '?'})`); + +/** The parts of a Sentry event worth reading in a log line. */ +function summarise(event) { + const exceptions = event.exception?.values ?? []; + + return { + level: event.level, + message: + event.message?.formatted ?? + (typeof event.message === 'string' ? event.message : undefined) ?? + event.logentry?.message, + exceptions: exceptions.map(e => ({ + type: e.type, + value: e.value, + frames: frames(e), + })), + url: event.request?.url, + release: event.release, + environment: event.environment, + tags: event.tags, + breadcrumbs: (event.breadcrumbs?.values ?? event.breadcrumbs ?? []) + .slice?.(-10) + .map(b => ({ category: b.category, message: b.message, data: b.data })), + }; +} + +/** Splits an envelope into items: a header line, then header/payload pairs, + * where a payload is `length` bytes when the item header says so. */ +function* items(body) { + let at = body.indexOf(0x0a); + if (at < 0) return; + at++; + + while (at < body.length) { + const end = body.indexOf(0x0a, at); + const header = JSON.parse( + body.subarray(at, end < 0 ? body.length : end).toString(), + ); + if (end < 0) return; + at = end + 1; + let payload; + + if (typeof header.length === 'number') { + payload = body.subarray(at, at + header.length); + at += header.length + 1; + } else { + const next = body.indexOf(0x0a, at); + payload = body.subarray(at, next < 0 ? body.length : next); + at = next < 0 ? body.length : next + 1; + } + + yield { header, payload }; + } +} + +function envelope(project, body) { + const source = PROJECTS[project] ?? `sentry-${project}`; + + for (const { header, payload } of items(body)) { + const type = header.type; + // Sessions, client reports and attachments are not stored. + if (!['event', 'feedback', 'user_report'].includes(type)) continue; + const item = JSON.parse(payload.toString()); + if (type === 'feedback') + write({ + via: 'sentry', + source, + type, + feedback: item.contexts?.feedback, + url: item.request?.url ?? item.contexts?.feedback?.url, + }); + else write({ via: 'sentry', source, type, ...summarise(item) }); + } +} + +function readBody(req) { + return new Promise((resolve, reject) => { + const chunks = []; + let size = 0; + req.on('data', chunk => { + size += chunk.length; + if (size > MAX_BODY) { + reject(new Error('too large')); + req.destroy(); + } else chunks.push(chunk); + }); + req.on('end', () => { + const raw = Buffer.concat(chunks); + const encoding = req.headers['content-encoding']; + resolve( + encoding === 'gzip' + ? gunzipSync(raw) + : encoding === 'deflate' + ? inflateSync(raw) + : raw, + ); + }); + req.on('error', reject); + }); +} + +const cors = { + 'access-control-allow-origin': '*', + 'access-control-allow-methods': 'POST, OPTIONS', + 'access-control-allow-headers': + 'content-type, content-encoding, x-sentry-auth, sentry-trace, baggage', + 'access-control-max-age': '600', +}; + +createServer(async (req, res) => { + const url = new URL(req.url ?? '/', 'http://collector'); + const done = (status, body = '') => { + res.writeHead(status, { ...cors, 'content-type': 'application/json' }); + res.end(body); + }; + + if (req.method === 'OPTIONS') return done(204); + if (req.method === 'GET' && url.pathname === '/health') + return done(200, '{"ok":true}'); + if (req.method !== 'POST') return done(405); + + try { + const body = await readBody(req); + const sentry = url.pathname.match(/^\/api\/(\d+)\/envelope\/?$/); + + if (sentry) { + envelope(sentry[1], body); + + return done(200, '{}'); + } + if (url.pathname === '/log') { + const parsed = JSON.parse(body.toString()); + for (const data of Array.isArray(parsed) ? parsed : [parsed]) + write({ + via: 'log', + source: typeof data?.source === 'string' ? data.source : 'unknown', + data, + }); + + return done(204); + } + + return done(404); + } catch (error) { + write({ via: 'collector', source: 'collector', error: String(error) }); + + return done(400); + } +}).listen(PORT, () => { + process.stdout.write(`collector on :${PORT}, writing to ${LOG_DIR}\n`); +}); diff --git a/usertest/deploy.sh b/usertest/deploy.sh index d0227fdb..b6cf344d 100755 --- a/usertest/deploy.sh +++ b/usertest/deploy.sh @@ -18,7 +18,7 @@ HERE=$(cd "$(dirname "$0")" && pwd) # COPYFILE_DISABLE keeps macOS tar from adding ._ metadata files. (cd "$HERE/out" && COPYFILE_DISABLE=1 tar czf - catalog.json apps) | ssh "$@" "$TARGET" 'mkdir -p /srv/catalog && tar xzf - -C /srv/catalog' -(cd "$HERE" && COPYFILE_DISABLE=1 tar czf - Caddyfile caddy-usertest.conf server.sh) | +(cd "$HERE" && COPYFILE_DISABLE=1 tar czf - Caddyfile caddy-usertest.conf server.sh collector) | ssh "$@" "$TARGET" 'set -e test -f /etc/caddy/usertest.env || { echo "create /etc/caddy/usertest.env first (README.md)" >&2; exit 1; } diff --git a/usertest/server.sh b/usertest/server.sh index 6492e298..1cb71501 100755 --- a/usertest/server.sh +++ b/usertest/server.sh @@ -7,6 +7,10 @@ # exposes test-only routes such as /app/prunetests. The image has no built-in # HTTPS, so Caddy terminates TLS in front of it (Caddyfile). # +# Errors (and the sidebar Feedback form) are reported to the collector +# (collector/), in Sentry's format: project 1 is the server, 2 the +# data-browser. +# # The store lives in the named volume atomic-plugins-store and survives # restarts. `docker volume rm atomic-plugins-store` (with the container # stopped) resets every tester's drive. @@ -25,4 +29,7 @@ docker run -d --name atomic-plugins --restart unless-stopped --init \ -e ATOMIC_DOMAIN="plugins.$BASE_DOMAIN" -e ATOMIC_PORT=80 \ -e ATOMIC_HOST_MODE=open \ -e ATOMIC_INTEGRATION_PROXY_URL=https://localthought.io \ + -e SENTRY_DSN="https://usertest@logs.$BASE_DOMAIN/1" \ + -e SENTRY_DSN_BROWSER="https://usertest@logs.$BASE_DOMAIN/2" \ + -e SENTRY_ENVIRONMENT=usertest \ "$IMAGE" From f388c571c1ab4fca439dc12556cac026ed0efc62 Mon Sep 17 00:00:00 2001 From: Michiel de Jong Date: Mon, 28 Sep 2026 12:46:45 +0200 Subject: [PATCH 3/5] usertest: document the collector; quieter deploy Co-Authored-By: Claude Opus 5.5 --- usertest/README.md | 28 ++++++++++++++++++++++++---- usertest/deploy.sh | 10 ++++++---- 2 files changed, 30 insertions(+), 8 deletions(-) diff --git a/usertest/README.md b/usertest/README.md index abdcd4a1..12e6444c 100644 --- a/usertest/README.md +++ b/usertest/README.md @@ -10,6 +10,7 @@ rebuild it. | --- | --- | | `https://plugins.` | atomic-server, the published e2e image of the pinned commit (`server.sh`) | | `https://catalog./catalog.json` | the test catalog and the app modules it points at (`catalog.mjs`) | +| `https://logs.` | the log collector (`collector/`) | | `https://localthought.io` | the integration proxy, shared with everyone else (not on the droplet) | The base domain is currently `178-62-223-35.sslip.io`. sslip.io resolves @@ -43,6 +44,7 @@ On a fresh Ubuntu 24.04 droplet, as root: apt-get update && apt-get install -y docker.io caddy ufw allow 22/tcp && ufw allow 80/tcp && ufw allow 443/tcp && ufw --force enable printf 'BASE_DOMAIN=%s\nACME_EMAIL=%s\n' 178-62-223-35.sslip.io you@example.org > /etc/caddy/usertest.env +mkdir -p /var/lib/usertest-logs ``` Docker publishes the server on `127.0.0.1` only, so ufw's rules are not @@ -57,6 +59,7 @@ for a in calendar money notion timesheets; do (cd integrations/$a && pnpm instal (cd integrations/issue-tracker/app && pnpm install --frozen-lockfile) node usertest/catalog.mjs sh usertest/deploy.sh root@178.62.223.35 +ssh root@178.62.223.35 sh /opt/usertest/collector/run.sh ssh root@178.62.223.35 sh /opt/usertest/server.sh 178-62-223-35.sslip.io ``` @@ -82,19 +85,36 @@ Step 2 goes away once the planned `/usertest` page sets it. ## Logs +The collector (`collector/server.mjs`, no dependencies, run in a +`node:22-alpine` container by `collector/run.sh`) writes one JSON line per +event to `/var/lib/usertest-logs/.jsonl`. It has two entrances: + +- **Sentry's envelope endpoint.** `server.sh` sets `SENTRY_DSN` (project 1, + atomic-server) and `SENTRY_DSN_BROWSER` (project 2, the data-browser, which + atomic-server injects into the page at runtime). This reports uncaught + errors in the page and the server's `error!` events. It also switches on + the sidebar's Feedback form, whose messages arrive as `type: feedback`. +- **`POST /log`**, a JSON object or an array of them, for anything else. A + drive app's errors stay inside its sandboxed frame: the host shows them + there and reports nothing. The app has to post them itself. + +Verified on 2026-09-28: an error thrown in the data-browser arrived within +seconds with its stack and URL. + ```sh +ssh root@178.62.223.35 'tail -f /var/lib/usertest-logs/$(date -u +%F).jsonl' ssh root@178.62.223.35 docker logs --since 1h atomic-plugins heroku logs -a integration-proxy -n 500 # needs access to the Heroku app ``` -A plugin's errors happen inside its sandboxed frame and do not reach either -log yet. Collecting them is the next step (a Sentry-compatible collector on -the droplet). +Both endpoints are public, like any Sentry DSN, and nothing checks who +sends. Bodies over 1 MB are refused. ## Privacy Testers connect real accounts. Their data is on the droplet, in the Docker volume `atomic-plugins-store`, and their provider tokens are in -localthought.io's database. Tell testers this before they start. Reset the +localthought.io's database. Error reports can contain what was on screen +(titles in messages, URLs), and they stay in `/var/lib/usertest-logs`. Tell testers this before they start. Reset the store with `docker rm -f atomic-plugins && docker volume rm atomic-plugins-store`, then run `server.sh` again. diff --git a/usertest/deploy.sh b/usertest/deploy.sh index b6cf344d..c92ded53 100755 --- a/usertest/deploy.sh +++ b/usertest/deploy.sh @@ -15,10 +15,11 @@ HERE=$(cd "$(dirname "$0")" && pwd) [ -f "$HERE/out/catalog.json" ] || { echo "no usertest/out/catalog.json; run node usertest/catalog.mjs first" >&2; exit 1; } -# COPYFILE_DISABLE keeps macOS tar from adding ._ metadata files. -(cd "$HERE/out" && COPYFILE_DISABLE=1 tar czf - catalog.json apps) | +# COPYFILE_DISABLE and --no-xattrs keep macOS tar from adding ._ files and +# extended attributes that GNU tar on the droplet warns about. +(cd "$HERE/out" && COPYFILE_DISABLE=1 tar --no-xattrs -czf - catalog.json apps) | ssh "$@" "$TARGET" 'mkdir -p /srv/catalog && tar xzf - -C /srv/catalog' -(cd "$HERE" && COPYFILE_DISABLE=1 tar czf - Caddyfile caddy-usertest.conf server.sh collector) | +(cd "$HERE" && COPYFILE_DISABLE=1 tar --no-xattrs -czf - Caddyfile caddy-usertest.conf server.sh collector) | ssh "$@" "$TARGET" 'set -e test -f /etc/caddy/usertest.env || { echo "create /etc/caddy/usertest.env first (README.md)" >&2; exit 1; } @@ -27,7 +28,8 @@ HERE=$(cd "$(dirname "$0")" && pwd) cp /opt/usertest/caddy-usertest.conf /etc/systemd/system/caddy.service.d/usertest.conf cp /opt/usertest/Caddyfile /etc/caddy/Caddyfile set -a; . /etc/caddy/usertest.env; set +a - caddy validate --config /etc/caddy/Caddyfile --adapter caddyfile >/dev/null + caddy validate --config /etc/caddy/Caddyfile --adapter caddyfile >/dev/null 2>/tmp/caddy-validate.log || + { cat /tmp/caddy-validate.log >&2; exit 1; } systemctl daemon-reload systemctl restart caddy' echo "deployed to $TARGET" From 905877f074855a11237bce5ba40f4b077fe3729d Mon Sep 17 00:00:00 2001 From: Michiel de Jong Date: Mon, 28 Sep 2026 12:57:57 +0200 Subject: [PATCH 4/5] usertest: prepend the collector hook to test builds; calendar usertest-3 Co-Authored-By: Claude Opus 5.5 --- usertest/README.md | 10 +++++++--- usertest/catalog.mjs | 20 +++++++++++++++++++- 2 files changed, 26 insertions(+), 4 deletions(-) diff --git a/usertest/README.md b/usertest/README.md index 12e6444c..6478b98a 100644 --- a/usertest/README.md +++ b/usertest/README.md @@ -57,7 +57,7 @@ dependencies installed: ```sh for a in calendar money notion timesheets; do (cd integrations/$a && pnpm install); done (cd integrations/issue-tracker/app && pnpm install --frozen-lockfile) -node usertest/catalog.mjs +USERTEST_LOG_URL=https://logs.178-62-223-35.sslip.io/log node usertest/catalog.mjs sh usertest/deploy.sh root@178.62.223.35 ssh root@178.62.223.35 sh /opt/usertest/collector/run.sh ssh root@178.62.223.35 sh /opt/usertest/server.sh 178-62-223-35.sslip.io @@ -66,7 +66,8 @@ ssh root@178.62.223.35 sh /opt/usertest/server.sh 178-62-223-35.sslip.io ## Updating an app 1. Change the app, then bump its entry in `VERSIONS` in `catalog.mjs`. -2. Run `node usertest/catalog.mjs && sh usertest/deploy.sh root@178.62.223.35`. +2. Run `USERTEST_LOG_URL=https://logs.178-62-223-35.sslip.io/log node usertest/catalog.mjs`, + then `sh usertest/deploy.sh root@178.62.223.35`. 3. On Integrations, testers who installed the old version see "Update to ". @@ -96,7 +97,10 @@ event to `/var/lib/usertest-logs/.jsonl`. It has two entrances: the sidebar's Feedback form, whose messages arrive as `type: feedback`. - **`POST /log`**, a JSON object or an array of them, for anything else. A drive app's errors stay inside its sandboxed frame: the host shows them - there and reports nothing. The app has to post them itself. + there and reports nothing. The calendar app hands its failures, a line + per finished sync and anything uncaught to a hook (`app/report.ts`), and + `catalog.mjs` prepends the hook that posts here when `USERTEST_LOG_URL` + is set. The app itself makes no network request. Verified on 2026-09-28: an error thrown in the data-browser arrived within seconds with its stack and URL. diff --git a/usertest/catalog.mjs b/usertest/catalog.mjs index e6525782..6c11c6a6 100755 --- a/usertest/catalog.mjs +++ b/usertest/catalog.mjs @@ -5,6 +5,14 @@ * * node usertest/catalog.mjs [out] # default out: usertest/out * + * With USERTEST_LOG_URL set (deploy it with + * USERTEST_LOG_URL=https://logs./log), every module built here + * starts with a prelude that defines `globalThis.__USERTEST_REPORT__`, which + * posts what an app hands it to the collector. Apps call it through their + * own `report.ts` (calendar so far). The apps themselves make no network + * request of their own (their build tests check for `fetch(`); only this + * prelude does, and only in these test builds. + * * It starts from this checkout's integrations/catalog.json and makes every * drive app installable and visible without the "Show experimental plugins" * toggle, including the ones the published catalog keeps disabled until @@ -35,7 +43,7 @@ const I = 'https://atomicdata.dev/integrations/properties/'; /** Drive apps built here. `base` is the catalog entry whose copy they reuse. */ const VERSIONS = { - calendar: 'usertest-2', + calendar: 'usertest-3', 'issue-tracker': 'usertest', money: 'usertest', notion: 'usertest', @@ -60,6 +68,15 @@ const APPS = { timesheets: { base: 'timesheets', row: ['Time entry', 'Time entries'] }, }; +const LOG_URL = process.env.USERTEST_LOG_URL; +if (LOG_URL && !/^https:\/\/[^/]+\/log$/.test(LOG_URL)) + throw new Error('USERTEST_LOG_URL must look like https:///log'); +/** text/plain keeps the post a simple request: no CORS preflight from the + * frame's null origin. A failing collector never affects the app. */ +const PRELUDE = LOG_URL + ? `globalThis.__USERTEST_REPORT__=e=>{try{fetch(${JSON.stringify(LOG_URL)},{method:"POST",keepalive:!0,headers:{"content-type":"text/plain"},body:JSON.stringify(e)}).catch(()=>{})}catch{}};\n` + : ''; + const catalog = JSON.parse( readFileSync(resolve(repo, 'integrations/catalog.json'), 'utf8'), ); @@ -73,6 +90,7 @@ for (const [id, app] of Object.entries(APPS)) { const file = resolve(out, 'apps', id, version, 'ui.js'); mkdirSync(dirname(file), { recursive: true }); await build({ outfile: file }); + if (PRELUDE) writeFileSync(file, PRELUDE + readFileSync(file, 'utf8')); const bytes = readFileSync(file); const source = byShortname(app.base); From 3d057647bb30878e54bea3f0637f559af84c2eef Mon Sep 17 00:00:00 2001 From: Michiel de Jong Date: Mon, 28 Sep 2026 12:58:50 +0200 Subject: [PATCH 5/5] usertest: prelude only for apps that report, so other builds keep their bytes Co-Authored-By: Claude Opus 5.5 --- usertest/catalog.mjs | 9 ++++++--- 1 file changed, 6 insertions(+), 3 deletions(-) diff --git a/usertest/catalog.mjs b/usertest/catalog.mjs index 6c11c6a6..ca146946 100755 --- a/usertest/catalog.mjs +++ b/usertest/catalog.mjs @@ -6,8 +6,8 @@ * node usertest/catalog.mjs [out] # default out: usertest/out * * With USERTEST_LOG_URL set (deploy it with - * USERTEST_LOG_URL=https://logs./log), every module built here - * starts with a prelude that defines `globalThis.__USERTEST_REPORT__`, which + * USERTEST_LOG_URL=https://logs./log), the modules of apps + * marked `report: true` start with a prelude that defines `globalThis.__USERTEST_REPORT__`, which * posts what an app hands it to the collector. Apps call it through their * own `report.ts` (calendar so far). The apps themselves make no network * request of their own (their build tests check for `fetch(`); only this @@ -55,6 +55,8 @@ const APPS = { name: 'Google Calendar', emoji: '📅', row: ['Event', 'Events'], + // Has app/report.ts; gets the collector prelude. + report: true, }, 'issue-tracker': { base: 'devonian-todoist', @@ -90,7 +92,8 @@ for (const [id, app] of Object.entries(APPS)) { const file = resolve(out, 'apps', id, version, 'ui.js'); mkdirSync(dirname(file), { recursive: true }); await build({ outfile: file }); - if (PRELUDE) writeFileSync(file, PRELUDE + readFileSync(file, 'utf8')); + if (PRELUDE && app.report) + writeFileSync(file, PRELUDE + readFileSync(file, 'utf8')); const bytes = readFileSync(file); const source = byShortname(app.base);