usertest: per-tester log lines, token redaction, findings to a private triage repo

[michielbdejong]

Makes moderated user-testing sessions (#185) produce findings without anyone copying files around, and keeps publication gated.

Per-tester log lines

• The collector tags each line with client, a salted hash (/etc/usertest-salt.env, created by collector/run.sh) of the sender's address. No addresses are stored.
• It also replaces credentials in URLs (token=, code=, key= and a few more) with [redacted]. Plugin stack traces carried the host's view token.
• The moderator remembers the hashes its tester's page came from and only passes those lines to Claude (atomic-server's own lines are always included). Two testers at the same time no longer hear about each other's errors.

Findings

• moderator/analyze.mjs runs when a session ends (the page ends it, or the moderator's [END]). It sends the transcript, the log lines and up to 12 screenshots to claude-opus-5 (effort high).
• It writes anonymized findings.json and findings.md into the session folder. The prompt forbids names, email addresses and calendar content, and scrub() removes the tester's name, email addresses and URL queries.
• With /etc/github-findings.env (a fine-grained token, Issues read/write on the private ontola/usertest-findings only, to be created and saved by Michiel), each finding becomes an issue there, labeled tooling, atomic-plugins or atomic-server. Without the file, findings stay on the droplet.
• Nothing reaches a public repository until Michiel labels a finding approved in that private repo.
• docker exec usertest-moderator node analyze.mjs <id> [--file] analyzes a session that ended without the page saying so.

Checked on the droplet

• Session 1 (about 11 minutes): 6 findings in 34 s, about 2,000 input and 2,400 output tokens. Four were tooling findings (two already fixed in usertest: voice-moderated sessions with a Claude moderator #185) and two matched the issues filed by hand (calendar: after an import the view stays on this week, even when every imported event is elsewhere #186, Empty drive: no visible path to connect another app (user testing) atomic-server#1867). The tester's name appears nowhere.
• A test /log line got a client hash, and its token= became [redacted].
• Both services restart cleanly with the new env files.

Not verified yet: filing into the private repo, which needs the token, and two concurrent sessions.

🤖 Generated with Claude Code