feat: load test results from uploaded archive files

.claude/rules/scanning.md

@@ -25,3 +25,16 @@ When a Watch tick or manual Re-scan re-runs the scanner, `updateScanModule` (in
 ## Result-source per profile
 
 Each profile declares `result_source: 'junit' | 'files'`. JUnit-driven profiles trust the XML's pass/fail summary (e.g. Paparazzi, Compose Screenshot Testing). File-only profiles use mtime clustering alone (legacy fallback for tools without JUnit). Adding a new tool means picking the right `result_source` — don't assume one or the other.
+
+## Overlaying CI artifacts: URL vs uploaded files
+
+Two ways to pull a CI run's `build/` outputs onto a project before scanning, both landing in the same `runScan` after overlay:
+
+- **Scan from URL** (`src/remoteFetch.js`, `scanJobs.startScanFromUrl`): download one `.tar`/`.tar.gz`, extract selected `<module>/build` dirs straight from the tar.
+- **Scan from uploaded files** (`src/localArchive.js`, `scanJobs.startScanFromUploads`): accept several `.zip`/`.tar`/`.tar.gz` files (raw-streamed via `POST /api/uploads` → temp file → opaque id), fully extract each to a stage dir, then **merge**.
+
+**Uploads merge + conflict rule:** the same archive-internal path appearing in two archives is fine **iff** the bytes are identical (sha256) — merged silently. Differing bytes is a **conflict**: the job aborts with `status:'failed'` + a `conflicts:[{path, archives}]` list and **writes nothing to the project**. Don't "resolve" conflicts by last-wins; the whole point is that we can't know which delta is authoritative.
+
+**Why full extraction for uploads (not tar member-selection like the URL flow):** we need file contents on disk to hash for conflict detection, it unifies zip+tar behind one pipeline, and it sidesteps the `[bracket]`-glob problem in Paparazzi parameterized names. Overlay copies only `<matchedRoot>/build` subtrees via `fs.cpSync` — never `src/`/goldens. Format is chosen by **magic-byte sniff**, not extension. Zip extraction shells out to `unzip` (macOS + Linux CI have it; Windows does not — zip-on-Windows is out of scope).
+
+The `needs_confirmation` park/resume and per-terminal-state temp cleanup are shared with the URL flow via the kind-aware `confirmScanJob` / `finishJob` in `src/scanJobs.js`.

CHANGES/2026-07-07_feat-scan-from-files.md

@@ -0,0 +1,68 @@
+# feat: load test results from uploaded archive files (multi-file, conflict-aware)
+
+**Date:** 2026-07-07
+**Type:** Feature
+
+## Intent
+Until now the only way to overlay a CI run's screenshot-test `build/` outputs onto a
+project was **Scan from URL** (download one `.tar`/`.tar.gz`). Users who have result
+archives on disk, or who produce several sharded CI archives, had no path in. This adds a
+**file picker** next to the URL input that accepts **multiple** `.zip`/`.tar`/`.tar.gz`
+files, **merges** them into a single overlay + scan, and — when two archives contain the
+same file with **different content** — **aborts** with an error dialog listing the
+conflicts. Identical-content duplicates are merged silently.
+
+### Prompts summary
+1. "For the load test result from zip file, we currently propose a URL input, I also want a
+   file picker that supports multiple files; if a duplicate file is found in multiple zip
+   files it should show an error dialog if the duplicate file name's content is different."
+2. Confirmed: accept zip + tar/tar.gz; merge into one scan; abort-all on any conflict.
+
+## Changes
+
+### `src/localArchive.js` (new)
+- Sibling of `remoteFetch.js` for local uploads. `sniffFormat` (magic bytes), `extractArchive`
+  (`unzip`/`tar`), `collectBuildMembers` (walk + `isUnsafeMember` guard + symlink skip),
+  `hashFile` (sha256), `mergeStages` (cross-archive conflict detection), `overlayBuildDirs`
+  (`fs.cpSync` of `<module>/build` subtrees only), and temp/stage cleanup. Reuses
+  `remoteFetch`'s `isBuildMember`/`isUnsafeMember`/`moduleRootOf`/`checkCompat`/`MAX_DOWNLOAD_BYTES`.
+
+### `src/remoteFetch.js`
+- Export `byteCapTransform` so the raw-upload route can reuse the streaming byte cap.
+
+### `src/scanJobs.js`
+- Upload registry (`registerUpload`/`takeUploads`) with TTL reaping of unconsumed uploads.
+- `startScanFromUploads` → `runUploads` pipeline: extract all → merge/conflict-detect →
+  abort on conflict (project untouched) → `checkCompat` → park `needs_confirmation` or
+  `doOverlayAndScan` → `runScan`.
+- `finishJob` now frees stage dirs + raw uploads and carries a `conflicts` payload; `getJob`
+  surfaces `conflicts`. `confirmScanFromUrl` generalized to kind-aware `confirmScanJob`.
+
+### `src/handler.js`
+- `POST /api/uploads` (raw stream → temp file, byte-capped, filename/extension validated) and
+  `POST /api/projects/:id/scan-from-uploads` (`{uploadIds}` → job). Confirm route now calls
+  `confirmScanJob`.
+
+### `static/js/home.js`, `static/css/app.css`
+- File picker added **inside the existing hidden Scan-from-URL form** (keeps project-card
+  screenshot baselines unchanged). `_scanFromFiles` uploads each file then starts the merge
+  scan; `_showConflictDialog` lists conflicting paths on a conflict abort.
+
+### Tests & docs
+- `tests/node/localArchive.test.js` (11 cases: pure helpers + e2e merge / identical-dup /
+  conflict-abort-project-untouched / zip+tar / needs_confirmation / size-cap 413 / validation).
+- `docs/index.html` and `.claude/rules/scanning.md` document the upload + conflict behavior.
+
+## Files modified
+
+| File | Change |
+|------|--------|
+| `src/localArchive.js` | New module: extract/merge/conflict-detect/overlay uploaded archives |
+| `src/remoteFetch.js` | Export `byteCapTransform` |
+| `src/scanJobs.js` | Upload registry, `startScanFromUploads`/`runUploads`/`doOverlayAndScan`, kind-aware `confirmScanJob`/`finishJob`, `getJob` conflicts |
+| `src/handler.js` | `POST /api/uploads`, `POST /api/projects/:id/scan-from-uploads`, confirm route rename |
+| `static/js/home.js` | File picker in the URL form, `_scanFromFiles`, conflict dialog |
+| `static/css/app.css` | `.url-form-or` divider |
+| `tests/node/localArchive.test.js` | New test suite |
+| `docs/index.html` | Document uploaded-archive workflow |
+| `.claude/rules/scanning.md` | Document URL-vs-uploads overlay + conflict rule |

docs/index.html

@@ -693,7 +693,7 @@ <h3>Live watch mode</h3>
         <svg width="18" height="18" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="1.75" stroke-linecap="round" stroke-linejoin="round" aria-hidden="true"><path d="M21 15v4a2 2 0 0 1-2 2H5a2 2 0 0 1-2-2v-4"/><polyline points="7 10 12 15 17 10"/><line x1="12" y1="15" x2="12" y2="3"/></svg>
       </div>
       <h3>Review CI results remotely</h3>
-      <p>Paste a CI run's result tarball URL (e.g. a Jenkins artifact) and Papa Stud.io downloads it, overlays the failures onto your local project, and scans — reviewing the remote run against your local goldens.</p>
+      <p>Paste a CI run's result tarball URL (e.g. a Jenkins artifact) — or upload one or more <code>.zip</code>/<code>.tar</code>/<code>.tar.gz</code> archives from disk. Papa Stud.io overlays the failures onto your local project and scans against your local goldens. Multiple archives are merged into one scan; if two disagree on a file's content, the scan is aborted and the conflicts are listed.</p>
     </div>
   </div>
 </section>

src/handler.js

@@ -15,8 +15,11 @@
 const apng = require('./apng');
 const imageSlice = require('./imageSlice');
 const remoteFetch = require('./remoteFetch');
+const localArchive = require('./localArchive');
+const { pipeline } = require('stream/promises');
 
 const STATIC_DIR = path.resolve(__dirname, '..', 'static');
+const UPLOAD_EXT_RE = /\.(zip|tar|tar\.gz|tgz)$/i;
 const INDEX_HTML = path.join(STATIC_DIR, 'index.html');
 
 function createRouter() {
@@ -288,10 +291,59 @@
     res.status(202).json({ jobId });
   });
 
-  // Resume a scan-from-url job that parked because the tarball's module layout
-  // didn't line up with the project (compat mismatch).
+  // Stream one uploaded archive to a temp file and return an opaque id the
+  // client later names in scan-from-uploads. Body is the raw file bytes
+  // (Content-Type is ignored; express.json() leaves non-JSON bodies intact, so
+  // the request stream reaches us here unread). We pipe straight to disk — never
+  // buffering the (up to 2 GB) body in memory — and enforce the same size cap as
+  // a scan-from-url download.
+  //
+  // codeql[js/missing-rate-limiting]: by design — single-user local app, server
+  // binds 127.0.0.1 (no remote attacker). See `.claude/rules/dev_workflow.md`.
+  router.post('/api/uploads', async (req, res) => {
+    const raw = req.query.filename;
+    const filename = typeof raw === 'string' ? path.basename(decodeURIComponent(raw)) : '';
+    if (!filename || !UPLOAD_EXT_RE.test(filename)) {
+      return res.status(400).json({ error: 'filename with .zip/.tar/.tar.gz extension required' });
+    }
+    const dir = fs.mkdtempSync(path.join(os.tmpdir(), 'papastud-upload-'));
+    const dest = path.join(dir, 'archive');
+    try {
+      await pipeline(req, remoteFetch.byteCapTransform(localArchive.MAX_UPLOAD_BYTES), fs.createWriteStream(dest));
+    } catch (e) {
+      localArchive.cleanupUpload(dest);
+      const tooLarge = /too large/.test(e.message || '');
+      return res.status(tooLarge ? 413 : 400).json({ error: e.message || 'upload failed' });
+    }
+    const size = fs.statSync(dest).size;
+    const uploadId = scanJobs.registerUpload({ path: dest, filename, dir });
+    res.status(201).json({ uploadId, filename, size });
+  });
+
+  // Merge several uploaded archives (by upload id) and overlay their build/
+  // artifacts onto a project, then scan — like scan-from-url but for local
+  // files. May park at `needs_confirmation`, or fail with a `conflicts` list
+  // when the archives disagree on a file's content.
+  router.post('/api/projects/:id/scan-from-uploads', (req, res) => {
+    const project = projects.getProject(req.params.id);
+    if (!project) return res.status(404).json({ error: 'project not found' });
+    const uploadIds = req.body && req.body.uploadIds;
+    if (!Array.isArray(uploadIds) || uploadIds.length === 0) {
+      return res.status(400).json({ error: 'uploadIds required' });
+    }
+    if (uploadIds.length > localArchive.MAX_UPLOAD_FILES) {
+      return res.status(400).json({ error: `too many files (max ${localArchive.MAX_UPLOAD_FILES})` });
+    }
+    const uploadList = scanJobs.takeUploads(uploadIds);
+    if (!uploadList) return res.status(400).json({ error: 'unknown or expired upload id' });
+    const jobId = scanJobs.startScanFromUploads(project, uploadList);
+    res.status(202).json({ jobId });
+  });
+
+  // Resume a scan job (URL or uploads) that parked because the archive's module
+  // layout didn't line up with the project (compat mismatch).
   router.post('/api/scan-jobs/:id/confirm', (req, res) => {
-    if (scanJobs.confirmScanFromUrl(req.params.id)) {
+    if (scanJobs.confirmScanJob(req.params.id)) {
       res.json({ ok: true });
     } else {
       res.status(404).json({ error: 'job not awaiting confirmation' });

src/localArchive.js

@@ -0,0 +1,190 @@
+/**
+ * Merge locally-uploaded screenshot-test result archives and overlay their
+ * build outputs onto a project directory.
+ *
+ * This is the file-upload sibling of `remoteFetch.js` (scan-from-URL). Where
+ * that module downloads a single `.tar` and extracts selected `build/` dirs
+ * straight out of the tar, this one accepts *several* archives (`.zip` or
+ * `.tar`/`.tar.gz`), extracts each fully into its own staging dir, and then:
+ *
+ *   1. collects the safe `build/` members of every stage,
+ *   2. hashes them to detect cross-archive conflicts (same path, different
+ *      bytes) — which abort the whole operation, and
+ *   3. copies the `<module>/build` subtrees into the project (identical
+ *      duplicates across archives are harmless overwrites).
+ *
+ * Full extraction (rather than tar member-selection) is deliberate: we need
+ * file contents on disk to hash for conflict detection, it unifies zip + tar
+ * handling behind one pipeline, and it sidesteps the `[bracket]` glob problem
+ * that Paparazzi parameterized snapshot names cause with `tar -T` member lists.
+ *
+ * No new npm deps: we shell out to the system `unzip` (zip) and `tar` (tar,
+ * gzip auto-detected). Both are present on macOS and Linux CI runners.
+ */
+
+'use strict';
+
+const fs = require('fs');
+const path = require('path');
+const crypto = require('crypto');
+const { spawnSync } = require('child_process');
+const remoteFetch = require('./remoteFetch');
+
+// Same 2 GB ceiling as a scan-from-url download, applied per uploaded file.
+const MAX_UPLOAD_BYTES = remoteFetch.MAX_DOWNLOAD_BYTES;
+
+// Cap on how many archives a single merge can combine. Well past any real
+// sharded-CI run; a backstop against a pathological request.
+const MAX_UPLOAD_FILES = 32;
+
+const SPAWN_OPTS = { encoding: 'utf8', maxBuffer: 64 * 1024 * 1024 };
+
+// Detect the archive format from its magic bytes rather than the filename.
+// Zip local-file/central-dir/end-of-central-dir headers all start `PK`.
+// Everything else is treated as tar (system `tar` auto-detects gzip, so plain
+// `.tar` and `.tar.gz` both extract; a genuinely bad file surfaces as a real
+// `tar -xf` error rather than being guessed here).
+function sniffFormat(filePath) {
+  const fd = fs.openSync(filePath, 'r');
+  try {
+    const buf = Buffer.alloc(4);
+    const n = fs.readSync(fd, buf, 0, 4, 0);
+    if (n >= 2 && buf[0] === 0x50 && buf[1] === 0x4b) return 'zip'; // "PK"
+    return 'tar';
+  } finally {
+    fs.closeSync(fd);
+  }
+}
+
+// Extract one archive fully into stageDir (created if needed). Throws with the
+// tail of the extractor's stderr on failure.
+function extractArchive(filePath, stageDir) {
+  fs.mkdirSync(stageDir, { recursive: true });
+  const format = sniffFormat(filePath);
+  const r = format === 'zip'
+    ? spawnSync('unzip', ['-o', '-q', filePath, '-d', stageDir], SPAWN_OPTS)
+    : spawnSync('tar', ['-xf', filePath, '-C', stageDir], SPAWN_OPTS);
+  if (r.status !== 0) {
+    const detail = (r.stderr || '').slice(-300) || `${format} extract failed`;
+    throw new Error(`could not extract archive: ${detail}`);
+  }
+}
+
+// Walk a staged archive and return its safe `build/` members as
+// { relPath (POSIX), absPath }. Directories and symlinks are skipped;
+// a member whose path escapes the stage root throws (defence-in-depth against
+// zip-slip on top of the extractor's own guards).
+function collectBuildMembers(stageDir) {
+  const members = [];
+  const walk = (dir) => {
+    for (const dirent of fs.readdirSync(dir, { withFileTypes: true })) {
+      const abs = path.join(dir, dirent.name);
+      if (dirent.isSymbolicLink()) continue;
+      if (dirent.isDirectory()) { walk(abs); continue; }
+      if (!dirent.isFile()) continue;
+      const relPath = path.relative(stageDir, abs).split(path.sep).join('/');
+      if (!remoteFetch.isBuildMember(relPath)) continue;
+      if (remoteFetch.isUnsafeMember(relPath)) {
+        throw new Error(`unsafe path in archive: ${relPath}`);
+      }
+      members.push({ relPath, absPath: abs });
+    }
+  };
+  walk(stageDir);
+  return members;
+}
+
+function hashFile(absPath) {
+  return crypto.createHash('sha256').update(fs.readFileSync(absPath)).digest('hex');
+}
+
+// Merge the build members of several staged archives, detecting conflicts.
+// `stages` is [{ name, dir }] (name = original filename, for reporting).
+//
+// Returns { members, moduleRoots, conflicts }:
+//  - members: one winning { relPath, absPath } per unique path.
+//  - moduleRoots: distinct module roots across all members (sorted).
+//  - conflicts: [{ path, archives }] — a path present in >1 archive with
+//    differing bytes. Empty when the merge is clean. Identical-content
+//    duplicates are merged silently (no conflict).
+function mergeStages(stages) {
+  const seen = new Map();       // relPath -> { hash, absPath, archives:Set }
+  const conflicts = new Map();  // relPath -> Set of archive names
+  for (const stage of stages) {
+    for (const { relPath, absPath } of collectBuildMembers(stage.dir)) {
+      const prior = seen.get(relPath);
+      if (!prior) {
+        seen.set(relPath, { hash: hashFile(absPath), absPath, archives: new Set([stage.name]) });
+        continue;
+      }
+      const hash = hashFile(absPath);
+      if (hash === prior.hash) {
+        prior.archives.add(stage.name); // identical duplicate — fine
+        continue;
+      }
+      let names = conflicts.get(relPath);
+      if (!names) { names = new Set(prior.archives); conflicts.set(relPath, names); }
+      names.add(stage.name);
+    }
+  }
+  const members = [...seen.entries()].map(([relPath, v]) => ({ relPath, absPath: v.absPath }));
+  const roots = new Set(members.map(m => remoteFetch.moduleRootOf(m.relPath)));
+  return {
+    members,
+    moduleRoots: [...roots].sort(),
+    conflicts: [...conflicts.entries()]
+      .map(([p, names]) => ({ path: p, archives: [...names].sort() }))
+      .sort((a, b) => a.path.localeCompare(b.path)),
+  };
+}
+
+// Copy the `<moduleRoot>/build` subtrees from every stage into projectRoot.
+// Only `build/` dirs are ever written, so source and goldens are never
+// clobbered. When no conflict was detected, cross-stage overwrites of a shared
+// path are byte-identical and therefore safe. Returns the number of build dirs
+// copied.
+function overlayBuildDirs(stages, moduleRoots, projectRoot) {
+  let copied = 0;
+  for (const stage of stages) {
+    for (const root of moduleRoots) {
+      const src = root ? path.join(stage.dir, root, 'build') : path.join(stage.dir, 'build');
+      let isDir = false;
+      try { isDir = fs.statSync(src).isDirectory(); } catch {}
+      if (!isDir) continue;
+      const dest = root ? path.join(projectRoot, root, 'build') : path.join(projectRoot, 'build');
+      fs.cpSync(src, dest, { recursive: true, force: true, dereference: false });
+      copied++;
+    }
+  }
+  return copied;
+}
+
+function cleanupStage(dir) {
+  if (!dir) return;
+  try { fs.rmSync(dir, { recursive: true, force: true }); } catch {}
+}
+
+function cleanupStages(stages) {
+  if (!stages) return;
+  for (const s of stages) cleanupStage(s.dir);
+}
+
+// Remove a raw upload's temp dir (each upload lives in its own mkdtemp dir).
+function cleanupUpload(uploadPath) {
+  if (!uploadPath) return;
+  try { fs.rmSync(path.dirname(uploadPath), { recursive: true, force: true }); } catch {}
+}
+
+module.exports = {
+  MAX_UPLOAD_BYTES,
+  MAX_UPLOAD_FILES,
+  sniffFormat,
+  extractArchive,
+  collectBuildMembers,
+  hashFile,
+  mergeStages,
+  overlayBuildDirs,
+  cleanupStage,
+  cleanupStages,
+  cleanupUpload,
+};

src/remoteFetch.js

@@ -210,4 +210,6 @@ module.exports = {
   isBuildMember,
   moduleRootOf,
   buildDirsForModules,
+  // shared with localArchive.js (raw upload byte cap)
+  byteCapTransform,
 };