Skip to content

refactor(evolve): collapse three scanners behind one runScanner core + collectFn seam#326

Merged
sethvoltz merged 1 commit into
mainfrom
seth/deepen-evolve-scanners
Jun 26, 2026
Merged

refactor(evolve): collapse three scanners behind one runScanner core + collectFn seam#326
sethvoltz merged 1 commit into
mainfrom
seth/deepen-evolve-scanners

Conversation

@sethvoltz

Copy link
Copy Markdown
Owner

What

Collapses the three hand-copied evolve scanner pipelines (scan-friction, scan-preferences, scan-dreaming) into one deep runScanner core fed by per-scanner Taxonomy adapters, and adds a second injectable seam (collectFn) so the full collect → batch → score → bucket pipeline is testable without a DB — closing the real test gap.

Why

The three scanners shared the same four-stage shape with ~35–40 lines of identical scaffolding hand-copied into each file. The model seam (scoreFn) already existed, but turn-collection hit Postgres directly, so no test drove the full pipeline — only the isolated bucket functions were covered.

Changes

  • collect.ts (new) — extracted OrchestratorTurn, collectOrchestratorTurns, dbTurnIdToLine to a neutral module (breaks the run-scannerscan-friction cycle).
  • run-scanner.ts (new) — runScanner<P, T, Ctx> + the Taxonomy interface + RunScannerOptions + SCAN_BATCH_SIZE, plus de-duplicated truncate/clamp/severityFor/severityRank. The core only ever touches the turn_id join key on T (via a ScoredRow constraint); every domain distinction stays inside each bucket.
  • scan-*.ts — each is now a Taxonomy + a thin scanX = (opts) => runScanner(taxonomy, opts, ctx) wrapper. Public function names/signatures unchanged. Dreaming's richer bucketing (payload-in-first-pointer, separate note-pointer cap, recall→severity bump, promote/reinforce verb) preserved whole; DreamEvidence threaded through as the opaque ctx.
  • run-scanner.test.ts (new) — end-to-end pipeline tests the seam unlocks: cross-session swap, default-30 + override batching, cross-batch merge into one bucket, log-and-continue, per-category caps, dreaming reinforce-merge + payload round-trip + evidence threading.

Scope

Pure internal refactor — no Signal shape change, no propose.ts/rank.ts change, public exports unchanged.

Verification

  • Package tsc clean; 140 tests pass (128 preserved verbatim + 12 new); daemon evolve-dreaming consumer green; prettier clean.
  • Adversarial multi-agent review confirmed behavior byte-identical (md5-verified) across all three bucket functions, all three scoring prompts, and the collect logic; export surface preserved; core extraction leak-free. The 4 real issues it surfaced (a stale OrchestratorTurn test import, two thin-coverage batching gaps, one redundant assertion) are fixed in this branch.

🤖 Generated with Claude Code

https://claude.ai/code/session_01SeavKpZNAxkrokvKhQF8Ai

…+ collectFn seam

Replace the three hand-copied scanner pipelines (friction / preferences /
dreaming) with one deep `runScanner` core fed by per-scanner `Taxonomy`
adapters. The collect → batch → score → bucket scaffolding now lives once;
each scanner contributes only its payload projection, default LLM scorer,
and (unchanged) bucketing rule.

- New `collect.ts`: extracted OrchestratorTurn + collectOrchestratorTurns +
  dbTurnIdToLine to a neutral module (breaks the run-scanner↔scan-friction cycle).
- New `run-scanner.ts`: runScanner<P,T,Ctx> + Taxonomy + RunScannerOptions +
  SCAN_BATCH_SIZE, plus de-duplicated truncate/clamp/severityFor/severityRank.
  Adds a second injectable seam, collectFn (defaults to collectOrchestratorTurns),
  so the whole pipeline is testable without a DB. The core only touches the
  turn_id join key on T (ScoredRow constraint); all domain variation stays in bucket.
- scan-*.ts: each is now a Taxonomy + a thin `scanX = (opts) => runScanner(...)`
  wrapper. Public names/signatures unchanged. Dreaming's richer bucketing
  (payload-in-first-pointer, separate note cap, recall→severity bump, promote/
  reinforce verb) preserved whole; DreamEvidence threaded as the opaque ctx.
- New run-scanner.test.ts: end-to-end pipeline coverage that the collectFn seam
  unlocks (cross-session swap, default+override batching, cross-batch merge,
  log-and-continue, per-category caps, dreaming reinforce-merge + payload round-trip).

Pure internal refactor: no Signal shape change, no propose.ts/rank.ts change.
Behavior verified byte-identical across all three buckets, prompts, and collect.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01SeavKpZNAxkrokvKhQF8Ai
@sethvoltz sethvoltz enabled auto-merge (squash) June 26, 2026 01:56
@sethvoltz sethvoltz merged commit aeb617c into main Jun 26, 2026
6 checks passed
@sethvoltz sethvoltz deleted the seth/deepen-evolve-scanners branch June 26, 2026 02:01
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant