fix: implement lib/retention module, resolve server.ts startup crash - #440
Merged
Osuochasam merged 3 commits intoAug 30, 2026
Merged
Conversation
Closes Open-audit-foundation#402 Two broken references in server.ts caused a crash on every boot: - import { startRetentionScheduler } from './lib/retention/scheduler' pointed at a module that didn't exist. - schedulePruner() on line 174 was called but never imported or defined anywhere in the repository. Root cause: two people implemented the same retention feature independently. startRetentionScheduler() was the scheduler entry-point; schedulePruner() was a duplicate one-shot prune call. Consolidated into a single coherent lib/retention/ module. Changes: - lib/retention/pruner.ts — pure pruneOldData() function; batch-delete via findMany(take)+deleteMany(id in) to bound transaction size; dry-run mode; structured JSON log output (counts, oldest/newest per table, duration); batchCap distributes budget across Event, DeadLetterEvent, WebhookDelivery in order - lib/retention/scheduler.ts — node-cron wrapper; reads RETENTION_ENABLED, RETENTION_DAYS, RETENTION_CRON_SCHEDULE, ARCHIVE_BATCH_SIZE; returns undefined (no-op) when RETENTION_ENABLED=false - lib/retention/index.ts — clean re-export surface - scripts/retention.ts — CLI with --dry-run / --days / --batch-cap - package.json — retention and retention:dry-run npm scripts - server.ts — added missing imports (persistExecutionDag, startRetentionScheduler) and startRetentionScheduler() call - .env.example — documented RETENTION_ENABLED variable Tests (6, all passing, zero regressions): - pruner: age threshold, batch cap, dry-run no-delete, lt boundary - scheduler: RETENTION_ENABLED=false no-op, enabled returns stoppable task
Contributor
Author
|
kindly review and merge |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Closes #402
Two broken references in
server.tscaused the process to crash on every boot vianpm run dev:ws:Root cause
Two independent retention implementations were left in-flight.
startRetentionSchedulerwas the scheduler entry-point;schedulePrunerwas a separate one-shot prune call. Neither was complete. This PR consolidates them into a single coherentlib/retention/module and removes the duplicate call site.Changes
lib/retention/pruner.ts(new)Pure
pruneOldData(options)function, injectedPrismaClientfor testability.findMany({ take, orderBy })+deleteMany({ id: { in: [...] } })— Prisma has no nativeDELETE ... LIMIT, so row IDs are fetched first, keeping the transaction short and bounded.batchCapbudget distributed acrossEvent → DeadLetterEvent → WebhookDeliveryin order; total deleted never exceeds cap.deleteManycalls are skipped. Reports exactly what would be removed.createdAtper table, wall-clock duration — for operator auditing.lib/retention/scheduler.ts(new)node-cronwrapper that callspruneOldDataon a configurable schedule.RETENTION_ENABLED,RETENTION_DAYS,RETENTION_CRON_SCHEDULE,ARCHIVE_BATCH_SIZE.undefined(no-op) whenRETENTION_ENABLED=false— tested.lib/retention/index.ts(new)Re-exports the public surface (
startRetentionScheduler,pruneOldData,logPruneResult, types).scripts/retention.ts(new)CLI script backing the two npm scripts below.
Flags:
--dry-run,--days <n>,--batch-cap <n>. Env vars (RETENTION_DAYS,ARCHIVE_BATCH_SIZE) used as defaults.package.jsonAdded:
server.tsimport { persistExecutionDag }(was used on line 73 but never imported — pre-existing compile error).import { startRetentionScheduler }and thestartRetentionScheduler()call inapp.prepare().schedulePruner()(undefined, duplicate intent)..env.exampleDocumented
RETENTION_ENABLED(default:true) in the existing Data Retention section.Acceptance criteria check
npm run dev:wsboots without throwing on either retention calllib/retention/module,schedulePrunerremovedRETENTION_ENABLED=falsefully disables pruningscheduler.test.tsnpm run retention:dry-runreports without touching DBdeleteManycalls in dry-run)npm run retentionprunes respecting batch cappruner.test.tsnpm run testTests
Out of scope (flagged for follow-up)
.csv.gzor S3-compatible sink before thedeleteMany. ThelogPruneResultstructured output already captures the timestamp range affected, making reconstruction feasible if logs are retained.