Skip to content

feat(indexer): implement retention and pruning policy for operations and snapshots (#214) - #375

Merged
blockchain-maxis merged 4 commits into
blockchain-maxis:mainfrom
ntttttbl123-blip:feat/retention-pruning-policy-214
Sep 3, 2026
Merged

feat(indexer): implement retention and pruning policy for operations and snapshots (#214)#375
blockchain-maxis merged 4 commits into
blockchain-maxis:mainfrom
ntttttbl123-blip:feat/retention-pruning-policy-214

Conversation

@ntttttbl123-blip

Copy link
Copy Markdown
Contributor

Closes #214

Summary of Changes

Implements a configurable retention and periodic background pruning policy for historical operations and contract snapshots in the indexer, and documents the operational bounds, storage trade-offs, and cost implications.

Key Changes

  • Pruning Worker (�pps/indexer/src/workers/prune.ts): Periodically deletes \Operation\ rows older than \operationsRetentionDays\ (default: 90 days) and \ContractSnapshot\ rows older than \snapshotsRetentionDays\ (default: 30 days). Setting either to 0 retains records indefinitely.
  • Config & Main Loop (�pps/indexer/src/config.ts, �pps/indexer/src/index.ts): Added \INDEXER_OPERATIONS_RETENTION_DAYS, \INDEXER_SNAPSHOTS_RETENTION_DAYS, and \INDEXER_PRUNE_INTERVAL_MS\ configuration with periodic maintenance execution and structured metrics logging.
  • Policy Documentation (docs/RETENTION.md): Documented the retention rationale, default active windows, pruning cadence, and storage/cost trade-offs.
  • Documentation Updates (docs/ENVIRONMENT.md, docs/INDEXER.md, .env.example): Added reference documentation for all retention variables.
  • Automated Tests (�pps/indexer/src/workers/prune.test.ts): Unit test coverage for operations and snapshot pruning logic.

Verification Status

  • \pnpm --filter @signet/indexer test: All 50 tests passed cleanly.

  • ode scripts/check-docs.mjs: Passed cleanly (23 markdown files, 30 env vars, no drift).

  • ode scripts/check-contract-errors.mjs: Passed.

@drips-wave

drips-wave Bot commented Aug 31, 2026

Copy link
Copy Markdown

@ntttttbl123-blip Great news! 🎉 Based on an automated assessment of this PR, the linked Wave issue(s) no longer count against your application limits.

You can now already apply to more issues while waiting for a review of this PR. Keep up the great work! 🚀

Learn more about application limits

@netlify

netlify Bot commented Aug 31, 2026

Copy link
Copy Markdown

Deploy Preview for stellar-signet ready!

Name Link
🔨 Latest commit 9f7f600
🔍 Latest deploy log https://app.netlify.com/projects/stellar-signet/deploys/6a995fa492e428000886da73
😎 Deploy Preview https://deploy-preview-375--stellar-signet.netlify.app
📱 Preview on mobile
Toggle QR Code...

QR Code

Use your smartphone camera to open QR code link.

To edit notification comments on pull requests, go to your Netlify project configuration.

@vercel

vercel Bot commented Aug 31, 2026

Copy link
Copy Markdown

@ntttttbl123-blip is attempting to deploy a commit to the blockchainmaxis-8449's projects Team on Vercel.

A member of the Team first needs to authorize it.

@blockchain-maxis blockchain-maxis left a comment

Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The retention worker itself reads well — retention expressed in days with 0 meaning disabled, the cutoff computed from an injected now so the tests don't depend on the clock, and operations and snapshots given independent windows. docs/RETENTION.md plus the .env.example / docs/ENVIRONMENT.md entries are the right supporting changes, and the docs guard is happy with them.

One thing blocks it: lint • typecheck • test • build fails on this branch, and main is green.

src/workers/prune.test.ts(52,16): error TS2532: Object is possibly 'undefined'.
src/workers/prune.test.ts(72,16): error TS2532: Object is possibly 'undefined'.

Both are the array index right after the length assertion:

assert.equal(operations.length, 1);
assert.equal(operations[0].id, 'op-recent');   // line 52
...
assert.equal(snapshots.length, 1);
assert.equal(snapshots[0].id, 's-recent');     // line 72

noUncheckedIndexedAccess is on for this workspace, so operations[0] is T | undefined regardless of the assertion above it — assert.equal isn't a type guard. operations[0]!.id / snapshots[0]!.id is the fix, matching how the existing indexer tests handle it.

pnpm --filter @signet/indexer typecheck reproduces both locally. Push that and I'll merge — nothing else here needs changing.

…xedAccess

tsc --noEmit failed with TS2532 on operations[0].id and snapshots[0].id
because tsconfig.base.json sets noUncheckedIndexedAccess.
@blockchain-maxis
blockchain-maxis merged commit f74a1ad into blockchain-maxis:main Sep 3, 2026
10 of 11 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

No retention or pruning policy for indexed operations

2 participants