fix(transactions): prevent TTL index from deleting confirmed purchases and donations (#94)

[samjay8]

Closes #94

Summary

• Fixes a data-loss bug: confirmed on-chain purchases and donations were being permanently deleted ~30 minutes after creation by MongoDB's TTL reaper, because the Transaction collection had a blanket TTL index on expiresAt and the schema default stamped a 30-minute expiry on every row regardless of status.
• Scopes the TTL index to status: "pending" via partialFilterExpression, makes the expiresAt default conditional on status, and adds a pre("save") schema hook that clears expiresAt for any terminal state — so reaping is structurally impossible for non-pending rows even if a future code path forgets.
• Adds an idempotent backfill migration that rescues existing non-pending rows and rebuilds the index, plus docs/transaction-lifecycle.md documenting the status → expiresAt invariant.

Root cause

src/models/Transaction.js declared:

expiresAt: { type: Date, default: () => new Date(Date.now() + 30 * 60 * 1000) } // every row
transactionSchema.index({ expiresAt: 1 }, { expireAfterSeconds: 0 })            // blanket TTL

expiresAt defaults to now + 30 minutes and was never cleared when a transaction was confirmed. The intent was to garbage-collect abandoned pending checkouts, but the index made no distinction by status, so:

• Confirmed purchases/donations were reaped ~30 minutes after creation — the row proving the buyer paid and the educator earned silently vanished.
• Earnings linkage was orphaned, not deleted. recordSaleEarnings (src/services/payoutService.js) credits EducatorBalance and writes a LedgerEntry whose txRef is stellarTxHash || _id. When the source Transaction was TTL-reaped, the balance and ledger entry survived but pointed at a row that no longer exists — leaving unauditable, unreconcilable earnings and no proof-of-purchase for buyer access.
• Worker-created confirmed rows were born doomed. createConfirmedDonation / createConfirmedPurchase (reconciliation service) built status: "confirmed" records without overriding expiresAt, so the schema default gave an already-confirmed record a 30-minute lifespan.

Changes

1. Partial TTL index (structural guarantee)

src/models/Transaction.js — the blanket index is now:

transactionSchema.index(
  { expiresAt: 1 },
  { expireAfterSeconds: 0, partialFilterExpression: { status: "pending" } }
);

MongoDB's TTL monitor only considers documents matching the partial filter, so a non-pending document is never a deletion candidate — even if it somehow still carries an expiresAt.

2. Conditional schema default

The expiresAt default only produces a timestamp when the status is pending (or unset at creation):

default: function () {
  return this.status === "pending" || !this.status
    ? new Date(Date.now() + 30 * 60 * 1000)
    : undefined;
}

Records created directly in a terminal state — e.g. worker-created confirmed donations/purchases — are never born with an expiry.

3. Pre-save hook (runtime defense in depth)

A pre("save") middleware clears expiresAt whenever a document is saved in a terminal status (confirmed, failed, expired, refunded, disputed). Even if a controller forgets to unset it explicitly, the model enforces the invariant — this is the key hardening added on top of the explicit $unsets.

4. Explicit $unset/expiresAt = undefined on every terminal transition

• src/controllers/stellar/paymentController.js — submitPayment: validation failure, Stellar error, verification failure, and confirmation paths; cancelTransaction: $set status to expired + $unset: { expiresAt: 1 }.
• src/controllers/stellar/donationController.js — submitDonation: same four terminal paths.
• src/controllers/stellar/refundController.js — submitRefund (→ refunded) and escalateDispute (→ disputed) via findByIdAndUpdate with $unset: { expiresAt: 1 }.
• src/services/stellar/reconciliationService.js — promoteTransaction clears before save; createConfirmedDonation / createConfirmedPurchase explicitly pass expiresAt: undefined (belt-and-braces alongside the conditional default).
• src/jobs/handlers.js — verifyPaymentOnChain clears on both the failed and confirmed paths.

5. Idempotent backfill migration

src/migrations/fixTtlTransactionExpiry.js (run via node src/migrations/fixTtlTransactionExpiry.js):

1. $unsets expiresAt on every existing non-pending transaction so already-confirmed records that still carry an expiry are rescued before the next TTL sweep.
2. Drops the blanket { expiresAt: 1 } TTL index (if present without the partial filter) and recreates it as the pending-scoped partial index. A schema .index() edit does not alter an already-built index, so this must be explicit.
3. Idempotent: a second run touches no documents and the createIndex is a no-op.

6. Lifecycle documentation

docs/transaction-lifecycle.md — documents the invariant, the status → expiresAt mapping table, the three defense-in-depth layers, and migration instructions, so the guarantee is discoverable.

Testing

• New test/ttlTransactionExpiry.test.js (MongoMemoryServer, mirrors test/reconciliation.test.js conventions) — 9 tests:
  • pending transactions get a 30-minute default expiry
  • direct creation in any terminal state (confirmed, failed, expired, refunded, disputed) produces no expiresAt
  • transient submitted/retrying states keep their expiry
  • the pre-save hook clears a stale/past expiresAt on transition to a terminal status (simulating a code path that forgets)
  • the migration unsets expiresAt on legacy non-pending rows (inserted directly to bypass hooks), keeps pending rows, and is idempotent (second run modifies 0)
  • the migration replaces a simulated blanket TTL index with the pending-scoped partial index, leaving exactly one expiresAt index
  • the index spec carries partialFilterExpression: { status: "pending" } and expireAfterSeconds: 0
  • a confirmed row survives past its original expiresAt (survival check asserts expiresAt unset + index scoping rather than triggering real TTL deletion)
• Existing suites updated with confirm-path assertions:
  • test/reconciliation.test.js — asserts promoteTransaction (reconciliation path) leaves expiresAt undefined
  • test/stellarPaymentController.test.js — asserts the submitPayment confirm path leaves expiresAt undefined
• Full suite: npm test → 33 suites, 323 tests passed (CI-green; the CI boot check also runs node --check on all sources and hits /health).

Related PR

This issue already has an implementation from ghzhost in #97. This PR is an independent take on the same fix, reviewed against the same acceptance criteria, that adds schema-level enforcement (pre-save hook) and explicit worker-created-row handling on top of the explicit $unsets, with the migration and tests built to match. Please compare the two and link them so the assignee's work is credited alongside — happy to fold in anything from #97 that reviewers prefer.

Checklist

• Confirmed transaction survives past its original expiresAt (asserted via unset + index scoping; no fake-timer real-TTL trigger)
• Confirming via both the submit path and the reconciliation path leaves expiresAt unset
• Worker-created confirmed donations and purchases are created without an expiresAt
• TTL index is partial on status: "pending"; test asserts the index spec
• Backfill migration clears expiresAt on existing non-pending rows and is idempotent
• Migration dropIndexes the blanket {expiresAt:1} index and createIndexes the partial one in the same run
• CI stays green (33 suites / 323 tests passed; boot check)

[coderabbitai]

Important

Review available on request

• 🔍 Trigger review

Reviews should be triggered manually for repositories with fewer than 10 stars. Select Trigger review above or comment @coderabbitai review to review the latest changes. For a full review, comment @coderabbitai full review.

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro Plus

Run ID: 3a035ff7-d506-416e-97d9-2c07ef178299

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}