feat(blockchain): Replace in-memory Maps with SQLite persistence

[Kekule17]

Closes #138

Summary

This PR replaces the three module-level Map objects in src/blockchain/repository.ts
(txStore, eventStore, trackerStore) with a durable SQLite backend powered by
better-sqlite3. All state written by the
Soroban ledger indexer now survives process restarts, Vercel cold starts, container
restarts, and pm2 recycling.

The public API surface of blockchainTransactionRepo, contractEventRepo, and
rollupTrackerRepo is 100% unchanged — every method signature is identical to the
previous in-memory implementation. No callers required modification.

---

Problem

The original implementation stored all ledger indexer state in plain in-process Map
objects. This caused several production issues:

Symptom	Root cause
GET /api/v1/admin/health always reported "behind by N ledgers" after a restart	lastProcessedLedger reset to 0 on every cold start
contractEventRepo.findUnprocessed() returned [] after restart	All ContractEvent rows lost on restart — fraud-detection pipeline silently stalled
Reorg detection flagged every previously-indexed ledger as orphaned	blockHash reset to '' on restart — every stored ledger looked like a reorg
Duplicate ContractEvent rows on re-scan after restart	Cursor reset caused overlap; deduplication only worked within a single process lifetime

The comment in the original repository.ts explicitly acknowledged this:

"This project has no persistent database. All state is kept in module-level in-memory Maps."

---

Solution

New file: src/blockchain/db.ts

A singleton Database instance backed by better-sqlite3 (synchronous, embeddable,
no network dependency). Key properties:

• WAL mode enabled on file-based databases via PRAGMA journal_mode=WAL — writers
  proceed concurrently with readers; an fsync only occurs at checkpoint, not on every
  write. A mid-scan crash loses at most the rows in the in-flight batch.
• BLOCKCHAIN_DB_PATH environment variable controls the database file path.
  Defaults to .data/blockchain.db in production. Set to :memory: in jest.setup.js
  so every Jest worker gets a fully isolated, zero-disk-I/O store.
• __resetDb() drops and recreates all three tables — used by __clearAllStores()
  in beforeEach hooks, giving each test the same clean-slate guarantee that Map.clear()
  previously provided.
• __closeDb() closes the singleton connection and clears the module-level reference,
  enabling cold-start simulation in integration tests without re-spawning a process.
• The .data/ directory is created automatically on first use — no manual setup required.

Schema:

-- Primary key: tx_hash (globally unique on Stellar)
CREATE TABLE IF NOT EXISTS blockchain_transactions (
  id               TEXT    NOT NULL,
  tx_hash          TEXT    NOT NULL PRIMARY KEY,
  block_number     INTEGER NOT NULL,
  block_hash       TEXT    NOT NULL DEFAULT '',
  status           TEXT    NOT NULL,
  "from"           TEXT    NOT NULL DEFAULT '',
  "to"             TEXT    NOT NULL DEFAULT '',
  amount           TEXT    NOT NULL DEFAULT '0',
  fee              TEXT    NOT NULL DEFAULT '0',
  operation_type   TEXT    NOT NULL DEFAULT '',
  memo             TEXT,
  created_at       TEXT    NOT NULL,
  indexed_at       TEXT    NOT NULL,
  processed        INTEGER NOT NULL DEFAULT 0
);

-- Composite primary key matches the existing eventCompositeKey() function
CREATE TABLE IF NOT EXISTS contract_events (
  id               TEXT    NOT NULL,
  tx_hash          TEXT    NOT NULL,
  contract_address TEXT    NOT NULL,
  event_name       TEXT    NOT NULL,
  ledger_sequence  INTEGER NOT NULL,
  event_index      INTEGER NOT NULL,
  parameters       TEXT    NOT NULL DEFAULT '{}',
  created_at       TEXT    NOT NULL,
  processed        INTEGER NOT NULL DEFAULT 0,
  PRIMARY KEY (tx_hash, contract_address, event_name, ledger_sequence, event_index)
);

-- Keyed by logical cursor type (e.g. 'soroban_indexer', 'soroban_events')
CREATE TABLE IF NOT EXISTS rollup_trackers (
  type                   TEXT    NOT NULL PRIMARY KEY,
  last_processed_ledger  INTEGER NOT NULL DEFAULT 0,
  last_event_cursor      TEXT    NOT NULL DEFAULT '',
  updated_at             TEXT    NOT NULL
);

Modified file: src/blockchain/repository.ts

Rewrote all three repository objects to use prepared SQL statements. Behavioural
contracts preserved exactly:

• blockchainTransactionRepo.upsert — INSERT OR IGNORE followed by UPDATE in a
  single SQLite transaction. The id and createdAt columns are written only on the
  first insert and never overwritten on subsequent upserts, matching the original
  existing?.id ?? data.id ?? randomUUID() and existing?.createdAt ?? data.createdAt
  logic.

• contractEventRepo.upsert — INSERT OR IGNORE against the composite primary key.
  Duplicate events silently no-op and the existing row is returned, identical to the
  original if (existing) return existing guard.

• contractEventRepo.updateTxHash — wrapped in a SQLite transaction: delete the
  sentinel row, check whether the real-hash row already exists (parallel resolution
  race), and insert the updated row preserving id, parameters, createdAt, and
  processed.

• rollupTrackerRepo.upsert — INSERT ... ON CONFLICT(type) DO UPDATE SET ....
  Partial updates (supplying only lastProcessedLedger or only lastEventCursor) are
  preserved by reading the existing row first and merging the supplied fields.

• __clearAllStores() — now delegates to __resetDb() (DROP + CREATE), which gives
  identical per-test isolation to the previous Map.clear() calls.

• eventCompositeKey() — function is unchanged and still exported; callers that use
  the string key directly are unaffected.

Modified file: jest.setup.js

process.env.BLOCKCHAIN_DB_PATH = ':memory:'

One line added at the top of the global Jest setup. Every test file automatically gets an
in-memory SQLite database with zero disk I/O and per-test isolation via __resetDb().

New file: src/blockchain/concurrent-writer-helper.js

A plain CommonJS script executed by child_process.fork in the AC4 concurrent-writer
integration test. It opens its own better-sqlite3 connection to a shared file-based DB
and inserts 1 000 rows inside a single transaction. Sets busy_timeout=30000 so the
SQLite write-lock retry loop handles contention transparently without SQLITE_BUSY
errors.

New file: src/blockchain/__tests__/repository.persistence.test.ts

Twelve integration tests covering the acceptance criteria for the migration:

Test	What it proves
AC1 — cursor written before close is readable after reopen	rollupTrackerRepo.find() returns the persisted cursor after __closeDb() + getDb() (cold-start simulation)
AC1 — find returns undefined after restart for uncreated cursor	No phantom rows on fresh DB
AC1 — upsert-then-restart preserves the latest cursor	Latest value wins across restart, not a stale one
AC2 — count before and after restart are identical	blockchainTransactionRepo.count() = 50 both before and after __closeDb()
AC2 — row contents are preserved across restart	Field values (blockNumber, blockHash, memo, processed) survive close+reopen
AC3 — committed batches survive simulated mid-scan crash	Rows from batches 1 and 2 present after closing mid-batch 3; cursor at 200
AC3 — WAL mode is active on file-based databases	PRAGMA journal_mode returns wal
AC4 — two concurrent fork workers produce exactly 2 000 rows	child_process.fork × 2 × 1 000 inserts = count() === 2000
AC5 — 10 000 sequential upserts complete in under 5 seconds	~700 ms observed (×7 headroom)
AC5 — 10 000 batched inserts complete in under 1 000 ms	~45 ms observed — demonstrates transactional throughput
ContractEvent — events readable after cold-start	parameters JSON round-trips correctly
ContractEvent — unprocessed events survive cold-start	findUnprocessed() returns all 5 rows after restart

---

Test results

Test Suites: 3 passed (repository.test.ts, soroban.indexer.test.ts, repository.persistence.test.ts)
Tests:       93 passed  (81 pre-existing + 12 new)

No previously passing tests were broken. The pre-existing failure in
src/app/campaigns/[id]/page.test.tsx (a Next.js SWC transform issue in UI code) is
unrelated to this PR and was present on master before this branch.

---

Acceptance criteria checklist

• rollupTrackerRepo.find('soroban_indexer') returns the persisted cursor after a
  simulated cold-start — verified by AC1 tests
• blockchainTransactionRepo.count() returns the same value before and after a
  process restart — verified by AC2 tests
• A kill-between-batches crash test demonstrates that only the in-flight batch is
  lost — verified by AC3 test
• Two concurrent writer processes writing 1 000 non-overlapping txHash values each
  produce exactly 2 000 distinct rows — verified by AC4 test (84 ms, no deadlocks)
• All 81 existing tests in repository.test.ts and soroban.indexer.test.ts pass
  without modification
• npm run type-check succeeds with zero new errors in blockchain files
• WAL mode confirmed active (PRAGMA journal_mode = wal) on file-based databases
• 10 000 sequential upserts complete in ~700 ms — well under the 5 s threshold
• Public method signatures (upsert, find, findByHash, findByLedger,
  orphanByLedger, findUnprocessed, count, __clear, __all) are unchanged

---

Configuration

Production

Set BLOCKCHAIN_DB_PATH to the desired file path (or omit to use the default
.data/blockchain.db):

# .env.local or container environment
BLOCKCHAIN_DB_PATH=/var/data/blockchain.db

The directory is created automatically on first start. WAL mode and synchronous=NORMAL
are applied automatically on file-based databases.

Concurrent instances (Vercel / Node cluster)

SQLite WAL mode serialises concurrent writers at the OS level using file locking. Each
serverless function instance opens its own better-sqlite3 connection; the busy_timeout
setting (inherited from the WAL pragma path) causes writers to retry for up to 30 seconds
before failing. For deployments with high write concurrency against a network-mounted
volume, an external store (e.g. Redis, Postgres) remains the recommended upgrade path —
this PR removes the in-memory-only blocker.

---

Files changed

File	Change
src/blockchain/db.ts	New — SQLite singleton, WAL init, schema DDL, __resetDb, __closeDb
src/blockchain/repository.ts	Modified — all three repos rewritten with SQL; API surface unchanged
src/blockchain/concurrent-writer-helper.js	New — fork worker for AC4 concurrent-write test
src/blockchain/__tests__/repository.persistence.test.ts	New — 12 persistence integration tests
jest.setup.js	Modified — sets BLOCKCHAIN_DB_PATH=':memory:' for test isolation
package.json	Modified — adds better-sqlite3@11.10.0, @types/better-sqlite3@7.6.13
package-lock.json	Modified — lockfile updated

---

Out of scope (not changed in this PR)

• Campaign data, beneficiary state, or any other store outside src/blockchain/
• Soroban indexer fetch strategy, batch size, or reorg logic
• API endpoint signatures or route handlers
• src/blockchain/types.ts field shapes
• Any distributed cache or Redis integration