Skip to content

feat: persistent applied-action ledger — replay-safe reindex#52

Merged
lucca65 merged 1 commit into
masterfrom
fix/processed-action-ledger
Jul 2, 2026
Merged

feat: persistent applied-action ledger — replay-safe reindex#52
lucca65 merged 1 commit into
masterfrom
fix/processed-action-ledger

Conversation

@lucca65

@lucca65 lucca65 commented Jul 2, 2026

Copy link
Copy Markdown
Member

Problem (the structural one)

Every data incident this year — duplicate claims/orders, the notification spam, the stale claim statuses, the phantom action ids — traced back to one gap: the indexer has no durable record of which chain actions it already applied. The in-memory seenGlobalSeqs Set dies on restart/seekToBlock, so any reindex re-invokes every updater over existing rows, and correctness depends entirely on hand-written per-updater guards. Miss one (as happened with reward/verifyClaim) and a reindex silently corrupts prod.

Fix

A _processed_actions ledger keyed on global_action_seq (globally unique; account_action_seq is per contract and collides between cambiatus.cm and cambiatus.tk):

  • GetActionsReader forwards global_action_seq as payload.globalSequence (it already fetched it for in-memory dedup).
  • app.js creates the table at boot (CREATE TABLE IF NOT EXISTS) — indexer infra, owned here like demux's _index_state / _block_number_txid, no backend migration needed.
  • updaters.js wraps every updater (incl. future ones, via a map at export): INSERT … ON CONFLICT DO NOTHING RETURNING atomically claims the seq inside the same block-level transaction demux opened for the updater's writes — ledger row and domain writes commit or roll back together. Already-claimed seq → skip, logged.

Per-updater guards stay as the second layer: they cover history processed before the ledger existed (which has no rows), and the pre-existing window where an updater's inner db.withTransaction commits on a separate connection.

Verified

  • StandardJS + node --check clean.
  • Smoke test against the dev DB: table auto-creates; first claim of a seq returns the row, replayed claim returns null → updater skipped.
  • Deploy is just pull + restart: the table appears on first boot, no data migration.

After this merges + deploys, block-range reindexes become a sanctioned operation (runbook to follow in the workspace docs).

🤖 Generated with Claude Code

Every data incident this year traced to the same structural gap: the
indexer has no durable record of which chain actions it already applied.
The in-memory seenGlobalSeqs Set dies on restart/seek, so any
reindex/seek re-invokes every updater over existing rows, and safety
depends entirely on hand-written per-updater guards (miss one — as with
reward/verifyClaim — and a reindex silently corrupts).

Add a _processed_actions ledger keyed on global_action_seq (unique
across contracts; account_action_seq is per contract and collides):

- GetActionsReader forwards global_action_seq as payload.globalSequence.
- app.js creates the table at boot (indexer infra, owned here like
  demux's _index_state / _block_number_txid).
- updaters.js wraps every updater (including future ones, via a map at
  export): INSERT .. ON CONFLICT DO NOTHING RETURNING atomically claims
  the seq inside the same block-level transaction demux opened for the
  updater's writes, so ledger row and domain writes commit or roll back
  together; an already-claimed seq skips the updater entirely.

Per-updater guards stay on as the second layer — they also cover
history processed before this ledger existed. With this, a block-range
reindex becomes a sanctioned, safe operation.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@lucca65
lucca65 merged commit fdf2aee into master Jul 2, 2026
1 check passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant