The service has no database — anchors, liquidity and settlements live in in-memory `Map`s and are destroyed on every restart

[Jagadeeshftw]

Priority: High  ·  Area: Persistence / architecture  ·  Est. effort: 12–16 h

📌 Problem

package.json lists exactly three runtime dependencies:

compression, cors, express

There is no database driver, no ORM, no migration tool. Every repository under src/repositories/ — anchorRepository.ts, liquidityRepository.ts, settlementRepository.ts — is backed by inMemoryRepository.ts, and settlementRepository.ts:13 keeps its secondary index as private readonly anchorIndex: Map<string, Set<number>> = new Map().

Consequences:

• Every restart wipes all state. A deploy, crash, or scale event destroys the settlement history, the registered anchors and all liquidity records.
• Horizontal scaling is impossible. Two replicas hold two disjoint datasets; which one answers a request decides what the client sees.
• Unbounded growth. Nothing evicts, so the process grows until it is restarted — which is also the only thing that clears it.

Settlements are financial records. A service that forgets them on restart cannot be operated.

🎯 Design decision required

Post your proposed schema and approach in a comment on this issue before writing code, and wait for maintainer agreement. This is too large to land unsolicited. State and defend:

1. Store choice. Postgres is the conventional answer for relational financial records; argue it against the alternatives rather than assuming it.
2. Migration tooling and how migrations run in CI, in development, and on deploy.
3. Interface preservation. The repository classes have method signatures that src/services/*.ts depend on. Keeping them stable means no service code changes — confirm that is achievable, and list any signature you must change.
4. Sequencing. All three repositories in one PR, or one at a time behind the existing interfaces? Argue which is reviewable.
5. Testing without a live database. The suite currently runs with no external dependency. How do you preserve that — an in-memory implementation retained for tests behind the same interface, or a containerised database in CI?

🧩 Requirements and context

• src/services/anchorService.ts, liquidityService.ts, settlementService.ts and quoteService.ts should require no changes. If one does, justify it.
• Schema must include the indexes the current code relies on — settlementRepository.ts's anchor index is the obvious one; find the rest by reading the query methods.
• Concurrency matters: two simultaneous settlements against the same liquidity must not both succeed if only one can be funded. State how your schema and transactions prevent that.
• All 42 existing test files must pass.
• Do not change API response shapes.
• Configuration must fail fast with a clear message when the database is unreachable — src/config.ts is the place.

🛠️ Suggested execution

1. Post the schema and approach; wait for agreement.
2. Add the driver and migration tooling; write migrations.
3. Implement one repository behind its existing interface; keep the in-memory one for tests if that is your choice.
4. Prove services are untouched.
5. Add a concurrency test for the double-spend case.

✅ Acceptance criteria

• The schema and approach were agreed on this issue before implementation.
• At least one repository is database-backed, with migrations committed.
• No file under src/services/ changed, or every change is justified.
• Indexes backing existing query methods exist in the schema.
• A test proves two concurrent settlements cannot both consume the same liquidity.
• All 42 test files pass, with no requirement for a live database if that was your stated approach.
• Startup fails fast and clearly when the database is unreachable.
• API response shapes are unchanged.

🚫 Out of scope

• Changing the public API surface.
• Migrating all three repositories, if you argued for sequencing.
• Adding caching.

🧪 Verification

npm ci
npm run lint
npm run build
npm test

📤 What your PR must include

1. A link to the agreed schema comment.
2. Your store, tooling, sequencing and test-strategy decisions with reasoning.
3. The concurrency test.
4. Confirmation services are unchanged.
5. Closes #<n> or Part of #<n>.

🔒 Security notes

Settlement records are the audit trail for value movement. Storing them only in process memory means there is no durable record of what the system did — a restart erases the evidence of every settlement executed before it. Beyond availability, that is an integrity and accountability failure: no reconstruction, no dispute resolution, and no way to detect after the fact that something went wrong.

📋 Guidelines

• Minimum 95% test coverage on changed lines
• Clear documentation
• Timeframe: 96 hours from assignment
• One logical change per commit; no merge commits

---

💬 Join our community

Working on this, or want to sanity-check your approach before you start? Come and ask — the maintainers are there and happy to help.

Telegram: https://t.me/Grainlify

[Baskarayelu]

I’d love to be assigned this issue and contribute across the relevant areas, whether it involves smart contracts, UI development, backend logic, design implementation, or overall system integration. I’m comfortable working across the full stack and can adapt to the project’s technical requirements to ensure a smooth and consistent contribution.

[grantfox-oss]

🦊 GrantFox — @Baskarayelu has been assigned to this issue as part of the Third Campaign campaign!

Next steps:

1. Open a Pull Request referencing this issue (e.g., Closes #220)
2. Your PR will be reviewed by the AnchorNet-Org maintainers

Good luck! Track your progress on GrantFox.

[Baskarayelu]

Proposed persistence architecture (before implementation)

I propose PostgreSQL as the durable store, using the official pg driver and SQL migrations managed by node-pg-migrate. PostgreSQL fits this domain better than SQLite (single-writer/file-oriented and awkward for horizontally scaled deploys), Redis (not the system of record and weak for relational audit queries), or a document store (the anchor/asset/settlement relationships and financial constraints are naturally relational). It also gives us row locks, transactional durability, and operational tooling for backups, replicas, and managed deployments.

Schema

The first migration would create:

anchors (
  id text primary key,
  name text not null,
  registered_at timestamptz not null,
  active boolean not null
)

liquidity_entries (
  anchor text not null references anchors(id),
  asset text not null,
  amount numeric(78,0) not null check (amount >= 0),
  updated_at timestamptz not null,
  primary key (anchor, asset)
)

settlements (
  id bigint generated always as identity primary key,
  anchor text not null references anchors(id),
  asset text not null,
  amount numeric(78,0) not null check (amount > 0),
  fee numeric(78,0) not null check (fee >= 0),
  status text not null check (status in ('pending','executed','cancelled')),
  created_at timestamptz not null,
  cancel_reason text
)

Indexes would mirror the existing repository access paths: settlements(anchor, id), settlements(asset, id), settlements(status, id), and liquidity_entries(asset, anchor). The primary keys cover anchor lookup and (anchor, asset) liquidity lookup. Settlement IDs remain monotonic so current list ordering is deterministic.

numeric(78,0) preserves the integer range represented by the current bigint values without converting financial amounts through JavaScript numbers. The driver layer will map these columns to strings/bigints at the repository boundary.

Interfaces and sequencing

I would migrate one repository at a time behind the current repository interfaces, beginning with anchors, then liquidity, then settlements. The service classes and public response shapes should remain unchanged. I expect the existing method signatures to remain unchanged; if the concurrency boundary requires an additional internal repository method, it will be kept private to the repository composition rather than pushed into the service API. Each step can therefore be reviewed and tested independently, while the final PR can include all three migrations if the maintainer prefers one atomic rollout.

For the double-spend case, settlement creation must execute as one database transaction: lock the relevant (asset, anchor) liquidity rows with SELECT ... FOR UPDATE, calculate the currently reserved amount under the same transaction, reject if the remaining available amount is insufficient, and insert the pending settlement before commit. A second transaction waits for the row lock and then observes the reduced availability, so both requests cannot succeed against the same balance. The concurrency test will issue two opens against one funded pool and assert exactly one succeeds.

Migration and configuration

• CI runs npm run migrate up against a disposable PostgreSQL service before integration tests, then rolls back/destroys that database after the job.
• Development runs the same command against DATABASE_URL, with a documented local Docker Compose PostgreSQL option.
• Deploy runs migrations as a release/pre-deploy step before application instances start; migrations are forward-only and acquire PostgreSQL advisory locking so two replicas cannot apply them concurrently.
• DATABASE_URL is required for the database-backed runtime. loadConfig will validate it and the startup path will open a connection and fail fast with an actionable message if the database is unreachable.
• The in-memory repositories remain the default test doubles for unit tests. Database integration tests use the disposable CI PostgreSQL service and cover migrations, mapping, rollback, and concurrent settlement creation. This preserves fast tests without hiding the important transaction behavior.

I will wait for maintainer agreement on this schema, tooling, sequencing, and test strategy before writing implementation code, as requested by this issue.