Skip to content

[Architecture] Migrate JSON Flat-File Persistence Layer to SQLite with WAL Mode and Transactional Integrity #17

Description

@Darkvader-ship-it

Estimated Effort: 3 weeks / 6 days

Impacted Subsystems / Files:

lib/phase-protocol.ts — Core Soroban protocol logic, contract simulation and RPC helpers
lib/server-data-paths.ts — Server data directory routing and file mapping
contracts/phase-protocol/src/lib.rs — Soroban smart contract logic and data keys
app/api/forge-agent/route.ts — x402 AI oracle, lore generation, and settlement verification
app/api/faucet/route.ts — Reward distribution, quest progress tracking, and trustline checks
Context & Problem Statement: The current implementation in lib/phase-protocol.ts and related server handlers faces architectural bottlenecks regarding issue #1: "Migrate JSON Flat-File Persistence Layer to SQLite with WAL Mode and Transactional Integrity". Currently, PHASE relies on uncoordinated JSON file storage (.data/ or /tmp/phase-server-data), monolithic client/server protocol utilities, and strict dependency on external testnet RPC endpoints without granular fallback coordination. Under high traffic or concurrent state updates, race conditions occur across market-store.ts, signal-store.ts, achievement-store.ts, and faucet-claims.json. Furthermore, the Soroban Rust contract in contracts/phase-protocol/src/lib.rs handles linear list lookups like OwnerTokenList which scales sub-optimally (O(N)) for high-frequency transfers.

Technical Requirements & Scope:

Audit affected components and define strict interface boundaries.
Refactor synchronous filesystem/contract bottlenecks to transactional, concurrent primitives.
Implement strict type safety, input validation schemas, and domain-level error handling.
Add state sync checks between client freighter signing and server verification pipelines.
Ensure full backward compatibility with SEP-50 metadata requirements and existing testnet deployments.
Multi-File Change Surface: This refactor impacts core server stores (lib/*-store.ts), route handlers under app/api/, client-side state hooks in components/, and smart contract methods in contracts/phase-protocol/src/lib.rs. Modifying state transitions without synchronized schema updates risks runtime deserialization crashes and corrupted state in production.

Acceptance Criteria:

[ ] High-concurrency operations execute safely without data loss or race condition crashes.
[ ] Zero unhandled domain errors across API route handlers.
[ ] Unit and integration test coverage over affected module routes exceeds 90%.
[ ] Performance benchmarks demonstrate <50ms response latency under simulated load.
Suggested Approach / Investigation Steps:

Benchmark current execution path latency and resource consumption.
Design isolated domain abstractions with clear fallback behavior.
Implement changes across contracts, API routes, and UI components.
Verify end-to-end functionality on Stellar testnet RPC.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions