refactor: RPC resilience, unified logging, centralized env config, and geospatial index optimization - #172
Open
noevidence1017 wants to merge 1 commit into
Conversation
…l indexes Resolves four related refactors across the config, service and model layers. Stellar/Soroban network resilience (SwiftChainn#119) - Extend utils/rpcRetry with a per-attempt timeout, so a hung socket becomes a prompt retryable error instead of blocking until the OS gives up. - Add onAttemptFailed/onRecovery hooks so failed attempts and recoveries are both visible in the logs. - Wrap every Soroban RPC call in stellarService (sendTransaction, getAccount, prepareTransaction, getTransaction) in the shared policy. - Classify transient failures (timeouts, socket errors, HTTP 408/425/429/5xx) and retry only those; deterministic 4xx and tx_bad_seq are left to their existing handling. Unified logging interface (SwiftChainn#121) - Rebuild config/logger on winston-daily-rotate-file with size and age based rotation, gzip archiving and bounded retention. - Add utils/piiMasker and install it as a logger-level format, so masking applies to every transport and every existing call site unchanged. - Redact sensitive keys wholesale; partially mask emails, phone numbers, card numbers and Stellar public keys so logs stay correlatable. - Handle transport errors and set exitOnError false so logging cannot take down the process. Centralized environment configuration (SwiftChainn#120) - Move every scattered process.env read into the Zod schema in config/env, covering socket tuning, location ingestion, ETA cache, routing, lifecycle, Stellar network, escrow indexing, logging and the Soroban circuit breaker. - Declare CB_SOROBAN_* which were already consumed via env but never defined. - Update all consumers to read the validated object; only JEST_WORKER_ID remains on process.env, as a test-runner signal rather than app config. - Add cross-field validation for the proximity radius and backoff bounds. Geospatial index optimization (SwiftChainn#122) - Add models/DriverLocation holding one current position per driver, so proximity search is an index scan rather than a per-driver sort of history. - Index availability fields before the geometry so the planner narrows to available drivers before walking the index, plus a plain 2dsphere for unfiltered searches and a TTL index to retire stale records. - Add driverLocationService using $geoNear with the filter pushed into the index walk, and an explain() based profiling method. - Expose versioned endpoints under /api/v1/drivers via a new controller. Also mounts the existing stellar routes and restores two route imports that were referenced in routes/index but never imported. Verification: 114 new tests pass (48 PII masking, 33 retry/timeout, 33 geospatial). Full-suite comparison against the base commit shows no newly failing test.
|
@noevidence1017 Great news! 🎉 Based on an automated assessment of this PR, the linked Wave issue(s) no longer count against your application limits. You can now already apply to more issues while waiting for a review of this PR. Keep up the great work! 🚀 |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This PR resolves four related refactors across the config, service, and model layers.
closes #119
closes #120
closes #121
closes #122
#119 - Stellar/Soroban network timeouts and retries
utils/rpcRetrywith a per-attempt timeout so a hung socket becomes a prompt retryable error instead of blocking until the OS gives up.onAttemptFailed/onRecoveryhooks so both failed attempts and recoveries are visible in logs.stellarService(sendTransaction, getAccount, prepareTransaction, getTransaction) in the shared policy.#121 - Unified logging with rotation and PII masking
config/loggeron winston-daily-rotate-file with size/age rotation, gzip archiving, and bounded retention.utils/piiMasker, installed as a logger-level format so masking applies to every transport and all 300+ existing call sites without changing any of them.exitOnErroris false, so logging cannot take down the process.#120 - Centralized environment variables
process.envread into the Zod schema inconfig/env: socket tuning, location ingestion, ETA cache, routing, lifecycle, Stellar network, escrow indexing, logging, and the Soroban circuit breaker.CB_SOROBAN_*variables, which were already consumed viaenvbut never defined - this was a live type error on main.JEST_WORKER_IDremains onprocess.env, as it is a test-runner signal rather than application config.#122 - Geospatial index optimization
models/DriverLocation, holding one current position per driver. Proximity search previously would have required finding the newest document per driver in the append-only history collection, which no index makes fast.{ isAvailable, status, location }) so the planner narrows to available drivers before walking the index, plus a plain 2dsphere for unfiltered searches and a TTL index to retire stale records.driverLocationServiceusing$geoNearwith the filter pushed into the index walk, so non-matching drivers are skipped rather than filtered afterwards.explain()-based profiling method, making index regressions observable rather than showing up only as production latency./api/v1/driversthrough a new controller.Architecture
Strict Controller -> Service -> Model throughout. Controllers handle transport concerns only; all database access is in the service layer. All response data is read from MongoDB, with no mock objects or hardcoded values. All new endpoints are versioned under
/api/v1/.Testing
114 new tests, all passing:
$geoNearpipeline are exercised for realFull-suite comparison against the base commit shows no newly failing test. Type errors drop from 449 to 416, with the reduction coming from declaring the previously-undefined
CB_SOROBAN_*variables.Pre-existing issues encountered but not addressed
These exist on main and are outside the scope of these four issues:
jest.config.jsthrowstsJestTransformCfg is not defined(introduced in 2b6751a)src/di/container.tsimports../controllers/stellarController, but the file isstellar.controller.ts@stryker-mutator/core@^7.3.1andawilix@^10.2.2no longer resolve, sonpm installfailsHappy to address any of these in a follow-up if useful.