feat: add @saga-ed/soa-postgres + redis/mongo loaders for 3-env mirror tier#102
Merged
Conversation
…r tier
Companion to iac PR #352 (saga-provision-credentials/saga-verify-credentials).
Together they enable the "roles in code, identities at provision time"
DB access pattern across the 7 services being stood up in prod.
New package:
@saga-ed/soa-postgres (packages/node/postgres) - PostgresProvider,
PostgresProviderConfig (zod), loadPostgresConfigFromAws. Lifted from
student-data-system/packages/node/ads-adm-db with Prisma coupling removed
so the package stays a thin pg-only library. Consumers wrap the pool
with PrismaPg themselves (one line). Reads the SM payload shape written
by saga-provision-credentials: {username, password, host, port, database}.
Extended packages:
@saga-ed/soa-redis-core - new loadRedisConfigFromAws reads
{env}/redis/{service} from Secrets Manager (+ SSM fallback for endpoint).
Returns the existing RedisConfig shape unchanged so RedisConnectionManager
works without modification.
@saga-ed/soa-db - aws-mongo-loader now accepts env='mirror' as the
canonical name; env='staging' is kept as a deprecated alias for one
cycle (emits a console.warn) so existing callers don't break.
Both names resolve to the same SSM/secret paths today; will switch to
/shared/infra/mirror/* when the IaC rename ships.
Tests:
- postgres: 9 tests (config validation, loader env mapping, SSL defaults,
SM payload shape compatibility)
- redis: 4 tests (endpoint from secret vs SSM fallback, TLS defaults per env)
- mongo: 4 tests (mirror alias works, staging deprecation warning, prod
unaffected, unknown env rejected)
All 32 tests pass; builds clean for all 3 packages.
Versions:
@saga-ed/soa-postgres: 0.1.0 (new)
@saga-ed/soa-redis-core: 1.1.1 -> 1.2.0
@saga-ed/soa-db: 1.1.3 -> 1.2.0
❌ Test Results
Package Results
Commits
LinksUpdated: 2026-06-01T20:06:02.129Z |
Pairs with hipponot/iac PR #352 commit 8b00466 — same alignment, other
side of the boundary.
postgres loader:
- env='mirror' now reads from /mirror/current/{service}-postgres-password
(the existing refresh workflow path; matches postgres_mirror_to_dev.yml
lines 207-263). Single source of truth instead of two parallel
secret universes.
- New `dbId` param for multi-DB services. Path appends -{dbId} before
the -postgres-password suffix for mirror (preserves workflow
pattern) or after the service name for dev/prod (clean separation).
- Accepts both `database` (CLI / prod) and `dbname` (mirror workflow)
in the secret payload — the workflow uses the libpq-style `dbname`
field name while the CLI uses the more common `database`.
- Exports `postgresServiceSecretName` helper for consumers that need
to compute the canonical path independently (e.g. in IAM policies).
mongo loader:
- env='mirror' now reads from the workflow's per-service path:
SSM /mirror/current/mongodb-shared/{endpoint,port,replica-set-name,ca-secret-arn}
Secret /mirror/current/{project}-mongo-password
instead of falling back to the staging/* paths. Mirror is the
canonical name and consumers get the up-to-date refresh-workflow
output directly.
- env='staging' still works (deprecated, console.warn) — resolves to
legacy /shared/infra/staging/* paths for one cycle.
- env='prod' unchanged.
Tests:
- postgres: new `postgresServiceSecretName` tests pin the path shape
(mirror with leading slash + -postgres-password suffix, prod with
{service}[-{dbId}] suffix). Multi-DB case covered (iam-api with pii
dbId). `dbname` vs `database` interchangeability verified.
- mongo: mirror test now stubs /mirror/current/* paths and verifies
the loader reads the per-service mirror secret. Staging deprecation
warning preserved. Prod path verified unchanged.
39/39 tests pass across all 3 packages (16 postgres + 4 redis-core + 19 db).
Contributor
Author
|
Companion iac patch in hipponot/iac#352 (commit 8b00466) aligns the CLI mirror mode to write to the same paths this loader now reads from. |
Contributor
Author
|
Downstream consumer: saga-ed/student-data-system#99 declares ledger-api's |
Rewrites loadPostgresConfigFromAws() for the validated chat-api
multi-role auth design. Single entry point now handles dev (static
parity password), mirror (IAM via SSM coords), and prod (IAM via
SSM coords) — same shape returned to consumers.
API surface changes:
loadPostgresConfigFromAws({
env: 'dev'|'mirror'|'prod',
service: string, // dir name; hyphenated
role?: 'owner'|'app'|'ro', // default 'app'
instanceName: string,
dbId?: string, // for multi-DB services
database?: string, // override derived db name
username?: string, // override derived username
region?: string,
}) -> PostgresPoolConfig
Returns a pg.Pool-compatible config:
{
instanceName, host, port, database, user, ssl,
password: string // dev: static parity
| (() => Promise<string>) // mirror/prod: IAM token
}
The password union lets consumers hand the config directly to
new pg.Pool(config); pg.Pool natively accepts either shape.
Why this shape:
- Dev keeps its existing parity flow (static password from SM
secret written by saga-provision-credentials --insecure-dev).
- Mirror/prod use IAM auth via @aws-sdk/rds-signer. Coords come
from SSM at /mirror/current/postgres-rds/{endpoint,port} or
/shared/infra/prod/postgres-{host,port}. The async password
callback mints a fresh 15-min token per new pool connection;
existing connections live as long as the server allows (RDS
doesn't invalidate connections on token expiry).
- Username + database are derived from convention
({service_snake}_{role} and {service_snake}); overridable for
legacy schemas.
Naming convention is suffix-matched and intentional:
- {service}_owner — DDL / migrations (matches the existing
postgres-role-isolation.md skill convention and avoids
*_admin collision with RDS master saga_admin)
- {service}_app — runtime DML
- {service}_ro — read-only
New helpers exported for IAM-policy authors:
- iamHostSsmPath(env) / iamPortSsmPath(env)
- devSecretName(service, role, dbId?)
18/18 tests pass; build clean.
Backed by validated chat-api auth pass:
- hipponot/iac#355 (cluster SG ingress, deployed dev)
- hipponot/iac#356 (SagaCap-RdsIamConnect-* + tier wiring, deployed)
- saga-ed/claude-plugins#79 (postgres-role-isolation.md rewrite)
Contributor
Author
|
Updated for IAM auth + multi-role per the validated chat-api auth design. The loader now handles dev/mirror/prod via the same entry point with a |
The prod branch of iamHostSsmPath/iamPortSsmPath pointed at
/shared/infra/prod/postgres-{host,port}, which is not deployed. Repoint
to /prod/postgres-rds/{endpoint,port} to match the RDS stack outputs.
Bump to 0.1.1.
…n (1.1.3) Production Express context previously required a logFile and routed logs through pino's worker-thread transport, which crashes on process fds in ECS Fargate — forcing a /tmp file that never reached CloudWatch. Use a main-thread pino.destination stream on fd 1 (stdout) so structured JSON is captured by the awslogs driver. An explicit logFile still overrides. Drops the "logFile required" throw; bump to 1.1.3.
… loss Attach a pool 'error' listener so an error on an idle pooled client (RDS failover/reboot, network partition) no longer bubbles out as an uncaught exception and crashes the process — the pool discards the dead client and reconnects on the next acquire. Enable TCP keepAlive to detect half-open sockets proactively, swallow rejections from the per-connection session-timeout SET statements, reset the health flag on idle error, and guard connect() on the live pool so it can't leak a second pool. Adds unit tests covering each path.
PostgresProvider now accepts either a static PostgresProviderConfig or a PostgresPoolConfig straight from loadPostgresConfigFromAws — including the mirror/prod IAM shape whose password is an async token callback. The pool is built from discrete pg options instead of a connection string, because pg silently drops a discrete password when a connectionString is also present (the token callback would be lost). ssl is expressed as a boolean (true => full verification, matching the previous sslmode=require) which also sidesteps the upcoming pg v9 sslmode reinterpretation. This means RDS/mirror consumers inherit the idle-error handler, keepAlive, connect retry, and per-connection token minting instead of hand-rolling a bare Pool that has none of them. PostgresPoolConfig gains optional pool tuning fields; the provider applies schema-matching defaults when omitted.
Bring MongoProvider's resilience posture in line with PostgresProvider: - connect() now retries with exponential backoff (3 attempts), but fails fast on non-retryable auth errors (AuthenticationFailed=18, Unauthorized=13) so a misconfigured deploy doesn't hang ~90s burning server-selection windows. The driver's own serverSelectionTimeoutMS still absorbs transient unavailability within each attempt. - isConnected() now reads a tracked _connected flag instead of poking the undocumented private (client as any).closed field; documented as a cheap sync flag, not a liveness check. - add ping() (admin.ping, returns false rather than throwing) for real readiness probes; exposed as an optional method on IMongoConnMgr. MockMongoProvider mirrors the flag + ping for parity. Adds unit tests (mocked mongodb driver) covering retry success, give-up, auth-skip, ping, and isConnected transitions.
resolveCAFile wrote the CA cert to tmp with mode 0o400 (read-only), so a second construction — a service restart on the same host, or any provider rebuilt with the same instanceName — failed with EACCES because the stale read-only file couldn't be overwritten. Remove any stale copy before writing. Adds a regression test that constructs twice without throwing.
Resolve pnpm-lock.yaml conflict by regenerating it with pnpm install (the only conflicting file; all branch source changes are in new files main does not touch). Lockfile verified in sync via --frozen-lockfile; soa-postgres + soa-db build clean and all tests pass against the merge.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Companion to iac PR hipponot/iac#352 (`saga-provision-credentials` + `saga-verify-credentials`). Together they enable the "roles in code, identities at provision time" DB access pattern across the 7 services being stood up in prod (saga-dash, qboard, coach, program-hub, rostering, student-data-system, janus).
The iac CLI writes per-service secrets at canonical SM paths; these soa libraries read those secrets and assemble typed provider configs.
New package
`@saga-ed/soa-postgres` (`packages/node/postgres`)
Lifted from `student-data-system/packages/node/ads-adm-db` with the Prisma coupling removed so the package stays a thin pg-only library. Consumers wrap the pool with PrismaPg themselves:
```ts
const provider = new PostgresProvider(await loadPostgresConfigFromAws({...}));
await provider.connect();
const adapter = new PrismaPg(provider.getPool());
const client = new PrismaClient({ adapter });
```
Extended packages
`@saga-ed/soa-redis-core` — new `loadRedisConfigFromAws` reads `{env}/redis/{service}` from SM with an SSM fallback for the endpoint. Returns the existing `RedisConfig` shape unchanged so `RedisConnectionManager` works without modification. Per-service users are server-side enforced via ElastiCache User Management AccessStrings provisioned by the iac CLI.
`@saga-ed/soa-db` — `aws-mongo-loader` now accepts `env='mirror'` as the canonical name. `env='staging'` is kept as a deprecated alias for one cycle (emits a `console.warn`) so existing callers don't break. Both names resolve to the same SSM/secret paths today; will switch `mirror` to `/shared/infra/mirror/*` when the IaC rename ships.
Tests
All 32 tests pass. All 3 packages build clean.
Versions
Test plan