feat(security): automated incident response and on-chain circuit breaker for critical pool alerts

[diegoveme]

Description

The security scan already detected critical alerts and wrote them to
security_alerts, but nothing ever acted on them. This adds the layer that
does: when enough critical alerts land against one pool, the circuit breaker
trips, the pool is paused, and an incident is recorded for an admin to review,
resolve and resume.

The pause is real on both sides. The platform halts the pool immediately, and the
contract's own pause is invoked automatically too, without the platform ever
holding an admin key and without changing the contract.

Closes #254

How the on-chain pause is automatic

This deserves spelling out, because the obvious reading is that it cannot be.

rotational::pause asserts admin.require_auth() and that the caller is the
pool's stored admin, which is the creator's own wallet (set in initialize,
passed as admin: address from the create-group forms). The platform cannot call
it on its own keys, and SPONSOR_SECRET_KEY cannot stand in: a fee bump pays for
a transaction, it authorises nothing inside it.

That is a key-custody problem, not a contract limitation. A
SorobanAuthorizationEntry is signed independently of the transaction envelope,
so the party that authorises a call and the party that submits it can be
different. The admin signs one entry covering exactly pause(admin) on exactly
their pool's contract. The platform stores it and, when the breaker trips, wraps
it in a transaction it pays for and signs the envelope of.

admin's wallet                     platform
     |                                |
     |  signs pause(admin) entry      |
     |------------------------------->|  stored, single use, expires
     |                                |
                                      |  breaker trips
                                      |  wraps entry in a tx, pays the fee
                                      |------------------> Soroban

Two signatures doing two jobs: the admin authorises the call, the platform
authorises the fee. No contract change, no shared key, and the credential the
platform holds can do exactly one thing.

An alternative exists and was deliberately not taken. require_auth for a
classic G address uses Stellar multisig at the medium threshold, so an admin
could add a platform signer with enough weight instead. Simpler to operate, but a
far wider grant, since that weight applies to the account in general rather than
to a single call.

The entry is validated, not trusted

A stored entry is a bearer credential the platform will submit later, so what it
refuses matters more than what it accepts. On arrival an entry must be
address-credentialed (a source-account credential authorises whoever submits,
which is not a delegation), invoke pause, take the signer as its only argument,
and carry no sub-invocations, so it cannot smuggle a second call alongside the
pause. It is then matched against the pool's contract and admin, and refused if
it expires too soon to be useful.

The tests build real entries with the XDR library and assert each refusal,
including an entry that authorises emergency_withdraw and one that hides it in
a sub-invocation.

The XDR is never returned by the API, and the table has no read policy outside
the service role. Whoever holds it can pause the pool, which would be a griefing
vector against the pool's own members.

When no authorization exists

Nothing is lost. The platform pause still happens immediately, the incident stays
at onchain_status = 'pending', the admin is told why in their notification, and
they sign the contract call themselves from the review screen. Pre-authorising
only removes the wait.

Entries are single-use and expire, so an admin re-signs one occasionally.
GET /api/admin/pause-authorizations reports armed: true while a usable one
exists.

emergency_withdraw

Nothing in the automated path can move funds, and that is enforced in two places
rather than promised. The breaker's IncidentAction has exactly two values,
pause and none, with a test asserting the set has not grown. And an
authorization entry that names emergency_withdraw, at the root or nested, is
refused before it is ever stored.

Type of Change

• feat: new feature
• fix: bug fix
• chore: maintenance, tooling, dependencies
• docs: documentation only
• refactor: code restructuring (no functional changes)
• test: adding or updating tests

How Has This Been Tested?

• cargo test passes (smart contracts): not applicable, no contract changed
• pnpm build succeeds (frontend): all three new routes appear in the output
• pnpm lint passes (frontend): no errors
• pnpm test:unit passes: 299 tests, 56 of them added here
• npx tsc --noEmit reports no errors in any file this PR touches

On formatting: pnpm format:check passes for every file in this PR. Running it
across the whole repo on a Windows checkout reports 373 files, including ones
this PR never touches, because git converts to CRLF locally while Prettier
defaults to LF. That does not happen on CI.

What the tests do and do not cover. The decision logic, the authorization
lifecycle, and the entry inspection are covered directly, and the inspection
tests run against real signed XDR rather than fixtures. What they do not cover is
a live submission: that needs a deployed pool, a funded sponsor and an RPC, so
submitOnChainPause is exercised through its callers' error paths rather than
end to end. It is written to degrade to "an admin needs to sign it" on every
failure, and the platform pause has already happened before it runs.

Area	What is covered
Cooldown	Blocked after the allowance is spent, a higher allowance lets a second through, the gate runs before the action so a pool is never paused then reverted, and a blocked pool still reports it would have fired
Dry-run	Decides but does not execute, identical to armed except for execution, and is the default
Escalation	Below threshold does nothing, reaching it exactly fires, the reason names the rules and the count
Grouping	Only critical alerts count, an alert naming several pools counts fully for each, distinct rules collected without repeats, stable order
Pool state	Already paused, completed, and unknown pool each skipped for their own stated reason
Config	Safe defaults, valid overrides, malformed or out-of-range values falling back instead of throwing
Authorization selection	Spent, revoked, expired, expiring inside the safety margin, wrong contract, and rotated admin each rejected for their own reason; the entry expiring soonest is spent first; ties break deterministically
Entry inspection	Real signed entries accepted; emergency_withdraw at the root refused; emergency_withdraw nested as a sub-invocation refused; pausing on behalf of another address refused; source-account credentials refused
Boundary	Pause is the only action the breaker can ever return
Revocation proof	A signature from another key refused, one for a different authorization refused, a stale or future-dated one refused, malformed input refused rather than thrown on

What is in here

Piece	File
Decision and authorization logic, pure and unit tested	frontend/lib/incident-response.ts
Execution against Supabase	frontend/lib/server/incident-actions.ts
On-chain submission and entry inspection	frontend/lib/server/pause-onchain.ts
Signing an authorization in the browser	frontend/lib/pause-authorization.ts
Wallet-signature proof	frontend/lib/wallet-proof.ts, frontend/lib/server/wallet-proof.ts
Admin review and recovery	frontend/app/api/admin/incidents/
Authorization endpoints	frontend/app/api/admin/pause-authorizations/
Wiring into both scans	app/api/cron/security-scan, app/api/admin/security/scan
Schema	supabase/migrations/20260827120000_incident_response.sql, 20260827130000_pause_authorizations.sql
Docs	docs/INCIDENT_RESPONSE.md

Acceptance criteria

Criterion	Where
Critical alerts can automatically pause the affected pool, with a dry-run toggle	decideAutoPause and runIncidentResponse, platform pause plus the contract call via submitOnChainPause; INCIDENT_AUTO_PAUSE_ENABLED arms it
Every auto-action produces an auditable incident record	incidents row written before the pause, plus a pool_activity row so it shows in /api/admin/audit-log
Cooldown prevents pause-flap; admins can review, resolve and resume	Cooldown gate in decideAutoPause; GET /api/admin/incidents and POST /api/admin/incidents/[id]
Unit tests cover cooldown, dry-run and escalation	lib/incident-response.test.ts and lib/server/pause-onchain.test.ts, both registered in test:unit
Linting, format and CI checks pass	See above

Design notes

• Incident before action. The incident row is written first, as "decided but
  not carried out", and promoted only once the pool is actually paused. A crash
  midway leaves a record saying no action was taken, which is true and
  recoverable. The other ordering leaves a paused pool with no explanation.
• The authorization is spent before submission. Its nonce may reach the
  network even when the response never reaches us, and a consumed nonce can never
  succeed again, so burning it on an uncertain outcome is the honest accounting.
  The claim is a conditional update, so two scans racing cannot both spend it.
• Only executed pauses count towards the cooldown. A dry-run decision must
  not consume a pool's allowance, or arming the breaker later would find it
  already spent.
• Below-threshold decisions are not persisted. Every scan would otherwise
  write a row for every pool with a single alert. Everything that met the
  thresholds is recorded, including what dry-run and cooldown held back.
• Failures are contained per pool, and a failure in the whole incident step
  does not lose the scan, since the alerts are already persisted.
• The pause write is conditional on the pool still being active, closing the
  race between the decision and the write.

Rollout

Dry-run is the default, and both scan endpoints report incidentResponse with a
wouldFire count regardless of dry-run state, so the flag produces real data on
threshold accuracy before anyone sets INCIDENT_AUTO_PAUSE_ENABLED=true.
Defaults and tuning are documented in .env.example and
docs/INCIDENT_RESPONSE.md.

Checklist

• My code follows the coding conventions of this project
• I have added/updated tests if needed
• I have updated documentation if needed
• My changes generate no new warnings or errors

On authorization of the admin endpoints

Most admin endpoints here compare a callerAddress from the request against the
pool's creator_address, mirroring /api/admin/audit-log and
/api/disputes/[id]/resolve. That is a claim rather than a proof, and this PR
keeps the convention where the stakes match it.

It does not keep it for revoking a pause authorization. Revoking disarms the
automatic on-chain pause, and both the pool id and the creator address are
public, so a spoofable check there would let an attacker switch off a pool's
defence before draining it. Revocation asks the wallet to sign a short,
timestamped message naming the exact authorization, verified under SEP-53 against
the pool's admin as recorded. No challenge table was needed: the message carries
its own timestamp and target, and the action is idempotent.

Registering needs no such proof and is left alone deliberately: an entry that was
not signed by the pool's real admin is refused by the inspector regardless of who
posted it, and the contract would reject it anyway.

The wider convention still applies to resume and to the read endpoints.
Changing it would touch every admin endpoint in the codebase, so it did not
belong in this PR, but I am glad to open an issue or a follow-up if you want it
hardened across the board.

[Sendi0011]

@diegoveme kindly resolve conflict

[diegoveme]

Conflict resolved and pushed. Two files were involved: frontend/lib/supabase.ts, where main added bridge_transactions to the generated Database types while this branch added pause_authorizations and incidents (all three kept, they are independent definitions), and frontend/package.json, where both sides appended to test:unit (kept as a union, 30 suites). Neither lockfile was touched by this branch and both still match main, so pnpm install --frozen-lockfile is unaffected.

While merging I noticed the new PUT /api/pools/[id]/archive and /unarchive authorise on the admin_address the caller sends in the request body, compared against the pool creator. Both of those values are public, so that check does not establish who is calling, and archiving a pool removes it from Explore and from every member's active list.

I have pushed a fix in the same branch, since it reuses the wallet-proof helper this PR already introduces:

• The endpoints verify a wallet signature against the creator address as the database records it. The body's admin_address no longer authorises anything.
• The signed message names the action and the pool and carries a timestamp, so a proof for one pool does not work on another, a proof to archive cannot be replayed to unarchive, and a captured one expires within minutes.
• The archive banner signs before it calls, so the admin flow is unchanged rather than broken.
• The daily sweep in /api/cron/archive-pools is untouched: it writes through the admin client and never goes near these routes.
• Five tests cover the forgery, cross-pool, cross-action and staleness cases.

Happy to split that into its own PR if you would rather review it separately, though it would need the proof helper from this branch to land first.

pnpm test:unit passes locally (352 tests) and pnpm lint is clean. The workflow runs are sitting in action_required since this is a fork PR, so they need an approval whenever you have a moment.

[Sendi0011]

Review — Automated Incident Response with On-Chain Circuit Breaker

This is the strongest PR in the project so far. The design solves a genuinely hard key-custody problem — submitting an on-chain pause that calls rotational::pause (which asserts admin.require_auth()) — without the platform ever holding an admin key, no contract change, and with the only automatic action being a pause. Thoroughly reasoned, thoroughly tested, and the security boundaries are enforced by a test rather than by convention.

The parts I verified most carefully

1. The boundary is real, not just claimed. IncidentAction has exactly "pause" | "none" and the test at lib/incident-response.test.ts asserts that union cannot have grown. emergency_withdraw and any funds-moving call are unreachable from the automated path.

2. The bearer-credential validation is correct and honest. inspectPauseAuthorization checks the entry is address-credentialed (a source-account credential authorises whoever submits — correctly refused), has no sub-invocations (so a hidden emergency_withdraw can't be smuggled), invokes pause, takes the signer as its only argument, and is matched against the pool's actual contract and admin. The tests build real XDR with the SDK and assert each refusal, including both a root emergency_withdraw and one nested as a sub-invocation. This is exactly the right thing to check on a bearer credential the platform will submit later.

3. Single-use accounting is done the honest way. The authorization is marked spent before submission (incident-actions.ts: the conditional claim with .is("used_at", null) prevents two racing scans from spending the same entry), because a consumed nonce can never succeed again regardless of whether the response reaches us. Good.

4. Ordering is safe under partial failure. Incident row is written before the pause ("decided, not carried out"), the pause write is conditional on status = 'active' (closes the decision→write race), and execution is contained per-pool so one failure doesn't stop the next. A crash midway leaves a recoverable record rather than an unexplained paused pool.

5. RLS on pause_authorizations is deliberately closed. No select policy, unlike incidents/security_alerts — the right call, since the stored XDR is a bearer credential and only the service-role routes may read it.

6. Dry-run is a real rollout mechanism, not a switch. In dry-run it still decides, records, and notifies ("would have paused"), and wouldFire is reported independently of execution — so you get genuine threshold data before arming INCIDENT_AUTO_PAUSE_ENABLED.

7. High-stakes endpoints got signature proof. Revocation of a pause authorization (which disables a pool's defence) and archive/unarchive (which change what everyone sees) now require a SEP-53 wallet signature verified against the pool's creator — closing the "caller-supplied admin_address is just a claim" gap where it actually matters, while leaving the lower-stakes read endpoints on the existing convention.

Non-blocking suggestions

1. verifySignedMessage accepts both SEP-53 framings — the SHA-256-prefixed digest and the raw prefixed bytes passed to ed25519. Accepting both is defensible for wallet compatibility, but it is a wider acceptance surface than the strict spec. If the specific wallets you support all implement the strict SEP-53 framing, consider narrowing it to the spec form to reject malformed signatures deterministically. Purely an availability/consistency trade-off, not a security regression.

2. incidents is SELECT USING (true) — consistent with security_alerts, but it exposes admin addresses (resolved_by) and pool IDs publicly. If you ever want that tightened, a separate policy mirroring the audit-log convention would fit. Not blocking.

3. submitOnChainPause is not covered end-to-end — as you correctly note in the PR body, the live submission path needs a deployed pool + funded sponsor + RPC. Its callers' error paths are covered and it degrades correctly. Worth adding an integration test in CI against a local testnet instance if one becomes available, so assembleTransaction auth-entry preservation is verified against a live simulation.

Verification

• All 5 CI checks green (Lint/Format, Component tests, Build Soroban, Node unit, Playwright)
• Mergeable state clean
• 56 new unit tests covering cooldown, dry-run, escalation, grouping, config bounds, auth selection, entry inspection, revocation proof, and the boundary test
• Docs (INCIDENT_RESPONSE.md) and .env.example are thorough and accurate
• test:unit script correctly registers all three new test files

Approved. Ready to merge. Closes #254.