From 5ed11629f200173f0b40709aef7cff90da712200 Mon Sep 17 00:00:00 2001 From: Deb-Auth Date: Sun, 30 Aug 2026 17:02:03 -0700 Subject: [PATCH 1/2] docs: add testing strategy, migration workflow, and security policy Adds three documents covering gaps in the contributor-facing documentation, plus the matching rows in the documentation index. docs/testing.md - cross-app testing philosophy and conventions: - States the no-external-services rule explicitly, with the reasoning and the approved substitute for each dependency (ioredis-mock for Redis, a hand-built db mock for Postgres, LocalDiskObjectStore for S3/MinIO, fake-indexeddb and the web setup shims, pytest-mock fixtures for the AI agent, and the Soroban Env harness for contracts). - Documents the per-app runners and commands: vitest for backend and web, pytest for the AI agent, cargo test for contracts, plus the include and exclude patterns that decide which files are collected. - Explains the established Drizzle mocking pattern, including why the module under test is imported with a dynamic import after the mocks, and why .values() sometimes has to be both thenable and expose .returning(): the message insert chains .returning() for the generated id while the envelope insert only awaits .values(), so a stub supporting one shape silently records nothing for the other. - Documents that socket handlers must be driven through the enveloped dispatch event rather than a raw socket.on listener, since there is no raw listener and grabbing one bypasses envelope validation, the auth gate, and eventId idempotency. - Notes the shared in-process rate-limit counters that leak between tests, with the reset hooks for that and the other module-level state. apps/backend/docs/migrations.md - drizzle-kit migration workflow: - Establishes schema.ts as the source of truth and migrations as generated output, never hand-written first, with the one documented exception. - Documents the drizzle/ layout and meta/_journal.json as the file that decides what actually runs, and that a .sql file missing from it is silently skipped. - Documents the merge hazard concretely against the incident already in this repository's history: seven colliding 0001_* files and a journal entry with duplicate JSON keys, which left nine migrations unlisted and forced a squash back to a single baseline. - Gives the recommended conflict-resolution procedure - merge schema.ts, take the base branch's drizzle/ wholesale, drop your own migration, and regenerate - plus the checks that catch a collision before it lands. - Notes that drizzle/meta/ is Prettier-ignored because it is generated output. SECURITY.md - vulnerability disclosure policy: - Gives GitHub private vulnerability reporting as the private channel, with acknowledgement, triage, update, and fix windows. - States explicitly that vulnerabilities must not be reported through public issues, pull requests, or discussions, and why. - Defines scope across the backend, web client crypto, contracts, AI agent, and supply chain, with an explicit out-of-scope list and testing rules. - Cross-links docs/threat-model.md so reporters can tell an accepted residual metadata risk from a real finding. - Adds a contract-specific path: only token_transfer is upgradeable, so a bug in group_treasury or proposals cannot be fixed by a redeploy and funds already in a vulnerable instance may be unrecoverable. Closes #545 Closes #546 Closes #548 --- SECURITY.md | 223 ++++++++++++++++++++ apps/backend/docs/migrations.md | 268 ++++++++++++++++++++++++ docs/README.md | 8 + docs/testing.md | 354 ++++++++++++++++++++++++++++++++ 4 files changed, 853 insertions(+) create mode 100644 SECURITY.md create mode 100644 apps/backend/docs/migrations.md create mode 100644 docs/testing.md diff --git a/SECURITY.md b/SECURITY.md new file mode 100644 index 0000000..2d4e6b7 --- /dev/null +++ b/SECURITY.md @@ -0,0 +1,223 @@ +# Security Policy + +Clicked is an end-to-end encrypted messaging product that also moves funds on-chain. A +vulnerability here can expose private conversations or drain a treasury, and in the on-chain +case the damage may be irreversible. We take reports seriously and we would much rather hear +about a problem from you than from an incident. + +--- + +## Reporting a vulnerability + +**Report privately, through GitHub's private vulnerability reporting:** + +> **** + +That form is private between you and the maintainers. It creates a draft security advisory, +gives us a place to discuss the issue and share a fix with you before it is public, and lets +us credit you when the advisory is published. + +If the form is unavailable to you for any reason, contact the maintainer +([@codebestia](https://github.com/codebestia)) directly on GitHub and ask for a private +channel. Do not include vulnerability details in that first message. + +### Do not report vulnerabilities publicly + +**Do not open a public issue, a public pull request, or a public discussion for a security +vulnerability, and do not post details on social media, in a chat channel, or in a commit +message before a fix has shipped.** + +A public issue is a working exploit advertisement. The report is visible to everyone the +moment it is filed, while a fix takes days to write, review, and deploy — and in the case of +a non-upgradeable contract it may not be deployable at all. A pull request is worse: the diff +itself explains the bug, and anyone watching the repository is notified. + +This applies even when the issue seems minor or you are not sure it is exploitable. Let us +make that call in private. If you are unsure whether something counts as a security issue, +report it privately anyway — a misfiled private report costs us nothing. + +### What to include + +The more of this you can supply, the faster we can triage: + +- Which component is affected (backend, web client, contracts, AI agent) and the commit, + branch, or deployed contract ID you tested against. +- A description of the vulnerability and the security property it breaks — for example + "the server can recover message plaintext", "device A can read device B's envelopes", + "a non-admin can withdraw from a treasury". +- Reproduction steps, ideally a minimal proof of concept: a failing test, a `curl` sequence, + or a Soroban invocation. +- Your assessment of the impact and of any preconditions (does it need an authenticated + session, a compromised device, a specific group state?). +- Whether you have disclosed this to anyone else, and any disclosure deadline you intend to + hold us to. + +### What to expect + +| Stage | Target | +| ------------------------------------------------------------------- | -------------------------- | +| Acknowledgement that we received the report | **3 business days** | +| Initial triage: severity, in scope or not, whether we can reproduce | **10 business days** | +| Status updates while a fix is in progress | At least every **14 days** | +| Fix or documented mitigation for a confirmed high-severity issue | **90 days** from triage | + +If you do not hear from us within the acknowledgement window, please ping the maintainer on +GitHub referencing the advisory — assume the notification was missed rather than ignored. + +We aim to publish an advisory once a fix is available, crediting you by the name or handle +you ask for (or keeping you anonymous, if you prefer). Please give us a chance to ship the +fix before disclosing publicly; if you have a fixed disclosure deadline, say so in your first +message so we can plan against it. + +There is no paid bug bounty for this project today. + +--- + +## Scope + +### In scope + +**Backend gateway and API** (`apps/backend`) + +- Authentication and session handling: wallet-signature challenge/verify, JWT issuance, + claim validation, session or device binding bypass. +- Authorization: reading or writing a conversation, message, device, file, or treasury record + you are not a member of or do not own. +- Anything that lets the server, or an attacker with server access, obtain message plaintext + or private key material — the ciphertext-only invariants are load-bearing for every claim + this product makes. +- Injection into the database, the object store, or the Socket.IO event path; envelope + validation or idempotency bypass on the `dispatch` event. +- Rate-limit bypass that enables credential stuffing, prekey exhaustion, or resource + exhaustion of the gateway. +- Presigned upload/download URL flaws: forgery, scope escalation, or access to another + conversation's blobs. + +**Web client cryptography** (`apps/web`) + +- Flaws in the E2EE implementation: X3DH, the double ratchet, MLS group operations, session + or epoch handling, safety-number computation. +- Key generation, storage, or lifetime bugs — weak randomness, a private key leaving the + device, keys surviving a revocation, keys readable from another origin. +- File encryption: key reuse, nonce reuse, unauthenticated ciphertext, a file key reaching + the wrong recipient. +- Identity-trust bugs: accepting an unverified identity key change, or failing to surface one. +- Client-side XSS or a content-injection path that reaches decrypted message content or + IndexedDB. + +**Smart contracts** (`contracts/`: `token_transfer`, `group_treasury`, `proposals`) + +- Any path that moves funds without correct authorization, or that lets a caller bypass + `require_auth`. +- Treasury multisig or voting logic that can be subverted: double-voting, quorum manipulation, + replaying or re-executing a proposal, executing a proposal that did not pass. +- Arithmetic overflow/underflow, accounting errors, and storage-key collisions. +- Abuse of the admin-gated `upgrade` entrypoint on `token_transfer`. + +**AI agent** (`apps/ai_agent`) — in scope where a flaw exposes user data or is reachable from +untrusted input; prompt-quality complaints are not security issues. + +**Repository and supply chain** — leaked credentials committed to the repository, a +compromised or typosquatted dependency, or a CI workflow that can be made to execute +attacker-controlled code. + +### Out of scope + +- **Denial of service through raw volume.** Load-testing a deployment, traffic floods, or + resource exhaustion that requires attacker bandwidth rather than an application flaw. +- **Automated scanner output with no demonstrated impact** — a CVE in a transitive dependency + on a code path this project does not reach, or a "missing header" report with no exploit. +- **Missing hardening that is not itself a vulnerability**: absent security headers, cookie + flags on endpoints that set no cookies, verbose non-sensitive error messages, TLS + configuration of a deployment we do not control. +- **Social engineering, phishing, or physical attacks** against maintainers or users. +- **Vulnerabilities requiring a fully compromised device or a malicious OS/browser + extension.** The threat model assumes the endpoint is trusted; an attacker with the user's + device already has their keys. +- **Third-party services** — the Stellar network itself, a wallet extension such as Freighter, + browser push providers, the LLM or vector-store vendors. Report those to their owners. +- **Anything already documented as an accepted residual risk** — see below. +- Findings in a fork, a stale branch, or code that was never merged to `dev` or `main`. + +### Testing rules + +Test against your own local deployment and your own accounts. Do not test against another +person's account or data, do not exfiltrate data beyond the minimum needed to demonstrate the +issue, and do not degrade a shared environment. For contracts, use **testnet** — see +[Contract-specific reporting](#contract-specific-reporting) below. + +--- + +## Known and accepted residual risk + +Before reporting, please read the **[threat model](docs/threat-model.md)**. It states exactly +what the server can and cannot see, and its "Residual metadata risk" section lists risks that +are inherent to operating a centralized delivery service and are **known and accepted**, +including: + +- the **social graph** — conversation membership reveals who talks to whom, +- **traffic analysis** — message timing and ciphertext size leak coarse signals, +- **presence and activity patterns**, +- **device fingerprinting** via device name, platform, and prekey consumption rate. + +A report that these are observable is not a vulnerability report; it is a restatement of the +documented design. A report that some _content_, _key material_, or _session state_ is +observable when the threat model says it is not — that is exactly what we want to hear about, +and it is a high-severity finding. + +Related reading, all of which describes intended behaviour rather than bugs: + +- [Threat model](docs/threat-model.md) — trust boundaries and what the server sees. +- [Backend security hardening](apps/backend/docs/security-hardening.md) — measures already in + place and the threats each closes. +- [Rate limits](docs/security/rate-limits.md) — every bucket and its threshold. +- [Audit logging](docs/security/audit-logging.md) — what is logged and what is deliberately + excluded. +- [TLS and pinning](docs/security/tls-and-pinning.md) — transport security expectations. +- [E2EE architecture](apps/web/docs/concepts-e2ee-architecture.md) — the client-side key model. + +--- + +## Contract-specific reporting + +**Treat an on-chain finding as more urgent than an equivalent server-side one, and be more +careful with it.** A backend bug is fixed by a deploy; an on-chain bug frequently is not. + +- **Most of this system's contracts cannot be patched in place.** Only `token_transfer` has an + `upgrade` entrypoint (admin-gated `update_current_contract_wasm`). `group_treasury` and + `proposals` expose **no upgrade function at all** — their deployed WASM is immutable for the + life of the contract instance. Fixing a bug in either one means deploying a _new_ contract, + migrating state, and repointing every consumer at the new contract ID. See + [Contract upgrades and versioning](contracts/docs/concepts-upgrades.md). +- **Funds already in a vulnerable contract may not be recoverable.** Because a redeploy does + not move existing balances or proposal state, a live exploit can be unfixable after the + fact. Time between report and mitigation matters much more than usual, and mitigation may + have to start with pausing usage and moving funds rather than with a code change. +- **Never test a contract vulnerability against mainnet or against a deployed instance holding + real funds.** Reproduce it in the Soroban test environment (`cargo test` against + `Env::default()`) or on testnet with your own deployment. A proof of concept executed + against a live treasury is an exploit, not research, regardless of intent. +- **The strongest proof of concept for a contract finding is a failing Rust test.** Add a + `#[test]` against the contract in question and send us the test — it removes all ambiguity + about preconditions and cannot be mistaken for an attack. See the + [contract testing guide](contracts/docs/testing.md). +- **Include the contract ID and network** you tested against, and say explicitly whether the + issue affects an already-deployed instance or only the current source. +- **If the finding involves the `upgrade` entrypoint or the admin key**, flag that in the first + line of your report. Admin compromise on `token_transfer` is the highest-severity class of + finding in this repository, because it converts an upgradeable contract into an + attacker-controlled one. + +--- + +## Supported versions + +This project is pre-1.0 and under active development. Security fixes are applied to the +`dev` branch and flow to `main`; there are no maintained release branches and no backports to +older tags. Run a current checkout of `main`. + +| Version | Supported | +| --------------- | --------- | +| `main` (latest) | Yes | +| `dev` (latest) | Yes | +| Anything older | No | diff --git a/apps/backend/docs/migrations.md b/apps/backend/docs/migrations.md new file mode 100644 index 0000000..538299a --- /dev/null +++ b/apps/backend/docs/migrations.md @@ -0,0 +1,268 @@ +# Database migration workflow + +How schema changes reach a database in this repository. Everything here is `drizzle-kit` +driven and lives under `apps/backend`; run every command from that directory (or with +`pnpm --filter backend`). + +--- + +## The one-sentence version + +Edit `src/db/schema.ts`, run `pnpm db:generate`, **read the SQL it emitted**, commit both +the schema change and the generated files together, and apply it with `pnpm db:migrate`. + +--- + +## `schema.ts` is the source of truth + +`src/db/schema.ts` is the single declarative description of the database. The files in +`drizzle/` are _output_: drizzle-kit diffs the schema against the snapshot of the last +generated state and writes the SQL needed to close the gap. + +**Never hand-write a migration first and then update the schema to match.** Doing it in that +order means the next `pnpm db:generate` diffs against a snapshot that does not reflect what +your SQL did, and it emits a second migration trying to re-apply — or worse, to undo — the +same change. The snapshot, not the database, is what drizzle-kit compares against, so it has +no way to notice that your SQL already handled it. + +There is one legitimate exception: a change drizzle-kit cannot express (a data backfill, a +`CREATE INDEX CONCURRENTLY`, a multi-step column rewrite). Handle it by generating the +migration normally first, then editing the emitted `.sql` file to add the extra statements — +so the journal entry and the snapshot still describe the change. Keep the edits inside the +generated file; do not add a `.sql` file that drizzle-kit did not create. + +Config lives in `drizzle.config.ts`: schema `./src/db/schema.ts`, output `./drizzle`, +dialect `postgresql`, and `DATABASE_URL` for credentials. + +--- + +## The workflow + +### 1. Edit the schema + +```ts +// src/db/schema.ts +export const users = pgTable('users', { + id: uuid('id').primaryKey().defaultRandom(), + username: text('username').unique(), + // ... new column here +}); +``` + +### 2. Generate + +```bash +pnpm --filter backend db:generate +``` + +This writes three things: + +- `drizzle/NNNN_.sql` — the DDL, +- `drizzle/meta/NNNN_snapshot.json` — the full schema state after this migration, +- a new entry appended to `drizzle/meta/_journal.json`. + +The `` suffix is generated (`0000_lean_scrambler`). You may rename the file to +something descriptive, but if you do you **must** update the matching `tag` in +`_journal.json` — the tag is how the runner finds the file. + +### 3. Review the emitted SQL + +This is the step people skip, and it is the one that matters. drizzle-kit infers intent from +a diff, and a diff is ambiguous in ways that lose data: + +- **A rename looks like a drop plus an add.** If you renamed a column, drizzle-kit will + usually prompt; if it guesses wrong you get `DROP COLUMN` followed by `ADD COLUMN`, and + every existing value is gone. Rewrite it as `ALTER TABLE ... RENAME COLUMN ...`. +- **A new `NOT NULL` column with no default fails on a non-empty table.** Add a default, or + split it into add-nullable, backfill, then set `NOT NULL`. +- **A type change may need a `USING` clause** that drizzle-kit will not write for you. +- **Dropping a column is silent and irreversible.** Confirm every `DROP` in the diff is one + you meant. + +If the SQL is wrong, fix the schema and regenerate rather than patching the SQL — unless it +is the "cannot be expressed" case above. + +### 4. Apply + +```bash +pnpm --filter backend db:migrate +``` + +`db:migrate` walks `_journal.json` in order and runs each migration that the target database +has not recorded yet, inside a transaction, tracking applied migrations in drizzle's own +bookkeeping table. It is the only command that should ever touch a shared database. + +`pnpm db:push` also exists. It diffs the schema straight onto a database with **no** +migration file and no journal entry, which puts that database into a state no migration +history describes. Use it for throwaway local experiments only, never against a shared or +production database, and never as a substitute for generating a migration. + +### 5. Commit together + +The schema change, the `.sql` file, the new snapshot, and the `_journal.json` change belong +in one commit. Splitting them produces a revision where the schema and the migration history +disagree. + +--- + +## The `drizzle/` directory + +``` +apps/backend/drizzle/ +├── 0000_lean_scrambler.sql # DDL for migration 0 +├── meta/ +│ ├── _journal.json # ordered list of migrations to run +│ └── 0000_snapshot.json # full schema state after migration 0 +``` + +**`.sql` files** are the migrations themselves, prefixed with a zero-padded index in apply +order. Statements are separated by drizzle's `--> statement-breakpoint` marker, which is what +`breakpoints: true` in the journal refers to; it tells the runner where one statement ends so +it can run them individually. + +**`meta/_journal.json`** is the index. Each entry carries `idx` (apply order), `version`, +`when` (generation timestamp), `tag` (the `.sql` filename without its extension), and +`breakpoints`. + +> **The journal decides what runs.** `db:migrate` reads `_journal.json`, not the directory +> listing. A `.sql` file sitting in `drizzle/` with no entry in the journal is **silently +> skipped** — no error, no warning, and the migration simply never happens. Because +> drizzle-kit rewrites the journal on every generate, this is exactly what a mishandled merge +> produces, and the failure surfaces much later as a "column does not exist" error in an +> environment where nobody remembers what changed. + +**`meta/NNNN_snapshot.json`** is the complete schema state after that migration. drizzle-kit +diffs the current `schema.ts` against the latest snapshot to work out what the next migration +should contain. A stale, missing, or hand-edited snapshot makes the _next_ contributor's +generate wrong, not yours — which is why snapshot conflicts must never be resolved by +guessing. + +`drizzle/meta/` is listed in `.prettierignore`. It is generated output, not hand-edited +source: reformatting it produces enormous noisy diffs, and drizzle-kit rewrites it in its own +format on the next generate anyway. Do not run Prettier over it, and do not "tidy" it by +hand. + +--- + +## The merge hazard + +This is the one thing to internalise, because **it has already broken this repository's +migration history once.** + +drizzle-kit numbers migrations by "one past the highest index I currently see" and appends to +the journal. It has no knowledge of other branches. So when two branches each add a +migration off the same base: + +- Branch A generates `0001_audit_logs.sql` and appends journal entry `idx: 1`. +- Branch B generates `0001_mls_key_packages.sql` and appends journal entry `idx: 1`. + +Both are correct in isolation. Merged, the result is two different migrations claiming index +`0001`, two snapshots claiming to be `0001_snapshot.json`, and two journal entries claiming +`idx: 1`. Git resolves the `.sql` files trivially — they have different names, so both are +simply kept — while the real damage happens inside `_journal.json`, where a careless +conflict resolution merges the two entries into one malformed object. + +That is precisely what happened here. Before commit `d60b648`, `drizzle/` held **seven** +distinct `0001_*.sql` files (`audit_logs`, `device_key_history`, `gc_background_jobs`, +`group_control_events`, `mls_group_state`, `mls_key_packages`, +`add_system_payload_to_messages`) — and a journal whose entry for `idx: 1` looked like this: + +```json +{ + "idx": 1, + "version": "7", + "when": 1785395646991, + "tag": "0001_mls_group_state", + "when": 1785395076224, + "tag": "0001_mls_key_packages", + "when": 1785129818340, + "tag": "0001_add_system_payload_to_messages", + "breakpoints": true +} +``` + +Duplicate keys in a JSON object are not an error — the last one wins. So that entry ran +`0001_add_system_payload_to_messages` and nothing else. Across the whole directory, thirteen +`.sql` files were present and four journal entries existed: **nine migrations were silently +skipped**, and the journal even referenced `0004_envelope_protocol` at `idx: 3` while +`0003_ciphertext_only_messages.sql` was never listed at all. The history was unrecoverable +and had to be squashed back to a single `0000_lean_scrambler.sql` baseline. + +Nothing warns you about this. `pnpm db:migrate` reports success, because from the runner's +point of view it did exactly what the journal asked. + +### Resolving a migration conflict during a merge + +When a merge touches `drizzle/`, do **not** hand-resolve `_journal.json` or the snapshots. +Regenerate instead: + +1. **Merge everything else first.** Resolve `src/db/schema.ts` on its own terms — both + branches' table and column definitions must survive, and this is the only file where a + real semantic decision is needed. +2. **Take the other branch's `drizzle/` wholesale** — the branch you are merging _into_, + normally `dev`: + + ```bash + git checkout --theirs apps/backend/drizzle # during a merge into your branch + git checkout dev -- apps/backend/drizzle # or, explicitly, from the base branch + ``` + +3. **Delete your own branch's migration files** — the `.sql` and its `meta/NNNN_snapshot.json` + — and make sure they are gone from `_journal.json`. Your change now exists only in + `schema.ts`, which is where it belongs. +4. **Regenerate:** + + ```bash + pnpm --filter backend db:generate + ``` + + drizzle-kit diffs your merged `schema.ts` against the base branch's latest snapshot and + emits a single migration at the correct next index, with a clean journal entry. + +5. **Review the new SQL.** It should contain your change and nothing from the other branch — + if it tries to re-create something the other branch's migration already made, the + snapshot you kept in step 2 was the wrong one. +6. **Verify before pushing**, against a scratch database: + + ```bash + pnpm --filter backend db:migrate + ``` + +Checks that catch the problem before it lands: + +- The number of `.sql` files in `drizzle/` equals the number of entries in `_journal.json`. +- Every `tag` in the journal names a file that exists, and every file is named by a tag. +- `idx` values are unique and contiguous from `0`. +- No index prefix appears on two files. +- `_journal.json` contains no duplicate keys within an entry. + +If you have already pushed a colliding migration, fix it by regenerating on a follow-up +commit as above. Do not renumber files by hand — the snapshots encode the chain, and renaming +a file without rebuilding its snapshot corrupts the next generate. + +--- + +## Rolling back + +There is no `db:rollback`. drizzle-kit generates forward migrations only. To undo an applied +change, generate a new forward migration that reverses it — which means destructive +migrations deserve extra review, since "revert the PR" does not revert the database. + +Where a destructive change has needed a documented undo path, this repository has used a +`drizzle/rollback/` directory holding `NNNN_.down.sql` files. Those are operator-run: +they are never listed in `_journal.json` and `db:migrate` never executes them. The directory +is absent whenever no migration currently needs one; recreate it if yours does. + +--- + +## Migrations and the test suite + +Backend tests never run migrations and never connect to Postgres — the database client is +mocked. See [Testing strategy and conventions](../../../docs/testing.md). Migrations are +validated in CI instead: the backend workflow starts a real Postgres service container and +runs `pnpm db:migrate` against it before the test step, so a migration that does not apply +cleanly to an empty database fails the build. + +That check is only as good as the journal, though. A migration missing from `_journal.json` +is skipped in CI exactly as silently as it is skipped everywhere else, so the review checks +above are not optional. diff --git a/docs/README.md b/docs/README.md index 2acc981..c3aa442 100644 --- a/docs/README.md +++ b/docs/README.md @@ -44,6 +44,8 @@ You want orientation and the shortest path to a working local environment. | [System architecture overview](architecture-overview.md) | The single diagram of all four apps and every external service, with two traced end-to-end paths. | | [Runbook](runbook.md) | Day-two operations: what to do when a service is unhealthy, and how to restart pieces safely. | | [Observability](observability.md) | Which metrics, logs, and traces exist and where they are emitted, so you can see what your change did. | +| [Testing strategy and conventions](testing.md) | The per-app test runners and commands, the rule that tests never start Redis, Postgres, or S3, and the conventions every new test must follow. | +| [Security policy](../SECURITY.md) | How to report a vulnerability privately, what is in scope, and the response windows you can expect. | ### Backend developer @@ -57,6 +59,7 @@ listener. | [Gateway architecture](../apps/backend/docs/concepts-gateway-architecture.md) | Socket.IO connection lifecycle, room semantics, and how the gateway scales horizontally over Redis pub/sub. | | [Delivery fan-out and receipts](../apps/backend/docs/concepts-delivery-fanout.md) | How one sent message reaches every recipient device, how receipts flow back, and which services are not actually wired into the live path. | | [Storage and push jobs](../apps/backend/docs/concepts-storage-push-jobs.md) | Object storage layout and the background jobs that expire files, devices, and envelopes. | +| [Testing strategy and conventions](testing.md) | The Drizzle mocking pattern, driving socket handlers through the `dispatch` envelope, and the in-process counters that leak between tests. | **API reference** @@ -84,6 +87,7 @@ listener. | Document | What it gives you | | --- | --- | +| [Database migration workflow](../apps/backend/docs/migrations.md) | The drizzle-kit loop from `schema.ts` to applied SQL, the `drizzle/` layout, and how to resolve the colliding-migration merge conflict that has already broken this history once. | | [E2EE onboarding](../apps/backend/docs/e2ee-onboarding.md) | Device registration and prekey upload flow for first-contact DM setup. | | [MLS key packages](../apps/backend/docs/mls-key-packages.md) | Key package publication, consumption, and replenishment. | | [MLS group membership](../apps/backend/docs/mls-group-membership.md) | Adding and removing members from an MLS group and the resulting epoch changes. | @@ -112,6 +116,7 @@ is about encryption and local state. | [Error handling and user feedback](../apps/web/docs/concepts-error-handling.md) | Toasts vs. inline error state, mapping backend errors to user-facing messages, and the rule that decryption failures never render as a generic crash. | | [Accessibility guide](../apps/web/docs/accessibility.md) | The WCAG 2.1 AA target, keyboard navigation, modal focus management, live-region announcements, and colour contrast. | | [Wallet and treasury UI](../apps/web/docs/concepts-wallet-treasury-ui.md) | How the wallet and treasury screens are composed and what they read from chain versus the backend. | +| [Testing strategy and conventions](testing.md) | The web Vitest setup, the `fake-indexeddb` and WebCrypto substitutes, and the include pattern that quietly skips `.tsx` test files. | **Client APIs and types** @@ -160,6 +165,7 @@ Threat model, hardening, and the crypto protocol documents. | Document | What it gives you | | --- | --- | +| [Security policy](../SECURITY.md) | The private disclosure channel, response windows, scope, and the extra care an on-chain finding needs. | | [Threat model](threat-model.md) | Assets, adversaries, trust boundaries, and the mitigations claimed for each threat. | | [Security fixes summary](../SECURITY_FIXES_SUMMARY.md) | A log of security issues found and the fixes applied for each. | | [Audit logging](security/audit-logging.md) | What is audit-logged, in what format, and what is deliberately excluded. | @@ -194,6 +200,8 @@ Documents about the repository itself rather than about the product. | Document | What it gives you | | --- | --- | +| [Security policy](../SECURITY.md) | How and where to report a vulnerability privately, and why never in a public issue or pull request. | +| [Testing strategy and conventions](testing.md) | Cross-app testing philosophy, runners, and the conventions a contributor must follow. | | [Pull request template](../.github/pull_request_template.md) | The checklist every pull request is opened against. | | [PR notes](../pr.md) | Scratch notes for an in-flight pull request; not a reference document. | diff --git a/docs/testing.md b/docs/testing.md new file mode 100644 index 0000000..1a79bb0 --- /dev/null +++ b/docs/testing.md @@ -0,0 +1,354 @@ +# Testing strategy and conventions + +How tests are written and run across the four apps in this repository. Read this before +adding a test to any suite — most of it is convention rather than tooling, and the +conventions exist because the alternatives have already broken this repository at least +once. + +The contract test suite has its own, deeper guide: +[Contract testing guide](../contracts/docs/testing.md). This document covers the +cross-app rules and the JavaScript/TypeScript and Python suites. + +--- + +## The rule that matters most: tests never start real external services + +**No test in this repository may require Redis, Postgres, or an S3/MinIO server to be +running.** The test suites must pass on a laptop with Docker stopped, on a fresh clone, +with nothing but `pnpm install` done first. + +This is not a stylistic preference: + +- **A test that needs Docker is a test nobody runs.** The suite is the thing that catches a + regression before review, and it only does that if running it costs one command and a few + seconds. Every service a contributor has to remember to start is a reason the suite gets + skipped locally and the failure gets discovered in CI instead. +- **Shared mutable state makes tests order-dependent.** A real Postgres or a real Redis is + shared across every test file in the run. One test that forgets to clean up a row or a key + produces a failure in an unrelated file, and the failure moves around as file order + changes. +- **Speed is correctness pressure.** An in-process fake answers in microseconds. A suite + that takes two minutes gets narrowed to "just the file I'm editing", and the cross-cutting + regressions are exactly the ones that narrowing hides. + +### Approved substitutes + +| Real dependency | What tests use instead | How | +| ------------------------------------------------------------- | ------------------------------------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| Redis | [`ioredis-mock`](https://github.com/stipsan/ioredis-mock) | `import RedisMock from 'ioredis-mock'`, then `vi.mock('../lib/redis.js', () => ({ redis: sharedRedis, ... }))`. It is a faithful in-memory implementation of the command surface this codebase uses, including `scan`, `del`, and `flushall`. | +| Redis, when the code path should behave as if Redis is _down_ | `null` | `vi.mock('../lib/redis.js', () => ({ get redis() { return null; } }))`. Several modules degrade deliberately when Redis is absent — rate limiting falls back to per-process counters, the conversation cache becomes a pass-through — and those fallbacks need coverage too. | +| Postgres | A hand-built `db` mock | `vi.mock('../db/index.js', ...)` with `vi.fn()` stubs for `query.*.findFirst` / `findMany`, `insert`, `update`, `delete`, `transaction`, and `execute`. See [The Drizzle mocking pattern](#the-drizzle-mocking-pattern). Drizzle's query builders are mocked, not driven — no SQL is generated and no connection is opened. | +| S3 / MinIO | `LocalDiskObjectStore` (`apps/backend/src/lib/localObjectStore.ts`) | Outside production, `lib/storage.ts` already routes presigned PUT/GET through the fs-backed store, so upload and download paths are exercised end to end against real files under `apps/backend/.local-storage/` (git-ignored) with no S3 client involved. Tests that target the S3 SDK path itself mock `../lib/objectStore.js`. | +| IndexedDB (web) | [`fake-indexeddb`](https://github.com/dumbmatter/fakeIndexedDB) | `import 'fake-indexeddb/auto'` at the top of the test file, before the module under test is imported. | +| WebCrypto / `window` / `btoa` (web) | `apps/web/src/test/setup.ts` | The setup file installs `node:crypto`'s `webcrypto`, a `window`/`self` alias, and `btoa`/`atob` onto `globalThis`, so browser crypto code runs unchanged under the `node` environment. Do not re-polyfill these per test. | +| OpenAI / Weaviate (AI agent) | `pytest-mock` fixtures in `apps/ai_agent/tests/conftest.py` | `mock_openai` patches `main.OpenAI`; `mock_weaviate` patches `main.weaviate.connect_to_local`. An autouse fixture sets a dummy `OPENAI_API_KEY` so the client constructor never 500s. No test makes a network call. | +| Stellar / Soroban RPC | The Soroban `Env` test harness | `Env::default()` is a complete in-process ledger. Contract tests never talk to a network. See the [Contract testing guide](../contracts/docs/testing.md). | + +### Environment variables, not services + +`apps/backend/src/__tests__/setup.ts` is registered as a Vitest `setupFiles` entry and sets +`JWT_SECRET`, `DATABASE_URL`, and the `OBJECT_STORE_*` variables to placeholder values. +Their only purpose is to satisfy config validation at import time — nothing connects to the +hosts they name. If a new module validates a new required variable at import, add a +placeholder there rather than mocking the config module in every test file. + +### What CI does, and why it is not a licence to depend on services + +The backend CI workflow does start Postgres, Redis, and MinIO containers. That is for the +`pnpm db:migrate` step, which validates that the generated migrations actually apply to a +real Postgres — not so that tests can reach them. The security workflow +(`.github/workflows/security-ci.yml`) runs backend tests with **no** service containers at +all, which is the standing proof that the suite is service-free. If a change makes the suite +depend on a running service, security CI is where it breaks. + +--- + +## Runners and commands + +| App | Runner | Command | Config | +| --------------- | ------------ | ----------------------------------- | ----------------------------------------------- | +| `apps/backend` | Vitest | `pnpm --filter backend test` | `apps/backend/vitest.config.ts` | +| `apps/web` | Vitest | `pnpm --filter web test` | `apps/web/vitest.config.ts` | +| `apps/ai_agent` | pytest | `cd apps/ai_agent && uv run pytest` | `[tool.pytest.ini_options]` in `pyproject.toml` | +| `contracts` | `cargo test` | `cd contracts && cargo test` | `contracts/Cargo.toml` workspace | + +Useful variants: + +```bash +# Backend: watch mode, and coverage +pnpm --filter backend test:watch +pnpm --filter backend test:coverage + +# Backend: a single file +pnpm --filter backend test -- rateLimiting.test.ts + +# Contracts: one package rather than the whole workspace (this is what CI does) +cd contracts && cargo test -p token_transfer + +# Backend tests plus the contract suite, as the Makefile defines "the tests" +make test +``` + +Conventions worth knowing: + +- **Backend test files** live in `apps/backend/src/__tests__/` as `*.test.ts`, with a few + co-located `*.spec.ts` files next to the module they cover (for example + `src/socket/dispatcher.spec.ts`). Both are collected — the Vitest `include` is + `src/**/*.{test,spec}.ts`. +- **`dist/` is excluded on purpose.** `pnpm build` emits a compiled copy of every spec, and + without the exclusion every test would run a second time against stale output. +- **Web test files** are co-located with the code (`src/lib/x3dh.test.ts`). The web `include` + is `src/**/*.test.ts` only — a `.tsx` test file is not picked up, so a component test must + be `.ts` or the include pattern has to be widened deliberately. +- **AI agent tests** live in `apps/ai_agent/tests/`, and coverage is on by default via + `addopts`, so a bare `uv run pytest` already prints the coverage table. + +--- + +## The Drizzle mocking pattern + +Backend tests do not run SQL. They replace `../db/index.js` with an object shaped like the +Drizzle client and assert on what the route or service asked that client to do. Three +modules are normally mocked together: + +```ts +vi.mock('../db/index.js', () => ({ + db: { + query: { + messages: { findFirst: mockFindMessage }, + conversationMembers: { findMany: mockFindMembers }, + }, + update: mockUpdate, + delete: mockDelete, + }, +})); + +// Column references become inert sentinels. The code under test passes them to +// eq()/and(), which are themselves mocked, so their only job is to be +// distinguishable in an assertion. +vi.mock('../db/schema.js', () => ({ + conversations: {}, + messages: { id: 'id', conversationId: 'conversationId', senderId: 'senderId' }, +})); + +// The operators become identity-ish stubs, so a test can assert on the shape a +// call site built rather than on generated SQL. +vi.mock('drizzle-orm', () => ({ + and: vi.fn((...args: unknown[]) => args), + eq: vi.fn((col: unknown, val: unknown) => ({ col, val })), + desc: vi.fn(), + lt: vi.fn(), + sql: vi.fn(), +})); +``` + +`vi.mock` calls are hoisted, but the mock factories close over `vi.fn()` handles declared +above them, so the module under test is imported **after** the mocks with a top-level +dynamic import: + +```ts +const { messagesRouter } = await import('../routes/messages.js'); +``` + +A static `import` at the top of the file would bind the real `db` before the mocks are +installed. Every backend suite that mocks the database uses the `await import(...)` form; +follow it. + +Builder chains are stubbed by returning the next link: + +```ts +const returning = vi.fn().mockResolvedValue([{ id, createdAt }]); +const values = vi.fn().mockReturnValue({ returning }); +mockInsert.mockReturnValue({ values }); +``` + +### Why `.values()` sometimes has to be both thenable and expose `.returning()` + +Drizzle's insert builder is a thenable. `db.insert(t).values(rows)` is itself awaitable and +executes the statement, _and_ `.returning()` can be chained onto it to execute the statement +and get the inserted rows back. Both forms are used in this codebase, sometimes inside the +same transaction: the message insert needs the generated `id` and `createdAt` so it calls +`.returning()`, while the envelope batch insert only cares that the rows landed and just +awaits `.values(...)`. + +A stub that returns `{ returning }` alone makes the awaited call resolve to a plain object +and silently record nothing — the test passes while asserting on an insert that never +happened. A stub that returns only a promise makes `.returning()` throw +`is not a function`. So a shared insert stub has to satisfy both shapes: + +```ts +function insertStub(table: string) { + return { + values: (vals: unknown) => ({ + returning: async () => recordInsert(table, vals), + then: (resolve: (value: unknown) => void) => resolve(recordInsert(table, vals)), + }), + }; +} +``` + +`apps/backend/src/__tests__/e2ee.integration.test.ts` is the canonical version. The `then` +property is what makes the returned object a thenable, so `await` resolves it and the call +is recorded either way. Two cautions: + +- **Do not add `then` to a stub whose call sites never await the builder directly.** A + thenable is awaited implicitly whenever it is returned from an `async` function, which can + fire the recording side effect a second time. Add it only for the chains that need it. +- **If both forms run against the same stub, make the recorder idempotent** — or assert on + call counts you have actually verified, rather than assuming one insert equals one + recorded row. + +When a test needs a transaction, mock `db.transaction` as a function that invokes its +callback with an object exposing the same stubbed builders: + +```ts +const mockTransaction = vi.fn(async (cb: (tx: unknown) => Promise) => + cb({ insert: insertStub }), +); +``` + +--- + +## Socket handlers are exercised through `dispatch`, never through a raw listener + +Every client-to-server socket event goes through a single enveloped `dispatch` event +(`apps/backend/src/socket/dispatcher.ts`). `dispatcher.register(type, handler)` stores the +handler in a map; `listen()` attaches exactly one `socket.on('dispatch', ...)` listener, +which checks that the socket is authenticated, validates the envelope against +`EventEnvelopeSchema`, discards unknown event types, and applies `eventId` idempotency +before the handler is reached. + +**There is no raw `socket.on(type, ...)` fallback.** A test that reaches into the emitter +looking for a listener registered for `'send_message'` will find nothing — and a test +written that way against an older revision passes while bypassing envelope validation, +idempotency, and the auth gate, which is precisely the surface those checks exist to +protect. + +Drive handlers by emitting a well-formed envelope on `dispatch`: + +```ts +let envelopeSeq = 0; + +function dispatchEvent(socket: EventEmitter, type: string) { + return async (payload: unknown) => { + envelopeSeq += 1; + // EventEmitter.prototype.emit.call bypasses any emit override on the fake + // socket, so this delivers to the listener instead of being captured as an + // outbound server -> client emit. + EventEmitter.prototype.emit.call(socket, 'dispatch', { + eventId: `test-evt-${envelopeSeq}`, + type, + timestamp: Date.now(), + payload, + }); + await new Promise((resolve) => setTimeout(resolve, 10)); + }; +} +``` + +Points to preserve when copying this: + +- **`eventId` must be unique per emit.** Replay protection is scoped to the sending device + and keyed by `eventId`, so a reused value turns the second and later events into no-ops and + produces a confusing "the handler never ran" failure. Note that the check _fails open_ when + Redis is `null`: a suite that mocks Redis away will not catch a duplicate `eventId`, and the + same test starts failing the moment someone gives it an `ioredis-mock` instance. Generate a + fresh id every time regardless. +- **`timestamp` must be current.** The envelope is rejected as stale before the handler runs, + so use `Date.now()` rather than a frozen constant — and if the test uses + `vi.useFakeTimers()`, keep the envelope timestamp inside the accepted window. +- **The dispatch listener is `async`.** `emit` returns synchronously, before the handler has + finished, so await a tick — the `setTimeout` above — before asserting. +- **Set `socket.auth` first.** An unauthenticated socket gets an `error` envelope back and + the handler is never reached. +- **Register handlers through the real registrar** (`registerMessagingHandlers(io, socket)`) + rather than pulling a handler function out of the module, so whatever the registrar + installs stays in the path. + +Worked examples: `apps/backend/src/__tests__/dispatcher.test.ts` (envelope validation and +idempotency) and `apps/backend/src/__tests__/askAssistant.test.ts` (a handler driven through +`dispatch`). + +--- + +## Trap: shared in-process counters leak between tests + +Several modules keep module-level state that survives across tests within a Vitest worker, +and `vi.clearAllMocks()` does not touch it. Rate limiting is the one that bites most often. + +`services/rateLimiter.ts` keeps a `localCounters` map used whenever Redis is unavailable — +which, in a suite that mocks `redis` to `null`, is always. The map is keyed by bucket, +window, and subject, and the window comes from wall-clock time, so several tests in the same +file hitting the same endpoint as the same subject are all charged against **one** budget. +The symptom is a test that passes alone and returns `429` when the file runs in order, or a +failure that moves when you reorder the file. + +Reset explicitly in `beforeEach`: + +```ts +const { resetRateLimitBucket, clearLocalRateLimitCounters } = + await import('../services/rateLimiter.js'); + +beforeEach(async () => { + vi.clearAllMocks(); + clearLocalRateLimitCounters(); // drops the process-local fallback map + await resetRateLimitBucket('auth_challenge'); // drops the bucket's Redis keys too + await resetRateLimitBucket('auth_verify'); + await resetRateLimitBucket('global_ip'); +}); +``` + +`clearLocalRateLimitCounters()` clears only the in-process map. +`resetRateLimitBucket(bucket)` clears the matching local keys _and_ scans and deletes the +bucket's keys in Redis (real or `ioredis-mock`). When a suite shares one `ioredis-mock` +instance, `await sharedRedis.flushall()` in `beforeEach` is the blunter equivalent for the +Redis half. + +The same shape of leak exists elsewhere. Each affected module exports its own reset hook — +use it rather than reaching into the module: + +| Module | State it keeps | Reset | +| --------------------------- | ---------------------------------------------------------------- | --------------------------------------------------------------- | +| `services/rateLimiter.ts` | Fallback counters plus Redis buckets | `clearLocalRateLimitCounters()`, `resetRateLimitBucket(bucket)` | +| `services/rateLimit.ts` | Socket repeat-violation counts | Cleared alongside the buckets the socket path charges | +| Prekey low-watermark alerts | One-shot latches, so a second low-prekey event does not re-alert | `__resetPrekeyLowLatches()` | +| Presence | Offline-broadcast dedupe set | `__resetOfflineBroadcastsForTesting()` | + +The general rule: **if a module keeps state outside a function so that production behaves +correctly across requests, it needs a test-visible reset, and every suite that touches it +must call that reset in `beforeEach`.** When you add such state, export the reset in the +same commit. + +--- + +## General conventions + +- **Assert on behaviour, not on generated SQL.** With `drizzle-orm` mocked there is no SQL to + assert on. Check the status code, the response body, what was emitted to which room, and + which rows were handed to `insert`/`update`. +- **Prefer `supertest` against a small Express app** built from the router under test + (`app.use('/messages', messagesRouter)`) over importing the whole `app.ts`, unless the test + is specifically about middleware ordering. +- **Mock `../middleware/auth.js` to inject `req.auth`** rather than minting real JWTs, in + tests that are not themselves about authentication. Auth tests use the real middleware. +- **Never weaken a ciphertext or key invariant to make a test pass.** The guards in + `apps/backend/src/__tests__/security.regression.test.ts` have a dedicated CI job; if a + change trips them, the change is wrong, not the test. See the + [threat model](threat-model.md) for what those invariants protect. +- **Keep tests deterministic.** No real timers over real durations, no network, no randomness + that is not seeded. Use `vi.useFakeTimers()` for time-dependent logic, and the Soroban + virtual ledger clock for contract expiry. +- **Formatting and lint apply to tests.** `pnpm --filter backend format:check` and + `pnpm --filter backend lint` cover `src/`, which includes `__tests__/`. + +--- + +## Related documents + +- [Contract testing guide](../contracts/docs/testing.md) — the Soroban `Env` harness, auth + mocking, and cross-contract test setup. +- [Database migration workflow](../apps/backend/docs/migrations.md) — why the suite never + runs migrations, and how migrations are validated instead. +- [Threat model](threat-model.md) — the invariants the security regression tests defend. +- [Rate limits](security/rate-limits.md) — every bucket and its threshold, which is what the + rate-limit tests assert against. From ed612f1be6b1cd7278a895c3ae78909c9209ddd4 Mon Sep 17 00:00:00 2001 From: Deb-Auth Date: Sun, 30 Aug 2026 17:23:43 -0700 Subject: [PATCH 2/2] docs(backend): add error code and error response catalog Adds apps/backend/docs/contracts-error-catalog.md, cataloguing every error shape the backend returns across both transports, plus the matching rows in the documentation index. Contents: - One REST table, deduplicated to a row per distinct (status, error) pair, listing the extra fields each carries and every route that emits it. Middleware-level errors are split into their own table since they can accompany any authenticated route. - The dynamic REST statuses that do not appear as literal res.status calls: the validateMessagePayload results behind POST /messages, and the checkEnvelopeProtocols 400/409 pair with its violations array. - Rate-limit responses, which do not have one shape: the standard rateLimit() middleware sets RateLimit-Limit/Remaining/Reset plus Retry-After, the upload byte quota sets only Retry-After, and the group-invite throttle sets no headers at all. Documents the safe client rule for reading a backoff from any of the three. - Socket errors, keyed by the emitted event value, including the condition-valued ones: device_set_mismatch, protocol_mismatch, rate_limited, envelope_too_large, device_revoked, and payload_too_large. Notes that rate_limited carries the throttled event in limitedEvent rather than event, and that envelope_too_large is a code value rather than an event value. - The two distinct socket error payload shapes: the dispatcher wraps its rejections in the standard event envelope, while the security middleware and every messaging handler emit a bare payload. Includes the normalising snippet a client needs to handle both. - Handshake failures, which arrive on connect_error as a plain Error rather than as an error event, so a client listening only on error sees nothing. - A retryable/terminal breakdown, including the cases where a naive retry is actively wrong: an auth nonce is single-use so a 401 there means restarting from the challenge, a message retry must reuse the same messageId to stay idempotent, and an MLS epoch conflict must be rebuilt against the returned currentEpoch. Two findings surfaced while cataloguing, documented rather than changed: there is no global Express error handler, so an exception escaping a route returns Express's HTML page instead of a JSON body; and auditLogsRouter is defined and tested but never mounted in app.ts, so its two errors are currently unreachable over HTTP. Closes #555 --- apps/backend/docs/contracts-error-catalog.md | 487 +++++++++++++++++++ docs/README.md | 2 + 2 files changed, 489 insertions(+) create mode 100644 apps/backend/docs/contracts-error-catalog.md diff --git a/apps/backend/docs/contracts-error-catalog.md b/apps/backend/docs/contracts-error-catalog.md new file mode 100644 index 0000000..4a543c3 --- /dev/null +++ b/apps/backend/docs/contracts-error-catalog.md @@ -0,0 +1,487 @@ +# Error code and error response catalog + +Every error shape the backend can return, across both transports, in one place. If a client +is handling a failure, the response it received is listed here. + +The backend has two error surfaces and they do **not** share a format: + +- **REST** — a JSON body `{ error: string }`, sometimes with extra fields, and an HTTP status + that carries most of the meaning. +- **Socket.IO** — an `error` event whose payload names the originating client event and + sometimes carries a `code`. There is no HTTP status, and — importantly — the payload comes + in [two different shapes](#socket-errors-come-in-two-shapes) depending on which layer + rejected the event. + +There is no machine-readable error code enum on the REST side. The `error` string is the +identifier, so **match on the status first and the string second**, and treat the string as +stable-ish rather than guaranteed: several are built with template literals and interpolate +runtime values. + +--- + +## REST errors + +### The common shape + +```jsonc +// The only field every REST error has: +{ "error": "Not a member of this conversation" } +``` + +Some errors add fields alongside `error` — never nested under it. The "Extra fields" column +below is exhaustive per error. + +Three things to know before reading the table: + +1. **There is no global Express error handler.** An exception that escapes a route handler + falls through to Express's built-in handler, which returns an HTML error page with a + `500` status, not `{ error }`. A client that assumes every non-2xx body is JSON will throw + while parsing. Always branch on `content-type`, or guard the `.json()` call. +2. **`GET /audit-logs` is not mounted.** `auditLogsRouter` exists in `src/routes/auditLogs.ts` + and is exercised in tests, but `src/app.ts` never mounts it, so its two errors are + currently unreachable over HTTP. They are listed for completeness and flagged. +3. **`4xx` bodies are safe to show to a developer, not to an end user.** Several leak internal + vocabulary (`sha256 mismatch`, `Commit epoch conflict`). Map them to user-facing copy on + the client — see the web app's [error handling guide](../../web/docs/concepts-error-handling.md). + +### Errors emitted by middleware + +These can accompany **any** authenticated route, so they are not repeated per-route in the +main table. + +| Status | `error` | Extra fields | Emitted by | +| ------ | ---------------------------------------------------------------------------- | --------------------------------- | ----------------------------------------------------------------------- | +| `401` | Missing or invalid Authorization header | — | `requireAuth` — no `Authorization: Bearer` header | +| `401` | Invalid or expired token | — | `requireAuth` — JWT signature or expiry check failed | +| `401` | Token missing deviceId | — | `requireAuth` — token predates device binding | +| `401` | Device not found or has been revoked | — | `requireAuth` — the token's device was revoked | +| `403` | `tls_required` | `message` | `transportSecurity` — plaintext HTTP against an https-only deployment | +| `403` | `origin_not_allowed` | `message` | `transportSecurity` — `Origin` not on the allow-list | +| `400` | Validation failed | `issues[]` (`{ field, message }`) | `validate(schema)` — Zod rejection on a schema-guarded route | +| `429` | Too many requests | `bucket`, `retryAfterSeconds` | `rateLimit(bucket)` — see [Rate-limit responses](#rate-limit-responses) | +| `503` | _(no `error` key)_ — `{ status: 'error', db: 'unreachable', node, version }` | — | `GET /health` when `SELECT 1` fails | + +Note the two odd ones out. `tls_required` and `origin_not_allowed` are the only REST errors +whose `error` value is a **machine code** rather than a sentence, and the only ones that pair +it with a separate human-readable `message`. `GET /health` is the only failure response with +no `error` key at all. + +### The catalog + +One row per distinct `(status, error)` pair, listing every route that emits it. + +| Status | `error` | Extra fields | Route(s) | +| ------ | ----------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| `400` | A device cannot be added and removed by the same commit | — | POST /conversations/:id/mls/commits | +| `400` | Added devices must be active devices of conversation members | `invalidDeviceIds` | POST /conversations/:id/mls/commits | +| `400` | addedDevices contains duplicate deviceIds | — | POST /conversations/:id/mls/commits | +| `400` | allowDirectMessages must be a boolean | — | PATCH /users/me | +| `400` | allowGroupInvites must be a boolean | — | PATCH /users/me | +| `400` | At least one of muted or archived is required | — | PATCH /conversations/:id/settings | +| `400` | At least one of name or avatarUrl must be provided | — | PATCH /conversations/:id | +| `400` | avatarUrl must be a string | — | PATCH /conversations/:id | +| `400` | cipherSuite must be a positive integer | — | GET /users/:userId/devices/:deviceId/mls-key-package | +| `400` | Conversation id is required | — | DELETE /conversations/:id/leave; GET /conversations/:id; GET /conversations/:id/devices; GET /conversations/:id/epoch; GET /conversations/:id/group-control; GET /conversations/:id/members; GET /conversations/:id/messages; GET /conversations/:id/transfers; PATCH /conversations/:id; PATCH /conversations/:id/settings; POST /conversations/:id/group-control; POST /conversations/:id/members; POST /conversations/:id/transfers; all `/conversations/:id/mls/*` | +| `400` | deviceId is required | — | GET /sync | +| `400` | DM conversations cannot add members | — | POST /conversations/:id/members | +| `400` | DM conversations cannot be left | — | DELETE /conversations/:id/leave | +| `400` | DM conversations cannot be updated | — | PATCH /conversations/:id | +| `400` | Endpoint is required | — | DELETE /push/subscriptions | +| `400` | expiresAt must be in the future | — | POST /devices/:id/mls-key-packages | +| `400` | File id is required | — | GET /files/:fileId | +| `400` | fileId is required | — | POST /uploads/:fileId/confirm | +| `400` | Invalid cursor | — | GET /conversations/:id/messages; GET /sync | +| `400` | Invalid membership change metadata | — | POST /messages | +| `400` | Invalid request | `details` (Zod issues) | POST /uploads | +| `400` | lastSeenVisible must be a boolean | — | PATCH /users/me | +| `400` | Message id is required | — | DELETE /messages/:id | +| `400` | Missing endpoint or keys | — | POST /push/subscriptions | +| `400` | MLS groups are only available for group conversations | — | POST /conversations/:id/mls/group | +| `400` | name must be a string | — | PATCH /conversations/:id | +| `400` | No wallet is associated with this account | — | POST /devices/link/challenge; POST /devices/link/verify | +| `400` | payload must be a non-empty string | — | POST /conversations/:id/group-control | +| `400` | presenceVisible must be a boolean | — | PATCH /users/me | +| `400` | Query parameter "q" is required | — | GET /users/search | +| `400` | recipientAddress, amount, tokenContractId, and txHash are required | — | POST /conversations/:id/transfers | +| `400` | sendReadReceipts must be a boolean | — | PATCH /users/me | +| `400` | Signed prekey signature is invalid | — | POST /devices/:id/prekeys | +| `400` | sinceEpoch must be a non-negative integer | — | GET /conversations/:id/mls/commits | +| `400` | sinceSequence must be a non-negative integer | — | GET /conversations/:id/group-control | +| `400` | Token is missing a deviceId | — | all `/conversations/:id/mls/*` | +| `400` | Unknown action filter | `allowed` | GET /audit-logs **(router not mounted)** | +| `400` | User id is required | — | GET /users/:id/key-history | +| `400` | userId is required | — | POST /conversations/:id/members | +| `400` | Username must be 3-30 alphanumeric characters and underscores only | — | PATCH /users/me | +| `401` | Device has been revoked | — | POST /auth/verify | +| `401` | Invalid or expired device link nonce | — | POST /devices/link/verify | +| `401` | Invalid or expired nonce | — | POST /auth/verify | +| `401` | Invalid signature or wallet address | — | POST /auth/verify | +| `401` | Signature verification failed | — | POST /auth/verify; POST /devices/link/verify | +| `403` | Device is not a member of this conversation MLS group | — | POST /uploads | +| `403` | Device is not a member of this MLS group | — | GET /conversations/:id/mls/commits | +| `403` | Device is revoked | — | POST /devices/:id/mls-key-packages; POST /devices/:id/prekeys | +| `403` | Device not found or not owned by this user | — | GET /sync | +| `403` | Device registration requires a fresh wallet signature. Use POST /devices/link/challenge then POST /devices/link/verify. | — | POST /devices | +| `403` | Invalid or expired signed URL | — | GET /local-storage/\*splat; PUT /local-storage/\*splat | +| `403` | No shared conversation with device owner | — | GET /user-devices/:id/public-key | +| `403` | Not a member of this conversation | — | GET /conversations/:id; GET /conversations/:id/devices; GET /conversations/:id/epoch; GET /conversations/:id/group-control; GET /conversations/:id/members; GET /conversations/:id/messages; GET /conversations/:id/transfers; PATCH /conversations/:id; PATCH /conversations/:id/settings; POST /conversations/:id/group-control; POST /conversations/:id/members; POST /conversations/:id/transfers; POST /messages; POST /uploads; all `/conversations/:id/mls/*` | +| `403` | Not authorized to access this file | — | GET /files/:fileId | +| `403` | Not authorized to confirm this upload | — | POST /uploads/:fileId/confirm | +| `403` | Only an active group member device may publish a commit | — | POST /conversations/:id/mls/commits | +| `403` | Only the device owner may upload MLS key packages | — | POST /devices/:id/mls-key-packages | +| `403` | Only the device owner may upload prekeys | — | POST /devices/:id/prekeys | +| `403` | system messages are reserved for the server | — | POST /messages | +| `403` | This device has no key for this file | `reason` | GET /files/:fileId | +| `403` | User is not accepting group invites | — | POST /conversations/:id/members | +| `403` | You can only delete your own messages | — | DELETE /messages/:id | +| `404` | Conversation has no MLS group | — | GET /conversations/:id/mls/commits; GET /conversations/:id/mls/group; GET /conversations/:id/mls/pending-devices; GET /conversations/:id/mls/welcome; POST /conversations/:id/mls/commits | +| `404` | Conversation membership not found | — | DELETE /conversations/:id/leave | +| `404` | Conversation not found | — | DELETE /conversations/:id/leave; GET /conversations/:id; GET /conversations/:id/epoch; GET /conversations/:id/group-control; PATCH /conversations/:id; POST /conversations/:id/members | +| `404` | Device not found | — | DELETE /devices/:id; POST /devices/:id/mls-key-packages; POST /devices/:id/prekeys | +| `404` | Device not found or has been revoked | — | GET /users/:userId/devices/:deviceId/key-bundle; GET /users/:userId/devices/:deviceId/mls-key-package | +| `404` | Device not found or revoked | — | GET /user-devices/:id/public-key | +| `404` | File not found | — | GET /files/:fileId; POST /uploads/:fileId/confirm | +| `404` | File not referenced by any message | — | GET /files/:fileId | +| `404` | Message not found | — | DELETE /messages/:id | +| `404` | No active devices found for this user | — | GET /users/:id/key-fingerprint | +| `404` | No pending Welcome for this device | — | GET /conversations/:id/mls/welcome | +| `404` | Object not found | — | GET /local-storage/\*splat | +| `404` | Proposal not found | — | POST /treasury/proposals/:id/approve; POST /treasury/proposals/:id/reject | +| `404` | User not found | — | GET /users/:id; GET /users/:id/key-fingerprint; GET /users/:id/key-history; GET /users/:id/presence; GET /users/me; PATCH /users/me | +| `409` | Already voted on this proposal | — | POST /treasury/proposals/:id/approve; POST /treasury/proposals/:id/reject | +| `409` | Cannot revoke your only active device | — | DELETE /devices/:id | +| `409` | Commit epoch conflict — another commit was applied first | `currentEpoch`, `expectedEpoch` | POST /conversations/:id/mls/commits | +| `409` | Conversation already has an MLS group | `groupId`, `currentEpoch` on the pre-check path; absent on the insert-race path | POST /conversations/:id/mls/group | +| `409` | Database conflict or validation error | — | POST /conversations/:id/members; POST /conversations/:id/transfers | +| `409` | Device already registered for this user | — | POST /devices/link/verify | +| `409` | Device has not uploaded a signed prekey yet | — | GET /users/:userId/devices/:deviceId/key-bundle | +| `409` | Envelope protocol is weaker than both devices support | `violations` | POST /messages | +| `409` | File has been deleted | — | POST /uploads/:fileId/confirm | +| `409` | File is already ready | — | POST /uploads/:fileId/confirm | +| `409` | No MLS key packages available for this device | `remaining` | GET /users/:userId/devices/:deviceId/mls-key-package | +| `409` | Proposal is no longer active | — | POST /treasury/proposals/:id/approve; POST /treasury/proposals/:id/reject | +| `409` | Transaction hash already exists | — | POST /conversations/:id/transfers | +| `409` | User is already a member | — | POST /conversations/:id/members | +| `409` | Username conflict or database error | — | PATCH /users/me | +| `409` | Username is already taken | — | PATCH /users/me | +| `410` | Server-side search removed; search is now client-side over decrypted messages | `docs` | GET /conversations/:id/search | +| `413` | payload exceeds `${MAX_GROUP_CONTROL_PAYLOAD_BYTES}` bytes | — | POST /conversations/:id/group-control | +| `415` | Unsupported media type | `mimeType` | POST /uploads | +| `422` | File integrity verification failed | `expectedHash`, `computedHash` | POST /uploads/:fileId/confirm | +| `422` | MLS key package cap of `${MLS_KEY_PACKAGE_CAP}` reached. … | — | POST /devices/:id/mls-key-packages | +| `422` | Object not found in storage | `storageKey` | POST /uploads/:fileId/confirm | +| `422` | Object size mismatch | `expectedSize`, `actualSize` | POST /uploads/:fileId/confirm | +| `422` | One-time prekey cap of `${OTP_CAP}` reached. … | — | POST /devices/:id/prekeys | +| `422` | sha256 is required | — | POST /uploads/:fileId/confirm | +| `422` | sha256 mismatch | — | POST /uploads/:fileId/confirm | +| `429` | Daily upload quota exceeded | `bucket`, `retryAfterSeconds` | POST /uploads | +| `429` | Too many group invites. Please try again later. | — | POST /conversations/:id/members | +| `500` | Failed to add conversation member | — | POST /conversations/:id/members | +| `500` | Failed to append group control event | — | POST /conversations/:id/group-control | +| `500` | Failed to claim MLS key package | — | GET /users/:userId/devices/:deviceId/mls-key-package | +| `500` | Failed to compute key fingerprint | — | GET /users/:id/key-fingerprint | +| `500` | Failed to create MLS group | — | POST /conversations/:id/mls/group | +| `500` | Failed to create user | — | POST /auth/verify | +| `500` | Failed to delete subscription | — | DELETE /push/subscriptions | +| `500` | Failed to fetch device public key | — | GET /user-devices/:id/public-key | +| `500` | Failed to generate download URL | — | GET /files/:fileId | +| `500` | Failed to issue device link challenge | — | POST /devices/link/challenge | +| `500` | Failed to list devices | — | GET /devices | +| `500` | Failed to persist message | — | POST /messages | +| `500` | Failed to read audit log | — | GET /audit-logs **(router not mounted)** | +| `500` | Failed to register device | — | POST /auth/verify; POST /devices/link/verify | +| `500` | Failed to register subscription | — | POST /push/subscriptions | +| `500` | Failed to retrieve transfers | — | GET /conversations/:id/transfers | +| `500` | Failed to store object | — | PUT /local-storage/\*splat | +| `500` | Failed to update conversation | — | PATCH /conversations/:id | +| `500` | Search failed | — | GET /users/search | + +### Message payload validation (`POST /messages`) + +`POST /messages` returns the status **chosen by the validator**, so these do not appear as +literal `res.status(400)` calls in the route. All are `{ error: }` with no extra +fields, and all come from `validateMessagePayload` (`src/lib/validateMessagePayload.ts`): + +| Status | `error` | Condition | +| ------ | --------------------------------------------------------------------------------- | ---------------------------------------------------- | +| `403` | system messages are reserved for the server | `contentType: 'system'` from a client | +| `400` | unsupported contentType: "…" | Not one of `text`, `file`, `image`, `video`, `audio` | +| `400` | MLS group messages carry a single group ciphertext, not per-device envelopes | `mlsEpoch` set _and_ `envelopes` non-empty | +| `400` | ciphertext is required for MLS group messages | `mlsEpoch` set, `ciphertext` empty | +| `400` | fileId is required for file-type messages | `file`/`image`/`video`/`audio` with no `fileId` | +| `400` | envelopes are required for file-type messages (they carry the encrypted file key) | File-type message with no envelopes | +| `400` | text messages require at least one envelope with an encrypted key | Text message with no envelopes | + +The same validator backs the socket `send_message` and `send_file_message` handlers, where +its `code` is surfaced as the payload's `code` field rather than an HTTP status. + +### Protocol enforcement (`POST /messages` and `send_message`) + +`checkEnvelopeProtocols` (`src/services/e2eeProtocol.ts`) rejects envelopes that name a +protocol the recipient cannot use. Both errors carry a `violations` array: + +```jsonc +{ + "error": "Envelope protocol is not supported by the recipient device", + "violations": [ + { + "recipientDeviceId": "…", + "declared": "…", // what the envelope claimed + "expected": "…", // what the two devices should have negotiated + "reason": "unsupported_by_recipient", // or "downgrade" + }, + ], +} +``` + +| Status | `error` | When | +| ------ | ---------------------------------------------------------- | -------------------------------------------------------------------------------------------------------------- | +| `400` | Envelope protocol is not supported by the recipient device | At least one violation has `reason: 'unsupported_by_recipient'` — the recipient literally could not decrypt it | +| `409` | Envelope protocol is weaker than both devices support | Every violation is a `downgrade` — decryptable, but weaker than negotiated | + +An undecryptable envelope is the more specific failure, so `400` wins when a batch contains +both kinds. + +--- + +## Rate-limit responses + +`429` does not have one shape. There are **three** rate limiters with three different bodies +and three different header sets — worth knowing, because a client that reads `Retry-After` +unconditionally will get `null` from one of them. + +| Source | Body | `Retry-After` | `RateLimit-*` | +| ------------------------------------------------------- | ------------------------------------------------------------------------------------------- | ------------- | ------------- | +| `rateLimit(bucket)` middleware — the standard path | `{ error: 'Too many requests', bucket, retryAfterSeconds }` | ✅ | ✅ | +| `POST /uploads` daily byte quota | `{ error: 'Daily upload quota exceeded', bucket: 'upload_bytes_daily', retryAfterSeconds }` | ✅ | ❌ | +| `POST /conversations/:id/members` group-invite throttle | `{ error: 'Too many group invites. Please try again later.' }` | ❌ | ❌ | + +The standard middleware path sets all four headers: + +```http +HTTP/1.1 429 Too Many Requests +RateLimit-Limit: 30 +RateLimit-Remaining: 0 +RateLimit-Reset: 42 +Retry-After: 42 +``` + +```jsonc +{ "error": "Too many requests", "bucket": "auth_challenge", "retryAfterSeconds": 42 } +``` + +`RateLimit-Reset` and `Retry-After` are both **seconds until the window rolls over**, not a +timestamp, and they always carry the same value. The `RateLimit-*` headers are set on +**successful** responses too, reporting the tightest remaining budget across every bucket +checked — so a client can back off before being rejected rather than after. + +**The safe client rule:** read `retryAfterSeconds` from the body when present, fall back to +the `Retry-After` header, and fall back to a local backoff constant when neither exists. Only +the first row above guarantees both. + +Per-bucket thresholds are in [`docs/security/rate-limits.md`](../../../docs/security/rate-limits.md); +the mechanism is in `src/services/rateLimiter.ts`. + +--- + +## Socket errors + +### Socket errors come in two shapes + +Both arrive on the `error` event, and clients must handle both. + +**Shape A — bare payload.** Emitted by the per-socket security middleware (`src/index.ts`) +and every handler in `src/socket/messaging.ts`: + +```jsonc +{ + "event": "send_message", // which client event failed + "message": "Not a member of this conversation", + "code": "envelope_too_large", // sometimes; see the table +} +``` + +**Shape B — enveloped.** Emitted by the dispatcher itself (`src/socket/dispatcher.ts`), which +wraps the payload in the standard event envelope: + +```jsonc +{ + "eventId": "…", + "type": "error", + "timestamp": 1730000000000, + "payload": { "message": "Malformed envelope", "details": { … } } +} +``` + +The difference is _which layer_ rejected the event. The dispatcher rejects before dispatch — +unauthenticated socket, unparseable envelope, unknown event type, stale timestamp — and it +speaks envelopes. Everything past that point emits the bare form. So: + +```ts +socket.on('error', (raw) => { + const payload = raw?.type === 'error' && raw.payload ? raw.payload : raw; + // payload.message is always present; payload.event and payload.code may not be +}); +``` + +Note also that Shape B carries **no `event` field** for three of its four cases — only the +`Unauthenticated` rejection names one (`event: 'dispatch'`). Do not key error handling on +`payload.event` being present. + +### Dispatcher errors (Shape B) + +Raised before any handler runs, so no client event has been processed. + +| `payload.message` | Extra fields | Cause | +| ----------------------------------- | ------------------------------- | ----------------------------------------------------- | +| Unauthenticated | `event: 'dispatch'` | Socket has no `auth` — the handshake did not complete | +| Malformed envelope | `details` (flattened Zod error) | The `dispatch` payload failed `EventEnvelopeSchema` | +| Unknown event type: "…" | `eventId` | Envelope parsed, but `type` is not a known event | +| Stale or invalid envelope timestamp | `eventId` | `timestamp` outside the accepted window | + +A duplicate `eventId` is **not** an error: the dispatcher replies `dispatch_ack` with +`{ eventId, duplicate: true }` and does not re-run the handler. Replay protection fails open +when Redis is unavailable. + +### Socket errors keyed by `event` (Shape A) + +The `event` value is the routing key. Most name the client event that failed; four name a +_condition_ instead, and those are the non-obvious ones a client is most likely to mishandle. + +#### Condition-valued `event` — the non-obvious ones + +| `event` | `message` | Extra fields | Meaning | +| --------------------- | ------------------------------------------------------------------------------------------------------------------ | -------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| `device_set_mismatch` | Missing envelopes for N sibling device(s) | `missingDeviceIds[]` | The send omitted envelopes for some of the **sender's own** other devices. Those devices would never be able to read the message. Re-fetch the device list, re-encrypt for the full set, resend. Emitted from `send_message`, `edit_message`, and `send_file_message` — the `event` field does **not** tell you which. | +| `protocol_mismatch` | Envelope protocol is not supported by the recipient device / Envelope protocol is weaker than both devices support | `code` (`400` or `409`), `violations[]` | An envelope named a protocol the recipient cannot use, or downgraded below what both devices support. `violations[]` is the same shape as the REST version above. | +| `rate_limited` | Rate limit exceeded | `limitedEvent`, `limit`, `retryAfterSeconds` | Per-event socket rate limit. **`limitedEvent`, not `event`, names what was throttled** — `event` is the literal string `rate_limited`. Three violations on one socket force a disconnect. | +| `envelope_too_large` | Envelope for device … exceeds size limit | — | Arrives as `code: 'envelope_too_large'` **inside** a normal `event: 'send_message'` / `'edit_message'` / `'send_file_message'` payload — it is a `code` value, not an `event` value. Listed here because it reads like a peer of the others. Distinct from `payload_too_large`, which is the whole-frame limit. | + +Two further condition-valued events come from the connection middleware in `src/index.ts`: + +| `event` | `message` | Extra fields | Meaning | +| ------------------- | ---------------------------- | ------------ | ------------------------------------------------------------------------------------------------ | +| `device_revoked` | Device has been revoked | — | The device was revoked mid-session. The socket is disconnected immediately afterwards. Terminal. | +| `payload_too_large` | Payload size N exceeds limit | — | The whole event frame exceeded `MAX_PAYLOAD_SIZE`. The event is dropped; the socket stays open. | + +#### Handler-valued `event` + +| `event` | `message` | Extra fields | +| --------------------- | -------------------------------------------------------------------------------- | ---------------------------- | +| `join_room` | Not a member of this conversation | — | +| `send_message` | messageId is required | — | +| `send_message` | Field "…" is not permitted: the server never stores session or private-key state | `code: 400` | +| `send_message` | _(from `validateMessagePayload`)_ | `code` (`400`/`403`) | +| `send_message` | Envelope for device … exceeds size limit | `code: 'envelope_too_large'` | +| `send_message` | Not a member of this conversation | — | +| `send_message` | Failed to persist message | — | +| `edit_message` | Field "…" is not permitted: the server never stores session or private-key state | `code: 400` | +| `edit_message` | originalMessageId and messageId are required | — | +| `edit_message` | Content (envelope ciphertext) must not be empty | — | +| `edit_message` | Envelope for device … exceeds size limit | `code: 'envelope_too_large'` | +| `edit_message` | Original message not found | — | +| `edit_message` | Only the original sender can edit this message | — | +| `edit_message` | Failed to persist message edit | — | +| `send_file_message` | messageId is required | — | +| `send_file_message` | Content (envelope ciphertext) must not be empty | — | +| `send_file_message` | _(from `validateMessagePayload`)_ | `code` (`400`/`403`) | +| `send_file_message` | Envelope for device … exceeds size limit | `code: 'envelope_too_large'` | +| `send_file_message` | Not a member of this conversation | — | +| `send_file_message` | File not found | — | +| `send_file_message` | File is not ready for use | — | +| `send_file_message` | File does not belong to this conversation | — | +| `send_file_message` | Access denied: you are not the uploader of this file | — | +| `send_file_message` | Failed to persist file message | — | +| `message_history` | Not a member of this conversation | — | +| `delete_message` | Message not found or not sender | — | +| `message_read` | Not a member of this conversation | — | +| `message_read` | Message not found in conversation | — | +| `message_delivered` | conversationId and messageId are required | — | +| `message_delivered` | Not a member of this conversation | — | +| `create_conversation` | One or more recipients are not accepting direct messages / … group invites | `blockedUserIds[]` | +| `create_conversation` | Too many new conversations. Please wait before starting another. | — | +| `create_conversation` | Failed to create conversation | — | +| `typing_start` | Invalid conversationId | — | +| `typing_start` | Not a member of this conversation | — | +| `typing_stop` | Invalid conversationId | — | +| `typing_stop` | Not a member of this conversation | — | +| `ask_assistant` | Not a member of this conversation | — | +| `ask_assistant` | Failed to get AI reply | — | + +`ask_assistant` also has its own `rate_limited` emission with `limitedEvent: 'ask_assistant'`, +separate from the generic per-event limiter. + +### Handshake failures + +These are **not** `error` events. They reject the connection during the Socket.IO handshake +and surface client-side on `connect_error` as a plain `Error` with only a message: + +| Message | Source | +| -------------------------------------------------- | ---------------------------------------------------------------- | +| Authentication token required | `socketAuth` — no token in the handshake | +| Invalid or expired token | `socketAuth` — JWT rejected | +| Device not found or has been revoked | `socketAuth` — token's device is gone or revoked | +| Insecure transport: connect over wss:// | `socketSecurity` — plaintext ws against an https-only deployment | +| Origin is not permitted to open a socket | `socketSecurity` — `Origin` not on the allow-list | +| Cookies may not be sent over an insecure handshake | `socketSecurity` — cookies on a non-TLS handshake | + +A client that only listens on `error` will see nothing at all for any of these. + +--- + +## Retryable vs terminal + +"Retryable" here means _the identical request may succeed later without the user changing +anything_. It is not the same as "recoverable" — most terminal errors are fixable, but only by +changing the request, the state, or the session. + +### Terminal — do not retry + +| Errors | Why | +| ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------- | +| Every `400`, plus socket `code: 400` | The request is malformed. Retrying sends the same malformed request. Fix the payload. | +| `401` on `/auth/*` and `/devices/link/verify` (`Invalid or expired nonce`, `Signature verification failed`, `Invalid signature or wallet address`) | The challenge is single-use and consumed. **Restart from `POST /auth/challenge`** — never resubmit the same nonce. | +| `403` authorization errors (`Not a member of this conversation`, `Only the device owner may …`, `Not authorized to access this file`) | Membership or ownership does not change because you asked twice. | +| `403` `system messages are reserved for the server` | The client may never send this. | +| `403` `tls_required`, `origin_not_allowed` | Deployment configuration. Reconnect over https/wss or from an allowed origin. | +| `404` on a resource id the client supplied | The row does not exist. `Conversation has no MLS group` is the one worth special-casing: create the group rather than retrying. | +| `409` uniqueness conflicts (`Username is already taken`, `User is already a member`, `Transaction hash already exists`, `Already voted on this proposal`, `Device already registered for this user`) | The conflicting state is durable. Several are effectively idempotency signals — `Transaction hash already exists` means your transfer already landed. | +| `409` `File is already ready` | The upload was already confirmed. Treat as success. | +| `410` `Server-side search removed` | The endpoint is gone permanently. Use client-side search. | +| `413`, `415`, and `422` integrity failures (`sha256 mismatch`, `Object size mismatch`, `File integrity verification failed`) | The bytes are wrong or too big. Re-encrypt or re-upload; do not resend the same request. | +| `422` prekey / key-package caps | Retrying fails identically until existing keys are consumed. | +| Socket `device_set_mismatch` | Re-fetch the device list and re-encrypt first. Resending the same envelope set fails identically. | +| Socket `protocol_mismatch` | Re-negotiate the protocol first. | +| Socket `device_revoked` | The session is over and the socket is disconnected. Re-authenticate. | +| Socket `payload_too_large`, `envelope_too_large` | Shrink the payload. | +| All handshake failures | Fix the token, transport, or origin. | + +### Retryable — with backoff + +| Errors | How | +| ------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | +| `429`, all three variants | Wait `retryAfterSeconds` / `Retry-After`, then retry once. Do not retry immediately — three socket violations force a disconnect. | +| `500` `Failed to …` | Transient server or database failure. Exponential backoff, bounded attempts. **`Failed to persist message` is the one to be careful with**: retry with the _same_ `messageId`, since the send path is idempotent on it — a new id would duplicate the message if the first attempt actually committed. | +| `503` from `GET /health` | The database is unreachable. Poll until healthy. | +| `409` `Commit epoch conflict` | A genuine race: another commit landed first. **Rebuild the commit against `currentEpoch` from the response**, then resend. Retrying the same commit body fails identically. | +| `409` `Database conflict or validation error`, `Username conflict or database error` | Ambiguous by design — these wrap both real conflicts and transient database errors. Retry once; if it repeats, treat as terminal. | +| Socket `Failed to persist message` / `Failed to persist message edit` / `Failed to create conversation` | Same as `500` above, with the same idempotency caution. | + +### Neither — re-authenticate + +`401` from `requireAuth` (`Missing or invalid Authorization header`, `Invalid or expired +token`, `Token missing deviceId`, `Device not found or has been revoked`) means the session is +no longer valid. Refresh or re-authenticate, then replay the original request once. +`Device not found or has been revoked` specifically means the device is gone — a token refresh +will not help; the user must link the device again. + +--- + +## Related documents + +- [WebSocket events](api-websocket-events.md) — every event the gateway accepts and emits. +- [WebSocket payloads](contracts-websocket-payloads.md) — success-path payload shapes. +- [REST schemas](contracts-rest-schemas.md) — success-path request and response bodies. +- [Rate limits](../../../docs/security/rate-limits.md) — every bucket and its threshold. +- [Frontend error handling](../../web/docs/concepts-error-handling.md) — how the client maps + these onto user-facing messages. diff --git a/docs/README.md b/docs/README.md index c3aa442..6be7af6 100644 --- a/docs/README.md +++ b/docs/README.md @@ -81,6 +81,7 @@ listener. | --- | --- | | [JWT auth contract](../apps/backend/docs/contracts-jwt-auth.md) | Token claim shape, signing algorithm, and expiry rules. | | [REST schemas](../apps/backend/docs/contracts-rest-schemas.md) | Request and response body schemas shared across the REST surface. | +| [Error code and response catalog](../apps/backend/docs/contracts-error-catalog.md) | Every error the backend can return on either transport: the REST status/`error` table, the socket `error` payload shapes, the rate-limit response, and which errors are retryable. | | [WebSocket payloads](../apps/backend/docs/contracts-websocket-payloads.md) | Payload shapes for each WebSocket event, as validated on the wire. | **Encryption and migrations** @@ -125,6 +126,7 @@ is about encryption and local state. | [REST client](../apps/web/docs/api-rest-client.md) | The typed wrapper around the backend REST surface. | | [WebSocket client](../apps/web/docs/api-websocket-client.md) | Socket lifecycle, reconnection, and event subscription on the client. | | [Soroban client](../apps/web/docs/api-soroban-client.md) | How the web app builds, signs, and submits Soroban contract invocations. | +| [Backend error catalog](../apps/backend/docs/contracts-error-catalog.md) | Every error the client can receive from the backend, both transports, and which ones are worth retrying. | | [Auth session contract](../apps/web/docs/contracts-auth-session.md) | The shape of the persisted session and what invalidates it. | | [IndexedDB schemas](../apps/web/docs/contracts-indexeddb-schemas.md) | Every IndexedDB object store, its keys, and its migration history. | | [Response types](../apps/web/docs/contracts-response-types.md) | Shared TypeScript response types used across the client. |