diff --git a/README.md b/README.md index f5e35b0..c793e85 100644 --- a/README.md +++ b/README.md @@ -152,8 +152,19 @@ pnpm test ## πŸ“š Documentation +**➑️ [Documentation index](docs/README.md) β€” every document in this repository, grouped by +reader: new contributor, backend, frontend, contract developer, operator, security reviewer.** +Start there; it links to everything below and is kept exhaustive. + +New to the project? Read the [system architecture overview](docs/architecture-overview.md) +first β€” one diagram of all four apps and the external services, plus two end-to-end traces. + +Frequently opened: + - [Devices & Prekeys API](apps/backend/docs/api-devices.md) β€” every `/devices` and `/user-devices` route: request/response shapes, ownership checks, prekey upload contract, and revocation side effects - [E2EE Onboarding Sequence](apps/backend/docs/e2ee-onboarding.md) β€” device registration and prekey upload flow for first-contact DM setup +- [Contract events reference](contracts/docs/contracts-events.md) β€” every on-chain event, its payload, and which ones the backend listener consumes +- [WASM size and resource budget](contracts/docs/concepts-resource-budget.md) β€” the 100 KB per-contract CI gate, current sizes, and headroom --- diff --git a/contracts/docs/concepts-resource-budget.md b/contracts/docs/concepts-resource-budget.md new file mode 100644 index 0000000..f8a0ddc --- /dev/null +++ b/contracts/docs/concepts-resource-budget.md @@ -0,0 +1,244 @@ +# WASM Size and Resource Budget + +Deployed contract size is enforced in CI. This document covers what the gate is, where it +lives, how much room each contract currently has, which build settings move the number, +what to do when a contract approaches the limit, and why raw size is only one of several +budgets a Soroban contract has to live within. + +--- + +## The gate + +**Every release WASM must be ≀ 102,400 bytes (100 KB). A contract over the limit fails +the build.** + +Enforced in [`.github/workflows/contracts-ci.yml`](../../.github/workflows/contracts-ci.yml), +in the `Report WASM binary sizes` step of the `test-and-build` job: + +```bash +THRESHOLD_BYTES=102400 # 100 KB +... +if [ "$bytes" -gt "$THRESHOLD_BYTES" ]; then + echo "::error file=${name}::WASM size ${bytes} bytes exceeds ${THRESHOLD_BYTES} byte limit" + FAILED=1 +fi +... +exit $FAILED +``` + +Mechanics worth knowing: + +- **Triggers** on any push or pull request touching `contracts/**` or the workflow file + itself, and on a weekly Monday 08:00 UTC schedule. +- **Runs per contract.** The job is a matrix over `token_transfer`, `group_treasury`, and + `proposals`, with `fail-fast: false`, so one oversized contract does not mask the others. +- **Measures the plain `cargo build --release` artifact** from + `target/wasm32-unknown-unknown/release/`. There is no post-processing step in CI β€” no + `wasm-opt`, no `stellar contract optimize` β€” so the number gated is the raw compiler + output. +- **Reports before it fails.** The step is `if: always()` and computes the whole size table + before exiting non-zero, so a failing run still shows every contract's size. +- **Posts a size table to the pull request** as an update-in-place comment, keyed on a + `` marker so repeated pushes edit one comment instead of + accumulating. The same table is written to the job summary. Comment failures are + swallowed with `|| true`; they do not fail the build. +- **The gate is per-contract, not cumulative.** Three contracts at 90 KB each all pass. + +> The size step shells out to `bc`. It is present on the `ubuntu-latest` runner but is not +> installed by default in every environment β€” worth knowing if you reproduce this step +> locally and it fails on a missing command rather than on size. + +--- + +## Current sizes and headroom + +Measured from `cargo build --release --target wasm32-unknown-unknown` at the current +workspace settings: + +| Contract | Size | % of gate | Headroom | +| --- | --- | --- | --- | +| `group_treasury` | 31,843 B (31.10 KB) | 31.1 % | 70,557 B (68.90 KB) | +| `proposals` | 25,931 B (25.32 KB) | 25.3 % | 76,469 B (74.68 KB) | +| `token_transfer` | 10,539 B (10.29 KB) | 10.3 % | 91,861 B (89.71 KB) | + +**Nothing is close to the limit.** The largest contract uses under a third of its budget, +and every contract could roughly triple in size before CI complains. Reproduce these +numbers with: + +```bash +cd contracts +cargo build --release --target wasm32-unknown-unknown +wc -c target/wasm32-unknown-unknown/release/*.wasm +``` + +Sizes will drift with `soroban-sdk` upgrades as much as with your own code, so treat the +table as a snapshot rather than a constant β€” the reproduction command above is the +authority. + +The relative ordering is what you would expect from the source: `group_treasury` carries +membership management, per-token balances, and a full proposal-voting flow; `proposals` +carries a voting flow plus a cross-contract client; `token_transfer` is a thin authorised +wrapper over a SEP-41 transfer. + +--- + +## Release profile settings that affect size + +From [`contracts/Cargo.toml`](../Cargo.toml): + +```toml +[profile.release] +opt-level = "z" +overflow-checks = true +debug = 0 +strip = "symbols" +debug-assertions = false +panic = "abort" +codegen-units = 1 +lto = true +``` + +Every one of these is already tuned for size. What each does, and what it costs to change: + +| Setting | Effect on size | Cost of changing it | +| --- | --- | --- | +| `opt-level = "z"` | Optimises for size over speed; the single biggest lever. | Moving to `"s"` trades a little size for a little speed; `3` inflates size significantly. On Soroban, CPU is metered separately and code size is charged at deploy, so `"z"` is usually right. | +| `lto = true` | Whole-program link-time optimisation; removes cross-crate dead code. Large win against a big dependency like `soroban-sdk`. | Slower builds. Disabling it inflates size noticeably. | +| `codegen-units = 1` | One codegen unit lets LLVM see everything at once, improving both inlining and dead-code elimination. | Slower, non-parallel builds. Purely a build-time cost. | +| `strip = "symbols"` | Drops the symbol table from the artifact. | Backtraces lose names β€” irrelevant here, since `panic = "abort"` means no unwinding anyway. | +| `debug = 0` | No debug info in the binary. | No source-level debugging of the deployed artifact. | +| `panic = "abort"` | No unwinding tables or landing pads. Required for Soroban regardless. | Not optional in practice. Panics trap; there is nothing to unwind into. | +| `debug-assertions = false` | Drops `debug_assert!` and related checks. | Those checks stop running in release. | +| `overflow-checks = true` | **Costs** size β€” it keeps arithmetic overflow checks in the release build. | **Do not turn this off to save bytes.** These contracts move funds; a silent `i128` wraparound in a balance is a critical bug. This is a deliberate trade of size for safety, and the current headroom means there is no reason to revisit it. | + +There is also a `release-with-logs` profile that inherits `release` and re-enables +`debug-assertions`, for local debugging. It is not what CI measures. + +--- + +## When a contract approaches the limit + +In rough order of effort-to-payoff. Measure after each step β€” guessing at what is large is +usually wrong. + +### 1. Find out what is actually big + +Before changing anything, look at the breakdown: + +```bash +cargo install twiggy +twiggy top -n 30 target/wasm32-unknown-unknown/release/group_treasury.wasm +twiggy dominators target/wasm32-unknown-unknown/release/group_treasury.wasm +``` + +`twiggy top` ranks by retained size; `dominators` shows which item's removal would actually +free space. Frequently the bulk is a formatting or serialisation path pulled in by a single +call site. + +### 2. Run the optimiser + +CI gates the raw build output, but a deployed contract does not have to be that artifact: + +```bash +stellar contract optimize --wasm target/wasm32-unknown-unknown/release/group_treasury.wasm +``` + +This runs `wasm-opt` and typically removes a meaningful fraction. Note the asymmetry β€” the +optimised artifact is smaller than the number CI gates, so a contract that passes CI is +comfortably deployable, and a contract that *fails* CI may still be deployable. Fix the +source rather than relying on that gap. + +### 3. Remove panic message strings + +Every distinct `panic!("...")` literal is bytes in the data section. These contracts panic +with descriptive strings throughout (`"insufficient funds"`, `"proposal not found"`, +`"already voted"`, and so on). Converting to a `#[contracterror]` enum with numeric codes +removes the strings and gives callers a typed error instead: + +```rust +#[contracterror] +#[derive(Copy, Clone, Debug, Eq, PartialEq)] +#[repr(u32)] +pub enum Error { + InsufficientFunds = 1, + ProposalNotFound = 2, + AlreadyVoted = 3, +} +``` + +This is usually the largest available win in a contract of this style, and it improves the +client experience rather than degrading it. + +### 4. Avoid formatting machinery + +`format!`, `write!`, and `{:?}` pull in Rust's formatting infrastructure, which is large +relative to a 100 KB budget. `#[derive(Debug)]` on a type that is never actually printed +is usually free after LTO, but an actual `{:?}` call site is not. In `no_std` Soroban code +these should not appear in the shipped path at all β€” keep them behind `#[cfg(test)]`. + +### 5. Deduplicate generics and large call sites + +A generic function instantiated over many types produces a copy per instantiation. If a +helper is used across many `DataKey` variants, having it take a concrete type β€” or funnel +through one non-generic inner function β€” collapses several copies into one. The same +applies to a large function inlined at many call sites; `#[inline(never)]` can shrink the +total. + +### 6. Split the contract + +If one contract genuinely needs more than 100 KB of logic, that is a signal to split it +along a real boundary and call across contracts, as `proposals` already does with +`group_treasury`. Cross-contract calls cost CPU instructions at runtime, so this trades +deploy size for execution cost β€” make it a design decision, not a size workaround. + +### 7. Raise the gate β€” last resort + +`THRESHOLD_BYTES` in the workflow is a project policy, not a protocol limit. Raising it is +legitimate if a contract has genuinely outgrown the budget, but do it deliberately and in +its own commit with the reasoning, not as a way to make a red build green. + +--- + +## Size is not the only budget + +A contract that fits in 100 KB can still be too expensive to use. Soroban meters several +resources independently, and each has its own ceiling and its own fee component: + +- **CPU instructions.** Metered per invocation against a per-transaction ceiling. Loops + over unbounded collections are the usual way to exceed it. Several functions here iterate + the full member set (`is_member`, `add_member`, `remove_member`) or the full proposal + range (`list_proposals`, `get_pending_proposals`), so their cost grows linearly with the + treasury's membership and proposal history. These are the parts that will hit a ceiling + long before code size does. +- **Memory.** A per-invocation limit on linear memory. Materialising a large `Vec` β€” for + example building a list of every proposal ever created β€” is the usual cause. +- **Ledger entry reads and writes.** Both the number of entries touched and their total + byte size are metered, and writes cost considerably more than reads. Note that these + contracts keep everything in **instance** storage, including per-proposal entries and + per-voter vote keys, so the instance entry grows without bound as proposals and votes + accumulate β€” and the whole entry is read and written on every call that touches it. This + is the most likely resource problem in this codebase. +- **Events.** Event topics and data count toward the transaction's resource usage. The + events published here are small, but emitting one per vote in a large group adds up. +- **Rent / TTL.** Contract data expires unless its TTL is extended, and the cost of + extension scales with entry size and duration. A large, long-lived instance entry is a + recurring cost, not a one-off. This is the second reason to care about instance-storage + growth: it is charged for as long as the contract lives, whereas code size is charged + once at deploy. +- **Transaction size.** The signed transaction, including arguments and the authorisation + footprint, has its own limit, independent of the contract's size. + +**Practical upshot:** the 100 KB gate protects deployability, and at 31 % of budget it is +not the binding constraint today. The constraint that will bind first is unbounded growth +in instance storage and the linear scans over it. When optimising a contract here, look at +storage layout and iteration before looking at code size. + +--- + +## Related documents + +- [Contracts README](../README.md) β€” workspace layout, toolchain, build and test +- [Deployment and invocation](api-deployment-invocation.md) β€” deploying and initialising each contract +- [Token transfer storage](contracts-token-transfer-storage.md) β€” storage keys and value types +- [Proposal lifecycle](concepts-proposal-lifecycle.md) β€” statuses and transitions +- [Contract events reference](contracts-events.md) β€” every published event and its payload diff --git a/contracts/docs/contracts-events.md b/contracts/docs/contracts-events.md new file mode 100644 index 0000000..576ed72 --- /dev/null +++ b/contracts/docs/contracts-events.md @@ -0,0 +1,484 @@ +# Contract Events Reference + +Every `env.events().publish(...)` call across the three Soroban contracts, with its topic +tuple, data payload, the state change it signals, and whether the event is emitted before +or after that state is written. + +These events are the only mechanism by which off-chain systems learn that on-chain state +changed. The backend's chain listener +([`apps/backend/src/services/stellarListener.ts`](../../apps/backend/src/services/stellarListener.ts)) +polls Soroban RPC `getEvents` on a cursor and turns them into database writes and +WebSocket pushes β€” see the [Treasury API doc](../../apps/backend/docs/api-treasury.md) and +the [deployment and invocation doc](api-deployment-invocation.md) for how the listener is +configured and started. + +**Emission order matters.** A consumer treats an event as proof that the state change +happened. Every event below is annotated with whether the `publish` call runs before or +after the corresponding `env.storage()` write, because two events in this codebase are +emitted *before* their write and one payload is therefore a projection of intended state +rather than committed state. + +--- + +## Summary + +| Contract | Topic | Consumed by the backend listener? | +| --- | --- | --- | +| `token_transfer` | `transfer` | βœ… Yes | +| `group_treasury` | `member_added` | ❌ No | +| `group_treasury` | `member_removed` | ❌ No | +| `group_treasury` | `deposit` | ❌ No | +| `group_treasury` | `withdraw` | ❌ No | +| `group_treasury` | `proposal_created` | βœ… Yes | +| `group_treasury` | `proposal_approved` | βœ… Yes | +| `group_treasury` | `proposal_rejected` | βœ… Yes | +| `group_treasury` | `withdraw_vote` | ❌ No | +| `proposals` | `proposal_created` | ⚠️ Only if the contract id is configured as a treasury contract | +| `proposals` | `vote_cast` | ❌ No | +| `proposals` | `proposal_finalized` | ❌ No | +| `proposals` | `proposal_expired` | ⚠️ Same caveat as above | +| `proposals` | `executed` | ❌ No | +| `proposals` | `execut` | ❌ No | + +The listener also subscribes to a topic named **`proposal_executed`, which no contract +publishes.** See [Consumption gaps](#consumption-gaps). + +--- + +## `token_transfer` + +Source: [`contracts/token_transfer/src/lib.rs`](../contracts/token_transfer/src/lib.rs), +event structs in [`storage.rs`](../contracts/token_transfer/src/storage.rs). + +### `transfer` + +| | | +| --- | --- | +| **Topic tuple** | `(Symbol::new(&env, "transfer"),)` β€” single-element | +| **Data type** | `TransferEvent` | +| **Emitted in** | `transfer()`, `lib.rs:56` | +| **Order** | **After** the token move. `token.transfer(...)` executes first, so the event follows the balance change. | +| **Consumed** | βœ… Yes | + +**Data payload** + +| Field | Type | Meaning | +| --- | --- | --- | +| `from` | `Address` | Sender; authorised via `from.require_auth()`. | +| `to` | `Address` | Recipient. | +| `amount` | `i128` | Amount in token units; guaranteed `> 0` (the call panics otherwise). | +| `memo` | `Bytes` | Opaque reference. When the transfer originated from a chat message this carries that message's UUID. | + +**State change signalled.** `amount` of the configured SEP-41 token has moved from `from` +to `to`. Note that `token_transfer` holds no balance state of its own β€” the authoritative +state change is in the token contract, and this event is the record that the routed +transfer succeeded. + +**How the backend consumes it.** `buildRpcFetcher` filters on `topics: [['transfer']]` for +the `TOKEN_TRANSFER_CONTRACT_ID`. `defaultPersistEvent` hex-decodes `memo`, and if it +parses as a UUID matching a row in `messages`, associates the transfer with that message's +conversation and sender. It then upserts into `token_transfers` keyed on `tx_hash`, so a +reconnect that re-reads a page produces no duplicates. + +> **Consumer caveat.** If the memo is absent or is not a UUID of an existing message, the +> listener falls back to *the first row* of `conversations` and `users` rather than +> skipping the row, because `conversationId` and `senderId` are non-nullable. Transfers +> not originating from a chat message are therefore attributed to an arbitrary +> conversation and user. + +--- + +## `group_treasury` + +Source: [`contracts/group_treasury/src/lib.rs`](../contracts/group_treasury/src/lib.rs), +event structs in [`storage.rs`](../contracts/group_treasury/src/storage.rs). + +This contract publishes nine events across membership, funds movement, and every proposal +transition. + +### Membership + +#### `member_added` + +| | | +| --- | --- | +| **Topic tuple** | `(Symbol::new(&env, "member_added"),)` | +| **Data type** | `MemberAddedEvent` | +| **Emitted in** | `add_member()`, `lib.rs:79` | +| **Order** | **After** the write β€” the updated `Members` vector is persisted first. | +| **Consumed** | ❌ No | + +| Field | Type | Meaning | +| --- | --- | --- | +| `member` | `Address` | The address added to the member set. | +| `added_by` | `Address` | The admin that performed the change. | + +**State change.** `DataKey::Members` now contains `member`. Admin-only; re-adding an +existing member panics. + +#### `member_removed` + +| | | +| --- | --- | +| **Topic tuple** | `(Symbol::new(&env, "member_removed"),)` | +| **Data type** | `MemberRemovedEvent` | +| **Emitted in** | `remove_member()`, `lib.rs:116` | +| **Order** | **After** the write. | +| **Consumed** | ❌ No | + +| Field | Type | Meaning | +| --- | --- | --- | +| `member` | `Address` | The address removed. | +| `removed_by` | `Address` | The admin that performed the change. | + +**State change.** `DataKey::Members` no longer contains `member`. Removing a non-member +panics. + +> **Note for consumers.** Membership changes the denominator of the rejection rule β€” the +> blocking minority is computed from the *current* member count at vote time. Since these +> two events are unconsumed, the backend's mirror of the member set can drift from chain. + +### Funds + +#### `deposit` + +| | | +| --- | --- | +| **Topic tuple** | `(Symbol::new(&env, "deposit"),)` | +| **Data type** | `DepositEvent` | +| **Emitted in** | `deposit()`, `lib.rs:168` | +| **Order** | **After** both the token transfer and the balance write. | +| **Consumed** | ❌ No | + +| Field | Type | Meaning | +| --- | --- | --- | +| `from` | `Address` | Depositor; authorised via `require_auth()`. | +| `amount` | `i128` | Amount deposited; guaranteed `> 0`. | + +**State change.** Tokens moved into the contract and `DataKey::Balances[token]` increased +by `amount`. + +> **Payload gap.** `DepositEvent` does not carry the `token` address, even though the +> function takes one and balances are tracked per token. A consumer cannot tell which +> token was deposited from the event alone. + +#### `withdraw` + +| | | +| --- | --- | +| **Topic tuple** | `(Symbol::new(&env, "withdraw"),)` | +| **Data type** | `WithdrawEvent` | +| **Emitted in** | `withdraw()`, `lib.rs:198` | +| **Order** | **After** both the token transfer and the balance write. | +| **Consumed** | ❌ No | + +| Field | Type | Meaning | +| --- | --- | --- | +| `to` | `Address` | Recipient of the funds. | +| `amount` | `i128` | Amount withdrawn. | + +**State change.** Tokens left the contract and `DataKey::Balances[token]` decreased by +`amount`. Admin-only, and panics if the balance is insufficient. + +This is also the event emitted when `proposals::execute_withdraw` calls into this contract +cross-contract, so it is the on-chain record that an approved governance withdrawal +actually moved funds. + +> **Payload gap.** As with `deposit`, `WithdrawEvent` omits the `token` address. + +### Proposal lifecycle + +#### `proposal_created` + +| | | +| --- | --- | +| **Topic tuple** | `(Symbol::new(&env, "proposal_created"),)` | +| **Data type** | `ProposalCreatedEvent` | +| **Emitted in** | `propose_withdraw()`, `lib.rs:275` | +| **Order** | **After** the proposal and the proposer's auto-approval vote are written. | +| **Consumed** | βœ… Yes β†’ status `active` | + +| Field | Type | Meaning | +| --- | --- | --- | +| `id` | `u32` | Proposal id, assigned from `ProposalCount`. | +| `proposer` | `Address` | Must be a member; auto-approves, so the proposal starts at `approvals = 1`. | +| `to` | `Address` | Withdrawal recipient if the proposal passes. | +| `token` | `Address` | Token to withdraw. | +| `amount` | `i128` | Amount requested; checked against the current balance at creation. | +| `expires_at` | `u64` | Unix timestamp, computed as `now + ttl_ledgers * 5` (β‰ˆ5 s per ledger). | + +**State change.** A new `WithdrawProposal` exists with `status = Active`, +`approvals = 1`, `rejections = 0`; `ProposalCount` incremented; the proposer's vote +recorded. + +#### `proposal_approved` + +| | | +| --- | --- | +| **Topic tuple** | `(Symbol::new(&env, "proposal_approved"),)` | +| **Data type** | `ProposalApprovedEvent` | +| **Emitted in** | `approve_withdraw()`, `lib.rs:311` β€” **conditional**, only when the threshold is reached | +| **Order** | ⚠️ **Before** the write. The status is set on the in-memory struct and the event published, and only then is the proposal persisted (`lib.rs:320`). | +| **Consumed** | βœ… Yes β†’ status `approved` | + +| Field | Type | Meaning | +| --- | --- | --- | +| `id` | `u32` | Proposal id. | +| `approvals` | `u32` | Approval count that met the threshold. | +| `threshold` | `u32` | The configured threshold, for context. | + +**State change.** The proposal transitioned `Active β†’ Passed`. It is now executable. + +Emitted at most once per proposal, only on the vote that crosses the threshold. Because +the whole call is one atomic transaction, the pre-write emission is not observable to an +off-chain consumer β€” if the transaction reverts, no event is delivered β€” but the ordering +is worth knowing when reading the contract. + +#### `proposal_rejected` + +| | | +| --- | --- | +| **Topic tuple** | `(Symbol::new(&env, "proposal_rejected"),)` | +| **Data type** | `ProposalRejectedEvent` | +| **Emitted in** | `reject_withdraw()`, `lib.rs:361` β€” **conditional**, only at the blocking minority | +| **Order** | ⚠️ **Before** the write, same pattern as `proposal_approved` (persisted at `lib.rs:369`). | +| **Consumed** | βœ… Yes β†’ status `rejected` | + +| Field | Type | Meaning | +| --- | --- | --- | +| `id` | `u32` | Proposal id. | +| `rejections` | `u32` | Rejection count that reached the blocking minority. | + +**State change.** The proposal transitioned `Active β†’ Rejected`. The blocking minority is +`member_count.saturating_sub(threshold) + 1` β€” the point at which the remaining members can +no longer reach `threshold` approvals. + +#### `withdraw_vote` + +| | | +| --- | --- | +| **Topic tuple** | `(Symbol::new(&env, "withdraw_vote"),)` | +| **Data type** | `WithdrawVoteCastEvent` | +| **Emitted in** | `approve_withdraw()` `lib.rs:325` **and** `reject_withdraw()` `lib.rs:374` | +| **Order** | **After** the proposal write in both paths. | +| **Consumed** | ❌ No | + +| Field | Type | Meaning | +| --- | --- | --- | +| `id` | `u32` | Proposal id. | +| `voter` | `Address` | The member who voted. | +| `approve` | `bool` | `true` = approve, `false` = reject. | + +**State change.** One vote recorded at `DataKey::Vote(id, voter)`, and the proposal's +`approvals` or `rejections` counter incremented by one. Each member may vote at most once. + +This is emitted on **every** vote, whereas `proposal_approved` / `proposal_rejected` fire +only on the transition. A vote that crosses the threshold therefore produces two events in +one transaction β€” the transition event first, then `withdraw_vote`. + +> **Note.** The proposer's auto-approval at creation does **not** emit a `withdraw_vote`, +> even though it is recorded as a vote. Counting `withdraw_vote` events undercounts +> approvals by one. + +--- + +## `proposals` + +Source: [`contracts/proposals/src/lib.rs`](../contracts/proposals/src/lib.rs), event +structs in [`storage.rs`](../contracts/proposals/src/storage.rs). Note this contract uses +`u64` proposal ids, whereas `group_treasury` uses `u32`. + +### `proposal_created` + +| | | +| --- | --- | +| **Topic tuple** | `(Symbol::new(&env, "proposal_created"),)` | +| **Data type** | `ProposalCreatedEvent` (distinct from the `group_treasury` type of the same name) | +| **Emitted in** | `create_proposal()`, `lib.rs:100` | +| **Order** | **After** the proposal and `NextProposalId` are written. | +| **Consumed** | ⚠️ Only if this contract id is configured as `GROUP_TREASURY_CONTRACT_ID` | + +| Field | Type | Meaning | +| --- | --- | --- | +| `id` | `u64` | Proposal id from `NextProposalId`. | +| `proposer` | `Address` | Creator; authorised via `require_auth()`. | +| `expires_at` | `u64` | Voting deadline; must be in the future. | +| `treasury` | `Address` | The `group_treasury` this proposal would withdraw from. | +| `token` | `Address` | Token to withdraw. | +| `to` | `Address` | Withdrawal recipient. | +| `amount` | `i128` | Amount requested; must be `> 0`. | + +**State change.** A new `Proposal` exists with `status = Active` and +`yes_votes = no_votes = 0`. Unlike `group_treasury`, the proposer does **not** auto-vote. + +> Note the shared topic name across two contracts with different payload shapes and +> different id widths. Consumers must disambiguate on the emitting contract id, not the +> topic alone. + +### `vote_cast` + +| | | +| --- | --- | +| **Topic tuple** | `(Symbol::new(&env, "vote_cast"),)` | +| **Data type** | `VoteCastEvent` | +| **Emitted in** | `vote()`, `lib.rs:144` | +| **Order** | **After** both the vote key and the updated proposal are written. | +| **Consumed** | ❌ No | + +| Field | Type | Meaning | +| --- | --- | --- | +| `id` | `u64` | Proposal id. | +| `voter` | `Address` | The voter. | +| `support` | `bool` | `true` = yes, `false` = no. | + +**State change.** `DataKey::Vote(id, voter)` set, and `yes_votes` or `no_votes` +incremented. One vote per address per proposal; voting after `expires_at` panics. + +### `proposal_finalized` + +| | | +| --- | --- | +| **Topic tuple** | `(Symbol::new(&env, "proposal_finalized"),)` | +| **Data type** | `ProposalFinalizedEvent` | +| **Emitted in** | `finalize_proposal()`, `lib.rs:180` | +| **Order** | **After** the status write. | +| **Consumed** | ❌ No | + +| Field | Type | Meaning | +| --- | --- | --- | +| `id` | `u64` | Proposal id. | +| `status` | `ProposalStatus` | The outcome: `Passed` or `Rejected`. | +| `yes_votes` | `u32` | Final yes tally. | +| `no_votes` | `u32` | Final no tally. | + +**State change.** `Active β†’ Passed` when `yes_votes > no_votes`, otherwise +`Active β†’ Rejected`. Callable by anyone, but only after `expires_at`. A tie rejects. + +> **Consumption gap.** This is the event that decides a governance proposal's outcome, and +> nothing consumes it. The backend has no listener-driven path to learn that a `proposals` +> vote passed. + +### `proposal_expired` + +| | | +| --- | --- | +| **Topic tuple** | `(Symbol::new(&env, "proposal_expired"),)` | +| **Data type** | `ProposalExpiredEvent` | +| **Emitted in** | `finalize_expired_proposal()`, `lib.rs:208` | +| **Order** | **After** the status write. | +| **Consumed** | ⚠️ Only if this contract id is configured as `GROUP_TREASURY_CONTRACT_ID` | + +| Field | Type | Meaning | +| --- | --- | --- | +| `id` | `u64` | Proposal id. | + +**State change.** `Active β†’ Expired`. This is an alternative terminal path to +`finalize_proposal`: whichever is called first wins, since both require `Active`. + +### `executed` + +| | | +| --- | --- | +| **Topic tuple** | `(symbol_short!("executed"),)` | +| **Data type** | `ProposalExecutedEvent` | +| **Emitted in** | `execute_proposal()`, `lib.rs:228` | +| **Order** | **After** the status write. | +| **Consumed** | ❌ No | + +| Field | Type | Meaning | +| --- | --- | --- | +| `id` | `u64` | Proposal id. | +| `executor` | `Address` | Whoever executed it. | + +**State change.** `Passed β†’ Executed`. This is the MVP execution path: it only flips the +status and emits, moving no funds. + +### `execut` + +| | | +| --- | --- | +| **Topic tuple** | `(symbol_short!("execut"),)` β€” note the truncated symbol | +| **Data type** | `ProposalExecutedEvent` | +| **Emitted in** | `execute_withdraw()`, `lib.rs:285` | +| **Order** | **After** the status write, and after the cross-contract withdrawal. | +| **Consumed** | ❌ No | + +| Field | Type | Meaning | +| --- | --- | --- | +| `id` | `u64` | Proposal id. | +| `executor` | `Address` | The treasury member who executed; must pass a membership check. | + +**State change.** `Passed β†’ Executed`, **and** funds have moved: the contract called +`group_treasury::withdraw` cross-contract before writing the status. That call causes +`group_treasury` to emit its own `withdraw` event in the same transaction. + +So an executed governance withdrawal produces two events from two contracts: +`group_treasury::withdraw` (funds moved) and `proposals::execut` (proposal closed). + +> **Naming defect.** Two different topics β€” `executed` and `execut` β€” signal the same +> logical transition with the same payload type. `symbol_short!` caps at 9 characters, so +> `"executed"` (8) was in range and the truncation in `execut` appears unintentional rather +> than forced. Consumers must match both. + +--- + +## Consumption gaps + +Collected here because these are the differences between what the contracts publish and +what the backend actually reacts to. Each is a real defect, not a design choice. + +### 1. The listener subscribes to a topic nothing publishes + +`stellarListener.ts` declares `TREASURY_TOPICS` as: + +``` +proposal_created, proposal_approved, proposal_rejected, proposal_executed, proposal_expired +``` + +**No contract publishes `proposal_executed`.** `proposals` publishes `executed` and +`execut`; `group_treasury` has no execution event at all. The listener's status map +contains `proposal_executed: 'executed'`, so the code path exists but is unreachable β€” +a proposal never reaches the `executed` status in the off-chain mirror by way of the +listener. The [treasury API doc](../../apps/backend/docs/api-treasury.md) and +[deployment doc](api-deployment-invocation.md) both list `proposal_executed` among the +watched events, which is accurate about the subscription and misleading about the effect. + +### 2. `group_treasury` has no execution event + +`group_treasury::withdraw` emits `withdraw`, but there is no +`proposal_executed`-equivalent tying a fund movement back to the proposal that authorised +it. A consumer watching only treasury topics cannot close the loop from +`proposal_approved` to "the money moved". + +### 3. Fund movement is entirely unconsumed + +`deposit` and `withdraw` β€” the two events that represent actual value movement β€” have no +consumer. Treasury balances shown off-chain cannot be maintained from the event stream as +it stands. + +### 4. Only one contract is watched for treasury topics + +The listener builds a single treasury fetcher from one `GROUP_TREASURY_CONTRACT_ID`. The +`proposals` contract is a separate deployment with its own id, so whichever id is +configured, the other contract's events are not polled. Since both contracts publish a +topic named `proposal_created` with different payloads and different id widths (`u32` vs +`u64`), pointing the listener at `proposals` would also mis-parse β€” the fetcher reads +`value.approvals` and `value.rejections`, which exist only on the `group_treasury` events. + +### 5. Vote-level events are unconsumed + +`withdraw_vote` and `vote_cast` are not consumed. Live vote tallies in the UI come from the +off-chain `proposalVotes` table written by the REST routes, not from chain. On-chain votes +cast directly against the contract are therefore invisible to the backend until a +transition event fires. + +--- + +## Related documents + +- [Backend chain listener source](../../apps/backend/src/services/stellarListener.ts) β€” the consumer +- [Treasury API](../../apps/backend/docs/api-treasury.md) β€” the REST surface and off-chain mirror +- [Deployment and invocation](api-deployment-invocation.md) β€” env vars that start the listener +- [Proposal lifecycle](concepts-proposal-lifecycle.md) β€” statuses and transitions in full +- [Token transfer flow](concepts-token-transfer-flow.md) β€” the in-chat payment path +- [System architecture overview](../../docs/architecture-overview.md) β€” how the listener fits the whole system diff --git a/docs/README.md b/docs/README.md new file mode 100644 index 0000000..85915a4 --- /dev/null +++ b/docs/README.md @@ -0,0 +1,212 @@ +# Documentation Index + +Every document in this repository, grouped by what you are trying to do rather than by +where the file happens to live. Each entry has a one-line description so you can pick the +right document without opening several first. + +If you are new here, follow [Start here](#start-here) and ignore the rest until you need it. + +> **Maintenance:** this index must be updated whenever a document is added, moved, or +> removed. A new `.md` file that is not listed here is effectively invisible β€” the whole +> point of this file is that it is exhaustive. See [Keeping this index honest](#keeping-this-index-honest). + +--- + +## Start here + +A first-time contributor should read these four, in this order. It is roughly an hour and +it is enough to make a scoped change and open a pull request. + +1. [Root README](../README.md) β€” what Clicked is, the tech stack, and how to install and + run the whole monorepo locally with `pnpm` and `docker compose`. +2. [System architecture overview](architecture-overview.md) β€” one diagram showing how the + four apps and the external services fit together, plus two end-to-end request traces. +3. The one app doc for the area you are touching β€” pick your entry point from + [By role](#by-role) below. +4. [Contributing guidelines](../README.md#-contributing) β€” branch naming, commit style, + and the pull request process. + +> **Note:** the contribution guidelines currently live in the "Contributing" section of the +> root README rather than in a top-level `CONTRIBUTING.md`. If a dedicated `CONTRIBUTING.md` +> is added later, this step and the link above should point at it instead. + +--- + +## By role + +### New contributor + +You want orientation and the shortest path to a working local environment. + +| Document | What it gives you | +| --- | --- | +| [Root README](../README.md) | Project pitch, tech stack, prerequisites, install, run, and test commands. | +| [System architecture overview](architecture-overview.md) | The single diagram of all four apps and every external service, with two traced end-to-end paths. | +| [Runbook](runbook.md) | Day-two operations: what to do when a service is unhealthy, and how to restart pieces safely. | +| [Observability](observability.md) | Which metrics, logs, and traces exist and where they are emitted, so you can see what your change did. | + +### Backend developer + +The Node.js gateway in `apps/backend`: REST, WebSockets, Postgres, Redis, and the chain +listener. + +**Architecture and concepts** + +| Document | What it gives you | +| --- | --- | +| [Gateway architecture](../apps/backend/docs/concepts-gateway-architecture.md) | Socket.IO connection lifecycle, room semantics, and how the gateway scales horizontally over Redis pub/sub. | +| [Delivery fan-out and receipts](../apps/backend/docs/concepts-delivery-fanout.md) | How one sent message reaches every recipient device, how receipts flow back, and which services are not actually wired into the live path. | +| [Storage and push jobs](../apps/backend/docs/concepts-storage-push-jobs.md) | Object storage layout and the background jobs that expire files, devices, and envelopes. | + +**API reference** + +| Document | What it gives you | +| --- | --- | +| [Auth API](../apps/backend/docs/api-auth.md) | Wallet-signature login, JWT issuance, and session refresh endpoints. | +| [Users API](../apps/backend/docs/api-users.md) | User profile read and update routes. | +| [Devices and prekeys API](../apps/backend/docs/api-devices.md) | Every `/devices` and `/user-devices` route: ownership checks, prekey upload contract, and revocation side effects. | +| [Conversations API](../apps/backend/docs/api-conversations.md) | Creating conversations, managing membership, and reading history. | +| [Messages and sync API](../apps/backend/docs/api-messages-sync.md) | Message history pagination and the cross-device sync cursor. | +| [Files and uploads API](../apps/backend/docs/api-files-uploads.md) | Encrypted attachment upload, download, and lifecycle. | +| [Push API](../apps/backend/docs/api-push.md) | Push subscription registration and notification dispatch. | +| [Treasury API](../apps/backend/docs/api-treasury.md) | REST routes for treasury proposals and votes, plus how they relate to the on-chain contracts. | +| [WebSocket events](../apps/backend/docs/api-websocket-events.md) | Every Socket.IO event the gateway emits and accepts, with direction and payload. | + +**Contracts and schemas** + +| Document | What it gives you | +| --- | --- | +| [JWT auth contract](../apps/backend/docs/contracts-jwt-auth.md) | Token claim shape, signing algorithm, and expiry rules. | +| [REST schemas](../apps/backend/docs/contracts-rest-schemas.md) | Request and response body schemas shared across the REST surface. | +| [WebSocket payloads](../apps/backend/docs/contracts-websocket-payloads.md) | Payload shapes for each WebSocket event, as validated on the wire. | + +**Encryption and migrations** + +| Document | What it gives you | +| --- | --- | +| [E2EE onboarding](../apps/backend/docs/e2ee-onboarding.md) | Device registration and prekey upload flow for first-contact DM setup. | +| [MLS key packages](../apps/backend/docs/mls-key-packages.md) | Key package publication, consumption, and replenishment. | +| [MLS group membership](../apps/backend/docs/mls-group-membership.md) | Adding and removing members from an MLS group and the resulting epoch changes. | +| [MLS group files](../apps/backend/docs/mls-group-files.md) | How file keys are distributed to an MLS group. | +| [Message encryption migration](../apps/backend/docs/message-encryption-migration.md) | Migrating stored messages onto the current encryption scheme. | +| [Signal migration](../apps/backend/docs/signal-migration.md) | Moving the double-ratchet implementation onto the Signal protocol. | +| [Backend security hardening](../apps/backend/docs/security-hardening.md) | Backend-specific hardening measures and the threats each one closes. | + +### Frontend developer + +The Next.js client in `apps/web`. It holds the private keys, so most of its documentation +is about encryption and local state. + +**Concepts** + +| Document | What it gives you | +| --- | --- | +| [Web app README](../apps/web/README.md) | Running the Next.js client on its own, its scripts, and its environment variables. | +| [E2EE architecture](../apps/web/docs/concepts-e2ee-architecture.md) | Where keys live in the browser, how sessions are established, and what never leaves the device. | +| [Message pipeline](../apps/web/docs/concepts-message-pipeline.md) | The client-side path from typed text to an encrypted envelope on the wire, and back. | +| [Auth and device lifecycle](../apps/web/docs/concepts-auth-device-lifecycle.md) | Wallet connection, device registration, session persistence, and revocation handling. | +| [File encryption](../apps/web/docs/concepts-file-encryption.md) | How attachments are encrypted client-side before upload. | +| [Local search](../apps/web/docs/concepts-local-search.md) | The on-device search index over decrypted message content. | +| [Push subscription](../apps/web/docs/concepts-push-subscription.md) | Service worker registration and push permission handling. | +| [Wallet and treasury UI](../apps/web/docs/concepts-wallet-treasury-ui.md) | How the wallet and treasury screens are composed and what they read from chain versus the backend. | + +**Client APIs and types** + +| Document | What it gives you | +| --- | --- | +| [REST client](../apps/web/docs/api-rest-client.md) | The typed wrapper around the backend REST surface. | +| [WebSocket client](../apps/web/docs/api-websocket-client.md) | Socket lifecycle, reconnection, and event subscription on the client. | +| [Soroban client](../apps/web/docs/api-soroban-client.md) | How the web app builds, signs, and submits Soroban contract invocations. | +| [Auth session contract](../apps/web/docs/contracts-auth-session.md) | The shape of the persisted session and what invalidates it. | +| [IndexedDB schemas](../apps/web/docs/contracts-indexeddb-schemas.md) | Every IndexedDB object store, its keys, and its migration history. | +| [Response types](../apps/web/docs/contracts-response-types.md) | Shared TypeScript response types used across the client. | +| [MLS integration notes](../apps/web/src/lib/mls-integration.md) | Implementation notes co-located with the MLS integration code. | +| [Search module README](../apps/web/src/lib/search/README.md) | Implementation notes for the local search module. | + +### Contract developer + +The Soroban workspace in `contracts/`: `token_transfer`, `group_treasury`, and `proposals`. + +| Document | What it gives you | +| --- | --- | +| [Contracts README](../contracts/README.md) | Workspace layout, toolchain, and how to build and test the contracts. | +| [Deployment and invocation](../contracts/docs/api-deployment-invocation.md) | Deploying each contract, initialising it, and invoking it from the CLI, including required environment variables. | +| [Contract events reference](../contracts/docs/contracts-events.md) | Every published event across all three contracts, its topic and data shape, the state change it signals, and whether the backend listener consumes it. | +| [Proposals API](../contracts/docs/api-proposals.md) | The `proposals` contract surface: creating, voting, finalising, and executing. | +| [Token transfer API](../contracts/docs/api-token-transfer.md) | The `token_transfer` contract surface, including the memo field used to correlate a transfer with a chat message. | +| [Proposal lifecycle](../contracts/docs/concepts-proposal-lifecycle.md) | Every proposal status, the transitions between them, and what triggers each one. | +| [Token transfer flow](../contracts/docs/concepts-token-transfer-flow.md) | The end-to-end flow of an in-chat payment through the contract. | +| [Token transfer storage](../contracts/docs/contracts-token-transfer-storage.md) | Storage keys and value types used by `token_transfer`. | +| [WASM size and resource budget](../contracts/docs/concepts-resource-budget.md) | The 100 KB per-contract CI gate, current sizes and headroom, and the levers available when a contract approaches the limit. | + +### Operator + +Running and monitoring a deployment. + +| Document | What it gives you | +| --- | --- | +| [Runbook](runbook.md) | Operational procedures: health checks, restarts, and incident response steps. | +| [Observability](observability.md) | Metrics, logs, and traces exposed by the services, and how to reach them. | +| [Deployment and invocation](../contracts/docs/api-deployment-invocation.md) | Contract deployment steps and the environment variables the backend needs to watch the chain. | +| [Storage and push jobs](../apps/backend/docs/concepts-storage-push-jobs.md) | The background jobs that run on a schedule and the storage they clean up. | + +### Security reviewer + +Threat model, hardening, and the crypto protocol documents. + +| Document | What it gives you | +| --- | --- | +| [Threat model](threat-model.md) | Assets, adversaries, trust boundaries, and the mitigations claimed for each threat. | +| [Security fixes summary](../SECURITY_FIXES_SUMMARY.md) | A log of security issues found and the fixes applied for each. | +| [Audit logging](security/audit-logging.md) | What is audit-logged, in what format, and what is deliberately excluded. | +| [Rate limits](security/rate-limits.md) | Every rate limit in the system, its scope, and its threshold. | +| [TLS and pinning](security/tls-and-pinning.md) | Transport security requirements and certificate pinning behaviour. | +| [Backend security hardening](../apps/backend/docs/security-hardening.md) | Backend hardening measures and the threats each one closes. | +| [Signal integration](signal-integration.md) | How the Signal protocol is integrated and which guarantees it provides. | +| [Group epoch sync](group-epoch-sync.md) | How MLS group epochs stay synchronised across devices and what happens when they diverge. | +| [E2EE architecture](../apps/web/docs/concepts-e2ee-architecture.md) | The client-side key model β€” the basis for any claim that the server cannot read messages. | + +### AI / data developer + +The FastAPI service in `apps/ai_agent`. + +| Document | What it gives you | +| --- | --- | +| [AI agent README](../apps/ai_agent/README.md) | Running the service locally with `uv`, and its environment variables. | +| [Chat API](../apps/ai_agent/docs/api-chat.md) | The assistant chat endpoint: request, response, and system prompt behaviour. | +| [Index and search API](../apps/ai_agent/docs/api-index-search.md) | Indexing documents into the vector store and querying them. | +| [Proposals summarise API](../apps/ai_agent/docs/api-proposals-summarise.md) | Summarising a governance proposal into a short digest. | +| [Transfers analyse API](../apps/ai_agent/docs/api-transfers-analyse.md) | Risk-scoring a transfer and the flagging threshold. | +| [RAG search architecture](../apps/ai_agent/docs/concepts-rag-search-architecture.md) | Retrieval-augmented search design: chunking, embedding, and retrieval. | +| [Transfer risk analysis](../apps/ai_agent/docs/concepts-transfer-risk-analysis.md) | The heuristics behind transfer risk scoring. | +| [Pydantic models](../apps/ai_agent/docs/contracts-pydantic-models.md) | Request and response model definitions for the service. | +| [Weaviate schema](../apps/ai_agent/docs/contracts-weaviate-schema.md) | The vector store collection schema and its properties. | + +--- + +## Repository meta + +Documents about the repository itself rather than about the product. + +| Document | What it gives you | +| --- | --- | +| [Pull request template](../.github/pull_request_template.md) | The checklist every pull request is opened against. | +| [PR notes](../pr.md) | Scratch notes for an in-flight pull request; not a reference document. | + +--- + +## Keeping this index honest + +This file is the only entry point into the documentation, which means a document missing +from it is a document nobody will find. + +- **Adding a document:** add a row to the section matching the *reader* who needs it, not + the directory it lives in. If two roles need it, list it under both β€” duplication across + role sections is intentional, since each section is meant to be read on its own. +- **Moving or deleting a document:** update or remove its row in the same commit. A broken + link here is worse than no link. +- **Adding a new area:** if a new app or subsystem arrives with its own `docs/` directory, + give it its own role section rather than appending to an existing one. + +The scope of this index is every `.md` file in the repository except generated output and +dependency directories (`node_modules/`, `target/`, `.venv/`, build artefacts). diff --git a/docs/architecture-overview.md b/docs/architecture-overview.md new file mode 100644 index 0000000..5357dd3 --- /dev/null +++ b/docs/architecture-overview.md @@ -0,0 +1,291 @@ +# System Architecture Overview + +**Scope: orientation only.** This document exists so that someone who has never seen the +codebase can understand how the pieces fit together in about ten minutes. It deliberately +stays shallow β€” every component section ends with a link to the document that actually +specifies it. When this document and a per-app document disagree, the per-app document is +right and this one needs fixing. + +For the exhaustive design specification, see the per-app documents linked throughout. For +a list of every document in the repository, see the [documentation index](README.md). + +--- + +## The whole system in one diagram + +```mermaid +flowchart TB + subgraph client["Web client β€” apps/web (Next.js, browser)"] + UI["React UI"] + CRYPTO["E2EE key store
(IndexedDB, non-extractable keys)"] + SOR["Soroban client
(@stellar/stellar-sdk)"] + end + + subgraph gateway["Backend gateway β€” apps/backend (Node.js)"] + REST["REST API (Express)"] + WS["WebSocket server (Socket.IO)"] + LISTEN["Stellar listener
services/stellarListener.ts"] + JOBS["Background jobs
(file / device / envelope GC)"] + end + + subgraph ai["AI agent β€” apps/ai_agent (FastAPI, Python)"] + AGENT["/chat, /transfers/analyse,
/proposals/summarise, /index, /search"] + end + + subgraph data["Stateful infrastructure"] + PG[("PostgreSQL
ciphertext, metadata, proposals")] + REDIS[("Redis
pub/sub, presence, rate limits")] + OBJ[("Object storage
MinIO / S3 / R2")] + VEC[("Weaviate
vector store")] + end + + subgraph chain["Stellar network"] + RPC["Soroban RPC"] + TT["token_transfer"] + GT["group_treasury"] + PROP["proposals"] + end + + LLM["LLM API
(external)"] + + UI --> CRYPTO + UI -->|"HTTPS/JSON"| REST + UI <-->|"WebSocket (Socket.IO)"| WS + UI --> SOR + SOR -->|"JSON-RPC, signed tx"| RPC + + REST --> PG + REST --> OBJ + WS --> PG + WS <-->|"pub/sub + presence"| REDIS + WS -->|"HTTP POST /chat"| AGENT + JOBS --> PG + JOBS --> OBJ + + LISTEN -->|"getEvents polling"| RPC + LISTEN --> PG + LISTEN -->|"treasury_proposal_updated"| WS + + AGENT --> VEC + AGENT --> LLM + + RPC --- TT + RPC --- GT + RPC --- PROP + PROP -->|"cross-contract call"| GT + TT -.->|"SEP-41 transfer"| GT +``` + +### Protocols on each edge + +| From | To | Protocol | +| --- | --- | --- | +| Web client | Backend REST | HTTPS, JSON, `Authorization: Bearer ` | +| Web client | Backend gateway | WebSocket (Socket.IO), JWT in the handshake `auth.token` | +| Web client | Soroban RPC | JSON-RPC over HTTPS, transactions signed in the browser wallet | +| Backend | PostgreSQL | TCP, via Drizzle ORM | +| Backend | Redis | RESP, pub/sub for cross-instance fan-out and presence | +| Backend | Object storage | S3 API (path-style against MinIO, virtual-host against S3/R2) | +| Backend | AI agent | HTTP POST, plaintext JSON | +| Backend listener | Soroban RPC | JSON-RPC `getEvents`, cursor-based polling every 5 s | +| AI agent | Weaviate | Weaviate Python client over HTTP | +| AI agent | LLM API | HTTPS to an external provider | +| `proposals` | `group_treasury` | Soroban cross-contract invocation | + +--- + +## Components + +### Web client β€” `apps/web` + +**Responsibility.** The Next.js browser application. It is the only component that holds +user private keys: it derives them, stores them in IndexedDB, encrypts every message and +attachment before it goes over the wire, and decrypts everything that comes back. It also +builds and signs Soroban transactions directly against the user's wallet, so payments do +not pass through the backend. + +**It must never** send a private key, a plaintext message body, or an unencrypted +attachment to the backend, and it must never trust the backend to tell it who a message +was from β€” sender identity is verified against the cryptographic session, not the +server-supplied metadata. + +See: [E2EE architecture](../apps/web/docs/concepts-e2ee-architecture.md), +[message pipeline](../apps/web/docs/concepts-message-pipeline.md), +[Soroban client](../apps/web/docs/api-soroban-client.md). + +### Backend gateway β€” `apps/backend` + +**Responsibility.** A Node.js process running Express for REST and Socket.IO for realtime. +It authenticates devices, stores and routes ciphertext, fans messages out to every +recipient device, tracks presence and delivery receipts, brokers uploads to object +storage, and runs the chain listener and the scheduled cleanup jobs. Multiple instances +run behind the Redis adapter, so any device may be connected to any instance. + +**It must never** be able to read message content β€” it stores and forwards opaque +ciphertext envelopes and must not acquire a decryption path. It must never accept a +client's claim about its own identity without verifying the JWT and confirming the device +row is present and unrevoked, and it must never hold custody of user funds; it observes +the chain, it does not sign for it. + +See: [gateway architecture](../apps/backend/docs/concepts-gateway-architecture.md), +[delivery fan-out](../apps/backend/docs/concepts-delivery-fanout.md), +[WebSocket events](../apps/backend/docs/api-websocket-events.md). + +### AI agent β€” `apps/ai_agent` + +**Responsibility.** A FastAPI service exposing chat assistance, transfer risk analysis, +proposal summarisation, and vector indexing/search. It calls an external LLM provider and +stores embeddings in Weaviate. + +**It must never** receive end-to-end encrypted message content that the user has not +explicitly submitted to it. Anything it is given leaves the E2EE trust boundary β€” it goes +to an external LLM provider in plaintext β€” so the boundary must stay explicit and +user-initiated. It is also not an authorisation component: nothing it returns should gate +a transfer or a withdrawal, since its output is advisory. + +See: [AI agent README](../apps/ai_agent/README.md), +[RAG search architecture](../apps/ai_agent/docs/concepts-rag-search-architecture.md), +[transfer risk analysis](../apps/ai_agent/docs/concepts-transfer-risk-analysis.md). + +### Soroban contracts β€” `contracts/` + +Three contracts make up the on-chain surface. + +- **`token_transfer`** β€” routes a SEP-41 token transfer between two addresses and emits a + `transfer` event carrying an opaque `memo`. The memo is how an on-chain payment is + correlated back to a chat message. +- **`group_treasury`** β€” holds pooled group funds. Tracks members and per-token balances, + and runs a threshold-approval withdraw-proposal flow. +- **`proposals`** β€” DAO-style governance: create a proposal, vote yes/no until expiry, + finalise on vote count, then execute β€” which, for a withdrawal, calls into + `group_treasury`. + +**They must never** trust a caller's claimed identity without `require_auth`, and must +never emit an event before the corresponding state write is committed β€” the backend +listener treats an event as proof that the state change happened. + +See: [contracts README](../contracts/README.md), +[contract events reference](../contracts/docs/contracts-events.md), +[proposal lifecycle](../contracts/docs/concepts-proposal-lifecycle.md), +[WASM size and resource budget](../contracts/docs/concepts-resource-budget.md). + +### PostgreSQL + +**Responsibility.** The system of record for everything off-chain: users, devices and +their public prekeys, conversations and membership, message ciphertext and per-device +envelopes, delivery receipts, file metadata, and the off-chain mirror of treasury +proposals and votes. + +**It must never** hold plaintext message bodies or user private keys. Ciphertext at rest +is the design; a schema change that lands readable content in a column is a break in the +threat model, not an optimisation. + +### Redis + +**Responsibility.** Cross-instance coordination β€” the Socket.IO adapter's pub/sub backbone, +presence state, per-device delivery channels, device-revocation notifications, and rate +limit counters. + +**It must never** be treated as durable. Everything in Redis is reconstructible from +PostgreSQL or from a reconnecting client; a flushed Redis must degrade the system, not +lose data. + +### Object storage (MinIO / S3 / R2) + +**Responsibility.** Stores encrypted file attachments. The backend brokers access; the +same S3 client path serves local MinIO and production S3 or R2, differing only by +environment variables. + +**It must never** receive an unencrypted attachment β€” files are encrypted in the browser +before upload β€” and must never be exposed as a public bucket, since object keys would then +be the only thing standing between an attacker and every stored file. + +### Stellar network / Soroban RPC + +**Responsibility.** Consensus, settlement, and the event stream. The backend reaches it +only through Soroban RPC's `getEvents`, polled on a cursor. + +**It must never** be assumed synchronously consistent with the backend's database. Events +arrive seconds later, may be re-delivered on reconnect, and the ledger is the authority +whenever the two disagree. + +See: [deployment and invocation](../contracts/docs/api-deployment-invocation.md). + +--- + +## End-to-end path 1: sending an encrypted message + +The point of this trace is that the backend never holds anything readable. + +1. **Compose.** The user types into the React UI. The client loads the cryptographic + session for the conversation from IndexedDB. +2. **Encrypt, per device.** The client encrypts the plaintext once per recipient *device* β€” + not per user. A conversation of three users with two devices each produces one envelope + per active device. Private keys never leave the browser. +3. **Emit.** The client emits `send_message` over its authenticated Socket.IO connection, + carrying the message id, content type, and the map of per-device ciphertext envelopes. +4. **Validate and persist.** The gateway validates the payload shape and size (16 KB cap), + applies the per-socket rate limit, and writes the message row plus its envelopes to + PostgreSQL. It stores opaque bytes; it cannot decrypt them. +5. **Fan out.** `services/deliveryPipeline.ts` loads the conversation's active, + non-revoked devices and emits a `message_envelope` event to each `device:${deviceId}` + room containing only that device's envelope, plus a ciphertext-free `new_message` to the + conversation room for unread counts. Recipients connected to a different gateway + instance are reached through the Redis adapter. +6. **Decrypt.** Each recipient client receives its envelope, decrypts it with its own key + material, advances the ratchet, and renders the message. +7. **Receipt.** The recipient acknowledges; `services/deliveryAggregation.ts` marks that + device delivered and, once every active device of a recipient user has acknowledged, + notifies the sender with `message_fully_delivered`. + +> The full trace, including which services are implemented but *not* wired into the live +> send path, is in [delivery fan-out](../apps/backend/docs/concepts-delivery-fanout.md). +> Read that before changing anything in this path. + +## End-to-end path 2: executing a treasury withdrawal + +The point of this trace is that authority lives on-chain and the backend only observes. + +1. **Propose.** A treasury member creates a withdrawal proposal. The proposal is recorded + on-chain β€” `group_treasury::propose_withdraw` for the treasury's own threshold flow, or + `proposals::create_proposal` for the governance flow β€” and the client signs the + transaction with the user's wallet. The backend's REST treasury routes maintain an + off-chain mirror in PostgreSQL for querying and UI, but that mirror is not the source of + truth. +2. **Observe.** The contract emits `proposal_created`. Within about five seconds the + Stellar listener's `getEvents` poll picks it up, upserts on + `(contractId, proposalId)`, and emits `treasury_proposal_updated` into the linked + conversation room. The upsert is what makes re-reading a page after a reconnect safe. +3. **Vote.** Each member signs an approve or reject transaction. `group_treasury` records + one vote per member per proposal, emits `withdraw_vote` for every vote, and separately + emits `proposal_approved` once approvals reach the configured threshold β€” or + `proposal_rejected` once rejections reach the blocking minority, the point at which + enough approvals can no longer be gathered. +4. **Execute.** Once the proposal has passed, execution moves the funds. In the governance + flow, `proposals::execute_withdraw` checks the caller is a treasury member, checks the + balance, calls `group_treasury::withdraw` cross-contract, and flips the status to + `Executed`. +5. **Settle.** `group_treasury` emits `withdraw` as the tokens move, and the ledger closes. + The listener picks up the events and updates the mirror; connected clients see the + status change pushed over WebSocket. + +> **Known gap.** The backend listener subscribes to a `proposal_executed` topic, but no +> contract publishes that topic β€” `proposals` publishes the truncated symbols `executed` +> and `execut` instead. An executed proposal therefore does not currently transition to +> `executed` in the off-chain mirror through the listener. This is documented per-event in +> the [contract events reference](../contracts/docs/contracts-events.md). + +--- + +## Where to go next + +| You want to… | Read | +| --- | --- | +| Change how messages are delivered | [delivery fan-out](../apps/backend/docs/concepts-delivery-fanout.md) | +| Change the WebSocket surface | [WebSocket events](../apps/backend/docs/api-websocket-events.md), [payloads](../apps/backend/docs/contracts-websocket-payloads.md) | +| Change client-side crypto | [E2EE architecture](../apps/web/docs/concepts-e2ee-architecture.md) | +| Change or deploy a contract | [deployment and invocation](../contracts/docs/api-deployment-invocation.md) | +| Consume a new on-chain event | [contract events reference](../contracts/docs/contracts-events.md) | +| Understand the security posture | [threat model](threat-model.md) | +| Operate a deployment | [runbook](runbook.md), [observability](observability.md) | +| Find any other document | [documentation index](README.md) |