From 3c88ae812bcbfb89bb6219176c7f2826645f6e6e Mon Sep 17 00:00:00 2001 From: bahdotsh Date: Wed, 30 Sep 2026 21:08:35 +0530 Subject: [PATCH] docs(spec): the local API chapter One server process fronting one engine for several local applications: JSON-RPC 2.0 over a WebSocket, a Unix domain socket by default and TCP on loopback with a per-launch token as the opt-in, a `hello` that declares the client's application id once and stamps it on every send, and the engine's events relayed unchanged as notifications. The chapter partitions the interface definition into the 126 methods a client may call and the 92 platform operations it never can, including the run loop and the drain, which the server owns because the engine delivers and acknowledges a message only from inside the drain. It is the first complete catalogue of the engine's 75 event tags with their fields, vocabularies and routing (by application id, by an identifier the server issued, or to everyone), and it keeps the 25-variant error taxonomy: the JSON-RPC code is the variant's position in the append-only enum. There is no HTTP request path. The pinned WebSocket library's handshake parser accepts only GET and drops any other method without a response, which replaces the program record's H5 wording and is recorded in the chapter so it is not re-added. An optional unauthenticated GET /health through the request hook is allowed. docs/bridges/local-api.md states which shared bridge rules a server inherits (C1, C2, C3, C6, C10, C12) and the new rule that a Rust guard pins the three tables to the definition, the engine and the reference server. Index rows added; docs/README.md corrected to thirteen shared rules and twenty-five ADRs, and given the peer-stream framing row it lacked. --- CHANGELOG.md | 21 + docs/README.md | 7 +- docs/bridges/README.md | 1 + docs/bridges/local-api.md | 121 +++++ docs/spec/README.md | 1 + docs/spec/local-api.md | 945 ++++++++++++++++++++++++++++++++++++++ 6 files changed, 1094 insertions(+), 2 deletions(-) create mode 100644 docs/bridges/local-api.md create mode 100644 docs/spec/local-api.md diff --git a/CHANGELOG.md b/CHANGELOG.md index 551396f87..42f60e319 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -15,6 +15,27 @@ archived by series under [docs/changelog/](docs/changelog/); see the ### Added +- **The local API chapter.** `docs/spec/local-api.md` specifies how one + server process fronts one engine for several local applications: JSON-RPC + 2.0 over a WebSocket on a Unix domain socket by default (TCP on loopback + with a per-launch token as the opt-in), a `hello` that declares the + client's application id once and stamps it on every send, and the events + relayed unchanged as notifications. It is the first complete catalogue of + the engine's events: all seventy-five tags with their fields and the + vocabularies of the enum-valued ones, and how each is routed (by + application id, by an identifier the server issued, or to everyone). The + method table partitions the interface definition into the 126 methods a + client may call and the 92 platform operations it never can, including the + run loop and the drain, which the server owns because the engine delivers + a message only when something drains. Errors keep the engine's + twenty-five-variant taxonomy: the JSON-RPC code is the variant's position + in the append-only enum. There is no HTTP request path: the pinned + WebSocket library drops any non-`GET` handshake without a response, which + the chapter records so it is not re-added. `docs/bridges/local-api.md` + states which shared bridge rules the server inherits and the new rule that + a Rust guard pins the three tables to the definition, the engine and the + reference server. Nothing ships in this entry but the contract; the + reference server follows it. - **The leaf node builds for ESP32 RISC-V parts and for Cortex-M0.** `offline-protocol-leaf` now compiles, and CI lints it, for `riscv32imac-unknown-none-elf` (ESP32-C6, ESP32-H2), diff --git a/docs/README.md b/docs/README.md index 174db4d38..a1c40a96f 100644 --- a/docs/README.md +++ b/docs/README.md @@ -60,8 +60,10 @@ implementation written against these documents should interoperate. | [Capability negotiation](spec/capability-negotiation.md) | What peers advertise, what it gates, what absence means | | [Leaf node provisioning](spec/leaf-provisioning.md) | What a constrained device owes at pairing, the never-committing profile | | [Bluetooth LE framing](spec/ble-framing.md) | The GATT contract, the fragment header, and what a receiver owes on reassembly | +| [Peer-stream framing](spec/stream-framing.md) | The preamble that proves a stream's peer, the length-prefixed message frame, and the LAN discovery hint | | [Username discovery and invites](spec/username-discovery.md) | The self-certifying invite payload, the username directory, and the signing-domain registry | | [The gateway contract](spec/gateway-contract.md) | What a gateway is, the five verbs, the daemon wire protocol, and the backbone | +| [The local API](spec/local-api.md) | One server, several local applications: JSON-RPC over a WebSocket, the method and event tables, routing, replay, errors | | [Conformance](spec/conformance.md) | The two profiles, what every implementation owes, and the vectors that decide it | ## Security @@ -87,7 +89,7 @@ implementation written against these documents should interoperate. | Document | Scope | |----------|-------| -| [ADR index](adr/README.md) | Twenty-four decisions that are expensive to reverse or easy to undo by accident | +| [ADR index](adr/README.md) | Twenty-five decisions that are expensive to reverse or easy to undo by accident | If something in the codebase looks redundant or over-engineered, check here before simplifying it. @@ -99,11 +101,12 @@ in here fails **silently** when violated. | Document | Scope | |----------|-------| -| [Shared contract](bridges/README.md) | The ten rules every binding shares | +| [Shared contract](bridges/README.md) | The thirteen rules every binding shares | | [Swift](bridges/swift.md) | iOS native and the React Native iOS bridge | | [Kotlin](bridges/kotlin.md) | Android native and the React Native Android bridge | | [Python](bridges/python.md) | Desktop and tooling | | [TypeScript](bridges/typescript.md) | The React Native JavaScript surface | +| [Local API](bridges/local-api.md) | A server fronting one engine for several local clients over a socket | ## Release history diff --git a/docs/bridges/README.md b/docs/bridges/README.md index da19a73c5..e7f5a5c00 100644 --- a/docs/bridges/README.md +++ b/docs/bridges/README.md @@ -11,6 +11,7 @@ binding. | [Kotlin](kotlin.md) | Android native, the Android library and the React Native Android bridge | | [Python](python.md) | Desktop and tooling | | [TypeScript](typescript.md) | The React Native JavaScript surface | +| [Local API](local-api.md) | A server fronting one engine for several local clients over a socket, and the reference server | Integration **guides** live elsewhere: [iOS](../ios-integration.md), [Android](../android-integration.md), diff --git a/docs/bridges/local-api.md b/docs/bridges/local-api.md new file mode 100644 index 000000000..3f8baad01 --- /dev/null +++ b/docs/bridges/local-api.md @@ -0,0 +1,121 @@ +# Local API contract + +Covers a server that fronts one engine for several local clients over the +[local API](../spec/local-api.md), and the reference server in the Python +package. + +Read [the shared contract](README.md) first. A server is a binding one level +up: it is written against a generated binding (the reference one against +Python), so everything that binding owes, the server owes through it, and +this document covers what is specific to sitting between the binding and a +socket. + +## Which shared rules apply verbatim + +| Rule | Why it reaches the server | +|---|---| +| [C1](README.md#c1-regenerate-every-binding-together) | The server calls a generated binding. A partial regeneration fails at the server's first call, at runtime, exactly as it does in an application | +| [C2](README.md#c2-the-error-enum-is-append-only) | The JSON-RPC error code is the variant's position in the enum. Append-only is what makes that number stable; an inserted variant would renumber every client's error handling as well as breaking the bindings | +| [C3](README.md#c3-events-cross-as-opaque-json) | An event notification's `params` is the engine's JSON, unchanged. The server never parses an event for anything but routing (`type`, `app_id`, the correlation identifiers), and never re-encodes one | +| [C6](README.md#c6-config-parsers-must-not-default-to-literals) | A dictionary parameter arrives partial. The server passes the fields the client sent and lets the definition's defaults fill the rest; a literal fallback in the server would reset every field a client did not mention | +| [C10](README.md#c10-lifecycle-rules-for-event-emission) | The per-application-id hold of stamped inbound events is C10's held inbound buffer, one level up: the same 256-entry capacity and the same reason (the engine has already acknowledged the message), over two of C10's three tags (`message_received`, `file_received`) plus `media_resend_required` because it is stamped; `message_decryption_failed` is excluded until it carries an `app_id` | +| [C12](README.md#c12-telemetry-is-callback-free-and-the-binding-fills-the-platform) | The server is the binding that fills the platform fields and owns the session boundary. Telemetry is therefore a server-owned operation, and no client can enable, disable or flush it | + +## L1. The method and event tables are pinned by a Rust guard + +The chapter carries three tables that name things the compiler never sees +together: the methods a client may call, the platform operations it may not, +and the events it may receive. Each rots silently. A method added to the +interface definition and to neither table is unreachable from every client +and unmentioned in the contract; an event added to the engine and not to the +catalogue is one no client is written for; a method the reference server +exposes that the chapter lists as platform-only is a hole in the boundary +the chapter's fifth invariant draws. + +A Rust guard in the FFI crate reads the chapter, the interface definition, +the engine's event enum and the reference server's dispatch table, and +asserts that the exposed set plus the platform set is exactly the set of +declarations, that the dispatch table exposes exactly the exposed set, and +that the catalogue's tags are exactly the enum's variants. It follows the +skip-if-tree-absent idiom the other document-reading guards use, so a +published crate's tests still build without the repository around them. + +The chapter states the shape the guard parses (a row's first cell is the +backticked name; the three tables sit under three named headings). Changing +one of those headings, or moving a row out from under them, is a change to +the guard and fails it, which is the point: the document and the code can +only move together. + +What this prevents: a new engine method quietly reachable by every local +application because the server's dispatcher was generated from the +definition and nobody classified it. The guard makes "unclassified" a +failing test rather than a default. + +## L2. The server owns the run loop, the drain and the lifecycle + +No client calls `process()`, `receive_message()`, `start()`, `stop()`, +`pause()` or `resume()`, and none of them is a method on the wire. The +server runs the loop the mobile bridges run (100 ms), drains on every tick, +and starts and stops the engine on the operator's instruction. + +This is the first invariant of the chapter and it is load-bearing in a way +that is easy to miss: the engine emits `message_received` from inside the +drain, and acknowledges and dedup-marks the message there. A client that +could drain would take messages out from under the server's routing; two +drainers would split one stream; a server that did not drain would leave +every inbound message unacknowledged while the sender retries into silence. +The Python manager's own documentation already says that a second caller of +`receive_message()` always sees `None`; a server makes that impossibility +structural instead of documented. + +**The drain's return value is never relayed.** `receive_message()` returns +the core `Message` as JSON, a different shape from the `message_received` +event the engine emits from inside the same call (`id` rather than +`message_id`, a capitalised `priority`, `forwarded_from` rather than +`forward_info`, no `transport`). The FFI crate discards that return value on +purpose, and the Python manager synthesises a second `message_received` from +it and hands it to the same handler as the engine's event. A server built on +that manager drops the synthesised event and relays only the engine's. The +failure otherwise is two `message_received` notifications per message, one +of them in a shape the catalogue does not describe, and a client that +deduplicates by `message_id` cannot even pair them, because the synthesised +one has no such field. + +## L3. Routing and the hold are the server's, and never reach the wire + +The three routing classes (stamped, correlated, broadcast), the per-application +hold, service ownership, space scoping, method denial and the refusal of an +unlisted id at `hello` once any allow-list or deny is configured are all +rules the server applies between the engine and the socket. Nothing about +them is sent to a peer, a relay or a gateway, and nothing about them changes +an event or a frame. + +The reason to say so here is the temptation to make one of them real: an +ownership claim in the service descriptor, a per-application member in a +space. Both would be a second membership system beside MLS, which +[document replication](../spec/data-sync.md) forbids for the reason it +states there, and both would mean a peer could see which applications sit +behind an identity. The server knows; the mesh never does. + +## L4. Session refusals reuse the engine's taxonomy + +A method before `hello` is `InvalidState`; a bad application id is +`InvalidArgument`; a wrong token, a service another application owns, a +space outside the client's allow-list, a denied method group, or an unlisted +id at `hello` once any allow-list or deny is configured, is +`PermissionDenied`. No error variant exists only on the wire. + +A server that invented its own codes for these would hand every client a +second taxonomy to handle, and would break the property that a client +written against the JSON-RPC surface handles the same variants an embedded +application does. + +## What the reference server owes + +| | | +|---|---| +| Carrier | A Unix domain socket by default, created `0600` in a `0700` directory; TCP on loopback only when enabled, with the per-launch token in a `0600` file | +| Frame limit | Raised from the WebSocket library's 1 MiB default to cover the engine's file size limit plus base64, so a media send is refused by the engine and not by a closed connection | +| Dispatch table | Checked in, classified against the two method tables, and read by the guard in L1 | +| Hold | Per application id, 256 entries, oldest dropped, each drop logged | +| Health | `GET /health` through the library's request hook, with the body the chapter shows; nothing else on HTTP | diff --git a/docs/spec/README.md b/docs/spec/README.md index b77d0f75c..91eb1b928 100644 --- a/docs/spec/README.md +++ b/docs/spec/README.md @@ -25,6 +25,7 @@ document says which reading is normative for the wire. | [Peer-stream framing](stream-framing.md) | The preamble that proves a stream's peer, the length-prefixed message frame, and the LAN discovery hint | | [Username discovery and invites](username-discovery.md) | The self-certifying invite payload, and the non-authoritative username directory | | [The gateway contract](gateway-contract.md) | What a gateway is, the five verbs it implements, the gateway-daemon wire protocol, and the backbone | +| [The local API](local-api.md) | One server fronting one engine for several local applications: JSON-RPC over a WebSocket, the `hello` handshake, the method and event tables, routing, replay, and the errors | | [Conformance](conformance.md) | The two profiles, what every implementation owes, and how the vectors decide it | ## Conformance vectors diff --git a/docs/spec/local-api.md b/docs/spec/local-api.md new file mode 100644 index 000000000..de63aa164 --- /dev/null +++ b/docs/spec/local-api.md @@ -0,0 +1,945 @@ +# The local API + +## What this chapter is for + +Every binding in this repository puts the engine inside the application's own +process: a Swift or Kotlin bridge, a Python module, a React Native package. +One application, one process, one engine. A long-lived host that serves +several local applications at once, on a desktop, a server, or a headless +box, has no shape to run the engine in, because nothing outside the process +can reach it. + +This chapter specifies that shape: one **server** process owns one engine +instance, and any number of local **clients** reach it over a socket. The +wire between them is JSON-RPC 2.0 over a WebSocket. A client written against +this chapter, in any language, works against any conforming server, and a +server written against it serves any conforming client. The reference server +ships in the Python package; it is one implementation of this contract, not +the contract. + +Three things this chapter deliberately does not do. It does not invent a +second API: every method is one of the interface definition's own methods +under its own name, with its own parameters, returning its own result, and +every event is the engine's own event JSON, relayed unchanged. It does not +change anything on the mesh: nothing here reaches another device, and a peer +cannot tell whether it is talking to an application with an embedded engine +or to a server fronting six of them. And it does not add a second identity: +one server is one identity, and the applications behind it share it by +construction. What separating them requires is stated in the invariants, and +it is the server's job, never the wire's. + +## Invariants + +Six things hold for every server and every client. A server that keeps the +mechanisms below and breaks one of these is not a conforming server. + +- **The server owns the run loop and the drain.** The server calls + `process()` on its timer and drains `receive_message()` on every tick; no + client ever does either, and neither is a method on the wire. This is + load-bearing rather than tidy: the engine emits `message_received` from + inside the drain, so a message is delivered, acknowledged and dedup-marked + only when something drains. Two drainers would split one stream between + them, and no drainer would leave every inbound message unacknowledged + forever while the sender retries into a black hole. +- **A connection sends as the application it declared.** The first request + on a connection is `hello`, which names an application id, and the server + stamps that id on every message and media transfer sent over that + connection, through the per-send application id the send options carry, + so a frame on the mesh names the application it came from and the + receiving side can route it. A connection cannot send as a different + application from the one it declared, and a server that let it would + defeat every routing rule below. The id is self-declared and it is + metadata, never an access control + ([R17](../security/threat-model.md#r17-the-application-id-on-every-frame-is-cleartext-and-unsigned)): + what stops a local process from declaring another application's id is the + socket boundary and the unlisted-id rule below, not the wire. +- **An event reaches the clients it is for.** An event that carries an + application id reaches only the clients that declared that id. An event + that names an identifier the server handed to one client reaches that + client. Everything else is engine-level and reaches every client. A server + never delivers a stamped or correlated event to a client it is not for, + because the alternative is every application reading every other + application's messages through the side door of a shared socket. +- **The socket is the authorization boundary.** Whoever can open the socket + is one of the operator's applications. There is no user database, no + session token that outlives a launch, and no permission model on the + wire. A Unix domain socket with owner-only permissions is that boundary by + construction; a TCP listener on loopback is that boundary only with the + per-launch token, and is never bound to anything but loopback. Inside the + boundary sit root and every process running as the owner's uid, and the + server does not check peer credentials, so nothing below distinguishes + one of those processes from another. Application ids are self-declared, + and the server-side rules (which application owns a service, which spaces + one may open, which ids are listed) separate the applications from each + other's mistakes: a name collision, a space opened by accident, a client + started with the wrong id. They do not separate them from a hostile + local application, which is inside the boundary by definition. Every one + of those rules is applied before a call reaches the engine, and none + appears on the mesh ([data replication](data-sync.md) invariant: a space + is an MLS scope with no second membership system). +- **The platform operations are never on the wire.** The interface + definition has two kinds of method: what an application calls, and what + the platform layer that drives a radio, a relay socket or the run loop + calls. The second kind (every `ble_*`, `internet_*`, `wifi_direct_*`, + `reticulum_*` and `nostr_*` operation, every callback installer, the + lifecycle, the storage attach, the telemetry pipe and the host feeds) is + the server's own and is not reachable from a client under any name. A + client that could inject an inbound frame or announce a peer would hold the + same authority the threat model's boundary 2 exists to keep on the far + side of a dedicated entry point. +- **Nothing here is a second implementation of anything.** Method names, + parameter names, result shapes, enum spellings, event tags and error + variants are the interface definition's. The server maps a JSON request to + the generated binding's call and the binding's result to JSON; it does not + reimplement, re-validate, or re-encode a message. The failure this + prevents is the one every hand-written bridge has had: a field renamed in + one copy and not the other, invisible until a device disagrees. + +## Framing + +### JSON-RPC 2.0 over one WebSocket + +A connection is one WebSocket. Every message on it is one JSON-RPC 2.0 +object as a text frame: a request from the client, a response from the +server, or a notification from the server. A client never sends a +notification, and a server never sends a request. + +``` +--> {"jsonrpc":"2.0","id":1,"method":"hello","params":{"app_id":"notes"}} +<-- {"jsonrpc":"2.0","id":1,"result":{"api_version":1,"server":{"name":"offline-protocol-service","version":"0.28.0"},"state":"Running","local_address":"off1..."}} +--> {"jsonrpc":"2.0","id":2,"method":"send_message","params":{"recipient":"off1...","content":"hi","priority":"Medium","reply_to_msg":null}} +<-- {"jsonrpc":"2.0","id":2,"result":"5f0c..."} +<-- {"jsonrpc":"2.0","method":"event","params":{"type":"message_delivered","message_id":"5f0c...","latency_ms":412,"hop_count":1,"transport":"ble"}} +``` + +- **Requests carry `params` by name**, as a JSON object whose keys are the + method's parameter names. Positional `params` are refused with + `-32602`. A method with no parameters takes `{}` or no `params` member. +- **The request `id`** is a string or a number; the response carries it back + unchanged. Responses correlate by `id` and MAY arrive in a different order + from the requests, so a client that pipelines keeps its own map. A request + with no `id` is a JSON-RPC notification, which a client never sends; the + server drops it without a response. +- **Events are notifications** with `method` set to `event` and `params` + set to the event object itself, exactly as the engine serialised it, with + its `type` tag. Nothing is added, removed or renamed + ([C3](../bridges/README.md#c3-events-cross-as-opaque-json)). +- **Batches** (a JSON array of requests) are not supported in this version. + A server answers one with `-32600`. +- **One request in flight per method call**: the server MAY execute requests + from one connection concurrently or in order, and a client MUST NOT depend + on either. The engine serialises calls on its own lock regardless. +- **A one-shot call is a connection that sends `hello`, one request, and + closes.** Every mainstream runtime carries a WebSocket client, so this is + the whole of the "curl-shaped" use case, and it is why no second protocol + is needed for it. + +### Why there is no HTTP + +An earlier draft of this chapter served plain HTTP `POST` on the same port +for one-shot calls, through the WebSocket library's request hook. That was +verified before this chapter was written against `websockets` 16.1, the +version the Python package's lock file pins (the package itself accepts any +release from 12 up to but not including 17, so 16.1 is the only version the +behaviour has been checked on), and it does not work: the library's +handshake parser accepts only `GET`, and a connection that opens with any +other method is closed with no HTTP response at all, before the hook runs. A +`POST` would fail with zero bytes back rather than with a JSON-RPC error, +which is the worst possible failure for the caller this feature was meant to +serve. So there is no HTTP request path in this contract, and a server MUST +NOT advertise one; a client that wants a single call opens a WebSocket for +it. The reference server pins the fact with a test, so a library upgrade +that changes it is noticed rather than assumed. + +What the hook can do is answer a plain `GET` that does not ask for an +upgrade. A server MAY serve `GET /health` that way, answering `200` with a +small JSON body: + +```json +{"server":{"name":"offline-protocol-service","version":"0.28.0"},"api_version":1,"carrier":"unix"} +``` + +It is unauthenticated because it reveals nothing beyond the fact that a server +is listening, which the open port already reveals. It carries no engine state, +no address and no client count. A server that does not implement it answers +the library's default (`426 Upgrade Required`), and a client MUST NOT depend +on it. + +### Carriers + +**A Unix domain socket is the default.** The server creates it with owner-only +permissions (`0600`) inside a directory only the owner can enter (`0700`), +and removes a stale socket file at the same path before binding. The path is +the operator's choice; the reference server documents its default. On this +carrier `hello` carries no token, because the file permissions are the +credential. + +**TCP on loopback is opt-in and carries a token.** When an operator enables +it, the server binds `127.0.0.1` or `::1` only, generates 32 random bytes at +launch, writes them hex-encoded to a file at a path the operator names, and +requires them in `hello.token` on every connection. The file is created with +mode `0600` in the same call that creates it, never created and then +narrowed, because the moment between the two is exactly the window the mode +exists to close. The token is regenerated at every launch, so a token read +once never outlives the server that issued it. On neither carrier does the +server check the peer's credentials: the socket permissions and the token +are the whole of the check, and root or any process of the owner's uid +passes it. A `hello` without the token, or +with the wrong one, is answered with `PermissionDenied` and the connection is +closed with WebSocket close code `1008`. A server MUST NOT bind a non-loopback +address in this version; a deployment that wants the API across a network +puts a reverse proxy with its own authentication in front, and that proxy is +outside this contract. + +**Frame size.** The WebSocket library's default limit on one message is +1 MiB. `send_media` and `send_file` carry the file bytes inside the request, +so a server MUST raise its inbound limit to at least the engine's file size +limit (100 MiB, and not configurable through the interface today) plus +base64 overhead, or every media send above 1 MiB fails as a closed connection +instead of as the engine's own refusal. A client's outbound limit is its own +business. + +## The session + +### `hello` + +The first request on every connection MUST be `hello`, with one exception: +the three instance-less functions at the end of the method table need no +engine and no application id, and MAY be called before it. Any other method +before `hello` is refused with `InvalidState`, and a second `hello` on the +same connection is refused the same way. A server that answered another +method first would have no application id to stamp on it. + +```json +{"jsonrpc":"2.0","id":1,"method":"hello","params":{ + "app_id": "notes", + "client": "notes-desktop/2.4", + "token": "…" +}} +``` + +| Parameter | | | +|---|---|---| +| `app_id` | required | The application this connection speaks for. Validated by the same rule as the engine's own `app_id`: at most 256 bytes, not empty, not `.` or `..`, no control character, no `/`, `\` or `:` ([wire format](wire-format.md#the-abstract-message)). An id that fails it is `InvalidArgument`. | +| `client` | optional | A free-form name and version for the server's log. Never on the mesh. | +| `token` | required on TCP | The per-launch token. Ignored on a Unix socket. | + +The result: + +| Field | | +|---|---| +| `api_version` | `1`. A client that needs something this version does not have checks here and refuses to proceed rather than probing. | +| `server` | `{"name","version"}` of the server implementation. | +| `state` | The engine's `ProtocolState`: `Stopped`, `Running` or `Paused`. A server MAY accept clients before `start()`; every method that needs a running engine then answers `NotStarted`, which is the same answer an embedded caller gets. | +| `local_address` | The identity's `off1…` address, or `null` before the identity exists. Every client of one server gets the same address, which is the one-identity rule made visible. | + +Several clients MAY declare the same application id, at once or in sequence; +a second process of the same application is the ordinary case. They are one +application to every rule below: an event for that id reaches all of them, +and any of them may unregister a service the other registered. Whether an +id the server has never seen is accepted depends on whether the operator +has configured a space allow-list or a method deny; see the unlisted-id +rule under server-side rules. + +### `subscribe` and `unsubscribe` + +After `hello`, a client receives every event the routing rules deliver to +it. `subscribe` narrows that to a list of tags, and `unsubscribe` widens it +back: + +```json +{"jsonrpc":"2.0","id":3,"method":"subscribe","params":{"types":["message_received","message_delivered"]}} +{"jsonrpc":"2.0","id":4,"method":"unsubscribe","params":{"types":["message_delivered"]}} +{"jsonrpc":"2.0","id":5,"method":"subscribe","params":{"types":"all"}} +``` + +The filter is the client's own convenience and changes nothing about +routing: a tag a client subscribed to still reaches it only if the rules +below deliver it. A tag the server does not know is accepted and never +fires, so a client built against a newer engine keeps working against an +older server. Both return `true`. + +### Session methods + +These three are the server's own and are not in the interface definition. +They are the only such methods; everything else on the wire is one of the +engine's. + +| Method | Params | Result | +|---|---|---| +| `hello` | `app_id`, `client?`, `token?` | the object above | +| `subscribe` | `types` (`[]` or `"all"`) | `true` | +| `unsubscribe` | `types` (`[]` or `"all"`) | `true` | + +## Encoding + +The interface definition's types map to JSON one way, in both directions, +and this is the whole of the mapping: + +| Interface type | JSON | +|---|---| +| `string`, `string?` | string; an optional one may be `null` or omitted | +| `boolean` | `true` / `false` | +| `u8`, `u16`, `u32`, `u64`, `i16`, `i64`, `f32`, `double` | number. A `u64` above 2^53 loses precision in a JavaScript client; none of the engine's counters and timestamps reach it in practice, and a client that must be exact parses the raw text | +| enums (`MessagePriority`, `TransportType`, `ProtocolState`, `ContentType`, …) | the definition's own spelling as a string: `"Medium"`, `"WiFiDirect"`, `"Running"`, `"Image"` | +| `sequence`, `bytes` | a base64 string (RFC 4648, standard alphabet, padded). This is the one place the wire departs from the events, which carry bytes as the engine serialises them (see the catalogue) | +| `sequence` | array of strings | +| `record` | object | +| dictionaries (`SendMessageOptions`, `DorsConfig`, `MlsWelcomeMessage`, …) | object with the dictionary's field names. A field with a default in the definition may be omitted and takes that default; a server passes only the fields the caller sent and never substitutes a literal of its own ([C6](../bridges/README.md#c6-config-parsers-must-not-default-to-literals)) | +| `void` | `null` | + +The legend used in the tables below: a bare name is a string; `#` is a +number; `!` a boolean; `b64` bytes as base64; `[]` an array of strings; +`{Name}` an object shaped as the named dictionary; `` a string from the +named enum; a trailing `?` is optional. + +## Method table + +Every method below is one of the interface definition's own, called with its +own parameters. The three objects of the definition keep their own prefix on +the wire, so the guard that pins this table can map every row back to its +declaration: engine methods are bare, `MeshServices` methods carry +`services.`, and `DataStore` methods carry `data.`. The instance-less +functions of the definition's namespace are bare too, and are the only +methods a client may call before `hello`. + +### Engine: identity and state + +| Method | Params | Result | +|---|---|---| +| `get_state` | | `` | +| `local_address` | | string or `null` | +| `get_identity_public_key` | | `b64` (32 bytes) | +| `derive_user_id_from_public_key` | `public_key b64` | string | +| `sign_data` | `data b64` | `b64` (64 bytes). Signs with the shared identity key, which every client of one server shares by construction; an operator who does not want every application to hold a signing oracle denies it by application id (see server-side rules) | +| `verify_signature` | `public_key b64`, `data b64`, `signature b64` | `!` | +| `create_invite` | `petname?`, `sign !` | string (the invite blob) | +| `resolve_username` | `username` | `!` | + +### Engine: messaging + +| Method | Params | Result | +|---|---|---| +| `send_message` | `recipient`, `content`, `priority `, `reply_to_msg?` | string (message id). Stamped with the connection's application id; the server calls the rich surface with only that option set | +| `send_message_rich` | `recipient`, `content`, `options {SendMessageOptions}` | string (message id). `options.app_id` is set by the server to the connection's id; a value the client sends is refused with `InvalidArgument` | +| `forward_message` | `original_message_json`, `new_recipient`, `priority ?` | string (message id) | +| `send_presence_update` | `recipient`, `status ` | string (message id) | +| `send_typing_indicator` | `recipient`, `conversation_id`, `is_typing !` | string (message id) | +| `send_read_receipt` | `recipient`, `message_ids []` | string (message id) | + +### Engine: connection requests + +| Method | Params | Result | +|---|---|---| +| `send_connection_request` | `recipient`, `sender_name`, `key_package b64?`, `initial_message?` | string (message id) | +| `accept_connection_request` | `recipient`, `accepter_name`, `key_package b64?` | string (message id) | +| `reject_connection_request` | `recipient` | string (message id) | +| `cancel_connection_request` | `recipient` | string (message id) | + +### Engine: media and files + +| Method | Params | Result | +|---|---|---| +| `send_media` | `recipient`, `file_data b64`, `file_name`, `content_type `, `media_metadata {MediaMetadata}?` | string (file id). Stamped like `send_message` | +| `send_media_rich` | `recipient`, `file_data b64`, `file_name`, `content_type `, `options {MediaSendOptions}` | string (file id). `options.app_id` as for `send_message_rich` | +| `send_file` | `recipient`, `file_data b64`, `file_name` | string (file id) | +| `get_file_progress` | `file_id` | `{FileProgress}` or `null` | +| `cancel_file_transfer` | `file_id` | `null` | + +### Engine: secure sessions + +| Method | Params | Result | +|---|---|---| +| `is_mls_initialized` | | `!` | +| `has_pending_key_package` | `peer_id` | `!` | +| `get_establishment_state` | `peer_id` | `` | +| `establish_secure_session` | `peer_id` | `{MlsWelcomeMessage}` or `null` | +| `rekey_session` | `peer_id` | `!` | +| `mls_has_session` | `other_user_id` | `!` | +| `mls_list_sessions` | | `[]` | +| `mls_delete_session` | `other_user_id` | `null` | + +### Engine: manual MLS + +The manual surface operates on the shared identity's sessions, exactly as it +does for an embedded application. A client that drives it is driving every +other client's sessions too. + +| Method | Params | Result | +|---|---|---| +| `mls_generate_key_package` | | `{MlsKeyPackageBundle}` | +| `mls_get_or_create_key_package` | | `{MlsKeyPackageBundle}` | +| `mls_import_key_package` | `user_id`, `key_package_data b64` | `null` | +| `mls_get_pending_key_packages` | | `[{MlsKeyPackageBundle}]` | +| `mls_mark_key_package_synced` | `package_id` | `null` | +| `mls_create_session` | `other_user_id` | `{MlsWelcomeMessage}` | +| `mls_join_session` | `welcome {MlsWelcomeMessage}` | `{MlsGroupInfo}` | +| `mls_encrypt_for_user` | `other_user_id`, `plaintext b64` | `{MlsEncryptedMessage}` | +| `mls_decrypt_from_user` | `encrypted {MlsEncryptedMessage}` | `b64` or `null` | +| `mls_get_pending_welcome` | `other_user_id` | `{MlsWelcomeMessage}` or `null` | +| `mls_clear_pending_welcome` | `other_user_id` | `null` | +| `mls_decrypt` | `encrypted {MlsEncryptedMessage}` | `b64` or `null` | +| `mls_process_welcome` | `welcome {MlsWelcomeMessage}` | `{MlsGroupInfo}` | + +### Engine: groups + +| Method | Params | Result | +|---|---|---| +| `create_group` | `group_name` | `{MlsGroupInfo}` | +| `send_group_message` | `group_id`, `content`, `priority ?`, `reply_to_msg?` | `[]` (message ids). Group sends carry the configured application id: the per-send id is a 1:1 option today | +| `forward_message_to_group` | `original_message_json`, `group_id`, `priority ?` | `[]` (message ids) | +| `invite_to_group` | `group_id`, `invitee_user_id` | `null` | +| `remove_from_group` | `group_id`, `member_id` | `null` | +| `leave_group` | `group_id` | `null` | +| `list_groups` | | `[]` | +| `get_group_info` | `group_id` | `{MlsGroupInfo}` or `null` | +| `group_rich_readiness` | `group_id` | `{GroupRichReadiness}` | +| `group_relay_sync_state` | `group_id` | `` | +| `request_group_relay_registration` | `group_id` | `!` | +| `set_member_role` | `group_id`, `user_id`, `role` | `null` | +| `get_member_role` | `group_id`, `user_id` | string | +| `get_group_roles` | `group_id` | object of user id to role | +| `rename_group` | `group_id`, `new_name` | `null` | + +### Engine: blocking + +| Method | Params | Result | +|---|---|---| +| `block_user` | `user_id` | `null` | +| `unblock_user` | `user_id` | `null` | +| `get_blocked_users` | | `[]` | +| `is_user_blocked` | `user_id` | `!` | + +### Engine: transports and metrics + +Read-only views of the instance. Everything here describes the whole server, +never one client's share of it. + +| Method | Params | Result | +|---|---|---| +| `get_active_transports` | | `[]` | +| `get_transport_metrics` | `transport_type ` | `{TransportMetrics}` or `null` | +| `should_escalate_to_wifi` | | `!` | +| `get_topology` | | `{NetworkTopology}` | +| `get_message_stats` | | `[{MessageStats}]` | +| `get_delivery_success_rate` | | `#` | +| `get_median_latency` | | `#` | +| `get_median_hops` | | `#` | +| `get_battery_level` | | `#` or `null` | +| `get_is_charging` | | `!` | +| `is_relay` | | `!` | +| `get_relay_priority` | | `` | +| `get_relay_config` | | `{RelayConfig}` | +| `get_dors_config` | | `{DorsConfig}` | +| `get_dedup_stats` | | `{DedupStats}` | +| `get_pending_ack_count` | | `#` | +| `get_retry_queue_size` | | `#` | +| `get_mesh_relay_stats` | | `{MeshRelayStats}` | +| `get_mesh_relay_tunables` | | `{MeshRelayTunables}` | + +### Engine: instance-wide tuning + +Every call here changes the engine for every client. They are on the wire +because an embedded application can call them, and because the socket is the +authorization boundary; an operator who does not want every application +retuning the relay denies the group by application id (see server-side +rules). None of them is per client and none is undone when the caller +disconnects. + +| Method | Params | Result | +|---|---|---| +| `set_relay_priority` | `priority ` | `null` | +| `update_relay_config` | `config {RelayConfig}` | `null` | +| `force_transport` | `transport_type ` | `null` | +| `release_transport_lock` | | `null` | +| `update_dors_config` | `config {DorsConfig}` | `null` | +| `update_ack_config` | `config {AckConfig}` | `null` | +| `update_retry_config` | `config {RetryConfig}` | `null` | +| `update_dedup_config` | `config {DedupConfig}` | `null` | + +### Services + +The `MeshServices` object, one per server, under `services.`. Ownership is a +server-side rule stated below: a service is owned by the application id that +registered it. + +| Method | Params | Result | +|---|---|---| +| `services.register_service` | `service_id`, `version`, `capabilities` (object) | `null` | +| `services.unregister_service` | `service_id` | `!` | +| `services.discover_services` | `service_id?` | string (query id) | +| `services.send_service_request` | `provider`, `service_id`, `method`, `body` | string (request id) | +| `services.respond_to_service_request` | `request_id`, `requester`, `service_id`, `status`, `body` | string (message id) | + +### Documents + +The `DataStore` object, one per server, under `data.`. Space scoping is a +server-side rule stated below. + +| Method | Params | Result | +|---|---|---| +| `data.create_doc` | `space_id`, `doc_id` | `null` | +| `data.delete_doc` | `space_id`, `doc_id` | `null` | +| `data.remove_doc` | `space_id`, `doc_id` | `null` | +| `data.remove_space` | `space_id` | `null` | +| `data.set_interest` | `space_id`, `patterns []` | `null` | +| `data.list_docs` | `space_id` | `[]` | +| `data.list_spaces` | | `[]`, filtered to the spaces the client may open | +| `data.map_set` | `space_id`, `doc_id`, `collection`, `key`, `value_json` | `null` | +| `data.map_delete` | `space_id`, `doc_id`, `collection`, `key` | `null` | +| `data.map_get_json` | `space_id`, `doc_id`, `collection`, `key` | string or `null` | +| `data.list_push` | `space_id`, `doc_id`, `collection`, `value_json` | `null` | +| `data.list_delete` | `space_id`, `doc_id`, `collection`, `index #`, `count #` | `null` | +| `data.list_len` | `space_id`, `doc_id`, `collection` | `#` | +| `data.text_insert` | `space_id`, `doc_id`, `collection`, `position #`, `text` | `null` | +| `data.text_delete` | `space_id`, `doc_id`, `collection`, `position #`, `count #` | `null` | +| `data.text_value` | `space_id`, `doc_id`, `collection` | string | +| `data.counter_increment` | `space_id`, `doc_id`, `collection`, `amount #` | `null` | +| `data.counter_value` | `space_id`, `doc_id`, `collection` | `#` | +| `data.doc_json` | `space_id`, `doc_id` | string | +| `data.export_raw` | `space_id`, `doc_id` | `b64` | +| `data.flush` | `space_id`, `doc_id` | `null` | +| `data.flush_all` | | `null` | +| `data.doc_size` | `space_id`, `doc_id` | `#` | +| `data.attachment_hash` | `data b64` | string | +| `data.fetch_attachment` | `space_id`, `hash` | `null` | +| `data.provide_attachment` | `space_id`, `peer_id`, `hash`, `data b64` | `null` | +| `data.decline_attachment` | `space_id`, `peer_id`, `hash` | `null` | +| `data.fetch_attachment_from` | `space_id`, `peer_id`, `hash` | `null` | + +### Instance-less + +The definition's namespace functions that need no engine. They are the one +group a client may call before `hello`, because there is nothing to stamp and +nothing to route. + +| Method | Params | Result | +|---|---|---| +| `derive_address` | `public_key b64` | string | +| `parse_invite` | `blob` | `{InviteInfo}` | +| `verify_identity_assertion` | `assertion b64` | string (the proved address) | + +## Platform operations + +Everything in the interface definition that is not in the method table is a +platform operation and is not on the wire. A request naming one is answered +with `-32601`, the same answer an unknown name gets, so a client cannot +discover the set by probing. The two tables together partition the +definition: every declaration is in exactly one, and the guard that reads +this chapter asserts it. + +| Operation | Why the server owns it | +|---|---| +| `constructor`, `start`, `stop`, `pause`, `resume`, `process`, `receive_message`, `set_event_callback`, `poll_event`, `emit_test_event` | The lifecycle, the run loop and the drain (first invariant) | +| `initialize_mls`, `initialize_mls_with_file_stores`, `close_file_stores` | The identity's storage is attached once, by the process that holds the store key | +| `enable_telemetry`, `disable_telemetry`, `set_telemetry_enabled`, `flush_telemetry`, `flush_telemetry_blocking`, `telemetry_stats`, `end_telemetry_session`, `notify_app_state`, `telemetry_install_id` | The binding fills the platform fields and owns the session boundary ([C12](../bridges/README.md#c12-telemetry-is-callback-free-and-the-binding-fills-the-platform)); the server is the binding | +| `set_ble_transport_callback`, `set_wifi_direct_transport_callback`, `set_reticulum_transport_callback`, `set_nostr_transport_callback` | A callback interface has no JSON form, and each installs the driver of a carrier | +| `ble_peer_discovered`, `ble_peer_lost`, `ble_status_changed`, `ble_fragment_received`, `ble_get_next_fragment`, `ble_return_fragment`, `ble_get_peer_count`, `ble_set_peer_mtu`, `ble_clear_peer_mtu`, `ble_undersized_mtu_reports`, `ble_fragment_fallback_count`, `ble_recipient_not_among_peers_count`, `protocol_lock_diagnostics` | The Bluetooth LE driver's feed and drain, and the host's lock diagnostics | +| `internet_status_changed`, `internet_message_received`, `internet_get_next_message`, `internet_confirm_sent`, `internet_send_failed`, `internet_send_failed_with_reason`, `internet_peer_presence`, `internet_presence_watchlist`, `internet_relay_capabilities`, `internet_group_report_received`, `internet_address_declared`, `internet_address_declaration_refused` | The relay client's feed and drain, including the dedicated entry points the threat model's boundary 2 requires | +| `wifi_direct_status_changed`, `wifi_direct_message_received`, `wifi_direct_get_next_message`, `wifi_direct_peer_connected`, `wifi_direct_peer_disconnected` | The peer-stream driver's feed and drain | +| `reticulum_status_changed`, `reticulum_message_received`, `reticulum_get_next_message`, `reticulum_confirm_sent`, `reticulum_send_failed`, `reticulum_send_failed_with_reason`, `reticulum_address_declared`, `reticulum_address_declaration_refused`, `reticulum_gateway_capabilities`, `reticulum_peer_presence`, `reticulum_presence_watchlist` | The gateway client's feed and drain | +| `nostr_status_changed`, `nostr_message_received`, `nostr_message_received_at`, `nostr_get_next_message`, `nostr_confirm_sent`, `nostr_send_failed`, `nostr_send_failed_with_reason`, `nostr_get_public_key`, `nostr_get_subscription_filter`, `nostr_get_next_query`, `nostr_query_event_received`, `nostr_query_completed` | The Nostr client's feed and drain | +| `update_transport_metrics`, `remove_transport`, `set_battery_level`, `set_battery_state` | Facts about the host that only the host process knows | +| `identity_assertion`, `gateway_address_declaration` | Signatures a carrier driver presents to prove this device to a peer or a gateway; a client that could mint them could impersonate the server's carriers | +| `process_file_chunk`, `finalize_file` | The inbound chunk driver of a platform transport | +| `services.constructor`, `data.constructor`, `data.with_storage` | The server constructs the two objects once, over the engine it owns; `with_storage` takes a callback interface | +| `data.wipe_all` | Erases every client's documents and the identity's key documents at once. That is the operator's logout, taken at the server, never one application's call | +| `run_storage_conformance` | Takes a callback interface; a storage backend is verified by the host that supplies it | + +## Event catalogue + +### Routing + +Every event the engine emits is relayed as a notification to the clients the +rules below select, in the order the engine emitted it. The rules are the +server's; they never change an event and they never reach the mesh. + +1. **Stamped.** An event carrying an `app_id` field reaches the clients that + declared that id, and no other. Three events are stamped today: + `message_received`, `file_received` and `media_resend_required`. On the + last two the field is optional, and the engine does emit `file_received` + without it, for a transfer whose assembly had no metadata entry to read + the id from. An event of a stamped tag whose `app_id` is absent is + broadcast, not held, and logged, because the server has nothing to route + or hold it by; dropping it would lose a message the engine counts as + delivered. +2. **Correlated.** An event naming an identifier the server handed to one + client as a method result (a message id, a file id, a query id, a request + id) reaches that client, and no other. An event naming a service id + reaches the clients whose application owns that service. The server keeps + these tables in memory for the life of the process; an identifier issued + before a restart, or one that came from a peer rather than from a client's + call, is unknown, and an event naming an unknown identifier is broadcast. + A `message_delivered` for a message a client sent therefore reaches that + client and not its neighbours, and a `message_delivered` for a message the + relay re-sent after a restart reaches everyone. + + The engine emits synchronously, on the thread that is inside its call, so + some events fire before the call that caused them has returned: + `message_sent` is emitted inside the send when a transport takes the + frame at once, and the server does not yet hold the id it will correlate + by. An event emitted while the server is executing one client's call is + therefore that client's, whatever it names, and the server records the + identifiers it carries as that client's too. Without this rule the very + first event of every send would be broadcast. +3. **Broadcast.** Everything else is engine-level and reaches every client. + Group, presence, session and transport events are shared state of the one + identity, so every application sees them. + +The `Routing` column of the catalogue names the key: `app_id`, one of the +identifiers, or `all`. + +### The catalogue + +Every event the engine can emit, its fields, and how it is routed. The tag +is the `type` member; fields are the object's other members, using the legend +above with one difference: bytes in an event are an array of numbers, because +the event is relayed as the engine serialises it and the engine has always +written them that way. Sub-object shapes and the enum vocabularies follow the +table. The API reference documents about twenty of these with prose; this +table is read from the event definitions, held to them by the guard below, +and is the one a client is written against. + +| Tag | Fields | Routing | +|---|---|---| +| `message_sent` | `message_id`, `sender`, `recipient`, `content`, `priority`, `requires_ack !`, `timestamp #`, `lamport_clock #`, `forward_info {ForwardInfo}?` | `message_id` | +| `message_received` | `message_id`, `sender`, `recipient`, `content`, `hop_count #`, `transport`, `timestamp #`, `lamport_clock #`, `reply_to_msg?`, `reply_context {ReplyContext}?`, `content_type`, `media_metadata {MediaMetadata}?`, `forward_info {ForwardInfo}?`, `encrypted !`, `app_id` | `app_id` | +| `message_delivered` | `message_id`, `latency_ms #`, `hop_count #`, `transport` | `message_id` | +| `message_failed` | `message_id`, `reason`, `retry_count #` | `message_id` | +| `message_decryption_failed` | `message_id`, `sender`, `code `, `reason` | `all` | +| `transport_switched` | `from?`, `to`, `reason` | `all` | +| `relay_promoted` | `connection_count #`, `battery_level #?` | `all` | +| `relay_demoted` | `reason` | `all` | +| `neighbor_discovered` | `peer_id`, `transport`, `rssi #?` | `all` | +| `neighbor_lost` | `peer_id` | `all` | +| `identity_ready` | `address` | `all` | +| `network_metrics` | `neighbor_count #`, `relay_count #`, `delivery_ratio #`, `avg_latency_ms #` | `all` | +| `file_progress` | `file_id`, `chunks_sent #`, `total_chunks #`, `percentage #` | `file_id` | +| `file_received` | `file_id`, `file_name`, `file_size #`, `sender`, `content_type`, `media_metadata {MediaMetadata}?`, `file_data` (base64), `timestamp #?`, `caption?`, `reply_to_msg?`, `reply_context {ReplyContext}?`, `forward_info {ForwardInfo}?`, `app_id?` | `app_id` | +| `file_receive_failed` | `file_id`, `file_name`, `sender`, `reason` | `all` | +| `media_sent` | `file_id`, `content_type`, `recipient` | `file_id` | +| `media_send_failed` | `file_id`, `recipient`, `reason` | `file_id` | +| `message_deferred` | `message_id`, `recipient`, `reason`, `retry_count #`, `next_retry_at #?` | `message_id` | +| `message_retrying` | `message_id`, `recipient`, `retry_count #`, `next_retry_at #` | `message_id` | +| `message_undeliverable` | `message_id`, `recipient`, `reason`, `file_id?` | `message_id` | +| `media_resend_required` | `file_id`, `recipient`, `file_name`, `file_size #`, `app_id?` | `app_id` | +| `ack_evicted` | `message_id`, `priority`, `reason` | `message_id` | +| `fragment_assembly_evicted` | `message_id`, `completion_percent #`, `reason` | `all` | +| `relay_demoted_battery` | `battery_level #`, `min_required #` | `all` | +| `secure_session_established` | `peer_id`, `group_id`, `is_session !`, `initiated_by_local !` | `all` | +| `secure_session_failed` | `peer_id`, `reason` | `all` | +| `convergence_diag` | `stage`, `peer_id`, `detail` | `all` | +| `welcome_send_attempted` | `peer_id`, `message_id`, `group_id`, `attempt #` | `all` | +| `welcome_send_succeeded` | `peer_id`, `message_id`, `group_id`, `attempt #` | `all` | +| `welcome_send_failed` | `peer_id`, `message_id`, `group_id`, `attempt #`, `reason_code `, `transport_error?`, `retryable !`, `next_retry_at #?` | `all` | +| `welcome_send_expired` | `peer_id`, `message_id`, `attempt #`, `reason_code ` | `all` | +| `connection_request_received` | `sender`, `sender_name`, `timestamp #`, `key_package [#]?`, `initial_message?` | `all` | +| `connection_request_undeliverable` | `recipient`, `message_id`, `reason` | `message_id` | +| `connection_accepted` | `accepted_by`, `accepted_by_name`, `timestamp #`, `key_package [#]?` | `all` | +| `connection_rejected` | `rejected_by` | `all` | +| `connection_request_cancelled` | `cancelled_by` | `all` | +| `group_created` | `group_id`, `name` | `all` | +| `group_message_received` | `group_id`, `sender`, `content`, `timestamp` (a string), `message_id`, `reply_to_msg?`, `forward_info {ForwardInfo}?`, `media_metadata {MediaMetadata}?`, `content_type?` | `all` | +| `group_member_added` | `group_id`, `user_id`, `added_by`, `group_name?`, `authorized !?` | `all` | +| `group_member_removed` | `group_id`, `user_id`, `removed_by`, `authorized !?` | `all` | +| `group_info` | `group_id`, `name`, `created_by`, `created_at`, `members [{GroupInfoMember}]` | `all` | +| `user_groups` | `groups [{UserGroupSummary}]` | `all` | +| `group_error` | `reason`, `group_id?` | `all` | +| `group_relay_sync_changed` | `group_id`, `synced !`, `reason` | `all` | +| `username_resolved` | `username`, `claims [{UsernameClaim}]`, `rejected #`, `truncated #` | `all` | +| `group_message_sent` | `group_id`, `message_ids []`, `member_count #` | `all` | +| `group_message_partial_failure` | `group_id`, `failed_members []`, `succeeded_members []` | `all` | +| `group_message_delivery_report` | `group_id`, `message_id`, `delivered []`, `pushed []`, `missed_reissued []` | `all` | +| `group_rich_extras_dropped` | `group_id`, `unknown_members []` | `all` | +| `group_unauthorized_membership_change` | `group_id`, `committer`, `added []`, `removed []`, `reason`, `enforced !` | `all` | +| `group_epoch_fork_detected` | `group_id`, `local_epoch #?` | `all` | +| `group_epoch_fork_resolved` | `group_id`, `resolved_epoch #`, `failed_members []` | `all` | +| `group_role_changed` | `group_id`, `user_id`, `new_role`, `changed_by` | `all` | +| `group_renamed` | `group_id`, `new_name`, `old_name?`, `renamed_by` | `all` | +| `service_discovered` | `query_id`, `service_id`, `version`, `provider_peer_id`, `capabilities` (object), `hop_count #` | `query_id` | +| `service_request_received` | `request_id`, `service_id`, `method`, `body`, `sender` | `service_id` | +| `service_response_received` | `request_id`, `service_id`, `status`, `body`, `provider_peer_id` | `request_id` | +| `presence_updated` | `peer_id`, `status `, `timestamp #`, `last_seen_ms #?`, `source ` | `all` | +| `typing_indicator_received` | `sender`, `conversation_id`, `is_typing !`, `timestamp #` | `all` | +| `read_receipt_received` | `sender`, `message_ids []`, `timestamp #` | `all` | +| `dors_score_updated` | `scores` (array of `[transport, score #]` pairs) | `all` | +| `dors_transport_selected` | `from?`, `transport`, `reason_code `, `score #` | `all` | +| `dors_transport_switched` | `from?`, `to`, `reason_code `, `reason_detail?` | `all` | +| `dors_escalation_triggered` | `phase `, `from`, `to`, `reason_code `, `reason_detail?` | `all` | +| `security_warning` | `peer_id`, `reason_code `, `reason` | `all` | +| `message_relayed` | `message_id`, `sender`, `recipient`, `hop_count #`, `remaining_ttl #` | `all` | +| `user_blocked` | `user_id` | `all` | +| `user_unblocked` | `user_id` | `all` | +| `data_changed` | `space_id`, `doc_id`, `delta_bytes #` | `all` | +| `data_doc_removed` | `space_id`, `doc_id`, `by ` | `all` | +| `data_doc_size_warning` | `space_id`, `doc_id`, `compacted_bytes #`, `cap_bytes #` | `all` | +| `data_attachment_requested` | `space_id`, `peer_id`, `hash` | `all` | +| `data_attachment_received` | `space_id`, `peer_id`, `hash`, `data` (base64) | `all` | +| `data_attachment_unavailable` | `space_id`, `peer_id`, `hash`, `reason` | `all` | +| `data_doc_unsyncable` | `space_id`, `doc_id`, `bytes #`, `reason` | `all` | + +Two rows deserve a note. `message_decryption_failed` stands in for a message +that could not be read, and the event does not carry the frame's application +id even though the frame did, so it is broadcast; when the engine adds the +id (additive under C3) it becomes stamped. `message_relayed` reports a frame +this device forwarded for others and names another identity's message id, +which is why it is broadcast rather than correlated. + +Document events (`data_*`) are broadcast, and a client MUST apply its own +space filter to them: the server's space allow-list gates calls, and the +events carry the space id for the client to match, so a client that may not +open a space still learns that it changed. Filtering them at the server is a +server-side rule an implementation MAY add, and the reference server does. + +### Shapes + +| Object | Members | +|---|---| +| `ForwardInfo` | `original_sender`, `original_message_id`, `original_timestamp #`, `forward_count #` | +| `ReplyContext` | `sender`, `text`, `timestamp #?`, `reply_media_label?`, `reply_content_type?` | +| `MediaMetadata` | `mime_type`, `file_name`, `file_size #`, `duration_ms #?`, `width #?`, `height #?`, `thumbnail_base64?`, `media_id?`, `download_url?`, `thumbnail_url?`, `encryption_key?`, `iv?`, `ciphertext_hash?`, `sticker_provider?`, `sticker_remote_id?`, `sticker_kind?` | +| `GroupInfoMember` | `user_id`, `role`, `joined_at` | +| `UserGroupSummary` | `group_id`, `name`, `created_at` | +| `UsernameClaim` | `address`, `public_key`, `issued_at_ms #` | + +### Vocabularies + +| Enum | Values | +|---|---| +| `PresenceStatus` | `online`, `away`, `offline` | +| `PresenceSource` | `internet`, `reticulum`, `peer` | +| `DocRemovedBy` | `local`, `peer` | +| `DecryptionFailureCode` | `INVALID_PAYLOAD`, `NOT_INITIALIZED`, `INVALID_CIPHERTEXT`, `IDENTITY_MISMATCH`, `CRYPTO_FAILURE`, `PENDING_QUEUE_DROPPED`, `UNKNOWN` | +| `WelcomeReasonCode` | `TRANSPORT_UNAVAILABLE`, `PEER_UNREACHABLE`, `PEER_DISCONNECTED`, `TIMEOUT`, `INTERNAL_ERROR`, `RETRY_EXHAUSTED` | +| `DorsReasonCode` | `INITIAL_SELECTION`, `PRIMARY_SELECTED`, `PRIMARY_SUCCESS`, `FALLBACK_SUCCESS`, `ESCALATION_APPLIED`, `CURRENT_UNAVAILABLE` | +| `DorsEscalationPhase` | `TRIGGERED`, `APPLIED` | +| `DorsEscalationReasonCode` | `FALLBACK_SUCCESS`, `RETRY_THRESHOLD`, `POOR_SIGNAL`, `CONGESTION`, `LOW_TTL`, `LOW_SUCCESS_RATE` | +| `SecurityWarningCode` | `SENDER_ADDRESS_MISMATCH`, `TRANSPORT_IDENTITY_MISMATCH`, `CONTROL_SIGNATURE_INVALID`, `UNSIGNED_CONTROL_REJECTED`, `MEDIA_SENDER_GROUP_MISMATCH`, `PLAINTEXT_SEND`, `PLAINTEXT_RECEIVE_REJECTED`, `SESSION_SENDER_GROUP_MISMATCH`, `SESSION_REKEY_TRIGGERED`, `NOSTR_KEY_PACKAGE_SLOT_EXHAUSTED`, `PUSH_KEY_PACKAGE_POOL_EXHAUSTED`, `RELAY_ADDRESS_BINDING_MISMATCH`, `RELAY_ADDRESS_DECLARATION_REFUSED`, `GROUP_LEAF_IDENTITY_UNPROVEN`, `STALE_CONTROL_FRAME`, `GATEWAY_ADDRESS_BINDING_MISMATCH`, `GATEWAY_ADDRESS_DECLARATION_REFUSED` | + +The transport name is spelled two ways by the engine, and a client accepts +both. `message_received`, `message_delivered`, the DORS events and a +`transport_switched` the engine emits carry the transport's label: `ble`, +`wifiDirect`, `internet`, `reticulum`, `nostr`, or `delayed` on a +`message_received` for a message that was held and decrypted later. +`neighbor_discovered`, and a `transport_switched` the FFI layer emits when a +carrier's status changes, carry the casing the platform bridge passes as a +string: `BLE`, `WiFiDirect`, `Internet`, `Reticulum`, `Nostr`, and on that +`transport_switched` the literal `None` as `to` when a carrier disconnects +and nothing takes over. Neither spelling is the interface definition's enum +(`Ble`), and `wifi_direct` never occurs. The `priority` string is spelled +two ways by the engine and a client accepts both: lowercase on `message_sent` +(`low`, `medium`, `high`, `critical`) and capitalised on `ack_evicted` (`Low`, +`Medium`, `High`, `Critical`). + +These are the spellings the events carry. A method parameter of an enum type +uses the interface definition's spelling, so a client sends `"Online"` to +`send_presence_update` and reads `"online"` back on `presence_updated`. + +The vocabularies above are the engine's, pinned against the TypeScript +declarations by the guards [T2](../bridges/typescript.md#t2-event-types-are-pinned-from-the-rust-side) +names; this chapter restates them so a client author has one place to read, +and the guard for this chapter (below) holds the tag column to the engine. + +## Replay + +The interface that drives a radio has a rule for events that fire before +anyone is listening ([C10](../bridges/README.md#c10-lifecycle-rules-for-event-emission)), +and a server has the same problem one level up: an application's client may +not be connected when its message arrives. The engine has already +acknowledged that message, dedup-marked its id and dropped its queued copy +by the time `message_received` is emitted, so the sender will not resend and +nothing in the engine will restate the event. A server that dropped it +would lose a message that the mesh counts as delivered. + +So a server holds **stamped inbound events** for an application id with no +connected client: two of C10's three tags, `message_received` and +`file_received`, plus `media_resend_required` because it is stamped too. +C10's third tag, `message_decryption_failed`, is excluded until it carries an +`app_id`, because there is no id to hold it under. Each held event is kept +whole, in arrival order, in a per-application-id FIFO capped at 256 entries +with the oldest dropped past the cap. The cap is the same real capacity the +mobile bridges use, and it is a capacity rather than a backstop: past it, +messages are lost, and a server SHOULD log each drop. When a client next +declares that application id and the server accepts the `hello`, the server +delivers the held events to it, in order, after the `hello` result and +before any newer event for that id. Delivery is **at least once**: a client +that disconnected mid-delivery may see an event twice, and message ids are +the idempotency key. An event of a stamped tag that arrives with no `app_id` +is not held (see routing), because there is nothing to hold it under. + +Nothing else is held. A broadcast event describes state the next event of +its kind restates, and a correlated event names an identifier whose client +was connected when it was issued; a server MAY hold correlated events for a +client that disconnected and reconnected under the same application id, and +MUST NOT hold them past the process's life. When `message_decryption_failed` +becomes stamped it joins the held set. + +The hold is per application id and shared by every client of that id: a +second process of the same application that connects while the first is +connected receives nothing from the hold, because the first already did. + +## Errors + +A failed call is a JSON-RPC error object. The `code` is an integer because +the framing requires one, and the client switches on the variant name, which +is what the engine's own bindings switch on: + +```json +{"jsonrpc":"2.0","id":2,"error":{"code":-32004,"message":"No key package available for recipient: off1...","data":{"variant":"NoKeyPackage"}}} +``` + +- **`error.data.variant`** is the `ProtocolError` variant name from the + interface definition, unchanged. The taxonomy is the engine's, all + twenty-five variants of it, and the distinctions the engine draws survive: + `NotStarted` is an engine that is not running, `InvalidState` is an + operation that is wrong for the engine's current state or for a + resource's, and `InvalidConfiguration` is a value the engine refused, + where retrying with the same value cannot help. +- **`error.message`** is the variant's display string, with the detail the + engine attached. +- **`error.code`** is `-32000 - n`, where `n` is the variant's position in + the definition's error enum, zero-based. The enum is append-only by + contract ([C2](../bridges/README.md#c2-the-error-enum-is-append-only)), + which is the same rule that keeps the generated bindings decoding it, so a + position never changes and a variant appended later takes the next + number without anyone assigning it. The position is zero-based, so + `NotStarted` is `-32000`; the discriminant the generated bindings use for + the same variant is one-based (`NotStarted` is `1`), and the two are not + meant to agree, so nobody should "correct" one to the other. The + server-defined range the JSON-RPC specification reserves runs to + `-32099`; the first fifty numbers are this taxonomy's, and nothing else is + ever assigned in them. +- **The standard codes** keep their standard meaning: `-32700` parse error, + `-32600` invalid request (including a batch), `-32601` method not found + (including every platform operation), `-32602` invalid params (a missing + required parameter, a positional array, a value of the wrong JSON type), + `-32603` an internal failure of the server itself, never of the engine. +- **Session refusals reuse the taxonomy.** A method before `hello`, or a + second `hello`, is `InvalidState`; a bad application id is + `InvalidArgument`; a wrong token, a service another application owns, a + space outside the client's allow-list, and a method group the operator + denied are all `PermissionDenied`. No variant exists only on this wire. + +| Variant | Code | The engine's meaning | +|---|---|---| +| `NotStarted` | `-32000` | The engine is not running. Retry after the server starts it | +| `AlreadyStarted` | `-32001` | Not reachable from a client; `start` is the server's | +| `InvalidConfiguration` | `-32002` | A value was refused. Retrying with the same value cannot help | +| `SendFailed` | `-32003` | The transport refused or lost the send | +| `NoKeyPackage` | `-32004` | No key package for the recipient yet | +| `SessionNotReady` | `-32005` | Establishment in progress; the message names the state | +| `EncryptFailed` | `-32006` | Outbound encryption failed | +| `InvalidState` | `-32007` | Wrong for the current state of the engine or of the named resource | +| `MlsNotInitialized` | `-32008` | The server has not attached storage | +| `MlsError` | `-32009` | An MLS operation failed | +| `UserBlocked` | `-32010` | The target is blocked | +| `MediaTransferLimit` | `-32011` | Too many transfers to one recipient; retry after one completes | +| `LockPoisoned` | `-32012` | A thread panicked inside the engine | +| `Other` | `-32013` | Unclassified | +| `TransportError` | `-32014` | A transport failure outside a send | +| `SerializationError` | `-32015` | A payload did not serialise or parse | +| `ServiceError` | `-32016` | A mesh service registry or request failure | +| `GroupNotFound` | `-32017` | The group does not exist locally | +| `PermissionDenied` | `-32018` | A role the caller lacks, or a server-side rule | +| `InvalidArgument` | `-32019` | A caller-supplied value failed validation | +| `DataDisabled` | `-32020` | The data layer is off in configuration | +| `DataStorageUnavailable` | `-32021` | The data layer has no storage yet | +| `DocTooLarge` | `-32022` | The document is over its cap | +| `DataCorrupted` | `-32023` | A stored document could not be read | +| `TelemetryConfigInvalid` | `-32024` | Not reachable from a client; telemetry is the server's | + +## Server-side rules + +Four rules separate the applications behind one server. Each is applied by +the server before a call reaches the engine, each refuses with +`PermissionDenied`, and none has any representation on the mesh: a peer +sees one identity registering services and syncing spaces, exactly as it +would from one application. They separate applications from each other's +mistakes, not from a hostile process inside the socket boundary (fourth +invariant). + +**Unlisted ids.** With no rule configured at all, a server accepts any +well-formed application id at `hello`. Once any space allow-list or method +deny is configured, the set of ids those entries name is the set of +applications the operator knows about, and a `hello` declaring an id with +no entry is refused with `PermissionDenied`. Service ownership does not +count: it is a runtime shadow the clients' own registrations build, not +something the operator configured, so a first registration never flips an +open server to default-deny. Without this rule the two configured rules are +void: an application whose id is denied a method, or scoped to a space, +reconnects under an id no rule names and is denied nothing and scoped to +nothing. An operator who wants an open server configures no rule; an +operator who configures one has listed every application. + +**Service ownership.** `services.register_service` records the calling +application id as the owner of that service id. `services.unregister_service` +and `services.respond_to_service_request` for a service another application +owns are refused; `service_request_received` for it is routed to the owner's +clients. The engine has no owner on a registration and no way to enumerate +them, so the server's registry is a shadow of the engine's, rebuilt from +the clients' calls, and empty after a restart until they register again. An +application that registers a service id another application already holds is +refused, which is the collision the registry exists to catch: without it the +second registration silently replaces the first in the engine and the first +application's requests start arriving at the second. + +**Space scoping.** The operator's configuration MAY map each application id +to a list of glob patterns over space ids. Every `data.*` call naming a space +outside the caller's patterns is refused, `data.list_spaces` is filtered to +them, and a document event for a space outside them is not delivered. An +application with no entry may open every space. The failure this prevents is +two applications choosing the same space name by accident and merging each +other's documents, which the engine cannot notice because to it there is one +member. + +**Method groups.** The operator's configuration MAY deny a named group of +methods to an application id: `sign_data`, the manual MLS group, the +instance-wide tuning group, or any single method. A denied method is refused +with `PermissionDenied`, not `-32601`, because the method exists and the +refusal is a policy the client should be able to read. Nothing is denied by +default. + +## What a guard pins + +The method table and the platform table together name every declaration in +the interface definition exactly once, and the catalogue names every event +tag the engine can emit. Both claims rot silently: a method added to the +definition and to no table is a method a client cannot find, or worse one the +reference server exposes without this chapter saying so; an event added to +the engine and not here is one no client is written for. + +A Rust guard in the FFI crate reads this chapter, the interface definition, +the engine's event definitions and the reference server's dispatch table, and +asserts: + +1. the set of names in the method table (with `services.` and `data.` + stripped) plus the set in the platform table equals the set of + declarations in the definition, with no name in both; +2. the reference server's dispatch table exposes exactly the method table's + names and no platform name; +3. every variant of the engine's event enum appears as a tag in the + catalogue, and every tag in the catalogue is a variant. + +The tables are written to be read by that guard as well as by a person: a +row's first cell is the backticked name, the method table runs from the +heading `Method table` to the heading `Platform operations`, the platform +table from there to `Event catalogue`, and the catalogue's rows are the ones +whose first cell is a backticked tag between `The catalogue` and `Shapes`. +Rows in the platform table list several names in one cell; the guard reads +every backticked token in the first cell. A change to those headings is a +change to the guard. + +## What this chapter does not cover + +- **The reference server's own configuration**: socket paths, the token + file, the allow-lists, the store key. Those are its guide's, and a second + server may configure them however it likes. +- **Transport drivers.** A server that wants Bluetooth LE or a peer stream + runs the driver in its own process, against the platform operations. This + chapter says only that a client cannot. +- **Authentication of anything but the socket.** There is no account, no + per-application credential, and no way for one application to prove to the + server that it is not another. An operator who needs that runs one server + per identity, which is one server per set of mutually trusting + applications. +- **A network-facing API.** Loopback only, by construction, in this version.