From e51a2bd826336bf700dbb531541e4b2601bf6ab3 Mon Sep 17 00:00:00 2001 From: bahdotsh Date: Wed, 30 Sep 2026 21:14:01 +0530 Subject: [PATCH 1/7] docs(spec): the custody chapter, with its receipt prefix reserved A forwarder holds a stranger's frame for five seconds; the device's own messages get seven days on disk. The custody chapter specifies how a device closes that gap for one class of traffic, and it is written before the code so the shape is agreed where the mechanism is invariant-laden. The chapter states seven invariants first: custody is replication, never transfer; a receipt settles nothing; the hold is strictly shorter than the outbox lifetime, because a custodian holds ciphertext it cannot re-seal; a custodian never blanks the route it is holding; Class A only, by the depositor's word, since a custodian cannot classify a sealed frame; a deposit comes from the depositor itself over the link that proved it, so the frame's sender must be the proven arrival peer and every forwarder strips the unsigned deposit key from any third-party frame it transmits; and custody is opt-in with its own erase. Then the deposit (the depositor's own frame with a reserved hop-local metadata key, accepted only where forwarding has already failed and the forwarding identifier is already released), the receipt (a signed control frame under the ordinary gate, sent once over the arrival link and gated on a capability entry so a peer that does not reserve the prefix never receives one), redelivery through a dedicated governor intake that skips the suppression check and the hop accounting and returns a frame that reaches no link to the store, a hold judged in wall time from the acceptance timestamp so a lowered hold applies to records already held and a custodian that was off for a week delivers nothing stale, the quotas with a stranger tier of zero, and the erase. The chapter states the condition on "cannot falsely settle": it holds with crypto recovery enabled, the default; with it disabled a stale copy is a terminal decrypt failure that is acknowledged, and for Class A the next version-offer exchange re-offers the change. Reservations, each in the registry that owns it: `__CUSTODY_RECEIPT__` in the control-message chapter and the engine's prefix macro, so the two-way registry test binds them and application text can never occupy the name; `__custody` in the wire-format chapter's reserved metadata keys; `data_versions` entry 7 in the capability chapter. No handler consumes the prefix yet and no peer advertises the entry, so nothing changes on the wire until the implementation lands. The threat model gains R19 (custody-borne re-key pressure, its condition, and what an integrator should read from the signal), R20 (a custodian retains third-party routing metadata for hours) and R21 (deposit spam, bounded by one proven address for depositor, tier, receipt and capability). The acknowledgement side-channel section says custody changes no acknowledgement decision. The replication state machine records the redelivery trigger, what each redelivered kind does at the recipient, and that the bottom rung of the catch-up ladder becomes reachable more often, which is one of the two reasons the hold is hours and not days. The conformance chapter lists custody among the chapters that are not yet surfaces: the receipt vector lands with the codec. --- CHANGELOG.md | 19 + .../offline-protocol/src/protocol/prefixes.rs | 8 + docs/README.md | 1 + docs/security/threat-model.md | 104 ++++ docs/spec/README.md | 1 + docs/spec/capability-negotiation.md | 21 +- docs/spec/conformance.md | 4 + docs/spec/control-messages.md | 20 + docs/spec/custody.md | 502 ++++++++++++++++++ docs/spec/wire-format.md | 1 + docs/state-machines/data-replication.md | 10 + 11 files changed, 690 insertions(+), 1 deletion(-) create mode 100644 docs/spec/custody.md diff --git a/CHANGELOG.md b/CHANGELOG.md index 551396f87..62201f19a 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -15,6 +15,25 @@ archived by series under [docs/changelog/](docs/changelog/); see the ### Added +- **The custody chapter.** `docs/spec/custody.md` specifies how a device + holds a neighbour's replication frame for hours instead of the five + seconds a forwarder gives it today: an explicit deposit in which the + depositor asserts the class and this version carries only `delta`, `snap`, + `vv` and `blob_gone`; a signed receipt that settles nothing; a hold that is + validated strictly shorter than the outbox lifetime, because a custodian + holds ciphertext it cannot re-seal; acceptance only for what the mesh could + not forward, at the point where the forwarding identifier is already + released, so a custodian never blanks its own route; redelivery as an + ordinary forward; entry and byte quotas per depositor with a stranger tier + of zero; and an erase of its own, because no global wipe exists. The + control-message registry and the engine's prefix list reserve + `__CUSTODY_RECEIPT__`, the wire-format chapter reserves the metadata key + `__custody`, and `data_versions` gains entry 7 for the receipt. The threat + model gains R19 (custody-borne re-key pressure), R20 (a custodian retains + third-party routing metadata) and R21 (deposit spam). Nothing ships in this + entry but the contract and the reservations; the store, the quotas and the + receipt follow it, and custody stays off until they do. + - **The leaf node builds for ESP32 RISC-V parts and for Cortex-M0.** `offline-protocol-leaf` now compiles, and CI lints it, for `riscv32imac-unknown-none-elf` (ESP32-C6, ESP32-H2), diff --git a/crates/offline-protocol/src/protocol/prefixes.rs b/crates/offline-protocol/src/protocol/prefixes.rs index d0447e809..02722a742 100644 --- a/crates/offline-protocol/src/protocol/prefixes.rs +++ b/crates/offline-protocol/src/protocol/prefixes.rs @@ -134,6 +134,14 @@ define_internal_prefixes! { TYPING_INDICATOR = "__TYPING__", /// Prefix for read receipt messages. READ_RECEIPT = "__READ_RECEIPT__", + /// Prefix for the custody receipt a custodian answers a deposit with + /// (`docs/spec/custody.md`). Reserved ahead of the implementation on the + /// same terms as `DATA_V1`: the name is registered before any frame uses + /// it, so no user message sent in the meantime can occupy it. Not in + /// `DATA_PLANE_PREFIXES`, so it is signature-gated by construction; no + /// handler consumes it yet, and no peer is sent one until it advertises + /// the custody capability entry. + CUSTODY_RECEIPT = "__CUSTODY_RECEIPT__", } /// Data-plane prefixes that are **excluded** from the security gate. diff --git a/docs/README.md b/docs/README.md index 174db4d38..e36c8f894 100644 --- a/docs/README.md +++ b/docs/README.md @@ -62,6 +62,7 @@ implementation written against these documents should interoperate. | [Bluetooth LE framing](spec/ble-framing.md) | The GATT contract, the fragment header, and what a receiver owes on reassembly | | [Username discovery and invites](spec/username-discovery.md) | The self-certifying invite payload, the username directory, and the signing-domain registry | | [The gateway contract](spec/gateway-contract.md) | What a gateway is, the five verbs, the daemon wire protocol, and the backbone | +| [Custody](spec/custody.md) | Holding a neighbour's replication frame for hours: the deposit, the receipt that settles nothing, the hold, redelivery, quotas, erase | | [Conformance](spec/conformance.md) | The two profiles, what every implementation owes, and the vectors that decide it | ## Security diff --git a/docs/security/threat-model.md b/docs/security/threat-model.md index a75cfd1a5..b8b92a6c7 100644 --- a/docs/security/threat-model.md +++ b/docs/security/threat-model.md @@ -200,6 +200,14 @@ Consequences for application teams are in [Delivery and ACKs](../state-machines/delivery-and-acks.md). The headline: **a missing acknowledgement is not proof of non-delivery.** +A frame delivered out of [custody](../spec/custody.md) hours after it was +sent is answered by exactly these rules and no others: the custodian's +receipt is not an acknowledgement, the recipient's acknowledgement travels +the ordinary ladder to the sender and never to the custodian, and a copy +sealed to an epoch the recipient has left is withheld like any other desync. +Custody widens nobody's view of the acknowledgement channel in this version, +because acknowledgements are not carried. + ## Known residual risks Stated plainly, because a threat model that lists only what it defeats is @@ -766,6 +774,102 @@ parent's access list. And a restored backup of the key directory rolls MLS state back: it cannot leak a message, but it reinstates spent ratchet secrets on disk and breaks the sessions until they are re-established. +### R19. Custody-borne re-key pressure + +A [custodian](../spec/custody.md) holds ciphertext it cannot re-seal, for +hours, and delivers it when a path appears. If the pair re-keyed in the +meantime, the copy classifies at the recipient as a session desync **when +crypto recovery is enabled, which is the default**: the acknowledgement is +withheld and the identifier unmarked, so nothing settles falsely, and the same +branch schedules a rate-limited re-key and emits the `SESSION_REKEY_TRIGGERED` +security warning. Benign late delivery therefore raises the rate of a signal +this document, and the configuration documentation, tell integrators to read +as injection. + +With crypto recovery disabled, the same copy is a terminal decrypt failure +instead: dropped and acknowledged, and the acknowledgement settles the +depositor's outbox entry for a frame the recipient never read. Custody is what +makes that path reachable, because a direct resend is re-sealed to the live +epoch and never arrives stale. For the one class carried, the recipient's +version vector still lacks the change and the next version-offer exchange +re-offers it, so the loss is a delay rather than a divergence; that is one of +the reasons only replication frames are carried, and a deployment that +disables crypto recovery anywhere should leave custody off. + +**Why it stands:** the recipient cannot tell a stale custody copy from an +injected wrong-epoch frame, for the reason R2 gives: every pre-authentication +verdict is structurally outside the reach of the credential check. The +re-key floor bounds the work, not the signal. + +**What bounds it:** this version carries only replication frames, and a +replication frame that dies with an epoch is re-derivable from state, so +the copy that fails is one the depositor's own re-sealing resend replaces. +The floor (`REKEY_INTERVAL_SECS`) holds one re-key per peer per window +whatever the cause. The hold is strictly shorter than the outbox lifetime, +so a copy never outlives the sender's own ladder. + +**What integrators should read:** in a deployment with custody enabled, a +re-key warning that follows a `peer_rediscovered` event for the same pair by +seconds is the expected shape of a late custody copy, and a sustained rate +with no rediscovery is still the injection signature. The implementation +does not distinguish the two on the warning itself; doing so would need the +custodian to be trusted about provenance, which it is not. + +### R20. A custodian retains routing metadata about third parties + +A sealed payload is opaque at every hop, but the outer frame is not: sender, +recipient, identifier, application id, priority, hop fields, timestamp and +size are readable by whoever holds it. A forwarder learns them for five +seconds in memory. A custodian keeps them for hours, on disk, for traffic +between two other parties, and an A1 or A2 adversary who volunteers as a +custodian collects them by design. + +**Why it stands:** routing needs the recipient, and a custodian needs the +identifier to deduplicate and the sender to key its quotas. Sealing the +held record protects the directory, not the custodian, which is the party +the exposure is to. + +**What bounds it:** custody is off by default; the stranger tier defaults to +zero, so a device that has met nobody holds nothing and a device holds only +for peers it has a session with; the hold is bounded and validated; every +record is erased on the data-layer wipe and at start when custody is +disabled. Quotas bound volume, not observation: an attacker who wants to +observe a pair needs only to be their neighbour with custody on, which is +the same position A1 already has for five seconds per frame. What custody +adds is retention, and the opt-in documentation MUST say so in those words. + +### R21. Deposit spam + +A deposit is a metadata key on the depositor's own frame, authenticated by +nothing beyond the transport identity of the peer that handed it over, and +the class it asserts cannot be checked by the custodian. An A2 adversary can +therefore offer frames for custody as fast as the mesh admits them, assert +`data` on any sealed body, and spend the custodian's durable writes and +storage. + +**Why it stands:** a custodian cannot see inside a sealed frame, so the +assertion cannot be verified, and a deposit cannot be signed by anything +the custodian could check that the transport identity does not already +prove. + +**What bounds it:** acceptance requires the frame's `sender` to be the +address the transport proved for the peer that handed it over, so a deposit +cannot be made on another device's behalf, and the quotas, the receipt and +the capability lookup all key on one address the transport proved. Every +forwarder strips the deposit key from a third-party frame before transmitting +it, so an attacker cannot recruit honest forwarders to deposit its frames at +custodians it never touched. The stranger tier is zero by default, so the +attacker must first complete a session, which costs a key package exchange +and gives the operator a name. Per-depositor entry and byte budgets cap what +one address can hold, the global budgets cap the store, the forwarding +governor's per-neighbour rate limit is applied before a frame ever reaches +the custody decision, and a duplicate identifier is never stored twice. A +false class assertion costs the depositor's own recipient exactly what a +direct resend would, because the recipient's deduplication and its terminal +data-layer outcomes apply unchanged, so it buys the attacker nothing a +direct send does not. Refusals are silent and counted, so the quotas are not +an oracle. + ## Network egress Until 0.26 the Rust crates opened no socket: every byte that left a device diff --git a/docs/spec/README.md b/docs/spec/README.md index b77d0f75c..822e55e16 100644 --- a/docs/spec/README.md +++ b/docs/spec/README.md @@ -25,6 +25,7 @@ document says which reading is normative for the wire. | [Peer-stream framing](stream-framing.md) | The preamble that proves a stream's peer, the length-prefixed message frame, and the LAN discovery hint | | [Username discovery and invites](username-discovery.md) | The self-certifying invite payload, and the non-authoritative username directory | | [The gateway contract](gateway-contract.md) | What a gateway is, the five verbs it implements, the gateway-daemon wire protocol, and the backbone | +| [Custody](custody.md) | Holding a neighbour's replication frame for hours instead of seconds: the deposit, the receipt that settles nothing, the hold, redelivery, quotas and erase | | [Conformance](conformance.md) | The two profiles, what every implementation owes, and how the vectors decide it | ## Conformance vectors diff --git a/docs/spec/capability-negotiation.md b/docs/spec/capability-negotiation.md index ba872f637..a53a1291b 100644 --- a/docs/spec/capability-negotiation.md +++ b/docs/spec/capability-negotiation.md @@ -10,7 +10,7 @@ Peers advertise what they can parse in the key package payload, the body of a | `wire_versions` | Hop-local | Which frame encodings we may emit to this peer | JSON only | | `env_versions` | End to end | Which `__MLS_ENC__` payload forms we may emit | Legacy JSON envelope only | | `rich_versions` | End to end | Whether we may seal a `__RICH_V1__` body, and the v2 media envelope | Plain text only, extras dropped | -| `data_versions` | End to end | Whether we may send `__DATA_V1__` document sync frames, and which document encoding they carry. Entry 1 is 1:1 replication; entry 2 additionally means the peer intercepts these frames inside a *group* ciphertext; entry 3 additionally means the peer speaks the blob-fetch frames and routes a data-purposed media transfer into its document layer; entry 4 additionally means the peer reads the removals a version offer carries; entry 5 additionally means it answers inside the interest an offer declares; entry 6 additionally means it carries attachment bytes inside a group | No replication with that peer | +| `data_versions` | End to end | Whether we may send `__DATA_V1__` document sync frames, and which document encoding they carry. Entry 1 is 1:1 replication; entry 2 additionally means the peer intercepts these frames inside a *group* ciphertext; entry 3 additionally means the peer speaks the blob-fetch frames and routes a data-purposed media transfer into its document layer; entry 4 additionally means the peer reads the removals a version offer carries; entry 5 additionally means it answers inside the interest an offer declares; entry 6 additionally means it carries attachment bytes inside a group; entry 7 additionally means it parses the custody receipt, so a custodian may answer its deposits | No replication with that peer | | `ctrl_versions` | End to end | Which control-frame signing payload we build for this peer. Entry 2 means the peer verifies `offline-ctrl-v2`, which binds the frame's timestamp | Build `offline-ctrl-v1`, which states no freshness | | `nostr_pubkey` | End to end | Which key metadata is sealed to on the Nostr path | Seal to the publicly computable key | @@ -114,6 +114,25 @@ answer that was never coming. A requester MUST refuse only on knowledge: a member this device has never exchanged key packages with is the ordinary case, and reading its absence as incapacity would refuse every such fetch. +Entry 7 gates one frame in one direction. A peer that advertises it parses +the `__CUSTODY_RECEIPT__` control frame ([Custody](custody.md)), so a +custodian that accepted one of its replication frames may answer with a +receipt. A custodian MUST NOT send a receipt to a peer without it: the peer +does not know the prefix as a control frame, so with encryption on it refuses +the receipt as inbound plaintext and records a security refusal, and on a +plaintext-only deployment it shows the receipt to its user as a message. The +gate exists to keep a helpful neighbour from ever causing either. It gates +nothing else. A deposit request is a metadata key an unaware receiver ignores, so a +depositor MAY write it toward any neighbour, and acceptance of a deposit is +decided by the custodian's quotas, never by this entry. It lives in the +replication family rather than in a list of its own because custody carries +replication frames only: a device that does not replicate has nothing to +deposit and nothing to hold, and a separate list would add a key package +field, which is a codec change with vectors, to state what the family +already scopes. It has no attested sibling, because the receipt is a 1:1 +control frame between neighbours and a group inviter has nothing to say +about one. + Entry 2 has a second source, because members of a group never exchange key packages with each other: a group inviter MAY attest it for a member on the Add commit and in the Welcome, exactly as it attests `rich_versions`. An diff --git a/docs/spec/conformance.md b/docs/spec/conformance.md index 0b14210d3..ded89ecb3 100644 --- a/docs/spec/conformance.md +++ b/docs/spec/conformance.md @@ -224,3 +224,7 @@ Some chapters specify behaviour that has no vector file: reads as a key problem rather than an encoding one. The rest of the chapter is a JSON message vocabulary whose mistakes surface as a rejected message, and it stays prose. +- [Custody](custody.md) adds one encoding, the receipt body, and has no + vector for it yet: the chapter precedes the codec, and a vector computed + from prose pins nothing. The vector lands with the implementation, in the + crate that holds the codec, and this list loses the entry then. diff --git a/docs/spec/control-messages.md b/docs/spec/control-messages.md index 7e4dee6a7..40cca800b 100644 --- a/docs/spec/control-messages.md +++ b/docs/spec/control-messages.md @@ -154,6 +154,26 @@ encryption requirement**, so discovery gossip and the application-supplied request and response bodies are sent in cleartext. See [residual risk R9](../security/threat-model.md#r9-service-discovery-and-service-bodies-are-signed-not-encrypted). +### Custody + +| Prefix | Direction | Body | +|--------|-----------|------| +| `__CUSTODY_RECEIPT__` | custodian to depositor | JSON `{"v":1,"id":,"hold_ms":}`. Signature-gated like every control frame; introduces no signing domain of its own | + +The receipt is the one frame [Custody](custody.md) adds. It is reserved +ahead of the implementation on the same terms `__DATA_V1__` was: the name is +registered before any frame uses it, so no application message sent in the +meantime can occupy it. A receipt settles nothing and is never routed as an +acknowledgement; a receiver that reserves the prefix but does not implement +custody MUST NOT advertise `data_versions` entry 7, and then never receives +one. + +The deposit itself is not a frame and has no prefix. It is the depositor's own +`__MLS_ENC__` frame carrying the reserved metadata key `__custody` +([reserved metadata keys](wire-format.md#reserved-metadata-keys)), which every +forwarder strips from a third-party frame before transmitting it, so a deposit +travels exactly one hop. + ## Document sync frames `__DATA_V1__` frames replicate documents. They are specified in their own diff --git a/docs/spec/custody.md b/docs/spec/custody.md new file mode 100644 index 000000000..d676dcd75 --- /dev/null +++ b/docs/spec/custody.md @@ -0,0 +1,502 @@ +# Custody + +## What this chapter is for + +A frame this device did not originate is held for five seconds. That is the +whole of what the mesh does for a stranger's message today: a forwarder queues +it, tries to put it on a link, and when no neighbour can take it within +`RELAY_QUEUE_MAX_OVERDUE` the frame is abandoned and its identifier released. +The device's own messages get the opposite treatment: sealed on disk, retried +for seven days sliding and twenty-eight absolute. Custody is the contract that +lets a device close that gap for one class of traffic, by holding a frame it +could not forward for hours rather than seconds and delivering it when a path +appears. + +This chapter specifies the deposit, the receipt, the hold, redelivery, the +quotas and the erase. It does not change what any existing frame means, and it +adds one control frame and one reserved metadata key to the wire. Nothing in it +is a delivery guarantee; the [invariants](#invariants) say why, and they come +first because every mechanism below is a consequence of one of them. + +Custody is off by default and needs no counterpart to be useful: a device that +never enables it, and a device that never meets a custodian, both behave +exactly as they do today. + +## Invariants + +Seven things hold regardless of implementation, and a change that breaks one is +a protocol break rather than a refactor. + +- **Custody is replication, never transfer.** A custodian is an additional + holder of a frame. The depositor's outbox entry, acknowledgement tracking, + retry ladder and park state are untouched by a deposit and untouched by a + receipt. The failure this prevents is the one the sender-holds-custody + invariant exists for + ([state machines](../state-machines/README.md#the-one-invariant-that-spans-all-six)): + a sender that released a message to a carrier which then walked away would + have lost it silently. Under this chapter that case costs latency and nothing + else. +- **A receipt settles nothing.** Only the recipient settles a message, and it + does so with an acknowledgement, never with anything a custodian sends. A + receipt MUST NOT settle an outbox entry, cancel an acknowledgement timer, + remove a retry entry or un-park a message. The receipt is a separate control + frame with its own prefix rather than an acknowledgement with a flag, because + a receipt that is called an acknowledgement is eventually routed like one, + and the attribution gate that refuses acknowledgements from anyone but the + recipient exists precisely because a false confirmation is strictly worse + than a refusal. +- **The hold is strictly shorter than the outbox lifetime.** A custodian holds + ciphertext it cannot re-seal. The depositor survives a session re-key because + it retains the plaintext and re-seals on resend + ([ADR 0007](../adr/0007-reseal-on-resend.md)); a custodian has neither the + plaintext nor the keys, so its copy has a validity horizon it cannot observe. + A hold that outlives the outbox would keep bytes whose sender has already + reported the message failed, and deliver a frame the recipient may no longer + be able to open, to settle an entry that no longer exists. The bound is + validated at configuration, not documented as advice, because the outbox + lifetime is itself a dial. It is validated on the custodian, against the + custodian's own outbox dial, and it holds network-wide only where depositor + and custodian share a configuration, which this chapter cannot enforce. + Where they do not, the consequence is bounded by the class carried: a Class + A copy delivered after the depositor's entry expired is absorbed by the + recipient, and its acknowledgement names nothing in the depositor's outbox + and settles nothing. A wasted write, never a false settlement. +- **A custodian never blanks the route it is holding.** Accepting a frame into + custody MUST NOT record its identifier in the forwarding suppression cache, + and MUST release it there if the forwarding path recorded it. The cache + retains an identifier for ten minutes against a five-second hold; a custodian + that both held a frame and suppressed its own forwarding of the depositor's + retransmissions of that frame would turn a helpful device into a black hole + on exactly the route now known to be slow. +- **Class A only, by the depositor's word.** A custodian cannot see inside a + sealed frame, so it cannot classify one. Custody is therefore an explicit + deposit in which the depositor asserts the class, and this version accepts + one class: the document replication frames `delta`, `snap`, `vv` and + `blob_gone`. Everything else is refused by name, and the + [class table](#the-classes) says why for each. +- **A deposit comes from the depositor itself, over the link that proved it.** + A custodian accepts a frame into custody only when the frame's `sender` is + the address the transport proved for the peer it arrived from, and every + device that forwards a third-party frame MUST strip the deposit request from + the copy it transmits, whether or not custody is enabled on it. Nothing else + keeps custody to one hop: the request is a metadata key, the control + signature covers the sender, identifier, recipient and content and never the + metadata, and an `__MLS_ENC__` frame carries no control signature at all, so + the key rides the wire unsigned and survives any forwarder that does not + remove it. Without both rules, one frame fanned out to three neighbours, each + of which fans out to three, is held for hours on every device in a zone, and + an honest two-hop network deposits a frame at a custodian that keys its + quotas on the forwarder while answering a sender it may hold no key package + for. With them, the depositor, the tier key, the capability lookup and the + receipt's recipient are one address, and it is one the transport proved. +- **Custody is opt-in and has its own erase.** A device holds other people's + traffic only because its operator said so, and stops holding it the moment + the operator asks. There is no global wipe in this protocol, so custody + cannot inherit one; the erase is its own verb and the data-layer wipe calls + it. + +## The classes + +Custody is defined per payload class, not per message. The classes are the +ones the design fixed, and only Class A is carried in this version. + +| Class | Frames | This version | Why | +|-------|--------|--------------|-----| +| **A. Replication state** | `delta`, `snap`, `vv`, `blob_gone`, sealed to a pairwise session | **Carried** | Idempotent, kind by kind. A duplicate `delta` or `snap` is short-circuited as already applied before the CRDT engine is touched, which is a safety verdict on a compacted replica rather than an optimisation. A redelivered `vv` is an offer, answered from the recipient's live state like any offer, so it costs one exchange that converges on nothing the recipient did not already hold. A redelivered `blob_gone` is a floor report the recipient has already applied. All four are unordered, terminal in every outcome, at most 32 KiB of blob per frame, and re-derivable from state if a carried copy dies with an epoch | +| **B. Replication requests and their answers** | `need_blob`, `need_snap`, `chunk` | **Refused** | Not idempotent in cost. Each acted-on copy of a request spends a whole media transfer or a snapshot export, and both endpoints bound those exchanges on the assumption that a request arrives once. `chunk` is the answer to a group blob request, up to 32 pieces of 32 KiB, and a carried copy repeats an answer the requester already had | +| **C. Direct messages** | Any other `__MLS_ENC__` body | Refused | Subject to epoch death for the whole hold, with no way for the custodian to notice. A carried copy that arrives after a re-key raises the recipient's re-key security signal with benign traffic (see [R19](../security/threat-model.md#r19-custody-borne-re-key-pressure)) | +| **D. Media chunks** | `FileChunk` frames | Refused | Media is never persisted, by policy; a custodian that persisted a chunk would break it, and the chunk-zero purpose mark has a refusal path a carrier must not be able to steer | +| **E. Control frames** | Signed frames under the control-plane gate | Refused | Deferred rather than refused on principle: the thirty-day freshness window already bounds any hold, but each kind has its own consequences on late arrival, and none has been examined for a delivery hours late | +| **F. Acknowledgements** | Frames carrying `ack_for` | Refused | The strongest candidate for the next version, because the attribution gate needs no change to accept a relayed acknowledgement, but it widens who can observe delivery timing and is not in this one | + +Group frames are Class A by content and are still refused: a frame sealed once +under a group key is carried by the group message path, never offered to the +mesh as the depositor's own direct frame, and a group epoch changes on every +membership commit. The depositor's engine never writes a deposit request on one. + +## Reserved names + +This chapter reserves the following. Each exists once, and the registry that +publishes it is named. + +| Name | Kind | Where it is registered | +|------|------|------------------------| +| `__CUSTODY_RECEIPT__` | Control-frame prefix, signature-gated | [Control messages](control-messages.md#custody) | +| `__custody` | Hop-local metadata key, engine-written, stripped by every forwarder | [Reserved metadata keys](wire-format.md#reserved-metadata-keys) | +| `data_versions` entry 7 | Capability | [Capability negotiation](capability-negotiation.md) | + +## The deposit + +A deposit is not a frame. It is the depositor's own frame, offered to its +neighbours exactly as it is offered today, with one reserved metadata key that +says what the frame carries. + +### What the depositor writes + +The depositor MUST write the key `__custody` with the value `data`, and MUST +write it only when all of the following hold: + +1. The frame is the depositor's own, sealed to a pairwise session (its outer + prefix is `__MLS_ENC__`), and the depositor is handing it to neighbours + because it cannot reach the recipient itself. +2. The sealed body is a Class A replication frame. The depositor knows this + from the plaintext it retains for re-sealing on resend; a frame whose + plaintext it no longer holds, for instance after a restart, is offered + without the key. +3. The frame is being offered over a mesh transport to a neighbour whose + address the transport has proved. The key is never written on a frame sent + to the relay, over the internet, or to a gateway, each of which has its own + store-and-forward. + +The depositor is always the frame's `sender`. A deposit is made only by the +device that originated the frame, and only directly: a device never writes the +key on a frame it is forwarding for somebody else, and a custodian never +deposits a held frame onward. + +The value is a class token. `data` names Class A and is the only token defined. +A custodian MUST refuse a deposit whose token it does not know, and MUST NOT +infer a class from anything but the token. + +### What a forwarder does + +A device that transmits a third-party frame MUST remove `__custody` from the +copy it transmits, at the point where it adjusts the frame's hop fields, whether +or not custody is enabled on it and whether or not it understands the key. The +key is not signed and is not inside the sealed body, so nothing else stops it +travelling on. The failure this rule prevents is stated under the invariants: +without it a two-hop network deposits a frame at a custodian that never saw +its sender, and a zone holds every frame everywhere. + +An application MUST NOT write `__custody`; it is engine-written, like the two +signature keys. Nothing authenticates the token beyond the transport identity +of the peer that handed the frame over, and nothing needs to: a false assertion +costs the depositor's own recipient exactly what a direct resend of the same +frame would have cost, and a depositor can already send that directly. + +### What a custodian does on arrival + +A frame carrying a deposit request is, first, an ordinary forward. The +custodian queues it, tries to put it on a link and hands it onward if it can, +under every gate and budget that governs forwarding today. Custody begins only +where forwarding ends. + +**A custodian holds only what it could not forward.** Acceptance happens at the +point where a forward has waited past `RELAY_QUEUE_MAX_OVERDUE` without reaching +a link and would be abandoned. At that point the forwarding path has already +released the frame's identifier from the suppression cache, which is what +makes the fourth invariant a consequence of existing code rather than a new +mechanism. A frame that transmitted is never held: holding every copy of every +fan-out would multiply storage by the fan-out at each hop, for frames another +device is already carrying. + +A queued forward that the governor marked as a held frame being redelivered +([Redelivery](#redelivery)) is returned to the store at this point, and is not +judged by the table below: it is not a deposit, its refusal would not be a +refusal, and it is not counted as one. The drop point MUST NOT touch the +suppression cache for a marked forward: the held intake never recorded the +identifier there, so releasing it is at best a no-op and at worst releases an +entry the depositor's own retransmission legitimately holds if the ordinary +intake admitted one in the meantime. + +At that point the custodian MUST accept the frame into custody when every one +of these holds, and MUST refuse otherwise: + +| Condition | Refusal reason when it fails | +|-----------|------------------------------| +| Custody is enabled | `disabled` | +| The frame carries `__custody` | `no_request` | +| The token is `data` | `unknown_class` | +| The outer prefix is `__MLS_ENC__` | `not_sealed` | +| The frame arrived from a peer whose address the transport proved | `unproven_peer` | +| The frame's `sender` is the address the transport proved for that peer | `not_depositor` | +| The frame is not addressed to this device | never fails: a frame for this device is delivered, not held | +| No frame with this identifier is already held | `duplicate` | +| The depositor's tier admits it: a peer with an established session under the session tier, any other proven peer under the stranger tier | `stranger_refused` | +| The depositor's entry and byte budgets have room, or the overflow policy makes room | `depositor_full` | +| The global entry and byte budgets have room, or the overflow policy makes room | `store_full` | +| The battery is above the soft relay floor | `battery` | + +The `unproven_peer` and `not_depositor` rows together make the depositor one +address: the peer that handed the frame over, the frame's `sender`, the key the +tiers are looked up under, the peer whose capabilities gate the receipt, and +the receipt's recipient. A frame whose `sender` is not the arrival peer reached +this device through a forwarder, which means the forwarder did not strip the +key; the custodian refuses it and MUST strip the key itself if it transmits the +frame onward. + +A refusal is silent. The depositor learns only that no receipt came, which it +cannot distinguish from a lost receipt or an absent custodian. A detailed +refusal would be an oracle for probing a device's quotas and its session list, +so refusals are counted by reason for the operator and never answered on the +wire. + +A duplicate deposit, one for an identifier already held, is not stored, not +answered and counted. The first receipt already suppressed re-deposit at a +depositor that received it; a depositor that did not will deposit again on its +next retry, and absorbing that here costs nothing. + +### What is held + +An accepted frame becomes one sealed record under the custody storage category, +holding: + +- the frame exactly as it arrived with the `__custody` key removed, hop fields + as the forwarding path adjusted them; +- the depositor's address, which is both the frame's `sender` and the peer the + frame arrived from, so redelivery never hands it back; +- the wall-clock time of acceptance, in milliseconds since the Unix epoch, + never a monotonic instant: a monotonic clock does not run while the device is + off, and a custodian powered off for a week would otherwise wake believing + its hold had just begun; +- the class token. + +The record is sealed for the same reason the pending-decrypt queue is sealed: +it is other people's ciphertext plus routing metadata about them, and it +outlives the process. Sealing authenticates presence, not absence, so a +custodian cannot detect that held records were deleted from under it; that is +the walked-away case, which costs latency and nothing else. + +## The receipt + +A custodian that accepted a frame answers with a receipt, and this is the only +frame custody adds to the wire. + +### Shape + +The receipt is a control frame with the reserved prefix `__CUSTODY_RECEIPT__` +followed by a JSON body: + +``` +__CUSTODY_RECEIPT__{"v":1,"id":"","hold_ms":} +``` + +| Field | Meaning | +|-------|---------| +| `v` | Body version, `1` | +| `id` | The identifier of the frame now held | +| `hold_ms` | How much longer the custodian will hold it, relative to the receipt's own timestamp. Relative rather than absolute so the depositor applies it to its own clock and skew cannot expire a valid receipt | + +The custodian's address is the frame's `sender`; the depositor's is its +`recipient`. Unknown fields MUST be ignored. A receiver MUST refuse a body it +cannot parse, or whose `v` it does not know, exactly as it refuses any +malformed control frame: silently, without an acknowledgement. + +### Authentication + +The receipt is signature-gated like every other control frame in the registry: +signed over the control-plane canonical payload (`offline-ctrl-v2`, or +`offline-ctrl-v1` where that is what the depositor negotiated), verified against +the key its sender's address derives from, and refused outside the freshness +window ([Control messages](control-messages.md#the-control-plane-signature-gate)). +It introduces no signing domain of its own. An unsigned receipt is refused by +the gate before this chapter is consulted. + +### Sending rules + +A custodian MUST send at most one receipt per accepted frame, at acceptance, +addressed to the depositor and transmitted once over the link the deposit +arrived on, to that peer. It is never placed in the custodian's outbox, never +offered to the mesh, and never sent through a relay or a gateway: it names a +neighbour that was on a link a moment ago, and a receipt that cannot be put on +that link is dropped and counted, because the depositor deposits again on its +next retry and the custodian absorbs the duplicate. The receipt MUST NOT +request an acknowledgement: it is advisory, and putting it on the retry ladder +toward a neighbour that may already have walked away spends a retry budget on +nothing. + +A custodian MUST send a receipt only to a depositor whose key package advertised +`data_versions` entry 7. A peer that does not reserve the prefix does not know +the frame as a control frame, so it treats the receipt as an application +message: with encryption on, it refuses it as inbound plaintext from a peer +that has proved it can encrypt, which its threat model classifies as a security +refusal and records as one; on a plaintext-only deployment, it hands the +receipt to its application as a text message. The gate exists so that a +custodian never provokes either. Acceptance of a deposit is not gated on the +entry: a depositor whose capabilities are unknown, which is the ordinary case +for a stranger, is judged by its tier and simply gets no receipt. + +### What the depositor does with it + +A depositor that receives a valid receipt for an identifier still in its outbox +MUST suppress further deposit requests for that identifier toward that +custodian until `hold_ms` elapses, and MAY surface the fact for observability. +It MUST NOT do anything else with it: the second invariant lists what a receipt +never touches. Suppression state is in memory; losing it at a restart costs one +duplicate deposit, which the custodian absorbs. + +A receipt whose `id` names nothing in the depositor's outbox is ignored and +counted. That is the ordinary shape of a receipt arriving after the recipient's +acknowledgement settled the message, and it is also the shape a forged receipt +for a never-sent identifier would take, which is why it is not an error. + +## Redelivery + +Held frames leave custody by delivery, by expiry or by erase. + +**On neighbour discovery.** When a neighbour is discovered on any mesh +transport, the custodian walks its held frames and queues each eligible one +toward that neighbour through a **dedicated intake** of the forwarding +governor, not the ordinary one. The ordinary intake would refuse or mangle a +held frame in three ways: it consults the suppression cache, which would refuse +a second re-origination toward another neighbour within the cache's ten-minute +window; it spends a hop, while a held frame is transmitted with the hop fields +it was stored with, because the custodian is the same forwarder it was when +the frame arrived; and it charges the per-peer rate of the frame's arrival +peer, hours after that peer sent anything. The held-frame intake: + +- MUST skip the suppression check and the hop accounting; +- MUST take the queue capacity, the send budget (the forward budget, never the + own-traffic reserve) and the per-neighbour rate toward the neighbour it + targets, and MUST obey `allow_relay` and the battery floors; +- MUST NOT target the depositor; +- MUST mark the queued forward by its held identifier, so that when it reaches + no link within the overdue window the drop point returns it to the store + rather than judging it as a deposit: not a refusal, not counted as one, no + new record and no new receipt; +- MUST record the single target neighbour on the marked forward, which is + transmitted to that neighbour only and never through target selection: a + fan-out at flush time would defeat at most once per neighbour. + +A frame whose recipient is the discovered neighbour is queued toward that +neighbour and released from the store on transmission: the custodian has no +way to know more than that the bytes left, and the recipient's +acknowledgement, if any, travels the ordinary ladder back to the depositor, +never to the custodian. Any other frame is re-originated toward the neighbour +at most once per distinct neighbour during its hold, with the set of +neighbours tried bounded the way tracked neighbours are bounded, and stays in +the store after transmission until its hold ends. + +**What the custodian transmits never carries `__custody`.** The key is removed +at acceptance, and the custodian is a forwarder like any other for the rule +that a forwarder strips it. + +**Expiry.** A record is expired when its acceptance time is older than the +`hold_ms` in force, judged in wall time at restore and at every sweep against +the configuration of that moment, the way a control frame's freshness is +judged against the window of the verifier that receives it. So a lowered +`hold_ms` applies to records already held, and a custodian that was powered off +for longer than its hold delivers nothing stale when it returns. An expired +record is dropped, counted under `expired`, and nothing is sent to anybody: the +depositor never lost anything, so there is nothing to report, and a +`message_failed` for a frame this device did not originate would be a lie to an +application that never sent it. + +**At the recipient.** A redelivered frame is an ordinary inbound forward. The +receiver's deduplication, its acknowledgement decision and its stale-epoch rule +apply unchanged, and nothing about custody changes any of them: + +- A copy that arrives after the original is deduplicated when it lands inside + the receiver's window (two thousand identifiers over twenty-four hours, + persisted). When it does not, a `delta` or `snap` is absorbed below the CRDT + engine as already applied, a `vv` is answered from live state as one more + offer, and a `blob_gone` is a floor report already applied. +- A copy sealed to an epoch the recipient has since left classifies as a + session desync **when the recipient has crypto recovery enabled, which is + the default**. The recipient then withholds the acknowledgement and unmarks + the identifier rather than confirming what it could not read, so the copy + cannot falsely settle the depositor's entry, and the depositor's own + re-sealing resend is the recovery. The same branch schedules a rate-limited + session re-key and emits the re-key security warning; what integrators + should read from that is stated under + [R19](../security/threat-model.md#r19-custody-borne-re-key-pressure). +- **With crypto recovery disabled, the same copy is a terminal decrypt + failure**: dropped and acknowledged, and that acknowledgement settles the + depositor's outbox entry for a frame the recipient never read. A direct + resend never reaches this path, because the depositor re-seals it to the + live epoch; a custody copy is exactly what does. For Class A the loss is + recoverable and the recovery is the exchange itself: the recipient's version + vector still lacks the change, so the next version offer between the two + re-offers it, and the depositor's document state, not its outbox, is the + source. This is a third reason Class A is the only class carried, and a + deployment that disables crypto recovery on any device SHOULD leave custody + off. + +## Quotas + +The scarce resource is durable writes, not bandwidth: every built-in storage +provider pays a device barrier per stored record and another per delete, +whatever the record's size, so a quota that metered bytes alone would let ten +thousand tiny deposits cost more than one large one. Entries and bytes are +metered together, per depositor and globally, in the shape the pending-decrypt +queue already has. + +| Dial | Reference default | Bound, and the failure the bound names | +|------|-------------------|----------------------------------------| +| `enabled` | `false` | The off switch. Refusing to hold other people's traffic is this dial's job, never a zeroed budget's | +| `hold_ms` | 6 hours | `0 < hold_ms < retry.outbox_max_lifetime_ms`, refused at configuration otherwise: a hold that outlives the outbox delivers frames whose sender has already reported them failed. Judged in wall time from each record's acceptance timestamp against the value in force at the sweep, so lowering it expires records already held | +| `max_entries_per_depositor`, `max_bytes_per_depositor` | 64 entries, 2 MiB | Session tier. Both must be positive when `enabled`, or the store is on and holds nothing, which is the silent dial the off switch exists to replace. The byte cap must admit at least one replication frame at its ceiling, or every deposit is refused as `depositor_full` | +| `max_entries`, `max_bytes` | 512 entries, 16 MiB | Global. Each must be at least its per-depositor sibling, or the per-depositor dial can never be reached | +| `stranger_max_entries`, `stranger_max_bytes` | 0, 0 | Stranger tier. Zero means deposits are accepted only from peers with an established session, which is the default and is stated rather than inferred; a deployment that wants strangers held raises both | +| `overflow_policy` | drop oldest | What happens when a budget is full: evict the oldest held frame to admit the new one, or refuse the new one. The per-depositor caps are what keep one depositor's flood from evicting everyone else's frames under drop-oldest | + +The defaults are shapes awaiting field data, exactly as the forwarding +governor's were before topology measurements tuned them. The bounds are +normative; the numbers are the reference implementation's. + +**Tiers are keyed on the proven arrival peer.** A depositor is under the +session tier when this device holds an established MLS session with the peer +that handed the frame over, and under the stranger tier otherwise. A session +costs a key package exchange, which is what makes the session tier resistant +to an attacker who can mint addresses for free. + +**Battery.** Below the soft relay floor a custodian accepts no new deposits. +Held frames are never dropped for battery: dropping them is the one local +optimisation that converts latency into loss, and redelivery simply pauses with +the forwarding gate until the floor is cleared. + +**Counters.** A custodian counts, at least: accepted, delivered, re-originated, +expired, duplicates, and each refusal reason in the acceptance table. Counters +are aggregate and never per depositor on any wire, for the oracle reason above. + +## Erase + +`erase_custody` deletes every held record and resets the counters. It MUST be +callable on its own, and the data-layer wipe MUST call it: a custody store that +survived a logout would hold other people's traffic past the point the user +asked for erasure. A device that starts with custody disabled MUST erase any +held records at restore rather than keep them, because nothing would ever +deliver them. + +At restore with custody enabled, a custodian walks its records under a budget, +drops every record whose acceptance time is older than the current `hold_ms`, +and keeps the rest exactly as stored. Restore never sends anything. + +## What a custodian learns + +A sealed payload is opaque at every hop, but the outer message is not: sender, +recipient, identifier, application id, priority, hop fields, timestamp and size +are readable by whoever holds the frame. Custody extends the retention of that +metadata, for traffic between two other parties, from five seconds in memory to +hours on disk. This is the design's primary privacy cost, it is not mitigable by +sealing because routing needs the recipient, and it is stated as +[R20](../security/threat-model.md#r20-a-custodian-retains-routing-metadata-about-third-parties) +so that an operator enabling custody to be a good citizen knows what they are +also volunteering to store. + +## Conformance + +The receipt body is the one encoding this chapter adds, and it has no frozen +vector yet: the chapter precedes the codec, and a vector computed from prose +alone pins nothing. The vector lands with the implementation, in the crate that +holds the codec, and [Conformance](conformance.md) lists this chapter among +those that are not yet surfaces until then. + +## What this chapter does not specify + +- **Custody transfer.** The sender never releases; the first invariant. +- **Any incentive scheme.** Credit needs a ledger and reputation needs gossip, + and a network whose defining condition is partition has neither. Quotas are + local state and are enough. +- **Ordering, sequencing or exactly-once.** The replication layer is + at-least-once and unordered by design, and custody supplies exactly that. +- **Classes B to F.** Named in the table so the refusal is by name, not + specified for carriage. +- **A custody role, promotion or election.** Every device can hold bytes and + several custodians are several copies, so custody is a flag and a set of + quotas, never a state machine. +- **Relay or gateway custody.** The relay has its own store-and-forward and its + own account model. A gateway is a device like any other here; if the gateway + contract gains a custody verb, that chapter defines it. +- **Custody-level retry against a data-layer refusal.** Every data-layer + outcome is terminal, and a held frame is delivered once per path, never + retried against the recipient's verdict. diff --git a/docs/spec/wire-format.md b/docs/spec/wire-format.md index 5af16610d..0a6c464ba 100644 --- a/docs/spec/wire-format.md +++ b/docs/spec/wire-format.md @@ -342,6 +342,7 @@ observed on the wire: | `ack_transport` | Transport the acknowledged message arrived on | | `transport_preference` | Requested transport for this message | | `original_content_type` | Pre-chunking content type of a file transfer | +| `__custody` | The class a depositor asserts for its own frame when it offers it into custody; engine-written, unsigned, and stripped by every device that forwards the frame, so it travels one hop. See [Custody](custody.md) | | `__ctrl_sig` | Base64 Ed25519 signature over the control-message canonical payload | | `__ctrl_pk` | Base64 Ed25519 public key of the signer, 32 raw bytes | diff --git a/docs/state-machines/data-replication.md b/docs/state-machines/data-replication.md index 18d086939..aa4a48bb5 100644 --- a/docs/state-machines/data-replication.md +++ b/docs/state-machines/data-replication.md @@ -160,6 +160,7 @@ Triggers, each of which names its cause in the logs: | Peer rediscovered on any transport | `peer_rediscovered` | That peer, and groups shared with them | | Start-up | `start` | 1:1 spaces only | | Group joined, or a member added | `group_joined`, `member_added` | That member | +| A neighbour appears while a custodian holds a frame for it | `custody_redelivery` ([Custody](../spec/custody.md)) | That frame, transmitted as stored. The custodian starts no exchange: it forwards, it does not offer. At the recipient a redelivered `vv` is answered from live state like any offer, a duplicate `delta` or `snap` is absorbed as already applied, and a `blob_gone` is a floor report | Offers to one peer are suppressed for 30 seconds after the last one. The window delays only the reconciliation sweep: a local change does not wait for @@ -198,6 +199,15 @@ The end of the ladder is reported rather than logged. `data_doc_unsyncable` means two replicas that will not converge while both keep accepting edits, and nothing else about that state looks like a problem. +[Custody](../spec/custody.md) makes the bottom rung reachable more often. A +held `delta` or `snap` arrives hours after it was sealed, which is more time +for the receiver to have compacted the history it is built on; a stale +`delta` still has `need_snap` above it, but a stale `snap` refused on trim +has no rung above it and the replicas stay apart until the next exchange +from live state. This is one of the two reasons the custody hold is hours +and not days, and it is why a custody-delivered frame is an ordinary +forward: the ladder that recovers from it is the one already here. + ## An attachment fetch The fetching side. Every path out of `Outstanding` reports, because a From 6e7f531827efffecde07a38d9bd4e27b4d0d7697 Mon Sep 17 00:00:00 2001 From: bahdotsh Date: Thu, 1 Oct 2026 00:10:39 +0530 Subject: [PATCH 2/7] feat(protocol): custody v1 for replicated-document frames A device can hold a neighbour's Class A replication frames for hours instead of the five seconds a forwarder gives them (docs/spec/custody.md), off by default. The depositor writes the one-hop request on its own sealed frame from the plaintext it retains for re-sealing; every forwarder strips it; a custodian judges a frame at the drop point after its id is released from the suppression cache, answers with the signed receipt once over the arrival link, redelivers through a dedicated governor intake, expires records in wall time against the hold in force, and keeps them sealed under the new custody storage category. A receipt settles nothing. The receipt body has frozen vectors. --- crates/offline-protocol/src/config.rs | 345 +++++ crates/offline-protocol/src/lib.rs | 6 +- .../src/protocol/custodian.rs | 609 +++++++++ .../offline-protocol/src/protocol/custody.rs | 1211 +++++++++++++++++ crates/offline-protocol/src/protocol/data.rs | 9 + .../src/protocol/data_sync.rs | 97 ++ .../src/protocol/mesh_relay.rs | 324 ++++- .../src/protocol/message_dispatch.rs | 20 +- crates/offline-protocol/src/protocol/mod.rs | 53 + .../offline-protocol/src/protocol/receive.rs | 98 +- crates/offline-protocol/src/protocol/send.rs | 29 +- .../offline-protocol/src/protocol/storage.rs | 23 +- .../src/protocol/tests/custody.rs | 852 ++++++++++++ .../src/protocol/tests/data_sync_group.rs | 7 +- .../src/protocol/tests/mod.rs | 5 +- crates/offline-protocol/src/protocol/types.rs | 52 + .../data/custody-receipt-v1.vectors.json | 71 + .../offline-protocol/tests/mesh_forwarding.rs | 146 +- docs/spec/README.md | 1 + docs/spec/conformance.md | 5 +- docs/spec/custody.md | 12 +- 21 files changed, 3918 insertions(+), 57 deletions(-) create mode 100644 crates/offline-protocol/src/protocol/custodian.rs create mode 100644 crates/offline-protocol/src/protocol/custody.rs create mode 100644 crates/offline-protocol/src/protocol/tests/custody.rs create mode 100644 crates/offline-protocol/tests/data/custody-receipt-v1.vectors.json diff --git a/crates/offline-protocol/src/config.rs b/crates/offline-protocol/src/config.rs index 8529a6147..a1be21217 100644 --- a/crates/offline-protocol/src/config.rs +++ b/crates/offline-protocol/src/config.rs @@ -690,6 +690,93 @@ impl std::fmt::Debug for DataConfig { } } +/// How long a custodian holds a deposited frame by default: six hours. +/// +/// Hours, not days, because a custodian holds ciphertext it cannot re-seal +/// (`docs/spec/custody.md`, invariant three): the depositor survives a +/// session re-key by re-sealing from retained plaintext, and the custodian's +/// copy has a validity horizon it cannot observe. Strictly under the seven-day +/// outbox lifetime by a wide margin, so the default configuration validates. +pub const DEFAULT_CUSTODY_HOLD_MS: u64 = 6 * 60 * 60 * 1000; + +/// Held frames one depositor may have in custody at once, by default. +pub const DEFAULT_CUSTODY_MAX_ENTRIES_PER_DEPOSITOR: usize = 64; + +/// Bytes one depositor may have in custody at once, by default: 2 MiB. +pub const DEFAULT_CUSTODY_MAX_BYTES_PER_DEPOSITOR: usize = 2 * 1024 * 1024; + +/// Held frames across every depositor, by default. +pub const DEFAULT_CUSTODY_MAX_ENTRIES: usize = 512; + +/// Bytes across every depositor, by default: 16 MiB. +pub const DEFAULT_CUSTODY_MAX_BYTES: usize = 16 * 1024 * 1024; + +/// The smallest per-depositor byte budget that admits one replication frame at +/// its ceiling: 64 KiB. +/// +/// A Class A frame carries at most 32 KiB of blob +/// (`data_sync::MAX_SYNC_BLOB_BYTES`), which base64 grows by a third, which +/// the MLS envelope then encodes again, so a sealed frame at the ceiling is +/// around 58 KiB of content with the outer message's addresses and metadata +/// on top. A byte cap below this admits no such frame and refuses every +/// deposit as `depositor_full`, which is the silent dial the `enabled` switch +/// exists to replace. +pub const CUSTODY_MIN_BYTES_PER_DEPOSITOR: usize = 64 * 1024; + +/// Custody: holding a neighbour's replication frame for hours instead of the +/// seconds a forwarder gives it (`docs/spec/custody.md`). +/// +/// Off by default, and every dial here is the custodian's. A device that +/// never enables it behaves exactly as before, and a device that enables it +/// takes on other people's ciphertext under the quotas below. The hold is +/// validated strictly shorter than `reliability.retry.outbox_max_lifetime_ms`, +/// because a custodian that outlived the depositor's outbox would deliver +/// frames whose sender already reported them failed. +#[derive(Debug, Clone, PartialEq, Serialize, Deserialize)] +pub struct CustodyConfig { + /// Whether this device accepts deposits. The off switch: refusing to hold + /// other people's traffic is this dial's job, never a zeroed budget's. + pub enabled: bool, + /// How long an accepted frame is held, in milliseconds, judged in wall + /// time from each record's acceptance timestamp against the value in + /// force at the sweep. Lowering it expires records already held. + pub hold_ms: u64, + /// Held frames one depositor with an established session may have at once. + pub max_entries_per_depositor: usize, + /// Bytes one depositor with an established session may have at once. + pub max_bytes_per_depositor: usize, + /// Held frames across every depositor. + pub max_entries: usize, + /// Bytes across every depositor. + pub max_bytes: usize, + /// Held frames one proven peer *without* an established session may have + /// at once. Zero, the default, means deposits are accepted only from peers + /// with a session, which costs a key package exchange and is what makes + /// the session tier resistant to an attacker who mints addresses for free. + pub stranger_max_entries: usize, + /// Bytes one proven peer without an established session may have at once. + pub stranger_max_bytes: usize, + /// What happens when a budget is full: evict the oldest held frame to + /// admit the new one, or refuse the new one. + pub overflow_policy: OverflowPolicy, +} + +impl Default for CustodyConfig { + fn default() -> Self { + Self { + enabled: false, + hold_ms: DEFAULT_CUSTODY_HOLD_MS, + max_entries_per_depositor: DEFAULT_CUSTODY_MAX_ENTRIES_PER_DEPOSITOR, + max_bytes_per_depositor: DEFAULT_CUSTODY_MAX_BYTES_PER_DEPOSITOR, + max_entries: DEFAULT_CUSTODY_MAX_ENTRIES, + max_bytes: DEFAULT_CUSTODY_MAX_BYTES, + stranger_max_entries: 0, + stranger_max_bytes: 0, + overflow_policy: OverflowPolicy::DropOldest, + } + } +} + /// Main configuration for the Offline Protocol. #[derive(Debug, Clone)] pub struct ProtocolConfig { @@ -773,6 +860,10 @@ pub struct ProtocolConfig { /// Replicated-document data layer configuration. pub data: DataConfig, + + /// Custody: holding a neighbour's replication frames for hours. Off by + /// default; see [`CustodyConfig`]. + pub custody: CustodyConfig, } impl ProtocolConfig { @@ -796,6 +887,7 @@ impl ProtocolConfig { group: GroupConfig::default(), security: SecurityConfig::default(), data: DataConfig::default(), + custody: CustodyConfig::default(), } } @@ -1134,6 +1226,109 @@ impl ProtocolConfig { )); } + self.validate_custody()?; + + Ok(()) + } + + /// The custody dials (`docs/spec/custody.md`, "Quotas"). + /// + /// A zero hold is refused whatever the switch says: it is not a shorter + /// hold, it is a store that expires everything at the first sweep. The + /// remaining bounds are checked only while custody is enabled. The + /// default hold is six hours, and a disabled section that could refuse a + /// configuration whose outbox lifetime is shorter than that would turn + /// every such configuration, which today holds nothing, into a startup + /// error for a feature it never switched on. + fn validate_custody(&self) -> crate::Result<()> { + let custody = &self.custody; + + if custody.hold_ms == 0 { + return Err(crate::Error::InvalidConfiguration( + "custody.hold_ms must be greater than 0".to_string(), + )); + } + + if !custody.enabled { + return Ok(()); + } + + // The inversion this refuses is silent: a custodian whose hold + // outlives the depositor's outbox keeps bytes whose sender has already + // reported the message failed, and delivers a frame to settle an entry + // that no longer exists. Nothing at runtime would notice, because the + // custodian cannot see the depositor's ladder. + let outbox_lifetime_ms = self.reliability.retry.outbox_max_lifetime_ms; + if custody.hold_ms >= outbox_lifetime_ms { + return Err(crate::Error::InvalidConfiguration(format!( + "custody.hold_ms ({}ms) must be strictly shorter than \ + retry.outbox_max_lifetime_ms ({}ms): a hold that outlives the outbox \ + delivers frames whose sender has already reported them failed", + custody.hold_ms, outbox_lifetime_ms, + ))); + } + + // The session tier is the one that admits anything by default, so a + // zero there is custody switched on and holding nothing: the silent + // dial `enabled` exists to replace. + if custody.max_entries_per_depositor == 0 || custody.max_bytes_per_depositor == 0 { + return Err(crate::Error::InvalidConfiguration( + "custody.max_entries_per_depositor and custody.max_bytes_per_depositor must \ + both be greater than 0 while custody is enabled: zero is not a smaller \ + quota, it is a store that refuses every deposit" + .to_string(), + )); + } + + if custody.max_bytes_per_depositor < CUSTODY_MIN_BYTES_PER_DEPOSITOR { + return Err(crate::Error::InvalidConfiguration(format!( + "custody.max_bytes_per_depositor ({}) must admit one replication frame at \ + its ceiling ({} bytes), or every deposit is refused as depositor_full", + custody.max_bytes_per_depositor, CUSTODY_MIN_BYTES_PER_DEPOSITOR, + ))); + } + + if custody.max_entries < custody.max_entries_per_depositor + || custody.max_bytes < custody.max_bytes_per_depositor + { + return Err(crate::Error::InvalidConfiguration( + "custody.max_entries and custody.max_bytes must each be at least their \ + per-depositor sibling, or the per-depositor dial can never be reached" + .to_string(), + )); + } + + // The stranger tier is off by both dials or on by both. One at zero + // with the other positive reads like a narrow allowance and is a + // refusal of every stranger, which is the default said in a way an + // operator would not recognise as the default. + let stranger_on = custody.stranger_max_entries > 0 || custody.stranger_max_bytes > 0; + if stranger_on && (custody.stranger_max_entries == 0 || custody.stranger_max_bytes == 0) { + return Err(crate::Error::InvalidConfiguration( + "custody.stranger_max_entries and custody.stranger_max_bytes must be both \ + zero (no stranger deposits) or both greater than 0" + .to_string(), + )); + } + + if stranger_on && custody.stranger_max_bytes < CUSTODY_MIN_BYTES_PER_DEPOSITOR { + return Err(crate::Error::InvalidConfiguration(format!( + "custody.stranger_max_bytes ({}) must admit one replication frame at its \ + ceiling ({} bytes), or every stranger deposit is refused as depositor_full", + custody.stranger_max_bytes, CUSTODY_MIN_BYTES_PER_DEPOSITOR, + ))); + } + + if custody.max_entries < custody.stranger_max_entries + || custody.max_bytes < custody.stranger_max_bytes + { + return Err(crate::Error::InvalidConfiguration( + "custody.max_entries and custody.max_bytes must each be at least the \ + stranger tier's dial, or that tier can never be reached" + .to_string(), + )); + } + Ok(()) } } @@ -1348,6 +1543,13 @@ impl ProtocolConfigBuilder { self } + /// Configures custody: whether this device holds a neighbour's replication + /// frames, and under what quotas. See [`CustodyConfig`]. + pub fn custody(mut self, config: CustodyConfig) -> Self { + self.config.custody = config; + self + } + /// Builds and validates the configuration. /// /// # Returns @@ -2121,4 +2323,147 @@ mod tests { } out } + + // ==================================================================== + // Custody + // ==================================================================== + + fn enabled_custody() -> CustodyConfig { + CustodyConfig { + enabled: true, + ..CustodyConfig::default() + } + } + + #[test] + fn custody_is_off_by_default_and_the_default_dials_validate_when_on() { + let config = ProtocolConfig::new("app", "user"); + assert!(!config.custody.enabled); + assert_eq!(config.custody.hold_ms, DEFAULT_CUSTODY_HOLD_MS); + assert_eq!(config.custody.stranger_max_entries, 0); + assert_eq!(config.custody.stranger_max_bytes, 0); + config.validate().unwrap(); + + let mut on = ProtocolConfig::new("app", "user"); + on.custody = enabled_custody(); + on.validate().unwrap(); + // The reference defaults respect their own bounds. + assert!(DEFAULT_CUSTODY_HOLD_MS < on.reliability.retry.outbox_max_lifetime_ms); + assert!(DEFAULT_CUSTODY_MAX_BYTES_PER_DEPOSITOR >= CUSTODY_MIN_BYTES_PER_DEPOSITOR); + } + + #[test] + fn test_config_validation_rejects_a_custody_hold_that_outlives_the_outbox() { + // The silent inversion: a custodian whose hold outlives the depositor's + // outbox delivers frames whose sender already reported them failed. + let mut config = ProtocolConfig::new("app", "user"); + config.custody = enabled_custody(); + config.custody.hold_ms = config.reliability.retry.outbox_max_lifetime_ms; + let err = config.validate().unwrap_err().to_string(); + assert!(err.contains("custody.hold_ms"), "{err}"); + assert!(err.contains("strictly shorter"), "{err}"); + + config.custody.hold_ms = config.reliability.retry.outbox_max_lifetime_ms - 1; + config.validate().unwrap(); + + // Disabled, the relation is not checked: a section that is off must + // not refuse a configuration for a feature it never switched on. + config.custody.enabled = false; + config.custody.hold_ms = config.reliability.retry.outbox_max_lifetime_ms * 2; + config.validate().unwrap(); + + // A zero hold is refused whatever the switch says. + config.custody.hold_ms = 0; + let err = config.validate().unwrap_err().to_string(); + assert!( + err.contains("custody.hold_ms must be greater than 0"), + "{err}" + ); + } + + #[test] + fn test_config_validation_rejects_custody_dials_that_silently_hold_nothing() { + let base = { + let mut config = ProtocolConfig::new("app", "user"); + config.custody = enabled_custody(); + config + }; + + let mut zero_entries = base.clone(); + zero_entries.custody.max_entries_per_depositor = 0; + assert!(zero_entries + .validate() + .unwrap_err() + .to_string() + .contains("max_entries_per_depositor")); + + let mut zero_bytes = base.clone(); + zero_bytes.custody.max_bytes_per_depositor = 0; + assert!(zero_bytes + .validate() + .unwrap_err() + .to_string() + .contains("max_bytes_per_depositor")); + + let mut too_small = base.clone(); + too_small.custody.max_bytes_per_depositor = CUSTODY_MIN_BYTES_PER_DEPOSITOR - 1; + let err = too_small.validate().unwrap_err().to_string(); + assert!(err.contains("one replication frame"), "{err}"); + + let mut inverted = base.clone(); + inverted.custody.max_entries = inverted.custody.max_entries_per_depositor - 1; + assert!(inverted + .validate() + .unwrap_err() + .to_string() + .contains("per-depositor sibling")); + + let mut inverted_bytes = base.clone(); + inverted_bytes.custody.max_bytes = inverted_bytes.custody.max_bytes_per_depositor - 1; + assert!(inverted_bytes + .validate() + .unwrap_err() + .to_string() + .contains("per-depositor sibling")); + + // Off, none of it is checked. + let mut off = zero_entries.clone(); + off.custody.enabled = false; + off.validate().unwrap(); + } + + #[test] + fn test_config_validation_holds_the_stranger_tier_to_both_dials() { + let mut config = ProtocolConfig::new("app", "user"); + config.custody = enabled_custody(); + + config.custody.stranger_max_entries = 4; + let err = config.validate().unwrap_err().to_string(); + assert!(err.contains("both zero"), "{err}"); + + config.custody.stranger_max_bytes = CUSTODY_MIN_BYTES_PER_DEPOSITOR - 1; + let err = config.validate().unwrap_err().to_string(); + assert!(err.contains("stranger_max_bytes"), "{err}"); + + config.custody.stranger_max_bytes = CUSTODY_MIN_BYTES_PER_DEPOSITOR; + config.validate().unwrap(); + + config.custody.stranger_max_entries = config.custody.max_entries + 1; + let err = config.validate().unwrap_err().to_string(); + assert!(err.contains("stranger tier"), "{err}"); + } + + #[test] + fn the_custody_builder_setter_reaches_validation() { + let err = ProtocolConfig::builder("app", "user") + .custody(CustodyConfig { + enabled: true, + hold_ms: u64::MAX, + ..CustodyConfig::default() + }) + .build() + .unwrap_err() + .to_string(); + assert!(err.contains("custody.hold_ms"), "{err}"); + } } diff --git a/crates/offline-protocol/src/lib.rs b/crates/offline-protocol/src/lib.rs index 281f88858..19e5beb3b 100644 --- a/crates/offline-protocol/src/lib.rs +++ b/crates/offline-protocol/src/lib.rs @@ -30,8 +30,9 @@ pub mod transport_manager; pub mod visualization; pub use config::{ - DataConfig, EncryptionConfig, GroupConfig, OverflowPolicy, PendingQueueConfig, ProtocolConfig, - SecurityConfig, DEFAULT_PENDING_TTL_MS, + CustodyConfig, DataConfig, EncryptionConfig, GroupConfig, OverflowPolicy, PendingQueueConfig, + ProtocolConfig, SecurityConfig, CUSTODY_MIN_BYTES_PER_DEPOSITOR, DEFAULT_CUSTODY_HOLD_MS, + DEFAULT_PENDING_TTL_MS, }; pub use error::{Error, EstablishmentState, Result, SessionStateError}; pub use events::{ @@ -57,6 +58,7 @@ pub use group_mesh::{ #[cfg(feature = "data")] pub use offline_protocol_data::{DataValue, DOC_SIZE_WARN_BYTES, MAX_DOC_BYTES, MAX_VALUE_BYTES}; pub use offline_protocol_services::MeshServices; +pub use protocol::custody::{CustodyRefusal, CustodyStats}; pub use protocol::mesh_relay::{MeshRelayConfig, MeshRelayStats}; pub use protocol::{GatewayCarrier, MediaSendOptions, OfflineProtocol, SendMessageOptions}; pub use protocol_state_storage::{ diff --git a/crates/offline-protocol/src/protocol/custodian.rs b/crates/offline-protocol/src/protocol/custodian.rs new file mode 100644 index 000000000..75936399b --- /dev/null +++ b/crates/offline-protocol/src/protocol/custodian.rs @@ -0,0 +1,609 @@ +//! The engine's custody seams (`docs/spec/custody.md`). +//! +//! The store in `custody.rs` decides; this file is where the engine feeds it: +//! the drop point at the end of the forwarding queue, redelivery on neighbour +//! discovery, the receipt in both directions, the sweep, the durable records +//! and the erase. Each seam is small on purpose, so the invariants the chapter +//! states can be read off the call sites: nothing here touches the suppression +//! cache, settles an outbox entry, or requests an acknowledgement. + +use std::time::{Duration, Instant}; + +use chrono::Utc; +use offline_protocol_core::{Message, MessageId, MessagePriority}; +use offline_protocol_router::RelayPriority; +use tracing::{debug, info, warn}; + +use super::custody::{ + decode_receipt, encode_receipt, CustodyCandidate, CustodyRefusal, CustodyStats, HeldFrame, + MAX_CUSTODY_RECEIPTS_PER_MESSAGE, +}; +use super::mesh_relay::{PendingRelay, RelayRejection}; +use super::prefixes::internal_prefixes; +use super::storage::PruneAllowance; +use super::types::{storage_keys, CustodyRecord, CUSTODY_RECORD_VERSION}; +use super::OfflineProtocol; +use crate::config::CustodyConfig; +use crate::{Error, ProtocolStateError, Result}; + +/// How often held records are swept for expiry, and receipt suppressions for +/// their end. Holds are hours, so a minute between sweeps costs nothing +/// observable and keeps the tick free of a walk over the store. +pub(crate) const CUSTODY_SWEEP_INTERVAL: Duration = Duration::from_secs(60); + +impl OfflineProtocol { + /// What this device has done as a custodian and as a depositor, and what + /// it is holding right now. See [`CustodyStats`]. + pub fn custody_stats(&self) -> CustodyStats { + self.custody.stats() + } + + /// The custody configuration in force. + pub fn custody_config(&self) -> &CustodyConfig { + self.custody.config() + } + + /// Deletes every held frame and resets the custody counters. + /// + /// Callable on its own, and called by the data-layer wipe: a custody + /// store that survived a logout would hold other people's traffic past + /// the point the user asked for erasure. Attempts every record and + /// reports the first failure, like the data wipe: answering `Ok` for an + /// erase that left records behind is the worst shape this call can take. + pub fn erase_custody(&mut self) -> Result<()> { + let ids = self.custody.erase(); + self.custody_receipts.clear(); + let Some(storage) = self.protocol_state_storage.clone() else { + return Ok(()); + }; + + let mut first_error: Option = None; + let mut record_error = |err: ProtocolStateError| { + if first_error.is_none() { + first_error = Some(err.to_string()); + } + }; + + for id in ids { + if let Err(err) = storage.delete(storage_keys::CUSTODY, &id.as_str()) { + record_error(err); + } + } + // Records the store never held this launch go too: a tail a restore + // left in place for lack of room, or a record written by a build + // whose version this one dropped. + match storage.list_keys(storage_keys::CUSTODY) { + Ok(keys) => { + for key in keys { + if let Err(err) = storage.delete(storage_keys::CUSTODY, &key) { + record_error(err); + } + } + } + Err(ProtocolStateError::NotFound(_)) => {} + Err(err) => record_error(err), + } + + match first_error { + Some(detail) => Err(Error::Other(format!( + "failed to delete every custody record: {detail}" + ))), + None => Ok(()), + } + } + + /// The drop point: a forward that waited past the overdue cut-off without + /// reaching a link. + /// + /// An ordinary forward is judged by the acceptance table; its id was + /// released from the suppression cache when the governor abandoned it, + /// which is what makes "a custodian never blanks the route it is holding" + /// a consequence of existing code. A marked held forward is returned to + /// the store: not a deposit, not a refusal, not counted. + pub(super) fn judge_at_drop_point(&mut self, relay: PendingRelay) { + if relay.custody_target.is_some() { + self.return_held_to_store(relay); + return; + } + + let PendingRelay { + message, + arrival_peer, + custody_request, + .. + } = relay; + + let session_tier = arrival_peer + .as_deref() + .is_some_and(|peer| self.confirmed_sessions.contains(peer)); + // The battery snapshot locks and allocates across every transport; + // skipped while custody is off, where the table refuses first anyway. + let battery_ok = !self.custody.is_enabled() || self.battery_allows_relaying(); + let now_ms = Utc::now().timestamp_millis(); + + let message_id = message.id.clone(); + let verdict = self.custody.judge(CustodyCandidate { + message, + request: custody_request.as_deref(), + arrival_peer: arrival_peer.as_deref(), + session_tier, + battery_ok, + now_ms, + }); + + match verdict { + Ok(accepted) => { + for evicted in &accepted.evicted { + self.delete_custody_record(evicted); + } + if let Some(held) = self.custody.get(&accepted.id) { + self.persist_custody_record(held); + } + debug!( + message_id = %message_id, + depositor = %accepted.depositor, + class = %accepted.class, + evicted = accepted.evicted.len(), + "Took a frame into custody" + ); + self.send_custody_receipt(&accepted.depositor, &accepted.id); + } + Err(CustodyRefusal::Duplicate) => { + debug!(message_id = %message_id, "Deposit of a frame already held; absorbed"); + } + Err(refusal) => { + debug!( + message_id = %message_id, + reason = refusal.as_str(), + "Frame abandoned at the drop point and not taken into custody" + ); + } + } + } + + /// A marked held forward reached no link: back to the store, unjudged. + pub(super) fn return_held_to_store(&mut self, relay: PendingRelay) { + self.custody.returned(&relay.message.id); + } + + /// A marked held forward went out. To its recipient, the frame leaves + /// custody; to anyone else, it stays held until its hold ends. + pub(super) fn record_custody_transmission( + &mut self, + relay: &PendingRelay, + reached_recipient: bool, + ) { + let id = &relay.message.id; + if reached_recipient { + if self.custody.record_delivered(id).is_some() { + self.delete_custody_record(id); + } + } else { + self.custody.record_re_originated(id); + } + } + + /// Queues every eligible held frame toward a neighbour that just + /// appeared, through the governor's held-frame intake. + /// + /// Obeys `allow_relay` and the battery floors like any forward: a + /// custodian is a forwarder for the frames it holds. A refusal for rate + /// or room leaves the frame held and untried, so the next discovery of + /// the same neighbour tries again; the store itself bounds how many + /// distinct neighbours a frame is offered to during its hold. + pub(super) fn redeliver_custody_to(&mut self, peer: &str) { + if !self.custody.is_enabled() || self.custody.is_empty() { + return; + } + if !self.config.relay.allow_relay + || matches!(self.config.relay.relay_priority, RelayPriority::Never) + { + return; + } + if !self.battery_allows_relaying() { + return; + } + + let now = Instant::now(); + for id in self.custody.candidates_for(peer) { + let Some(message) = self.custody.get(&id).map(|held| held.message.clone()) else { + continue; + }; + match self.mesh_relay.admit_held(message, peer, now) { + Ok(()) => { + self.custody.mark_queued(&id, peer); + debug!( + message_id = %id, + neighbor = %peer, + cause = "custody_redelivery", + "Queued a held frame toward a neighbor" + ); + } + Err(RelayRejection::QueueFull) => break, + Err(_) => continue, + } + } + } + + /// Drops held records past the hold in force and receipt suppressions + /// past their end. Throttled to [`CUSTODY_SWEEP_INTERVAL`]. + pub(super) fn sweep_custody(&mut self) { + let now = Instant::now(); + if now.duration_since(self.custody_last_sweep) < CUSTODY_SWEEP_INTERVAL { + return; + } + self.custody_last_sweep = now; + self.sweep_custody_now(now, Utc::now().timestamp_millis()); + } + + /// [`Self::sweep_custody`] without the throttle, at the given clocks. + pub(super) fn sweep_custody_now(&mut self, now: Instant, now_ms: i64) { + self.custody_receipts.retain(|_, custodians| { + custodians.retain(|_, until| *until > now); + !custodians.is_empty() + }); + + if self.custody.is_empty() { + return; + } + let hold_ms = self.custody.config().hold_ms; + let expired = self.custody.expire(now_ms, hold_ms); + for id in &expired { + self.delete_custody_record(id); + } + if !expired.is_empty() { + debug!( + count = expired.len(), + "Expired held frames at the end of their hold" + ); + } + } + + /// Answers a deposit with the receipt, once, over the arrival link. + /// + /// Never through the outbox, the mesh offer, a relay or a gateway: it + /// names a neighbour that was on a link a moment ago, and a receipt that + /// cannot be put on that link is dropped and counted. Never requests an + /// acknowledgement. Sent only to a depositor whose key package advertised + /// the custody entry, because a peer without it does not know the prefix + /// as a control frame. + fn send_custody_receipt(&mut self, depositor: &str, held_id: &MessageId) { + if !self.peer_data_custody.contains(depositor) { + debug!( + depositor = %depositor, + "Receipt withheld: the depositor does not advertise the custody entry" + ); + self.custody.record_receipt_dropped(); + return; + } + if self.mls_manager.is_none() { + // An unsigned receipt is refused by the depositor's control gate + // before it is read, so there is nothing to send. + self.custody.record_receipt_dropped(); + return; + } + + let content = encode_receipt(&held_id.as_str(), self.custody.config().hold_ms); + let mut message = match self.create_message( + depositor, + content, + Some(MessagePriority::Low), + None, + ) { + Ok(message) => message, + Err(err) => { + warn!(depositor = %depositor, error = %err, "Could not build a custody receipt"); + self.custody.record_receipt_dropped(); + return; + } + }; + message.requires_ack = false; + if let Err(err) = self.sign_control_message(&mut message) { + warn!(depositor = %depositor, error = %err, "Could not sign a custody receipt"); + self.custody.record_receipt_dropped(); + return; + } + + // Ours: a copy circling back through the mesh is recognised rather + // than carried, exactly as an offered frame is. + self.deduplicator.mark_seen_local(message.id.clone()); + self.mesh_relay.mark_handled(&message.id.as_str()); + + match self.transport_manager.send_to_neighbor(depositor, &message) { + Ok(transport) => { + debug!( + depositor = %depositor, + held = %held_id, + transport = ?transport, + "Sent a custody receipt" + ); + self.custody.record_receipt_sent(); + } + Err(err) => { + debug!( + depositor = %depositor, + held = %held_id, + error = %err, + "Dropped a custody receipt: the arrival link is gone" + ); + self.custody.record_receipt_dropped(); + } + } + } + + /// A receipt from a custodian, past the control gate. + /// + /// Suppresses further deposit requests for that identifier toward that + /// custodian until the hold elapses, and nothing else: the outbox entry, + /// the acknowledgement timer, the retry entry and any park are untouched. + /// A receipt naming nothing in the outbox is ignored and counted; that is + /// the ordinary shape of one arriving after the recipient's + /// acknowledgement settled the message, and the shape a forged receipt + /// for a never-sent identifier would take. + pub(super) fn handle_custody_receipt(&mut self, custodian: &str, body: &str) { + let receipt = match decode_receipt(body) { + Ok(receipt) => receipt, + Err(err) => { + debug!(custodian = %custodian, error = ?err, "Refused a malformed custody receipt"); + self.custody.record_receipt_ignored(); + return; + } + }; + let Ok(id) = MessageId::from_str(&receipt.id) else { + self.custody.record_receipt_ignored(); + return; + }; + if !self.outbox.contains_key(&id) { + debug!(custodian = %custodian, message_id = %id, "Custody receipt names nothing in the outbox"); + self.custody.record_receipt_ignored(); + return; + } + + // A hold longer than this device's own outbox window buys nothing: + // the entry is gone before the suppression would end. Clamping here + // also keeps the instant arithmetic in range. + let hold_ms = receipt + .hold_ms + .min(self.config.reliability.retry.outbox_max_lifetime_ms); + let until = Instant::now() + .checked_add(Duration::from_millis(hold_ms)) + .unwrap_or_else(Instant::now); + + let custodians = self.custody_receipts.entry(id.clone()).or_default(); + if !custodians.contains_key(custodian) + && custodians.len() >= MAX_CUSTODY_RECEIPTS_PER_MESSAGE + { + // Bounded against a neighbourhood that mints addresses: the + // suppression that ends soonest is the one worth least. + if let Some(oldest) = custodians + .iter() + .min_by_key(|(_, until)| **until) + .map(|(peer, _)| peer.clone()) + { + custodians.remove(&oldest); + } + } + custodians.insert(custodian.to_string(), until); + debug!(custodian = %custodian, message_id = %id, hold_ms, "Recorded a custody receipt"); + self.custody.record_receipt_received(); + } + + /// The class token to write on this device's own frame when it is + /// offered to neighbours, or `None` when the frame is not a deposit. + /// + /// Judged from the plaintext retained for re-sealing on resend: a frame + /// whose plaintext this device no longer holds, after a restart, is + /// offered without the key, exactly as the chapter says. Only a sealed + /// 1:1 frame qualifies, so a group frame, a plaintext frame and a media + /// chunk never carry one. + pub(super) fn custody_class_for(&self, message: &Message) -> Option<&'static str> { + if !message.content.starts_with(internal_prefixes::ENCRYPTED) { + return None; + } + let plaintext = self + .outbox + .get(&message.id) + .and_then(|entry| entry.reseal.as_ref()) + .map(|reseal| reseal.content.as_str()) + .or_else(|| { + self.pending_reseal + .get(&message.id) + .map(|reseal| reseal.content.as_str()) + })?; + #[cfg(feature = "data")] + { + super::data_sync::custody_class_of_plaintext(plaintext) + } + #[cfg(not(feature = "data"))] + { + // Without the data layer there are no replication frames to + // deposit, so nothing this device sends is Class A. + let _ = plaintext; + None + } + } + + /// Whether a receipt from `custodian` still suppresses a deposit request + /// for `id` toward it. + pub(super) fn custody_suppressed_toward(&self, id: &MessageId, custodian: &str) -> bool { + self.custody_receipts + .get(id) + .and_then(|custodians| custodians.get(custodian)) + .is_some_and(|until| *until > Instant::now()) + } + + /// Writes one held frame under its own message id. Best-effort, like the + /// pending-decrypt records: a failed write is logged and the frame still + /// lives in the store, it just will not survive a restart. + fn persist_custody_record(&self, held: &HeldFrame) { + let Some(storage) = &self.protocol_state_storage else { + return; + }; + let record = CustodyRecord { + version: CUSTODY_RECORD_VERSION, + depositor: held.depositor.clone(), + message: held.message.clone(), + accepted_at_ms: held.accepted_at_ms, + class: held.class.clone(), + }; + let data = match serde_json::to_vec(&record) { + Ok(data) => data, + Err(err) => { + warn!(message_id = %held.message.id, error = %err, "Failed to serialize a custody record"); + return; + } + }; + if let Err(err) = self.write_state_record( + storage.as_ref(), + storage_keys::CUSTODY, + &held.message.id.as_str(), + &data, + ) { + warn!(message_id = %held.message.id, error = %err, "Failed to persist a custody record"); + } + } + + /// Removes one held frame's record. Logged rather than swallowed: a + /// delete that silently failed is a record the next launch restores and + /// redelivers, past the point the store had given it up. + fn delete_custody_record(&self, id: &MessageId) { + if let Some(storage) = &self.protocol_state_storage { + if let Err(err) = storage.delete(storage_keys::CUSTODY, &id.as_str()) { + warn!(message_id = %id, error = %err, "Failed to delete a custody record"); + } + } + } + + /// Restores held frames at launch, under a budget. + /// + /// With custody disabled, every record is erased rather than kept: + /// nothing would ever deliver it. With custody enabled, each record is + /// dropped when its acceptance time is older than the hold in force and + /// kept exactly as stored otherwise, oldest first so the budgets in force + /// admit the frames that have waited longest. Restore never sends + /// anything. Deletes draw on `allowance` and may be refused, leaving the + /// record for a later launch; a record the store has no room for is + /// deleted rather than left, because nothing would ever restore it. + pub(super) fn restore_custody(&mut self, allowance: &mut PruneAllowance) { + let Some(storage) = self.protocol_state_storage.clone() else { + return; + }; + let keys = match storage.list_keys(storage_keys::CUSTODY) { + Ok(keys) => keys, + Err(ProtocolStateError::NotFound(_)) => return, + Err(err) => { + warn!(error = %err, "Failed to list custody records"); + return; + } + }; + if keys.is_empty() { + return; + } + + let cap = self.config.custody.max_entries.saturating_mul(4).max(1); + let enabled = self.custody.is_enabled(); + let hold = i64::try_from(self.custody.config().hold_ms).unwrap_or(i64::MAX); + let now_ms = Utc::now().timestamp_millis(); + // A refusing budget on the inbound pool, which no advisory walk + // shares (see `PruneAllowance::refusing_private`). + let mut budget = allowance.refusing_private(); + let mut erased = 0usize; + let mut expired = 0usize; + let mut deferred = 0usize; + + let mut records: Vec = Vec::new(); + for key in keys.into_iter().take(cap) { + if !enabled { + if budget.claim() { + self.delete_custody_record_by_key(&key); + erased += 1; + } else { + deferred += 1; + } + continue; + } + let data = match self.read_state_record(storage.as_ref(), storage_keys::CUSTODY, &key) { + Ok(Some(data)) => data, + Ok(None) => continue, + Err(err) => { + debug!(key = %key, error = %err, "Custody record could not be read; leaving it"); + continue; + } + }; + let record = match serde_json::from_slice::(&data) { + Ok(record) if record.version == CUSTODY_RECORD_VERSION => record, + Ok(record) => { + warn!(key = %key, version = record.version, "Dropping a custody record of an unknown version"); + if budget.claim() { + self.delete_custody_record_by_key(&key); + } + continue; + } + Err(err) => { + warn!(key = %key, error = %err, "Dropping a corrupted custody record"); + if budget.claim() { + self.delete_custody_record_by_key(&key); + } + continue; + } + }; + if record.message.id.as_str() != key { + warn!(key = %key, message_id = %record.message.id, "Dropping a custody record filed under another id"); + if budget.claim() { + self.delete_custody_record_by_key(&key); + } + continue; + } + if now_ms.saturating_sub(record.accepted_at_ms) > hold { + if budget.claim() { + self.delete_custody_record_by_key(&key); + expired += 1; + } else { + deferred += 1; + } + continue; + } + records.push(record); + } + + records.sort_by(|left, right| { + left.accepted_at_ms + .cmp(&right.accepted_at_ms) + .then_with(|| left.message.id.as_str().cmp(&right.message.id.as_str())) + }); + + let mut restored = 0usize; + for record in records { + let key = record.message.id.as_str(); + if self.custody.admit_restored( + record.message, + record.depositor, + record.accepted_at_ms, + record.class, + ) { + restored += 1; + } else if budget.claim() { + self.delete_custody_record_by_key(&key); + } else { + deferred += 1; + } + } + + if enabled { + info!(restored, expired, deferred, "Restored custody records"); + } else if erased > 0 || deferred > 0 { + info!( + erased, + deferred, "Erased custody records left by a launch with custody enabled" + ); + } + } + + fn delete_custody_record_by_key(&self, key: &str) { + if let Some(storage) = &self.protocol_state_storage { + if let Err(err) = storage.delete(storage_keys::CUSTODY, key) { + warn!(key = %key, error = %err, "Failed to delete a custody record"); + } + } + } +} diff --git a/crates/offline-protocol/src/protocol/custody.rs b/crates/offline-protocol/src/protocol/custody.rs new file mode 100644 index 000000000..9b1140ca8 --- /dev/null +++ b/crates/offline-protocol/src/protocol/custody.rs @@ -0,0 +1,1211 @@ +//! Custody v1: holding a neighbour's replication frame for hours. +//! +//! The store, the acceptance table, the quotas and the receipt codec from +//! `docs/spec/custody.md`. The seams that feed it sit beside the forwarding +//! path: `receive.rs` judges a frame at the drop point and transmits +//! redeliveries, `send.rs` writes the deposit request on the depositor's own +//! frame, and `mod.rs` restores, sweeps and answers receipts. +//! +//! Nothing here touches the forwarding suppression cache, settles an outbox +//! entry, or sends anything. Those are the chapter's invariants, and keeping +//! this module free of the engine's other state is what lets a test pin them +//! against the store alone. + +use std::collections::{HashMap, HashSet, VecDeque}; + +use offline_protocol_core::{Message, MessageId}; +use serde::{Deserialize, Serialize}; + +use crate::config::{CustodyConfig, OverflowPolicy}; +use crate::protocol::prefixes::internal_prefixes; + +/// The reserved metadata key a depositor writes on its own frame +/// (`docs/spec/wire-format.md`, reserved metadata keys). Engine-written, +/// unsigned, and stripped by every device that forwards the frame. +pub const CUSTODY_META_KEY: &str = "__custody"; + +/// The one class token this version defines: Class A replication frames. +pub const CUSTODY_CLASS_DATA: &str = "data"; + +/// The receipt body version this build writes and reads. +pub const CUSTODY_RECEIPT_VERSION: u8 = 1; + +/// Neighbours a held frame is re-originated toward during one hold, at most. +/// +/// The chapter bounds the set of neighbours tried "the way tracked neighbours +/// are bounded"; this is that bound, sized well above any neighbourhood the +/// mesh targets so it is a ceiling against a pathological churn of addresses +/// rather than a dial. +pub const MAX_CUSTODY_NEIGHBOURS_PER_HOLD: usize = 64; + +/// Custodians one depositor remembers a receipt from, per outbox entry. +/// +/// A receipt suppresses re-deposit toward the custodian that sent it. The set +/// is keyed by outbox id, so it is bounded by the outbox; this bounds the other +/// axis against a neighbourhood that mints addresses. +pub const MAX_CUSTODY_RECEIPTS_PER_MESSAGE: usize = 64; + +/// Why a frame at the drop point was not taken into custody, in the order the +/// chapter's acceptance table applies them. +#[derive(Debug, Clone, Copy, PartialEq, Eq, Hash)] +pub enum CustodyRefusal { + /// Custody is off on this device. + Disabled, + /// The frame carries no deposit request. + NoRequest, + /// The request names a class this version does not define. + UnknownClass, + /// The outer prefix is not `__MLS_ENC__`. + NotSealed, + /// The carrier did not identify the link the frame arrived on. + UnprovenPeer, + /// The frame's sender is not the peer it arrived from: it came through a + /// forwarder that did not strip the request. + NotDepositor, + /// A frame with this identifier is already held: not stored, not + /// answered, counted under `duplicates` rather than as a refusal. + Duplicate, + /// The depositor is a stranger and the stranger tier is closed. + StrangerRefused, + /// The depositor's own budget is full and the policy does not make room. + DepositorFull, + /// The global budget is full and the policy does not make room. + StoreFull, + /// The battery is below the soft relay floor. + Battery, +} + +impl CustodyRefusal { + /// Every refusal reason, in table order, for the counters and their tests. + pub const ALL: &'static [Self] = &[ + Self::Disabled, + Self::NoRequest, + Self::UnknownClass, + Self::NotSealed, + Self::UnprovenPeer, + Self::NotDepositor, + Self::Duplicate, + Self::StrangerRefused, + Self::DepositorFull, + Self::StoreFull, + Self::Battery, + ]; + + /// The reason as the chapter names it. + pub fn as_str(self) -> &'static str { + match self { + Self::Disabled => "disabled", + Self::NoRequest => "no_request", + Self::UnknownClass => "unknown_class", + Self::NotSealed => "not_sealed", + Self::UnprovenPeer => "unproven_peer", + Self::NotDepositor => "not_depositor", + Self::Duplicate => "duplicate", + Self::StrangerRefused => "stranger_refused", + Self::DepositorFull => "depositor_full", + Self::StoreFull => "store_full", + Self::Battery => "battery", + } + } +} + +/// What a custodian has done, and is holding. +/// +/// A snapshot, safe to poll. `held` and `held_bytes` are gauges; everything +/// else is cumulative since start-up or the last erase. Aggregate and never per +/// depositor, because a per-depositor answer on any wire would be the quota +/// oracle the chapter refuses to be. +#[derive(Debug, Clone, Default, PartialEq, Eq)] +pub struct CustodyStats { + /// Frames in custody right now. + pub held: u64, + /// Bytes in custody right now. + pub held_bytes: u64, + /// Deposits accepted. + pub accepted: u64, + /// Held frames handed to their recipient directly, and released. + pub delivered: u64, + /// Held frames re-originated toward a neighbour that is not the recipient. + pub re_originated: u64, + /// Held frames dropped at the end of their hold. + pub expired: u64, + /// Deposits of an identifier already held: not stored, not answered. + pub duplicates: u64, + /// Held frames evicted to admit a newer deposit under drop-oldest. + pub evicted: u64, + /// Receipts put on the arrival link. + pub receipts_sent: u64, + /// Receipts not sent: the depositor does not parse them, the link was + /// gone, or this device cannot sign. + pub receipts_dropped: u64, + /// Receipts this device received for an entry still in its outbox. + pub receipts_received: u64, + /// Receipts this device received naming nothing in its outbox, or that it + /// could not parse. + pub receipts_ignored: u64, + /// Refused: custody is off. + pub refused_disabled: u64, + /// Refused: no deposit request on the frame. + pub refused_no_request: u64, + /// Refused: unknown class token. + pub refused_unknown_class: u64, + /// Refused: not an `__MLS_ENC__` frame. + pub refused_not_sealed: u64, + /// Refused: the arrival link was not identified. + pub refused_unproven_peer: u64, + /// Refused: the sender is not the arrival peer. + pub refused_not_depositor: u64, + /// Refused: a stranger while the stranger tier is closed. + pub refused_stranger: u64, + /// Refused: the depositor's budget is full. + pub refused_depositor_full: u64, + /// Refused: the global budget is full. + pub refused_store_full: u64, + /// Refused: the battery is below the relay floor. + pub refused_battery: u64, +} + +impl CustodyStats { + fn count(&mut self, refusal: CustodyRefusal) { + let slot = match refusal { + CustodyRefusal::Disabled => &mut self.refused_disabled, + CustodyRefusal::NoRequest => &mut self.refused_no_request, + CustodyRefusal::UnknownClass => &mut self.refused_unknown_class, + CustodyRefusal::NotSealed => &mut self.refused_not_sealed, + CustodyRefusal::UnprovenPeer => &mut self.refused_unproven_peer, + CustodyRefusal::NotDepositor => &mut self.refused_not_depositor, + CustodyRefusal::Duplicate => &mut self.duplicates, + CustodyRefusal::StrangerRefused => &mut self.refused_stranger, + CustodyRefusal::DepositorFull => &mut self.refused_depositor_full, + CustodyRefusal::StoreFull => &mut self.refused_store_full, + CustodyRefusal::Battery => &mut self.refused_battery, + }; + *slot = slot.saturating_add(1); + } + + /// The counter a refusal reason lands in, for tests that walk the table. + pub fn refusals(&self, refusal: CustodyRefusal) -> u64 { + match refusal { + CustodyRefusal::Disabled => self.refused_disabled, + CustodyRefusal::NoRequest => self.refused_no_request, + CustodyRefusal::UnknownClass => self.refused_unknown_class, + CustodyRefusal::NotSealed => self.refused_not_sealed, + CustodyRefusal::UnprovenPeer => self.refused_unproven_peer, + CustodyRefusal::NotDepositor => self.refused_not_depositor, + CustodyRefusal::Duplicate => self.duplicates, + CustodyRefusal::StrangerRefused => self.refused_stranger, + CustodyRefusal::DepositorFull => self.refused_depositor_full, + CustodyRefusal::StoreFull => self.refused_store_full, + CustodyRefusal::Battery => self.refused_battery, + } + } +} + +/// One frame in custody. +#[derive(Debug, Clone)] +pub(crate) struct HeldFrame { + /// The frame as it arrived, hop fields as the forwarding path adjusted + /// them, with the deposit request removed. + pub(crate) message: Message, + /// The frame's sender, which is also the peer it arrived from. + pub(crate) depositor: String, + /// Wall-clock acceptance time, milliseconds since the Unix epoch. Never a + /// monotonic instant: that clock does not run while the device is off. + pub(crate) accepted_at_ms: i64, + /// The class token the depositor asserted. + pub(crate) class: String, + /// The frame's footprint against the byte budgets. + bytes: usize, + /// Neighbours this frame has been re-originated toward during its hold. + tried: HashSet, + /// The neighbour a marked forward is queued toward right now, if any. + in_flight: Option, +} + +/// A frame at the drop point, with everything the acceptance table reads. +pub(crate) struct CustodyCandidate<'a> { + /// The forwarded copy: hop fields adjusted, deposit request removed. + pub(crate) message: Message, + /// The class token the frame carried on arrival, if any. + pub(crate) request: Option<&'a str>, + /// The peer the frame arrived from, when the carrier proved the link. + pub(crate) arrival_peer: Option<&'a str>, + /// Whether this device holds an established session with that peer. + pub(crate) session_tier: bool, + /// Whether the battery is above the soft relay floor. + pub(crate) battery_ok: bool, + /// Wall-clock now, milliseconds since the Unix epoch. + pub(crate) now_ms: i64, +} + +/// A deposit the store took on. +pub(crate) struct Accepted { + /// The held frame's identifier. + pub(crate) id: MessageId, + /// The depositor, which is the arrival peer and the frame's sender. + pub(crate) depositor: String, + /// The class token, for the durable record. + pub(crate) class: String, + /// Frames evicted under drop-oldest to admit this one; their records are + /// the caller's to delete. + pub(crate) evicted: Vec, +} + +/// The footprint a held frame is charged against the byte budgets. +/// +/// The same accounting the pending-decrypt queue uses: the variable fields +/// (content, binary content, metadata, media metadata) plus the addresses. The +/// fixed cost of a record is bounded by the entry caps, so it is not charged +/// here; the byte budget exists to bound payload bloat. +fn frame_footprint(message: &Message) -> usize { + let metadata_bytes: usize = message + .metadata + .iter() + .map(|(key, value)| key.len() + value.len()) + .sum(); + let media_bytes = message + .media_metadata + .as_ref() + .map(|media| { + media.mime_type.len() + + media.file_name.len() + + media + .thumbnail_base64 + .as_ref() + .map(String::len) + .unwrap_or(0) + }) + .unwrap_or(0); + message.content.len() + + message + .binary_content + .as_ref() + .map(|binary| binary.len()) + .unwrap_or(0) + + message.sender.as_str().len() + + message.recipient.as_str().len() + + message.app_id.as_str().len() + + metadata_bytes + + media_bytes +} + +/// Which quota a depositor is judged under. +#[derive(Debug, Clone, Copy, PartialEq, Eq)] +enum Tier { + Session, + Stranger, +} + +#[derive(Debug, Clone, Copy, Default)] +struct Usage { + entries: usize, + bytes: usize, +} + +/// The custody store: held frames, per-depositor and global budgets, and the +/// counters. +/// +/// Memory-only; the sealed records are written and deleted by the engine's +/// storage seam beside the pending-decrypt records, and restored through +/// [`Self::admit_restored`]. +pub(crate) struct CustodyStore { + config: CustodyConfig, + entries: HashMap, + /// Acceptance order, oldest first. Ids are removed lazily: an id here that + /// is no longer in `entries` is skipped. + order: VecDeque, + per_depositor: HashMap, + total: Usage, + stats: CustodyStats, +} + +impl CustodyStore { + pub(crate) fn new(config: CustodyConfig) -> Self { + Self { + config, + entries: HashMap::new(), + order: VecDeque::new(), + per_depositor: HashMap::new(), + total: Usage::default(), + stats: CustodyStats::default(), + } + } + + pub(crate) fn config(&self) -> &CustodyConfig { + &self.config + } + + pub(crate) fn is_enabled(&self) -> bool { + self.config.enabled + } + + pub(crate) fn is_empty(&self) -> bool { + self.entries.is_empty() + } + + #[cfg(test)] + pub(crate) fn len(&self) -> usize { + self.entries.len() + } + + #[cfg(test)] + pub(crate) fn contains(&self, id: &MessageId) -> bool { + self.entries.contains_key(id) + } + + pub(crate) fn get(&self, id: &MessageId) -> Option<&HeldFrame> { + self.entries.get(id) + } + + /// The counters, with the gauges filled from the store. + pub(crate) fn stats(&self) -> CustodyStats { + let mut stats = self.stats.clone(); + stats.held = self.entries.len() as u64; + stats.held_bytes = self.total.bytes as u64; + stats + } + + /// Judges a frame at the drop point and takes it into custody when every + /// row of the acceptance table holds. + /// + /// Counts the refusal or the acceptance. Touches nothing outside the + /// store: the caller persists the record, deletes the evicted ones and + /// answers with the receipt. + pub(crate) fn judge( + &mut self, + candidate: CustodyCandidate<'_>, + ) -> Result { + let tier = match self.check(&candidate) { + Ok(tier) => tier, + Err(refusal) => { + // Every row is counted, `disabled` included, so an operator + // who expected deposits and sees none can tell "off" from + // "nobody asked"; a duplicate lands in its own counter. + self.stats.count(refusal); + return Err(refusal); + } + }; + + let depositor = candidate + .arrival_peer + .expect("check proved the arrival peer") + .to_string(); + let bytes = frame_footprint(&candidate.message); + let (max_entries, max_bytes) = match tier { + Tier::Session => ( + self.config.max_entries_per_depositor, + self.config.max_bytes_per_depositor, + ), + Tier::Stranger => ( + self.config.stranger_max_entries, + self.config.stranger_max_bytes, + ), + }; + + let mut evicted = Vec::new(); + + // The depositor's own budget first: under drop-oldest it is the + // depositor's oldest frame that goes, which is what keeps one + // depositor's flood from evicting everyone else's frames. + if bytes > max_bytes { + self.stats.count(CustodyRefusal::DepositorFull); + return Err(CustodyRefusal::DepositorFull); + } + while !self.fits_depositor(&depositor, bytes, max_entries, max_bytes) { + match self.config.overflow_policy { + OverflowPolicy::DropOldest => match self.oldest_of(&depositor) { + Some(victim) => { + self.remove_inner(&victim); + self.stats.evicted = self.stats.evicted.saturating_add(1); + evicted.push(victim); + } + None => { + self.stats.count(CustodyRefusal::DepositorFull); + return Err(CustodyRefusal::DepositorFull); + } + }, + OverflowPolicy::DropNewest => { + self.stats.count(CustodyRefusal::DepositorFull); + return Err(CustodyRefusal::DepositorFull); + } + } + } + + if bytes > self.config.max_bytes { + self.stats.count(CustodyRefusal::StoreFull); + return Err(CustodyRefusal::StoreFull); + } + while !self.fits_store(bytes) { + match self.config.overflow_policy { + OverflowPolicy::DropOldest => match self.oldest() { + Some(victim) => { + self.remove_inner(&victim); + self.stats.evicted = self.stats.evicted.saturating_add(1); + evicted.push(victim); + } + None => { + self.stats.count(CustodyRefusal::StoreFull); + return Err(CustodyRefusal::StoreFull); + } + }, + OverflowPolicy::DropNewest => { + self.stats.count(CustodyRefusal::StoreFull); + return Err(CustodyRefusal::StoreFull); + } + } + } + + let id = candidate.message.id.clone(); + let class = candidate + .request + .expect("check proved the request") + .to_string(); + self.insert(HeldFrame { + message: candidate.message, + depositor: depositor.clone(), + accepted_at_ms: candidate.now_ms, + class: class.clone(), + bytes, + tried: HashSet::new(), + in_flight: None, + }); + self.stats.accepted = self.stats.accepted.saturating_add(1); + + Ok(Accepted { + id, + depositor, + class, + evicted, + }) + } + + /// The acceptance table, in the chapter's order. + fn check(&self, c: &CustodyCandidate<'_>) -> Result { + if !self.config.enabled { + return Err(CustodyRefusal::Disabled); + } + let Some(token) = c.request else { + return Err(CustodyRefusal::NoRequest); + }; + if token != CUSTODY_CLASS_DATA { + return Err(CustodyRefusal::UnknownClass); + } + if !c.message.content.starts_with(internal_prefixes::ENCRYPTED) { + return Err(CustodyRefusal::NotSealed); + } + let Some(peer) = c.arrival_peer else { + return Err(CustodyRefusal::UnprovenPeer); + }; + if c.message.sender.as_str() != peer { + return Err(CustodyRefusal::NotDepositor); + } + if self.entries.contains_key(&c.message.id) { + // Not a refusal in the table's sense: the first receipt already + // suppressed re-deposit at a depositor that received it, and one + // that did not deposits again on its next retry. Absorbed here. + return Err(CustodyRefusal::Duplicate); + } + let tier = if c.session_tier { + Tier::Session + } else { + Tier::Stranger + }; + if tier == Tier::Stranger + && (self.config.stranger_max_entries == 0 || self.config.stranger_max_bytes == 0) + { + return Err(CustodyRefusal::StrangerRefused); + } + if !c.battery_ok { + return Err(CustodyRefusal::Battery); + } + Ok(tier) + } + + fn fits_depositor( + &self, + depositor: &str, + bytes: usize, + max_entries: usize, + max_bytes: usize, + ) -> bool { + let usage = self + .per_depositor + .get(depositor) + .copied() + .unwrap_or_default(); + usage.entries < max_entries && usage.bytes.saturating_add(bytes) <= max_bytes + } + + fn fits_store(&self, bytes: usize) -> bool { + self.total.entries < self.config.max_entries + && self.total.bytes.saturating_add(bytes) <= self.config.max_bytes + } + + fn oldest(&self) -> Option { + self.order + .iter() + .find(|id| self.entries.contains_key(*id)) + .cloned() + } + + fn oldest_of(&self, depositor: &str) -> Option { + self.order + .iter() + .find(|id| { + self.entries + .get(*id) + .is_some_and(|held| held.depositor == depositor) + }) + .cloned() + } + + fn insert(&mut self, held: HeldFrame) { + let usage = self + .per_depositor + .entry(held.depositor.clone()) + .or_default(); + usage.entries += 1; + usage.bytes = usage.bytes.saturating_add(held.bytes); + self.total.entries += 1; + self.total.bytes = self.total.bytes.saturating_add(held.bytes); + self.order.push_back(held.message.id.clone()); + self.entries.insert(held.message.id.clone(), held); + } + + fn remove_inner(&mut self, id: &MessageId) -> Option { + let held = self.entries.remove(id)?; + if let Some(usage) = self.per_depositor.get_mut(&held.depositor) { + usage.entries = usage.entries.saturating_sub(1); + usage.bytes = usage.bytes.saturating_sub(held.bytes); + if usage.entries == 0 { + self.per_depositor.remove(&held.depositor); + } + } + self.total.entries = self.total.entries.saturating_sub(1); + self.total.bytes = self.total.bytes.saturating_sub(held.bytes); + self.order.retain(|queued| queued != id); + Some(held) + } + + /// Puts a record read back from storage into the store, or refuses it. + /// + /// Refused when the identifier is already held or the budgets in force + /// have no room: restore keeps records "exactly as stored" but under the + /// configuration of the launch, so a lowered cap drops the excess rather + /// than exceeding the cap the operator set. Never evicts, never counts an + /// acceptance: nothing was deposited at restore. + pub(crate) fn admit_restored( + &mut self, + message: Message, + depositor: String, + accepted_at_ms: i64, + class: String, + ) -> bool { + if self.entries.contains_key(&message.id) { + return false; + } + let bytes = frame_footprint(&message); + let (max_entries, max_bytes) = ( + self.config + .max_entries_per_depositor + .max(self.config.stranger_max_entries), + self.config + .max_bytes_per_depositor + .max(self.config.stranger_max_bytes), + ); + if !self.fits_depositor(&depositor, bytes, max_entries, max_bytes) + || !self.fits_store(bytes) + { + return false; + } + self.insert(HeldFrame { + message, + depositor, + accepted_at_ms, + class, + bytes, + tried: HashSet::new(), + in_flight: None, + }); + true + } + + /// Drops every record whose acceptance time is older than `hold_ms` at + /// `now_ms`, returning their identifiers so the caller deletes the + /// records. Judged against the hold in force now, not the one at + /// acceptance, so lowering the dial expires records already held. + /// + /// A record dated in the future is kept: a clock that moved back is not a + /// reason to drop other people's traffic, and it ages out once the clock + /// passes its acceptance time again. + pub(crate) fn expire(&mut self, now_ms: i64, hold_ms: u64) -> Vec { + let hold = i64::try_from(hold_ms).unwrap_or(i64::MAX); + let expired: Vec = self + .entries + .values() + .filter(|held| now_ms.saturating_sub(held.accepted_at_ms) > hold) + .map(|held| held.message.id.clone()) + .collect(); + for id in &expired { + self.remove_inner(id); + self.stats.expired = self.stats.expired.saturating_add(1); + } + expired + } + + /// Held frames eligible for re-origination toward `peer`, oldest first. + /// + /// Never the depositor's own frames back to it; never a frame already + /// queued toward someone; at most once per distinct neighbour during the + /// hold, except toward the frame's own recipient, which is the delivery + /// attempt itself and is retried whenever that neighbour appears. + pub(crate) fn candidates_for(&self, peer: &str) -> Vec { + self.order + .iter() + .filter_map(|id| self.entries.get(id)) + .filter(|held| held.depositor != peer && held.in_flight.is_none()) + .filter(|held| { + held.message.recipient.as_str() == peer + || (!held.tried.contains(peer) + && held.tried.len() < MAX_CUSTODY_NEIGHBOURS_PER_HOLD) + }) + .map(|held| held.message.id.clone()) + .collect() + } + + /// Records that a marked forward of `id` is queued toward `peer`. + pub(crate) fn mark_queued(&mut self, id: &MessageId, peer: &str) { + if let Some(held) = self.entries.get_mut(id) { + held.tried.insert(peer.to_string()); + held.in_flight = Some(peer.to_string()); + } + } + + /// A marked forward reached no link and came back from the drop point. + /// Not a deposit, not a refusal, not counted. + pub(crate) fn returned(&mut self, id: &MessageId) { + if let Some(held) = self.entries.get_mut(id) { + held.in_flight = None; + } + } + + /// A marked forward was transmitted to a neighbour that is not the + /// recipient. The frame stays held until its hold ends. + pub(crate) fn record_re_originated(&mut self, id: &MessageId) { + if let Some(held) = self.entries.get_mut(id) { + held.in_flight = None; + } + self.stats.re_originated = self.stats.re_originated.saturating_add(1); + } + + /// A marked forward was handed to its recipient: the frame leaves custody. + /// Returns the record so the caller deletes it. + pub(crate) fn record_delivered(&mut self, id: &MessageId) -> Option { + let held = self.remove_inner(id)?; + self.stats.delivered = self.stats.delivered.saturating_add(1); + Some(held) + } + + pub(crate) fn record_receipt_sent(&mut self) { + self.stats.receipts_sent = self.stats.receipts_sent.saturating_add(1); + } + + pub(crate) fn record_receipt_dropped(&mut self) { + self.stats.receipts_dropped = self.stats.receipts_dropped.saturating_add(1); + } + + pub(crate) fn record_receipt_received(&mut self) { + self.stats.receipts_received = self.stats.receipts_received.saturating_add(1); + } + + pub(crate) fn record_receipt_ignored(&mut self) { + self.stats.receipts_ignored = self.stats.receipts_ignored.saturating_add(1); + } + + /// Drops every held frame and resets the counters, returning the + /// identifiers so the caller deletes the records. + pub(crate) fn erase(&mut self) -> Vec { + let ids: Vec = self.entries.keys().cloned().collect(); + self.entries.clear(); + self.order.clear(); + self.per_depositor.clear(); + self.total = Usage::default(); + self.stats = CustodyStats::default(); + ids + } +} + +/// The receipt body: `{"v":1,"id":"","hold_ms":}`. +/// +/// `hold_ms` is relative to the receipt's own timestamp, so the depositor +/// applies it to its own clock and skew cannot expire a valid receipt. +#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)] +pub(crate) struct CustodyReceipt { + pub(crate) v: u8, + pub(crate) id: String, + pub(crate) hold_ms: u64, +} + +/// Why a receipt body was refused. +#[derive(Debug, Clone, PartialEq, Eq)] +pub(crate) enum ReceiptDecodeError { + /// Not a JSON object, or a field of the wrong shape. + Malformed, + /// A body version this build does not know. + UnknownVersion, + /// An empty identifier names nothing. + EmptyId, +} + +/// Encodes a receipt for `id`, prefix included. +pub(crate) fn encode_receipt(id: &str, hold_ms: u64) -> String { + let body = CustodyReceipt { + v: CUSTODY_RECEIPT_VERSION, + id: id.to_string(), + hold_ms, + }; + // Three scalar fields cannot fail to serialize. + let json = serde_json::to_string(&body).unwrap_or_default(); + format!("{}{}", internal_prefixes::CUSTODY_RECEIPT, json) +} + +/// Decodes a receipt body (the text after the prefix). +/// +/// The version is read before the body, so a future shape is refused on its +/// version rather than failing to parse; unknown fields are ignored. +pub(crate) fn decode_receipt(body: &str) -> Result { + let value: serde_json::Value = + serde_json::from_str(body).map_err(|_| ReceiptDecodeError::Malformed)?; + if !value.is_object() { + return Err(ReceiptDecodeError::Malformed); + } + match value.get("v").and_then(serde_json::Value::as_u64) { + Some(v) if v == u64::from(CUSTODY_RECEIPT_VERSION) => {} + Some(_) => return Err(ReceiptDecodeError::UnknownVersion), + None => return Err(ReceiptDecodeError::Malformed), + } + let receipt: CustodyReceipt = + serde_json::from_value(value).map_err(|_| ReceiptDecodeError::Malformed)?; + if receipt.id.is_empty() { + return Err(ReceiptDecodeError::EmptyId); + } + Ok(receipt) +} + +#[cfg(test)] +mod tests { + use super::*; + use offline_protocol_core::{AppId, UserId}; + + fn config() -> CustodyConfig { + CustodyConfig { + enabled: true, + ..CustodyConfig::default() + } + } + + /// A message id derived from a label: ids are UUIDs on the wire, and a + /// test reads better naming them. + fn mid(label: &str) -> MessageId { + let mut acc: u64 = 0xcbf2_9ce4_8422_2325; + for byte in label.bytes() { + acc ^= u64::from(byte); + acc = acc.wrapping_mul(0x0100_0000_01b3); + } + MessageId::from_str(&format!( + "00000000-0000-4000-8000-{:012x}", + acc & 0xffff_ffff_ffff + )) + .unwrap() + } + + fn sealed(from: &str, to: &str, id: &str) -> Message { + let mut message = Message::new( + UserId::new(from).unwrap(), + UserId::new(to).unwrap(), + AppId::new("app").unwrap(), + format!("{}opaque-{id}", internal_prefixes::ENCRYPTED), + ); + message.id = mid(id); + message + } + + fn candidate<'a>(message: Message, from: &'a str) -> CustodyCandidate<'a> { + CustodyCandidate { + message, + request: Some(CUSTODY_CLASS_DATA), + arrival_peer: Some(from), + session_tier: true, + battery_ok: true, + now_ms: 1_000, + } + } + + #[test] + fn a_deposit_from_the_depositor_over_its_own_link_is_accepted() { + let mut store = CustodyStore::new(config()); + let accepted = store + .judge(candidate(sealed("alice", "carol", "m1"), "alice")) + .expect("accepted"); + assert_eq!(accepted.depositor, "alice"); + assert_eq!(accepted.class, CUSTODY_CLASS_DATA); + assert!(store.contains(&mid("m1"))); + assert_eq!(store.stats().accepted, 1); + assert_eq!(store.stats().held, 1); + } + + #[test] + fn every_refusal_reason_is_reachable_and_counted() { + let refuse = |store: &mut CustodyStore, c: CustodyCandidate<'_>, expected| { + let before = store.stats().refusals(expected); + assert_eq!(store.judge(c).err(), Some(expected)); + assert_eq!(store.stats().refusals(expected), before + 1); + }; + + let mut off = CustodyStore::new(CustodyConfig::default()); + refuse( + &mut off, + candidate(sealed("alice", "carol", "m1"), "alice"), + CustodyRefusal::Disabled, + ); + + let mut store = CustodyStore::new(config()); + let mut no_request = candidate(sealed("alice", "carol", "m1"), "alice"); + no_request.request = None; + refuse(&mut store, no_request, CustodyRefusal::NoRequest); + + let mut unknown = candidate(sealed("alice", "carol", "m1"), "alice"); + unknown.request = Some("media"); + refuse(&mut store, unknown, CustodyRefusal::UnknownClass); + + let mut plain = sealed("alice", "carol", "m1"); + plain.content = "hello".to_string(); + refuse( + &mut store, + candidate(plain, "alice"), + CustodyRefusal::NotSealed, + ); + + let mut unproven = candidate(sealed("alice", "carol", "m1"), "alice"); + unproven.arrival_peer = None; + refuse(&mut store, unproven, CustodyRefusal::UnprovenPeer); + + refuse( + &mut store, + candidate(sealed("alice", "carol", "m1"), "bob"), + CustodyRefusal::NotDepositor, + ); + + let mut stranger = candidate(sealed("alice", "carol", "m1"), "alice"); + stranger.session_tier = false; + refuse(&mut store, stranger, CustodyRefusal::StrangerRefused); + + let mut flat = candidate(sealed("alice", "carol", "m1"), "alice"); + flat.battery_ok = false; + refuse(&mut store, flat, CustodyRefusal::Battery); + + // Depositor full, under drop-newest so nothing is evicted. + let mut tight = CustodyStore::new(CustodyConfig { + max_entries_per_depositor: 1, + overflow_policy: OverflowPolicy::DropNewest, + ..config() + }); + tight + .judge(candidate(sealed("alice", "carol", "m1"), "alice")) + .unwrap(); + refuse( + &mut tight, + candidate(sealed("alice", "carol", "m2"), "alice"), + CustodyRefusal::DepositorFull, + ); + + // Store full: two depositors, a global cap of one. + let mut global = CustodyStore::new(CustodyConfig { + max_entries: 1, + overflow_policy: OverflowPolicy::DropNewest, + ..config() + }); + global + .judge(candidate(sealed("alice", "carol", "m1"), "alice")) + .unwrap(); + refuse( + &mut global, + candidate(sealed("bob", "carol", "m2"), "bob"), + CustodyRefusal::StoreFull, + ); + } + + #[test] + fn a_duplicate_is_neither_stored_nor_counted_as_a_refusal() { + let mut store = CustodyStore::new(config()); + store + .judge(candidate(sealed("alice", "carol", "m1"), "alice")) + .unwrap(); + let verdict = store.judge(candidate(sealed("alice", "carol", "m1"), "alice")); + assert_eq!(verdict.err(), Some(CustodyRefusal::Duplicate)); + assert_eq!(store.len(), 1); + assert_eq!(store.stats().accepted, 1); + assert_eq!(store.stats().duplicates, 1); + for reason in CustodyRefusal::ALL + .iter() + .filter(|reason| **reason != CustodyRefusal::Duplicate) + { + assert_eq!(store.stats().refusals(*reason), 0, "{reason:?}"); + } + } + + #[test] + fn drop_oldest_evicts_the_depositors_own_oldest_frame_first() { + let mut store = CustodyStore::new(CustodyConfig { + max_entries_per_depositor: 1, + ..config() + }); + store + .judge(candidate(sealed("alice", "carol", "a1"), "alice")) + .unwrap(); + store + .judge(candidate(sealed("bob", "carol", "b1"), "bob")) + .unwrap(); + let accepted = store + .judge(candidate(sealed("alice", "carol", "a2"), "alice")) + .unwrap(); + assert_eq!(accepted.evicted, vec![mid("a1")]); + assert!(store.contains(&mid("b1"))); + assert!(store.contains(&mid("a2"))); + assert_eq!(store.stats().evicted, 1); + } + + #[test] + fn a_stranger_is_admitted_only_under_the_stranger_tier() { + let mut store = CustodyStore::new(CustodyConfig { + stranger_max_entries: 1, + stranger_max_bytes: 128 * 1024, + ..config() + }); + let mut stranger = candidate(sealed("mallory", "carol", "s1"), "mallory"); + stranger.session_tier = false; + store.judge(stranger).expect("one stranger frame"); + let mut second = candidate(sealed("mallory", "carol", "s2"), "mallory"); + second.session_tier = false; + // The tier is one entry deep, and drop-oldest keeps it that deep. + let accepted = store.judge(second).expect("evicts the first"); + assert_eq!(accepted.evicted.len(), 1); + } + + #[test] + fn expiry_is_judged_in_wall_time_against_the_hold_in_force() { + let mut store = CustodyStore::new(config()); + store + .judge(candidate(sealed("alice", "carol", "m1"), "alice")) + .unwrap(); + assert!(store.expire(1_000 + 5_000, 6_000).is_empty()); + // A lowered hold expires a record accepted under a longer one. + assert_eq!(store.expire(1_000 + 5_000, 4_000).len(), 1); + assert_eq!(store.stats().expired, 1); + assert!(store.is_empty()); + } + + #[test] + fn a_record_dated_in_the_future_is_kept() { + let mut store = CustodyStore::new(config()); + let mut future = candidate(sealed("alice", "carol", "m1"), "alice"); + future.now_ms = 10_000; + store.judge(future).unwrap(); + assert!(store.expire(1_000, 1).is_empty()); + } + + #[test] + fn redelivery_candidates_skip_the_depositor_and_neighbours_already_tried() { + let mut store = CustodyStore::new(config()); + store + .judge(candidate(sealed("alice", "carol", "m1"), "alice")) + .unwrap(); + let id = mid("m1"); + assert!( + store.candidates_for("alice").is_empty(), + "never back to the depositor" + ); + assert_eq!(store.candidates_for("bob"), vec![id.clone()]); + store.mark_queued(&id, "bob"); + assert!(store.candidates_for("bob").is_empty(), "in flight"); + store.record_re_originated(&id); + assert!(store.candidates_for("bob").is_empty(), "once per neighbour"); + assert_eq!(store.candidates_for("dave"), vec![id.clone()]); + // The recipient is always worth trying again. + store.mark_queued(&id, "carol"); + store.returned(&id); + assert_eq!(store.candidates_for("carol"), vec![id]); + } + + #[test] + fn delivery_releases_the_frame_and_re_origination_keeps_it() { + let mut store = CustodyStore::new(config()); + store + .judge(candidate(sealed("alice", "carol", "m1"), "alice")) + .unwrap(); + let id = mid("m1"); + store.mark_queued(&id, "bob"); + store.record_re_originated(&id); + assert!(store.contains(&id)); + assert_eq!(store.stats().re_originated, 1); + store.mark_queued(&id, "carol"); + assert!(store.record_delivered(&id).is_some()); + assert!(!store.contains(&id)); + assert_eq!(store.stats().delivered, 1); + assert_eq!(store.stats().held, 0); + } + + #[test] + fn erase_drops_everything_and_resets_the_counters() { + let mut store = CustodyStore::new(config()); + store + .judge(candidate(sealed("alice", "carol", "m1"), "alice")) + .unwrap(); + let ids = store.erase(); + assert_eq!(ids, vec![mid("m1")]); + assert!(store.is_empty()); + assert_eq!(store.stats(), CustodyStats::default()); + } + + #[test] + fn restore_refuses_a_duplicate_and_never_counts_an_acceptance() { + let mut store = CustodyStore::new(config()); + assert!(store.admit_restored( + sealed("alice", "carol", "m1"), + "alice".into(), + 5, + CUSTODY_CLASS_DATA.into() + )); + assert!(!store.admit_restored( + sealed("alice", "carol", "m1"), + "alice".into(), + 5, + CUSTODY_CLASS_DATA.into() + )); + assert_eq!(store.stats().accepted, 0); + assert_eq!(store.stats().held, 1); + } + + #[test] + fn the_receipt_round_trips_and_refuses_what_the_chapter_refuses() { + let wire = encode_receipt("m1", 21_600_000); + assert_eq!( + wire, + format!( + "{}{{\"v\":1,\"id\":\"m1\",\"hold_ms\":21600000}}", + internal_prefixes::CUSTODY_RECEIPT + ) + ); + let body = wire + .strip_prefix(internal_prefixes::CUSTODY_RECEIPT) + .unwrap(); + let receipt = decode_receipt(body).unwrap(); + assert_eq!(receipt.id, "m1"); + assert_eq!(receipt.hold_ms, 21_600_000); + + assert_eq!( + decode_receipt("{\"v\":2,\"id\":\"m1\",\"hold_ms\":1}").err(), + Some(ReceiptDecodeError::UnknownVersion) + ); + assert_eq!( + decode_receipt("{\"v\":1,\"id\":\"\",\"hold_ms\":1}").err(), + Some(ReceiptDecodeError::EmptyId) + ); + assert_eq!( + decode_receipt("{\"id\":\"m1\",\"hold_ms\":1}").err(), + Some(ReceiptDecodeError::Malformed) + ); + assert_eq!( + decode_receipt("not json").err(), + Some(ReceiptDecodeError::Malformed) + ); + // Unknown fields are ignored, as the chapter requires. + assert!(decode_receipt("{\"v\":1,\"id\":\"m1\",\"hold_ms\":1,\"x\":true}").is_ok()); + } +} + +/// The frozen wire vectors for the custody receipt. +/// +/// The conformance surface a second implementation is written against. When +/// one of these fails the wire format has changed: bump the body version and +/// negotiate a new `data_versions` entry. Do NOT edit the expected values to +/// make the test pass; every shipped install still speaks the old ones. +#[cfg(test)] +mod golden_vectors { + use super::*; + + const VECTORS: &str = include_str!("../../tests/data/custody-receipt-v1.vectors.json"); + + fn vectors() -> serde_json::Value { + serde_json::from_str(VECTORS).expect("the vector file is JSON") + } + + fn cases<'a>(vectors: &'a serde_json::Value, name: &str) -> &'a Vec { + vectors[name] + .as_array() + .unwrap_or_else(|| panic!("{name} must be an array")) + } + + #[test] + fn the_vector_file_is_the_size_it_was() { + let vectors = vectors(); + assert_eq!(cases(&vectors, "frames").len(), 3); + assert_eq!(cases(&vectors, "decode_only").len(), 2); + assert_eq!(cases(&vectors, "rejects").len(), 6); + assert_eq!( + vectors["version"].as_u64(), + Some(u64::from(CUSTODY_RECEIPT_VERSION)) + ); + assert_eq!( + vectors["prefix"].as_str(), + Some(internal_prefixes::CUSTODY_RECEIPT) + ); + } + + #[test] + fn every_frame_encodes_to_its_vector_and_decodes_back() { + let vectors = vectors(); + for case in cases(&vectors, "frames") { + let name = case["name"].as_str().unwrap(); + let id = case["id"].as_str().unwrap(); + let hold_ms = case["hold_ms"].as_u64().unwrap(); + let wire = case["wire"].as_str().unwrap(); + assert_eq!(encode_receipt(id, hold_ms), wire, "{name}: encode"); + let body = wire + .strip_prefix(internal_prefixes::CUSTODY_RECEIPT) + .unwrap_or_else(|| panic!("{name}: the vector carries the prefix")); + let decoded = decode_receipt(body).unwrap_or_else(|e| panic!("{name}: {e:?}")); + assert_eq!(decoded.id, id, "{name}: id"); + assert_eq!(decoded.hold_ms, hold_ms, "{name}: hold_ms"); + assert_eq!(decoded.v, CUSTODY_RECEIPT_VERSION, "{name}: version"); + } + } + + #[test] + fn every_decode_only_case_is_accepted() { + let vectors = vectors(); + for case in cases(&vectors, "decode_only") { + let name = case["name"].as_str().unwrap(); + let wire = case["wire"].as_str().unwrap(); + let body = wire + .strip_prefix(internal_prefixes::CUSTODY_RECEIPT) + .unwrap(); + let decoded = decode_receipt(body).unwrap_or_else(|e| panic!("{name}: {e:?}")); + assert_eq!(decoded.id, case["id"].as_str().unwrap(), "{name}"); + assert_eq!(decoded.hold_ms, case["hold_ms"].as_u64().unwrap(), "{name}"); + } + } + + #[test] + fn every_reject_case_is_refused() { + let vectors = vectors(); + for case in cases(&vectors, "rejects") { + let name = case["name"].as_str().unwrap(); + let wire = case["wire"].as_str().unwrap(); + let body = wire + .strip_prefix(internal_prefixes::CUSTODY_RECEIPT) + .unwrap(); + assert!(decode_receipt(body).is_err(), "{name} must be refused"); + } + } +} diff --git a/crates/offline-protocol/src/protocol/data.rs b/crates/offline-protocol/src/protocol/data.rs index 19aaa752d..70f5e95d1 100644 --- a/crates/offline-protocol/src/protocol/data.rs +++ b/crates/offline-protocol/src/protocol/data.rs @@ -2251,6 +2251,15 @@ impl OfflineProtocol { // logout path, so the records it failed to remove outlive the account // that made them, and the application has no symptom to notice it by. let mut first_error: Option = None; + + // Custody goes with the documents. A custody store that survived a + // logout would hold other people's traffic past the point the user + // asked for erasure, and there is no global wipe for it to inherit + // (`docs/spec/custody.md`, "Erase"), so this wipe calls its own. + if let Err(err) = self.erase_custody() { + first_error = Some(err.to_string()); + } + let mut record_error = |err: crate::protocol_state_storage::ProtocolStateError| { if first_error.is_none() { first_error = Some(err.to_string()); diff --git a/crates/offline-protocol/src/protocol/data_sync.rs b/crates/offline-protocol/src/protocol/data_sync.rs index dd6634e29..eb60efed4 100644 --- a/crates/offline-protocol/src/protocol/data_sync.rs +++ b/crates/offline-protocol/src/protocol/data_sync.rs @@ -3957,3 +3957,100 @@ mod golden_vectors { } } } + +impl SyncBody { + /// The custody class of this frame, or `None` for a kind custody never + /// carries (`docs/spec/custody.md`, "The classes"). + /// + /// Exhaustive on purpose: a new kind has to decide here whether a copy of + /// it is idempotent in cost, rather than inheriting an answer. + pub(crate) fn custody_class(&self) -> Option<&'static str> { + match self { + SyncBody::Versions { .. } + | SyncBody::Delta { .. } + | SyncBody::Snapshot { .. } + | SyncBody::BlobGone { .. } => Some(crate::protocol::custody::CUSTODY_CLASS_DATA), + // Not idempotent in cost: each acted-on copy of a request spends a + // whole media transfer or a snapshot export, and a carried chunk + // repeats an answer the requester already had. + SyncBody::NeedSnapshot { .. } | SyncBody::NeedBlob { .. } | SyncBody::Chunk { .. } => { + None + } + } + } +} + +/// The custody class of a `__DATA_V1__` plaintext the depositor retains for +/// re-sealing, or `None` when it is not a sync frame this build reads, or is a +/// kind custody refuses by name. +/// +/// Read the way an arriving frame is read: version before body, so a frame of +/// a future version is simply not deposited rather than misclassified. +pub(crate) fn custody_class_of_plaintext(plaintext: &str) -> Option<&'static str> { + let body = plaintext.strip_prefix(internal_prefixes::DATA_V1)?; + let value: serde_json::Value = serde_json::from_str(body).ok()?; + if value.get("v").and_then(serde_json::Value::as_u64) != Some(u64::from(DATA_SYNC_V1)) { + return None; + } + serde_json::from_value::(value) + .ok()? + .custody_class() +} + +#[cfg(test)] +mod custody_class_tests { + use super::*; + + fn framed(body: &str) -> String { + format!( + "{}{{\"v\":{},{}", + internal_prefixes::DATA_V1, + DATA_SYNC_V1, + &body[1..] + ) + } + + #[test] + fn class_a_kinds_are_deposited_and_the_rest_are_not() { + for (kind, expected) in [ + ( + "{\"k\":\"delta\",\"doc\":\"d\",\"blob\":\"AA==\"}", + Some("data"), + ), + ( + "{\"k\":\"snap\",\"doc\":\"d\",\"blob\":\"AA==\"}", + Some("data"), + ), + ("{\"k\":\"vv\",\"docs\":{}}", Some("data")), + ("{\"k\":\"blob_gone\",\"hash\":\"h\"}", Some("data")), + ("{\"k\":\"need_snap\",\"doc\":\"d\"}", None), + ("{\"k\":\"need_blob\",\"hash\":\"h\"}", None), + ( + "{\"k\":\"chunk\",\"hash\":\"h\",\"i\":0,\"n\":1,\"blob\":\"AA==\"}", + None, + ), + ] { + assert_eq!( + custody_class_of_plaintext(&framed(kind)), + expected, + "{kind}" + ); + } + } + + #[test] + fn anything_but_a_readable_sync_frame_is_not_deposited() { + assert_eq!(custody_class_of_plaintext("hello"), None); + assert_eq!( + custody_class_of_plaintext(&format!("{}not json", internal_prefixes::DATA_V1)), + None + ); + // A future version is not deposited rather than misclassified. + let future = format!( + "{}{{\"v\":{},\"k\":\"delta\",\"doc\":\"d\",\"blob\":\"AA==\"}}", + internal_prefixes::DATA_V1, + DATA_SYNC_V1 + 1 + ); + assert_eq!(custody_class_of_plaintext(&future), None); + } +} diff --git a/crates/offline-protocol/src/protocol/mesh_relay.rs b/crates/offline-protocol/src/protocol/mesh_relay.rs index be9eb7be8..1d8f93a67 100644 --- a/crates/offline-protocol/src/protocol/mesh_relay.rs +++ b/crates/offline-protocol/src/protocol/mesh_relay.rs @@ -61,6 +61,8 @@ use std::collections::HashMap; use std::hash::{Hash, Hasher}; use std::time::{Duration, Instant}; +use super::custody::CUSTODY_META_KEY; + /// Hop budget a forwarded frame is clamped to under normal density. pub const DEFAULT_RELAY_MAX_TTL: u8 = 8; @@ -314,6 +316,19 @@ pub struct PendingRelay { pub arrival_peer: Option, /// When it becomes eligible to transmit. pub due_at: Instant, + /// The class token the frame carried as a deposit request on arrival + /// (`docs/spec/custody.md`). Kept here because the forwarded copy has + /// the key stripped, and the drop point needs it to judge the frame for + /// custody. `None` on a frame that asked for nothing. + pub custody_request: Option, + /// For a held frame being redelivered: the one neighbor it is + /// transmitted to, never through target selection. `None` on an + /// ordinary forward. A forward carrying this is a *marked* forward: the + /// drop point returns it to the custody store instead of judging it, and + /// neither the overdue cut-off nor a refused requeue touches the + /// suppression cache for it, because the held intake never recorded its + /// id there. + pub custody_target: Option, } /// Running totals, for telemetry and for tests that assert a flood stayed @@ -804,6 +819,10 @@ impl MeshRelayGovernor { } } + // The deposit request, read before the hop rewrite strips it from the + // copy that travels on. Only the drop point acts on it. + let custody_request = message.metadata.get(CUSTODY_META_KEY).cloned(); + // Hop accounting. The arriving budget is clamped to what our own policy // would have issued before it is spent, so an inflated claim buys at // most this one hop. @@ -829,12 +848,56 @@ impl MeshRelayGovernor { message: forwarded, arrival_peer: arrival_peer.map(str::to_string), due_at, + custody_request, + custody_target: None, }); self.counters.queued = self.counters.queued.saturating_add(1); RelayAdmission::Queued } + /// Queues a held frame for redelivery toward one neighbor: the dedicated + /// intake custody uses instead of [`Self::admit`] + /// (`docs/spec/custody.md`, "Redelivery"). + /// + /// The ordinary intake would refuse or mangle a held frame in three + /// ways, and this one skips exactly those: it does not consult the + /// suppression cache, which would refuse a second re-origination toward + /// another neighbor within the cache's window; it does not spend a hop, + /// because the custodian is the same forwarder it was when the frame + /// arrived and the frame goes out with the hop fields it was stored + /// with; and it does not charge the arrival peer's rate hours after that + /// peer sent anything. It still takes the queue capacity (evicting a + /// lower-priority forward as `admit` would) and the per-neighbor rate + /// toward the target, and the send budget is checked at release like + /// every other forward. + /// + /// Nothing is recorded in the suppression cache, so a refusal here and + /// an abandonment later both leave the cache exactly as they found it. + /// Refusals are not counted: the frame is still held, and the caller + /// tries again when the neighbor next appears. + pub fn admit_held( + &mut self, + message: Message, + target: &str, + now: Instant, + ) -> Result<(), RelayRejection> { + if !self.take_peer_token(target, now) { + return Err(RelayRejection::PeerRateLimited); + } + if self.pending.len() >= self.config.queue_capacity && !self.evict_for(&message, now) { + return Err(RelayRejection::QueueFull); + } + self.pending.push(PendingRelay { + message, + arrival_peer: None, + due_at: now, + custody_request: None, + custody_target: Some(target.to_string()), + }); + Ok(()) + } + /// Returns the forwards whose delay has elapsed, removing them from the /// queue. /// @@ -856,10 +919,27 @@ impl MeshRelayGovernor { /// back with [`Self::requeue`] rather than drop it — the id is already /// recorded as handled here, so dropping it would lose this copy *and* /// refuse the copies and retransmissions that follow. + #[cfg(test)] pub fn take_due(&mut self, now: Instant) -> Vec { + self.release_due(now).0 + } + + /// [`Self::take_due`], also handing back the forwards it abandoned. + /// + /// The second list is the drop point (`docs/spec/custody.md`, "What a + /// custodian does on arrival"). An ordinary forward in it waited past + /// [`RELAY_QUEUE_MAX_OVERDUE`] without reaching a link, its id has + /// already been released from the suppression cache here, and custody + /// may take it on. A marked held forward (`custody_target` set) in it is + /// returned to the custody store by the caller; this never touches the + /// suppression cache for one, because the held intake never recorded its + /// id there, and releasing it would at best be a no-op and at worst + /// release an entry the depositor's own retransmission legitimately holds. + pub fn release_due(&mut self, now: Instant) -> (Vec, Vec) { self.seen.expire(now); let mut due = Vec::new(); + let mut abandoned = Vec::new(); let mut keep: Vec = Vec::with_capacity(self.pending.len()); for relay in std::mem::take(&mut self.pending) { @@ -869,12 +949,17 @@ impl MeshRelayGovernor { } if now.saturating_duration_since(relay.due_at) > RELAY_QUEUE_MAX_OVERDUE { - // Never made it onto a link, so its id must not stay - // suppressed: the sender's retransmissions carry the same id, - // and refusing them would close this route for the whole - // retention window over a few seconds of congestion. - self.seen.forget(&relay.message.id.as_str()); - self.counters.abandoned_overdue = self.counters.abandoned_overdue.saturating_add(1); + if relay.custody_target.is_none() { + // Never made it onto a link, so its id must not stay + // suppressed: the sender's retransmissions carry the same + // id, and refusing them would close this route for the + // whole retention window over a few seconds of + // congestion. + self.seen.forget(&relay.message.id.as_str()); + self.counters.abandoned_overdue = + self.counters.abandoned_overdue.saturating_add(1); + } + abandoned.push(relay); continue; } @@ -887,7 +972,7 @@ impl MeshRelayGovernor { } self.pending = keep; - due + (due, abandoned) } /// Whether any budget remains to forward right now. @@ -973,19 +1058,39 @@ impl MeshRelayGovernor { /// instead of being retried forever. /// /// Refused only if the queue has filled meanwhile, which is the same - /// bound every other queued frame is subject to. - pub fn requeue(&mut self, relay: PendingRelay) { + /// bound every other queued frame is subject to. A marked held forward + /// refused room is handed back rather than dropped: the custody store + /// still holds the frame, so nothing is lost, and its id was never + /// recorded in the suppression cache, so nothing is released. The + /// caller returns it to the store; `None` means the frame was requeued + /// or, for an ordinary forward, dropped. + pub fn requeue(&mut self, relay: PendingRelay) -> Option { if self.pending.len() >= self.config.queue_capacity { + if relay.custody_target.is_some() { + return Some(relay); + } // Refused room on the way back, so this frame is being dropped // having reached nobody. Release its id for the same reason the // overdue cut-off does — a copy behind it, or the sender's own // retransmission, is now the only way it travels. self.seen.forget(&relay.message.id.as_str()); self.counters.queue_full = self.counters.queue_full.saturating_add(1); - return; + return None; } self.counters.requeued = self.counters.requeued.saturating_add(1); self.pending.push(relay); + None + } + + /// Whether `message_id` is recorded as handled in the suppression cache. + /// + /// For the tests that pin custody and the cache disjoint: a custodian + /// that both held a frame and suppressed its own forwarding of the + /// depositor's retransmissions would be a black hole on exactly the + /// route now known to be slow. + #[cfg(test)] + pub fn is_suppressed(&self, message_id: &str) -> bool { + self.seen.contains(message_id) } /// Records an id as handled without queueing a forward for it. @@ -1171,6 +1276,15 @@ impl MeshRelayGovernor { // past this hop. forwarded.ttl = remaining; let _ = forwarded.increment_hop(); + // Every device that transmits a third-party frame strips the deposit + // request from the copy it transmits, custody-enabled or not + // (`docs/spec/custody.md`, "What a forwarder does"). The key is + // unsigned metadata outside the sealed body, so nothing else stops + // it travelling on; left in place, a two-hop network deposits a + // frame at a custodian that never saw its sender, and a zone holds + // every frame everywhere. Only the outer metadata is touched, never + // `content`. + forwarded.metadata.remove(CUSTODY_META_KEY); Some(forwarded) } @@ -2510,4 +2624,194 @@ mod tests { assert_eq!(gov.observe_activity(start + Duration::from_secs(22)), None); assert!(!gov.is_active_relay()); } + + // ==================================================================== + // Custody: the held-frame intake and the drop point + // ==================================================================== + + /// A frame carrying a deposit request, as a depositor offers it. + fn deposit_frame() -> Message { + let mut msg = frame(); + msg.metadata + .insert(CUSTODY_META_KEY.to_string(), "data".to_string()); + msg + } + + #[test] + fn a_forwarder_strips_the_deposit_request_and_the_drop_point_keeps_it() { + // The key is unsigned metadata outside the sealed body, so nothing + // but this strip keeps a deposit to one hop: left in place, a two-hop + // network deposits a frame at a custodian that never saw its sender. + let mut gov = governor(); + let msg = deposit_frame(); + assert_eq!( + gov.admit(&msg, Some("alice"), 3, false), + RelayAdmission::Queued + ); + + let (due, abandoned) = gov.release_due(Instant::now()); + assert!(abandoned.is_empty()); + let relay = &due[0]; + assert!( + !relay.message.metadata.contains_key(CUSTODY_META_KEY), + "the copy that travels on carries no request" + ); + assert_eq!( + relay.custody_request.as_deref(), + Some("data"), + "the request is kept beside the copy, for the drop point alone" + ); + assert_eq!( + relay.custody_target, None, + "an ordinary forward is not marked" + ); + assert_eq!( + relay.message.content, msg.content, + "only the outer metadata is touched" + ); + } + + #[test] + fn an_abandoned_forward_is_handed_to_the_drop_point_with_its_id_released() { + let mut gov = governor(); + let msg = deposit_frame(); + gov.admit(&msg, Some("alice"), 3, false); + let (due, _) = gov.release_due(Instant::now()); + gov.requeue(due.into_iter().next().unwrap()); + + let later = Instant::now() + RELAY_QUEUE_MAX_OVERDUE + Duration::from_millis(1); + let (due, abandoned) = gov.release_due(later); + assert!(due.is_empty()); + assert_eq!( + abandoned.len(), + 1, + "the drop point sees the abandoned forward" + ); + assert_eq!(abandoned[0].custody_request.as_deref(), Some("data")); + assert!( + !gov.is_suppressed(&msg.id.as_str()), + "released before custody judges it, so a custodian never blanks its own route" + ); + assert_eq!(gov.counters().abandoned_overdue, 1); + } + + #[test] + fn the_held_intake_skips_suppression_and_hop_accounting() { + let mut gov = governor(); + let msg = frame_with(3, MessagePriority::Medium); + + // The ordinary path has handled this id: a copy would be refused. + assert_eq!( + gov.admit(&msg, Some("alice"), 3, false), + RelayAdmission::Queued + ); + let (due, _) = gov.release_due(Instant::now()); + assert_eq!(due.len(), 1); + assert_eq!( + gov.admit(&msg, Some("alice"), 3, false), + RelayAdmission::Rejected(RelayRejection::AlreadySeen) + ); + + // The held intake queues it anyway, toward one neighbor, with the hop + // fields exactly as handed in. + let held = due.into_iter().next().unwrap().message; + let hop_before = held.hop_count.value(); + let ttl_before = held.ttl.value(); + gov.admit_held(held, "dave", Instant::now()) + .expect("queued"); + let (due, _) = gov.release_due(Instant::now()); + assert_eq!(due.len(), 1); + let relay = &due[0]; + assert_eq!(relay.custody_target.as_deref(), Some("dave")); + assert_eq!(relay.arrival_peer, None); + assert_eq!(relay.message.hop_count.value(), hop_before, "no hop spent"); + assert_eq!(relay.message.ttl.value(), ttl_before, "no budget spent"); + + // And it recorded nothing: the cache is exactly as it was. + let seen_before = gov.seen.len(); + gov.admit_held(relay.message.clone(), "erin", Instant::now()) + .expect("queued"); + assert_eq!(gov.seen.len(), seen_before); + } + + #[test] + fn a_marked_forward_never_touches_the_suppression_cache() { + let mut gov = governor(); + let msg = frame(); + // The depositor's own retransmission is legitimately handled: taken + // on, released to the radio, and its id recorded for the window. + gov.admit(&msg, Some("alice"), 3, false); + let (due, _) = gov.release_due(Instant::now()); + assert_eq!(due.len(), 1, "transmitted"); + assert!(gov.is_suppressed(&msg.id.as_str())); + + // A held copy of the same id, abandoned: the ordinary entry stays. + gov.admit_held(msg.clone(), "dave", Instant::now()) + .expect("queued"); + let later = Instant::now() + RELAY_QUEUE_MAX_OVERDUE + Duration::from_millis(1); + let (_, abandoned) = gov.release_due(later); + let held: Vec<_> = abandoned + .iter() + .filter(|r| r.custody_target.is_some()) + .collect(); + assert_eq!(held.len(), 1); + assert!( + gov.is_suppressed(&msg.id.as_str()), + "abandoning a marked forward releases nothing" + ); + // Nor does refusing it room on the way back. + let counters_before = gov.counters().clone(); + let mut full = MeshRelayGovernor::with_config( + "relay-node", + MeshRelayConfig { + queue_capacity: 1, + ..immediate_config() + }, + ); + full.admit(&frame(), Some("alice"), 3, false); + let returned = full.requeue(PendingRelay { + message: frame(), + arrival_peer: None, + due_at: Instant::now(), + custody_request: None, + custody_target: Some("dave".to_string()), + }); + assert!(returned.is_some(), "handed back to the custody store"); + assert_eq!(full.counters().queue_full, 0, "not counted as a loss"); + assert_eq!( + gov.counters().abandoned_overdue, + counters_before.abandoned_overdue + ); + } + + #[test] + fn the_held_intake_still_takes_the_queue_and_the_neighbor_rate() { + let mut gov = MeshRelayGovernor::with_config( + "relay-node", + MeshRelayConfig { + queue_capacity: 1, + peer_burst: 1.0, + peer_rate_per_sec: 0.0001, + ..immediate_config() + }, + ); + let now = Instant::now(); + gov.admit_held(frame(), "dave", now) + .expect("first one queues"); + assert_eq!( + gov.admit_held(frame(), "erin", now).err(), + Some(RelayRejection::QueueFull), + "a full queue is a full queue" + ); + let (due, _) = gov.release_due(now); + assert_eq!(due.len(), 1); + assert_eq!( + gov.admit_held(frame(), "dave", now).err(), + Some(RelayRejection::PeerRateLimited), + "one neighbor's share is spent" + ); + // Erin's share went with the refusal above, as it does on the + // ordinary path; a neighbor nobody has charged still admits one. + assert!(gov.admit_held(frame(), "frank", now).is_ok()); + } } diff --git a/crates/offline-protocol/src/protocol/message_dispatch.rs b/crates/offline-protocol/src/protocol/message_dispatch.rs index 1ec6ea971..32c0c79fc 100644 --- a/crates/offline-protocol/src/protocol/message_dispatch.rs +++ b/crates/offline-protocol/src/protocol/message_dispatch.rs @@ -6,8 +6,8 @@ use super::{ GroupMemberAddedPayload, GroupMemberRemovedPayload, GroupMessageReceivedPayload, InternalMessageResult, KeyPackagePayload, OfflineProtocol, PeerCapabilities, PresencePayload, ReadReceiptPayload, ReceivedKeyPackage, TypingIndicatorPayload, UserGroupsPayload, - DATA_GROUP_BLOB_V1, DATA_GROUP_V1, DATA_INTEREST_V1, DATA_MEDIA_V1, DATA_SYNC_V1, - DATA_TOMBSTONE_V1, MAX_KEY_PACKAGE_LIFETIME_MS, MAX_KEY_PACKAGE_SENT_TO, + DATA_CUSTODY_V1, DATA_GROUP_BLOB_V1, DATA_GROUP_V1, DATA_INTEREST_V1, DATA_MEDIA_V1, + DATA_SYNC_V1, DATA_TOMBSTONE_V1, MAX_KEY_PACKAGE_LIFETIME_MS, MAX_KEY_PACKAGE_SENT_TO, MAX_PENDING_KEY_PACKAGES, MAX_READ_RECEIPT_IDS, MLS_ENVELOPE_COMPACT_V1, RICH_PAYLOAD_V1, }; use crate::events::{DecryptionFailureCode, Event, SecurityWarningCode}; @@ -254,6 +254,22 @@ impl OfflineProtocol { } self.peer_data_group_blob_attested.remove(sender); + // Whether this peer parses a custody receipt, so a deposit of + // theirs this device takes on may be answered. Gates the receipt + // only: a deposit is judged by the quotas, never by this entry. + // Same shape as the sets above: gated by our own data switch, + // removed when a fresh key package stops advertising it. + if self.config.data.enabled && payload.data_versions.contains(&DATA_CUSTODY_V1) { + if !self.peer_data_custody.contains(sender) + && self.peer_data_custody.len() >= MAX_KEY_PACKAGE_SENT_TO + { + self.peer_data_custody.clear(); + } + self.peer_data_custody.insert(sender.to_string()); + } else { + self.peer_data_custody.remove(sender); + } + // Direct knowledge is authoritative for the group capability // too: a key package from the peer itself evicts whatever an // inviter attested about them, in either direction. The durable diff --git a/crates/offline-protocol/src/protocol/mod.rs b/crates/offline-protocol/src/protocol/mod.rs index c9cf6dc99..eac811d1e 100644 --- a/crates/offline-protocol/src/protocol/mod.rs +++ b/crates/offline-protocol/src/protocol/mod.rs @@ -2,6 +2,8 @@ mod blocking; mod config_accessors; +mod custodian; +pub(crate) mod custody; #[cfg(feature = "data")] pub(crate) mod data; #[cfg(feature = "data")] @@ -23,6 +25,7 @@ pub(crate) mod state_crypto; mod storage; mod types; +pub use custody::{CustodyRefusal, CustodyStats}; pub(crate) use decryption_queue::PendingDecryptionQueue; pub use decryption_queue::PendingQueueMetrics; pub(crate) use prefixes::*; @@ -133,6 +136,23 @@ pub struct OfflineProtocol { /// delivery has to be retried. pub(crate) mesh_relay: MeshRelayGovernor, + /// Frames held in custody for a neighbour, with the quotas and the + /// counters (`docs/spec/custody.md`). Disjoint from + /// [`Self::mesh_relay`]'s suppression cache by construction: accepting a + /// frame never records its id there, and redelivering one never consults + /// it, so a custodian never blanks the route it is holding. + pub(crate) custody: custody::CustodyStore, + + /// Receipts this device holds as a *depositor*: for each outbox entry, + /// the custodians holding it and until when a further deposit request + /// toward each is suppressed. In memory only, bounded by the outbox and + /// by [`custody::MAX_CUSTODY_RECEIPTS_PER_MESSAGE`]; losing it at a + /// restart costs one duplicate deposit, which the custodian absorbs. + pub(crate) custody_receipts: HashMap>, + + /// When the custody store was last swept for expired records. + custody_last_sweep: Instant, + /// Shared mutable state. shared_state: Arc>, @@ -480,6 +500,15 @@ pub struct OfflineProtocol { /// received key package, in either direction. peer_data_group_blob_attested: std::collections::HashSet, + /// Peers whose key package advertised the custody receipt + /// ([`DATA_CUSTODY_V1`] in `data_versions`), so this device may answer + /// their deposits with one. Gates the receipt and nothing else: a deposit + /// is judged by the quotas. Persisted inside `PeerCapabilities`, restored + /// on `initialize_mls`, bounded like `key_package_sent_to`. + /// + /// [`DATA_CUSTODY_V1`]: crate::protocol::types::DATA_CUSTODY_V1 + peer_data_custody: std::collections::HashSet, + /// Peers already flagged with a `PlaintextSend` security warning, so the /// explicit-opt-out plaintext path warns once per peer instead of once /// per message. @@ -1052,6 +1081,9 @@ impl OfflineProtocol { config.profile.clone(), config.mesh_relay.clone(), ), + custody: custody::CustodyStore::new(config.custody.clone()), + custody_receipts: HashMap::new(), + custody_last_sweep: Instant::now(), local_id: config.profile.clone(), identity_established: false, shared_state: Arc::new(Mutex::new(SharedState::new())), @@ -1084,6 +1116,7 @@ impl OfflineProtocol { peer_data_interest: std::collections::HashSet::new(), peer_data_group_blob: std::collections::HashSet::new(), peer_data_group_blob_attested: std::collections::HashSet::new(), + peer_data_custody: std::collections::HashSet::new(), peer_rich_attested: std::collections::HashSet::new(), plaintext_send_warned: std::collections::HashSet::new(), plaintext_receive_warned: std::collections::HashSet::new(), @@ -1393,6 +1426,7 @@ impl OfflineProtocol { let restore_result = (|| { self.restore_pending_messages(&mut pending_prunes)?; self.restore_pending_decrypt_entries(&mut inbound_prunes); + self.restore_custody(&mut inbound_prunes); self.restore_lamport_clock(); self.restore_dedup_seen(&mut inbound_prunes); self.restore_encryption_capable_peers(); @@ -1532,6 +1566,7 @@ impl OfflineProtocol { let mut inbound_prunes = PruneAllowance::pool(); self.restore_pending_messages(&mut pending_prunes)?; self.restore_pending_decrypt_entries(&mut inbound_prunes); + self.restore_custody(&mut inbound_prunes); self.restore_lamport_clock(); self.restore_dedup_seen(&mut inbound_prunes); self.restore_encryption_capable_peers(); @@ -2208,6 +2243,7 @@ impl OfflineProtocol { self.peer_data_interest.remove(peer); self.peer_data_group_blob.remove(peer); self.peer_data_group_blob_attested.remove(peer); + self.peer_data_custody.remove(peer); // A peer we have stopped replicating with cannot answer anything we // asked them for, so the questions go too. Left behind they would // hold slots against the fetch bound until they timed out. @@ -2297,6 +2333,10 @@ impl OfflineProtocol { // Flush any pending outbox messages destined for this peer self.flush_outbox_for_peer_via(peer_id, unpark_via); + // Held frames for this neighbour, or to try through it + // (`docs/spec/custody.md`, "Redelivery"). + self.redeliver_custody_to(peer_id); + // A Welcome that stalled or expired while this peer was unreachable now // has a fresh delivery opportunity over the carrier that surfaced this // peer — re-arm it. No-op when there is no pending Welcome for the peer. @@ -3297,6 +3337,16 @@ impl OfflineProtocol { return Some(InternalMessageResult::Consumed); } + // A custody receipt from a neighbour holding one of our frames. Past + // the control gate like every signed frame; it suppresses re-deposit + // toward that custodian and settles nothing (`docs/spec/custody.md`). + // Consumed whatever the body says: a malformed one is refused + // silently, and a receipt never requests an acknowledgement. + if let Some(data) = content.strip_prefix(internal_prefixes::CUSTODY_RECEIPT) { + self.handle_custody_receipt(sender, data); + return Some(InternalMessageResult::Consumed); + } + // --- Group (mesh/MLS) messages --- if let Some(data) = content.strip_prefix(internal_prefixes::GROUP_MLS_MSG) { @@ -3388,6 +3438,9 @@ impl OfflineProtocol { // own retries are not — a frame held too long is dropped rather than // sent late. self.flush_mesh_relays(); + // Held frames past their hold, and receipt suppressions past their + // end. Throttled inside; hours-long holds need no per-tick walk. + self.sweep_custody(); self.process_retry_queue()?; self.process_welcome_retry_queue()?; diff --git a/crates/offline-protocol/src/protocol/receive.rs b/crates/offline-protocol/src/protocol/receive.rs index 9b8bdbb7b..2cdb85a71 100644 --- a/crates/offline-protocol/src/protocol/receive.rs +++ b/crates/offline-protocol/src/protocol/receive.rs @@ -500,7 +500,7 @@ impl OfflineProtocol { /// /// Takes its own battery reading; the per-tick caller already holds one and /// uses [`Self::battery_allows_relaying_with`] instead. - fn battery_allows_relaying(&self) -> bool { + pub(super) fn battery_allows_relaying(&self) -> bool { let (_statuses, available) = self.transport_manager.snapshot_status_and_available(); let (battery_level, is_charging) = crate::telemetry::aggregator::device_battery_from_available( @@ -554,7 +554,23 @@ impl OfflineProtocol { /// to the peer that wrote it. A frame whose recipient is a neighbor of ours /// is handed straight to them instead — the shortest path we can see. pub(super) fn flush_mesh_relays(&mut self) { - let due = self.mesh_relay.take_due(Instant::now()); + self.flush_mesh_relays_at(Instant::now()); + } + + /// [`Self::flush_mesh_relays`] at a given instant, so a test can walk a + /// forward past the overdue cut-off without waiting it out. + pub(super) fn flush_mesh_relays_at(&mut self, now: Instant) { + let (due, abandoned) = self.mesh_relay.release_due(now); + + // The drop point (`docs/spec/custody.md`): an ordinary forward here + // waited past the overdue cut-off without reaching a link, and its + // id is already released from the suppression cache; custody may + // take it on. A marked held forward goes back to the store, unjudged + // and uncounted. + for relay in abandoned { + self.judge_at_drop_point(relay); + } + if due.is_empty() { return; } @@ -571,30 +587,48 @@ impl OfflineProtocol { let message_id = message.id.as_str(); let hop_count = message.hop_count.value(); let remaining_ttl = message.ttl.value(); + let held = relay.custody_target.is_some(); + let cause = if held { + "custody_redelivery" + } else { + "forward" + }; - let mut exclude: Vec<&str> = vec![message.sender.as_str()]; - if let Some(peer) = relay.arrival_peer.as_deref() { - exclude.push(peer); - } - let onward = self.mesh_relay.select_targets( - neighbors - .iter() - .map(|n| (n.peer_id.as_str(), n.link_quality())), - &exclude, - &message_id, - ); - - // If the destination is one of our own neighbors, hand it over - // directly: no fan-out is worth more than arriving. Should that - // link fail between choosing it and writing to it, fall back to - // carrying it onward rather than dropping a frame we could still - // move. - let targets = if neighbors.iter().any(|n| n.peer_id == recipient) { - let mut ordered = vec![recipient.clone()]; - ordered.extend(onward.into_iter().filter(|peer| peer != &recipient)); - ordered + let targets = if let Some(target) = relay.custody_target.as_deref() { + // A held frame goes to the one neighbor it was queued toward, + // never through target selection: a fan-out here would defeat + // at most once per neighbor. Gone from range, it goes nowhere + // and back to the store below. + if neighbors.iter().any(|n| n.peer_id == target) { + vec![target.to_string()] + } else { + Vec::new() + } } else { - onward + let mut exclude: Vec<&str> = vec![message.sender.as_str()]; + if let Some(peer) = relay.arrival_peer.as_deref() { + exclude.push(peer); + } + let onward = self.mesh_relay.select_targets( + neighbors + .iter() + .map(|n| (n.peer_id.as_str(), n.link_quality())), + &exclude, + &message_id, + ); + + // If the destination is one of our own neighbors, hand it over + // directly: no fan-out is worth more than arriving. Should that + // link fail between choosing it and writing to it, fall back to + // carrying it onward rather than dropping a frame we could still + // move. + if neighbors.iter().any(|n| n.peer_id == recipient) { + let mut ordered = vec![recipient.clone()]; + ordered.extend(onward.into_iter().filter(|peer| peer != &recipient)); + ordered + } else { + onward + } }; let deliver_direct = targets.first() == Some(&recipient); @@ -606,11 +640,13 @@ impl OfflineProtocol { // the frame is still worth carrying. debug!( message_id = %message.id, + cause, "No onward neighbor for this frame" ); } let mut delivered_to = 0usize; + let mut reached_recipient = false; for target in &targets { // Each link this frame crosses is one transmission against the // device's ceiling. Running out mid-fan-out stops the fan-out @@ -631,10 +667,12 @@ impl OfflineProtocol { // journey; the remaining neighbors are only a fallback // for that link failing. if deliver_direct && target == &recipient { + reached_recipient = true; debug!( message_id = %message.id, next_hop = %target, transport = ?transport, + cause, "Delivered to its recipient directly" ); break; @@ -645,6 +683,7 @@ impl OfflineProtocol { transport = ?transport, hop_count, remaining_ttl, + cause, "Forwarded frame to neighbor" ); } @@ -674,11 +713,20 @@ impl OfflineProtocol { // thing that still remembers it. It keeps its due time, so one that // stays stuck is abandoned by the overdue cut-off rather than // retried forever. + // + // A held frame refused room on the way back goes to the custody + // store instead: the store still holds it, nothing is lost, and + // its id was never recorded here. if delivered_to == 0 { - self.mesh_relay.requeue(relay); + if let Some(returned) = self.mesh_relay.requeue(relay) { + self.return_held_to_store(returned); + } continue; } + if held { + self.record_custody_transmission(&relay, reached_recipient); + } self.mesh_relay.record_forwarded(); self.emit_event(Event::message_relayed( message.id.as_str(), diff --git a/crates/offline-protocol/src/protocol/send.rs b/crates/offline-protocol/src/protocol/send.rs index 02fb7e0dd..e1ed1b148 100644 --- a/crates/offline-protocol/src/protocol/send.rs +++ b/crates/offline-protocol/src/protocol/send.rs @@ -1,5 +1,6 @@ //! Send pipeline, outbox management, and delivery tracking. +use super::custody::CUSTODY_META_KEY; use super::{ base64_encode, internal_prefixes, lifetime_expired, lock_shared_state, ConnectionAcceptedPayload, ConnectionRequestPayload, KeyPackagePayload, MediaSendOptions, @@ -1017,6 +1018,7 @@ impl OfflineProtocol { super::types::DATA_TOMBSTONE_V1, super::types::DATA_INTEREST_V1, super::types::DATA_GROUP_BLOB_V1, + super::types::DATA_CUSTODY_V1, ]; } } @@ -3261,6 +3263,9 @@ impl OfflineProtocol { // staging is consumed at entry creation; this covers the id being torn // down before that ever happened. self.pending_reseal.remove(message_id); + // A receipt names an outbox entry; with the entry gone it names + // nothing, and keeping it would only grow the map. + self.custody_receipts.remove(message_id); if let Some(entry) = self.outbox.remove(message_id) { self.clear_outbox_entry_from_storage(message_id); return Some(entry); @@ -3678,6 +3683,14 @@ impl OfflineProtocol { ) }; + // The deposit request (`docs/spec/custody.md`, "What the depositor + // writes"): a class token on this device's own sealed replication + // frame, judged from the plaintext retained for re-sealing, written + // only here, where the frame is handed to proven mesh neighbours + // because the recipient is out of reach. Never toward a custodian + // whose receipt for this frame is still live. + let custody_class = self.custody_class_for(message); + let mut handed_to = 0usize; for target in targets { // Metered against the same ceiling as carrying other people's @@ -3697,7 +3710,20 @@ impl OfflineProtocol { break; } - match self.transport_manager.send_to_neighbor(&target, message) { + let deposit; + let frame: &Message = match custody_class { + Some(class) if !self.custody_suppressed_toward(&message.id, &target) => { + let mut stamped = message.clone(); + stamped + .metadata + .insert(CUSTODY_META_KEY.to_string(), class.to_string()); + deposit = stamped; + &deposit + } + _ => message, + }; + + match self.transport_manager.send_to_neighbor(&target, frame) { Ok(transport) => { handed_to += 1; debug!( @@ -3705,6 +3731,7 @@ impl OfflineProtocol { recipient = %message.recipient, next_hop = %target, transport = ?transport, + deposit = frame.metadata.contains_key(CUSTODY_META_KEY), "Handed message to a neighbor to carry" ); } diff --git a/crates/offline-protocol/src/protocol/storage.rs b/crates/offline-protocol/src/protocol/storage.rs index 4f24834c5..5f5e8dc7b 100644 --- a/crates/offline-protocol/src/protocol/storage.rs +++ b/crates/offline-protocol/src/protocol/storage.rs @@ -5,8 +5,8 @@ use super::{ lifetime_expired, storage_keys, MediaTransferDescriptor, OfflineProtocol, OutboxEntry, PeerCapabilities, PendingDecryptRecord, PendingMessage, PendingMessageRecord, ReceivedKeyPackage, SessionState, WelcomeDeliveryState, WelcomeLifecycleRecord, - DATA_GROUP_BLOB_V1, DATA_GROUP_V1, DATA_INTEREST_V1, DATA_MEDIA_V1, DATA_SYNC_V1, - DATA_TOMBSTONE_V1, MAX_BLOCKED_USERS, MAX_KEY_PACKAGE_SENT_TO, + DATA_CUSTODY_V1, DATA_GROUP_BLOB_V1, DATA_GROUP_V1, DATA_INTEREST_V1, DATA_MEDIA_V1, + DATA_SYNC_V1, DATA_TOMBSTONE_V1, MAX_BLOCKED_USERS, MAX_KEY_PACKAGE_SENT_TO, MAX_MIGRATED_PENDING_WRITES_PER_LAUNCH, MAX_PENDING_KEY_PACKAGES, MAX_PENDING_MESSAGES_GLOBAL, MAX_PENDING_MESSAGES_PER_PEER, MAX_PENDING_MESSAGE_BYTES_GLOBAL, MAX_PENDING_MESSAGE_BYTES_PER_PEER, MAX_PERSISTED_CAPABILITY_VERSIONS, @@ -58,6 +58,12 @@ pub(crate) enum StateCategory { /// [`storage_keys::ADOPTABLE_STATE_KEY_TYPES`], which has no pre-split /// data to inherit for it. PendingDecryptEntries, + /// Frames held in custody for a neighbour. Sealed for the reason the + /// pending-decrypt records are: other people's ciphertext plus routing + /// metadata about them, outliving the process. Post-split only, and + /// absent from [`storage_keys::ADOPTABLE_STATE_KEY_TYPES`] like + /// [`Self::PendingDecryptEntries`]. + Custody, Outbox, MediaDescriptors, PeerKeyPackages, @@ -120,6 +126,7 @@ impl StateCategory { storage_keys::PENDING_MESSAGES => Self::PendingMessages, storage_keys::PENDING_MESSAGE_ENTRIES => Self::PendingMessageEntries, storage_keys::PENDING_DECRYPT_ENTRIES => Self::PendingDecryptEntries, + storage_keys::CUSTODY => Self::Custody, storage_keys::OUTBOX => Self::Outbox, storage_keys::MEDIA_DESCRIPTORS => Self::MediaDescriptors, storage_keys::PEER_KEY_PACKAGES => Self::PeerKeyPackages, @@ -159,6 +166,7 @@ impl StateCategory { Self::PendingMessages, Self::PendingMessageEntries, Self::PendingDecryptEntries, + Self::Custody, Self::Outbox, Self::MediaDescriptors, Self::PeerKeyPackages, @@ -190,6 +198,7 @@ impl StateCategory { Self::PendingMessages => storage_keys::PENDING_MESSAGES, Self::PendingMessageEntries => storage_keys::PENDING_MESSAGE_ENTRIES, Self::PendingDecryptEntries => storage_keys::PENDING_DECRYPT_ENTRIES, + Self::Custody => storage_keys::CUSTODY, Self::Outbox => storage_keys::OUTBOX, Self::MediaDescriptors => storage_keys::MEDIA_DESCRIPTORS, Self::PeerKeyPackages => storage_keys::PEER_KEY_PACKAGES, @@ -303,6 +312,7 @@ impl StateCategory { Self::PendingMessages | Self::PendingMessageEntries | Self::PendingDecryptEntries + | Self::Custody | Self::Outbox | Self::MediaDescriptors | Self::PeerKeyPackages @@ -992,7 +1002,7 @@ impl<'a> PruneBudget<'a> { /// Claims one delete. Refuses — and records that this walk's share is gone /// — only for a refusing budget; a counting one always allows the delete /// and just charges for it. - fn claim(&mut self) -> bool { + pub(super) fn claim(&mut self) -> bool { if self.is_spent() { self.exhausted = true; if self.refusable { @@ -3234,6 +3244,13 @@ impl OfflineProtocol { if self.config.data.enabled && caps.data_versions.contains(&DATA_GROUP_BLOB_V1) { self.peer_data_group_blob.insert(peer_id.clone()); } + // And the custody receipt. Cheapest of all to skip: a deposit + // taken on before the peer's next key package would simply go + // unanswered, and the depositor deposits again on its next retry. + // Restored anyway, because the record is already here. + if self.config.data.enabled && caps.data_versions.contains(&DATA_CUSTODY_V1) { + self.peer_data_custody.insert(peer_id.clone()); + } if self.config.data.enabled && caps.attested_data_versions.contains(&DATA_GROUP_BLOB_V1) { self.peer_data_group_blob_attested.insert(peer_id.clone()); diff --git a/crates/offline-protocol/src/protocol/tests/custody.rs b/crates/offline-protocol/src/protocol/tests/custody.rs new file mode 100644 index 000000000..6eeeede5b --- /dev/null +++ b/crates/offline-protocol/src/protocol/tests/custody.rs @@ -0,0 +1,852 @@ +//! Custody over the real send, forward and receive paths. +//! +//! Three replicas: a depositor with a live session to a recipient it cannot +//! reach, a custodian in range of the depositor, and the recipient. Every +//! frame goes through MLS sealing, the transport, the forwarding governor and +//! the prefix dispatch, because the chapter's claims are about that machinery: +//! the deposit request rides the depositor's own sealed frame, acceptance +//! happens at the end of the forwarding queue, and redelivery is an ordinary +//! forward. What is pinned here is what a device can observe on its links and +//! in its store, plus the one thing a device must never do (blank its own +//! route) and the one thing a receipt must never do (settle anything). + +use std::sync::{Arc, Mutex}; +use std::time::{Duration, Instant}; + +use chrono::Utc; +use offline_protocol_data::DataValue; +use offline_protocol_transport::{MockTransport, Transport, TransportType}; + +use crate::config::CustodyConfig; +use crate::mls::InMemoryStorage; +use crate::protocol::custody::{encode_receipt, CUSTODY_CLASS_DATA, CUSTODY_META_KEY}; +use crate::protocol::mesh_relay::RELAY_QUEUE_MAX_OVERDUE; +use crate::protocol::prefixes::internal_prefixes; +use crate::protocol::tests::{create_test_config_for_user, id}; +use crate::protocol::types::{storage_keys, SessionState}; +use crate::protocol::{OfflineProtocol, TestProtocolStateStorage}; +use crate::ProtocolConfig; +use offline_protocol_core::{Message, MessageId, MessagePriority}; +use offline_protocol_mls::MlsStorage; + +/// One device, with the radio its frames actually go through. +struct Node { + protocol: OfflineProtocol, + transport: MockTransport, + address: String, + label: String, + secure: Arc, + state: Arc, + events: Arc>>, +} + +/// A custodian that admits strangers, so a deposit from a peer it holds no +/// session with is judged by the quotas rather than refused at the tier. +fn open_custody() -> CustodyConfig { + CustodyConfig { + enabled: true, + stranger_max_entries: 8, + stranger_max_bytes: 512 * 1024, + ..CustodyConfig::default() + } +} + +fn base_config(label: &str) -> ProtocolConfig { + let mut config = create_test_config_for_user(label); + config.encryption.enabled = true; + config.data.enabled = true; + // No hold before forwarding: the tests drive the queue by hand. + config.mesh_relay.jitter_min = Duration::from_millis(0); + config.mesh_relay.jitter_max = Duration::from_millis(0); + config +} + +impl Node { + fn new(label: &str) -> Self { + Self::with_config(label, base_config(label)) + } + + fn custodian(label: &str, custody: CustodyConfig) -> Self { + let mut config = base_config(label); + config.custody = custody; + Self::with_config(label, config) + } + + fn with_config(label: &str, config: ProtocolConfig) -> Self { + let secure = crate::test_identity::seeded_storage(label); + let state = Arc::new(InMemoryStorage::new()); + Self::launch(label, config, secure, state) + } + + fn launch( + label: &str, + config: ProtocolConfig, + secure: Arc, + state: Arc, + ) -> Self { + let mut protocol = OfflineProtocol::new(config).expect("protocol"); + protocol + .initialize_mls( + secure.clone(), + Arc::new(TestProtocolStateStorage { + storage: state.clone(), + }), + ) + .expect("initialize_mls"); + + let mock = MockTransport::new(TransportType::BLE); + mock.start().expect("transport start"); + // A device can only put a frame on a link it actually has, which is + // what makes a recipient out of range in the first place. + mock.set_reject_unknown_recipients(true); + let transport = mock.clone(); + protocol + .transport_manager_mut() + .add_transport(TransportType::BLE, Box::new(mock)); + let events: Arc>> = Arc::new(Mutex::new(Vec::new())); + let sink = events.clone(); + protocol.on_event(move |event| sink.lock().unwrap().push(event)); + protocol.start().expect("start"); + + Self { + protocol, + transport, + address: id(label), + label: label.to_string(), + secure, + state, + events, + } + } + + /// Relaunch on the same storage with a different configuration, as an + /// application does between two sessions. + fn relaunch(self, custody: CustodyConfig) -> Self { + let mut config = base_config(&self.label); + config.custody = custody; + let Node { + label, + secure, + state, + .. + } = self; + Self::launch(&label, config, secure, state) + } + + /// The space this node replicates with `peer`: the peer's own address. + fn space_for(peer: &Node) -> String { + peer.address.clone() + } + + /// Puts `peer` in range, as discovery would. + fn link(&mut self, peer: &Node) { + self.transport.add_connected_peer(peer.address.clone(), -55); + self.protocol.on_neighbor_discovered(&peer.address); + } + + fn unlink(&mut self, peer: &Node) { + self.transport.remove_connected_peer(&peer.address); + self.protocol.on_neighbor_lost(&peer.address); + } + + /// Everything handed to a neighbour since the last take, as + /// `(neighbour, frame)`. + fn take_peer_sends(&mut self) -> Vec<(String, Message)> { + let sends = self.transport.peer_sends(); + self.transport.clear_peer_sends(); + sends + } + + /// Hands `message` to this node over the link from `from`, and lets it + /// process everything it holds. + fn receive_from(&mut self, message: Message, from: &str) { + self.transport.queue_message_from(message, from.to_string()); + while self.protocol.receive_message().is_some() {} + } + + /// Runs the forwarding queue as if `elapsed` had passed since now. + fn flush_after(&mut self, elapsed: Duration) { + self.protocol.flush_mesh_relays_at(Instant::now() + elapsed); + } + + fn held_records(&self) -> Vec { + self.state + .list_keys(storage_keys::CUSTODY) + .unwrap_or_default() + } +} + +/// A real 1:1 MLS session between two nodes, with the sync capabilities +/// recorded on both sides as a key-package exchange would leave them. +fn pair(alice: &mut Node, carol: &mut Node) { + let carol_kp = { + let manager = carol.protocol.mls_manager.as_ref().unwrap().read().unwrap(); + manager.get_or_create_key_package().unwrap() + }; + let welcome = { + let manager = alice.protocol.mls_manager.as_ref().unwrap().read().unwrap(); + manager + .import_key_package(&carol.address, &carol_kp.key_package_data) + .unwrap(); + manager.create_session(&carol.address).unwrap() + }; + { + let manager = carol.protocol.mls_manager.as_ref().unwrap().read().unwrap(); + manager.join_session(&welcome).unwrap(); + } + confirm(alice, &carol.address); + confirm(carol, &alice.address); + let alice_address = alice.address.clone(); + let carol_address = carol.address.clone(); + for (node, peer) in [(&mut *alice, carol_address), (&mut *carol, alice_address)] { + node.protocol.peer_data_sync.insert(peer.clone()); + node.protocol.peer_data_tombstones.insert(peer.clone()); + node.protocol.peer_data_interest.insert(peer.clone()); + node.protocol.peer_data_custody.insert(peer); + } +} + +fn confirm(node: &mut Node, peer: &str) { + node.protocol.record_encryption_capable(peer); + node.protocol + .persist_session_state(peer, SessionState::Confirmed, "test") + .unwrap(); + node.protocol.confirmed_sessions.insert(peer.to_string()); +} + +fn write(node: &mut Node, space: &str, doc: &str, key: &str, value: &str) { + node.protocol + .data_map_set(space, doc, "m", key, DataValue::text(value)) + .expect("set"); + node.protocol.data_flush(space, doc).expect("flush"); +} + +fn read(node: &mut Node, space: &str, doc: &str, key: &str) -> Option { + node.protocol + .data_map_get(space, doc, "m", key) + .expect("get") +} + +/// The depositor's frame as it leaves for the custodian: alice edits a +/// document she replicates with carol while only bob is in range. +/// +/// Returns the frame handed to bob, which carries the deposit request. +fn deposit_from(alice: &mut Node, bob: &Node, carol: &Node) -> Message { + write(alice, &Node::space_for(carol), "notes", "k", "v"); + let sends = alice.take_peer_sends(); + let (to, frame) = sends + .into_iter() + .find(|(_, frame)| frame.content.starts_with(internal_prefixes::ENCRYPTED)) + .expect("the sync frame was handed to a neighbour"); + assert_eq!(to, bob.address, "handed to the one neighbour in range"); + assert_eq!(frame.recipient.as_str(), carol.address); + assert_eq!( + frame.metadata.get(CUSTODY_META_KEY).map(String::as_str), + Some(CUSTODY_CLASS_DATA), + "the depositor writes the class token on its own sealed replication frame" + ); + frame +} + +/// Three nodes in the chapter's opening topology: alice paired with carol, +/// alice in range of bob only, bob in range of alice only. +fn topology(custody: CustodyConfig) -> (Node, Node, Node) { + let mut alice = Node::new("alice"); + let mut carol = Node::new("carol"); + let mut bob = Node::custodian("bob", custody); + pair(&mut alice, &mut carol); + alice.link(&bob); + bob.link(&alice); + // Bob knows alice parses receipts, as her key package would have said. + bob.protocol.peer_data_custody.insert(alice.address.clone()); + (alice, bob, carol) +} + +/// Walks bob's queue to the drop point for a frame he could not forward. +fn drop_point(bob: &mut Node) { + // Due, nowhere to go (the only link is the depositor's), requeued. + bob.flush_after(Duration::ZERO); + // Past the overdue cut-off: abandoned, and judged. + bob.flush_after(RELAY_QUEUE_MAX_OVERDUE + Duration::from_millis(1)); +} + +#[test] +fn a_deposited_frame_is_held_and_delivered_when_the_recipient_appears() { + let (mut alice, mut bob, mut carol) = topology(open_custody()); + let frame = deposit_from(&mut alice, &bob, &carol); + let frame_id = frame.id.clone(); + + bob.receive_from(frame.clone(), &alice.address); + assert_eq!( + bob.protocol.custody_stats().held, + 0, + "a forward is not a deposit yet" + ); + drop_point(&mut bob); + + let stats = bob.protocol.custody_stats(); + assert_eq!(stats.accepted, 1); + assert_eq!(stats.held, 1); + assert_eq!( + bob.held_records(), + vec![frame_id.as_str()], + "sealed record on disk" + ); + assert_eq!( + bob.protocol.mesh_relay_stats().abandoned_overdue, + 1, + "custody begins only where forwarding ends" + ); + assert!( + !bob.protocol.mesh_relay.is_suppressed(&frame_id.as_str()), + "acceptance must not record the id in the suppression cache" + ); + + // The receipt: once, over the arrival link, signed, no acknowledgement + // requested, and it settles nothing at the depositor. + let sends = bob.take_peer_sends(); + let (to, receipt) = sends + .into_iter() + .find(|(_, m)| m.content.starts_with(internal_prefixes::CUSTODY_RECEIPT)) + .expect("a receipt was sent"); + assert_eq!(to, alice.address); + assert!( + !receipt.requires_ack, + "a receipt never requests an acknowledgement" + ); + assert!( + receipt.metadata.contains_key("__ctrl_sig"), + "signed like every control frame" + ); + assert_eq!(bob.protocol.custody_stats().receipts_sent, 1); + + assert!(alice.protocol.outbox.contains_key(&frame_id)); + let retries_before = alice.protocol.retry_queue_size(); + alice.receive_from(receipt, &bob.address); + assert_eq!(alice.protocol.custody_stats().receipts_received, 1); + assert!( + alice.protocol.outbox.contains_key(&frame_id), + "a receipt settles nothing: the outbox entry stands" + ); + assert_eq!( + alice.protocol.retry_queue_size(), + retries_before, + "and the retry entry stands" + ); + + // A re-offer toward the custodian that answered carries no request; the + // frame itself still goes, as an ordinary forward attempt. (Contact with + // bob also re-drove alice's outbox toward him before the receipt was + // read; only what leaves after it counts.) + alice.take_peer_sends(); + let stored = alice + .protocol + .outbox + .get(&frame_id) + .expect("still in the outbox") + .message + .clone(); + assert!( + !stored.metadata.contains_key(CUSTODY_META_KEY), + "the request is written on the copy handed over, never on the stored frame" + ); + alice.protocol.offer_to_mesh(&stored); + let sends = alice.take_peer_sends(); + let reoffers: Vec<&Message> = sends + .iter() + .filter(|(to, m)| *to == bob.address && m.id == frame_id) + .map(|(_, m)| m) + .collect(); + assert!(!reoffers.is_empty(), "re-offered to bob"); + assert!( + reoffers + .iter() + .all(|m| !m.metadata.contains_key(CUSTODY_META_KEY)), + "a live receipt suppresses the deposit request toward that custodian" + ); + + // The duplicate deposit bob absorbs: not stored twice, not answered. + bob.receive_from(frame.clone(), &alice.address); + drop_point(&mut bob); + let stats = bob.protocol.custody_stats(); + assert_eq!(stats.held, 1); + assert_eq!(stats.duplicates, 1); + assert_eq!(stats.receipts_sent, 1, "a duplicate is not answered"); + for reason in crate::protocol::custody::CustodyRefusal::ALL { + if *reason != crate::protocol::custody::CustodyRefusal::Duplicate { + assert_eq!(stats.refusals(*reason), 0, "{reason:?}"); + } + } + + // Alice walks away; carol appears. The held frame is delivered. + bob.unlink(&alice); + bob.link(&carol); + bob.flush_after(Duration::ZERO); + let sends = bob.take_peer_sends(); + let (to, delivered) = sends + .into_iter() + .find(|(_, m)| m.id == frame_id) + .expect("the held frame went out"); + assert_eq!(to, carol.address, "handed straight to the recipient"); + assert!( + !delivered.metadata.contains_key(CUSTODY_META_KEY), + "what the custodian transmits never carries the request" + ); + assert_eq!( + delivered.content, frame.content, + "the ciphertext is untouched" + ); + assert_eq!(delivered.hop_count.value(), 1, "one hop, as stored"); + + let stats = bob.protocol.custody_stats(); + assert_eq!(stats.delivered, 1); + assert_eq!( + stats.held, 0, + "delivery to the recipient releases the frame" + ); + assert!(bob.held_records().is_empty(), "and its record"); + assert!( + bob.events + .lock() + .unwrap() + .iter() + .any(|event| matches!(event, crate::Event::MessageRelayed { message_id, .. } if *message_id == frame_id.as_str())), + "a redelivery is an ordinary forward to the application" + ); + + // Carol opens it and the document lands, hours after alice wrote it. + carol.receive_from(delivered, &bob.address); + assert_eq!( + read(&mut carol, &Node::space_for(&alice), "notes", "k"), + Some(DataValue::text("v")) + ); +} + +#[test] +fn a_held_frame_is_re_originated_at_most_once_per_neighbour_and_stays_held() { + let (mut alice, mut bob, carol) = topology(open_custody()); + let frame = deposit_from(&mut alice, &bob, &carol); + bob.receive_from(frame.clone(), &alice.address); + drop_point(&mut bob); + assert_eq!(bob.protocol.custody_stats().held, 1); + + // Dave is not the recipient: the frame is re-originated toward him + // once, and stays held. + let dave = Node::new("dave"); + bob.link(&dave); + bob.flush_after(Duration::ZERO); + assert_eq!(bob.transport.peer_send_count_for(&frame.id.as_str()), 1); + let stats = bob.protocol.custody_stats(); + assert_eq!(stats.re_originated, 1); + assert_eq!(stats.held, 1, "a re-origination is not a delivery"); + + // Seen again: nothing more toward dave during this hold. + bob.protocol.on_neighbor_discovered(&dave.address); + bob.flush_after(Duration::ZERO); + assert_eq!(bob.transport.peer_send_count_for(&frame.id.as_str()), 1); + assert_eq!(bob.protocol.custody_stats().re_originated, 1); + + // The depositor never gets its own frame back. + bob.protocol.on_neighbor_discovered(&alice.address); + bob.flush_after(Duration::ZERO); + assert_eq!(bob.transport.peer_send_count_for(&frame.id.as_str()), 1); + + // The suppression cache saw none of it: alice's own retransmission of + // the same id is still carried, which is the black hole the chapter + // names. + assert!(!bob.protocol.mesh_relay.is_suppressed(&frame.id.as_str())); + bob.receive_from(frame.clone(), &alice.address); + bob.flush_after(Duration::ZERO); + assert_eq!( + bob.transport.peer_send_count_for(&frame.id.as_str()), + 2, + "the depositor's retransmission is forwarded, not suppressed" + ); +} + +#[test] +fn a_marked_forward_that_reaches_no_link_returns_to_the_store_uncounted() { + let (mut alice, mut bob, mut carol) = topology(open_custody()); + let frame = deposit_from(&mut alice, &bob, &carol); + bob.receive_from(frame.clone(), &alice.address); + drop_point(&mut bob); + let before = bob.protocol.mesh_relay_stats(); + + // Carol appears and is gone again before the queue is flushed. + bob.link(&carol); + bob.transport.remove_connected_peer(&carol.address); + bob.flush_after(Duration::ZERO); + bob.flush_after(RELAY_QUEUE_MAX_OVERDUE + Duration::from_millis(1)); + + let stats = bob.protocol.custody_stats(); + assert_eq!(stats.held, 1, "still held"); + assert_eq!(stats.delivered, 0); + assert_eq!(stats.re_originated, 0); + for reason in crate::protocol::custody::CustodyRefusal::ALL { + assert_eq!( + stats.refusals(*reason), + 0, + "not judged as a deposit: {reason:?}" + ); + } + assert_eq!( + bob.protocol.mesh_relay_stats().abandoned_overdue, + before.abandoned_overdue, + "a marked forward is not an abandoned forward" + ); + assert!(!bob.protocol.mesh_relay.is_suppressed(&frame.id.as_str())); + + // The recipient is always worth trying again. + bob.transport.add_connected_peer(carol.address.clone(), -55); + bob.protocol.on_neighbor_discovered(&carol.address); + bob.flush_after(Duration::ZERO); + assert_eq!(bob.protocol.custody_stats().delivered, 1); + let sends = bob.take_peer_sends(); + let (_, delivered) = sends.into_iter().find(|(_, m)| m.id == frame.id).unwrap(); + carol.receive_from(delivered, &bob.address); + assert_eq!( + read(&mut carol, &Node::space_for(&alice), "notes", "k"), + Some(DataValue::text("v")) + ); +} + +#[test] +fn a_forwarder_strips_the_request_whether_or_not_it_holds_custody() { + let (mut alice, mut bob, carol) = topology(CustodyConfig::default()); + let frame = deposit_from(&mut alice, &bob, &carol); + let dave = Node::new("dave"); + bob.link(&dave); + + bob.receive_from(frame.clone(), &alice.address); + bob.flush_after(Duration::ZERO); + + let sends = bob.take_peer_sends(); + let (to, forwarded) = sends + .into_iter() + .find(|(_, m)| m.id == frame.id) + .expect("forwarded onward"); + assert_eq!(to, dave.address); + assert!( + !forwarded.metadata.contains_key(CUSTODY_META_KEY), + "the request travels exactly one hop" + ); + assert_eq!( + forwarded.content, frame.content, + "only the outer metadata is touched" + ); + assert_eq!( + bob.protocol.custody_stats().accepted, + 0, + "a transmitted frame is never held" + ); +} + +#[test] +fn the_acceptance_table_refuses_what_the_chapter_refuses() { + // Custody off: judged, refused, counted, so "off" can be told from + // "nobody asked". + let (mut alice, mut bob, carol) = topology(CustodyConfig::default()); + let frame = deposit_from(&mut alice, &bob, &carol); + bob.receive_from(frame.clone(), &alice.address); + drop_point(&mut bob); + assert_eq!(bob.protocol.custody_stats().refused_disabled, 1); + assert!(bob.held_records().is_empty()); + assert!( + bob.take_peer_sends().is_empty(), + "nothing answers a refusal" + ); + + // Stranger tier closed, which is the default. + let (mut alice, mut bob, carol) = topology(CustodyConfig { + enabled: true, + ..CustodyConfig::default() + }); + let frame = deposit_from(&mut alice, &bob, &carol); + bob.receive_from(frame.clone(), &alice.address); + drop_point(&mut bob); + assert_eq!(bob.protocol.custody_stats().refused_stranger, 1); + + // The same peer with an established session is admitted under the + // session tier. + bob.protocol + .confirmed_sessions + .insert(alice.address.clone()); + bob.receive_from(frame.clone(), &alice.address); + drop_point(&mut bob); + assert_eq!(bob.protocol.custody_stats().accepted, 1); + + // Through a forwarder: the sender is not the arrival peer. + let (mut alice, mut bob, carol) = topology(open_custody()); + let frame = deposit_from(&mut alice, &bob, &carol); + let dave = Node::new("dave"); + bob.link(&dave); + bob.unlink(&alice); + bob.receive_from(frame.clone(), &dave.address); + // Dave is excluded as the arrival peer, alice as the sender: nowhere. + drop_point(&mut bob); + assert_eq!(bob.protocol.custody_stats().refused_not_depositor, 1); + assert_eq!(bob.protocol.custody_stats().held, 0); + + // No request on the frame at all. + let (mut alice, mut bob, carol) = topology(open_custody()); + let mut frame = deposit_from(&mut alice, &bob, &carol); + frame.metadata.remove(CUSTODY_META_KEY); + bob.receive_from(frame, &alice.address); + drop_point(&mut bob); + assert_eq!(bob.protocol.custody_stats().refused_no_request, 1); + + // An unknown class token. + let (mut alice, mut bob, carol) = topology(open_custody()); + let mut frame = deposit_from(&mut alice, &bob, &carol); + frame + .metadata + .insert(CUSTODY_META_KEY.to_string(), "media".to_string()); + bob.receive_from(frame, &alice.address); + drop_point(&mut bob); + assert_eq!(bob.protocol.custody_stats().refused_unknown_class, 1); + + // Below the soft relay floor. + let (mut alice, mut bob, carol) = topology(open_custody()); + let frame = deposit_from(&mut alice, &bob, &carol); + bob.protocol.set_device_battery(5, false); + bob.receive_from(frame.clone(), &alice.address); + // Relaying is refused outright below the floor, so the frame is never + // queued; raise the battery to queue it, then drop it before the drop + // point. + assert_eq!(bob.protocol.mesh_relay_stats().queued, 0); + bob.protocol.set_device_battery(90, false); + bob.receive_from(frame, &alice.address); + bob.protocol.set_device_battery(5, false); + drop_point(&mut bob); + assert_eq!(bob.protocol.custody_stats().refused_battery, 1); +} + +#[test] +fn a_held_frame_expires_at_the_end_of_the_hold_in_force() { + let (mut alice, mut bob, carol) = topology(open_custody()); + let frame = deposit_from(&mut alice, &bob, &carol); + bob.receive_from(frame.clone(), &alice.address); + drop_point(&mut bob); + assert_eq!(bob.protocol.custody_stats().held, 1); + // The receipt at acceptance is the last thing that leaves. + bob.take_peer_sends(); + + let hold_ms = bob.protocol.custody_config().hold_ms; + let now_ms = Utc::now().timestamp_millis(); + bob.protocol + .sweep_custody_now(Instant::now(), now_ms + hold_ms as i64 - 1); + assert_eq!(bob.protocol.custody_stats().held, 1, "inside the hold"); + + bob.protocol + .sweep_custody_now(Instant::now(), now_ms + hold_ms as i64 + 1); + let stats = bob.protocol.custody_stats(); + assert_eq!(stats.expired, 1); + assert_eq!(stats.held, 0); + assert!(bob.held_records().is_empty(), "the record goes with it"); + assert!( + bob.take_peer_sends().is_empty(), + "expiry sends nothing to anybody" + ); + // Carol appearing now finds nothing to deliver. + bob.link(&carol); + bob.flush_after(Duration::ZERO); + assert_eq!(bob.transport.peer_send_count_for(&frame.id.as_str()), 0); +} + +#[test] +fn held_frames_survive_a_relaunch_and_a_lowered_hold_expires_them_at_restore() { + let (mut alice, mut bob, mut carol) = topology(open_custody()); + let frame = deposit_from(&mut alice, &bob, &carol); + bob.receive_from(frame.clone(), &alice.address); + drop_point(&mut bob); + assert_eq!(bob.held_records().len(), 1); + + // Same configuration: restored as stored, and deliverable. + let mut bob = bob.relaunch(open_custody()); + assert_eq!(bob.protocol.custody_stats().held, 1); + assert_eq!( + bob.protocol.custody_stats().accepted, + 0, + "restore counts no acceptance" + ); + bob.link(&carol); + bob.flush_after(Duration::ZERO); + let sends = bob.take_peer_sends(); + let (_, delivered) = sends + .into_iter() + .find(|(_, m)| m.id == frame.id) + .expect("delivered after the relaunch"); + carol.receive_from(delivered, &bob.address); + assert_eq!( + read(&mut carol, &Node::space_for(&alice), "notes", "k"), + Some(DataValue::text("v")) + ); + + // A fresh deposit, then a relaunch under a hold shorter than the record's + // age: dropped at restore, record and all. + let (mut alice, mut bob, carol) = topology(open_custody()); + let frame = deposit_from(&mut alice, &bob, &carol); + bob.receive_from(frame, &alice.address); + drop_point(&mut bob); + std::thread::sleep(Duration::from_millis(5)); + let bob = bob.relaunch(CustodyConfig { + hold_ms: 1, + ..open_custody() + }); + assert_eq!(bob.protocol.custody_stats().held, 0); + assert!(bob.held_records().is_empty()); + + // A relaunch with custody disabled erases what it finds rather than + // keeping it: nothing would ever deliver it. + let (mut alice, mut bob, carol) = topology(open_custody()); + let frame = deposit_from(&mut alice, &bob, &carol); + bob.receive_from(frame, &alice.address); + drop_point(&mut bob); + let bob = bob.relaunch(CustodyConfig::default()); + assert_eq!(bob.protocol.custody_stats().held, 0); + assert!(bob.held_records().is_empty()); +} + +#[test] +fn erase_drops_every_record_and_the_data_wipe_calls_it() { + let (mut alice, mut bob, carol) = topology(open_custody()); + let frame = deposit_from(&mut alice, &bob, &carol); + bob.receive_from(frame.clone(), &alice.address); + drop_point(&mut bob); + assert_eq!(bob.held_records().len(), 1); + + bob.protocol.erase_custody().unwrap(); + let stats = bob.protocol.custody_stats(); + assert_eq!(stats.held, 0); + assert_eq!(stats.accepted, 0, "counters reset"); + assert!(bob.held_records().is_empty()); + + bob.receive_from(frame, &alice.address); + drop_point(&mut bob); + assert_eq!(bob.held_records().len(), 1); + bob.protocol.data_wipe_all().unwrap(); + assert_eq!(bob.protocol.custody_stats().held, 0); + assert!( + bob.held_records().is_empty(), + "the data-layer wipe erases custody" + ); +} + +#[test] +fn a_receipt_naming_nothing_or_arriving_unsigned_changes_nothing() { + let (mut alice, mut bob, carol) = topology(open_custody()); + let frame = deposit_from(&mut alice, &bob, &carol); + + // A receipt for an identifier the outbox does not hold: ignored, counted. + let unknown = encode_receipt(&MessageId::new().as_str(), 1_000); + let mut receipt = bob + .protocol + .create_message(&alice.address, unknown, Some(MessagePriority::Low), None) + .unwrap(); + receipt.requires_ack = false; + bob.protocol.sign_control_message(&mut receipt).unwrap(); + alice.receive_from(receipt, &bob.address); + let stats = alice.protocol.custody_stats(); + assert_eq!(stats.receipts_ignored, 1); + assert_eq!(stats.receipts_received, 0); + assert!(alice.protocol.custody_receipts.is_empty()); + + // An unsigned receipt for a real entry is refused by the control gate + // before it is read. + let mut unsigned = bob + .protocol + .create_message( + &alice.address, + encode_receipt(&frame.id.as_str(), 1_000), + Some(MessagePriority::Low), + None, + ) + .unwrap(); + unsigned.requires_ack = false; + alice.receive_from(unsigned, &bob.address); + assert_eq!(alice.protocol.custody_stats().receipts_received, 0); + assert!(alice.protocol.custody_receipts.is_empty()); + + // A malformed body past the gate: refused silently. + let mut malformed = bob + .protocol + .create_message( + &alice.address, + format!( + "{}{{\"v\":9,\"id\":\"x\"}}", + internal_prefixes::CUSTODY_RECEIPT + ), + Some(MessagePriority::Low), + None, + ) + .unwrap(); + malformed.requires_ack = false; + bob.protocol.sign_control_message(&mut malformed).unwrap(); + alice.receive_from(malformed, &bob.address); + assert_eq!(alice.protocol.custody_stats().receipts_ignored, 2); + + // No receipt without the capability entry, whatever the quotas say. + bob.protocol.peer_data_custody.remove(&alice.address); + bob.receive_from(frame, &alice.address); + drop_point(&mut bob); + assert_eq!( + bob.protocol.custody_stats().accepted, + 1, + "the deposit is still taken" + ); + assert_eq!(bob.protocol.custody_stats().receipts_sent, 0); + assert_eq!(bob.protocol.custody_stats().receipts_dropped, 1); + assert!(bob.take_peer_sends().is_empty()); +} + +#[test] +fn only_a_class_a_frame_with_retained_plaintext_carries_a_request() { + let mut alice = Node::new("alice"); + let mut carol = Node::new("carol"); + let bob = Node::new("bob"); + pair(&mut alice, &mut carol); + alice.link(&bob); + + // A direct message is sealed the same way and is never deposited. + alice + .protocol + .send_message(&carol.address, "hello", None, None::) + .unwrap(); + let sends = alice.take_peer_sends(); + assert!(!sends.is_empty(), "offered to bob"); + assert!( + sends + .iter() + .all(|(_, m)| !m.metadata.contains_key(CUSTODY_META_KEY)), + "a direct message is Class C and carries no request" + ); + + // After a relaunch the retained plaintext is gone, so the same frame is + // offered without the key. + write(&mut alice, &Node::space_for(&carol), "notes", "k", "v"); + let sends = alice.take_peer_sends(); + let (_, frame) = sends + .into_iter() + .find(|(_, m)| m.metadata.contains_key(CUSTODY_META_KEY)) + .expect("deposited while the plaintext is retained"); + let mut alice = alice.relaunch(CustodyConfig::default()); + alice.protocol.peer_data_sync.insert(carol.address.clone()); + alice.link(&bob); + let restored = alice + .protocol + .outbox + .get(&frame.id) + .expect("the outbox entry was restored") + .message + .clone(); + alice.protocol.offer_to_mesh(&restored); + let sends = alice.take_peer_sends(); + let (_, reoffer) = sends + .into_iter() + .find(|(_, m)| m.id == frame.id) + .expect("re-offered after the relaunch"); + assert!( + !reoffer.metadata.contains_key(CUSTODY_META_KEY), + "a frame whose plaintext this device no longer holds is offered without the key" + ); +} diff --git a/crates/offline-protocol/src/protocol/tests/data_sync_group.rs b/crates/offline-protocol/src/protocol/tests/data_sync_group.rs index 51fba88b5..efee2b2df 100644 --- a/crates/offline-protocol/src/protocol/tests/data_sync_group.rs +++ b/crates/offline-protocol/src/protocol/tests/data_sync_group.rs @@ -22,8 +22,8 @@ use crate::protocol::data_sync::{SyncChannel, MAX_GROUP_BLOB_CHUNKS, MAX_SYNC_BL use crate::protocol::prefixes::internal_prefixes; use crate::protocol::tests::{create_test_config_for_user, id}; use crate::protocol::types::{ - DATA_GROUP_BLOB_V1, DATA_GROUP_V1, DATA_INTEREST_V1, DATA_MEDIA_V1, DATA_SYNC_V1, - DATA_TOMBSTONE_V1, + DATA_CUSTODY_V1, DATA_GROUP_BLOB_V1, DATA_GROUP_V1, DATA_INTEREST_V1, DATA_MEDIA_V1, + DATA_SYNC_V1, DATA_TOMBSTONE_V1, }; use crate::protocol::{OfflineProtocol, TestProtocolStateStorage}; @@ -753,7 +753,8 @@ fn the_group_capability_is_advertised_and_recorded() { DATA_MEDIA_V1, DATA_TOMBSTONE_V1, DATA_INTEREST_V1, - DATA_GROUP_BLOB_V1 + DATA_GROUP_BLOB_V1, + DATA_CUSTODY_V1 ], "a build that intercepts group frames has to say so, or no peer \ will ever send it one. The media entry rides the same list and is \ diff --git a/crates/offline-protocol/src/protocol/tests/mod.rs b/crates/offline-protocol/src/protocol/tests/mod.rs index b13bfd7f3..83276c820 100644 --- a/crates/offline-protocol/src/protocol/tests/mod.rs +++ b/crates/offline-protocol/src/protocol/tests/mod.rs @@ -1,4 +1,6 @@ #[cfg(feature = "data")] +mod custody; +#[cfg(feature = "data")] mod data_layer; #[cfg(feature = "data")] mod data_sync; @@ -2938,7 +2940,8 @@ fn data_sync_is_advertised_only_when_the_layer_is_on() { DATA_MEDIA_V1, DATA_TOMBSTONE_V1, DATA_INTEREST_V1, - DATA_GROUP_BLOB_V1 + DATA_GROUP_BLOB_V1, + DATA_CUSTODY_V1 ], "every entry, and the order is append-only. Each says something a \ build advertising only its predecessors does not do: intercept a \ diff --git a/crates/offline-protocol/src/protocol/types.rs b/crates/offline-protocol/src/protocol/types.rs index eed4c2e33..d854041e8 100644 --- a/crates/offline-protocol/src/protocol/types.rs +++ b/crates/offline-protocol/src/protocol/types.rs @@ -930,6 +930,25 @@ pub(crate) const DATA_INTEREST_V1: u8 = 5; /// coming. pub(crate) const DATA_GROUP_BLOB_V1: u8 = 6; +/// The custody receipt, advertised in [`KeyPackagePayload::data_versions`] +/// alongside [`DATA_SYNC_V1`]: the peer parses the `__CUSTODY_RECEIPT__` +/// control frame, so a custodian that accepted one of its replication frames +/// may answer with a receipt (`docs/spec/custody.md`). +/// +/// It gates one frame in one direction and nothing else. A deposit request is +/// a metadata key an unaware receiver ignores, so a depositor writes it toward +/// any neighbour, and acceptance is decided by the custodian's quotas. The +/// gate exists because a peer without the entry does not know the prefix as a +/// control frame: with encryption on it refuses the receipt as inbound +/// plaintext and records a security refusal, and on a plaintext-only +/// deployment it shows the receipt to its user as a message. +/// +/// In the replication family rather than a list of its own because custody +/// carries replication frames only. No attested sibling: the receipt is a 1:1 +/// control frame between neighbours, and a group inviter has nothing to say +/// about one. +pub(crate) const DATA_CUSTODY_V1: u8 = 7; + /// Rich fields accepted by the `send_message_with` surface. Only ever /// delivered inside the sealed [`RichPayloadV1`] body — toward a recipient /// that did not advertise [`RICH_PAYLOAD_V1`] they are silently dropped, @@ -1796,6 +1815,31 @@ fn pending_decrypt_record_version() -> u8 { PENDING_DECRYPT_RECORD_VERSION } +/// One frame in custody, persisted under [`storage_keys::CUSTODY`]. +/// +/// The frame as it arrived with the deposit request removed and the hop +/// fields as the forwarding path adjusted them; the depositor, which is both +/// the frame's sender and the peer it arrived from; the wall-clock acceptance +/// time in Unix milliseconds, never a monotonic instant; and the class token. +/// `version` is the same forward-compatibility hinge as +/// [`PendingDecryptRecord::version`]. +#[derive(Serialize, Deserialize)] +pub(crate) struct CustodyRecord { + #[serde(default = "custody_record_version")] + pub(crate) version: u8, + pub(crate) depositor: String, + pub(crate) message: Message, + pub(crate) accepted_at_ms: i64, + pub(crate) class: String, +} + +/// The only custody record version this build writes or reads. +pub(crate) const CUSTODY_RECORD_VERSION: u8 = 1; + +fn custody_record_version() -> u8 { + CUSTODY_RECORD_VERSION +} + impl PendingMessage { /// Recomputes [`Self::serialized_bytes`] from the current field values. /// @@ -1944,6 +1988,14 @@ pub(crate) mod storage_keys { /// per message keyed by message id — the receive-side mirror of /// [`PENDING_MESSAGE_ENTRIES`]. See `PendingDecryptRecord`. pub const PENDING_DECRYPT_ENTRIES: &str = "pending_decrypt_entries"; + + /// Frames held in custody for a neighbour (`docs/spec/custody.md`), one + /// record per held frame keyed by its message id. Sealed like the + /// pending-decrypt records: other people's ciphertext plus routing + /// metadata about them, outliving the process. Post-split only, and + /// absent from [`ADOPTABLE_STATE_KEY_TYPES`] for the same reason as + /// [`PENDING_DECRYPT_ENTRIES`]. + pub const CUSTODY: &str = "custody_entries"; /// Key type for persisted per-peer MLS session confirmation state. pub const SESSION_STATES: &str = "session_states"; /// Key type for persisted per-peer received key packages (survives restart). diff --git a/crates/offline-protocol/tests/data/custody-receipt-v1.vectors.json b/crates/offline-protocol/tests/data/custody-receipt-v1.vectors.json new file mode 100644 index 000000000..ad24d0d17 --- /dev/null +++ b/crates/offline-protocol/tests/data/custody-receipt-v1.vectors.json @@ -0,0 +1,71 @@ +{ + "chapter": "docs/spec/custody.md", + "prefix": "__CUSTODY_RECEIPT__", + "version": 1, + "notes": [ + "A receipt is the control frame a custodian answers a deposit with: prefix, then a JSON object.", + "Fields serialize in the order v, id, hold_ms. A decoder reads v before the body and ignores unknown fields.", + "hold_ms is relative to the receipt's own timestamp. Every case below carries the prefix.", + "Computed by hand from the chapter, independently of the code that pins them." + ], + "frames": [ + { + "name": "six_hour_hold", + "id": "7f3a1c2e-4b5d-4e6f-8a9b-0c1d2e3f4a5b", + "hold_ms": 21600000, + "wire": "__CUSTODY_RECEIPT__{\"v\":1,\"id\":\"7f3a1c2e-4b5d-4e6f-8a9b-0c1d2e3f4a5b\",\"hold_ms\":21600000}" + }, + { + "name": "zero_hold", + "id": "a", + "hold_ms": 0, + "wire": "__CUSTODY_RECEIPT__{\"v\":1,\"id\":\"a\",\"hold_ms\":0}" + }, + { + "name": "largest_hold", + "id": "max", + "hold_ms": 18446744073709551615, + "wire": "__CUSTODY_RECEIPT__{\"v\":1,\"id\":\"max\",\"hold_ms\":18446744073709551615}" + } + ], + "decode_only": [ + { + "name": "unknown_field_ignored", + "wire": "__CUSTODY_RECEIPT__{\"v\":1,\"id\":\"m1\",\"hold_ms\":5,\"note\":\"a future field\"}", + "id": "m1", + "hold_ms": 5 + }, + { + "name": "field_order_is_free", + "wire": "__CUSTODY_RECEIPT__{\"hold_ms\":5,\"id\":\"m1\",\"v\":1}", + "id": "m1", + "hold_ms": 5 + } + ], + "rejects": [ + { + "name": "unknown_version", + "wire": "__CUSTODY_RECEIPT__{\"v\":2,\"id\":\"m1\",\"hold_ms\":5}" + }, + { + "name": "missing_version", + "wire": "__CUSTODY_RECEIPT__{\"id\":\"m1\",\"hold_ms\":5}" + }, + { + "name": "empty_id", + "wire": "__CUSTODY_RECEIPT__{\"v\":1,\"id\":\"\",\"hold_ms\":5}" + }, + { + "name": "not_an_object", + "wire": "__CUSTODY_RECEIPT__[1,\"m1\",5]" + }, + { + "name": "negative_hold", + "wire": "__CUSTODY_RECEIPT__{\"v\":1,\"id\":\"m1\",\"hold_ms\":-1}" + }, + { + "name": "missing_hold", + "wire": "__CUSTODY_RECEIPT__{\"v\":1,\"id\":\"m1\"}" + } + ] +} diff --git a/crates/offline-protocol/tests/mesh_forwarding.rs b/crates/offline-protocol/tests/mesh_forwarding.rs index 965adba03..bb930ec7b 100644 --- a/crates/offline-protocol/tests/mesh_forwarding.rs +++ b/crates/offline-protocol/tests/mesh_forwarding.rs @@ -14,7 +14,7 @@ //! Reach alone is easy to get by repeating everything endlessly; the counts in //! these tests are what separate a working mesh from one that floods. -use offline_protocol::{Event, OfflineProtocol, ProtocolConfig}; +use offline_protocol::{CustodyConfig, Event, OfflineProtocol, ProtocolConfig}; use offline_protocol_core::{AppId, Message, UserId}; use offline_protocol_transport::{mock::MockTransport, Transport, TransportType}; use std::collections::HashMap; @@ -32,6 +32,9 @@ struct Neighborhood { links: HashMap>, /// Every hand-off that has crossed a link, as `(from, to, message_id)`. transmissions: Vec<(String, String, String)>, + /// The frames themselves, in the same order, for a test that asserts on + /// what a device was actually handed rather than only that it was. + frames: Vec<(String, String, Message)>, /// What each device has surfaced to its app, kept because stepping the /// network is what drains it. inboxes: HashMap>, @@ -44,6 +47,7 @@ impl Neighborhood { radios: HashMap::new(), links: HashMap::new(), transmissions: Vec::new(), + frames: Vec::new(), inboxes: HashMap::new(), }; @@ -184,6 +188,8 @@ impl Neighborhood { ); self.transmissions .push((from.clone(), to.clone(), message.id.as_str())); + self.frames + .push((from.clone(), to.clone(), message.clone())); // The receiver sees which link it arrived on, as a radio reports. self.radios[&to].queue_message_from(message, from.clone()); moved += 1; @@ -820,3 +826,141 @@ fn a_frame_claiming_an_absurd_reach_is_cut_down() { onward.ttl.value() ); } + +// ============================================================================ +// Custody (docs/spec/custody.md) +// ============================================================================ + +/// A device that holds a neighbour's replication frames, admitting strangers +/// so the deposit below is judged by the quotas rather than refused at the +/// tier (these devices hold no sessions with each other). +fn custodian_config(user_id: &str) -> ProtocolConfig { + let mut config = default_config(user_id); + config.custody = CustodyConfig { + enabled: true, + stranger_max_entries: 8, + stranger_max_bytes: 512 * 1024, + ..CustodyConfig::default() + }; + config +} + +/// The frame a depositor offers into custody: its own sealed replication +/// frame with the class token on it. Built by hand because these devices +/// hold no sessions; the custodian cannot see inside a sealed frame either +/// way, and judges the outer message alone. +fn deposit(from: &str, to: &str) -> Message { + let mut frame = message( + from, + to, + &format!( + "{}opaque-ciphertext", + offline_protocol_sealed::prefixes::ENCRYPTED + ), + ); + // The reserved key from the wire-format chapter, as the depositor's + // engine writes it; pinned against the engine's constant by its own + // tests. + frame + .metadata + .insert("__custody".to_string(), "data".to_string()); + frame +} + +#[test] +fn a_deposited_frame_outlives_the_carrier_walking_away() { + // The sibling of `a_message_survives_the_carrier_walking_away`. There the + // carrier that left took the message with it and the sender's own retry + // was the recovery. Here the carrier holds custody: it keeps the frame + // past the seconds a forwarder gives it, and delivers it when the + // recipient appears, with no help from the sender at all. + let mut net = Neighborhood::new(&["alice", "carol"]); + net.add_node("bob", custodian_config("bob")); + net.link("alice", "bob"); + + let frame = deposit("alice", "carol"); + let frame_id = frame.id.as_str(); + // Alice hands it over her link to bob, as a depositor does. + net.radios["bob"].queue_message_from(frame, "alice".to_string()); + net.step(); + assert_eq!( + net.node("bob").custody_stats().held, + 0, + "a forward is not a deposit while the mesh can still carry it" + ); + + // Bob walks out of range of everyone with the frame, and nobody can take + // it from him for longer than a forwarder is willing to wait. That wait + // is the governor's overdue cut-off, five seconds of wall-clock time, + // which is why this test is slower than its neighbours. + net.unlink_all("bob"); + std::thread::sleep(std::time::Duration::from_millis(5_200)); + net.run_until_quiet(8); + + let stats = net.node("bob").custody_stats(); + assert_eq!(stats.accepted, 1, "taken into custody at the drop point"); + assert_eq!(stats.held, 1); + assert!(net.inbox("carol").is_empty()); + assert_eq!(net.deliveries_to("carol", &frame_id), 0); + + // Carol comes into range of bob, and bob alone. + net.link("bob", "carol"); + net.run_until_quiet(8); + + assert_eq!( + net.deliveries_to("carol", &frame_id), + 1, + "the custodian delivers the held frame to its recipient, once" + ); + let stats = net.node("bob").custody_stats(); + assert_eq!(stats.delivered, 1); + assert_eq!( + stats.held, 0, + "delivery to the recipient releases the frame" + ); + let (_, _, delivered) = net + .frames + .iter() + .find(|(from, to, m)| from == "bob" && to == "carol" && m.id.as_str() == frame_id) + .expect("crossed the bob-carol link"); + assert!( + !delivered.metadata.contains_key("__custody"), + "what the custodian transmits never carries the request" + ); +} + +#[test] +fn a_device_with_custody_off_still_strips_the_request_and_holds_nothing() { + // Every forwarder strips the request, custody-enabled or not: it is what + // keeps a deposit to one hop. And the default is off, so the ordinary + // mesh is exactly as it was. + let mut net = Neighborhood::new(&["alice", "bob", "dave"]); + net.link("alice", "bob"); + net.link("bob", "dave"); + + let frame = deposit("alice", "carol"); + let frame_id = frame.id.as_str(); + net.radios["bob"].queue_message_from(frame.clone(), "alice".to_string()); + net.run_until_quiet(8); + + assert_eq!( + net.deliveries_to("dave", &frame_id), + 1, + "carried on as before" + ); + let (_, _, forwarded) = net + .frames + .iter() + .find(|(from, to, m)| from == "bob" && to == "dave" && m.id.as_str() == frame_id) + .expect("crossed the bob-dave link"); + assert!( + !forwarded.metadata.contains_key("__custody"), + "the request travels exactly one hop" + ); + assert_eq!( + forwarded.content, frame.content, + "only the outer metadata is touched" + ); + assert_eq!(net.node("bob").custody_stats().held, 0); + assert_eq!(net.node("bob").custody_stats().accepted, 0); +} diff --git a/docs/spec/README.md b/docs/spec/README.md index 822e55e16..29a2b5514 100644 --- a/docs/spec/README.md +++ b/docs/spec/README.md @@ -45,6 +45,7 @@ the crate whose code they pin, so a packaged build carries its own vectors: | `crates/offline-protocol-sealed/tests/data/key-package-v1.vectors.json` | [Capability negotiation](capability-negotiation.md) | | `crates/offline-protocol-sealed/tests/data/identity-assertion-v1.vectors.json` | [Bluetooth LE framing](ble-framing.md#the-identity-assertion) | | `crates/offline-protocol/tests/data/data-sync-v1.vectors.json` | [Document replication](data-sync.md) | +| `crates/offline-protocol/tests/data/custody-receipt-v1.vectors.json` | [Custody](custody.md) | | `crates/offline-protocol-transport/tests/data/ble-framing-v1.vectors.json` | [Bluetooth LE framing](ble-framing.md) | | `crates/offline-protocol-transport/tests/data/stream-framing-v1.vectors.json` | [Peer-stream framing](stream-framing.md) | | `crates/offline-protocol-transport/tests/data/nip44.vectors.json` | None. These are the NIP-44 spec's own published vectors, vendored for the Nostr carrier's sealing and pinned to the checksum that spec publishes. Transport framing is out of scope here, so there is no chapter for them to pin | diff --git a/docs/spec/conformance.md b/docs/spec/conformance.md index ded89ecb3..1ec9194ee 100644 --- a/docs/spec/conformance.md +++ b/docs/spec/conformance.md @@ -117,6 +117,7 @@ are computed independently of that code. | `crates/offline-protocol-sealed/tests/data/key-package-v1.vectors.json` | [Capability negotiation](capability-negotiation.md) | Parse | | `crates/offline-protocol-sealed/tests/data/identity-assertion-v1.vectors.json` | [Bluetooth LE framing](ble-framing.md#the-identity-assertion) | Both | | `crates/offline-protocol/tests/data/data-sync-v1.vectors.json` | [Document replication](data-sync.md) | Both | +| `crates/offline-protocol/tests/data/custody-receipt-v1.vectors.json` | [Custody](custody.md) | Both | | `crates/offline-protocol-transport/tests/data/ble-framing-v1.vectors.json` | [Bluetooth LE framing](ble-framing.md) | Both | | `crates/offline-protocol-transport/tests/data/stream-framing-v1.vectors.json` | [Peer-stream framing](stream-framing.md) | Both | @@ -224,7 +225,3 @@ Some chapters specify behaviour that has no vector file: reads as a key problem rather than an encoding one. The rest of the chapter is a JSON message vocabulary whose mistakes surface as a rejected message, and it stays prose. -- [Custody](custody.md) adds one encoding, the receipt body, and has no - vector for it yet: the chapter precedes the codec, and a vector computed - from prose pins nothing. The vector lands with the implementation, in the - crate that holds the codec, and this list loses the entry then. diff --git a/docs/spec/custody.md b/docs/spec/custody.md index d676dcd75..69128ef59 100644 --- a/docs/spec/custody.md +++ b/docs/spec/custody.md @@ -475,11 +475,13 @@ also volunteering to store. ## Conformance -The receipt body is the one encoding this chapter adds, and it has no frozen -vector yet: the chapter precedes the codec, and a vector computed from prose -alone pins nothing. The vector lands with the implementation, in the crate that -holds the codec, and [Conformance](conformance.md) lists this chapter among -those that are not yet surfaces until then. +The receipt body is the one encoding this chapter adds. Its frozen vectors are +`crates/offline-protocol/tests/data/custody-receipt-v1.vectors.json`, in the +crate that holds the codec: three bodies to encode and decode, two that only a +decoder sees (an unknown field, and the fields in another order), and six that +MUST be refused (an unknown or missing version, an empty identifier, a body +that is not an object, a negative or missing hold). [Conformance](conformance.md) +lists the file with the others. ## What this chapter does not specify From 1357e966cbb89cb5485bfcc90f46fe6b173836eb Mon Sep 17 00:00:00 2001 From: bahdotsh Date: Thu, 1 Oct 2026 00:10:39 +0530 Subject: [PATCH 3/7] feat(uniffi,bindings): custody config, counters and erase in every binding CustodyConfig (all-optional, overlaid on the core defaults), CustodyStats, get_custody_stats() and erase_custody() over the FFI, with the section, the counters and the erase in TypeScript, Swift and Kotlin, a Foundation-only CustodyConfigReader, the C6 guard, and Python tests. Also fixes the iOS readers reading JSON 0 and 1 as booleans. --- .../offline_protocol_sdk/offline_protocol.py | 596 ++++++++++++++---- bindings/python/tests/test_custody.py | 92 +++ bindings/react-native/MeshSdk.podspec | 1 + .../offlineprotocol/OfflineProtocolModule.kt | 52 ++ .../offlineprotocol/ProtocolConfigParser.kt | 36 +- .../offline_protocol/offline_protocol.kt | 320 ++++++++++ .../ProtocolConfigParserTest.kt | 79 +++ .../ios/CustodyConfigReader.swift | 89 +++ .../ios/Generated/offline_protocol.swift | 291 ++++++++- .../ios/Generated/offline_protocolFFI.h | 22 + .../ios/MeshRelayConfigReader.swift | 8 +- .../react-native/ios/OfflineProtocolModule.m | 6 + .../ios/OfflineProtocolModule.swift | 91 ++- bindings/react-native/ios/Package.swift | 2 + .../ios/tests/CustodyConfigReaderTests.swift | 105 +++ .../tests/MeshRelayConfigReaderTests.swift | 10 + .../js-ci-harness/custody-config.test.js | 232 +++++++ bindings/react-native/node_modules | 1 + bindings/react-native/package.json | 2 +- bindings/react-native/src/index.ts | 51 ++ bindings/react-native/src/types.ts | 108 ++++ crates/offline-protocol-uniffi/src/lib.rs | 544 +++++++++++++++- .../src/offline_protocol.udl | 107 ++++ 23 files changed, 2718 insertions(+), 127 deletions(-) create mode 100644 bindings/python/tests/test_custody.py create mode 100644 bindings/react-native/ios/CustodyConfigReader.swift create mode 100644 bindings/react-native/ios/tests/CustodyConfigReaderTests.swift create mode 100644 bindings/react-native/js-ci-harness/custody-config.test.js create mode 120000 bindings/react-native/node_modules diff --git a/bindings/python/offline_protocol_sdk/offline_protocol.py b/bindings/python/offline_protocol_sdk/offline_protocol.py index 10e059a3f..2ccf26e5e 100644 --- a/bindings/python/offline_protocol_sdk/offline_protocol.py +++ b/bindings/python/offline_protocol_sdk/offline_protocol.py @@ -641,6 +641,8 @@ def _uniffi_check_api_checksums(lib): raise InternalError("UniFFI API checksum mismatch: try cleaning and rebuilding your project") if lib.uniffi_offline_protocol_uniffi_checksum_method_offlineprotocol_end_telemetry_session() != 31162: raise InternalError("UniFFI API checksum mismatch: try cleaning and rebuilding your project") + if lib.uniffi_offline_protocol_uniffi_checksum_method_offlineprotocol_erase_custody() != 61003: + raise InternalError("UniFFI API checksum mismatch: try cleaning and rebuilding your project") if lib.uniffi_offline_protocol_uniffi_checksum_method_offlineprotocol_establish_secure_session() != 56452: raise InternalError("UniFFI API checksum mismatch: try cleaning and rebuilding your project") if lib.uniffi_offline_protocol_uniffi_checksum_method_offlineprotocol_finalize_file() != 63518: @@ -663,6 +665,8 @@ def _uniffi_check_api_checksums(lib): raise InternalError("UniFFI API checksum mismatch: try cleaning and rebuilding your project") if lib.uniffi_offline_protocol_uniffi_checksum_method_offlineprotocol_get_blocked_users() != 56869: raise InternalError("UniFFI API checksum mismatch: try cleaning and rebuilding your project") + if lib.uniffi_offline_protocol_uniffi_checksum_method_offlineprotocol_get_custody_stats() != 53054: + raise InternalError("UniFFI API checksum mismatch: try cleaning and rebuilding your project") if lib.uniffi_offline_protocol_uniffi_checksum_method_offlineprotocol_get_dedup_stats() != 26483: raise InternalError("UniFFI API checksum mismatch: try cleaning and rebuilding your project") if lib.uniffi_offline_protocol_uniffi_checksum_method_offlineprotocol_get_delivery_success_rate() != 63625: @@ -1823,6 +1827,11 @@ class _UniffiVTableCallbackInterfaceOfflineProtocolWifiDirectTransportCallback(c ctypes.POINTER(_UniffiRustCallStatus), ) _UniffiLib.uniffi_offline_protocol_uniffi_fn_method_offlineprotocol_end_telemetry_session.restype = None +_UniffiLib.uniffi_offline_protocol_uniffi_fn_method_offlineprotocol_erase_custody.argtypes = ( + ctypes.c_uint64, + ctypes.POINTER(_UniffiRustCallStatus), +) +_UniffiLib.uniffi_offline_protocol_uniffi_fn_method_offlineprotocol_erase_custody.restype = None _UniffiLib.uniffi_offline_protocol_uniffi_fn_method_offlineprotocol_establish_secure_session.argtypes = ( ctypes.c_uint64, _UniffiRustBuffer, @@ -1889,6 +1898,11 @@ class _UniffiVTableCallbackInterfaceOfflineProtocolWifiDirectTransportCallback(c ctypes.POINTER(_UniffiRustCallStatus), ) _UniffiLib.uniffi_offline_protocol_uniffi_fn_method_offlineprotocol_get_blocked_users.restype = _UniffiRustBuffer +_UniffiLib.uniffi_offline_protocol_uniffi_fn_method_offlineprotocol_get_custody_stats.argtypes = ( + ctypes.c_uint64, + ctypes.POINTER(_UniffiRustCallStatus), +) +_UniffiLib.uniffi_offline_protocol_uniffi_fn_method_offlineprotocol_get_custody_stats.restype = _UniffiRustBuffer _UniffiLib.uniffi_offline_protocol_uniffi_fn_method_offlineprotocol_get_dedup_stats.argtypes = ( ctypes.c_uint64, ctypes.POINTER(_UniffiRustCallStatus), @@ -2952,6 +2966,9 @@ class _UniffiVTableCallbackInterfaceOfflineProtocolWifiDirectTransportCallback(c _UniffiLib.uniffi_offline_protocol_uniffi_checksum_method_offlineprotocol_end_telemetry_session.argtypes = ( ) _UniffiLib.uniffi_offline_protocol_uniffi_checksum_method_offlineprotocol_end_telemetry_session.restype = ctypes.c_uint16 +_UniffiLib.uniffi_offline_protocol_uniffi_checksum_method_offlineprotocol_erase_custody.argtypes = ( +) +_UniffiLib.uniffi_offline_protocol_uniffi_checksum_method_offlineprotocol_erase_custody.restype = ctypes.c_uint16 _UniffiLib.uniffi_offline_protocol_uniffi_checksum_method_offlineprotocol_establish_secure_session.argtypes = ( ) _UniffiLib.uniffi_offline_protocol_uniffi_checksum_method_offlineprotocol_establish_secure_session.restype = ctypes.c_uint16 @@ -2985,6 +3002,9 @@ class _UniffiVTableCallbackInterfaceOfflineProtocolWifiDirectTransportCallback(c _UniffiLib.uniffi_offline_protocol_uniffi_checksum_method_offlineprotocol_get_blocked_users.argtypes = ( ) _UniffiLib.uniffi_offline_protocol_uniffi_checksum_method_offlineprotocol_get_blocked_users.restype = ctypes.c_uint16 +_UniffiLib.uniffi_offline_protocol_uniffi_checksum_method_offlineprotocol_get_custody_stats.argtypes = ( +) +_UniffiLib.uniffi_offline_protocol_uniffi_checksum_method_offlineprotocol_get_custody_stats.restype = ctypes.c_uint16 _UniffiLib.uniffi_offline_protocol_uniffi_checksum_method_offlineprotocol_get_dedup_stats.argtypes = ( ) _UniffiLib.uniffi_offline_protocol_uniffi_checksum_method_offlineprotocol_get_dedup_stats.restype = ctypes.c_uint16 @@ -3570,6 +3590,403 @@ def write(value, buf): _UniffiFfiConverterString.write(value.recipient_id, buf) _UniffiFfiConverterSequenceUInt8.write(value.data, buf) +class _UniffiFfiConverterBoolean: + @classmethod + def check_lower(cls, value): + return not not value + + @classmethod + def lower(cls, value): + return 1 if value else 0 + + @staticmethod + def lift(value): + return value != 0 + + @classmethod + def read(cls, buf): + return cls.lift(buf.read_u8()) + + @classmethod + def write(cls, value, buf): + buf.write_u8(value) + +class _UniffiFfiConverterOptionalBoolean(_UniffiConverterRustBuffer): + @classmethod + def check_lower(cls, value): + if value is not None: + _UniffiFfiConverterBoolean.check_lower(value) + + @classmethod + def write(cls, value, buf): + if value is None: + buf.write_u8(0) + return + + buf.write_u8(1) + _UniffiFfiConverterBoolean.write(value, buf) + + @classmethod + def read(cls, buf): + flag = buf.read_u8() + if flag == 0: + return None + elif flag == 1: + return _UniffiFfiConverterBoolean.read(buf) + else: + raise InternalError("Unexpected flag byte for optional type") + +class _UniffiFfiConverterOptionalUInt64(_UniffiConverterRustBuffer): + @classmethod + def check_lower(cls, value): + if value is not None: + _UniffiFfiConverterUInt64.check_lower(value) + + @classmethod + def write(cls, value, buf): + if value is None: + buf.write_u8(0) + return + + buf.write_u8(1) + _UniffiFfiConverterUInt64.write(value, buf) + + @classmethod + def read(cls, buf): + flag = buf.read_u8() + if flag == 0: + return None + elif flag == 1: + return _UniffiFfiConverterUInt64.read(buf) + else: + raise InternalError("Unexpected flag byte for optional type") + + + + + + +class OverflowPolicy(enum.Enum): + + DROP_OLDEST = 0 + + DROP_NEWEST = 1 + + + +class _UniffiFfiConverterTypeOverflowPolicy(_UniffiConverterRustBuffer): + @staticmethod + def read(buf): + variant = buf.read_i32() + if variant == 1: + return OverflowPolicy.DROP_OLDEST + if variant == 2: + return OverflowPolicy.DROP_NEWEST + raise InternalError("Raw enum value doesn't match any cases") + + @staticmethod + def check_lower(value): + if value == OverflowPolicy.DROP_OLDEST: + return + if value == OverflowPolicy.DROP_NEWEST: + return + raise ValueError(value) + + @staticmethod + def write(value, buf): + if value == OverflowPolicy.DROP_OLDEST: + buf.write_i32(1) + if value == OverflowPolicy.DROP_NEWEST: + buf.write_i32(2) + + + +class _UniffiFfiConverterOptionalTypeOverflowPolicy(_UniffiConverterRustBuffer): + @classmethod + def check_lower(cls, value): + if value is not None: + _UniffiFfiConverterTypeOverflowPolicy.check_lower(value) + + @classmethod + def write(cls, value, buf): + if value is None: + buf.write_u8(0) + return + + buf.write_u8(1) + _UniffiFfiConverterTypeOverflowPolicy.write(value, buf) + + @classmethod + def read(cls, buf): + flag = buf.read_u8() + if flag == 0: + return None + elif flag == 1: + return _UniffiFfiConverterTypeOverflowPolicy.read(buf) + else: + raise InternalError("Unexpected flag byte for optional type") + +@dataclass +class CustodyConfig: + def __init__(self, *, enabled:typing.Optional[bool] = _DEFAULT, hold_ms:typing.Optional[int] = _DEFAULT, max_entries_per_depositor:typing.Optional[int] = _DEFAULT, max_bytes_per_depositor:typing.Optional[int] = _DEFAULT, max_entries:typing.Optional[int] = _DEFAULT, max_bytes:typing.Optional[int] = _DEFAULT, stranger_max_entries:typing.Optional[int] = _DEFAULT, stranger_max_bytes:typing.Optional[int] = _DEFAULT, overflow_policy:typing.Optional[OverflowPolicy] = _DEFAULT): + if enabled is _DEFAULT: + self.enabled = None + else: + self.enabled = enabled + if hold_ms is _DEFAULT: + self.hold_ms = None + else: + self.hold_ms = hold_ms + if max_entries_per_depositor is _DEFAULT: + self.max_entries_per_depositor = None + else: + self.max_entries_per_depositor = max_entries_per_depositor + if max_bytes_per_depositor is _DEFAULT: + self.max_bytes_per_depositor = None + else: + self.max_bytes_per_depositor = max_bytes_per_depositor + if max_entries is _DEFAULT: + self.max_entries = None + else: + self.max_entries = max_entries + if max_bytes is _DEFAULT: + self.max_bytes = None + else: + self.max_bytes = max_bytes + if stranger_max_entries is _DEFAULT: + self.stranger_max_entries = None + else: + self.stranger_max_entries = stranger_max_entries + if stranger_max_bytes is _DEFAULT: + self.stranger_max_bytes = None + else: + self.stranger_max_bytes = stranger_max_bytes + if overflow_policy is _DEFAULT: + self.overflow_policy = None + else: + self.overflow_policy = overflow_policy + + + + + def __str__(self): + return "CustodyConfig(enabled={}, hold_ms={}, max_entries_per_depositor={}, max_bytes_per_depositor={}, max_entries={}, max_bytes={}, stranger_max_entries={}, stranger_max_bytes={}, overflow_policy={})".format(self.enabled, self.hold_ms, self.max_entries_per_depositor, self.max_bytes_per_depositor, self.max_entries, self.max_bytes, self.stranger_max_entries, self.stranger_max_bytes, self.overflow_policy) + def __eq__(self, other): + if self.enabled != other.enabled: + return False + if self.hold_ms != other.hold_ms: + return False + if self.max_entries_per_depositor != other.max_entries_per_depositor: + return False + if self.max_bytes_per_depositor != other.max_bytes_per_depositor: + return False + if self.max_entries != other.max_entries: + return False + if self.max_bytes != other.max_bytes: + return False + if self.stranger_max_entries != other.stranger_max_entries: + return False + if self.stranger_max_bytes != other.stranger_max_bytes: + return False + if self.overflow_policy != other.overflow_policy: + return False + return True + +class _UniffiFfiConverterTypeCustodyConfig(_UniffiConverterRustBuffer): + @staticmethod + def read(buf): + return CustodyConfig( + enabled=_UniffiFfiConverterOptionalBoolean.read(buf), + hold_ms=_UniffiFfiConverterOptionalUInt64.read(buf), + max_entries_per_depositor=_UniffiFfiConverterOptionalUInt64.read(buf), + max_bytes_per_depositor=_UniffiFfiConverterOptionalUInt64.read(buf), + max_entries=_UniffiFfiConverterOptionalUInt64.read(buf), + max_bytes=_UniffiFfiConverterOptionalUInt64.read(buf), + stranger_max_entries=_UniffiFfiConverterOptionalUInt64.read(buf), + stranger_max_bytes=_UniffiFfiConverterOptionalUInt64.read(buf), + overflow_policy=_UniffiFfiConverterOptionalTypeOverflowPolicy.read(buf), + ) + + @staticmethod + def check_lower(value): + _UniffiFfiConverterOptionalBoolean.check_lower(value.enabled) + _UniffiFfiConverterOptionalUInt64.check_lower(value.hold_ms) + _UniffiFfiConverterOptionalUInt64.check_lower(value.max_entries_per_depositor) + _UniffiFfiConverterOptionalUInt64.check_lower(value.max_bytes_per_depositor) + _UniffiFfiConverterOptionalUInt64.check_lower(value.max_entries) + _UniffiFfiConverterOptionalUInt64.check_lower(value.max_bytes) + _UniffiFfiConverterOptionalUInt64.check_lower(value.stranger_max_entries) + _UniffiFfiConverterOptionalUInt64.check_lower(value.stranger_max_bytes) + _UniffiFfiConverterOptionalTypeOverflowPolicy.check_lower(value.overflow_policy) + + @staticmethod + def write(value, buf): + _UniffiFfiConverterOptionalBoolean.write(value.enabled, buf) + _UniffiFfiConverterOptionalUInt64.write(value.hold_ms, buf) + _UniffiFfiConverterOptionalUInt64.write(value.max_entries_per_depositor, buf) + _UniffiFfiConverterOptionalUInt64.write(value.max_bytes_per_depositor, buf) + _UniffiFfiConverterOptionalUInt64.write(value.max_entries, buf) + _UniffiFfiConverterOptionalUInt64.write(value.max_bytes, buf) + _UniffiFfiConverterOptionalUInt64.write(value.stranger_max_entries, buf) + _UniffiFfiConverterOptionalUInt64.write(value.stranger_max_bytes, buf) + _UniffiFfiConverterOptionalTypeOverflowPolicy.write(value.overflow_policy, buf) + +@dataclass +class CustodyStats: + def __init__(self, *, held:int, held_bytes:int, accepted:int, delivered:int, re_originated:int, expired:int, duplicates:int, evicted:int, receipts_sent:int, receipts_dropped:int, receipts_received:int, receipts_ignored:int, refused_disabled:int, refused_no_request:int, refused_unknown_class:int, refused_not_sealed:int, refused_unproven_peer:int, refused_not_depositor:int, refused_stranger:int, refused_depositor_full:int, refused_store_full:int, refused_battery:int): + self.held = held + self.held_bytes = held_bytes + self.accepted = accepted + self.delivered = delivered + self.re_originated = re_originated + self.expired = expired + self.duplicates = duplicates + self.evicted = evicted + self.receipts_sent = receipts_sent + self.receipts_dropped = receipts_dropped + self.receipts_received = receipts_received + self.receipts_ignored = receipts_ignored + self.refused_disabled = refused_disabled + self.refused_no_request = refused_no_request + self.refused_unknown_class = refused_unknown_class + self.refused_not_sealed = refused_not_sealed + self.refused_unproven_peer = refused_unproven_peer + self.refused_not_depositor = refused_not_depositor + self.refused_stranger = refused_stranger + self.refused_depositor_full = refused_depositor_full + self.refused_store_full = refused_store_full + self.refused_battery = refused_battery + + + + + def __str__(self): + return "CustodyStats(held={}, held_bytes={}, accepted={}, delivered={}, re_originated={}, expired={}, duplicates={}, evicted={}, receipts_sent={}, receipts_dropped={}, receipts_received={}, receipts_ignored={}, refused_disabled={}, refused_no_request={}, refused_unknown_class={}, refused_not_sealed={}, refused_unproven_peer={}, refused_not_depositor={}, refused_stranger={}, refused_depositor_full={}, refused_store_full={}, refused_battery={})".format(self.held, self.held_bytes, self.accepted, self.delivered, self.re_originated, self.expired, self.duplicates, self.evicted, self.receipts_sent, self.receipts_dropped, self.receipts_received, self.receipts_ignored, self.refused_disabled, self.refused_no_request, self.refused_unknown_class, self.refused_not_sealed, self.refused_unproven_peer, self.refused_not_depositor, self.refused_stranger, self.refused_depositor_full, self.refused_store_full, self.refused_battery) + def __eq__(self, other): + if self.held != other.held: + return False + if self.held_bytes != other.held_bytes: + return False + if self.accepted != other.accepted: + return False + if self.delivered != other.delivered: + return False + if self.re_originated != other.re_originated: + return False + if self.expired != other.expired: + return False + if self.duplicates != other.duplicates: + return False + if self.evicted != other.evicted: + return False + if self.receipts_sent != other.receipts_sent: + return False + if self.receipts_dropped != other.receipts_dropped: + return False + if self.receipts_received != other.receipts_received: + return False + if self.receipts_ignored != other.receipts_ignored: + return False + if self.refused_disabled != other.refused_disabled: + return False + if self.refused_no_request != other.refused_no_request: + return False + if self.refused_unknown_class != other.refused_unknown_class: + return False + if self.refused_not_sealed != other.refused_not_sealed: + return False + if self.refused_unproven_peer != other.refused_unproven_peer: + return False + if self.refused_not_depositor != other.refused_not_depositor: + return False + if self.refused_stranger != other.refused_stranger: + return False + if self.refused_depositor_full != other.refused_depositor_full: + return False + if self.refused_store_full != other.refused_store_full: + return False + if self.refused_battery != other.refused_battery: + return False + return True + +class _UniffiFfiConverterTypeCustodyStats(_UniffiConverterRustBuffer): + @staticmethod + def read(buf): + return CustodyStats( + held=_UniffiFfiConverterUInt64.read(buf), + held_bytes=_UniffiFfiConverterUInt64.read(buf), + accepted=_UniffiFfiConverterUInt64.read(buf), + delivered=_UniffiFfiConverterUInt64.read(buf), + re_originated=_UniffiFfiConverterUInt64.read(buf), + expired=_UniffiFfiConverterUInt64.read(buf), + duplicates=_UniffiFfiConverterUInt64.read(buf), + evicted=_UniffiFfiConverterUInt64.read(buf), + receipts_sent=_UniffiFfiConverterUInt64.read(buf), + receipts_dropped=_UniffiFfiConverterUInt64.read(buf), + receipts_received=_UniffiFfiConverterUInt64.read(buf), + receipts_ignored=_UniffiFfiConverterUInt64.read(buf), + refused_disabled=_UniffiFfiConverterUInt64.read(buf), + refused_no_request=_UniffiFfiConverterUInt64.read(buf), + refused_unknown_class=_UniffiFfiConverterUInt64.read(buf), + refused_not_sealed=_UniffiFfiConverterUInt64.read(buf), + refused_unproven_peer=_UniffiFfiConverterUInt64.read(buf), + refused_not_depositor=_UniffiFfiConverterUInt64.read(buf), + refused_stranger=_UniffiFfiConverterUInt64.read(buf), + refused_depositor_full=_UniffiFfiConverterUInt64.read(buf), + refused_store_full=_UniffiFfiConverterUInt64.read(buf), + refused_battery=_UniffiFfiConverterUInt64.read(buf), + ) + + @staticmethod + def check_lower(value): + _UniffiFfiConverterUInt64.check_lower(value.held) + _UniffiFfiConverterUInt64.check_lower(value.held_bytes) + _UniffiFfiConverterUInt64.check_lower(value.accepted) + _UniffiFfiConverterUInt64.check_lower(value.delivered) + _UniffiFfiConverterUInt64.check_lower(value.re_originated) + _UniffiFfiConverterUInt64.check_lower(value.expired) + _UniffiFfiConverterUInt64.check_lower(value.duplicates) + _UniffiFfiConverterUInt64.check_lower(value.evicted) + _UniffiFfiConverterUInt64.check_lower(value.receipts_sent) + _UniffiFfiConverterUInt64.check_lower(value.receipts_dropped) + _UniffiFfiConverterUInt64.check_lower(value.receipts_received) + _UniffiFfiConverterUInt64.check_lower(value.receipts_ignored) + _UniffiFfiConverterUInt64.check_lower(value.refused_disabled) + _UniffiFfiConverterUInt64.check_lower(value.refused_no_request) + _UniffiFfiConverterUInt64.check_lower(value.refused_unknown_class) + _UniffiFfiConverterUInt64.check_lower(value.refused_not_sealed) + _UniffiFfiConverterUInt64.check_lower(value.refused_unproven_peer) + _UniffiFfiConverterUInt64.check_lower(value.refused_not_depositor) + _UniffiFfiConverterUInt64.check_lower(value.refused_stranger) + _UniffiFfiConverterUInt64.check_lower(value.refused_depositor_full) + _UniffiFfiConverterUInt64.check_lower(value.refused_store_full) + _UniffiFfiConverterUInt64.check_lower(value.refused_battery) + + @staticmethod + def write(value, buf): + _UniffiFfiConverterUInt64.write(value.held, buf) + _UniffiFfiConverterUInt64.write(value.held_bytes, buf) + _UniffiFfiConverterUInt64.write(value.accepted, buf) + _UniffiFfiConverterUInt64.write(value.delivered, buf) + _UniffiFfiConverterUInt64.write(value.re_originated, buf) + _UniffiFfiConverterUInt64.write(value.expired, buf) + _UniffiFfiConverterUInt64.write(value.duplicates, buf) + _UniffiFfiConverterUInt64.write(value.evicted, buf) + _UniffiFfiConverterUInt64.write(value.receipts_sent, buf) + _UniffiFfiConverterUInt64.write(value.receipts_dropped, buf) + _UniffiFfiConverterUInt64.write(value.receipts_received, buf) + _UniffiFfiConverterUInt64.write(value.receipts_ignored, buf) + _UniffiFfiConverterUInt64.write(value.refused_disabled, buf) + _UniffiFfiConverterUInt64.write(value.refused_no_request, buf) + _UniffiFfiConverterUInt64.write(value.refused_unknown_class, buf) + _UniffiFfiConverterUInt64.write(value.refused_not_sealed, buf) + _UniffiFfiConverterUInt64.write(value.refused_unproven_peer, buf) + _UniffiFfiConverterUInt64.write(value.refused_not_depositor, buf) + _UniffiFfiConverterUInt64.write(value.refused_stranger, buf) + _UniffiFfiConverterUInt64.write(value.refused_depositor_full, buf) + _UniffiFfiConverterUInt64.write(value.refused_store_full, buf) + _UniffiFfiConverterUInt64.write(value.refused_battery, buf) + @dataclass class DedupConfig: def __init__(self, *, max_tracked_messages:int, retention_time_secs:int): @@ -3654,27 +4071,6 @@ def write(value, buf): _UniffiFfiConverterUInt8.write(value.capacity_used_percent, buf) _UniffiFfiConverterString.write(value.mode, buf) -class _UniffiFfiConverterBoolean: - @classmethod - def check_lower(cls, value): - return not not value - - @classmethod - def lower(cls, value): - return 1 if value else 0 - - @staticmethod - def lift(value): - return value != 0 - - @classmethod - def read(cls, buf): - return cls.lift(buf.read_u8()) - - @classmethod - def write(cls, value, buf): - buf.write_u8(value) - class _UniffiFfiConverterFloat32(_UniffiConverterPrimitiveFloat): @staticmethod def read(buf): @@ -3842,46 +4238,6 @@ def write(value, buf): _UniffiFfiConverterUInt8.write(value.relay_min_battery_level, buf) _UniffiFfiConverterUInt8.write(value.relay_optimal_connection_count, buf) - - - - - -class OverflowPolicy(enum.Enum): - - DROP_OLDEST = 0 - - DROP_NEWEST = 1 - - - -class _UniffiFfiConverterTypeOverflowPolicy(_UniffiConverterRustBuffer): - @staticmethod - def read(buf): - variant = buf.read_i32() - if variant == 1: - return OverflowPolicy.DROP_OLDEST - if variant == 2: - return OverflowPolicy.DROP_NEWEST - raise InternalError("Raw enum value doesn't match any cases") - - @staticmethod - def check_lower(value): - if value == OverflowPolicy.DROP_OLDEST: - return - if value == OverflowPolicy.DROP_NEWEST: - return - raise ValueError(value) - - @staticmethod - def write(value, buf): - if value == OverflowPolicy.DROP_OLDEST: - buf.write_i32(1) - if value == OverflowPolicy.DROP_NEWEST: - buf.write_i32(2) - - - @dataclass class PendingQueueConfig: def __init__(self, *, max_pending_per_peer:int, max_pending_global:int, pending_ttl_ms:int, overflow_policy:OverflowPolicy): @@ -4327,31 +4683,6 @@ def write(value, buf): _UniffiFfiConverterOptionalString.write(value.petname, buf) _UniffiFfiConverterBoolean.write(value.signed, buf) -class _UniffiFfiConverterOptionalUInt64(_UniffiConverterRustBuffer): - @classmethod - def check_lower(cls, value): - if value is not None: - _UniffiFfiConverterUInt64.check_lower(value) - - @classmethod - def write(cls, value, buf): - if value is None: - buf.write_u8(0) - return - - buf.write_u8(1) - _UniffiFfiConverterUInt64.write(value, buf) - - @classmethod - def read(cls, buf): - flag = buf.read_u8() - if flag == 0: - return None - elif flag == 1: - return _UniffiFfiConverterUInt64.read(buf) - else: - raise InternalError("Unexpected flag byte for optional type") - class _UniffiFfiConverterOptionalUInt32(_UniffiConverterRustBuffer): @classmethod def check_lower(cls, value): @@ -5933,9 +6264,34 @@ def read(cls, buf): else: raise InternalError("Unexpected flag byte for optional type") +class _UniffiFfiConverterOptionalTypeCustodyConfig(_UniffiConverterRustBuffer): + @classmethod + def check_lower(cls, value): + if value is not None: + _UniffiFfiConverterTypeCustodyConfig.check_lower(value) + + @classmethod + def write(cls, value, buf): + if value is None: + buf.write_u8(0) + return + + buf.write_u8(1) + _UniffiFfiConverterTypeCustodyConfig.write(value, buf) + + @classmethod + def read(cls, buf): + flag = buf.read_u8() + if flag == 0: + return None + elif flag == 1: + return _UniffiFfiConverterTypeCustodyConfig.read(buf) + else: + raise InternalError("Unexpected flag byte for optional type") + @dataclass class ProtocolConfig: - def __init__(self, *, app_id:str, profile:str, ble_enabled:bool, wifi_direct_enabled:bool, internet_enabled:bool, reticulum_enabled:bool, nostr_enabled:bool, prefer_online:bool, initial_ttl:int, encryption_enabled:bool, auto_key_exchange:bool, store_pending:bool, require_encryption:bool = True, max_pending_per_peer:int, max_pending_global:int, pending_ttl_ms:int, overflow_policy:OverflowPolicy, edge_driven_unreachable_dm:bool = False, max_group_members:int = 256, group_relay_enabled:bool = True, group_relay_broadcast_enabled:bool = True, group_enforce_admin_commits:bool = False, require_transport_identity:bool = False, binary_wire_enabled:bool = True, nostr_sealing_enabled:bool = True, nostr_cold_contact_enabled:bool = True, nostr_username_discovery_enabled:bool = False, compact_envelope_enabled:bool = True, rich_payload_enabled:bool = True, crypto_recovery_enabled:bool = True, mesh_relay:typing.Optional[MeshRelayConfig] = _DEFAULT, data_enabled:bool = True, control_freshness_enforced:bool = True): + def __init__(self, *, app_id:str, profile:str, ble_enabled:bool, wifi_direct_enabled:bool, internet_enabled:bool, reticulum_enabled:bool, nostr_enabled:bool, prefer_online:bool, initial_ttl:int, encryption_enabled:bool, auto_key_exchange:bool, store_pending:bool, require_encryption:bool = True, max_pending_per_peer:int, max_pending_global:int, pending_ttl_ms:int, overflow_policy:OverflowPolicy, edge_driven_unreachable_dm:bool = False, max_group_members:int = 256, group_relay_enabled:bool = True, group_relay_broadcast_enabled:bool = True, group_enforce_admin_commits:bool = False, require_transport_identity:bool = False, binary_wire_enabled:bool = True, nostr_sealing_enabled:bool = True, nostr_cold_contact_enabled:bool = True, nostr_username_discovery_enabled:bool = False, compact_envelope_enabled:bool = True, rich_payload_enabled:bool = True, crypto_recovery_enabled:bool = True, mesh_relay:typing.Optional[MeshRelayConfig] = _DEFAULT, custody:typing.Optional[CustodyConfig] = _DEFAULT, data_enabled:bool = True, control_freshness_enforced:bool = True): self.app_id = app_id self.profile = profile self.ble_enabled = ble_enabled @@ -5970,6 +6326,10 @@ def __init__(self, *, app_id:str, profile:str, ble_enabled:bool, wifi_direct_ena self.mesh_relay = None else: self.mesh_relay = mesh_relay + if custody is _DEFAULT: + self.custody = None + else: + self.custody = custody self.data_enabled = data_enabled self.control_freshness_enforced = control_freshness_enforced @@ -5977,7 +6337,7 @@ def __init__(self, *, app_id:str, profile:str, ble_enabled:bool, wifi_direct_ena def __str__(self): - return "ProtocolConfig(app_id={}, profile={}, ble_enabled={}, wifi_direct_enabled={}, internet_enabled={}, reticulum_enabled={}, nostr_enabled={}, prefer_online={}, initial_ttl={}, encryption_enabled={}, auto_key_exchange={}, store_pending={}, require_encryption={}, max_pending_per_peer={}, max_pending_global={}, pending_ttl_ms={}, overflow_policy={}, edge_driven_unreachable_dm={}, max_group_members={}, group_relay_enabled={}, group_relay_broadcast_enabled={}, group_enforce_admin_commits={}, require_transport_identity={}, binary_wire_enabled={}, nostr_sealing_enabled={}, nostr_cold_contact_enabled={}, nostr_username_discovery_enabled={}, compact_envelope_enabled={}, rich_payload_enabled={}, crypto_recovery_enabled={}, mesh_relay={}, data_enabled={}, control_freshness_enforced={})".format(self.app_id, self.profile, self.ble_enabled, self.wifi_direct_enabled, self.internet_enabled, self.reticulum_enabled, self.nostr_enabled, self.prefer_online, self.initial_ttl, self.encryption_enabled, self.auto_key_exchange, self.store_pending, self.require_encryption, self.max_pending_per_peer, self.max_pending_global, self.pending_ttl_ms, self.overflow_policy, self.edge_driven_unreachable_dm, self.max_group_members, self.group_relay_enabled, self.group_relay_broadcast_enabled, self.group_enforce_admin_commits, self.require_transport_identity, self.binary_wire_enabled, self.nostr_sealing_enabled, self.nostr_cold_contact_enabled, self.nostr_username_discovery_enabled, self.compact_envelope_enabled, self.rich_payload_enabled, self.crypto_recovery_enabled, self.mesh_relay, self.data_enabled, self.control_freshness_enforced) + return "ProtocolConfig(app_id={}, profile={}, ble_enabled={}, wifi_direct_enabled={}, internet_enabled={}, reticulum_enabled={}, nostr_enabled={}, prefer_online={}, initial_ttl={}, encryption_enabled={}, auto_key_exchange={}, store_pending={}, require_encryption={}, max_pending_per_peer={}, max_pending_global={}, pending_ttl_ms={}, overflow_policy={}, edge_driven_unreachable_dm={}, max_group_members={}, group_relay_enabled={}, group_relay_broadcast_enabled={}, group_enforce_admin_commits={}, require_transport_identity={}, binary_wire_enabled={}, nostr_sealing_enabled={}, nostr_cold_contact_enabled={}, nostr_username_discovery_enabled={}, compact_envelope_enabled={}, rich_payload_enabled={}, crypto_recovery_enabled={}, mesh_relay={}, custody={}, data_enabled={}, control_freshness_enforced={})".format(self.app_id, self.profile, self.ble_enabled, self.wifi_direct_enabled, self.internet_enabled, self.reticulum_enabled, self.nostr_enabled, self.prefer_online, self.initial_ttl, self.encryption_enabled, self.auto_key_exchange, self.store_pending, self.require_encryption, self.max_pending_per_peer, self.max_pending_global, self.pending_ttl_ms, self.overflow_policy, self.edge_driven_unreachable_dm, self.max_group_members, self.group_relay_enabled, self.group_relay_broadcast_enabled, self.group_enforce_admin_commits, self.require_transport_identity, self.binary_wire_enabled, self.nostr_sealing_enabled, self.nostr_cold_contact_enabled, self.nostr_username_discovery_enabled, self.compact_envelope_enabled, self.rich_payload_enabled, self.crypto_recovery_enabled, self.mesh_relay, self.custody, self.data_enabled, self.control_freshness_enforced) def __eq__(self, other): if self.app_id != other.app_id: return False @@ -6041,6 +6401,8 @@ def __eq__(self, other): return False if self.mesh_relay != other.mesh_relay: return False + if self.custody != other.custody: + return False if self.data_enabled != other.data_enabled: return False if self.control_freshness_enforced != other.control_freshness_enforced: @@ -6082,6 +6444,7 @@ def read(buf): rich_payload_enabled=_UniffiFfiConverterBoolean.read(buf), crypto_recovery_enabled=_UniffiFfiConverterBoolean.read(buf), mesh_relay=_UniffiFfiConverterOptionalTypeMeshRelayConfig.read(buf), + custody=_UniffiFfiConverterOptionalTypeCustodyConfig.read(buf), data_enabled=_UniffiFfiConverterBoolean.read(buf), control_freshness_enforced=_UniffiFfiConverterBoolean.read(buf), ) @@ -6119,6 +6482,7 @@ def check_lower(value): _UniffiFfiConverterBoolean.check_lower(value.rich_payload_enabled) _UniffiFfiConverterBoolean.check_lower(value.crypto_recovery_enabled) _UniffiFfiConverterOptionalTypeMeshRelayConfig.check_lower(value.mesh_relay) + _UniffiFfiConverterOptionalTypeCustodyConfig.check_lower(value.custody) _UniffiFfiConverterBoolean.check_lower(value.data_enabled) _UniffiFfiConverterBoolean.check_lower(value.control_freshness_enforced) @@ -6155,6 +6519,7 @@ def write(value, buf): _UniffiFfiConverterBoolean.write(value.rich_payload_enabled, buf) _UniffiFfiConverterBoolean.write(value.crypto_recovery_enabled, buf) _UniffiFfiConverterOptionalTypeMeshRelayConfig.write(value.mesh_relay, buf) + _UniffiFfiConverterOptionalTypeCustodyConfig.write(value.custody, buf) _UniffiFfiConverterBoolean.write(value.data_enabled, buf) _UniffiFfiConverterBoolean.write(value.control_freshness_enforced, buf) @@ -6826,31 +7191,6 @@ def read(buf): def write(value, buf): buf.write_u16(value) -class _UniffiFfiConverterOptionalBoolean(_UniffiConverterRustBuffer): - @classmethod - def check_lower(cls, value): - if value is not None: - _UniffiFfiConverterBoolean.check_lower(value) - - @classmethod - def write(cls, value, buf): - if value is None: - buf.write_u8(0) - return - - buf.write_u8(1) - _UniffiFfiConverterBoolean.write(value, buf) - - @classmethod - def read(cls, buf): - flag = buf.read_u8() - if flag == 0: - return None - elif flag == 1: - return _UniffiFfiConverterBoolean.read(buf) - else: - raise InternalError("Unexpected flag byte for optional type") - @@ -9954,6 +10294,8 @@ def enable_telemetry(self, config: TelemetryConfig,app_state: AppState) -> None: raise NotImplementedError def end_telemetry_session(self, ) -> None: raise NotImplementedError + def erase_custody(self, ) -> None: + raise NotImplementedError def establish_secure_session(self, peer_id: str) -> typing.Optional[MlsWelcomeMessage]: raise NotImplementedError def finalize_file(self, file_id: str) -> None: @@ -9976,6 +10318,8 @@ def get_battery_level(self, ) -> typing.Optional[int]: raise NotImplementedError def get_blocked_users(self, ) -> typing.List[str]: raise NotImplementedError + def get_custody_stats(self, ) -> CustodyStats: + raise NotImplementedError def get_dedup_stats(self, ) -> DedupStats: raise NotImplementedError def get_delivery_success_rate(self, ) -> float: @@ -10645,6 +10989,18 @@ def end_telemetry_session(self, ) -> None: *_uniffi_lowered_args, ) return _uniffi_lift_return(_uniffi_ffi_result) + def erase_custody(self, ) -> None: + _uniffi_lowered_args = ( + self._uniffi_clone_handle(), + ) + _uniffi_lift_return = lambda val: None + _uniffi_error_converter = _UniffiFfiConverterTypeProtocolError + _uniffi_ffi_result = _uniffi_rust_call_with_error( + _uniffi_error_converter, + _UniffiLib.uniffi_offline_protocol_uniffi_fn_method_offlineprotocol_erase_custody, + *_uniffi_lowered_args, + ) + return _uniffi_lift_return(_uniffi_ffi_result) def establish_secure_session(self, peer_id: str) -> typing.Optional[MlsWelcomeMessage]: _UniffiFfiConverterString.check_lower(peer_id) @@ -10810,6 +11166,18 @@ def get_blocked_users(self, ) -> typing.List[str]: *_uniffi_lowered_args, ) return _uniffi_lift_return(_uniffi_ffi_result) + def get_custody_stats(self, ) -> CustodyStats: + _uniffi_lowered_args = ( + self._uniffi_clone_handle(), + ) + _uniffi_lift_return = _UniffiFfiConverterTypeCustodyStats.lift + _uniffi_error_converter = None + _uniffi_ffi_result = _uniffi_rust_call_with_error( + _uniffi_error_converter, + _UniffiLib.uniffi_offline_protocol_uniffi_fn_method_offlineprotocol_get_custody_stats, + *_uniffi_lowered_args, + ) + return _uniffi_lift_return(_uniffi_ffi_result) def get_dedup_stats(self, ) -> DedupStats: _uniffi_lowered_args = ( self._uniffi_clone_handle(), @@ -13077,6 +13445,8 @@ def verify_identity_assertion(assertion: typing.List[int]) -> str: "TransportType", "AckConfig", "BleFragment", + "CustodyConfig", + "CustodyStats", "DedupConfig", "DedupStats", "DorsConfig", diff --git a/bindings/python/tests/test_custody.py b/bindings/python/tests/test_custody.py new file mode 100644 index 000000000..493ac9437 --- /dev/null +++ b/bindings/python/tests/test_custody.py @@ -0,0 +1,92 @@ +"""The custody section and the two custody calls over the Python binding. + +Python has no config parser of its own: the generated ``ProtocolConfig`` +record carries the section, so what this pins is that the section is there, +that it reaches the core's validation, and that the counters and the erase +come back through the generated class. The behaviour itself is pinned by the +engine's tests; this file is the binding's half of C9. +""" + +import pytest + +from offline_protocol_sdk import ( + CustodyConfig, + OfflineProtocol, + OverflowPolicy, + ProtocolConfig, + ProtocolError, +) + + +def _config(custody: CustodyConfig | None) -> ProtocolConfig: + return ProtocolConfig( + app_id="test-app", + profile="custody-user", + ble_enabled=False, + wifi_direct_enabled=False, + internet_enabled=True, + reticulum_enabled=False, + nostr_enabled=False, + prefer_online=True, + initial_ttl=3, + encryption_enabled=True, + auto_key_exchange=True, + store_pending=True, + require_encryption=False, + max_pending_per_peer=100, + max_pending_global=1000, + pending_ttl_ms=60000, + overflow_policy=OverflowPolicy.DROP_OLDEST, + custody=custody, + ) + + +def test_custody_is_absent_by_default_and_the_counters_start_at_zero() -> None: + # No section at all: the core's default is off, and every counter reads + # zero, including the refusal counters the acceptance table names. + proto = OfflineProtocol(_config(None)) + stats = proto.get_custody_stats() + assert stats.held == 0 + assert stats.held_bytes == 0 + assert stats.accepted == 0 + assert stats.refused_disabled == 0 + assert stats.refused_stranger == 0 + proto.erase_custody() + + +def test_a_partial_section_reaches_the_core_and_validates() -> None: + # Every field is optional and an omitted one keeps the core default: + # enabling custody alone is a valid configuration. + proto = OfflineProtocol(_config(CustodyConfig(enabled=True))) + assert proto.get_custody_stats().held == 0 + + +def test_the_core_refuses_a_hold_that_outlives_the_outbox() -> None: + # The bound is the core's, not the binding's: a hold that outlives the + # outbox would deliver frames whose sender already reported them failed. + with pytest.raises(ProtocolError.InvalidConfiguration, match="custody.hold_ms"): + OfflineProtocol(_config(CustodyConfig(enabled=True, hold_ms=10**15))) + + +def test_the_core_refuses_a_stranger_tier_set_by_one_dial() -> None: + with pytest.raises(ProtocolError.InvalidConfiguration, match="stranger"): + OfflineProtocol(_config(CustodyConfig(enabled=True, stranger_max_entries=4))) + + +def test_every_counter_the_acceptance_table_names_is_a_field() -> None: + proto = OfflineProtocol(_config(CustodyConfig(enabled=True))) + stats = proto.get_custody_stats() + for name in ( + "refused_disabled", + "refused_no_request", + "refused_unknown_class", + "refused_not_sealed", + "refused_unproven_peer", + "refused_not_depositor", + "duplicates", + "refused_stranger", + "refused_depositor_full", + "refused_store_full", + "refused_battery", + ): + assert getattr(stats, name) == 0 diff --git a/bindings/react-native/MeshSdk.podspec b/bindings/react-native/MeshSdk.podspec index 362c9114f..14f84c1cc 100644 --- a/bindings/react-native/MeshSdk.podspec +++ b/bindings/react-native/MeshSdk.podspec @@ -23,6 +23,7 @@ Pod::Spec.new do |s| "ios/OfflineProtocolModule.{m,swift}", "ios/EncryptionConfigReader.swift", "ios/MeshRelayConfigReader.swift", + "ios/CustodyConfigReader.swift", "ios/ProtocolErrorBridge.swift", "ios/TransportManager.swift", "ios/BleManager.swift", diff --git a/bindings/react-native/android/src/main/java/com/offlineprotocol/OfflineProtocolModule.kt b/bindings/react-native/android/src/main/java/com/offlineprotocol/OfflineProtocolModule.kt index 6aad68c0f..ea01bcbef 100644 --- a/bindings/react-native/android/src/main/java/com/offlineprotocol/OfflineProtocolModule.kt +++ b/bindings/react-native/android/src/main/java/com/offlineprotocol/OfflineProtocolModule.kt @@ -3628,6 +3628,58 @@ class OfflineProtocolModule(reactContext: ReactApplicationContext) : } } + // Custody counters, read through to the Rust core (docs/spec/custody.md). + @ReactMethod + fun getCustodyStats(promise: Promise) { + try { + val stats = protocol?.getCustodyStats() + if (stats != null) { + val map = Arguments.createMap() + map.putDouble("held", stats.held.toDouble()) + map.putDouble("heldBytes", stats.heldBytes.toDouble()) + map.putDouble("accepted", stats.accepted.toDouble()) + map.putDouble("delivered", stats.delivered.toDouble()) + map.putDouble("reOriginated", stats.reOriginated.toDouble()) + map.putDouble("expired", stats.expired.toDouble()) + map.putDouble("duplicates", stats.duplicates.toDouble()) + map.putDouble("evicted", stats.evicted.toDouble()) + map.putDouble("receiptsSent", stats.receiptsSent.toDouble()) + map.putDouble("receiptsDropped", stats.receiptsDropped.toDouble()) + map.putDouble("receiptsReceived", stats.receiptsReceived.toDouble()) + map.putDouble("receiptsIgnored", stats.receiptsIgnored.toDouble()) + map.putDouble("refusedDisabled", stats.refusedDisabled.toDouble()) + map.putDouble("refusedNoRequest", stats.refusedNoRequest.toDouble()) + map.putDouble("refusedUnknownClass", stats.refusedUnknownClass.toDouble()) + map.putDouble("refusedNotSealed", stats.refusedNotSealed.toDouble()) + map.putDouble("refusedUnprovenPeer", stats.refusedUnprovenPeer.toDouble()) + map.putDouble("refusedNotDepositor", stats.refusedNotDepositor.toDouble()) + map.putDouble("refusedStranger", stats.refusedStranger.toDouble()) + map.putDouble("refusedDepositorFull", stats.refusedDepositorFull.toDouble()) + map.putDouble("refusedStoreFull", stats.refusedStoreFull.toDouble()) + map.putDouble("refusedBattery", stats.refusedBattery.toDouble()) + promise.resolve(map) + } else { + promise.resolve(null) + } + } catch (e: Exception) { + promise.reject("ERROR_STATS", "Failed to get custody stats: ${e.message}", e) + } + } + + // Drops every held frame and resets the custody counters. The data + // layer's wipe calls the same erase in the core; this is the standalone + // verb. + @ReactMethod + fun eraseCustody(promise: Promise) { + try { + val proto = protocol ?: throw IllegalStateException("Protocol not initialized") + proto.eraseCustody() + promise.resolve(null) + } catch (e: Exception) { + rejectWithProtocolError(promise, e, "ERROR_ERASECUSTODY", "eraseCustody failed") + } + } + @ReactMethod fun getPendingAckCount(promise: Promise) { try { diff --git a/bindings/react-native/android/src/main/java/com/offlineprotocol/ProtocolConfigParser.kt b/bindings/react-native/android/src/main/java/com/offlineprotocol/ProtocolConfigParser.kt index 66a8ebf54..e6f3b5bbc 100644 --- a/bindings/react-native/android/src/main/java/com/offlineprotocol/ProtocolConfigParser.kt +++ b/bindings/react-native/android/src/main/java/com/offlineprotocol/ProtocolConfigParser.kt @@ -1,6 +1,7 @@ package com.offlineprotocol import org.json.JSONObject +import uniffi.offline_protocol.CustodyConfig import uniffi.offline_protocol.MeshRelayConfig import uniffi.offline_protocol.OverflowPolicy import uniffi.offline_protocol.ProtocolConfig @@ -187,6 +188,38 @@ internal object ProtocolConfigParser { ) } + // Custody section (nested home under `custody`). Absent stays absent + // all the way to the core, whose default is off: every field is + // nullable and null means "keep the Rust default", so this parser + // never states a default of its own. Mirrors CustodyConfigReader.swift; + // keep the read order in sync. Widths are coerced before the unsigned + // conversion for the reason the mesh block gives, and an overflow + // policy spelled in a way this build does not know stays null rather + // than becoming a default chosen here. + val custodyJson = json.optJSONObject("custody") + val custody = custodyJson?.let { section -> + fun uLong(vararg keys: String): ULong? = + section.optLongCompat(*keys)?.coerceAtLeast(0L)?.toULong() + val overflowRaw = section.optStringCompat("overflowPolicy", "overflow_policy") + val overflow = when (overflowRaw?.lowercase()) { + "drop_newest", "dropnewest" -> OverflowPolicy.DROP_NEWEST + "drop_oldest", "dropoldest" -> OverflowPolicy.DROP_OLDEST + else -> null + } + + CustodyConfig( + enabled = section.optBooleanCompat("enabled"), + holdMs = uLong("holdMs", "hold_ms"), + maxEntriesPerDepositor = uLong("maxEntriesPerDepositor", "max_entries_per_depositor"), + maxBytesPerDepositor = uLong("maxBytesPerDepositor", "max_bytes_per_depositor"), + maxEntries = uLong("maxEntries", "max_entries"), + maxBytes = uLong("maxBytes", "max_bytes"), + strangerMaxEntries = uLong("strangerMaxEntries", "stranger_max_entries"), + strangerMaxBytes = uLong("strangerMaxBytes", "stranger_max_bytes"), + overflowPolicy = overflow + ) + } + // Data layer section (nested home under `data`, both cases). Same // rule as meshRelay: absent stays absent, so the Rust default is the // only default. The flag is read out of the section rather than as a @@ -244,7 +277,8 @@ internal object ProtocolConfigParser { compactEnvelopeEnabled = compactEnvelopeEnabled, richPayloadEnabled = richPayloadEnabled, cryptoRecoveryEnabled = cryptoRecoveryEnabled, - meshRelay = meshRelay + meshRelay = meshRelay, + custody = custody ) // Assigned only when the app actually sent it. Writing diff --git a/bindings/react-native/android/src/main/java/uniffi/offline_protocol/offline_protocol.kt b/bindings/react-native/android/src/main/java/uniffi/offline_protocol/offline_protocol.kt index 383f51c3b..55c8e05e4 100644 --- a/bindings/react-native/android/src/main/java/uniffi/offline_protocol/offline_protocol.kt +++ b/bindings/react-native/android/src/main/java/uniffi/offline_protocol/offline_protocol.kt @@ -948,6 +948,8 @@ external fun uniffi_offline_protocol_uniffi_checksum_method_offlineprotocol_enab ): Short external fun uniffi_offline_protocol_uniffi_checksum_method_offlineprotocol_end_telemetry_session( ): Short +external fun uniffi_offline_protocol_uniffi_checksum_method_offlineprotocol_erase_custody( +): Short external fun uniffi_offline_protocol_uniffi_checksum_method_offlineprotocol_establish_secure_session( ): Short external fun uniffi_offline_protocol_uniffi_checksum_method_offlineprotocol_finalize_file( @@ -970,6 +972,8 @@ external fun uniffi_offline_protocol_uniffi_checksum_method_offlineprotocol_get_ ): Short external fun uniffi_offline_protocol_uniffi_checksum_method_offlineprotocol_get_blocked_users( ): Short +external fun uniffi_offline_protocol_uniffi_checksum_method_offlineprotocol_get_custody_stats( +): Short external fun uniffi_offline_protocol_uniffi_checksum_method_offlineprotocol_get_dedup_stats( ): Short external fun uniffi_offline_protocol_uniffi_checksum_method_offlineprotocol_get_delivery_success_rate( @@ -1447,6 +1451,8 @@ external fun uniffi_offline_protocol_uniffi_fn_method_offlineprotocol_enable_tel ): Unit external fun uniffi_offline_protocol_uniffi_fn_method_offlineprotocol_end_telemetry_session(`ptr`: Long,uniffi_out_err: UniffiRustCallStatus, ): Unit +external fun uniffi_offline_protocol_uniffi_fn_method_offlineprotocol_erase_custody(`ptr`: Long,uniffi_out_err: UniffiRustCallStatus, +): Unit external fun uniffi_offline_protocol_uniffi_fn_method_offlineprotocol_establish_secure_session(`ptr`: Long,`peerId`: RustBuffer.ByValue,uniffi_out_err: UniffiRustCallStatus, ): RustBuffer.ByValue external fun uniffi_offline_protocol_uniffi_fn_method_offlineprotocol_finalize_file(`ptr`: Long,`fileId`: RustBuffer.ByValue,uniffi_out_err: UniffiRustCallStatus, @@ -1469,6 +1475,8 @@ external fun uniffi_offline_protocol_uniffi_fn_method_offlineprotocol_get_batter ): RustBuffer.ByValue external fun uniffi_offline_protocol_uniffi_fn_method_offlineprotocol_get_blocked_users(`ptr`: Long,uniffi_out_err: UniffiRustCallStatus, ): RustBuffer.ByValue +external fun uniffi_offline_protocol_uniffi_fn_method_offlineprotocol_get_custody_stats(`ptr`: Long,uniffi_out_err: UniffiRustCallStatus, +): RustBuffer.ByValue external fun uniffi_offline_protocol_uniffi_fn_method_offlineprotocol_get_dedup_stats(`ptr`: Long,uniffi_out_err: UniffiRustCallStatus, ): RustBuffer.ByValue external fun uniffi_offline_protocol_uniffi_fn_method_offlineprotocol_get_delivery_success_rate(`ptr`: Long,uniffi_out_err: UniffiRustCallStatus, @@ -2078,6 +2086,9 @@ private fun uniffiCheckApiChecksums(lib: IntegrityCheckingUniffiLib) { if (lib.uniffi_offline_protocol_uniffi_checksum_method_offlineprotocol_end_telemetry_session() != 51941.toShort()) { throw RuntimeException("UniFFI API checksum mismatch: try cleaning and rebuilding your project") } + if (lib.uniffi_offline_protocol_uniffi_checksum_method_offlineprotocol_erase_custody() != 5086.toShort()) { + throw RuntimeException("UniFFI API checksum mismatch: try cleaning and rebuilding your project") + } if (lib.uniffi_offline_protocol_uniffi_checksum_method_offlineprotocol_establish_secure_session() != 25919.toShort()) { throw RuntimeException("UniFFI API checksum mismatch: try cleaning and rebuilding your project") } @@ -2111,6 +2122,9 @@ private fun uniffiCheckApiChecksums(lib: IntegrityCheckingUniffiLib) { if (lib.uniffi_offline_protocol_uniffi_checksum_method_offlineprotocol_get_blocked_users() != 24603.toShort()) { throw RuntimeException("UniFFI API checksum mismatch: try cleaning and rebuilding your project") } + if (lib.uniffi_offline_protocol_uniffi_checksum_method_offlineprotocol_get_custody_stats() != 42535.toShort()) { + throw RuntimeException("UniFFI API checksum mismatch: try cleaning and rebuilding your project") + } if (lib.uniffi_offline_protocol_uniffi_checksum_method_offlineprotocol_get_dedup_stats() != 43759.toShort()) { throw RuntimeException("UniFFI API checksum mismatch: try cleaning and rebuilding your project") } @@ -4224,6 +4238,8 @@ public interface OfflineProtocolInterface { fun `endTelemetrySession`() + fun `eraseCustody`() + fun `establishSecureSession`(`peerId`: kotlin.String): MlsWelcomeMessage? fun `finalizeFile`(`fileId`: kotlin.String) @@ -4246,6 +4262,8 @@ public interface OfflineProtocolInterface { fun `getBlockedUsers`(): List + fun `getCustodyStats`(): CustodyStats + fun `getDedupStats`(): DedupStats fun `getDeliverySuccessRate`(): kotlin.Float @@ -4949,6 +4967,19 @@ open class OfflineProtocol: Disposable, AutoCloseable, OfflineProtocolInterface + @Throws(ProtocolException::class)override fun `eraseCustody`() + = + callWithHandle { + uniffiRustCallWithError(ProtocolException) { _status -> + UniffiLib.uniffi_offline_protocol_uniffi_fn_method_offlineprotocol_erase_custody( + it, + _status) +} + } + + + + @Throws(ProtocolException::class)override fun `establishSecureSession`(`peerId`: kotlin.String): MlsWelcomeMessage? { return FfiConverterOptionalTypeMlsWelcomeMessage.lift( callWithHandle { @@ -5095,6 +5126,19 @@ open class OfflineProtocol: Disposable, AutoCloseable, OfflineProtocolInterface } + override fun `getCustodyStats`(): CustodyStats { + return FfiConverterTypeCustodyStats.lift( + callWithHandle { + uniffiRustCall() { _status -> + UniffiLib.uniffi_offline_protocol_uniffi_fn_method_offlineprotocol_get_custody_stats( + it, + _status) +} + } + ) + } + + override fun `getDedupStats`(): DedupStats { return FfiConverterTypeDedupStats.lift( callWithHandle { @@ -7051,6 +7095,213 @@ public object FfiConverterTypeBleFragment: FfiConverterRustBuffer { +data class CustodyConfig ( + var `enabled`: kotlin.Boolean? = null + , + var `holdMs`: kotlin.ULong? = null + , + var `maxEntriesPerDepositor`: kotlin.ULong? = null + , + var `maxBytesPerDepositor`: kotlin.ULong? = null + , + var `maxEntries`: kotlin.ULong? = null + , + var `maxBytes`: kotlin.ULong? = null + , + var `strangerMaxEntries`: kotlin.ULong? = null + , + var `strangerMaxBytes`: kotlin.ULong? = null + , + var `overflowPolicy`: OverflowPolicy? = null + +){ + + + + companion object +} + +/** + * @suppress + */ +public object FfiConverterTypeCustodyConfig: FfiConverterRustBuffer { + override fun read(buf: ByteBuffer): CustodyConfig { + return CustodyConfig( + FfiConverterOptionalBoolean.read(buf), + FfiConverterOptionalULong.read(buf), + FfiConverterOptionalULong.read(buf), + FfiConverterOptionalULong.read(buf), + FfiConverterOptionalULong.read(buf), + FfiConverterOptionalULong.read(buf), + FfiConverterOptionalULong.read(buf), + FfiConverterOptionalULong.read(buf), + FfiConverterOptionalTypeOverflowPolicy.read(buf), + ) + } + + override fun allocationSize(value: CustodyConfig) = ( + FfiConverterOptionalBoolean.allocationSize(value.`enabled`) + + FfiConverterOptionalULong.allocationSize(value.`holdMs`) + + FfiConverterOptionalULong.allocationSize(value.`maxEntriesPerDepositor`) + + FfiConverterOptionalULong.allocationSize(value.`maxBytesPerDepositor`) + + FfiConverterOptionalULong.allocationSize(value.`maxEntries`) + + FfiConverterOptionalULong.allocationSize(value.`maxBytes`) + + FfiConverterOptionalULong.allocationSize(value.`strangerMaxEntries`) + + FfiConverterOptionalULong.allocationSize(value.`strangerMaxBytes`) + + FfiConverterOptionalTypeOverflowPolicy.allocationSize(value.`overflowPolicy`) + ) + + override fun write(value: CustodyConfig, buf: ByteBuffer) { + FfiConverterOptionalBoolean.write(value.`enabled`, buf) + FfiConverterOptionalULong.write(value.`holdMs`, buf) + FfiConverterOptionalULong.write(value.`maxEntriesPerDepositor`, buf) + FfiConverterOptionalULong.write(value.`maxBytesPerDepositor`, buf) + FfiConverterOptionalULong.write(value.`maxEntries`, buf) + FfiConverterOptionalULong.write(value.`maxBytes`, buf) + FfiConverterOptionalULong.write(value.`strangerMaxEntries`, buf) + FfiConverterOptionalULong.write(value.`strangerMaxBytes`, buf) + FfiConverterOptionalTypeOverflowPolicy.write(value.`overflowPolicy`, buf) + } +} + + + +data class CustodyStats ( + var `held`: kotlin.ULong + , + var `heldBytes`: kotlin.ULong + , + var `accepted`: kotlin.ULong + , + var `delivered`: kotlin.ULong + , + var `reOriginated`: kotlin.ULong + , + var `expired`: kotlin.ULong + , + var `duplicates`: kotlin.ULong + , + var `evicted`: kotlin.ULong + , + var `receiptsSent`: kotlin.ULong + , + var `receiptsDropped`: kotlin.ULong + , + var `receiptsReceived`: kotlin.ULong + , + var `receiptsIgnored`: kotlin.ULong + , + var `refusedDisabled`: kotlin.ULong + , + var `refusedNoRequest`: kotlin.ULong + , + var `refusedUnknownClass`: kotlin.ULong + , + var `refusedNotSealed`: kotlin.ULong + , + var `refusedUnprovenPeer`: kotlin.ULong + , + var `refusedNotDepositor`: kotlin.ULong + , + var `refusedStranger`: kotlin.ULong + , + var `refusedDepositorFull`: kotlin.ULong + , + var `refusedStoreFull`: kotlin.ULong + , + var `refusedBattery`: kotlin.ULong + +){ + + + + companion object +} + +/** + * @suppress + */ +public object FfiConverterTypeCustodyStats: FfiConverterRustBuffer { + override fun read(buf: ByteBuffer): CustodyStats { + return CustodyStats( + FfiConverterULong.read(buf), + FfiConverterULong.read(buf), + FfiConverterULong.read(buf), + FfiConverterULong.read(buf), + FfiConverterULong.read(buf), + FfiConverterULong.read(buf), + FfiConverterULong.read(buf), + FfiConverterULong.read(buf), + FfiConverterULong.read(buf), + FfiConverterULong.read(buf), + FfiConverterULong.read(buf), + FfiConverterULong.read(buf), + FfiConverterULong.read(buf), + FfiConverterULong.read(buf), + FfiConverterULong.read(buf), + FfiConverterULong.read(buf), + FfiConverterULong.read(buf), + FfiConverterULong.read(buf), + FfiConverterULong.read(buf), + FfiConverterULong.read(buf), + FfiConverterULong.read(buf), + FfiConverterULong.read(buf), + ) + } + + override fun allocationSize(value: CustodyStats) = ( + FfiConverterULong.allocationSize(value.`held`) + + FfiConverterULong.allocationSize(value.`heldBytes`) + + FfiConverterULong.allocationSize(value.`accepted`) + + FfiConverterULong.allocationSize(value.`delivered`) + + FfiConverterULong.allocationSize(value.`reOriginated`) + + FfiConverterULong.allocationSize(value.`expired`) + + FfiConverterULong.allocationSize(value.`duplicates`) + + FfiConverterULong.allocationSize(value.`evicted`) + + FfiConverterULong.allocationSize(value.`receiptsSent`) + + FfiConverterULong.allocationSize(value.`receiptsDropped`) + + FfiConverterULong.allocationSize(value.`receiptsReceived`) + + FfiConverterULong.allocationSize(value.`receiptsIgnored`) + + FfiConverterULong.allocationSize(value.`refusedDisabled`) + + FfiConverterULong.allocationSize(value.`refusedNoRequest`) + + FfiConverterULong.allocationSize(value.`refusedUnknownClass`) + + FfiConverterULong.allocationSize(value.`refusedNotSealed`) + + FfiConverterULong.allocationSize(value.`refusedUnprovenPeer`) + + FfiConverterULong.allocationSize(value.`refusedNotDepositor`) + + FfiConverterULong.allocationSize(value.`refusedStranger`) + + FfiConverterULong.allocationSize(value.`refusedDepositorFull`) + + FfiConverterULong.allocationSize(value.`refusedStoreFull`) + + FfiConverterULong.allocationSize(value.`refusedBattery`) + ) + + override fun write(value: CustodyStats, buf: ByteBuffer) { + FfiConverterULong.write(value.`held`, buf) + FfiConverterULong.write(value.`heldBytes`, buf) + FfiConverterULong.write(value.`accepted`, buf) + FfiConverterULong.write(value.`delivered`, buf) + FfiConverterULong.write(value.`reOriginated`, buf) + FfiConverterULong.write(value.`expired`, buf) + FfiConverterULong.write(value.`duplicates`, buf) + FfiConverterULong.write(value.`evicted`, buf) + FfiConverterULong.write(value.`receiptsSent`, buf) + FfiConverterULong.write(value.`receiptsDropped`, buf) + FfiConverterULong.write(value.`receiptsReceived`, buf) + FfiConverterULong.write(value.`receiptsIgnored`, buf) + FfiConverterULong.write(value.`refusedDisabled`, buf) + FfiConverterULong.write(value.`refusedNoRequest`, buf) + FfiConverterULong.write(value.`refusedUnknownClass`, buf) + FfiConverterULong.write(value.`refusedNotSealed`, buf) + FfiConverterULong.write(value.`refusedUnprovenPeer`, buf) + FfiConverterULong.write(value.`refusedNotDepositor`, buf) + FfiConverterULong.write(value.`refusedStranger`, buf) + FfiConverterULong.write(value.`refusedDepositorFull`, buf) + FfiConverterULong.write(value.`refusedStoreFull`, buf) + FfiConverterULong.write(value.`refusedBattery`, buf) + } +} + + + data class DedupConfig ( var `maxTrackedMessages`: kotlin.ULong , @@ -8722,6 +8973,8 @@ data class ProtocolConfig ( , var `meshRelay`: MeshRelayConfig? = null , + var `custody`: CustodyConfig? = null + , var `dataEnabled`: kotlin.Boolean = true , var `controlFreshnessEnforced`: kotlin.Boolean = true @@ -8770,6 +9023,7 @@ public object FfiConverterTypeProtocolConfig: FfiConverterRustBuffer { + override fun read(buf: ByteBuffer): CustodyConfig? { + if (buf.get().toInt() == 0) { + return null + } + return FfiConverterTypeCustodyConfig.read(buf) + } + + override fun allocationSize(value: CustodyConfig?): ULong { + if (value == null) { + return 1UL + } else { + return 1UL + FfiConverterTypeCustodyConfig.allocationSize(value) + } + } + + override fun write(value: CustodyConfig?, buf: ByteBuffer) { + if (value == null) { + buf.put(0) + } else { + buf.put(1) + FfiConverterTypeCustodyConfig.write(value, buf) + } + } +} + + + + /** * @suppress */ @@ -11660,6 +11948,38 @@ public object FfiConverterOptionalTypeMlsVerbosity: FfiConverterRustBuffer { + override fun read(buf: ByteBuffer): OverflowPolicy? { + if (buf.get().toInt() == 0) { + return null + } + return FfiConverterTypeOverflowPolicy.read(buf) + } + + override fun allocationSize(value: OverflowPolicy?): ULong { + if (value == null) { + return 1UL + } else { + return 1UL + FfiConverterTypeOverflowPolicy.allocationSize(value) + } + } + + override fun write(value: OverflowPolicy?, buf: ByteBuffer) { + if (value == null) { + buf.put(0) + } else { + buf.put(1) + FfiConverterTypeOverflowPolicy.write(value, buf) + } + } +} + + + + /** * @suppress */ diff --git a/bindings/react-native/android/src/test/java/com/offlineprotocol/ProtocolConfigParserTest.kt b/bindings/react-native/android/src/test/java/com/offlineprotocol/ProtocolConfigParserTest.kt index 5ca62f417..5d27c23e3 100644 --- a/bindings/react-native/android/src/test/java/com/offlineprotocol/ProtocolConfigParserTest.kt +++ b/bindings/react-native/android/src/test/java/com/offlineprotocol/ProtocolConfigParserTest.kt @@ -5,6 +5,7 @@ import org.junit.Assert.assertFalse import org.junit.Assert.assertNull import org.junit.Assert.assertTrue import org.junit.Test +import uniffi.offline_protocol.OverflowPolicy /** * Locks down the create()-config parsing: a silent regression here reverts a @@ -450,6 +451,84 @@ class ProtocolConfigParserTest { assertNull(mesh.activityIdleWindows) } + // ------------------------------------------------------------------ + // Custody section + // ------------------------------------------------------------------ + + @Test + fun custodySectionIsAbsentWhenOmitted() { + // Nil, not an object of nulls: the core's default is off, and a + // section materialised here would be this parser deciding it. + val config = parse("""{"appId":"app","userId":"alice"}""") + assertNull(config.custody) + } + + @Test + fun custodySectionReadsItsNestedHome() { + val config = parse( + """{"appId":"app","userId":"alice","custody":{"enabled":true,"holdMs":3600000,"maxEntriesPerDepositor":16,"maxBytesPerDepositor":131072,"maxEntries":128,"maxBytes":4194304,"strangerMaxEntries":2,"strangerMaxBytes":65536,"overflowPolicy":"drop_newest"}}""" + ) + val custody = config.custody!! + assertEquals(true, custody.enabled) + assertEquals(3600000L, custody.holdMs!!.toLong()) + assertEquals(16L, custody.maxEntriesPerDepositor!!.toLong()) + assertEquals(131072L, custody.maxBytesPerDepositor!!.toLong()) + assertEquals(128L, custody.maxEntries!!.toLong()) + assertEquals(4194304L, custody.maxBytes!!.toLong()) + assertEquals(2L, custody.strangerMaxEntries!!.toLong()) + assertEquals(65536L, custody.strangerMaxBytes!!.toLong()) + assertEquals(OverflowPolicy.DROP_NEWEST, custody.overflowPolicy) + } + + @Test + fun custodySectionReadsNestedSnakeCase() { + val config = parse( + """{"appId":"app","userId":"alice","custody":{"enabled":false,"hold_ms":7200000,"max_entries_per_depositor":8,"max_bytes_per_depositor":65536,"max_entries":64,"max_bytes":1048576,"stranger_max_entries":1,"stranger_max_bytes":65536,"overflow_policy":"drop_oldest"}}""" + ) + val custody = config.custody!! + assertEquals(false, custody.enabled) + assertEquals(7200000L, custody.holdMs!!.toLong()) + assertEquals(8L, custody.maxEntriesPerDepositor!!.toLong()) + assertEquals(65536L, custody.maxBytesPerDepositor!!.toLong()) + assertEquals(64L, custody.maxEntries!!.toLong()) + assertEquals(1048576L, custody.maxBytes!!.toLong()) + assertEquals(1L, custody.strangerMaxEntries!!.toLong()) + assertEquals(65536L, custody.strangerMaxBytes!!.toLong()) + assertEquals(OverflowPolicy.DROP_OLDEST, custody.overflowPolicy) + } + + @Test + fun custodyLeavesUnnamedFieldsNull() { + // The ordinary case: an app switches custody on and names nothing + // else. Every other field must arrive null so the core keeps its own + // value, and an unknown policy spelling stays null too. + val config = parse( + """{"appId":"app","userId":"alice","custody":{"enabled":true,"overflowPolicy":"keep_everything"}}""" + ) + val custody = config.custody!! + assertEquals(true, custody.enabled) + assertNull(custody.holdMs) + assertNull(custody.maxEntriesPerDepositor) + assertNull(custody.maxBytesPerDepositor) + assertNull(custody.maxEntries) + assertNull(custody.maxBytes) + assertNull(custody.strangerMaxEntries) + assertNull(custody.strangerMaxBytes) + assertNull(custody.overflowPolicy) + } + + @Test + fun custodyNegativeNumbersClampToZeroRatherThanWrapping() { + // App-supplied JS: a negative would wrap to something enormous through + // toULong(). Clamped low it reaches the core's own validation. + val config = parse( + """{"appId":"app","userId":"alice","custody":{"holdMs":-5,"maxEntries":-1}}""" + ) + val custody = config.custody!! + assertEquals(0L, custody.holdMs!!.toLong()) + assertEquals(0L, custody.maxEntries!!.toLong()) + } + @Test fun meshRelayNegativesAreClampedRatherThanWrapped() { // These fields are unsigned across the FFI. A bare conversion would diff --git a/bindings/react-native/ios/CustodyConfigReader.swift b/bindings/react-native/ios/CustodyConfigReader.swift new file mode 100644 index 000000000..a6dd260cf --- /dev/null +++ b/bindings/react-native/ios/CustodyConfigReader.swift @@ -0,0 +1,89 @@ +import Foundation + +/// The custody section of the `create()` config JSON (`docs/spec/custody.md`). +/// +/// Mirrors android/ `ProtocolConfigParser`'s custody block, and keeps the +/// read order and precedence in sync: nested home under `custody`, camelCase +/// or snake_case within it. +/// +/// Every value is optional and stays optional, for the reason the mesh +/// forwarding reader beside it gives: an absent field must reach the core +/// absent, because the core owns every default, and here the default that +/// matters most is "off". A reader that filled `enabled` in with `false` +/// would be a second copy of that default, and the release that ever flips +/// it would keep forcing `false` for every app that omitted the section. +/// +/// Foundation-only on purpose: the SwiftPM test harness (Package.swift) +/// compiles this file without React or the Generated UniFFI module, so the +/// overflow policy is carried as the string the app wrote and mapped onto the +/// UniFFI enum by `OfflineProtocolModule`. +struct CustodyConfigValues: Equatable { + var enabled: Bool? + var holdMs: UInt64? + var maxEntriesPerDepositor: UInt64? + var maxBytesPerDepositor: UInt64? + var maxEntries: UInt64? + var maxBytes: UInt64? + var strangerMaxEntries: UInt64? + var strangerMaxBytes: UInt64? + /// The policy as the app spelled it (`drop_oldest` or `drop_newest`). + var overflowPolicy: String? +} + +enum CustodyConfigReader { + + /// Returns nil when the app set no custody section at all, so the module + /// passes nil across the FFI and the core keeps every default. + static func read(_ raw: [String: Any]) -> CustodyConfigValues? { + guard let nested = raw["custody"] as? [String: Any] else { + return nil + } + + return CustodyConfigValues( + enabled: bool(nested, "enabled"), + holdMs: uint64(nested, "holdMs", "hold_ms"), + maxEntriesPerDepositor: uint64(nested, "maxEntriesPerDepositor", "max_entries_per_depositor"), + maxBytesPerDepositor: uint64(nested, "maxBytesPerDepositor", "max_bytes_per_depositor"), + maxEntries: uint64(nested, "maxEntries", "max_entries"), + maxBytes: uint64(nested, "maxBytes", "max_bytes"), + strangerMaxEntries: uint64(nested, "strangerMaxEntries", "stranger_max_entries"), + strangerMaxBytes: uint64(nested, "strangerMaxBytes", "stranger_max_bytes"), + overflowPolicy: string(nested, "overflowPolicy", "overflow_policy") + ) + } + + private static func bool(_ dict: [String: Any], _ keys: String...) -> Bool? { + for key in keys { + if let value = dict[key] as? Bool { + return value + } + } + return nil + } + + // Clamped rather than converted: the value is app-supplied JS, so a + // negative would trap the unsigned initializer outright. Clamped to zero + // it reaches the core's own validation, which is the one place that gets + // to decide what is legal. + private static func uint64(_ dict: [String: Any], _ keys: String...) -> UInt64? { + for key in keys { + // A JSON boolean also arrives as an NSNumber, so it is excluded by + // its CoreFoundation type rather than by `is Bool`: Swift bridges + // the numbers 0 and 1 to Bool as well, and testing that would + // read a legitimate `1` as unset. + if let value = dict[key] as? NSNumber, CFGetTypeID(value) != CFBooleanGetTypeID() { + return UInt64(clamping: value.int64Value) + } + } + return nil + } + + private static func string(_ dict: [String: Any], _ keys: String...) -> String? { + for key in keys { + if let value = dict[key] as? String { + return value + } + } + return nil + } +} diff --git a/bindings/react-native/ios/Generated/offline_protocol.swift b/bindings/react-native/ios/Generated/offline_protocol.swift index ac0fa7a2f..14a725ea8 100644 --- a/bindings/react-native/ios/Generated/offline_protocol.swift +++ b/bindings/react-native/ios/Generated/offline_protocol.swift @@ -1323,6 +1323,8 @@ public protocol OfflineProtocolProtocol: AnyObject, Sendable { func endTelemetrySession() + func eraseCustody() throws + func establishSecureSession(peerId: String) throws -> MlsWelcomeMessage? func finalizeFile(fileId: String) throws @@ -1345,6 +1347,8 @@ public protocol OfflineProtocolProtocol: AnyObject, Sendable { func getBlockedUsers() throws -> [String] + func getCustodyStats() -> CustodyStats + func getDedupStats() -> DedupStats func getDeliverySuccessRate() -> Float @@ -1883,6 +1887,13 @@ open func endTelemetrySession() {try! rustCall() { } } +open func eraseCustody()throws {try rustCallWithError(FfiConverterTypeProtocolError_lift) { + uniffi_offline_protocol_uniffi_fn_method_offlineprotocol_erase_custody( + self.uniffiCloneHandle(),$0 + ) +} +} + open func establishSecureSession(peerId: String)throws -> MlsWelcomeMessage? { return try FfiConverterOptionTypeMlsWelcomeMessage.lift(try rustCallWithError(FfiConverterTypeProtocolError_lift) { uniffi_offline_protocol_uniffi_fn_method_offlineprotocol_establish_secure_session( @@ -1979,6 +1990,14 @@ open func getBlockedUsers()throws -> [String] { }) } +open func getCustodyStats() -> CustodyStats { + return try! FfiConverterTypeCustodyStats_lift(try! rustCall() { + uniffi_offline_protocol_uniffi_fn_method_offlineprotocol_get_custody_stats( + self.uniffiCloneHandle(),$0 + ) +}) +} + open func getDedupStats() -> DedupStats { return try! FfiConverterTypeDedupStats_lift(try! rustCall() { uniffi_offline_protocol_uniffi_fn_method_offlineprotocol_get_dedup_stats( @@ -3349,6 +3368,218 @@ public func FfiConverterTypeBleFragment_lower(_ value: BleFragment) -> RustBuffe } +public struct CustodyConfig: Equatable, Hashable { + public var enabled: Bool? + public var holdMs: UInt64? + public var maxEntriesPerDepositor: UInt64? + public var maxBytesPerDepositor: UInt64? + public var maxEntries: UInt64? + public var maxBytes: UInt64? + public var strangerMaxEntries: UInt64? + public var strangerMaxBytes: UInt64? + public var overflowPolicy: OverflowPolicy? + + // Default memberwise initializers are never public by default, so we + // declare one manually. + public init(enabled: Bool? = nil, holdMs: UInt64? = nil, maxEntriesPerDepositor: UInt64? = nil, maxBytesPerDepositor: UInt64? = nil, maxEntries: UInt64? = nil, maxBytes: UInt64? = nil, strangerMaxEntries: UInt64? = nil, strangerMaxBytes: UInt64? = nil, overflowPolicy: OverflowPolicy? = nil) { + self.enabled = enabled + self.holdMs = holdMs + self.maxEntriesPerDepositor = maxEntriesPerDepositor + self.maxBytesPerDepositor = maxBytesPerDepositor + self.maxEntries = maxEntries + self.maxBytes = maxBytes + self.strangerMaxEntries = strangerMaxEntries + self.strangerMaxBytes = strangerMaxBytes + self.overflowPolicy = overflowPolicy + } + + +} + +#if compiler(>=6) +extension CustodyConfig: Sendable {} +#endif + +#if swift(>=5.8) +@_documentation(visibility: private) +#endif +public struct FfiConverterTypeCustodyConfig: FfiConverterRustBuffer { + public static func read(from buf: inout (data: Data, offset: Data.Index)) throws -> CustodyConfig { + return + try CustodyConfig( + enabled: FfiConverterOptionBool.read(from: &buf), + holdMs: FfiConverterOptionUInt64.read(from: &buf), + maxEntriesPerDepositor: FfiConverterOptionUInt64.read(from: &buf), + maxBytesPerDepositor: FfiConverterOptionUInt64.read(from: &buf), + maxEntries: FfiConverterOptionUInt64.read(from: &buf), + maxBytes: FfiConverterOptionUInt64.read(from: &buf), + strangerMaxEntries: FfiConverterOptionUInt64.read(from: &buf), + strangerMaxBytes: FfiConverterOptionUInt64.read(from: &buf), + overflowPolicy: FfiConverterOptionTypeOverflowPolicy.read(from: &buf) + ) + } + + public static func write(_ value: CustodyConfig, into buf: inout [UInt8]) { + FfiConverterOptionBool.write(value.enabled, into: &buf) + FfiConverterOptionUInt64.write(value.holdMs, into: &buf) + FfiConverterOptionUInt64.write(value.maxEntriesPerDepositor, into: &buf) + FfiConverterOptionUInt64.write(value.maxBytesPerDepositor, into: &buf) + FfiConverterOptionUInt64.write(value.maxEntries, into: &buf) + FfiConverterOptionUInt64.write(value.maxBytes, into: &buf) + FfiConverterOptionUInt64.write(value.strangerMaxEntries, into: &buf) + FfiConverterOptionUInt64.write(value.strangerMaxBytes, into: &buf) + FfiConverterOptionTypeOverflowPolicy.write(value.overflowPolicy, into: &buf) + } +} + + +#if swift(>=5.8) +@_documentation(visibility: private) +#endif +public func FfiConverterTypeCustodyConfig_lift(_ buf: RustBuffer) throws -> CustodyConfig { + return try FfiConverterTypeCustodyConfig.lift(buf) +} + +#if swift(>=5.8) +@_documentation(visibility: private) +#endif +public func FfiConverterTypeCustodyConfig_lower(_ value: CustodyConfig) -> RustBuffer { + return FfiConverterTypeCustodyConfig.lower(value) +} + + +public struct CustodyStats: Equatable, Hashable { + public var held: UInt64 + public var heldBytes: UInt64 + public var accepted: UInt64 + public var delivered: UInt64 + public var reOriginated: UInt64 + public var expired: UInt64 + public var duplicates: UInt64 + public var evicted: UInt64 + public var receiptsSent: UInt64 + public var receiptsDropped: UInt64 + public var receiptsReceived: UInt64 + public var receiptsIgnored: UInt64 + public var refusedDisabled: UInt64 + public var refusedNoRequest: UInt64 + public var refusedUnknownClass: UInt64 + public var refusedNotSealed: UInt64 + public var refusedUnprovenPeer: UInt64 + public var refusedNotDepositor: UInt64 + public var refusedStranger: UInt64 + public var refusedDepositorFull: UInt64 + public var refusedStoreFull: UInt64 + public var refusedBattery: UInt64 + + // Default memberwise initializers are never public by default, so we + // declare one manually. + public init(held: UInt64, heldBytes: UInt64, accepted: UInt64, delivered: UInt64, reOriginated: UInt64, expired: UInt64, duplicates: UInt64, evicted: UInt64, receiptsSent: UInt64, receiptsDropped: UInt64, receiptsReceived: UInt64, receiptsIgnored: UInt64, refusedDisabled: UInt64, refusedNoRequest: UInt64, refusedUnknownClass: UInt64, refusedNotSealed: UInt64, refusedUnprovenPeer: UInt64, refusedNotDepositor: UInt64, refusedStranger: UInt64, refusedDepositorFull: UInt64, refusedStoreFull: UInt64, refusedBattery: UInt64) { + self.held = held + self.heldBytes = heldBytes + self.accepted = accepted + self.delivered = delivered + self.reOriginated = reOriginated + self.expired = expired + self.duplicates = duplicates + self.evicted = evicted + self.receiptsSent = receiptsSent + self.receiptsDropped = receiptsDropped + self.receiptsReceived = receiptsReceived + self.receiptsIgnored = receiptsIgnored + self.refusedDisabled = refusedDisabled + self.refusedNoRequest = refusedNoRequest + self.refusedUnknownClass = refusedUnknownClass + self.refusedNotSealed = refusedNotSealed + self.refusedUnprovenPeer = refusedUnprovenPeer + self.refusedNotDepositor = refusedNotDepositor + self.refusedStranger = refusedStranger + self.refusedDepositorFull = refusedDepositorFull + self.refusedStoreFull = refusedStoreFull + self.refusedBattery = refusedBattery + } + + +} + +#if compiler(>=6) +extension CustodyStats: Sendable {} +#endif + +#if swift(>=5.8) +@_documentation(visibility: private) +#endif +public struct FfiConverterTypeCustodyStats: FfiConverterRustBuffer { + public static func read(from buf: inout (data: Data, offset: Data.Index)) throws -> CustodyStats { + return + try CustodyStats( + held: FfiConverterUInt64.read(from: &buf), + heldBytes: FfiConverterUInt64.read(from: &buf), + accepted: FfiConverterUInt64.read(from: &buf), + delivered: FfiConverterUInt64.read(from: &buf), + reOriginated: FfiConverterUInt64.read(from: &buf), + expired: FfiConverterUInt64.read(from: &buf), + duplicates: FfiConverterUInt64.read(from: &buf), + evicted: FfiConverterUInt64.read(from: &buf), + receiptsSent: FfiConverterUInt64.read(from: &buf), + receiptsDropped: FfiConverterUInt64.read(from: &buf), + receiptsReceived: FfiConverterUInt64.read(from: &buf), + receiptsIgnored: FfiConverterUInt64.read(from: &buf), + refusedDisabled: FfiConverterUInt64.read(from: &buf), + refusedNoRequest: FfiConverterUInt64.read(from: &buf), + refusedUnknownClass: FfiConverterUInt64.read(from: &buf), + refusedNotSealed: FfiConverterUInt64.read(from: &buf), + refusedUnprovenPeer: FfiConverterUInt64.read(from: &buf), + refusedNotDepositor: FfiConverterUInt64.read(from: &buf), + refusedStranger: FfiConverterUInt64.read(from: &buf), + refusedDepositorFull: FfiConverterUInt64.read(from: &buf), + refusedStoreFull: FfiConverterUInt64.read(from: &buf), + refusedBattery: FfiConverterUInt64.read(from: &buf) + ) + } + + public static func write(_ value: CustodyStats, into buf: inout [UInt8]) { + FfiConverterUInt64.write(value.held, into: &buf) + FfiConverterUInt64.write(value.heldBytes, into: &buf) + FfiConverterUInt64.write(value.accepted, into: &buf) + FfiConverterUInt64.write(value.delivered, into: &buf) + FfiConverterUInt64.write(value.reOriginated, into: &buf) + FfiConverterUInt64.write(value.expired, into: &buf) + FfiConverterUInt64.write(value.duplicates, into: &buf) + FfiConverterUInt64.write(value.evicted, into: &buf) + FfiConverterUInt64.write(value.receiptsSent, into: &buf) + FfiConverterUInt64.write(value.receiptsDropped, into: &buf) + FfiConverterUInt64.write(value.receiptsReceived, into: &buf) + FfiConverterUInt64.write(value.receiptsIgnored, into: &buf) + FfiConverterUInt64.write(value.refusedDisabled, into: &buf) + FfiConverterUInt64.write(value.refusedNoRequest, into: &buf) + FfiConverterUInt64.write(value.refusedUnknownClass, into: &buf) + FfiConverterUInt64.write(value.refusedNotSealed, into: &buf) + FfiConverterUInt64.write(value.refusedUnprovenPeer, into: &buf) + FfiConverterUInt64.write(value.refusedNotDepositor, into: &buf) + FfiConverterUInt64.write(value.refusedStranger, into: &buf) + FfiConverterUInt64.write(value.refusedDepositorFull, into: &buf) + FfiConverterUInt64.write(value.refusedStoreFull, into: &buf) + FfiConverterUInt64.write(value.refusedBattery, into: &buf) + } +} + + +#if swift(>=5.8) +@_documentation(visibility: private) +#endif +public func FfiConverterTypeCustodyStats_lift(_ buf: RustBuffer) throws -> CustodyStats { + return try FfiConverterTypeCustodyStats.lift(buf) +} + +#if swift(>=5.8) +@_documentation(visibility: private) +#endif +public func FfiConverterTypeCustodyStats_lower(_ value: CustodyStats) -> RustBuffer { + return FfiConverterTypeCustodyStats.lower(value) +} + + public struct DedupConfig: Equatable, Hashable { public var maxTrackedMessages: UInt64 public var retentionTimeSecs: UInt64 @@ -5317,12 +5548,13 @@ public struct ProtocolConfig: Equatable, Hashable { public var richPayloadEnabled: Bool public var cryptoRecoveryEnabled: Bool public var meshRelay: MeshRelayConfig? + public var custody: CustodyConfig? public var dataEnabled: Bool public var controlFreshnessEnforced: Bool // Default memberwise initializers are never public by default, so we // declare one manually. - public init(appId: String, profile: String, bleEnabled: Bool, wifiDirectEnabled: Bool, internetEnabled: Bool, reticulumEnabled: Bool, nostrEnabled: Bool, preferOnline: Bool, initialTtl: UInt8, encryptionEnabled: Bool, autoKeyExchange: Bool, storePending: Bool, requireEncryption: Bool = true, maxPendingPerPeer: UInt64, maxPendingGlobal: UInt64, pendingTtlMs: UInt64, overflowPolicy: OverflowPolicy, edgeDrivenUnreachableDm: Bool = false, maxGroupMembers: UInt32 = UInt32(256), groupRelayEnabled: Bool = true, groupRelayBroadcastEnabled: Bool = true, groupEnforceAdminCommits: Bool = false, requireTransportIdentity: Bool = false, binaryWireEnabled: Bool = true, nostrSealingEnabled: Bool = true, nostrColdContactEnabled: Bool = true, nostrUsernameDiscoveryEnabled: Bool = false, compactEnvelopeEnabled: Bool = true, richPayloadEnabled: Bool = true, cryptoRecoveryEnabled: Bool = true, meshRelay: MeshRelayConfig? = nil, dataEnabled: Bool = true, controlFreshnessEnforced: Bool = true) { + public init(appId: String, profile: String, bleEnabled: Bool, wifiDirectEnabled: Bool, internetEnabled: Bool, reticulumEnabled: Bool, nostrEnabled: Bool, preferOnline: Bool, initialTtl: UInt8, encryptionEnabled: Bool, autoKeyExchange: Bool, storePending: Bool, requireEncryption: Bool = true, maxPendingPerPeer: UInt64, maxPendingGlobal: UInt64, pendingTtlMs: UInt64, overflowPolicy: OverflowPolicy, edgeDrivenUnreachableDm: Bool = false, maxGroupMembers: UInt32 = UInt32(256), groupRelayEnabled: Bool = true, groupRelayBroadcastEnabled: Bool = true, groupEnforceAdminCommits: Bool = false, requireTransportIdentity: Bool = false, binaryWireEnabled: Bool = true, nostrSealingEnabled: Bool = true, nostrColdContactEnabled: Bool = true, nostrUsernameDiscoveryEnabled: Bool = false, compactEnvelopeEnabled: Bool = true, richPayloadEnabled: Bool = true, cryptoRecoveryEnabled: Bool = true, meshRelay: MeshRelayConfig? = nil, custody: CustodyConfig? = nil, dataEnabled: Bool = true, controlFreshnessEnforced: Bool = true) { self.appId = appId self.profile = profile self.bleEnabled = bleEnabled @@ -5354,6 +5586,7 @@ public struct ProtocolConfig: Equatable, Hashable { self.richPayloadEnabled = richPayloadEnabled self.cryptoRecoveryEnabled = cryptoRecoveryEnabled self.meshRelay = meshRelay + self.custody = custody self.dataEnabled = dataEnabled self.controlFreshnessEnforced = controlFreshnessEnforced } @@ -5403,6 +5636,7 @@ public struct FfiConverterTypeProtocolConfig: FfiConverterRustBuffer { richPayloadEnabled: FfiConverterBool.read(from: &buf), cryptoRecoveryEnabled: FfiConverterBool.read(from: &buf), meshRelay: FfiConverterOptionTypeMeshRelayConfig.read(from: &buf), + custody: FfiConverterOptionTypeCustodyConfig.read(from: &buf), dataEnabled: FfiConverterBool.read(from: &buf), controlFreshnessEnforced: FfiConverterBool.read(from: &buf) ) @@ -5440,6 +5674,7 @@ public struct FfiConverterTypeProtocolConfig: FfiConverterRustBuffer { FfiConverterBool.write(value.richPayloadEnabled, into: &buf) FfiConverterBool.write(value.cryptoRecoveryEnabled, into: &buf) FfiConverterOptionTypeMeshRelayConfig.write(value.meshRelay, into: &buf) + FfiConverterOptionTypeCustodyConfig.write(value.custody, into: &buf) FfiConverterBool.write(value.dataEnabled, into: &buf) FfiConverterBool.write(value.controlFreshnessEnforced, into: &buf) } @@ -8916,6 +9151,30 @@ fileprivate struct FfiConverterOptionTypeBleFragment: FfiConverterRustBuffer { } } +#if swift(>=5.8) +@_documentation(visibility: private) +#endif +fileprivate struct FfiConverterOptionTypeCustodyConfig: FfiConverterRustBuffer { + typealias SwiftType = CustodyConfig? + + public static func write(_ value: SwiftType, into buf: inout [UInt8]) { + guard let value = value else { + writeInt(&buf, Int8(0)) + return + } + writeInt(&buf, Int8(1)) + FfiConverterTypeCustodyConfig.write(value, into: &buf) + } + + public static func read(from buf: inout (data: Data, offset: Data.Index)) throws -> SwiftType { + switch try readInt(&buf) as Int8 { + case 0: return nil + case 1: return try FfiConverterTypeCustodyConfig.read(from: &buf) + default: throw UniffiInternalError.unexpectedOptionalTag + } + } +} + #if swift(>=5.8) @_documentation(visibility: private) #endif @@ -9324,6 +9583,30 @@ fileprivate struct FfiConverterOptionTypeMlsVerbosity: FfiConverterRustBuffer { } } +#if swift(>=5.8) +@_documentation(visibility: private) +#endif +fileprivate struct FfiConverterOptionTypeOverflowPolicy: FfiConverterRustBuffer { + typealias SwiftType = OverflowPolicy? + + public static func write(_ value: SwiftType, into buf: inout [UInt8]) { + guard let value = value else { + writeInt(&buf, Int8(0)) + return + } + writeInt(&buf, Int8(1)) + FfiConverterTypeOverflowPolicy.write(value, into: &buf) + } + + public static func read(from buf: inout (data: Data, offset: Data.Index)) throws -> SwiftType { + switch try readInt(&buf) as Int8 { + case 0: return nil + case 1: return try FfiConverterTypeOverflowPolicy.read(from: &buf) + default: throw UniffiInternalError.unexpectedOptionalTag + } + } +} + #if swift(>=5.8) @_documentation(visibility: private) #endif @@ -9753,6 +10036,9 @@ private let initializationResult: InitializationResult = { if (uniffi_offline_protocol_uniffi_checksum_method_offlineprotocol_end_telemetry_session() != 51941) { return InitializationResult.apiChecksumMismatch } + if (uniffi_offline_protocol_uniffi_checksum_method_offlineprotocol_erase_custody() != 5086) { + return InitializationResult.apiChecksumMismatch + } if (uniffi_offline_protocol_uniffi_checksum_method_offlineprotocol_establish_secure_session() != 25919) { return InitializationResult.apiChecksumMismatch } @@ -9786,6 +10072,9 @@ private let initializationResult: InitializationResult = { if (uniffi_offline_protocol_uniffi_checksum_method_offlineprotocol_get_blocked_users() != 24603) { return InitializationResult.apiChecksumMismatch } + if (uniffi_offline_protocol_uniffi_checksum_method_offlineprotocol_get_custody_stats() != 42535) { + return InitializationResult.apiChecksumMismatch + } if (uniffi_offline_protocol_uniffi_checksum_method_offlineprotocol_get_dedup_stats() != 43759) { return InitializationResult.apiChecksumMismatch } diff --git a/bindings/react-native/ios/Generated/offline_protocolFFI.h b/bindings/react-native/ios/Generated/offline_protocolFFI.h index 30d0ffaf9..a734ef480 100644 --- a/bindings/react-native/ios/Generated/offline_protocolFFI.h +++ b/bindings/react-native/ios/Generated/offline_protocolFFI.h @@ -743,6 +743,11 @@ void uniffi_offline_protocol_uniffi_fn_method_offlineprotocol_enable_telemetry(u void uniffi_offline_protocol_uniffi_fn_method_offlineprotocol_end_telemetry_session(uint64_t ptr, RustCallStatus *_Nonnull out_status ); #endif +#ifndef UNIFFI_FFIDEF_UNIFFI_OFFLINE_PROTOCOL_UNIFFI_FN_METHOD_OFFLINEPROTOCOL_ERASE_CUSTODY +#define UNIFFI_FFIDEF_UNIFFI_OFFLINE_PROTOCOL_UNIFFI_FN_METHOD_OFFLINEPROTOCOL_ERASE_CUSTODY +void uniffi_offline_protocol_uniffi_fn_method_offlineprotocol_erase_custody(uint64_t ptr, RustCallStatus *_Nonnull out_status +); +#endif #ifndef UNIFFI_FFIDEF_UNIFFI_OFFLINE_PROTOCOL_UNIFFI_FN_METHOD_OFFLINEPROTOCOL_ESTABLISH_SECURE_SESSION #define UNIFFI_FFIDEF_UNIFFI_OFFLINE_PROTOCOL_UNIFFI_FN_METHOD_OFFLINEPROTOCOL_ESTABLISH_SECURE_SESSION RustBuffer uniffi_offline_protocol_uniffi_fn_method_offlineprotocol_establish_secure_session(uint64_t ptr, RustBuffer peer_id, RustCallStatus *_Nonnull out_status @@ -798,6 +803,11 @@ RustBuffer uniffi_offline_protocol_uniffi_fn_method_offlineprotocol_get_battery_ RustBuffer uniffi_offline_protocol_uniffi_fn_method_offlineprotocol_get_blocked_users(uint64_t ptr, RustCallStatus *_Nonnull out_status ); #endif +#ifndef UNIFFI_FFIDEF_UNIFFI_OFFLINE_PROTOCOL_UNIFFI_FN_METHOD_OFFLINEPROTOCOL_GET_CUSTODY_STATS +#define UNIFFI_FFIDEF_UNIFFI_OFFLINE_PROTOCOL_UNIFFI_FN_METHOD_OFFLINEPROTOCOL_GET_CUSTODY_STATS +RustBuffer uniffi_offline_protocol_uniffi_fn_method_offlineprotocol_get_custody_stats(uint64_t ptr, RustCallStatus *_Nonnull out_status +); +#endif #ifndef UNIFFI_FFIDEF_UNIFFI_OFFLINE_PROTOCOL_UNIFFI_FN_METHOD_OFFLINEPROTOCOL_GET_DEDUP_STATS #define UNIFFI_FFIDEF_UNIFFI_OFFLINE_PROTOCOL_UNIFFI_FN_METHOD_OFFLINEPROTOCOL_GET_DEDUP_STATS RustBuffer uniffi_offline_protocol_uniffi_fn_method_offlineprotocol_get_dedup_stats(uint64_t ptr, RustCallStatus *_Nonnull out_status @@ -2188,6 +2198,12 @@ uint16_t uniffi_offline_protocol_uniffi_checksum_method_offlineprotocol_enable_t #define UNIFFI_FFIDEF_UNIFFI_OFFLINE_PROTOCOL_UNIFFI_CHECKSUM_METHOD_OFFLINEPROTOCOL_END_TELEMETRY_SESSION uint16_t uniffi_offline_protocol_uniffi_checksum_method_offlineprotocol_end_telemetry_session(void +); +#endif +#ifndef UNIFFI_FFIDEF_UNIFFI_OFFLINE_PROTOCOL_UNIFFI_CHECKSUM_METHOD_OFFLINEPROTOCOL_ERASE_CUSTODY +#define UNIFFI_FFIDEF_UNIFFI_OFFLINE_PROTOCOL_UNIFFI_CHECKSUM_METHOD_OFFLINEPROTOCOL_ERASE_CUSTODY +uint16_t uniffi_offline_protocol_uniffi_checksum_method_offlineprotocol_erase_custody(void + ); #endif #ifndef UNIFFI_FFIDEF_UNIFFI_OFFLINE_PROTOCOL_UNIFFI_CHECKSUM_METHOD_OFFLINEPROTOCOL_ESTABLISH_SECURE_SESSION @@ -2254,6 +2270,12 @@ uint16_t uniffi_offline_protocol_uniffi_checksum_method_offlineprotocol_get_batt #define UNIFFI_FFIDEF_UNIFFI_OFFLINE_PROTOCOL_UNIFFI_CHECKSUM_METHOD_OFFLINEPROTOCOL_GET_BLOCKED_USERS uint16_t uniffi_offline_protocol_uniffi_checksum_method_offlineprotocol_get_blocked_users(void +); +#endif +#ifndef UNIFFI_FFIDEF_UNIFFI_OFFLINE_PROTOCOL_UNIFFI_CHECKSUM_METHOD_OFFLINEPROTOCOL_GET_CUSTODY_STATS +#define UNIFFI_FFIDEF_UNIFFI_OFFLINE_PROTOCOL_UNIFFI_CHECKSUM_METHOD_OFFLINEPROTOCOL_GET_CUSTODY_STATS +uint16_t uniffi_offline_protocol_uniffi_checksum_method_offlineprotocol_get_custody_stats(void + ); #endif #ifndef UNIFFI_FFIDEF_UNIFFI_OFFLINE_PROTOCOL_UNIFFI_CHECKSUM_METHOD_OFFLINEPROTOCOL_GET_DEDUP_STATS diff --git a/bindings/react-native/ios/MeshRelayConfigReader.swift b/bindings/react-native/ios/MeshRelayConfigReader.swift index 6ad152e75..75201ce3c 100644 --- a/bindings/react-native/ios/MeshRelayConfigReader.swift +++ b/bindings/react-native/ios/MeshRelayConfigReader.swift @@ -93,9 +93,11 @@ enum MeshRelayConfigReader { private static func number(_ dict: [String: Any], _ keys: [String]) -> NSNumber? { for key in keys { - // Bool is bridged as NSNumber, so an explicit exclusion keeps a - // stray `true` from arriving as the number 1. - if let value = dict[key] as? NSNumber, !(dict[key] is Bool) { + // A JSON boolean also arrives as an NSNumber, so it is excluded by + // its CoreFoundation type rather than by `is Bool`: Swift bridges + // the numbers 0 and 1 to Bool as well, and testing that read a + // legitimate `fanout: 1` or `activityIdleWindows: 1` as unset. + if let value = dict[key] as? NSNumber, CFGetTypeID(value) != CFBooleanGetTypeID() { return value } } diff --git a/bindings/react-native/ios/OfflineProtocolModule.m b/bindings/react-native/ios/OfflineProtocolModule.m index 348a6d6de..3ce6e010b 100644 --- a/bindings/react-native/ios/OfflineProtocolModule.m +++ b/bindings/react-native/ios/OfflineProtocolModule.m @@ -280,6 +280,12 @@ @interface RCT_EXTERN_MODULE(OfflineProtocolModule, RCTEventEmitter) RCT_EXTERN_METHOD(getMeshRelayTunables:(RCTPromiseResolveBlock)resolve rejecter:(RCTPromiseRejectBlock)reject) +RCT_EXTERN_METHOD(getCustodyStats:(RCTPromiseResolveBlock)resolve + rejecter:(RCTPromiseRejectBlock)reject) + +RCT_EXTERN_METHOD(eraseCustody:(RCTPromiseResolveBlock)resolve + rejecter:(RCTPromiseRejectBlock)reject) + RCT_EXTERN_METHOD(getPendingAckCount:(RCTPromiseResolveBlock)resolve rejecter:(RCTPromiseRejectBlock)reject) diff --git a/bindings/react-native/ios/OfflineProtocolModule.swift b/bindings/react-native/ios/OfflineProtocolModule.swift index 189ea8ddb..a08d2f14b 100644 --- a/bindings/react-native/ios/OfflineProtocolModule.swift +++ b/bindings/react-native/ios/OfflineProtocolModule.swift @@ -580,6 +580,26 @@ class OfflineProtocolModule: RCTEventEmitter { ) } + // Custody section: read by CustodyConfigReader, Foundation-only like + // the mesh forwarding reader and for the same reason; mirrors + // ProtocolConfigParser.kt, keep the read order in sync. Absent stays + // absent all the way to the core, whose default is off; the reader + // carries the overflow policy as the app spelled it, and an unknown + // spelling stays absent rather than becoming a default written here. + let custody = CustodyConfigReader.read(raw).map { values in + CustodyConfig( + enabled: values.enabled, + holdMs: values.holdMs, + maxEntriesPerDepositor: values.maxEntriesPerDepositor, + maxBytesPerDepositor: values.maxBytesPerDepositor, + maxEntries: values.maxEntries, + maxBytes: values.maxBytes, + strangerMaxEntries: values.strangerMaxEntries, + strangerMaxBytes: values.strangerMaxBytes, + overflowPolicy: values.overflowPolicy.flatMap(custodyOverflowPolicy) + ) + } + // Data layer section (nested home under `data`, both cases). Same // rule as meshRelay: absent stays absent, so the Rust default is the // only default. Kept in step with android/ ProtocolConfigParser — @@ -637,7 +657,8 @@ class OfflineProtocolModule: RCTEventEmitter { compactEnvelopeEnabled: encryption.compactEnvelopeEnabled, richPayloadEnabled: encryption.richPayloadEnabled, cryptoRecoveryEnabled: encryption.cryptoRecoveryEnabled, - meshRelay: meshRelay + meshRelay: meshRelay, + custody: custody ) // Assigned only when the app actually sent it. Writing `?? false` @@ -656,6 +677,20 @@ class OfflineProtocolModule: RCTEventEmitter { return (config, raw) } + /// The custody overflow policy as the app spelled it, or nil for a + /// spelling this build does not know: nil reaches the core as "keep the + /// default", never as a default chosen here. + private func custodyOverflowPolicy(_ raw: String) -> OverflowPolicy? { + switch raw.lowercased() { + case "drop_newest", "dropnewest": + return .dropNewest + case "drop_oldest", "dropoldest": + return .dropOldest + default: + return nil + } + } + /// Accepts both the current vocabulary and the pre-0.22 `low`/`medium`/`high` /// spelling of the same three values, so an app that has not migrated its /// config keeps working. @@ -3832,6 +3867,60 @@ class OfflineProtocolModule: RCTEventEmitter { resolver(tunablesDict) } + /// Custody counters, read through to the Rust core (docs/spec/custody.md). + @objc func getCustodyStats(_ resolver: @escaping RCTPromiseResolveBlock, + rejecter: @escaping RCTPromiseRejectBlock) { + guard let proto = protocolInstance else { + rejecter("ERROR_STATS", "Protocol not initialized", nil) + return + } + let stats = proto.getCustodyStats() + let statsDict: [String: Any] = [ + "held": stats.held, + "heldBytes": stats.heldBytes, + "accepted": stats.accepted, + "delivered": stats.delivered, + "reOriginated": stats.reOriginated, + "expired": stats.expired, + "duplicates": stats.duplicates, + "evicted": stats.evicted, + "receiptsSent": stats.receiptsSent, + "receiptsDropped": stats.receiptsDropped, + "receiptsReceived": stats.receiptsReceived, + "receiptsIgnored": stats.receiptsIgnored, + "refusedDisabled": stats.refusedDisabled, + "refusedNoRequest": stats.refusedNoRequest, + "refusedUnknownClass": stats.refusedUnknownClass, + "refusedNotSealed": stats.refusedNotSealed, + "refusedUnprovenPeer": stats.refusedUnprovenPeer, + "refusedNotDepositor": stats.refusedNotDepositor, + "refusedStranger": stats.refusedStranger, + "refusedDepositorFull": stats.refusedDepositorFull, + "refusedStoreFull": stats.refusedStoreFull, + "refusedBattery": stats.refusedBattery + ] + resolver(statsDict) + } + + /// Drops every held frame and resets the custody counters. The data + /// layer's wipe calls the same erase in the core; this is the standalone + /// verb. + @objc func eraseCustody(_ resolver: @escaping RCTPromiseResolveBlock, + rejecter: @escaping RCTPromiseRejectBlock) { + do { + guard let proto = protocolInstance else { + throw NSError(domain: "OfflineProtocol", code: -1, + userInfo: [NSLocalizedDescriptionKey: "Protocol not initialized"]) + } + try proto.eraseCustody() + resolver(nil) + } catch { + rejectWithProtocolError(error, rejecter, + fallbackCode: "ERROR_ERASECUSTODY", + fallbackMessage: "eraseCustody failed") + } + } + @objc func getPendingAckCount(_ resolver: @escaping RCTPromiseResolveBlock, rejecter: @escaping RCTPromiseRejectBlock) { guard let proto = protocolInstance else { diff --git a/bindings/react-native/ios/Package.swift b/bindings/react-native/ios/Package.swift index 20718dc46..b47a8482a 100644 --- a/bindings/react-native/ios/Package.swift +++ b/bindings/react-native/ios/Package.swift @@ -68,6 +68,7 @@ let package = Package( "InboundFragmentBuffer.swift", "LegacyRelayMessage.swift", "LegacyStoreAdoption.swift", + "CustodyConfigReader.swift", "MeshRelayConfigReader.swift", "MlsSecureStorage.swift", "MonotonicClock.swift", @@ -126,6 +127,7 @@ let package = Package( "InboundFragmentBufferTests.swift", "LegacyRelayMessageTests.swift", "LegacyStoreAdoptionTests.swift", + "CustodyConfigReaderTests.swift", "MeshRelayConfigReaderTests.swift", "NostrQueryTrackerTests.swift", "OutboundFragmentQueueTests.swift", diff --git a/bindings/react-native/ios/tests/CustodyConfigReaderTests.swift b/bindings/react-native/ios/tests/CustodyConfigReaderTests.swift new file mode 100644 index 000000000..077ab28ec --- /dev/null +++ b/bindings/react-native/ios/tests/CustodyConfigReaderTests.swift @@ -0,0 +1,105 @@ +import XCTest +@testable import OfflineProtocol + +/// Mirrors android/ `ProtocolConfigParserTest`'s custody cases, and keeps the +/// two suites in sync. +/// +/// The property under test is the one the mesh forwarding reader pins: this +/// reader resolves nothing. Absent must stay absent all the way to the core, +/// whose default for custody is off, and a reader that wrote that default as a +/// literal would keep every app that omitted the section on it forever. +final class CustodyConfigReaderTests: XCTestCase { + + private func read(_ json: String) throws -> CustodyConfigValues? { + let data = try XCTUnwrap(json.data(using: .utf8)) + let raw = try XCTUnwrap( + JSONSerialization.jsonObject(with: data) as? [String: Any] + ) + return CustodyConfigReader.read(raw) + } + + func testSectionIsAbsentWhenOmitted() throws { + // Nil, not an object of nils: the module passes nil across the FFI and + // the core keeps every default untouched, off included. + XCTAssertNil(try read(#"{"appId":"app","userId":"alice"}"#)) + } + + func testSectionReadsItsNestedCamelCaseHome() throws { + let values = try XCTUnwrap(try read(#""" + {"appId":"app","custody":{"enabled":true,"holdMs":3600000,"maxEntriesPerDepositor":16,"maxBytesPerDepositor":131072,"maxEntries":128,"maxBytes":4194304,"strangerMaxEntries":2,"strangerMaxBytes":65536,"overflowPolicy":"drop_newest"}} + """#)) + + XCTAssertEqual(values.enabled, true) + XCTAssertEqual(values.holdMs, 3_600_000) + XCTAssertEqual(values.maxEntriesPerDepositor, 16) + XCTAssertEqual(values.maxBytesPerDepositor, 131_072) + XCTAssertEqual(values.maxEntries, 128) + XCTAssertEqual(values.maxBytes, 4_194_304) + XCTAssertEqual(values.strangerMaxEntries, 2) + XCTAssertEqual(values.strangerMaxBytes, 65_536) + XCTAssertEqual(values.overflowPolicy, "drop_newest") + } + + func testSectionReadsNestedSnakeCase() throws { + let values = try XCTUnwrap(try read(#""" + {"appId":"app","custody":{"enabled":false,"hold_ms":7200000,"max_entries_per_depositor":8,"max_bytes_per_depositor":65536,"max_entries":64,"max_bytes":1048576,"stranger_max_entries":1,"stranger_max_bytes":65536,"overflow_policy":"drop_oldest"}} + """#)) + + XCTAssertEqual(values.enabled, false) + XCTAssertEqual(values.holdMs, 7_200_000) + XCTAssertEqual(values.maxEntriesPerDepositor, 8) + XCTAssertEqual(values.maxBytesPerDepositor, 65_536) + XCTAssertEqual(values.maxEntries, 64) + XCTAssertEqual(values.maxBytes, 1_048_576) + XCTAssertEqual(values.strangerMaxEntries, 1) + XCTAssertEqual(values.strangerMaxBytes, 65_536) + XCTAssertEqual(values.overflowPolicy, "drop_oldest") + } + + func testLeavesUnnamedFieldsNil() throws { + // The ordinary case: an app switches custody on and names nothing + // else. Every other field must arrive nil so the core keeps its own + // value. + let values = try XCTUnwrap(try read(#"{"appId":"app","custody":{"enabled":true}}"#)) + XCTAssertEqual(values.enabled, true) + XCTAssertNil(values.holdMs) + XCTAssertNil(values.maxEntriesPerDepositor) + XCTAssertNil(values.maxBytesPerDepositor) + XCTAssertNil(values.maxEntries) + XCTAssertNil(values.maxBytes) + XCTAssertNil(values.strangerMaxEntries) + XCTAssertNil(values.strangerMaxBytes) + XCTAssertNil(values.overflowPolicy) + } + + func testAnEmptySectionIsPresentAndEmpty() throws { + // Present but naming nothing: still an object, every field nil, so the + // module sends an all-nil dictionary and the core still keeps every + // default. Distinguished from absent only in that it proves the app + // meant to mention custody. + let values = try XCTUnwrap(try read(#"{"appId":"app","custody":{}}"#)) + XCTAssertEqual(values, CustodyConfigValues()) + } + + func testNegativeNumbersClampToZeroRatherThanTrapping() throws { + // App-supplied JS. A negative would trap the unsigned initializer; + // clamped low it reaches the core's own validation. + let values = try XCTUnwrap(try read(#"{"appId":"app","custody":{"holdMs":-5,"maxEntries":-1}}"#)) + XCTAssertEqual(values.holdMs, 0) + XCTAssertEqual(values.maxEntries, 0) + } + + func testABooleanIsNotReadAsANumber() throws { + let values = try XCTUnwrap(try read(#"{"appId":"app","custody":{"holdMs":true}}"#)) + XCTAssertNil(values.holdMs) + } + + func testZeroAndOneAreNumbersNotBooleans() throws { + // Swift bridges the numbers 0 and 1 to Bool, so an `is Bool` exclusion + // reads both as unset. A stranger tier of one entry, or a hold the core + // must refuse, has to arrive. + let values = try XCTUnwrap(try read(#"{"appId":"app","custody":{"strangerMaxEntries":1,"holdMs":0}}"#)) + XCTAssertEqual(values.strangerMaxEntries, 1) + XCTAssertEqual(values.holdMs, 0) + } +} diff --git a/bindings/react-native/ios/tests/MeshRelayConfigReaderTests.swift b/bindings/react-native/ios/tests/MeshRelayConfigReaderTests.swift index db60278e3..5fb20b657 100644 --- a/bindings/react-native/ios/tests/MeshRelayConfigReaderTests.swift +++ b/bindings/react-native/ios/tests/MeshRelayConfigReaderTests.swift @@ -18,6 +18,16 @@ final class MeshRelayConfigReaderTests: XCTestCase { return MeshRelayConfigReader.read(raw) } + func testZeroAndOneAreNumbersNotBooleans() throws { + // Swift bridges the numbers 0 and 1 to Bool, so an `is Bool` exclusion + // read `fanout: 1` and `activityIdleWindows: 1` as unset, and the app's + // dial silently stayed at the default. + let values = try XCTUnwrap(try read(#"{"appId":"app","meshRelay":{"fanout":1,"activityIdleWindows":1,"jitterMinMs":0}}"#)) + XCTAssertEqual(values.fanout, 1) + XCTAssertEqual(values.activityIdleWindows, 1) + XCTAssertEqual(values.jitterMinMs, 0) + } + func testSectionIsAbsentWhenOmitted() throws { // Nil, not an object of nils: the module passes nil across the FFI and // the core keeps every default untouched. diff --git a/bindings/react-native/js-ci-harness/custody-config.test.js b/bindings/react-native/js-ci-harness/custody-config.test.js new file mode 100644 index 000000000..c7bd7a27e --- /dev/null +++ b/bindings/react-native/js-ci-harness/custody-config.test.js @@ -0,0 +1,232 @@ +#!/usr/bin/env node +/** + * Behavioral tests for the JS-layer marshalling of the custody configuration + * section and the two custody methods (`src/index.ts`). + * + * Drives the *real compiled* SDK against a stubbed native module and asserts + * on the payloads it hands over, because every failure in this layer is + * silent: a config field the bridge fills in with a literal makes the Rust + * default unreachable, and for custody the default that matters is "off". A + * bridge that sent `{ enabled: false }` for an app that never mentioned + * custody would keep every such app off forever, after the release that + * flips the default. That is why the assertions below check for *absence* as + * hard as they check for presence. + * + * The sibling Rust guard `every_bridge_reads_the_custody_config_section` pins + * that both native parsers read the section this file proves JS sends. + * + * See README.md for why the package has no other JS test setup. + */ +'use strict'; + +const assert = require('node:assert/strict'); +const { execFileSync } = require('node:child_process'); +const fs = require('node:fs'); +const Module = require('node:module'); +const os = require('node:os'); +const path = require('node:path'); + +const PACKAGE_DIR = path.resolve(__dirname, '..'); + +function compileSdk() { + const tsc = path.join(PACKAGE_DIR, 'node_modules', 'typescript', 'bin', 'tsc'); + if (!fs.existsSync(tsc)) { + throw new Error(`TypeScript not found at ${tsc}: run \`npm ci\` in ${PACKAGE_DIR} first.`); + } + const outDir = fs.mkdtempSync(path.join(os.tmpdir(), 'op-rn-custody-')); + execFileSync( + process.execPath, + [tsc, '--outDir', outDir, '--declaration', 'false', '--declarationMap', 'false'], + { cwd: PACKAGE_DIR, stdio: 'inherit' } + ); + return outDir; +} + +let nativeOverrides = {}; +let nativeCalls = []; + +const nativeModule = new Proxy( + {}, + { + get(_target, method) { + if (typeof method !== 'string') return undefined; + return (...args) => { + nativeCalls.push({ method, args }); + const override = nativeOverrides[method]; + return override ? override(...args) : Promise.resolve(); + }; + }, + } +); + +class StubNativeEventEmitter { + addListener() { + return { remove: () => {} }; + } +} + +const realLoad = Module._load; +Module._load = function loadWithReactNativeStub(request) { + if (request === 'react-native') { + return { + NativeModules: { OfflineProtocolModule: nativeModule }, + NativeEventEmitter: StubNativeEventEmitter, + }; + } + return realLoad.apply(this, arguments); +}; + +const realConsole = { log: console.log, warn: console.warn, error: console.error }; + +function captureConsole() { + console.log = () => {}; + console.warn = () => {}; + console.error = () => {}; +} + +function releaseConsole() { + Object.assign(console, realConsole); +} + +const tests = []; +const test = (name, fn) => tests.push({ name, fn }); + +let OfflineProtocol; + +const newSdk = (config = {}) => + new OfflineProtocol({ appId: 'harness', profile: 'harness-profile', ...config }); + +function onlyCall(method) { + const matches = nativeCalls.filter((c) => c.method === method); + assert.equal(matches.length, 1, `expected exactly one ${method} call, saw ${matches.length}`); + return matches[0]; +} + +function payloadOf(method) { + return JSON.parse(onlyCall(method).args[0]); +} + +// --------------------------------------------------------------------------- +// The create-time custody section +// --------------------------------------------------------------------------- + +test('the custody section reaches native field-for-field when the app sets it', async () => { + const sdk = newSdk({ + custody: { + enabled: true, + holdMs: 3600000, + maxEntriesPerDepositor: 16, + maxBytesPerDepositor: 131072, + maxEntries: 128, + maxBytes: 4194304, + strangerMaxEntries: 2, + strangerMaxBytes: 65536, + overflowPolicy: 'drop_newest', + }, + }); + await sdk.start(); + + assert.deepEqual(payloadOf('create').custody, { + enabled: true, + holdMs: 3600000, + maxEntriesPerDepositor: 16, + maxBytesPerDepositor: 131072, + maxEntries: 128, + maxBytes: 4194304, + strangerMaxEntries: 2, + strangerMaxBytes: 65536, + overflowPolicy: 'drop_newest', + }); +}); + +test('an unset custody section is absent from the create payload', async () => { + // Absence is the assertion. A bridge that sent `{ enabled: false }` here + // would make the Rust default unreachable, and nothing would report it. + const sdk = newSdk({}); + await sdk.start(); + + assert.equal( + 'custody' in payloadOf('create'), + false, + 'an unconfigured custody section must not be materialised by the bridge' + ); +}); + +test('a partial custody section carries only the fields it names', async () => { + // The ordinary case: an app switches custody on and names nothing else. + // Every unnamed field must stay absent so the core keeps its own value. + const sdk = newSdk({ custody: { enabled: true } }); + await sdk.start(); + + assert.deepEqual(payloadOf('create').custody, { enabled: true }); +}); + +test('a custody section that switches it off still crosses the bridge', async () => { + // Explicitly off is not the same as unset: it must reach native, so an app + // can turn custody off after the default ever flips on. + const sdk = newSdk({ custody: { enabled: false } }); + await sdk.start(); + + assert.deepEqual(payloadOf('create').custody, { enabled: false }); +}); + +// --------------------------------------------------------------------------- +// The two custody methods +// --------------------------------------------------------------------------- + +test('getCustodyStats reads through to native unchanged', async () => { + const stats = { held: 2, heldBytes: 4096, accepted: 3, delivered: 1, refusedStranger: 4 }; + nativeOverrides.getCustodyStats = () => Promise.resolve(stats); + const sdk = newSdk({}); + await sdk.start(); + + assert.deepEqual(await sdk.getCustodyStats(), stats); + onlyCall('getCustodyStats'); +}); + +test('eraseCustody is its own call, distinct from the data wipe', async () => { + const sdk = newSdk({}); + await sdk.start(); + await sdk.eraseCustody(); + + onlyCall('eraseCustody'); + assert.equal( + nativeCalls.some((c) => c.method === 'dataWipeAll'), + false + ); +}); + +// --------------------------------------------------------------------------- +// Runner +// --------------------------------------------------------------------------- + +(async () => { + const outDir = compileSdk(); + try { + ({ OfflineProtocol } = require(path.join(outDir, 'index.js'))); + + let failed = 0; + for (const { name, fn } of tests) { + nativeOverrides = {}; + nativeCalls = []; + captureConsole(); + try { + await fn(); + releaseConsole(); + realConsole.log(` ✓ ${name}`); + } catch (error) { + failed += 1; + releaseConsole(); + realConsole.log(` ✗ ${name}\n ${error.message}`); + } + } + + realConsole.log( + failed === 0 ? `\n${tests.length} passed.` : `\n${failed} of ${tests.length} FAILED.` + ); + process.exitCode = failed === 0 ? 0 : 1; + } finally { + releaseConsole(); + fs.rmSync(outDir, { recursive: true, force: true }); + } +})(); diff --git a/bindings/react-native/node_modules b/bindings/react-native/node_modules new file mode 120000 index 000000000..ef573485e --- /dev/null +++ b/bindings/react-native/node_modules @@ -0,0 +1 @@ +/Users/goku/projects/offline/offline-protocol-sdk/bindings/react-native/node_modules \ No newline at end of file diff --git a/bindings/react-native/package.json b/bindings/react-native/package.json index 3332bff4e..2b9599263 100644 --- a/bindings/react-native/package.json +++ b/bindings/react-native/package.json @@ -29,7 +29,7 @@ "scripts": { "prepare": "tsc", "build": "tsc", - "test:js": "node js-ci-harness/one-shot-hold.test.js && node js-ci-harness/local-address.test.js && node js-ci-harness/forward-priority.test.js && node js-ci-harness/rich-send-app-id.test.js && node js-ci-harness/relay-config.test.js && node js-ci-harness/data-config.test.js && node js-ci-harness/security-config.test.js && node js-ci-harness/telemetry-config.test.js", + "test:js": "node js-ci-harness/one-shot-hold.test.js && node js-ci-harness/local-address.test.js && node js-ci-harness/forward-priority.test.js && node js-ci-harness/rich-send-app-id.test.js && node js-ci-harness/relay-config.test.js && node js-ci-harness/data-config.test.js && node js-ci-harness/security-config.test.js && node js-ci-harness/telemetry-config.test.js && node js-ci-harness/custody-config.test.js", "build:ios": "bash scripts/build-ios.sh", "build:android": "bash scripts/build-android.sh", "build:all": "bash scripts/build-all.sh", diff --git a/bindings/react-native/src/index.ts b/bindings/react-native/src/index.ts index b5e6e34e4..0980552c5 100644 --- a/bindings/react-native/src/index.ts +++ b/bindings/react-native/src/index.ts @@ -46,6 +46,8 @@ import type { MeshRelayConfig, MeshRelayStats, MeshRelayTunables, + CustodyConfig, + CustodyStats, MlsKeyPackage, MlsEncryptedMessage, MlsWelcome, @@ -180,6 +182,7 @@ interface NativeConfig { enforceAdminCommits?: boolean; }; meshRelay?: MeshRelayConfig; + custody?: CustodyConfig; data?: { enabled?: boolean; }; @@ -545,6 +548,27 @@ export class OfflineProtocol { } } + // Custody section. Nested only, forwarded field-for-field with no + // defaults filled in, for the reason the mesh forwarding section gives: + // the core owns every default, and its default here is off. A `?? false` + // on `enabled` would keep every app that omits the section off forever. + if (this.config.custody) { + const custodyConfig = sanitize({ + enabled: this.config.custody.enabled, + holdMs: this.config.custody.holdMs, + maxEntriesPerDepositor: this.config.custody.maxEntriesPerDepositor, + maxBytesPerDepositor: this.config.custody.maxBytesPerDepositor, + maxEntries: this.config.custody.maxEntries, + maxBytes: this.config.custody.maxBytes, + strangerMaxEntries: this.config.custody.strangerMaxEntries, + strangerMaxBytes: this.config.custody.strangerMaxBytes, + overflowPolicy: this.config.custody.overflowPolicy, + }); + if (custodyConfig) { + nativeConfig.custody = custodyConfig; + } + } + // Data layer section. Same rule as meshRelay above: forwarded // field-for-field with no defaults filled in, so an omitted field stays // omitted all the way to the core and the default lives in exactly one @@ -2304,6 +2328,33 @@ export class OfflineProtocol { return await OfflineProtocolNativeModule.getMeshRelayTunables(); } + /** + * What this device holds for its neighbours and has done as a depositor + * (docs/spec/custody.md), read through to the Rust core. + * + * `held` and `heldBytes` are gauges; everything else is cumulative. Every + * refusal reason in the acceptance table has a counter, so an operator who + * enabled custody and sees nothing held can tell "off" from "nobody asked" + * from "everyone refused". + * + * @returns Custody counters + */ + async getCustodyStats(): Promise { + return await OfflineProtocolNativeModule.getCustodyStats(); + } + + /** + * Drops every held frame and resets the custody counters. + * + * Callable on its own; the data layer's `wipeAll()` erases custody in the + * core as well, because a custody store that survived a logout would hold + * other people's traffic past the point the user asked for erasure. Fails + * only when a record could not be deleted, after attempting every one. + */ + async eraseCustody(): Promise { + await OfflineProtocolNativeModule.eraseCustody(); + } + /** * Gets the number of pending ACKs waiting for confirmation * diff --git a/bindings/react-native/src/types.ts b/bindings/react-native/src/types.ts index 0720ca217..37a82ec1d 100644 --- a/bindings/react-native/src/types.ts +++ b/bindings/react-native/src/types.ts @@ -227,6 +227,107 @@ export interface MeshRelayStats { droppedForCapacity: number; } +/** + * Custody: holding a neighbour's replication frames for hours instead of the + * seconds a forwarder gives them (docs/spec/custody.md). + * + * Off by default, and every dial is the custodian's. Every field is optional + * and an omitted one keeps the core's default rather than being restated by + * the bridge; the default that matters most is `enabled: false`, and a + * literal written here would keep every app that omits the section on it + * after the release that ever flips it. The core validates the bounds: the + * hold strictly shorter than the outbox lifetime, positive session-tier caps, + * a byte cap that admits one replication frame, and a stranger tier that is + * both zero or both set. Applied at construction; there is no runtime update. + */ +export interface CustodyConfig { + /** Whether this device accepts deposits. The off switch (default: false) */ + enabled?: boolean; + /** + * How long an accepted frame is held, in ms (default: 21600000, six hours). + * Judged in wall time against the value in force at each sweep, so lowering + * it expires records already held. + */ + holdMs?: number; + /** Held frames one depositor with an established session may have at once (default: 64) */ + maxEntriesPerDepositor?: number; + /** Bytes one depositor with an established session may have at once (default: 2 MiB) */ + maxBytesPerDepositor?: number; + /** Held frames across every depositor (default: 512) */ + maxEntries?: number; + /** Bytes across every depositor (default: 16 MiB) */ + maxBytes?: number; + /** + * Held frames one proven peer without a session may have at once + * (default: 0, which refuses such peers). Set together with + * `strangerMaxBytes` or not at all. + */ + strangerMaxEntries?: number; + /** Bytes one proven peer without a session may have at once (default: 0) */ + strangerMaxBytes?: number; + /** + * What happens when a budget is full: evict the oldest held frame to admit + * the new one, or refuse the new one (default: `drop_oldest`). + */ + overflowPolicy?: 'drop_oldest' | 'drop_newest'; +} + +/** + * What this device has done as a custodian and as a depositor, and what it + * is holding right now. See `getCustodyStats()`. + * + * `held` and `heldBytes` are gauges. Everything else is cumulative since + * start-up or the last `eraseCustody()`, so a rate is a difference between + * two reads. Every refusal reason in the acceptance table has a counter, so + * "custody is off" can be told from "nobody asked". + */ +export interface CustodyStats { + /** Frames in custody right now */ + held: number; + /** Bytes in custody right now */ + heldBytes: number; + /** Deposits accepted */ + accepted: number; + /** Held frames handed to their recipient directly, and released */ + delivered: number; + /** Held frames re-originated toward a neighbour that is not the recipient */ + reOriginated: number; + /** Held frames dropped at the end of their hold */ + expired: number; + /** Deposits of an identifier already held: not stored, not answered */ + duplicates: number; + /** Held frames evicted to admit a newer deposit under drop-oldest */ + evicted: number; + /** Receipts put on the arrival link */ + receiptsSent: number; + /** Receipts not sent: the depositor does not parse them, the link was gone, or this device cannot sign */ + receiptsDropped: number; + /** Receipts this device received for an entry still in its outbox */ + receiptsReceived: number; + /** Receipts this device received naming nothing in its outbox, or that it could not parse */ + receiptsIgnored: number; + /** Refused: custody is off */ + refusedDisabled: number; + /** Refused: no deposit request on the frame */ + refusedNoRequest: number; + /** Refused: unknown class token */ + refusedUnknownClass: number; + /** Refused: not a sealed frame */ + refusedNotSealed: number; + /** Refused: the arrival link was not identified */ + refusedUnprovenPeer: number; + /** Refused: the sender is not the peer the frame arrived from */ + refusedNotDepositor: number; + /** Refused: a peer without a session while the stranger tier is closed */ + refusedStranger: number; + /** Refused: the depositor's budget is full */ + refusedDepositorFull: number; + /** Refused: the global budget is full */ + refusedStoreFull: number; + /** Refused: the battery is below the relay floor */ + refusedBattery: number; +} + /** * Deduplicator statistics for monitoring */ @@ -779,6 +880,13 @@ export interface ProtocolConfig { * `getMeshRelayTunables()`. */ meshRelay?: MeshRelayConfig; + /** + * Custody: whether this device holds a neighbour's replication frames for + * hours, and under what quotas (docs/spec/custody.md). Off by default, and + * an omitted field keeps the core's default rather than being restated + * here. Applied at construction; there is no runtime update. + */ + custody?: CustodyConfig; /** Relay configuration (optional) */ relay?: RelayConfig; /** Network configuration (optional) */ diff --git a/crates/offline-protocol-uniffi/src/lib.rs b/crates/offline-protocol-uniffi/src/lib.rs index e0b5d276a..6c40e96c3 100644 --- a/crates/offline-protocol-uniffi/src/lib.rs +++ b/crates/offline-protocol-uniffi/src/lib.rs @@ -14,13 +14,14 @@ mod host_log; use offline_protocol::{ - AppState as CoreAppState, EstablishmentState as CoreEstablishmentState, Event as CoreEvent, - GatewayCarrier, MediaSendOptions as CoreMediaSendOptions, - MeshRelayConfig as CoreMeshRelayConfig, MeshRelayStats as CoreMeshRelayStats, - MlsVerbosity as CoreMlsVerbosity, NetworkVisualizer, OfflineProtocol as CoreProtocol, - OverflowPolicy as CoreOverflowPolicy, PendingQueueConfig as CorePendingQueueConfig, - PresenceStatus as CorePresenceStatus, ProtocolConfig as CoreConfig, - ProtocolStateError as CoreProtocolStateError, ProtocolStateResult as CoreProtocolStateResult, + AppState as CoreAppState, CustodyConfig as CoreCustodyConfig, CustodyStats as CoreCustodyStats, + EstablishmentState as CoreEstablishmentState, Event as CoreEvent, GatewayCarrier, + MediaSendOptions as CoreMediaSendOptions, MeshRelayConfig as CoreMeshRelayConfig, + MeshRelayStats as CoreMeshRelayStats, MlsVerbosity as CoreMlsVerbosity, NetworkVisualizer, + OfflineProtocol as CoreProtocol, OverflowPolicy as CoreOverflowPolicy, + PendingQueueConfig as CorePendingQueueConfig, PresenceStatus as CorePresenceStatus, + ProtocolConfig as CoreConfig, ProtocolStateError as CoreProtocolStateError, + ProtocolStateResult as CoreProtocolStateResult, ProtocolStateStorage as CoreProtocolStateStorage, SendMessageOptions as CoreSendMessageOptions, TelemetryConfig as CoreTelemetryConfig, TelemetryHost as CoreTelemetryHost, TelemetryOs as CoreTelemetryOs, TelemetryPipe as CoreTelemetryPipe, @@ -2019,6 +2020,9 @@ pub struct ProtocolConfig { /// Mesh forwarding tunables. `None` (and any `None` field inside it) /// leaves the core default alone — see [`MeshRelayConfig`]. pub mesh_relay: Option, + /// Custody dials. `None` (and any `None` field inside it) leaves the core + /// default alone, which is off. See [`CustodyConfig`]. + pub custody: Option, /// Whether the replicated-document layer accepts work (default off). See /// the UDL dictionary and `DataConfig::enabled` for semantics. pub data_enabled: bool, @@ -2096,6 +2100,147 @@ pub struct MeshRelayStats { pub dropped_for_capacity: u64, } +/// Custody dials, every field optional (`docs/spec/custody.md`). +/// +/// Absent means "leave the core default alone", for the reason +/// [`MeshRelayConfig`] gives: the defaults live in the core and nowhere else, +/// and a partial section from an app moves only the dials it names. +#[derive(Debug, Clone, Default)] +pub struct CustodyConfig { + pub enabled: Option, + pub hold_ms: Option, + pub max_entries_per_depositor: Option, + pub max_bytes_per_depositor: Option, + pub max_entries: Option, + pub max_bytes: Option, + pub stranger_max_entries: Option, + pub stranger_max_bytes: Option, + pub overflow_policy: Option, +} + +impl CustodyConfig { + /// Lays the fields the caller actually set over the core defaults, with + /// the same saturating `u64` to `usize` conversion [`MeshRelayConfig`] + /// uses, for the same 32-bit reason. + fn overlay(self, base: CoreCustodyConfig) -> CoreCustodyConfig { + fn to_usize(value: u64) -> usize { + usize::try_from(value).unwrap_or(usize::MAX) + } + + CoreCustodyConfig { + enabled: self.enabled.unwrap_or(base.enabled), + hold_ms: self.hold_ms.unwrap_or(base.hold_ms), + max_entries_per_depositor: self + .max_entries_per_depositor + .map(to_usize) + .unwrap_or(base.max_entries_per_depositor), + max_bytes_per_depositor: self + .max_bytes_per_depositor + .map(to_usize) + .unwrap_or(base.max_bytes_per_depositor), + max_entries: self.max_entries.map(to_usize).unwrap_or(base.max_entries), + max_bytes: self.max_bytes.map(to_usize).unwrap_or(base.max_bytes), + stranger_max_entries: self + .stranger_max_entries + .map(to_usize) + .unwrap_or(base.stranger_max_entries), + stranger_max_bytes: self + .stranger_max_bytes + .map(to_usize) + .unwrap_or(base.stranger_max_bytes), + overflow_policy: match self.overflow_policy { + Some(OverflowPolicy::DropOldest) => CoreOverflowPolicy::DropOldest, + Some(OverflowPolicy::DropNewest) => CoreOverflowPolicy::DropNewest, + None => base.overflow_policy, + }, + } + } +} + +/// What this device has done as a custodian and as a depositor, and what it +/// is holding right now. See the UDL dictionary for each counter. +#[derive(Debug, Clone)] +pub struct CustodyStats { + pub held: u64, + pub held_bytes: u64, + pub accepted: u64, + pub delivered: u64, + pub re_originated: u64, + pub expired: u64, + pub duplicates: u64, + pub evicted: u64, + pub receipts_sent: u64, + pub receipts_dropped: u64, + pub receipts_received: u64, + pub receipts_ignored: u64, + pub refused_disabled: u64, + pub refused_no_request: u64, + pub refused_unknown_class: u64, + pub refused_not_sealed: u64, + pub refused_unproven_peer: u64, + pub refused_not_depositor: u64, + pub refused_stranger: u64, + pub refused_depositor_full: u64, + pub refused_store_full: u64, + pub refused_battery: u64, +} + +impl From for CustodyStats { + /// Destructured for the reason [`MeshRelayStats`] is: a counter added to + /// the core must break this build rather than stop at the FFI boundary. + fn from(stats: CoreCustodyStats) -> Self { + let CoreCustodyStats { + held, + held_bytes, + accepted, + delivered, + re_originated, + expired, + duplicates, + evicted, + receipts_sent, + receipts_dropped, + receipts_received, + receipts_ignored, + refused_disabled, + refused_no_request, + refused_unknown_class, + refused_not_sealed, + refused_unproven_peer, + refused_not_depositor, + refused_stranger, + refused_depositor_full, + refused_store_full, + refused_battery, + } = stats; + + Self { + held, + held_bytes, + accepted, + delivered, + re_originated, + expired, + duplicates, + evicted, + receipts_sent, + receipts_dropped, + receipts_received, + receipts_ignored, + refused_disabled, + refused_no_request, + refused_unknown_class, + refused_not_sealed, + refused_unproven_peer, + refused_not_depositor, + refused_stranger, + refused_depositor_full, + refused_store_full, + refused_battery, + } + } +} + impl MeshRelayConfig { /// Lays the fields the caller actually set over the core defaults. /// @@ -2289,6 +2434,9 @@ impl From for CoreConfig { if let Some(mesh_relay) = config.mesh_relay { core_config.mesh_relay = mesh_relay.overlay(core_config.mesh_relay); } + if let Some(custody) = config.custody { + core_config.custody = custody.overlay(core_config.custody); + } core_config.data.enabled = config.data_enabled; core_config } @@ -6177,6 +6325,20 @@ impl OfflineProtocol { protocol.mesh_relay_config().into() } + /// What this device holds for its neighbours and has done as a depositor + /// (`docs/spec/custody.md`). Read through to the core. + pub fn get_custody_stats(&self) -> CustodyStats { + let protocol = self.lock_inner_recovering(); + protocol.custody_stats().into() + } + + /// Drops every held frame and resets the custody counters. The data + /// layer's `wipe_all` calls the same erase; this is the standalone verb. + pub fn erase_custody(&self) -> Result<(), ProtocolError> { + let mut protocol = self.lock_inner_recovering(); + protocol.erase_custody().map_err(ProtocolError::from) + } + /// Gets the number of pending ACKs. pub fn get_pending_ack_count(&self) -> u64 { let protocol = self.lock_inner_recovering(); @@ -8147,6 +8309,7 @@ mod tests { fn create_test_config() -> ProtocolConfig { ProtocolConfig { mesh_relay: None, + custody: None, data_enabled: false, binary_wire_enabled: true, nostr_sealing_enabled: true, @@ -8185,6 +8348,7 @@ mod tests { fn create_ble_only_config() -> ProtocolConfig { ProtocolConfig { mesh_relay: None, + custody: None, data_enabled: false, binary_wire_enabled: true, nostr_sealing_enabled: true, @@ -8500,6 +8664,7 @@ mod tests { fn create_reticulum_config() -> ProtocolConfig { ProtocolConfig { mesh_relay: None, + custody: None, data_enabled: false, binary_wire_enabled: true, nostr_sealing_enabled: true, @@ -18166,6 +18331,371 @@ mod tests { } } + // ==================================================================== + // Custody: the config section and the counters across the bridges + // ==================================================================== + + /// Omitting the custody section, or sending one with nothing set, must + /// leave every core default alone, off included. + #[test] + fn an_absent_custody_section_keeps_every_core_default() { + let defaults = CoreCustodyConfig::default(); + + let core: CoreConfig = create_test_config().into(); + assert_eq!(core.custody, defaults); + assert!(!core.custody.enabled, "the core's default is off"); + + let mut config = create_test_config(); + config.custody = Some(CustodyConfig::default()); + let core: CoreConfig = config.into(); + assert_eq!(core.custody, defaults); + } + + /// A partial section must move exactly the fields it names: switching + /// custody on is the ordinary case, and it must not reset a quota. + #[test] + fn a_partial_custody_section_moves_only_what_it_names() { + let defaults = CoreCustodyConfig::default(); + + let mut config = create_test_config(); + config.custody = Some(CustodyConfig { + enabled: Some(true), + hold_ms: Some(3_600_000), + ..CustodyConfig::default() + }); + let core: CoreConfig = config.into(); + + assert!(core.custody.enabled); + assert_eq!(core.custody.hold_ms, 3_600_000); + assert_eq!( + core.custody.max_entries_per_depositor, + defaults.max_entries_per_depositor + ); + assert_eq!(core.custody.max_bytes, defaults.max_bytes); + assert_eq!( + core.custody.stranger_max_entries, + defaults.stranger_max_entries + ); + assert_eq!(core.custody.overflow_policy, defaults.overflow_policy); + } + + /// Every dial must survive the trip, set to values distinct from the + /// defaults and from each other, so a field wired to the wrong source or + /// dropped entirely fails here rather than in an app. + #[test] + fn every_custody_dial_survives_the_round_trip() { + let mut config = create_test_config(); + config.custody = Some(CustodyConfig { + enabled: Some(true), + hold_ms: Some(1_800_000), + max_entries_per_depositor: Some(11), + max_bytes_per_depositor: Some(131_072), + max_entries: Some(77), + max_bytes: Some(4_194_304), + stranger_max_entries: Some(3), + stranger_max_bytes: Some(65_536), + overflow_policy: Some(OverflowPolicy::DropNewest), + }); + let core: CoreConfig = config.into(); + + assert!(core.custody.enabled); + assert_eq!(core.custody.hold_ms, 1_800_000); + assert_eq!(core.custody.max_entries_per_depositor, 11); + assert_eq!(core.custody.max_bytes_per_depositor, 131_072); + assert_eq!(core.custody.max_entries, 77); + assert_eq!(core.custody.max_bytes, 4_194_304); + assert_eq!(core.custody.stranger_max_entries, 3); + assert_eq!(core.custody.stranger_max_bytes, 65_536); + assert_eq!(core.custody.overflow_policy, CoreOverflowPolicy::DropNewest); + core.validate() + .expect("a complete, consistent section validates"); + } + + /// Every custody dial must be readable by every layer that carries it, in + /// both spellings each bridge accepts, and no layer may restate a default. + /// + /// The same failure the mesh-relay guard pins, with a sharper default: a + /// bridge that wrote `enabled: false` for an app that omitted the section + /// would keep every such app off after the release that ever flips the + /// core's default, with no error anywhere. + #[test] + fn every_bridge_reads_the_custody_config_section() { + fn code_only(source: &str) -> String { + source + .lines() + .map(str::trim) + .filter(|l| !l.starts_with("//") && !l.starts_with('*') && !l.starts_with("/*")) + .collect::>() + .join(" ") + .split_whitespace() + .collect::>() + .join(" ") + } + + fn slice_between<'a>(source: &'a str, start: &str, end: &str) -> &'a str { + let after = source + .split_once(start) + .unwrap_or_else(|| panic!("expected {start:?} in source")) + .1; + after + .split_once(end) + .unwrap_or_else(|| panic!("expected {end:?} after {start:?}")) + .0 + } + + fn camel(snake: &str) -> String { + let mut out = String::new(); + let mut upper = false; + for c in snake.chars() { + if c == '_' { + upper = true; + } else if upper { + out.extend(c.to_uppercase()); + upper = false; + } else { + out.push(c); + } + } + out + } + + let rn_dir = + std::path::Path::new(env!("CARGO_MANIFEST_DIR")).join("../../bindings/react-native"); + let read = |rel: &str| -> String { + let path = rn_dir.join(rel); + std::fs::read_to_string(&path) + .unwrap_or_else(|e| panic!("cannot read {}: {e}", path.display())) + }; + + // camelCase, in UDL order. + const FIELDS: &[&str] = &[ + "enabled", + "holdMs", + "maxEntriesPerDepositor", + "maxBytesPerDepositor", + "maxEntries", + "maxBytes", + "strangerMaxEntries", + "strangerMaxBytes", + "overflowPolicy", + ]; + + let udl = std::fs::read_to_string( + std::path::Path::new(env!("CARGO_MANIFEST_DIR")).join("src/offline_protocol.udl"), + ) + .expect("read udl"); + let udl_fields: Vec = slice_between(&udl, "dictionary CustodyConfig {", "};") + .lines() + .map(str::trim) + .filter(|l| !l.is_empty() && !l.starts_with("//")) + .filter_map(|l| l.trim_end_matches(';').split_whitespace().nth(1)) + .map(camel) + .collect(); + assert_eq!( + udl_fields, FIELDS, + "CustodyConfig gained or lost a field in the UDL. Add it to FIELDS *and* to both \ + bridge parsers, the TypeScript interface and the JS transform, or an app setting \ + it is silently ignored" + ); + assert!( + udl.contains("CustodyConfig? custody = null;"), + "the UDL must default the section to null: absent means off, and only the core \ + may say so" + ); + + let swift = code_only(&read("ios/CustodyConfigReader.swift")); + let kotlin = code_only(slice_between( + &read("android/src/main/java/com/offlineprotocol/ProtocolConfigParser.kt"), + "val custodyJson =", + "val config = ProtocolConfig(", + )); + let types_ts = code_only(slice_between( + &read("src/types.ts"), + "export interface CustodyConfig {", + "}", + )); + let index_ts = code_only(slice_between( + &read("src/index.ts"), + "if (this.config.custody) {", + "nativeConfig.custody = custodyConfig;", + )); + + for field in FIELDS { + let snake = { + let mut out = String::new(); + for c in field.chars() { + if c.is_uppercase() { + out.push('_'); + out.extend(c.to_lowercase()); + } else { + out.push(c); + } + } + out + }; + + assert!( + swift.contains(&format!("\"{field}\"")), + "CustodyConfigReader.swift must read `{field}`" + ); + assert!( + kotlin.contains(&format!("\"{field}\"")), + "ProtocolConfigParser.kt must read `{field}`" + ); + if snake != *field { + assert!( + swift.contains(&format!("\"{snake}\"")), + "CustodyConfigReader.swift must also accept the snake_case `{snake}`" + ); + assert!( + kotlin.contains(&format!("\"{snake}\"")), + "ProtocolConfigParser.kt must also accept the snake_case `{snake}`" + ); + } + assert!( + types_ts.contains(&format!("{field}?:")), + "types.ts CustodyConfig must declare `{field}?:`, or no app can set it" + ); + assert!( + index_ts.contains(&format!("{field}: this.config.custody.{field}")), + "index.ts must forward `{field}` to the native payload, or it never leaves JS" + ); + } + + // No layer restates a default. The Swift reader resolves nothing, the + // Kotlin block leaves every field nullable, and the JS transform + // forwards what the app wrote; a literal in any of them is a second + // copy of a core default, free to drift. + assert!( + !swift.contains("?? "), + "CustodyConfigReader.swift must not fill a field in with a literal" + ); + assert!( + !kotlin.contains("?: false") && !kotlin.contains("?: 0") && !kotlin.contains("?: true"), + "the Kotlin custody block must not restate a default as a literal" + ); + assert!( + kotlin.contains("else -> null") && !kotlin.contains("else -> OverflowPolicy"), + "an overflow policy spelling the Kotlin block does not know must stay null, \ + never become a default chosen in the parser" + ); + assert!( + !index_ts.contains("?? "), + "the JS transform must not restate a custody default" + ); + } + + /// Every custody counter must be reported by both native modules and + /// declared in the TypeScript interface, or apps read a number that is + /// never populated. + #[test] + fn react_native_bridges_report_every_custody_counter() { + fn code_only(source: &str) -> String { + source + .lines() + .map(str::trim) + .filter(|l| !l.starts_with("//") && !l.starts_with('*') && !l.starts_with("/*")) + .collect::>() + .join(" ") + .split_whitespace() + .collect::>() + .join(" ") + } + + fn slice_between<'a>(source: &'a str, start: &str, end: &str) -> &'a str { + let after = source + .split_once(start) + .unwrap_or_else(|| panic!("expected {start:?} in source")) + .1; + after + .split_once(end) + .unwrap_or_else(|| panic!("expected {end:?} after {start:?}")) + .0 + } + + let rn_dir = + std::path::Path::new(env!("CARGO_MANIFEST_DIR")).join("../../bindings/react-native"); + let read = |rel: &str| -> String { + let path = rn_dir.join(rel); + std::fs::read_to_string(&path) + .unwrap_or_else(|e| panic!("cannot read {}: {e}", path.display())) + }; + + let udl = std::fs::read_to_string( + std::path::Path::new(env!("CARGO_MANIFEST_DIR")).join("src/offline_protocol.udl"), + ) + .expect("read udl"); + let fields: Vec = slice_between(&udl, "dictionary CustodyStats {", "};") + .lines() + .map(str::trim) + .filter(|l| !l.is_empty() && !l.starts_with("//")) + .filter_map(|l| l.trim_end_matches(';').split_whitespace().nth(1)) + .map(|snake| { + let mut out = String::new(); + let mut upper = false; + for c in snake.chars() { + if c == '_' { + upper = true; + } else if upper { + out.extend(c.to_uppercase()); + upper = false; + } else { + out.push(c); + } + } + out + }) + .collect(); + assert_eq!( + fields.len(), + 22, + "CustodyStats gained or lost a counter in the UDL; check both native modules and \ + the TypeScript interface, and update this count" + ); + assert_eq!(fields[0], "held"); + assert_eq!(fields[fields.len() - 1], "refusedBattery"); + + let kotlin = code_only(slice_between( + &read("android/src/main/java/com/offlineprotocol/OfflineProtocolModule.kt"), + "fun getCustodyStats(", + "promise.resolve(map)", + )); + let swift = code_only(slice_between( + &read("ios/OfflineProtocolModule.swift"), + "func getCustodyStats(", + "resolver(statsDict)", + )); + let types_ts = code_only(slice_between( + &read("src/types.ts"), + "export interface CustodyStats {", + "}", + )); + + for field in &fields { + assert!( + kotlin.contains(&format!("\"{field}\"")), + "OfflineProtocolModule.kt getCustodyStats must report `{field}`" + ); + assert!( + swift.contains(&format!("\"{field}\"")), + "OfflineProtocolModule.swift getCustodyStats must report `{field}`" + ); + assert!( + types_ts.contains(&format!("{field}:")), + "types.ts CustodyStats must declare `{field}:`, or no app can read it" + ); + } + + // The erase verb crosses every layer too. + assert!(read("ios/OfflineProtocolModule.m").contains("RCT_EXTERN_METHOD(eraseCustody:")); + assert!(read("ios/OfflineProtocolModule.swift").contains("func eraseCustody(")); + assert!( + read("android/src/main/java/com/offlineprotocol/OfflineProtocolModule.kt") + .contains("fun eraseCustody(") + ); + assert!(read("src/index.ts").contains("OfflineProtocolNativeModule.eraseCustody()")); + } + /// battery/relay fields used to be missing from the FFI shape, so *any* /// runtime DORS update silently reset them to 20/30/4 — including one that /// meant to change something else. diff --git a/crates/offline-protocol-uniffi/src/offline_protocol.udl b/crates/offline-protocol-uniffi/src/offline_protocol.udl index eaa701a3e..fb4353389 100644 --- a/crates/offline-protocol-uniffi/src/offline_protocol.udl +++ b/crates/offline-protocol-uniffi/src/offline_protocol.udl @@ -634,6 +634,98 @@ dictionary MeshRelayStats { u64 dropped_for_capacity; }; +// Custody: holding a neighbour's replication frames for hours instead of the +// seconds a forwarder gives them (docs/spec/custody.md). Off by default, and +// every dial is the custodian's: a device that never enables it behaves +// exactly as before. Absent means "keep the core default", like +// MeshRelayConfig, so a section naming one dial moves only that dial. The +// core validates the bounds: the hold strictly shorter than +// retry.outbox_max_lifetime_ms, positive session-tier caps, a byte cap that +// admits one replication frame at its ceiling, and a stranger tier that is +// both zero (the default: no deposits from peers without a session) or both +// set. Applied at construction; there is no runtime update. +dictionary CustodyConfig { + // Whether this device accepts deposits. The off switch. + boolean? enabled = null; + // How long an accepted frame is held, in milliseconds, judged in wall + // time against the value in force at each sweep: lowering it expires + // records already held. + u64? hold_ms = null; + // Held frames one depositor with an established session may have at once. + u64? max_entries_per_depositor = null; + // Bytes one depositor with an established session may have at once. + u64? max_bytes_per_depositor = null; + // Held frames across every depositor. + u64? max_entries = null; + // Bytes across every depositor. + u64? max_bytes = null; + // Held frames one proven peer without a session may have at once. Zero + // means such peers are refused, which is the default. + u64? stranger_max_entries = null; + // Bytes one proven peer without a session may have at once. + u64? stranger_max_bytes = null; + // What happens when a budget is full: evict the oldest held frame to + // admit the new one, or refuse the new one. + OverflowPolicy? overflow_policy = null; +}; + +// What this device has done as a custodian and as a depositor, and what it +// is holding right now (docs/spec/custody.md). +// +// `held` and `held_bytes` are gauges. Everything else is cumulative since +// start-up or the last erase_custody(). Aggregate and never per depositor, +// because a per-depositor answer would be the quota oracle the chapter +// refuses to be. Every refusal reason in the acceptance table has a counter, +// so "custody is off" can be told from "nobody asked". +dictionary CustodyStats { + // Frames in custody right now. + u64 held; + // Bytes in custody right now. + u64 held_bytes; + // Deposits accepted. + u64 accepted; + // Held frames handed to their recipient directly, and released. + u64 delivered; + // Held frames re-originated toward a neighbour that is not the recipient. + u64 re_originated; + // Held frames dropped at the end of their hold. + u64 expired; + // Deposits of an identifier already held: not stored, not answered. + u64 duplicates; + // Held frames evicted to admit a newer deposit under drop-oldest. + u64 evicted; + // Receipts put on the arrival link. + u64 receipts_sent; + // Receipts not sent: the depositor does not parse them, the link was + // gone, or this device cannot sign. + u64 receipts_dropped; + // Receipts this device received for an entry still in its outbox. + u64 receipts_received; + // Receipts this device received naming nothing in its outbox, or that it + // could not parse. + u64 receipts_ignored; + // Refused: custody is off. + u64 refused_disabled; + // Refused: no deposit request on the frame. + u64 refused_no_request; + // Refused: unknown class token. + u64 refused_unknown_class; + // Refused: not a sealed frame. + u64 refused_not_sealed; + // Refused: the arrival link was not identified. + u64 refused_unproven_peer; + // Refused: the sender is not the peer the frame arrived from. + u64 refused_not_depositor; + // Refused: a peer without a session while the stranger tier is closed. + u64 refused_stranger; + // Refused: the depositor's budget is full. + u64 refused_depositor_full; + // Refused: the global budget is full. + u64 refused_store_full; + // Refused: the battery is below the relay floor. + u64 refused_battery; +}; + // ACK configuration dictionary AckConfig { u64 default_timeout_ms; @@ -865,6 +957,10 @@ dictionary ProtocolConfig { // inside it — see MeshRelayConfig. Applied at construction: the governor // takes its snapshot there, and there is no runtime update path. MeshRelayConfig? mesh_relay = null; + // Whether this device holds a neighbour's replication frames for hours, + // and under what quotas (docs/spec/custody.md). Null, and any field left + // null inside it, keeps the core default: off. Applied at construction. + CustodyConfig? custody = null; // Whether the replicated-document layer accepts work. On by default, // now that replication has landed: a space replicates with the peer // whose address names it, and an application that never opens a store @@ -1584,6 +1680,17 @@ interface OfflineProtocol { // exact thing the optional *input* exists to prevent. MeshRelayStats get_mesh_relay_stats(); MeshRelayTunables get_mesh_relay_tunables(); + + // Custody: what this device holds for its neighbours and has done as a + // depositor (docs/spec/custody.md). Read through to the core. + CustodyStats get_custody_stats(); + // Drops every held frame and resets the custody counters. Callable on + // its own; the data layer's wipe_all() calls it too, because a custody + // store that survived a logout would hold other people's traffic past + // the point the user asked for erasure. Attempts every record and fails + // only when one could not be deleted. + [Throws=ProtocolError] + void erase_custody(); // ======================================================================== // MLS (END-TO-END ENCRYPTION) OPERATIONS From 5d6ca5353a7ba8c19758daae518d1f3ea39c2a80 Mon Sep 17 00:00:00 2001 From: bahdotsh Date: Thu, 1 Oct 2026 00:10:39 +0530 Subject: [PATCH 4/7] docs: custody in the mesh, delivery and configuration guides --- CHANGELOG.md | 29 ++++++++++++++ docs/configuration.md | 65 ++++++++++++++++++++++++++++++ docs/mesh.md | 87 ++++++++++++++++++++++++++++++++++++++++ docs/message-delivery.md | 9 +++++ 4 files changed, 190 insertions(+) diff --git a/CHANGELOG.md b/CHANGELOG.md index 62201f19a..a94d652ce 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -15,6 +15,35 @@ archived by series under [docs/changelog/](docs/changelog/); see the ### Added +- **Custody v1.** A device can now hold a neighbour's replication frames for + hours instead of the five seconds a forwarder gives them + (`docs/spec/custody.md`), off by default. `ProtocolConfig::custody` + (`custody` in every binding) switches it on and sets the hold and the + quotas; the hold is validated strictly shorter than the outbox lifetime. The + depositor's engine writes the one-hop request on its own sealed `delta`, + `snap`, `vv` and `blob_gone` frames when it offers them to neighbours, from + the plaintext it retains for re-sealing and never after a restart; every + forwarder strips the request from a third-party frame it transmits. A + custodian judges a frame at the drop point, after the forwarding identifier + is released, so it never blanks its own route; answers a depositor that + advertises `data_versions` entry 7 with the signed `__CUSTODY_RECEIPT__` + once, over the arrival link; redelivers on neighbour discovery through a + dedicated governor intake, at most once per neighbour per hold and straight + to the recipient when it appears; expires records in wall time against the + hold in force; and keeps them sealed under the new `custody_entries` storage + category, restored at launch. A receipt settles nothing. `custody_stats()` + (`get_custody_stats()` over the FFI) reports the counters, every refusal + reason included, and `erase_custody()` drops every record; the data-layer + wipe calls it. The receipt body has frozen vectors at + `crates/offline-protocol/tests/data/custody-receipt-v1.vectors.json`. + +- **The iOS config readers read `0` and `1` as numbers.** The Foundation-only + readers behind `meshRelay` and `custody` excluded JSON booleans with an + `is Bool` test that Swift also answers true for the numbers 0 and 1, so a + `fanout: 1`, an `activityIdleWindows: 1` or a `jitterMinMs: 0` written from + React Native reached the core as unset and the dial silently stayed at its + default. Both readers now exclude booleans by their CoreFoundation type. + - **The custody chapter.** `docs/spec/custody.md` specifies how a device holds a neighbour's replication frame for hours instead of the five seconds a forwarder gives it today: an explicit deposit in which the diff --git a/docs/configuration.md b/docs/configuration.md index e5c99956e..e556ac681 100644 --- a/docs/configuration.md +++ b/docs/configuration.md @@ -500,6 +500,47 @@ populated, so no caller needs a fallback literal. Counters are `getMeshRelayStats()`; see [mesh.md](mesh.md#reading-the-numbers) for how to read them. +### Custody Configuration + +Holding a neighbour's replication frames for hours instead of the seconds a +forwarder gives them ([spec](spec/custody.md)). Off by default. A device that +enables it takes on other people's ciphertext under the quotas below, holds +each frame until its recipient appears or the hold ends (re-originating it +toward other neighbours meanwhile), and settles nothing: only the recipient's acknowledgement settles a message, and the +receipt a custodian sends the depositor is advisory. + +Applied at construction. There is no runtime update. + +| Parameter | Type | Default | Description | +|-----------|------|---------|-------------| +| `custody.enabled` | boolean | `false` | Whether this device accepts deposits. The off switch | +| `custody.holdMs` | number | 21600000 (6 hours) | How long an accepted frame is held, judged in wall time against the value in force at each sweep, so lowering it expires records already held | +| `custody.maxEntriesPerDepositor` | number | 64 | Held frames one depositor with an established session may have at once | +| `custody.maxBytesPerDepositor` | number | 2097152 (2 MiB) | Bytes one such depositor may have at once | +| `custody.maxEntries` | number | 512 | Held frames across every depositor | +| `custody.maxBytes` | number | 16777216 (16 MiB) | Bytes across every depositor | +| `custody.strangerMaxEntries` | number | 0 | Held frames one proven peer *without* a session may have at once. Zero refuses such peers | +| `custody.strangerMaxBytes` | number | 0 | Bytes one such peer may have at once | +| `custody.overflowPolicy` | `'drop_oldest'` or `'drop_newest'` | `drop_oldest` | Evict the oldest held frame to admit a new one, or refuse the new one | + +Every field is optional, and an omitted one keeps the default above rather than +being restated by a binding, for the reason the mesh forwarding section gives. +Here the default that matters is `enabled: false`: a binding that wrote it as a +literal would keep every app that omits the section off after the release that +ever flips it. + +```typescript +const config: ProtocolConfig = { + appId: 'my-app', + profile: 'default', + custody: { enabled: true }, +}; +``` + +The counters are `getCustodyStats()`, and `eraseCustody()` drops every held +frame; see [mesh.md](mesh.md#holding-a-frame-for-hours-custody) for what the +numbers mean and what a custodian owes. + ### Path Configuration | Parameter | Type | Default | Description | @@ -814,6 +855,30 @@ work. Nothing is partially applied. 24. `meshRelay.activityWindowMs`, `activityMinForwards` and `activityIdleWindows` must each be > 0 +**Custody** + +25. `custody.holdMs` must be > 0, whether or not custody is enabled. Zero is + not a shorter hold but a store that expires everything at the first sweep +26. While `custody.enabled`: `custody.holdMs` must be strictly shorter than + `reliability.retry.outboxMaxLifetimeMs`. A custodian holds ciphertext it + cannot re-seal, and a hold that outlives the depositor's outbox delivers + frames whose sender has already reported them failed; nothing at runtime + would notice, because the custodian cannot see the depositor's ladder +27. While enabled: `custody.maxEntriesPerDepositor` and + `custody.maxBytesPerDepositor` must each be > 0, the byte cap must be at + least 65536 (one replication frame at its ceiling, or every deposit is + refused as `depositor_full`), and `custody.maxEntries` and + `custody.maxBytes` must each be at least their per-depositor sibling +28. While enabled: `custody.strangerMaxEntries` and `custody.strangerMaxBytes` + must be both zero (no deposits from peers without a session, the default) + or both > 0, a positive byte cap must be at least 65536, and neither may + exceed the global dial + +Rules 26 through 28 are checked only while custody is enabled. A disabled +section that could refuse a configuration would turn every outbox lifetime +shorter than the six-hour default hold into a startup error for a feature the +app never switched on. + Rules 17 through 20 all guard one failure: a dial that reads like a conservative setting but is in fact an off switch, leaving the device running, reporting no error, and carrying nothing. Refusing them at construction is what diff --git a/docs/mesh.md b/docs/mesh.md index 2b6d5009a..a2071886f 100644 --- a/docs/mesh.md +++ b/docs/mesh.md @@ -574,6 +574,93 @@ Because parking removes the pending acknowledgement, a parked message that is th What is still not covered: a device whose only infrastructure is **Nostr** never receives an unreachable verdict at all (a broadcast relay reports no per-recipient delivery), so nothing contradicts the initial "reachable" answer and no mesh fallback fires for it. That gap is permanent for Nostr rather than unfinished: there is no verdict to be had. Reticulum is no longer in that position. Its managers speak [the gateway contract](spec/gateway-contract.md), so a gateway's `recipient_unreachable` verdict reaches the same parking machinery the relay's does (it was always keyed to the verdict rather than to the relay), and a device attached to a gateway gets mesh fallback for a recipient that gateway cannot reach. What remains carrier-specific is only that a zone with no gateway has no verdicts to receive, which is the same as having no infrastructure at all. Note also that carrier status is reported by the platform bridge and means "this carrier is up", not "the relay connection is authenticated"; a bridge that reports a connection it never authenticates produces no verdicts either, and its messages settle by acknowledgement timeout as they always did. +### Holding a frame for hours: custody + +Everything above gives a frame this device did not originate five seconds: +queued, tried on a link, abandoned when no neighbour could take it. Custody +([spec](spec/custody.md)) lets a device close that gap for one class of +traffic. It is off by default (`custody.enabled`); a device that never enables +it, and a device that never meets a custodian, both behave exactly as +described above. + +What happens, in the order it happens: + +1. **The depositor asks, on its own frame.** When a device offers its own + sealed replication frame (a document delta, a snapshot, a version offer or + a blob-gone report) to neighbours because the recipient is out of reach, it + writes a one-hop request on the copy it hands over. Only those frames: a + direct message, a media chunk, and a request for a snapshot or a blob are + never deposited, because a custodian cannot see inside a sealed frame and + the depositor asserts the class from the plaintext it retains for + re-sealing. After a restart that plaintext is gone, so the frame is offered + without the request. +2. **Every forwarder strips the request.** The key is unsigned metadata outside + the sealed body, so it is removed from any third-party frame a device + transmits, custody-enabled or not. That is what keeps a deposit to one hop: + a custodian accepts a frame only from its own sender, over the link that + proved it. +3. **The custodian holds only what it could not forward.** A frame carrying a + request is an ordinary forward first. Custody begins where forwarding ends: + at the point a forward has waited past the five seconds without reaching a + link and would be abandoned. By then the forwarding path has already + released the frame's identifier from the handled-once cache, and accepting + the frame never puts it back. That is the one thing a custodian must never + do, and the reason the drop point is where acceptance lives: a device that + both held a frame and suppressed its own forwarding of the sender's + retransmissions of it would be a black hole on exactly the route now known + to be slow. Acceptance is judged by the quotas (per depositor, global, and + a stranger tier that is closed by default), by whether the frame is sealed + and sent by the peer it arrived from, and by the battery floor; a refusal + is silent and counted. +4. **A receipt that settles nothing.** The custodian answers the depositor + once, over the arrival link, with a signed `__CUSTODY_RECEIPT__`, and only + when the depositor advertised the custody entry in its key package. The + depositor uses it for exactly one thing: not asking that custodian again + for the same frame while the hold lasts. The outbox entry, the + acknowledgement timer, the retry ladder and any park are untouched. +5. **Redelivery is an ordinary forward.** When a neighbour appears, the + custodian queues each eligible held frame toward it through a dedicated + intake of the forwarding governor: no handled-once check, no hop spent, the + forward budget rather than the device's own reserve, and one target only. + A frame is re-originated at most once per distinct neighbour during its + hold and stays held; a frame whose recipient is the neighbour is handed + straight over and leaves custody. Expiry, at the end of the hold in force, + is a silent drop with a counter and no event: the depositor never lost + anything. +6. **Its own erase.** `eraseCustody()` drops every held frame and resets the + counters. There is no global wipe in this protocol for custody to inherit, + so the data layer's `wipeAll()` calls it, and a launch with custody + disabled erases whatever a previous launch held, under the same per-launch + delete budget every restore walk draws on, finishing at a later launch if + the store was large. + +Held frames are sealed on disk under their own storage category and restored +at launch, oldest first under the quotas in force. + +#### Reading the custody numbers + +`getCustodyStats()` reports what a device has done as a custodian and as a +depositor. `held` and `heldBytes` are gauges; everything else is cumulative +since start-up or the last erase. + +- `accepted`, `delivered`, `reOriginated`, `expired` are the life of a held + frame. `delivered` counts frames handed straight to their recipient, which + is the number that says custody paid for itself. +- Every refusal reason in the acceptance table has a counter. `refusedDisabled` + climbs on a device with custody off for every frame abandoned at the drop + point, which is how "off" is told from "nobody asked". `refusedStranger` is + the one to read on a device that enabled custody and holds nothing: the + stranger tier is closed by default, so only peers with an established + session are admitted. +- `duplicates` and `evicted` say the quotas are doing work. +- `receiptsSent` and `receiptsDropped` are the custodian's side of the + receipt; `receiptsReceived` and `receiptsIgnored` are the depositor's. A + receipt naming nothing in the outbox is ignored and counted, which is the + ordinary shape of one arriving after the recipient's acknowledgement. + +Tunables live in `ProtocolConfig::custody` (`custody` in every binding); see +[configuration.md](configuration.md#custody-configuration). + ### How a forwarding device chooses There is no routing table and no remembered path. A device that decides to diff --git a/docs/message-delivery.md b/docs/message-delivery.md index 1c4cd21b2..72c8635ba 100644 --- a/docs/message-delivery.md +++ b/docs/message-delivery.md @@ -241,6 +241,15 @@ When the outbox is full, the oldest entry is evicted with a terminal `message_fa **Important**: When a message storage backend is configured, regular-message outbox entries are persisted and restored on the next `start()` with a refreshed delivery window. Media chunks are never persisted — an interrupted transfer surfaces as `media_resend_required` instead. See [Client-Side Persistence](#client-side-persistence) for the app-side layer. +**Custody does not change any of this.** A neighbour holding one of this +device's replication frames in custody ([spec](spec/custody.md)) is an +additional holder of the frame, never its owner: the outbox entry, its +lifetime, the acknowledgement tracking and the retry ladder are untouched by a +deposit and untouched by the custodian's receipt. The receipt only stops this +device from asking the same custodian again while the hold lasts. Whichever +path carries the frame, delivery is settled by the recipient's acknowledgement +and nothing else. + ## Unreachable Recipients: Parking When the internet relay reports a recipient unreachable for an in-flight regular message (its `recipient_unreachable` delivery verdict), the message does not burn its ACK retry budget against a peer that is provably offline. Instead it is **parked**: From ea98b0ddaa0bfc5b7377e79f9273bfb37d58d592 Mon Sep 17 00:00:00 2001 From: bahdotsh Date: Thu, 1 Oct 2026 00:33:58 +0530 Subject: [PATCH 5/7] fix(protocol): custody review round one Redelivery queues nothing toward a peer no mesh link reaches and an abandoned forward leaves its neighbour untried; no_request is judged before disabled and battery before the budgets, in the chapter's table on this branch too; the receipt suppression runs on the wall clock from the receipt's own timestamp; erase drops queued redeliveries; the drop point takes the battery snapshot only for a frame that asked; the acceptance table returns what it proved so no expect remains. The custody vectors are now generated by tools/spec-vectors/generate.py, with the largest hold within the double-safe range; the tracked node_modules symlink is removed and both ignore rules match a symlink too; the iOS reader fix moves under Fixed. --- .gitignore | 2 +- CHANGELOG.md | 14 +- bindings/react-native/.gitignore | 2 +- bindings/react-native/node_modules | 1 - .../src/protocol/custodian.rs | 66 +++++--- .../offline-protocol/src/protocol/custody.rs | 134 ++++++++++++---- .../src/protocol/mesh_relay.rs | 25 +++ crates/offline-protocol/src/protocol/mod.rs | 7 +- .../src/protocol/tests/custody.rs | 144 +++++++++++++++++- .../data/custody-receipt-v1.vectors.json | 10 +- docs/mesh.md | 10 +- docs/spec/conformance.md | 2 +- docs/spec/custody.md | 6 +- tools/spec-vectors/generate.py | 60 ++++++++ 14 files changed, 409 insertions(+), 74 deletions(-) delete mode 120000 bindings/react-native/node_modules diff --git a/.gitignore b/.gitignore index 11ce840d7..8265df7ee 100644 --- a/.gitignore +++ b/.gitignore @@ -30,7 +30,7 @@ __pycache__/ *.py[cod] # Node -node_modules/ +node_modules examples/react-native-app/node_modules/ *.log npm-debug.log* diff --git a/CHANGELOG.md b/CHANGELOG.md index a94d652ce..df2f3794f 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -37,13 +37,6 @@ archived by series under [docs/changelog/](docs/changelog/); see the wipe calls it. The receipt body has frozen vectors at `crates/offline-protocol/tests/data/custody-receipt-v1.vectors.json`. -- **The iOS config readers read `0` and `1` as numbers.** The Foundation-only - readers behind `meshRelay` and `custody` excluded JSON booleans with an - `is Bool` test that Swift also answers true for the numbers 0 and 1, so a - `fanout: 1`, an `activityIdleWindows: 1` or a `jitterMinMs: 0` written from - React Native reached the core as unset and the dial silently stayed at its - default. Both readers now exclude booleans by their CoreFoundation type. - - **The custody chapter.** `docs/spec/custody.md` specifies how a device holds a neighbour's replication frame for hours instead of the five seconds a forwarder gives it today: an explicit deposit in which the @@ -222,6 +215,13 @@ archived by series under [docs/changelog/](docs/changelog/); see the ### Fixed +- **The iOS config readers read `0` and `1` as numbers.** The Foundation-only + readers behind `meshRelay` and `custody` excluded JSON booleans with an + `is Bool` test that Swift also answers true for the numbers 0 and 1, so a + `fanout: 1`, an `activityIdleWindows: 1` or a `jitterMinMs: 0` written from + React Native reached the core as unset and the dial silently stayed at its + default. Both readers now exclude booleans by their CoreFoundation type. + - **The storage conformance suite no longer deletes a merging backend's records.** `runStorageConformance` cleaned up its probe records by listing a probe key type and deleting what it listed, before any check had run. diff --git a/bindings/react-native/.gitignore b/bindings/react-native/.gitignore index 184bac521..50468bed7 100644 --- a/bindings/react-native/.gitignore +++ b/bindings/react-native/.gitignore @@ -5,7 +5,7 @@ lib/ # Node modules -node_modules/ +node_modules # Logs *.log diff --git a/bindings/react-native/node_modules b/bindings/react-native/node_modules deleted file mode 120000 index ef573485e..000000000 --- a/bindings/react-native/node_modules +++ /dev/null @@ -1 +0,0 @@ -/Users/goku/projects/offline/offline-protocol-sdk/bindings/react-native/node_modules \ No newline at end of file diff --git a/crates/offline-protocol/src/protocol/custodian.rs b/crates/offline-protocol/src/protocol/custodian.rs index 75936399b..78ede3c48 100644 --- a/crates/offline-protocol/src/protocol/custodian.rs +++ b/crates/offline-protocol/src/protocol/custodian.rs @@ -52,6 +52,8 @@ impl OfflineProtocol { /// erase that left records behind is the worst shape this call can take. pub fn erase_custody(&mut self) -> Result<()> { let ids = self.custody.erase(); + // A redelivery queued seconds ago must not go out after the erase. + self.mesh_relay.drop_held(); self.custody_receipts.clear(); let Some(storage) = self.protocol_state_storage.clone() else { return Ok(()); @@ -117,8 +119,11 @@ impl OfflineProtocol { .as_deref() .is_some_and(|peer| self.confirmed_sessions.contains(peer)); // The battery snapshot locks and allocates across every transport; - // skipped while custody is off, where the table refuses first anyway. - let battery_ok = !self.custody.is_enabled() || self.battery_allows_relaying(); + // taken only for a frame the table can reach that row for, which is + // one that asked for custody on a device that has it on. + let battery_ok = !self.custody.is_enabled() + || custody_request.is_none() + || self.battery_allows_relaying(); let now_ms = Utc::now().timestamp_millis(); let message_id = message.id.clone(); @@ -184,13 +189,18 @@ impl OfflineProtocol { } /// Queues every eligible held frame toward a neighbour that just - /// appeared, through the governor's held-frame intake. + /// appeared on a mesh link, through the governor's held-frame intake. /// - /// Obeys `allow_relay` and the battery floors like any forward: a - /// custodian is a forwarder for the frames it holds. A refusal for rate - /// or room leaves the frame held and untried, so the next discovery of - /// the same neighbour tries again; the store itself bounds how many - /// distinct neighbours a frame is offered to during its hold. + /// The discovery hook also fires for a peer a carrier with no link + /// reported present (a gateway presence edge, a relay's presence + /// answer); nothing is queued for those, because a marked forward + /// toward a peer no mesh link reaches would only sit out the overdue + /// window and come back. Obeys `allow_relay` and the battery floors like + /// any forward: a custodian is a forwarder for the frames it holds. A + /// refusal for rate or room, and a forward that comes back untransmitted, + /// both leave the frame held and the neighbour untried, so the next + /// discovery of the same neighbour tries again; the store itself bounds + /// how many distinct neighbours a frame is offered to during its hold. pub(super) fn redeliver_custody_to(&mut self, peer: &str) { if !self.custody.is_enabled() || self.custody.is_empty() { return; @@ -200,6 +210,14 @@ impl OfflineProtocol { { return; } + if !self + .transport_manager + .mesh_neighbors() + .iter() + .any(|neighbor| neighbor.peer_id == peer) + { + return; + } if !self.battery_allows_relaying() { return; } @@ -237,9 +255,9 @@ impl OfflineProtocol { } /// [`Self::sweep_custody`] without the throttle, at the given clocks. - pub(super) fn sweep_custody_now(&mut self, now: Instant, now_ms: i64) { + pub(super) fn sweep_custody_now(&mut self, _now: Instant, now_ms: i64) { self.custody_receipts.retain(|_, custodians| { - custodians.retain(|_, until| *until > now); + custodians.retain(|_, until_ms| *until_ms > now_ms); !custodians.is_empty() }); @@ -340,7 +358,17 @@ impl OfflineProtocol { /// the ordinary shape of one arriving after the recipient's /// acknowledgement settled the message, and the shape a forged receipt /// for a never-sent identifier would take. - pub(super) fn handle_custody_receipt(&mut self, custodian: &str, body: &str) { + /// + /// `minted_at_ms` is the receipt frame's own timestamp: the hold is + /// relative to it, as the chapter says, so the suppression ends when the + /// custodian's does, however long the receipt took to arrive and whatever + /// this device's monotonic clock did in the meantime. + pub(super) fn handle_custody_receipt( + &mut self, + custodian: &str, + body: &str, + minted_at_ms: i64, + ) { let receipt = match decode_receipt(body) { Ok(receipt) => receipt, Err(err) => { @@ -360,14 +388,14 @@ impl OfflineProtocol { } // A hold longer than this device's own outbox window buys nothing: - // the entry is gone before the suppression would end. Clamping here - // also keeps the instant arithmetic in range. + // the entry is gone before the suppression would end. Wall clock, + // from the receipt's own timestamp: a monotonic clock pauses in + // suspend and knows nothing of transit, and either would keep asking + // nothing of a custodian that had already let the frame go. let hold_ms = receipt .hold_ms .min(self.config.reliability.retry.outbox_max_lifetime_ms); - let until = Instant::now() - .checked_add(Duration::from_millis(hold_ms)) - .unwrap_or_else(Instant::now); + let until_ms = minted_at_ms.saturating_add(i64::try_from(hold_ms).unwrap_or(i64::MAX)); let custodians = self.custody_receipts.entry(id.clone()).or_default(); if !custodians.contains_key(custodian) @@ -377,13 +405,13 @@ impl OfflineProtocol { // suppression that ends soonest is the one worth least. if let Some(oldest) = custodians .iter() - .min_by_key(|(_, until)| **until) + .min_by_key(|(_, until_ms)| **until_ms) .map(|(peer, _)| peer.clone()) { custodians.remove(&oldest); } } - custodians.insert(custodian.to_string(), until); + custodians.insert(custodian.to_string(), until_ms); debug!(custodian = %custodian, message_id = %id, hold_ms, "Recorded a custody receipt"); self.custody.record_receipt_received(); } @@ -429,7 +457,7 @@ impl OfflineProtocol { self.custody_receipts .get(id) .and_then(|custodians| custodians.get(custodian)) - .is_some_and(|until| *until > Instant::now()) + .is_some_and(|until_ms| *until_ms > Utc::now().timestamp_millis()) } /// Writes one held frame under its own message id. Best-effort, like the diff --git a/crates/offline-protocol/src/protocol/custody.rs b/crates/offline-protocol/src/protocol/custody.rs index 9b1140ca8..32ef66b3b 100644 --- a/crates/offline-protocol/src/protocol/custody.rs +++ b/crates/offline-protocol/src/protocol/custody.rs @@ -49,10 +49,12 @@ pub const MAX_CUSTODY_RECEIPTS_PER_MESSAGE: usize = 64; /// chapter's acceptance table applies them. #[derive(Debug, Clone, Copy, PartialEq, Eq, Hash)] pub enum CustodyRefusal { + /// The frame carries no deposit request. Judged first, on every device, + /// so `refused_disabled` counts deposits an off device turned away rather + /// than every frame it ever abandoned. + NoRequest, /// Custody is off on this device. Disabled, - /// The frame carries no deposit request. - NoRequest, /// The request names a class this version does not define. UnknownClass, /// The outer prefix is not `__MLS_ENC__`. @@ -67,28 +69,30 @@ pub enum CustodyRefusal { Duplicate, /// The depositor is a stranger and the stranger tier is closed. StrangerRefused, + /// The battery is below the soft relay floor. Judged before the budgets, + /// so a refusal for battery never evicts a held frame to make room for a + /// deposit that is then refused. + Battery, /// The depositor's own budget is full and the policy does not make room. DepositorFull, /// The global budget is full and the policy does not make room. StoreFull, - /// The battery is below the soft relay floor. - Battery, } impl CustodyRefusal { /// Every refusal reason, in table order, for the counters and their tests. pub const ALL: &'static [Self] = &[ - Self::Disabled, Self::NoRequest, + Self::Disabled, Self::UnknownClass, Self::NotSealed, Self::UnprovenPeer, Self::NotDepositor, Self::Duplicate, Self::StrangerRefused, + Self::Battery, Self::DepositorFull, Self::StoreFull, - Self::Battery, ]; /// The reason as the chapter names it. @@ -296,6 +300,16 @@ enum Tier { Stranger, } +/// What the acceptance table established about a candidate it did not +/// refuse, so the budgets can be applied without re-proving any of it. +struct Admitted<'a> { + tier: Tier, + /// The arrival peer, which the table proved is also the frame's sender. + depositor: &'a str, + /// The class token, which the table proved is one this version defines. + token: &'a str, +} + #[derive(Debug, Clone, Copy, Default)] struct Usage { entries: usize, @@ -375,21 +389,24 @@ impl CustodyStore { &mut self, candidate: CustodyCandidate<'_>, ) -> Result { - let tier = match self.check(&candidate) { - Ok(tier) => tier, + let Admitted { + tier, + depositor, + token, + } = match self.check(&candidate) { + Ok(admitted) => admitted, Err(refusal) => { - // Every row is counted, `disabled` included, so an operator - // who expected deposits and sees none can tell "off" from - // "nobody asked"; a duplicate lands in its own counter. + // Every row is counted. `no_request` is judged before + // `disabled`, so on an off device the latter counts deposits + // turned away rather than every abandoned forward, and an + // operator can tell "off" from "nobody asked"; a duplicate + // lands in its own counter. self.stats.count(refusal); return Err(refusal); } }; - - let depositor = candidate - .arrival_peer - .expect("check proved the arrival peer") - .to_string(); + let depositor = depositor.to_string(); + let class = token.to_string(); let bytes = frame_footprint(&candidate.message); let (max_entries, max_bytes) = match tier { Tier::Session => ( @@ -456,10 +473,6 @@ impl CustodyStore { } let id = candidate.message.id.clone(); - let class = candidate - .request - .expect("check proved the request") - .to_string(); self.insert(HeldFrame { message: candidate.message, depositor: depositor.clone(), @@ -479,14 +492,16 @@ impl CustodyStore { }) } - /// The acceptance table, in the chapter's order. - fn check(&self, c: &CustodyCandidate<'_>) -> Result { - if !self.config.enabled { - return Err(CustodyRefusal::Disabled); - } + /// The acceptance table, in the chapter's order: the request before the + /// switch, the depositor's identity before the tiers, and the battery + /// before the budgets, which [`Self::judge`] applies after this returns. + fn check<'a>(&self, c: &CustodyCandidate<'a>) -> Result, CustodyRefusal> { let Some(token) = c.request else { return Err(CustodyRefusal::NoRequest); }; + if !self.config.enabled { + return Err(CustodyRefusal::Disabled); + } if token != CUSTODY_CLASS_DATA { return Err(CustodyRefusal::UnknownClass); } @@ -518,7 +533,11 @@ impl CustodyStore { if !c.battery_ok { return Err(CustodyRefusal::Battery); } - Ok(tier) + Ok(Admitted { + tier, + depositor: peer, + token, + }) } fn fits_depositor( @@ -682,10 +701,14 @@ impl CustodyStore { } /// A marked forward reached no link and came back from the drop point. - /// Not a deposit, not a refusal, not counted. + /// Not a deposit, not a refusal, not counted, and the neighbour it was + /// queued toward is untried again: an attempt that never transmitted has + /// not spent the one re-origination the hold allows toward that neighbour. pub(crate) fn returned(&mut self, id: &MessageId) { if let Some(held) = self.entries.get_mut(id) { - held.in_flight = None; + if let Some(peer) = held.in_flight.take() { + held.tried.remove(&peer); + } } } @@ -1086,6 +1109,61 @@ mod tests { assert_eq!(store.stats().held, 1); } + #[test] + fn a_frame_asking_for_nothing_is_no_request_before_disabled() { + let mut off = CustodyStore::new(CustodyConfig::default()); + let mut silent = candidate(sealed("alice", "carol", "m1"), "alice"); + silent.request = None; + assert_eq!(off.judge(silent).err(), Some(CustodyRefusal::NoRequest)); + assert_eq!(off.stats().refused_no_request, 1); + assert_eq!(off.stats().refused_disabled, 0, "nobody asked"); + + let verdict = off.judge(candidate(sealed("alice", "carol", "m1"), "alice")); + assert_eq!(verdict.err(), Some(CustodyRefusal::Disabled)); + assert_eq!(off.stats().refused_disabled, 1, "a deposit turned away"); + } + + #[test] + fn battery_is_judged_before_the_quotas_so_a_refusal_evicts_nothing() { + let mut store = CustodyStore::new(CustodyConfig { + max_entries_per_depositor: 1, + ..config() + }); + store + .judge(candidate(sealed("alice", "carol", "m1"), "alice")) + .unwrap(); + let mut flat = candidate(sealed("alice", "carol", "m2"), "alice"); + flat.battery_ok = false; + assert_eq!(store.judge(flat).err(), Some(CustodyRefusal::Battery)); + assert_eq!( + store.stats().evicted, + 0, + "nothing made room for a refused deposit" + ); + assert!(store.contains(&mid("m1"))); + assert_eq!(store.stats().refused_battery, 1); + } + + #[test] + fn a_returned_forward_leaves_its_neighbour_untried() { + let mut store = CustodyStore::new(config()); + store + .judge(candidate(sealed("alice", "carol", "m1"), "alice")) + .unwrap(); + let id = mid("m1"); + store.mark_queued(&id, "dave"); + assert!(store.candidates_for("dave").is_empty(), "in flight"); + store.returned(&id); + assert_eq!( + store.candidates_for("dave"), + vec![id.clone()], + "an attempt that never transmitted is not spent" + ); + store.mark_queued(&id, "dave"); + store.record_re_originated(&id); + assert!(store.candidates_for("dave").is_empty(), "a transmission is"); + } + #[test] fn the_receipt_round_trips_and_refuses_what_the_chapter_refuses() { let wire = encode_receipt("m1", 21_600_000); diff --git a/crates/offline-protocol/src/protocol/mesh_relay.rs b/crates/offline-protocol/src/protocol/mesh_relay.rs index 1d8f93a67..e9a28861c 100644 --- a/crates/offline-protocol/src/protocol/mesh_relay.rs +++ b/crates/offline-protocol/src/protocol/mesh_relay.rs @@ -1082,6 +1082,13 @@ impl MeshRelayGovernor { None } + /// Drops every marked held forward from the queue, leaving the ordinary + /// ones. For the custody erase: a frame queued for redelivery seconds + /// before the erase must not go out after it. + pub fn drop_held(&mut self) { + self.pending.retain(|relay| relay.custody_target.is_none()); + } + /// Whether `message_id` is recorded as handled in the suppression cache. /// /// For the tests that pin custody and the cache disjoint: a custodian @@ -2784,6 +2791,24 @@ mod tests { ); } + #[test] + fn dropping_held_forwards_leaves_the_ordinary_ones_queued() { + let mut gov = governor(); + gov.admit(&frame(), Some("alice"), 3, false); + gov.admit_held(frame(), "dave", Instant::now()) + .expect("queued"); + gov.admit_held(frame(), "erin", Instant::now()) + .expect("queued"); + assert_eq!(gov.pending_len(), 3); + + gov.drop_held(); + + assert_eq!(gov.pending_len(), 1); + let (due, _) = gov.release_due(Instant::now()); + assert_eq!(due.len(), 1); + assert!(due[0].custody_target.is_none()); + } + #[test] fn the_held_intake_still_takes_the_queue_and_the_neighbor_rate() { let mut gov = MeshRelayGovernor::with_config( diff --git a/crates/offline-protocol/src/protocol/mod.rs b/crates/offline-protocol/src/protocol/mod.rs index eac811d1e..a9397de89 100644 --- a/crates/offline-protocol/src/protocol/mod.rs +++ b/crates/offline-protocol/src/protocol/mod.rs @@ -145,10 +145,11 @@ pub struct OfflineProtocol { /// Receipts this device holds as a *depositor*: for each outbox entry, /// the custodians holding it and until when a further deposit request - /// toward each is suppressed. In memory only, bounded by the outbox and + /// toward each is suppressed, in Unix milliseconds from the receipt's own + /// timestamp. In memory only, bounded by the outbox and /// by [`custody::MAX_CUSTODY_RECEIPTS_PER_MESSAGE`]; losing it at a /// restart costs one duplicate deposit, which the custodian absorbs. - pub(crate) custody_receipts: HashMap>, + pub(crate) custody_receipts: HashMap>, /// When the custody store was last swept for expired records. custody_last_sweep: Instant, @@ -3343,7 +3344,7 @@ impl OfflineProtocol { // Consumed whatever the body says: a malformed one is refused // silently, and a receipt never requests an acknowledgement. if let Some(data) = content.strip_prefix(internal_prefixes::CUSTODY_RECEIPT) { - self.handle_custody_receipt(sender, data); + self.handle_custody_receipt(sender, data, message.timestamp.as_millis()); return Some(InternalMessageResult::Consumed); } diff --git a/crates/offline-protocol/src/protocol/tests/custody.rs b/crates/offline-protocol/src/protocol/tests/custody.rs index 6eeeede5b..d2640639b 100644 --- a/crates/offline-protocol/src/protocol/tests/custody.rs +++ b/crates/offline-protocol/src/protocol/tests/custody.rs @@ -26,7 +26,7 @@ use crate::protocol::tests::{create_test_config_for_user, id}; use crate::protocol::types::{storage_keys, SessionState}; use crate::protocol::{OfflineProtocol, TestProtocolStateStorage}; use crate::ProtocolConfig; -use offline_protocol_core::{Message, MessageId, MessagePriority}; +use offline_protocol_core::{Message, MessageId, MessagePriority, Timestamp}; use offline_protocol_mls::MlsStorage; /// One device, with the radio its frames actually go through. @@ -799,6 +799,148 @@ fn a_receipt_naming_nothing_or_arriving_unsigned_changes_nothing() { assert!(bob.take_peer_sends().is_empty()); } +#[test] +fn a_neighbour_without_a_mesh_link_queues_nothing_and_stays_untried() { + let (mut alice, mut bob, mut carol) = topology(open_custody()); + let frame = deposit_from(&mut alice, &bob, &carol); + bob.receive_from(frame.clone(), &alice.address); + drop_point(&mut bob); + bob.take_peer_sends(); + + // A presence edge from a carrier holding no link to carol runs the + // discovery hook; nothing is queued, and nothing is spent. + bob.protocol.on_neighbor_discovered(&carol.address); + bob.flush_after(Duration::ZERO); + assert_eq!(bob.transport.peer_send_count_for(&frame.id.as_str()), 0); + assert_eq!(bob.protocol.mesh_relay_stats().awaiting_transmission, 0); + assert_eq!(bob.protocol.custody_stats().held, 1); + + // The mesh link appears: delivered on the first flush. + bob.link(&carol); + bob.flush_after(Duration::ZERO); + assert_eq!(bob.protocol.custody_stats().delivered, 1); + let sends = bob.take_peer_sends(); + let (_, delivered) = sends.into_iter().find(|(_, m)| m.id == frame.id).unwrap(); + carol.receive_from(delivered, &bob.address); + assert_eq!( + read(&mut carol, &Node::space_for(&alice), "notes", "k"), + Some(DataValue::text("v")) + ); +} + +#[test] +fn an_abandoned_forward_leaves_the_neighbour_untried() { + let (mut alice, mut bob, carol) = topology(open_custody()); + let frame = deposit_from(&mut alice, &bob, &carol); + bob.receive_from(frame.clone(), &alice.address); + drop_point(&mut bob); + bob.take_peer_sends(); + + // Dave is not the recipient. Queued toward him, gone before the flush, + // abandoned at the overdue cut-off: returned, and dave untried. + let dave = Node::new("dave"); + bob.link(&dave); + bob.transport.remove_connected_peer(&dave.address); + bob.flush_after(Duration::ZERO); + bob.flush_after(RELAY_QUEUE_MAX_OVERDUE + Duration::from_millis(1)); + assert_eq!(bob.protocol.custody_stats().re_originated, 0); + assert_eq!(bob.transport.peer_send_count_for(&frame.id.as_str()), 0); + + // Seen again with a link: the one attempt the hold allows toward dave + // is still available. + bob.transport.add_connected_peer(dave.address.clone(), -55); + bob.protocol.on_neighbor_discovered(&dave.address); + bob.flush_after(Duration::ZERO); + assert_eq!(bob.transport.peer_send_count_for(&frame.id.as_str()), 1); + assert_eq!(bob.protocol.custody_stats().re_originated, 1); +} + +#[test] +fn a_frame_asking_for_nothing_on_an_off_device_is_no_request() { + let (mut alice, mut bob, carol) = topology(CustodyConfig::default()); + let mut frame = deposit_from(&mut alice, &bob, &carol); + frame.metadata.remove(CUSTODY_META_KEY); + bob.receive_from(frame, &alice.address); + drop_point(&mut bob); + let stats = bob.protocol.custody_stats(); + assert_eq!(stats.refused_no_request, 1, "nobody asked"); + assert_eq!( + stats.refused_disabled, 0, + "so nothing was turned away for being off" + ); +} + +#[test] +fn a_receipt_is_judged_from_its_own_timestamp_not_from_arrival() { + let (mut alice, mut bob, carol) = topology(open_custody()); + let frame = deposit_from(&mut alice, &bob, &carol); + let hold_ms = bob.protocol.custody_config().hold_ms; + let now_ms = Utc::now().timestamp_millis(); + let depositor = alice.address.clone(); + let held_id = frame.id.as_str(); + + let receipt_at = |bob: &mut Node, minted_at_ms: i64| -> Message { + let mut receipt = bob + .protocol + .create_message( + &depositor, + encode_receipt(&held_id, hold_ms), + Some(MessagePriority::Low), + None, + ) + .unwrap(); + receipt.requires_ack = false; + receipt.timestamp = Timestamp::from_millis(minted_at_ms); + bob.protocol.sign_control_message(&mut receipt).unwrap(); + receipt + }; + + // Minted longer ago than its hold: received, and nothing left to + // suppress, so the next offer toward bob carries the request. + let stale = receipt_at(&mut bob, now_ms - hold_ms as i64 - 60_000); + alice.receive_from(stale, &bob.address); + assert_eq!(alice.protocol.custody_stats().receipts_received, 1); + assert!(!alice + .protocol + .custody_suppressed_toward(&frame.id, &bob.address)); + + // A fresh one suppresses until its hold elapses on the wall clock. + let fresh = receipt_at(&mut bob, now_ms); + alice.receive_from(fresh, &bob.address); + assert!(alice + .protocol + .custody_suppressed_toward(&frame.id, &bob.address)); + alice + .protocol + .sweep_custody_now(Instant::now(), now_ms + hold_ms as i64 + 1); + assert!( + !alice + .protocol + .custody_suppressed_toward(&frame.id, &bob.address), + "the sweep drops a suppression whose hold has ended" + ); +} + +#[test] +fn erase_drops_redeliveries_already_queued() { + let (mut alice, mut bob, carol) = topology(open_custody()); + let frame = deposit_from(&mut alice, &bob, &carol); + bob.receive_from(frame.clone(), &alice.address); + drop_point(&mut bob); + bob.take_peer_sends(); + + // Carol appears: a marked forward is queued and not yet flushed. + bob.link(&carol); + bob.protocol.erase_custody().unwrap(); + bob.flush_after(Duration::ZERO); + assert_eq!( + bob.transport.peer_send_count_for(&frame.id.as_str()), + 0, + "a redelivery queued before the erase must not go out after it" + ); + assert_eq!(bob.protocol.custody_stats().delivered, 0); +} + #[test] fn only_a_class_a_frame_with_retained_plaintext_carries_a_request() { let mut alice = Node::new("alice"); diff --git a/crates/offline-protocol/tests/data/custody-receipt-v1.vectors.json b/crates/offline-protocol/tests/data/custody-receipt-v1.vectors.json index ad24d0d17..cb1af16dc 100644 --- a/crates/offline-protocol/tests/data/custody-receipt-v1.vectors.json +++ b/crates/offline-protocol/tests/data/custody-receipt-v1.vectors.json @@ -5,8 +5,8 @@ "notes": [ "A receipt is the control frame a custodian answers a deposit with: prefix, then a JSON object.", "Fields serialize in the order v, id, hold_ms. A decoder reads v before the body and ignores unknown fields.", - "hold_ms is relative to the receipt's own timestamp. Every case below carries the prefix.", - "Computed by hand from the chapter, independently of the code that pins them." + "hold_ms is relative to the receipt's own timestamp and a u64 on the wire; the vectors stay within the double-safe integer range so every JSON decoder can run them.", + "Every case carries the prefix. Computed by tools/spec-vectors/generate.py from the chapter, independently of the code that pins them." ], "frames": [ { @@ -22,10 +22,10 @@ "wire": "__CUSTODY_RECEIPT__{\"v\":1,\"id\":\"a\",\"hold_ms\":0}" }, { - "name": "largest_hold", + "name": "largest_double_safe_hold", "id": "max", - "hold_ms": 18446744073709551615, - "wire": "__CUSTODY_RECEIPT__{\"v\":1,\"id\":\"max\",\"hold_ms\":18446744073709551615}" + "hold_ms": 9007199254740991, + "wire": "__CUSTODY_RECEIPT__{\"v\":1,\"id\":\"max\",\"hold_ms\":9007199254740991}" } ], "decode_only": [ diff --git a/docs/mesh.md b/docs/mesh.md index a2071886f..3bc1b7c94 100644 --- a/docs/mesh.md +++ b/docs/mesh.md @@ -613,7 +613,8 @@ What happens, in the order it happens: and sent by the peer it arrived from, and by the battery floor; a refusal is silent and counted. 4. **A receipt that settles nothing.** The custodian answers the depositor - once, over the arrival link, with a signed `__CUSTODY_RECEIPT__`, and only + once, over a mesh link to the peer that handed the frame over, with a + signed `__CUSTODY_RECEIPT__`, and only when the depositor advertised the custody entry in its key package. The depositor uses it for exactly one thing: not asking that custodian again for the same frame while the hold lasts. The outbox entry, the @@ -646,9 +647,10 @@ since start-up or the last erase. - `accepted`, `delivered`, `reOriginated`, `expired` are the life of a held frame. `delivered` counts frames handed straight to their recipient, which is the number that says custody paid for itself. -- Every refusal reason in the acceptance table has a counter. `refusedDisabled` - climbs on a device with custody off for every frame abandoned at the drop - point, which is how "off" is told from "nobody asked". `refusedStranger` is +- Every refusal reason in the acceptance table has a counter. A frame that + asked for nothing lands in `refusedNoRequest` on any device, so + `refusedDisabled` counts the deposits a device with custody off turned away, + which is how "off" is told from "nobody asked". `refusedStranger` is the one to read on a device that enabled custody and holds nothing: the stranger tier is closed by default, so only peers with an established session are admitted. diff --git a/docs/spec/conformance.md b/docs/spec/conformance.md index 1ec9194ee..9c84e657b 100644 --- a/docs/spec/conformance.md +++ b/docs/spec/conformance.md @@ -126,7 +126,7 @@ are computed independently of that code. `tools/spec-vectors/generate.py` is a second implementation of these encodings, written from the chapters and forbidden from importing, linking against or shelling out to the Rust crates it pins. Running it with `--check` regenerates -the nine files it owns (every row above except document replication and the +the ten files it owns (every row above except document replication and the Bluetooth LE fragment framing) and fails on any difference, which is what CI does. diff --git a/docs/spec/custody.md b/docs/spec/custody.md index 69128ef59..5e28ed80c 100644 --- a/docs/spec/custody.md +++ b/docs/spec/custody.md @@ -203,8 +203,8 @@ of these holds, and MUST refuse otherwise: | Condition | Refusal reason when it fails | |-----------|------------------------------| -| Custody is enabled | `disabled` | | The frame carries `__custody` | `no_request` | +| Custody is enabled | `disabled` | | The token is `data` | `unknown_class` | | The outer prefix is `__MLS_ENC__` | `not_sealed` | | The frame arrived from a peer whose address the transport proved | `unproven_peer` | @@ -212,9 +212,9 @@ of these holds, and MUST refuse otherwise: | The frame is not addressed to this device | never fails: a frame for this device is delivered, not held | | No frame with this identifier is already held | `duplicate` | | The depositor's tier admits it: a peer with an established session under the session tier, any other proven peer under the stranger tier | `stranger_refused` | +| The battery is above the soft relay floor, judged before the budgets so a refusal evicts nothing | `battery` | | The depositor's entry and byte budgets have room, or the overflow policy makes room | `depositor_full` | | The global entry and byte budgets have room, or the overflow policy makes room | `store_full` | -| The battery is above the soft relay floor | `battery` | The `unproven_peer` and `not_depositor` rows together make the depositor one address: the peer that handed the frame over, the frame's `sender`, the key the @@ -274,7 +274,7 @@ __CUSTODY_RECEIPT__{"v":1,"id":"","hold_ms":} |-------|---------| | `v` | Body version, `1` | | `id` | The identifier of the frame now held | -| `hold_ms` | How much longer the custodian will hold it, relative to the receipt's own timestamp. Relative rather than absolute so the depositor applies it to its own clock and skew cannot expire a valid receipt | +| `hold_ms` | How much longer the custodian will hold it, relative to the receipt's own timestamp. Relative rather than absolute so the depositor applies it to its own clock and skew cannot expire a valid receipt. A `u64` on the wire; the vectors stay within the double-safe integer range (2^53 - 1) so every JSON decoder can run them | The custodian's address is the frame's `sender`; the depositor's is its `recipient`. Unknown fields MUST be ignored. A receiver MUST refuse a body it diff --git a/tools/spec-vectors/generate.py b/tools/spec-vectors/generate.py index 1ee0ac374..409473789 100644 --- a/tools/spec-vectors/generate.py +++ b/tools/spec-vectors/generate.py @@ -35,6 +35,7 @@ CORE_DATA = REPO / "crates" / "offline-protocol-core" / "tests" / "data" SEALED_DATA = REPO / "crates" / "offline-protocol-sealed" / "tests" / "data" TRANSPORT_DATA = REPO / "crates" / "offline-protocol-transport" / "tests" / "data" +PROTOCOL_DATA = REPO / "crates" / "offline-protocol" / "tests" / "data" # -------------------------------------------------------------------------- @@ -1685,6 +1686,64 @@ def case(name: str, note: str, address: str, challenge: bytes) -> dict: } +def build_custody_receipt_vectors() -> dict: + """The custody receipt body, from docs/spec/custody.md ("The receipt"). + + The prefix, then a compact JSON object with the fields `v`, `id` and + `hold_ms` in that order. A decoder reads `v` before the body and ignores + unknown fields. `hold_ms` is a u64 on the wire; the vectors stay within + the double-safe integer range so every JSON decoder can run them. + """ + PREFIX = "__CUSTODY_RECEIPT__" + DOUBLE_SAFE_MAX = 2**53 - 1 + + def wire(msg_id: str, hold_ms: int) -> str: + body = {"v": 1, "id": msg_id, "hold_ms": hold_ms} + return PREFIX + json.dumps(body, separators=(",", ":")) + + def frame(name: str, msg_id: str, hold_ms: int) -> dict: + return {"name": name, "id": msg_id, "hold_ms": hold_ms, "wire": wire(msg_id, hold_ms)} + + return { + "chapter": "docs/spec/custody.md", + "prefix": PREFIX, + "version": 1, + "notes": [ + "A receipt is the control frame a custodian answers a deposit with: prefix, then a JSON object.", + "Fields serialize in the order v, id, hold_ms. A decoder reads v before the body and ignores unknown fields.", + "hold_ms is relative to the receipt's own timestamp and a u64 on the wire; the vectors stay within the double-safe integer range so every JSON decoder can run them.", + "Every case carries the prefix. Computed by tools/spec-vectors/generate.py from the chapter, independently of the code that pins them.", + ], + "frames": [ + frame("six_hour_hold", "7f3a1c2e-4b5d-4e6f-8a9b-0c1d2e3f4a5b", 21_600_000), + frame("zero_hold", "a", 0), + frame("largest_double_safe_hold", "max", DOUBLE_SAFE_MAX), + ], + "decode_only": [ + { + "name": "unknown_field_ignored", + "wire": PREFIX + '{"v":1,"id":"m1","hold_ms":5,"note":"a future field"}', + "id": "m1", + "hold_ms": 5, + }, + { + "name": "field_order_is_free", + "wire": PREFIX + '{"hold_ms":5,"id":"m1","v":1}', + "id": "m1", + "hold_ms": 5, + }, + ], + "rejects": [ + {"name": "unknown_version", "wire": PREFIX + '{"v":2,"id":"m1","hold_ms":5}'}, + {"name": "missing_version", "wire": PREFIX + '{"id":"m1","hold_ms":5}'}, + {"name": "empty_id", "wire": PREFIX + '{"v":1,"id":"","hold_ms":5}'}, + {"name": "not_an_object", "wire": PREFIX + '[1,"m1",5]'}, + {"name": "negative_hold", "wire": PREFIX + '{"v":1,"id":"m1","hold_ms":-1}'}, + {"name": "missing_hold", "wire": PREFIX + '{"v":1,"id":"m1"}'}, + ], + } + + FILES = [ (CORE_DATA / "wire-v1.vectors.json", build_wire_vectors), (CORE_DATA / "address-v1.vectors.json", build_address_vectors), @@ -1695,6 +1754,7 @@ def case(name: str, note: str, address: str, challenge: bytes) -> dict: (SEALED_DATA / "key-package-v1.vectors.json", build_key_package_vectors), (SEALED_DATA / "identity-assertion-v1.vectors.json", build_identity_assertion_vectors), (TRANSPORT_DATA / "stream-framing-v1.vectors.json", build_stream_framing_vectors), + (PROTOCOL_DATA / "custody-receipt-v1.vectors.json", build_custody_receipt_vectors), ] From 299357f11735f21ff25bac5b064e3e5289139454 Mon Sep 17 00:00:00 2001 From: bahdotsh Date: Thu, 1 Oct 2026 00:47:17 +0530 Subject: [PATCH 6/7] fix(protocol): a receipt from a fast clock cannot extend its suppression --- crates/offline-protocol/src/protocol/custodian.rs | 8 +++++++- .../offline-protocol/src/protocol/tests/custody.rs | 14 ++++++++++++++ 2 files changed, 21 insertions(+), 1 deletion(-) diff --git a/crates/offline-protocol/src/protocol/custodian.rs b/crates/offline-protocol/src/protocol/custodian.rs index 78ede3c48..4110fbc57 100644 --- a/crates/offline-protocol/src/protocol/custodian.rs +++ b/crates/offline-protocol/src/protocol/custodian.rs @@ -395,7 +395,13 @@ impl OfflineProtocol { let hold_ms = receipt .hold_ms .min(self.config.reliability.retry.outbox_max_lifetime_ms); - let until_ms = minted_at_ms.saturating_add(i64::try_from(hold_ms).unwrap_or(i64::MAX)); + // Never from a timestamp ahead of this device's clock: the control + // gate admits one up to two days ahead, and never checks it on an + // unbound signature, so a fast-clock custodian would otherwise + // suppress re-deposit toward itself past its own hold. + let until_ms = minted_at_ms + .min(Utc::now().timestamp_millis()) + .saturating_add(i64::try_from(hold_ms).unwrap_or(i64::MAX)); let custodians = self.custody_receipts.entry(id.clone()).or_default(); if !custodians.contains_key(custodian) diff --git a/crates/offline-protocol/src/protocol/tests/custody.rs b/crates/offline-protocol/src/protocol/tests/custody.rs index d2640639b..7018b15e8 100644 --- a/crates/offline-protocol/src/protocol/tests/custody.rs +++ b/crates/offline-protocol/src/protocol/tests/custody.rs @@ -904,6 +904,20 @@ fn a_receipt_is_judged_from_its_own_timestamp_not_from_arrival() { .protocol .custody_suppressed_toward(&frame.id, &bob.address)); + // One stamped a day ahead of this clock is judged from local now: the + // suppression ends by now + hold, not a day later. + let ahead = receipt_at(&mut bob, now_ms + 86_400_000); + alice.receive_from(ahead, &bob.address); + assert!(alice + .protocol + .custody_suppressed_toward(&frame.id, &bob.address)); + let until = alice.protocol.custody_receipts[&frame.id][&bob.address]; + assert!( + until <= Utc::now().timestamp_millis() + hold_ms as i64, + "a future timestamp must not extend the suppression" + ); + alice.protocol.custody_receipts.clear(); + // A fresh one suppresses until its hold elapses on the wall clock. let fresh = receipt_at(&mut bob, now_ms); alice.receive_from(fresh, &bob.address); From f39e069eacbcd87bac2fc1a8ebc6a5293b1dd21d Mon Sep 17 00:00:00 2001 From: bahdotsh Date: Thu, 1 Oct 2026 01:34:18 +0530 Subject: [PATCH 7/7] fix(bindings): classify the custody methods in the local API get_custody_stats is a read-only instance view beside get_mesh_relay_stats. erase_custody is a platform operation: it drops what every client's traffic deposited, so it is the operator's call, like data.wipe_all. Regenerates the local API table for the custody additions to the definition. --- .../local_api/dispatch.py | 3 ++ .../offline_protocol_sdk/local_api/table.py | 36 ++++++++++++++++++- docs/spec/local-api.md | 2 ++ 3 files changed, 40 insertions(+), 1 deletion(-) diff --git a/bindings/python/offline_protocol_sdk/local_api/dispatch.py b/bindings/python/offline_protocol_sdk/local_api/dispatch.py index b2f8583b9..19fecd512 100644 --- a/bindings/python/offline_protocol_sdk/local_api/dispatch.py +++ b/bindings/python/offline_protocol_sdk/local_api/dispatch.py @@ -121,6 +121,7 @@ "get_retry_queue_size", "get_mesh_relay_stats", "get_mesh_relay_tunables", + "get_custody_stats", # instance-wide tuning "set_relay_priority", "update_relay_config", @@ -282,6 +283,8 @@ "data.with_storage", # the operator's logout "data.wipe_all", + # erases what every client's traffic deposited + "erase_custody", # takes a callback interface "run_storage_conformance", } diff --git a/bindings/python/offline_protocol_sdk/local_api/table.py b/bindings/python/offline_protocol_sdk/local_api/table.py index 56dd28f7e..8a137537f 100644 --- a/bindings/python/offline_protocol_sdk/local_api/table.py +++ b/bindings/python/offline_protocol_sdk/local_api/table.py @@ -9,7 +9,7 @@ from __future__ import annotations -UDL_SHA256 = "6e46f0053aa0b08d7b1b5e31cb2e3802fc126351fed72a97dee28558f70ee424" +UDL_SHA256 = "d2b3a4e23be560b35388bfe45c5164ce4b7c002f5e11022700618c7e102823cb" TABLE = {'callbacks': ('MlsStorageProvider', 'ProtocolStateStorageProvider', @@ -219,6 +219,7 @@ ('app_state', 'AppState')), 'void'), 'end_telemetry_session': ((), 'void'), + 'erase_custody': ((), 'void'), 'establish_secure_session': ((('peer_id', 'string'),), 'MlsWelcomeMessage?'), 'finalize_file': ((('file_id', 'string'),), 'void'), @@ -239,6 +240,7 @@ 'get_active_transports': ((), 'sequence'), 'get_battery_level': ((), 'u8?'), 'get_blocked_users': ((), 'sequence'), + 'get_custody_stats': ((), 'CustodyStats'), 'get_dedup_stats': ((), 'DedupStats'), 'get_delivery_success_rate': ((), 'f32'), 'get_dors_config': ((), 'DorsConfig'), @@ -522,6 +524,37 @@ 'records': {'AckConfig': (('default_timeout_ms', 'u64', False), ('max_pending_acks', 'u64', False)), 'BleFragment': (('recipient_id', 'string', False), ('data', 'sequence', False)), + 'CustodyConfig': (('enabled', 'boolean?', True), + ('hold_ms', 'u64?', True), + ('max_entries_per_depositor', 'u64?', True), + ('max_bytes_per_depositor', 'u64?', True), + ('max_entries', 'u64?', True), + ('max_bytes', 'u64?', True), + ('stranger_max_entries', 'u64?', True), + ('stranger_max_bytes', 'u64?', True), + ('overflow_policy', 'OverflowPolicy?', True)), + 'CustodyStats': (('held', 'u64', False), + ('held_bytes', 'u64', False), + ('accepted', 'u64', False), + ('delivered', 'u64', False), + ('re_originated', 'u64', False), + ('expired', 'u64', False), + ('duplicates', 'u64', False), + ('evicted', 'u64', False), + ('receipts_sent', 'u64', False), + ('receipts_dropped', 'u64', False), + ('receipts_received', 'u64', False), + ('receipts_ignored', 'u64', False), + ('refused_disabled', 'u64', False), + ('refused_no_request', 'u64', False), + ('refused_unknown_class', 'u64', False), + ('refused_not_sealed', 'u64', False), + ('refused_unproven_peer', 'u64', False), + ('refused_not_depositor', 'u64', False), + ('refused_stranger', 'u64', False), + ('refused_depositor_full', 'u64', False), + ('refused_store_full', 'u64', False), + ('refused_battery', 'u64', False)), 'DedupConfig': (('max_tracked_messages', 'u64', False), ('retention_time_secs', 'u64', False)), 'DedupStats': (('total_tracked', 'u64', False), @@ -728,6 +761,7 @@ ('rich_payload_enabled', 'boolean', True), ('crypto_recovery_enabled', 'boolean', True), ('mesh_relay', 'MeshRelayConfig?', True), + ('custody', 'CustodyConfig?', True), ('data_enabled', 'boolean', True), ('control_freshness_enforced', 'boolean', True)), 'ProtocolLockDiagnostics': (('held', 'boolean', False), diff --git a/docs/spec/local-api.md b/docs/spec/local-api.md index de63aa164..45beb92a2 100644 --- a/docs/spec/local-api.md +++ b/docs/spec/local-api.md @@ -436,6 +436,7 @@ never one client's share of it. | `get_retry_queue_size` | | `#` | | `get_mesh_relay_stats` | | `{MeshRelayStats}` | | `get_mesh_relay_tunables` | | `{MeshRelayTunables}` | +| `get_custody_stats` | | `{CustodyStats}` | ### Engine: instance-wide tuning @@ -544,6 +545,7 @@ this chapter asserts it. | `process_file_chunk`, `finalize_file` | The inbound chunk driver of a platform transport | | `services.constructor`, `data.constructor`, `data.with_storage` | The server constructs the two objects once, over the engine it owns; `with_storage` takes a callback interface | | `data.wipe_all` | Erases every client's documents and the identity's key documents at once. That is the operator's logout, taken at the server, never one application's call | +| `erase_custody` | Drops every frame this device holds for its neighbours, whichever client's traffic brought them, and resets the counters. The operator's erase, like `data.wipe_all`, never one application's call | | `run_storage_conformance` | Takes a callback interface; a storage backend is verified by the host that supplies it | ## Event catalogue