Skip to content

feat(protocol): custody v1 for replicated-document frames - #493

Merged
bahdotsh merged 8 commits into
mainfrom
feat/headless-custody-v1
Sep 30, 2026
Merged

bahdotsh merged 8 commits into
mainfrom
feat/headless-custody-v1

Conversation

@bahdotsh

@bahdotsh bahdotsh commented Sep 30, 2026 •

Copy link
Copy Markdown
Member

Summary

Custody v1, as docs/spec/custody.md (#488) specifies it: a device can hold a neighbour's replication frames for hours instead of the five seconds a forwarder gives them, off by default. Stacked on #488, which is the chapter this implements; it gets CI only once #488 merges and this PR retargets to main.

  • The depositor asks on its own frame. When the engine offers its own sealed frame to mesh neighbours because the recipient is out of reach, it writes __custody=data on the copy it hands over, and only for a Class A replication frame (delta, snap, vv, blob_gone) judged from the plaintext retained for re-sealing. A direct message, a media chunk, a need_blob, a need_snap, a group frame, and any frame after a restart (no retained plaintext) are offered without it.
  • Every forwarder strips it. prepare_hop, which already clones the outer message to rewrite the hop fields, removes the key from the copy that travels on, custody-enabled or not. The request is kept beside the queued forward for the drop point alone.
  • A custodian holds only what it could not forward. release_due hands the governor's abandoned forwards to the drop point after their ids are released from the suppression cache, and the acceptance table (custody.rs) judges them in the chapter's order. Acceptance never records an id in the cache, and redelivery never consults it: is_suppressed pins the two disjoint.
  • A receipt that settles nothing. A signed __CUSTODY_RECEIPT__ goes once to the depositor over a mesh link, with requires_ack false, only when the depositor advertised data_versions entry 7. At the depositor it suppresses re-deposit toward that custodian until the hold elapses and touches nothing else.
  • Redelivery through a dedicated intake. MeshRelayGovernor::admit_held skips the suppression check and the hop accounting, takes the queue capacity and the per-neighbour rate toward its single target, and marks the forward; the flush transmits it to that neighbour only, the drop point returns it to the store uncounted, and a refused requeue hands it back rather than dropping it. A frame reaching its recipient leaves custody; any other transmission counts as a re-origination, at most once per distinct neighbour per hold.
  • Durable, expiring, erasable. Records are sealed under the new StateCategory::Custody (custody_entries), restored at launch oldest-first under the quotas in force, expired in wall time against the hold in force at every sweep and at restore, and erased by erase_custody(), which the data-layer wipe calls. A launch with custody disabled erases what a previous launch held.
  • Every binding. CustodyConfig (all-optional, overlaid on the core defaults like MeshRelayConfig), CustodyStats (22 counters), get_custody_stats() and erase_custody() over the FFI; the section, the counters and the erase in TypeScript, Swift (a Foundation-only CustodyConfigReader) and Kotlin, with the C6 rule held by a new Rust guard.

The acceptance table

Judged at the drop point, in this order; every row has a counter.

Row Refusal
The frame carried __custody refused_no_request
Custody enabled refused_disabled
The token is data refused_unknown_class
Outer prefix __MLS_ENC__ refused_not_sealed
The carrier proved the arrival link refused_unproven_peer
sender equals the arrival peer (exact string compare) refused_not_depositor
Identifier not already held duplicates (absorbed, not answered)
Session tier, or the stranger tier is open refused_stranger
Battery above the soft relay floor refused_battery
Depositor budget has room, or drop-oldest makes it refused_depositor_full
Global budget has room, or drop-oldest makes it refused_store_full

Counters beyond the refusals: held, held_bytes (gauges), accepted, delivered, re_originated, expired, evicted, receipts_sent, receipts_dropped, receipts_received, receipts_ignored.

The dials

Dial Default Validation
enabled false
hold_ms 6 h > 0 always; < retry.outbox_max_lifetime_ms while enabled
max_entries_per_depositor / max_bytes_per_depositor 64 / 2 MiB both > 0 while enabled; bytes >= 64 KiB (one replication frame at its ceiling)
max_entries / max_bytes 512 / 16 MiB each >= its per-depositor sibling and the stranger dial
stranger_max_entries / stranger_max_bytes 0 / 0 both zero or both > 0; a positive byte cap >= 64 KiB
overflow_policy drop_oldest

Deviations from the chapter, and why

  1. The hold bound is validated only while custody is enabled (hold_ms > 0 is refused regardless). The chapter says "refused at configuration"; a disabled section that refused a configuration would turn every outbox lifetime shorter than the six-hour default hold (test fixtures use 100 ms) into a startup error for a feature the app never switched on.
  2. The receipt goes over a mesh link to the arrival peer, not provably the same link. TransportManager::send_to_neighbor picks the first mesh transport holding a link to the peer; the engine does not record which transport a forward arrived on. Same peer, one transmission, never the outbox, offer, relay or gateway.
  3. The stranger tier must be set by both dials or neither. The chapter says a deployment "raises both"; one at zero with the other positive is refused rather than read as a narrow allowance.
  4. The receipt suppression window is clamped to this device's own outbox_max_lifetime_ms before it is applied, since a longer window would outlive the entry it protects. (Also keeps the instant arithmetic in range.)
  5. The recipient is exempt from "at most once per neighbour". A frame whose recipient is the discovered neighbour is queued toward it on every discovery until it transmits; re-origination toward anyone else is once per hold.
  6. no_request is judged before disabled. The chapter's table listed the switch first; applied literally, refused_disabled counted every abandoned forward on an off device. Now a frame that asked for nothing lands in refused_no_request on any device, and refused_disabled counts deposits an off device turned away, which is what tells "off" from "nobody asked". The row order is corrected in the chapter on this branch.
  7. Restore draws on the inbound delete pool (refusing_private, the pending-decrypt walk's shape) rather than a fifth pool, so the launch ceiling test is unchanged.
  8. Battery is judged before the budgets. The chapter's table put it last; taken literally, a refusal for battery would first evict the depositor's oldest frame under drop-oldest to make room for a deposit it then refuses. The code judges it before the eviction loops, and the row moved up in the chapter on this branch.

Validation

  • cargo fmt --all -- --check, cargo clippy --workspace -- -D warnings, the messaging-only cargo clippy --package offline-protocol --no-default-features --locked -- -D warnings, rustdoc under -D warnings: clean.
  • cargo test --workspace --lib: every binary green (engine 1983). cargo test -p offline-protocol --test mesh_forwarding: green, including two new tests (one sleeps 5.2 s to reach the governor's real overdue cut-off). cargo test -p offline-protocol-uniffi --lib: green, including the two new guards, negative-controlled by hand.
  • New engine tests: 12 store unit tests, 4 receipt vector tests, 2 classifier tests, 5 governor tests, 5 config tests, 9 three-node tests over real MLS sessions (protocol/tests/custody.rs).
  • Mutation (round one): six mutants for the six new behaviours, six killed (a returned forward keeping its neighbour tried, redelivery ignoring the mesh-link gate, disabled before no_request, suppression from arrival instead of the receipt's timestamp, erase leaving queued redeliveries, battery after the budgets).
  • Mutation: 16 mutants, 16 killed (each mutant reverts one rule: the forwarder strip, the drop point touching the suppression cache for a marked forward, not_depositor, the receipt suppression at offer, expiry against the hold, the hold bound, an unknown-id receipt, the delivered record delete, redelivery to the depositor, the held intake skipping the neighbour rate, restore keeping expired records, a receipt without the capability, the closed stranger tier, the data wipe skipping custody, a marked forward judged as a deposit, and at most once per neighbour).
  • Bindings regenerated with ./scripts/generate-bindings.sh; drift check clean; scripts/tests/test-generate-bindings.sh 54 guards pass.
  • React Native: npm run build, npm run test:js (a new custody-config.test.js, 6 cases). Swift: swift test 418 pass, with the new CustodyConfigReaderTests. Android: 566 JVM tests pass in a clean copy of the module (5 new parser cases).
  • Python: the full suite on 3.14 (455 plus the 5 new test_custody.py cases); the new file under pytest --count 40 on 3.12, 3.13 and 3.14. 3.10 is not installed here; CI covers it.

Review round one

Twelve findings, all landed in ea98b0dd:

  • The tracked bindings/react-native/node_modules symlink is removed, both ignore rules now say node_modules (a symlink is not a directory), and the JS gates ran again over a real npm ci in this worktree.
  • The iOS reader fix sits under ### Fixed.
  • tools/spec-vectors/generate.py now owns custody-receipt-v1.vectors.json (build_custody_receipt_vectors, registered in FILES); --check passes with ten files, and conformance.md says ten.
  • The largest-hold vector is 9007199254740991, and the chapter's receipt table says hold_ms is a u64 on the wire with the vectors kept double-safe.
  • Redelivery queues nothing toward a peer no mesh link reaches (a gateway presence edge runs the same hook), and a marked forward that comes back untransmitted leaves its neighbour untried.
  • no_request before disabled, and battery before the budgets, in the code, the chapter's table on this branch, the guide and the table above.
  • The receipt suppression runs on the wall clock from the receipt's own timestamp, swept against Unix time.
  • erase_custody drops redeliveries already queued (MeshRelayGovernor::drop_held).
  • The guide says the receipt goes over a mesh link to the peer that handed the frame over.
  • The acceptance table returns what it proved, so no expect remains; the battery snapshot is taken only for a frame that asked.

Review round two

One finding, landed in its own commit: the receipt suppression is computed from min(receipt timestamp, local now), because the control gate admits a timestamp up to two days ahead and never checks it on an unbound signature, so a fast-clock custodian could otherwise suppress re-deposit toward itself past its own hold. The timestamp test gained a receipt stamped a day ahead (the suppression ends by local now plus the hold); mutating the clamp away fails it.

Also fixed

The iOS config readers read JSON 0 and 1 as booleans. MeshRelayConfigReader excluded booleans with !(dict[key] is Bool), which Swift also answers true for NSNumber(1) and NSNumber(0), so a fanout: 1, activityIdleWindows: 1 or jitterMinMs: 0 from React Native reached the core as unset. Found by the new custody reader's test (stranger_max_entries: 1 came back nil); both readers now exclude booleans by CoreFoundation type, with a regression test each. Changelog entry added.

Not in this PR

  • No custody event. Counters only, as the chapter has it; nothing new crosses on_event.
  • No Swift or Kotlin application-level wrappers beyond the config, the counters and the erase.
  • No device run. Everything above is mock transports and JVM/simulator tests.
  • Classes B to F, any incentive scheme, and relay or gateway custody, as the chapter's "does not specify" list has them.

Notes for reviewers

  • Stacked on docs(spec): the custody chapter, with its receipt prefix reserved #488. The base is at its final reviewed tip; this branch will rebase if it moves.
  • PendingRelay gained two fields; it is constructed only inside the governor.
  • take_due is now #[cfg(test)] and wraps release_due, which the flush uses so the abandoned forwards reach the drop point.
  • The CustodyStats UDL dictionary is read by two guards that derive the field list from the UDL; adding a counter means updating the count in react_native_bridges_report_every_custody_counter and the three bridges.

A forwarder holds a stranger's frame for five seconds; the device's own
messages get seven days on disk. The custody chapter specifies how a device
closes that gap for one class of traffic, and it is written before the code
so the shape is agreed where the mechanism is invariant-laden.

The chapter states seven invariants first: custody is replication, never
transfer; a receipt settles nothing; the hold is strictly shorter than the
outbox lifetime, because a custodian holds ciphertext it cannot re-seal; a
custodian never blanks the route it is holding; Class A only, by the
depositor's word, since a custodian cannot classify a sealed frame; a deposit
comes from the depositor itself over the link that proved it, so the frame's
sender must be the proven arrival peer and every forwarder strips the unsigned
deposit key from any third-party frame it transmits; and custody is opt-in
with its own erase. Then the deposit (the depositor's own frame with a
reserved hop-local metadata key, accepted only where forwarding has already
failed and the forwarding identifier is already released), the receipt (a
signed control frame under the ordinary gate, sent once over the arrival link
and gated on a capability entry so a peer that does not reserve the prefix
never receives one), redelivery through a dedicated governor intake that
skips the suppression check and the hop accounting and returns a frame that
reaches no link to the store, a hold judged in wall time from the acceptance
timestamp so a lowered hold applies to records already held and a custodian
that was off for a week delivers nothing stale, the quotas with a stranger
tier of zero, and the erase. The chapter states the condition on "cannot
falsely settle": it holds with crypto recovery enabled, the default; with it
disabled a stale copy is a terminal decrypt failure that is acknowledged, and
for Class A the next version-offer exchange re-offers the change.

Reservations, each in the registry that owns it: `__CUSTODY_RECEIPT__` in
the control-message chapter and the engine's prefix macro, so the two-way
registry test binds them and application text can never occupy the name;
`__custody` in the wire-format chapter's reserved metadata keys;
`data_versions` entry 7 in the capability chapter. No handler consumes the
prefix yet and no peer advertises the entry, so nothing changes on the wire
until the implementation lands.

The threat model gains R19 (custody-borne re-key pressure, its condition,
and what an integrator should read from the signal), R20 (a custodian
retains third-party routing metadata for hours) and R21 (deposit spam,
bounded by one proven address for depositor, tier, receipt and capability).
The acknowledgement side-channel section says custody changes no
acknowledgement decision. The replication state machine records the
redelivery trigger, what each redelivered kind does at the recipient, and
that the bottom rung of the catch-up ladder becomes reachable more often,
which is one of the two reasons the hold is hours and not days. The
conformance chapter lists custody among the chapters that are not yet
surfaces: the receipt vector lands with the codec.
A device can hold a neighbour's Class A replication frames for hours instead of the five seconds a forwarder gives them (docs/spec/custody.md), off by default. The depositor writes the one-hop request on its own sealed frame from the plaintext it retains for re-sealing; every forwarder strips it; a custodian judges a frame at the drop point after its id is released from the suppression cache, answers with the signed receipt once over the arrival link, redelivers through a dedicated governor intake, expires records in wall time against the hold in force, and keeps them sealed under the new custody storage category. A receipt settles nothing. The receipt body has frozen vectors.
…nding

CustodyConfig (all-optional, overlaid on the core defaults), CustodyStats, get_custody_stats() and erase_custody() over the FFI, with the section, the counters and the erase in TypeScript, Swift and Kotlin, a Foundation-only CustodyConfigReader, the C6 guard, and Python tests. Also fixes the iOS readers reading JSON 0 and 1 as booleans.
Redelivery queues nothing toward a peer no mesh link reaches and an abandoned forward leaves its neighbour untried; no_request is judged before disabled and battery before the budgets, in the chapter's table on this branch too; the receipt suppression runs on the wall clock from the receipt's own timestamp; erase drops queued redeliveries; the drop point takes the battery snapshot only for a frame that asked; the acceptance table returns what it proved so no expect remains. The custody vectors are now generated by tools/spec-vectors/generate.py, with the largest hold within the double-safe range; the tracked node_modules symlink is removed and both ignore rules match a symlink too; the iOS reader fix moves under Fixed.
@bahdotsh
bahdotsh force-pushed the docs/headless-custody-chapter branch from e51a2bd to b88a3c6 Compare September 30, 2026 19:35
# Conflicts:
#	CHANGELOG.md
#	docs/README.md
#	docs/spec/README.md
#	docs/spec/custody.md
get_custody_stats is a read-only instance view beside get_mesh_relay_stats.
erase_custody is a platform operation: it drops what every client's traffic
deposited, so it is the operator's call, like data.wipe_all. Regenerates the
local API table for the custody additions to the definition.
@bahdotsh
bahdotsh changed the base branch from docs/headless-custody-chapter to main September 30, 2026 20:04
@bahdotsh bahdotsh closed this Sep 30, 2026
@bahdotsh bahdotsh reopened this Sep 30, 2026
@github-actions github-actions Bot locked and limited conversation to collaborators Sep 30, 2026
@bahdotsh
bahdotsh merged commit 2977ff9 into main Sep 30, 2026
43 of 44 checks passed
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant