Skip to content

docs(backend): protocol negotiation, replay protection, caching, and testing guides - #600

Merged
codebestia merged 1 commit into
codebestia:devfrom
AdaBliss:docs/backend-protocol-replay-caching-testing
Aug 31, 2026
Merged

docs(backend): protocol negotiation, replay protection, caching, and testing guides#600
codebestia merged 1 commit into
codebestia:devfrom
AdaBliss:docs/backend-protocol-replay-caching-testing

Conversation

@AdaBliss

Copy link
Copy Markdown

Summary

Adds four backend documents covering areas whose behaviour is currently only learnable by reading the source, and indexes them in docs/README.md as the index's maintenance rule requires.

This is a documentation-only change. No source file is touched.

Document Issue
apps/backend/docs/concepts-protocol-negotiation.md #557
apps/backend/docs/concepts-replay-protection.md #558
apps/backend/docs/testing.md #566
apps/backend/docs/concepts-caching.md #567

closes #557
closes #558
closes #566
closes #567

Device capability and E2EE protocol negotiation (#557)

Covers lib/capabilities.ts and services/e2eeProtocol.ts:

  • The capability payload shape (protocols, ciphersuites, fileTransfer), why every field is optional and unrecognised values are preserved rather than rejected.
  • BASELINE_PROTOCOL and the three ways a client predating the field ends up on the sealed-box baseline: the column default, an omitted capabilities, and a document normalizeCapabilities cannot parse.
  • How a device advertises at registration (POST /auth/verify, POST /devices/link/verify), including the update-in-place upgrade path, and the three endpoints that read capabilities back, one of which also returns the server-computed negotiatedProtocol.
  • selectProtocol and the strongest-first PROTOCOL_PRIORITY, plus MLS ciphersuite selection and the per-device-pair rationale.
  • The per-envelope message_envelopes.protocol column and why it is recorded per envelope rather than per device: capabilities are mutable, so a device-level record would make all pre-cutover history appear to have been built with the newer construction.
  • checkEnvelopeProtocols, both violation reasons, the 400/409 split, and why unresolvable device ids are skipped rather than rejected.
  • The protocol_mismatch rejection on both transports, with the four-step client recovery, including "do not fall back to a weaker protocol".
  • A section on how the pieces together give a staged sealed-box to Signal rollout with no coordinated cutover, and the contributor rule that a new construction must be added to KNOWN_PROTOCOLS, the e2ee_protocol enum, and PROTOCOL_PRIORITY together.

Deliberately does not restate the migration plan in signal-migration.md; it links to it and documents the mechanism instead.

Replay protection and event idempotency (#558)

Covers services/replay-protection.service.ts and the dispatcher path:

  • The two layers side by side, and why neither subsumes the other: the transport window is finite and Redis-backed, while most events have no durable id to be idempotent on and the message layer cannot see a deliberate frame replay at all.
  • The replay:{deviceId}:{eventId} key and why the device component is load-bearing: with a global key, one device's replayed id would silently drop another device's legitimate first-time event.
  • The TTL: REPLAY_PROTECTION_TTL_SECONDS, default 300, accepted range 1-86400, invalid values degrading to the default, and its relationship to SOCKET_EVENT_MAX_AGE_MS.
  • Fail-open behaviour when Redis is null or errors, why hardening is not worth trading total availability for, and the consequence for tests.
  • dispatch_ack with duplicate: true/false, and that duplicate: true is an acknowledgement rather than an error.
  • The message-level messageId check on all four send paths, spanning transports and with no window.
  • The fixed dispatcher ordering, and the fact that dispatcher.register handlers are reachable only through the enveloped dispatch path, so every registered event gets the check.

One accuracy note for reviewers: send_file_message is still attached with a raw socket.on in src/socket/messaging.ts, so it does not currently get envelope validation or the eventId check; its duplicate suppression is the messageId layer alone. The document states the rule and flags that single exception rather than overstating the coverage.

Backend testing guide (#566)

The backend-specific companion to docs/testing.md, which it links to rather than duplicating:

  • A copyable route-test skeleton mocking db/index.js, db/schema.js, drizzle-orm, lib/redis.js, and middleware/auth.js, with the reasoning attached to each block, and why the subject is imported with await import(...).
  • The socket-handler pattern: emit a well-formed envelope on dispatch, why grabbing a raw listener finds nothing (and why a test written that way used to pass while bypassing the auth gate, envelope validation, and idempotency), and the five properties to preserve when copying the helper.
  • The Drizzle chain traps: .values() needing to be both thenable and expose .returning(), with the two wrong stubs and their opposite failure modes; transaction mocks; select().from().where().groupBy(); sql as both tag and namespace; and the two import-time failures from a missing table or operator.
  • The rule that a route gaining a new service dependency needs the corresponding vi.mock added to every existing test file for that route in the same change, with a table of the frequently mocked services.
  • Resetting shared in-process state, with the rate-limit counters explained in full plus a table of the other modules and their exported reset hooks.
  • A nine-point checklist for a new backend test.

Backend caching reference (#567)

Covers lib/conversationCache.ts and the Redis cache key in lib/redis.ts:

  • The conversations:{userId} key, the 30-second CONV_CACHE_TTL, the cached payload, and why the archived view is not cached at all.
  • The read and write paths, both guarded on redis being non-null and by try/catch.
  • A table of all thirteen invalidation sites with the user-visible event that triggers each, including the one direct redis.del in PATCH /conversations/:id/settings (correct, because mute and archive are per-member) and the device-added/revoked path in emitDeviceChangeEvent, plus the general rule for new call sites.
  • The degraded behaviour when Redis is unavailable, and why the cache is an optimisation rather than a correctness dependency: nothing is gated on it, no path reads it to make a decision, and nothing stored is unreconstructible from Postgres.
  • The per-device scoping hazard. GET /conversations builds its preview through getConversationRelations(req.auth.deviceId), which filters envelopes to the requesting device, so the response is device-specific. convCacheKey is conversations:{userId} and does not carry the device, so for a multi-device user the first device to populate the entry serves its own ciphertext to the user's other devices for up to the TTL. The document states this as a live, TTL-bounded hazard, names convCacheKey as the single place a fix belongs, and notes that a device-scoped key also changes invalidation, since invalidateConversationCaches takes user ids. Flagged for maintainer triage rather than changing behaviour in a documentation PR.
  • A short table of the other Redis key namespaces so they are not mistaken for caches and cleared as if they were.

Checks

  • npx prettier --check on all five changed files: clean.
  • All relative links in the new documents and the new index rows verified to resolve to existing files.
  • Every code reference (function name, env var, key format, TTL, status code, call-site line, exported reset hook) taken from the source on dev at the time of writing.
  • No source file changed, so no test behaviour is affected.

…and testing guides

Adds four backend documents and indexes them in docs/README.md.

concepts-protocol-negotiation.md (codebestia#557): the device capability payload
shape, the sealed_box BASELINE_PROTOCOL applied to clients predating the
field, how selectProtocol picks a mutually supported protocol, why the
protocol is recorded per envelope rather than per device, the
protocol_mismatch rejection and the client recovery steps, and how the
whole mechanism supports a staged sealed_box to Signal rollout.

concepts-replay-protection.md (codebestia#558): the two distinct layers of
duplicate suppression - transport-level eventId dedup and message-level
messageId idempotency - the device-scoped Redis key, the env-configurable
TTL and its fail-open behaviour, the dispatch_ack duplicate flag, and the
dispatcher path that applies the check to every registered handler.

concepts-caching.md (codebestia#567): the conversation-list cache key, TTL and
payload, every invalidateConversationCaches call site with its trigger,
the degraded behaviour when Redis is unavailable, and the per-device
scoping hazard created by a device-specific preview under a key that does
not name the device.

testing.md (codebestia#566): the backend companion to the cross-app testing guide -
the standard route-test mock set with a copyable skeleton, the Drizzle
chain-mocking traps, driving socket handlers through the enveloped
dispatch event, adding service mocks when a route gains a dependency, and
resetting shared in-process state between tests.
@drips-wave

drips-wave Bot commented Aug 31, 2026

Copy link
Copy Markdown

@AdaBliss Great news! 🎉 Based on an automated assessment of this PR, the linked Wave issue(s) no longer count against your application limits.

You can now already apply to more issues while waiting for a review of this PR. Keep up the great work! 🚀

Learn more about application limits

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants