feat(redis): ElastiCache Serverless compatibility with full standard-Redis parity - #4114
Open
tlongwell-block wants to merge 15 commits into
Open
feat(redis): ElastiCache Serverless compatibility with full standard-Redis parity#4114tlongwell-block wants to merge 15 commits into
tlongwell-block wants to merge 15 commits into
Conversation
Co-authored-by: npub1t2tgm7d8f995uqvmnm8h88sg3wnpp9a5xysjf6dg3tjmgt3ltulqdp8ehr <5a968df9a7494b4e019b9ecf739e088ba61097b4312124e9a88ae5b42e3f5f3e@buzz.block.builderlab.xyz> Signed-off-by: npub1t2tgm7d8f995uqvmnm8h88sg3wnpp9a5xysjf6dg3tjmgt3ltulqdp8ehr <5a968df9a7494b4e019b9ecf739e088ba61097b4312124e9a88ae5b42e3f5f3e@buzz.block.builderlab.xyz>
Bulk presence read issued a single `MGET` over one key per pubkey. Presence
keys are `buzz:{community}:presence:{pubkey_hex}` with no hash tag, so they
hash to different cluster slots and the command fails `CROSSSLOT` on any
cluster-mode server, including ElastiCache Serverless.
Replace it with one single-key `GET` per pubkey in a non-atomic pipeline:
same one round trip, same key format, works identically on standalone Redis.
The key format is deliberately left untagged — tagging presence by community
would co-slot an entire community's keys and concentrate load on one slot.
Reply reassembly is positional, so a short reply would silently attribute one
member's status to another (user-visible as other people's online status).
Length is now checked and mismatches error instead of truncating under `zip`.
`.atomic()` must never be added to this pipeline: it wraps the batch in
MULTI/EXEC, which is dispatched as a unit and restores the cross-slot failure.
Documented at the call site, since single-node tests cannot catch it.
Also route the test Redis endpoint through one `test_redis_url()` helper.
`make_test_pool()` hardcoded `127.0.0.1:6379` while pub/sub tests passed their
own literal URL, so pointing the suite at another server split the pool and the
pub/sub client across two hosts — which surfaced as three unrelated timeouts
that looked like cluster incompatibility.
Tests: bulk read asserts the reassembled `pubkey -> status` map (not vector
length) across distinct slots, shuffled order, interleaved missing keys, and
duplicates, plus permutation-equality so misalignment fails without knowing
the expected order. A test-local CRC16-XMODEM (checked against the Redis
Cluster reference vector) asserts presence keys span multiple slots, which
also fails if someone later hash-tags them.
CI now runs the buzz-pubsub suite including `#[ignore]`d Redis tests against
the standalone Redis the backend-integration job already starts; previously
nothing ran them. This keeps standard Redis in the matrix permanently.
Verified at this tree: 39/39 with `--include-ignored` against standalone
Redis 7 and against a single-shard cluster-mode server owning all 16384
slots. Negative control: restoring the `MGET` on the same rig fails with
`CrossSlot: Keys in request don't hash to the same slot`, confirming the rig
detects the bug the fix removes.
Co-authored-by: Dawn (sprout agent) <c6237ef84fa537c78dcee78efd2d4e59f728859c7f194da42ac51ededfa0be05@sprout-oss.stage.blox.sqprod.co>
Signed-off-by: tlongwell-block <109685178+tlongwell-block@users.noreply.github.com>
Co-authored-by: npub1t2tgm7d8f995uqvmnm8h88sg3wnpp9a5xysjf6dg3tjmgt3ltulqdp8ehr <5a968df9a7494b4e019b9ecf739e088ba61097b4312124e9a88ae5b42e3f5f3e@buzz.block.builderlab.xyz> Signed-off-by: npub1t2tgm7d8f995uqvmnm8h88sg3wnpp9a5xysjf6dg3tjmgt3ltulqdp8ehr <5a968df9a7494b4e019b9ecf739e088ba61097b4312124e9a88ae5b42e3f5f3e@buzz.block.builderlab.xyz>
Co-authored-by: npub1t2tgm7d8f995uqvmnm8h88sg3wnpp9a5xysjf6dg3tjmgt3ltulqdp8ehr <5a968df9a7494b4e019b9ecf739e088ba61097b4312124e9a88ae5b42e3f5f3e@buzz.block.builderlab.xyz> Signed-off-by: npub1t2tgm7d8f995uqvmnm8h88sg3wnpp9a5xysjf6dg3tjmgt3ltulqdp8ehr <5a968df9a7494b4e019b9ecf739e088ba61097b4312124e9a88ae5b42e3f5f3e@buzz.block.builderlab.xyz>
Co-authored-by: npub1t2tgm7d8f995uqvmnm8h88sg3wnpp9a5xysjf6dg3tjmgt3ltulqdp8ehr <5a968df9a7494b4e019b9ecf739e088ba61097b4312124e9a88ae5b42e3f5f3e@buzz.block.builderlab.xyz> Signed-off-by: npub1t2tgm7d8f995uqvmnm8h88sg3wnpp9a5xysjf6dg3tjmgt3ltulqdp8ehr <5a968df9a7494b4e019b9ecf739e088ba61097b4312124e9a88ae5b42e3f5f3e@buzz.block.builderlab.xyz>
…ng @ b25be0f) Wren-approved at exact head b25be0f (9/9/9.5). Cross-slot Lua fencing hash-tag cutover behind strict BUZZ_TUNNEL_FENCE_KEY_FORMAT gate (default legacy), watermark-transfer runbook, six ignored Redis fence tests wired into backend-integration. Co-authored-by: npub1qyvc0c5kl4gqv2fd97fsk46tu378sqgy35vc83rvgfwne90sel7s0ed67d <011987e296fd5006292d2f930b574be47c7801048d1983c46c425d3c95f0cffd@buzz.block.builderlab.xyz> Signed-off-by: npub1qyvc0c5kl4gqv2fd97fsk46tu378sqgy35vc83rvgfwne90sel7s0ed67d <011987e296fd5006292d2f930b574be47c7801048d1983c46c425d3c95f0cffd@buzz.block.builderlab.xyz>
…scriptions Two subscribers rode wildcard patterns: `buzz:*:cache-invalidate` and `buzz:*:conn-control`. ElastiCache Serverless restricts `PSUBSCRIBE`, so both fail loudly there. Replace each with exact `SUBSCRIBE`/`UNSUBSCRIBE` on the per-community channels this pod actually needs. Standalone Redis and cluster-mode behave identically — there is no separate serverless mode, and no `SSUBSCRIBE`. Reconciliation is level-triggered. A `desired` closure is re-read inside the subscriber task immediately before each command, never snapshotted, so a queued unsubscribe cannot be applied against interest that has since been renewed. Establishment is recorded only after Redis acks the `SUBSCRIBE`: `sink.subscribe().await` returning `Ok` *is* the acknowledgement, which is what lets a cross-crate gate read acknowledged state without adding a oneshot ack path to fire-and-forget retain. Convergence is driven by a 1s tick, with wakes as a latency shortcut only — a lost or coalesced wake costs latency, never convergence, and the tick is the restore path that fires with no external event. On disconnect the whole established set is cleared, and the next connect rebuilds from `desired` rather than from a remembered active set. The two families have different interest owners. Connection control follows live sockets: a pod needs a community's disconnect commands only while it holds a socket a remote ban could close. Cache invalidation follows *cache residency*, not connection lifetime, because a cached authorization decision outlives the socket that populated it; scoping it to connections would leave a pod serving authz for a community it no longer listens to. Residency lapses on a timer, so no call path owes an unsubscribe. It cannot be driven off a moka eviction listener: moka's notifications are lazy and do not fire at TTL on an idle cache, so an event-driven release would never run on the pod that most needs it. `CacheResidency::admit` deliberately both records residency and returns the gate. Split into a separate gate and a separate record, only an already-admitted community would ever be recorded — so on a fresh pod nothing enters the desired set, nothing is subscribed, nothing is established, and all three authorization caches read through to Postgres for the life of the process with no error and no log. Folding them makes that ordering unrepresentable. The gate covers cache *inserts only*, never reads. Gating reads would discard a live, still-invalidatable cache on every reconnect and turn a pub/sub blip into a DB stampede. A refused insert means the pod reads through — degraded, not stale — and bumps `buzz_authz_cache_insert_skipped_total`. `AUTHZ_CACHE_TTL` is now one constant shared by the three moka caches and by residency, because residency must outlast every entry it protects. Both subscribers moved from an early spawn to alongside their consumer loops: each needs a desired-set closure read off `AppState`, which did not exist at the old spawn point. Tests: five Redis-backed reconciliation tests, asserting on `PUBSUB NUMSUB` rather than `PUBLISH`'s return. `PUBLISH` counts pattern matches, so any unrelated `PSUBSCRIBE buzz:*` client on a shared server inflates it — and exact-subscriber count is precisely the property this change is about, since it cannot be satisfied by a leftover wildcard. 100 exact subscriptions on one connection route every payload to its own community (no cross-wiring under the wrong tenant label); a withdrawn subset unsubscribes while the rest keep routing; a newly desired community converges on the tick with no wake at all; and a severed connection rebuilds the entire desired set. The sever kills only its own client by id — `CLIENT KILL TYPE pubsub` would take down every other pub/sub client on the server, including concurrent tests. Relay-side tests assert the bootstrap closes: the refused first `admit` must still make the community desired, which is the property whose absence is the deadlock above. Co-authored-by: npub17jjz49l9jjmhhk7cac63j8yt9z555n9cw8vk7v5jz4vzw4ppld5qgj57cc <f4a42a97e594b77bdbd8ee35191c8b28a94a4cb871d96f32921558275421fb68@buzz.block.builderlab.xyz> Signed-off-by: npub17jjz49l9jjmhhk7cac63j8yt9z555n9cw8vk7v5jz4vzw4ppld5qgj57cc <f4a42a97e594b77bdbd8ee35191c8b28a94a4cb871d96f32921558275421fb68@buzz.block.builderlab.xyz>
…ishment lock Review (Wren, ratified by Eva) found a renewal-vs-withdrawal race in the reconciler. `reconcile()` snapshotted `desired()` and `established` once, then ran both loops against those snapshots with awaited Redis round-trips in between. Its own doc comment claimed the opposite — "read immediately before each command" — describing a contract the code did not implement. I wrote that comment, and it is exactly how the next reader would have re-introduced this. The lost interleaving: desire drops community C, the pass records `have` containing C, and before the unsubscribe loop reaches C a new authz resolution calls `admit(C)`. `admit` sees the still-present establishment bit, permits a cache insert, and the reconciler then withdraws establishment and unsubscribes from its stale view. The entry stays readable for its whole 10s TTL with no invalidation channel behind it — the silent stale-authz hole this lane exists to close, reached through our own unsubscribe path instead of AWS's restriction. A per-command re-read does not fix it: renewal can still land between the check and the removal. `admit()`'s wake shortens the gap to milliseconds but cannot close it, because pub/sub has no replay — the tick restores state, nothing restores a dropped message. Fix: `CommunityTopics::withdraw_undesired` re-reads `desired()` *inside* the establishment write lock and returns only the communities to unsubscribe. `admit()` already recorded residency before reading establishment, so the two critical sections are mutually exclusive and only two orders exist: either the renewal's residency write precedes the withdrawal's re-read (interest is seen, the subscription survives) or it follows the withdrawal (establishment reads false, nothing is admitted). An admitted entry can no longer outlive its subscription. `remove_established` is gone, so no caller can withdraw outside the lock; the ordering requirement is documented at both call sites, since each is correct alone and only the pairing carries the property. Subscribe stays on the pass's initial read: holding a subscription nobody wants costs one idle channel until the next pass and can never make an entry readable without coverage. Both families inherit the invariant because it lives in the shared reconciler; conn-control's milder exposure (a lost disconnect leaves a socket the DB ban row still refuses at next authz) is now stated in code rather than implied. Tests, all mutation-verified to bite: - `a_renewal_racing_withdrawal_never_sees_establishment_it_loses` parks the `desired` closure inside the withdrawal critical section and releases a renewal thread from there, forcing the interleaving rather than sleeping for it. Moving the `desired()` read back outside the lock fails it. - `interest_renewed_mid_reconcile_keeps_its_exact_subscription` proves it end-to-end through the real path against live Redis, asserting on `PUBSUB NUMSUB`. Its first draft did NOT bite the reinstated bug twice over: renewing on the pass's first read meant no withdrawal was ever attempted, and a coarse end-state assertion passed vacuously because the next tick re-subscribes. It now discriminates on read count within a window well under `RECONCILE_INTERVAL` and asserts establishment never transiently drops — the transient loss is the bug, the repaired end state is not proof. - `withdrawal_reads_live_desire_not_the_passs_stale_view`, `withdrawal_still_removes_communities_nobody_wants`, and relay-side `admitted_residency_is_visible_to_a_concurrent_withdrawal` cover both directions and the cross-crate pairing. Also corrects two stale comments Eva flagged: `state.rs` named the renamed `may_cache` in an intra-doc link (a real rustdoc `broken_intra_doc_links` warning I introduced), and `event.rs` test prose said "PSUBSCRIBEs" for a path that has always used exact `SUBSCRIBE`. Verified at this tree: buzz-pubsub 52/52 `--include-ignored` against standalone Redis 8.6.2 and against a cluster-mode rig (`cluster_state:ok`, 16384 slots); dead-port control fails all 6 Redis-backed reconciliation tests by name while the 7 pure-logic tests still pass; buzz-relay --lib 807 passed / 0 failed; workspace `clippy --all-targets -D warnings` rc=0; `cargo fmt --all --check` clean. Co-authored-by: npub17jjz49l9jjmhhk7cac63j8yt9z555n9cw8vk7v5jz4vzw4ppld5qgj57cc <f4a42a97e594b77bdbd8ee35191c8b28a94a4cb871d96f32921558275421fb68@buzz.block.builderlab.xyz> Signed-off-by: npub17jjz49l9jjmhhk7cac63j8yt9z555n9cw8vk7v5jz4vzw4ppld5qgj57cc <f4a42a97e594b77bdbd8ee35191c8b28a94a4cb871d96f32921558275421fb68@buzz.block.builderlab.xyz>
Review (Wren, ratified by Eva) found the same "an admitted insert must not outlive its protection" invariant failing at a second boundary. Round 1 closed the lock boundary; this is the lifetime boundary. `CacheResidency::admit()` stamped `deadline = now + AUTHZ_CACHE_TTL`, read establishment, and returned a bool. All three callers inserted into moka only afterwards, and moka starts its own identical TTL at that later instant. So the residency window that protects an entry started *before* the entry's own clock — under-covering it by the call gap at minimum, and by an unbounded amount if the thread is preempted in between. The lost interleaving: admit(C) renews residency to T+10s and returns true; the caller is delayed; after T+10s the reconciler prunes residency and correctly withdraws establishment and unsubscribes; the caller resumes and inserts an entry readable for nearly a full TTL with no invalidation channel behind it. The gate no longer hands out a bool for the caller to act on. `CommunityTopics::with_established(community, f)` runs `f` inside the establishment read lock, and `CacheResidency::admit_and_insert(community, insert)` performs three ordered steps in one call: 1. residency is recorded before the establishment read (round 1's invariant, unchanged): withdrawal re-reads desire under the matching write lock, so either this renewal is seen and the subscription survives, or the withdrawal already happened and nothing is admitted; 2. `insert()` runs inside the read lock, so no withdrawal can interleave between the gate and the entry landing; 3. the residency deadline is re-stamped after `insert()` returns and still inside that section, so it is based at or after the entry's own clock start and therefore outlives it. `admit()` is deleted rather than deprecated, so read-then-act is unrepresentable rather than warned against — the same structural exclusion that removed `remove_established` in round 1. The three call sites cache three different value types (membership, accessible-channels, channel-visibility), so the insert travels as a closure rather than making the API generic over the cache. TTL padding was rejected: it narrows the window without closing arbitrary descheduling. The tests assert the exclusion *structurally*, with no threads and no sleeps. An earlier draft of this commit parked a withdrawal thread and slept 50ms from inside the critical section. That is deterministic on correct code, but its *mutation bite* is not: under the unlocked mutation a late-scheduled withdrawer still loses the race and the test passes vacuously. Wren caught it pre-push. A probabilistic detector is not a deterministic test, so both concurrency tests were rewritten to assert `try_write()`/`WouldBlock` from inside the critical section — the lock state itself rather than a racing thread's outcome. `CommunityTopics::try_withdraw_undesired_for_test` is the seam for that; it shares `withdraw_locked` with the real `withdraw_undesired` so the twin cannot drift from the path it stands in for. `residency_outlives_the_entry_it_protects` previously compared TTL constants and was green while the bug was live — equal durations do not nest when their clocks start in the wrong order. It now records `Instant::now()` from inside the insert closure and asserts `deadline >= inserted_at + AUTHZ_CACHE_TTL`, the nesting relation itself. Mutation results, each re-run to confirm determinism rather than a lucky pass: * releasing the establishment lock before calling `f` (round 2's read-then-act, restored) fails `no_entry_can_land_after_its_subscription_is_withdrawn` and `with_established_holds_the_gate_for_the_whole_closure` on 5 of 5 runs, and leaves `residency_outlives_the_entry_it_protects` green; * dropping the step-3 re-stamp fails both relay tests on 3 of 3 runs. It bites `no_entry_can_land...` too because that test's post-insert assertion — the next withdrawal must withdraw *nothing* — is exactly what the re-stamp buys; the lock alone does not survive a residency that lapsed during the insert. So the lock and the re-stamp are separately load-bearing: the first mutation isolates the lock, the second shows the re-stamp is required even with the lock intact. Also resolves the `rustdoc::private_intra_doc_links` warning introduced in the previous commit: `cargo doc -p buzz-pubsub --no-deps` emitted 4 warnings at c05cdb1 and emits 3 here, the remainder pre-existing and unrelated. Co-authored-by: npub17jjz49l9jjmhhk7cac63j8yt9z555n9cw8vk7v5jz4vzw4ppld5qgj57cc <f4a42a97e594b77bdbd8ee35191c8b28a94a4cb871d96f32921558275421fb68@buzz.block.builderlab.xyz> Signed-off-by: npub17jjz49l9jjmhhk7cac63j8yt9z555n9cw8vk7v5jz4vzw4ppld5qgj57cc <f4a42a97e594b77bdbd8ee35191c8b28a94a4cb871d96f32921558275421fb68@buzz.block.builderlab.xyz>
…bsub * origin/main: fix(buzz-acp): thread cache-read tokens into NIP-AM kind:44200 events (#3999) feat(relay): accept kind:30621 multi-repo projects at ingest (#3171) fix(release): preserve main in desktop PR body (#3979) chore(release): release Buzz Desktop version 0.5.3 (#3972) fix(release): require exact-head approval for desktop tags (#3973) fix(release): make desktop tagging squash-safe (#3965) Revert "chore(release): release Buzz Desktop version 0.5.3" (#3960) docs(nips): add single-coordinate manual-unread override layer and verification model to NIP-RS (#2864) chore(release): release Buzz Desktop version 0.5.3 Co-authored-by: npub17jjz49l9jjmhhk7cac63j8yt9z555n9cw8vk7v5jz4vzw4ppld5qgj57cc <f4a42a97e594b77bdbd8ee35191c8b28a94a4cb871d96f32921558275421fb68@buzz.block.builderlab.xyz> Signed-off-by: npub17jjz49l9jjmhhk7cac63j8yt9z555n9cw8vk7v5jz4vzw4ppld5qgj57cc <f4a42a97e594b77bdbd8ee35191c8b28a94a4cb871d96f32921558275421fb68@buzz.block.builderlab.xyz>
…che-compat B1 (PSUBSCRIBE removal): exact SUBSCRIBE/UNSUBSCRIBE reconciliation with the establishment-gated authz caches. Reviewed CLEAN by Wren at 5419e86 (tree 60a6c43); Eva spot-verified the narrow delta and controls at the same head. Co-authored-by: npub1qyvc0c5kl4gqv2fd97fsk46tu378sqgy35vc83rvgfwne90sel7s0ed67d <011987e296fd5006292d2f930b574be47c7801048d1983c46c425d3c95f0cffd@buzz.block.builderlab.xyz> Signed-off-by: npub1qyvc0c5kl4gqv2fd97fsk46tu378sqgy35vc83rvgfwne90sel7s0ed67d <011987e296fd5006292d2f930b574be47c7801048d1983c46c425d3c95f0cffd@buzz.block.builderlab.xyz>
The residency gate's two halves were each unit-tested in isolation against `insert_established_for_test`, but nothing exercised the composition: a community whose invalidation topic is severed *while a caller is resolving authorization*. That is the silent-stale-authz hole the whole B1 design exists to close — an entry admitted in the reconnect gap stays readable for a full TTL with nothing able to drop it, and reports no error anywhere. The drill composes the production objects `main.rs` wires (`PubSubManager`, its own `cache_invalidation_topics()`, a `CacheResidency` at `AUTHZ_CACHE_TTL`) and walks bootstrap -> open gate -> `CLIENT KILL` sever -> suppression -> resume, asserting real delivery either side of the sever rather than establishment bookkeeping. No production code changes, and no new production seam. It lives in buzz-relay because it must: `CacheResidency` is here, `CommunityTopics` is in buzz-pubsub, and the dependency edge only runs relay -> pubsub. The `clients_with_exact_subs` helper is therefore a test-private twin of the buzz-pubsub one rather than a new `pub` cross-crate surface whose only consumer is one test. Which forces the ci.yml change. The Redis-ignored step selected `package(buzz-pubsub)` alone, so an `#[ignore]`d test added here would pass locally and gate nothing; the selector now names `state::redis_tests` too. Verified under that exact expression via nextest, not a package run: 54/54 pass, and a dead-port control (`REDIS_URL` -> 127.0.0.1:6399) makes the drill FAIL rather than self-skip. Two bounds in the drill are load-bearing and were both found by mutation, not by reading: * The sever's gate-close wait is 500ms, far below the 1s reconnect backoff. `connect_and_serve` also clears establishment on connect, so a generous bound is satisfied by that later clear — verified: with a 15s bound the test still passes with the reconnect-gap `clear_established()` deleted. Only the tight bound can be met by the clear that protects the gap. * Resume asserts the still-resident community only. The suppressed admission re-stamps its own residency; the other six age out at `AUTHZ_CACHE_TTL` and are correctly never re-subscribed. Asserting all seven claims a property the level-triggered design deliberately lacks and turns the test into a race against a 10s TTL. Mutation results: reconnect-gap clear removed -> fails at the sever assert in 0.53s; residency gate bypassed -> fails at bootstrap; residency recorded only when already established (the closed-loop deadlock) -> fails to converge. Stale-snapshot withdrawal does not bite this drill and should not: three existing buzz-pubsub tests already cover that invariant. The two `buzz-relay --lib` failures on this branch (`telemetry::tests::trace_context_lookup_does_not_enable_callsites`, `api::mesh_demo::tests::demo_join_forwarded_arm_round_trips_echo`) are pre-existing — reproduced identically at a clean 784a5f3 with this diff stashed, 834 passed / 2 failed both ways. Co-authored-by: npub17jjz49l9jjmhhk7cac63j8yt9z555n9cw8vk7v5jz4vzw4ppld5qgj57cc <f4a42a97e594b77bdbd8ee35191c8b28a94a4cb871d96f32921558275421fb68@buzz.block.builderlab.xyz> Signed-off-by: npub17jjz49l9jjmhhk7cac63j8yt9z555n9cw8vk7v5jz4vzw4ppld5qgj57cc <f4a42a97e594b77bdbd8ee35191c8b28a94a4cb871d96f32921558275421fb68@buzz.block.builderlab.xyz>
tlongwell-block
marked this pull request as ready for review
August 1, 2026 15:00
added 3 commits
August 1, 2026 12:38
Brings the branch up to date with main at 756dd7f. One conflict in .github/workflows/ci.yml: both sides appended a step after the invite security tests — kept both (tunnel fence Redis integration tests from this branch, workspace profile kind:9033 gate tests from main). Co-authored-by: npub1qyvc0c5kl4gqv2fd97fsk46tu378sqgy35vc83rvgfwne90sel7s0ed67d <011987e296fd5006292d2f930b574be47c7801048d1983c46c425d3c95f0cffd@buzz.block.builderlab.xyz> Signed-off-by: npub1qyvc0c5kl4gqv2fd97fsk46tu378sqgy35vc83rvgfwne90sel7s0ed67d <011987e296fd5006292d2f930b574be47c7801048d1983c46c425d3c95f0cffd@buzz.block.builderlab.xyz>
Security update: nostr-relay-pool 0.44.1 -> 0.44.2 (RUSTSEC-2026-0224, verification-cache poisoning allowing forged events to bypass signature validation on redelivery). Cargo.lock-only change, no conflicts. Co-authored-by: npub1qyvc0c5kl4gqv2fd97fsk46tu378sqgy35vc83rvgfwne90sel7s0ed67d <011987e296fd5006292d2f930b574be47c7801048d1983c46c425d3c95f0cffd@buzz.block.builderlab.xyz> Signed-off-by: npub1qyvc0c5kl4gqv2fd97fsk46tu378sqgy35vc83rvgfwne90sel7s0ed67d <011987e296fd5006292d2f930b574be47c7801048d1983c46c425d3c95f0cffd@buzz.block.builderlab.xyz>
Brings in #4124: relay-membership checks served from the read replica via bounded routed read. No conflicts. Co-authored-by: npub1qyvc0c5kl4gqv2fd97fsk46tu378sqgy35vc83rvgfwne90sel7s0ed67d <011987e296fd5006292d2f930b574be47c7801048d1983c46c425d3c95f0cffd@buzz.block.builderlab.xyz> Signed-off-by: npub1qyvc0c5kl4gqv2fd97fsk46tu378sqgy35vc83rvgfwne90sel7s0ed67d <011987e296fd5006292d2f930b574be47c7801048d1983c46c425d3c95f0cffd@buzz.block.builderlab.xyz>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Makes Buzz compatible with ElastiCache Serverless while retaining complete compatibility with standard Redis — one design, no serverless mode, no cluster-client rewrite.
Spec of record:
PLANS/BUZZ_SERVERLESS_ELASTICACHE_COMPATIBILITY_SPEC.md(negotiated in buzz-redis-cluster-mode).The three serverless blockers, closed
1.
PSUBSCRIBEremoval (B1)Serverless rejects pattern subscribe. Both
buzz:*:cache-invalidateandbuzz:*:conn-controlpatterns are replaced by exact per-communitySUBSCRIBE/UNSUBSCRIBEthrough one new level-triggered reconciler (buzz-pubsub/src/community_topics.rs): a desired community set is diffed against the Redis-acked set on connect / wake / 1s tick. NoSSUBSCRIBE, so standard Redis is unaffected.Safety model (per spec):
sink.subscribe().await == Okis the acknowledgement — a community is established only after Redis acks. Cache inserts are gated on establishment viaCacheResidency::admit_and_insert, the only populate path for the three covered authz caches: residency recorded → insert inside the establishment read lock (withdrawal's write lock excluded) → residency deadline re-stamped after insert. An admitted entry structurally cannot outlive the subscription protecting it. Disconnect clears established (desired retained; reconnect re-acks); broadcastLagged(n)does not touch established — counted, bounded by the centralized ≤10s TTL, which is a security-reviewed correctness constant, not a tuning knob. No cache flush ever (stampede-by-design rejected on the fan-out path).2. Cross-slot tunnel lease fencing (B2)
The four two-key fencing Lua scripts would
CROSSSLOTunder slot enforcement. First-position{session_id}hash tags co-slot them; on standalone Redis these are ordinary keys. Migration is gated by strictBUZZ_TUNNEL_FENCE_KEY_FORMAT(unset ⇒legacy; invalid ⇒ hard config error), with generation-monotonic legacy→tagged cutover and an executable watermark-transfer runbook (docs/tunnel-fence-key-migration.md) that aborts at scale-zero on any mismatch.3. Presence
MGET(B3)Cross-slot
MGETreplaced by a non-atomic pipeline of single-keyGETs (same single round trip;MULTI/EXECwould itselfCROSSSLOT). Length mismatch errors loudly (PresenceReplyMismatch) instead of zip-truncating into misattributed presence; downstream degrades to empty presence, never a wrong map.Verification (all at exact head
784a5f3a5, live-local per TESTING.md)buzz-pubsub --include-ignored: 53/53 standalone and 53/53 one-node cluster mode owning all 16,384 slots (rig verified rejectingMGET alpha betawithCROSSSLOT) — includes 100-channel byte-exact routing, subset unsub/re-sub, realCLIENT KILLsever + full desired-set rebuild, tick-only convergence, presence missing/duplicate/permutation maps.ci.ymlselects both suites explicitly).buzz-relay --lib: 834/2/41; both failures proven pre-existing by base-SHA control at911cfb22a, not by flake folklore.PSUBSCRIBE, RedisMGET, or.atomic()undercrates/.Every lane was independently reviewed at its exact pushed SHA (Wren, reviewer of record) and re-verified at the merged head.
Verification amendment (A1)
The one gap Max's integrated audit found is closed at
d93bde42e: a composed Redis-backed drill (state::redis_tests::a_severed_invalidation_topic_suppresses_caching_until_it_is_rebuilt) walks bootstrap → acked gate + real delivery → targetedCLIENT KILL IDsever → establishment closed within 500ms (below the 1s reconnect backoff, so only the gap-protecting clear can satisfy it) → suppressed admission → autonomous rebuild from residency alone → resumed admission + real post-rebuild delivery. Test-only + one CI selector line so the gate actually selects it; mutation-verified (3 mutations bite by name), 54/54 under the exact CI selector on standalone and cluster rigs, dead-port control fails by name rather than skipping.Deliberately not in this PR (minimalness): a test seam for the
Lagged(n)arm — the arm (main.rs) is two statements touching neither establishment nor residency, so "established unchanged" is structural; adding production plumbing solely to test it fails the minimalness bar. A desired-vs-established divergence gauge is deferred to the rollout/staging follow-up alongside ECPU metering,EVALbilling, andSCANchecks.