Ask each relay about the people who write there, not about everyone - #205
Merged
Conversation
Plaza routed reads for the four relays it discovered and asked its own eight about every followed author anyway. That is a hybrid: the routing bought reach and none of it bought the pool a smaller question. Jumble and Amethyst both route the whole feed. This is that. Each pool relay is now asked about the follows who write THERE, plus the residual, and every routed relay keeps asking only about its own. The residual is the whole risk and defining it is the entire change. It is NOT "authors with no relay list". It is every author no chosen relay covers, list or no list, because somebody who publishes only to a relay that did not make the cut has a list and would otherwise be asked of nobody at all. They would vanish from the feed with no error and no empty state, which is precisely the failure the outbox model exists to fix and the easiest one to reintroduce while fixing it. So the invariant, with a test named after it: every followed author appears in at least one relay's filters. Live, on my own account, 257 follows: nos.lol 164 of 257 was 257 relay.damus.io 162 was 257 relay.primal.net 122 was 257 eden.nostr.land 94 was 257 with 63 residual carried by all of them. Each relay is told less about who this reader follows, and no follow is dropped. The fallback matters as much as the routing: an empty route table means ask about everyone. The first dial happens before any relay list has been read back, and asking about nobody then opens the app to a blank feed that fills only when somebody's kind:10002 arrives. Two things about how this was tested, both worth more than the change. My probe helper could not tell a BUILD FAILURE from a passing test. It grepped for test-failure lines, and a compile error matches none of them, so three probes in a row reported "not caught" when they had never run. One of them was hiding a duplicate declaration that meant the test suite had not compiled for several steps. The helper now checks the build first and says "probe invalid" instead of implying a pass. And the first version of the coverage test could not fail. It used two relays against four routed slots, so everything was routed and the residual was never exercised; deleting the residual entirely failed nothing. It uses ten authors across eight relays now, more than the budget holds, and asserts the residual is non-empty before asserting coverage. Not covered by a test: that `ingestOnce` calls the routed path at all, because it needs a socket. That is what the live run above is for. This is the first of four. Still to come: greedy set cover with a coverage target of two, per-slot generations so one relay changing does not redial the rest, in-place REQ replacement, hysteresis, and then the routed budget.
Merged
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Plaza routed reads for the four relays it discovered, and asked its own eight about every followed author anyway. That is a hybrid: the routing bought reach, and none of it bought the pool a smaller question. Jumble and Amethyst both route the whole feed. This is that.
Each pool relay is now asked about the follows who write there, plus the residual. Routed relays are unchanged: only their own.
The residual is the whole risk
It is not "authors with no relay list". It is every author no chosen relay covers, list or no list.
Somebody who publishes only to a relay too unpopular to be chosen has a list, and would otherwise be asked of nobody at all. They vanish from the feed with no error and no empty state, which is exactly the failure the outbox model exists to fix and the easiest one to reintroduce while fixing it.
So: every followed author appears in at least one relay's filters, with a test named after it.
Live, 257 follows
63 residual authors carried by all of them. Each relay is told less about who this reader follows, and no follow is dropped.
The fallback matters as much as the routing: an empty route table means ask about everyone. The first dial happens before any relay list has been read back, and asking about nobody then opens the app to a blank feed that only fills when somebody's kind:10002 arrives.
Two things about the testing, both worth more than the change
My probe helper could not tell a build failure from a passing test. It grepped for test-failure lines, and a compile error matches none of them, so three probes in a row reported "not caught" when they had never run. One was hiding a duplicate declaration that meant the suite had not compiled for several steps. It checks the build first now and says "probe invalid" rather than implying a pass.
The first coverage test could not fail. Two relays against four routed slots meant everything got routed and the residual was never exercised; deleting the residual entirely failed nothing. It now uses ten authors across eight relays, more than the budget holds, and asserts the residual is non-empty before asserting coverage.
Not covered by a test: that
ingestOncecalls the routed path at all, since it needs a socket. That is what the live run is for, and it is stated rather than implied.Also fixed on the way
fillPoolRoutesinitially calledrelaySnapshotwhile its caller held the relay-table lock — the same non-reentrant spinlock deadlock that hung the suite earlier today, in this same call chain. The pool URLs are passed in already snapshotted, and there is now a small audit that lists every function holding a lock and calling a known lock-taker.First of four
Still to come: greedy set cover with a coverage target of two, per-slot generations so one relay changing does not redial the rest, in-place REQ replacement, hysteresis and a settle window, then the routed budget 4 to 8.