Choose relays by who they reach, not by how popular they are - #206
Merged
Conversation
Ranking by popularity answers "which relay should this reader consider adding". It does not answer "which four relays reach the most people", and the top four of a popularity list is not that: they all carry the same crowd, and somebody publishing to one quiet relay is never reached however many popular ones are dialled. Measured on a real account: 215 of 257 follows have a relay list, naming more than 128 distinct relays between them. That is a long tail of relays with one or two writers each, and no top-N of it covers everybody. So the two questions get two algorithms. Suggestions stay ranked by how many follows write there, which is the right answer for a human reading a list. Connections are chosen by marginal coverage: repeatedly take the relay reaching the most people not yet reached twice, and stop when the next one would reach nobody new. Two is the target because one is enough to see somebody and two is what keeps a single relay going down from hiding them. Jumble uses the same number from the other end, pruning a relay only when every pubkey on it is covered twice elsewhere. Amethyst wrote this exact algorithm and never called it. The reader's own relays are pre-seeded as already-paid-for coverage and are never dialled again. Without that the greedy spends its whole budget re-reaching the crowd that is already on nos.lol. Live, 257 follows: reached 193, of those 134 twice, 64 residual riding the pool. Chosen were nostr.wine, snort, nostr.land and premium.primal, which is not the top of the popularity list. `relay_rank_candidates` was 128 and it was a guess. On the same account it was hit EXACTLY and dropped 63 further relay references without a word, which is the silent cap this codebase has a rule against and I wrote anyway. Now 384, and `RouteCoverage.candidates_dropped` says so out loud when even that is not enough. Raising it changed nothing about coverage on this account, 193/134/64 before and after, because the dropped relays were tail entries that would not have been chosen. Saying that rather than implying the fix bought reach: it bought an honest number. A bug worth recording. `g_route_bits` is indexed by table position, and the suggestions sorted the table in place, so every bitset silently re-pointed at a different relay: the routing read one relay's authors and dialled another. The symptom was choosing a relay with two writers over one with four. An ORDER is sorted now, never the table. Second of four. Still to come: per-slot generations so one relay changing does not redial the rest, in-place REQ replacement, hysteresis and a settle window, then the routed budget.
Merged
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Ranking by popularity answers which relay should this reader consider adding. It does not answer which four relays reach the most people, and the top four of a popularity list is not that: they all carry the same crowd, and somebody publishing to one quiet relay is never reached however many popular ones are dialled.
Measured on a real account: 215 of 257 follows have a relay list, naming more than 128 distinct relays between them. A long tail of relays with one or two writers each, which no top-N can cover.
So the two questions get two algorithms. Suggestions stay ranked by how many follows write there, which is right for a human reading a list. Connections are chosen by marginal coverage: repeatedly take the relay reaching the most people not yet reached twice, and stop when the next would reach nobody new.
Two is the target because one is enough to see somebody and two keeps a single relay going down from hiding them. Jumble uses the same number from the other end (prune only when every pubkey is covered twice elsewhere). Amethyst wrote this exact algorithm and never called it.
The reader's own relays are pre-seeded as already-paid-for coverage and never dialled again. Without that the greedy spends its whole budget re-reaching the crowd already on nos.lol.
Live, 257 follows
Reached 193, of those 134 twice, 64 residual riding the pool. Chosen: nostr.wine, snort.social, nostr.land, premium.primal — which is not the top of the popularity list.
The silent cap, found and fixed
relay_rank_candidateswas 128 and it was a guess. On the same account it was hit exactly, dropping 63 further relay references without a word. That is the silent-cap rule this codebase has, and I wrote the violation myself in #196.Now 384, with
RouteCoverage.candidates_droppedsaying so out loud when even that is not enough.Raising it changed nothing about coverage — 193/134/64 before and after — because the dropped relays were tail entries that would not have been chosen. Saying that rather than implying the fix bought reach: it bought an honest number.
A bug worth recording
g_route_bitsis indexed by table position, and the suggestions sorted the table in place. Every bitset silently re-pointed at a different relay, so the routing read one relay's authors and dialled another. The symptom was choosing a relay with two writers over one with four. An order is sorted now, never the table.Tests
Five, each verified by breaking it:
The pre-seed probe needed a second attempt: the first version's assertions did not depend on it, so removing it failed nothing.
Second of four
Still to come: per-slot generations so one relay changing does not redial the rest, in-place REQ replacement, hysteresis and a settle window, then the routed budget.