Repository navigation
[fix] Report a rewritten outbound port as not reachable - #22
Merged
Merged
Conversation
A relay whose outbound UDP port is rewritten reported reachable while hyperdht stopped offering clients a direct connection. Reachability now follows dht.remoteAddress(): /readyz returns 200 only for "reachable", with new states port-unstable and unknown, a relay_reachability_state gauge, page guidance and transition log lines.
On a public-IP VM the state tracked real client outcomes, but a rewrite shows intermittently: replies on inbound flows keep the real port, so a running relay can settle on it and connect directly again, and a new rewrite can take ~30 min to show. Docker -p stayed reachable for 30 min, so it moves from first to last among the causes.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Why
A relay whose outbound UDP port is rewritten on the way out (Docker
-ppublishing, a NAT or tunnel in front of the host, stale conntrack state) reportedreachableeverywhere while no client could connect. hyperdht only offers clients a direct connection whendht.remoteAddress()is non-null; that is null for an unknown host, a randomized port, or a consistent public port that differs from the bound one. The last case never setsrandomized, so even the status page's symmetric-NAT note missed it.What
networkInfo().publicAddress=dht.remoteAddress();reachabilityState()becomesreachableonly with it. New stateport-unstable(host known, no stable address);unknownnow also covers an unsettled address./readyz: 200 only forstate: "reachable"; body gainsstateanddirectlyReachable. Behaviour change: 503 for port-unstable/unknown. Use it for readiness, never liveness; the DockerHEALTHCHECKuses/healthz.relay_reachability_state{state}one-hot gauge, set on scrape.src/reachability-watch.js: transition log lines (warnfor port-unstable and its recovery,infofor unknown, onewarnafter 10 min unsettled).docker-compose.ymlcomment, docs guard test,.claude/lessons.md.Tests
Integration tests drive the real dht-rpc NAT sampler (
dht._natAdd) into each state rather than stubbingnetworkInfo. The six new/changed/readyztests fail againstmainand pass here.npm run lintclean;npm test432/432.E2E on a public-IP VM (2026-09-21)
Hetzner cx23, public IPv4 on
eth0, image built from851a118, probes (scripts/probe.js) from a different public IP. Faults injected with a separate nft table ahead of Docker's srcnat./readyz/ pagereachable,91.107.206.23:49737, probedmasquerade randomon relay-initiated flows, relay already runningreachablefor 22 min (sampler re-samples known nodes every 30 min)port-unstablewithin seconds; pill, gauge,warnlog line all correctreachable(inbound samples outweigh rewritten ones)snatto one fixed port 31562, fresh restartport-unstablefor ~20 sreachable-p 49737:49737/udppublishing, 30 minreachablethroughout, no transitionsThe state matched the client outcome in every case, including the windows where the old code said
reachablewhile probes failed. Findings folded into the docs (second commit): a rewrite of relay-initiated flows shows intermittently and can take ~30 min to appear on a running relay; Docker-pmoves from first to last among the causes. Not reproduced: the sampler settling on the wrong consistent port (covered by the integration test only).