Skip to content

[fix] Report a rewritten outbound port as not reachable - #22

Merged
ok merged 2 commits into
mainfrom
ok/relay-port-stability
Sep 22, 2026
Merged

ok merged 2 commits into
mainfrom
ok/relay-port-stability

Conversation

@ok

@ok ok commented Sep 21, 2026 •

Copy link
Copy Markdown
Owner

Why

A relay whose outbound UDP port is rewritten on the way out (Docker -p publishing, a NAT or tunnel in front of the host, stale conntrack state) reported reachable everywhere while no client could connect. hyperdht only offers clients a direct connection when dht.remoteAddress() is non-null; that is null for an unknown host, a randomized port, or a consistent public port that differs from the bound one. The last case never sets randomized, so even the status page's symmetric-NAT note missed it.

What

  • networkInfo().publicAddress = dht.remoteAddress(); reachabilityState() becomes reachable only with it. New state port-unstable (host known, no stable address); unknown now also covers an unsettled address.
  • /readyz: 200 only for state: "reachable"; body gains state and directlyReachable. Behaviour change: 503 for port-unstable/unknown. Use it for readiness, never liveness; the Docker HEALTHCHECK uses /healthz.
  • relay_reachability_state{state} one-hot gauge, set on scrape.
  • src/reachability-watch.js: transition log lines (warn for port-unstable and its recovery, info for unknown, one warn after 10 min unsettled).
  • Status and members pages: Port unstable pill, sentence, a note naming which rewrite it is, and remediation steps.
  • README, OPERATIONS.md, Prometheus alert example, docker-compose.yml comment, docs guard test, .claude/lessons.md.

Tests

Integration tests drive the real dht-rpc NAT sampler (dht._natAdd) into each state rather than stubbing networkInfo. The six new/changed /readyz tests fail against main and pass here. npm run lint clean; npm test 432/432.

E2E on a public-IP VM (2026-09-21)

Hetzner cx23, public IPv4 on eth0, image built from 851a118, probes (scripts/probe.js) from a different public IP. Faults injected with a separate nft table ahead of Docker's srcnat.

Scenario /readyz / page Probe from outside
Host networking, no fault 200 reachable, 91.107.206.23:49737, probed bridged
Fault A: masquerade random on relay-initiated flows, relay already running stayed reachable for 22 min (sampler re-samples known nodes every 30 min) —
Fault A, fresh restart 503 port-unstable within seconds; pill, gauge, warn log line all correct HOLEPUNCH_ABORTED
Fault A, ~18 s later back to 200 reachable (inbound samples outweigh rewritten ones) bridged 3/3
Fault B: snat to one fixed port 31562, fresh restart 503 port-unstable for ~20 s HOLEPUNCH_ABORTED 2/2
Fault B, settled 200 reachable bridged 2/2
-p 49737:49737/udp publishing, 30 min 200 reachable throughout, no transitions bridged

The state matched the client outcome in every case, including the windows where the old code said reachable while probes failed. Findings folded into the docs (second commit): a rewrite of relay-initiated flows shows intermittently and can take ~30 min to appear on a running relay; Docker -p moves from first to last among the causes. Not reproduced: the sampler settling on the wrong consistent port (covered by the integration test only).

ok added 2 commits September 21, 2026 20:28
A relay whose outbound UDP port is rewritten reported reachable while
hyperdht stopped offering clients a direct connection. Reachability now
follows dht.remoteAddress(): /readyz returns 200 only for "reachable",
with new states port-unstable and unknown, a relay_reachability_state
gauge, page guidance and transition log lines.
On a public-IP VM the state tracked real client outcomes, but a
rewrite shows intermittently: replies on inbound flows keep the real
port, so a running relay can settle on it and connect directly again,
and a new rewrite can take ~30 min to show. Docker -p stayed reachable
for 30 min, so it moves from first to last among the causes.
@ok
ok merged commit f49f88b into main Sep 22, 2026
4 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant