Skip to content

Replace the home-relay watchdog with in-place relay failover - #123

Merged
andrewchen5678 merged 5 commits into
mainfrom
relay-failover
Sep 5, 2026
Merged

andrewchen5678 merged 5 commits into
mainfrom
relay-failover

Conversation

@andrewchen5678

Copy link
Copy Markdown
Contributor

Summary

First consumer of flexaccess-iroh 0.0.7 (flexaccessdev/flexaccess-iroh#6), which deletes the home-relay watchdog and its endpoint rebuild and replaces them with relay_failover::fail_over_home_relay. Shared docs: flexaccessdev/iroh-common-architecture#7 (relay-failover.md).

  • After 60 s without a connected home relay, the server removes the wedged relay from the relay map so iroh's forced net report re-homes it on another configured relay in place (no connections dropped). The relay is re-inserted once a probe shows it connectable again (checked every 90 s).
  • Policy change: custom relays (--relay-url / relay_urls) now require at least two distinct URLs. Default relay mode is unchanged.
  • Policy change: the startup probe fails only when every custom relay is unreachable; otherwise it warns per relay and continues, so servers and clients can still start during a single-relay outage.
  • --relay-only requires at least two custom relays.
  • Removed the rebuild deadline, EndpointFactory, REBUILD_* constants and the watch_home_relay_if plumbing from multi_source.rs / endpoint.rs.
  • README, ARCHITECTURE, config examples and AGENTS.md updated; examples use two relays.

Cargo.toml pins the crate by branch = "relay-failover"; switch to tag = "v0.0.7" once the crate PR is merged and tagged.

Test plan

  • cargo clippy clean, cargo test -q (71 tests) pass
  • test-scripts/run_relay_failover_e2e.sh ALL PASS (A0 both relays down → start fails; A1 one relay down → start with warnings, echo via survivor; A2 single relay rejected; B1–B4 relay kill/restart re-homing; C0–C2 wedged relay via fake_relay.py + delay_proxy.py → "Removed … from the relay map", server re-homed on relay2 without restarting, relay1 restored after it comes back)
  • test-scripts/run_e2e.sh with two dev relays passes in relay-only and normal mode

Caveats

  • The failover runs on servers only. A freshly started client whose net report prefers a relay that answers /ping but refuses relay connections still fails at endpoint creation.
  • ezvpn and flextunnel still pin flexaccess-iroh v0.0.3 and adopt later.

🤖 Generated with Claude Code

https://claude.ai/code/session_01HnozJMiCj3v5Xfdb9gB5S8

Andrew Chen and others added 5 commits September 4, 2026 20:31
Adopt flexaccess-iroh 0.0.7 (branch relay-failover), which deletes the
watchdog (a 60 s network_change() nudge that is a no-op on a stable host
and a 180 s endpoint rebuild that dropped every connection) and replaces
it with relay_failover::fail_over_home_relay: after 60 s without a
connected home relay the server removes the wedged relay from the relay
map so iroh's forced net report homes it on another configured relay in
place, and re-inserts the relay once a probe shows it connectable again.

Consequences for tunnel-rs:
- custom relays (--relay-url / relay_urls) now require at least two
  distinct URLs; default relay mode is unchanged
- the startup probe fails only when every custom relay is unreachable,
  and warns per relay otherwise
- the server serve loop runs the failover alongside the accept loop;
  the rebuild deadline, EndpointFactory plumbing and REBUILD_* constants
  are gone
- --relay-only requires at least two custom relays

Tests:
- test-scripts/run_relay_failover_e2e.sh rewritten: two dev relays plus
  fake_relay.py (answers /ping, refuses relay connections) and
  delay_proxy.py (makes relay2 deterministically slower) so net_report
  keeps preferring the broken relay and the failover itself is exercised
  (phases A/B/C, all pass)
- test-scripts/run_e2e.sh takes a whitespace-separated RELAY_URL list
  and runs with two dev relays (relay-dev.toml, relay-dev-2.toml)

Docs point at iroh-common-architecture's relay-failover.md.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HnozJMiCj3v5Xfdb9gB5S8
Picks up de88179 on relay-failover: a second wedged relay is failed over
while the first is still out, the last relay in the map is never removed,
restore reports success only after re-insertion, and an empty custom
relay set no longer panics.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HnozJMiCj3v5Xfdb9gB5S8
The crate's relay-failover branch is merged and released; consume the
release tag instead of the deleted branch.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HnozJMiCj3v5Xfdb9gB5S8
The relay failover suite and its fake-relay and delay-proxy fixtures test
the shared crate, not the tunnel; they now run in flexaccess-iroh against
its own harness. run_e2e.sh keeps everything tunnel-rs adds and drops the
distinct-ephemeral-identity assertion, which is covered there too.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0163AtLdT9AWeDE3EYmhFYXK
@andrewchen5678
andrewchen5678 merged commit a7c55bf into main Sep 5, 2026
@andrewchen5678
andrewchen5678 deleted the relay-failover branch September 5, 2026 14:18
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant