Skip to content

Rebuild the client endpoint after repeated reconnect failures - #90

Merged
andrewchen5678 merged 3 commits into
mainfrom
reconnect-endpoint-rebuild
Sep 1, 2026
Merged

andrewchen5678 merged 3 commits into
mainfrom
reconnect-endpoint-rebuild

Conversation

@andrewchen5678

Copy link
Copy Markdown
Contributor

Summary

  • Escalate the client reconnect loop: every third consecutive failed attempt rebuilds the iroh endpoint from scratch (ClientEndpoint::rebuild) — fresh sockets, fresh relay connections, fresh discovery — the in-process equivalent of the manual restart that recovered a wedged client (network_change() alone couldn't repair a relay link lost to a ping timeout). Other retries keep the network_change() nudge.
  • New ClientEndpoint handle (shared by CLI, desktop, and FFI) wraps the endpoint with a rebuild factory; the rebuild skips the startup per-relay probe and tolerates the online wait failing so a partial outage never blocks recovery. The old endpoint is closed gracefully on a background task (never dropped unclosed — fatal under panic=abort).
  • Close the just-bound endpoint before propagating a failed startup online wait, on both the client and server creation paths (same panic=abort hazard).
  • Docs: architecture reconnect policy, systemd notes, bridge module rationale (bridges never rebuild — they dial on the server's own endpoint).

Testing

  • New e2e test reconnect_rebuilds_a_dead_endpoint (kills the client's endpoint, asserts reconnection lands on a freshly built one).
  • ci/all.sh green on all three host platforms (mac, linux, windows) for the feature commit; Linux clippy -D warnings + 194 core/CLI tests green on the follow-up fixes.

🤖 Generated with Claude Code

https://claude.ai/code/session_01UA6a2mvmPMsnymfox6nK7K

Andrew Chen and others added 3 commits August 31, 2026 21:27
A wedged iroh endpoint can be broken beyond what the network_change()
nudge repairs: a relay link lost to a ping timeout that never
re-establishes, stale cached paths for the server. Reconnect attempts
then time out forever while a process restart connects instantly (seen
on WSL2 with custom relays and a LAN server: heartbeat detected the
drop in 33s, but every reconnect timed out at 30s until a manual
restart).

Introduce ClientEndpoint, a shared handle wrapping the live endpoint
plus a rebuild recipe. Every third consecutive failed reconnect attempt
the loop swaps in a freshly bound endpoint (new sockets, relay
connections, discovery) and closes the old one in the background — the
in-process equivalent of a restart, keeping the proxy listeners bound
and held requests alive. Rebuilds skip the startup all-relays probe (a
partial relay outage must not block recovery) and tolerate the online
wait failing (mDNS can still reach a LAN server). Quick clients keep
their fixed identity; keypair clients get a fresh ephemeral node id
(the CLI status panel now tracks it).

CLI, desktop, and FFI embedders all adopt the handle. Bridges
deliberately do not rebuild: they dial on the server's own endpoint,
and rebuilding it would sever every connected client.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01UA6a2mvmPMsnymfox6nK7K
Review fixes on the endpoint-rebuild change: in ClientEndpoint creation a
wait_online failure returned via `?` and dropped the just-bound endpoint
without close(), which is fatal under the release profile's panic=abort.
Also correct the architecture doc: network_change() applies only to
retries that don't trigger a rebuild, not to every retry.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01UA6a2mvmPMsnymfox6nK7K
Same pre-existing hazard as the client fix in the previous commit:
create_server_endpoint propagated a wait_online failure via `?` and
dropped the bound endpoint unclosed, fatal under panic=abort.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01UA6a2mvmPMsnymfox6nK7K
@andrewchen5678
andrewchen5678 merged commit 0612c67 into main Sep 1, 2026
4 checks passed
@andrewchen5678
andrewchen5678 deleted the reconnect-endpoint-rebuild branch September 1, 2026 04:39
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant