Skip to content

(remote): bind watch-channel handlers to their child, add ssh keepalive and failure logs - #258

Merged
devsuitup merged 1 commit into
mainfrom
fix/watch-restart-leak
Sep 11, 2026
Merged

devsuitup merged 1 commit into
mainfrom
fix/watch-restart-leak

Conversation

@devsuitup

@devsuitup devsuitup commented Sep 11, 2026

Copy link
Copy Markdown
Owner

Audit 2026-09-11, findings 1, 4 and 11 (.work-files/switchboard/audit-fable-2026-09-11.md, local note).

Why

stop() followed by start() on the same host raced the previous child's asynchronous close: its handler was bound to the per-host state, so when it fired it dropped the new child from the state and scheduled a third one. Result: an unkillable ssh -tt … inotifywait per host on every manual refresh, per-host reconnect, or host toggle in Settings (the toggle path ships in v0.0.75). Reproduced with a fake child whose close arrives one tick late: 2 children alive after stop+start, 3 spawned after the restart timer, 1 survivor after stopAll().

What

  • Every child handler (data, stderr, close) checks s.child === child before touching the state; a superseded child's late close or chunk is ignored.
  • The watch ssh gets ConnectTimeout=10, ServerAliveInterval=30, ServerAliveCountMax=3, so a half-open session dies and gets restarted instead of staying "running" forever.
  • A quick failure logs the exit code and the last 200 bytes of stderr once per backoff tier (same throttle as the inventory backoff) — a dead watch channel is no longer silent in main.log.
  • stop() clears pending coalesce timers so onEvent cannot fire after stop.
  • Test helper now emits close asynchronously, like a real ChildProcess.

Proof

Mutations: remove the identity guard in onExit → the F1 race test and the F4 duplicate-close test go red; drop the stderr capture → the warn-content test red. task check: 1256 pass, 0 fail, 8 pre-existing skips.

…ve and failure logs

stop() then start() raced the previous child's asynchronous close: its handler,
bound to the per-host state, dropped the new child and scheduled a third, leaving
an unkillable ssh per host on every manual refresh or host toggle. Handlers now
check the child's identity. The watch ssh gets ConnectTimeout and ServerAlive so a
half-open session dies instead of staying "running", quick failures log their
stderr tail once per backoff tier, and stop() clears pending coalesce timers.
@devsuitup
devsuitup merged commit 3133410 into main Sep 11, 2026
10 checks passed
@devsuitup
devsuitup deleted the fix/watch-restart-leak branch September 11, 2026 06:20
@devsuitup devsuitup mentioned this pull request Sep 11, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant