fix: FCM reconnect storm hardening + phantom doorbell ring on restart (v0.16.4-beta.2) - #14
Merged
Merged
Conversation
…4-beta.2) A poisoned StreamReader in firebase_messaging's _listen loop re-raises the same exception object every iteration, growing its traceback while logging.exception formats it inside the HA event loop - quadratic on Python 3.14 and fatal (Supervisor watchdog kill -> crash loop). - Rate-limit filter on the firebase_messaging.fcmpushclient logger: strips exc_info after 3 tracebacks per 5-minute window, keeping one-line messages. Filters run before formatting, so this defuses the CPU bomb even while the upstream loop spins. - Bounded abort (abort_on_sequential_error_count=3) instead of unbounded retries; the watchdog restarts the client with a delayed doubling backoff (5 -> 15 min cap) that resets when healthy. Closes #12
Event entities (doorbell ring, door opened, camera on) inherited availability from the device connection_state. A transient intercom disconnect - or the brief window during an HA restart - flapped them to unavailable and back; on recovery the EventEntity restores its last event (e.g. ring) and HA fires state triggers, producing a phantom doorbell ring with no FCM message involved (confirmed in logs: no 'Doorbell notification' line, grace-period and dedup never engaged). Events are momentary historical markers, so their availability is now constant True instead of tracking connectivity. Real events still fire via _trigger_event.
bvis
changed the title
anthonws
added a commit
to anthonws/fermax-blue-hass
that referenced
this pull request
Jul 4, 2026
… (v0.13.0) Addresses 46 findings from a multi-agent performance/reliability/security audit. Highlights below; each change is annotated with its audit id in-code. ## Performance — stream startup latency (the headline) - Video now goes live independently of the ~10s Fermax "pickup": the video relay + frame grabber + is_active are started right after the video consumer, and the audio relay is pre-created from a silent switchable source so its m-line is negotiated up front. When pickup completes, the real intercom audio is swapped into that relay — no WebRTC renegotiation. Cuts perceived first-frame latency from ~13s to ~3-4s (bvis#2, bvis#6). - Frame grabber: decode+JPEG-encode moved off the event loop via to_thread, encoded once (overlay burned in) instead of twice, and throttled to ~5fps when not recording since the WebRTC path consumes raw relay frames (bvis#3,bvis#7,bvis#8). - WebRTC offer handler polls the relay at 0.1s (was 0.5s) and caps ICE gathering at 4s (was 10s) (bvis#18, bvis#19). ## Reliability - CRITICAL: re-register a rotated FCM token with Fermax. Previously a token rotation silently killed the doorbell until a full reload (bvis#1). - Notification de-duplication now persists persistent_ids across restarts, so FCM re-deliveries are recognised by id; the blunt 10s time-grace is used only on first run — a genuine ring right after restart is no longer dropped (bvis#9,bvis#24). - Serialise stream start/stop with a lock + session-identity guards so overlapping notifications can't orphan a half-started session; the grabber's end-callback no longer clobbers a newer session; stop() is idempotent and the session is fully torn down on natural end (bvis#11, bvis#26, bvis#27, bvis#45). - FCM decrypt: escalate to ERROR after 5 consecutive failures (systemic outage) instead of masking it as healthy (bvis#10). - Socket.IO connect() is bounded by a 15s timeout; startup HTTP calls carry an explicit per-request timeout (bvis#13, bvis#30). - Partial-setup teardown: a later pairing failing no longer leaks earlier coordinators' FCM listeners / API client (bvis#14). - Auto-stop timer defers while a card viewer is connected (G3); call_mode and stream_duration persist across restarts via RestoreEntity (G2). - Camera closes in-flight PCs on unload; lock cancels its auto-lock timer on removal; send_audio resolves coordinators dynamically and is unregistered on the last unload; shared API client closed once (bvis#12, G5, G4, G7). - Misc: recv() raises MediaStreamError not StopIteration; get_running_loop; access-token / credentials-save / callDate parse hardened (bvis#29,bvis#28,bvis#31,bvis#33,bvis#39). ## Security / hardening - TLS enforcement uses a urlparse allowlist and rejects whitespace, closing the " http://…" leading-space bypass; signaling URL must be https/wss on *.fermax.io (bvis#16, bvis#37). - Diagnostics redacts all Firebase identifiers + OAuth tokens; log redaction recurses into lists; join_call errors no longer logged verbatim (bvis#35,bvis#34,bvis#36). - send_audio and auto-response file paths validated before av.open (bvis#38); recording temp files already use mkstemp; ffmpeg shell=False. - Rate-limit log filter is now thread-safe (bvis#44). ## Custom Lovelace card (v1.1.1) - Mic is never acquired during idle polling (no OS mic-indicator flicker); talkback is enabled on demand via a mic-inclusive sendrecv reconnect, which is how a single-offer server negotiates outbound audio (bvis#5). - Epoch guard prevents PC/subscription leaks when the card is removed mid- connect; synchronous re-entrancy lock stops duplicate connects; explicit hangup no longer auto-reconnects; aspect_ratio is sanitised (bvis#4,bvis#17,bvis#40,bvis#41). ## Dependencies - Declare av>=11 and numpy>=1.20 (used on core streaming paths; permissive bounds satisfied by HA, no re-conflict with the av 17 situation) (G1). Deferred (negligible/intentional): per-frame relay list copy (bvis#22), in-memory recording buffer bounded by auto-stop (bvis#23), FCM restart backoff by design (bvis#25), minor stop() sleeps (bvis#42), reload token ordering (bvis#43). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Two independent reliability fixes bundled into the
0.16.4-beta.2release.1. Phantom doorbell ring on HA restart
After an HA restart, the doorbell automation fired "someone is at the door" with no FCM message involved (confirmed in logs: no
Doorbell notificationline, the coordinator grace-period and dedup never engaged).Root cause: the event entities (doorbell ring, door opened, camera on) inherited
availablefrom the deviceconnection_state. A transient intercom disconnect — or the brief window during an HA restart — flapped them tounavailableand back. On recovery theEventEntityrestores its last event (ring) and HA fires state triggers, so automations watching the doorbell ran. Verified against the live instance:event.videoportero_badalonawentring → unavailable (22:18:02) → ring (22:23:03), and the automation ran at 22:23:03 on that recovery edge.Fix: event availability is decoupled from connectivity (events are momentary historical markers, so
availableis constantTrue). Real events still fire via_trigger_event.2. FCM reconnect storm hardening (closes #12)
logging.Filteron thefirebase_messaging.fcmpushclientlogger stripsexc_infoafter 3 tracebacks per 5-minute window. Filters run before formatting, so the quadratic Python 3.14 traceback formatting never runs on the ever-growing exception chain even while the upstream loop spins hot.abort_on_sequential_error_countgoes fromNoneback to 3, andensure_running()defers restarts with a doubling backoff (5 → 15 min cap) that resets once the listener is healthy. Converts the crash loop into "push down for a few minutes".Tests
TDD throughout. 12 new tests (rate-limit filter, backoff escalation/reset, bounded-abort config, event availability decoupling).
make pre-pushgreen: lint, format, mypy, vulture, 150 tests on Python 3.12 + 3.13.Closes #12