Harden encrypted push payload decoding - #37
MultivitaminJuice wants to merge 1 commit into
Conversation
sdb9696
left a comment
There was a problem hiding this comment.
@MultivitaminJuice thanks for the PR! Could you please provide some more information on what kind of messages were causing these issues (type, device model etc) and include some samples in the test fixtures of actual malformed and unusually-formatted messages?
The crypto-key and salt headers (and webpush keys generally, per RFC 8291)
are transmitted without base64 '=' padding. Decoding them with a bare
urlsafe_b64decode raises binascii.Error on otherwise valid input, which in
downstream Home Assistant Ring use repeatedly crashed the push client:
ERROR Unknown error: Incorrect padding, shutting down FcmPushClient.
File "fcmpushclient.py", line 439, in _handle_data_message
File "fcmpushclient.py", line 378, in _decrypt_raw_data
crypto_key = urlsafe_b64decode(crypto_key_str.encode("ascii"))
binascii.Error: Incorrect padding
Add a small padding helper and use it for all four base64 fields in
_decrypt_raw_data. Additionally, wrap the decrypt call in
_handle_data_message so a single undecryptable message is logged and
skipped (binascii.Error is a ValueError subclass) instead of propagating
and tearing down the listen loop.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
62dcd6f to
94a8f83
Compare
|
Thanks for taking a look — same disclaimer as on #38: I'm coming at this as a downstream Home Assistant / Ring user whose push receiver kept dying, not as a firebase-messaging expert. So please reshape, rename, narrow, or drop anything that doesn't fit the library's direction — I trust your judgement on the right form far more than mine. Rebased onto What I actually saw. On my HA/Ring box the push client was repeatedly killed outright while decrypting an incoming data message: This happened 4 times over 2026-03-04 … 03-06 (14:17:34, then 08:28:22, 04:22:11, 08:02:32). Each time the listener exited and Ring What this PR does — two small things:
A regression test decodes a deliberately unpadded The honest caveats — very much your call:
|
|
Independent corroboration from another downstream integration: Fermax Blue for Home Assistant hits the identical crash. So it isn't Ring-specific — Fermax's push backend also emits the I independently arrived at the same fix (padding the four base64 fields before decode) and confirmed it on a live HA instance — your |
* feat(diagnostics): read-only WebRTC live-video feasibility probe (#322)
Ajax's app pulls remote live video exclusively over WebRTC (the
WebrtcService.initiate signalling stream returns ICE/TURN servers and an
SDP offer from the camera); there is no cloud RTSP/HLS URL to hand Home
Assistant. Before investing in a camera entity for cloud-hosted HA (where
the local ONVIF/RTSP path from #282 is unreachable), we need to know
whether a normal account is authorised to start that session or whether
it is permission-walled server-side like photo-on-demand v3.
Adds DevicesApi.probe_webrtc_initiate: opens the initiate signalling
stream, reads only the first message and closes it without sending any
SDP offer/answer, so no media is negotiated and no session is
established. Returns a PII-free summary (authorized flag, first signalling
message type, ICE server count/schemes) — never credentials, URLs or SDP.
Wired into diagnostics under the existing video_edge probe, keyed per
video_edge_id, best-effort and skipped when there are no video devices.
This is the go/no-go probe before any aiortc bridge work: it settles the
permission-wall question using a reporter's real camera without asking
them to run an emulator/mitmproxy capture.
* fix(hts): stop spurious mains-power flapping during outage (#323)
While the hub runs on battery during a mains outage, Home Assistant flooded
the logbook with Mains power Unplugged/Plugged-in events even though the Ajax
app showed a stable "no power" state.
Root causes and fixes:
- Direct-delta mis-parse (primary): the positionally-paired "direct delta"
network path could surface a stray 0x03 byte (KEY_HUB_POWERED) from a
mis-aligned per-device delta (escape handling can also shift byte boundaries)
and wrongly report mains power restored. The mains-power flag is now only
trusted from the authoritative full STATUS/SETTINGS snapshot, which locates
the hub section by an exact hub-id marker and is re-requested frequently.
- Reconnect reset: a fresh HtsClient is created on every (re)connect with empty
hub state, so the first post-reconnect frame parsed fields from scratch and
any key not repeated fell back to its default (externally_powered -> False =
"Unplugged"). The client now seeds last-known hub state across reconnects.
- Diagnostic: a DEBUG log traces every externally_powered change with the
source frame and raw KEY_HUB_POWERED bytes to confirm the fix in the field.
Adds regression tests for the delta hardening and reconnect seeding.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* fix(hts): refresh on genuine power delta; address #324 review
Addresses @bvis's review on PR #324 (fix for #323):
- Schedule a single-flight hub refresh when a direct-delta power flag differs from the last-known state, so a genuine power change is confirmed within seconds instead of waiting for the periodic STATUS_BODY poll (and never becomes permanently invisible on firmware that omits the power key).
- Drop the now-unreachable KEY_HUB_POWERED entry from _is_network_state_delta.
- Restore test_malformed_payload_drops_message_without_raising (#108) as its own test.
- Assert refresh is/ isn't scheduled for changed/unchanged power deltas, and add coordinator tests that seed_hub_states is wired into the HTS lifecycle.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* docs(readme): FCM event delivery is gated by account role, not the notifications toggle (#319)
* feat(update): add per-device firmware update entities (2.1)
Surface field 200 (device_firmware_updates) from the read-only
streamHubObject snapshot as per-device `update.<device>_firmware`
entities, mirroring the existing hub-level firmware update entity
shipped in 1.4.0-beta.5.
Each non-hub device gets an entity exposing the pending target
firmware version, download progress (during the download phase) and a
security-critical flag; it renders `Up to date` when Ajax has no
update queued for that device. Like the hub entity these are
informational only - no install button and the integration never calls
the install RPC.
Entities are disabled by default because a typical install has 10-30
devices and most users only care about a specific device's firmware
when it is failing to update. Localized name added to all 14
translation files.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* chore: bump version to 1.14.0-beta.2
* refactor(update): address Copilot review on per-device firmware (2.1)
- latest_version: return self.installed_version instead of duplicating
the module-level placeholder constant, keeping the `Up to date`
fallback correct if installed_version logic changes later.
- _maybe_refresh_sim_and_firmware: dedupe hub_ids so a hub backing
multiple spaces (group mode) triggers only one streamHubObject fetch
per cycle instead of one per space, avoiding redundant RPC load.
- Add coordinator test locking in the shared-hub dedup behaviour.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* chore: release 1.14.0
Consolidates the 1.14.0-beta series: read-only WebRTC live-video
feasibility probe in diagnostics (#322) and the mains-power flapping
fixes contributed by @aavdberg (#323). Both field-confirmed on real
hardware: the probe returned a well-formed authorization result on a
cloud-hosted HA install, and a controlled mains unplug/replug on a
live Hub 2 showed exactly one transition per direction (<1 s via the
untrusted-delta refresh, confirmed by the authoritative STATUS_BODY)
with zero flapping on battery.
* feat: persistent notifications for security events (2.2)
Add an option to surface selected Ajax security events as Home Assistant
persistent notifications that remain visible until dismissed. The event set
is configurable and defaults to real incidents (alarm, panic, tamper, fire,
CO, flood, glass break); off by default. Repeats of the same event on the
same device refresh the existing card instead of stacking duplicates.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* fix: address Copilot review on persistent notifications (2.2)
- notification_id no longer collapses all device-less events onto one
'space' card: fall back to group_id, then space_id, before a bare
'space' suffix, so group- and space-scoped events stay distinct.
- Forward the hub entity's space_id in the event payload so the notifier
can disambiguate space-scoped events (no device_id) per space.
- Attach None as the persistent notifier when the feature is off or no
event types are selected, keeping the coordinator's 'is None' fast-path
meaningful and skipping per-event work when disabled.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* fix(update): harden per-device firmware entities after review (2.1)
- Declare UpdateEntityFeature.PROGRESS on both firmware entities: without
the flag HA's UpdateEntity.state_attributes ignores the in_progress /
update_percentage properties entirely (verified against HA 2025.1.4),
so the advertised download progress could never render. Regression
tests now assert through state_attributes, not the raw properties.
- Gate availability on device presence: a device removed from the hub
otherwise keeps its orphan entity reporting 'Up to date' forever.
- Normalize device_id casing (.upper()) on both the coordinator write and
the entity lookup — the update map comes from streamHubObject while
entities key off the devices-snapshot Device.id, and cross-service hex
casing is unverified.
- Render 'failed' and 'completed' update states truthfully: failed says
so in release_summary (the case the user most needs to see); completed
renders as up to date instead of a stale pending update.
- Guard the per-device setup loop against unrecognized hub models
(device_type 'unknown') that would collide with the hub entity's
unique_id.
- Dump hub_firmware_updates / device_firmware_updates in diagnostics
(project rule from #148); include exc_info in the stream-failure debug
log.
* docs(changelog): move per-device firmware entry to 1.15.0 unreleased
1.14.0 shipped on main while this branch was open; the 2.1 feature is
new functionality and lands in the next MINOR per SemVer.
* chore: bump version to 1.15.0-beta.1
* feat: adjustable siren volume and alarm duration (#310)
Expose the StreetSiren/HomeSiren writable settings the Ajax app offers as
Home Assistant config entities:
- select "Siren volume" (very loud / loud / quiet / disabled)
- number "Alarm duration" (seconds)
Read path: values live only in the rich per-device StreamHubDevice snapshot
(common_siren_part.siren_settings), so they are fetched on the same throttled
timer as the per-device temperature and merged into device.statuses (carried
forward across light-stream snapshots).
Write path: DeviceCommand.set_siren_settings -> UpdateHubDeviceService
(Update.siren_settings), reusing the bypass permission/error mapping.
Entities are created only for sirens that report the settings. Adds unit
tests for the parser, the StreamHubDevice fetch, the command dispatch, the
coordinator merge/carry-forward, and both entities.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* style: ruff format test_coordinator.py
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* Address Copilot review on #310: create siren entities at setup by device type
Copilot flagged that the number/select entities were gated on the presence of
the siren settings status keys, which are only merged later by the throttled
per-device snapshot refresh. On first boot the entities could therefore be
missing until a reload.
- number/select: create entities for every device whose device_type is in
SIREN_DEVICE_TYPES instead of on status-key presence. The value reads
unknown until the first snapshot merges it (no reload required).
- const: expand the SIREN_DEVICE_TYPES comment to explain it is exactly the
siren `device` oneof cases the rich HubDevice proto models; other siren SKUs
are not in the proto yet so their settings are unreadable (proto-drift, #229).
- tests: give _make_device a device_type param; setup tests now assert
entities are created for siren device types even with no status key, and not
for non-siren devices.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* fix: harden persistent notifications after review (2.2)
- Translate the new options-flow strings and all 17 event-type selector
options into the remaining 13 locales (selector labels reuse each
locale's existing security_event event-type strings, so terminology
stays consistent with the event entity).
- Add an end-to-end parser->notifier contract test: a real-proto push
(HubEventQualifier + HubNotificationSource) is driven through the
production _parse_and_fire_event path and a real AjaxPersistentNotifier
with only persistent_notification.async_create patched. Guards the
data-key contract (device_name/device_id) the whole feature depends on
and that the hand-written-dict unit tests could not.
- Only append the account email to notification titles when more than one
Ajax config entry exists — on single-account installs it was pure noise
on every card.
- Clarify in CHANGELOG/strings that persistent notifications do not
survive a Home Assistant restart, and move the CHANGELOG entry to the
1.15.0 unreleased section (1.14.0 shipped while the branch was open).
* chore: bump version to 1.15.0-beta.2
* fix(blueprints): guard event-entity state triggers against restore false-fires
security_event_notification, tamper_alert and intrusion_alarm_capture
triggered on any state change of the event entity. When the entity is
restored (integration reload, options change, HA restart) it re-delivers
its last event with a fresh last_changed, which slips past the 10s
recency condition and re-sends the last notification (or re-captures
photos). Add the same not_from unavailable/unknown guard PR #293 gave
the other state-trigger blueprints; it skipped these three because their
bare triggers looked protected by the recency condition, but a restored
state's last_changed is always fresh.
Confirmed live on a real install: an options-change reload re-sent the
last security-event notification; with the guard the same reload stays
quiet.
* i18n: add siren volume/duration strings to all locales (#310)
* chore: bump version to 1.15.0-beta.3
* chore(actions): bump softprops/action-gh-release
Bumps the actions-minor-patch group with 1 update: [softprops/action-gh-release](https://github.com/softprops/action-gh-release).
Updates `softprops/action-gh-release` from 3.0.1 to 3.0.2
- [Release notes](https://github.com/softprops/action-gh-release/releases)
- [Changelog](https://github.com/softprops/action-gh-release/blob/master/CHANGELOG.md)
- [Commits](https://github.com/softprops/action-gh-release/compare/718ea10b132b3b2eba29c1007bb80653f286566b...3d0d9888cb7fd7b750713d6e236d1fcb99157228)
---
updated-dependencies:
- dependency-name: softprops/action-gh-release
dependency-version: 3.0.2
dependency-type: direct:production
update-type: version-update:semver-patch
dependency-group: actions-minor-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
* fix(siren): confirm settings value with targeted re-read after write (#310)
Siren volume/duration entities had no optimistic state and only re-read on
the shared 900 s snapshot timer, so after a successful write the entity kept
showing the previous value for up to ~15 min — indistinguishable in the UI
from a rejected write.
A successful write now schedules a single-flight, per-device settings re-read
after a short settle delay, so the entity confirms the real hub value within
seconds. Kept as a read-back rather than an optimistic set: UpdateHubDevice
has a real accept-but-inert failure mode on this service (as on the bypass
switch), so the entity must only ever show what an independent read returns.
Surfaced by @wip3out3r's 2x StreetSiren hardware validation on v1.15.0-beta.3.
* fix(tamper): fold real case-tampering signals into the tamper sensor (#339)
The per-device tamper binary_sensor of ~40 device families binds to a
'tamper' status key sourced from a LightDeviceStatus oneof case that no
vendored proto revision (v0.3.0 → today) has ever had — the branch was
unreachable, so the sensor could never turn on. Confirmed live: pulling a
Door Protect off its SmartBracket alarmed Ajax (monitoring-station callout)
while HA showed nothing.
The signals the wire actually carries are granular: lid_opened,
smart_bracket_unlocked, case_drilling_detected. Fold all three into the
shared 'tamper' key in both entry points:
- snapshot parser (_parse_statuses): sets 'tamper' alongside the granular
key, which is kept for profiles that surface it individually (hub lids)
- realtime delta (_handle_status_update): mirrors ADD/UPDATE onto 'tamper';
on REMOVE clears it only when no other tamper source remains active
The dead 'tamper' oneof branch stays as defensive forward-compat. The
alarm panel's issues summary (statuses['tamper']) becomes functional again.
Tests: real-proto coverage for all three signals plus negative controls —
the old test for this was a MagicMock fabricating the impossible 'tamper'
oneof case, which is exactly how the regression stayed invisible (the #119
lesson); it now carries a warning note.
Refs #339.
* chore: bump version to 1.15.0-beta.4
* fix(push): decode push events by content type and walk the payload correctly (#339, #320)
Two independent defects made push security events unusable.
Wrong type. `_extract_event_with_compiled_protos` guessed the qualifier
type by trying Space > Hub > Video > SmartLock and breaking on the first
successful decode. The tag protos share field numbers, so hub qualifier
bytes decode cleanly as an unrelated space event: `tamper_opened` (17)
reads as `space_group_duress_disarmed` -> `disarm`, `malfunction` (14) as
`space_night_mode_on_with_malfunctions` -> `arm_night`, `door_opened` (1)
as `space_armed` -> `arm`. Every mapped hub tag up to field 18 collides
with a mapped space tag, and unmapped hub tags manufacture arm/disarm
events out of nothing — which then feed `_apply_security_state_from_
event`. A genuine space-arm push also resolved to `disarm`, because a
spurious candidate from the unstructured scan won the priority tie.
Payloads are `PushNotificationDispatchEvent` messages, so they are now
decoded structurally and the qualifier is read from the
`NotificationContent` oneof that carries it — the only reliable type
discriminator. `media_enriched_notification` (photo-on-demand) is
handled too. When the content type is known but its tag is unmapped the
result is None: reporting no event beats reporting a cross-decoded one.
Payloads that don't decode as a dispatch notification still fall through
to the scan, which keeps the legacy precedence for lack of a wrapper.
Lost events. `_find_embedded_messages` scanned byte-wise for wire-type-2
headers instead of walking the wire format: it never consumed varint /
64-bit / 32-bit payloads (a value byte read as a field tag threw the walk
out of step and over the qualifier), read tags as single bytes (field
numbers above 15 need two), rejected 4-byte tag-only qualifiers, and put
the recursive descent inside the `4 < length < 500` filter so a wrapper
over 500 bytes yielded zero candidates. It is now a proper wire walk with
no size bounds, plus depth and candidate caps as loop guards.
Tests build genuine dispatch-event payloads from the vendored protos —
the type-collision class cannot be reproduced with mocks. Also covers
the >500 B push, the 4-byte qualifier, media-enriched pushes, and the
unmapped-tag case. DEBUG lines record which path resolved a push so a
hardware capture can confirm it.
Refs #339, #320, #287
* fix(push): never scan a recognised-but-unmapped notification content
Audit follow-up. `company` / `accounting` / `blank` contents DO carry a
qualifier (Company/Accounting/BlankEventQualifier) — the previous comment
claiming otherwise was wrong — so they fell through to the candidate scan
and could still be cross-decoded into a phantom arm/disarm. A recognised
content case is now authoritative in every branch: mapped vocabularies
resolve, known-unmapped ones report no event, and a content case newer
than the vendored protos logs a WARNING (protos need refreshing) and also
reports nothing. The scan is now reachable only for payloads that aren't
dispatch notifications at all.
The captured payload the suite already replays turns out to be a
photo-on-demand notification (`photo_on_demand_with_name`, unmapped): the
scan reported it as `arm_night`/`space_night_mode_on`, which also applied
a NIGHT_MODE security state. Locked in as a regression test asserting
both the old phantom and the new None, so the fixture can't silently stop
covering it.
Descriptor-validated mapping tests (tests/unit/test_event_tag_maps.py):
every tag in the four vocabularies exists in its proto; TAG_PRIORITY and
both RAW_TAG_TO_*_SECURITY_STATE maps only reference tags the parser can
actually report; SECURITY_STATE_EVENT_TYPES are reachable; every
NotificationContent case is either mapped or explicitly unmapped, so a
new Ajax content type fails a test instead of silently changing routing.
Plus the collision canary documenting why the content case must decide
the type, including why hub arm/disarm (fields 83-96) escaped the bug
while door/tamper/malfunction did not.
Also pins the property that made the wrong-event class survivable: a push
that resolves to no event still triggers the coordinator re-read, so the
"push says something happened, ask Ajax what" path never depends on the
parser succeeding.
Refs #339, #320
* chore: bump version to 1.15.0-beta.5
* chore(339): read-only DEBUG probe for the HTS case-tamper candidate keys
A Hub Plus capture showed per-device kv keys 0x04 and 0x0f flipping 00->01
in lockstep with a physical SmartBracket detach, on a hub whose gRPC status
snapshot carries no tamper signal at all — so the #340 fold cannot help
there. One hub is not enough evidence to wire a user-visible alarm signal:
a key meaning something else on another firmware would raise phantom
tampers. This logs the values (with the current tamper status alongside, so
the 'keys flip while tamper stays None' pattern is visible) without acting
on them, ahead of the routing itself.
Probe runs before the temperature merge, which returns early for its gated
families — otherwise a Curtain Outdoor or a siren would never report.
Refs #339
* fix(tamper): route the HTS case-tamper keys onto the tamper sensor (#339)
On hubs whose gRPC device stream carries no case-tampering status at all,
the #340 fold has nothing to fold: the tamper sensor stays off through a
physical detach. Those hubs report it on the HTS status stream instead --
per-device keys 0x04/0x0f, both flipping 00->01 on a SmartBracket detach
and back on re-attach (Hub Plus, MotionProtect Curtain, two runs).
Wired conservatively, because a key that means something else on another
firmware would raise a phantom tamper alert:
- only 00/01 are acted on; any other byte is traced and ignored;
- clearing mirrors the gRPC delta path -- tamper only goes away once no
granular gRPC source (lid_opened / smart_bracket_unlocked /
case_drilling) is active either;
- the value is marked as HTS-sourced (hts_case_tamper) so a fresh device
snapshot -- which on these hubs has no tamper field -- carries it forward
instead of silently wiping the sensor, and so an HTS 00 only withdraws
what HTS itself raised;
- an unchanged value re-reported on the 60 s status refresh is a no-op.
Evidence base, both hardware: the positive transition above, plus nine
intact devices across four families (door_protect, door_protect_plus,
keypad_combi, motion_cam_phod) on a second hub all reading 00 -- so the
keys are not routinely set for unrelated reasons. Which key is the lid and
which the bracket is still unconfirmed; the shared tamper sensor doesn't
need the distinction. The DEBUG trace logs both values against the current
tamper status so a reporter can confirm the semantics on their own hub.
Refs #339
* chore: bump version to 1.15.0-beta.6
* chore: release 1.15.0
Consolidates the 1.15.0-beta series into the stable line.
- Push security events are decoded against their own event vocabulary
instead of guessing, so hub events are no longer relabelled as
arm/disarm (see #320, #339, #287).
- Device case-tamper sensors work for the first time: the gRPC statuses
fold plus the HTS status-stream source (see #339).
- Siren volume and alarm duration are configurable, with a targeted
read-back after each write (see #310, #337).
- Per-device firmware update entities and optional persistent
notifications for security events.
README audited for the release: the three new feature families are
documented, the data-source table records the dual-sourced case tamper
and the HTS-carried per-device temperature, and the device-settings
roadmap item is marked as partially shipped.
Hardware validation on two independent installs by @wip3out3r; log
evidence for the push cross-decode by @Daniel-Vitanza.
* chore(actions): bump actions/checkout in the actions-minor-patch group
Bumps the actions-minor-patch group with 1 update: [actions/checkout](https://github.com/actions/checkout).
Updates `actions/checkout` from 7.0.0 to 7.0.1
- [Release notes](https://github.com/actions/checkout/releases)
- [Changelog](https://github.com/actions/checkout/blob/main/CHANGELOG.md)
- [Commits](https://github.com/actions/checkout/compare/9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0...3d3c42e5aac5ba805825da76410c181273ba90b1)
---
updated-dependencies:
- dependency-name: actions/checkout
dependency-version: 7.0.1
dependency-type: direct:production
update-type: version-update:semver-patch
dependency-group: actions-minor-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
* chore(actions): bump home-assistant/actions/hassfest
Bumps [home-assistant/actions/hassfest](https://github.com/home-assistant/actions) from f4ca6f671bd429efb108c0f2fa0ae8af0215986c to e3fb68ebda13d88a0d695082f471ba2c83d025fb.
- [Release notes](https://github.com/home-assistant/actions/releases)
- [Commits](https://github.com/home-assistant/actions/compare/f4ca6f671bd429efb108c0f2fa0ae8af0215986c...e3fb68ebda13d88a0d695082f471ba2c83d025fb)
---
updated-dependencies:
- dependency-name: home-assistant/actions/hassfest
dependency-version: e3fb68ebda13d88a0d695082f471ba2c83d025fb
dependency-type: direct:production
...
Signed-off-by: dependabot[bot] <support@github.com>
* fix(bypass): read the panel's real deactivation state and confirm writes (#338)
A device deactivated from the Ajax app leaves `profile.bypassed` False and
reports one of the `*_deactivation_*` statuses instead, none of which the
parser read — so HA showed a panel-disabled sensor as live protection and
its bypass switch as `off`.
Read them on both paths (snapshot parser and realtime delta), folded into a
shared `deactivated` key the bypass switch binds to, with the granular mode
exposed as a `deactivation_kinds` attribute. `*_deactivation_tamper` means
the tamper protection is disabled, NOT a tamper alarm, and is deliberately
kept out of the #339/#340 tamper fold living in the same function. The
inverted Ajax naming (`temporary_` = permanent, `one_time_` = one arming
cycle) is documented at the mapping site.
Writes get the #337 confirm-read treatment: `DeviceCommandBypass` answers
success and applies nothing when the account can't set the requested mode,
so a successful command is not evidence of a state change. Every write
schedules an independent per-device read-back (single-flight, 5 s settle)
and logs a warning when the panel disagrees with what was asked.
Option B (switching the write lever to BYPASS_ONE_TIME_DISABLE) stays gated
on the hardware datapoint requested in the issue.
* chore: bump version to 1.15.1-beta.1
Ships the #338 bypass read-first fix plus the post-write confirm read for
hardware validation against a live deactivation fixture.
* fix(bypass): drop the wrong cause from the accept-but-inert warning (#338)
Hardware validation on 1.15.1-beta.1 disproved the diagnosis the warning
asserted. wip3out3r's fixture: the three deactivated devices were deactivated
by the system administrator from the STANDARD Ajax app and it took effect —
so the account can deactivate. Only `DeviceCommandDeviceBypass` from an Aegis
session is inert, and enum 4 fails the same way. "The account most likely
lacks the rights" sent readers chasing a permission problem that isn't there.
The warning now reports the symptom, names the device, points at the app as
the working path, and links the issue. A test pins the absence of a cause
claim, because this string is what users paste into issues.
* chore(338): read-only DEBUG probe for the HTS bypass-configuration keys
wip3out3r's 1.15.1-beta.1 validation surfaced a lead nothing in the
integration reads: `CommonJewellerPart.bypass_part` (a `BypassPart` carrying
the device's `capabilities` and its list of enabled `bypass_mode`s). If a
device's enabled modes exclude the one the bypass command sends, then
"accepted and applied to nothing" is the expected outcome — system
configuration rather than an account right, which would also explain why
enum 4 fails while the Ajax app succeeds.
Two hints put the data on HTS rather than behind a new request: his
reactivation log showed `0xb7` on the device's STATUS_UPDATE row and `0xb6`
on the SETTINGS_UPDATE row, both `00` once protected, and `bypass_part` is
field `0xB7` in the gRPC model. If the numbering spaces coincide it is
already arriving on a stream we consume.
Logged with the device's deactivation state on the same line — a bare byte
proves nothing, the same byte next to deactivated=True on one device and
False on another identifies the field. Nothing is routed off these keys:
misreading one onto the bypass switch would misreport whether a sensor is
protecting anything.
Refs #338
* chore: bump version to 1.15.1-beta.2
Corrects the accept-but-inert warning's wrong cause attribution and adds the
read-only bypass-configuration probe, both for #338.
* fix(fcm): read back Google's actual api-key refusal reason (#344)
A Firebase Installations 403 has two causes needing opposite fixes:
API_KEY_SERVICE_BLOCKED (the extracted AIza is not the FCM one — change the
key) and API_KEY_ANDROID_APP_BLOCKED (the key may be right; Google's Android
restriction rejected our request — the key is not the problem).
`firebase-messaging` 0.4.5 collapses both into "Unable to register with fcm"
and drops the HTTP body carrying the reason, so the warning asserted the
first explanation for both. Reporters with the correct key were sent off
extracting more keys, and establishing which case they were in cost a
round trip each time (#344, and #182/#194 before it).
The integration now re-issues the Installations call itself, purely to read
the reason back, and logs it with advice matching the actual answer — for
the restriction case, naming the package the request identified itself as,
since an unmapped co-brand sends none at all and is refused whatever key was
extracted. Only on a path that has already failed, never raises, and answers
nothing rather than guessing when the response isn't a 403 it can read. The
api-key is sent (it is the thing under test) and never logged.
The general message no longer picks a side between the two causes.
* chore: bump version to 1.15.1-beta.3
Ships the FCM api-key refusal-reason read-back so #344 can be diagnosed from
a single log line instead of a round trip.
* fix(fcm): read the api-key rejection reason on any status, not only 403
The reason #344 actually returns is HTTP 400 / API_KEY_INVALID — Google does
not recognise the string as one of its api-keys at all. The probe shipped in
1.15.1-beta.3 parsed the body only on a 403, so it stayed silent on the exact
report it was written for, and the message asserted a status the branch
cannot guarantee.
Any non-2xx body is now read: the status code was never part of the contract,
the error body is. API_KEY_INVALID gets its own advice, because it is neither
a scope nor a restriction problem — the value is damaged in transit, or that
`AIza…` was never a live key (the native library carries several strings of
the right shape, and keys rotate between app builds), so both "try another
key" and "check the package" would send the reporter the wrong way.
Also corrects a factual claim in the module comment: the library does not
discard the response body, it writes it to its own logger before raising a
bare RuntimeError. That is why the data existed all along and only a second
logger revealed it.
Refs #344
* feat(hts): probe the Button activity-timestamp keys 0x39/0x40 (#348)
A control-mode Button press produces no user-visible signal today. A
hardware log from #348 shows why it might be cheap to change that, and
also why it isn't yet safe to.
What the capture establishes: HTS key `0x39` on the Button's
STATUS_UPDATE row moves to the exact second of each press. Two
independent presses, on a UTC+3 install — `6a683b9f` = 05:18:23Z for a
press logged 08:18:25 local, `6a683cbd` = 05:23:09Z for one logged
08:23:10. It decodes as a big-endian Unix epoch, and it is the only
signal the press produces at all: the `button_1_on` / `button_2_on` FCM
tags never appeared, and the coincident gRPC delta is a plain `battery`
status.
What it does not establish, and why this is a probe rather than an event
entity:
1. Short and long click are indistinguishable in it. Both moved `0x39`
and nothing else, so one key cannot carry the two separate actions
the issue asks for. `0x40` is not the second action either — it held
a three-week-old epoch on the install's other Button, so it tracks
something rarer than a press.
2. Whether `0x39` moves without a press is unknown. Ajax peripherals
ping the hub for supervision, and if this were last-radio-contact
rather than last-press, routing an HA event off it would fire phantom
presses into users' automations on every ping.
So: log transitions only, act on nothing. Transition-only is what makes
the next capture conclusive — an untouched Button stays silent, so
silence over a quiet window falsifies the ping reading while a 1:1 match
with presses confirms it. A first sighting is recorded silently, since
the boot snapshot re-reports whatever the last press left behind and
would otherwise look like a press at every restart.
Values are rendered as UTC so a reporter can line a line up against the
moment they pressed, and a value that is not a plausible epoch is
reported as raw rather than dressed up as a date — the keys may carry
something else entirely on another firmware.
Refs #348
* chore: bump version to 1.15.1-beta.4
* fix(fcm): read a bare PERMISSION_DENIED as a fourth refusal reason (#344)
@Thomas-v87 has two candidate api-keys and they fail *differently*. One
returns HTTP 400 `API_KEY_INVALID`. The other returns:
{"error": {"code": 403,
"message": "The caller does not have permission",
"status": "PERMISSION_DENIED"}}
No `details`, no `ErrorInfo`, so no `reason` — which meant the probe
found nothing structured and the listener fell through to "please
include this line when reporting the problem". That threw away the most
informative signal in the whole issue: Google *recognises* this string
as one of its api-keys (or it would have said `API_KEY_INVALID`) and
refuses to authorise it for this project's app. A real key, failing for
a reason that is neither scope nor restriction.
So `_extract_refusal_reason` now falls back to `error.status` when there
is no `ErrorInfo`, and `PERMISSION_DENIED` gets its own advice. A
structured reason still wins over the status — the specific answer beats
the coarse one when a body carries both.
The advice deliberately points somewhere none of the other three do.
Sending this user to extract another `AIza…` is the `API_KEY_INVALID`
remedy and would waste their time again; pointing at the app label is
the `ANDROID_APP_BLOCKED` remedy and is equally wrong here. The usual
cause is four values that don't all come from the same app build, and
the api-key is precisely the value nothing can catch offline: the shape
check ties `fcm_sender_id` to `fcm_app_id`, but no local check can tie
the key to either. So a mismatched pair passes validation and only
Google can reject it.
Stated as the usual cause rather than the certain one — asserting a
single cause for a multi-cause branch is the original defect in this
issue and it is not worth repeating a third time.
Refs #344
* chore: bump version to 1.15.1-beta.5
* fix(hts): read the Relay's voltage sub-key as millivolts (#325)
The per-device electrical sub-key carrying line voltage is shared by the
WallSwitch, Socket and Relay families, and the integration mapped all of
them through one key map that treated the raw value as whole volts. That
is right for the metering families but wrong for the Jeweller Relay,
which reports millivolts: a 12 V-fed Relay surfaced as 11,671 V.
`relay` now gets its own key map that scales the sub-key to volts. The
factor is unconditional — the Relay accepts 7-24 V DC or 110-230 V AC
and reports millivolts either way, so a mains unit sends 230000 — which
is what makes a fixed per-family scale safe here. The opt-in derived
Power sensor reads the same field and is corrected by the same change.
`relay_fibra_base` is deliberately left on the WallSwitch map: its unit
is unobserved, so matching it to the Jeweller Relay would be a guess. A
test pins that decision so a future change has to be deliberate.
DeviceReadings' numeric fields widen to float to carry the converted
value; an integer scale still yields an integer, so the WallSwitch,
Socket and Outlet paths are byte-identical.
Reported by @AdamG100.
* feat(siren): cover the DoubleDeck / Fibra / S siren SKUs (#354)
Six siren models showed only case tamper, bypass and battery: the Street
Siren DoubleDeck, S DoubleDeck, DoubleDeck Fibra, S, Fibra and Plus
Fibra. They were excluded from SIREN_DEVICE_TYPES on purpose — the
`HubDevice.device` oneof did not model their cases, so their snapshot
decoded as an unknown case, no settings could be read, and creating the
volume/duration entities would have left them permanently empty.
The oneof now models those cases (62-67, plus HomeSirenPlus at 68) and
`street_siren.proto` / `home_siren.proto` gain the matching messages.
Each of the six carries only `common_siren_part` upstream — no
temperature, tamper or battery part — so their internal temperature keeps
coming from the status stream, not from here.
`parse_hub_device_siren_settings` needed no change: it already resolves
`common_siren_part` off whatever oneof case is set rather than
enumerating device types. Nor did the write path, which addresses the
device by ObjectType and so never depended on the oneof at all — these
SKUs were always writable, only unreadable.
`home_siren_plus` is modelled in the proto but deliberately kept out of
SIREN_DEVICE_TYPES: ObjectType has no such case, so `parse_device` can
never produce that device_type. A test pins both directions of that
invariant — every gated type must be a real ObjectType *and* a decodable
oneof case — so the two can't drift apart silently.
Only the three touched protos were recompiled, not the whole tree.
Reported by @nimahel.
* fix(hts): log the first delta sighting of a delta-only probe key (#348)
The Button activity probe records each key's first sighting silently so
the boot snapshot doesn't read as a press at every restart. That is right
for a key the hub re-reports in its periodic STATUS_BODY, and wrong for
one that only ever arrives in a STATUS_UPDATE delta: there the first
sighting IS the event, so the first press after every restart was being
swallowed.
The per-device kv callback now reports which kind of message a row came
from, and the probe uses it: a first sighting in a body stays silent, and
a first sighting in a delta is logged once the device's body row has
arrived without that key — which is what proves the key is delta-only.
Before any body has arrived the two cases are indistinguishable, so a
delta that early is still recorded silently rather than guessed at.
The callback contract becomes a Protocol rather than a Callable alias,
because `from_body` has to be keyword-only and Callable cannot express
that. Body/delta reporting is now covered for all four sub-keys.
DEBUG diagnostics only; no entity or state behaviour changes.
Found by @wip3out3r, whose first press produced no log line at all
because he pressed after a restart but before the key had ever appeared
in a body.
* chore: bump version to 1.15.1-beta.6
* fix(siren): report why a siren settings snapshot came back empty (#354)
On 1.15.1-beta.6 a Street Siren DoubleDeck materialises both config
entities and accepts writes, but the values never populate. An empty
settings dict has three causes needing three different fixes and the log
could not tell them apart: an unmodelled `device` oneof case (nothing
decodes), a modelled case carrying no `common_siren_part` (this SKU does
not expose its settings on this stream), or a part whose values are both
unset.
The read path now DEBUG-logs the oneof case the server actually sent plus
whether a siren part was present, on the empty path only — mirroring the
probe the temperature path has had since #229. A message whose type has
no such field at all reports `None` rather than False, so "field absent"
stays distinct from "field unset".
Silent on the healthy path, so the 900 s sweep adds no noise.
No behaviour change; diagnostics only.
* feat(button): fire an event for an Ajax Button in control mode (#348)
A Button in control mode was invisible to Home Assistant. The hub sends no
push for those presses, so there was nothing to trigger an automation
with; only panic mode had a path, via the hub-level `panic` event.
Each Button now gets an `event` entity (`device_class: button`) firing
`pressed`, sourced from HTS sub-key 0x39 — a big-endian epoch of the
press. It therefore works without FCM configured, armed or disarmed.
ONE event type, not two: short and long click move that one key
identically, and the hub does not push control-mode presses at all
(confirmed on an install with push working), so nothing can tell them
apart. Advertising two would imply a distinction the data cannot support.
Why 0x39 is a press and not a supervision ping, from @raven2k24's
captures: 14 presses produced 14 transitions each within 1-2 s; 64
minutes and 61 snapshot cycles afterwards showed no movement; and
decisively, the value present at boot was 20 hours old, where a
contact-tracking key would have moved hundreds of times. That last one
needs no log coverage to hold.
Four guards keep it from inventing a press: gated on device type (the
same sub-key is a roller-shutter flag on a DoorProtect Plus and a
hub-wide counter on a StreetSiren), a first sighting never fires, the
value must be a plausible epoch, and time must move forward.
Both snapshot and delta rows feed it through one shared cache. The deltas
catch every individual press while the snapshot collapses a burst to its
last value, so a missed delta still surfaces — at a slightly older
timestamp — and neither path can double-fire.
`0x40` is deliberately not wired up: it advances in lockstep across
sirens on the same hub, so it tracks something hub-wide, not the device.
Strings added in all 14 locales; the last press epoch is dumped in
diagnostics so a silent entity can be told from a silent hub.
* chore: bump version to 1.15.1-beta.7
* fix(triggers): scope device triggers to the hub that fired the event (#358)
An automation using an arm/disarm device trigger on one hub also ran
for every other Ajax system: the device trigger delegated to a bus
event trigger that matched on event_type alone, and the bus payload
carried no hub identity to match on.
The hub-level event entity now stamps its own hub_id and space_id onto
the aegis_ajax_event payload (after the spread, so the source device's
fields cannot collide), and the device trigger resolves the automation's
HA device back to its aegis hub id and filters on it. A device that can
no longer be resolved degrades to the old event-type-only match instead
of detaching the automation.
* test(triggers): stub HA's TRIGGER_SCHEMA in the attach test
Validating the delegated event trigger's schema needs a hass in the
context — its template validator reports validating outside the event
loop, which the Python 3.13 image treats as an error. That check is
HA's, not ours, so stub it and assert on the config we hand it.
* chore: bump version to 1.15.1-beta.8
* fix(push): route each push to the space that produced it (#358)
Arm/disarm device triggers still fired for every configured Ajax system
after beta.8. That release made the bus event carry the firing hub and
made the device trigger filter on it, which was necessary but not
sufficient: the leak was upstream, in how a push is matched to a space.
`_find_space_for_event` scanned the payload for `bytes.fromhex(hub_id)`,
but a hub's id travels as an 8-char ASCII string (`HubOrigin.hex_id`),
never as those raw bytes — verified against the captured push already in
the test suite, where the scan returns False and the space id is present.
So routing always failed and every push took the fallback that delivered
it to *every* space. Each copy was stamped with a genuine hub_id, so the
trigger filter matched legitimately five times over.
Route on `Notification.space.id` instead, read structurally from the
dispatch message rather than by substring scan, and check it against the
spaces the entry knows. The hub-id scan stays as a fallback for shapes
the structural path can't decode, now guarded against a non-hex id.
Drop the fan-out, distinguishing two cases so the log stays honest:
- one space -> the destination is unambiguous, deliver there;
- the payload names a space this entry doesn't manage -> DEBUG, since
the FCM stream is per account and a user who added 2 of their 5
systems legitimately receives the other 3;
- several spaces and no name -> WARNING and discard, rather than
deliver to the wrong hubs. The unconditional snapshot nudge already
re-reads authoritative state.
This also fixes a second symptom nobody had reported: the fan-out ran
`_apply_security_state_from_event` for every space, so disarming one hub
briefly showed all of them disarmed.
Extract `_dispatch_notification` for the dispatch-unwrapping now shared
by the event and space-id extractors.
* fix(siren): report the real cause when a settings read fails (#354)
nimahel's DoubleDeck shows `unknown` for Siren volume / Alarm duration
while writes reach the hub, and the beta.7 probe added to explain it
never appeared in their log. The probe sits on the empty-snapshot
branch, but the read never gets there: the RPC itself raises.
The handler logged that exception with `exc_info=True` alone, so the
status code that separates a permission denial from a timeout lived at
the tail of a traceback — the part their log viewer truncated. Three
causes were modelled for an empty read; a raising read was a fourth
nobody had instrumented.
Add `_describe_rpc_error` to render type + gRPC status (+ details) on
the message line itself, so it survives a truncated paste, and use it
here. `exc_info` is kept for the full picture when the log is complete.
Raise the first failure per device to WARNING: the symptom is two
entities stuck on `unknown` indefinitely, which at HA's default level
was invisible until the reporter was asked to turn debug on. Repeats
drop back to DEBUG so a permanently unreadable siren can't spam every
900 s sweep, and a successful read clears the latch so a device that
recovers and breaks again warns again.
Diagnostics only — no entity behaviour changes.
* chore: bump version to 1.15.1-beta.9
* fix(push): decode each event source with its own vocabulary (#367)
`_extract_source_info` located an event's source by scanning the payload
for a `HubNotificationSource` and mapping its numeric `type` through the
hub vocabulary. All four `*NotificationSource` messages share the same
wire layout (`1 type` varint, `2 id` string, `3 name` string), so a
`SpaceNotificationSource` — what a whole-space arm/disarm carries —
parses cleanly as the hub variant and its type is translated with the
wrong dictionary. The vocabularies disagree on every value: 2 is
HUB_PLUS in one and SPACE_MEMBER in the other.
Verified before and after on a real space disarm by a person:
before: {'device_name': 'Carlos Lopez', ..., 'device_type': 'HUB_PLUS'}
after: {'device_name': 'Carlos Lopez', ..., 'device_type': 'SPACE_MEMBER'}
Same class as #320/#339, fixed there for event tags by letting the
content wrapper identify the vocabulary. The source field never got the
same treatment; this applies it.
Read the source from the content case that names its kind, via a
`_CONTENT_SOURCE_FIELDS` table mirroring `_CONTENT_TAG_MAPS`.
`space_notification_content` carries both `space_source` and
`hub_source`; prefer the former, since it is the one that answers "who
armed this?" (#362, #359).
Derive the vocabulary from the source message's *own* descriptor rather
than a hard-coded enum table, so each is necessarily translated with its
own enum and the four cannot be crossed again.
A recognised content with no usable source returns {} rather than
falling back to the scan — a known container is authoritative, and
rescanning it is exactly how a neighbouring message gets mistaken for
the source. Payloads that aren't typed dispatch notifications still use
the legacy scan.
Behaviour change: automations filtering on device_type for arm/disarm
see real values instead of meaningless hub model names. device_name,
which is what users were told to use, is unaffected.
* chore: bump version to 1.15.1-beta.10
* fix(proto): recompile stray stubs and pin the protobuf toolchain (#354)
Three of the 1614 generated stubs — the ecosystem `hub_device`, `home_siren`
and `street_siren` on the StreamHubDevice path — were regenerated by a newer
grpcio-tools than the rest while adding the DoubleDeck siren SKUs, stamping
Protobuf gencode 7.35.1 into them where the other 1611 carry 6.31.1. Protobuf
refuses at import time to load a stub built by a version newer than the
installed runtime, and Home Assistant ships protobuf 6.x, so importing those
three raised VersionError on every install from 1.15.1-beta.6 onwards.
Both consumers of that path were dead as a result: the writable siren settings
(#310, hence "siren volume and alarm duration never load" on every siren model,
not just the new ones) and the per-device internal temperature (#220, #229).
It also explains why beta.7's diagnostic probe never printed anything — it sits
downstream of the failing import.
Recompiles the three with the pinned compiler so all 1614 agree, and fixes two
latent faults of the same kind found on the way: the manifest advertised
protobuf>=4.25.0 and grpcio>=1.60.0 while the generated code has long required
6.31.1 and 1.75.1, so an install resolving to either floor would have failed on
import. Both floors now state what the code needs, in manifest.json and its
pyproject.toml mirror.
Prevention, since this had already happened once (cb57480) and recurred:
- grpcio-tools is pinned exactly rather than floored, so codegen is
reproducible and a rebuilt dev image cannot silently change what a
regenerated stub demands of the user's runtime.
- compile_protos.sh refuses to run when the installed compiler disagrees with
that pin, before it writes anything.
- Regenerating a subset is now first-class (`make proto PROTOS=...`), removing
the reason to reach for the ad-hoc protoc invocation that caused this.
- A unit test asserts every stub agrees on one gencode version and that both
stamped versions are loadable by the oldest protobuf/grpcio the manifest
accepts.
- A new CI job installs those exact floors in a throwaway container and imports
all 3228 stubs. The test suite could never have caught this on its own: the
dev image installs the newest runtime satisfying our floors, and a stub only
fails to load under an older one.
Both guards were verified to fail against the state that shipped and pass after
the recompile.
* chore: bump version to 1.15.1-beta.11
* docs(notification): correct the stale space-routing comment (#358)
The comment above `_find_space_for_event` still described the pre-#358
behaviour — scanning the payload for the hub id as raw bytes — which is
precisely the wrong mental model that caused pushes to be delivered to every
configured space. The function itself was fixed in 1.15.1-beta.9; only its
call-site comment was left behind, where it reads as documentation of what the
code does rather than of what it must never do again.
* fix(fcm): survive an undecodable push frame instead of dying on it (#373)
One push message the library cannot decrypt was enough to end real-time
push permanently, and silently.
`_decrypt_raw_data` decodes the `crypto-key` and `encryption` header values
without padding, while padding the two stored key values in the same
function. Those headers are URL-safe base64 that may legitimately arrive
without trailing `=`, so an unpadded one raises `binascii.Error`. That is a
`ValueError`, so the listen loop's `except (OSError, EOFError)` misses it, it
reaches the outer `except Exception`, and the client shuts down.
The severe part is where it raises: before the library appends the persistent
id and sends the selective ack. The message is therefore never acknowledged,
so it is redelivered on the next connection — and the supervision from #285
faithfully restarts straight back into it. The reporter measured the same
message killing the client 16 times over 3.5 hours, each death 3-9 ms after
receiving it, surviving a host reboot because the queue is server-side.
Three changes:
- `install_fcm_decrypt_guard` pads both header values before delegating, and
contains any remaining decode failure by returning empty bytes. The handler
then stays on its normal path and reaches the acknowledgement, so one event
is lost instead of every future one.
- Repeated deaths on the same persistent_id raise a Repair. The failure is
otherwise invisible: alarm state comes from polling and HTS, never from
push, so nothing looks wrong from the outside.
- At that same threshold the stored FCM registration is discarded and renewed
once, which is what the reporter did by hand. Without it the only recovery
available to a user is editing `.storage`.
Upstream fixes the root cause in sdb9696/firebase-messaging#37, open and
mergeable since June with no release carrying it; the guard goes away when a
release ships it.
Also fixes two things found on the way: `test_start_without_firebase_messaging`
simulated an absent package by nulling only the parent module, which stops
working once anything imports a submodule, and `strings.json` was missing the
`fcm_not_configured` entry that `translations/en.json` already had.
Refs #373, #359
* chore: bump version to 1.15.1-beta.12
* feat(siren): Add StreetSiren Double Deck HTS temperature support
- Added street_siren_double_deck and street_siren_double_deck_fibra to HUB_DEVICE_TEMPERATURE_DEVICE_TYPES and HTS_TEMPERATURE_DEVICE_TYPES.
- Ensures these siren variants are treated as HTS-sourced temperature
devices
- Updated sensor setup so temperature entities are created for
hub-device temperature types even before device.statuses["temperature"]
is present
- Refactored sensor entity creation into _should_create_status_sensor()
for cleaner readability
* fix(repairs): make the "Learn more" links on Repair notices resolve
Three push-notification Repairs and the hub-network-sensors one built
`learn_more_url` from README fragments that match no section: the headings had
been reworded since. A fragment that matches nothing does not 404 — it silently
drops the user at the top of a 600-line README — so nothing surfaced it.
Rather than rewrite the code to match the current prose (which would break again
at the next rewording), the README now carries explicit `<a id="...">` anchors
for the two stable targets the code references, and a test asserts every
`DOCS_BASE_URL` anchor in `repairs.py` resolves to either an explicit anchor or a
generated heading slug. Verified the test fails when an anchor is removed.
The push anchor is the one the new `fcm_push_stuck` notice uses, so it mattered
more than usual: that card tells a user their push has been dead and then sent
them to the wrong place.
Also consolidates the duplicated `### Added` heading in the 1.15.1 changelog
block so it matches the one-section-per-type format the other releases use.
* chore(deps): stop Dependabot proposing grpcio-tools minor/major bumps
`grpcio-tools` is pinned exactly rather than floored, because the compiler bakes
its Protobuf gencode version into every generated stub. Moving the pin is
therefore a three-part job — bump, recompile the whole proto tree, raise the
runtime `protobuf` floor to match — and the third part is blocked on Home
Assistant itself shipping protobuf 7.x, which it does not.
Dependabot proposed 1.75.1 → 1.83.0 (#377) as a routine bump. 1.75.1 emits
gencode 6.31.1; 1.83.0 emits 7.35.1, which is the exact version from the #354
failure ("gencode 7.35.1 runtime 6.32.0") that broke every install on
1.15.1-beta.6 through beta.10.
A review habit is not enough of a guard here, because CI cannot see the problem:
the bump changes no stub, so every check passes — including proto-runtime-floor —
and the breakage lands later, on whoever next runs `make proto`. So this is a
rule instead.
Patch bumps stay allowed: those keep gencode at 6.31.1.
* chore(actions): bump home-assistant/actions/hassfest
Bumps [home-assistant/actions/hassfest](https://github.com/home-assistant/actions) from e3fb68ebda13d88a0d695082f471ba2c83d025fb to a7c616ce81ccda50150bf1595786c71b1883fabb.
- [Release notes](https://github.com/home-assistant/actions/releases)
- [Commits](https://github.com/home-assistant/actions/compare/e3fb68ebda13d88a0d695082f471ba2c83d025fb...a7c616ce81ccda50150bf1595786c71b1883fabb)
---
updated-dependencies:
- dependency-name: home-assistant/actions/hassfest
dependency-version: a7c616ce81ccda50150bf1595786c71b1883fabb
dependency-type: direct:production
...
Signed-off-by: dependabot[bot] <support@github.com>
* chore: release 1.15.1
Consolidates the 1.15.1-beta.1..beta.12 series (#325, #338, #344, #348, #354, #358, #367, #373, #378) and applies the stable-cut README audit: push-recovery Repair listed, siren temperature coverage clarified, Relay metering explained, arm/disarm source types documented.
* fix: restrict temperature sensor creation to HTS-backed devices
* fix: add street_siren_s_double_deck to temperature device sets
* style: fix import order and formatting
* fix(sensor): import HTS_TEMPERATURE_DEVICE_TYPES from its real module, add tests
The helper imported the set from `const`, where it does not live — it is
defined in `api/hts/hub_state`. That broke type-check and every test that
imports the sensor platform.
Adds the creation-gate tests the PR checklist left open: one per Double Deck
variant asserting the temperature entity exists before the first HTS value,
one asserting the Curtain Outdoor Mini stays excluded (#269) since it has no
HTS source and would otherwise get a permanently-unknown entity, one asserting
it still appears once a real value arrives, and one asserting the gate does not
leak to the other status keys.
Also merges main (1.15.1), records the change under a new CHANGELOG section and
corrects the README sirens row, which said the Double Decks report no
internal temperature.
* fix(hub): surface why a hub's SIM read failed, and dump it in diagnostics
Reference #379.
The IMEI sensor is only created for hubs present in `sim_info`, so a failed
read means the entity is never offered — and one created on an earlier start
stays `unavailable` forever, since HA does not evict entities a platform
stops offering. That was invisible: `get_sim_info` swallowed every exception
into a DEBUG line naming neither the cause nor the gRPC status code, and
returned `None` for both "call failed" and "hub has no SIM".
The read now lets errors propagate so those two states are distinguishable.
The coordinator owns them and logs the first failure per hub at WARNING with
the cause and the affected entity, repeats at DEBUG. A hub with no modem
still returns None and is not an error.
Diagnostics gains a `sim_info` section — it drives an entity, so the project
rule says it belongs there — reporting per hub whether the read ever
succeeded. Only the IMEI's length is included, never the value.
* feat(groups): expose which devices belong to each Ajax group
Reference #366.
Group membership was readable only from the Ajax mobile app. The integration
already fetches it — it drives the per-group alarm panels — but never surfaced
it, so a finding about a device's group (#348's siren activity counter) could
not be reproduced from Home Assistant alone, and multi-group automations had
to hard-code the layout. Rooms do not answer this: a device has a room and a
group independently, and rooms already map to areas.
Each group's alarm panel gains `member_device_ids` and `member_device_names`,
sorted by name. Diagnostics gains each device's `group_id` and its resolved
`group_name`, since a bare id means nothing to whoever reads the dump.
No new entities: the data rides the group panels that already exist in group
mode, which also answers what a space with group mode off should report —
there is no panel and no group, and diagnostics reports null.
A per-device attribute would be the more direct shape, but there is no shared
entity base to hang it on today and introducing one belongs to #332.
* chore: bump version to 1.16.0-beta.1
Carries the Double Deck siren temperature (#375/#354), the SIM-read
diagnosability fix (#379) and Ajax group membership (#366).
* fix(hts): stop reading a device status row as the hub's mains-power flag
A per-device STATUS_UPDATE whose first key is 0x03 — the near-universal
operational-state byte — was reaching the hub-network delta path and being
read as KEY_HUB_POWERED. Disagreeing with the stored state, it scheduled a
full ~8.6 KB snapshot request on every such push, every few minutes, while
the reporter's hub was running on battery mid-outage.
The `is_per_device_shape` guard only recognises a device id at params[1];
this hub frames the marker one slot later, so the frame fell through to
`_extract_direct_kv`, which pairs positionally and produced kv[0x03].
The pop stays unconditional so #323's invariant holds — this path never
writes the power flag. Only the refresh is now gated on the frame not
carrying a populated device row. A genuine power delta is flat and still
fires immediately.
Refs #386
* fix(logbook): mark an arm that happened with malfunctions present
Ajax sends arming-despite-a-fault as its own qualifier
(`space_armed_with_malfunctions` and friends). `SPACE_EVENT_TAG_MAP` /
`HUB_EVENT_TAG_MAP` normalise all of them to plain `arm` / `arm_night`
so automations keep matching, which left the logbook rendering them
identically to a clean arm — the detail survived only on `raw_tag`.
The logbook callback already receives `raw_tag` (event.py spreads the
payload onto the bus event), so this reads it and appends a marker.
Keyed on the tag suffix rather than an explicit list, since every Ajax
qualifier for this shares it.
Event types and panel states are unchanged.
Refs #387
* chore: bump version to 1.16.0-beta.2
Carries the mains-power snapshot storm fix (#386) and the logbook marker
for arming with malfunctions present (#387).
* Fix IMEI sensor availability
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
---------
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: Basilio Vera <basilio.vera@gmail.com>
Co-authored-by: Basi <bvis@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Pg <pg.developper.fr@gmail.com>
|
I'm also seeing this problem in my app (which is not Home Assistant or Ring), and it's due to CPython closing a security hole in b64 decoding. I think given this #42 the correct fix is related to the first caveat in the PR description -- dh should be parsed out, and then given to base64 decoding with the pad fix. Here is a commit that I'm currently testing in my system: filament-dm@76d641e |
|
@sdb9696, independent downstream confirmation from the Elektronny Gorod Home Assistant integration. A user running Home Assistant 2025.11.3 on Python 3.13 with firebase-messaging 0.4.5 is hitting the exact Incorrect padding crash in _decrypt_raw_data(): In our case the impact is amplified because each config entry owns an FCM listener. The failed message is not acknowledged, the client terminates, and our recovery watchdog reconnects it, so the same condition can repeat for every configured account and generate substantial log volume. We are adding a downstream circuit breaker to bound retries and protect users, but PR #37 addresses the root decode/isolation problem. The related header parsing case from #42 also looks relevant for making the recovery complete. We would be happy to test an updated commit or pre-release against our integration and report the result. Please let us know if any additional reproduction details would help move this forward. |
|
Our patched fork is working for us so far. I'd be happy to turn this into a real PR if the approach looks ok. That commit is also cherry pickable. |
The previous commit fixed the wrong defect. Its premise - that Web Push always
sends crypto-key and salt unpadded, so every encrypted push fails - is falsified
by evidence: _decrypt_raw_data is byte-identical across firebase-messaging
0.4.0-0.4.5, and on 2026-06-22 that same code decrypted 16 real CALL_INCOMING
pushes from this operator with no error at all
(research/intercom-call-probe/logs/fcm.log). Padding alone also moved the
production failure rather than removing it: binascii.Error became
ValueError: Invalid EC key.
A DIAG probe on production captured the real shapes:
crypto-key: len=189 shape=dh=87, p256ecdsa=87
encryption: len=29 shape=salt=24
Crypto-Key and Encryption are parameter lists separated by ';' (RFC 8188 2.1,
RFC 8291 4). The operator now signs pushes with VAPID, so crypto-key carries a
second p256ecdsa segment, while the library strips the prefix by position -
[3:] for dh=, [5:] for salt= (fcmpushclient.py:425-426). The remainder is
'<dh>; p256ecdsa=<...>'; the base64 decoder silently drops ';', the space and
the label letters, yielding 137 bytes where a P-256 point needs 65. Salt already
arrives padded (24 chars) and dh does not (87), so padding is neither necessary
nor sufficient - dh only decodes once separated from the VAPID segment.
Causal chain: operator adds VAPID -> positional strip leaves a trailing segment
-> base64 decodes a 137-byte soup -> from_encoded_point rejects it -> the
exception escapes _handle_data_message -> _listen shuts the client down before
the ACK at fcmpushclient.py:605-608 -> Google redelivers the same message on
every reconnect and the client dies within a second, so no doorbell call ever
reaches Home Assistant.
_patch_push_headers now rewrites both headers in app_data before the library
reads them: the segment is selected by label, so its position does not matter,
and then padded. The library's positional strip lands exactly on dh and salt. A
value that arrives without its label is given one back, since the blind strip
would otherwise eat real key bytes. Unrecognised shapes are passed through
untouched. No cryptography, call order or protocol state is involved; the patch
is idempotent and becomes a no-op once upstream parses the parameter list.
Upstream does not cover this: PR #37 pads and isolates per message, but its
author explicitly dropped ';'-parameter parsing as speculative, and issue #42
proposes removeprefix("dh="), which still leaves the '; p256ecdsa=...' tail. PR
#37's isolation would stop the crash while ACKing the message - the call would
be lost silently instead of loudly.
Verified in production 2026-08-13: listener up at 10:38:36 and still alive with
no decryption error, where every previous start died after ~0.6s.
Tests: 616 passed; 614 passed + 2 skipped with the dependency hidden, as in CI.
The regression test drives real http_ece crypto through the exact production
header shape and asserts both historical errors plus the fix.
Refs A-80/A-86, issue #77, upstream sdb9696/firebase-messaging#37 and #42.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The previous commit fixed the wrong defect. Its premise - that Web Push always
sends crypto-key and salt unpadded, so every encrypted push fails - is falsified
by evidence: _decrypt_raw_data is byte-identical across firebase-messaging
0.4.0-0.4.5, and on 2026-06-22 that same code decrypted 16 real CALL_INCOMING
pushes from this operator with no error at all
(research/intercom-call-probe/logs/fcm.log). Padding alone also moved the
production failure rather than removing it: binascii.Error became
ValueError: Invalid EC key.
A DIAG probe on production captured the real shapes:
crypto-key: len=189 shape=dh=87, p256ecdsa=87
encryption: len=29 shape=salt=24
Crypto-Key and Encryption are parameter lists separated by ';' (RFC 8188 2.1,
RFC 8291 4). The operator now signs pushes with VAPID, so crypto-key carries a
second p256ecdsa segment, while the library strips the prefix by position -
[3:] for dh=, [5:] for salt= (fcmpushclient.py:425-426). The remainder is
'<dh>; p256ecdsa=<...>'; the base64 decoder silently drops ';', the space and
the label letters, yielding 137 bytes where a P-256 point needs 65. Salt already
arrives padded (24 chars) and dh does not (87), so padding is neither necessary
nor sufficient - dh only decodes once separated from the VAPID segment.
Causal chain: operator adds VAPID -> positional strip leaves a trailing segment
-> base64 decodes a 137-byte soup -> from_encoded_point rejects it -> the
exception escapes _handle_data_message -> _listen shuts the client down before
the ACK at fcmpushclient.py:605-608 -> Google redelivers the same message on
every reconnect and the client dies within a second, so no doorbell call ever
reaches Home Assistant.
_patch_push_headers now rewrites both headers in app_data before the library
reads them: the segment is selected by label, so its position does not matter,
and then padded. The library's positional strip lands exactly on dh and salt. A
value that arrives without its label is given one back, since the blind strip
would otherwise eat real key bytes. Unrecognised shapes are passed through
untouched. No cryptography, call order or protocol state is involved; the patch
is idempotent and becomes a no-op once upstream parses the parameter list.
Upstream does not cover this: PR #37 pads and isolates per message, but its
author explicitly dropped ';'-parameter parsing as speculative, and issue #42
proposes removeprefix("dh="), which still leaves the '; p256ecdsa=...' tail. PR
#37's isolation would stop the crash while ACKing the message - the call would
be lost silently instead of loudly.
Verified in production 2026-08-13: listener up at 10:38:36 and still alive with
no decryption error, where every previous start died after ~0.6s.
http_ece is imported inside the crypto test rather than at module scope: it
arrives transitively with firebase-messaging, which CI does not install, and a
module-level import took the whole test_fcm module down with ModuleNotFoundError
instead of skipping one test.
Tests: 616 passed; 614 passed + 2 skipped with firebase-messaging and http_ece
both hidden, matching what CI installs. The regression test drives real http_ece
crypto through the exact production header shape and asserts both historical
errors plus the fix.
Refs A-80/A-86, issue #77, upstream sdb9696/firebase-messaging#37 and #42.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Doorbell calls stopped reaching Home Assistant entirely. firebase-messaging
0.4.5 tore down FcmPushClient on every encrypted push, first with
binascii.Error: Incorrect padding and, once padding was restored, with
ValueError: Invalid EC key.
A DIAG probe on production captured the real shapes:
crypto-key: len=189 shape=dh=87, p256ecdsa=87
encryption: len=29 shape=salt=24
Crypto-Key and Encryption are parameter lists separated by ';' (RFC 8188 2.1,
RFC 8291 4). The operator now signs pushes with VAPID, so crypto-key carries a
second p256ecdsa segment, while the library strips the prefix by position -
[3:] for dh=, [5:] for salt= (fcmpushclient.py:425-426). The remainder is
'<dh>; p256ecdsa=<...>'; the base64 decoder silently drops ';', the space and
the label letters, yielding 137 bytes where a P-256 point needs 65.
Padding alone is neither necessary nor sufficient, which is why treating this as
a padding bug only moved the error: salt already arrives padded (24 chars) and
dh does not (87), and dh only decodes once separated from the VAPID segment.
Nor is it a library regression - _decrypt_raw_data is byte-identical across
0.4.0-0.4.5, and on 2026-06-22 that same code decrypted 16 real CALL_INCOMING
pushes from this operator (research/intercom-call-probe/logs/fcm.log), back when
crypto-key still carried only the dh segment.
Causal chain: operator adds VAPID -> positional strip leaves a trailing segment
-> base64 decodes a 137-byte soup -> from_encoded_point rejects it -> the
exception escapes _handle_data_message -> _listen shuts the client down before
the ACK at fcmpushclient.py:605-608 -> Google redelivers the same message on
every reconnect and the client dies within a second.
_patch_push_headers rewrites both headers in app_data before the library reads
them: the segment is selected by label, so its position does not matter, and
then padded. The library's positional strip then lands exactly on dh and salt. A
value that arrives without its label is given one back, since the blind strip
would otherwise eat real key bytes. Unrecognised shapes are passed through
untouched. No cryptography, call order or protocol state is involved; the patch
is idempotent and becomes a no-op once upstream parses the parameter list.
Upstream does not cover this: PR #37 pads and isolates per message, but its
author explicitly dropped ';'-parameter parsing as speculative, and issue #42
proposes removeprefix("dh="), which still leaves the '; p256ecdsa=...' tail. PR
would be lost silently instead of loudly.
http_ece is imported inside the crypto test rather than at module scope: it
arrives transitively with firebase-messaging, which CI does not install, and a
module-level import took the whole test_fcm module down with ModuleNotFoundError
instead of skipping one test.
Verified in production 2026-08-13: the listener survives, and a real doorbell
press reached Home Assistant.
Tests: 616 passed; 614 passed + 2 skipped with firebase-messaging and http_ece
both hidden, matching what CI installs. The regression test drives real http_ece
crypto through the exact production header shape and asserts both historical
errors plus the fix.
Refs A-80/A-86, issue #77, upstream sdb9696/firebase-messaging#37 and #42.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
|
Re the open question in your Notes:
They are in the wild. Different downstream (a Home Assistant custom integration for a Russian intercom operator), same library. The operator started signing pushes with VAPID and Worth noting for this PR specifically: Your per-message isolation is still valuable on its own, but one caveat: the ACK at I opened #43 with the parameter-by-name lookup, deliberately orthogonal to this PR — |
…itHub #68) Beta-1 (this same session) fixed the FCM supervisor's registration-storm symptom (escalating backoff on repeated confirmed hard-heal failure). This follow-up fixes the actual root cause behind the reporter's original complaint: the pinned firebase-messaging==0.4.5 library's FcmPushClient._decrypt_raw_data decodes the push message's crypto-key/salt headers with plain base64.urlsafe_b64decode, which raises binascii.Error on the RFC-8291-legal *unpadded* base64 those headers actually arrive as. This integration's own earlier fix (issue #65) only caught that error and silently dropped the message, falling back to the coordinator's ~15-90s polling path to eventually discover the same event — reproduced live on our own test HA within the hour as a measured 2:59 delay on a real motion event, matching the "always late" report. Ports the fix from a real, open, unmerged upstream PR (sdb9696/firebase-messaging#37) that pads correctly so decryption actually succeeds instead of failing. Deliberately decoupled from the pre-existing issue #33 _listen fix via a new _build_decrypt_raw_data_override() helper, so a future unrelated library signature change degrades only this improvement, never the foundational one. THREE_PER_ISSUE_PER_CHANGE 3-agent adversarial bug-hunt (all three independently converging on the same finding) caught a real regression in the first version of this fix: once headers decode successfully, a message whose ciphertext body is genuinely corrupt (a stale key after a registration rotation, a message meant for a different subtype) now reaches http_ece.decrypt() and raises ECEException instead of binascii.Error — previously unreachable, since every message failed earlier at the header-padding step first — which would have escaped the existing narrow catch and torn down the whole FCM client over one bad payload. Fixed by widening the catch alongside the padding fix, with a regression test. Deploy-verified live on test HA: a real push arrived with last_push_seconds_ago=0 immediately after restart, zero bosch_shc_camera exceptions. 6717 pytest / mypy --strict / ruff / codespell clean, 100% coverage.
|
Hi @sdb9696, friendly ping to see if you might have any time to review or merge this PR (along with #38 and #39) and cut a new release when convenient. Several downstream Home Assistant integrations using |
…vis#373) One push message the library cannot decrypt was enough to end real-time push permanently, and silently. `_decrypt_raw_data` decodes the `crypto-key` and `encryption` header values without padding, while padding the two stored key values in the same function. Those headers are URL-safe base64 that may legitimately arrive without trailing `=`, so an unpadded one raises `binascii.Error`. That is a `ValueError`, so the listen loop's `except (OSError, EOFError)` misses it, it reaches the outer `except Exception`, and the client shuts down. The severe part is where it raises: before the library appends the persistent id and sends the selective ack. The message is therefore never acknowledged, so it is redelivered on the next connection — and the supervision from bvis#285 faithfully restarts straight back into it. The reporter measured the same message killing the client 16 times over 3.5 hours, each death 3-9 ms after receiving it, surviving a host reboot because the queue is server-side. Three changes: - `install_fcm_decrypt_guard` pads both header values before delegating, and contains any remaining decode failure by returning empty bytes. The handler then stays on its normal path and reaches the acknowledgement, so one event is lost instead of every future one. - Repeated deaths on the same persistent_id raise a Repair. The failure is otherwise invisible: alarm state comes from polling and HTS, never from push, so nothing looks wrong from the outside. - At that same threshold the stored FCM registration is discarded and renewed once, which is what the reporter did by hand. Without it the only recovery available to a user is editing `.storage`. Upstream fixes the root cause in sdb9696/firebase-messaging#37, open and mergeable since June with no release carrying it; the guard goes away when a release ships it. Also fixes two things found on the way: `test_start_without_firebase_messaging` simulated an absent package by nulling only the parent module, which stops working once anything imports a submodule, and `strings.json` was missing the `fcm_not_configured` entry that `translations/en.json` already had. Refs bvis#373, bvis#359
_handle_data_message reads crypto-key, encryption and subtype without do_not_raise, so a message lacking any of them raises RuntimeError straight into _listen, which shuts the client down. The message is never acked, so it is redelivered on every reconnect -- the same permanent wedge this patch module exists to prevent, reached by a different door. Degrade a missing one of those three to an empty value. Decryption then fails for that message alone, so it is skipped and acked while the connection survives. Headers outside that set still raise, so unrelated faults are not swallowed. Found while reviewing upstream PRs sdb9696/firebase-messaging#37 and #43, which together fix the header parsing and padding but call the header extraction outside the try/except that makes decrypt failures non-fatal. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
|
Can confirm this fixes the same crash for me. I hand-patched these exact three changes (padded-b64 helper, applied to all four decode sites, try/except around |
Summary
Hardens the path that decodes incoming encrypted FCM data messages so a single malformed payload no longer crashes the listener.
In the wild (Home Assistant's Ring integration) the push listener was dying outright while decrypting a message:
(4× over 2026-03-04 … 03-06; each time push stopped until the integration was reloaded.) Root causes:
crypto-key/encryptionheader values and the stored private/secret keys are URL-safe base64 that may arrive without trailing=padding (webpush keys per RFC 8291), which makesurlsafe_b64decoderaisebinascii.Error.Changes
_urlsafe_b64decode_padded()and use it for the four base64 fields in_decrypt_raw_data, so missing=padding is tolerated.try/except ValueError(binascii.Erroris aValueErrorsubclass): log a warning and skip that single payload instead of tearing down the listener.Testing
uv run pytest tests/test_fcmpushclient.py– all green, incl. a new regression test decoding a deliberately unpaddedcrypto-key/salt.ruff check/ruff format --checkclean.Notes
Kept intentionally narrow. An earlier local version also parsed
;-separated header parameters (dh=…;keyid=…) and validated the DH key shape (65-byte0x04prefix), but I never actually saw;-parameters in my own captured headers, so I dropped both as speculative — happy to add back if you've seen such headers in the wild.This is the first of two independent PRs; the second (#38) addresses transient read/connection recovery. They touch different code paths and can be reviewed/merged separately.
🤖 Generated with Claude Code