Skip to content

Stop one unformattable number from destroying the whole message - #37

Merged
matuszeg merged 1 commit into
mainfrom
fix-nonfinite-timestamps
Aug 24, 2026
Merged

matuszeg merged 1 commit into
mainfrom
fix-nonfinite-timestamps

Conversation

@matuszeg

Copy link
Copy Markdown
Owner

Fixes the parse failures in #27, which turned out not to be truncation.

Every one of the 892 failures in the log buffer was identical, down to the
column: Expecting ',' delimiter: line 1 column 47 (char 46). That offset is
the first byte of the utcunixtime value in a /pings body, and substituting
1.#INF00 for the timestamp reproduces the error exactly. So do 1.#QNAN0 and
-1.#IND00. Those are what a C printf("%f") writes for a value that isn't
finite, so the device appears to format its clock out even when it doesn't have
one.

A constant offset is the part that settles it. A body cut short mid-flight,
which is what this looked like, would fail somewhere different every time.

The message itself is fine apart from that one field, but json.loads gives up
on all of it, so the readings alongside are thrown away too. For /pings that's
the signal strength, and it's the only field in there we read. The same thing in
a /bbs_json message would take the battery voltage and any pump run with it.
There's no recorded instance of that yet, which is luck.

What changed

parse_request retries once with those renderings replaced by null, and only
on a body that has already failed to parse. The healthy path never sees the
substitution, so there's no way for it to damage a message that was fine. A
repaired message logs at debug. One that's still unreadable keeps the warning
and the traceback it always had, which is now rare enough to deserve one.

Running it over the raw bytes also reaches the second JSON document /bbs_json
keeps inside a string, because none of these characters need escaping there.

_device_time already tolerated an unreadable timestamp; it just never got the
chance, because the decode failed first. Now it does. One related fix came with
it: a run whose current came through as null no longer counts as evidence the
primary pump is working.

Verification

Six new tests, including the exact byte sequence from production, which
reproduces the live error string character for character.

Then replayed at a real listener on the dev instance rather than trusted to the
tests, since this sits in the request path:

DEBUG ... core.parser] /pings carried a non-finite number, read without it
INFO  ... discovered device 99999999999999

No warning, and the reading was recorded even though the forward upstream
failed, which is the condition these arrive under most.

229 tests pass.

Every parse failure recorded on the live install turned out to be the same
thing, byte for byte: the device writes utcunixtime as 1.#INF00, which is what
a C printf("%f") produces for a value that isn't finite. That isn't JSON, so
the decoder abandons the entire body rather than the one bad field, and every
reading beside it goes too. For a /pings message that's the signal strength,
which is the only thing in there we read at all.

There were 892 of them sitting in the log buffer, each with a full traceback,
all reporting the same column. The constant column is what gave it away. A
truncated body, which is what this looked like, would fail somewhere different
every time.

parse_request now retries once with those renderings replaced by null, and only
for a body that has already failed. The healthy path is untouched, so the
repair can't mangle good data. A message that comes back clean is logged at
debug; one that's still unreadable keeps the warning and the traceback it
always had.

The substitution runs over the raw bytes, so it also reaches the second JSON
document /bbs_json hides inside a string. None of these characters need
escaping there, so they appear verbatim.

A run whose current came through as null no longer counts as evidence that the
primary pump is healthy.

Verified by replaying the bytes at a real listener rather than only in tests:
the reading is recorded and the warning is gone, including when the forward
upstream fails, which is when this happens most.

Refs #27.
@matuszeg
matuszeg merged commit 28d2b28 into main Aug 24, 2026
3 checks passed
@matuszeg
matuszeg deleted the fix-nonfinite-timestamps branch August 24, 2026 23:10
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant