fix: ADS-B response truncation and radar longitude projection - #69
fix: ADS-B response truncation and radar longitude projection#69davidjconnolly wants to merge 2 commits into
Conversation
Roughly 90% of ADS-B fetches failed with "JSON parse error: InvalidInput" on the ESP32-C3 Super Mini. The response body was buffered whole into a String before parsing. A busy sector returns ~21 kB, but on-device diagnostics showed only ~30 kB of free heap with a largest contiguous block of ~9 kB, so the reserve() and the subsequent concat() calls failed. The socket delivered all ~21 kB, yet the String only ever held 10-14 kB. Because payload.length() could then never reach content_length, the read loop also spun out its full 10 s timeout on every fetch before handing the truncated body to the parser. Parse straight off the response stream instead, one aircraft at a time, with an ArduinoJson filter limiting the document to the fields the radar renders. Peak memory drops from ~21 kB contiguous to a few hundred bytes, so a fragmented heap no longer matters. A PollingStream wrapper preserves the existing behaviour of servicing the network poll hook during blocking reads and bounds the transfer with a single deadline. Verified on hardware: 25 consecutive fetches, 0 parse errors, each now completing at the 3 s poll interval rather than stalling for 10 s. Also drop the `namespace fonts = lgfx::v1::fonts;` aliases, which collide with the namespace LovyanGFX 1.2.26 exposes at global scope and broke the build before anything could be flashed. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
offsetKmFromCenter() converted longitude degrees to km with a flat 111 km/°, applying no cos(latitude) correction. Away from the equator that overstates every east-west offset — ~38% at 43°N. Two visible consequences at short ranges: * Aircraft well inside the ring were measured as beyond it and demoted from a full symbol + callsign/type/altitude tag to a 2 px rim dot. On hardware at the 10 km preset every aircraft collapsed this way (n=4 inside=0 dots=4), which reads as "the planes disappeared". * Projected x could land off the 240 px panel entirely (x=250, x=251). The 25 km preset masked it: its 33 km outer radius is wide enough that even inflated distances still fall inside the ring. Verified on an ESP32-C3 Super Mini at Toronto latitude: aircraft that previously rendered as rim dots now project inside the ring (n=8 inside=7 dots=1), and off-panel x coordinates are gone. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
davidjconnolly
changed the title
|
Hardware & testing methodology: ESP32-C3 Super Mini ( Results: merged cleanly, no conflicts. Ran 30+ minutes cumulative across sessions with steady |
2 participants
Two independent bugs found while bringing this up on an ESP32-C3 Super Mini with the 1.28" round GC9A01, plus the build fix needed to flash anything at all. Each is a separate commit.
1. ~90% of ADS-B fetches fail with a JSON parse error
Root cause
fetchUpdate()buffered the whole response body into aStringbefore parsing. Instrumenting the read loop on hardware:read=21040— the socket delivers the full body and it starts as valid JSON. Not a transport, TLS, or API problem, and not chunked encoding (te=[]).heap=30788/maxblk=9204— ~30 kB free heap, but the largest contiguous block is ~9 kB.payload.reserve(21041)needs one contiguous 21 kB allocation and fails.concat_fail=1— the followingString::concat()calls fail, so the buffer tops out at 10–14 kB anddeserializeJson()gets a truncated body.why=deadline ms=10001— a second bug falls out of the first:payload.length()can never reachcontent_length, so the loop never takes its completion branch and burns the full 10 s timeout on every fetch.This is heap fragmentation, not payload size as such — there is enough free heap for 21 kB, just never in one piece. Responses small enough to fit still succeed, which is the surviving ~10%.
Fix
Parse directly off the response stream, one aircraft at a time, with a
DeserializationOption::Filterrestricting the document to the fields the radar renders. Peak memory drops from ~21 kB contiguous to a few hundred bytes. A smallPollingStreampreserves the existing network-poll-hook behaviour during blocking reads and bounds the transfer with one deadline. Parsing stops early oncekMaxAircraftis reached.Verification
Before — ~90% failure. After — 25 consecutive fetches, 0 parse errors, each completing at the real 3 s poll interval instead of stalling 10 s:
2. Aircraft vanish at the 5 km and 10 km presets
Everything looked frozen at short ranges, while the 25 km preset worked fine.
Root cause
offsetKmFromCenter()converts longitude degrees to km with a flat111 km/°and nocos(latitude)correction, so every east-west offset is overstated — ~38% at 43°N.On hardware at the 10 km preset, radar centred at
43.6485,-79.4795:Aircraft genuinely inside the ring measure as beyond it and are demoted from a full symbol + callsign/type/altitude tag to a 2 px rim dot —
inside=0, i.e. every aircraft. That reads as "the planes disappeared". Projected x can also land off-panel entirely. The 25 km preset masks it because its 33 km outer radius is wide enough that even inflated distances still fall inside.Fix
Scale the longitude delta by
cos(centre latitude).Verification
Same location and traffic, after the fix:
Aircraft that previously rendered as rim dots now project inside the ring, and off-panel x coordinates are gone.
3. Build fix (required to flash anything)
LovyanGFX resolves to 1.2.26 under
@^1.2.7and now exposesfontsat global scope, so the three local aliases collide:Removed the three
namespace fonts = lgfx::v1::fonts;aliases;fonts::already resolves without them. Pinning the library version instead would also work if you'd prefer that — happy to switch.🤖 Generated with Claude Code