Skip to content

Emit zone-aware UTC timestamp with real sub-second (not naive local + fake .000000)#14

Open
ACETyr wants to merge 2 commits into
agessaman:mqtt-bridge-implementationfrom
ACETyr:fix/mqtt-observer-utc-timestamp
Open

Emit zone-aware UTC timestamp with real sub-second (not naive local + fake .000000)#14
ACETyr wants to merge 2 commits into
agessaman:mqtt-bridge-implementationfrom
ACETyr:fix/mqtt-observer-utc-timestamp

Conversation

@ACETyr

@ACETyr ACETyr commented Jun 14, 2026

Copy link
Copy Markdown

What

Make the MQTT observer's packet/raw timestamp a zone-aware UTC value with a real sub-second,
instead of naive local time stamped at publish with a hardcoded .000000.

src/helpers/MQTTMessageBuilder.cpp (all three builders): format from gmtime() with an explicit Z
suffix, and take the fractional second from gettimeofday() (genuine microseconds, SNTP-maintained on
ESP32) rather than the literal .000000.

before:  2026-06-14T17:29:29.000000      (naive local, fake sub-second)
after:   2026-06-14T15:29:29.123456Z     (UTC, real microseconds)

Why

The timestamp was built with localtime(timezone ? timezone->toLocal(now) : now) → a zone-less
string, and time(nullptr) → whole seconds with a literal .000000.

Verified against a representative consumer's ingest code (CoreScope,
github.com/Kpa-clawbot/CoreScope, cmd/ingestor/main.go resolveRxTime):

  • A naive (zone-less) timestamp more than 15 min off the server clock is discarded and replaced
    with ingest time
    , and the observer is flagged clock-skewed. So an observer that emits local time
    (e.g. UTC+2) has every packet treated as ~2 h in the future and clamped — emitting local time
    actively degrades ingest. A Z-suffixed UTC value is trusted (even for buffered/late uploads).
  • The .000000 is fabricated (there is no sub-second source today). That's misleading. CoreScope
    truncates any fraction to whole seconds — but that's its current choice; consumers self-logging and
    correlating directly off the MQTT feed (single-observer ordering, inter-arrival, airtime/collision
    windows) genuinely benefit from a real sub-second, so this PR provides one rather than dropping it.

Measured impact context: across the public observer fleet, ~27% of observers already trip the
naive-clamp (local-time publishers); this change keeps an observer on the trusted path.

Testing

  • Compiles clean: pio run -e Heltec_v3_repeater_observer_mqtt → SUCCESS (ESP32-S3 image built).
  • Not yet hardware-validated by me on a live broker; behavior is a localized formatting change.

Scope / possible follow-ups (not in this PR)

  • The status message timestamp is formatted separately in MQTTBridge.cpp (publishStatus) — same
    treatment should apply there for consistency; happy to include if preferred.
  • RX-time vs publish-time: this stamps publish time. QueuedPacket already captures a millis()
    at reception; threading a captured wall-clock from there would make the timestamp the true receive
    instant even under queue backlog. Larger change — left as a follow-up.

… fake .000000)

The packet/raw `timestamp` was built from localtime() (naive, no zone) at publish
time with a hardcoded ".000000" sub-second. Aggregators that assume UTC (e.g.
CoreScope) clamp a naive value landing >15 min off their clock to ingest time and
flag the observer as clock-skewed — so emitting local time actively degrades ingest.

Now format from gmtime() with an explicit Z, and use gettimeofday() for a genuine
microsecond fraction (SNTP-maintained on ESP32) instead of the literal ".000000".
Real sub-second helps consumers correlating off the raw MQTT feed; second-resolution
consumers (CoreScope) simply truncate it.

Note: this stamps publish time; threading the queued RX capture (QueuedPacket has a
millis() timestamp) for true receive time is a worthwhile follow-up.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
ACETyr added a commit to ACETyr/MeshCore that referenced this pull request Jun 14, 2026
PR agessaman#14 fixed the packet timestamp, but the two status-message builders in
MQTTBridge.cpp still formatted a hardcoded ".000000" via getLocalTime().
Use the same gettimeofday() + gmtime() + ".%06ldZ" pattern so the status
timestamp is unambiguous UTC with a real sub-second, consistent with the
packet builder (and not naive local, which a CoreScope-style aggregator
would clamp/flag).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
PR agessaman#14 fixed the packet timestamp, but the two status-message builders in
MQTTBridge.cpp still formatted a hardcoded ".000000" via getLocalTime().
Use the same gettimeofday() + gmtime() + ".%06ldZ" pattern so the status
timestamp is unambiguous UTC with a real sub-second, consistent with the
packet builder (and not naive local, which a CoreScope-style aggregator
would clamp/flag).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@ACETyr

ACETyr commented Jun 14, 2026

Copy link
Copy Markdown
Author

Added the status-message timestamp fix per the "possible follow-ups" note above. Both status builders in MQTTBridge.cpp (publishStatus + the analyzer status path) now emit zone-aware UTC with an explicit Z and a real gettimeofday() sub-second, consistent with the packet builder — previously they formatted a hardcoded .000000 via getLocalTime() (same naive-local issue this PR fixes for packets). Builds clean (pio run -e Heltec_v3_repeater_observer_mqtt).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant