Skip to content

feat(telemetry): add messageId and nudgeSentAtMs to nudge event for TLON-5728#155

Merged
patosullivan merged 2 commits into
masterfrom
po/tlon-5728-nudge-notification-tracking
Jun 4, 2026
Merged

feat(telemetry): add messageId and nudgeSentAtMs to nudge event for TLON-5728#155
patosullivan merged 2 commits into
masterfrom
po/tlon-5728-nudge-notification-tracking

Conversation

@patosullivan

@patosullivan patosullivan commented May 21, 2026

Copy link
Copy Markdown
Member

The send-side half of the cross-repo nudge → tap funnel. Adds the join key
(and a sibling timestamp) to the existing TlonBot Heartbeat Nudge Sent
event so it can be correlated, in PostHog, with the mobile
Tapped DM Push Notification event Homestead now emits.

Why

We want to know whether bot nudge DMs actually bring users back into the
app. Homestead emits a Tapped DM Push Notification event when a user taps
a DM push; PostHog joins it to this repo's TlonBot Heartbeat Nudge Sent
event on messageId to compute a send-to-tap conversion rate.

Today the send event carries no messageId, so there is nothing to join
on. This PR adds it. The change is additive — same event name, new
fields — so it is safe to land independently of the Homestead PR.

What changed

sendDmWithStory returns its sentAt

  • src/urbit/send.ts — export formatSentAt; both return branches of
    sendDmWithStory (and therefore sendDm) now return
    { channel, messageId, sentAt }.

Nudge runner threads the canonical id + timestamp

  • src/monitor/nudge-runner.ts — tighten NudgeRunnerDeps.sendDm to the
    concrete return shape; capture messageId / sentAt from the send
    result; pass them into captureHeartbeatNudge; use the canonical
    sentAt for the success-path setLocalPendingNudge write so the pending
    record agrees with the telemetry event. On send failure both stay null.

Telemetry event gains two fields

  • src/telemetry.ts — extend TlonHeartbeatNudgeEvent with
    messageId: string | null and nudgeSentAtMs: number | null, and write
    both into the PostHog event properties. Event name is unchanged.

Tests

  • src/urbit/send.test.ts — assert the new sentAt / channel return
    fields (existing @urbit/aura mock kept).
  • src/urbit/send.fixtures.test.ts (new) — cross-repo fixture canary that
    pins formatSentAt output to the exact messageId strings Homestead's
    getIdParts inverts. Deliberately has no top-level @urbit/aura
    mock so it exercises the real encoding pipeline.
  • src/monitor/nudge-runner.test.ts, src/telemetry.test.ts — assert the
    new fields on a successful send and null on failure.

Dev tooling

  • dev/trigger-nudge.sh (new) — primes the bot's %settings
    (lastOwnerMessageAt, clears lastNudgeStage, 24h UTC active hours) so
    the next runner tick fires a nudge; used to drive the end-to-end PostHog
    preflight. --reset clears the primed settings.

Testing

  • pnpm test — full suite passes (35 files / 541 tests).
  • node ./node_modules/typescript/bin/tsc --noEmit — passes.
  • Cross-repo canary green on both sides: this repo's send.fixtures.test.ts
    and Homestead's dmTapTelemetryRoundTrip.test.ts pin the same
    (sentAt, messageId) fixtures.
  • End-to-end verified on Android against the Tlon Staging PostHog project:
    a nudge send joined to a mobile DM tap on messageId
    (sends = 1 / converted_sends = 1).

Notes

@linear

linear Bot commented May 21, 2026

Copy link
Copy Markdown

TLON-5728

@patosullivan patosullivan marked this pull request as ready for review June 1, 2026 19:12

@claude claude Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Claude Code Review

This repository is configured for manual code reviews. Comment @claude review to trigger a review and subscribe this PR to future pushes, or @claude review once for a one-time review.

Tip: disable this comment in your organization's Code Review settings.

@patosullivan

Copy link
Copy Markdown
Member Author

@claude review once

@claude claude Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM — clean additive telemetry change with strong test coverage (cross-repo fixture pinning, null-on-failure assertions) and end-to-end verification against staging.

Extended reasoning...

Overview

This PR adds two fields (messageId, nudgeSentAtMs) to the existing TlonBot Heartbeat Nudge Sent PostHog event so it can be joined to Homestead's Tapped DM Push Notification event for a send-to-tap funnel (TLON-5728). sendDmWithStory now returns the sentAt it already used internally to build the messageId; the nudge runner threads that through into both the telemetry call and the setLocalPendingNudge write. formatSentAt is exported so a new fixture test can pin the encoding cross-repo against Homestead's inverse. A dev-only trigger-nudge.sh script is added for preflight testing.

Security risks

None apparent. This is observability-only: PostHog properties carry a message id and a unix-ms timestamp — no secrets, no auth, no permissions, no user input crossing boundaries. The new dev script only mutates the bot's own %settings and explicitly only runs from a developer's host shell.

Level of scrutiny

Low. The change is mechanical and additive: existing event name, new optional-shaped fields, both nullable on send failure. The send path's behavior is unchanged (the returned object just gains a field). The runner's only behavioral shift is using the canonical sentAt for the pending-nudge sentAt write instead of a separate now() call — which is the intended correctness improvement (one timestamp, two consumers).

Other factors

  • Test coverage is thorough: explicit assertions for both success (messageId+timestamp populated) and failure (both null), plus a cross-repo fixture canary that exercises the real @urbit/aura encoding pipeline rather than the mock.
  • The fixture file deliberately has no top-level vi.mock('@urbit/aura') and the comment explains why — that's exactly the right discipline for a cross-repo pin.
  • End-to-end verified on Android against the Tlon Staging PostHog project per the PR description.
  • The localSentAt fallback in the runner is dead-but-defensive (only fires if send 'succeeded' but result is missing, which shouldn't happen given the typed shape) — fine to keep as belt-and-braces.

@patosullivan patosullivan requested a review from arthyn June 1, 2026 20:07

@arthyn arthyn left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm

@patosullivan patosullivan merged commit 9871a16 into master Jun 4, 2026
3 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants