perf(contacts): coalesce per-contact-frame work during a sync

[andyshinn]

Connecting to a radio had become slow — the renderer sat at ~157% CPU while contacts loaded.

Root cause

The library emitted a full contacts and a full discovered snapshot on every RESP_CONTACT frame, and coresense responded to each one by doing whole-collection work. A sync was quadratic in sqlite statements, bytes on the wire, and renderer re-renders.

The coresense-side regression was cab268c, which changed the discovered handler from a pure re-read into a per-row setOnRadio + setFavourite write-through. Correct fix for a real stale-flag bug; it just became quadratic against the per-frame emit cadence.

Measured, N=300 through the real adapter → sqlite → bus path

	before	after
main-process wall clock	1,930 ms	203 ms
discovered broadcasts	601	2
contacts broadcasts	301	1
discovered_contacts writes	91,200	600
conversations_fts writes	45,450	300
JSON pushed to renderer	37.55 MB	0.16 MB

Re-measured with realistic pacing (300 contacts over ~15 s, one frame per ~50 ms, which is what BLE actually does): 17 broadcasts / 0.86 MB. Broadcast count under a coalescer is burst_duration ÷ interval, so the single-tick number above flatters it — the paced figure is the honest one.

Changes

• coalesce helper — leading edge, then at most one run per interval, with flush/cancel. Applied to the contacts and discovered broadcasts, the dock-badge recompute, and holder persistence. Interval 1 s.
• discoveredStore.applyRadioFlags — one transaction, skips rows whose flags already match. Replaces two unbatched UPDATEs per row per frame. Every other write path invalidates the cache.
• holder.setContacts/setChannels no longer rewrite the whole JSON file and DROP+rebuild conversations_fts on every call; flushed on quit.
• Memoised the discovered-name index on array identity. It was rebuilt once per rendered message row per websocket message — the main source of renderer CPU.

State stays synchronous throughout; only broadcasts and derived persistence are coalesced, so anything reading the holder or sqlite sees the latest value immediately.

Also here

• meshcore-ts 0.7.0, which coalesces the snapshots library-side too. Both layers are needed: contactObserved still fires per contact by design, and coresense projects the whole pool per call.
• Channel-relay attribution fix. Crossing 0.6.0 activated the matcher rewrite, which attributes a heard relay by the timestamp sealed in the packet. sendMessage never passed it, so attribution silently fell back to a newest-first guess — the ✓×N chip landing on the wrong message with two sends in flight. The test's fixture was stale in the same way (it fed the library deadbeef as ciphertext, which 0.6.0+ rejects outright); it now builds a genuinely encrypted GRP_TXT payload, plus a case that fails without the timestamp.
• Sync is now verifiable. Nothing on this path was logged at all — coresense logged nothing, and MeshCoreSession was built without a logger, so the library discarded its own iterator counts. Now: the library's logger is wired in, contactsSynced produces a summary line comparing delivered vs stored (warning when short), and a per-contact ingest line sits at trace.

Verification

972 tests (10 new), typecheck and lint clean. Written test-first — each assertion was watched failing with the real quadratic numbers (81 emits, 1,760 SQL writes, 860 FTS inserts at N=40) before implementing. Before/after figures come from the same harness with src/ stashed to HEAD.

Not verified on real hardware.

Known limitation

timestampUnix is second-granularity, so two channel sends inside the same second remain indistinguishable and fall back to newest-first. That is the library's documented behaviour.

🤖 Generated with Claude Code

[Copilot]

Pull request overview

This PR reduces contact-sync CPU, sqlite write amplification, and renderer re-rendering by coalescing per-contact-frame work into bounded-rate broadcasts/persistence, batching discovered flag write-through, and memoizing an expensive renderer index. It also updates meshcore-ts and fixes channel relay attribution by passing the encrypted packet timestamp through send registration, while adding logging/summary signals to make sync outcomes verifiable.

Changes:

• Introduces a coalesce helper and applies it to contacts/discovered broadcasts, dock badge recompute, and holder persistence (contacts/channels JSON + conversations FTS rebuild).
• Adds discoveredStore.applyRadioFlags to batch and skip redundant flag writes, plus sync-summary emission/logging and test harness support for counting sqlite writes.
• Memoizes the renderer’s discovered-name index on array identity; bumps @andyshinn/meshcore-ts to 0.7.0 and updates relay attribution tests/fixtures.

Reviewed changes

Copilot reviewed 25 out of 26 changed files in this pull request and generated 2 comments.

Show a summary per file

File	Description
tests/unit/renderer/lib/identity-index-cache.test.ts	Adds unit coverage for the discovered-name index memoization behavior.
tests/unit/notifications/badge-coalescing.test.ts	Verifies dock badge recompute is coalesced for bursty events.
tests/unit/main/coalesce.test.ts	Adds unit tests for the new coalescing helper contract (leading/trailing/flush/cancel).
tests/support/sqlite-temp.ts	Resets discovered-flag module cache when swapping temp userData DBs in tests.
tests/support/sql-counter.ts	Adds helper to count executed sqlite write statements for perf/regression tests.
tests/integration/storage/discovered-flags.test.ts	Adds integration tests for applyRadioFlags batching/skip/invalidation behavior.
tests/integration/outbound/channel-relay-ack.test.ts	Updates relay-ack fixture to real decryptable payload + asserts timestamp-based attribution.
tests/integration/inbound/contacts-iterator.test.ts	Updates contacts iterator test to flush coalesced broadcasts before asserting.
tests/integration/adapter/sync-summary.test.ts	Adds coverage for contact sync summary signal emitted post-sync.
tests/integration/adapter/sync-coalescing.test.ts	Pins “sync work is O(N), not O(N²)” via bounded emit/write assertions.
src/renderer/shell/rightrail/sections/peopleModel.ts	Switches to memoized discovered-name index for roster row building.
src/renderer/lib/identity.ts	Adds discoveredNameIndex memoized on discovered array identity.
src/renderer/hooks/useIdentityHash.ts	Uses discoveredNameIndex to avoid rebuilding name index per render.
src/main/storage/discoveredContacts.ts	Implements applyRadioFlags batching + per-pubkey flag cache invalidation.
src/main/state/holder.ts	Coalesces contacts/channels persistence and conversations FTS rebuild; adds flush helper.
src/main/state/contactSync.ts	Coalesces contacts/discovered broadcasts; adds trace logging + flush helper for tests.
src/main/protocol/sessionAdapter.ts	Wires meshcore-ts logger into coresense logging.
src/main/protocol/adapterEvents.ts	Uses applyRadioFlags + coalesced discovered emit; emits sync summary on contactsSynced.
src/main/notifications/index.ts	Routes badge recompute subscriptions through coalescing helper.
src/main/notifications/badge.ts	Adds coalesced badge recompute subscription with flush/stop.
src/main/messaging/sendMessage.ts	Passes timestampUnix to registerChannelSend for authoritative relay attribution.
src/main/index.ts	Flushes coalesced holder persistence during shutdown to avoid losing last changes.
src/main/events/coalesce.ts	Adds coalescing utility (leading + bounded trailing) with flush/cancel.
src/main/events/bus.ts	Adds contactSyncSummary event type + emitter entry.
pnpm-lock.yaml	Updates lockfile for @andyshinn/meshcore-ts@0.7.0 and related dependency resolution.
package.json	Bumps @andyshinn/meshcore-ts dependency to ^0.7.0.

Files not reviewed (1)

• pnpm-lock.yaml: Generated file

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.