Skip to content

fix(tui): stop Home telemetry sqlite3 process churn#659

Open
TZZheng wants to merge 5 commits into
mainfrom
fix/issue-643-inprocess-molt-query
Open

fix(tui): stop Home telemetry sqlite3 process churn#659
TZZheng wants to merge 5 commits into
mainfrom
fix/issue-643-inprocess-molt-query

Conversation

@TZZheng

@TZZheng TZZheng commented Jul 15, 2026

Copy link
Copy Markdown
Collaborator

Summary

  • stop once-per-second Home telemetry from forking external sqlite3 to resolve molt-session boundaries, including when canonical events are missing, non-regular, or unreadable
  • resolve the latest two psyche_molt boundaries from canonical logs/events.jsonl with a stateless in-process reverse scan that normally reads only the two-session tail
  • keep derived-SQLite compatibility only in the explicitly opened detail view; unavailable canonical events make the Home session fragment empty rather than subprocess-backed
  • add production-entry no-subprocess coverage plus reverse-scan correctness, bounded-read, rewrite, EOF, availability, race, and cross-platform validation

Addresses #643.

Why

On macOS under Rosetta, each sqlite3 child could emit:

MallocStackLogging: can't turn off malloc stack logging because it was not enabled

Home telemetry already ran asynchronously after #540, but active tool traffic changed log.sqlite often enough to invalidate the short session-window cache. The one-second Home refresh therefore kept launching child PIDs and could leak unsolicited warnings into the terminal.

logs/events.jsonl is the canonical source for molt boundaries; log.sqlite is derived. Reading the small relevant tail directly removes the child-process boundary without introducing an in-process SQLite dependency.

Test-first chronology

Initial RED on untouched production code at base 10c2822812add0ae07ed26ffcfff8d6cec202a4f:

go test ./internal/tui \
  -run '^TestFetchHomeTelemetryDoesNotSpawnSQLiteForMoltWindows$' \
  -count=1 -v
home telemetry spawned sqlite3 to resolve molt windows; canonical events.jsonl must serve this high-frequency path in-process
FAIL
exit 1

The fixture puts a recording sqlite3 executable first on PATH, supplies a sidecar that would exercise the old fallback, calls the production Home entry point, and fails on the forbidden process launch itself—not on a mocked helper or environment error.

Two additional REDs then pinned unavailable-canonical behavior and ordinary same-inode truncate/regrow recovery. A final adversarial RED defeated the intermediate bounded-anchor cache: it rewrote equal-length earlier molt rows on the same inode, preserved the exact prior 4 KiB tail sample, grew beyond the old horizon, and observed stale current = 2000, want 5000. That showed that no finite byte sample can prove an unchanged semantic prefix.

All of these cases are GREEN in final candidate 3d6f53bc12f5564cfd1584250140fe077d9048f1.

Implementation notes

  • SumMoltSessionTokenLedgerFromEvents is the Home-only, process-free source; an unavailable canonical log returns empty session stats and never reaches sqlitelog
  • SumMoltSessionTokenLedger remains canonical-first with derived-SQLite fallback for the explicit detail-view compatibility path
  • each Home read is stateless: open one canonical file, snapshot it, read fixed 64 KiB chunks newest-to-oldest, and stop after the latest two valid psyche_molt rows
  • only the single JSONL row crossing a chunk boundary is retained; memory is bounded by one chunk plus the longest encountered row
  • complete malformed rows are skipped; a valid unterminated EOF row is accepted; an invalid partial EOF row is skipped for that tick and naturally retried after completion
  • missing/non-regular/unreadable files return unavailable; empty or valid no-molt files are authoritative zero windows
  • post-scan file/path identity, size, and mtime checks reject ordinary concurrent mutation or replacement rather than publishing a mixed snapshot
  • there is no cross-call offset, inode, prefix-anchor, or boundary cache, so arbitrary prior-prefix rewrites cannot pin stale state
  • the early-stop test proves one 64 KiB ReadAt when both boundaries are in the tail; histories with fewer than two boundaries necessarily scan to byte zero

Validation

Candidate: 3d6f53bc12f5564cfd1584250140fe077d9048f1 (five commits; 9 files, +635/−109)

  • focused production-entry and reverse-scanner regressions
  • focused go test -race ./internal/fs ./internal/tui
  • unfiltered exact-HEAD go test ./... -count=1
  • exact-HEAD go vet ./...
  • exact-HEAD go build ./...
  • exact-HEAD go test -race ./internal/fs ./internal/tui -count=1, except the proven-baseline test described below
  • exact-HEAD CGO_ENABLED=0 GOOS=darwin GOARCH=amd64 go build ./...
  • exact-HEAD CGO_ENABLED=0 GOOS=darwin GOARCH=arm64 go build ./...
  • git diff --check
  • all 16 changed Anatomy citations are in bounds
  • independent Claude review: APPROVE, no blockers; 27,000 randomized differential cases across chunk sizes down to one byte plus 130 seam alignments, zero divergence from a forward oracle
  • independent Codex review: APPROVE, no blockers; independent focused fs/tui/sqlitelog tests passed

The unfiltered full suite, vet, and build pass at the exact candidate. The race gate reproduces one unrelated baseline timing flake, TestPortalURLTimeoutKillsChild (fake portal never wrote its pid). An untouched base checkout reproduces the same failure. A supplemental exact-HEAD run skips only that one proven-baseline test and passes the full suite, fs/tui race, vet, build, both Darwin no-CGO builds, diff-check, and Anatomy checks. No unrelated portal-test change is included.

After acceptance, current main advanced by one release/install commit (45482ced); its changed paths are disjoint from this PR and git merge-tree reports no conflicts, so the reviewed candidate SHA was intentionally preserved rather than rebased without cause.

Scope / related work

No duplicate or linked PR was found for #643 before publication. #526 is the broader historical SQLite/slow-volume issue; #540 moved Home telemetry off the UI thread but retained this fork; closed #578 was a deliberately partial /kanban timeout attempt; #611 and #632 do not remove the Home session-window subprocess.

Generic SQLite-backed detail and diagnostic helpers remain in place. This PR changes only the high-frequency Home token/session-boundary path and removes the superseded full-JSONL/SQLite fallback behavior from that path.

@TZZheng
TZZheng marked this pull request as ready for review July 15, 2026 21:24
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant