Skip to content

feat(self): /self/owner-storage adds messageCount + lastWriteToken for silent-reject visibility#12

Merged
elkimek merged 2 commits into
mainfrom
feat/owner-status-health-probe
May 11, 2026
Merged

feat(self): /self/owner-storage adds messageCount + lastWriteToken for silent-reject visibility#12
elkimek merged 2 commits into
mainfrom
feat/owner-status-health-probe

Conversation

@elkimek
Copy link
Copy Markdown
Owner

@elkimek elkimek commented May 11, 2026

Summary

Widens /self/owner-storage so a client can verify "is the relay actually persisting my pushes?" without operator help.

New fields, purely additive:

  • messageCountCOUNT(*) FROM evolu_message WHERE ownerId = ?
  • lastWriteToken — hex of evolu_usage.lastTimestamp, or null when no writes have landed

A client that just pushed can poll this endpoint, compare a before/after pair, and detect when the relay silently dropped a push (the WS round-trip reports "Push committed" cleanly but neither storedBytes nor messageCount advanced and lastWriteToken didn't change).

Why

Real production incident 2026-05-11. Owner 41CFA6304F7AD97F9C486A1EFD2F2345 was wedged: every client push reported Push committed mnsvhkhg (67ms) with a clean WebSocket round-trip, but evolu_usage.storedBytes stayed at 0 and evolu_message had zero rows. Bug lives upstream in @evolu/nodejs@2.4.0 (already the latest published) and emits no quota / error / reject event we can log. Operator had to SSH in and query the SQLite by hand to confirm. With this in place, the next wedge becomes a red dot in the client diagnose modal within 30 seconds instead of "I noticed sun data was different across devices today."

Cost

One extra SELECT COUNT(*) per probe call, same readonly connection. evolu_message primary key is (ownerId, timestamp), so it's an index range scan — not a table scan. Existing rate limit (60/min/IP, "storage" bucket) unchanged and still adequate.

Backwards compatibility

Existing callers (e.g. the bundled-bytes estimate replacement) keep getting storedBytes + quotaBytes exactly as before. The new fields are additive.

Tests

  • storage probe returns live storedBytes from the DB extended to assert messageCount=5 and lastWriteToken matches hex of the seeded lastTimestamp BLOB.
  • storage probe still works after compact extended to assert the exact wedged-owner shape: messageCount=0, lastWriteToken=null.
  • All 26 tests pass.

Test plan

  • npm test — 26/26 pass
  • Deploy: pull on VPS, rebuild + restart. Smoke: hit /self/owner-storage for an active owner, expect the new fields populated; for a deliberately-empty owner, expect messageCount=0 + lastWriteToken=null.

🤖 Generated with Claude Code

…ilent-reject visibility)

Real production incident 2026-05-11: owner `41CFA6304F7AD97F9C486A1EFD2F2345`
was wedged. Every client push reported "Push committed (67ms)" with a clean
WebSocket round-trip — but `evolu_usage.storedBytes` stayed at 0 and
`evolu_message` had zero rows for the owner. Bug lives upstream in
`@evolu/nodejs@2.4.0` (latest published); no quota/error event logged.
Took an SSH + sqlite3 query to even confirm the silent-reject.

This patch widens `/self/owner-storage` so a client can verify "is the
relay actually persisting my pushes?" without operator help:

  storedBytes      — unchanged
  quotaBytes       — unchanged
  messageCount     — NEW: COUNT(*) from evolu_message for the owner
  lastWriteToken   — NEW: hex of evolu_usage.lastTimestamp, or null when
                     no writes have landed

A client that just pushed can poll the endpoint and check whether
storedBytes/messageCount advanced and lastWriteToken changed. If it
believed the push succeeded but the relay shows no progress, surface
the divergence (e.g. red dot in Settings → Data → Sync). With this
in place the next wedge becomes visible in 30s instead of "I noticed
data was different across devices today."

Cheap on the SQLite side — `evolu_message`'s primary key is
(ownerId, timestamp), so the COUNT is an index range scan, not a full
table scan. One additional SELECT per probe call, same readonly
connection.

Backwards-compatible: existing callers (the bundled-bytes estimate
replacement) still get `storedBytes` + `quotaBytes`; the new fields
are additive.

Tests
─────
- `storage probe returns live storedBytes from the DB` extended to
  assert `messageCount=5` and `lastWriteToken` matches hex of the
  seeded `lastTimestamp` BLOB.
- `storage probe still works after compact` extended to assert the
  exact wedged-owner shape: `messageCount=0`, `lastWriteToken=null`.
- All 26 tests pass; no schema change.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@greptile-apps
Copy link
Copy Markdown

greptile-apps Bot commented May 11, 2026

Greptile Summary

This PR widens /self/owner-storage to return two new diagnostic fields — messageCount (row count from evolu_message) and lastWriteToken (hex of evolu_usage.lastTimestamp) — so clients can detect the silent-push-reject scenario without operator intervention. It also wraps the two SQL reads in an explicit readDb.transaction(), addressing the snapshot-consistency concern raised in a previous review thread.

  • New response fields: messageCount and lastWriteToken are appended to the existing storedBytes/quotaBytes response; existing callers are unaffected.
  • Atomicity fix: Both SELECTs now execute inside a single BEGIN DEFERRED transaction on the readonly connection, ensuring storedBytes and messageCount always reflect the same DB snapshot.
  • Test coverage: Two existing integration tests are extended to assert the new fields, covering both the populated-owner and post-compact (null/0) cases.

Confidence Score: 5/5

Safe to merge — additive response fields, correct null handling, and the previously-flagged snapshot consistency gap is now resolved by the explicit transaction wrapper.

The change is purely additive: no existing fields are modified, auth and rate-limiting paths are untouched, and the two new SELECTs are wrapped in a single transaction so they always observe the same DB snapshot. Null/undefined edge cases for lastTimestamp are handled correctly, and two integration tests cover both the populated-owner and post-compact shapes end-to-end.

No files require special attention.

Important Files Changed

Filename Overview
src/lib/self-server.ts Adds messageCount and lastWriteToken to the storage-probe response; wraps both reads in a transaction for a consistent snapshot. Logic is correct, null/undefined cases handled, no issues found.
test/self-server.integration.test.mjs Extends two existing tests to assert messageCount and lastWriteToken in both the populated and post-compact states. Coverage is complete for the new fields.
package.json Version bump 1.2.2 → 1.2.3 and minor engines formatting; no functional changes.

Sequence Diagram

sequenceDiagram
    participant Client
    participant SelfServer
    participant SQLite as SQLite (readonly)

    Client->>SelfServer: GET /self/owner-storage (ownerId + timestamp + auth)
    SelfServer->>SelfServer: Validate auth and replay window
    SelfServer->>SQLite: Open readonly connection
    SelfServer->>SQLite: BEGIN DEFERRED (single snapshot)
    SelfServer->>SQLite: SELECT storedBytes + lastTimestamp FROM evolu_usage
    SQLite-->>SelfServer: usage row or undefined
    SelfServer->>SQLite: SELECT COUNT from evolu_message
    SQLite-->>SelfServer: message count
    SelfServer->>SQLite: COMMIT + close
    SelfServer->>SelfServer: Compute lastWriteToken from lastTimestamp blob
    SelfServer-->>Client: 200 JSON storedBytes, quotaBytes, messageCount, writeToken
Loading

Reviews (2): Last reviewed commit: "fix(self): wrap probe queries in one tra..." | Re-trigger Greptile

Comment thread src/lib/self-server.ts Outdated
)

Greptile flagged that better-sqlite3 runs each statement in its own
implicit transaction, so the two SELECTs in /self/owner-storage
(`evolu_usage` + `COUNT(*) FROM evolu_message`) can observe different
DB snapshots if a writer commits between them. Window is narrow but
the cost of a spurious 'wedged' verdict on the client side is exactly
the false alarm we built the new health probe to avoid.

Wrapping both reads in `readDb.transaction(() => {...})()` makes them
share one snapshot. The transaction is a no-op on writes (readonly
connection), purely a consistency barrier across the two SELECTs.

All 26 tests still pass; no behavior change for a quiescent DB.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@elkimek elkimek merged commit 355b855 into main May 11, 2026
4 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant