Durable afterCommit for user-installed plugins, behind --plugin-after-commit (#1851)

[michielbdejong]

Closes #1851

Implements the design in #1888 (planning/durable-after-commit.md, carried in this branch and updated) with Michiel's four decisions. Everything is behind --plugin-after-commit / ATOMIC_PLUGIN_AFTER_COMMIT, which is off by default. With the flag off, no subscription is recorded, no wake-up marker is written, and the entrypoint does nothing.

What this is stacked on

Base: feat/plugin-debug. The branch starts from claude/atomic-plugins-pin-candidate14 (1432e24). On top of that it carries:

• Per-table change list with tombstones (#1850) #1885 (change list), cherry-picked with -x: e77fe3c. Conflicts with candidate14's db.rs, trees.rs, redb_store.rs, tests.rs and index.ts were resolved by keeping both sides. candidate14 has no Db::from_kv or Tree::ALL, so the new tree is wired into its explicit lists.
• Allow editing scope covers a plugin's provider extras on rows (provider id, ETag, baseline) #1849 (row-grant extras, PR Cover a plugin's row extras with the table's row grant (#1849) #1900), merged: 64be088. Hook writes under a grant can therefore include the app's declared row-extras.
• Design: durable afterCommit for user-installed plugins (#1851) #1888 (the design doc), cherry-picked: 937f12b.

Until the stack lands, the diff shows all of it. Please review these commits:

Commit	What
bf22a7c	atomic_lib: wake-up markers staged inside the change log's transaction; table_changes_head
73c522e	Server: subscriptions, delivery, cursor, review, retries, loop cap, reads scope, runtime entry, /app-after-commit, tests
69e0ca8	createApp apps; reserved routes; @tomic/lib manifest mirror plus shared fixtures; browser/lib integration test
ba17a3c	UI for decisions 1–3
98dba07	Docs, planning status and deviations, TESTING_COVERAGE.md

Decisions, as built

1. Adding the view subscribes the app. The "Let edit rows?" dialog gains the line " is told when rows change, also when this tab is closed." The line appears only when the server has the flag on and the app exports afterCommit. Allow editing (op: "grant") subscribes on the server in the same request. Read-only now sends op: "follow".
2. Proposals wait per table. A bar above the app's tab reads " wants to change 3 rows", with Apply, Allow all edits by this view on this table, and Decline. The second button is offered only when every change stays within what a row grant can cover. It records a grant with via: "hook-review". The tab gets a mark, and the Installation page shows the count, linking to the table.
3. Failures stay quiet. After 8 failed attempts, the tab and the Installation page show a warning mark with " stopped following changes to :", the error, and Retry. There is no notification.
4. The app pauses per table. While a proposal waits there is one delivery in flight, so nothing new is claimed. After the answer, the app catches up on the markers that coalesced meanwhile.

Server

• lib/src/after_commit_wake.rs: a table → marker index on the store. stage_change_log upserts markers in the same transaction as the change-list entries. That covers commits, replicated resources, imports and sync removals.
• server/src/plugins/after_commit.rs:
  • Subscriptions per (drive, app, table).
  • Eligibility: a view, a table, the activator can read it, the installation resolves with its own key, and the manifest declares afterCommit.
  • Claim after a debounce (2 s after the last change, or 10 s after the first), then a page of the change list read as the activator.
  • Echoes are dropped by Loro version. The cursor moves only in the acknowledging batch. A delivery whose journal already finished is acknowledged without running again.
  • CURSOR_EXPIRED → reset: 'expired'. A new subscription starts at the head with reset: 'initial'. hasMore continues on the next pass and does not count toward the loop cap.
  • Backoff runs from 30 s, doubling to 1 h. After 8 attempts delivery stops until Retry, or until the source changes.
  • The loop cap is 30 woken runs a minute. Each run gets 60 s of wall clock plus the JS fuel and memory limits.
  • A sweep runs at startup and every 10 minutes, and the grant backfill runs at startup.
• Writes: with a live grant, changes within check_scope (extras included) and changes to the app's own subtree apply unattended, signed by the app agent. Anything else is held for review.
• Reads: the host wrapper limits ctx.read and ctx.query to the table, its rows, the row class and its properties, and the app's subtree. ctx.changes(table, {since, limit}) works for the event's table only.
• Runtime: plugin-runtime calls input.entry (run or afterCommit), and ctx.changes goes through host-call. The WIT is unchanged.
• The same worker loop as triggers runs a sibling drain (after_commit::tick).
• GET/POST /app-after-commit: status, answers and Retry.

Deviations from the design

Details are in the doc's "Implementation notes".

• Records live under after-commit/v1/{sub,wake,delivery,own}/, not plugin-event/v1/. The trigger drain reads every plugin-event/v1/ entry as a QueuedEvent, and it deletes any whose trigger record is missing, so a shared prefix would have lost deliveries. The worker loop, journal and run log are shared.
• The marker is claimed (under the change-log lock) before the page is read. A crash between the two loses only the marker, and the sweep recovers it.
• Subscriptions are keyed by the app that view-kind names, as the grant is. Code comes from the Installation above the app if there is one. Otherwise it comes from the app's own plugin-source or its entrypoint (createApp apps). Apps without an Installation show their state on the tab only.
• follow records nothing when the app does not export afterCommit.
• A page that holds only the plugin's own echoes is acknowledged without a run.
• A reviewed "Apply" cannot delete rows the app does not own.

Left for later

• Deleting subscriptions on uninstall or revoke. For now they wait, as for a paused installation.
• Ending a subscription immediately when the activator loses read. For now it ends at the next attempt.
• Refusing a waiting proposal after a release change.
• A Playwright spec (optional in the brief).
• A test for "a person's edit just before the hook's write".

Tests

• atomic_lib (change_log_test.rs): a marker in the same batch, none for other tables; coalescing; replicated and sync-removed rows; no index, no marker; the head cursor.
• atomic-server (plugins/after_commit_test.rs, 22 tests with a fixture JS plugin installed from a release), covering:
  • delivery at the head, then per change;
  • a paused installation skipped without counting attempts;
  • a throw redelivering the same page with attempt: 2;
  • 8 failures stopping the table with the cursor unchanged, then Retry;
  • the crash window (a finished journal acknowledged once);
  • CURSOR_EXPIRED;
  • 210 rows arriving in 3 pages outside the cap, and the loop cap itself;
  • echo dropping, with a person's edit after it still delivered;
  • a write under a grant signed by the app agent;
  • review: Apply, Allow all and Decline, with the pause and catch-up;
  • out-of-scope writes;
  • ctx.changes refused for another table;
  • view-kind set by hand, and a kind change ending the subscription;
  • the sweep, the grant backfill and the flag off;
  • durability across a hard restart (subprocess exit, reopen, delivered).
• manifest.rs plus shared fixtures: afterCommit is accepted in extension and refused in server-extension; onResourceGet and beforeCommit are refused.
• browser/lib integration (tests/after-commit.integration.test.ts): a real server with the flag. A row saved through store.save reaches the hook, and its edit waits for review.
• vitest: AfterCommitBar.test.tsx, afterCommit.test.ts, appViewGrant.test.ts (Read-only follows; Allow editing sends no second request; retried while the view saves).
• Run:
  • cargo test -p atomic_lib --features db-redb --lib: one failure, drive_mapping_tests::reconcile_skips_empty_hosts_and_drives. It passes on its own, reuses a fixed temp-dir name, and has nothing to do with this change.
  • cargo test -p atomic-server --lib, with and without --features plugin-routes: all green after the fixes in 69e0ca8. Those were the pinned release id for the new fixture and the reserved routes app-after-commit and Per-table change list: rows changed since a cursor, including deletions (tombstones) #1850's changes.
  • Clippy: -D warnings for light,wasm-plugins (the hook's set) and for plugin-routes.
  • pnpm typecheck, oxlint and oxfmt.
  • The .po files are settled: a dev-server run afterwards leaves them unchanged.
  • useDriveApps.live.test.tsx fails on Node 26 (localStorage). That failure predates this change.

Screenshots

Taken from the real app (vite plus a server with the flag). The setup: an app made with createApp inside an Installation, shown on a "Team calendar" table outside the app; three rows saved; the hook proposes a description for each.

• the dialog line, at desktop and 375 px;
• the review bar with the tab mark, at desktop and 375 px, followed by Allow all applying the edits;
• the Installation page count, at desktop and 375 px;
• the warning mark with Retry, at desktop and 375 px.

The stopped state takes 8 failures over about 2 hours, so for that shot alone the status response is rewritten in the browser to stopped. The screenshots are shared in the coordination thread.

🤖 Generated with Claude Code

[gitguardian]

⚠️ GitGuardian has uncovered 1 secret following the scan of your pull request.

Please consider investigating the findings and remediating the incidents. Failure to do so may lead to compromising the associated services or software components.

🔎 Detected hardcoded secret in your pull request

GitGuardian id	GitGuardian status	Secret	Commit	Filename
37584408	Triggered	RSA Private Key	7d10b63	testdata/http-signatures/vectors.json	View secret

🛠 Guidelines to remediate hardcoded secrets

1. Understand the implications of revoking this secret by investigating where it is used in your code.
2. Replace and store your secret safely. Learn here the best practices.
3. Revoke and rotate this secret.
4. If possible, rewrite git history. Rewriting git history is not a trivial act. You might completely break other contributing developers' workflow and you risk accidentally deleting legitimate data.

To avoid such incidents in the future consider

• following these best practices for managing and storing secrets including API keys and other credentials
• install secret detection on pre-commit to catch secret before it leaves your machine and ease remediation.

---

^{🦉 GitGuardian detects secrets in your source code to help developers and security teams secure the modern development process. You are seeing this because you or someone else with access to this repository has authorized GitGuardian to scan your pull request.}

[michielbdejong]

Closing as part of the plugin-host stack cleanup (Q-101): not needed by atomic-plugins now. See the inventory at #1726 (comment). The branch is kept, and pin candidates are unaffected. Closed by Claude (VPS coordinator) on behalf of Michiel.