Unblock the server event loop: timeline build coalescing, deferred pruning, websocket backpressure

[vburojevic]

What was wrong

The server's event loop showed p95 578ms / max 1.18s, and every stall delays every phone RPC and websocket flush (better-sqlite3 is synchronous; plugins run in-process). Four verified mechanisms: (1) timeline rebuilds didn't coalesce across clients — the response cache is cold on every appended event and refuses >200-row (streaming) windows, so desktop+browser+phone+plugin windows each triggered an independent synchronous 130–260ms rebuild that serialized on the loop; (2) maybePruneActiveThreadEventHistory ran inline in POST /internal/session/events every ~30s per active thread, unattributable to the stall monitor, with two DELETEs walking delta rows through correlated subqueries even when nothing was prunable; (3) the app websocket broadcast paths looped bare socket.send — no bufferedAmount check, no queue cap, no try/catch — so a suspended phone grew unbounded buffers and one throwing socket truncated fan-outs; (4) bb.realtime.publish triple-serialized with no rate/size bound, bb.log did mkdir+stat+append per line on the loop, and thread lists ran unbounded queries.

What changed

• Timeline cache: a per-params-key last-build slot shares one build across same-key requests inside a 250ms window regardless of row count, and floors rebuilds of over-LRU-cap windows (the streaming shape) to one per 250ms; LRU-cacheable windows keep eager rebuilds and status flips are never floored. The route records latest-rows under the response's own maxSeq so floored responses stay delta-coherent.
• The active-thread prune moved into the existing post-response follow-up batch, wrapped in runEventLoopWorkSync for stall attribution, with a LIMIT-1 covering-index probe per prune class so the no-op case skips the correlated-subquery DELETEs.
• The terminal socket queue/drain/drop machinery is generalized into per-lane sendWithBackpressure; all five app notify paths route through a 1 MiB high-water / 4 MiB budget lane with try/caught sends and drop-and-close (1013) for wedged or throwing sockets (clients reconnect and catch up via the watermark). Terminal lane semantics unchanged.
• bb.realtime.publish serializes exactly once, rejects payloads over 64KB, and drops publishes over a per-plugin 10/s (burst 20) token bucket with a coalesced warning. bb.log batches lines into one async append per 200ms/8KB with a cached rotation counter; the plugin handle's dispose hooks flush the writer.
• Thread lists default to a 200-row cap in both list functions; GET /threads reports truncation via an x-bb-thread-list-has-more header. Known deliberate behavior change: bb thread list and unpaged SDK lists now return at most 200 rows; in-repo programmatic consumers already pass explicit limits.

No daemon-facing payload shapes changed (no HOST_DAEMON_PROTOCOL_VERSION bump); the plugin-signal wire frame is byte-compatible.

How you verified

New tests proven fail-before/pass-after per step: one build shared across concurrent same-key requests (builder spy); floored request returns the prior window, never floored below the row cap or across params keys; prune runs post-response with identical timeline rows read between append and deferred prune; queue-then-drain above high water, drop-at-cap while siblings receive, throwing-socket isolation; publish rate-limit and 64KB rejection through a real installed plugin; log batching with no sync writes on the call path; db default-cap (201→200) and has-more header tests. Full @bb/server (2,033 tests after rebase) and @bb/db (423) suites green via turbo; typecheck clean. Rebased onto current origin/main (clean; verified by merge-tree). Remaining operational verification: the 24h bb perf issues event-loop-lag soak post-deploy.

AGENT GENERATED

[SawyerHood]

Closing this PR for rework after an adversarial review of the #2385–#2393 set. Method: two independent multi-agent review passes over a worktree at this PR head; every finding went to three independent refuters (code trace, a throwaway experiment against the real code, and an impact judge), and only findings that survived at least two of three are listed. The branch stays as-is; please reopen this PR or open a new one when the blocking items are addressed.

Blocking

1. Sidebar bootstrap is silently capped at 200 live threads across all projects — packages/db/src/data/threads.ts:1277. listThreadsWithPendingInteractionStateForProjects now applies DEFAULT_THREAD_LIST_LIMIT to a query ordered by projectId first, and its only production caller (buildProjectsWithThreadsResponseFromRows, behind GET /projects/sidebar-bootstrap and GET /projects?include=threads) passes no limit. With project A (201 active threads) and project B (5), the bootstrap returns 200 rows for A and 0 for B; B renders empty in the desktop and mobile sidebars, and useChildThreads loses children the same way. Reproduced with the real query on in-memory SQLite (3 projects × 150 threads → 150 / 50 / 0). The body's claim that in-repo programmatic consumers already pass explicit limits is false for this call site. Fix: no silent default inside the db layer; make limit explicit and fill the 200 default once in GET /threads (AGENTS.md: fill defaults once at the boundary); the sidebar passes what it needs, or caps per project with an explicit hasMore.
2. The timeline rebuild floor serves a stale window with no catch-up — apps/server/src/services/threads/timeline-cache.ts:122. For >200-row (streaming) windows, a request with a newer maxSeq inside the 250 ms window gets the previous build. Clients refetch only on the events-appended hint plus one trailing refetch and never poll, so when the last event before a pause (for example item/started of a long tool call) lands during an in-flight fetch, every client misses it until the next append, which can be minutes. Fix: keep same-key sharing but drop the cross-revision floor, or arm a one-shot trailing rebuild for the rest of the window (or emit events-appended when full.maxSeq < maxSeq).
3. The bb.realtime.publish token bucket tail-drops terminal signals — apps/server/src/services/plugins/plugin-api.ts:948. Twelve script automations on one cron minute publish 24 frames synchronously inside sweepDueAutomations; the bucket admits 20; the finally publishes ~400 ms later find ~5 tokens. The automations UI has no poll or reconnect fallback, so finished runs stay "running". Fix: coalesce latest-per-(plugin, channel) and flush when a token frees instead of dropping, or exempt terminal signals. Document the 64 KB rejection and the rate limit in the PluginRealtime.publish JSDoc (packages/plugin-sdk/src/backend-contract.ts:229), the bb-plugin-authoring skill, and the SDK changelog; today every plugin-facing surface still promises unconditional broadcast.
4. The 200-row GET /threads cap is invisible to the SDK and CLI — apps/cli/src/commands/thread/list.ts:51. The only signal is the x-bb-thread-list-has-more header: sdk.threads.list discards it, bb thread list has no --limit/--offset and prints no notice, CORS does not expose the header, and the bb-cli skill and guide do not mention the cap. An agent running bb thread list --json on an install with 250 threads cannot find or reach the missing 50. The same truncation hits the "Assign parent thread" dropdown (apps/app/src/hooks/queries/thread-queries.ts:436) and the mobile home list. Per AGENTS.md, a behavior change must ship its SDK and CLI surfaces in the same change.

Also found

• apps/server/test/services/plugins/plugin-log.test.ts:58 races appendFile open vs write and read an empty file under load (it failed once in our runs).
• apps/server/test/services/plugins/plugin-wire.test.ts:708 depends on wall clock: a ≥100 ms stall mid-loop refills a token and yields 21 frames. Inject a clock.
• plugin-api.ts:898: the refill does not clamp negative deltas, so a backward clock step (NTP) drops every publish until the deficit refills. Use a monotonic clock.
• On the relayed path (bb connect), socket.raw.bufferedAmount measures a loopback socket that always drains, so the new lane never trips for a relayed phone; the bytes accumulate in the tunnel instead. That is pre-existing, but the body's "suspended phone" framing only holds for LAN clients.

What was clean

The deferred active-thread prune, the bb.log batching, and the backpressure lane on LAN had no findings.

Suggested salvage

Split it: land prune + log batching + backpressure now. Then fix the cap (explicit limits, hasMore in the SDK response, --limit/--offset or --all in the CLI, docs), the floor (trailing rebuild), and the bucket (coalesce), and reopen.

AGENT GENERATED