perf(history): serve long threads from stored turn slices and keep agent state to the window

[Chen-zexi]

Note

Claude Opus 5.5 writing on behalf of Zexi

Branch perf/long-thread-history, base fix/fork-turn-and-reasoning-space. Nine commits in five parts. Each commit passes the suite on its own, so it reads well commit by commit:

Part	Commits
1. Turn slices	2c8974ff2 feat(history): add the turn slice and replay facts tables ed075e392 perf(history): replay a thread from stored turn slices
2. Replay facts	916292ce2 feat(runs): record replay facts when a run settles e7f80f939 feat(scripts): backfill replay facts for turns settled before them
3. Sliding window	16bb69fa0 perf(agent): keep the main agent's state to the window the model sees
4. Operator scripts	34326ffec feat(scripts): bring every turn's stored replay up to date 0953391a4 feat(ops): thin message snapshots on old checkpoints be2166a91 perf(agent): store a full messages snapshot every 250 updates
5. Crossed-thread repair	8d7c5ae75 perf(ops): repair crossed threads so the window can trim them

Overview

A long thread cost three things that grew with its length:

• Opening it. History was rebuilt from checkpoint state. One missing cached turn rebuilt the whole thread, one full state load per turn, serialized by the saver lock, and some turns could never be cached: a subagent launch counted as live once its Redis stream expired, a day after the task ended. Even a full cache hit read every turn's stored SSE events.
• Every turn. The agent loaded the whole messages history, though after a summary the model sees only the summary and what follows. On the longest bench thread that is 1,823 messages and 9.1 MB loaded to send about a twentieth of it.
• Storage. Most checkpoint bytes are full copies of messages: a snapshot every 50 updates, and on threads from before DeltaChannel a full copy per update.

1. Turn slices

When a turn settles, what it added to its checkpoints (new messages, ui records, compaction and offload changes) is cut once and stored in turn_slices, keyed by the turn's response row, together with its projected replay lines. A subagent run gets the same in task_run_slices. Opening a thread now:

1. reads light rows (queries and settled responses, never sse_events);
2. walks the branch in Postgres, metadata only;
3. streams each turn's stored lines, and projects again only a turn whose slice key or lines key no longer matches, two state loads for that turn alone.

A slice is valid while its input and tail checkpoints match the walk and its slice_key names the code that cut it. The lines key is a deploy epoch (projection version, a digest of every module that shapes the text, the token threshold) plus a digest of the turn's rows, so a deploy that changes projection code re-projects from slices without loading any state.

• Background refresh. A settled turn projects the newest two turns, and a settled subagent run re-projects the turn that launched it. A read projects whatever those left undone, so a crash between finalize and refresh costs one read.
• Paging on the endpoint. GET /api/v1/threads/{id}/messages/replay accepts before_turn and limit. With either, a history_page frame comes first so the client knows where the page goes. The web PR on top uses them.
• Committed output only. Replay shows what a turn's checkpoints committed. Text from a step that never reached a checkpoint (a model call stopped mid stream, a worker that died) is no longer resurrected from the stored stream.
• One branch walk. walk_current_branch_boundaries returns typed Boundary records from Postgres, with no silent in-process fallback. The history reader, /turns (pair_turns), the automation excerpt and the parity script all read through it.
• MainAgentState. The main agent and the history reader share one state schema, so a ui record appended by the reader keeps every private field.
• Removed. The Redis projection cache (projection_cache.py, REPLAY_PROJECTION_CACHE_TTL_SECONDS), the stream gate, the task lane rebuild and the cross-turn segment claimer. Legacy launches with no ledger stamp are claimed once per thread and pinned in turn_slices.run_claims.

2. Replay facts

A few things a turn shows on replay were only ever in the stream it emitted, never in its checkpoints: how long each reasoning block took, which sandbox images a subagent captured to which links, and steering a subagent handed back unread. Replay, public shares, the Chief of Staff and automation settlement all read them back from conversation_responses.sse_events, a second full record of every turn.

A settled run now carries them as replay facts on its own row (replay_facts, migration 065), merged in after the status settles. Reasoning durations are keyed by the committed message id, so a model call that committed nothing takes its durations with it. Captured images are recorded per subagent lane, so image capture no longer writes a root checkpoint after finalize and a settled turn's tail stays put. Returned steering leaves the Redis queue only once its fact has landed.

Every other reader moved off the stored events. Public share pages are built by the owner's replay assembly, with error events dropped and error text stripped. The Chief of Staff's agent_output reads the text each run committed from its turn slice, main lane only. Provenance is written per lane, and automation credit pause messages are stamped into the response's metadata at finalize, by recovery too. sse_events is still written exactly as before, so no stored data changes that a rollback would miss.

Along the way sse_producer.py shrinks from 2,060 lines on main to 1,718 (stream errors, the event archive and reasoning durations move to their own modules) and task chain repair moves out of subagent_runs.py (951 to 831 lines) into subagent_repair.py.

scripts/utils/backfill_replay_facts.py resolves what only a legacy turn's stored events supplied (widget data, evicted results, reasoning durations, positioned signal rows, captured images) into replay_facts.legacy, after which that turn's replay never reads its events. It writes only when the row version is unchanged and the new facts place every row the same.

3. Sliding window

The main agent's live state now equals the model's view. After a summary, the compaction middleware's before_agent hook cuts messages back to the start of the transcript run holding the summary's boundary: the window the model already sees, rounded back to a run boundary so the current turn never loses a message. It runs on every fresh turn and orchestrator re-entry, and trims only when the summary event has an anchor, the cut is past the first run, and the server confirms within 5 s, by a chained content digest rather than a message count, that stored slices hold every run the cut drops (services/history/window.runs_held). A refusal is kept on the anchor and asked again only after the next summary. The trim never runs in a subagent's namespace or without the server to ask (the CLI).

What the model reads stays byte-identical across a trim, because everything that numbers turns counts from one absolute run base kept in state: turn maps, offload markers, the summary span and index, the earlier turns heading, notes' tool call counts and the transcript pointer. Transcripts render trimmed runs from the stored manifest or read them back from slices, refusing a copy from another branch by its identity digest. MainAgentState declares every private field the middleware keep, so a write through a build that lacks one (thread maintenance, the reader's ui append) no longer erases it. Manual /compact and offload write at the checkpoint they read and advance the thread's commit pointer by compare-and-set.

4. Operator scripts

• scripts/utils/backfill_turn_slices.py projects every turn lacking stored lines or keyed under an old epoch or slice key, confirmed by the read path's own unserved_turns test, newest thread first. --recheck confirms every selected thread, for bulk row changes the SQL pass cannot see, such as the facts backfill.
• scripts/ops/thin_message_snapshots.py drops a messages snapshot only when every checkpoint loading it replays byte for byte from the next kept one. It always keeps branch tips, the current tip, the newest two turns' anchors and one snapshot per 1,000 updates. Each thread is one REPEATABLE READ transaction that loads every affected checkpoint through the server's reader before and after the write and rolls back on any digest change, any longer protected walk or any change to the turn anchors. It skips threads with unserved turns or a run in progress, runs dry by default (--rehearse, --apply), and refuses any langgraph or saver release it was not verified against.
• --keep all drops nothing. It flags every stored pre-delta message list as a snapshot and seeds the top of each walk that crossed one, so no migrated thread replays legacy writes from its root. Those writes carry no ids and come back reordered, and the next snapshot would keep that order for good.
• MESSAGES_SNAPSHOT_FREQUENCY moves from 50 to 250, about a fifth of the snapshot bytes on a long thread while a tip load still reads one scan page.

All three scripts count or rehearse until given --apply, and share one harness (scripts/utils/_thread_job.py), which backfill_transcripts.py now uses too.

5. Crossed-thread repair

--keep all stops new crossings, but a thread that already crossed (its first DeltaChannel turn replayed id-less legacy writes from the root, and a snapshot then kept the reordered head) holds its oldest messages in another order than its turn slices. The window's coverage check compares the two, so it refuses every trim on that thread and its checkpoint keeps growing.

scripts/ops/repair_crossed_threads.py does that trim offline, with the window taken from the slices. It writes the trimmed state through a graph node so the new tip stores a snapshot, removes the step's writes from the old tip so no load replays them, and moves the thread's commit pointer by compare-and-set as the last step before commit, so a user's message never waits on its verification. It refuses unless the next model call, the call after an idle pause, the summary request and the call after the summary all stay byte-identical, and it refuses a thread with a run in progress or a turn before the cut with no stored slice. Like the others it is dry by default (--rehearse, --apply).

Size

	Files	Added	Removed	Net
Production, new	34	8,005	0	+8,005
Production, modified	67	2,962	2,828	+134
Production, deleted	3	0	883	-883
Tests, new	33	8,749	0	+8,749
Tests, modified	40	1,355	991	+364
Tests, deleted	1	0	556	-556
Total	178	21,071	5,258	+15,813

By module, production only (tests excluded), +7,214 net:

Module	Net	What changed
src/server/services/history/	+2,073	replay/ package: cold.py, turn.py, run_lane.py, keys.py, lines.py, refresh.py, stored_events.py, facts.py, legacy.py; slices.py, task_streams.py, committed.py, window.py; reader.py 675 to 473 lines; projection_cache.py, task_lane.py, segment_claim.py removed
scripts/ops/	+2,559	thin_message_snapshots.py, its planner _thin_plan.py and _delta_loads.py; repair_crossed_threads.py; point_app_pool_at shared in _db.py
scripts/	+879	backfill_replay_facts.py, backfill_turn_slices.py, _thread_job.py, replay_parity.py (--compare sse, sse-merged, facts; a stopped turn is excused only for output streamed past its last checkpoint), _errors.py
src/server/database/	+601	turn_slices.py store, light rows in replay_rows.py, replay_facts.py, per-lane provenance.py, RunOutcome in runs/lifecycle.py, subagent_repair.py split out; the app-data pool tags its timeouts, so a replay still falls back to stored events when only the checkpointer pool is busy
src/ptc_agent/agent/	+581	compaction/window.py and the middleware hook; transcript/identity.py; main_state.py; snapshot frequency in state.py; returned steering recorded before it leaves Redis
src/server/services/	+345	runs/: stream_errors.py, event_archive.py, reasoning_facts.py out of sse_producer.py, facts at finalize and recovery; the coordinator re-projects a launching turn; image capture, settlement, transcripts
src/server/utils/	+156	checkpoint_helpers.py: the Postgres branch walk, Boundary, update_at_commit
migrations/	+112	064 turn_slices, task_run_slices; 065 replay_facts; both with lock_timeout = 5s
src/tools/secretary/	+52	committed main-lane text for agent_output
src/server/contracts/	+12	one classify_interrupts
src/server/handlers/	-10	/turns pairs through the shared walk; maintenance writes at the read checkpoint
src/server/app/	-94	public.py through the owner's assembly; the replay endpoint's paging
src/config/	-11	the projection cache TTL setting

Measured

On a bench copy of 113 long threads, the same harness, container and data as a run of main (31 threads timed over HTTP). The two largest are a 97-turn compacted thread and a 71-turn thread with 5,038 replay items. A gate run on this final tree was flat against the first complete build (HTTP full open, cold and warm: 9.35 and 0.80 s to 8.70 and 0.79 s).

Opening a thread

Measure	97-turn thread, main to branch	71-turn thread, main to branch
Full open, warm	1.73 s to 0.069 s	4.33 s to 0.153 s
Full open, cold (now once per turn: at finalize, backfill or a branch change)	1.79 s to 1.84 s	4.30 s to 4.72 s
Newest 20 turns, cold	1.79 s, 5.2 MB to 0.50 s, 0.57 MB	4.30 s, 11.4 MB to 1.16 s, 1.94 MB
Row read p50	50 ms to 8 ms	165 ms to 20 ms
Worker peak RSS	1,209 MB to 627 MB	1,557 MB to 786 MB
Post-turn refresh, 2 new turns	0.13 s to 0.08 s	0.37 s to 0.18 s
Two warm opens at once, one worker	3.57 s to 0.13 s wall

A cold open fetches 0 bytes of sse_events. Cold full opens are 3 to 10% slower than main: the claims pass for legacy launches, now paid once per turn instead of on every open. The derived store is 48.9 MB on disk for the 31 threads, against main's expiring Redis entries.

Turn start (real trim path)

Thread	Messages	Loaded	Load at tip	One turn later
71-turn thread	1,823 to 116	9.12 to 0.58 MB	70.3 to 7.6 ms	67.2 to 19.4 ms
a 352-message thread	352 to 35	3.01 to 0.36 MB	27.4 to 5.8 ms	30.8 to 14.7 ms
97-turn thread, forced	672 to 228	6.47 to 1.36 MB	41.9 to 14.1 ms	40.3 to 21.0 ms

The model request is byte-identical on every trimmed thread. The trim costs 152 ms once per summary, almost all of it the coverage check. Live on a dev stack, a new turn on the 71-turn thread ran on the window with state going from 1,823 to 126 messages. Before the repair, the 97-turn thread refused every trim: it crossed onto DeltaChannel before seeding existed. After it, a turn start on that thread goes from 672 to 228 messages, 6.47 to 1.36 MB fetched, 46.9 to 24.4 ms p50, and the coverage check from 91 ms refused to 0.2 ms.

Storage and finalize

Measure	Result
The three checkpoint tables after thinning, VACUUM FULL	942 MB to 390 MB (58.6% less)
Thinning rehearsal with --verify all	10,581 checkpoints, 0 mismatches
--keep all	760 lists flagged, 2 threads seeded, 0 mismatches; walks of 180, 401 and 331 checkpoints to 14, 3 and 3
Facts backfill	405 turns in 16.1 s, a rerun writes nothing
Finalize CPU per turn, pooled p95	12.5 ms to 4.6 ms
Crossed-thread repair over 109 threads	1 repaired, 1 crossed with no summary yet (rerun after its first), 107 not crossed; --verify all 1,431 checkpoints, 0 mismatches; a rerun finds nothing

The Chief of Staff's committed text read is slower on the giant threads (warm, one turn, 8.0 to 31.7 ms), because it now runs the full validity check; the text is identical on all 404 turns.

Parity against main, 109 bench threads: 104 of 109 match in full, 104 of 109 in the last ten turns, 108 of 109 in the last two, identical with the replay cache off and on, after the facts backfill with sse_events set to NULL, after thinning, and on the repaired thread before and after its repair. Every turn that differs was decided: five where a stopped or errored turn loses its uncommitted output or an errored resume puts its error after the committed tool result, and seven legacy turns whose images had never resolved, which now show their committed tool calls and resolved image links instead of stream-only chunks. Reasoning and all other text are identical.

What a user sees differently

• A long thread opens fast on every open after its first. The first open of a turn with no stored slice costs about what main costs, once.
• After reload, a stopped or failed turn shows only what its checkpoints committed. Text that streamed after the last checkpoint is gone; the Stopped marker, the error card and every committed step remain. The live view keeps the streamed text until reload, as before. An errored resume shows its error after the committed tool result.
• A few legacy turns whose sandbox images never resolved now show resolved images and their committed tool calls.
• A public share page shows what the owner sees after reload, error events and error text removed.
• The Chief of Staff's report of a workspace thread's output quotes what the main agent said, without subagent text mixed in.
• An automation paused for credits shows its denial message even when the pause settled through recovery. Steering a subagent never read shows as returned on replay.
• /compact and offload report a message count for the window, not the whole history.
• source=checkpoint on replay answers 409 when a legacy claims pass fails and 500 when the Postgres walk fails. source=auto, which the web uses, still falls back to stored events with 200.
• The model request, its input token count and the web client are unchanged.

Deploy

Migrations 064 and 065 run first. Then, inside the backend container, in this order:

1. backfill_replay_facts.py --apply.
2. backfill_turn_slices.py --apply --recheck. The facts backfill is a bulk UPDATE, so it stales every lines key; on the bench a plain --apply found 1 thread and --recheck found 108, and until then thinning refused 16 threads.
3. thin_message_snapshots.py --keep all --apply, right after the deploy. This PR also moves the snapshot frequency to 250, and until step 3 runs, a migrated thread with no snapshot yet walks to its root for up to 250 updates instead of 50. That is a load cost, not a correctness one, and it is the same walk such threads take on main today.
4. thin_message_snapshots.py --apply in batches (--thread, --limit), whenever convenient. Disk returns only after VACUUM FULL or pg_repack.
5. repair_crossed_threads.py --apply, after steps 2 and 3. A thread that crossed before its first summary has nothing to trim yet; rerun the script after its first anchored compaction. Do not run replay_parity.py --cache on beforehand: it deletes the thread's slices, and the repair then refuses.

Existing turns are projected on their first read if step 2 has not reached them. Rerun step 1 immediately before the sse_events column is ever dropped, for rows written in between. A thread starts trimming only after its next anchored summary and once its slices are stored.

Manual verification suggested

• Open a long thread twice and compare the replay request's time in the Network tab; ?limit=2 returns the history_page frame first.
• Stop a turn mid-answer, reload: the in-flight text is gone, the Stopped marker and committed steps remain.
• Launch a background subagent, let it finish, reload: the launching turn's task card shows the run's final state. Have it capture a chart image and check the image resolves.
• With two backend workers, finalize a turn on one and replay on the other right after (scripts/multiworker_gate/).
• Share a thread with a stopped and an errored turn, and compare the share page with the owner's view after reload.
• On a stack with a low compaction threshold, grow a thread past a summary and look for [Compaction] window trimmed N messages, runs up to K; the first model call's input tokens match a regenerate on a copy without slices. Delete the thread's turn_slices rows and the log says window waits for the next summary instead.
• Edit a message inside the trimmed range: the fork restores the full history and replay of earlier turns is unchanged.
• On a copy, run the deploy steps above, check that a second pass of each finds nothing, and that editing an old turn of a thinned thread still forks and replays. On a repaired thread, send a turn and look for the window trim in the log. alembic downgrade 063 rolls 064 and 065 back cleanly.

Covered by the DB suites tests/integration/test_turn_slices_db.py, test_checkpoint_walk_db.py, test_replay_walk_failure_db.py, test_finalize_run_db.py, test_replay_facts_db.py, test_legacy_facts_db.py, test_window_identity_db.py, test_main_state_db.py, test_backfill_turn_slices_db.py, test_thin_message_snapshots_db.py and test_repair_crossed_threads_db.py (twin threads, one crossed with six parallel tools per legacy turn, refused before the repair and sending the same model and summary requests as its uncrossed twin after). The window has a deterministic 12-turn harness on a real create_agent graph that byte-compares every model and summary request with the trim on and off (test_window_identity.py), and test_projection_sources.py fails when a digested module imports a text-shaping module that is neither digested nor excluded with a reason.

Place in the stack

2 of 3, on the fixes PR (#510). The web PR on top (#512) pages the transcript through this PR's before_turn and limit; with this PR alone the web client still loads the whole history, from slices.

[coderabbitai]

Important

Review skipped

Too many files!

This PR contains 178 files, which is 28 over the limit of 150.

To get a review, reduce the PR to 150 files or fewer by splitting it into smaller PRs or changing its base branch.

Upgrade to a paid plan to raise the limit.

This review couldn't start because sufficient usage credits or metered capacity aren't available. Add credits or update usage-based reviews in the billing tab, then retry.

⚙️ Run configuration

• Configuration used: Repository UI
• Review profile: CHILL
• Plan: Advanced
• Run ID: 9d36dea8-2f06-48ce-9c06-236a7c871d05

---

📥 Commits

Reviewing files that changed from the base of the PR and between 0c9776e and 8d7c5ae.

---

📒 Files selected for processing (178)

• migrations/versions/064_turn_slices.py
• migrations/versions/065_replay_facts.py
• scripts/_errors.py
• scripts/ops/_db.py
• scripts/ops/_delta_loads.py
• scripts/ops/_thin_plan.py
• scripts/ops/repair_crossed_threads.py
• scripts/ops/thin_message_snapshots.py
• scripts/utils/_thread_job.py
• scripts/utils/backfill_replay_facts.py
• scripts/utils/backfill_transcripts.py
• scripts/utils/backfill_turn_slices.py
• scripts/utils/replay_parity.py
• src/config/settings.py
• src/llms/error_classification.py
• src/ptc_agent/agent/agent.py
• src/ptc_agent/agent/graph.py
• src/ptc_agent/agent/main_state.py
• src/ptc_agent/agent/middleware/background_subagent/redis_stream.py
• src/ptc_agent/agent/middleware/background_subagent/run_executor.py
• src/ptc_agent/agent/middleware/background_subagent/steering.py
• src/ptc_agent/agent/middleware/compaction/compact.py
• src/ptc_agent/agent/middleware/compaction/middleware.py
• src/ptc_agent/agent/middleware/compaction/notes.py
• src/ptc_agent/agent/middleware/compaction/offloading.py
• src/ptc_agent/agent/middleware/compaction/summarize.py
• src/ptc_agent/agent/middleware/compaction/types.py
• src/ptc_agent/agent/middleware/compaction/window.py
• src/ptc_agent/agent/middleware/skills/content.py
• src/ptc_agent/agent/middleware/subagent_switch.py
• src/ptc_agent/agent/state.py
• src/ptc_agent/agent/transcript/__init__.py
• src/ptc_agent/agent/transcript/identity.py
• src/ptc_agent/agent/transcript/pointer.py
• src/ptc_agent/agent/transcript/render.py
• src/ptc_agent/agent/transcript/store.py
• src/ptc_agent/core/sandbox/livefs_mount.py
• src/server/AGENTS.md
• src/server/app/public.py
• src/server/app/threads/messaging.py
• src/server/app/threads/tasks.py
• src/server/contracts/status.py
• src/server/database/automation_executions.py
• src/server/database/conversation/__init__.py
• src/server/database/conversation/_sql.py
• src/server/database/conversation/replay_rows.py
• src/server/database/conversation/responses.py
• src/server/database/conversation/threads_write.py
• src/server/database/conversation/turn_slices.py
• src/server/database/pool.py
• src/server/database/provenance.py
• src/server/database/replay_facts.py
• src/server/database/runs/lifecycle.py
• src/server/database/runs/subagent_repair.py
• src/server/database/runs/subagent_runs.py
• src/server/handlers/chat/ptc_run.py
• src/server/handlers/checkpoint_handler.py
• src/server/handlers/thread_maintenance.py
• src/server/services/automation_excerpt.py
• src/server/services/automation_settlement.py
• src/server/services/history/__init__.py
• src/server/services/history/committed.py
• src/server/services/history/projection_cache.py
• src/server/services/history/projector.py
• src/server/services/history/reader.py
• src/server/services/history/replay/__init__.py
• src/server/services/history/replay/cold.py
• src/server/services/history/replay/errors.py
• src/server/services/history/replay/facts.py
• src/server/services/history/replay/items.py
• src/server/services/history/replay/keys.py
• src/server/services/history/replay/legacy.py
• src/server/services/history/replay/lines.py
• src/server/services/history/replay/refresh.py
• src/server/services/history/replay/run_lane.py
• src/server/services/history/replay/segment_claim.py
• src/server/services/history/replay/stopped.py
• src/server/services/history/replay/stored_events.py
• src/server/services/history/replay/stored_merge.py
• src/server/services/history/replay/task_lane.py
• src/server/services/history/replay/turn.py
• src/server/services/history/slices.py
• src/server/services/history/task_status.py
• src/server/services/history/task_streams.py
• src/server/services/history/window.py
• src/server/services/livefs/mount.py
• src/server/services/persistence/image_capture.py
• src/server/services/runs/coordinator.py
• src/server/services/runs/credit_usage.py
• src/server/services/runs/event_archive.py
• src/server/services/runs/executor.py
• src/server/services/runs/finalization.py
• src/server/services/runs/reasoning_facts.py
• src/server/services/runs/recovery.py
• src/server/services/runs/sse_producer.py
• src/server/services/runs/stream_errors.py
• src/server/services/runs/subagent_archive.py
• src/server/services/runs/subagent_collection.py
• src/server/services/share_redaction.py
• src/server/services/subagent_run_coordinator.py
• src/server/services/transcripts.py
• src/server/utils/checkpoint_helpers.py
• src/server/utils/pg_sanitize.py
• src/tools/secretary/utils.py
• tests/integration/test_backfill_turn_slices_db.py
• tests/integration/test_checkpoint_walk_db.py
• tests/integration/test_finalize_run_db.py
• tests/integration/test_fork_turns_db.py
• tests/integration/test_legacy_facts_db.py
• tests/integration/test_main_state_db.py
• tests/integration/test_repair_crossed_threads_db.py
• tests/integration/test_replace_agent_events_db.py
• tests/integration/test_replay_facts_db.py
• tests/integration/test_replay_walk_failure_db.py
• tests/integration/test_thin_message_snapshots_db.py
• tests/integration/test_turn_slices_db.py
• tests/integration/test_window_identity_db.py
• tests/unit/middleware/compaction/test_offload_view.py
• tests/unit/middleware/compaction/test_scratchpad_notes.py
• tests/unit/middleware/compaction/test_window.py
• tests/unit/middleware/compaction/test_window_identity.py
• tests/unit/middleware/compaction/window_harness.py
• tests/unit/middleware/skills/test_skill_content.py
• tests/unit/middleware/test_compaction_request_shape.py
• tests/unit/middleware/test_credit_gate.py
• tests/unit/middleware/test_returned_steering_facts.py
• tests/unit/ptc_agent/agent/middleware/compaction/test_summary_message.py
• tests/unit/ptc_agent/agent/middleware/compaction/test_transcript_pointer.py
• tests/unit/ptc_agent/agent/test_delta_channel_determinism.py
• tests/unit/ptc_agent/agent/test_main_state.py
• tests/unit/ptc_agent/agent/test_subagent_switch_wiring.py
• tests/unit/ptc_agent/agent/test_transcript_render.py
• tests/unit/scripts/utils/test_replay_parity.py
• tests/unit/server/app/test_public_order_privacy.py
• tests/unit/server/app/test_public_redaction.py
• tests/unit/server/app/test_public_replay_assembly.py
• tests/unit/server/app/test_public_replay_duration.py
• tests/unit/server/app/test_public_replay_stopped.py
• tests/unit/server/app/test_replay_snapshot_frame.py
• tests/unit/server/contracts/test_status.py
• tests/unit/server/database/test_conversation_db.py
• tests/unit/server/database/test_provenance_insert.py
• tests/unit/server/database/test_provenance_lane_scope.py
• tests/unit/server/database/test_provenance_sync_skip.py
• tests/unit/server/database/test_start_run_fork_order_refusal.py
• tests/unit/server/database/test_start_run_fork_task_guard.py
• tests/unit/server/database/test_subagent_runs_repair.py
• tests/unit/server/handlers/test_checkpoint_handler.py
• tests/unit/server/handlers/test_streaming_handler.py
• tests/unit/server/handlers/test_trigger_compaction.py
• tests/unit/server/services/history/reader_graphs.py
• tests/unit/server/services/history/replay_builders.py
• tests/unit/server/services/history/test_committed.py
• tests/unit/server/services/history/test_event_ledger.py
• tests/unit/server/services/history/test_legacy_facts.py
• tests/unit/server/services/history/test_pair_turns.py
• tests/unit/server/services/history/test_projection_cache.py
• tests/unit/server/services/history/test_projection_sources.py
• tests/unit/server/services/history/test_reader.py
• tests/unit/server/services/history/test_reader_ui.py
• tests/unit/server/services/history/test_reasoning_durations.py
• tests/unit/server/services/history/test_replay.py
• tests/unit/server/services/history/test_replay_facts.py
• tests/unit/server/services/history/test_replay_refresh.py
• tests/unit/server/services/history/test_replay_stopped.py
• tests/unit/server/services/history/test_replay_store.py
• tests/unit/server/services/history/test_slices.py
• tests/unit/server/services/history/test_window.py
• tests/unit/server/services/runs/test_coordinator.py
• tests/unit/server/services/runs/test_finalize_credit_pause_stamp.py
• tests/unit/server/services/runs/test_recovery_task_chains.py
• tests/unit/server/services/runs/test_sse_producer_reasoning_facts.py
• tests/unit/server/services/runs/test_subagent_archive_run_facts.py
• tests/unit/server/services/test_automation_excerpt.py
• tests/unit/server/services/test_automation_settlement.py
• tests/unit/server/services/test_transcript_sync.py
• tests/unit/tools/secretary/conftest.py
• tests/unit/tools/secretary/test_extract_from_db.py

---

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

• Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

---

_{Comment @coderabbitai help to get the list of available commands.}

[chatgpt-codex-connector]

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review	Status	Commit	Review trigger
📝 Code Review	✅ Completed 2026-10-10T13:13:51.418847Z	8d7c5ae	Manual request

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

[claude]

Review scope note: GitHub refuses to serve this diff (over 20,000 lines), so I could not review it in full. I read the PR description and spot-checked src/server/services/history/window.py, the coverage check that gates the sliding-window trim.

No blocking issues found in that file. It checks coverage by content identity, reads slice rows in batches of 32 with the decoding run off the event loop (asyncio.to_thread), and leaves the thread untrimmed when the check misses.

I did not review the rest: the migrations (064/065), the operator scripts, compaction/window.py, the replay facts and finalize path, or the Postgres branch walk. Because of the size, I suggest a human reviewer go through it commit by commit, as the description proposes. The multi-worker gate (scripts/multiworker_gate/) and the DB suites it lists are worth running before merge.

[chatgpt-codex-connector]

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: cbc98d2ac6

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

[claude]

Review scope: this PR's diff exceeds GitHub's 20,000-line limit (176 files), so gh pr diff fails. I read the description and comments, plus compaction/window.py and migrations 064/065 from the merged checkout. I did not review the rest line by line.

No blocking issues found in what I read.

💡 Non-blocking

• migrations/versions/064_turn_slices.py: turn_slices.conversation_thread_id has no FK to conversation_threads, unlike task_run_slices. It cascades only through conversation_responses, so this is fine unless responses can be deleted without the thread. If that's intended, add a one-line comment saying so.
• migrations/versions/065_replay_facts.py: the SET LOCAL lock_timeout in upgrade is good. Confirm Alembic runs the migration in a transaction, because outside one SET LOCAL has no effect.
• compaction/window.py window_cut: a run-boundary scan runs on every before_agent. It is O(n) per turn, which is fine, but caching the result keyed on the event would make the no-op path free.
• This PR bundles the slices, replay facts, sliding window and three operator scripts. It is hard to review as one unit. The commit-by-commit structure helps, but splitting the PR (the window and repair scripts in particular, since they carry the most risk) would give them a proper review.

The deploy ordering in the description (facts backfill, then slices --recheck, then --keep all) is load-bearing. Consider a runbook doc next to the scripts.

[Chen-zexi]

Claude Opus 5.5, automated agent reply

Per item, no change in this round:

1. As intended. A slice belongs to its response row, so it cascades from conversation_responses, which itself cascades from conversation_threads (001_initial_schema.py). Fork truncation deletes responses from the fork turn on (threads_write.py:575) and takes their slices with them, which a thread FK alone would not do. conversation_thread_id is there for the (conversation_thread_id, slice_key) index.
2. It runs in a transaction. migrations/env.py wraps context.run_migrations() in context.begin_transaction() on both the online and offline paths, so SET LOCAL lock_timeout applies to the migration.
3. Left as is. The scan is over the messages already in state, and the measured turn-start load with the window trimmed is 9.1 ms on a thread whose window spans 1,707 messages.
4. Kept as one PR on purpose. The stack was consolidated to three PRs at the owner's request, and the nine commits are the review units: each one passes the suites on its own.
5. The deploy order lives in the description's Deploy section and in each script's docstring, which is where an operator running them looks. A separate runbook can follow if the order grows.

[chatgpt-codex-connector]

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 192ff51b39

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

[Chen-zexi]

@codex review

[claude]

Review status: not a full review.

The PR is about 21k lines. GitHub refuses to serve the diff (HTTP 406, over 20,000 lines) and the checkout has no base commit to diff against. I could not read the code line by line, so I'm not signing off on it.

What I did check:

• The description is detailed and matches the stated scope.
• The two earlier Codex threads (the task_stream_key() builder and _thread_job.py using DB_* rather than MEMORY_DB_*) have author replies, and I found no reason to dispute them. The pagination reply (a page can carry a few stub turns beyond limit) is stated openly.

I found no blocking issues, but that is because I could not review the code, not because I verified it.

💡 Non-blocking: split the PR. The five parts (turn slices, replay facts, sliding window, operator scripts, crossed-thread repair) are independently reviewable and have different risk profiles.

• The sliding-window trim changes live agent state.
• The thinning and repair scripts delete checkpoint data.

Splitting them, or at least stacking each part as its own PR, would allow a real line-level review. The commits are described as each passing the suite, so a split looks feasible.

Areas I'd want a human to look at:

• services/history/window.runs_held: it has a 5 s timeout, and a refusal is kept on the anchor.
• The compare-and-set updates of the commit pointer.
• The thinning script's rollback logic.
• The migrations, which use lock_timeout = 5s.

🤖 Generated with Claude Code

[Chen-zexi]

Claude Opus 5.5, automated agent reply

1. Split: kept as one PR on purpose, as answered on the previous review. The stack was consolidated to three PRs at the owner's request, and each of the nine commits passes the suites on its own, so the commits are the review units. The largest, d7c65be, is about 11.8k changed lines, which GitHub serves.
2. The four areas you name (runs_held, the commit-pointer compare-and-set, the thinning rollback, the migration lock timeouts) are the right places for a human pass, and I have passed them to the owner.

[chatgpt-codex-connector]

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 192ff51b39

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

[claude]

Review scope note: gh pr diff refuses this PR (over GitHub's 20,000-line diff limit), so I could not review it in full. I read the sliding-window core (src/ptc_agent/agent/middleware/compaction/window.py, src/server/services/history/window.py) and the PR description. I did not review the replay/slice stores, migrations 064/065, the finalize path or the operator scripts.

No blocking issues found in the code I read. The trim is gated on a server-side identity check that the stored slices hold the dropped runs. It falls back to no trim when the server can't be asked. Slice reads are batched (32 per query) and decoding runs off the event loop.

💡 Non-blocking

• The PR is about 21k lines across 176 files. Please consider landing it as the nine commits' five parts in separate PRs (slices, facts, window, scripts), or have a reviewer check it commit by commit. Reviewing it as one diff is not practical.
• The deploy order (backfills before thinning, --recheck, --keep all right after the deploy) is easy to get wrong. A runbook under docs/ or scripts/ops/README would help.

LGTM on what I could inspect. Please get a human review of the unreviewed parts, especially the multi-worker paths (finalize, refresh and the CAS commit pointer). Per src/server/AGENTS.md, run scripts/multiworker_gate/ before merge.

[Chen-zexi]

@codex review

[Chen-zexi]

Claude Opus 5.5, automated agent reply

1. Split: unchanged, as answered on the earlier reviews. The nine commits each pass the suites on their own, and the body maps them to the five parts for a commit-by-commit read.
2. Runbook: the deploy order lives in the body's deploy section for now. A runbook under scripts/ops/ is on the follow-up list.
3. Multi-worker gate: it ran on an earlier tip of this work. Cells 1, 2, 7, 9, 10, 13, 14 and 17 passed, and cell 18 stopped on a stack config value this branch does not touch. Extra cross-worker checks also passed: finalize against concurrent replays on the other worker, a worker killed between finalize and refresh, and the CAS-advanced pointer after /summarize on the other worker. It has not been rerun on this head, so it stays on the pre-merge list.

[chatgpt-codex-connector]

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 0930c2ed18

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

[Chen-zexi]

@codex review

[claude]

No blocking issues found, but this review is partial. The diff is 21k lines across 178 files, and GitHub refuses to serve it (HTTP 406, over 20,000 lines). I read only the migrations (064/065 header and 064 body), compaction/window.py and services/history/window.py.

🚫 Blocking: none in what I read. The trim is guarded by anchor, run boundary and a server-side coverage check before it drops anything, and failures leave the thread untrimmed. Migration 064 is idempotent, sets lock_timeout, and has a clean downgrade.

💡 Non-blocking

• A PR this size is hard to review safely. The commit-by-commit split in the description helps, but the operator scripts (thin_message_snapshots, repair_crossed_threads) rewrite checkpoint data and deserve their own review pass. Consider splitting them out. They run dry by default and have rollback checks, which is good.
• Deploy ordering is load-bearing: backfill facts, then slices with --recheck, then --keep all. A deploy note or runbook doc in the repo, not only the PR body, would help operators.

I did not review the replay/refresh pipeline, sse_producer extraction, public share rewrite, or the scripts. Treat the result as unreviewed for those areas.

[Chen-zexi]

Claude Opus 5.5, automated agent reply

1. Split and the operator scripts: unchanged, as answered on the earlier reviews. The scripts are their own commits (34326ffec, 0953391a4, 8d7c5ae75), so a reviewer can take them as a separate pass, and the human review of the areas you did not read is on the pre-merge list.
2. Runbook: the deploy order stays in the body's deploy section for now; a runbook under scripts/ops/ is on the follow-up list.

[chatgpt-codex-connector]

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 8d7c5ae75a

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

[claude]

Review scope: partial. GitHub's API refuses this diff (21k lines, 178 files, over the 20,000-line limit), and CodeRabbit skipped it for the same reason. I read the PR description and migrations 064 and 065 only. I did not review the rest of the code.

No blocking issues in the migrations.

💡 Non-blocking

• This PR is too large for anyone to review properly. The description already splits it into five independent parts, each of which passes the suite on its own. Landing them as stacked PRs (turn slices, replay facts, sliding window, ops scripts, crossed-thread repair) would give each part a real review. The sliding-window change to main agent state (main_state.py, graph.py, compaction) changes what the model sees, so it deserves its own review.
• migrations/versions/064_turn_slices.py: turn_slices.conversation_thread_id has no foreign key to conversation_threads. Cleanup relies on the conversation_responses cascade. That works, but task_run_slices does have the FK, so the two tables are inconsistent.

[github-actions]

Sandbox Integration Test Metrics

Daytona

Category	Ops	Mean	Median	P95	Max
code_run	6	0.10s	0.08s	0.17s	0.20s
exec	6	0.28s	0.24s	0.46s	0.49s
file_io	8	0.14s	0.10s	0.35s	0.47s
full_lifecycle	11	0.28s	0.08s	0.81s	0.84s
lifecycle	13	0.82s	0.48s	2.80s	5.00s
metadata	4	0.04s	0ms	0.15s	0.17s
persistence	3	2.98s	1.50s	5.61s	6.07s

Total: 51 ops, 26.32s, 100% success

---

_{Generated by sandbox-integration CI}

[github-actions]

Hot Path Integration Test Results

⏱️ Operation Timings

_{Actual operation duration vs pytest total (which includes fixture setup and cold path prerequisites)}

Operation	Actual	Test Total
cold has ready	6µs	180ms
cold create	5.91s	6.23s
warm has ready	12µs	7.16s
warm get session	621ms	6.84s
warm zero db	617ms	7.32s
warm cooldown	632ms	8.08s
expired resync	605ms	6.31s
cold create ws1	6.08s	11.99s
cold create ws2	5.43s	11.99s

✅ Cold / Warm Session Path

_{Session resolution with real PostgreSQL + Daytona sandbox}

Test	Duration	Result
cold path has ready session returns false	180ms	✅
cold path creates initialized session	6.23s	✅
warm path has ready session returns true	7.16s	✅
warm path returns same session object	6.84s	✅
warm path makes only the narrow identity query	7.32s	✅
sync cooldown respected	8.08s	✅
cooldown expired triggers sync	6.31s	✅
multiple workspaces independent sessions	11.99s	✅

✅ has_ready_session Edge Cases

_{Sync check accuracy for various session states}

Test	Duration	Result
no session cached	6ms	✅
session not initialized	5ms	✅
sandbox none	9ms	✅
sandbox not ready	5ms	✅
full ready state	7.88s	✅

✅ Conditional update_workspace_activity

_{60-second conditional SQL UPDATE behavior}

Test	Duration	Result
first call writes	180ms	✅
immediate second call skips	190ms	✅
nonexistent workspace returns false	175ms	✅
deleted workspace returns false	181ms	✅

Total: 17 tests, 62.75s, 100% pass

---

_{Generated by backend-integration CI}