fix: cover the trim-to-compaction amnesia window with a continuity note#1439
Conversation
trigger_compaction_for_dropped advances the trim watermark synchronously, so dropped rows vanish from LLM context on the very next message, but the compaction LLM call that extracts their facts into MEMORY.md is async and may still be in flight (or may have failed). In that window the agent has amnesia for the dropped range, immediately after the trim, when the dropped content is most likely still topical. The deterministic trim summary the trim turn saw was never persisted, so it existed for one turn only. While the user has a recent (under 60 minutes) 'pending' compaction event, a terse deterministic summary of the covered rows is rebuilt from the durable message rows (same summarize_dropped_messages shape) and injected as a dynamic system-prompt section. Once the event flips to 'completed', the note disappears and MEMORY.md carries the facts: the context now contains either the compacted facts or the note, never neither. Design choices: the summary is recomputed per turn instead of persisted (inputs are durable, the summarizer is cheap and deterministic, no migration needed); the note rides the dynamic half of the prompt so it never busts the history cache; the 60-minute window keeps a permanently stuck event from pinning a stale note forever (that case belongs to the retry sweep, issue #1431). Watermark semantics are untouched. Fixes #1432 Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
|
Warning Review limit reached
More reviews will be available in 10 minutes and 45 seconds. Learn how PR review limits work. Your organization has run out of usage credits. Purchase more in the billing tab. ⌛ How to resolve this issue?After more reviews become available, a review can be triggered using the We recommend that you space out your commits to avoid hitting the rate limit. 🚦 How do rate limits work?CodeRabbit enforces hourly rate limits for each developer per organization. Our paid plans include higher PR review limits than trial, open-source, and free plans. In all cases, reviews become available again over time. During sustained high-volume PR review activity, CodeRabbit may temporarily slow when the next review becomes available. Please see our Fair Usage Limits Policy for further information. ℹ️ Review info⚙️ Run configurationConfiguration used: Path: .coderabbit.yaml Review profile: CHILL Plan: Pro Run ID: 📒 Files selected for processing (3)
✨ Finishing Touches🧪 Generate unit tests (beta)
✨ Simplify code
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
1 participant
Description
Fixes #1432.
trigger_compaction_for_droppedadvances the trim watermark synchronously, so dropped rows vanish from LLM context on the very next message, but the compaction LLM call that extracts their facts into MEMORY.md is async and may still be in flight (or may have failed). In that window the agent has amnesia for the dropped range, at the worst possible moment: immediately after the trim, when the dropped content is most likely still topical. The deterministic trim summary the trim turn itself saw is never persisted, so it exists for one turn only.Mitigation, preserving the watermark semantics untouched:
build_pending_compaction_note(user_id)rebuilds a terse deterministic summary (the existingsummarize_dropped_messagesshape) of the rows covered by the user's recent (under 60 minutes)'pending'compaction events, straight from the durable message rows.'completed', the note disappears and MEMORY.md carries the facts. The acceptance criterion from the issue holds: on the message immediately following a trim, the context contains either the compacted facts or the note, never neither (the pending row is inserted synchronously in phase 1).Deliberate deviation from the issue: the issue proposed persisting the summary on the
CompactionEventrow. Recomputing per turn instead avoids a schema migration (also avoiding an Alembic head collision with #1438's migration 039): the inputs are durable (seq range + message rows are never deleted), the summarizer is deterministic and cheap (no LLM, output capped at 500 chars), and the recompute only runs while a pending event exists. The common case costs one indexed SELECT per turn.Bounds:
Type
Checklist
uv run pytest -v) (2891 passed, 2 skipped)ruff check backend/ && ruff format --check backend/)AI Usage