Skip to content

ENG-664: expose compaction result for host persistence + skip low-yield re-summarization - #249

Draft
ea-rus wants to merge 4 commits into
stagingfrom
feat/eng-664-conversations-resend-the-full-history
Draft

ENG-664: expose compaction result for host persistence + skip low-yield re-summarization#249
ea-rus wants to merge 4 commits into
stagingfrom
feat/eng-664-conversations-resend-the-full-history

Conversation

@ea-rus

@ea-rus ea-rus commented Jul 14, 2026

Copy link
Copy Markdown
Contributor

Summary

  • Adds ChatSession.last_compaction — a read-only result of the most recent history compaction this session ran ({"summary": str, "covered_through": int} or None). summary is the exact compacted message content (including the _COMPACTED_MARKER prefix) so a host can seed it back verbatim next turn and have _summarize_history recognize + extend it in place. covered_through is how many of the session's seed (initial_history) messages are now folded into it, letting a host map that count onto its own message list to
    find the cutoff.
  • _summarize_history now skips the LLM round-trip when there's little new material to fold in (excluding any already-carried-forward prior summary from the measurement) — avoids paying for a summarization call that would barely extend an already-large summary.
  • No change to the compaction algorithm itself, and no new required constructor parameters — existing hosts are unaffected.

This is the anton-side half of ENG-664 (cowork-server persists last_compaction across turns instead of resending full history every turn). The once-per-turn compaction guard (_compacted_this_turn) already existed and needed no change.

Test plan

  • New tests/test_history_compaction.py: last_compaction is None before any compaction, populated correctly after one, and the low-yield-skip guard (including the "prior summary doesn't count as new material" case).

Fixes https://linear.app/mindsdb/issue/ENG-664/conversations-resend-the-full-history-every-turn-and-throw-away-the

@Dav1dF

Dav1dF commented Jul 17, 2026

Copy link
Copy Markdown

Read this alongside the prompt-caching work (the gateway's cache alignment from mindsdb/mindshub_inference — ENG-601/ENG-586) to check for a conflict, and wanted to note the opposite on the record: this design is exactly what prefix caching needs.

  • Replaying the persisted summary verbatim keeps the seed byte-stable between compactions, and the tail is append-only — so a cached prefix survives across turns.
  • The low-yield re-summarization skip is also the cache-friendly behavior: every re-summarization rewrites the front of history and resets the cached prefix, so avoiding marginal ones saves twice.

One expectation to write down somewhere (here is fine): when cache alignment turns on in prod, cache-hit dips will correlate with compaction events. That is inherent to any history trimming, and last_compaction / the compaction event give us the attribution — no change needed in this PR, just flagging so nobody reads the dips as a caching bug later.

Linear: ENG-664; caching context on ENG-586.

@ea-rus
ea-rus changed the base branch from jorge/eng-648-reduce-anton-cowork-server-token-usage-promptskills to staging July 27, 2026 14:15
@ea-rus
ea-rus changed the base branch from staging to jorge/eng-648-reduce-anton-cowork-server-token-usage-promptskills July 27, 2026 14:16
@ea-rus
ea-rus force-pushed the feat/eng-664-conversations-resend-the-full-history branch from 108732e to 67cb4a2 Compare July 28, 2026 07:50
@ea-rus
ea-rus changed the base branch from jorge/eng-648-reduce-anton-cowork-server-token-usage-promptskills to staging July 28, 2026 07:50
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants