Skip to content

fix(compaction): enable overflow recovery for Auto (router) sessions - #1152

Open
iamhatesz wants to merge 2 commits into
masterfrom
fix/auto-model-compaction
Open

iamhatesz wants to merge 2 commits into
masterfrom
fix/auto-model-compaction

Conversation

@iamhatesz

Copy link
Copy Markdown
Contributor

Problem

Auto (router) sessions cannot recover from context-window overflow. Upstream's _checkCompaction gates overflow recovery behind a sameModel check that compares the assistant message's provider/model against this.model. In Auto sessions, this.model is kimchi-dev/auto (the user-selected model, kept by design), but the assistant message carries the routed concrete model (e.g. ai-enabler/routed) — stamped by the Auto API provider when it delegates to the concrete model's stream(). So sameModel is always false, the overflow recovery branch never fires, and the session wedges on the 400 instead of compacting and retrying.

The existing installCompactionRecoveryPatch wrapper already handles the other half of this problem — restoring the raw provider error text so isContextOverflow can match it — but it cannot help if sameModel never lets execution reach that check.

Fix

Extended the existing installCompactionRecoveryPatch wrapper in src/upstream-retry-patch.ts to stamp the throwaway message copy with the session model's identity (kimchi-dev/auto) when an Auto session has resolved routing. This makes sameModel pass because the copy's provider/model now match this.model. The original message is never mutated — the stamping operates on the same throwaway copy the wrapper already creates for raw-error restoration. Non-Auto sessions are a no-op, preserving the model-switch guard.

Alternatives Considered

  1. Temporarily swap this.agent.state.model — Set the session model to the routed concrete model during _checkCompaction, then restore it in a finally block. Rejected because _checkCompaction sets this._overflowRecoveryAttempted = true internally — the flag would correctly land on the real session, but concurrent reads (e.g. status-line renders) could briefly see the routed model instead of auto. Stamping the copy avoids mutating session state entirely.

  2. Patch upstream _checkCompaction source via patches/ — Add a source-level patch to modify the sameModel gate in agent-session.js. Rejected because patches are debt — they require tracking issues and expire when upstream changes. Additionally, the effective (routed) model is tracked in Kimchi's router state (stateBySession), which is not accessible from upstream's AgentSession context, making it hard to resolve the correct model identity inside a source patch. The existing wrapper approach already intercepts _checkCompaction cleanly and has access to the session via this, where the router state can be resolved.

Tests

  • 4 unit tests in src/upstream-compaction-recovery.test.ts: Auto stamping, unresolved routing no-op, concrete-session no-op, combined raw-error + model stamping.
  • 1 e2e test in tests/e2e/tui/auto-model.test.ts: Auto session routes to a concrete model, hits a context-length 400, runs compaction summary, and retries successfully.

Upstream's _checkCompaction gates overflow recovery behind a sameModel
check that compares the assistant message's provider/model against
this.model. In Auto sessions, this.model is kimchi-dev/auto but the
assistant message carries the routed concrete model (e.g.
ai-enabler/routed), so sameModel is always false and the session wedges
on the context-length 400 instead of compacting and retrying.

Extend the existing installCompactionRecoveryPatch wrapper in
src/upstream-retry-patch.ts to stamp the throwaway message copy with
the session model's provider/id when an Auto session has resolved
routing. This makes sameModel pass so upstream's overflow recovery
fires. The original message is never mutated; non-Auto sessions are
unaffected, preserving the model-switch guard.

Add 4 unit tests in src/upstream-compaction-recovery.test.ts covering
Auto stamping, unresolved routing, concrete-session no-op, and combined
raw-error + model stamping.

Add an e2e test in tests/e2e/tui/auto-model.test.ts verifying that Auto
sessions compact and recover from a context-window overflow end-to-end.

Co-Authored-By: Kimchi <noreply@kimchi.dev>
@iamhatesz iamhatesz added the bug Something isn't working label Sep 7, 2026
@kimchi-review

kimchi-review Bot commented Sep 7, 2026

Copy link
Copy Markdown

Kimchi Code Review

Property Value
Commit f68a0f6
Author @iamhatesz
Files changed 3
Review status Completed
Comments 1 (1 warning)
Duration 96s

Summary

📊 Review Score: 85/100 (overall code quality — 0 lowest, 100 highest)
⏱️ Estimated effort to review: 3/5 (1 = trivial, 5 = very complex)

🧪 Tests: yes — The PR adds focused unit tests in src/upstream-compaction-recovery.test.ts covering resolved/unresolved Auto routing, concrete (non-Auto) sessions, and the combined raw-error + model-stamping path. It also adds an end-to-end TUI test in tests/e2e/tui/auto-model.test.ts that exercises the full Auto overflow-recovery flow. Coverage is strong for the changed behavior.

📝 Found 1 issue(s). See inline comments for details.

What to expect

Kimchi will analyze the changes in this pull request and post:

  • A summary of the overall changes
  • Inline comments on specific lines with findings categorized by issue type

The review typically completes within a few minutes. This comment will be updated once the review is ready.

Interact with Kimchi
  • @getkimchi review — re-trigger a full review on the latest commit
  • @getkimchi summary — regenerate the PR summary
  • @getkimchi ignore — skip this PR (no review will be posted)
  • Reply to any inline comment to ask follow-up questions or request clarification
Configuration

Reviews are configured by your organization admin.
Review instructions, excluded directories, and severity thresholds can be adjusted per repository in the Kimchi dashboard.


Powered by Kimchi — AI-powered code review by CAST AI

@kimchi-review kimchi-review Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

📊 Review Score: 85/100 (overall code quality — 0 lowest, 100 highest)
⏱️ Estimated effort to review: 3/5 (1 = trivial, 5 = very complex)

🧪 Tests: yes — The PR adds focused unit tests in src/upstream-compaction-recovery.test.ts covering resolved/unresolved Auto routing, concrete (non-Auto) sessions, and the combined raw-error + model-stamping path. It also adds an end-to-end TUI test in tests/e2e/tui/auto-model.test.ts that exercises the full Auto overflow-recovery flow. Coverage is strong for the changed behavior.

📝 Found 1 issue(s). See inline comments for details.

Comment thread src/upstream-retry-patch.ts
…tiveModel

Address review feedback: reference equality on the resolveEffectiveModel
result is fragile against future refactoring that may return a distinct
object representing the same Auto model. Compare by provider/id instead.

Co-Authored-By: Kimchi <noreply@kimchi.dev>
@iamhatesz
iamhatesz marked this pull request as ready for review September 7, 2026 14:50
@readme-ai-writer

readme-ai-writer Bot commented Sep 7, 2026

Copy link
Copy Markdown

📚 No documentation changes were needed for this PR.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bug Something isn't working

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant