From a45346cadb5691510d9f59fb247c6ba0bd8a3f20 Mon Sep 17 00:00:00 2001 From: Joob1n Date: Thu, 3 Sep 2026 22:15:37 +0800 Subject: [PATCH] fix(runtime): retreat a rejected fold to a span the provider has accepted When the summarizer's own provider rejects a fold as too large, the planner halved the covered range and tried again. Halving is a guess in both directions: it can discard verbatim history the summarizer would have taken, and it can still be too large, paying another round trip to find out. There is a boundary that needs no guessing. The last accepted request's input covered everything before the newest model reply began; that span was accepted by this model on this connection, so it is provably within the provider's capacity. The fold retreats to it once. A rejection of that span too is the provider saying this fold cannot be made, and the fold fails open with the summarizer's own reason rather than a span-selection one. The boundary is read from the ledger rather than persisted: the newest model reply is the end of the proven span whether or not it sits at the tail, so a turn's first request finds the previous turn's reply. A ledger with no model reply has nothing proven and gets no retreat, because inventing a boundary is the guess this change removes. Refs #4559, #4634 Generated-by: Claude Code Claude-Session: https://claude.ai/code/session_014ajaRxC4jydavY9nYUFj5J --- CHANGELOG.md | 1 + .../src/__tests__/history-compaction.test.ts | 214 +++++++++++++++++- .../overflow-reactive-recovery.test.ts | 12 +- packages/runtime/src/ai-sdk-compaction.ts | 5 + packages/runtime/src/history-compaction.ts | 71 +++++- 5 files changed, 287 insertions(+), 16 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index bab1cf0dc6..ead049df6d 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -48,6 +48,7 @@ owner, immutable request snapshots remain enforced at AgentRun acceptance and backend dispatch, and SessionEvent-to-RuntimeEvent conversion remains a pure mapper. - Retired the Task Ledger domain: SessionTodo is now the sole authority for in-session work items, and the operational-state schema drops the `workflow_task_ledger_events` table on first open. **Unfinished Tasks are not migrated and are permanently deleted.** This affects workspaces last opened by `v0.1.0` through `v0.1.11`, `cli-v0.1.0-beta.1`, `v0.2.0-incubating-rc1`, or a `v0.2.0-dev` build; those releases wrote Tasks to a table that no shipped build ever bridged into SessionTodo. Before opening such a workspace with this build, finish or export the Tasks you still need, or copy the workspace's `runtime.sqlite` aside — the migration removes the only live copy, so afterwards recovery requires a backup made in advance. +- A compaction rejected as too large for the summarizer's own window now retreats to the span the last accepted request's input covered, instead of halving the covered range. That span is the newest reply this route produced, found through the run headers, so it was accepted by this model on this connection and is provably within capacity; halving can overshoot (discarding verbatim history for nothing) or undershoot (paying another round trip), and a span another route accepted proves nothing at all. One retreat, then the fold fails open and the provider decides. - `token_usage` anchors now record the model and connection that produced them. A token count is a number in one model's tokenizer against one connection; carrying the route on the record lets any reader apply the rule the runtime already enforces, instead of pairing one model's usage with another model's window. The record decodes against a closed allowlist, so sessions written with these keys do not open in earlier releases, and the Runtime Host compatibility epoch moves to 107. - Let the provider decide whether a request fits. Proactive compaction now uses only a user-declared Maka window and the previous accepted request's provider-reported `inputTokens + outputTokens`; no declaration means no proactive capacity threshold. `/models` and generated model metadata are display hints, not limits. `token_usage` records persist the last-request anchor under `lastRequestAnchor`; its new `{ inputTokens, outputTokens }` shape still decodes the retired `payloadChars` key from older sessions. Requests that are too large are compacted and retried once after a real provider rejection, then reported as a `context_overflow` provider error. Compaction is entered at most once per send, and a request rejected after a fold was actually applied is reported as still too large after compaction. A fold that failed open makes no such claim: that request went out with its full raw history. A reply cut at `finishReason: length` no longer triggers a fold, because the provider running out of window room and the provider's own lower output cap are indistinguishable from outside. Five system notes explain the provider-side cases: dropping context, a window worth declaring, an exchange past the declared window, a request accepted past the window the model reports (once per crossing, while nothing is declared), and a request still too large after compaction. The reply reserve that arms the proactive threshold is twice the last real reply, bounded at 8,000 tokens, rather than the model's maximum output. **Sessions this build writes do not open in earlier releases:** those decode `token_usage` against a closed allowlist, so the reshaped `lastRequestAnchor` key fails the record and, with it, the Session that contains it; downgrading therefore needs a copy of the workspace's `runtime.sqlite` taken before the upgrade. Nothing produces the `context_budget_exhausted` stop reason any more — a request that really is too large is compacted and retried once, then reported as a `context_overflow` provider error — though sessions that already recorded it still decode and present. The Runtime Host compatibility epoch moves to 106. - Unified context management under one Runtime-owned policy. `MAKA_CONTEXT_*` environment overrides no longer tune or disable compaction and Tool Result pruning; model-visible archive placeholders are read on demand through bounded `ArchiveRead` calls instead of eager hydration. Previously supported overrides are ignored on upgrade: if Tool Result pruning was set to `off`, pruning is re-enabled, and there is currently no supported replacement opt-out. diff --git a/packages/runtime/src/__tests__/history-compaction.test.ts b/packages/runtime/src/__tests__/history-compaction.test.ts index 7dd3f7e580..64432af88e 100644 --- a/packages/runtime/src/__tests__/history-compaction.test.ts +++ b/packages/runtime/src/__tests__/history-compaction.test.ts @@ -19,6 +19,7 @@ import assert from 'node:assert/strict'; import { describe, test } from 'node:test'; +import type { AgentRunHeader } from '@maka/core/agent-run'; import type { RuntimeEvent } from '@maka/core/runtime-event'; import { applyRuntimeEventHistoryCompact, @@ -176,7 +177,10 @@ describe('plan context compaction', () => { assert.deepEqual(result.replacementEvents[1], events[2]); }); - test('retreats the safe prefix by half for each input-too-large rejection', async () => { + test('retreats to the span the last accepted input covered', async () => { + // The newest reply's events end that span: everything before the first of + // them was in the request the provider accepted, so it is provably within + // capacity. Halving would be a guess in either direction (#4559). const events = [ user('old-user', 'old-turn'), model('old-model', 'old-turn', 'old result'), @@ -188,10 +192,12 @@ describe('plan context compaction', () => { planInput({ phase: 'standalone', orderedEvents: events, + runHeaders: HEADERS_A, + acceptedRoute: ROUTE_A, reserveTailEvents: 0, summarize: ({ coveredRuntimeEvents }) => { attemptedCoverage.push(coveredRuntimeEvents.map((event) => event.id)); - if (attemptedCoverage.length <= 2) { + if (attemptedCoverage.length === 1) { throw new HistoryCompactSummarizerError('input_too_large'); } return structuredSummary('A bounded automatic summary.'); @@ -203,24 +209,195 @@ describe('plan context compaction', () => { if (result.decision !== 'compacted') return; assert.deepEqual(attemptedCoverage, [ ['old-user', 'old-model', 'recent-user', 'recent-model'], - ['old-user', 'old-model'], - ['old-user'], + ['old-user', 'old-model', 'recent-user'], ]); assert.deepEqual( result.tailRuntimeEvents.map((event) => event.id), - ['old-model', 'recent-user', 'recent-model'], + ['recent-model'], + ); + }); + + test('a later fold rolls over the reply the retreat left verbatim', async () => { + // The retreat keeps the newest reply out of the fold, so it stays in the + // request as raw text. It does not stay there: the next fold covers it, + // rolling the checkpoint forward, because by then a newer reply ends the + // proven span. This bounds how long a retreat's leftover survives. + const first = [ + user('old-user', 'old-turn'), + model('old-model', 'old-turn', 'old result'), + user('recent-user', 'recent-turn'), + model('recent-model', 'recent-turn', 'recent result'), + ]; + let attempts = 0; + const retreated = await planHistoryCompaction( + planInput({ + phase: 'standalone', + orderedEvents: first, + runHeaders: HEADERS_A, + acceptedRoute: ROUTE_A, + reserveTailEvents: 0, + summarize: () => { + attempts += 1; + if (attempts === 1) throw new HistoryCompactSummarizerError('input_too_large'); + return structuredSummary('A bounded automatic summary.'); + }, + }), + ); + assert.equal(retreated.decision, 'compacted'); + if (retreated.decision !== 'compacted') return; + assert.deepEqual( + retreated.tailRuntimeEvents.map((event) => event.id), + ['recent-model'], + ); + + // The turn continues: a newer reply arrives, so the proven span now ends + // after the one the retreat spared. + const later = [...first, user('next-user', 'next-turn'), model('next-model', 'next-turn')]; + const rolled = await planHistoryCompaction( + planInput({ + phase: 'standalone', + orderedEvents: later, + runHeaders: HEADERS_A, + acceptedRoute: ROUTE_A, + reserveTailEvents: 0, + previousCheckpoint: retreated.checkpoint, + summarize: () => structuredSummary('A bounded automatic summary.'), + }), + ); + assert.equal(rolled.decision, 'compacted'); + if (rolled.decision !== 'compacted') return; + assert.equal( + rolled.coveredRuntimeEvents.some((event) => event.id === 'recent-model'), + true, + ); + }); + + test('fails open when the proven boundary is refused too', async () => { + // One retreat, because there is one proven boundary. A rejection of that + // span is the provider saying this fold cannot be made. + const events = [ + user('old-user', 'old-turn'), + model('old-model', 'old-turn', 'old result'), + user('recent-user', 'recent-turn'), + model('recent-model', 'recent-turn', 'recent result'), + ]; + let attempts = 0; + const result = await planHistoryCompaction( + planInput({ + phase: 'standalone', + orderedEvents: events, + runHeaders: HEADERS_A, + acceptedRoute: ROUTE_A, + reserveTailEvents: 0, + summarize: () => { + attempts += 1; + throw new HistoryCompactSummarizerError('input_too_large'); + }, + }), + ); + + assert.equal(attempts, 2); + assert.equal(result.decision, 'fail_open'); + if (result.decision !== 'fail_open') return; + assert.equal(result.diagnosticReason, 'input_too_large'); + }); + + test("a mixed-route session retreats to this route's own newest reply", async () => { + // History can span runs on several routes. A span another model accepted + // proves nothing about this summarizer's window, so the retreat targets the + // newest reply THIS route produced, found through the run headers. + const events = [ + user('old-user', 'old-turn'), + modelOnRun('mine', 'old-turn', 'run-1', 'accepted by this route'), + user('switch-user', 'switch-turn'), + modelOnRun('theirs', 'switch-turn', 'run-2', 'accepted by another route'), + ]; + const attemptedCoverage: string[][] = []; + const result = await planHistoryCompaction( + planInput({ + phase: 'standalone', + orderedEvents: events, + runHeaders: [ + runHeader('run-1', 'model-a', 'conn-a'), + runHeader('run-2', 'model-b', 'conn-b'), + ], + acceptedRoute: ROUTE_A, + reserveTailEvents: 0, + summarize: ({ coveredRuntimeEvents }) => { + attemptedCoverage.push(coveredRuntimeEvents.map((event) => event.id)); + if (attemptedCoverage.length === 1) { + throw new HistoryCompactSummarizerError('input_too_large'); + } + return structuredSummary('A bounded automatic summary.'); + }, + }), ); + + assert.equal(result.decision, 'compacted'); + // Not ['old-user', 'mine', 'switch-user'], which stops at the other route's + // reply: that span is proven only for the model that accepted it. + assert.deepEqual(attemptedCoverage, [ + ['old-user', 'mine', 'switch-user', 'theirs'], + ['old-user'], + ]); }); - test('fails open after repeated input-too-large retreat reaches no safe span', async () => { + test('fails open when only another route has ever been accepted', async () => { + let attempts = 0; const result = await planHistoryCompaction( planInput({ + phase: 'standalone', + orderedEvents: [user('u1', 't1'), modelOnRun('theirs', 't1', 'run-2')], + runHeaders: [runHeader('run-2', 'model-b', 'conn-b')], + acceptedRoute: ROUTE_A, + reserveTailEvents: 0, summarize: () => { + attempts += 1; throw new HistoryCompactSummarizerError('input_too_large'); }, }), ); - assert.deepEqual(result, { decision: 'fail_open', reason: 'no_safe_completed_span' }); + + assert.equal(attempts, 1); + assert.equal(result.decision, 'fail_open'); + }); + + test('fails open without retrying when no accepted reply proves a boundary', async () => { + // No model reply anywhere, so no request has ever been accepted on this + // ledger; inventing a boundary would be the guess this change removes. + let attempts = 0; + const result = await planHistoryCompaction( + planInput({ + phase: 'standalone', + orderedEvents: [user('u1', 't1'), user('u2', 't1'), user('u3', 't2')], + runHeaders: HEADERS_A, + acceptedRoute: ROUTE_A, + reserveTailEvents: 0, + summarize: () => { + attempts += 1; + throw new HistoryCompactSummarizerError('input_too_large'); + }, + }), + ); + + assert.equal(attempts, 1); + assert.equal(result.decision, 'fail_open'); + }); + + test('fails open on an input-too-large rejection with the summarizer reason', async () => { + // The retreat is bounded by what a provider has already accepted, so a + // rejection that outlives it fails open carrying the summarizer's own + // reason rather than a span-selection one. + const result = await planHistoryCompaction( + planInput({ + summarize: () => { + throw new HistoryCompactSummarizerError('input_too_large'); + }, + }), + ); + assert.equal(result.decision, 'fail_open'); + if (result.decision !== 'fail_open') return; + assert.equal(result.diagnosticReason, 'input_too_large'); }); test('persisted checkpoint replay-validates against the same ledger prefix (recovery)', async () => { @@ -379,6 +556,29 @@ function user(id: string, turnId: string): RuntimeEvent { function model(id: string, turnId: string, text: string = id): RuntimeEvent { return { ...base(id, turnId), role: 'model', author: 'agent', content: { kind: 'text', text } }; } +/** A model reply produced by a named run, so a route can be attached to it. */ +function modelOnRun(id: string, turnId: string, runId: string, text: string = id): RuntimeEvent { + return { ...model(id, turnId, text), runId, invocationId: runId }; +} +function runHeader(runId: string, modelId: string, llmConnectionId: string): AgentRunHeader { + return { + runId, + sessionId: 'session-1', + turnId: 'turn-1', + status: 'completed', + backendKind: 'ai-sdk', + llmConnectionId, + llmConnectionSlug: llmConnectionId, + modelId, + cwd: '/tmp/maka', + permissionMode: 'ask', + createdAt: 1_800_000_000_000, + updatedAt: 1_800_000_000_000, + }; +} +const ROUTE_A = { modelId: 'model-a', connectionId: 'conn-a' }; +const HEADERS_A = [runHeader('run-1', 'model-a', 'conn-a')]; + function call(id: string, callId: string, turnId: string): RuntimeEvent { return { ...base(id, turnId), diff --git a/packages/runtime/src/__tests__/overflow-reactive-recovery.test.ts b/packages/runtime/src/__tests__/overflow-reactive-recovery.test.ts index 442dc2104f..8166923445 100644 --- a/packages/runtime/src/__tests__/overflow-reactive-recovery.test.ts +++ b/packages/runtime/src/__tests__/overflow-reactive-recovery.test.ts @@ -1175,6 +1175,10 @@ describe('reactive overflow recovery in the streaming backend', () => { }); test('step-0 overflow recovery gates reasoning on retry and durable reload', async () => { + // The subject here is reasoning gating across the retry and the durable + // reload, not how many times a fold may retreat: the retreat is bounded by + // the one span a provider has already accepted, so the summarizer answers + // on its first call. let summarizeCalls = 0; const fixture = buildReactiveFixture({ script: ['overflow', 'tool', 'done'], @@ -1182,16 +1186,16 @@ describe('reactive overflow recovery in the streaming backend', () => { reasoningReplayTail: true, summarize: (input) => { summarizeCalls += 1; - if (summarizeCalls <= 2) { - throw new HistoryCompactSummarizerError('input_too_large'); - } + if (summarizeCalls === 1) throw new HistoryCompactSummarizerError('input_too_large'); return reactiveStructuredSummary(input.source.foldedRuntimeEvents); }, }); await runTurn(fixture); assert.equal(fixture.model.doStreamCalls.length, 3); - assert.equal(fixture.summarizerCalls(), 3); + // One retreat, to the span the last accepted input covered, which leaves + // the reasoning tail verbatim. + assert.equal(fixture.summarizerCalls(), 2); for (const call of fixture.model.doStreamCalls.slice(1)) { const prompt = JSON.stringify(call.prompt); assert.match(prompt, /REACTIVE_SUMMARY_SENTINEL/); diff --git a/packages/runtime/src/ai-sdk-compaction.ts b/packages/runtime/src/ai-sdk-compaction.ts index 09fb46fb0d..256b4fd0e0 100644 --- a/packages/runtime/src/ai-sdk-compaction.ts +++ b/packages/runtime/src/ai-sdk-compaction.ts @@ -1069,6 +1069,11 @@ export class AiSdkCompaction { phase: input.phase ?? 'mid_turn', orderedEvents, headAnchor: { runtimeEventId: state.headAnchor.id, turnId }, + runHeaders: state.priorRunHeaders, + acceptedRoute: { + modelId: this.input.modelId, + ...(this.targetConnectionId !== undefined ? { connectionId: this.targetConnectionId } : {}), + }, reserveTailEvents: 1, charsPerToken, now: this.now(), diff --git a/packages/runtime/src/history-compaction.ts b/packages/runtime/src/history-compaction.ts index ef4acfdd49..268aa59f50 100644 --- a/packages/runtime/src/history-compaction.ts +++ b/packages/runtime/src/history-compaction.ts @@ -17,6 +17,7 @@ * under the License. */ +import type { AgentRunHeader } from '@maka/core/agent-run'; import type { RuntimeEvent } from '@maka/core/runtime-event'; import type { ContextBudgetDiagnostic } from '@maka/core/usage-stats/types'; import { finitePositive } from './context-budget-helpers.js'; @@ -183,6 +184,14 @@ export interface PlanHistoryCompactionInput { highWaterName?: string; highWaterSeq?: number; previousCheckpoint?: HistoryCompactCheckpoint; + /** + * Run headers for the ordered events, and the route this fold is dispatched + * on. Together they name the newest reply this route produced, which is the + * only span a retreat may target: a rejection of a larger one says nothing + * about a span another model accepted. + */ + runHeaders?: readonly AgentRunHeader[]; + acceptedRoute?: { modelId: string; connectionId?: string }; /** Present only when this automatic Compaction should create a Memory task. */ memoryExtractionBoundary?: HistoryCompactMemoryExtractionBoundary; summarize: HistoryCompactionSummarizer; @@ -214,6 +223,41 @@ export type HistoryCompactionFailReason = 'no_safe_completed_span' | 'summarizer * when it cannot fold a safe completed prefix it FAILS OPEN (keep the raw * projection + diagnostic) and the request goes out unchanged. */ +/** + * How many ordered events the last request THIS ROUTE had accepted covered. + * + * A span is only proven for the model and connection that accepted it: a token + * count is a number in one tokenizer, and a session's history can span runs on + * several routes. So the newest reply produced on the summarizer's own route + * ends the span, found through the run headers rather than by role alone — + * everything before its first event was in a request that route accepted. + * A ledger with no reply from this route has nothing proven, and the caller + * must not invent a boundary. + */ +function acceptedInputBoundary( + events: readonly RuntimeEvent[], + runHeaders: readonly AgentRunHeader[], + route: { modelId: string; connectionId?: string } | undefined, +): number | undefined { + if (!route) return undefined; + const onRoute = (event: RuntimeEvent | undefined): boolean => { + if (event?.role !== 'model') return false; + const header = runHeaders.find((candidate) => candidate.runId === event.runId); + if (!header || header.modelId !== route.modelId) return false; + return header.llmConnectionId === route.connectionId; + }; + let index = -1; + for (let cursor = events.length - 1; cursor >= 0; cursor -= 1) { + if (onRoute(events[cursor])) { + index = cursor; + break; + } + } + if (index < 0) return undefined; + while (index > 0 && onRoute(events[index - 1])) index -= 1; + return index; +} + export async function planHistoryCompaction( input: PlanHistoryCompactionInput, ): Promise { @@ -282,11 +326,28 @@ export async function planHistoryCompaction( if (error instanceof HistoryCompactSummarizerError) { if (error.reason === 'input_too_large') { // The summarizer's provider said this span does not fit its own - // window; that is the only fit signal the fold listens to. Retreat by - // half rather than by one event: each retreat is a real provider - // round trip, and the loop exits through no_safe_completed_span when - // even the smallest legal span is refused. - maxCoveredCount = Math.floor(boundary.coveredCount / 2); + // window; that is the only fit signal the fold listens to. Retreat to + // the span the last accepted request's input covered: that span was + // accepted by this model on this connection, so it is provably within + // capacity, where halving the range is a guess that can overshoot + // (throwing away verbatim history for nothing) or undershoot (paying + // another round trip). Only one retreat is available, because there + // is only one proven boundary; a rejection of that span too is the + // provider saying this fold cannot be made, and the fold fails open + // (#4559). + const proven = acceptedInputBoundary( + input.orderedEvents, + input.runHeaders ?? [], + input.acceptedRoute, + ); + if (proven === undefined || proven >= boundary.coveredCount) { + return { + decision: 'fail_open', + reason: 'summarizer_failed', + diagnosticReason: error.reason, + }; + } + maxCoveredCount = proven; continue; } return {