feat(synthesize): self-heal length-truncated batches with minimal-reasoning retry - #68
Merged
Merged
Conversation
…soning retry (#62) - batch synthesis that ends with stop reason 'length' retries once with reasoning pinned to 'minimal': thinking tokens share the output budget on many local OpenAI-compatible servers, so the configured level (or provider default when unset) can starve the batch contract - retry skipped when the configured level is already minimal (no-op) and skipped for non-length format failures; a failed retry still throws so the existing deterministic fallback covers the batch - preflight info notice when summaryThinkingLevel is unset: the provider default may consume the batch output budget
This was referenced Sep 17, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Implements the retry portion of #62 (option 1: self-healing retry, plus the preflight notice from option 3). Review context: #62 (comment)
Why
On local OpenAI-compatible reasoning servers, thinking tokens share the per-call
max_tokensbudget with the answer. When thinking runs long, the batch synthesis ends with stop reasonlength— a well-formed prefix, but the batch contract (one### CHUNK n:section per chunk, terminal stop) can never be met. Until now that path went straight to the deterministic fallback, losing LLM summary quality for the whole batch.What
Retry once at minimal reasoning (
src/phases/synthesize.ts):lengthstop → one retry withreasoning: "minimal"explicitly pinned in the call opts. This caps thinking for providers that honor it, whether the configured level was high or the level was unset (provider default — the trap from the issue).minimal(retry would be a no-op).lengthformat failures (duplicate ids, missing sections — those indicate a different problem).BatchSummaryFormatError, so the existing deterministic fallback + warning path is unchanged.Preflight notice (
src/app/steps/synthesize.ts): oneinfonotice per run whensummaryThinkingLevelis unset — the provider default may consume the batch output budget; now users know why a batch took two calls instead of guessing.Budget interaction
A retry at most doubles one batch's wall time; with the raised watchdog ceilings from #67 (merged) the interaction is safe. Retry metrics flow through the normal
trackedCompletebudget accounting — two real calls are recorded as two.Verification