Skip to content

feat(synthesize): self-heal length-truncated batches with minimal-reasoning retry - #68

Merged
alpertarhan merged 1 commit into
mainfrom
fix/issue-62-length-retry
Sep 17, 2026
Merged

alpertarhan merged 1 commit into
mainfrom
fix/issue-62-length-retry

Conversation

@alpertarhan

Copy link
Copy Markdown
Owner

Implements the retry portion of #62 (option 1: self-healing retry, plus the preflight notice from option 3). Review context: #62 (comment)

Why

On local OpenAI-compatible reasoning servers, thinking tokens share the per-call max_tokens budget with the answer. When thinking runs long, the batch synthesis ends with stop reason length — a well-formed prefix, but the batch contract (one ### CHUNK n: section per chunk, terminal stop) can never be met. Until now that path went straight to the deterministic fallback, losing LLM summary quality for the whole batch.

What

Retry once at minimal reasoning (src/phases/synthesize.ts):

  • length stop → one retry with reasoning: "minimal" explicitly pinned in the call opts. This caps thinking for providers that honor it, whether the configured level was high or the level was unset (provider default — the trap from the issue).
  • Skipped when the configured level is already minimal (retry would be a no-op).
  • Skipped for non-length format failures (duplicate ids, missing sections — those indicate a different problem).
  • A failed retry still throws BatchSummaryFormatError, so the existing deterministic fallback + warning path is unchanged.

Preflight notice (src/app/steps/synthesize.ts): one info notice per run when summaryThinkingLevel is unset — the provider default may consume the batch output budget; now users know why a batch took two calls instead of guessing.

Budget interaction

A retry at most doubles one batch's wall time; with the raised watchdog ceilings from #67 (merged) the interaction is safe. Retry metrics flow through the normal trackedComplete budget accounting — two real calls are recorded as two.

Verification

  • 985 tests (4 new: retry success asserting per-call reasoning opts, retry-also-fails, minimal-config skip, non-length no-retry), bench, build, release audit — all green

…soning retry (#62)

- batch synthesis that ends with stop reason 'length' retries once with
  reasoning pinned to 'minimal': thinking tokens share the output budget
  on many local OpenAI-compatible servers, so the configured level (or
  provider default when unset) can starve the batch contract
- retry skipped when the configured level is already minimal (no-op) and
  skipped for non-length format failures; a failed retry still throws so
  the existing deterministic fallback covers the batch
- preflight info notice when summaryThinkingLevel is unset: the provider
  default may consume the batch output budget
@alpertarhan
alpertarhan merged commit d45ec90 into main Sep 17, 2026
2 checks passed
@alpertarhan
alpertarhan deleted the fix/issue-62-length-retry branch September 17, 2026 18:37
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant