Skip to content

Make a rejected plan diagnosable - #30

Merged
justinmclean merged 3 commits into
mainfrom
feat/decompose-rejection-diagnostics
Sep 3, 2026
Merged

Make a rejected plan diagnosable#30
justinmclean merged 3 commits into
mainfrom
feat/decompose-rejection-diagnostics

Conversation

@justinmclean

Copy link
Copy Markdown
Owner

What this changes

Two changes, both about the same failure: a decompose attempt is reported as "response is not valid JSON" and there is no way to see what the model actually said.

_extract_json now decodes the first complete object with JSONDecoder.raw_decode and ignores whatever follows it. json.loads is strict about trailing content, so a model that closes its object and then adds a sentence of commentary -- or leaves a ``` fence unclosed, which defeats the regex path -- was rejected even though a complete, usable plan was sitting right there.

The rejection journal entry gains agent_stdout_path. A 500-character stderr head is enough to see that a plan was rejected and useless for seeing why, because the malformed part of a plan is almost always near its end; the full response now goes to runs//artifacts/ so a rejection is diagnosable after the fact rather than only reproducible. Best effort: a journal without a path, an empty response, or an unwritable directory leaves the entry as it was.

Checklist

  • One work item only — nothing unrelated bundled in
  • make check is green locally
  • Tests cover the change, including refusal paths (the negative cases
    are the product)
  • No test, acceptance criterion, or check was weakened to get green
  • No changes under specs/ (spec changes go through an issue first)
  • Docs updated where behaviour changed (README / USAGE.md)
  • Commit subjects are imperative; Generated-by: trailer present if an
    agent wrote the change

justinmclean and others added 3 commits September 3, 2026 03:43
Two changes, both about the same failure: a decompose attempt is reported
as "response is not valid JSON" and there is no way to see what the model
actually said.

_extract_json now decodes the first complete object with
JSONDecoder.raw_decode and ignores whatever follows it. json.loads is
strict about trailing content, so a model that closes its object and then
adds a sentence of commentary -- or leaves a ``` fence unclosed, which
defeats the regex path -- was rejected even though a complete, usable plan
was sitting right there.

The rejection journal entry gains agent_stdout_path. A 500-character
stderr head is enough to see *that* a plan was rejected and useless for
seeing *why*, because the malformed part of a plan is almost always near
its end; the full response now goes to runs/<id>/artifacts/ so a rejection
is diagnosable after the fact rather than only reproducible. Best effort:
a journal without a path, an empty response, or an unwritable directory
leaves the entry as it was.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01S7Y1ASxt4ox8riauVcYCsF
@justinmclean
justinmclean merged commit a1e79be into main Sep 3, 2026
2 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant