Skip to content

fix(chat): reject empty Claude Code responses - #5091

Merged
huangruiteng merged 1 commit into
loopx-project:mainfrom
Duang777:codex/fix-claude-empty-success
Sep 26, 2026
Merged

huangruiteng merged 1 commit into
loopx-project:mainfrom
Duang777:codex/fix-claude-empty-success

Conversation

@Duang777

Copy link
Copy Markdown
Collaborator

Goal And Delivered Outcome

  • Goal/source and gap: Claude Code could exit successfully without emitting a usable stream-json answer, but LoopX still parsed an empty string and completed the Chat Turn.
  • Observable before -> after, with the validation row that proves it: empty stdout, malformed-only stdout, and an empty result event now raise provider_empty_response without publishing answer.final; non-empty result and text_delta events still complete normally.
  • Issue/task and intended base: standalone correctness fix against main; no linked issue.

Scope And Continuation

  • Completed scope and remaining work: complete within the Claude Code CLI adapter. Other provider adapters use different terminal protocols and are unchanged.
  • Slice boundary / successor: N/A for this fix. The fail-closed check is local to Claude Code's stream-json contract.

Validation

  • Tested revision: d9119bd366c778b69cefd3dc189f86a335f6192c
  • Run state: finished
  • Input classes: synthetic
Check kind Result Public-safe evidence / limitation
regression_parity passed The new empty and malformed stdout cases failed before the implementation and passed after it.
unit passed python -m pytest tests/test_chat_agent.py -q: 43 passed.
integration passed All Chat-related Python test modules: 403 passed, 1 existing duplicate-archive warning.
static passed python -m ruff check loopx/chat_providers.py tests/test_chat_agent.py.
real_entrypoint passed loopx canary premerge --from-git-diff: all selected checks passed.
  • Coverage and gaps: tests cover no stdout, malformed-only stdout, an empty valid result event, and both accepted answer event forms. No live Claude account was used; subprocess protocol behavior is exercised with synthetic stdout.

Frontend / Visual Evidence

  • UI impact: none
  • Before: N/A
  • After: N/A
  • States and viewports shown: N/A
  • Source data: none
  • Attention review: N/A

Type of Change

  • Bug fix
  • New feature
  • Breaking change
  • Refactoring (no functional changes)
  • Documentation update
  • Test update

LoopX Area

  • Control plane (goals, todos, quota, scheduler, registry, runtime)
  • Benchmark boundary (adapters, runners, verifiers, scoring, evidence)
  • Capability or extension (providers, adapters, skills)
  • Public docs or presentation surface (README, protocols, dashboard)
  • Build, packaging, installer, or CI
  • Host or runtime integration

Technical Direction

  • Direction / acceptance reference, when applicable: fail closed at the provider protocol boundary before publishing a successful terminal event.

Shared-authority RFC fixture impact

N/A. This change does not alter shared Goal Authority or TypeScript control-plane behavior.

Boundary Checklist

  • Neither the diff nor this PR body/comments/attachments disclose private state, credentials, raw traces or verifier output, internal links, or local machine paths (including .loopx/, .codex/goals/, and live ACTIVE_GOAL_STATE.md).
  • I did not duplicate maintainer-owned benchmark work unless a maintainer split out a public issue for it.
  • I kept the change scoped to the linked issue/task.
  • I completed the visual evidence section for UI changes, or marked UI impact none.
  • Every commit includes a DCO Signed-off-by trailer (git commit -s).

Signed-off-by: duanjialing.777 <duanjialing.777@bytedance.com>

@Duang777 Duang777 left a comment

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Exact-head self-review for d9119bd: APPROVE for maintainer review. Claude Code exit zero with no assistant text now raises a provider error instead of emitting answer.final and completing the Turn; focused empty-output cases are covered. The remaining installed-artifact failure is the main-branch packaging defect tracked by #5096.

@huangruiteng huangruiteng left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Exact-head independent review: d9119bd366c778b69cefd3dc189f86a335f6192c,base 71dbfd5e605da4e7efdb58c87cd3f4a96d39e0e5。

动机

Claude 进程 exit 0 只表示进程退出正常,不证明它产生了回答。原实现会把空 stdout、无法识别的行或者空 result 解析为 message="",仍发送 answer.final,让 Chat Turn 进入 completed。用户看到“完成”却没有收到回答,诊断与重试也失去真实失败原因。本次修复的是 Claude stream-json 传输输出边界,不应外推为所有 provider/结构化回答语义都已解决。

改动思路

在既有 ClaudeCodeAdapter 收集输出之后、parse_agent_response 和 answer.final 之前增加明确的空回答检查。text_delta 拼接后若只有空白,可以使用终端 result;两者都空则抛 provider_empty_response,并复用现有 host_tool_gate 和 Chat runtime 的失败持久化路径。有效回答仍走原共享 parser/过滤器,不新增 parser、协议版本、配置或权限。

具体改动

  • loopx/chat_providers.py 的 Claude adapter 增加 12 行、删除 1 行:区分空白 stream 与可用 result,禁止原始空回答成功终结。其他 adapters 和非零退出分支没有改变。
  • tests/test_chat_agent.py 增加 87 行:三种空/无效输出拒绝,没有 answer.final;有效 result 与 text_delta 仍完成,upstream session 与 resumed 状态正常。

关键代码讲解:

  1. ClaudeCodeAdapter.start_turn(chat_providers.py:168,新增分支 :243):空输出异常发生在成功终结和 resumed=True 前;existing finally 已清理 current_process。它拒绝的是确定的空输出,不是根据消息中的词语猜测“失败”。
  2. parse_agent_response(chat.py:372):有效输出仍由已有共享规范化 owner 处理,保留路径脱敏和结构化 proposal/gate 语义,没有为 Claude 再造决策规则。
  3. ChatRuntimeController._fail_turn(chat_runtime.py:1362):existing catch 接到 typed error 后写 failed/error event,释放 active_turn。前端 data/chat.ts 的 turn.failed 消费路径抛 ChatApiError,而有效回答从 turn.completed 读取最终 response,故不需要新的 frontend 字段。

对主干的风险

我独立测试了完整的 adapter → runtime → File ChatSessionStore 链路,subprocess 没有被 Popen mock 替代;只有上游 stream-json producer 是确定的合成可执行程序,没有调用真实 Claude 账户:

  • 同一 fixture 在 base 对 empty/malformed/empty result/whitespace 四类全部 completed 且 message="";head 全部 failed/provider_empty_response,无 answer.final,并通过 turn.failed 传递 host_tool_gate。
  • 随后在同一 Session 提交有效 result 或 text_delta,head 均 completed/"Completed.",active_turn 清理;纯空白 stream + 有效 result 也恢复为正确最终回答。非零退出仍 provider_unavailable,后续有效 Turn 仍可继续。
  • 79 个 agent/parser/session/event 专项测试通过;changed Python Ruff、仓库配置 mypy 的 23 source files、5 个直接 + 2 个选中 premerge 检查通过。第一次 premerge 缺 Node 依赖,补齐后复跑通过。
  • 扩大到整个 Chat 相关测试选择:base 508 passed,head 513 passed(pytest -q tests -k chat)。新 worktree 最初 12 个失败都因 packaged Chat bundle 未构建;在 dashboard 目录补齐 npm ci --ignore-scripts 和 npm run build:chat 后,同一 head 同一命令全部通过,没有修改源码或降低断言。
  • 未配置的扩大 typing 检查在 immutable base/head 产生完全一致的 4,139 个已有错误签名;不因无关基础问题要求修改本 PR。
  • 最新 main 相对本 PR base 的 chat_runtime 改动是 queue worker 唤醒/退出竞争处理,不修改本次调用的 _run_turn/_fail_turn 语义;本次未将作者评论或其他 PR 的绿灯当作证据。

非阻塞的已有边界:若 result 是非空 <loopx-review-json>{"message":"","proposals":[]}</loopx-review-json> 外壳,规范化后仍可能是空回答并 completed。我在 base 与 head 都复现,属于当前共享 parser 的既有语义,不是这 12 行新增的回归,也不否定已声明的三类空传输修复。建议后续在规范化 owner 增加“没有可见 message 且没有可执行 proposal/gate/handoff”检查和负例,保留合法的 action-only 回答;不能简单把所有空 message 都拒绝。这是相邻加固建议,不是本次 REQUEST_CHANGES。

真实 Claude CLI 的账户状态、session 文件及模型输出没有做在线验证;这里的证据覆盖 production subprocess/持久化/前端消费协议,不声称已完成 live provider 资格验证。空白 delta 到 result 的中间 streaming 展示可能短暂空白,但前端最终 response 消费路径已核对,不会丢失终端回答。

我的整体评价

APPROVE,无阻塞发现。修复明确、范围合适,错误状态真实且有重试/切换 endpoint 路径;重复 Turn 和失败后继续的行为有独立验证。未来-facing 检查采用已有 parser/runtime owner,不需要抽一个只被这一分支使用的框架;规范化后空内容的加固可在共享语义 owner 中独立处理。结论只适用于上述 exact head,未执行合并。

English verdict: APPROVE - d9119bd; real subprocess-to-File-store base/head fixtures verify empty-output failure and subsequent successful turns, 79 focused tests, 513 Chat-selected tests and selected premerge checks passed. No blocking finding; normalized empty structured envelopes are an unchanged adjacent hardening opportunity.

@huangruiteng
huangruiteng merged commit 13ab5cc into loopx-project:main Sep 26, 2026
26 of 30 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants