Skip to content

preserve full ResultMessage field set on response_metadata (both paths) - #2

Open
jasoncarreira wants to merge 1 commit into
thehumanworks:mainfrom
jasoncarreira:fix/preserve-result-message-fields-in-streaming
Open

preserve full ResultMessage field set on response_metadata (both paths)#2
jasoncarreira wants to merge 1 commit into
thehumanworks:mainfrom
jasoncarreira:fix/preserve-result-message-fields-in-streaming

Conversation

@jasoncarreira

Copy link
Copy Markdown

Summary

_generate and _astream emit asymmetric generation_info shapes, so calling invoke vs ainvoke/astream against ClaudeCodeChatModel produces AIMessages with different response_metadata keys — even when the underlying SDK ResultMessage is identical.

Field _generate _astream
num_turns ✅ preserved ❌ dropped
is_error ✅ preserved ❌ collapsed into binary finish_reason
stop_reason ❌ dropped ❌ dropped
finish_reason (LangChain convention) ❌ missing ✅ emitted

Downstream impact: code that reads ai_message.response_metadata to detect e.g. max_turns truncation or to count SDK-internal turn iterations gets different results depending on which call mode was used. The _astream path loses granularity that ResultMessage originally carries.

Change

Extract a _generation_info_from_result(msg) helper that builds one canonical shape from a ResultMessage, and use it in both _generate and _astream. Net effect:

  • Both paths emit the same set of keys.
  • stop_reason is preserved when the SDK carries it. Access is getattr-guarded so the package keeps working on the >= 0.1.10 SDK floor (the field was added in a later SDK release).
  • LangChain's finish_reason convention is honored uniformly on both paths.
  • num_turns / is_error continue to surface as structured fields, not just as derived finish_reason.

Net diff: −19 / +225 across 2 files (most additions are tests + a thorough helper docstring).

Test plan

  • Adds 4 new tests in tests/test_claude_chat_model.py:
    • test_astream_preserves_full_result_message_fields — streaming path emits the full set
    • test_astream_finish_reason_error_on_is_error_trueis_errorfinish_reason="error" mapping correct
    • test_generate_preserves_full_result_message_fields — non-streaming path emits the matching set
    • test_generation_info_from_result_helper — direct unit test of the helper, covers the omit-when-missing semantics for stop_reason (older SDK) and usage (None)
  • All 15 tests pass on SDK 0.1.10 (the version pinned in uv.lock).

Backwards compatibility

The change is purely additive on the streaming path (new keys), and additive on the non-streaming path (one new key: finish_reason). Existing callers that read only the keys that were already emitted continue to work unchanged.

Context

mimir hit this on a real production deployment after migrating to deepagents (which streams via astream). We worked around it with a local monkey-patch in our _langchain_claude_code_patches module — same pattern your existing docstrings describe for the _arun config-kwarg fix. Once this lands, the mimir-side patch can be retired.

Happy to iterate on style / scope if you'd prefer the change shaped differently.

The non-streaming ``_generate`` and streaming ``_astream`` paths
emit asymmetric ``generation_info`` keys, so the AIMessage produced
by ``invoke`` vs ``ainvoke``/``astream`` carries different metadata
fields even when the underlying SDK ``ResultMessage`` is identical:

  - ``_generate`` preserves ``num_turns`` and ``is_error`` directly
    but emits NO ``finish_reason`` (the LangChain convention).
  - ``_astream`` emits a binary ``finish_reason`` ("stop"/"error")
    derived from ``is_error`` but drops ``num_turns`` and
    ``is_error`` themselves.
  - Neither path preserves ``stop_reason``, even though the SDK's
    ``ResultMessage.stop_reason`` carries granular signals like
    ``"end_turn"`` / ``"max_turns"`` / ``"max_tokens"`` that
    downstream callers want for distinguishing successful end-of-
    turn from truncation-driven termination.

Downstream impact: code reading
``ai_message.response_metadata`` to derive things like "did this
hit max_turns" or "how many internal turns did the SDK run" gets
different results depending on whether the caller used
``ainvoke``/``astream`` or ``invoke``. In mimir we worked around
this with a wrapper monkey-patch
(jasoncarreira/mimir#193), but the right
fix is at the source.

## Change

Extract a ``_generation_info_from_result(msg)`` helper that mirrors
``ResultMessage`` onto a single canonical ``generation_info`` shape
and use it from both ``_generate`` and ``_astream``. Net effect:

  - Both paths emit the SAME set of keys.
  - Granular ``stop_reason`` is preserved when the SDK carries it
    (``getattr``-guarded so older ``>= 0.1.10`` SDKs without the
    field stay supported).
  - LangChain's ``finish_reason`` convention is honored uniformly.
  - ``num_turns`` / ``is_error`` continue to be available as
    structured fields, not just as the binary ``finish_reason``.

## Tests

Adds 4 new tests:
  - ``test_astream_preserves_full_result_message_fields``
  - ``test_astream_finish_reason_error_on_is_error_true``
  - ``test_generate_preserves_full_result_message_fields``
  - ``test_generation_info_from_result_helper``

All 15 tests pass.
jasoncarreira added a commit to jasoncarreira/mimir that referenced this pull request May 22, 2026
Pins the dep to the ``mimir-bundled-fixes`` integration branch on
jasoncarreira/langchain-claude-code, which combines three fixes
currently open as PRs upstream:

* thehumanworks/langchain-claude-code#2 — preserve full
  ``ResultMessage`` field set on ``response_metadata`` in both
  streaming and non-streaming paths
* thehumanworks/langchain-claude-code#4 — capture all tool
  calls/results via SDK hooks instead of inferring from the
  message stream (closes upstream issue #3)
* thehumanworks/langchain-claude-code#6 — skip tools with
  langgraph-injected args in the bind_tools MCP bridge, fixing
  ``TypeError: missing 1 required positional argument: 'runtime'``
  on every ``mcp__langchain-tools__read_file`` / ``write_file`` /
  ``edit_file`` invocation when running on deepagents 0.6+

The third one is the load-bearing fix for the production-observed
duplicate-tools failure on mimirbot (turn 078eb0f98a1b on
2026-05-22): the framework's middleware-injected filesystem tools
were also being bridged through MCP, and every bridged invocation
failed with the missing-runtime error. The agent retried a few
times before falling back to the native framework tool (``Read``),
wasting prompt tokens and turn cycles in the process.

Adds ``[tool.hatch.metadata] allow-direct-references = true`` so
hatchling permits the ``pkg @ git+https://...`` form in
``optional-dependencies``. Reverts to the registry version (and
drops the flag) once upstream PRs land + a new PyPI release ships.

Three places in pyproject.toml carry the pin (claude-code extra,
dev extra, deepagents extra) — all updated to the same SHA.

Co-authored-by: Jason Carreira <jason@visotrust.com>
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
jasoncarreira added a commit to jasoncarreira/mimir that referenced this pull request May 24, 2026
Pins the dep to the ``mimir-bundled-fixes`` integration branch on
jasoncarreira/langchain-claude-code, which combines three fixes
currently open as PRs upstream:

* thehumanworks/langchain-claude-code#2 — preserve full
  ``ResultMessage`` field set on ``response_metadata`` in both
  streaming and non-streaming paths
* thehumanworks/langchain-claude-code#4 — capture all tool
  calls/results via SDK hooks instead of inferring from the
  message stream (closes upstream issue #3)
* thehumanworks/langchain-claude-code#6 — skip tools with
  langgraph-injected args in the bind_tools MCP bridge, fixing
  ``TypeError: missing 1 required positional argument: 'runtime'``
  on every ``mcp__langchain-tools__read_file`` / ``write_file`` /
  ``edit_file`` invocation when running on deepagents 0.6+

The third one is the load-bearing fix for the production-observed
duplicate-tools failure on mimirbot (turn 078eb0f98a1b on
2026-05-22): the framework's middleware-injected filesystem tools
were also being bridged through MCP, and every bridged invocation
failed with the missing-runtime error. The agent retried a few
times before falling back to the native framework tool (``Read``),
wasting prompt tokens and turn cycles in the process.

Adds ``[tool.hatch.metadata] allow-direct-references = true`` so
hatchling permits the ``pkg @ git+https://...`` form in
``optional-dependencies``. Reverts to the registry version (and
drops the flag) once upstream PRs land + a new PyPI release ships.

Three places in pyproject.toml carry the pin (claude-code extra,
dev extra, deepagents extra) — all updated to the same SHA.

Co-authored-by: Jason Carreira <jason@visotrust.com>
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
jasoncarreira added a commit to jasoncarreira/mimir that referenced this pull request May 25, 2026
Pins the dep to the ``mimir-bundled-fixes`` integration branch on
jasoncarreira/langchain-claude-code, which combines three fixes
currently open as PRs upstream:

* thehumanworks/langchain-claude-code#2 — preserve full
  ``ResultMessage`` field set on ``response_metadata`` in both
  streaming and non-streaming paths
* thehumanworks/langchain-claude-code#4 — capture all tool
  calls/results via SDK hooks instead of inferring from the
  message stream (closes upstream issue #3)
* thehumanworks/langchain-claude-code#6 — skip tools with
  langgraph-injected args in the bind_tools MCP bridge, fixing
  ``TypeError: missing 1 required positional argument: 'runtime'``
  on every ``mcp__langchain-tools__read_file`` / ``write_file`` /
  ``edit_file`` invocation when running on deepagents 0.6+

The third one is the load-bearing fix for the production-observed
duplicate-tools failure on mimirbot (turn 078eb0f98a1b on
2026-05-22): the framework's middleware-injected filesystem tools
were also being bridged through MCP, and every bridged invocation
failed with the missing-runtime error. The agent retried a few
times before falling back to the native framework tool (``Read``),
wasting prompt tokens and turn cycles in the process.

Adds ``[tool.hatch.metadata] allow-direct-references = true`` so
hatchling permits the ``pkg @ git+https://...`` form in
``optional-dependencies``. Reverts to the registry version (and
drops the flag) once upstream PRs land + a new PyPI release ships.

Three places in pyproject.toml carry the pin (claude-code extra,
dev extra, deepagents extra) — all updated to the same SHA.

Co-authored-by: Jason Carreira <jason@visotrust.com>
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants