preserve full ResultMessage field set on response_metadata (both paths) - #2
Open
jasoncarreira wants to merge 1 commit into
Conversation
The non-streaming ``_generate`` and streaming ``_astream`` paths
emit asymmetric ``generation_info`` keys, so the AIMessage produced
by ``invoke`` vs ``ainvoke``/``astream`` carries different metadata
fields even when the underlying SDK ``ResultMessage`` is identical:
- ``_generate`` preserves ``num_turns`` and ``is_error`` directly
but emits NO ``finish_reason`` (the LangChain convention).
- ``_astream`` emits a binary ``finish_reason`` ("stop"/"error")
derived from ``is_error`` but drops ``num_turns`` and
``is_error`` themselves.
- Neither path preserves ``stop_reason``, even though the SDK's
``ResultMessage.stop_reason`` carries granular signals like
``"end_turn"`` / ``"max_turns"`` / ``"max_tokens"`` that
downstream callers want for distinguishing successful end-of-
turn from truncation-driven termination.
Downstream impact: code reading
``ai_message.response_metadata`` to derive things like "did this
hit max_turns" or "how many internal turns did the SDK run" gets
different results depending on whether the caller used
``ainvoke``/``astream`` or ``invoke``. In mimir we worked around
this with a wrapper monkey-patch
(jasoncarreira/mimir#193), but the right
fix is at the source.
## Change
Extract a ``_generation_info_from_result(msg)`` helper that mirrors
``ResultMessage`` onto a single canonical ``generation_info`` shape
and use it from both ``_generate`` and ``_astream``. Net effect:
- Both paths emit the SAME set of keys.
- Granular ``stop_reason`` is preserved when the SDK carries it
(``getattr``-guarded so older ``>= 0.1.10`` SDKs without the
field stay supported).
- LangChain's ``finish_reason`` convention is honored uniformly.
- ``num_turns`` / ``is_error`` continue to be available as
structured fields, not just as the binary ``finish_reason``.
## Tests
Adds 4 new tests:
- ``test_astream_preserves_full_result_message_fields``
- ``test_astream_finish_reason_error_on_is_error_true``
- ``test_generate_preserves_full_result_message_fields``
- ``test_generation_info_from_result_helper``
All 15 tests pass.
This was referenced May 18, 2026
jasoncarreira
added a commit
to jasoncarreira/mimir
that referenced
this pull request
May 22, 2026
Pins the dep to the ``mimir-bundled-fixes`` integration branch on jasoncarreira/langchain-claude-code, which combines three fixes currently open as PRs upstream: * thehumanworks/langchain-claude-code#2 — preserve full ``ResultMessage`` field set on ``response_metadata`` in both streaming and non-streaming paths * thehumanworks/langchain-claude-code#4 — capture all tool calls/results via SDK hooks instead of inferring from the message stream (closes upstream issue #3) * thehumanworks/langchain-claude-code#6 — skip tools with langgraph-injected args in the bind_tools MCP bridge, fixing ``TypeError: missing 1 required positional argument: 'runtime'`` on every ``mcp__langchain-tools__read_file`` / ``write_file`` / ``edit_file`` invocation when running on deepagents 0.6+ The third one is the load-bearing fix for the production-observed duplicate-tools failure on mimirbot (turn 078eb0f98a1b on 2026-05-22): the framework's middleware-injected filesystem tools were also being bridged through MCP, and every bridged invocation failed with the missing-runtime error. The agent retried a few times before falling back to the native framework tool (``Read``), wasting prompt tokens and turn cycles in the process. Adds ``[tool.hatch.metadata] allow-direct-references = true`` so hatchling permits the ``pkg @ git+https://...`` form in ``optional-dependencies``. Reverts to the registry version (and drops the flag) once upstream PRs land + a new PyPI release ships. Three places in pyproject.toml carry the pin (claude-code extra, dev extra, deepagents extra) — all updated to the same SHA. Co-authored-by: Jason Carreira <jason@visotrust.com> Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
jasoncarreira
added a commit
to jasoncarreira/mimir
that referenced
this pull request
May 24, 2026
Pins the dep to the ``mimir-bundled-fixes`` integration branch on jasoncarreira/langchain-claude-code, which combines three fixes currently open as PRs upstream: * thehumanworks/langchain-claude-code#2 — preserve full ``ResultMessage`` field set on ``response_metadata`` in both streaming and non-streaming paths * thehumanworks/langchain-claude-code#4 — capture all tool calls/results via SDK hooks instead of inferring from the message stream (closes upstream issue #3) * thehumanworks/langchain-claude-code#6 — skip tools with langgraph-injected args in the bind_tools MCP bridge, fixing ``TypeError: missing 1 required positional argument: 'runtime'`` on every ``mcp__langchain-tools__read_file`` / ``write_file`` / ``edit_file`` invocation when running on deepagents 0.6+ The third one is the load-bearing fix for the production-observed duplicate-tools failure on mimirbot (turn 078eb0f98a1b on 2026-05-22): the framework's middleware-injected filesystem tools were also being bridged through MCP, and every bridged invocation failed with the missing-runtime error. The agent retried a few times before falling back to the native framework tool (``Read``), wasting prompt tokens and turn cycles in the process. Adds ``[tool.hatch.metadata] allow-direct-references = true`` so hatchling permits the ``pkg @ git+https://...`` form in ``optional-dependencies``. Reverts to the registry version (and drops the flag) once upstream PRs land + a new PyPI release ships. Three places in pyproject.toml carry the pin (claude-code extra, dev extra, deepagents extra) — all updated to the same SHA. Co-authored-by: Jason Carreira <jason@visotrust.com> Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
jasoncarreira
added a commit
to jasoncarreira/mimir
that referenced
this pull request
May 25, 2026
Pins the dep to the ``mimir-bundled-fixes`` integration branch on jasoncarreira/langchain-claude-code, which combines three fixes currently open as PRs upstream: * thehumanworks/langchain-claude-code#2 — preserve full ``ResultMessage`` field set on ``response_metadata`` in both streaming and non-streaming paths * thehumanworks/langchain-claude-code#4 — capture all tool calls/results via SDK hooks instead of inferring from the message stream (closes upstream issue #3) * thehumanworks/langchain-claude-code#6 — skip tools with langgraph-injected args in the bind_tools MCP bridge, fixing ``TypeError: missing 1 required positional argument: 'runtime'`` on every ``mcp__langchain-tools__read_file`` / ``write_file`` / ``edit_file`` invocation when running on deepagents 0.6+ The third one is the load-bearing fix for the production-observed duplicate-tools failure on mimirbot (turn 078eb0f98a1b on 2026-05-22): the framework's middleware-injected filesystem tools were also being bridged through MCP, and every bridged invocation failed with the missing-runtime error. The agent retried a few times before falling back to the native framework tool (``Read``), wasting prompt tokens and turn cycles in the process. Adds ``[tool.hatch.metadata] allow-direct-references = true`` so hatchling permits the ``pkg @ git+https://...`` form in ``optional-dependencies``. Reverts to the registry version (and drops the flag) once upstream PRs land + a new PyPI release ships. Three places in pyproject.toml carry the pin (claude-code extra, dev extra, deepagents extra) — all updated to the same SHA. Co-authored-by: Jason Carreira <jason@visotrust.com> Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
_generateand_astreamemit asymmetricgeneration_infoshapes, so callinginvokevsainvoke/astreamagainstClaudeCodeChatModelproduces AIMessages with differentresponse_metadatakeys — even when the underlying SDKResultMessageis identical._generate_astreamnum_turnsis_errorfinish_reasonstop_reasonfinish_reason(LangChain convention)Downstream impact: code that reads
ai_message.response_metadatato detect e.g.max_turnstruncation or to count SDK-internal turn iterations gets different results depending on which call mode was used. The_astreampath loses granularity thatResultMessageoriginally carries.Change
Extract a
_generation_info_from_result(msg)helper that builds one canonical shape from aResultMessage, and use it in both_generateand_astream. Net effect:stop_reasonis preserved when the SDK carries it. Access isgetattr-guarded so the package keeps working on the>= 0.1.10SDK floor (the field was added in a later SDK release).finish_reasonconvention is honored uniformly on both paths.num_turns/is_errorcontinue to surface as structured fields, not just as derivedfinish_reason.Net diff: −19 / +225 across 2 files (most additions are tests + a thorough helper docstring).
Test plan
tests/test_claude_chat_model.py:test_astream_preserves_full_result_message_fields— streaming path emits the full settest_astream_finish_reason_error_on_is_error_true—is_error→finish_reason="error"mapping correcttest_generate_preserves_full_result_message_fields— non-streaming path emits the matching settest_generation_info_from_result_helper— direct unit test of the helper, covers the omit-when-missing semantics forstop_reason(older SDK) andusage(None)uv.lock).Backwards compatibility
The change is purely additive on the streaming path (new keys), and additive on the non-streaming path (one new key:
finish_reason). Existing callers that read only the keys that were already emitted continue to work unchanged.Context
mimir hit this on a real production deployment after migrating to deepagents (which streams via
astream). We worked around it with a local monkey-patch in our_langchain_claude_code_patchesmodule — same pattern your existing docstrings describe for the_arunconfig-kwarg fix. Once this lands, the mimir-side patch can be retired.Happy to iterate on style / scope if you'd prefer the change shaped differently.