Fix: recover lossy TOOL-span args from the LLM tool_call (LangChain/LangGraph) - #54
Merged
Merged
Conversation
…aph)
The OTel adapter reads tool arguments from the TOOL span's input.value.
LangChain/LangGraph records that as a lossy bare scalar (e.g. "A100"
instead of {"order_id": "A100"}), so the args parsed as empty and:
- R6 (malformed_arguments) fired a false hard_defect, and
- R1 (schema) saw no args and reported the required field missing,
both hard_defects -> the run failed CI (exit 2) on a perfectly valid
call. The model's real, valid arguments are in the trace all along, on
the originating LLM span's tool_calls.*.tool_call.function.arguments.
Index the LLM spans' output tool_calls by tool name (in span order) and,
when a TOOL span's own input does not parse to an argument object,
recover the arguments from the matching LLM tool_call and clear raw_text
(the args were never malformed). Instrumentors that already put full
args on the TOOL span (e.g. smolagents) are unaffected — recovery only
runs when the TOOL span's own args are empty. A genuinely malformed
model call yields no structured LLM args either, so R6 still fires.
Verified on a real gpt-4o-mini LangGraph create_react_agent trace:
keyless and with tools.json both go from exit 2 (two false hard_defects)
to 0 findings, exit 0; smolagents traces unchanged.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
The OTel/OpenInference adapter reads a tool call's arguments from the TOOL
span's
input.value. LangChain / LangGraph records that as a lossy barescalar — e.g.
"A100"instead of{"order_id": "A100"}. The args then parse asempty, and on a perfectly valid call:
hard_defect, andboth
hard_defects → the run fails CI (exit 2).The model's real, valid arguments are in the trace the whole time — on the
originating LLM span's
tool_calls.*.tool_call.function.arguments(
{"order_id": "A100"}). The adapter just wasn't using them when a TOOL spanwas present.
Fix
Index each LLM span's output
tool_callsby tool name (in span order). When aTOOL span's own
input.valuedoes not parse to an argument object, recover thearguments from the matching LLM tool_call and clear
raw_text(the argumentswere never malformed — the TOOL span just under-recorded them).
{"args":[],"kwargs":{...}}) are unaffected — recovery only runs when theTOOL span's own args are empty.
either, so R6 still fires in that case (only
output_messages— the newlyemitted calls — are read, so a replayed turn never double-counts).
Why it matters
Second per-instrumentor shape difference found while running tracelint against
real third-party OpenInference traces (after the nested content-parts fix,
#52). Instrumentors place the authoritative tool arguments in different spans
— smolagents on the TOOL span, LangChain on the LLM span — and the shared OTel
adapter must normalize that. Without this, any LangGraph/LangChain user gets a
spurious CI failure.
Verification
Built from a real gpt-4o-mini LangGraph
create_react_agenttrace. Before:R6(keyless) andR1 + R6(with tools.json), both exit 2. After: 0findings, exit 0 in both cases;
R1coverage 1/1. smolagents real tracesunchanged.
Tests
tests/test_adapter_otel.py::test_lossy_tool_span_input_recovers_args_from_llm_tool_call(built from the real LangGraph shape).
ruff check+ full suite green.