Skip to content

Fix: recover lossy TOOL-span args from the LLM tool_call (LangChain/LangGraph) - #54

Merged
AshwinUgale merged 1 commit into
mainfrom
fix/openinference-langchain-tool-args
Sep 3, 2026
Merged

Fix: recover lossy TOOL-span args from the LLM tool_call (LangChain/LangGraph)#54
AshwinUgale merged 1 commit into
mainfrom
fix/openinference-langchain-tool-args

Conversation

@AshwinUgale

Copy link
Copy Markdown
Owner

What

The OTel/OpenInference adapter reads a tool call's arguments from the TOOL
span's
input.value. LangChain / LangGraph records that as a lossy bare
scalar
— e.g. "A100" instead of {"order_id": "A100"}. The args then parse as
empty, and on a perfectly valid call:

  • R6 (malformed_arguments) fires a false hard_defect, and
  • R1 (schema) sees no args and reports the required field missing,

both hard_defects → the run fails CI (exit 2).

The model's real, valid arguments are in the trace the whole time — on the
originating LLM span's tool_calls.*.tool_call.function.arguments
({"order_id": "A100"}). The adapter just wasn't using them when a TOOL span
was present.

Fix

Index each LLM span's output tool_calls by tool name (in span order). When a
TOOL span's own input.value does not parse to an argument object, recover the
arguments from the matching LLM tool_call and clear raw_text (the arguments
were never malformed — the TOOL span just under-recorded them).

  • Instrumentors that already put full args on the TOOL span (smolagents:
    {"args":[],"kwargs":{...}}) are unaffected — recovery only runs when the
    TOOL span's own args are empty.
  • A genuinely malformed model call yields no structured args on the LLM span
    either, so R6 still fires in that case (only output_messages — the newly
    emitted calls — are read, so a replayed turn never double-counts).

Why it matters

Second per-instrumentor shape difference found while running tracelint against
real third-party OpenInference traces (after the nested content-parts fix,
#52). Instrumentors place the authoritative tool arguments in different spans
— smolagents on the TOOL span, LangChain on the LLM span — and the shared OTel
adapter must normalize that. Without this, any LangGraph/LangChain user gets a
spurious CI failure.

Verification

Built from a real gpt-4o-mini LangGraph create_react_agent trace. Before:
R6 (keyless) and R1 + R6 (with tools.json), both exit 2. After: 0
findings, exit 0
in both cases; R1 coverage 1/1. smolagents real traces
unchanged.

Tests

tests/test_adapter_otel.py::test_lossy_tool_span_input_recovers_args_from_llm_tool_call
(built from the real LangGraph shape). ruff check + full suite green.

…aph)

The OTel adapter reads tool arguments from the TOOL span's input.value.
LangChain/LangGraph records that as a lossy bare scalar (e.g. "A100"
instead of {"order_id": "A100"}), so the args parsed as empty and:

- R6 (malformed_arguments) fired a false hard_defect, and
- R1 (schema) saw no args and reported the required field missing,

both hard_defects -> the run failed CI (exit 2) on a perfectly valid
call. The model's real, valid arguments are in the trace all along, on
the originating LLM span's tool_calls.*.tool_call.function.arguments.

Index the LLM spans' output tool_calls by tool name (in span order) and,
when a TOOL span's own input does not parse to an argument object,
recover the arguments from the matching LLM tool_call and clear raw_text
(the args were never malformed). Instrumentors that already put full
args on the TOOL span (e.g. smolagents) are unaffected — recovery only
runs when the TOOL span's own args are empty. A genuinely malformed
model call yields no structured LLM args either, so R6 still fires.

Verified on a real gpt-4o-mini LangGraph create_react_agent trace:
keyless and with tools.json both go from exit 2 (two false hard_defects)
to 0 findings, exit 0; smolagents traces unchanged.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
@AshwinUgale
AshwinUgale merged commit fc0b7ed into main Sep 3, 2026
3 checks passed
@AshwinUgale
AshwinUgale deleted the fix/openinference-langchain-tool-args branch September 3, 2026 20:37
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant