Fix: seed the user turn from OpenInference nested content-parts messages - #52
Merged
Merged
Conversation
The OTel adapter's _collect_messages read only the flat llm.input_messages.N.message.content string. smolagents (and other OpenInference instrumentors) emit the content-parts shape instead — llm.input_messages.N.message.contents.M.message_content.text — so the user's own request was dropped from the seeded input turns. With no record of what the agent observed, R3 falsely flagged an argument the user actually supplied (order_id) as an underivable/hallucinated value. Collect the indexed text parts and join them into content (a flat message.content still wins when present). Verified against a real gpt-4o-mini smolagents ToolCallingAgent trace: R3's false positive on order_id=A100 disappears and the trace lints clean. Not smolagents- specific — this shape is common across OpenInference instrumentation. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
The OTel/OpenInference adapter's
_collect_messagesread only the flatllm.input_messages.N.message.contentstring. Many OpenInference instrumentors(smolagents among them) instead emit the content-parts shape:
Because that shape wasn't parsed, the user's own request was dropped from the
seeded input turns. With no record of what the agent observed, R3
(hallucinated-arg) falsely flagged an argument the user actually supplied
(e.g.
order_id) as an underivable/hallucinated value.This change collects the indexed text parts and joins them into
content(a flat
message.contentstill wins when present).Why it matters
This is a general OpenInference correctness fix, not a framework-specific
one — the content-parts shape is common across OpenInference instrumentation, so
the false positive affected any such trace.
How it was found & verified
Surfaced while linting a real agent trace via
--format openinference. Verifiedon a real gpt-4o-mini
ToolCallingAgentrun (genuine LLM spans, so theuser-prompt provenance was genuinely present in the trace, ruling out a
scripted-model artifact):
R3 [candidate] order_id=A100 not derivable— false positive.Independently confirmed by collapsing the nested content into the flat form on a
copy of the spans and watching R3 vanish — isolating this parsing as the sole
cause.
Tests
tests/test_adapter_otel.py::test_input_messages_seed_from_nested_content_parts_shape(built from the real trace). Full suite green.