Skip to content

fix(llmobs): stop inlining base64 images into langchain span content - #19803

Draft
joizddog wants to merge 2 commits into
mainfrom
jose/mlob-6408-langchain-image-parts
Draft

fix(llmobs): stop inlining base64 images into langchain span content#19803
joizddog wants to merge 2 commits into
mainfrom
jose/mlob-6408-langchain-image-parts

Conversation

@joizddog

@joizddog joizddog commented Aug 20, 2026

Copy link
Copy Markdown
Contributor

fix(llmobs): stop inlining base64 images into langchain span content

What does this PR do?

Stops the langchain integration writing raw base64 image payloads into span content, and captures
them as typed image_parts / audio_parts on the llm path instead.

The defect

langchain builds its own LLMObs message content rather than delegating to the provider integration,
and did so with a bare str() and no branch for list-shaped multimodal content
(_integrations/langchain.py, non-stream chat and both streamed call sites). A
data:image/...;base64,... URL therefore landed in span content verbatim.

This happens on both paths, because input_messages is built before the is_workflow branch;
that branch only chooses which key the result lands under. When demoted, it is JSON-serialized into
input.value.

Past DD_LLMOBS_EVENT_SIZE_BYTES, _truncate_span_event blanks the span's entire input and
output
, not just the image. So a single vision prompt can cost the whole record of the call.

The delegated path does not save you either: three cases produce no provider llm span at all
(proxy requests, raw-response streaming, and a provider registered-but-never-imported), and in those
the langchain span carrying the inlined base64 is the only record.

Precedent

No new design here. List content is routed through the shared _extract_content_parts, the same
helper #19690 extended to return image parts, so the wire shape and the capture semantics are
identical to the merged OpenAI and Anthropic work (#19148, #19690), which in turn build on
#18809.

Changes

ddtrace/llmobs/_integrations/langchain.py (+73/-12):

  • _extract_message_content(content, is_workflow) — list content goes through
    _extract_content_parts; a data URL becomes a structured part and never reaches the text. Blocks
    the shared extractor does not recognise (langchain's own image/audio standard blocks) still
    leave a [image] / [audio] marker rather than the payload.
  • _build_message(content, role, is_workflow) — attaches media keys only when something was
    captured, so a text-only message keeps byte-identical shape to before.
  • Wired into the non-stream chat path and both streamed call sites
    (_handle_stream_input_messages now takes is_workflow).

Audio is fixed by the same change; langchain had never handled it either.

Size guarding (the question asked on #19148 and #19690)

This inherits the per-image budget from _capture_inline_image, which #19690 added to
_extract_content_parts — an over-budget image degrades to [image omitted: too large] before it
can push the event over the cap. There is no new guard here and no guard bypass.

The cumulative case is still open, exactly as on the merged PRs: N images that each fit the
per-image budget can together exceed the per-event limit. That is unchanged by this PR and remains
tracked under MLOB-6408 as a writer-side fix. Flagging it explicitly so it is not re-litigated here.

Why the demoted path gets markers, not typed parts

On a demoted (workflow) span, typed messages have nowhere to land — those spans carry
input_value, not messages — so structured parts would be serialized straight back into the
value, which is the bug this PR fixes. Those keep a marker only; the provider's child llm span
carries the real payload.

Note this interacts with the pending non-LLM span-kind widening: once workflow / task / tool /
step can keep typed messages, this branch becomes eligible to emit real parts there instead of
markers. Deliberately left as a follow-up rather than pre-empting a change that has not landed.

Testing

tests/contrib/langchain/test_langchain_llmobs.py (+216). 8/8 of the new tests pass; lint fmt,
style, typing and spelling clean; banned-terms clean.

The existing test_langchain.py:99-123 already fed an image_url block through ChatOpenAI, but
asserted only "no error" and used an https URL, so it could not catch this. The new cases use a
data: URL and assert the base64 is absent from span content.

Note conftest.py sets integrations_enabled=False, so the suite exercises the non-delegated path;
the demoted path needs a fixture that patches a provider and is called out rather than silently
uncovered.

Known adjacent defects, deliberately not in scope

  • The chain path (format_langchain_io) can still stringify content blocks.
  • Streamed output aggregation is untouched by this PR.

Both are pre-existing, independent of this change, and worth their own issues rather than widening
this diff.

@datadog-datadog-prod-us1

datadog-datadog-prod-us1 Bot commented Aug 20, 2026

Copy link
Copy Markdown
Contributor

Tests

🎉 All green!

🧪 All tests passed
❄️ No new flaky tests detected

This comment will be updated automatically if new data arrives.
🔗 Commit SHA: 340de84 | Docs | View more details | Give us feedback!

@pr-commenter

pr-commenter Bot commented Aug 20, 2026

Copy link
Copy Markdown

Benchmarks

Benchmark execution time: 2026-08-20 23:26:25

Comparing candidate commit 1623095 in PR branch jose/mlob-6408-langchain-image-parts with baseline commit f559280 in branch main.

📊 Benchmarking dashboard

Found 0 performance improvements and 9 performance regressions! Performance is the same for 613 metrics, 10 unstable metrics.

Explanation

This is an A/B test comparing a candidate commit's performance against that of a baseline commit. Performance changes are noted in the tables below as:

  • 🟩 = significantly better candidate vs. baseline
  • 🟥 = significantly worse candidate vs. baseline

We compute a confidence interval (CI) over the relative difference of means between metrics from the candidate and baseline commits, considering the baseline as the reference.

If the CI is entirely outside the configured SIGNIFICANT_IMPACT_THRESHOLD (or the deprecated UNCONFIDENCE_THRESHOLD), the change is considered significant.

Feel free to reach out to #apm-benchmarking-platform on Slack if you have any questions.

More details about the CI and significant changes

You can imagine this CI as a range of values that is likely to contain the true difference of means between the candidate and baseline commits.

CIs of the difference of means are often centered around 0%, because often changes are not that big:

---------------------------------(------|---^--------)-------------------------------->
                              -0.6%    0%  0.3%     +1.2%
                                 |          |        |
         lower bound of the CI --'          |        |
sample mean (center of the CI) -------------'        |
         upper bound of the CI ----------------------'

As described above, a change is considered significant if the CI is entirely outside the configured SIGNIFICANT_IMPACT_THRESHOLD (or the deprecated UNCONFIDENCE_THRESHOLD).

For instance, for an execution time metric, this confidence interval indicates a significantly worse performance:

----------------------------------------|---------|---(---------^---------)---------->
                                       0%        1%  1.3%      2.2%      3.1%
                                                  |   |         |         |
       significant impact threshold --------------'   |         |         |
                      lower bound of CI --------------'         |         |
       sample mean (center of the CI) --------------------------'         |
                      upper bound of CI ----------------------------------'

scenario:httppropagationextract-empty_headers

  • 🟥 execution_time [+127.687ns; +155.250ns] or [+12.262%; +14.909%]

scenario:httppropagationinject-ids_only

  • 🟥 execution_time [+1.651µs; +1.785µs] or [+8.585%; +9.282%]

scenario:iastaspects-casefold_noaspect

  • 🟥 execution_time [+41.129µs; +49.149µs] or [+16.690%; +19.944%]

scenario:iastaspects-swapcase_noaspect

  • 🟥 execution_time [+19.988µs; +27.512µs] or [+9.869%; +13.584%]

scenario:iastaspectsospath-ospathbasename_aspect

  • 🟥 execution_time [+107.511µs; +116.569µs] or [+25.827%; +28.003%]

scenario:iastaspectssplit-rsplit_aspect

  • 🟥 execution_time [+14.958µs; +19.881µs] or [+10.160%; +13.504%]

scenario:span-start

  • 🟥 execution_time [+1.566ms; +1.730ms] or [+10.774%; +11.902%]

scenario:telemetryaddmetric-1-count-metric-1-times

  • 🟥 execution_time [+394.807ns; +421.374ns] or [+14.934%; +15.938%]

scenario:tracer-small

  • 🟥 execution_time [+25.336µs; +28.217µs] or [+7.500%; +8.353%]

Unstable benchmarks

These benchmarks have a confidence interval too wide to call a change; treat them as noise rather than signal.

scenario:coreapiscenario-context_with_data_listeners

  • unstable execution_time [-714.082ns; +764.144ns] or [-6.555%; +7.015%]

scenario:coreapiscenario-core_dispatch_1_listener

  • unstable execution_time [-34.607ns; +32.069ns] or [-5.635%; +5.222%]

scenario:coreapiscenario-core_dispatch_50_listeners

  • unstable execution_time [-1627.676ns; +1663.497ns] or [-9.608%; +9.819%]

scenario:coreapiscenario-core_dispatch_exception_listeners

  • unstable execution_time [-1288.430ns; +1256.829ns] or [-9.646%; +9.410%]

scenario:coreapiscenario-core_dispatch_listeners

  • unstable execution_time [-319.569ns; +329.432ns] or [-8.744%; +9.014%]

scenario:coreapiscenario-core_dispatch_no_args_listeners

  • unstable execution_time [-246.822ns; +254.920ns] or [-8.546%; +8.826%]

scenario:coreapiscenario-core_dispatch_with_results_1_listener

  • unstable execution_time [-74.405ns; +70.791ns] or [-6.585%; +6.266%]

scenario:coreapiscenario-core_dispatch_with_results_50_listeners

  • unstable execution_time [-3976.198ns; +3919.422ns] or [-9.856%; +9.715%]

scenario:coreapiscenario-core_dispatch_with_results_listeners

  • unstable execution_time [-767.425ns; +788.852ns] or [-9.551%; +9.818%]

scenario:packagesupdateimporteddependencies-import_many_stdlib_cached

  • unstable execution_time [-64.660µs; +56.055µs] or [-10.039%; +8.703%]

langchain builds its own LLMObs message content with a bare str() and no branch
for list-shaped multimodal content, so a data:image/...;base64 URL landed
verbatim in span content. This hit both paths: input_messages is built before
the is_workflow branch, and when demoted it is serialized into input.value.
Past DD_LLMOBS_EVENT_SIZE_BYTES, _truncate_span_event blanks the span's whole
input and output, not just the offending field.

Route list content through the shared _extract_content_parts, which parses the
OpenAI-shaped image_url blocks langchain passes. The llm path gets image_parts
and audio_parts; the demoted path keeps a marker instead, since a workflow span
has nowhere to put structured parts and the provider's child span carries the
payload.

Covers both input sites, streaming and not. Bare strings in the content list
are normalized first, since langchain allows them and the extractor does not
read them.
@joizddog
joizddog force-pushed the jose/mlob-6408-langchain-image-parts branch from 1623095 to 807ec2e Compare August 21, 2026 14:15
@cit-pr-commenter-54b7da

Copy link
Copy Markdown

Codeowners resolved as

Resolved from the full PR diff against main using the target branch CODEOWNERS file.
CODEOWNERS team requests not listed below are not required by the current file set.

ddtrace/llmobs/_integrations/langchain.py                               @DataDog/ml-observability
releasenotes/notes/fix-langchain-inline-base64-images-54f331500a30449f.yaml  @DataDog/apm-python
tests/contrib/langchain/test_langchain_llmobs.py                        @DataDog/ml-observability

@cit-pr-commenter-54b7da

cit-pr-commenter-54b7da Bot commented Aug 21, 2026

Copy link
Copy Markdown

Dependency direction analysis

⚠️ Existing dependency direction violations

There are 250 dependency direction violations that already exist on the base branch and have not been changed by this PR.

Show existing violations (showing 5 of 250 highest severity)
ddtrace.internal.tracemethods -×-> ddtrace.trace  (internal-core -> product:tracing, score=135)
ddtrace.profiling.collector.stack -×-> ddtrace.trace  (product:profiling -> product:tracing, score=133)
ddtrace.aiguard._api_client -×-> ddtrace.trace  (product:aiguard -> product:tracing, score=133)
ddtrace.llmobs._integrations.llama_index -×-> ddtrace.trace  (product:llmobs -> product:tracing, score=133)
ddtrace.llmobs._telemetry -×-> ddtrace.trace  (product:llmobs -> product:tracing, score=133)

To see all violations, download the layers-base.json and layers-pr.json artifacts from this CI job and run:

uv run --script scripts/import-analysis/layers.py compare layers-base.json layers-pr.json

@cit-pr-commenter-54b7da

cit-pr-commenter-54b7da Bot commented Aug 21, 2026

Copy link
Copy Markdown

Circular import analysis

⚠️ Existing circular imports

There are 3 circular imports that already exist on the base branch and have not been changed by this PR.

ddtrace.llmobs -> ddtrace.llmobs._evaluators -> ddtrace.llmobs._evaluators.format -> ddtrace.llmobs._experiment -> ddtrace.llmobs
ddtrace.errortracking._handled_exceptions.bytecode_injector -> ddtrace.errortracking._handled_exceptions.callbacks -> ddtrace.errortracking._handled_exceptions.collector -> ddtrace.errortracking._handled_exceptions.bytecode_reporting -> ddtrace.errortracking._handled_exceptions.bytecode_injector
ddtrace.appsec._asm_request_context -> ddtrace.appsec._iast._iast_request_context_base -> ddtrace.appsec._iast._iast_env -> ddtrace.appsec._iast.reporter -> ddtrace.appsec._exploit_prevention.stack_traces -> ddtrace.appsec._asm_request_context

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR updates the LangChain LLMObs integration to prevent inline base64 media (notably data:image/...;base64,...) from being stringified into span message text, instead capturing it as structured image_parts / audio_parts where applicable, and falling back to markers on demoted (“workflow”) spans.

Changes:

  • Add LangChain-specific message-content extraction that routes list-shaped multimodal content through the shared _extract_content_parts helper to avoid base64-in-text.
  • Ensure demoted/workflow spans emit markers rather than structured parts (to avoid re-serializing media back into input_value).
  • Add targeted LangChain LLMObs tests covering inline image/audio capture, oversize degradation, streaming/non-streaming paths, and regression pins; add a release note.

Reviewed changes

Copilot reviewed 3 out of 3 changed files in this pull request and generated 1 comment.

File Description
ddtrace/llmobs/_integrations/langchain.py Parse list-shaped multimodal message content via shared extractor, attach image_parts/audio_parts only when captured, and add workflow-marker behavior.
tests/contrib/langchain/test_langchain_llmobs.py Adds coverage to ensure base64 payloads are not inlined into span content (including streaming and workflow/demoted behavior).
releasenotes/notes/fix-langchain-inline-base64-images-54f331500a30449f.yaml Documents the behavior change and the size-guard/marker semantics for LangChain LLMObs.

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

Comment on lines +6 to +7
render in LLM Observability. Images referenced by a remote URL are not fetched and keep their
existing text reference. A single inline image whose base64 payload exceeds 80% of the
… note

_capture_inline_image returns no marker for a non-data URL, so the shared
extractor falls back to "[image]" and the URL string is not recorded. The note
claimed the reference was preserved. Pin the behavior with a test, since the
note now asserts it.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants