Skip to content

fix(llmobs): parse additional headers with an explicit separator to support values containing spaces [do not merge] - #19782

Draft
KowalskiThomas wants to merge 2 commits into
mainfrom
kowalski/psaulue-ddog-paul.laborde/aipaw-1308-parse-tags-str-separator
Draft

fix(llmobs): parse additional headers with an explicit separator to support values containing spaces [do not merge]#19782
KowalskiThomas wants to merge 2 commits into
mainfrom
kowalski/psaulue-ddog-paul.laborde/aipaw-1308-parse-tags-str-separator

Conversation

@KowalskiThomas

@KowalskiThomas KowalskiThomas commented Aug 20, 2026

Copy link
Copy Markdown
Collaborator

This is only opened to support #19506 which is from a fork.

psaulue-ddog and others added 2 commits August 10, 2026 08:11
…upport values containing spaces

_DD_TRACE_WRITER_ADDITIONAL_HEADERS values like "Authorization:Bearer <token>" (space in the
value, no comma anywhere) were incorrectly whitespace-split by parse_tags_str's auto-detection,
corrupting the header. parse_tags_str now accepts an optional explicit separator, and the LLMObs
writer passes "," to avoid the whitespace fallback.

Environment: Datadog workspace

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
…dback

Environment: Datadog workspace

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
@datadog-datadog-us1-prod

datadog-datadog-us1-prod Bot commented Aug 20, 2026

Copy link
Copy Markdown

Pipelines  Tests

Unblock PR with BitsAI

⚠️ Warnings

🚦 11 Pipeline jobs failed

DataDog/apm-reliability/dd-trace-py | build linux: [arm64, cp315-cp315, v113741357-d2b8243-manylinux2014_aarch64] — 🔧 Needs a code fix, caused by this PR

View in Datadog · View in GitLab

DataDog/apm-reliability/dd-trace-py | check-slo-breaches — 🔧 Needs a code fix, caused by this PR

View in Datadog · View in GitLab

DataDog/apm-reliability/dd-trace-py | detect_circular_imports — 🔧 Needs a code fix, caused by this PR

View in Datadog · View in GitLab

View all 11 failed jobs.

ℹ️ Info

No other issues found (see more)

🧪 All tests passed
❄️ No new flaky tests detected

🔄 Datadog auto-retried 1 job - 0 passed on retry View in Datadog

Useful? React with 👍 / 👎

This comment will be updated automatically if new data arrives.
🔗 Commit SHA: 4c9b145 | Docs | View more details | Give us feedback!

@cit-pr-commenter-54b7da

Copy link
Copy Markdown

Circular import analysis

🚨 New circular imports detected 🚨

5 new circular import(s) have been introduced by this PR:

ddtrace.appsec._common_module_patches -> ddtrace.appsec._listeners -> ddtrace.appsec._common_module_patches
ddtrace.contrib.internal.django.patch -> ddtrace.contrib.internal.django.response -> ddtrace.contrib.internal.django.patch
ddtrace.aiguard._api_client -> ddtrace.aiguard._redaction -> ddtrace.aiguard._api_client
ddtrace.internal.openfeature._native -> ddtrace.internal.openfeature._provider -> ddtrace.internal.openfeature._native
ddtrace.contrib.internal.pytorch._distributed -> ddtrace.contrib.internal.pytorch._rank_root -> ddtrace.contrib.internal.pytorch._distributed

Please consider refactoring your changes in accordance to the Separation of Concerns principle.

⚠️ Existing circular imports

There are 3 circular imports that already exist on the base branch and have not been changed by this PR.

ddtrace.llmobs -> ddtrace.llmobs._evaluators -> ddtrace.llmobs._evaluators.format -> ddtrace.llmobs._experiment -> ddtrace.llmobs
ddtrace.errortracking._handled_exceptions.bytecode_injector -> ddtrace.errortracking._handled_exceptions.callbacks -> ddtrace.errortracking._handled_exceptions.collector -> ddtrace.errortracking._handled_exceptions.bytecode_reporting -> ddtrace.errortracking._handled_exceptions.bytecode_injector
ddtrace.appsec._asm_request_context -> ddtrace.appsec._iast._iast_request_context_base -> ddtrace.appsec._iast._iast_env -> ddtrace.appsec._iast.reporter -> ddtrace.appsec._exploit_prevention.stack_traces -> ddtrace.appsec._asm_request_context

@pr-commenter

pr-commenter Bot commented Aug 20, 2026

Copy link
Copy Markdown

Benchmarks

Benchmark execution time: 2026-08-20 08:27:04

Comparing candidate commit 4c9b145 in PR branch kowalski/psaulue-ddog-paul.laborde/aipaw-1308-parse-tags-str-separator with baseline commit c78292f in branch main.

📊 Benchmarking dashboard

Found 0 performance improvements and 4 performance regressions! Performance is the same for 618 metrics, 10 unstable metrics.

Explanation

This is an A/B test comparing a candidate commit's performance against that of a baseline commit. Performance changes are noted in the tables below as:

  • 🟩 = significantly better candidate vs. baseline
  • 🟥 = significantly worse candidate vs. baseline

We compute a confidence interval (CI) over the relative difference of means between metrics from the candidate and baseline commits, considering the baseline as the reference.

If the CI is entirely outside the configured SIGNIFICANT_IMPACT_THRESHOLD (or the deprecated UNCONFIDENCE_THRESHOLD), the change is considered significant.

Feel free to reach out to #apm-benchmarking-platform on Slack if you have any questions.

More details about the CI and significant changes

You can imagine this CI as a range of values that is likely to contain the true difference of means between the candidate and baseline commits.

CIs of the difference of means are often centered around 0%, because often changes are not that big:

---------------------------------(------|---^--------)-------------------------------->
                              -0.6%    0%  0.3%     +1.2%
                                 |          |        |
         lower bound of the CI --'          |        |
sample mean (center of the CI) -------------'        |
         upper bound of the CI ----------------------'

As described above, a change is considered significant if the CI is entirely outside the configured SIGNIFICANT_IMPACT_THRESHOLD (or the deprecated UNCONFIDENCE_THRESHOLD).

For instance, for an execution time metric, this confidence interval indicates a significantly worse performance:

----------------------------------------|---------|---(---------^---------)---------->
                                       0%        1%  1.3%      2.2%      3.1%
                                                  |   |         |         |
       significant impact threshold --------------'   |         |         |
                      lower bound of CI --------------'         |         |
       sample mean (center of the CI) --------------------------'         |
                      upper bound of CI ----------------------------------'

scenario:httppropagationinject-ids_only

  • 🟥 execution_time [+2.102µs; +2.266µs] or [+9.642%; +10.397%]

scenario:iastaspectsospath-ospathbasename_aspect

  • 🟥 execution_time [+83.452µs; +92.252µs] or [+18.476%; +20.424%]

scenario:telemetryaddmetric-1-count-metric-1-times

  • 🟥 execution_time [+503.502ns; +537.554ns] or [+19.097%; +20.389%]

scenario:tracer-small

  • 🟥 execution_time [+27.515µs; +29.513µs] or [+8.201%; +8.796%]

Unstable benchmarks

These benchmarks have a confidence interval too wide to call a change; treat them as noise rather than signal.

scenario:coreapiscenario-context_with_data_listeners

  • unstable execution_time [-815.098ns; +667.066ns] or [-7.377%; +6.038%]

scenario:coreapiscenario-core_dispatch_1_listener

  • unstable execution_time [-28.999ns; +36.356ns] or [-4.779%; +5.991%]

scenario:coreapiscenario-core_dispatch_50_listeners

  • unstable execution_time [-1626.830ns; +1656.632ns] or [-9.645%; +9.821%]

scenario:coreapiscenario-core_dispatch_exception_listeners

  • unstable execution_time [-1254.262ns; +1230.387ns] or [-9.608%; +9.425%]

scenario:coreapiscenario-core_dispatch_listeners

  • unstable execution_time [-324.603ns; +319.976ns] or [-8.936%; +8.809%]

scenario:coreapiscenario-core_dispatch_no_args_listeners

  • unstable execution_time [-246.179ns; +259.339ns] or [-8.503%; +8.958%]

scenario:coreapiscenario-core_dispatch_with_results_1_listener

  • unstable execution_time [-80.578ns; +64.380ns] or [-7.119%; +5.688%]

scenario:coreapiscenario-core_dispatch_with_results_50_listeners

  • unstable execution_time [-3943.010ns; +4155.303ns] or [-9.672%; +10.192%]

scenario:coreapiscenario-core_dispatch_with_results_listeners

  • unstable execution_time [-860.899ns; +700.644ns] or [-10.604%; +8.630%]

scenario:packagesupdateimporteddependencies-import_many_stdlib_cached

  • unstable execution_time [-62.563µs; +56.920µs] or [-9.779%; +8.897%]

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants