refactor(internal): use fork-safe locks and threads consistently

[P403n1x87]

Description

Several modules still constructed threading.Lock/RLock or threading.Thread directly. Mixing those with the library's own real OS threads can deadlock. Switch remaining call sites to ddtrace.internal.threads' fork-safe Lock/RLock, and replace llmobs' PromptManager background refresh thread with a new Thread helper built on the existing fork-safe PeriodicThread/PERIODIC_STOP machinery.

[datadog-datadog-us1-prod]

🎉 All green!

🧪 All tests passed
❄️ No new flaky tests detected

_{This comment will be updated automatically if new data arrives.
🔗 Commit SHA: 8d74be3 | Docs | View more details | Give us feedback!}

[cit-pr-commenter-54b7da]

Circular import analysis

⚠️ Existing circular imports

There are 3 circular imports that already exist on the base branch and have not been changed by this PR.

ddtrace.llmobs -> ddtrace.llmobs._evaluators -> ddtrace.llmobs._evaluators.format -> ddtrace.llmobs._experiment -> ddtrace.llmobs
ddtrace.errortracking._handled_exceptions.bytecode_injector -> ddtrace.errortracking._handled_exceptions.callbacks -> ddtrace.errortracking._handled_exceptions.collector -> ddtrace.errortracking._handled_exceptions.bytecode_reporting -> ddtrace.errortracking._handled_exceptions.bytecode_injector
ddtrace.appsec._asm_request_context -> ddtrace.appsec._iast._iast_request_context_base -> ddtrace.appsec._iast._iast_env -> ddtrace.appsec._iast.reporter -> ddtrace.appsec._exploit_prevention.stack_traces -> ddtrace.appsec._asm_request_context

[cit-pr-commenter-54b7da]

Dependency direction analysis

⚠️ Existing dependency direction violations

There are 252 dependency direction violations that already exist on the base branch and have not been changed by this PR.

Show existing violations (showing 5 of 252 highest severity)

ddtrace.internal.tracemethods -×-> ddtrace.trace  (internal-core -> product:tracing, score=135)
ddtrace.internal.ci_visibility.git_client -×-> ddtrace.trace  (product:ci_visibility -> product:tracing, score=133)
ddtrace.internal.opentelemetry.span -×-> ddtrace.trace  (product:opentelemetry -> product:tracing, score=133)
ddtrace.llmobs._integrations.openai -×-> ddtrace.trace  (product:llmobs -> product:tracing, score=133)
ddtrace.llmobs._utils -×-> ddtrace.trace  (product:llmobs -> product:tracing, score=133)

To see all violations, download the layers-base.json and layers-pr.json artifacts from this CI job and run:

uv run --script scripts/import-analysis/layers.py compare layers-base.json layers-pr.json

[cit-pr-commenter-54b7da]

Codeowners resolved as

Resolved from the full PR diff against main using the target branch CODEOWNERS file.
CODEOWNERS team requests not listed below are not required by the current file set.

.sg/rules/threading-lock-usage.yml                                      @DataDog/python-guild
.sg/tests/__snapshots__/threading-lock-usage-snapshot.yml               @DataDog/python-guild
.sg/tests/threading-lock-usage-test.yml                                 @DataDog/python-guild
ddtrace/_trace/processor/__init__.py                                    @DataDog/apm-sdk-capabilities-python
ddtrace/_trace/tracer.py                                                @DataDog/apm-sdk-capabilities-python
ddtrace/appsec/sca/_instrumenter.py                                     @DataDog/asm-python
ddtrace/appsec/sca/_registry.py                                         @DataDog/asm-python
ddtrace/internal/_threads.pyi                                           @DataDog/apm-core-python
ddtrace/internal/ci_visibility/encoder.py                               @DataDog/ci-app-libraries
ddtrace/internal/datastreams/schemas/schema_sampler.py                  @DataDog/data-streams-monitoring
ddtrace/internal/excepthook.py                                          @DataDog/apm-core-python
ddtrace/internal/rate_limiter.py                                        @DataDog/apm-sdk-capabilities-python
ddtrace/internal/telemetry/dependency_tracker.py                        @DataDog/apm-python
ddtrace/internal/threads.py                                             @DataDog/apm-core-python
ddtrace/internal/writer/writer.py                                       @DataDog/apm-core-python
ddtrace/llmobs/_prompts/cache.py                                        @DataDog/ml-observability
ddtrace/llmobs/_prompts/manager.py                                      @DataDog/ml-observability
tests/llmobs/test_prompts.py                                            @DataDog/ml-observability

[datadog-datadog-us1-prod]

A fork can restart a completed prompt refresh thread in both the parent and child. This can send duplicate refresh requests and can remove the record for a newer refresh.

_{Open Bits AI session}

_{🤖 Datadog Autotest · Commit a6d8b87 · What is Autotest? · @DataDog review to ask questions · Any feedback? Reach out in #autotest}

[chatgpt-codex-connector]

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: a6d8b878d5

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

[pr-commenter]

Benchmarks

Benchmark execution time: 2026-08-21 13:32:13

Comparing candidate commit 8d74be3 in PR branch chore/safe-threading with baseline commit 1530b7b in branch main.

📊 Benchmarking dashboard

Found 0 performance improvements and 11 performance regressions! Performance is the same for 607 metrics, 10 unstable metrics.

Explanation

This is an A/B test comparing a candidate commit's performance against that of a baseline commit. Performance changes are noted in the tables below as:

• 🟩 = significantly better candidate vs. baseline
• 🟥 = significantly worse candidate vs. baseline

We compute a confidence interval (CI) over the relative difference of means between metrics from the candidate and baseline commits, considering the baseline as the reference.

If the CI is entirely outside the configured SIGNIFICANT_IMPACT_THRESHOLD (or the deprecated UNCONFIDENCE_THRESHOLD), the change is considered significant.

Feel free to reach out to #apm-benchmarking-platform on Slack if you have any questions.

More details about the CI and significant changes

You can imagine this CI as a range of values that is likely to contain the true difference of means between the candidate and baseline commits.

CIs of the difference of means are often centered around 0%, because often changes are not that big:

---------------------------------(------|---^--------)-------------------------------->
                              -0.6%    0%  0.3%     +1.2%
                                 |          |        |
         lower bound of the CI --'          |        |
sample mean (center of the CI) -------------'        |
         upper bound of the CI ----------------------'

As described above, a change is considered significant if the CI is entirely outside the configured SIGNIFICANT_IMPACT_THRESHOLD (or the deprecated UNCONFIDENCE_THRESHOLD).

For instance, for an execution time metric, this confidence interval indicates a significantly worse performance:

----------------------------------------|---------|---(---------^---------)---------->
                                       0%        1%  1.3%      2.2%      3.1%
                                                  |   |         |         |
       significant impact threshold --------------'   |         |         |
                      lower bound of CI --------------'         |         |
       sample mean (center of the CI) --------------------------'         |
                      upper bound of CI ----------------------------------'

scenario:httppropagationextract-empty_headers

• 🟥 execution_time [+189.969ns; +220.012ns] or [+19.529%; +22.617%]

scenario:httppropagationextract-wsgi_medium_valid_headers_all

• 🟥 execution_time [+726.468ns; +812.373ns] or [+7.171%; +8.019%]

scenario:httppropagationinject-ids_only

• 🟥 execution_time [+1.848µs; +1.971µs] or [+9.907%; +10.570%]

scenario:iastaspects-capitalize_noaspect

• 🟥 execution_time [+21.419µs; +27.746µs] or [+11.541%; +14.951%]

scenario:iastaspects-ljust_aspect

• 🟥 execution_time [+112.446µs; +130.001µs] or [+20.820%; +24.070%]

scenario:iastaspects-swapcase_aspect

• 🟥 execution_time [+46.311µs; +55.450µs] or [+16.427%; +19.669%]

scenario:iastaspectsospath-ospathbasename_aspect

• 🟥 execution_time [+107.626µs; +115.378µs] or [+25.961%; +27.831%]

scenario:iastaspectssplit-rsplit_aspect

• 🟥 execution_time [+11.548µs; +18.108µs] or [+7.638%; +11.978%]

scenario:span-start

• 🟥 execution_time [+1.319ms; +1.499ms] or [+8.709%; +9.898%]

scenario:telemetryaddmetric-1-count-metric-1-times

• 🟥 execution_time [+411.770ns; +455.756ns] or [+15.680%; +17.355%]

scenario:tracer-small

• 🟥 execution_time [+25.897µs; +30.053µs] or [+7.670%; +8.901%]

Unstable benchmarks

These benchmarks have a confidence interval too wide to call a change; treat them as noise rather than signal.

scenario:coreapiscenario-context_with_data_listeners

• unstable execution_time [-825.959ns; +658.070ns] or [-7.507%; +5.981%]

scenario:coreapiscenario-core_dispatch_1_listener

• unstable execution_time [-34.749ns; +31.938ns] or [-5.656%; +5.199%]

scenario:coreapiscenario-core_dispatch_50_listeners

• unstable execution_time [-1751.216ns; +1541.266ns] or [-10.266%; +9.035%]

scenario:coreapiscenario-core_dispatch_exception_listeners

• unstable execution_time [-1234.831ns; +1302.773ns] or [-9.310%; +9.822%]

scenario:coreapiscenario-core_dispatch_listeners

• unstable execution_time [-340.666ns; +312.358ns] or [-9.256%; +8.487%]

scenario:coreapiscenario-core_dispatch_no_args_listeners

• unstable execution_time [-277.763ns; +228.968ns] or [-9.516%; +7.844%]

scenario:coreapiscenario-core_dispatch_with_results_1_listener

• unstable execution_time [-76.791ns; +66.241ns] or [-6.737%; +5.811%]

scenario:coreapiscenario-core_dispatch_with_results_50_listeners

• unstable execution_time [-4429.019ns; +3472.216ns] or [-10.888%; +8.536%]

scenario:coreapiscenario-core_dispatch_with_results_listeners

• unstable execution_time [-861.307ns; +689.113ns] or [-10.586%; +8.470%]

scenario:packagesupdateimporteddependencies-import_many_stdlib_cached

• unstable execution_time [-60950.353ns; +59851.185ns] or [-9.481%; +9.310%]

[datadog-datadog-us1-prod]

A one-shot start queued during a fork runs in both processes. Prompt refresh threads also enter an exit join with no time limit, so a stalled HTTP call can block shutdown.

_{Open Bits AI session}

_{🤖 Datadog Autotest · Commit d4f0832 · What is Autotest? · @DataDog review to ask questions · Any feedback? Reach out in #autotest}