fix(profiling): discover Python 3.14 asyncio offsets by section

[taegyunkim]

Description

Python 3.14 exposes the private asyncio task-list layouts through the runtime .AsyncioDebug table, but Echion currently compiles the two list-head offsets into _stack.so with offsetof(). A normal cp314 wheel can then use the wrong offsets on a different Python 3.14 patch release. For example, the ddtrace 4.13.1 wheel was built with CPython 3.14.4 offsets (0x370 for the thread head and 0x2a68 for the interpreter head), while CPython 3.14.2 uses 0x368 and 0x2a60. The mismatch can drop task attribution or traverse malformed memory and severely inflate task profiles.

CPython emits .AsyncioDebug as runtime metadata for remote unwinders. The fixed 104-byte table describes the relevant task, interpreter, and thread layouts for the running interpreter, so reading it at initialization avoids assuming that private CPython structures remain stable across patch releases. This change validates the complete 13-field table and publishes the two list-head offsets Echion needs to the lock-free sampler.

An alternative was to get the _asyncio module state and read its final debug_offsets pointer. That works with the CPython 3.14 versions inspected because debug_offsets is currently the last member of the private asyncio_state structure. However, its position and the structure's field ordering are not an ABI contract. Assuming it remains last would replace one patch-sensitive private-layout dependency with another. Discovering the named section follows CPython's remote-unwinding mechanism and does not depend on the _asyncio module-state layout.

Linux discovery enumerates loaded images, maps each candidate ELF file, finds .AsyncioDebug, and verifies that the section belongs to a loaded PT_LOAD segment. macOS discovery scans loaded Mach-O commands and locates AsyncioDebug in __DATA. Invalid candidates are skipped, and unavailable or invalid metadata fails closed. Discovery is cached after the first initialization attempt because asyncio is already loaded and its layout cannot change during the process lifetime.

Scope is intentionally limited to:

• _PyThreadStateImpl.asyncio_tasks_head
• PyInterpreterState.asyncio_tasks_head

TaskObj.task_node remains unchanged. Pure-Python tasks remain covered by _scheduled_tasks. The separate linked-list consistency checks in #19798 are not included here.

The deterministic cross-patch reproducer and explicit-offset audit are in DataDog/experimental#13678.

Testing

CI passing.

Risks

• AsyncioDebug has no cookie. Discovery requires the exact section name and expected 104-byte schema, validates every field against its reported containing-structure size and alignment, and skips invalid candidate images.
• Linux requires the loaded image path and ELF section headers to remain available. Stripped or deleted images without the section metadata fail closed and omit native linked-list task attribution.
• Sampling remains lock-free. Discovery, file access, and validation happen once during Python initialization; the sampler performs only relaxed atomic integer loads.
• No imports or Python C API calls were added to the sampler thread.
• Fork safety: no locks or condition variables were added. The immutable offsets remain valid after fork.

[cit-pr-commenter-54b7da]

Codeowners resolved as

Resolved from the full PR diff against main using the target branch CODEOWNERS file.
CODEOWNERS team requests not listed below are not required by the current file set.

ddtrace/internal/datadog/profiling/stack/CMakeLists.txt                 @DataDog/profiling-python
ddtrace/internal/datadog/profiling/stack/echion/echion/cpython/asyncio_debug.h  @DataDog/profiling-python
ddtrace/internal/datadog/profiling/stack/echion/echion/echion_sampler.h  @DataDog/profiling-python
ddtrace/internal/datadog/profiling/stack/src/echion/asyncio_debug.cc    @DataDog/profiling-python
ddtrace/internal/datadog/profiling/stack/src/echion/threads.cc          @DataDog/profiling-python
ddtrace/internal/datadog/profiling/stack/src/stack.cpp                  @DataDog/profiling-python
ddtrace/internal/datadog/profiling/stack/test/test_sampling_cycle_state.cpp  @DataDog/profiling-python
releasenotes/notes/fix-py314-asyncio-task-offsets-b08edd4c76321a26.yaml  @DataDog/apm-python

[cit-pr-commenter-54b7da]

Circular import analysis

⚠️ Existing circular imports

There are 3 circular imports that already exist on the base branch and have not been changed by this PR.

ddtrace.errortracking._handled_exceptions.bytecode_injector -> ddtrace.errortracking._handled_exceptions.callbacks -> ddtrace.errortracking._handled_exceptions.collector -> ddtrace.errortracking._handled_exceptions.bytecode_reporting -> ddtrace.errortracking._handled_exceptions.bytecode_injector
ddtrace.llmobs -> ddtrace.llmobs._evaluators -> ddtrace.llmobs._evaluators.format -> ddtrace.llmobs._experiment -> ddtrace.llmobs
ddtrace.appsec._asm_request_context -> ddtrace.appsec._iast._iast_request_context_base -> ddtrace.appsec._iast._iast_env -> ddtrace.appsec._iast.reporter -> ddtrace.appsec._exploit_prevention.stack_traces -> ddtrace.appsec._asm_request_context

[cit-pr-commenter-54b7da]

Dependency direction analysis

⚠️ Existing dependency direction violations

There are 250 dependency direction violations that already exist on the base branch and have not been changed by this PR.

Show existing violations (showing 5 of 250 highest severity)

ddtrace.internal.tracemethods -×-> ddtrace.trace  (internal-core -> product:tracing, score=135)
ddtrace.llmobs._integrations.llama_index -×-> ddtrace.trace  (product:llmobs -> product:tracing, score=133)
ddtrace.aiguard._api_client -×-> ddtrace.trace  (product:aiguard -> product:tracing, score=133)
ddtrace.profiling.collector.stack -×-> ddtrace.trace  (product:profiling -> product:tracing, score=133)
ddtrace.llmobs._integrations.google_adk -×-> ddtrace.trace  (product:llmobs -> product:tracing, score=133)

To see all violations, download the layers-base.json and layers-pr.json artifacts from this CI job and run:

uv run --script scripts/import-analysis/layers.py compare layers-base.json layers-pr.json

[datadog-datadog-prod-us1]

🎉 All green!

🧪 All tests passed
❄️ No new flaky tests detected

_{This comment will be updated automatically if new data arrives.
🔗 Commit SHA: 80c19b7 | Docs | View more details | Give us feedback!}

[taegyunkim]

@codex review

[chatgpt-codex-connector]

Codex Review: Didn't find any major issues. More of your lovely PRs please.

Reviewed commit: 28dd8f2b80

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

[Copilot]

Pull request overview

Replaces Python 3.14 compile-time asyncio list offsets with validated runtime discovery, preventing missing or inflated task profiles across patch releases.

Changes:

• Discovers and validates .AsyncioDebug metadata on Linux and macOS.
• Publishes offsets atomically and fails closed when unavailable.
• Adds validation tests, build integration, and a release note.

Reviewed changes

Copilot reviewed 8 out of 8 changed files in this pull request and generated no comments.

Show a summary per file

File	Description
releasenotes/notes/fix-py314-asyncio-task-offsets-b08edd4c76321a26.yaml	Documents the customer-visible fix.
ddtrace/internal/datadog/profiling/stack/test/test_sampling_cycle_state.cpp	Tests offset validation and storage.
ddtrace/internal/datadog/profiling/stack/src/stack.cpp	Initializes runtime offsets.
ddtrace/internal/datadog/profiling/stack/src/echion/threads.cc	Uses discovered offsets with safety checks.
ddtrace/internal/datadog/profiling/stack/src/echion/asyncio_debug.cc	Implements ELF and Mach-O discovery.
ddtrace/internal/datadog/profiling/stack/echion/echion/echion_sampler.h	Stores offsets atomically.
ddtrace/internal/datadog/profiling/stack/echion/echion/cpython/asyncio_debug.h	Defines and validates the runtime table.
ddtrace/internal/datadog/profiling/stack/CMakeLists.txt	Builds and links the discovery implementation.

---

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.