refactor(profiling): drop bytecode-lib dependency from _asyncio.py - #17995
refactor(profiling): drop bytecode-lib dependency from _asyncio.py#17995vlad-scherbich wants to merge 19 commits into
_asyncio.py#17995Conversation
This comment has been minimized.
This comment has been minimized.
BenchmarksBenchmark execution time: 2026-05-10 20:21:06 Comparing candidate commit a982199 in PR branch Found 0 performance improvements and 5 performance regressions! Performance is the same for 592 metrics, 4 unstable metrics. scenario:iast_aspects-re_expand_aspect
scenario:iastaspects-lower_aspect
scenario:iastaspectsospath-ospathbasename_aspect
scenario:span-start
scenario:telemetryaddmetric-1-count-metric-1-times
|
a982199 to
87bd294
Compare
Codeowners resolved as |
|
@hatgpt-codex-connector please review |
There was a problem hiding this comment.
Pull request overview
This PR updates the asyncio profiling integration to avoid ddtrace.internal.wrapping.wrap() (and its third-party bytecode dependency) by introducing a local _wrap() helper that uses functools.wraps + setattr, including explicit alias mirroring for asyncio.* / asyncio.tasks.* bindings.
Changes:
- Replaced bytecode-based wrapping in
ddtrace/profiling/_asyncio.pywith an in-module_wrap()helper and added alias mirroring where needed. - Added a comprehensive new subprocess-based test suite covering alias identity, metadata preservation, callback firing, and argument/kwarg substitution behaviors.
Reviewed changes
Copilot reviewed 2 out of 2 changed files in this pull request and generated 3 comments.
| File | Description |
|---|---|
ddtrace/profiling/_asyncio.py |
Introduces _wrap() and migrates asyncio/uvloop hook points off bytecode-based wrapping, with alias mirroring for selected APIs. |
tests/profiling/collector/test_asyncio_wrapping.py |
Adds subprocess-isolated behavioral tests validating wrapping invariants and callback firing across asyncio APIs. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
02c593b to
9133537
Compare
|
@chatgpt-codex-connector please review |
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 9133537072
ℹ️ About Codex in GitHub
Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".
2e404db to
348bb66
Compare
_asyncio.py
ebb1828 to
7211d79
Compare
| return await wrapper(original_copy, args, kwargs) | ||
|
|
||
|
|
||
| def _wrap( |
There was a problem hiding this comment.
Is there a reason why we need all this extra machinery here, instead of simply defining a wrapper function, e.g.:
original_tasks_wait = asyncio.tasks._wait
def tasks_wait(*args, **kwargs):
...
return original_tasks_wait(*args, **kwargs)
asyncio.tasks._wait = tasks_waitThere was a problem hiding this comment.
I've actually tried something like that at first. it breaks for callers that captured a reference via from asyncio.tasks import _wait before the profiler started. Those references still point at the un-wrapped original.
For example: uvloop's Cython modules do exactly that, which caused all 6 uvloop CI jobs to fail. This PR does basically what the bytecode.wrap machinery does under the hood, which is mutating __code__ in place so identity is preserved.
This regression test: test_pre_cached_reference_still_triggers_callback should fail with the simple approach.
There was a problem hiding this comment.
Ah I see. Have we considered using the wrapping context instead then? That's what the tracer would need to migrate to anyway to get rid of bytecode wrapping
There was a problem hiding this comment.
Yes, we did in a very general way, so I would need to better understand how that would work.
What are the pros / cons of "plain" wrap vs wrapping context, why is context preferable?
There was a problem hiding this comment.
Would the tracing's WrappingContext be used here as you suggest? If so, wouldn't it still depend on bytecode? Or is that the future WrappingContext that's rewritten in your sys.monitoring API? (forgive my n00b questions).
If it's actually the latter, where bytecode dependency is removed, then I would be all for adopting it.
What do you think of merging this change for profiling only, which would fully unblock testing a 3.15 demo service and proving the migration will work? And then replace this implementation with your, better, approach?
There was a problem hiding this comment.
Or is that the future WrappingContext that's rewritten in your sys.monitoring API?
Yes. This is the wrapping machinery that is getting migrated to the monitoring API. So if we really need a more "advanced" wrapping mechanism than just a plain Python wrapper, we should consider using this if possible.
…-cached references (uvloop)
…tity-preservation tests
…ty-preserving wrap
…s_available in helper
…plementation-agnostic
06190ee to
7c9f453
Compare
|
Blocked on #17849, which will replace the need for custom wrapper created here. |
|
Closing in favor of #18389 |
Description
ddtrace/profiling/_asyncio.pypreviously hooked into asyncio internals viaddtrace.internal.wrapping.wrap, which rewrites the target's__code__using the third-partybytecodelibrary.This PR replaces that with a small in-module
_wrap()helper that clones a template trampoline's__code__viaCodeType.replace()and grafts it onto the original function. It is the same identity-preserving trick the previousbytecode.wrap()used, but without the dependency!Why
bytecodedependency during migration to v3.15 (@gab is working on the tracing-side implementation), because it blocks the entire minor version release until it become compatibleScope
This PR affects profiling consumers only.
ddtrace.internal.wrapping.wrapitself is unchanged — the tracing path still genuinely needs bytecode for async/generator function wrapping with stack-effect preservation. The other (legitimate)bytecodeconsumers (ddtrace/debugging/_expressions.py,ddtrace/internal/assembly.py,ddtrace/internal/bytecode_injection/__init__.py) are untouched.Testing
tests/profiling/collector/test_asyncio_wrapping.py(16 tests).Performance
TL;DR:
Microbenchmarked on Python 3.14.3 / macOS arm64.
Three-way:
N = 12 trials per variant; each trial:
create_task: 200,000 iterations per timinggather3/shield: 2,000 iterations per timingRisks
bytecode-dependency implementation.