feat(iast): enable Code Security on Python 3.15 - #19698
Conversation
Lifts the IAST version gate in ASMConfig from < 3.15 to < 3.16 so DD_IAST_ENABLED takes effect on Python 3.15 instead of being silently ignored. No build change is needed: the IAST native extensions are gated on platform only, never on Python version. Validated on Python 3.15.0rc1+dev: both native extensions build for cp315, the native taint-tracking gtest suite passes 186/186 against real 3.15 headers, and tests/appsec/iast/ is green (19,625 passed). No regression on 3.13 or 3.14. Two bugs surfaced during validation and are fixed here: - The code-injection aspect resolved the wrong caller scope, raising NameError in application code that calls eval(). It assumed a fixed inspect.currentframe().f_back depth, which only holds while wrapt uses its C extension. wrapt has no cp315 wheel, so 3.15 falls back to the pure-Python FunctionWrapper, which adds a frame. This is pre-existing rather than 3.15-specific: it reproduces on 3.14 with WRAPT_DISABLE_EXTENSIONS=1. - CPython 3.15 dropped the "or None" clause from its slice TypeError, so test_slice_aspect_fixtures.py asserted a message that no longer exists. IAST on 3.15 stays unreachable until the tracer wrapping work (#17849) and the requires-python / riot matrix bumps land, so this is correct but inert until then. Closes #17843 APPSEC-69649 Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Circular import analysis
|
Codeowners resolved asResolved from the full PR diff against No remaining files require a CODEOWNERS review. |
Dependency direction analysis
|
BenchmarksBenchmark execution time: 2026-08-24 15:06:38 Comparing candidate commit 7509b17 in PR branch Found 0 performance improvements and 2 performance regressions! Performance is the same for 82 metrics, 0 unstable metrics.
|
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: bbe46c7f1f
ℹ️ About Codex in GitHub
Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".
wrapt's C extension adds no frame of its own, but the pure-Python FunctionWrapper does, so _resolve_caller_frame had no coverage: CI always runs with the C extension, where the walk returns on its first iteration. Add a fixture that evals an expression referencing a caller-local, plus unit tests for the walk and end-to-end tests that recreate the extra wrapt frame via types.FunctionType. Recreating the frame instead of using wrapt.wrappers.FunctionWrapper keeps this correct on wrapt 1.x, where wrappers.py re-imports the C classes over its own names. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
christophe-papazian
left a comment
christophe-papazian
left a comment
There was a problem hiding this comment.
Non-blocking review. I verified the core claims independently and they hold up: setup.py really does gate the IAST extensions on platform only (the three sys.version_info < (3, 15) gates cover profiling/crashtracker/memalloc), asm.py really was the only runtime gate, and the eval() bug reproduces exactly as described — with the wrapt C extension present wrapt.FunctionWrapper is the C class and adds no frame, with it disabled a wrapt.wrappers frame appears and eval("local_value + 1") raises NameError on every wrapt version I tried (1.17.2 → 2.3.0). So the heuristic targets the right frame and is a genuine no-op on the common path.
One finding worth acting on before merge (the _PURE_PYTHON_WRAPT_AVAILABLE line breaks collection of the whole file on most supported wrapt versions), plus four cheap cleanups inline.
- Drop the module-level wrapt __module__ probe in the regression tests. ObjectProxy declares __module__ as a property, so on the class it is a property object, not a str, before wrapt 2.2 — the guard raised AttributeError at import time, a collection error for the whole file on the ~90 lockfiles pinning wrapt==1.17.x. It was also unnecessary: wrapt.wrappers.FunctionWrapper is the pure-Python class on every allowed version, so the test now always runs instead of being skipif-guarded. - Rename _MAX_WRAPPER_FRAMES to _MAX_FRAMES_TO_INSPECT; range(8) inspects 8 candidates, so at most 7 wrapt frames were ever skipped. - Re-nest the func_globals assignment so a None caller frame still raises into the except and falls back to a native wrapped(*args, **kwargs) eval rather than evaluating against ddtrace's own globals with kwargs dropped. - Assert the slice TypeError with one anchored regex instead of two disconnected substrings that accepted arbitrary text between them. - Correct the AIDEV-NOTE: the bound intentionally leads requires-python. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
🎉 All green!🧪 All tests passed 🔗 Commit SHA: 7509b17 | Docs | View more details | Give us feedback! |
The "left unguarded on purpose" comment justified the except fallback as avoiding ddtrace's own globals, which is false: for a plain eval(code), wrapped(*args, **kwargs) and wrapped(code, None, None) both resolve against _iast_coi's module globals. The fallback's actual merit is re-dispatching the caller's original arguments, so an explicitly-passed-but-falsy globals or locals survives the truthiness checks above instead of becoming None. Also give the slice TypeError assertion a message, so a mismatch prints the actual text rather than a bare "assert None". Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
|
/merge |
|
View all feedbacks in Devflow UI.
The expected merge time in
|
2 participants
Description
Part of the Python 3.15 integration parity effort (parent tracker: #17809).
Closes #17843
APPSEC-69649
This PR enables IAST (Code Security) on Python 3.15.
There was no upstream blocker to bump. IAST's native extensions were never version-gated —
setup.pyadds_iast._ast.iastpatchand_iast._taint_tracking._nativeunder a platform-onlyguard — so the only thing disabling Code Security on 3.15 was the runtime tuple in
ASMConfig._iast_supported. This widens it from< (3, 15, 0)to< (3, 16, 0)and corrects theadjacent comment, which already said "3.6 to 3.13" while the code allowed 3.14.
Validating that gate surfaced two real bugs, both fixed here (see Bugs found below). The
eval()one matters: without it, turning IAST on for 3.15 would raiseNameErrorinside customerapplication code.
Changes
ddtrace/internal/settings/asm.py— widen_iast_supportedto 3.15, fix the stale comment, addan
AIDEV-NOTErecording that this is the only version gate and that the native extensions haveno build-time gate.
ddtrace/appsec/_iast/taint_sinks/code_injection.py— add_resolve_caller_frame()so thecode-injection aspect walks past
wrapt's wrapper frame when locating the caller ofeval().tests/appsec/iast/aspects/test_slice_aspect_fixtures.py— accept CPython 3.15's reworded sliceTypeError.featuresfor 3.15 support,fixesfor theeval()bug).Bugs found during validation
1.
eval()raisedNameErrorin application code whenever wrapt's C extension is absent._iast_coiresolved its caller with a fixedinspect.currentframe().f_back. That only lands on theuser's frame while wrapt uses its C
FunctionWrapper, which creates no Python frame. wrapt ships nocp315 wheel, so 3.15 falls back to the pure-Python implementation, which does add a frame — the
aspect then passed wrapt's globals/locals to
eval(), so a lambda default likeeval("lambda v,fun=fun: not fun(v)")could not seefun.This is pre-existing, not 3.15-specific. It reproduces on 3.14:
2. CPython 3.15 reworded its slice error message, dropping the
or Noneclause(
slice indices must be integers or None or have an __index__ method→slice indices must be integers or have an __index__ method). The aspect raises correctly; only thetest's asserted substring was stale.
Checklist
Adapted — several template items don't apply, because the blocker here was a runtime version check
rather than an upstream package pin:
Bumped upstream pin in— N/A, no upstream dependency involved.riotfile.pyLifted— N/A, the cap was amax_versioncap on the affected venv(s)sys.version_infocheck, not a riot cap.Ran— N/A,riot generateand committed lockfilesriotfile.pyis unchanged. TheIAST venvs already use
select_pys(), so they pick up 3.15 automatically once it joinsSUPPORTED_PYTHON_VERSIONS.scripts/run-tests; see the caveat there.Updated— N/A, that file tracks per-integration packagesupported_versions.jsonversions and has no IAST row.
releasenotes/notes/.Testing
Built CPython 3.15.0rc1+dev via
pyenv install 3.15-dev(matching.python-versionand.gitlab/testrunner.yml), stacked #17849 in a scratch worktree to get past theimport ddtraceblocker, and locally bumped
requires-pythonsopip install -e .would run. Neither of thoselocal-only edits is in this PR.
pip install -e .on 3.15_native.cpython-315*.soandiastpatch.cpython-315*.soboth producedappsec_iast_native)tests/appsec/iast/tests/appsec/iast_tdd_propagation/>= (3, 14)skips)appsec_iast_defaulton 3.14 (riot)appsec_iast_defaulton 3.13 (riot)CMake was confirmed to resolve the right interpreter (3.15 headers +
libpython3.15.so) rather thansilently picking the system Python.
Notable positives:
test_template_string_aspect.py(PEP-750 t-strings, 3.14+) collects andpasses on 3.15, exercising the
PY_VERSION_HEX >= 0x030E0000branch ofutils/string_utils.cpp; andtest_native_taint_range.py's refcount assertions hold under 3.15.Why raw
pytestfor the 3.15 leg.AGENTS.mdsays never to invoke pytest directly, but riotcannot target 3.15 at all today:
riotfile.py:SUPPORTED_PYTHON_VERSIONSandscripts/gen_gitlab_config.py:ALL_PYTHON_VERSIONSboth stop at 3.14, and there are zero 3.15lockfiles. The
env=/pkgs=blocks were replicated by hand from theappsec_iast_defaultvenv.Both 3.13/3.14 regression runs went through
scripts/run-testsas normal. Excluded on 3.15:test_grpc_iast.py(grpcio has no cp315 wheel).Risks
import ddtracestill raisesNotImplementedErrorfromddtrace/internal/wrapping/context.py([3.15] Tracer wrapping 3.15 #17810 / chore: wrapping context support for Python 3.15 #17849), andpyproject.tomlstill capsrequires-pythonat<3.15([3.15] AddProgramming Language :: Python :: 3.15classifier topyproject.toml#17815). The gate can only widen, sothere is no risk to existing users on 3.9–3.14.
the manual run above until 3.15 joins the riot matrix ([3.15] cp315 smoke tests (
tests/smoke_test.py,tests/lib-injection) green → flip cp315 wheel job fromallow-failto required #17816).< (3, 16, 0)optimistically covers all of 3.15.eval()fix changes frame resolution on all Python versions, not just 3.15. It is ano-op wherever wrapt's C extension is present (the common case), and 3.13/3.14 regression runs
are green.
Additional Notes
Findings handed to the Python 3.15 migration owners — all outside this PR's scope:
PYO3_USE_ABI3_FORWARD_COMPATIBILITY=1breaks the 3.15 build. It forces limited-API mode,which excludes
PyContextVar_New/Get/Set, sosrc/native/contextvar.rsfails withE0425.pyo3 0.28 supports 3.15 natively, so the flag is counterproductive there.
tests/conftest.pyblocks every test on 3.15. The autouseenable_crashtrackingfixtureasserts
crashtracking.is_started(), butsetup.pydrops the crashtracker Rust feature on 3.15,so
is_availableisFalseandstart()returns early. Needsyield platform.system() == "Linux" and crashtracking.is_available.pip<25cannot be satisfied on 3.15.appsec_iast_defaultpins it to work around IASTfirst-party detection, but pip 24.x fails to import on 3.15 (
typing.no_type_check_decoratorwasremoved). Resolving the
iastpatch.cTODO is a prerequisite for that suite on 3.15.ddtrace/internal/coverage/import_instrumentation_py3_12.pyimportsINJECTION_ASSEMBLY, which no longer exists inddtrace.internal.bytecode_injection. This breaksddtrace's own pytest plugin on all Python versions, not just 3.15.
python-314-f80c8356eaaf8392.yaml,more-314-suites-bf99a7d7ef1b2128.yaml)still list IAST as not working on 3.14, which was already inaccurate before this PR. Left
untouched, since both shipped in v3.16.0rc1.
🤖 Generated with Claude Code