Avoid final field modifications in Karate instrumentation by daniel-mohedano · Pull Request #11753 · DataDog/dd-trace-java

daniel-mohedano · 2026-06-26T12:32:46Z

What Does This Do

Avoids the final field modifications in Karate instrumentation:
- Instead of overwriting the field after the retries, the original ScenarioResult is updated through Karate's public addStepResult API, and SuppressErrorAdvice now suppresses every failing step of a retried attempt (not just the first).

Motivation

Avoid failures in future JDKs due to JEP 500: Prepare to Make Final Mean Final.

Additional Notes

Running an example project on JDK26 with JAVA_TOOL_OPTIONS=--illegal-final-field-mutation=debug produced the following logs:

Final field result in class com.intuit.karate.core.ScenarioRuntime has been unreflected for mutation ...

The build containing the fix doesn't produce the logs.

Contributor Checklist

Format the title according to the contribution guidelines
Assign the type: and (comp: or inst:) labels in addition to any other useful labels
Avoid using close, fix, or any linking keywords when referencing an issue
Use solves instead, and assign the PR milestone to the issue
Update the CODEOWNERS file on source file addition, migration, or deletion
Update public documentation with any new configuration flags or behaviors
Add your completed PR to the merge queue by commenting /merge. You can also:
- Customize the commit message associated with the merge with /merge --commit-message "..."
- Remove your PR from the merge queue with /merge -c
- Skip all merge queue checks with /merge -f --reason "reason"; please use this judiciously, as some checks do not run at the PR-level (note: the PR still needs to be mergeable, this will only skip the pre-merge build)
- Get more information in this doc

Jira ticket: SDTEST-3867

cit-pr-commenter-54b7da · 2026-06-26T12:45:27Z

Test Environment - sbt-scalatest

Job Status: 🟢 success

Scenario	This PR (%)	7d median	Δ 7d	30d median	Δ 30d	runs (7d/30d)
agent	56.06	55.43	$\color{red}{\blacktriangle}$ +0.63	55.43	$\color{red}{\blacktriangle}$ +0.63	54/200
agentEvpProxy	55.46	n/a	n/a	n/a	n/a	-

Baseline: median of @test.tracer_overhead on main (gitlab) over the last 7/30 days, per OSS project & scenario. Δ = this PR − baseline median; red ▲ = more overhead, green ▽ = less overhead than baseline.

cit-pr-commenter-54b7da · 2026-06-26T12:46:33Z

Test Environment - nebula-release-plugin

Job Status: 🟢 success

Scenario	This PR (%)	7d median	Δ 7d	30d median	Δ 30d	runs (7d/30d)
agent	35.47	36.42	$\color{green}{\blacktriangledown}$ -0.95	36.42	$\color{green}{\blacktriangledown}$ -0.95	29/99
agentless	35.84	35.70	$\color{red}{\blacktriangle}$ +0.14	35.70	$\color{red}{\blacktriangle}$ +0.14	29/99
agentlessCodeCoverage	43.69	43.60	$\color{red}{\blacktriangle}$ +0.09	43.60	$\color{red}{\blacktriangle}$ +0.09	29/99
agentlessLineCoverage	74.51	74.82	$\color{green}{\blacktriangledown}$ -0.31	74.82	$\color{green}{\blacktriangledown}$ -0.31	28/98

Baseline: median of @test.tracer_overhead on main (gitlab) over the last 7/30 days, per OSS project & scenario. Δ = this PR − baseline median; red ▲ = more overhead, green ▽ = less overhead than baseline.

cit-pr-commenter-54b7da · 2026-06-26T12:48:24Z

Test Environment - netflix-zuul

Job Status: 🟢 success

Scenario	This PR (%)	7d median	Δ 7d	30d median	Δ 30d	runs (7d/30d)
agent	88.52	86.07	$\color{red}{\blacktriangle}$ +2.45	86.07	$\color{red}{\blacktriangle}$ +2.45	30/146
agentless	81.45	81.05	$\color{red}{\blacktriangle}$ +0.40	81.05	$\color{red}{\blacktriangle}$ +0.40	31/146
agentlessCodeCoverage	96.98	95.12	$\color{red}{\blacktriangle}$ +1.86	95.12	$\color{red}{\blacktriangle}$ +1.86	30/143
agentlessLineCoverage	113.31	111.62	$\color{red}{\blacktriangle}$ +1.69	111.62	$\color{red}{\blacktriangle}$ +1.69	29/142

Baseline: median of @test.tracer_overhead on main (gitlab) over the last 7/30 days, per OSS project & scenario. Δ = this PR − baseline median; red ▲ = more overhead, green ▽ = less overhead than baseline.

dd-octo-sts · 2026-06-26T12:53:50Z

🟢 Java Benchmark SLOs — All performance SLOs passed

Suite	Status
Startup	🟢 pass

SLO thresholds are defined here based on automatically generated metrics. A warning is raised when results are within 5% of the threshold.

PR vs. master results

Scenario	Candidate	master	Δ (95% CI of mean)
startup:insecure-bank:iast:Agent	14.01 s	13.98 s	[-0.5%; +0.9%] (no difference)
startup:insecure-bank:tracing:Agent	12.91 s	13.05 s	[-1.9%; -0.2%] (maybe better)
startup:petclinic:appsec:Agent	16.83 s	16.72 s	[-0.2%; +1.5%] (no difference)
startup:petclinic:iast:Agent	16.28 s	16.84 s	[-7.6%; +0.9%] (no difference)
startup:petclinic:profiling:Agent	16.57 s	16.67 s	[-1.6%; +0.4%] (no difference)
startup:petclinic:sca:Agent	16.84 s	16.73 s	[-0.3%; +1.6%] (no difference)
startup:petclinic:tracing:Agent	15.95 s	16.15 s	[-2.2%; -0.3%] (maybe better)

Commit: eda96c22 · CI Pipeline · Benchmarking Platform UI

Load and DaCapo benchmarks can be triggered manually in the GitLab pipeline. Results will appear in the Benchmarking Platform UI after completion.

cit-pr-commenter-54b7da · 2026-06-26T12:56:20Z

Test Environment - jolokia

Job Status: 🟢 success

Scenario	This PR (%)	7d median	Δ 7d	30d median	Δ 30d	runs (7d/30d)
agent	95.75	95.12	$\color{red}{\blacktriangle}$ +0.63	93.23	$\color{red}{\blacktriangle}$ +2.52	28/102
agentless	91.18	89.58	$\color{red}{\blacktriangle}$ +1.60	89.58	$\color{red}{\blacktriangle}$ +1.60	28/102
agentlessCodeCoverage	101.11	99.00	$\color{red}{\blacktriangle}$ +2.11	99.00	$\color{red}{\blacktriangle}$ +2.11	27/99
agentlessLineCoverage	101.27	101.00	$\color{red}{\blacktriangle}$ +0.27	99.00	$\color{red}{\blacktriangle}$ +2.27	27/99

Baseline: median of @test.tracer_overhead on main (gitlab) over the last 7/30 days, per OSS project & scenario. Δ = this PR − baseline median; red ▲ = more overhead, green ▽ = less overhead than baseline.

cit-pr-commenter-54b7da · 2026-06-26T12:57:34Z

Test Environment - sonar-kotlin

Job Status: 🟢 success

Scenario	This PR (%)	7d median	Δ 7d	30d median	Δ 30d	runs (7d/30d)
agent	11.98	13.13	$\color{green}{\blacktriangledown}$ -1.15	13.13	$\color{green}{\blacktriangledown}$ -1.15	29/101
agentless	12.02	12.12	$\color{green}{\blacktriangledown}$ -0.10	11.88	$\color{red}{\blacktriangle}$ +0.14	29/100
agentlessCodeCoverage	16.65	15.41	$\color{red}{\blacktriangle}$ +1.24	15.11	$\color{red}{\blacktriangle}$ +1.54	29/100
agentlessLineCoverage	17.74	19.20	$\color{green}{\blacktriangledown}$ -1.46	19.20	$\color{green}{\blacktriangledown}$ -1.46	28/100

Baseline: median of @test.tracer_overhead on main (gitlab) over the last 7/30 days, per OSS project & scenario. Δ = this PR − baseline median; red ▲ = more overhead, green ▽ = less overhead than baseline.

cit-pr-commenter-54b7da · 2026-06-26T13:05:16Z

Test Environment - okhttp

Job Status: 🟢 success

Scenario	This PR (%)	7d median	Δ 7d	30d median	Δ 30d	runs (7d/30d)
agent	19.27	19.20	$\color{red}{\blacktriangle}$ +0.07	19.59	$\color{green}{\blacktriangledown}$ -0.32	31/107
agentless	19.18	19.20	$\color{green}{\blacktriangledown}$ -0.02	19.20	$\color{green}{\blacktriangledown}$ -0.02	31/107
agentlessCodeCoverage	22.06	22.09	$\color{green}{\blacktriangledown}$ -0.03	22.09	$\color{green}{\blacktriangledown}$ -0.03	30/104
agentlessLineCoverage	44.71	44.48	$\color{red}{\blacktriangle}$ +0.23	44.48	$\color{red}{\blacktriangle}$ +0.23	30/109

Baseline: median of @test.tracer_overhead on main (gitlab) over the last 7/30 days, per OSS project & scenario. Δ = this PR − baseline median; red ▲ = more overhead, green ▽ = less overhead than baseline.

cit-pr-commenter-54b7da · 2026-06-26T13:10:09Z

Test Environment - spring_boot

Job Status: 🟢 success

Scenario	This PR (%)	7d median	Δ 7d	30d median	Δ 30d	runs (7d/30d)
agent	16.32	16.04	$\color{red}{\blacktriangle}$ +0.28	16.04	$\color{red}{\blacktriangle}$ +0.28	26/96
agentless	9.28	9.73	$\color{green}{\blacktriangledown}$ -0.45	9.73	$\color{green}{\blacktriangledown}$ -0.45	26/96
agentlessCodeCoverage	13.28	13.40	$\color{green}{\blacktriangledown}$ -0.12	13.40	$\color{green}{\blacktriangledown}$ -0.12	26/95
agentlessLineCoverage	33.51	32.95	$\color{red}{\blacktriangle}$ +0.56	32.30	$\color{red}{\blacktriangle}$ +1.21	26/96

Baseline: median of @test.tracer_overhead on main (gitlab) over the last 7/30 days, per OSS project & scenario. Δ = this PR − baseline median; red ▲ = more overhead, green ▽ = less overhead than baseline.

cit-pr-commenter-54b7da · 2026-06-26T13:31:52Z

Test Environment - sonar-java

Job Status: 🟢 success

Scenario	This PR (%)	7d median	Δ 7d	30d median	Δ 30d	runs (7d/30d)
agent	41.56	17.38	$\color{red}{\blacktriangle}$ +24.18	15.72	$\color{red}{\blacktriangle}$ +25.84	29/103
agentless	4.57	24.41	$\color{green}{\blacktriangledown}$ -19.84	17.73	$\color{green}{\blacktriangledown}$ -13.16	29/102
agentlessCodeCoverage	99.86	89.58	$\color{red}{\blacktriangle}$ +10.28	89.58	$\color{red}{\blacktriangle}$ +10.28	29/102
agentlessLineCoverage	152.21	144.77	$\color{red}{\blacktriangle}$ +7.44	141.90	$\color{red}{\blacktriangle}$ +10.31	28/101

Baseline: median of @test.tracer_overhead on main (gitlab) over the last 7/30 days, per OSS project & scenario. Δ = this PR − baseline median; red ▲ = more overhead, green ▽ = less overhead than baseline.

cit-pr-commenter-54b7da · 2026-06-29T12:28:59Z

Test Environment - pass4s

Job Status: 🟢 success

Scenario	This PR (%)	7d median	Δ 7d	30d median	Δ 30d	runs (7d/30d)
agent	10.77	10.75	$\color{red}{\blacktriangle}$ +0.02	10.33	$\color{red}{\blacktriangle}$ +0.44	27/99
agentless	11.63	10.75	$\color{red}{\blacktriangle}$ +0.88	9.92	$\color{red}{\blacktriangle}$ +1.71	27/100
agentlessCodeCoverage	16.99	18.08	$\color{green}{\blacktriangledown}$ -1.09	17.73	$\color{green}{\blacktriangledown}$ -0.74	26/97

Baseline: median of @test.tracer_overhead on main (gitlab) over the last 7/30 days, per OSS project & scenario. Δ = this PR − baseline median; red ▲ = more overhead, green ▽ = less overhead than baseline.

cit-pr-commenter-54b7da · 2026-06-29T12:29:54Z

Test Environment - reactive-streams-jvm

Job Status: 🟢 success

Scenario	This PR (%)	7d median	Δ 7d	30d median	Δ 30d	runs (7d/30d)
agent	21.66	21.65	$\color{red}{\blacktriangle}$ +0.01	21.65	$\color{red}{\blacktriangle}$ +0.01	29/144
agentless	19.92	18.45	$\color{red}{\blacktriangle}$ +1.47	18.82	$\color{red}{\blacktriangle}$ +1.10	30/143
agentlessCodeCoverage	20.53	20.39	$\color{red}{\blacktriangle}$ +0.14	19.99	$\color{red}{\blacktriangle}$ +0.54	29/142
agentlessLineCoverage	30.81	29.82	$\color{red}{\blacktriangle}$ +0.99	29.82	$\color{red}{\blacktriangle}$ +0.99	28/141

Baseline: median of @test.tracer_overhead on main (gitlab) over the last 7/30 days, per OSS project & scenario. Δ = this PR − baseline median; red ▲ = more overhead, green ▽ = less overhead than baseline.

chatgpt-codex-connector

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 3eaec0c7b3

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

chatgpt-codex-connector · 2026-06-29T15:05:30Z

+    "test-failed"                | false   | [TestFailedKarate]                 | []
+    "test-retry-failed"          | false   | [TestFailedKarate]                 | [new TestFQN("[org/example/test_failed] test failed", "second scenario")]
+    "test-failed-then-succeed"   | true    | [TestFailedThenSucceedKarate]      | [new TestFQN("[org/example/test_failed_then_succeed] test failed", "flaky scenario")]
+    "test-retry-continue-on-step-failure" | true | [TestContinueOnStepFailureKarate] | [


Migrate the new retry coverage to JUnit 5

The repo instructions in AGENTS.md say, "Do not write new Groovy / Spock tests and migrate the existing one to JUnit 5 if it is written in Groovy." This new data-table case adds coverage to the existing Spock spec instead, so the change leaves the new Karate retry scenario in the framework the repo is trying to retire; please move it to a JUnit 5 test or migrate the spec.

Useful? React with 👍 / 👎.

datadog-datadog-prod-us1-2

More details

The core behavioral change — replacing getAndResetSuppressFailures() (which reset after the first call, leaving subsequent failing steps in a continueOnStepFailure attempt unsuppressed) with shouldSuppressFailures() (no reset) — is correct and necessary. Karate 1.0.0 source confirms getFailedStep() is non-null when isFailed() is true, addStepResult() correctly propagates failure state, and the SuppressErrorAdvice is not re-triggered problematically because suppressFailures is set to false by the last retry's beforeExecute for all non-quarantined scenarios.

Was this helpful? React 👍 or 👎

_{📊 Validated against 7 scenarios · Open Bits AI session}

_{🤖 Datadog Autotest · Commit 3eaec0c · What is Autotest? · Any feedback? Reach out in #autotest}

datadog-datadog-prod-us1-2 · 2026-07-01T11:40:35Z

🎯 Code Coverage (details)
• Patch Coverage: 100.00%
• Overall Coverage: 56.94% (+0.00%)

_{This comment will be updated automatically if new data arrives.

🔗 Commit SHA: eda96c2 | Docs | Datadog PR Page | Give us feedback!}

daniel-mohedano · 2026-07-03T08:32:54Z

/merge

gh-worker-devflow-routing-ef8351 · 2026-07-03T08:32:58Z

View all feedbacks in Devflow UI.

2026-07-03 08:32:58 UTC ℹ️ Start processing command /merge

2026-07-03 08:33:03 UTC ℹ️ MergeQueue: pull request added to the queue

The expected merge time in master is approximately 2h (p90).

2026-07-03 10:33:37 UTC ❌ MergeQueue: The build pipeline has timeout

The merge request has been interrupted because the build 2431498075535147252 took longer than expected. The current limit for the base branch 'master' is 120 minutes.

daniel-mohedano · 2026-07-03T10:34:47Z

/merge

gh-worker-devflow-routing-ef8351 · 2026-07-03T10:34:52Z

View all feedbacks in Devflow UI.

2026-07-03 10:34:51 UTC ℹ️ Start processing command /merge

2026-07-03 10:34:57 UTC ℹ️ MergeQueue: pull request added to the queue

The expected merge time in master is approximately 1h (p90).

2026-07-03 12:35:18 UTC ❌ MergeQueue: The build pipeline has timeout

The merge request has been interrupted because the build 1070108256444918620 took longer than expected. The current limit for the base branch 'master' is 120 minutes.

feat: avoid final field modifications in karate instrumentation

acf6e29

daniel-mohedano added type: enhancement Enhancements and improvements comp: ci visibility Continuous Integration Visibility labels Jun 26, 2026

daniel-mohedano added 3 commits June 29, 2026 14:06

fix: revert changes

277ad1a

feat: propagate failed results to original

eb735e2

fix: spotless

9c0449a

nit: comment

3eaec0c

daniel-mohedano marked this pull request as ready for review June 29, 2026 14:58

daniel-mohedano requested review from a team as code owners June 29, 2026 14:58

daniel-mohedano requested a review from mhlidd June 29, 2026 14:58

chatgpt-codex-connector Bot reviewed Jun 29, 2026

View reviewed changes

juan-fernandez approved these changes Jun 29, 2026

View reviewed changes

datadog-datadog-prod-us1-2 Bot reviewed Jun 29, 2026

View reviewed changes

Merge branch 'master' into daniel.mohedano/jep-500-karate

5205120

Merge branch 'master' into daniel.mohedano/jep-500-karate

eda96c2

Uh oh!

Conversation

daniel-mohedano commented Jun 26, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What Does This Do

Motivation

Additional Notes

Contributor Checklist

Uh oh!

cit-pr-commenter-54b7da Bot commented Jun 26, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Test Environment - sbt-scalatest

Uh oh!

cit-pr-commenter-54b7da Bot commented Jun 26, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Test Environment - nebula-release-plugin

Uh oh!

cit-pr-commenter-54b7da Bot commented Jun 26, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Test Environment - netflix-zuul

Uh oh!

dd-octo-sts Bot commented Jun 26, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

🟢 Java Benchmark SLOs — All performance SLOs passed

Uh oh!

cit-pr-commenter-54b7da Bot commented Jun 26, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Test Environment - jolokia

Uh oh!

cit-pr-commenter-54b7da Bot commented Jun 26, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Test Environment - sonar-kotlin

Uh oh!

cit-pr-commenter-54b7da Bot commented Jun 26, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Test Environment - okhttp

Uh oh!

cit-pr-commenter-54b7da Bot commented Jun 26, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Test Environment - spring_boot

Uh oh!

cit-pr-commenter-54b7da Bot commented Jun 26, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Test Environment - sonar-java

Uh oh!

cit-pr-commenter-54b7da Bot commented Jun 29, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Test Environment - pass4s

Uh oh!

cit-pr-commenter-54b7da Bot commented Jun 29, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Test Environment - reactive-streams-jvm

Uh oh!

chatgpt-codex-connector Bot left a comment

Choose a reason for hiding this comment

💡 Codex Review

Uh oh!

chatgpt-codex-connector Bot Jun 29, 2026

Choose a reason for hiding this comment

Uh oh!

datadog-datadog-prod-us1-2 Bot left a comment

Choose a reason for hiding this comment

Uh oh!

datadog-datadog-prod-us1-2 Bot commented Jul 1, 2026 • edited by datadog-prod-us1-5 Bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

daniel-mohedano commented Jul 3, 2026

Uh oh!

gh-worker-devflow-routing-ef8351 Bot commented Jul 3, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

daniel-mohedano commented Jul 3, 2026

Uh oh!

gh-worker-devflow-routing-ef8351 Bot commented Jul 3, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

daniel-mohedano commented Jun 26, 2026 •

edited

Loading

cit-pr-commenter-54b7da Bot commented Jun 26, 2026 •

edited

Loading

cit-pr-commenter-54b7da Bot commented Jun 26, 2026 •

edited

Loading

cit-pr-commenter-54b7da Bot commented Jun 26, 2026 •

edited

Loading

dd-octo-sts Bot commented Jun 26, 2026 •

edited

Loading

cit-pr-commenter-54b7da Bot commented Jun 26, 2026 •

edited

Loading

cit-pr-commenter-54b7da Bot commented Jun 26, 2026 •

edited

Loading

cit-pr-commenter-54b7da Bot commented Jun 26, 2026 •

edited

Loading

cit-pr-commenter-54b7da Bot commented Jun 26, 2026 •

edited

Loading

cit-pr-commenter-54b7da Bot commented Jun 26, 2026 •

edited

Loading

cit-pr-commenter-54b7da Bot commented Jun 29, 2026 •

edited

Loading

cit-pr-commenter-54b7da Bot commented Jun 29, 2026 •

edited

Loading

datadog-datadog-prod-us1-2 Bot commented Jul 1, 2026 •

edited by datadog-prod-us1-5 Bot

Loading

gh-worker-devflow-routing-ef8351 Bot commented Jul 3, 2026 •

edited

Loading

gh-worker-devflow-routing-ef8351 Bot commented Jul 3, 2026 •

edited

Loading