Compare release artifacts with jardiff before publishing to Maven Central by bric3 · Pull Request #11856 · DataDog/dd-trace-java

bric3 · 2026-07-03T18:37:32Z

What Does This Do

Adds a jar comparison between the artifacts produced by the build job and those produced by the deploy_to_maven_central job before the jar gets published to maven central, i.e. before publishMavenPublicationToSonatypeRepository executes.

If a difference is found the deploy_to_maven_central fails. It will also prevent to publish the jar to github releases.

This PR introduces a dd-trace-java.jardiff plugin that uses under the hood the jardiff. The tool is run with --stat --exit-code to compare a freshly built jar against a reference jar from build.
Failing the build if they differ.

Trimmed task graph (--task-graph) when invoking publishToSonatype :

Tasks graph for: :dd-java-agent:publishToSonatype
\--- :dd-java-agent:publishToSonatype (org.gradle.api.DefaultTask)
     \--- :dd-java-agent:publishMavenPublicationToSonatypeRepository (org.gradle.api.publish.maven.tasks.PublishToMavenRepository)
          +--- :initializeSonatypeStagingRepository (io.github.gradlenexus.publishplugin.InitializeNexusStagingRepository)
          +--- :dd-java-agent:compareToReferenceJar (datadog.gradle.plugin.jardiff.JardiffTask)
          |    \--- :dd-java-agent:shadowJar (com.github.jengelman.gradle.plugins.shadow.tasks.ShadowJar) (+)
          +--- :dd-java-agent:generatePomFileForMavenPublication (org.gradle.api.publish.maven.tasks.GenerateMavenPom)
          +--- :dd-java-agent:javadocJar (org.gradle.api.tasks.bundling.Jar)
          |    \--- :dd-java-agent:javadoc (org.gradle.api.tasks.javadoc.Javadoc) (+)
          +--- :dd-java-agent:shadowJar (*)
          +--- :dd-java-agent:signMavenPublication (org.gradle.plugins.signing.Sign)
          |    +--- :dd-java-agent:generatePomFileForMavenPublication (*)
          |    +--- :dd-java-agent:javadocJar (*)
          |    +--- :dd-java-agent:shadowJar (*)
          |    \--- :dd-java-agent:sourcesJar (org.gradle.api.tasks.bundling.Jar) (+)
          \--- :dd-java-agent:sourcesJar (*)

Motivation

The build job produces deliverable jar artifacts, those are used during the whole pipeline for all tests, system-tests, benchmark included. It is also used for OCI images. The build job also pushes to the lib gitlab cache, the Gradle build cache.
However, the deploy_to_maven_central depends on build job and reuse the build cache rehydrated from the lib gitlab cache.

That allows to reuse what was computed in the build job, including the produced artifact, and usually that works fine.

However, if for any reason the build script has a bug, it might result in a different artifact being built, possibly a faulty artifact.
That artifact is the one that gets published to customer of maven central or github release.

Note

The above description iswhat happened in release 1.60.0: a faulty jar reached Maven Central and GitHub.

In order to prevent such a faulty artifact to be published, this PR introduces an intermediate tasks whose role is to compare the artifacts produced from the build job with the one from the current deploy_to_maven_central Gradle invocation.

Additional Notes

This task runs on all publication unless explicitly disabled:
- feature-flagging-api dd-openfeature in particular disable the task at this point because build do not build these jars
- deploy_snapshot_with_ddprof_snapshot excludes it (-x compareToReferenceJar) since its ddprof-snapshot jar differs by design
publishToMavenLocal stays ungated (it is a PublishToMavenLocal, not a PublishToMavenRepository), so local installs are unaffected.

Simple tests to compare to another jar

Run it directly, no credentials or network:

Build a "reference" jar (stand-in for the build job's validated artifact)

./gradlew :dd-trace-api:jar
mkdir -p /tmp/jardiff-ref && cp dd-trace-api/build/libs/dd-trace-api-*.jar /tmp/jardiff-ref/

Compare the freshly built jar against it, if identical this succeeds

./gradlew :dd-trace-api:compareToReferenceJar --reference-jar=/tmp/jardiff-ref/dd-trace-api-<version>.jar

The current wiring do not handle publishToMavenLocal. E.g. ./gradlew :dd-trace-api:publishToMavenLocal installs to ~/.m2 without triggering it.

Simulating a divergent rebuild

Reproducing a second build that produces a different jar, with a real subproject change flowing into a published shadow jar.

dd-trace-ot bundles the :dd-trace-ot:correlation-id-injection subproject (relocated under ddtrot/):

Reference: the shadow jar as it stands today.

./gradlew :dd-trace-ot:shadowJar --build-cache
mkdir -p /tmp/jardiff-ref && cp dd-trace-ot/build/libs/dd-trace-ot-*.jar /tmp/jardiff-ref/

Patch a class in the bundled subproject, e.g. add a method to

dd-trace-ot/correlation-id-injection/src/main/java/datadog/trace/correlation/Slf4jCorrelationIdInjector.java
Any change to a bundled class works: add/remove/rename a method or class)

Rebuild and compare, the changed class is recompiled and re-bundled, as compareToReferenceJar has the jar task as dependency.

The comparison should fail and should pinpoint what's changed.

./gradlew :dd-trace-ot:compareToReferenceJar -PjardiffReferenceDir=/tmp/jardiff-ref  --build-cache
...
> Built jar differs from the reference jar — refusing to publish.
    ddtrot/dd/trace/correlation/Slf4jCorrelationIdInjector.class | 9 +++++++++
    1 files changed, 9 insertions(+), 0 deletions(-)

Other alternatives

During the previous attempt in #11733, an idea emerged to use a non Gradle tool to make the publication, like jreleaser. However, this require a deeper change (build has to make all the necessary artifacts, they need to be in a proper shape to be handed over to jreleaser, configuring jreleaser, make team aware of jreleaser failure modes).

datadog-datadog-prod-us1 · 2026-07-03T18:53:27Z

🎯 Code Coverage (details)
• Patch Coverage: 100.00%
• Overall Coverage: 56.94% (-0.02%)

_{This comment will be updated automatically if new data arrives.

🔗 Commit SHA: 23288b0 | Docs | Datadog PR Page | Give us feedback!}

dd-octo-sts · 2026-07-03T19:01:34Z

🟢 Java Benchmark SLOs — All performance SLOs passed

Suite	Status
Startup	🟢 pass

SLO thresholds are defined here based on automatically generated metrics. A warning is raised when results are within 5% of the threshold.

PR vs. master results

Scenario	Candidate	master	Δ (95% CI of mean)
startup:insecure-bank:iast:Agent	14.06 s	13.96 s	[-0.0%; +1.5%] (no difference)
startup:insecure-bank:tracing:Agent	12.90 s	12.96 s	[-1.2%; +0.2%] (no difference)
startup:petclinic:appsec:Agent	16.95 s	16.68 s	[+0.7%; +2.5%] (maybe worse)
startup:petclinic:iast:Agent	16.34 s	16.94 s	[-8.0%; +0.8%] (no difference)
startup:petclinic:profiling:Agent	16.85 s	16.54 s	[-2.8%; +6.5%] (no difference)
startup:petclinic:sca:Agent	16.89 s	16.81 s	[-0.5%; +1.4%] (no difference)
startup:petclinic:tracing:Agent	16.11 s	15.92 s	[+0.1%; +2.2%] (maybe worse)

Commit: 23288b07 · CI Pipeline · Benchmarking Platform UI

Load and DaCapo benchmarks can be triggered manually in the GitLab pipeline. Results will appear in the Benchmarking Platform UI after completion.

`deploy_to_maven_central` rebuilds the artifacts (reusing the build cache) before publishing, so a fault in the build script could make the rebuild to produce a faulty jar that differs from the one the build job. Which was validated across the whole test chain. This add a `compareToReferenceJar` task (via a new `dd-trace-java.jardiff` plugin) that runs jardiff `--stat --exit-code` against a reference jar (the one from `build` job) and fails on any difference. This allows to gate maven publication (and github releases).

mhdatie · 2026-07-06T15:48:52Z

@bric3

However, if for any reason the build script has a bug, it might result in a different artifact being built, possibly a faulty artifact

Is the motivation here that if the build was faulty, then it would result in a cache miss during the deploy to maven step, and that's why we're comparing the two artifacts?

.. Or would the maven deployment step use the faulty build cache anyways? Because that would result in the same build, no?

bric3 · 2026-07-06T16:07:27Z

Is the motivation here that if the build was faulty, then it would result in a cache miss during the deploy to maven step, and that's why we're comparing the two artifacts?

Not necessarily a cache miss, but anything that alters the output in an unintended way, that could include how gradle uses it's build cache and if the build infrastructure we have has "bugs" that surface in these conditions.

.. Or would the maven deployment step use the faulty build cache anyways? Because that would result in the same build, no?

It's supposed to result in the same, however, if Gradle detects out-of-date items, it can re-execute parts of the build to account for that, this is the whole idea behind that feature, only rebuild what changed.

It's unlikely to happen, but if our customization to the build script have an error this can lead to unintended bad interaction. This is what happened during 1.60.0 release where parts of the classes disappeared.

bric3 added type: feature Enhancements and improvements tag: no release notes Changes to exclude from release notes comp: tooling Build & Tooling labels Jul 3, 2026

bric3 force-pushed the bdu/jardiff-task-before-publish branch 2 times, most recently from c8a141d to 2b41fdf Compare July 3, 2026 21:12

bric3 force-pushed the bdu/jardiff-task-before-publish branch from 2b41fdf to 23288b0 Compare July 3, 2026 21:33

bric3 marked this pull request as ready for review July 3, 2026 21:39

bric3 requested review from a team as code owners July 3, 2026 21:39

bric3 requested review from AlexeyKuznetsov-DD, PerfectSlayer, mcculls, mhlidd, randomanderson, sameerank, sarahchen6 and typotter and removed request for a team July 3, 2026 21:39

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Compare release artifacts with jardiff before publishing to Maven Central#11856

Compare release artifacts with jardiff before publishing to Maven Central#11856
bric3 wants to merge 1 commit into
masterfrom
bdu/jardiff-task-before-publish

bric3 commented Jul 3, 2026 •

edited

Loading

Uh oh!

datadog-datadog-prod-us1 Bot commented Jul 3, 2026 •

edited by datadog-official Bot

Loading

Uh oh!

dd-octo-sts Bot commented Jul 3, 2026 •

edited

Loading

Uh oh!

mhdatie commented Jul 6, 2026

Uh oh!

bric3 commented Jul 6, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Uh oh!

Conversation

bric3 commented Jul 3, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What Does This Do

Motivation

Additional Notes

Simple tests to compare to another jar

Simulating a divergent rebuild

Other alternatives

Uh oh!

datadog-datadog-prod-us1 Bot commented Jul 3, 2026 • edited by datadog-official Bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

dd-octo-sts Bot commented Jul 3, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

🟢 Java Benchmark SLOs — All performance SLOs passed

Uh oh!

mhdatie commented Jul 6, 2026

Uh oh!

bric3 commented Jul 6, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

bric3 commented Jul 3, 2026 •

edited

Loading

datadog-datadog-prod-us1 Bot commented Jul 3, 2026 •

edited by datadog-official Bot

Loading

dd-octo-sts Bot commented Jul 3, 2026 •

edited

Loading