Retry dependency download failures in Maven and Gradle CI Visibility tests by daniel-mohedano · Pull Request #12907 · DataDog/dd-trace-java

daniel-mohedano · 2026-10-16T10:07:30Z

What Does This Do

Makes two CI Visibility test suites resilient to transient dependency download failures.

Embedded Maven tests (MavenProjectConfiguratorTest, MavenUtilsTest): when a Maven build fails because an artifact could not be transferred, it is retried up to 3 times. Builds that fail for other reasons are not retried.
GradleDaemonSmokeTest: the Gradle distribution is downloaded in advance into the Gradle user home that TestKit uses. Before, the advance download went into the TestKit folder, which TestKit does not use for distributions. So TestKit downloaded every distribution a second time with its own 10 second timeout.

Motivation

On master, over the last 3 weeks:

MavenProjectConfiguratorTest failed 3 times because the Maven repository proxy returned HTTP 500.
GradleDaemonSmokeTest (succeed-new-latest) failed 2 times because TestKit timed out downloading the Gradle distribution.

Additional Notes

Both changes were checked locally with a proxy that injects failures:

Maven: with a 500 injected on a Surefire artifact, master fails like on CI, and this branch passes. The test and latestDepTest suites of the module pass.
Gradle: each distribution is now downloaded once, by the advance download.

test-environment-trigger: skip

…or them

cit-pr-commenter-54b7da · 2026-10-16T10:20:33Z

CI Visibility Test Environment - sbt-scalatest

Job Status: 🟢 success

Scenario	This PR (%)	7d median	Δ 7d	30d median	Δ 30d	runs (7d/30d)
agent	58.19	57.69	$\color{red}{\blacktriangle}$ +0.50	56.55	$\color{red}{\blacktriangle}$ +1.64	62/310
agentEvpProxy	57.85	n/a	n/a	n/a	n/a	-

Baseline: median of @test.tracer_overhead on main (gitlab) over the last 7/30 days, per OSS project & scenario. Δ = this PR − baseline median; red ▲ = more overhead, green ▽ = less overhead than baseline.

Tests run to verify that CI Visibility behavior has not regressed in the current PR.

cit-pr-commenter-54b7da · 2026-10-16T10:23:08Z

CI Visibility Test Environment - nebula-release-plugin

Job Status: 🟢 success

Scenario	This PR (%)	7d median	Δ 7d	30d median	Δ 30d	runs (7d/30d)
agent	36.33	37.15	$\color{green}{\blacktriangledown}$ -0.82	37.15	$\color{green}{\blacktriangledown}$ -0.82	32/169
agentless	36.32	36.42	$\color{green}{\blacktriangledown}$ -0.10	36.42	$\color{green}{\blacktriangledown}$ -0.10	32/142
agentlessCodeCoverage	46.21	45.38	$\color{red}{\blacktriangle}$ +0.83	45.38	$\color{red}{\blacktriangle}$ +0.83	32/141
agentlessLineCoverage	56.97	56.55	$\color{red}{\blacktriangle}$ +0.42	56.55	$\color{red}{\blacktriangle}$ +0.42	31/140

Baseline: median of @test.tracer_overhead on main (gitlab) over the last 7/30 days, per OSS project & scenario. Δ = this PR − baseline median; red ▲ = more overhead, green ▽ = less overhead than baseline.

Tests run to verify that CI Visibility behavior has not regressed in the current PR.

cit-pr-commenter-54b7da · 2026-10-16T10:24:57Z

CI Visibility Test Environment - pass4s

Job Status: 🟢 success

Scenario	This PR (%)	7d median	Δ 7d	30d median	Δ 30d	runs (7d/30d)
agent	5.53	10.54	$\color{green}{\blacktriangledown}$ -5.01	10.13	$\color{green}{\blacktriangledown}$ -4.60	34/165
agentless	8.42	10.33	$\color{green}{\blacktriangledown}$ -1.91	10.13	$\color{green}{\blacktriangledown}$ -1.71	32/134
agentlessCodeCoverage	16.06	18.45	$\color{green}{\blacktriangledown}$ -2.39	17.73	$\color{green}{\blacktriangledown}$ -1.67	27/121

Baseline: median of @test.tracer_overhead on main (gitlab) over the last 7/30 days, per OSS project & scenario. Δ = this PR − baseline median; red ▲ = more overhead, green ▽ = less overhead than baseline.

Tests run to verify that CI Visibility behavior has not regressed in the current PR.

cit-pr-commenter-54b7da · 2026-10-16T10:25:08Z

CI Visibility Test Environment - netflix-zuul

Job Status: 🟢 success

Scenario	This PR (%)	7d median	Δ 7d	30d median	Δ 30d	runs (7d/30d)
agent	90.30	91.39	$\color{green}{\blacktriangledown}$ -1.09	89.58	$\color{red}{\blacktriangle}$ +0.72	33/171
agentless	81.66	82.69	$\color{green}{\blacktriangledown}$ -1.03	82.69	$\color{green}{\blacktriangledown}$ -1.03	33/141
agentlessCodeCoverage	97.24	99.00	$\color{green}{\blacktriangledown}$ -1.76	97.04	$\color{red}{\blacktriangle}$ +0.20	32/140
agentlessLineCoverage	113.45	113.88	$\color{green}{\blacktriangledown}$ -0.43	113.88	$\color{green}{\blacktriangledown}$ -0.43	32/141

Baseline: median of @test.tracer_overhead on main (gitlab) over the last 7/30 days, per OSS project & scenario. Δ = this PR − baseline median; red ▲ = more overhead, green ▽ = less overhead than baseline.

Tests run to verify that CI Visibility behavior has not regressed in the current PR.

cit-pr-commenter-54b7da · 2026-10-16T10:26:25Z

CI Visibility Test Environment - reactive-streams-jvm

Job Status: 🟢 success

Scenario	This PR (%)	7d median	Δ 7d	30d median	Δ 30d	runs (7d/30d)
agent	22.58	22.54	$\color{red}{\blacktriangle}$ +0.04	22.09	$\color{red}{\blacktriangle}$ +0.49	35/171
agentless	17.39	19.59	$\color{green}{\blacktriangledown}$ -2.20	19.59	$\color{green}{\blacktriangledown}$ -2.20	33/137
agentlessCodeCoverage	18.02	20.39	$\color{green}{\blacktriangledown}$ -2.37	19.99	$\color{green}{\blacktriangledown}$ -1.97	32/136
agentlessLineCoverage	24.71	27.52	$\color{green}{\blacktriangledown}$ -2.81	27.52	$\color{green}{\blacktriangledown}$ -2.81	20/93

Baseline: median of @test.tracer_overhead on main (gitlab) over the last 7/30 days, per OSS project & scenario. Δ = this PR − baseline median; red ▲ = more overhead, green ▽ = less overhead than baseline.

Tests run to verify that CI Visibility behavior has not regressed in the current PR.

cit-pr-commenter-54b7da · 2026-10-16T10:27:20Z

CI Visibility Test Environment - heliboard

Job Status: 🟢 success

Scenario	This PR (%)	7d median	Δ 7d	30d median	Δ 30d	runs (7d/30d)
agent	9.62	9.92	$\color{green}{\blacktriangledown}$ -0.30	9.92	$\color{green}{\blacktriangledown}$ -0.30	31/138

Baseline: median of @test.tracer_overhead on main (gitlab) over the last 7/30 days, per OSS project & scenario. Δ = this PR − baseline median; red ▲ = more overhead, green ▽ = less overhead than baseline.

Tests run to verify that CI Visibility behavior has not regressed in the current PR.

dd-octo-sts · 2026-10-16T10:31:24Z

🟢 Java Benchmark SLOs — All performance SLOs passed

Suite	Status
Startup	🟢 pass

SLO thresholds are defined here based on automatically generated metrics. A warning is raised when results are within 5% of the threshold.

PR vs. master results

Scenario	Candidate	master	Δ (95% CI of mean)
startup:insecure-bank:iast:Agent	13.99 s	13.95 s	[-0.5%; +1.1%] (no difference)
startup:insecure-bank:tracing:Agent	12.85 s	12.96 s	[-1.5%; -0.2%] (maybe better)
startup:petclinic:appsec:Agent	17.08 s	17.11 s	[-1.3%; +1.0%] (no difference)
startup:petclinic:iast:Agent	16.93 s	16.97 s	[-1.0%; +0.6%] (no difference)
startup:petclinic:profiling:Agent	16.82 s	16.81 s	[-1.2%; +1.3%] (no difference)
startup:petclinic:sca:Agent	17.05 s	16.74 s	[+1.1%; +2.7%] (significantly worse)
startup:petclinic:tracing:Agent	16.01 s	16.14 s	[-1.6%; +0.1%] (no difference)

Commit: 8f1be3b7 · CI Pipeline · Benchmarking Platform UI

Load and DaCapo benchmarks can be triggered manually in the GitLab pipeline. Results will appear in the Benchmarking Platform UI after completion.

cit-pr-commenter-54b7da · 2026-10-16T10:32:25Z

CI Visibility Test Environment - sonar-kotlin

Job Status: 🟢 success

Scenario	This PR (%)	7d median	Δ 7d	30d median	Δ 30d	runs (7d/30d)
agent	11.52	13.67	$\color{green}{\blacktriangledown}$ -2.15	13.40	$\color{green}{\blacktriangledown}$ -1.88	35/171
agentless	10.74	12.12	$\color{green}{\blacktriangledown}$ -1.38	12.12	$\color{green}{\blacktriangledown}$ -1.38	34/140
agentlessCodeCoverage	13.43	15.72	$\color{green}{\blacktriangledown}$ -2.29	15.41	$\color{green}{\blacktriangledown}$ -1.98	33/139
agentlessLineCoverage	15.55	17.38	$\color{green}{\blacktriangledown}$ -1.83	17.38	$\color{green}{\blacktriangledown}$ -1.83	31/137

Baseline: median of @test.tracer_overhead on main (gitlab) over the last 7/30 days, per OSS project & scenario. Δ = this PR − baseline median; red ▲ = more overhead, green ▽ = less overhead than baseline.

Tests run to verify that CI Visibility behavior has not regressed in the current PR.

cit-pr-commenter-54b7da · 2026-10-16T10:35:34Z

CI Visibility Test Environment - jolokia

Job Status: 🟢 success

Scenario	This PR (%)	7d median	Δ 7d	30d median	Δ 30d	runs (7d/30d)
agent	94.69	95.12	$\color{green}{\blacktriangledown}$ -0.43	95.12	$\color{green}{\blacktriangledown}$ -0.43	36/172
agentless	89.26	91.39	$\color{green}{\blacktriangledown}$ -2.13	89.58	$\color{green}{\blacktriangledown}$ -0.32	35/143
agentlessCodeCoverage	99.74	101.00	$\color{green}{\blacktriangledown}$ -1.26	99.00	$\color{red}{\blacktriangle}$ +0.74	33/141
agentlessLineCoverage	101.72	103.04	$\color{green}{\blacktriangledown}$ -1.32	101.00	$\color{red}{\blacktriangle}$ +0.72	38/149

Baseline: median of @test.tracer_overhead on main (gitlab) over the last 7/30 days, per OSS project & scenario. Δ = this PR − baseline median; red ▲ = more overhead, green ▽ = less overhead than baseline.

Tests run to verify that CI Visibility behavior has not regressed in the current PR.

cit-pr-commenter-54b7da · 2026-10-16T10:40:14Z

CI Visibility Test Environment - okhttp

Job Status: 🟢 success

Scenario	This PR (%)	7d median	Δ 7d	30d median	Δ 30d	runs (7d/30d)
agent	18.70	19.20	$\color{green}{\blacktriangledown}$ -0.50	19.99	$\color{green}{\blacktriangledown}$ -1.29	35/169
agentless	18.59	18.45	$\color{red}{\blacktriangle}$ +0.14	19.20	$\color{green}{\blacktriangledown}$ -0.61	36/145
agentlessCodeCoverage	21.40	21.65	$\color{green}{\blacktriangledown}$ -0.25	22.99	$\color{green}{\blacktriangledown}$ -1.59	36/146
agentlessLineCoverage	39.04	37.91	$\color{red}{\blacktriangle}$ +1.13	39.45	$\color{green}{\blacktriangledown}$ -0.41	32/139

Baseline: median of @test.tracer_overhead on main (gitlab) over the last 7/30 days, per OSS project & scenario. Δ = this PR − baseline median; red ▲ = more overhead, green ▽ = less overhead than baseline.

Tests run to verify that CI Visibility behavior has not regressed in the current PR.

cit-pr-commenter-54b7da · 2026-10-16T10:48:21Z

CI Visibility Test Environment - spring_boot

Job Status: 🟢 success

Scenario	This PR (%)	7d median	Δ 7d	30d median	Δ 30d	runs (7d/30d)
agent	16.28	16.36	$\color{green}{\blacktriangledown}$ -0.08	16.36	$\color{green}{\blacktriangledown}$ -0.08	35/168
agentless	10.96	9.92	$\color{red}{\blacktriangle}$ +1.04	9.92	$\color{red}{\blacktriangle}$ +1.04	35/140
agentlessCodeCoverage	13.62	13.67	$\color{green}{\blacktriangledown}$ -0.05	13.67	$\color{green}{\blacktriangledown}$ -0.05	34/141
agentlessLineCoverage	22.82	22.54	$\color{red}{\blacktriangle}$ +0.28	22.54	$\color{red}{\blacktriangle}$ +0.28	31/137

Baseline: median of @test.tracer_overhead on main (gitlab) over the last 7/30 days, per OSS project & scenario. Δ = this PR − baseline median; red ▲ = more overhead, green ▽ = less overhead than baseline.

Tests run to verify that CI Visibility behavior has not regressed in the current PR.

chatgpt-codex-connector · 2026-10-16T10:51:56Z

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review	Status	Commit	Review trigger
📝 Code Review	⚠️ Failed 2026-10-16T11:04:33.683660Z	`8f1be3b`	Draft marked ready
🔒 Security Review	✅ Completed 2026-10-16T10:54:45.141444Z	`8f1be3b`	Draft marked ready

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

datadog-datadog-prod-us1

More details

Maven retries are capped at three attempts and triggered by artifact-transfer failures. The Gradle pre-download now targets the user-home distribution cache used by TestKit.

Was this helpful? React 👍 or 👎

_{Open Bits AI session}

_{🤖 Bits Code Review · Commit 8f1be3b}

cit-pr-commenter-54b7da · 2026-10-16T11:12:42Z

CI Visibility Test Environment - sonar-java

Job Status: 🟢 success

Scenario	This PR (%)	7d median	Δ 7d	30d median	Δ 30d	runs (7d/30d)
agent	12.76	17.38	$\color{green}{\blacktriangledown}$ -4.62	15.41	$\color{green}{\blacktriangledown}$ -2.65	35/169
agentless	13.01	13.94	$\color{green}{\blacktriangledown}$ -0.93	13.13	$\color{green}{\blacktriangledown}$ -0.12	35/141
agentlessCodeCoverage	69.61	79.45	$\color{green}{\blacktriangledown}$ -9.84	79.45	$\color{green}{\blacktriangledown}$ -9.84	35/142
agentlessLineCoverage	78.28	123.36	$\color{green}{\blacktriangledown}$ -45.08	123.36	$\color{green}{\blacktriangledown}$ -45.08	33/143

Baseline: median of @test.tracer_overhead on main (gitlab) over the last 7/30 days, per OSS project & scenario. Δ = this PR − baseline median; red ▲ = more overhead, green ▽ = less overhead than baseline.

Tests run to verify that CI Visibility behavior has not regressed in the current PR.

AlexeyKuznetsov-DD

LGTM, left a comment about delay between attempts.

AlexeyKuznetsov-DD · 2026-10-16T15:22:49Z

+    MojoStartedSpy spy;
+    int attempt = 0;
+    do {
+      spy = new MojoStartedSpy(mojoStartedHandler);
+      MojoStartedSpy attemptSpy = spy;
+      MavenCli mavenCli =
+          new MavenCli() {
+            @Override
+            protected void customizeContainer(PlexusContainer container) {
+              container.addComponent(attemptSpy, EventSpy.class, null);
+            }
+          };
+      mavenCli.doMain(
+          arguments.toArray(new String[0]),
+          WORKING_DIRECTORY.toAbsolutePath().toString(),
+          stdOut,
+          stderr);
+    } while (spy.artifactTransferFailed && ++attempt < MAVEN_BUILD_ATTEMPTS);


Maybe worth to have some sleep(5-10 seconds) between attempts? or am I missing something?

daniel-mohedano added 2 commits October 16, 2026 10:23

test(maven): retry embedded Maven builds that fail to transfer artifacts

e50ce00

test(gradle): pre-download Gradle distributions where TestKit looks f…

8f1be3b

…or them

This comment has been minimized.

Sign in to view

daniel-mohedano marked this pull request as ready for review October 16, 2026 10:51

daniel-mohedano requested review from a team as code owners October 16, 2026 10:51

daniel-mohedano requested review from jordan-wong and removed request for a team October 16, 2026 10:51

datadog-datadog-prod-us1 Bot reviewed Oct 16, 2026

View reviewed changes

AlexeyKuznetsov-DD approved these changes Oct 16, 2026

View reviewed changes

Conversation

daniel-mohedano commented Oct 16, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What Does This Do

Motivation

Additional Notes

Uh oh!

cit-pr-commenter-54b7da Bot commented Oct 16, 2026

CI Visibility Test Environment - sbt-scalatest

Uh oh!

cit-pr-commenter-54b7da Bot commented Oct 16, 2026

CI Visibility Test Environment - nebula-release-plugin

Uh oh!

cit-pr-commenter-54b7da Bot commented Oct 16, 2026

CI Visibility Test Environment - pass4s

Uh oh!

cit-pr-commenter-54b7da Bot commented Oct 16, 2026

CI Visibility Test Environment - netflix-zuul

Uh oh!

This comment has been minimized.

cit-pr-commenter-54b7da Bot commented Oct 16, 2026

CI Visibility Test Environment - reactive-streams-jvm

Uh oh!

cit-pr-commenter-54b7da Bot commented Oct 16, 2026

CI Visibility Test Environment - heliboard

Uh oh!

dd-octo-sts Bot commented Oct 16, 2026

🟢 Java Benchmark SLOs — All performance SLOs passed

Uh oh!

cit-pr-commenter-54b7da Bot commented Oct 16, 2026

CI Visibility Test Environment - sonar-kotlin

Uh oh!

cit-pr-commenter-54b7da Bot commented Oct 16, 2026

CI Visibility Test Environment - jolokia

Uh oh!

cit-pr-commenter-54b7da Bot commented Oct 16, 2026

CI Visibility Test Environment - okhttp

Uh oh!

cit-pr-commenter-54b7da Bot commented Oct 16, 2026

CI Visibility Test Environment - spring_boot

Uh oh!

chatgpt-codex-connector Bot commented Oct 16, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codex Review Summary

Uh oh!

datadog-datadog-prod-us1 Bot left a comment

Choose a reason for hiding this comment

Uh oh!

cit-pr-commenter-54b7da Bot commented Oct 16, 2026

CI Visibility Test Environment - sonar-java

Uh oh!

AlexeyKuznetsov-DD left a comment

Choose a reason for hiding this comment

Uh oh!

AlexeyKuznetsov-DD Oct 16, 2026

Choose a reason for hiding this comment

Uh oh!

This branch has not been deployed

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

daniel-mohedano commented Oct 16, 2026 •

edited

Loading

chatgpt-codex-connector Bot commented Oct 16, 2026 •

edited

Loading