Improve Test Optimization's smoke tests clarity by daniel-mohedano · Pull Request #10028 · DataDog/dd-trace-java

daniel-mohedano · 2025-11-25T10:05:37Z

What Does This Do

Includes several improvements to Test Optimization's smoke test to reduce noise in test and CI logs:
- Disables Git client. This will avoid logging the failed Git commands due to smoke tests not being run on a real repository.
- Disables Code Origins. Although there wasn't any noticeable negative impact in the smoke tests' performance, this will reduce the complexity of the tracer when running them.
- Set TRACE_AGENT_URL to the mocked backend URL. This avoids error logs like EXCLUDE_TELEMETRY datadog.communication.ddagent.DDAgentFeaturesDiscovery - Error querying v0.4/traces at http://localhost:8126/ from being reported by the smoke tests, reducing the noise. We also had reports a couple of weeks back of these errors appearing in the Logs product and causing confusion on teams.
- Increase Gradle smoke test timeout to 25m. Last week we identified two failures due to timeouts + retries. Increasing the timeout value in case it wasn't a transient issue with CI and to try to have the task finish without being killed in order to generate the XML reports. Reports not being generated on timeout will hopefully be addressed in a future PR.
- Add error logging to GradleLauncherSmokeTest on failure. This will provide better context on the failures if the reports are not generated.

Motivation

These changes will hopefully reduce the noise and the confusion when trying to identify failures in the smoke test jobs.

Contributor Checklist

Format the title according the contribution guidelines
Assign the type: and (comp: or inst:) labels in addition to any useful labels
Don't use close, fix or any linking keywords when referencing an issue.
Use solves instead, and assign the PR milestone to the issue
Update the CODEOWNERS file on source file addition, move, or deletion
Update the public documentation in case of new configuration flag or behavior

Jira ticket: SDTEST-2749

Co-authored-by: Nikita Tkachenko <121111529+nikita-tkachenko-datadog@users.noreply.github.com>

cit-pr-commenter · 2025-11-25T11:15:27Z

Test Environment - sbt-scalatest

Job Status: success

Scenario	Overhead (%)
agent	57.36
agentEvpProxy	58.22

cit-pr-commenter · 2025-11-25T11:15:54Z

Test Environment - nebula-release-plugin

Job Status: success

Scenario	Overhead (%)
agent	38.37
agentless	37.60
agentlessCodeCoverage	45.50
agentlessLineCoverage	76.78

cit-pr-commenter · 2025-11-25T11:18:07Z

Test Environment - pass4s

Job Status: success

Scenario	Overhead (%)
agent	13.07
agentless	9.70
agentlessCodeCoverage	16.30

cit-pr-commenter · 2025-11-25T11:18:39Z

Test Environment - reactive-streams-jvm

Job Status: success

Scenario	Overhead (%)
agent	22.33
agentless	19.25
agentlessCodeCoverage	20.29
agentlessLineCoverage	30.33

cit-pr-commenter · 2025-11-25T11:18:50Z

Test Environment - netflix-zuul

Job Status: success

Scenario	Overhead (%)
agent	91.28
agentless	84.31
agentlessCodeCoverage	100.26
agentlessLineCoverage	113.93

cit-pr-commenter · 2025-11-25T11:24:50Z

Test Environment - sonar-kotlin

Job Status: success

Scenario	Overhead (%)
agent	13.06
agentless	11.42
agentlessCodeCoverage	15.84
agentlessLineCoverage	19.22

cit-pr-commenter · 2025-11-25T11:26:54Z

Test Environment - jolokia

Job Status: success

Scenario	Overhead (%)
agent	87.35
agentless	83.20
agentlessCodeCoverage	92.57
agentlessLineCoverage	94.16

cit-pr-commenter · 2025-11-25T11:33:05Z

Test Environment - okhttp

Job Status: success

Scenario	Overhead (%)
agent	20.03
agentless	18.53
agentlessCodeCoverage	21.79
agentlessLineCoverage	37.11

pr-commenter · 2025-11-25T11:48:27Z

Benchmarks

Startup

Parameters

	Baseline	Candidate
baseline_or_candidate	baseline	candidate
git_branch	master	daniel.mohedano/testopt-smoke-disable-git
git_commit_date	1763974460	1764068421
git_commit_sha	`c79dfc6`	`aa62b32`
release_version	1.57.0-SNAPSHOT~c79dfc656d	1.57.0-SNAPSHOT~aa62b32a6a

See matching parameters

	Baseline	Candidate
application	insecure-bank	insecure-bank
ci_job_date	1764070294	1764070294
ci_job_id	1255500840	1255500840
ci_pipeline_id	83921622	83921622
cpu_model	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
kernel_version	Linux runner-zfyrx7zua-project-304-concurrent-0-43f4y2n5 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux	Linux runner-zfyrx7zua-project-304-concurrent-0-43f4y2n5 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux
module	Agent	Agent
parent	None	None

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 60 metrics, 5 unstable metrics.

Startup time reports for insecure-bank

gantt
    title insecure-bank - global startup overhead: candidate=1.57.0-SNAPSHOT~aa62b32a6a, baseline=1.57.0-SNAPSHOT~c79dfc656d

    dateFormat X
    axisFormat %s
section tracing
Agent [baseline] (1.112 s) : 0, 1112100
Total [baseline] (8.9 s) : 0, 8899914
Agent [candidate] (1.103 s) : 0, 1102802
Total [candidate] (8.88 s) : 0, 8880409
section iast
Agent [baseline] (1.237 s) : 0, 1237337
Total [baseline] (9.557 s) : 0, 9557225
Agent [candidate] (1.241 s) : 0, 1240934
Total [candidate] (9.596 s) : 0, 9596369

baseline results

Module	Variant	Duration	Δ tracing
Agent	tracing	1.112 s	-
Agent	iast	1.237 s	125.237 ms (11.3%)
Total	tracing	8.9 s	-
Total	iast	9.557 s	657.311 ms (7.4%)

candidate results

Module	Variant	Duration	Δ tracing
Agent	tracing	1.103 s	-
Agent	iast	1.241 s	138.132 ms (12.5%)
Total	tracing	8.88 s	-
Total	iast	9.596 s	715.96 ms (8.1%)

gantt
    title insecure-bank - break down per module: candidate=1.57.0-SNAPSHOT~aa62b32a6a, baseline=1.57.0-SNAPSHOT~c79dfc656d

    dateFormat X
    axisFormat %s
section tracing
crashtracking [baseline] (1.522 ms) : 0, 1522
crashtracking [candidate] (1.49 ms) : 0, 1490
BytebuddyAgent [baseline] (716.524 ms) : 0, 716524
BytebuddyAgent [candidate] (709.065 ms) : 0, 709065
GlobalTracer [baseline] (250.44 ms) : 0, 250440
GlobalTracer [candidate] (249.554 ms) : 0, 249554
AppSec [baseline] (32.333 ms) : 0, 32333
AppSec [candidate] (32.078 ms) : 0, 32078
Debugger [baseline] (63.452 ms) : 0, 63452
Debugger [candidate] (63.103 ms) : 0, 63103
Remote Config [baseline] (630.562 µs) : 0, 631
Remote Config [candidate] (644.458 µs) : 0, 644
Telemetry [baseline] (8.359 ms) : 0, 8359
Telemetry [candidate] (8.226 ms) : 0, 8226
Flare Poller [baseline] (3.733 ms) : 0, 3733
Flare Poller [candidate] (3.734 ms) : 0, 3734
section iast
crashtracking [baseline] (1.504 ms) : 0, 1504
crashtracking [candidate] (1.487 ms) : 0, 1487
BytebuddyAgent [baseline] (831.014 ms) : 0, 831014
BytebuddyAgent [candidate] (833.142 ms) : 0, 833142
GlobalTracer [baseline] (237.433 ms) : 0, 237433
GlobalTracer [candidate] (237.968 ms) : 0, 237968
IAST [baseline] (27.282 ms) : 0, 27282
IAST [candidate] (26.622 ms) : 0, 26622
AppSec [baseline] (33.874 ms) : 0, 33874
AppSec [candidate] (34.827 ms) : 0, 34827
Debugger [baseline] (59.823 ms) : 0, 59823
Debugger [candidate] (60.339 ms) : 0, 60339
Remote Config [baseline] (541.042 µs) : 0, 541
Remote Config [candidate] (540.675 µs) : 0, 541
Telemetry [baseline] (7.545 ms) : 0, 7545
Telemetry [candidate] (7.592 ms) : 0, 7592
Flare Poller [baseline] (3.436 ms) : 0, 3436
Flare Poller [candidate] (3.518 ms) : 0, 3518

Startup time reports for petclinic

gantt
    title petclinic - global startup overhead: candidate=1.57.0-SNAPSHOT~aa62b32a6a, baseline=1.57.0-SNAPSHOT~c79dfc656d

    dateFormat X
    axisFormat %s
section tracing
Agent [baseline] (1.103 s) : 0, 1102717
Total [baseline] (10.916 s) : 0, 10915831
Agent [candidate] (1.105 s) : 0, 1104711
Total [candidate] (10.81 s) : 0, 10809999
section appsec
Agent [baseline] (1.286 s) : 0, 1285649
Total [baseline] (11.217 s) : 0, 11216545
Agent [candidate] (1.288 s) : 0, 1288447
Total [candidate] (11.398 s) : 0, 11397887
section iast
Agent [baseline] (1.242 s) : 0, 1242234
Total [baseline] (11.327 s) : 0, 11327244
Agent [candidate] (1.253 s) : 0, 1253054
Total [candidate] (11.428 s) : 0, 11427785
section profiling
Agent [baseline] (1.239 s) : 0, 1238926
Total [baseline] (11.156 s) : 0, 11156273
Agent [candidate] (1.243 s) : 0, 1242798
Total [candidate] (11.187 s) : 0, 11186547

baseline results

Module	Variant	Duration	Δ tracing
Agent	tracing	1.103 s	-
Agent	appsec	1.286 s	182.932 ms (16.6%)
Agent	iast	1.242 s	139.518 ms (12.7%)
Agent	profiling	1.239 s	136.209 ms (12.4%)
Total	tracing	10.916 s	-
Total	appsec	11.217 s	300.713 ms (2.8%)
Total	iast	11.327 s	411.413 ms (3.8%)
Total	profiling	11.156 s	240.442 ms (2.2%)

candidate results

Module	Variant	Duration	Δ tracing
Agent	tracing	1.105 s	-
Agent	appsec	1.288 s	183.736 ms (16.6%)
Agent	iast	1.253 s	148.343 ms (13.4%)
Agent	profiling	1.243 s	138.088 ms (12.5%)
Total	tracing	10.81 s	-
Total	appsec	11.398 s	587.888 ms (5.4%)
Total	iast	11.428 s	617.786 ms (5.7%)
Total	profiling	11.187 s	376.548 ms (3.5%)

gantt
    title petclinic - break down per module: candidate=1.57.0-SNAPSHOT~aa62b32a6a, baseline=1.57.0-SNAPSHOT~c79dfc656d

    dateFormat X
    axisFormat %s
section tracing
crashtracking [baseline] (1.484 ms) : 0, 1484
crashtracking [candidate] (1.478 ms) : 0, 1478
BytebuddyAgent [baseline] (708.692 ms) : 0, 708692
BytebuddyAgent [candidate] (709.222 ms) : 0, 709222
GlobalTracer [baseline] (249.365 ms) : 0, 249365
GlobalTracer [candidate] (249.741 ms) : 0, 249741
AppSec [baseline] (31.817 ms) : 0, 31817
AppSec [candidate] (32.086 ms) : 0, 32086
Debugger [baseline] (63.924 ms) : 0, 63924
Debugger [candidate] (64.571 ms) : 0, 64571
Remote Config [baseline] (640.439 µs) : 0, 640
Remote Config [candidate] (625.0 µs) : 0, 625
Telemetry [baseline] (8.25 ms) : 0, 8250
Telemetry [candidate] (8.317 ms) : 0, 8317
Flare Poller [baseline] (3.696 ms) : 0, 3696
Flare Poller [candidate] (3.795 ms) : 0, 3795
section appsec
crashtracking [baseline] (1.505 ms) : 0, 1505
crashtracking [candidate] (1.498 ms) : 0, 1498
BytebuddyAgent [baseline] (733.62 ms) : 0, 733620
BytebuddyAgent [candidate] (735.123 ms) : 0, 735123
GlobalTracer [baseline] (241.552 ms) : 0, 241552
GlobalTracer [candidate] (242.042 ms) : 0, 242042
IAST [baseline] (24.821 ms) : 0, 24821
IAST [candidate] (24.765 ms) : 0, 24765
AppSec [baseline] (174.716 ms) : 0, 174716
AppSec [candidate] (175.001 ms) : 0, 175001
Debugger [baseline] (61.452 ms) : 0, 61452
Debugger [candidate] (62.122 ms) : 0, 62122
Remote Config [baseline] (671.432 µs) : 0, 671
Remote Config [candidate] (666.957 µs) : 0, 667
Telemetry [baseline] (8.313 ms) : 0, 8313
Telemetry [candidate] (8.229 ms) : 0, 8229
Flare Poller [baseline] (3.996 ms) : 0, 3996
Flare Poller [candidate] (3.973 ms) : 0, 3973
section iast
crashtracking [baseline] (1.482 ms) : 0, 1482
crashtracking [candidate] (1.508 ms) : 0, 1508
BytebuddyAgent [baseline] (833.577 ms) : 0, 833577
BytebuddyAgent [candidate] (841.251 ms) : 0, 841251
GlobalTracer [baseline] (238.296 ms) : 0, 238296
GlobalTracer [candidate] (239.754 ms) : 0, 239754
IAST [baseline] (28.406 ms) : 0, 28406
IAST [candidate] (27.01 ms) : 0, 27010
AppSec [baseline] (32.96 ms) : 0, 32960
AppSec [candidate] (35.01 ms) : 0, 35010
Debugger [baseline] (60.943 ms) : 0, 60943
Debugger [candidate] (61.488 ms) : 0, 61488
Remote Config [baseline] (538.349 µs) : 0, 538
Remote Config [candidate] (555.101 µs) : 0, 555
Telemetry [baseline] (7.568 ms) : 0, 7568
Telemetry [candidate] (7.698 ms) : 0, 7698
Flare Poller [baseline] (3.496 ms) : 0, 3496
Flare Poller [candidate] (3.595 ms) : 0, 3595
section profiling
crashtracking [baseline] (1.447 ms) : 0, 1447
crashtracking [candidate] (1.448 ms) : 0, 1448
BytebuddyAgent [baseline] (739.387 ms) : 0, 739387
BytebuddyAgent [candidate] (740.671 ms) : 0, 740671
GlobalTracer [baseline] (223.81 ms) : 0, 223810
GlobalTracer [candidate] (224.168 ms) : 0, 224168
AppSec [baseline] (32.203 ms) : 0, 32203
AppSec [candidate] (32.586 ms) : 0, 32586
Debugger [baseline] (63.301 ms) : 0, 63301
Debugger [candidate] (63.744 ms) : 0, 63744
Remote Config [baseline] (641.913 µs) : 0, 642
Remote Config [candidate] (662.298 µs) : 0, 662
Telemetry [baseline] (7.905 ms) : 0, 7905
Telemetry [candidate] (8.125 ms) : 0, 8125
Flare Poller [baseline] (3.753 ms) : 0, 3753
Flare Poller [candidate] (3.821 ms) : 0, 3821
ProfilingAgent [baseline] (96.929 ms) : 0, 96929
ProfilingAgent [candidate] (97.986 ms) : 0, 97986
Profiling [baseline] (97.51 ms) : 0, 97510
Profiling [candidate] (98.576 ms) : 0, 98576

Load

Parameters

	Baseline	Candidate
baseline_or_candidate	baseline	candidate
git_branch	master	daniel.mohedano/testopt-smoke-disable-git
git_commit_date	1763974460	1764068421
git_commit_sha	`c79dfc6`	`aa62b32`
release_version	1.57.0-SNAPSHOT~c79dfc656d	1.57.0-SNAPSHOT~aa62b32a6a

See matching parameters

	Baseline	Candidate
application	insecure-bank	insecure-bank
ci_job_date	1764070770	1764070770
ci_job_id	1255500841	1255500841
ci_pipeline_id	83921622	83921622
cpu_model	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
kernel_version	Linux runner-zfyrx7zua-project-304-concurrent-1-98e4z9kr 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux	Linux runner-zfyrx7zua-project-304-concurrent-1-98e4z9kr 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux

Summary

Found 4 performance improvements and 1 performance regressions! Performance is the same for 15 metrics, 16 unstable metrics.

scenario	Δ mean agg_http_req_duration_p50	Δ mean agg_http_req_duration_p95	Δ mean throughput	candidate mean agg_http_req_duration_p50	candidate mean agg_http_req_duration_p95	candidate mean throughput	baseline mean agg_http_req_duration_p50	baseline mean agg_http_req_duration_p95	baseline mean throughput
scenario:load:petclinic:profiling:high_load	better [-1.742ms; -0.814ms] or [-9.005%; -4.208%]	better [-2.511ms; -1.060ms] or [-8.101%; -3.421%]	unstable [-9.254op/s; +43.504op/s] or [-3.892%; +18.298%]	18.070ms	29.207ms	254.875op/s	19.349ms	30.993ms	237.750op/s
scenario:load:petclinic:appsec:high_load	better [-1.683ms; -0.701ms] or [-8.717%; -3.630%]	unsure [-2.589ms; -0.624ms] or [-8.253%; -1.988%]	unstable [-11.722op/s; +41.097op/s] or [-4.927%; +17.275%]	18.120ms	29.764ms	252.594op/s	19.313ms	31.371ms	237.906op/s
scenario:load:petclinic:code_origins:high_load	worse [+506.079µs; +1220.353µs] or [+2.921%; +7.043%]	unsure [+0.553ms; +2.258ms] or [+1.954%; +7.975%]	unstable [-40.447op/s; +15.134op/s] or [-15.339%; +5.739%]	18.189ms	29.722ms	251.031op/s	17.326ms	28.316ms	263.688op/s
scenario:load:petclinic:no_agent:high_load	better [-2.243ms; -0.419ms] or [-11.808%; -2.207%]	unstable [-4.179ms; -0.570ms] or [-13.229%; -1.803%]	unstable [-14.486op/s; +42.986op/s] or [-6.009%; +17.832%]	17.663ms	29.217ms	255.312op/s	18.994ms	31.592ms	241.062op/s

Request duration reports for petclinic

gantt
    title petclinic - request duration [CI 0.99] : candidate=1.57.0-SNAPSHOT~aa62b32a6a, baseline=1.57.0-SNAPSHOT~c79dfc656d
    dateFormat X
    axisFormat %s
section baseline
no_agent (19.361 ms) : 19163, 19558
.   : milestone, 19361,
appsec (19.619 ms) : 19415, 19823
.   : milestone, 19619,
code_origins (17.693 ms) : 17518, 17867
.   : milestone, 17693,
iast (17.703 ms) : 17528, 17879
.   : milestone, 17703,
profiling (19.637 ms) : 19439, 19834
.   : milestone, 19637,
tracing (17.821 ms) : 17644, 17998
.   : milestone, 17821,
section candidate
no_agent (18.278 ms) : 18091, 18465
.   : milestone, 18278,
appsec (18.476 ms) : 18289, 18663
.   : milestone, 18476,
code_origins (18.59 ms) : 18403, 18776
.   : milestone, 18590,
iast (17.58 ms) : 17404, 17756
.   : milestone, 17580,
profiling (18.314 ms) : 18132, 18496
.   : milestone, 18314,
tracing (17.701 ms) : 17526, 17876
.   : milestone, 17701,

baseline results

Variant	Request duration [CI 0.99]	Δ no_agent
no_agent	19.361 ms [19.163 ms, 19.558 ms]	-
appsec	19.619 ms [19.415 ms, 19.823 ms]	258.579 µs (1.3%)
code_origins	17.693 ms [17.518 ms, 17.867 ms]	-1.668 ms (-8.6%)
iast	17.703 ms [17.528 ms, 17.879 ms]	-1.657 ms (-8.6%)
profiling	19.637 ms [19.439 ms, 19.834 ms]	275.893 µs (1.4%)
tracing	17.821 ms [17.644 ms, 17.998 ms]	-1.539 ms (-8.0%)

candidate results

Variant	Request duration [CI 0.99]	Δ no_agent
no_agent	18.278 ms [18.091 ms, 18.465 ms]	-
appsec	18.476 ms [18.289 ms, 18.663 ms]	198.04 µs (1.1%)
code_origins	18.59 ms [18.403 ms, 18.776 ms]	311.664 µs (1.7%)
iast	17.58 ms [17.404 ms, 17.756 ms]	-698.088 µs (-3.8%)
profiling	18.314 ms [18.132 ms, 18.496 ms]	36.206 µs (0.2%)
tracing	17.701 ms [17.526 ms, 17.876 ms]	-577.164 µs (-3.2%)

Request duration reports for insecure-bank

gantt
    title insecure-bank - request duration [CI 0.99] : candidate=1.57.0-SNAPSHOT~aa62b32a6a, baseline=1.57.0-SNAPSHOT~c79dfc656d
    dateFormat X
    axisFormat %s
section baseline
no_agent (1.205 ms) : 1193, 1217
.   : milestone, 1205,
iast (3.218 ms) : 3171, 3265
.   : milestone, 3218,
iast_FULL (5.819 ms) : 5760, 5877
.   : milestone, 5819,
iast_GLOBAL (3.635 ms) : 3582, 3688
.   : milestone, 3635,
profiling (2.035 ms) : 2016, 2053
.   : milestone, 2035,
tracing (1.813 ms) : 1798, 1828
.   : milestone, 1813,
section candidate
no_agent (1.21 ms) : 1198, 1222
.   : milestone, 1210,
iast (3.2 ms) : 3158, 3242
.   : milestone, 3200,
iast_FULL (5.745 ms) : 5688, 5802
.   : milestone, 5745,
iast_GLOBAL (3.7 ms) : 3636, 3764
.   : milestone, 3700,
profiling (2.08 ms) : 2061, 2099
.   : milestone, 2080,
tracing (1.874 ms) : 1857, 1890
.   : milestone, 1874,

baseline results

Variant	Request duration [CI 0.99]	Δ no_agent
no_agent	1.205 ms [1.193 ms, 1.217 ms]	-
iast	3.218 ms [3.171 ms, 3.265 ms]	2.013 ms (167.0%)
iast_FULL	5.819 ms [5.76 ms, 5.877 ms]	4.614 ms (382.9%)
iast_GLOBAL	3.635 ms [3.582 ms, 3.688 ms]	2.43 ms (201.7%)
profiling	2.035 ms [2.016 ms, 2.053 ms]	829.884 µs (68.9%)
tracing	1.813 ms [1.798 ms, 1.828 ms]	607.802 µs (50.4%)

candidate results

Variant	Request duration [CI 0.99]	Δ no_agent
no_agent	1.21 ms [1.198 ms, 1.222 ms]	-
iast	3.2 ms [3.158 ms, 3.242 ms]	1.99 ms (164.5%)
iast_FULL	5.745 ms [5.688 ms, 5.802 ms]	4.535 ms (374.8%)
iast_GLOBAL	3.7 ms [3.636 ms, 3.764 ms]	2.49 ms (205.8%)
profiling	2.08 ms [2.061 ms, 2.099 ms]	870.197 µs (71.9%)
tracing	1.874 ms [1.857 ms, 1.89 ms]	663.751 µs (54.9%)

Dacapo

Parameters

	Baseline	Candidate
baseline_or_candidate	baseline	candidate
git_branch	master	daniel.mohedano/testopt-smoke-disable-git
git_commit_date	1763974460	1764068421
git_commit_sha	`c79dfc6`	`aa62b32`
release_version	1.57.0-SNAPSHOT~c79dfc656d	1.57.0-SNAPSHOT~aa62b32a6a

See matching parameters

	Baseline	Candidate
application	biojava	biojava
ci_job_date	1764070493	1764070493
ci_job_id	1255500842	1255500842
ci_pipeline_id	83921622	83921622
cpu_model	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
kernel_version	Linux runner-zfyrx7zua-project-304-concurrent-0-36xli1ud 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux	Linux runner-zfyrx7zua-project-304-concurrent-0-36xli1ud 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 12 metrics, 0 unstable metrics.

Execution time for biojava

gantt
    title biojava - execution time [CI 0.99] : candidate=1.57.0-SNAPSHOT~aa62b32a6a, baseline=1.57.0-SNAPSHOT~c79dfc656d
    dateFormat X
    axisFormat %s
section baseline
no_agent (15.059 s) : 15059000, 15059000
.   : milestone, 15059000,
appsec (14.475 s) : 14475000, 14475000
.   : milestone, 14475000,
iast (18.319 s) : 18319000, 18319000
.   : milestone, 18319000,
iast_GLOBAL (18.004 s) : 18004000, 18004000
.   : milestone, 18004000,
profiling (15.232 s) : 15232000, 15232000
.   : milestone, 15232000,
tracing (14.674 s) : 14674000, 14674000
.   : milestone, 14674000,
section candidate
no_agent (14.821 s) : 14821000, 14821000
.   : milestone, 14821000,
appsec (14.541 s) : 14541000, 14541000
.   : milestone, 14541000,
iast (18.722 s) : 18722000, 18722000
.   : milestone, 18722000,
iast_GLOBAL (18.054 s) : 18054000, 18054000
.   : milestone, 18054000,
profiling (14.824 s) : 14824000, 14824000
.   : milestone, 14824000,
tracing (14.812 s) : 14812000, 14812000
.   : milestone, 14812000,

baseline results

Variant	Execution Time [CI 0.99]	Δ no_agent
no_agent	15.059 s [15.059 s, 15.059 s]	-
appsec	14.475 s [14.475 s, 14.475 s]	-584.0 ms (-3.9%)
iast	18.319 s [18.319 s, 18.319 s]	3.26 s (21.6%)
iast_GLOBAL	18.004 s [18.004 s, 18.004 s]	2.945 s (19.6%)
profiling	15.232 s [15.232 s, 15.232 s]	173.0 ms (1.1%)
tracing	14.674 s [14.674 s, 14.674 s]	-385.0 ms (-2.6%)

candidate results

Variant	Execution Time [CI 0.99]	Δ no_agent
no_agent	14.821 s [14.821 s, 14.821 s]	-
appsec	14.541 s [14.541 s, 14.541 s]	-280.0 ms (-1.9%)
iast	18.722 s [18.722 s, 18.722 s]	3.901 s (26.3%)
iast_GLOBAL	18.054 s [18.054 s, 18.054 s]	3.233 s (21.8%)
profiling	14.824 s [14.824 s, 14.824 s]	3.0 ms (0.0%)
tracing	14.812 s [14.812 s, 14.812 s]	-9.0 ms (-0.1%)

Execution time for tomcat

gantt
    title tomcat - execution time [CI 0.99] : candidate=1.57.0-SNAPSHOT~aa62b32a6a, baseline=1.57.0-SNAPSHOT~c79dfc656d
    dateFormat X
    axisFormat %s
section baseline
no_agent (1.469 ms) : 1457, 1480
.   : milestone, 1469,
appsec (2.457 ms) : 2405, 2509
.   : milestone, 2457,
iast (2.203 ms) : 2138, 2267
.   : milestone, 2203,
iast_GLOBAL (2.251 ms) : 2186, 2315
.   : milestone, 2251,
profiling (2.049 ms) : 1997, 2101
.   : milestone, 2049,
tracing (2.039 ms) : 1988, 2089
.   : milestone, 2039,
section candidate
no_agent (1.469 ms) : 1457, 1480
.   : milestone, 1469,
appsec (2.442 ms) : 2390, 2493
.   : milestone, 2442,
iast (2.198 ms) : 2134, 2262
.   : milestone, 2198,
iast_GLOBAL (2.242 ms) : 2177, 2307
.   : milestone, 2242,
profiling (2.062 ms) : 2010, 2114
.   : milestone, 2062,
tracing (2.045 ms) : 1994, 2097
.   : milestone, 2045,

baseline results

Variant	Execution Time [CI 0.99]	Δ no_agent
no_agent	1.469 ms [1.457 ms, 1.48 ms]	-
appsec	2.457 ms [2.405 ms, 2.509 ms]	988.326 µs (67.3%)
iast	2.203 ms [2.138 ms, 2.267 ms]	733.649 µs (49.9%)
iast_GLOBAL	2.251 ms [2.186 ms, 2.315 ms]	781.742 µs (53.2%)
profiling	2.049 ms [1.997 ms, 2.101 ms]	580.15 µs (39.5%)
tracing	2.039 ms [1.988 ms, 2.089 ms]	570.023 µs (38.8%)

candidate results

Variant	Execution Time [CI 0.99]	Δ no_agent
no_agent	1.469 ms [1.457 ms, 1.48 ms]	-
appsec	2.442 ms [2.39 ms, 2.493 ms]	972.801 µs (66.2%)
iast	2.198 ms [2.134 ms, 2.262 ms]	729.537 µs (49.7%)
iast_GLOBAL	2.242 ms [2.177 ms, 2.307 ms]	773.184 µs (52.6%)
profiling	2.062 ms [2.01 ms, 2.114 ms]	593.428 µs (40.4%)
tracing	2.045 ms [1.994 ms, 2.097 ms]	576.662 µs (39.3%)

feat: clean CI logs and disable unused configurations

664da59

daniel-mohedano changed the title ~~feat: clean CI logs and disable unused configurations~~ Improvements to Test Optimization smoke tests Nov 25, 2025

daniel-mohedano added comp: ci visibility Continuous Integration Visibility type: enhancement Enhancements and improvements tag: no release notes Changes to exclude from release notes labels Nov 25, 2025

daniel-mohedano changed the title ~~Improvements to Test Optimization smoke tests~~ Improve Test Optimization's smoke tests clarity Nov 25, 2025

daniel-mohedano marked this pull request as ready for review November 25, 2025 10:44

daniel-mohedano requested a review from a team as a code owner November 25, 2025 10:44

daniel-mohedano requested a review from nikita-tkachenko-datadog November 25, 2025 10:44

nikita-tkachenko-datadog approved these changes Nov 25, 2025

View reviewed changes

Comment thread dd-smoke-tests/gradle/build.gradle Outdated

fix: rename comment

aa62b32

Co-authored-by: Nikita Tkachenko <121111529+nikita-tkachenko-datadog@users.noreply.github.com>

daniel-mohedano merged commit 9dde14c into master Nov 25, 2025
537 checks passed

daniel-mohedano deleted the daniel.mohedano/testopt-smoke-disable-git branch November 25, 2025 13:17

github-actions Bot added this to the 1.57.0 milestone Nov 25, 2025

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Improve Test Optimization's smoke tests clarity#10028

Improve Test Optimization's smoke tests clarity#10028
daniel-mohedano merged 2 commits into
masterfrom
daniel.mohedano/testopt-smoke-disable-git

daniel-mohedano commented Nov 25, 2025 •

edited

Loading

Uh oh!

Uh oh!

cit-pr-commenter Bot commented Nov 25, 2025

Uh oh!

cit-pr-commenter Bot commented Nov 25, 2025

Uh oh!

cit-pr-commenter Bot commented Nov 25, 2025

Uh oh!

cit-pr-commenter Bot commented Nov 25, 2025

Uh oh!

cit-pr-commenter Bot commented Nov 25, 2025

Uh oh!

cit-pr-commenter Bot commented Nov 25, 2025

Uh oh!

cit-pr-commenter Bot commented Nov 25, 2025

Uh oh!

cit-pr-commenter Bot commented Nov 25, 2025

Uh oh!

pr-commenter Bot commented Nov 25, 2025

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Conversation

daniel-mohedano commented Nov 25, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What Does This Do

Motivation

Contributor Checklist

Uh oh!

Uh oh!

cit-pr-commenter Bot commented Nov 25, 2025

Test Environment - sbt-scalatest

Uh oh!

cit-pr-commenter Bot commented Nov 25, 2025

Test Environment - nebula-release-plugin

Uh oh!

cit-pr-commenter Bot commented Nov 25, 2025

Test Environment - pass4s

Uh oh!

cit-pr-commenter Bot commented Nov 25, 2025

Test Environment - reactive-streams-jvm

Uh oh!

cit-pr-commenter Bot commented Nov 25, 2025

Test Environment - netflix-zuul

Uh oh!

cit-pr-commenter Bot commented Nov 25, 2025

Test Environment - sonar-kotlin

Uh oh!

cit-pr-commenter Bot commented Nov 25, 2025

Test Environment - jolokia

Uh oh!

cit-pr-commenter Bot commented Nov 25, 2025

Test Environment - okhttp

Uh oh!

pr-commenter Bot commented Nov 25, 2025

Benchmarks

Startup

Parameters

Summary

Load

Parameters

Summary

Dacapo

Parameters

Summary

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

daniel-mohedano commented Nov 25, 2025 •

edited

Loading