Skip to content

Fix race condition Exception Replay smoke tests#10011

Merged
jpbempel merged 1 commit into
masterfrom
jpbempel/fix-exception-replay-flakiness
Nov 25, 2025
Merged

Fix race condition Exception Replay smoke tests#10011
jpbempel merged 1 commit into
masterfrom
jpbempel/fix-exception-replay-flakiness

Conversation

@jpbempel
Copy link
Copy Markdown
Member

@jpbempel jpbempel commented Nov 21, 2025

What Does This Do

need to wait for having added the fingerprint into the Concurrent Map otherwise it will try to reinstrument the exception on the second call Generalize the usage of NonRetryable annotation because all debugger smoke tests rely on the fact that they are run only once (log file naming)

Motivation

Additional Notes

Contributor Checklist

Jira ticket: [PROJ-IDENT]

@jpbempel
Fix race condition Exception Replay smoke tests
b8761e4 
need to wait for having added the fingerprint into the Concurrent Map
otherwise it will try to reinstrument the exception on the second call
Generalize the usage of NonRetryable annotation because all debugger
smoke tests rely on the fact that they are run only once (log file
naming)
@jpbempel jpbempel requested a review from a team as a code owner November 21, 2025 10:10
@jpbempel jpbempel requested review from tylfin and removed request for a team November 21, 2025 10:10
@github-actions
Copy link
Copy Markdown
Contributor

github-actions Bot commented Nov 21, 2025

Hi! 👋 Thanks for your pull request! 🎉

To help us review it, please make sure to:

  • Add at least one type, and one component or instrumentation label to the pull request

If you need help, please check our contributing guidelines.

@jpbempel jpbempel added comp: debugger Dynamic Instrumentation tag: no release notes Changes to exclude from release notes type: bug Bug report and fix labels Nov 21, 2025
@pr-commenter
Copy link
Copy Markdown

pr-commenter Bot commented Nov 21, 2025

Debugger benchmarks

Parameters

Baseline Candidate
baseline_or_candidate baseline candidate
ci_job_date 1763720383 1763720718
end_time 2025-11-21T10:21:05 2025-11-21T10:26:39
git_branch master jpbempel/fix-exception-replay-flakiness
git_commit_sha 6b32bdd b8761e4
start_time 2025-11-21T10:19:44 2025-11-21T10:25:19
See matching parameters
Baseline Candidate
ci_job_id 1250027527 1250027527
ci_pipeline_id 83548474 83548474
cpu_model Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
git_commit_date 1763719813 1763719813

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 8 metrics, 7 unstable metrics.

See unchanged results
scenario Δ mean agg_http_req_duration_min Δ mean agg_http_req_duration_p50 Δ mean agg_http_req_duration_p75 Δ mean agg_http_req_duration_p99 Δ mean throughput
scenario:noprobe unstable
[-26919.276ns; +28679.541ns] or [-9.435%; +10.052%]		 unstable
[-41044.506ns; +41008.039ns] or [-12.559%; +12.548%]		 unstable
[-50.859µs; +58.050µs] or [-14.918%; +17.028%]		 unstable
[-196.635µs; +242.245µs] or [-19.385%; +23.882%]		 same
scenario:basic same same same unstable
[-160.550µs; +37.372µs] or [-19.121%; +4.451%]		 unstable
[-137.573op/s; +137.573op/s] or [-5.365%; +5.365%]
scenario:loop same same same unstable
[-581.340µs; +807.971µs] or [-6.603%; +9.177%]		 same
Request duration reports for reports 
gantt
    title reports - request duration [CI 0.99] : candidate=None, baseline=None
    dateFormat X
    axisFormat %s
section baseline
noprobe (326.805 µs) : 293, 361
.   : milestone, 327,
basic (292.338 µs) : 285, 300
.   : milestone, 292,
loop (8.44 ms) : 8378, 8502
.   : milestone, 8440,
section candidate
noprobe (326.787 µs) : 283, 370
.   : milestone, 327,
basic (292.368 µs) : 285, 300
.   : milestone, 292,
loop (8.447 ms) : 8385, 8509
.   : milestone, 8447,
Loading
  • baseline results
Scenario Request median duration [CI 0.99]
noprobe 326.805 µs [293.013 µs, 360.597 µs]
basic 292.338 µs [284.846 µs, 299.829 µs]
loop 8.44 ms [8.378 ms, 8.502 ms]
  • candidate results
Scenario Request median duration [CI 0.99]
noprobe 326.787 µs [283.105 µs, 370.469 µs]
basic 292.368 µs [285.11 µs, 299.625 µs]
loop 8.447 ms [8.385 ms, 8.509 ms]

@pr-commenter
Copy link
Copy Markdown

pr-commenter Bot commented Nov 21, 2025

Benchmarks

Startup

Parameters

Baseline Candidate
baseline_or_candidate baseline candidate
git_branch master jpbempel/fix-exception-replay-flakiness
git_commit_date 1763710237 1763719813
git_commit_sha 6b32bdd b8761e4
release_version 1.57.0-SNAPSHOT~6b32bddf07 1.57.0-SNAPSHOT~b8761e406b
See matching parameters
Baseline Candidate
application insecure-bank insecure-bank
ci_job_date 1763721610 1763721610
ci_job_id 1250027520 1250027520
ci_pipeline_id 83548474 83548474
cpu_model Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
kernel_version Linux runner-zfyrx7zua-project-304-concurrent-0-dgg0eimb 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux Linux runner-zfyrx7zua-project-304-concurrent-0-dgg0eimb 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux
module Agent Agent
parent None None

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 60 metrics, 5 unstable metrics.

Startup time reports for petclinic 
gantt
    title petclinic - global startup overhead: candidate=1.57.0-SNAPSHOT~b8761e406b, baseline=1.57.0-SNAPSHOT~6b32bddf07

    dateFormat X
    axisFormat %s
section tracing
Agent [baseline] (1.111 s) : 0, 1111307
Total [baseline] (10.847 s) : 0, 10847370
Agent [candidate] (1.109 s) : 0, 1108886
Total [candidate] (10.81 s) : 0, 10810363
section appsec
Agent [baseline] (1.287 s) : 0, 1286773
Total [baseline] (11.121 s) : 0, 11120793
Agent [candidate] (1.279 s) : 0, 1278569
Total [candidate] (11.115 s) : 0, 11114765
section iast
Agent [baseline] (1.245 s) : 0, 1245331
Total [baseline] (11.319 s) : 0, 11319246
Agent [candidate] (1.24 s) : 0, 1240201
Total [candidate] (11.277 s) : 0, 11277035
section profiling
Agent [baseline] (1.233 s) : 0, 1232886
Total [baseline] (11.057 s) : 0, 11056703
Agent [candidate] (1.232 s) : 0, 1231674
Total [candidate] (11.128 s) : 0, 11127996
Loading
  • baseline results
Module Variant Duration Δ tracing
Agent tracing 1.111 s -
Agent appsec 1.287 s 175.466 ms (15.8%)
Agent iast 1.245 s 134.024 ms (12.1%)
Agent profiling 1.233 s 121.579 ms (10.9%)
Total tracing 10.847 s -
Total appsec 11.121 s 273.423 ms (2.5%)
Total iast 11.319 s 471.876 ms (4.4%)
Total profiling 11.057 s 209.333 ms (1.9%)
  • candidate results
Module Variant Duration Δ tracing
Agent tracing 1.109 s -
Agent appsec 1.279 s 169.682 ms (15.3%)
Agent iast 1.24 s 131.315 ms (11.8%)
Agent profiling 1.232 s 122.788 ms (11.1%)
Total tracing 10.81 s -
Total appsec 11.115 s 304.402 ms (2.8%)
Total iast 11.277 s 466.672 ms (4.3%)
Total profiling 11.128 s 317.633 ms (2.9%) 
gantt
    title petclinic - break down per module: candidate=1.57.0-SNAPSHOT~b8761e406b, baseline=1.57.0-SNAPSHOT~6b32bddf07

    dateFormat X
    axisFormat %s
section tracing
crashtracking [baseline] (1.488 ms) : 0, 1488
crashtracking [candidate] (1.481 ms) : 0, 1481
BytebuddyAgent [baseline] (713.76 ms) : 0, 713760
BytebuddyAgent [candidate] (712.653 ms) : 0, 712653
GlobalTracer [baseline] (251.636 ms) : 0, 251636
GlobalTracer [candidate] (250.479 ms) : 0, 250479
AppSec [baseline] (32.214 ms) : 0, 32214
AppSec [candidate] (32.252 ms) : 0, 32252
Debugger [baseline] (64.546 ms) : 0, 64546
Debugger [candidate] (64.272 ms) : 0, 64272
Remote Config [baseline] (633.657 µs) : 0, 634
Remote Config [candidate] (636.431 µs) : 0, 636
Telemetry [baseline] (8.355 ms) : 0, 8355
Telemetry [candidate] (8.419 ms) : 0, 8419
Flare Poller [baseline] (3.782 ms) : 0, 3782
Flare Poller [candidate] (3.787 ms) : 0, 3787
section appsec
crashtracking [baseline] (1.486 ms) : 0, 1486
crashtracking [candidate] (1.468 ms) : 0, 1468
BytebuddyAgent [baseline] (734.421 ms) : 0, 734421
BytebuddyAgent [candidate] (730.767 ms) : 0, 730767
GlobalTracer [baseline] (241.687 ms) : 0, 241687
GlobalTracer [candidate] (239.978 ms) : 0, 239978
AppSec [baseline] (174.535 ms) : 0, 174535
AppSec [candidate] (173.178 ms) : 0, 173178
Debugger [baseline] (61.936 ms) : 0, 61936
Debugger [candidate] (61.08 ms) : 0, 61080
Remote Config [baseline] (670.231 µs) : 0, 670
Remote Config [candidate] (655.18 µs) : 0, 655
Telemetry [baseline] (8.224 ms) : 0, 8224
Telemetry [candidate] (8.063 ms) : 0, 8063
Flare Poller [baseline] (3.886 ms) : 0, 3886
Flare Poller [candidate] (3.878 ms) : 0, 3878
IAST [baseline] (25.034 ms) : 0, 25034
IAST [candidate] (24.614 ms) : 0, 24614
section iast
crashtracking [baseline] (1.485 ms) : 0, 1485
crashtracking [candidate] (1.483 ms) : 0, 1483
BytebuddyAgent [baseline] (834.931 ms) : 0, 834931
BytebuddyAgent [candidate] (832.018 ms) : 0, 832018
GlobalTracer [baseline] (238.791 ms) : 0, 238791
GlobalTracer [candidate] (237.673 ms) : 0, 237673
AppSec [baseline] (33.459 ms) : 0, 33459
AppSec [candidate] (33.167 ms) : 0, 33167
Debugger [baseline] (61.403 ms) : 0, 61403
Debugger [candidate] (60.877 ms) : 0, 60877
Remote Config [baseline] (557.451 µs) : 0, 557
Remote Config [candidate] (541.168 µs) : 0, 541
Telemetry [baseline] (7.708 ms) : 0, 7708
Telemetry [candidate] (7.595 ms) : 0, 7595
Flare Poller [baseline] (3.551 ms) : 0, 3551
Flare Poller [candidate] (3.498 ms) : 0, 3498
IAST [baseline] (28.601 ms) : 0, 28601
IAST [candidate] (28.427 ms) : 0, 28427
section profiling
ProfilingAgent [baseline] (97.195 ms) : 0, 97195
ProfilingAgent [candidate] (96.91 ms) : 0, 96910
crashtracking [baseline] (1.426 ms) : 0, 1426
crashtracking [candidate] (1.426 ms) : 0, 1426
BytebuddyAgent [baseline] (734.811 ms) : 0, 734811
BytebuddyAgent [candidate] (734.444 ms) : 0, 734444
GlobalTracer [baseline] (221.994 ms) : 0, 221994
GlobalTracer [candidate] (222.126 ms) : 0, 222126
AppSec [baseline] (32.142 ms) : 0, 32142
AppSec [candidate] (32.249 ms) : 0, 32249
Debugger [baseline] (63.574 ms) : 0, 63574
Debugger [candidate] (63.076 ms) : 0, 63076
Remote Config [baseline] (656.156 µs) : 0, 656
Remote Config [candidate] (666.84 µs) : 0, 667
Telemetry [baseline] (8.133 ms) : 0, 8133
Telemetry [candidate] (8.02 ms) : 0, 8020
Flare Poller [baseline] (3.782 ms) : 0, 3782
Flare Poller [candidate] (3.707 ms) : 0, 3707
Profiling [baseline] (97.786 ms) : 0, 97786
Profiling [candidate] (97.508 ms) : 0, 97508
Loading
Startup time reports for insecure-bank 
gantt
    title insecure-bank - global startup overhead: candidate=1.57.0-SNAPSHOT~b8761e406b, baseline=1.57.0-SNAPSHOT~6b32bddf07

    dateFormat X
    axisFormat %s
section tracing
Agent [baseline] (1.105 s) : 0, 1105344
Total [baseline] (8.847 s) : 0, 8846924
Agent [candidate] (1.1 s) : 0, 1099625
Total [candidate] (8.826 s) : 0, 8825851
section iast
Agent [baseline] (1.239 s) : 0, 1238946
Total [baseline] (9.534 s) : 0, 9534437
Agent [candidate] (1.249 s) : 0, 1249494
Total [candidate] (9.61 s) : 0, 9609821
Loading
  • baseline results
Module Variant Duration Δ tracing
Agent tracing 1.105 s -
Agent iast 1.239 s 133.602 ms (12.1%)
Total tracing 8.847 s -
Total iast 9.534 s 687.513 ms (7.8%)
  • candidate results
Module Variant Duration Δ tracing
Agent tracing 1.1 s -
Agent iast 1.249 s 149.869 ms (13.6%)
Total tracing 8.826 s -
Total iast 9.61 s 783.97 ms (8.9%) 
gantt
    title insecure-bank - break down per module: candidate=1.57.0-SNAPSHOT~b8761e406b, baseline=1.57.0-SNAPSHOT~6b32bddf07

    dateFormat X
    axisFormat %s
section tracing
crashtracking [baseline] (1.482 ms) : 0, 1482
crashtracking [candidate] (1.475 ms) : 0, 1475
BytebuddyAgent [baseline] (710.508 ms) : 0, 710508
BytebuddyAgent [candidate] (706.632 ms) : 0, 706632
GlobalTracer [baseline] (249.368 ms) : 0, 249368
GlobalTracer [candidate] (249.076 ms) : 0, 249076
AppSec [baseline] (32.245 ms) : 0, 32245
AppSec [candidate] (32.011 ms) : 0, 32011
Debugger [baseline] (64.036 ms) : 0, 64036
Debugger [candidate] (62.985 ms) : 0, 62985
Remote Config [baseline] (645.132 µs) : 0, 645
Remote Config [candidate] (633.099 µs) : 0, 633
Telemetry [baseline] (8.387 ms) : 0, 8387
Telemetry [candidate] (8.358 ms) : 0, 8358
Flare Poller [baseline] (3.801 ms) : 0, 3801
Flare Poller [candidate] (3.692 ms) : 0, 3692
section iast
crashtracking [baseline] (1.482 ms) : 0, 1482
crashtracking [candidate] (1.482 ms) : 0, 1482
BytebuddyAgent [baseline] (831.478 ms) : 0, 831478
BytebuddyAgent [candidate] (839.314 ms) : 0, 839314
GlobalTracer [baseline] (237.405 ms) : 0, 237405
GlobalTracer [candidate] (239.529 ms) : 0, 239529
IAST [baseline] (28.428 ms) : 0, 28428
IAST [candidate] (28.862 ms) : 0, 28862
AppSec [baseline] (33.206 ms) : 0, 33206
AppSec [candidate] (33.178 ms) : 0, 33178
Debugger [baseline] (60.345 ms) : 0, 60345
Debugger [candidate] (60.436 ms) : 0, 60436
Remote Config [baseline] (539.284 µs) : 0, 539
Remote Config [candidate] (545.587 µs) : 0, 546
Telemetry [baseline] (7.639 ms) : 0, 7639
Telemetry [candidate] (7.66 ms) : 0, 7660
Flare Poller [baseline] (3.547 ms) : 0, 3547
Flare Poller [candidate] (3.487 ms) : 0, 3487
Loading

Load

Parameters

Baseline Candidate
baseline_or_candidate baseline candidate
git_branch master jpbempel/fix-exception-replay-flakiness
git_commit_date 1763710237 1763719813
git_commit_sha 6b32bdd b8761e4
release_version 1.57.0-SNAPSHOT~6b32bddf07 1.57.0-SNAPSHOT~b8761e406b
See matching parameters
Baseline Candidate
application insecure-bank insecure-bank
ci_job_date 1763722104 1763722104
ci_job_id 1250027521 1250027521
ci_pipeline_id 83548474 83548474
cpu_model Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
kernel_version Linux runner-zfyrx7zua-project-304-concurrent-0-9yahaihx 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux Linux runner-zfyrx7zua-project-304-concurrent-0-9yahaihx 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux

Summary

Found 7 performance improvements and 0 performance regressions! Performance is the same for 13 metrics, 16 unstable metrics.

scenario Δ mean agg_http_req_duration_p50 Δ mean agg_http_req_duration_p95 Δ mean throughput candidate mean agg_http_req_duration_p50 candidate mean agg_http_req_duration_p95 candidate mean throughput baseline mean agg_http_req_duration_p50 baseline mean agg_http_req_duration_p95 baseline mean throughput
scenario:load:insecure-bank:iast:high_load better
[-269.779µs; -142.656µs] or [-10.287%; -5.440%]		 unstable
[-3.074ms; +7.560ms] or [-40.294%; +99.089%]		 unstable
[-69.464op/s; +259.776op/s] or [-5.108%; +19.102%]		 2.416ms 9.872ms 1455.094op/s 2.623ms 7.629ms 1359.938op/s
scenario:load:insecure-bank:iast_FULL:high_load better
[-502.674µs; -274.334µs] or [-9.025%; -4.925%]		 better
[-1265.003µs; -535.187µs] or [-9.630%; -4.074%]		 unstable
[-31.444op/s; +125.569op/s] or [-4.217%; +16.839%]		 5.182ms 12.236ms 792.750op/s 5.570ms 13.136ms 745.688op/s
scenario:load:petclinic:appsec:high_load better
[-1383.738µs; -473.805µs] or [-7.274%; -2.491%]		 better
[-2.165ms; -1.041ms] or [-6.988%; -3.359%]		 unstable
[-14.135op/s; +41.697op/s] or [-5.912%; +17.440%]		 18.094ms 29.377ms 252.875op/s 19.023ms 30.980ms 239.094op/s
scenario:load:petclinic:no_agent:high_load better
[-2.954ms; -1.524ms] or [-15.946%; -8.225%]		 better
[-5.327ms; -2.335ms] or [-16.873%; -7.397%]		 unstable
[+4.653op/s; +65.035op/s] or [+1.926%; +26.916%]		 16.287ms 27.738ms 276.469op/s 18.526ms 31.569ms 241.625op/s
Request duration reports for insecure-bank 
gantt
    title insecure-bank - request duration [CI 0.99] : candidate=1.57.0-SNAPSHOT~b8761e406b, baseline=1.57.0-SNAPSHOT~6b32bddf07
    dateFormat X
    axisFormat %s
section baseline
no_agent (1.198 ms) : 1186, 1209
.   : milestone, 1198,
iast (3.367 ms) : 3318, 3417
.   : milestone, 3367,
iast_FULL (6.208 ms) : 6144, 6271
.   : milestone, 6208,
iast_GLOBAL (3.562 ms) : 3506, 3618
.   : milestone, 3562,
profiling (2.262 ms) : 2242, 2282
.   : milestone, 2262,
tracing (1.837 ms) : 1822, 1853
.   : milestone, 1837,
section candidate
no_agent (1.219 ms) : 1207, 1230
.   : milestone, 1219,
iast (3.143 ms) : 3098, 3189
.   : milestone, 3143,
iast_FULL (5.831 ms) : 5772, 5889
.   : milestone, 5831,
iast_GLOBAL (3.699 ms) : 3637, 3761
.   : milestone, 3699,
profiling (2.365 ms) : 2341, 2390
.   : milestone, 2365,
tracing (1.914 ms) : 1896, 1932
.   : milestone, 1914,
Loading
  • baseline results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 1.198 ms [1.186 ms, 1.209 ms] -
iast 3.367 ms [3.318 ms, 3.417 ms] 2.17 ms (181.1%)
iast_FULL 6.208 ms [6.144 ms, 6.271 ms] 5.01 ms (418.3%)
iast_GLOBAL 3.562 ms [3.506 ms, 3.618 ms] 2.364 ms (197.4%)
profiling 2.262 ms [2.242 ms, 2.282 ms] 1.064 ms (88.9%)
tracing 1.837 ms [1.822 ms, 1.853 ms] 639.562 µs (53.4%)
  • candidate results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 1.219 ms [1.207 ms, 1.23 ms] -
iast 3.143 ms [3.098 ms, 3.189 ms] 1.925 ms (157.9%)
iast_FULL 5.831 ms [5.772 ms, 5.889 ms] 4.612 ms (378.5%)
iast_GLOBAL 3.699 ms [3.637 ms, 3.761 ms] 2.48 ms (203.5%)
profiling 2.365 ms [2.341 ms, 2.39 ms] 1.147 ms (94.1%)
tracing 1.914 ms [1.896 ms, 1.932 ms] 695.745 µs (57.1%)
Request duration reports for petclinic 
gantt
    title petclinic - request duration [CI 0.99] : candidate=1.57.0-SNAPSHOT~b8761e406b, baseline=1.57.0-SNAPSHOT~6b32bddf07
    dateFormat X
    axisFormat %s
section baseline
no_agent (19.322 ms) : 19121, 19524
.   : milestone, 19322,
appsec (19.524 ms) : 19324, 19723
.   : milestone, 19524,
code_origins (17.629 ms) : 17451, 17807
.   : milestone, 17629,
iast (17.57 ms) : 17398, 17742
.   : milestone, 17570,
profiling (18.602 ms) : 18416, 18789
.   : milestone, 18602,
tracing (17.584 ms) : 17406, 17762
.   : milestone, 17584,
section candidate
no_agent (16.875 ms) : 16707, 17043
.   : milestone, 16875,
appsec (18.455 ms) : 18269, 18640
.   : milestone, 18455,
code_origins (17.829 ms) : 17648, 18010
.   : milestone, 17829,
iast (17.583 ms) : 17409, 17757
.   : milestone, 17583,
profiling (18.658 ms) : 18473, 18843
.   : milestone, 18658,
tracing (17.483 ms) : 17300, 17666
.   : milestone, 17483,
Loading
  • baseline results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 19.322 ms [19.121 ms, 19.524 ms] -
appsec 19.524 ms [19.324 ms, 19.723 ms] 201.223 µs (1.0%)
code_origins 17.629 ms [17.451 ms, 17.807 ms] -1.694 ms (-8.8%)
iast 17.57 ms [17.398 ms, 17.742 ms] -1.752 ms (-9.1%)
profiling 18.602 ms [18.416 ms, 18.789 ms] -720.316 µs (-3.7%)
tracing 17.584 ms [17.406 ms, 17.762 ms] -1.738 ms (-9.0%)
  • candidate results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 16.875 ms [16.707 ms, 17.043 ms] -
appsec 18.455 ms [18.269 ms, 18.64 ms] 1.579 ms (9.4%)
code_origins 17.829 ms [17.648 ms, 18.01 ms] 953.715 µs (5.7%)
iast 17.583 ms [17.409 ms, 17.757 ms] 707.606 µs (4.2%)
profiling 18.658 ms [18.473 ms, 18.843 ms] 1.783 ms (10.6%)
tracing 17.483 ms [17.3 ms, 17.666 ms] 608.051 µs (3.6%)

Dacapo

Parameters

Baseline Candidate
baseline_or_candidate baseline candidate
git_branch master jpbempel/fix-exception-replay-flakiness
git_commit_date 1763710237 1763719813
git_commit_sha 6b32bdd b8761e4
release_version 1.57.0-SNAPSHOT~6b32bddf07 1.57.0-SNAPSHOT~b8761e406b
See matching parameters
Baseline Candidate
application biojava biojava
ci_job_date 1763721884 1763721884
ci_job_id 1250027522 1250027522
ci_pipeline_id 83548474 83548474
cpu_model Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
kernel_version Linux runner-zfyrx7zua-project-304-concurrent-1-wpo5wt79 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux Linux runner-zfyrx7zua-project-304-concurrent-1-wpo5wt79 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 11 metrics, 1 unstable metrics.

Execution time for tomcat 
gantt
    title tomcat - execution time [CI 0.99] : candidate=1.57.0-SNAPSHOT~b8761e406b, baseline=1.57.0-SNAPSHOT~6b32bddf07
    dateFormat X
    axisFormat %s
section baseline
no_agent (1.483 ms) : 1471, 1494
.   : milestone, 1483,
appsec (3.725 ms) : 3505, 3945
.   : milestone, 3725,
iast (2.225 ms) : 2160, 2290
.   : milestone, 2225,
iast_GLOBAL (2.261 ms) : 2196, 2326
.   : milestone, 2261,
profiling (2.066 ms) : 2013, 2118
.   : milestone, 2066,
tracing (2.053 ms) : 2002, 2104
.   : milestone, 2053,
section candidate
no_agent (1.477 ms) : 1465, 1488
.   : milestone, 1477,
appsec (3.708 ms) : 3490, 3927
.   : milestone, 3708,
iast (2.215 ms) : 2151, 2280
.   : milestone, 2215,
iast_GLOBAL (2.269 ms) : 2204, 2334
.   : milestone, 2269,
profiling (2.073 ms) : 2021, 2125
.   : milestone, 2073,
tracing (2.046 ms) : 1995, 2097
.   : milestone, 2046,
Loading
  • baseline results
Variant Execution Time [CI 0.99] Δ no_agent
no_agent 1.483 ms [1.471 ms, 1.494 ms] -
appsec 3.725 ms [3.505 ms, 3.945 ms] 2.242 ms (151.2%)
iast 2.225 ms [2.16 ms, 2.29 ms] 742.295 µs (50.1%)
iast_GLOBAL 2.261 ms [2.196 ms, 2.326 ms] 778.349 µs (52.5%)
profiling 2.066 ms [2.013 ms, 2.118 ms] 582.996 µs (39.3%)
tracing 2.053 ms [2.002 ms, 2.104 ms] 570.382 µs (38.5%)
  • candidate results
Variant Execution Time [CI 0.99] Δ no_agent
no_agent 1.477 ms [1.465 ms, 1.488 ms] -
appsec 3.708 ms [3.49 ms, 3.927 ms] 2.231 ms (151.1%)
iast 2.215 ms [2.151 ms, 2.28 ms] 738.393 µs (50.0%)
iast_GLOBAL 2.269 ms [2.204 ms, 2.334 ms] 791.772 µs (53.6%)
profiling 2.073 ms [2.021 ms, 2.125 ms] 596.075 µs (40.4%)
tracing 2.046 ms [1.995 ms, 2.097 ms] 569.181 µs (38.5%)
Execution time for biojava 
gantt
    title biojava - execution time [CI 0.99] : candidate=1.57.0-SNAPSHOT~b8761e406b, baseline=1.57.0-SNAPSHOT~6b32bddf07
    dateFormat X
    axisFormat %s
section baseline
no_agent (14.947 s) : 14947000, 14947000
.   : milestone, 14947000,
appsec (14.825 s) : 14825000, 14825000
.   : milestone, 14825000,
iast (18.284 s) : 18284000, 18284000
.   : milestone, 18284000,
iast_GLOBAL (17.745 s) : 17745000, 17745000
.   : milestone, 17745000,
profiling (14.704 s) : 14704000, 14704000
.   : milestone, 14704000,
tracing (14.962 s) : 14962000, 14962000
.   : milestone, 14962000,
section candidate
no_agent (15.476 s) : 15476000, 15476000
.   : milestone, 15476000,
appsec (14.979 s) : 14979000, 14979000
.   : milestone, 14979000,
iast (18.708 s) : 18708000, 18708000
.   : milestone, 18708000,
iast_GLOBAL (17.895 s) : 17895000, 17895000
.   : milestone, 17895000,
profiling (14.756 s) : 14756000, 14756000
.   : milestone, 14756000,
tracing (14.806 s) : 14806000, 14806000
.   : milestone, 14806000,
Loading
  • baseline results
Variant Execution Time [CI 0.99] Δ no_agent
no_agent 14.947 s [14.947 s, 14.947 s] -
appsec 14.825 s [14.825 s, 14.825 s] -122.0 ms (-0.8%)
iast 18.284 s [18.284 s, 18.284 s] 3.337 s (22.3%)
iast_GLOBAL 17.745 s [17.745 s, 17.745 s] 2.798 s (18.7%)
profiling 14.704 s [14.704 s, 14.704 s] -243.0 ms (-1.6%)
tracing 14.962 s [14.962 s, 14.962 s] 15.0 ms (0.1%)
  • candidate results
Variant Execution Time [CI 0.99] Δ no_agent
no_agent 15.476 s [15.476 s, 15.476 s] -
appsec 14.979 s [14.979 s, 14.979 s] -497.0 ms (-3.2%)
iast 18.708 s [18.708 s, 18.708 s] 3.232 s (20.9%)
iast_GLOBAL 17.895 s [17.895 s, 17.895 s] 2.419 s (15.6%)
profiling 14.756 s [14.756 s, 14.756 s] -720.0 ms (-4.7%)
tracing 14.806 s [14.806 s, 14.806 s] -670.0 ms (-4.3%)

bric3
bric3 approved these changes Nov 24, 2025
View reviewed changes
Copy link
Copy Markdown
Contributor

@bric3 bric3 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good, but another pair of eyes working on this part of the codebase could be useful

@jpbempel jpbempel merged commit e5347b6 into master Nov 25, 2025
547 of 552 checks passed
@jpbempel jpbempel deleted the jpbempel/fix-exception-replay-flakiness branch November 25, 2025 10:03
@github-actions github-actions Bot added this to the 1.57.0 milestone Nov 25, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@bric3 bric3 bric3 approved these changes

@tylfin tylfin tylfin approved these changes

Assignees

No one assigned

Labels

comp: debugger Dynamic Instrumentation tag: no release notes Changes to exclude from release notes type: bug Bug report and fix

Projects

None yet

Milestone

1.57.0

Development

Successfully merging this pull request may close these issues.

3 participants

@jpbempel @bric3 @tylfin