Skip to content

Configuration changes to support triage mode#6262

Merged
mcculls merged 1 commit into
masterfrom
mcculls/triage-mode
Nov 30, 2023
Merged

Configuration changes to support triage mode#6262
mcculls merged 1 commit into
masterfrom
mcculls/triage-mode

Conversation

@mcculls
Copy link
Copy Markdown
Contributor

@mcculls mcculls commented Nov 22, 2023 •
edited by atlassian Bot
Loading

What Does This Do

Adds a triage mode to static and dynamic config. Explicitly enabling full debug implicitly enables triage mode.

Motivation

Triage mode will enable lightweight checks to help triage common issues without having to enable full debug mode.

Usage

JVM option:

-Ddd.trace.triage=true

Environment variable:

DD_TRACE_TRIAGE=true

Jira ticket: APMJAVA-1134

@mcculls mcculls added the tag: diagnostics Diagnostics related changes label Nov 22, 2023
@mcculls mcculls force-pushed the mcculls/triage-mode branch 2 times, most recently from 45c51da to ca8059e Compare November 22, 2023 12:06
@pr-commenter
Copy link
Copy Markdown

pr-commenter Bot commented Nov 22, 2023 •
edited
Loading

Benchmarks

Startup

Parameters

Baseline Candidate
baseline_or_candidate baseline candidate
git_branch master mcculls/triage-mode
git_commit_date 1700671175 1700684191
git_commit_sha 46fcb7c 987256f
release_version 1.25.0-SNAPSHOT~46fcb7cb58 1.25.0-SNAPSHOT~987256f79d
See matching parameters
Baseline Candidate
application insecure-bank insecure-bank
ci_job_date 1700686901 1700686901
ci_job_id 375388214 375388214
ci_pipeline_id 23742299 23742299
cpu_model Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
module Agent Agent
parent None None
variant iast iast

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 47 metrics, 7 unstable metrics.

Startup time reports for insecure-bank 
gantt
    title insecure-bank - global startup overhead: candidate=1.25.0-SNAPSHOT~987256f79d, baseline=1.25.0-SNAPSHOT~46fcb7cb58

    dateFormat X
    axisFormat %s
section tracing
Agent [baseline] (1.036 s) : 0, 1035953
Total [baseline] (8.703 s) : 0, 8702569
Agent [candidate] (1.034 s) : 0, 1034485
Total [candidate] (8.681 s) : 0, 8681404
section iast
Agent [baseline] (1.158 s) : 0, 1157866
Total [baseline] (9.261 s) : 0, 9261104
Agent [candidate] (1.149 s) : 0, 1149374
Total [candidate] (9.217 s) : 0, 9217374
section iast_TELEMETRY_OFF
Agent [baseline] (1.139 s) : 0, 1139304
Total [baseline] (9.225 s) : 0, 9225480
Agent [candidate] (1.138 s) : 0, 1138235
Total [candidate] (9.218 s) : 0, 9217900
Loading
  • baseline results
Module Variant Duration Δ tracing
Agent tracing 1.036 s -
Agent iast 1.158 s 121.913 ms (11.8%)
Agent iast_TELEMETRY_OFF 1.139 s 103.351 ms (10.0%)
Total tracing 8.703 s -
Total iast 9.261 s 558.535 ms (6.4%)
Total iast_TELEMETRY_OFF 9.225 s 522.91 ms (6.0%)
  • candidate results
Module Variant Duration Δ tracing
Agent tracing 1.034 s -
Agent iast 1.149 s 114.889 ms (11.1%)
Agent iast_TELEMETRY_OFF 1.138 s 103.75 ms (10.0%)
Total tracing 8.681 s -
Total iast 9.217 s 535.97 ms (6.2%)
Total iast_TELEMETRY_OFF 9.218 s 536.496 ms (6.2%) 
gantt
    title insecure-bank - break down per module: candidate=1.25.0-SNAPSHOT~987256f79d, baseline=1.25.0-SNAPSHOT~46fcb7cb58

    dateFormat X
    axisFormat %s
section tracing
BytebuddyAgent [baseline] (646.609 ms) : 0, 646609
BytebuddyAgent [candidate] (645.529 ms) : 0, 645529
GlobalTracer [baseline] (298.72 ms) : 0, 298720
GlobalTracer [candidate] (298.458 ms) : 0, 298458
AppSec [baseline] (48.281 ms) : 0, 48281
AppSec [candidate] (48.201 ms) : 0, 48201
Remote Config [baseline] (685.971 µs) : 0, 686
Remote Config [candidate] (680.7 µs) : 0, 681
Telemetry [baseline] (7.319 ms) : 0, 7319
Telemetry [candidate] (7.349 ms) : 0, 7349
section iast
BytebuddyAgent [baseline] (768.782 ms) : 0, 768782
BytebuddyAgent [candidate] (763.127 ms) : 0, 763127
GlobalTracer [baseline] (279.188 ms) : 0, 279188
GlobalTracer [candidate] (278.166 ms) : 0, 278166
AppSec [baseline] (49.904 ms) : 0, 49904
AppSec [candidate] (50.532 ms) : 0, 50532
IAST [baseline] (16.992 ms) : 0, 16992
IAST [candidate] (13.98 ms) : 0, 13980
Remote Config [baseline] (573.95 µs) : 0, 574
Remote Config [candidate] (569.247 µs) : 0, 569
Telemetry [baseline] (7.916 ms) : 0, 7916
Telemetry [candidate] (8.72 ms) : 0, 8720
section iast_TELEMETRY_OFF
BytebuddyAgent [baseline] (754.824 ms) : 0, 754824
BytebuddyAgent [candidate] (755.199 ms) : 0, 755199
GlobalTracer [baseline] (277.803 ms) : 0, 277803
GlobalTracer [candidate] (278.161 ms) : 0, 278161
AppSec [baseline] (47.018 ms) : 0, 47018
AppSec [candidate] (46.499 ms) : 0, 46499
IAST [baseline] (18.63 ms) : 0, 18630
IAST [candidate] (16.565 ms) : 0, 16565
Remote Config [baseline] (553.691 µs) : 0, 554
Remote Config [candidate] (560.359 µs) : 0, 560
Telemetry [baseline] (6.322 ms) : 0, 6322
Telemetry [candidate] (7.122 ms) : 0, 7122
Loading
Startup time reports for petclinic 
gantt
    title petclinic - global startup overhead: candidate=1.25.0-SNAPSHOT~987256f79d, baseline=1.25.0-SNAPSHOT~46fcb7cb58

    dateFormat X
    axisFormat %s
section tracing
Agent [baseline] (1.029 s) : 0, 1029056
Total [baseline] (9.266 s) : 0, 9265563
Agent [candidate] (1.03 s) : 0, 1029924
Total [candidate] (9.331 s) : 0, 9331021
section appsec
Agent [baseline] (1.133 s) : 0, 1133415
Total [baseline] (9.453 s) : 0, 9452504
Agent [candidate] (1.12 s) : 0, 1120355
Total [candidate] (9.398 s) : 0, 9398323
section iast
Agent [baseline] (1.144 s) : 0, 1143677
Total [baseline] (9.479 s) : 0, 9479307
Agent [candidate] (1.158 s) : 0, 1157994
Total [candidate] (9.566 s) : 0, 9565504
section profiling
Agent [baseline] (1.219 s) : 0, 1219189
Total [baseline] (9.58 s) : 0, 9580214
Agent [candidate] (1.22 s) : 0, 1219976
Total [candidate] (9.63 s) : 0, 9630295
Loading
  • baseline results
Module Variant Duration Δ tracing
Agent tracing 1.029 s -
Agent appsec 1.133 s 104.359 ms (10.1%)
Agent iast 1.144 s 114.622 ms (11.1%)
Agent profiling 1.219 s 190.133 ms (18.5%)
Total tracing 9.266 s -
Total appsec 9.453 s 186.941 ms (2.0%)
Total iast 9.479 s 213.744 ms (2.3%)
Total profiling 9.58 s 314.651 ms (3.4%)
  • candidate results
Module Variant Duration Δ tracing
Agent tracing 1.03 s -
Agent appsec 1.12 s 90.431 ms (8.8%)
Agent iast 1.158 s 128.07 ms (12.4%)
Agent profiling 1.22 s 190.052 ms (18.5%)
Total tracing 9.331 s -
Total appsec 9.398 s 67.302 ms (0.7%)
Total iast 9.566 s 234.482 ms (2.5%)
Total profiling 9.63 s 299.274 ms (3.2%) 
gantt
    title petclinic - break down per module: candidate=1.25.0-SNAPSHOT~987256f79d, baseline=1.25.0-SNAPSHOT~46fcb7cb58

    dateFormat X
    axisFormat %s
section tracing
BytebuddyAgent [baseline] (641.895 ms) : 0, 641895
BytebuddyAgent [candidate] (642.451 ms) : 0, 642451
GlobalTracer [baseline] (296.944 ms) : 0, 296944
GlobalTracer [candidate] (297.387 ms) : 0, 297387
AppSec [baseline] (48.0 ms) : 0, 48000
AppSec [candidate] (48.115 ms) : 0, 48115
Remote Config [baseline] (674.175 µs) : 0, 674
Remote Config [candidate] (673.676 µs) : 0, 674
Telemetry [baseline] (7.311 ms) : 0, 7311
Telemetry [candidate] (7.24 ms) : 0, 7240
section appsec
BytebuddyAgent [baseline] (652.167 ms) : 0, 652167
BytebuddyAgent [candidate] (644.368 ms) : 0, 644368
GlobalTracer [baseline] (301.274 ms) : 0, 301274
GlobalTracer [candidate] (297.971 ms) : 0, 297971
AppSec [baseline] (137.643 ms) : 0, 137643
AppSec [candidate] (136.338 ms) : 0, 136338
Remote Config [baseline] (665.017 µs) : 0, 665
Remote Config [candidate] (649.177 µs) : 0, 649
Telemetry [baseline] (6.924 ms) : 0, 6924
Telemetry [candidate] (6.745 ms) : 0, 6745
section iast
BytebuddyAgent [baseline] (760.354 ms) : 0, 760354
BytebuddyAgent [candidate] (768.832 ms) : 0, 768832
GlobalTracer [baseline] (276.39 ms) : 0, 276390
GlobalTracer [candidate] (279.873 ms) : 0, 279873
AppSec [baseline] (48.463 ms) : 0, 48463
AppSec [candidate] (51.798 ms) : 0, 51798
Remote Config [baseline] (564.335 µs) : 0, 564
Remote Config [candidate] (576.177 µs) : 0, 576
Telemetry [baseline] (9.949 ms) : 0, 9949
Telemetry [candidate] (8.547 ms) : 0, 8547
IAST [baseline] (13.899 ms) : 0, 13899
IAST [candidate] (13.944 ms) : 0, 13944
section profiling
ProfilingAgent [baseline] (88.434 ms) : 0, 88434
ProfilingAgent [candidate] (87.913 ms) : 0, 87913
BytebuddyAgent [baseline] (653.223 ms) : 0, 653223
BytebuddyAgent [candidate] (654.12 ms) : 0, 654120
GlobalTracer [baseline] (366.918 ms) : 0, 366918
GlobalTracer [candidate] (367.44 ms) : 0, 367440
AppSec [baseline] (48.652 ms) : 0, 48652
AppSec [candidate] (48.619 ms) : 0, 48619
Remote Config [baseline] (694.315 µs) : 0, 694
Remote Config [candidate] (695.969 µs) : 0, 696
Telemetry [baseline] (7.373 ms) : 0, 7373
Telemetry [candidate] (7.356 ms) : 0, 7356
Profiling [baseline] (88.457 ms) : 0, 88457
Profiling [candidate] (87.936 ms) : 0, 87936
Loading

Load

Parameters

Baseline Candidate
baseline_or_candidate baseline candidate
end_time 2023-11-22T20:40:55 2023-11-22T20:57:27
git_branch master mcculls/triage-mode
git_commit_date 1700671175 1700684191
git_commit_sha 46fcb7c 987256f
release_version 1.25.0-SNAPSHOT~46fcb7cb58 1.25.0-SNAPSHOT~987256f79d
start_time 2023-11-22T20:40:42 2023-11-22T20:57:14
See matching parameters
Baseline Candidate
application insecure-bank insecure-bank
ci_job_date 1700686901 1700686901
ci_job_id 375388214 375388214
ci_pipeline_id 23742299 23742299
cpu_model Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
variant iast iast

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 9 metrics, 13 unstable metrics.

Request duration reports for insecure-bank 
gantt
    title insecure-bank - request duration [CI 0.99] : candidate=1.25.0-SNAPSHOT~987256f79d, baseline=1.25.0-SNAPSHOT~46fcb7cb58
    dateFormat X
    axisFormat %s
section baseline
no_agent (377.139 µs) : 356, 398
.   : milestone, 377,
iast (491.445 µs) : 471, 512
.   : milestone, 491,
iast_FULL (537.606 µs) : 517, 558
.   : milestone, 538,
iast_INACTIVE (451.644 µs) : 430, 473
.   : milestone, 452,
iast_TELEMETRY_OFF (474.388 µs) : 454, 495
.   : milestone, 474,
tracing (440.49 µs) : 420, 461
.   : milestone, 440,
section candidate
no_agent (371.887 µs) : 351, 393
.   : milestone, 372,
iast (479.158 µs) : 459, 500
.   : milestone, 479,
iast_FULL (533.872 µs) : 513, 554
.   : milestone, 534,
iast_INACTIVE (441.844 µs) : 421, 462
.   : milestone, 442,
iast_TELEMETRY_OFF (463.736 µs) : 443, 484
.   : milestone, 464,
tracing (446.028 µs) : 426, 466
.   : milestone, 446,
Loading
  • baseline results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 377.139 µs [356.017 µs, 398.26 µs] -
iast 491.445 µs [470.848 µs, 512.041 µs] 114.306 µs (30.3%)
iast_FULL 537.606 µs [517.322 µs, 557.89 µs] 160.467 µs (42.5%)
iast_INACTIVE 451.644 µs [430.433 µs, 472.855 µs] 74.505 µs (19.8%)
iast_TELEMETRY_OFF 474.388 µs [453.504 µs, 495.273 µs] 97.25 µs (25.8%)
tracing 440.49 µs [419.866 µs, 461.114 µs] 63.352 µs (16.8%)
  • candidate results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 371.887 µs [351.188 µs, 392.585 µs] -
iast 479.158 µs [458.706 µs, 499.611 µs] 107.272 µs (28.8%)
iast_FULL 533.872 µs [513.454 µs, 554.29 µs] 161.985 µs (43.6%)
iast_INACTIVE 441.844 µs [421.374 µs, 462.314 µs] 69.958 µs (18.8%)
iast_TELEMETRY_OFF 463.736 µs [443.327 µs, 484.146 µs] 91.85 µs (24.7%)
tracing 446.028 µs [425.687 µs, 466.368 µs] 74.141 µs (19.9%)
Request duration reports for petclinic 
gantt
    title petclinic - request duration [CI 0.99] : candidate=1.25.0-SNAPSHOT~987256f79d, baseline=1.25.0-SNAPSHOT~46fcb7cb58
    dateFormat X
    axisFormat %s
section baseline
no_agent (1.354 ms) : 1335, 1373
.   : milestone, 1354,
appsec (1.769 ms) : 1744, 1794
.   : milestone, 1769,
iast (1.534 ms) : 1510, 1559
.   : milestone, 1534,
profiling (1.508 ms) : 1483, 1533
.   : milestone, 1508,
tracing (1.52 ms) : 1495, 1544
.   : milestone, 1520,
section candidate
no_agent (1.339 ms) : 1320, 1358
.   : milestone, 1339,
appsec (1.769 ms) : 1744, 1794
.   : milestone, 1769,
iast (1.545 ms) : 1521, 1570
.   : milestone, 1545,
profiling (1.562 ms) : 1536, 1589
.   : milestone, 1562,
tracing (1.514 ms) : 1489, 1539
.   : milestone, 1514,
Loading
  • baseline results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 1.354 ms [1.335 ms, 1.373 ms] -
appsec 1.769 ms [1.744 ms, 1.794 ms] 415.398 µs (30.7%)
iast 1.534 ms [1.51 ms, 1.559 ms] 180.554 µs (13.3%)
profiling 1.508 ms [1.483 ms, 1.533 ms] 153.946 µs (11.4%)
tracing 1.52 ms [1.495 ms, 1.544 ms] 165.665 µs (12.2%)
  • candidate results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 1.339 ms [1.32 ms, 1.358 ms] -
appsec 1.769 ms [1.744 ms, 1.794 ms] 429.89 µs (32.1%)
iast 1.545 ms [1.521 ms, 1.57 ms] 206.44 µs (15.4%)
profiling 1.562 ms [1.536 ms, 1.589 ms] 223.439 µs (16.7%)
tracing 1.514 ms [1.489 ms, 1.539 ms] 175.272 µs (13.1%)

@mcculls mcculls force-pushed the mcculls/triage-mode branch from ca8059e to 987256f Compare November 22, 2023 20:16
@mcculls mcculls marked this pull request as ready for review November 22, 2023 20:16
@mcculls mcculls requested a review from a team as a code owner November 22, 2023 20:16
PerfectSlayer
PerfectSlayer approved these changes Nov 30, 2023
View reviewed changes
Copy link
Copy Markdown
Contributor

@PerfectSlayer PerfectSlayer left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM! Do we consider publicly documenting it?

@mcculls mcculls merged commit b449d63 into master Nov 30, 2023
@mcculls mcculls deleted the mcculls/triage-mode branch November 30, 2023 11:19
@github-actions github-actions Bot added this to the 1.25.0 milestone Nov 30, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@PerfectSlayer PerfectSlayer PerfectSlayer approved these changes

@ygree ygree Awaiting requested review from ygree ygree is a code owner automatically assigned from DataDog/apm-java

@am312 am312 Awaiting requested review from am312 am312 is a code owner automatically assigned from DataDog/apm-java

Assignees

No one assigned

Labels

tag: diagnostics Diagnostics related changes

Projects

None yet

Milestone

1.25.0

Development

Successfully merging this pull request may close these issues.

2 participants

@mcculls @PerfectSlayer