Skip to content

Fix CI triggers with dynamic config#5773

Merged
smola merged 1 commit into
masterfrom
smola/fix-triggers
Aug 24, 2023
Merged

Fix CI triggers with dynamic config#5773
smola merged 1 commit into
masterfrom
smola/fix-triggers

Conversation

@smola
Copy link
Copy Markdown
Member

@smola smola commented Aug 24, 2023 •
edited
Loading

What Does This Do

Regular triggers do not play well with dynamic config in Circle CI. We need to use pipeline triggers at:
https://app.circleci.com/settings/project/github/DataDog/dd-trace-java/triggers

Motivation

Additional Notes

  • Con: triggers no longer specified in code.
  • Pro: we can now use the "Trigger Pipeline" button in Circle CI to manually trigger nightlies or weeklies.

image

@smola smola added comp: testing Testing tag: no release notes Changes to exclude from release notes labels Aug 24, 2023
@smola smola requested a review from a team as a code owner August 24, 2023 07:10
@smola smola requested a review from bantonsson August 24, 2023 07:10
@smola smola force-pushed the smola/fix-triggers branch from 9290743 to ffa3340 Compare August 24, 2023 07:30
@smola
Fix CI triggers with dynamic config
c555940 
Regular triggers do not play well with dynamic config in Circle CI. We
need to use pipeline triggers at:
https://app.circleci.com/settings/project/github/DataDog/dd-trace-java/triggers
@smola smola force-pushed the smola/fix-triggers branch from ffa3340 to c555940 Compare August 24, 2023 07:34
@pr-commenter
Copy link
Copy Markdown

pr-commenter Bot commented Aug 24, 2023 •
edited
Loading

Benchmarks

Startup

Parameters

Baseline Candidate
commit 1.20.0-SNAPSHOT~65849d8869 1.20.0-SNAPSHOT~c5559400b1
config baseline candidate
See matching parameters
Baseline Candidate
application insecure-bank insecure-bank
module Agent Agent
parent None None
variant appsec appsec

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 62 cases.

Startup time reports for insecure-bank 
gantt
    title insecure-bank - global startup overhead: candidate=1.20.0-SNAPSHOT~c5559400b1, baseline=1.20.0-SNAPSHOT~65849d8869

    dateFormat X
    axisFormat %s
section tracing
Agent [baseline] (991.76 ms) : 0, 991760
Total [baseline] (8.635 s) : 0, 8634812
Agent [candidate] (995.689 ms) : 0, 995689
Total [candidate] (8.636 s) : 0, 8635570
section appsec
Agent [baseline] (1.07 s) : 0, 1070405
Total [baseline] (8.685 s) : 0, 8684552
Agent [candidate] (1.076 s) : 0, 1075689
Total [candidate] (8.749 s) : 0, 8748974
section iast
Agent [baseline] (1.09 s) : 0, 1089770
Total [baseline] (9.165 s) : 0, 9165422
Agent [candidate] (1.089 s) : 0, 1089226
Total [candidate] (9.159 s) : 0, 9158983
section profiling
Agent [baseline] (1.157 s) : 0, 1157115
Total [baseline] (8.904 s) : 0, 8904120
Agent [candidate] (1.159 s) : 0, 1159103
Total [candidate] (8.868 s) : 0, 8868133
Loading
  • baseline results
Module Variant Duration Δ tracing
Agent tracing 991.76 ms -
Agent appsec 1.07 s 78.645 ms (7.9%)
Agent iast 1.09 s 98.011 ms (9.9%)
Agent profiling 1.157 s 165.356 ms (16.7%)
Total tracing 8.635 s -
Total appsec 8.685 s 49.74 ms (0.6%)
Total iast 9.165 s 530.61 ms (6.1%)
Total profiling 8.904 s 269.308 ms (3.1%)
  • candidate results
Module Variant Duration Δ tracing
Agent tracing 995.689 ms -
Agent appsec 1.076 s 80.0 ms (8.0%)
Agent iast 1.089 s 93.537 ms (9.4%)
Agent profiling 1.159 s 163.414 ms (16.4%)
Total tracing 8.636 s -
Total appsec 8.749 s 113.404 ms (1.3%)
Total iast 9.159 s 523.414 ms (6.1%)
Total profiling 8.868 s 232.564 ms (2.7%) 
gantt
    title insecure-bank - break down per module: candidate=1.20.0-SNAPSHOT~c5559400b1, baseline=1.20.0-SNAPSHOT~65849d8869

    dateFormat X
    axisFormat %s
section tracing
BytebuddyAgent [baseline] (615.106 ms) : 0, 615106
BytebuddyAgent [candidate] (617.484 ms) : 0, 617484
GlobalTracer [baseline] (285.145 ms) : 0, 285145
GlobalTracer [candidate] (286.747 ms) : 0, 286747
AppSec [baseline] (50.222 ms) : 0, 50222
AppSec [candidate] (50.206 ms) : 0, 50206
Remote Config [baseline] (671.059 µs) : 0, 671
Remote Config [candidate] (668.773 µs) : 0, 669
Telemetry [baseline] (6.059 ms) : 0, 6059
Telemetry [candidate] (6.039 ms) : 0, 6039
section appsec
BytebuddyAgent [baseline] (611.174 ms) : 0, 611174
BytebuddyAgent [candidate] (612.733 ms) : 0, 612733
GlobalTracer [baseline] (283.252 ms) : 0, 283252
GlobalTracer [candidate] (286.451 ms) : 0, 286451
AppSec [baseline] (135.288 ms) : 0, 135288
AppSec [candidate] (135.776 ms) : 0, 135776
Remote Config [baseline] (723.116 µs) : 0, 723
Remote Config [candidate] (717.495 µs) : 0, 717
Telemetry [baseline] (5.734 ms) : 0, 5734
Telemetry [candidate] (5.767 ms) : 0, 5767
section iast
BytebuddyAgent [baseline] (718.21 ms) : 0, 718210
BytebuddyAgent [candidate] (716.998 ms) : 0, 716998
GlobalTracer [baseline] (269.954 ms) : 0, 269954
GlobalTracer [candidate] (270.546 ms) : 0, 270546
AppSec [baseline] (47.241 ms) : 0, 47241
AppSec [candidate] (47.385 ms) : 0, 47385
Remote Config [baseline] (548.358 µs) : 0, 548
Remote Config [candidate] (539.346 µs) : 0, 539
Telemetry [baseline] (5.724 ms) : 0, 5724
Telemetry [candidate] (5.627 ms) : 0, 5627
IAST [baseline] (13.858 ms) : 0, 13858
IAST [candidate] (13.811 ms) : 0, 13811
section profiling
BytebuddyAgent [baseline] (624.254 ms) : 0, 624254
BytebuddyAgent [candidate] (626.733 ms) : 0, 626733
GlobalTracer [baseline] (346.417 ms) : 0, 346417
GlobalTracer [candidate] (347.045 ms) : 0, 347045
AppSec [baseline] (49.277 ms) : 0, 49277
AppSec [candidate] (49.134 ms) : 0, 49134
Remote Config [baseline] (649.092 µs) : 0, 649
Remote Config [candidate] (648.812 µs) : 0, 649
Telemetry [baseline] (6.029 ms) : 0, 6029
Telemetry [candidate] (5.986 ms) : 0, 5986
ProfilingAgent [baseline] (77.684 ms) : 0, 77684
ProfilingAgent [candidate] (76.642 ms) : 0, 76642
Profiling [baseline] (77.708 ms) : 0, 77708
Profiling [candidate] (76.666 ms) : 0, 76666
Loading
Startup time reports for petclinic 
gantt
    title petclinic - global startup overhead: candidate=1.20.0-SNAPSHOT~c5559400b1, baseline=1.20.0-SNAPSHOT~65849d8869

    dateFormat X
    axisFormat %s
section tracing
Agent [baseline] (994.415 ms) : 0, 994415
Total [baseline] (9.213 s) : 0, 9213486
Agent [candidate] (989.164 ms) : 0, 989164
Total [candidate] (9.194 s) : 0, 9193523
section appsec
Agent [baseline] (1.073 s) : 0, 1073480
Total [baseline] (9.272 s) : 0, 9271857
Agent [candidate] (1.079 s) : 0, 1079056
Total [candidate] (9.308 s) : 0, 9308458
section iast
Agent [baseline] (1.106 s) : 0, 1105513
Total [baseline] (9.434 s) : 0, 9433514
Agent [candidate] (1.096 s) : 0, 1096131
Total [candidate] (9.494 s) : 0, 9494498
section profiling
Agent [baseline] (1.168 s) : 0, 1167529
Total [baseline] (9.442 s) : 0, 9441803
Agent [candidate] (1.156 s) : 0, 1155790
Total [candidate] (9.491 s) : 0, 9490946
Loading
  • baseline results
Module Variant Duration Δ tracing
Agent tracing 994.415 ms -
Agent appsec 1.073 s 79.066 ms (8.0%)
Agent iast 1.106 s 111.098 ms (11.2%)
Agent profiling 1.168 s 173.115 ms (17.4%)
Total tracing 9.213 s -
Total appsec 9.272 s 58.371 ms (0.6%)
Total iast 9.434 s 220.028 ms (2.4%)
Total profiling 9.442 s 228.317 ms (2.5%)
  • candidate results
Module Variant Duration Δ tracing
Agent tracing 989.164 ms -
Agent appsec 1.079 s 89.892 ms (9.1%)
Agent iast 1.096 s 106.967 ms (10.8%)
Agent profiling 1.156 s 166.626 ms (16.8%)
Total tracing 9.194 s -
Total appsec 9.308 s 114.936 ms (1.3%)
Total iast 9.494 s 300.975 ms (3.3%)
Total profiling 9.491 s 297.424 ms (3.2%) 
gantt
    title petclinic - break down per module: candidate=1.20.0-SNAPSHOT~c5559400b1, baseline=1.20.0-SNAPSHOT~65849d8869

    dateFormat X
    axisFormat %s
section tracing
BytebuddyAgent [baseline] (617.086 ms) : 0, 617086
BytebuddyAgent [candidate] (613.847 ms) : 0, 613847
GlobalTracer [baseline] (286.025 ms) : 0, 286025
GlobalTracer [candidate] (284.424 ms) : 0, 284424
AppSec [baseline] (50.03 ms) : 0, 50030
AppSec [candidate] (49.722 ms) : 0, 49722
Remote Config [baseline] (670.656 µs) : 0, 671
Remote Config [candidate] (664.327 µs) : 0, 664
Telemetry [baseline] (6.049 ms) : 0, 6049
Telemetry [candidate] (5.994 ms) : 0, 5994
section appsec
BytebuddyAgent [baseline] (612.979 ms) : 0, 612979
BytebuddyAgent [candidate] (616.038 ms) : 0, 616038
GlobalTracer [baseline] (284.433 ms) : 0, 284433
GlobalTracer [candidate] (286.435 ms) : 0, 286435
AppSec [baseline] (135.171 ms) : 0, 135171
AppSec [candidate] (135.651 ms) : 0, 135651
Remote Config [baseline] (723.252 µs) : 0, 723
Remote Config [candidate] (733.713 µs) : 0, 734
Telemetry [baseline] (5.773 ms) : 0, 5773
Telemetry [candidate] (5.781 ms) : 0, 5781
section iast
BytebuddyAgent [baseline] (729.173 ms) : 0, 729173
BytebuddyAgent [candidate] (722.531 ms) : 0, 722531
GlobalTracer [baseline] (273.3 ms) : 0, 273300
GlobalTracer [candidate] (271.55 ms) : 0, 271550
AppSec [baseline] (47.776 ms) : 0, 47776
AppSec [candidate] (47.506 ms) : 0, 47506
Remote Config [baseline] (561.084 µs) : 0, 561
Remote Config [candidate] (543.436 µs) : 0, 543
Telemetry [baseline] (5.794 ms) : 0, 5794
Telemetry [candidate] (5.699 ms) : 0, 5699
IAST [baseline] (14.11 ms) : 0, 14110
IAST [candidate] (13.866 ms) : 0, 13866
section profiling
BytebuddyAgent [baseline] (632.004 ms) : 0, 632004
BytebuddyAgent [candidate] (624.422 ms) : 0, 624422
GlobalTracer [baseline] (349.138 ms) : 0, 349138
GlobalTracer [candidate] (345.909 ms) : 0, 345909
AppSec [baseline] (49.383 ms) : 0, 49383
AppSec [candidate] (49.199 ms) : 0, 49199
Remote Config [baseline] (652.883 µs) : 0, 653
Remote Config [candidate] (643.678 µs) : 0, 644
Telemetry [baseline] (6.004 ms) : 0, 6004
Telemetry [candidate] (5.945 ms) : 0, 5945
ProfilingAgent [baseline] (76.824 ms) : 0, 76824
ProfilingAgent [candidate] (76.781 ms) : 0, 76781
Profiling [baseline] (76.847 ms) : 0, 76847
Profiling [candidate] (76.805 ms) : 0, 76805
Loading

Load

Parameters

Baseline Candidate
commit 1.20.0-SNAPSHOT~65849d8869 1.20.0-SNAPSHOT~c5559400b1
config baseline candidate
end_time 2023-08-24T07:58:03 2023-08-24T08:16:09
start_time 2023-08-24T07:57:46 2023-08-24T08:15:51
See matching parameters
Baseline Candidate
application insecure-bank insecure-bank
variant appsec appsec

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 24 cases.

Request duration reports for insecure-bank 
gantt
    title insecure-bank - request duration [CI 0.99] : candidate=1.20.0-SNAPSHOT~c5559400b1, baseline=1.20.0-SNAPSHOT~65849d8869
    dateFormat X
    axisFormat %s
section baseline
no_agent (362.118 µs) : 341, 383
.   : milestone, 362,
appsec (663.23 µs) : 641, 685
.   : milestone, 663,
iast (448.155 µs) : 428, 469
.   : milestone, 448,
iast_FULL (515.864 µs) : 495, 537
.   : milestone, 516,
iast_INACTIVE (429.181 µs) : 408, 450
.   : milestone, 429,
profiling (426.824 µs) : 407, 447
.   : milestone, 427,
tracing (428.648 µs) : 408, 450
.   : milestone, 429,
section candidate
no_agent (361.272 µs) : 341, 382
.   : milestone, 361,
appsec (664.289 µs) : 643, 686
.   : milestone, 664,
iast (450.429 µs) : 429, 471
.   : milestone, 450,
iast_FULL (513.94 µs) : 493, 535
.   : milestone, 514,
iast_INACTIVE (426.929 µs) : 405, 448
.   : milestone, 427,
profiling (438.991 µs) : 418, 460
.   : milestone, 439,
tracing (443.172 µs) : 422, 464
.   : milestone, 443,
Loading
  • baseline results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 362.118 µs [341.492 µs, 382.743 µs] -
appsec 663.23 µs [641.496 µs, 684.965 µs] 301.112 µs (83.2%)
iast 448.155 µs [427.625 µs, 468.685 µs] 86.038 µs (23.8%)
iast_FULL 515.864 µs [495.1 µs, 536.629 µs] 153.747 µs (42.5%)
iast_INACTIVE 429.181 µs [407.998 µs, 450.364 µs] 67.063 µs (18.5%)
profiling 426.824 µs [406.65 µs, 446.998 µs] 64.706 µs (17.9%)
tracing 428.648 µs [407.691 µs, 449.606 µs] 66.53 µs (18.4%)
  • candidate results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 361.272 µs [340.885 µs, 381.658 µs] -
appsec 664.289 µs [642.908 µs, 685.67 µs] 303.017 µs (83.9%)
iast 450.429 µs [429.393 µs, 471.465 µs] 89.157 µs (24.7%)
iast_FULL 513.94 µs [493.288 µs, 534.592 µs] 152.669 µs (42.3%)
iast_INACTIVE 426.929 µs [405.416 µs, 448.441 µs] 65.657 µs (18.2%)
profiling 438.991 µs [417.683 µs, 460.298 µs] 77.719 µs (21.5%)
tracing 443.172 µs [421.858 µs, 464.485 µs] 81.9 µs (22.7%)
Request duration reports for petclinic 
gantt
    title petclinic - request duration [CI 0.99] : candidate=1.20.0-SNAPSHOT~c5559400b1, baseline=1.20.0-SNAPSHOT~65849d8869
    dateFormat X
    axisFormat %s
section baseline
no_agent (1.343 ms) : 1324, 1362
.   : milestone, 1343,
appsec (1.651 ms) : 1627, 1676
.   : milestone, 1651,
iast (1.451 ms) : 1428, 1475
.   : milestone, 1451,
profiling (1.469 ms) : 1445, 1493
.   : milestone, 1469,
tracing (1.438 ms) : 1413, 1462
.   : milestone, 1438,
section candidate
no_agent (1.316 ms) : 1297, 1335
.   : milestone, 1316,
appsec (1.677 ms) : 1652, 1701
.   : milestone, 1677,
iast (1.468 ms) : 1444, 1492
.   : milestone, 1468,
profiling (1.46 ms) : 1434, 1486
.   : milestone, 1460,
tracing (1.433 ms) : 1409, 1458
.   : milestone, 1433,
Loading
  • baseline results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 1.343 ms [1.324 ms, 1.362 ms] -
appsec 1.651 ms [1.627 ms, 1.676 ms] 308.899 µs (23.0%)
iast 1.451 ms [1.428 ms, 1.475 ms] 108.821 µs (8.1%)
profiling 1.469 ms [1.445 ms, 1.493 ms] 126.512 µs (9.4%)
tracing 1.438 ms [1.413 ms, 1.462 ms] 95.106 µs (7.1%)
  • candidate results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 1.316 ms [1.297 ms, 1.335 ms] -
appsec 1.677 ms [1.652 ms, 1.701 ms] 360.564 µs (27.4%)
iast 1.468 ms [1.444 ms, 1.492 ms] 151.445 µs (11.5%)
profiling 1.46 ms [1.434 ms, 1.486 ms] 143.978 µs (10.9%)
tracing 1.433 ms [1.409 ms, 1.458 ms] 117.205 µs (8.9%)

PerfectSlayer
PerfectSlayer reviewed Aug 24, 2023
View reviewed changes
Comment thread .circleci/config.continue.yml.j2
Comment on lines +35 to +40
nightly:
type: boolean
default: false
weekly:
type: boolean
default: false
Copy link
Copy Markdown
Contributor

@PerfectSlayer PerfectSlayer Aug 24, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There is an enum type parameters.
What about using it to encode a the trigger type? (push, nightly, weekly, maybe release also?)

Copy link
Copy Markdown
Member Author

@smola smola Aug 24, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not sure how to make push/release (not triggered by us) set such parameter in config.yml. Also note that these are only used in config.yml, but Circle CI will pass them to the generated pipeline too, and it will fail if they are not declared.

@smola smola merged commit bb95c7a into master Aug 24, 2023
@smola smola deleted the smola/fix-triggers branch August 24, 2023 13:34
@github-actions github-actions Bot added this to the 1.20.0 milestone Aug 24, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@PerfectSlayer PerfectSlayer PerfectSlayer left review comments

@bantonsson bantonsson bantonsson approved these changes

Assignees

No one assigned

Labels

comp: testing Testing tag: no release notes Changes to exclude from release notes

Projects

None yet

Milestone

1.20.0

Development

Successfully merging this pull request may close these issues.

3 participants

@smola @bantonsson @PerfectSlayer