Refactor of the PropagationModule and removal of WebModule by manuel-alvarez-alvarez · Pull Request #6033 · DataDog/dd-trace-java

manuel-alvarez-alvarez · 2023-10-11T19:57:00Z

What Does This Do

Refactoring of the tainting API by moving all tainting operations to the PropagationModule with methods according to:

taint methods are used to taint values : taint(context?, target, source_attributes+)
taintIfTainted methods are used to taint values only if the input is tainted : taintIfTainted(context?, target, input, source_attributes+)
taintIfAnyTainted methods are used to taint values only if any of the inputs is tainted : taintIfAnyTainted(context?, target, input[], source_attributes+)

Motivation

Both modules WebModule and PropagationModule have grown organically bringing a lot of duplication, this PR tries to merge all tainting operations providing a cohesive API.

Additional Notes

This PR is quite big but it does not introduce any new functionality, just a reorganization of the tainting API.

Jira ticket: APPSEC-10454

pr-commenter · 2023-10-11T20:38:28Z

Benchmarks

Startup

Parameters

	Baseline	Candidate
commit	1.23.0-SNAPSHOT~a012761e16	1.23.0-SNAPSHOT~072e68a521
config	baseline	candidate

See matching parameters

	Baseline	Candidate
application	insecure-bank	insecure-bank
module	Agent	Agent
parent	None	None
variant	iast	iast

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 54 cases.

Startup time reports for insecure-bank

gantt
    title insecure-bank - global startup overhead: candidate=1.23.0-SNAPSHOT~072e68a521, baseline=1.23.0-SNAPSHOT~a012761e16

    dateFormat X
    axisFormat %s
section tracing
Agent [baseline] (1.032 s) : 0, 1031748
Total [baseline] (8.759 s) : 0, 8759255
Agent [candidate] (1.027 s) : 0, 1026874
Total [candidate] (8.764 s) : 0, 8763927
section iast
Agent [baseline] (1.171 s) : 0, 1171310
Total [baseline] (9.349 s) : 0, 9348814
Agent [candidate] (1.155 s) : 0, 1154731
Total [candidate] (9.271 s) : 0, 9270759
section iast_TELEMETRY_OFF
Agent [baseline] (1.143 s) : 0, 1143493
Total [baseline] (9.312 s) : 0, 9312482
Agent [candidate] (1.147 s) : 0, 1147284
Total [candidate] (9.246 s) : 0, 9245937

baseline results

Module	Variant	Duration	Δ tracing
Agent	tracing	1.032 s	-
Agent	iast	1.171 s	139.562 ms (13.5%)
Agent	iast_TELEMETRY_OFF	1.143 s	111.745 ms (10.8%)
Total	tracing	8.759 s	-
Total	iast	9.349 s	589.559 ms (6.7%)
Total	iast_TELEMETRY_OFF	9.312 s	553.227 ms (6.3%)

candidate results

Module	Variant	Duration	Δ tracing
Agent	tracing	1.027 s	-
Agent	iast	1.155 s	127.857 ms (12.5%)
Agent	iast_TELEMETRY_OFF	1.147 s	120.41 ms (11.7%)
Total	tracing	8.764 s	-
Total	iast	9.271 s	506.832 ms (5.8%)
Total	iast_TELEMETRY_OFF	9.246 s	482.01 ms (5.5%)

gantt
    title insecure-bank - break down per module: candidate=1.23.0-SNAPSHOT~072e68a521, baseline=1.23.0-SNAPSHOT~a012761e16

    dateFormat X
    axisFormat %s
section tracing
BytebuddyAgent [baseline] (645.19 ms) : 0, 645190
BytebuddyAgent [candidate] (642.159 ms) : 0, 642159
GlobalTracer [baseline] (295.556 ms) : 0, 295556
GlobalTracer [candidate] (294.505 ms) : 0, 294505
AppSec [baseline] (49.647 ms) : 0, 49647
AppSec [candidate] (48.964 ms) : 0, 48964
Remote Config [baseline] (683.009 µs) : 0, 683
Remote Config [candidate] (677.767 µs) : 0, 678
Telemetry [baseline] (6.146 ms) : 0, 6146
Telemetry [candidate] (6.058 ms) : 0, 6058
section iast
BytebuddyAgent [baseline] (785.25 ms) : 0, 785250
BytebuddyAgent [candidate] (773.117 ms) : 0, 773117
GlobalTracer [baseline] (277.418 ms) : 0, 277418
GlobalTracer [candidate] (274.07 ms) : 0, 274070
AppSec [baseline] (48.892 ms) : 0, 48892
AppSec [candidate] (46.311 ms) : 0, 46311
Remote Config [baseline] (1.247 ms) : 0, 1247
Remote Config [candidate] (578.151 µs) : 0, 578
Telemetry [baseline] (8.238 ms) : 0, 8238
Telemetry [candidate] (11.272 ms) : 0, 11272
IAST [baseline] (15.406 ms) : 0, 15406
IAST [candidate] (14.954 ms) : 0, 14954
section iast_TELEMETRY_OFF
BytebuddyAgent [baseline] (763.725 ms) : 0, 763725
BytebuddyAgent [candidate] (766.779 ms) : 0, 766779
GlobalTracer [baseline] (273.667 ms) : 0, 273667
GlobalTracer [candidate] (274.258 ms) : 0, 274258
AppSec [baseline] (46.74 ms) : 0, 46740
AppSec [candidate] (46.287 ms) : 0, 46287
Remote Config [baseline] (582.785 µs) : 0, 583
Remote Config [candidate] (607.005 µs) : 0, 607
Telemetry [baseline] (6.796 ms) : 0, 6796
Telemetry [candidate] (9.01 ms) : 0, 9010
IAST [baseline] (17.649 ms) : 0, 17649
IAST [candidate] (15.876 ms) : 0, 15876

Startup time reports for petclinic

gantt
    title petclinic - global startup overhead: candidate=1.23.0-SNAPSHOT~072e68a521, baseline=1.23.0-SNAPSHOT~a012761e16

    dateFormat X
    axisFormat %s
section tracing
Agent [baseline] (1.027 s) : 0, 1027447
Total [baseline] (9.348 s) : 0, 9347892
Agent [candidate] (1.033 s) : 0, 1032660
Total [candidate] (9.326 s) : 0, 9326114
section appsec
Agent [baseline] (1.121 s) : 0, 1121300
Total [baseline] (9.389 s) : 0, 9388998
Agent [candidate] (1.114 s) : 0, 1114184
Total [candidate] (9.355 s) : 0, 9354766
section iast
Agent [baseline] (1.159 s) : 0, 1158952
Total [baseline] (9.517 s) : 0, 9516657
Agent [candidate] (1.152 s) : 0, 1151679
Total [candidate] (9.511 s) : 0, 9510660
section profiling
Agent [baseline] (1.209 s) : 0, 1208580
Total [baseline] (9.589 s) : 0, 9588982
Agent [candidate] (1.217 s) : 0, 1216876
Total [candidate] (9.576 s) : 0, 9575790

baseline results

Module	Variant	Duration	Δ tracing
Agent	tracing	1.027 s	-
Agent	appsec	1.121 s	93.854 ms (9.1%)
Agent	iast	1.159 s	131.505 ms (12.8%)
Agent	profiling	1.209 s	181.133 ms (17.6%)
Total	tracing	9.348 s	-
Total	appsec	9.389 s	41.107 ms (0.4%)
Total	iast	9.517 s	168.765 ms (1.8%)
Total	profiling	9.589 s	241.09 ms (2.6%)

candidate results

Module	Variant	Duration	Δ tracing
Agent	tracing	1.033 s	-
Agent	appsec	1.114 s	81.524 ms (7.9%)
Agent	iast	1.152 s	119.019 ms (11.5%)
Agent	profiling	1.217 s	184.215 ms (17.8%)
Total	tracing	9.326 s	-
Total	appsec	9.355 s	28.652 ms (0.3%)
Total	iast	9.511 s	184.546 ms (2.0%)
Total	profiling	9.576 s	249.676 ms (2.7%)

gantt
    title petclinic - break down per module: candidate=1.23.0-SNAPSHOT~072e68a521, baseline=1.23.0-SNAPSHOT~a012761e16

    dateFormat X
    axisFormat %s
section tracing
BytebuddyAgent [baseline] (642.296 ms) : 0, 642296
BytebuddyAgent [candidate] (646.206 ms) : 0, 646206
GlobalTracer [baseline] (294.646 ms) : 0, 294646
GlobalTracer [candidate] (295.76 ms) : 0, 295760
AppSec [baseline] (49.208 ms) : 0, 49208
AppSec [candidate] (49.209 ms) : 0, 49209
Remote Config [baseline] (680.815 µs) : 0, 681
Remote Config [candidate] (697.338 µs) : 0, 697
Telemetry [baseline] (6.105 ms) : 0, 6105
Telemetry [candidate] (6.161 ms) : 0, 6161
section appsec
BytebuddyAgent [baseline] (645.984 ms) : 0, 645984
BytebuddyAgent [candidate] (642.272 ms) : 0, 642272
GlobalTracer [baseline] (295.352 ms) : 0, 295352
GlobalTracer [candidate] (293.272 ms) : 0, 293272
AppSec [baseline] (139.032 ms) : 0, 139032
AppSec [candidate] (138.026 ms) : 0, 138026
Remote Config [baseline] (643.199 µs) : 0, 643
Remote Config [candidate] (638.492 µs) : 0, 638
Telemetry [baseline] (5.751 ms) : 0, 5751
Telemetry [candidate] (5.704 ms) : 0, 5704
section iast
BytebuddyAgent [baseline] (777.439 ms) : 0, 777439
BytebuddyAgent [candidate] (771.787 ms) : 0, 771787
GlobalTracer [baseline] (275.153 ms) : 0, 275153
GlobalTracer [candidate] (273.491 ms) : 0, 273491
AppSec [baseline] (46.694 ms) : 0, 46694
AppSec [candidate] (47.134 ms) : 0, 47134
Remote Config [baseline] (1.244 ms) : 0, 1244
Remote Config [candidate] (579.982 µs) : 0, 580
Telemetry [baseline] (6.946 ms) : 0, 6946
Telemetry [candidate] (7.524 ms) : 0, 7524
IAST [baseline] (16.794 ms) : 0, 16794
IAST [candidate] (16.831 ms) : 0, 16831
section profiling
BytebuddyAgent [baseline] (657.163 ms) : 0, 657163
BytebuddyAgent [candidate] (662.71 ms) : 0, 662710
GlobalTracer [baseline] (359.635 ms) : 0, 359635
GlobalTracer [candidate] (362.227 ms) : 0, 362227
AppSec [baseline] (50.321 ms) : 0, 50321
AppSec [candidate] (49.8 ms) : 0, 49800
Remote Config [baseline] (647.036 µs) : 0, 647
Remote Config [candidate] (659.882 µs) : 0, 660
Telemetry [baseline] (6.042 ms) : 0, 6042
Telemetry [candidate] (6.168 ms) : 0, 6168
ProfilingAgent [baseline] (81.015 ms) : 0, 81015
ProfilingAgent [candidate] (81.204 ms) : 0, 81204
Profiling [baseline] (81.041 ms) : 0, 81041
Profiling [candidate] (81.23 ms) : 0, 81230

Load

Parameters

	Baseline	Candidate
commit	1.23.0-SNAPSHOT~a012761e16	1.23.0-SNAPSHOT~072e68a521
config	baseline	candidate
end_time	2023-10-27T09:17:42	2023-10-27T09:34:04
start_time	2023-10-27T09:17:29	2023-10-27T09:33:51

See matching parameters

	Baseline	Candidate
application	insecure-bank	insecure-bank
variant	iast	iast

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 22 cases.

Request duration reports for insecure-bank

gantt
    title insecure-bank - request duration [CI 0.99] : candidate=1.23.0-SNAPSHOT~072e68a521, baseline=1.23.0-SNAPSHOT~a012761e16
    dateFormat X
    axisFormat %s
section baseline
no_agent (360.147 µs) : 339, 381
.   : milestone, 360,
iast (471.176 µs) : 450, 492
.   : milestone, 471,
iast_FULL (528.003 µs) : 507, 549
.   : milestone, 528,
iast_INACTIVE (430.327 µs) : 409, 451
.   : milestone, 430,
iast_TELEMETRY_OFF (461.486 µs) : 440, 483
.   : milestone, 461,
tracing (430.152 µs) : 409, 451
.   : milestone, 430,
section candidate
no_agent (363.093 µs) : 342, 384
.   : milestone, 363,
iast (461.19 µs) : 441, 482
.   : milestone, 461,
iast_FULL (526.994 µs) : 506, 548
.   : milestone, 527,
iast_INACTIVE (431.855 µs) : 410, 454
.   : milestone, 432,
iast_TELEMETRY_OFF (463.163 µs) : 442, 485
.   : milestone, 463,
tracing (431.538 µs) : 411, 452
.   : milestone, 432,

baseline results

Variant	Request duration [CI 0.99]	Δ no_agent
no_agent	360.147 µs [339.297 µs, 380.996 µs]	-
iast	471.176 µs [449.963 µs, 492.388 µs]	111.029 µs (30.8%)
iast_FULL	528.003 µs [507.274 µs, 548.732 µs]	167.857 µs (46.6%)
iast_INACTIVE	430.327 µs [409.372 µs, 451.282 µs]	70.18 µs (19.5%)
iast_TELEMETRY_OFF	461.486 µs [440.386 µs, 482.587 µs]	101.34 µs (28.1%)
tracing	430.152 µs [408.982 µs, 451.322 µs]	70.006 µs (19.4%)

candidate results

Variant	Request duration [CI 0.99]	Δ no_agent
no_agent	363.093 µs [341.895 µs, 384.292 µs]	-
iast	461.19 µs [440.545 µs, 481.835 µs]	98.097 µs (27.0%)
iast_FULL	526.994 µs [506.101 µs, 547.886 µs]	163.9 µs (45.1%)
iast_INACTIVE	431.855 µs [410.184 µs, 453.526 µs]	68.762 µs (18.9%)
iast_TELEMETRY_OFF	463.163 µs [441.569 µs, 484.758 µs]	100.07 µs (27.6%)
tracing	431.538 µs [410.945 µs, 452.131 µs]	68.445 µs (18.9%)

Request duration reports for petclinic

gantt
    title petclinic - request duration [CI 0.99] : candidate=1.23.0-SNAPSHOT~072e68a521, baseline=1.23.0-SNAPSHOT~a012761e16
    dateFormat X
    axisFormat %s
section baseline
no_agent (1.348 ms) : 1329, 1367
.   : milestone, 1348,
appsec (1.713 ms) : 1689, 1738
.   : milestone, 1713,
iast (1.448 ms) : 1424, 1472
.   : milestone, 1448,
profiling (1.451 ms) : 1426, 1475
.   : milestone, 1451,
tracing (1.461 ms) : 1436, 1486
.   : milestone, 1461,
section candidate
no_agent (1.369 ms) : 1349, 1388
.   : milestone, 1369,
appsec (1.685 ms) : 1661, 1710
.   : milestone, 1685,
iast (1.457 ms) : 1433, 1481
.   : milestone, 1457,
profiling (1.497 ms) : 1472, 1523
.   : milestone, 1497,
tracing (1.457 ms) : 1432, 1482
.   : milestone, 1457,

baseline results

Variant	Request duration [CI 0.99]	Δ no_agent
no_agent	1.348 ms [1.329 ms, 1.367 ms]	-
appsec	1.713 ms [1.689 ms, 1.738 ms]	365.532 µs (27.1%)
iast	1.448 ms [1.424 ms, 1.472 ms]	100.156 µs (7.4%)
profiling	1.451 ms [1.426 ms, 1.475 ms]	102.938 µs (7.6%)
tracing	1.461 ms [1.436 ms, 1.486 ms]	113.483 µs (8.4%)

candidate results

Variant	Request duration [CI 0.99]	Δ no_agent
no_agent	1.369 ms [1.349 ms, 1.388 ms]	-
appsec	1.685 ms [1.661 ms, 1.71 ms]	316.519 µs (23.1%)
iast	1.457 ms [1.433 ms, 1.481 ms]	88.442 µs (6.5%)
profiling	1.497 ms [1.472 ms, 1.523 ms]	128.675 µs (9.4%)
tracing	1.457 ms [1.432 ms, 1.482 ms]	88.262 µs (6.4%)

manuel-alvarez-alvarez added type: refactoring comp: asm iast Application Security Management (IAST) labels Oct 11, 2023

smola approved these changes Oct 16, 2023

View reviewed changes

manuel-alvarez-alvarez mentioned this pull request Oct 16, 2023

Fix IAST ranges for StringBuffer.toString #6050

Closed

manuel-alvarez-alvarez force-pushed the malvarez/iast-taint-api-refactor branch 3 times, most recently from 0efacb4 to 995f5c4 Compare October 22, 2023 20:27

manuel-alvarez-alvarez marked this pull request as ready for review October 22, 2023 21:23

manuel-alvarez-alvarez requested review from a team as code owners October 22, 2023 21:23

manuel-alvarez-alvarez requested review from dougqh, jandro996 and mcculls October 22, 2023 21:23

manuel-alvarez-alvarez self-assigned this Oct 23, 2023

manuel-alvarez-alvarez added run-tests: all Run all tests tag: no release notes Changes to exclude from release notes labels Oct 23, 2023

manuel-alvarez-alvarez requested review from DDJavierSantos and smola October 23, 2023 07:45

manuel-alvarez-alvarez force-pushed the malvarez/iast-taint-api-refactor branch from 03bcf09 to 333c7a8 Compare October 23, 2023 09:20

smola approved these changes Oct 23, 2023

View reviewed changes

manuel-alvarez-alvarez force-pushed the malvarez/iast-taint-api-refactor branch from 333c7a8 to 9f8489a Compare October 23, 2023 16:06

jandro996 approved these changes Oct 24, 2023

View reviewed changes

manuel-alvarez-alvarez force-pushed the malvarez/iast-taint-api-refactor branch from 9f8489a to 29e3032 Compare October 24, 2023 09:18

manuel-alvarez-alvarez mentioned this pull request Oct 24, 2023

Improve http.request.path IAST source for servlet #6091

Closed

DDJavierSantos approved these changes Oct 25, 2023

View reviewed changes

manuel-alvarez-alvarez force-pushed the malvarez/iast-taint-api-refactor branch 4 times, most recently from 2116662 to 469769d Compare October 25, 2023 15:24

manuel-alvarez-alvarez force-pushed the malvarez/iast-taint-api-refactor branch from 469769d to e52e4ee Compare October 26, 2023 09:58

smola approved these changes Oct 26, 2023

View reviewed changes

manuel-alvarez-alvarez force-pushed the malvarez/iast-taint-api-refactor branch 2 times, most recently from 547364a to 3a6030d Compare October 27, 2023 08:32

Refactor of the PropagationModule and removal of WebModule

072e68a

manuel-alvarez-alvarez force-pushed the malvarez/iast-taint-api-refactor branch from 3a6030d to 072e68a Compare October 27, 2023 08:50

manuel-alvarez-alvarez merged commit 00a81bb into master Oct 27, 2023

manuel-alvarez-alvarez deleted the malvarez/iast-taint-api-refactor branch October 27, 2023 10:00

github-actions Bot added this to the 1.23.0 milestone Oct 27, 2023

jandro996 pushed a commit that referenced this pull request Oct 27, 2023

Refactor of the PropagationModule and removal of WebModule (#6033)

b68b8f0

manuel-alvarez-alvarez mentioned this pull request Oct 30, 2023

Add name in sources for vertx 4 header names and param values #6114

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Refactor of the PropagationModule and removal of WebModule#6033

Refactor of the PropagationModule and removal of WebModule#6033
manuel-alvarez-alvarez merged 1 commit into
masterfrom
malvarez/iast-taint-api-refactor

manuel-alvarez-alvarez commented Oct 11, 2023 •

edited

Loading

Uh oh!

pr-commenter Bot commented Oct 11, 2023 •

edited

Loading

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

Conversation

manuel-alvarez-alvarez commented Oct 11, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What Does This Do

Motivation

Additional Notes

Uh oh!

pr-commenter Bot commented Oct 11, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Benchmarks

Startup

Parameters

Summary

Load

Parameters

Summary

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

manuel-alvarez-alvarez commented Oct 11, 2023 •

edited

Loading

pr-commenter Bot commented Oct 11, 2023 •

edited

Loading