Skip to content

ReDoS: testing a parameterised ReDoS module #10604

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
erik-krogh wants to merge 35 commits into from
Closed

ReDoS: testing a parameterised ReDoS module #10604

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
35 commits
Select commit Hold shift + click to select a range
5ec22bc
add a shared regex pack
erik-krogh Nov 1, 2022
09275a5
remove files from identical files that soon won't be identical
erik-krogh Nov 1, 2022
f5daee2
port canonicalization fix from #11071 to the shared pack
erik-krogh Nov 7, 2022
c42979d
add codeql/regex as a dependency
erik-krogh Nov 1, 2022
f4faab9
move existing regex-tree into a module
erik-krogh Nov 1, 2022
c765d69
add a Java implementation of `RegexTreeViewSig`
erik-krogh Nov 1, 2022
a0b3066
use namespace in `PrintAst.qll` to avoid conflict with `Top`
erik-krogh Nov 1, 2022
2d66cc9
port the Java regex/redos queries to use the shared pack
erik-krogh Nov 1, 2022
acf412e
add change-note
erik-krogh Nov 1, 2022
9cd9755
use instead of a fixed version number
erik-krogh Nov 7, 2022
2b13992
add codeql/regex as a dependency
erik-krogh Nov 1, 2022
5fbcbbc
move existing regex-tree into a module
erik-krogh Nov 1, 2022
1aeaefc
add a Python implementation of `RegexTreeViewSig`
erik-krogh Nov 1, 2022
0560548
drive-by simplification of the python regex-tree
erik-krogh Nov 1, 2022
4f11e2d
port the Python regex/redos queries to use the shared pack
erik-krogh Nov 1, 2022
6184386
update expected output of the queries (some sorting changed due to lo…
erik-krogh Nov 1, 2022
c733648
add change-note
erik-krogh Nov 1, 2022
c89016b
use instead of a fixed version number
erik-krogh Nov 7, 2022
dddf550
add codeql/regex as a dependency
erik-krogh Nov 1, 2022
af92270
move existing regex-tree into a module
erik-krogh Nov 1, 2022
3432e81
add a Ruby implementation of `RegexTreeViewSig`
erik-krogh Nov 1, 2022
40e4359
port the Ruby regex/redos queries to use the shared pack
erik-krogh Nov 1, 2022
860c3c4
update expected output of the queries (some sorting changed due to lo…
erik-krogh Nov 1, 2022
f2d980b
update ruby build to include the regex pack (depend on #10668)
erik-krogh Nov 1, 2022
7a8e715
add change-note
erik-krogh Nov 1, 2022
26dc923
add codeql/regex as a dependency
erik-krogh Nov 1, 2022
898b1dc
add `hasLocationInfo` predicate to regexp terms
erik-krogh Nov 1, 2022
1379fa3
add a JS implementation of `RegexTreeViewSig`
erik-krogh Nov 1, 2022
d8e4020
port the JS regex/redos queries to use the shared pack
erik-krogh Nov 1, 2022
23b413f
update expected output of the queries (some sorting changed due to lo…
erik-krogh Nov 1, 2022
030f3fd
add change-note
erik-krogh Nov 1, 2022
68aac59
use instead of a fixed version number
erik-krogh Nov 7, 2022
d84b37f
Merge branch 'rb-redosMod' into redosMod
erik-krogh Nov 7, 2022
fd74e12
Merge branch 'java-redosMod' into redosMod
erik-krogh Nov 7, 2022
d7350f6
Merge branch 'py-redosMod' into redosMod
erik-krogh Nov 7, 2022
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions .github/workflows/ruby-build.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -97,6 +97,7 @@ jobs:
run: |
codeql pack create ../shared/ssa --output target/packs
codeql pack create ../misc/suite-helpers --output target/packs
codeql pack create ../shared/regex --output target/packs
codeql pack create ql/lib --output target/packs
codeql pack create ql/src --output target/packs
PACK_FOLDER=$(readlink -f target/packs/codeql/ruby-queries/*)
Expand Down
34 changes: 0 additions & 34 deletions config/identical-files.json
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -486,40 +486,6 @@
"python/ql/lib/semmle/python/security/internal/SensitiveDataHeuristics.qll",
"ruby/ql/lib/codeql/ruby/security/internal/SensitiveDataHeuristics.qll"
],
"ReDoS Util Python/JS/Ruby/Java": [
"javascript/ql/lib/semmle/javascript/security/regexp/NfaUtils.qll",
"python/ql/lib/semmle/python/security/regexp/NfaUtils.qll",
"ruby/ql/lib/codeql/ruby/security/regexp/NfaUtils.qll",
"java/ql/lib/semmle/code/java/security/regexp/NfaUtils.qll"
],
"ReDoS Exponential Python/JS/Ruby/Java": [
"javascript/ql/lib/semmle/javascript/security/regexp/ExponentialBackTracking.qll",
"python/ql/lib/semmle/python/security/regexp/ExponentialBackTracking.qll",
"ruby/ql/lib/codeql/ruby/security/regexp/ExponentialBackTracking.qll",
"java/ql/lib/semmle/code/java/security/regexp/ExponentialBackTracking.qll"
],
"ReDoS Polynomial Python/JS/Ruby/Java": [
"javascript/ql/lib/semmle/javascript/security/regexp/SuperlinearBackTracking.qll",
"python/ql/lib/semmle/python/security/regexp/SuperlinearBackTracking.qll",
"ruby/ql/lib/codeql/ruby/security/regexp/SuperlinearBackTracking.qll",
"java/ql/lib/semmle/code/java/security/regexp/SuperlinearBackTracking.qll"
],
"RegexpMatching Python/JS/Ruby": [
"javascript/ql/lib/semmle/javascript/security/regexp/RegexpMatching.qll",
"python/ql/lib/semmle/python/security/regexp/RegexpMatching.qll",
"ruby/ql/lib/codeql/ruby/security/regexp/RegexpMatching.qll"
],
"BadTagFilterQuery Python/JS/Ruby": [
"javascript/ql/lib/semmle/javascript/security/BadTagFilterQuery.qll",
"python/ql/lib/semmle/python/security/BadTagFilterQuery.qll",
"ruby/ql/lib/codeql/ruby/security/BadTagFilterQuery.qll"
],
"OverlyLargeRange Python/JS/Ruby/Java": [
"javascript/ql/lib/semmle/javascript/security/OverlyLargeRangeQuery.qll",
"python/ql/lib/semmle/python/security/OverlyLargeRangeQuery.qll",
"ruby/ql/lib/codeql/ruby/security/OverlyLargeRangeQuery.qll",
"java/ql/lib/semmle/code/java/security/OverlyLargeRangeQuery.qll"
],
"CFG": [
"csharp/ql/lib/semmle/code/csharp/controlflow/internal/ControlFlowGraphImplShared.qll",
"ruby/ql/lib/codeql/ruby/controlflow/internal/ControlFlowGraphImplShared.qll",
Expand Down
4 changes: 4 additions & 0 deletions java/ql/lib/change-notes/2022-10-31-shared-redos-pack.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
---
category: minorAnalysis
---
* The ReDoS libraries in `semmle.code.java.security.regexp` has been moved to a shared pack inside the `shared/` folder, and the previous location has been deprecated.
2 changes: 2 additions & 0 deletions java/ql/lib/qlpack.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -5,3 +5,5 @@ dbscheme: config/semmlecode.dbscheme
extractor: java
library: true
upgrades: upgrades
dependencies:
codeql/regex: ${workspace}
14 changes: 8 additions & 6 deletions java/ql/lib/semmle/code/java/PrintAst.qll
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@
*/

import java
import semmle.code.java.regex.RegexTreeView
import semmle.code.java.regex.RegexTreeView as RegexTreeView

private newtype TPrintAstConfiguration = MkPrintAstConfiguration()

Expand Down Expand Up @@ -134,8 +134,10 @@ private newtype TPrintAstNode =
TImportsNode(CompilationUnit cu) {
shouldPrint(cu, _) and exists(Import i | i.getCompilationUnit() = cu)
} or
TRegExpTermNode(RegExpTerm term) {
exists(StringLiteral str | term.getRootTerm() = getParsedRegExp(str) and shouldPrint(str, _))
TRegExpTermNode(RegexTreeView::RegExpTerm term) {
exists(StringLiteral str |
term.getRootTerm() = RegexTreeView::getParsedRegExp(str) and shouldPrint(str, _)
)
}

/**
Expand Down Expand Up @@ -316,20 +318,20 @@ final class StringLiteralNode extends ExprStmtNode {

override PrintAstNode getChild(int childIndex) {
childIndex = 0 and
result.(RegExpTermNode).getTerm() = getParsedRegExp(element)
result.(RegExpTermNode).getTerm() = RegexTreeView::getParsedRegExp(element)
}
}

/**
* A node representing a regular expression term.
*/
class RegExpTermNode extends TRegExpTermNode, PrintAstNode {
RegExpTerm term;
RegexTreeView::RegExpTerm term;

RegExpTermNode() { this = TRegExpTermNode(term) }

/** Gets the `RegExpTerm` for this node. */
RegExpTerm getTerm() { result = term }
RegexTreeView::RegExpTerm getTerm() { result = term }

override PrintAstNode getChild(int childIndex) {
result.(RegExpTermNode).getTerm() = term.getChild(childIndex)
Expand Down
Loading