You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: _posts/2023-07-22-gsoc-sqlancer-final.md
+16-6Lines changed: 16 additions & 6 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -10,7 +10,7 @@ tags:
10
10
11
11
## Overview
12
12
13
-
SQLancer generates a large number of statements, but not all of them are relevant to the bug. To automatically reduce the test cases, [Yutan Yang](https://github.com/ColinYoungTaro) implemented two reducers in SQLancer, the statement reducer and the AST-based reducer.
13
+
SQLancer generates a large number of statements, but not all of them are relevant to the bug. To automatically reduce the test cases, I implemented two reducers in SQLancer, the statement reducer and the AST-based reducer.
14
14
15
15
16
16
## Statement Reducer
@@ -26,21 +26,31 @@ Using the statement reducer, SQLancer reduces the set of statements to a minimal
26
26
## AST-Based Reducer
27
27
28
28
The AST-based reducer can shorten a statement by applying AST level transformations. They are mostly implemented using [JSQLParser](https://github.com/JSQLParser/JSqlParser), a RDBMS agnostic SQL statement parser that can translate SQL statements into a traversable hierarchy of Java classes. JSQLParser provides support for the SQL standard as well as major SQL dialects. The AST-based reducer works for any SQL dialects that can be parsed by this tool.
29
-
q
30
29
31
30
Currently, the AST-based transformations include:
32
31
33
32
+ Remove union selects. e.g.,
33
+
34
34
`SELECT 1 UNION SELECT 2` -> `SELECT 1`
35
+
35
36
+ Remove irrelevant clauses. e.g.,
37
+
36
38
`SELECT * FROM t OFFSET 20 LIMIT 5` -> `SELECT * FROM t`
39
+
37
40
+ Remove list elements. e.g.,
41
+
38
42
`SELECT a, b, c FROM t` -> `SELECT a FROM t`
39
-
+ Remove rows of an insert statement. e.g.,
43
+
44
+
+ Remove rows of an insert statement. e.g.,
45
+
40
46
`INSERT INTO t VALUES (1, 2), (3, 4)` -> `INSERT INTO t VALUES (1, 2)`
47
+
41
48
+ Replace complicated expressions with their sub expressions. e.g.,
49
+
42
50
`(a+b)+c` -> `c`
43
-
+ Simplify constant values. e.g.,
51
+
52
+
+ Simplify constant values. e.g.,
53
+
44
54
`3.27842156` -> `3.278`
45
55
46
56
The AST-based reducer is designed with extensibility. It walks through the list of transformations and applies them to statements until a fixed point is reached. Adding new transformations is straightforward. Simply create a new transformation and include it in the list. This flexibility allows for easy customization and expansion of the reducer's functionality. Additionally, it is also possible to support transformations that do not depend on JSQLParser.
@@ -49,7 +59,7 @@ The AST-based reducer is designed with extensibility. It walks through the list
49
59
50
60
## Framework for reducers Testing
51
61
52
-
Yutan designed a virtual database engine to facilitate the testing of reducers. Instead of executing the statements, this engine records them for analysis. One of its notable features is the ability to customize the conditions that trigger a bug. For instance, the testing framework allows specifying an interestingness check that tests whether certain words are part of the reduced SQL test case.
62
+
I designed a virtual database engine to facilitate the testing of reducers. Instead of executing the statements, this engine records them for analysis. One of its notable features is the ability to customize the conditions that trigger a bug. For instance, the testing framework allows specifying an interestingness check that tests whether certain words are part of the reduced SQL test case.
53
63
54
64
This approach provides a testing environment and allows for thorough evaluation of the reducers' performance. It offers convenience and flexibility in refining and polishing the functionality of the reducers without impacting real databases.
55
65
@@ -68,4 +78,4 @@ Note: if `--reduce-ast` is set, `--use-reducer` option must be enabled first.
68
78
69
79
There are also options to define timeout seconds and max steps of reduction for both statement reducer and AST-based reducer.
70
80
71
-
For more details, you can refer to this [doc](https://github.com/sqlancer/sqlancer/blob/main/docs/testCaseReduction.md).
81
+
For more details, you can refer to this [doc](https://github.com/sqlancer/sqlancer/blob/7804a3adec0962ad6d24687c42ec473aa49669fe/docs/testCaseReduction.md).
0 commit comments