Skip to content

Python: Dataflow, test magic methods #3981

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
tausbn merged 29 commits into from
Aug 14, 2020
Merged

Python: Dataflow, test magic methods #3981

Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
29 commits
Select commit Hold shift + click to select a range
38acea6
Python: Dataflow, expand callable to classes
yoff Jul 27, 2020
eab64f1
Python: Dataflow, start on test for classes
yoff Jul 28, 2020
488a7f4
Python: update test expectations
yoff Jul 28, 2020
d32e277
Python: some doc, a generator, and a corotuine
yoff Jul 29, 2020
d7c08f7
Merge branch 'master' of github.com:github/codeql into SharedDataflow…
yoff Aug 4, 2020
81ad455
Python: full list of magic methods to be tested
yoff Aug 5, 2020
a896246
Python: format ql
yoff Aug 5, 2020
e642808
Update python/ql/test/experimental/dataflow/coverage/classes.py
yoff Aug 5, 2020
614103c
Python: Test calls rather than flows
yoff Aug 6, 2020
3db1cee
Python: format ql
yoff Aug 6, 2020
d84294d
Python: Check that tests are valid
yoff Aug 7, 2020
aff4535
Python: fix tests for descriptors
yoff Aug 7, 2020
f6d6f91
Python: tests for containers
yoff Aug 7, 2020
5b7c7f9
Python: tests for numeric classes
yoff Aug 7, 2020
0247877
Python: tests for context managers
yoff Aug 10, 2020
639d914
Python: test Awaitable, framework for async test
yoff Aug 10, 2020
959c631
Python: update reference to fix tests
yoff Aug 10, 2020
a963f15
Python: format strings are unnecessary and mess up
yoff Aug 10, 2020
5da37f5
Python: Update test expectations
yoff Aug 10, 2020
681657f
Merge branch 'master' of github.com:github/codeql into SharedDataflow…
yoff Aug 11, 2020
3929e01
Python: tests for async iterators/context managers
yoff Aug 11, 2020
12dfc4a
Python: clean up validity check code
yoff Aug 11, 2020
2c5de7f
Python: fix r/l confusion
yoff Aug 11, 2020
f834d71
Python: split out data model tests
yoff Aug 11, 2020
3949911
Python: Update test expectations
yoff Aug 11, 2020
dd4d002
Python: remaining class tests
yoff Aug 11, 2020
4b336e9
Update python/ql/test/experimental/dataflow/coverage/classes.py
yoff Aug 14, 2020
8d49ad7
Update python/ql/test/experimental/dataflow/coverage/datamodel.py
yoff Aug 14, 2020
9556937
Python: address review comments
yoff Aug 14, 2020
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
Python: Test calls rather than flows
  • Loading branch information
@yoff
yoff committed Aug 6, 2020
commit 614103c3b6bcea797cdeaeda2733862fa5cad0ec
2 changes: 1 addition & 1 deletion python/ql/test/experimental/dataflow/basic/callGraph.ql
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
import callGraphConfig
import experimental.dataflow.callGraphConfig

from DataFlow::Node source, DataFlow::Node sink
where exists(CallGraphConfig cfg | cfg.hasFlow(source, sink))
Expand Down
0 ...mental/dataflow/basic/callGraphConfig.qll → ...experimental/dataflow/callGraphConfig.qll
View file
Open in desktop
File renamed without changes.
12 changes: 6 additions & 6 deletions python/ql/test/experimental/dataflow/coverage/classes.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -128,6 +128,10 @@ async def agen(x):
# Class Instances
# Instances of arbitrary classes can be made callable by defining a __call__() method in their class.

# If a class sets __iter__() to None, calling iter() on its instances will raise a TypeError (without falling back to __getitem__()).

# 3.3.1. Basic customization

class Customized:

a = NONSOURCE
Expand All @@ -140,19 +144,15 @@ def __new__(cls):
def __init__(self):
self.b = SOURCE

# object.__new__(cls[, ...])
# object.__init__(self[, ...])
customized = Customized()
SINK(Customized.a)
SINK_F(Customized.b)
SINK(customized.a)
SINK(customized.b)

# def __del__(self):

# If a class sets __iter__() to None, calling iter() on its instances will raise a TypeError (without falling back to __getitem__()).

# 3.3.1. Basic customization
# object.__new__(cls[, ...])
# object.__init__(self[, ...])
# object.__del__(self)
# object.__repr__(self)
# object.__str__(self)¶
Expand Down
81 changes: 81 additions & 0 deletions python/ql/test/experimental/dataflow/coverage/classesCallGraph.expected
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,81 @@
| classes.py:29:10:29:10 | ControlFlowNode for a | classes.py:17:10:17:10 | SSA variable x |
| classes.py:29:10:29:10 | ControlFlowNode for a | classes.py:31:6:31:17 | ControlFlowNode for f() |
| classes.py:31:6:31:17 | ControlFlowNode for f() | classes.py:17:10:17:10 | SSA variable x |
| classes.py:31:6:31:17 | ControlFlowNode for f() | classes.py:31:6:31:17 | ControlFlowNode for f() |
| classes.py:31:8:31:13 | ControlFlowNode for SOURCE | classes.py:17:10:17:10 | SSA variable x |
| classes.py:31:8:31:13 | ControlFlowNode for SOURCE | classes.py:28:7:28:7 | SSA variable a |
| classes.py:31:8:31:13 | ControlFlowNode for SOURCE | classes.py:31:6:31:17 | ControlFlowNode for f() |
| classes.py:31:16:31:16 | ControlFlowNode for IntegerLiteral | classes.py:28:10:28:10 | SSA variable b |
| classes.py:39:16:39:16 | ControlFlowNode for x | classes.py:17:10:17:10 | SSA variable x |
| classes.py:39:16:39:16 | ControlFlowNode for x | classes.py:64:6:64:24 | ControlFlowNode for Attribute() |
| classes.py:39:16:39:16 | ControlFlowNode for x | classes.py:65:6:65:27 | ControlFlowNode for Attribute() |
| classes.py:43:16:43:16 | ControlFlowNode for x | classes.py:17:10:17:10 | SSA variable x |
| classes.py:43:16:43:16 | ControlFlowNode for x | classes.py:73:6:73:26 | ControlFlowNode for Attribute() |
| classes.py:43:16:43:16 | ControlFlowNode for x | classes.py:74:6:74:26 | ControlFlowNode for Attribute() |
| classes.py:56:14:56:14 | ControlFlowNode for x | classes.py:100:18:100:31 | ControlFlowNode for Attribute() |
| classes.py:64:6:64:24 | ControlFlowNode for Attribute() | classes.py:17:10:17:10 | SSA variable x |
| classes.py:64:6:64:24 | ControlFlowNode for Attribute() | classes.py:64:6:64:24 | ControlFlowNode for Attribute() |
| classes.py:64:15:64:20 | ControlFlowNode for SOURCE | classes.py:17:10:17:10 | SSA variable x |
| classes.py:64:15:64:20 | ControlFlowNode for SOURCE | classes.py:37:16:37:19 | SSA variable self |
| classes.py:64:15:64:20 | ControlFlowNode for SOURCE | classes.py:37:22:37:22 | SSA variable x |
| classes.py:64:15:64:20 | ControlFlowNode for SOURCE | classes.py:64:6:64:24 | ControlFlowNode for Attribute() |
| classes.py:64:23:64:23 | ControlFlowNode for C | classes.py:17:10:17:10 | SSA variable x |
| classes.py:64:23:64:23 | ControlFlowNode for C | classes.py:37:22:37:22 | SSA variable x |
| classes.py:64:23:64:23 | ControlFlowNode for C | classes.py:37:25:37:27 | SSA variable cls |
| classes.py:64:23:64:23 | ControlFlowNode for C | classes.py:64:6:64:24 | ControlFlowNode for Attribute() |
| classes.py:65:6:65:27 | ControlFlowNode for Attribute() | classes.py:17:10:17:10 | SSA variable x |
| classes.py:65:6:65:27 | ControlFlowNode for Attribute() | classes.py:65:6:65:27 | ControlFlowNode for Attribute() |
| classes.py:65:15:65:15 | ControlFlowNode for c | classes.py:37:16:37:19 | SSA variable self |
| classes.py:65:18:65:23 | ControlFlowNode for SOURCE | classes.py:17:10:17:10 | SSA variable x |
| classes.py:65:18:65:23 | ControlFlowNode for SOURCE | classes.py:37:22:37:22 | SSA variable x |
| classes.py:65:18:65:23 | ControlFlowNode for SOURCE | classes.py:65:6:65:27 | ControlFlowNode for Attribute() |
| classes.py:65:26:65:26 | ControlFlowNode for C | classes.py:37:25:37:27 | SSA variable cls |
| classes.py:66:6:66:27 | ControlFlowNode for func_obj() | classes.py:17:10:17:10 | SSA variable x |
| classes.py:66:6:66:27 | ControlFlowNode for func_obj() | classes.py:66:6:66:27 | ControlFlowNode for func_obj() |
| classes.py:73:6:73:26 | ControlFlowNode for Attribute() | classes.py:17:10:17:10 | SSA variable x |
| classes.py:73:6:73:26 | ControlFlowNode for Attribute() | classes.py:73:6:73:26 | ControlFlowNode for Attribute() |
| classes.py:73:20:73:25 | ControlFlowNode for SOURCE | classes.py:17:10:17:10 | SSA variable x |
| classes.py:73:20:73:25 | ControlFlowNode for SOURCE | classes.py:42:21:42:23 | SSA variable cls |
| classes.py:73:20:73:25 | ControlFlowNode for SOURCE | classes.py:42:26:42:26 | SSA variable x |
| classes.py:73:20:73:25 | ControlFlowNode for SOURCE | classes.py:73:6:73:26 | ControlFlowNode for Attribute() |
| classes.py:74:6:74:26 | ControlFlowNode for Attribute() | classes.py:17:10:17:10 | SSA variable x |
| classes.py:74:6:74:26 | ControlFlowNode for Attribute() | classes.py:74:6:74:26 | ControlFlowNode for Attribute() |
| classes.py:74:20:74:25 | ControlFlowNode for SOURCE | classes.py:17:10:17:10 | SSA variable x |
| classes.py:74:20:74:25 | ControlFlowNode for SOURCE | classes.py:42:21:42:23 | SSA variable cls |
| classes.py:74:20:74:25 | ControlFlowNode for SOURCE | classes.py:42:26:42:26 | SSA variable x |
| classes.py:74:20:74:25 | ControlFlowNode for SOURCE | classes.py:74:6:74:26 | ControlFlowNode for Attribute() |
| classes.py:75:6:75:26 | ControlFlowNode for c_func_obj() | classes.py:17:10:17:10 | SSA variable x |
| classes.py:75:6:75:26 | ControlFlowNode for c_func_obj() | classes.py:75:6:75:26 | ControlFlowNode for c_func_obj() |
| classes.py:85:12:85:17 | ControlFlowNode for SOURCE | classes.py:79:9:79:9 | SSA variable x |
| classes.py:85:20:85:20 | ControlFlowNode for IntegerLiteral | classes.py:79:12:79:16 | SSA variable count |
| classes.py:86:6:86:20 | ControlFlowNode for Attribute() | classes.py:17:10:17:10 | SSA variable x |
| classes.py:86:6:86:20 | ControlFlowNode for Attribute() | classes.py:86:6:86:20 | ControlFlowNode for Attribute() |
| classes.py:87:8:87:22 | ControlFlowNode for Attribute() | classes.py:20:12:20:12 | SSA variable x |
| classes.py:87:8:87:22 | ControlFlowNode for Attribute() | classes.py:87:8:87:22 | ControlFlowNode for Attribute() |
| classes.py:89:15:89:20 | ControlFlowNode for SOURCE | classes.py:49:13:49:16 | SSA variable self |
| classes.py:89:15:89:20 | ControlFlowNode for SOURCE | classes.py:49:19:49:19 | SSA variable x |
| classes.py:89:23:89:23 | ControlFlowNode for IntegerLiteral | classes.py:49:19:49:19 | SSA variable x |
| classes.py:89:23:89:23 | ControlFlowNode for IntegerLiteral | classes.py:49:22:49:26 | SSA variable count |
| classes.py:90:6:90:21 | ControlFlowNode for Attribute() | classes.py:17:10:17:10 | SSA variable x |
| classes.py:90:6:90:21 | ControlFlowNode for Attribute() | classes.py:90:6:90:21 | ControlFlowNode for Attribute() |
| classes.py:91:8:91:23 | ControlFlowNode for Attribute() | classes.py:20:12:20:12 | SSA variable x |
| classes.py:91:8:91:23 | ControlFlowNode for Attribute() | classes.py:91:8:91:23 | ControlFlowNode for Attribute() |
| classes.py:96:10:96:10 | ControlFlowNode for x | classes.py:99:18:99:29 | ControlFlowNode for coro() |
| classes.py:99:6:99:30 | ControlFlowNode for Attribute() | classes.py:17:10:17:10 | SSA variable x |
| classes.py:99:6:99:30 | ControlFlowNode for Attribute() | classes.py:99:6:99:30 | ControlFlowNode for Attribute() |
| classes.py:99:23:99:28 | ControlFlowNode for SOURCE | classes.py:95:16:95:16 | SSA variable x |
| classes.py:99:23:99:28 | ControlFlowNode for SOURCE | classes.py:99:18:99:29 | ControlFlowNode for coro() |
| classes.py:100:6:100:32 | ControlFlowNode for Attribute() | classes.py:17:10:17:10 | SSA variable x |
| classes.py:100:6:100:32 | ControlFlowNode for Attribute() | classes.py:100:6:100:32 | ControlFlowNode for Attribute() |
| classes.py:100:25:100:30 | ControlFlowNode for SOURCE | classes.py:55:20:55:23 | SSA variable self |
| classes.py:100:25:100:30 | ControlFlowNode for SOURCE | classes.py:55:26:55:26 | SSA variable x |
| classes.py:100:25:100:30 | ControlFlowNode for SOURCE | classes.py:100:18:100:31 | ControlFlowNode for Attribute() |
| classes.py:110:10:110:16 | ControlFlowNode for Await | classes.py:112:18:112:29 | ControlFlowNode for agen() |
| classes.py:112:6:112:30 | ControlFlowNode for Attribute() | classes.py:17:10:17:10 | SSA variable x |
| classes.py:112:6:112:30 | ControlFlowNode for Attribute() | classes.py:112:6:112:30 | ControlFlowNode for Attribute() |
| classes.py:112:23:112:28 | ControlFlowNode for SOURCE | classes.py:108:16:108:16 | SSA variable x |
| classes.py:142:12:142:31 | ControlFlowNode for Attribute() | classes.py:142:12:142:31 | ControlFlowNode for Attribute() |
| classes.py:150:6:150:17 | ControlFlowNode for Attribute | classes.py:17:10:17:10 | SSA variable x |
| classes.py:151:8:151:19 | ControlFlowNode for Attribute | classes.py:20:12:20:12 | SSA variable x |
| classes.py:152:6:152:17 | ControlFlowNode for Attribute | classes.py:17:10:17:10 | SSA variable x |
| classes.py:153:6:153:17 | ControlFlowNode for Attribute | classes.py:17:10:17:10 | SSA variable x |
10 changes: 10 additions & 0 deletions python/ql/test/experimental/dataflow/coverage/classesCallGraph.ql
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,10 @@
import experimental.dataflow.callGraphConfig

from DataFlow::Node source, DataFlow::Node sink
where
source.getLocation().getFile().getBaseName() = "classes.py" and
sink.getLocation().getFile().getBaseName() = "classes.py" and
exists(CallGraphConfig cfg | cfg.hasFlow(source, sink))
select source, sink

// Rewrite this to just have 1-step paths?

@tausbn tausbn Aug 13, 2020

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this comment is a bit too cryptic for me. Can you elaborate? 🙂