Skip to content

gh-107265: Remove all ENTER_EXECUTOR when execute _Py_Instrument #108539

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
corona10 merged 17 commits into from
Sep 7, 2023
Merged

gh-107265: Remove all ENTER_EXECUTOR when execute _Py_Instrument #108539

Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
17 commits
Select commit Hold shift + click to select a range
23e1f48
Revert "gh-107265: Fix initialize/remove_tools for ENTER_EXECUTOR cas…
corona10 Aug 27, 2023
8820df8
gh-107265: Remove all ENTER_EXECUTOR when execute _Py_Instrument
corona10 Aug 27, 2023
e015f14
nit
corona10 Aug 27, 2023
7612b8e
Add assert
corona10 Aug 27, 2023
7d38c77
nit
corona10 Aug 27, 2023
a01354d
Address code review
corona10 Sep 1, 2023
9a631d7
nit
corona10 Sep 1, 2023
7e81bf0
Add test
corona10 Sep 1, 2023
162a870
Merge remote-tracking branch 'upstream/main' into gh-107265-mark
corona10 Sep 1, 2023
05773f9
Add test
corona10 Sep 1, 2023
9ddbfe7
Add space between +=
corona10 Sep 1, 2023
1199dfe
Return -1 if there is ENTER_EXECUTOR
corona10 Sep 1, 2023
1232432
Update
corona10 Sep 1, 2023
e051be1
Merge remote-tracking branch 'upstream/main' into gh-107265-mark
corona10 Sep 6, 2023
4a0be4d
fix
corona10 Sep 6, 2023
bc251b5
Fix
corona10 Sep 6, 2023
83d8829
Address Guido's review
corona10 Sep 6, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
Add test
  • Loading branch information
@corona10
corona10 committed Sep 1, 2023
commit 05773f94cd9ec95501fab8149ab8b892a57ff2e9
9 changes: 4 additions & 5 deletions Lib/test/test_monitoring.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -1700,15 +1700,17 @@ def run():
sys.monitoring.set_events(TEST_TOOL, 0)


import _testinternalcapi
class TestOptimizer(MonitoringTestBase, unittest.TestCase):
Copy link
Copy Markdown
Member Author

@corona10 corona10 Sep 1, 2023 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@markshannon cc @gvanrossum

I added a test, but not sure what you wanted :)
Without this PR, Python/instrumentation.c#L1589 will not be able to pass the assert through this test.

Copy link
Copy Markdown
Member

@gvanrossum gvanrossum Sep 6, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What is this test supposed to do? When I copy this test into the main branch, it passes (in debug mode, so failing asserts would cause crashes), so I don't think this proves your fix works.

By adding some dis.dis(test_func, adaptive=True) calls to the test I think I see the difference -- in main, an ENTER_EXECUTOR opcode remains in the code object, whereas with this PR, the ENTER_EXECUTOR opcode appears after the function is called, but disappears again after the second set_local_events call.

You could check for that ENTER_EXECUTOR (there are examples in test_capi/test_misc.py) but it would be nice if you had example code that actually crashed or triggered an assert (in debug mode) in main without your fix.

Copy link
Copy Markdown
Member Author

@corona10 corona10 Sep 6, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hmm,, interesting, It looks like I watched the Hallucination when I wrote the test in the first place :(
I will soon update the PR.

Copy link
Copy Markdown
Member Author

@corona10 corona10 Sep 6, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

See: #108539 (comment)


def setUp(self):
import _testinternalcapi
self.old_opt = _testinternalcapi.get_optimizer()
opt = _testinternalcapi.get_counter_optimizer()
_testinternalcapi.set_optimizer(opt)
super(TestOptimizer, self).setUp()

def tearDown(self):
import _testinternalcapi
super(TestOptimizer, self).tearDown()
_testinternalcapi.set_optimizer(self.old_opt)

Expand All @@ -1721,9 +1723,6 @@ def test_func(x):
code = test_func.__code__
sys.monitoring.set_local_events(TEST_TOOL, code, E.PY_START)
self.assertEqual(sys.monitoring.get_local_events(TEST_TOOL, code), E.PY_START)
sys.monitoring.set_local_events(TEST_TOOL2, code, E.PY_START)
self.assertEqual(sys.monitoring.get_local_events(TEST_TOOL2, code), E.PY_START)
test_func(1000)
sys.monitoring.set_local_events(TEST_TOOL, code, 0)
self.assertEqual(sys.monitoring.get_local_events(TEST_TOOL, code), 0)
sys.monitoring.set_local_events(TEST_TOOL2, code, 0)
self.assertEqual(sys.monitoring.get_local_events(TEST_TOOL2, code), 0)
1 change: 1 addition & 0 deletions Python/instrumentation.c
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -1586,6 +1586,7 @@ _Py_Instrument(PyCodeObject *code, PyInterpreterState *interp)
_Py_CODEUNIT *instr = &_PyCode_CODE(code)[i];
CHECK(instr->op.code != 0);
int base_opcode = _Py_GetBaseOpcode(code, i);
assert(base_opcode != ENTER_EXECUTOR);
if (opcode_has_event(base_opcode)) {
int8_t event;
if (base_opcode == RESUME) {
Expand Down