Skip to content

gh-114763: Protect lazy loading modules from attribute access race #114781

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
brettcannon merged 12 commits into from
Feb 24, 2024
+94 −32
Merged

gh-114763: Protect lazy loading modules from attribute access race #114781

Changes from 1 commit
Commits
Show all changes
12 commits
Select commit Hold shift + click to select a range
d57f4eb
gh-114763: Protect lazy loading modules from attribute access race
effigies Jan 31, 2024
3db717c
TEST: Reproduce gh-114763
effigies Jan 31, 2024
5424d0c
Add new blurb
effigies Jan 31, 2024
96c08c5
Improve comments around the branching
effigies Feb 1, 2024
3642ddc
TEST: Use TestCase.assert* methods
effigies Feb 1, 2024
46238e5
Merge branch 'main' into fix-issue-114763
brettcannon Feb 6, 2024
6accf0c
Apply suggestions from code review
effigies Feb 17, 2024
bb2292f
TEST: Require working threading for test_module_load_race
effigies Feb 17, 2024
57fe084
Move threading import to module level of importlib.util
effigies Feb 17, 2024
40383a0
Make is_loading a bool instead of Event
effigies Feb 17, 2024
6322dd6
Merge branch 'main' into fix-issue-114763
effigies Feb 23, 2024
023d65d
Merge branch 'main' into fix-issue-114763
brettcannon Feb 23, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
Improve comments around the branching
  • Loading branch information
@effigies
effigies committed Feb 5, 2024
commit 96c08c5ad9e63f9f56941ebdfa8afa7296f2e060
9 changes: 7 additions & 2 deletions Lib/importlib/util.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -174,9 +174,14 @@ def __getattribute__(self, attr):
__spec__ = object.__getattribute__(self, '__spec__')
loader_state = __spec__.loader_state
with loader_state['lock']:
# Only the first thread to get the lock should trigger the load
# and reset the module's class. The rest can now getattr().
if object.__getattribute__(self, '__class__') is _LazyModule:
Comment thread
effigies marked this conversation as resolved.
# exec_module() will access dunder attributes, so we use a reentrant
# lock and an event to prevent infinite recursion.
# The first thread comes here multiple times as it descends the
# call stack. The first time, it sets is_loading and triggers
# exec_module(), which will access module.__dict__, module.__name__,
# and/or module.__spec__, reentering this method. These accesses
# need to be allowed to proceed without triggering the load again.
if loader_state['is_loading'].is_set() and attr[:2] == attr[-2:] == '__':
Comment thread
brettcannon marked this conversation as resolved.
Outdated
Comment thread
effigies marked this conversation as resolved.
Outdated
return object.__getattribute__(self, attr)
loader_state['is_loading'].set()
Comment thread
effigies marked this conversation as resolved.
Outdated
Expand Down