gh-149728: Fix free-threaded race in importlib lazy-submodule fast path#149729
Open
SwayamInSync wants to merge 2 commits into
Open
gh-149728: Fix free-threaded race in importlib lazy-submodule fast path#149729SwayamInSync wants to merge 2 commits into
SwayamInSync wants to merge 2 commits into
Conversation
|
Most changes to Python require a NEWS entry. Add one using the blurb_it web app or the blurb command-line tool. If this change has little impact on Python users, wait for a maintainer to apply the |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes #149728.
Cause
_load_unlockedclearsspec._initializingat the end of its body, before_find_and_load_unlockedrunssetattr(parent_module, child, module). Between those two events,sys.modules[name]is set and_initializing == False, butparent.__dict__[child]is still missing. The fast path in_find_and_loadreturns the module without taking the import lock once it sees_initializing == False, so under free-threaded CPython a second thread can observe this window.IMPORT_FROM 'child'on that thread doesgetattr(parent, 'child'), falls into a lazy__getattr__, runs the sameimport parent.child as childline, fast-paths again, and recurses toRecursionError. See the linked issue for the full walkthrough and reproducer.Change
Keep
spec._initializing == Trueuntil after the parent setattr in_find_and_load_unlocked:_load_unlockedno longer clears_initializingon the success path. Failure paths still clear it._find_and_load_unlockedclears_initializingin afinallyblock aftersetattr(parent_module, child, module)and_imp._set_lazy_attributes._load_unlocked(_loadand_builtin_from_name) have no parent setattr, so they clear_initializingin a localfinally.This preserves the invariant the fast path needs:
_initializing == Falseimplies the module is reachable viagetattr(parent_module, child).Test
Lib/test/test_importlib/test_threaded_import.py::ThreadedImportTests::test_lazy_submodule_getattr_no_recursionwidens the natural microsecond race window with onethreading.Eventand verifies that an observer thread does not recurse on the lazy__getattr__. Fails on the unpatched interpreter, passes on this branch. Fulltest_importlibsuite remains green (1220/1220).Notes
__getattr__#149728