importlib: later import hangs forever after a RecursionError in a small-stack thread leaves a module lock held #159652

If an import in a thread with a small stack fails with RecursionError and the exception is kept alive, the module lock it held is never released, and a later import of the same module in another thread blocks forever in _ModuleLock.acquire.

import threading
threading.stack_size(131072)
errors = []
def worker():
    try:
        import _colorize
    except RecursionError as e:
        errors.append(e)
t = threading.Thread(target=worker)
t.start(); t.join()
print(errors, flush=True)
import collections.abc
print("done")

timeout 20 python reduced.py

Release build:

[RecursionError('Stack overflow (used 99 kB)')]

It then hangs and is killed by timeout (exit 124); done is never printed. In a variant that ends with an uncaught exception instead, so that the import that hangs is import collections.abc at line 3 of Lib/traceback.py, a faulthandler dump at 3646ea5 shows the main thread in _ModuleLock.acquire (<frozen importlib._bootstrap> line 359), called from __enter__ (line 452) and _find_and_load.

I expected import collections.abc to succeed, since the failed import should have released its locks. Without errors.append(e) the lock objects are garbage collected and nothing hangs.

Found at commit 3646ea5; still reproduces at e5ef47e (the release build hangs the same way). The checking build (assertions and ASAN) does not hang here: at this stack size it aborts first with Parser/pegen.h:120: _PyPegen_stack_exhausted: Assertion `!_Py_ReachedRecursionLimitWithMargin(p->tstate, 1)' failed (rc 134). The stack size that triggers the hang depends on the build; on the release build I saw it for sizes from about 124928 to 188416.

This looks like _HierarchicalLockManager.__enter__ in Lib/importlib/_bootstrap.py: a lock taken by lock.acquire() is only appended to self._locks afterwards, so a RecursionError in that window (or in the cleanup release() calls) leaves it owned by the dead thread. I checked the leak itself (after the thread exits, importlib._bootstrap._module_locks still holds the collections.abc lock, owned by the dead thread); the exact line is a guess. PRs #153575 and #157413 change parser recursion depth and may be related; I did not test them, as their diffs do not apply on the latest commit.

I can send a patch if that helps.

Found with an LLM-based testing tool; I used Claude to reduce it and look for the cause.

Linked PRs

• gh-159652: Fix import lock leak when the C stack is nearly exhausted #159660