Skip to content

gh-152754: Fix crash when an os.scandir iterator is shared between threads#153462

Draft
nascheme wants to merge 3 commits into
python:mainfrom
nascheme:gh-152754-os-race-mutex
Draft

gh-152754: Fix crash when an os.scandir iterator is shared between threads#153462
nascheme wants to merge 3 commits into
python:mainfrom
nascheme:gh-152754-os-race-mutex

Conversation

@nascheme

@nascheme nascheme commented Jul 9, 2026

Copy link
Copy Markdown
Member

This is revision of GH-153402.

The issue was a race between ScandirIterator_iternext() and ScandirIterator.close(): one thread could be inside readdir() / FindNextFileW() while another thread closed the underlying DIR * / Windows handle.

Changes

  • Add a free-threaded-only mutex to ScandirIterator.
  • Use it to serialize access to the native directory stream / handle.
  • Keep Py_BEGIN_ALLOW_THREADS around readdir(), FindNextFileW(), closedir(), and FindClose(), preserving existing GIL-enabled behavior around potentially slow OS calls.
  • Make the mutex and lock/unlock operations no-ops in GIL-enabled builds.
  • Avoid holding the mutex while creating Python objects or setting Python exceptions.
  • Add free-threading regression tests for concurrent access patterns that previously could crash.

Notes

This change is intended to prevent crashes and memory corruption. It does not document or guarantee os.scandir() iterators as generally thread-safe.

A PyMutex is used rather than critical sections because critical sections may be suspended across Py_BEGIN_ALLOW_THREADS; the native iterator must remain protected while the OS directory iteration call runs without the GIL.

…een threads

Co-authored-by: Neil Schemenauer <nas-github@arctrix.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants