gh-151640: avoid sharing BytesIO buffer in free-threaded builds#151651
gh-151640: avoid sharing BytesIO buffer in free-threaded builds#151651pedramkarimii wants to merge 3 commits into
Conversation
|
Most changes to Python require a NEWS entry. Add one using the blurb_it web app or the blurb command-line tool. If this change has little impact on Python users, wait for a maintainer to apply the |
|
All commit authors signed the Contributor License Agreement. |
|
Most changes to Python require a NEWS entry. Add one using the blurb_it web app or the blurb command-line tool. If this change has little impact on Python users, wait for a maintainer to apply the |
|
Additional context: This PR is intended to fix the specific While investigating the code path, I also found that The chosen fix is intentionally conservative: in free-threaded builds, avoid returning the internal I also considered fixing this lower in the resize/shared-buffer ownership path, but that seemed more invasive because it would require changing the interaction between |
Fixes gh-151640.
In free-threaded builds,
BytesIO.read()could returnself->bufdirectly for whole-buffer reads viaPy_NewRef().BytesIO.getvalue()could also return the internal buffer directly. A concurrent writer may then resize the same internal bytes object while another thread decrefs the returned reference, producing a TSAN-reported race between_PyBytes_Resize()and_Py_DecRefShared().This change avoids exposing the internal
BytesIObuffer inPy_GIL_DISABLEDbuilds. Whole-bufferread()andgetvalue()now return a copy in free-threaded builds, while keeping the existing fast path unchanged for regular GIL builds.A focused free-threading regression test was added for concurrent whole-buffer
read()/getvalue()and buffer-resizing writes.Tests run:
./python -m test test_free_threading.test_io -v -m test_concurrent_whole_buffer_read_and_resize./python -m test test_free_threading.test_io -v./python -m test test_io -v./python -m test test_free_threading -v./python ../gh151640_reproducer.py