Skip to content

gh-115999: Make list and tuple iteration more thread-safe. #128637

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
Yhg1s merged 17 commits into from
Feb 19, 2025
+177 −18
Merged

gh-115999: Make list and tuple iteration more thread-safe. #128637

Changes from 1 commit
Commits
Show all changes
17 commits
Select commit Hold shift + click to select a range
1533d1d
Make list, tuple and range iteration more thread-safe, and actually test
Yhg1s Jan 8, 2025
a069f9b
Fix lint issues.
Yhg1s Jan 8, 2025
661dc7b
Restore the original guard for list_get_item_ref's fast path, since it's
Yhg1s Jan 9, 2025
7263d1e
Merge branch 'main' into list-realloc
Yhg1s Jan 9, 2025
484c6b2
Fix test failures in test_sys because of the changed size of range
Yhg1s Jan 9, 2025
89c4629
Merge branch 'list-realloc' of github.com:Yhg1s/cpython into list-rea…
Yhg1s Jan 9, 2025
daacf82
Fix tupleiter_reduce for test_iter's assumptions, and make tupleiter_…
Yhg1s Jan 10, 2025
94ab0c6
Undo the range iterator changes (leave them for later), adjust the range
Yhg1s Jan 12, 2025
c6b9200
Remove edit snafu.
Yhg1s Jan 12, 2025
ed7e8c7
Merge branch 'main' into list-realloc
Yhg1s Jan 12, 2025
dd56d89
Extend tests a little, add suppressions with links to GH issues.
Yhg1s Jan 20, 2025
4028dfe
Address reviewer comments.
Yhg1s Jan 20, 2025
ef5ca7c
Turn the last few stores/loads of list iterator/reversed iterator into
Yhg1s Jan 21, 2025
e1dd5f8
Merge branch 'main' into list-realloc
Yhg1s Jan 21, 2025
f689a8a
Address reviewer comments.
Yhg1s Jan 26, 2025
2923f6a
Merge branch 'list-realloc' of github.com:Yhg1s/cpython into list-rea…
Yhg1s Jan 26, 2025
08f0647
Merge branch 'main' into list-realloc
Yhg1s Feb 18, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
Turn the last few stores/loads of list iterator/reversed iterator into 
relaxed atomics. (These are not performance-sensitive parts of the code, and
this lets us detect races elsewhere involving the iterators.)
  • Loading branch information
@Yhg1s
Yhg1s committed Jan 21, 2025
commit ef5ca7c029454a41e5a343cf3243f3ca27b1acbc
15 changes: 8 additions & 7 deletions Objects/listobject.c
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -4000,7 +4000,7 @@ listiter_setstate(PyObject *self, PyObject *state)
index = -1;
else if (index > PyList_GET_SIZE(it->it_seq))
index = PyList_GET_SIZE(it->it_seq); /* iterator exhausted */
it->it_index = index;
FT_ATOMIC_STORE_SSIZE_RELAXED(it->it_index, index);
}
Py_RETURN_NONE;
}
Expand Down Expand Up @@ -4152,7 +4152,7 @@ listreviter_setstate(PyObject *self, PyObject *state)
index = -1;
else if (index > PyList_GET_SIZE(it->it_seq) - 1)
index = PyList_GET_SIZE(it->it_seq) - 1;
it->it_index = index;
FT_ATOMIC_STORE_SSIZE_RELAXED(it->it_index, index);
}
Py_RETURN_NONE;
}
Expand All @@ -4169,18 +4169,19 @@ listiter_reduce_general(void *_it, int forward)
* call must be before access of iterator pointers.
* see issue #101765 */

/* the objects are not the same, index is of different types! */
if (forward) {
iter = _PyEval_GetBuiltin(&_Py_ID(iter));
_PyListIterObject *it = (_PyListIterObject *)_it;
if (it->it_index >= 0) {
return Py_BuildValue("N(O)n", iter, it->it_seq, it->it_index);
Py_ssize_t idx = FT_ATOMIC_LOAD_SSIZE_RELAXED(it->it_index);
if (idx >= 0) {
return Py_BuildValue("N(O)n", iter, it->it_seq, idx);
}
} else {
iter = _PyEval_GetBuiltin(&_Py_ID(reversed));
listreviterobject *it = (listreviterobject *)_it;
if (it->it_index >= 0) {
return Py_BuildValue("N(O)n", iter, it->it_seq, it->it_index);
Py_ssize_t idx = FT_ATOMIC_LOAD_SSIZE_RELAXED(it->it_index);
if (idx >= 0) {
return Py_BuildValue("N(O)n", iter, it->it_seq, idx);
}
}
/* empty iterator, create an empty list */
Expand Down