Skip to content

gh-124397: Add threading.iter_locked #133908

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
eendebakpt wants to merge 13 commits into
base: main
Choose a base branch
from
Open

gh-124397: Add threading.iter_locked #133908

Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
13 commits
Select commit Hold shift + click to select a range
6313b3d
Add iter_locked
eendebakpt May 10, 2025
5ced0a9
cleanup
eendebakpt May 11, 2025
110332e
cleanup
eendebakpt May 11, 2025
b675631
📜🤖 Added by blurb_it.
blurb-it[bot] May 11, 2025
4406050
update docs
eendebakpt May 12, 2025
d30ace8
review comments
eendebakpt Jun 1, 2025
baec5bc
use explicit lock
eendebakpt Jun 1, 2025
e6ea285
Merge branch 'main' into iter_locked
eendebakpt Jun 1, 2025
7315c49
fix import
eendebakpt Jun 1, 2025
b7c9d77
cleanup after merge to main
eendebakpt Jun 1, 2025
bb98c52
reduce duration of test
eendebakpt Jun 1, 2025
946ee51
use recursive lock
eendebakpt Jul 9, 2025
7e3653a
Merge branch 'main' into iter_locked
eendebakpt Jul 9, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
review comments
  • Loading branch information
@eendebakpt
eendebakpt committed Jun 1, 2025
commit d30ace8e7746d3f6c3ed3c8e3cf19999fab8ad03
2 changes: 2 additions & 0 deletions Lib/test/test_free_threading/test_threading_iter_locked.py
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,3 +1,4 @@
import time
import unittest
from threading import Thread, Barrier, iter_locked
from test.support import threading_helper
Expand All @@ -15,6 +16,7 @@ def __iter__(self):

def __next__(self):
a = next(self.it)
time.sleep(0)
b = next(self.it)
Copy link
Copy Markdown
Member

@serhiy-storchaka serhiy-storchaka May 12, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Add time.sleep(0) (or with small non-zero argument) or Event.wait() with a timeout to force or significantly increase the chance of concurrent access.

return a, b

Expand Down
9 changes: 7 additions & 2 deletions Modules/_threadmodule.c
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -747,6 +747,7 @@ static PyType_Spec ThreadHandle_Type_spec = {
typedef struct {
PyObject_HEAD
PyObject *it;
PyMutex lock;
} iter_locked_object;

#define iter_locked_object_CAST(op) ((iter_locked_object *)(op))
Expand Down Expand Up @@ -774,6 +775,7 @@ _thread_iter_locked_impl(PyTypeObject *type, PyObject *iterable)
return NULL;
}
il->it = it;
il->lock = (PyMutex){0};

return (PyObject *)il;
}
Expand Down Expand Up @@ -804,14 +806,17 @@ iter_locked_next(PyObject *op)
iter_locked_object *lz = iter_locked_object_CAST(op);
PyObject *result = NULL;

Py_BEGIN_CRITICAL_SECTION(op); // lock on op or lz->it?
// we cannot use Py_BEGIN_CRITICAL_SECTION as it is not available in the normal build
PyMutex_Lock(&(lz->lock));

PyObject *it = lz->it;
result = PyIter_Next(it);
Copy link
Copy Markdown
Member

@ZeroIntensity ZeroIntensity Jun 1, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let's use PyIter_NextItem, PyIter_Next has a weird return value.

if (result == NULL) {
/* Note: StopIteration is already cleared by PyIter_Next() */
/* If PyErr_Occurred() we will also return NULL*/
}
Py_END_CRITICAL_SECTION();
PyMutex_Unlock(&(lz->lock));

return result;
}

Expand Down