Skip to content

gh-159558: Avoid un-needed nested locks in set operations - #159559

Open
johng wants to merge 10 commits into
python:mainfrom
johng:ft-cs-reentry
Open

johng wants to merge 10 commits into
python:mainfrom
johng:ft-cs-reentry

Conversation

@johng

@johng johng commented Oct 16, 2026 •

Copy link
Copy Markdown
Contributor

These are fixed by adding a lock free version internal function to give caller the option of calling with a lock or not.

johng added 5 commits October 16, 2026 17:47
…uild

`set_issuperset_impl()` is already locked but calls `set_issubset` which
just wraps a lock to `set_issubset_impl`

(cherry picked from commit d4e412b)
…e-threaded build

set_lookkey() is shared by callers that run without the set's lock and by
callers that already hold it, so it takes the critical section on a
mutable set itself.  set.discard() and set.remove() hold that lock, taken
by their Argument Clinic wrappers, and reached set_lookkey() through
set_discard_entry(), so every call re-entered the critical section and
went through _PyCriticalSection_BeginSlow().

Split set_lookkey() into a lock-held core and a locking wrapper, and use
the core from set_discard_entry(): all of its callers (set.discard,
set.remove, PySet_Discard and the difference-update helpers) hold the
lock, and the helpers already assert it.

Measured on the free-threaded build: s.discard(1); s.add(1) 34 -> 23 ns.
…ree-threaded build

KeepRef() takes the critical section on its target so that it is safe
from any caller, and for a type that keeps a reference to the stored
object it then calls PyCData_GetContainer(), which takes the critical
section on the root object.  The Simple.value setter already holds the
lock on self, taken by its Argument Clinic wrapper, and for a top-level
object the root is self as well, so `c_char_p().value = b"x"` entered the
same critical section three times and `c_int().value = 1` twice (the
"nothing to keep" early return sits inside the locked region).

Add PyCData_GetContainer_lock_held() for a caller that holds the root's
lock, use it from KeepRef_lock_held() when the target is its own root,
and call KeepRef_lock_held() from the setter.

Measured on the free-threaded build: c_int().value = 1 35 -> 22 ns,
c_char_p().value = b"x" 39 -> 22 ns.
…nts setter

The setter runs under a two-object critical section on the pointer and
the new target, then calls KeepRef() twice on the pointer and
GetKeepedObjects() on the target; both helpers lock their argument
again.  Call the lock-held variants instead, adding
GetKeepedObjects_lock_held() for the target.

Measured on the free-threaded build: p.contents = c_int(2) 86 -> 58 ns.
PyCData_set() already requires its caller to hold the critical section
on the destination object and asserts it, but finished by calling
KeepRef(), which takes that lock again.  Every store of a Python object
into a Structure or Union field, an Array item or a Pointer item went
through it.  Call KeepRef_lock_held() instead.

Measured on the free-threaded build: Structure int field assignment
38 -> 24 ns, c_char_p field assignment 69 -> 54 ns.
@picnixz

picnixz commented Oct 16, 2026

Copy link
Copy Markdown
Member

Sorry for saying one PR for all. Let's have one PR for the set operations and one PR for the ctypes. Keep this one for either of the two. The ctypes change can be reviewed separately that way.

@johng johng changed the title gh-159558: Avoid various un-needed nested locks gh-159558: Avoid un-needed nested locks in set operations Oct 16, 2026
@bedevere-app bedevere-app Bot added the type-feature A feature request or enhancement label Oct 16, 2026
johng added 2 commits October 16, 2026 18:22
Revert the three ctypes commits and drop their NEWS entry so that this PR
only changes set; the ctypes part is resubmitted on its own.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

awaiting review type-feature A feature request or enhancement

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants