Skip to content

bpo-44962 fix a race in WeakKeyDict, WeakValueDict and WeakSet when two threads attempt to commit the last pending removal #27921

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
ambv merged 10 commits into from
Aug 28, 2021
Merged

bpo-44962 fix a race in WeakKeyDict, WeakValueDict and WeakSet when two threads attempt to commit the last pending removal #27921

Changes from 1 commit
Commits
Show all changes
10 commits
Select commit Hold shift + click to select a range
5a0dc76
bpo-44962 fix a race in WeakSet
graingert Aug 23, 2021
3f74f24
📜🤖 Added by blurb_it.
blurb-it[bot] Aug 23, 2021
08f31cb
Update Misc/NEWS.d/next/Core and Builtins/2021-08-23-19-55-08.bpo-449…
graingert Aug 24, 2021
738dc9a
fix races in WeakKeyDict and WeakValueDict
graingert Aug 24, 2021
f3585ef
catch KeyError in WeakKeyDict callback
graingert Aug 24, 2021
6476722
Update Misc/NEWS.d/next/Core and Builtins/2021-08-23-19-55-08.bpo-449…
graingert Aug 24, 2021
36fd3f0
Update Misc/NEWS.d/next/Core and Builtins/2021-08-23-19-55-08.bpo-449…
graingert Aug 24, 2021
54a0032
Update Lib/weakref.py
graingert Aug 28, 2021
aada0e4
cache _atomic_removal in WeakValueDictionary._commit_removals
graingert Aug 28, 2021
b2b25d0
remove redundant else clauses
graingert Aug 28, 2021
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
fix races in WeakKeyDict and WeakValueDict
  • Loading branch information
@graingert
graingert committed Aug 24, 2021
commit 738dc9acf73d8721702b9300019b06eb7223259f
27 changes: 18 additions & 9 deletions Lib/weakref.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -120,13 +120,17 @@ def remove(wr, selfref=ref(self), _atomic_removal=_remove_dead_weakref):
self.update(other, **kw)

def _commit_removals(self):
l = self._pending_removals
pop = self._pending_removals.pop
Comment thread
graingert marked this conversation as resolved.
d = self.data
# We shouldn't encounter any KeyError, because this method should
# always be called *before* mutating the dict.
while l:
key = l.pop()
_remove_dead_weakref(d, key)
while True:
try:
key = pop()
except IndexError:
return
else:
_remove_dead_weakref(d, key)
Comment thread
ambv marked this conversation as resolved.
Outdated
Comment thread
graingert marked this conversation as resolved.
Outdated

def __getitem__(self, key):
if self._pending_removals:
Expand Down Expand Up @@ -384,13 +388,18 @@ def _commit_removals(self):
# because a dead weakref never compares equal to a live weakref,
# even if they happened to refer to equal objects.
# However, it means keys may already have been removed.
l = self._pending_removals
pop = self._pending_removals.pop
d = self.data
while l:
while True:
try:
del d[l.pop()]
except KeyError:
pass
key = pop()
except IndexError:
return
else:
try:
del d[key]
except KeyError:
pass
Comment thread
graingert marked this conversation as resolved.
Outdated

def _scrub_removals(self):
d = self.data
Expand Down