bpo-35615: Fix crashes when copying a Weak{Key,Value}Dictionary.#11384
Merged
Conversation
Protect dict iterations by wrapping them with _IterationGuard in the following methods: - WeakValueDictionary.copy() - WeakValueDictionary.__deepcopy__() - WeakKeyDictionary.copy() - WeakKeyDictionary.__deepcopy__()
|
Hello, and thanks for your contribution! I'm a bot set up to make sure that the project can legally accept your contribution by verifying you have signed the PSF contributor agreement (CLA). Unfortunately we couldn't find an account corresponding to your GitHub username on bugs.python.org (b.p.o) to verify you have signed the CLA (this might be simply due to a missing "GitHub Name" entry in your b.p.o account settings). This is necessary for legal reasons before we can look at your contribution. Please follow the steps outlined in the CPython devguide to rectify this issue. You can check yourself to see if the CLA has been received. Thanks again for your contribution, we look forward to reviewing it! |
pitrou
approved these changes
Dec 31, 2018
pitrou
left a comment
Member
There was a problem hiding this comment.
LGTM. We might want to add a test (see check_weak_destroy_and_mutate_while_iterating in test_weakref.py).
vstinner
reviewed
Jan 4, 2019
vstinner
left a comment
Member
There was a problem hiding this comment.
Is it possible to write a test reproducing the bug? (to then make sure that the fix works as expected)
Contributor
Author
As much as I want, I couldn't write a reliable test case in a standalone Python file to trigger this bug. I could, however, reliably trigger it when debugging my multithreaded PySide2 application. We also happened to trigger it once on our CI after hundreds of runs. I'm open to ideas of how to write a reliable test case to trigger this bug! |
Member
|
I explained above where to look for and how to add a test. Please take a look. Edit: ok, the challenge here is to execute arbitrary code during the |
Contributor
Author
|
@pitrou I took a look at |
Member
|
@ltfish You're right, I was too optimistic. I think we're left to using two threads: one that heavily copies a very large weakdict, another that loses references to the weakdict keys' or values', one at a time, in more or less random order. |
Contributor
Author
|
@pitrou Thanks. Lemme give it a try! |
a8af3a7 to
bfaeb71
Compare
bfaeb71 to
e2b9d24
Compare
Contributor
Author
|
On master branch: With this PR: |
Contributor
Author
|
Hi there, is there any update on this PR? |
Member
|
Sorry @ltfish. I'm gonna review soon. |
Contributor
Author
Please see my previous reply. Without the fix, my newly added test case fails reliably on my machine. I can also open a PR without my fix if needed (to make sure the test case works as expected on the CI). |
rhelmot
added a commit
to angr/angr
that referenced
this pull request
Feb 5, 2019
rhelmot
added a commit
to angr/angr
that referenced
this pull request
Feb 5, 2019
Contributor
Member
|
Thanks @ltfish ! |
miss-islington
pushed a commit
to miss-islington/cpython
that referenced
this pull request
Feb 7, 2019
…honGH-11384) Protect dict iterations by wrapping them with _IterationGuard in the following methods: - WeakValueDictionary.copy() - WeakValueDictionary.__deepcopy__() - WeakKeyDictionary.copy() - WeakKeyDictionary.__deepcopy__() (cherry picked from commit 96d37db) Co-authored-by: Fish <ltfish@users.noreply.github.com>
|
GH-11785 is a backport of this pull request to the 3.7 branch. |
pitrou
pushed a commit
that referenced
this pull request
Feb 7, 2019
…11384) (GH-11785) Protect dict iterations by wrapping them with _IterationGuard in the following methods: - WeakValueDictionary.copy() - WeakValueDictionary.__deepcopy__() - WeakKeyDictionary.copy() - WeakKeyDictionary.__deepcopy__() (cherry picked from commit 96d37db) Co-authored-by: Fish <ltfish@users.noreply.github.com>
Contributor
Author
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
7 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Protect dict iterations by wrapping them with _IterationGuard in the
following methods:
https://bugs.python.org/issue35615