Updates Tz Singletons and Caches with WeakValueDictionarys#672
Updates Tz Singletons and Caches with WeakValueDictionarys#672pganssle merged 6 commits intodateutil:masterfrom cs-cordero:weakrefs
WeakValueDictionarys#672Conversation
|
@pganssle - I'm trying to figure out how my pytest failed on the AppVeyor CI. It failed on a line that asserts that the weak references are cleared immediately following a couple Any ideas how this could happen? Is it possible that the |
| from datetime import datetime, time | ||
|
|
||
|
|
||
| class FakeWeakRef: |
There was a problem hiding this comment.
.utils is a public module and very much not the right place for this. It should probably be in tz._common.py
| from datetime import timedelta | ||
|
|
||
| try: | ||
| import weakref |
There was a problem hiding this comment.
I would change this to:
try:
from weakref import WeakRefDictionary
except ImportError:
WeakRefDictionary = dictThere was a problem hiding this comment.
Actually it turns out that Python 2 has weak references, so we can just drop the whole compatibility shim. My bad.
| from functools import partial | ||
|
|
||
| IS_WIN = sys.platform.startswith('win') | ||
| NOT_PY3 = not sys.version_info[0] == 3 |
There was a problem hiding this comment.
Normally you would not have to define this. There's already a six.PY3 and six.PY2 that will give you that.
That said, I may have been wrong about weak references being a Python 3 thing. Seems like they are available in Python 2.7, so we should drop this skip test.
|
Hard to tell if this is just holding a strong reference somewhere we don't realize or if the assumptions about one of the inputs is wrong. Seems to only happen on Windows on old versions of Python, though. |
|
Sorry for the flurry of seemingly random commits. I can't seem to replicate AppVeyor's failure on my local machine even after matching the Python version, so I'm trying to troubleshoot with small changes. |
|
Just reverted all the troubleshooting commits, but I'm kind of at a loss here for how this is failing on the AppVeyor CI. The stdout from the 46bdab7 commit is really what's really tripping me up: In this commit, I print the keys inside the I'll keep thinking about how to get around this. One idea I had was to have some kind of |
|
I'm guessing that this is a real issue with Windows and not a problem with the tests. I'll grab this branch and try debugging it on a Windows VM. |
|
Ah, I figured this out. It's because |
|
Cleaned up the commit history. Will merge when tests pass. Thanks so much @cs-cordero! |
|
Happy to help. Thanks for taking it home with that clutch Windows |
|
Hm.. The Python 3.5 test failed but succeeded on reboot. This "id won't get reused" gambit may be less stable than I hoped. I just had a new idea for a more robust test, though. Create a new weak reference to the object before deleting it and make sure its referrent is gone after the gc step. |
|
@cs-cordero One last thing, can you add yourself to |
|
Will do re: |
|
@cs-cordero No this test assumes that the If you do: # Checkout base of the branch
git checkout e6644a4
# Checkout the commit where the tests were added
git checkout 75ae199 -- dateutil/test/test_tz.py
pytest -k test_tz.pyYou'll see that these three tests fail, which means the problem - that the cache itself holds a strong reference to the cached time zones - is solved by this patch. |
|
Ah yes that makes perfect sense! Thank you for explaining. |
This is a more reliable mechanism to test that nothing continues to hold a strong reference to the cached values. I have also switched the tests over to using pytest-style bare functions, and marked them all as smoke tests, since this will not actually be a guaranteed behavior of the interface.
|
@cs-cordero You'll probably need to do a force pull when you edit this branch again, because I've rebased your branch and rewritten the history. If you don't know how to do that it's |
|
🎉 |
Closes #635.