Handle ReferenceError except while count rlock#1042
Conversation
4383
left a comment
There was a problem hiding this comment.
Hello,
Thanks for this well detailed PR, much appreciated.
It looks globally ok, I'd just suggest to add some logging here and there.
Also, I'd love to know if your initial scenario is more or less similar to the one described in #1032, so please can you double check this issue and let us know your point of view about the described problem and the solution you propose. Thanks in advance.
|
@4383 I've updated the application to log exceptions immediately upon occurrence. The logs are now more descriptive and should aid in debugging. Please review if this meets your requirements or if any further adjustments are needed. Feel free to reach out for any clarifications or changes. |
|
Thanks @psychomantys , will review it again early next week. |
|
Blocked on #1048 |
Codecov ReportAttention: Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## master #1042 +/- ##
======================================
- Coverage 56% 56% -1%
======================================
Files 89 89
Lines 9852 9868 +16
Branches 1645 1647 +2
======================================
- Hits 5570 5564 -6
- Misses 3916 3936 +20
- Partials 366 368 +2
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Sentry. 🚀 New features to boost your workflow:
|
|
The py312-asyncio job failure looks legit. Itamar, through #1047, made some changes related to that. I think we have to merge Itamar is PR first, and then yours. @psychomantys: Do you mind testing your change and your scenario by rebasing your branch over #1047? And then, if you do not mind, tell us what are your observations. |
|
@4383 No doubt about it I'm going to proceed with building and run the changes of branch #1047. Internally, I have already been using a modified branch which has been working flawlessly for me; however, when I run with this specific branch version and confirm that it works well, I will come back here to report my findings. |
When counting remaining_rlocks on patcher.py, there is a chance of get a object how is already collected by the GC. - Add a try block to get and make the correct handle of this count. A minor change, but can exist in some envs. - Improved error messaging when a ReferenceError occurs during rlock count adjustment.
|
Please find below a screenshot of the relevant log excerpt where the exception occurred, which supports the running flawless:
|
|
Hi @psychomantys . Even with the latest version of your patch, we're still getting the same error in Nova (ie: nova-compute looses reference to the SessionClient object when trying to establish an API connection to Neutron, with "AttributeError: 'SessionClient' object has no attribute 'endpoint_override'"). |
|
Hey @psychomantys, #1047 is now merged, please rebase your branch over the HEAD of master, and please resolve the conflict. I'm gonna go to prepare a new release, would be good to have your changes landed with, else we will prepare a new release once your patch will be merged. |
|
@4383 tonight i will be resolving the conflicts of the branch so before that we can proceed with the merge. @thomasgoirand Its a unusual behavior; it might be due to the GC running between threads. However, I cant imagine how my branch would have a positive or negative effect on this behavior. |
|
@psychomantys thank you |
681b26d to
0d75e92
Compare
|
@4383 You have successfully performed a rebase operation on the branch, aligning it with the |
4383
left a comment
There was a problem hiding this comment.
LGTM, thank for you contribution, much appreciated!
Handling
ReferenceError: weakly-referenced object no longer existsDescription
This pull request aims to address an uncommon issue where a
ReferenceError: weakly-referenced object no longer existsoccurs when accessing objects returned bygc.get_objects(). According to the Python documentation, this error is raised when attempting to access an object that has been garbage collected.Motivation and Context
The motivation for this change is a rare issue encountered during my development, where accessing objects returned by
gc.get_objects()could lead to aReferenceError. Although the condition to trigger this error might be difficult to reproduce, it is crucial to handle any exceptions to gracefully maintain the stability.Changes and Fixes
The proposed change involves adding a try-except block when counting remaining
RLocksin thepatcher.pyfile. This modification aims to catch potential instances where an object might already have been collected by the garbage collector (GC), thus preventing the raisedReferenceError.By wrapping this specific line of code within a try-except, we ensure that any such exceptions are handled gracefully, allowing our application to continue functioning without interruptions or crashes due to unhandled errors.
Testing and Validation
This change has been tested in controlled environments, albeit with a low frequency of occurrence. The added try-except block does not affect normal operations but provides an additional layer of protection against potential runtime errors.
Further testing across different systems and configurations is recommended to validate the effectiveness of this fix.
Impact and Risk Assessment
The impact of this change is minimal since it only affects specific lines of code within the
patcher.pyfile, and the default handling for exceptions does not alter normal application behavior.The risk associated with this change is low.