Repository navigation
gc.get_objects can corrupt in-progress GC in free threading build #125859
Description
Bug report
Background
The free threading GC uses two queue-like data structures to keep track of objects:
-
struct worklist, which is a singly linked list that repurposesob_tidfor the linked list pointer -
_PyObjectStack, which is effectively a dynamically sized array ofPyObject*. (Implemented using a linked list of fixed size array buffers).
The struct worklist data structure is convenient because enqueueing objects doesn't require a memory allocation and so can't fail. However, an object can only be part of one "worklist" at a time, because each object has only one ob_tid field.
Bug
Other threads can run while the GC is finalizing cyclic garbage and while it's calling tp_clear() and other clean-up.
During that time, some thread may call gc.get_objects(), which can return otherwise "unreachable" objects. The implementation of _PyGC_GetObjects temporarily pushes objects to a struct worklist, including objects that might already be part of some other worklist, overwriting the linked list pointer. This essentially corrupts the state of the in-progress GC and causes assertion failures.
Proposed fix
- We should probably exclude objects in the "unreachable" (i.e.
_PyGC_BITS_UNREACHABLE) from being returned bygc.get_objects() - We should limit the use of
struct worklistto the actual GC and use_PyObjectStack(or some other data structure) in_PyGC_GetObjects(). This reduces the risk of bugs causing an object to be added to more than one worklist.
Activity
rruuaanng commentedon Oct 23, 2024
Hi @colesbury, if there is no objection, I would like to take over this task. I will submit a PR in the next few days. I will take plan
We should probably exclude objects in the "unreachable" (i.e. _PyGC_BITS_UNREACHABLE) from being returned by gc.get_objects().I think using plan
We should limit the use of struct worklist to the actual GC and use _PyObjectStack (or some other data structure) in _PyGC_GetObjects(). This reduces the risk of bugs causing an object to be added to more than one worklist.may cause the memory usage to become higher. So I choose a simple conditional return
colesbury commentedon Oct 23, 2024
@rruuaanng - thank you for the offer, but I've already started working on this
pythongh-125859: Fix crash when `gc.get_objects` is called during GC
gh-125859: Fix crash when `gc.get_objects` is called during GC (#125882)
pythongh-125859: Fix crash when `gc.get_objects` is called during GC (p…
[3.13] gh-125859: Fix crash when `gc.get_objects` is called during GC (…
pythongh-125859: Fix crash when `gc.get_objects` is called during GC (p…