Creating a dict item-iterator (iter(d.items()) or reversed(d.items())) while
allocations are failing aborts a debug build with:
Objects/dictobject.c:5532: _PyObject_GC_UNTRACK:
Assertion "_PyObject_GC_IS_TRACKED(op)" failed:
object not tracked by the garbage collector
object type name: dict_itemiterator
Fatal Python error: _PyObject_AssertFailed
dictiter_new() allocates the iterator with PyObject_GC_New() but only calls
_PyObject_GC_TRACK(di) at the very end. For item iterators it first allocates
di->di_result via _PyTuple_FromPairSteal(). If that allocation fails (OOM), the
error path runs Py_DECREF(di) on the still-untracked iterator, and
dictiter_dealloc() unconditionally calls _PyObject_GC_UNTRACK(di).
- Debug build: the
_PyObject_GC_IS_TRACKED assertion fires → Py_FatalError / abort.
- Release build (NDEBUG): the assert is compiled out and untracking a never-tracked
object corrupts the GC list, leading to a later segfault in _Py_Dealloc.
Only item iterators are affected (dict_itemiterator / dict_reverseitemiterator) —
they are the only dict iterators whose di_result requires an allocation between
PyObject_GC_New and _PyObject_GC_TRACK.
This was found via allocation-failure fuzzing as OOM-0006 in the umbrella issue
#151763. Filing a focused issue per that issue's process.
Reproducer
Minimal, stdlib-only, deterministic (requires a --with-pydebug interpreter so
_testcapi.set_nomemory is available):
import faulthandler, _strptime
faulthandler.enable()
from _testcapi import set_nomemory
for start in range(60):
set_nomemory(start)
try:
_strptime._strptime("", "")
except BaseException:
pass
print("done, no crash")
_strptime._strptime("", "") performs an iter(d.items()) once the size-2 tuple
freelist is drained, so _PyTuple_FromPairSteal() reaches the failing allocator and
drives the dictiter_new() OOM error path. (Reducer by @devdanzin; gist:
https://gist.github.com/devdanzin/c809eb4072c0c787c0c890f54ba1c843)
Backtrace (abbreviated)
_PyObject_AssertFailed Objects/object.c
_PyObject_GC_UNTRACK Include/internal/pycore_gc.h:254
dictiter_dealloc (self=dict_itemiterator) Objects/dictobject.c:5532
_Py_Dealloc Objects/object.c
Py_DECREF Include/refcount.h
dictiter_new (itertype=PyDictIterItem_Type) Objects/dictobject.c <- Py_DECREF(di) after di_result alloc fails
PyObject_GetIter Objects/abstract.c
Suggested fix
Either guard the untracked against the never-tracked error path:
if (_PyObject_GC_IS_TRACKED(di)) {
_PyObject_GC_UNTRACK(di);
}
or move _PyObject_GC_TRACK(di) ahead of the fallible di_result allocation (after
initializing di->di_result = NULL so traverse/dealloc stay safe). A PR with the first
approach and a regression test is ready.
CPython versions tested on:
CPython main branch
Operating systems tested on:
macOS
Output from running 'python -VV' on the command line:
No response
Linked PRs
Creating a dict item-iterator (
iter(d.items())orreversed(d.items())) whileallocations are failing aborts a debug build with:
dictiter_new()allocates the iterator withPyObject_GC_New()but only calls_PyObject_GC_TRACK(di)at the very end. For item iterators it first allocatesdi->di_resultvia_PyTuple_FromPairSteal(). If that allocation fails (OOM), theerror path runs
Py_DECREF(di)on the still-untracked iterator, anddictiter_dealloc()unconditionally calls_PyObject_GC_UNTRACK(di)._PyObject_GC_IS_TRACKEDassertion fires →Py_FatalError/ abort.object corrupts the GC list, leading to a later segfault in
_Py_Dealloc.Only item iterators are affected (
dict_itemiterator/dict_reverseitemiterator) —they are the only dict iterators whose
di_resultrequires an allocation betweenPyObject_GC_Newand_PyObject_GC_TRACK.This was found via allocation-failure fuzzing as OOM-0006 in the umbrella issue
#151763. Filing a focused issue per that issue's process.
Reproducer
Minimal, stdlib-only, deterministic (requires a
--with-pydebuginterpreter so_testcapi.set_nomemoryis available):_strptime._strptime("", "")performs aniter(d.items())once the size-2 tuplefreelist is drained, so
_PyTuple_FromPairSteal()reaches the failing allocator anddrives the
dictiter_new()OOM error path. (Reducer by @devdanzin; gist:https://gist.github.com/devdanzin/c809eb4072c0c787c0c890f54ba1c843)
Backtrace (abbreviated)
Suggested fix
Either guard the untracked against the never-tracked error path:
or move
_PyObject_GC_TRACK(di)ahead of the fallibledi_resultallocation (afterinitializing
di->di_result = NULLso traverse/dealloc stay safe). A PR with the firstapproach and a regression test is ready.
CPython versions tested on:
CPython main branch
Operating systems tested on:
macOS
Output from running 'python -VV' on the command line:
No response
Linked PRs