Bug report
Script bug.py (from issue gh-149146, but modified):
import resource
import signal
import os
def segfault_handler(signum, frame):
os.abort()
raise MemoryError
#signal.signal(signal.SIGSEGV, segfault_handler)
# deallocation the chain uses a lot of C stack memory
CHAIN_LENGTH = 1024 * 1024
# limit process memory to not crash the whole machine
max_memory = 1024 * 1024 * 1024
resource.setrlimit(resource.RLIMIT_AS, (max_memory, max_memory))
print("create long chain...")
long_chain = None
for _ in range(CHAIN_LENGTH):
long_chain = (long_chain, None)
print("fill the process memory...")
memory = []
try:
large = 1024 * 1024
while True:
memory.append(b'x' * large)
except MemoryError:
pass
try:
small = 1024
os.uname()
while True:
memory.append(b'x' * small)
except MemoryError:
pass
print("delete the long chain...")
long_chain = None
# at exit (on error), deallocating long_chain will use a lot of C stack memory
# ... and does crash!Sometimes, running this script triggers a bug when memory.append(b'x' * small) raises a MemoryError. Sadly, the reproducer is very fragile. So far, I failed to write a more reliable reproducer.
The fix is to set stack_pointer[-1] to PyStackRef_NULL in _CALL_LIST_APPEND before calling JUMP_TO_LABEL(error);.
Steps:
- PyErr_NoMemory() is called by _PyList_AppendTakeRefListResize() in
_CALL_LIST_APPEND opcode
- At this point,
stack_pointer[-1] (arg) points to freed memory (all bytes are set to 0xdd)
_PyList_AppendTakeRef() returns non-zero result, so _CALL_LIST_APPEND opcode goes to JUMP_TO_LABEL(error);- The
error label calls JUMP_TO_LABEL(exception_unwind);
exception_unwind label clears the stack: call ref = _PyFrame_StackPop() and PyStackRef_XCLOSE(ref) until the stack is emptyexception_unwind: PyStackRef_XCLOSE(ref) fails with an assertion error (Py_DECREF_MORTAL: Assertion !_Py_IsStaticImmortal(op)') because stack_pointer[-1]` points to freed memory
$ gdb -args ./python bug.py
(gdb) b os_uname
(gdb) run
Breakpoint 2, os_uname (...)
(gdb) b PyErr_NoMemory
(gdb) c
Continuing.
Breakpoint 3, PyErr_NoMemory () at Python/errors.c:803
803 {
(gdb) n
(...)
(gdb)
_PyList_AppendTakeRefListResize (...)
531 Py_DECREF(newitem);
(gdb)
532 return -1;
(gdb)
_PyEval_EvalFrameDefault (tstate=<optimized out>, frame=0x7ffff7fb0020, throwflag=0) at Python/generated_cases.c.h:3983
3983 if (err) {
(gdb) p stack_pointer[-1]
$2 = <unknown at remote 0xcc08e0>
(gdb) p /x *(PyObject*)stack_pointer[-1]
$3 = {
{
ob_refcnt_full = 0xdddddddddddddddd,
{
ob_refcnt = 0xdddddddd,
ob_overflow = 0xdddd,
ob_flags = 0xdddd
},
_aligner = 0xdd
},
ob_type = 0xdddddddddddddddd
}
(gdb) n
3984 JUMP_TO_LABEL(error);
(gdb) n
(...)
(gdb)
13843 JUMP_TO_LABEL(exception_unwind);
(gdb)
13849 int offset = INSTR_OFFSET()-1;
(...)
(gdb)
13865 while (frame->stackpointer > new_top) {
(gdb)
13866 _PyStackRef ref = _PyFrame_StackPop(frame);
(gdb)
13867 PyStackRef_XCLOSE(ref);
(gdb) p ref
$5 = <unknown at remote 0xcc08e0>
(gdb) p /x *(PyObject*)ref
$6 = {
{
ob_refcnt_full = 0xdddddddddddddddd,
{
ob_refcnt = 0xdddddddd,
ob_overflow = 0xdddd,
ob_flags = 0xdddd
},
_aligner = 0xdd
},
ob_type = 0xdddddddddddddddd
}
(gdb) n
python: ./Include/internal/pycore_object.h:414: Py_DECREF_MORTAL: Assertion `!_Py_IsStaticImmortal(op)' failed.
Program received signal SIGABRT, Aborted.
0x00007ffff7cfedcc in __pthread_kill_implementation () from /lib64/libc.so.6
Linked PRs
Bug report
Script
bug.py(from issue gh-149146, but modified):Sometimes, running this script triggers a bug when
memory.append(b'x' * small)raises aMemoryError. Sadly, the reproducer is very fragile. So far, I failed to write a more reliable reproducer.The fix is to set
stack_pointer[-1]toPyStackRef_NULLin_CALL_LIST_APPENDbefore callingJUMP_TO_LABEL(error);.Steps:
_CALL_LIST_APPENDopcodestack_pointer[-1](arg) points to freed memory (all bytes are set to0xdd)_PyList_AppendTakeRef()returns non-zero result, so_CALL_LIST_APPENDopcode goes toJUMP_TO_LABEL(error);errorlabel callsJUMP_TO_LABEL(exception_unwind);exception_unwindlabel clears the stack: callref = _PyFrame_StackPop()andPyStackRef_XCLOSE(ref)until the stack is emptyexception_unwind:PyStackRef_XCLOSE(ref)fails with an assertion error (Py_DECREF_MORTAL: Assertion!_Py_IsStaticImmortal(op)') becausestack_pointer[-1]` points to freed memoryLinked PRs