Skip to content

gh-153083: Defer GC tracking of an array to the end of construction.#153284

Open
corona10 wants to merge 2 commits into
python:mainfrom
corona10:gh-153083
Open

gh-153083: Defer GC tracking of an array to the end of construction.#153284
corona10 wants to merge 2 commits into
python:mainfrom
corona10:gh-153083

Conversation

@corona10

@corona10 corona10 commented Jul 7, 2026

Copy link
Copy Markdown
Member

@corona10 corona10 requested a review from vstinner July 7, 2026 16:49
@corona10 corona10 added needs backport to 3.13 bugs and security fixes needs backport to 3.14 bugs and security fixes needs backport to 3.15 pre-release feature fixes, bugs and security fixes topic-free-threading labels Jul 7, 2026
@corona10

corona10 commented Jul 7, 2026

Copy link
Copy Markdown
Member Author

TSAN Script

# TSAN_OPTIONS=halt_on_error=0 ./python.exe tsan_array.py
import gc, threading
from array import array

N = 4000
box = []
done = False

def reader():
    while not done:
        if box:
            try:
                len(box[0]); repr(box[0])
            except Exception:
                pass

def gen():
    for i in range(N):
        if i == 5 and not box:  # leak the half-built array
            box.extend(o for o in gc.get_objects()
                       if type(o) is array and 0 < len(o) < N)
        yield i

t = threading.Thread(target=reader)
t.start()
for _ in range(30):
    box.clear()
    array('q', gen())
done = True
t.join()
print("observed half-built:", bool(box))

AS-IS

SUMMARY: ThreadSanitizer: data race arraymodule.c:195 in array_resize
==================
==29994==ERROR: ThreadSanitizer: SEGV on unknown address 0x000104000100 (pc 0x00010601baec bp 0x00016c046010 sp 0x00016c045fd0 T5647275)
==29994==The signal is caused by a READ memory access.
    #0 __tsan_atomic64_load <null> (libclang_rt.tsan_osx_dynamic.dylib:arm64e+0x5faec)
    #1 _PyMem_MiFree obmalloc.c:298 (python.exe:arm64+0x100164ac8)
    #2 PyMem_Free obmalloc.c:1286 (python.exe:arm64+0x100166b48)
    #3 list_dealloc listobject.c:569 (python.exe:arm64+0x1000e8524)
    #4 _Py_Dealloc object.c:3319 (python.exe:arm64+0x10013ee18)
    #5 _Py_MergeZeroLocalRefcount object.c (python.exe:arm64+0x10013f074)
    #6 array_repr arraymodule.c:2633 (array.cpython-316t-darwin.so:arm64+0x536c)
    #7 PyObject_Repr object.c:784 (python.exe:arm64+0x100140264)
    #8 builtin_repr bltinmodule.c:2677 (python.exe:arm64+0x100286d28)
    #9 _PyEval_EvalFrameDefault generated_cases.c.h:2712 (python.exe:arm64+0x100292e58)
    #10 _PyEval_Vector ceval.c:2172 (python.exe:arm64+0x10028b578)
    #11 _PyFunction_Vectorcall call.c (python.exe:arm64+0x100090918)
    #12 _PyObject_VectorcallPrepend call.c:855 (python.exe:arm64+0x10009205c)
    #13 method_vectorcall classobject.c:55 (python.exe:arm64+0x10009536c)
    #14 context_run context.c:731 (python.exe:arm64+0x1002d761c)
    #15 method_vectorcall_FASTCALL_KEYWORDS descrobject.c:421 (python.exe:arm64+0x1000a785c)
    #16 PyObject_Vectorcall call.c:327 (python.exe:arm64+0x1000902f0)
    #17 _Py_VectorCallInstrumentation_StackRefSteal ceval.c:768 (python.exe:arm64+0x10028c058)
    #18 _PyEval_EvalFrameDefault generated_cases.c.h:1906 (python.exe:arm64+0x100290f88)
    #19 _PyEval_Vector ceval.c:2172 (python.exe:arm64+0x10028b578)
    #20 _PyFunction_Vectorcall call.c (python.exe:arm64+0x100090918)
    #21 _PyObject_VectorcallPrepend call.c:855 (python.exe:arm64+0x10009205c)
    #22 method_vectorcall classobject.c:55 (python.exe:arm64+0x10009536c)
    #23 _PyObject_Call call.c:348 (python.exe:arm64+0x100090598)
    #24 PyObject_Call call.c:373 (python.exe:arm64+0x10009060c)
    #25 thread_run _threadmodule.c:388 (python.exe:arm64+0x100444458)
    #26 pythread_wrapper thread_pthread.h:234 (python.exe:arm64+0x100383b00)
    #27 __tsan_thread_start_func <null> (libclang_rt.tsan_osx_dynamic.dylib:arm64e+0x337a0)
    #28 _pthread_start <null> (libsystem_pthread.dylib:arm64e+0x6c54)
    #29 thread_start <null> (libsystem_pthread.dylib:arm64e+0x1c18)

==29994==Register values:
 x[0] = 0x0000000105b24000   x[1] = 0x0000100208000200   x[2] = 0x00000000c0b414ff   x[3] = 0x0000000000000008
 x[4] = 0x0000000000000003   x[5] = 0x0000000000000000   x[6] = 0x0000000000000029   x[7] = 0x00000000e6477557
 x[8] = 0x0000000000000000   x[9] = 0x0000000000000014  x[10] = 0x00000000c0000000  x[11] = 0x0000000000000000
x[12] = 0x0000000105ab1ef0  x[13] = 0x0000100208000200  x[14] = 0x0000000000000003  x[15] = 0x0000000000001f8c
x[16] = 0x0000000000000000  x[17] = 0x0000000106070a40  x[18] = 0x0000000000000000  x[19] = 0x0000000104000100
x[20] = 0x0000000105b24000  x[21] = 0x0000000104fb0acc  x[22] = 0x0000000000000000  x[23] = 0x000000016c0470e0
x[24] = 0x0000000000000000  x[25] = 0x0000000000000000  x[26] = 0x00000003023644c0  x[27] = 0x0000000105f6c208
x[28] = 0x0000000105f6c1a8     fp = 0x000000016c046010     lr = 0x000000010601badc     sp = 0x000000016c045fd0
ThreadSanitizer can not provide additional info.
SUMMARY: ThreadSanitizer: SEGV obmalloc.c:298 in _PyMem_MiFree
==29994==ABORTING

TO-BE

observed half-built: False

@vstinner vstinner left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There are many methods still calling newarrayobject(). I would prefer to fix all methods at once to defer the GC tracking.

}

extern PyObject* _PyType_AllocNoTrack(PyTypeObject *type, Py_ssize_t nitems);
PyAPI_FUNC(PyObject*) _PyType_AllocNoTrack(PyTypeObject *type, Py_ssize_t nitems);

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
PyAPI_FUNC(PyObject*) _PyType_AllocNoTrack(PyTypeObject *type, Py_ssize_t nitems);
// Export for 'array' shared extension.
PyAPI_FUNC(PyObject*) _PyType_AllocNoTrack(PyTypeObject *type, Py_ssize_t nitems);

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

awaiting core review needs backport to 3.13 bugs and security fixes needs backport to 3.14 bugs and security fixes needs backport to 3.15 pre-release feature fixes, bugs and security fixes topic-free-threading

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants