Repository navigation
Closed as duplicate
Closed as duplicate
Copy link
Labels
3.13only security fixesonly security fixes3.14bugs and security fixesbugs and security fixesinterpreter-core(Objects, Python, Grammar, and Parser dirs)(Objects, Python, Grammar, and Parser dirs)topic-free-threadingtype-bugAn unexpected behavior, bug, or errorAn unexpected behavior, bug, or error
Description
Bug report
Bug description:
I built 3.13 branch and observed the following race under free-threading in cpython 3.13 (Python 3.13.2+ experimental free-threading build (heads/3.13:6afab81f72c, Apr 7 2025, 13:34:51) [Clang 18.1.3 (1ubuntu1)])
TSAN Report
WARNING: ThreadSanitizer: data race (pid=98556)
Atomic write of size 8 at 0x7fffb695e118 by thread T2 (mutexes: read M0):
#0 _Py_atomic_store_ssize_relaxed /project/cpython/./Include/cpython/pyatomic_gcc.h:481:3 (python3.13+0x273c84) (BuildId: ec7bf2b242541cde4f3e3f0459fbc7b3819b41b0)
#1 split_keys_entry_added /project/cpython/Objects/dictobject.c:225:5 (python3.13+0x273c84)
#2 insert_split_key /project/cpython/Objects/dictobject.c:1743:9 (python3.13+0x273c84)
#3 insertdict /project/cpython/Objects/dictobject.c:1796:25 (python3.13+0x263ec6) (BuildId: ec7bf2b242541cde4f3e3f0459fbc7b3819b41b0)
#4 dict_dict_merge /project/cpython/./Include/cpython/pyatomic_gcc.h (python3.13+0x2698e2) (BuildId: ec7bf2b242541cde4f3e3f0459fbc7b3819b41b0)
#5 dict_merge /project/cpython/Objects/dictobject.c:3807:15 (python3.13+0x2698e2)
#6 PyDict_Merge /project/cpython/Objects/dictobject.c:3897:12 (python3.13+0x27767c) (BuildId: ec7bf2b242541cde4f3e3f0459fbc7b3819b41b0)
#7 dict_update_arg /project/cpython/Objects/dictobject.c (python3.13+0x27767c)
#8 dict_update_common /project/cpython/Objects/dictobject.c:3555:18 (python3.13+0x27767c)
#9 dict_update /project/cpython/Objects/dictobject.c:3573:9 (python3.13+0x27733c) (BuildId: ec7bf2b242541cde4f3e3f0459fbc7b3819b41b0)
#10 method_vectorcall_VARARGS_KEYWORDS /project/cpython/Objects/descrobject.c:358:14 (python3.13+0x2009ca) (BuildId: ec7bf2b242541cde4f3e3f0459fbc7b3819b41b0)
#11 _PyObject_VectorcallTstate /project/cpython/./Include/internal/pycore_call.h:168:11 (python3.13+0x1eb0fa) (BuildId: ec7bf2b242541cde4f3e3f0459fbc7b3819b41b0)
#12 PyObject_Vectorcall /project/cpython/Objects/call.c:327:12 (python3.13+0x1eb0fa)
#13 _PyEval_EvalFrameDefault /project/cpython/Python/generated_cases.c.h:813:23 (python3.13+0x3e38f2) (BuildId: ec7bf2b242541cde4f3e3f0459fbc7b3819b41b0)
#14 _PyEval_EvalFrame /project/cpython/./Include/internal/pycore_ceval.h:119:16 (python3.13+0x3df9aa) (BuildId: ec7bf2b242541cde4f3e3f0459fbc7b3819b41b0)
#15 _PyEval_Vector /project/cpython/Python/ceval.c:1814:12 (python3.13+0x3df9aa)
#16 _PyFunction_Vectorcall /project/cpython/Objects/call.c (python3.13+0x1eb76f) (BuildId: ec7bf2b242541cde4f3e3f0459fbc7b3819b41b0)
#17 _PyVectorcall_Call /project/cpython/Objects/call.c:285:24 (python3.13+0x1eb336) (BuildId: ec7bf2b242541cde4f3e3f0459fbc7b3819b41b0)
#18 _PyObject_Call /project/cpython/Objects/call.c:348:16 (python3.13+0x1eb336)
#19 PyObject_Call /project/cpython/Objects/call.c:373:12 (python3.13+0x1eb465) (BuildId: ec7bf2b242541cde4f3e3f0459fbc7b3819b41b0)
#20 bounded_lru_cache_wrapper /project/cpython/./Modules/_functoolsmodule.c:1068:14 (python3.13+0x577a42) (BuildId: ec7bf2b242541cde4f3e3f0459fbc7b3819b41b0)
#21 lru_cache_call /project/cpython/./Modules/_functoolsmodule.c:1334:14 (python3.13+0x5762aa) (BuildId: ec7bf2b242541cde4f3e3f0459fbc7b3819b41b0)
#22 _PyObject_Call /project/cpython/Objects/call.c:361:18 (python3.13+0x1eb39b) (BuildId: ec7bf2b242541cde4f3e3f0459fbc7b3819b41b0)
#23 PyObject_Call /project/cpython/Objects/call.c:373:12 (python3.13+0x1eb465) (BuildId: ec7bf2b242541cde4f3e3f0459fbc7b3819b41b0)
#24 _PyEval_EvalFrameDefault /project/cpython/Python/generated_cases.c.h:1355:26 (python3.13+0x3e5b12) (BuildId: ec7bf2b242541cde4f3e3f0459fbc7b3819b41b0)
#25 _PyEval_EvalFrame /project/cpython/./Include/internal/pycore_ceval.h:119:16 (python3.13+0x3df9aa) (BuildId: ec7bf2b242541cde4f3e3f0459fbc7b3819b41b0)
#26 _PyEval_Vector /project/cpython/Python/ceval.c:1814:12 (python3.13+0x3df9aa)
#27 _PyFunction_Vectorcall /project/cpython/Objects/call.c (python3.13+0x1eb76f) (BuildId: ec7bf2b242541cde4f3e3f0459fbc7b3819b41b0)
#28 _PyObject_VectorcallTstate /project/cpython/./Include/internal/pycore_call.h:168:11 (python3.13+0x575e83) (BuildId: ec7bf2b242541cde4f3e3f0459fbc7b3819b41b0)
#29 partial_vectorcall /project/cpython/./Modules/_functoolsmodule.c:262:25 (python3.13+0x575e83)
#30 _PyVectorcall_Call /project/cpython/Objects/call.c:285:24 (python3.13+0x1eb336) (BuildId: ec7bf2b242541cde4f3e3f0459fbc7b3819b41b0)
#31 _PyObject_Call /project/cpython/Objects/call.c:348:16 (python3.13+0x1eb336)
#32 PyObject_Call /project/cpython/Objects/call.c:373:12 (python3.13+0x1eb465) (BuildId: ec7bf2b242541cde4f3e3f0459fbc7b3819b41b0)
#33 _PyEval_EvalFrameDefault /project/cpython/Python/generated_cases.c.h:1355:26 (python3.13+0x3e5b12) (BuildId: ec7bf2b242541cde4f3e3f0459fbc7b3819b41b0)
#34 _PyEval_EvalFrame /project/cpython/./Include/internal/pycore_ceval.h:119:16 (python3.13+0x3df9aa) (BuildId: ec7bf2b242541cde4f3e3f0459fbc7b3819b41b0)
#35 _PyEval_Vector /project/cpython/Python/ceval.c:1814:12 (python3.13+0x3df9aa)
#36 _PyFunction_Vectorcall /project/cpython/Objects/call.c (python3.13+0x1eb76f) (BuildId: ec7bf2b242541cde4f3e3f0459fbc7b3819b41b0)
#37 _PyObject_VectorcallTstate /project/cpython/./Include/internal/pycore_call.h:168:11 (python3.13+0x1ef685) (BuildId: ec7bf2b242541cde4f3e3f0459fbc7b3819b41b0)
#38 method_vectorcall /project/cpython/Objects/classobject.c:62:18 (python3.13+0x1ef685)
#39 _PyVectorcall_Call /project/cpython/Objects/call.c:285:24 (python3.13+0x1eb336) (BuildId: ec7bf2b242541cde4f3e3f0459fbc7b3819b41b0)
#40 _PyObject_Call /project/cpython/Objects/call.c:348:16 (python3.13+0x1eb336)
#41 PyObject_Call /project/cpython/Objects/call.c:373:12 (python3.13+0x1eb465) (BuildId: ec7bf2b242541cde4f3e3f0459fbc7b3819b41b0)
#42 _PyEval_EvalFrameDefault /project/cpython/Python/generated_cases.c.h:1355:26 (python3.13+0x3e5b12) (BuildId: ec7bf2b242541cde4f3e3f0459fbc7b3819b41b0)
#43 _PyEval_EvalFrame /project/cpython/./Include/internal/pycore_ceval.h:119:16 (python3.13+0x3df9aa) (BuildId: ec7bf2b242541cde4f3e3f0459fbc7b3819b41b0)
#44 _PyEval_Vector /project/cpython/Python/ceval.c:1814:12 (python3.13+0x3df9aa)
#45 _PyFunction_Vectorcall /project/cpython/Objects/call.c (python3.13+0x1eb76f) (BuildId: ec7bf2b242541cde4f3e3f0459fbc7b3819b41b0)
#46 _PyObject_VectorcallTstate /project/cpython/./Include/internal/pycore_call.h:168:11 (python3.13+0x1ef685) (BuildId: ec7bf2b242541cde4f3e3f0459fbc7b3819b41b0)
#47 method_vectorcall /project/cpython/Objects/classobject.c:62:18 (python3.13+0x1ef685)
#48 _PyVectorcall_Call /project/cpython/Objects/call.c:285:24 (python3.13+0x1eb336) (BuildId: ec7bf2b242541cde4f3e3f0459fbc7b3819b41b0)
#49 _PyObject_Call /project/cpython/Objects/call.c:348:16 (python3.13+0x1eb336)
#50 PyObject_Call /project/cpython/Objects/call.c:373:12 (python3.13+0x1eb465) (BuildId: ec7bf2b242541cde4f3e3f0459fbc7b3819b41b0)
#51 _PyEval_EvalFrameDefault /project/cpython/Python/generated_cases.c.h:1355:26 (python3.13+0x3e5b12) (BuildId: ec7bf2b242541cde4f3e3f0459fbc7b3819b41b0)
#52 _PyEval_EvalFrame /project/cpython/./Include/internal/pycore_ceval.h:119:16 (python3.13+0x3df9aa) (BuildId: ec7bf2b242541cde4f3e3f0459fbc7b3819b41b0)
#53 _PyEval_Vector /project/cpython/Python/ceval.c:1814:12 (python3.13+0x3df9aa)
#54 _PyFunction_Vectorcall /project/cpython/Objects/call.c (python3.13+0x1eb76f) (BuildId: ec7bf2b242541cde4f3e3f0459fbc7b3819b41b0)
#55 _PyObject_VectorcallTstate /project/cpython/./Include/internal/pycore_call.h:168:11 (python3.13+0x1ef7f0) (BuildId: ec7bf2b242541cde4f3e3f0459fbc7b3819b41b0)
#56 method_vectorcall /project/cpython/Objects/classobject.c:92:18 (python3.13+0x1ef7f0)
#57 _PyVectorcall_Call /project/cpython/Objects/call.c:273:16 (python3.13+0x1eb3e3) (BuildId: ec7bf2b242541cde4f3e3f0459fbc7b3819b41b0)
#58 _PyObject_Call /project/cpython/Objects/call.c:348:16 (python3.13+0x1eb3e3)
#59 PyObject_Call /project/cpython/Objects/call.c:373:12 (python3.13+0x1eb465) (BuildId: ec7bf2b242541cde4f3e3f0459fbc7b3819b41b0)
#60 _PyEval_EvalFrameDefault /project/cpython/Python/generated_cases.c.h:1355:26 (python3.13+0x3e5b12) (BuildId: ec7bf2b242541cde4f3e3f0459fbc7b3819b41b0)
#61 _PyEval_EvalFrame /project/cpython/./Include/internal/pycore_ceval.h:119:16 (python3.13+0x3df9aa) (BuildId: ec7bf2b242541cde4f3e3f0459fbc7b3819b41b0)
#62 _PyEval_Vector /project/cpython/Python/ceval.c:1814:12 (python3.13+0x3df9aa)
#63 _PyFunction_Vectorcall /project/cpython/Objects/call.c (python3.13+0x1eb76f) (BuildId: ec7bf2b242541cde4f3e3f0459fbc7b3819b41b0)
#64 _PyObject_VectorcallDictTstate /project/cpython/Objects/call.c:135:15 (python3.13+0x1ea2ed) (BuildId: ec7bf2b242541cde4f3e3f0459fbc7b3819b41b0)
#65 _PyObject_Call_Prepend /project/cpython/Objects/call.c:504:24 (python3.13+0x1ebde7) (BuildId: ec7bf2b242541cde4f3e3f0459fbc7b3819b41b0)
#66 slot_tp_call /project/cpython/Objects/typeobject.c:9556:15 (python3.13+0x2f9738) (BuildId: ec7bf2b242541cde4f3e3f0459fbc7b3819b41b0)
#67 _PyObject_MakeTpCall /project/cpython/Objects/call.c:242:18 (python3.13+0x1ea55c) (BuildId: ec7bf2b242541cde4f3e3f0459fbc7b3819b41b0)
#68 _PyObject_VectorcallTstate /project/cpython/./Include/internal/pycore_call.h:166:16 (python3.13+0x1eb1b8) (BuildId: ec7bf2b242541cde4f3e3f0459fbc7b3819b41b0)
#69 PyObject_Vectorcall /project/cpython/Objects/call.c:327:12 (python3.13+0x1eb1b8)
#70 _PyEval_EvalFrameDefault /project/cpython/Python/generated_cases.c.h:813:23 (python3.13+0x3e38f2) (BuildId: ec7bf2b242541cde4f3e3f0459fbc7b3819b41b0)
#71 _PyEval_EvalFrame /project/cpython/./Include/internal/pycore_ceval.h:119:16 (python3.13+0x3df9aa) (BuildId: ec7bf2b242541cde4f3e3f0459fbc7b3819b41b0)
#72 _PyEval_Vector /project/cpython/Python/ceval.c:1814:12 (python3.13+0x3df9aa)
#73 _PyFunction_Vectorcall /project/cpython/Objects/call.c (python3.13+0x1eb76f) (BuildId: ec7bf2b242541cde4f3e3f0459fbc7b3819b41b0)
#74 _PyObject_VectorcallTstate /project/cpython/./Include/internal/pycore_call.h:168:11 (python3.13+0x1ef73f) (BuildId: ec7bf2b242541cde4f3e3f0459fbc7b3819b41b0)
#75 method_vectorcall /project/cpython/Objects/classobject.c:70:20 (python3.13+0x1ef73f)
#76 _PyVectorcall_Call /project/cpython/Objects/call.c:273:16 (python3.13+0x1eb3e3) (BuildId: ec7bf2b242541cde4f3e3f0459fbc7b3819b41b0)
#77 _PyObject_Call /project/cpython/Objects/call.c:348:16 (python3.13+0x1eb3e3)
#78 PyObject_Call /project/cpython/Objects/call.c:373:12 (python3.13+0x1eb465) (BuildId: ec7bf2b242541cde4f3e3f0459fbc7b3819b41b0)
#79 thread_run /project/cpython/./Modules/_threadmodule.c:337:21 (python3.13+0x568562) (BuildId: ec7bf2b242541cde4f3e3f0459fbc7b3819b41b0)
#80 pythread_wrapper /project/cpython/Python/thread_pthread.h:243:5 (python3.13+0x4c1477) (BuildId: ec7bf2b242541cde4f3e3f0459fbc7b3819b41b0)
Previous read of size 8 at 0x7fffb695e118 by thread T1 (mutexes: read M0):
#0 dict_dict_merge /project/cpython/Objects/dictobject.c:3776:42 (python3.13+0x2699cb) (BuildId: ec7bf2b242541cde4f3e3f0459fbc7b3819b41b0)
#1 dict_merge /project/cpython/Objects/dictobject.c:3807:15 (python3.13+0x2699cb)
#2 PyDict_Merge /project/cpython/Objects/dictobject.c:3897:12 (python3.13+0x27767c) (BuildId: ec7bf2b242541cde4f3e3f0459fbc7b3819b41b0)
#3 dict_update_arg /project/cpython/Objects/dictobject.c (python3.13+0x27767c)
#4 dict_update_common /project/cpython/Objects/dictobject.c:3555:18 (python3.13+0x27767c)
#5 dict_update /project/cpython/Objects/dictobject.c:3573:9 (python3.13+0x27733c) (BuildId: ec7bf2b242541cde4f3e3f0459fbc7b3819b41b0)
#6 method_vectorcall_VARARGS_KEYWORDS /project/cpython/Objects/descrobject.c:358:14 (python3.13+0x2009ca) (BuildId: ec7bf2b242541cde4f3e3f0459fbc7b3819b41b0)
#7 _PyObject_VectorcallTstate /project/cpython/./Include/internal/pycore_call.h:168:11 (python3.13+0x1eb0fa) (BuildId: ec7bf2b242541cde4f3e3f0459fbc7b3819b41b0)
#8 PyObject_Vectorcall /project/cpython/Objects/call.c:327:12 (python3.13+0x1eb0fa)
#9 _PyEval_EvalFrameDefault /project/cpython/Python/generated_cases.c.h:813:23 (python3.13+0x3e38f2) (BuildId: ec7bf2b242541cde4f3e3f0459fbc7b3819b41b0)
#10 _PyEval_EvalFrame /project/cpython/./Include/internal/pycore_ceval.h:119:16 (python3.13+0x3df9aa) (BuildId: ec7bf2b242541cde4f3e3f0459fbc7b3819b41b0)
#11 _PyEval_Vector /project/cpython/Python/ceval.c:1814:12 (python3.13+0x3df9aa)
#12 _PyFunction_Vectorcall /project/cpython/Objects/call.c (python3.13+0x1eb76f) (BuildId: ec7bf2b242541cde4f3e3f0459fbc7b3819b41b0)
#13 _PyObject_VectorcallTstate /project/cpython/./Include/internal/pycore_call.h:168:11 (python3.13+0x1ef7f0) (BuildId: ec7bf2b242541cde4f3e3f0459fbc7b3819b41b0)
#14 method_vectorcall /project/cpython/Objects/classobject.c:92:18 (python3.13+0x1ef7f0)
#15 _PyVectorcall_Call /project/cpython/Objects/call.c:273:16 (python3.13+0x1eb3e3) (BuildId: ec7bf2b242541cde4f3e3f0459fbc7b3819b41b0)
#16 _PyObject_Call /project/cpython/Objects/call.c:348:16 (python3.13+0x1eb3e3)
#17 PyObject_Call /project/cpython/Objects/call.c:373:12 (python3.13+0x1eb465) (BuildId: ec7bf2b242541cde4f3e3f0459fbc7b3819b41b0)
#18 _PyEval_EvalFrameDefault /project/cpython/Python/generated_cases.c.h:1355:26 (python3.13+0x3e5b12) (BuildId: ec7bf2b242541cde4f3e3f0459fbc7b3819b41b0)
#19 _PyEval_EvalFrame /project/cpython/./Include/internal/pycore_ceval.h:119:16 (python3.13+0x3df9aa) (BuildId: ec7bf2b242541cde4f3e3f0459fbc7b3819b41b0)
#20 _PyEval_Vector /project/cpython/Python/ceval.c:1814:12 (python3.13+0x3df9aa)
#21 _PyFunction_Vectorcall /project/cpython/Objects/call.c (python3.13+0x1eb76f) (BuildId: ec7bf2b242541cde4f3e3f0459fbc7b3819b41b0)
#22 _PyObject_VectorcallDictTstate /project/cpython/Objects/call.c:135:15 (python3.13+0x1ea2ed) (BuildId: ec7bf2b242541cde4f3e3f0459fbc7b3819b41b0)
#23 _PyObject_Call_Prepend /project/cpython/Objects/call.c:504:24 (python3.13+0x1ebde7) (BuildId: ec7bf2b242541cde4f3e3f0459fbc7b3819b41b0)
#24 slot_tp_call /project/cpython/Objects/typeobject.c:9556:15 (python3.13+0x2f9738) (BuildId: ec7bf2b242541cde4f3e3f0459fbc7b3819b41b0)
#25 _PyObject_MakeTpCall /project/cpython/Objects/call.c:242:18 (python3.13+0x1ea55c) (BuildId: ec7bf2b242541cde4f3e3f0459fbc7b3819b41b0)
#26 _PyObject_VectorcallTstate /project/cpython/./Include/internal/pycore_call.h:166:16 (python3.13+0x1eb1b8) (BuildId: ec7bf2b242541cde4f3e3f0459fbc7b3819b41b0)
#27 PyObject_Vectorcall /project/cpython/Objects/call.c:327:12 (python3.13+0x1eb1b8)
#28 _PyEval_EvalFrameDefault /project/cpython/Python/generated_cases.c.h:813:23 (python3.13+0x3e38f2) (BuildId: ec7bf2b242541cde4f3e3f0459fbc7b3819b41b0)
#29 _PyEval_EvalFrame /project/cpython/./Include/internal/pycore_ceval.h:119:16 (python3.13+0x3df9aa) (BuildId: ec7bf2b242541cde4f3e3f0459fbc7b3819b41b0)
#30 _PyEval_Vector /project/cpython/Python/ceval.c:1814:12 (python3.13+0x3df9aa)
#31 _PyFunction_Vectorcall /project/cpython/Objects/call.c (python3.13+0x1eb76f) (BuildId: ec7bf2b242541cde4f3e3f0459fbc7b3819b41b0)
#32 _PyObject_VectorcallTstate /project/cpython/./Include/internal/pycore_call.h:168:11 (python3.13+0x1ef73f) (BuildId: ec7bf2b242541cde4f3e3f0459fbc7b3819b41b0)
#33 method_vectorcall /project/cpython/Objects/classobject.c:70:20 (python3.13+0x1ef73f)
#34 _PyVectorcall_Call /project/cpython/Objects/call.c:273:16 (python3.13+0x1eb3e3) (BuildId: ec7bf2b242541cde4f3e3f0459fbc7b3819b41b0)
#35 _PyObject_Call /project/cpython/Objects/call.c:348:16 (python3.13+0x1eb3e3)
#36 PyObject_Call /project/cpython/Objects/call.c:373:12 (python3.13+0x1eb465) (BuildId: ec7bf2b242541cde4f3e3f0459fbc7b3819b41b0)
#37 thread_run /project/cpython/./Modules/_threadmodule.c:337:21 (python3.13+0x568562) (BuildId: ec7bf2b242541cde4f3e3f0459fbc7b3819b41b0)
#38 pythread_wrapper /project/cpython/Python/thread_pthread.h:243:5 (python3.13+0x4c1477) (BuildId: ec7bf2b242541cde4f3e3f0459fbc7b3819b41b0)
I know that there were already a bunch of data race fixes in dictobject and happy to close this issue if it is already fixed.
I do not have a minimal reproducer, but can try to get it if needed.
CPython versions tested on:
3.13
Operating systems tested on:
Linux
Similar previously reported race: #130547
Metadata
Metadata
Assignees
Labels
3.13only security fixesonly security fixes3.14bugs and security fixesbugs and security fixesinterpreter-core(Objects, Python, Grammar, and Parser dirs)(Objects, Python, Grammar, and Parser dirs)topic-free-threadingtype-bugAn unexpected behavior, bug, or errorAn unexpected behavior, bug, or error
Activity
hawkinsp commentedon Apr 16, 2025
Certainly
#0 dict_dict_merge /project/cpython/Objects/dictobject.c:3776:42 (python3.13+0x2699cb) (BuildId: ec7bf2b242541cde4f3e3f0459fbc7b3819b41b0)is a non-atomic read (cpython/Objects/dictobject.c
Line 3776 in f9578dc
colesbury commentedon Apr 16, 2025
That check looks wrong to me. There should only be an error if the actual dictionary was mutated. Not if the shared keys were changed.
colesbury commentedon Apr 16, 2025
Filed:
dict.update()mutation check too broad #132617colesbury commentedon Apr 16, 2025
The non-atomic read is only an issue with shared keys objects, but checking
dk_nentriesbefore and after isn't reliable in that case.(It's not an issue for other dictionaries because
dict_dict_mergeis called with the dictionary locks held)vfdev-5 commentedon May 16, 2025
We also see this now on 3.14 branch in JAX CI: https://github.com/jax-ml/jax/actions/runs/15061081634/job/42336229305
colesbury commentedon Jun 16, 2025
I think this should be fixed now in 3.13, 3.14, and main. I'm going to mark this as a duplicate of #132617 because I think the race was due to an incorrect mutation check.
dict.update()mutation check too broad #132617