Repository navigation
Debugging crash in Python 3.6 #884
Description
Environment
- Pythonnet version: A slightly modified version of the code from Initial coreclr and build tooling #612
- Python version: 3.6
- Operating System: Centos 7 x64
Details
I'm trying to debug a crash in pythonnet. I've built a debuggable version of .NET core, and can get stack trace etc from lldb.
CORE_ROOT=$HOME/dotnet_debug/ python -c 'import clr'
Segmentation fault (core dumped)
Activity
Cronan commentedon Jun 19, 2019
(lldb) clrstack -f OS Thread Id: 0x4422 (1) Child SP IP Call Site 00007FFFB92B1378 00007F8AA9F2835F libpython3.6m.so!Py_DecRef + 15 00007FFFB92B1380 00007F8A3CED902D 00007FFFB92B1390 [InlinedCallFrame: 00007fffb92b1390] Python.Runtime.dll!Python.Runtime.Runtime.Py_DecRef(IntPtr) 00007FFFB92B1390 [InlinedCallFrame: 00007fffb92b1390] Python.Runtime.dll!Python.Runtime.Runtime.Py_DecRef(IntPtr) 00007FFFB92B1380 00007F8A3CED902D Python.Runtime.dll!DomainBoundILStubClass.IL_STUB_PInvoke(IntPtr) + 157 00007FFFB92B1430 00007F8A3CED90CC Python.Runtime.dll!Python.Runtime.Runtime.XDecref(IntPtr) + 44 [/users/isys/my-user/git/medusa-recipes/pythonnet/2.3.0/ws/src/src/runtime/runtime.cs @ 547] 00007FFFB92B1450 00007F8A3CED6BE5 Python.Runtime.dll!Python.Runtime.Runtime.Initialize() + 1861 [/users/isys/my-user/git/medusa-recipes/pythonnet/2.3.0/ws/src/src/runtime/runtime.cs @ 292] 00007FFFB92B1610 00007F8A3CED5479 Python.Runtime.dll!Python.Runtime.PythonEngine.Initialize(System.Collections.Generic.IEnumerable`1<System.String>, Boolean) + 217 [/users/isys/my-user/git/medusa-recipes/pythonnet/2.3.0/ws/src/src/runtime/pythonengine.cs @ 167] 00007FFFB92B1850 00007F8A3CED2C6B Python.Runtime.dll!Python.Runtime.PythonEngine.Initialize(Boolean) + 75 [/users/isys/my-user/git/medusa-recipes/pythonnet/2.3.0/ws/src/src/runtime/pythonengine.cs @ 145] 00007FFFB92B1870 00007F8A3CED25E4 Python.Runtime.dll!Python.Runtime.PythonEngine.InitExt() + 68 [/users/isys/my-user/git/medusa-recipes/pythonnet/2.3.0/ws/src/src/runtime/pythonengine.cs @ 239] 00007FFFB92B1910 00007F8AB58E354F libcoreclr.so!UMThunkStub + 294 at /coreclr/src/vm/amd64/umthunkstub.S:144 00007FFFB92B19E0 00007F8AB653E044 clr.cpython-36m-x86_64-linux-gnu.so!init + 580 at /users/isys/my-user/git/medusa-recipes/pythonnet/2.3.0/ws/src/src/coreclr/pynetinit.c:190 00007FFFB92B1A40 00007F8AB653E0E8 clr.cpython-36m-x86_64-linux-gnu.so!PyNet_Init + 120 at /users/isys/my-user/git/medusa-recipes/pythonnet/2.3.0/ws/src/src/coreclr/pynetinit.c:52 00007FFFB92B1A60 00007F8AB653E156 clr.cpython-36m-x86_64-linux-gnu.so!PyInit_clr + 70 at /users/isys/my-user/git/medusa-recipes/pythonnet/2.3.0/ws/src/src/coreclr/clrmod.c:47 00007FFFB92B1A60 00007F8AB653E122 clr.cpython-36m-x86_64-linux-gnu.so!PyInit_clr + 18 at /users/isys/my-user/git/medusa-recipes/pythonnet/2.3.0/ws/src/src/coreclr/clrmod.c:37 00007FFFB92B1A80 00005586BBBC48E5 python!_PyImport_LoadDynamicModuleWithSpec + 389 00007FFFB92B1AD0 00005586BBBC4AE5 python!_imp_create_dynamic + 197 00007FFFB92B1B30 00005586BBAC0A61 python!PyCFunction_Call + 305 00007FFFB92B1B70 00005586BBB74FDB python!_PyEval_EvalFrameDefault + 22043 00007FFFB92B1C90 00005586BBB46A94 python!_PyEval_EvalCodeWithName + 820 00007FFFB92B1D70 00005586BBB47941 python!fast_function + 833 00007FFFB92B1E00 00005586BBB4D755 python!call_function + 341 00007FFFB92B1E50 00005586BBB6FCBA python!_PyEval_EvalFrameDefault + 762 00007FFFB92B1F70 00005586BBB4770B python!fast_function + 267 00007FFFB92B1FB0 00005586BBB4D755 python!call_function + 341 00007FFFB92B2000 00005586BBB6FCBA python!_PyEval_EvalFrameDefault + 762 00007FFFB92B2120 00005586BBB4770B python!fast_function + 267 00007FFFB92B2160 00005586BBB4D755 python!call_function + 341 00007FFFB92B21B0 00005586BBB6FCBA python!_PyEval_EvalFrameDefault + 762 00007FFFB92B22D0 00005586BBB4770B python!fast_function + 267 00007FFFB92B2310 00005586BBB4D755 python!call_function + 341 00007FFFB92B2360 00005586BBB6FCBA python!_PyEval_EvalFrameDefault + 762 00007FFFB92B2480 00005586BBB4770B python!fast_function + 267 00007FFFB92B24C0 00005586BBB4D755 python!call_function + 341 00007FFFB92B2510 00005586BBB6FCBA python!_PyEval_EvalFrameDefault + 762 00007FFFB92B2630 00005586BBB47D7B python!_PyFunction_FastCallDict + 283 00007FFFB92B26C0 00005586BBABDF5F python!_PyObject_FastCallDict + 623 00007FFFB92B2710 00005586BBB02670 python!_PyObject_CallMethodIdObjArgs + 256 00007FFFB92B2850 00005586BBAB4A70 python!PyImport_ImportModuleLevelObject + 832 00007FFFB92B28C0 00005586BBB72033 python!_PyEval_EvalFrameDefault + 9843 00007FFFB92B29E0 00005586BBB48459 python!PyEval_EvalCodeEx + 809 00007FFFB92B2AA0 00005586BBB491EC python!PyEval_EvalCode + 28 00007FFFB92B2AE0 00005586BBBC39A4 python!run_mod + 52 00007FFFB92B2B00 00005586BBBC3A3D python!PyRun_StringFlags + 125 00007FFFB92B2B50 00005586BBBC3A9F python!PyRun_SimpleStringFlags + 63 00007FFFB92B2B70 00005586BBBC789B python!Py_Main + 1083 00007FFFB92B2C80 00005586BBA8F4BE python!main + 238 00007FFFB92B2CD0 00007F8ABD5DE445 libc.so.6!__libc_start_main + 245 00007FFFB92B2D90 00005586BBB76773 python!_start + 41Cronan commentedon Jun 19, 2019
Cronan commentedon Jun 19, 2019
Cronan commentedon Jun 19, 2019
Cronan commentedon Jun 19, 2019
According to Jan in the linked thread, it's:
Does this help anyone more than it helps me? :-)
amos402 commentedon Jun 21, 2019
It you are debugging on a python with Py_DEBUG, you can simply insert
to
src/runtime/interop36.csto get the right offset.Cronan commentedon Jun 21, 2019
Wow. Can you explain to me what this is doing please, I’d love to understand.
amos402 commentedon Jun 22, 2019
I wonder if you are debugging on a
debugversion python, if so, you should care about thePy_XDECREFdue to thedebugversion do more things than release version.As you see, there is a
_Py_ForgetReference(op);call in_Py_Decalloc, it would deal with the_ob_nextand_ob_prev, but in pythonnetYou can see that it just call
Py_TYPE(op)->tp_deallocwithout_Py_ForgetReference.All these operations would mess your memory, thus crashing would be just sooner or later.
Well, it consider as a minor bug for pythonnet
Cronan commentedon Jun 22, 2019
amos402 commentedon Jun 22, 2019
Sometimes you can just use the release configure but set -O0 to make it easy for debugging.
Cronan commentedon Jun 24, 2019
@amos402 Looks like it's not the problem - back to the drawing board.
But thanks for the suggestion.
Cronan commentedon Jun 25, 2019
It seems to be this:
When called from here:
Cronan commentedon Jun 25, 2019
Something to do with the small int list?
Works:
Fails:
Cronan commentedon Jun 25, 2019
OK - it seems the Python interpreter isn't initialized in Python 3, which is why the small_int cache is missing - anyone have any ideas why?
amos402 commentedon Jun 26, 2019
Do you have the C code of your library? Is you interpreter a official version or the one you have been modified?
Cronan commentedon Jun 27, 2019
amos402 commentedon Jun 27, 2019
You can use GDB attach to the target process and use
info shareto validate your loaded library is correct or not.Cronan commentedon Jun 27, 2019
So, to make a long story short .....
Some versions of python are compiled with the shared library support disabled.
See this discussion.
For example, the one my company uses:
https://anaconda.org/anaconda/python/3.6.5/download/linux-32/python-3.6.5-hc3d631a_2.tar.bz2
This does not mean that the libpython3.6m.so.1.0 is not created, it means that it is not used by the interpreter. The effect of this is that some Python C calls will work
(e.g.
PyObject_Type(PyLong_FromLong(257))but others will fail
(e.g.
PyObject_Type(PyLong_FromLong(0)))You can test for the setting by running the following (the result below means pythonnet will segfault):
The only solution for me is to alter the version of the interpreter to use one that was compiled with the shared library support enabled.
We're going to be using this moving forward:
https://anaconda.org/conda-forge/python/3.6.5/download/linux-64/python-3.6.5-1.tar.bz2
Thank you to @amos402 for all your help.