Hi, I ran across this quite nasty bug today. In certain situations (I have been unable to determine when exactly), an exception of any kind being thrown in a custom object_hook while decoding, will make the entire Python interpreter segfault. I have included a test case for both a non-segfaulting exception and a segfaulting exception.
Test case:
import msgpack
class TestObject:
def _unpack(self, data):
return msgpack.unpackb(data, object_hook=self._decode_unpack)
def _decode_unpack(self, obj):
raise Exception("test")
def testrun_segfault(self):
# msgpack-encoded version of {"test": "just sending some test data...", "number": 41, "file": open("test.py", "r")}
# read custom encoding function that was used at the bottom
encoded = "\x83\xa4test\xbejust sending some test data...\xa6number)\xa4file\x83\xa8__type__\xa4file\xa6__id__\x0b\xa8__size__\xcd\x02`"
try:
decoded = self._unpack(encoded)
except:
print "Exception (with segfault) happened successfully"
def testrun_nosegfault(self):
# msgpack-encoded version of {"test": "just sending some test data...", "number": 41}
encoded = "\x82\xa4test\xbejust sending some test data...\xa6number)"
try:
decoded = self._unpack(encoded)
except:
print "Exception (without segfault) happened successfully"
test = TestObject()
print "Attempting testrun without segfault..."
test.testrun_nosegfault()
print "Attempting testrun with segfault..."
test.testrun_segfault()
''' Custom encoding code used to encode the data:
def _pack(self, data):
return msgpack.packb(data, default=self._encode_pack)
def _encode_pack(self, obj):
if hasattr(obj, "read"):
datastream_id = self._create_datastream(obj) # <- Unrelated project code
# Determine the total size of the file
current_pos = obj.tell()
obj.seek(0, os.SEEK_END)
total_size = obj.tell()
obj.seek(current_pos)
obj = {"__type__": "file", "__id__": datastream_id, "__size__": total_size}
return obj
'''
Output:
[occupy@edge13 pyreactor]$ python ~/segfault.py
Attempting testrun without segfault...
Exception (without segfault) happened successfully
Attempting testrun with segfault...
Segmentation fault (core dumped)
Hi, I ran across this quite nasty bug today. In certain situations (I have been unable to determine when exactly), an exception of any kind being thrown in a custom object_hook while decoding, will make the entire Python interpreter segfault. I have included a test case for both a non-segfaulting exception and a segfaulting exception.
Test case:
Output: