Skip to content

Segmentation fault when exceptions of any kind happen within an object_hook in some situations #39

Description

@joepie91

Hi, I ran across this quite nasty bug today. In certain situations (I have been unable to determine when exactly), an exception of any kind being thrown in a custom object_hook while decoding, will make the entire Python interpreter segfault. I have included a test case for both a non-segfaulting exception and a segfaulting exception.

Test case:

import msgpack

class TestObject:
    def _unpack(self, data):
        return msgpack.unpackb(data, object_hook=self._decode_unpack)

    def _decode_unpack(self, obj):
        raise Exception("test")

    def testrun_segfault(self):
        # msgpack-encoded version of {"test": "just sending some test data...", "number": 41, "file": open("test.py", "r")}
        # read custom encoding function that was used at the bottom
        encoded = "\x83\xa4test\xbejust sending some test data...\xa6number)\xa4file\x83\xa8__type__\xa4file\xa6__id__\x0b\xa8__size__\xcd\x02`"

        try:
            decoded = self._unpack(encoded)
        except:
            print "Exception (with segfault) happened successfully"

    def testrun_nosegfault(self):
        # msgpack-encoded version of {"test": "just sending some test data...", "number": 41}
        encoded = "\x82\xa4test\xbejust sending some test data...\xa6number)"

        try:
            decoded = self._unpack(encoded)
        except:
            print "Exception (without segfault) happened successfully"


test = TestObject()
print "Attempting testrun without segfault..."
test.testrun_nosegfault()
print "Attempting testrun with segfault..."
test.testrun_segfault()


''' Custom encoding code used to encode the data:

    def _pack(self, data):
        return msgpack.packb(data, default=self._encode_pack)

    def _encode_pack(self, obj):
        if hasattr(obj, "read"):
            datastream_id = self._create_datastream(obj)  # <- Unrelated project code

            # Determine the total size of the file
            current_pos = obj.tell()
            obj.seek(0, os.SEEK_END)
            total_size = obj.tell()
            obj.seek(current_pos)

            obj = {"__type__": "file", "__id__": datastream_id, "__size__": total_size}

        return obj
'''

Output:

[occupy@edge13 pyreactor]$ python ~/segfault.py 
Attempting testrun without segfault...
Exception (without segfault) happened successfully
Attempting testrun with segfault...
Segmentation fault (core dumped)

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions