NumPy 1.24 seems to use multi-dimensional memoryviews when getting array data with array.data. msgpack does not play nicely with such multi-dimensional views.

Example

Here's a crashing example without NumPy, but creating a multi-dimensional memoryview by hand:

import msgpack

def mutli_dim_memoryview():

    view = memoryview(b"\00" * 6)
    for data in [
        view,  # fine
        view.cast(view.format, (3, 2))  # crashes when unpacking
    ]:
        print("view shape:", data.shape)
        packed = msgpack.packb(data)
        print("packed:", packed)
        unpacked = msgpack.unpackb(packed)
        print("unpacked:", unpacked)

Output

view shape: (6,)
packed: b'\xc4\x06\x00\x00\x00\x00\x00\x00'
unpacked: b'\x00\x00\x00\x00\x00\x00'
view shape: (3, 2)
packed: b'\xc4\x03\x00\x00\x00\x00\x00\x00'
Traceback (most recent call last):
    [...]
    unpacked = msgpack.unpackb(packed)
               ^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/xxx/lib64/python3.11/site-packages/msgpack/fallback.py", line 133, in unpackb
    raise ExtraData(ret, unpacker._get_extradata())
msgpack.exceptions.ExtraData: unpack(b) received extra data.

Mitigations

• Casting the view w/o a shape creates a 1D view without re-allocating memory (see Python docs): data.case(data.format)
• Generate bytes either by data.tobytes() or bytes(data). I'm not sure if that duplicates the memory buffer.

Of course, it would be great if msgpack could do that internally when packing a memoryview.

Thanks for this great package! ❤️