Skip to content

gh-154568: Fix array._array_reconstructor() ignoring requested byte order for float16 #154569

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
PhysicistJohn wants to merge 3 commits into
base: main
Choose a base branch
from
+32 −1
Open

gh-154568: Fix array._array_reconstructor() ignoring requested byte order for float16 #154569

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
30d9822
gh-154568: Fix array._array_reconstructor() ignoring requested byte o…
PhysicistJohn Jul 24, 2026
fc94566
Fix NEWS entry: don't use :func: role for an undocumented private fun…
PhysicistJohn Jul 24, 2026
e8631d1
Address review feedback: add half-floats case to test_numbers(), simp…
PhysicistJohn Jul 24, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
26 changes: 26 additions & 0 deletions Lib/test/test_array.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -209,6 +209,10 @@ def test_numbers(self):
[-1<<63, (1<<63)-1, 0]),
(['l'], SIGNED_INT64_BE, '>qqq',
[-1<<63, (1<<63)-1, 0]),
(['e'], IEEE_754_FLOAT16_LE, '<eeee',
[1.0, float('inf'), float('-inf'), -0.0]),
(['e'], IEEE_754_FLOAT16_BE, '>eeee',
[1.0, float('inf'), float('-inf'), -0.0]),
(['f'], IEEE_754_FLOAT_LE, '<ffff',
[16711938.0, float('inf'), float('-inf'), -0.0]),
(['f'], IEEE_754_FLOAT_BE, '>ffff',
Expand Down Expand Up @@ -239,6 +243,28 @@ def test_numbers(self):
self.assertEqual(a, b,
msg="{0!r} != {1!r}; testcase={2!r}".format(a, b, testcase))

def test_float16_endianness(self):
# gh-issue: the slow-path decoder for IEEE_754_FLOAT16_LE/BE

@skirpichev skirpichev Jul 24, 2026

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
# gh-issue: the slow-path decoder for IEEE_754_FLOAT16_LE/BE
# gh-154568: the slow-path decoder for IEEE_754_FLOAT16_LE/BE

Second thought, I think this test is fine. Just please add half-floats case to the test_numbers().

# compared mformat_code against the 32-bit float constant
# (IEEE_754_FLOAT_LE) instead of the float16 one, so it always
# decoded as big-endian regardless of what was requested.
#
# A real 'e'-typecode array only exercises the slow (converting)
# path when the requested mformat disagrees with the *native*
# float16 format, so which of LE/BE actually exercises the buggy
# branch depends on the test machine's endianness, and the other
# direction happens to come out "correct by coincidence" because
# the bug unconditionally decodes as big-endian. Using a
# typecode ('d') whose native mformat can never match
# IEEE_754_FLOAT16_LE/BE forces the slow path deterministically
# on any machine, so both directions are actually exercised.
le_bytes = struct.pack('<e', 1.0)
be_bytes = struct.pack('>e', 1.0)
b_le = array_reconstructor(array.array, 'd', IEEE_754_FLOAT16_LE, le_bytes)
b_be = array_reconstructor(array.array, 'd', IEEE_754_FLOAT16_BE, be_bytes)
self.assertEqual(b_le.tolist(), [1.0])
self.assertEqual(b_be.tolist(), [1.0])

Comment thread
skirpichev marked this conversation as resolved.
def test_unicode(self):
teststr = "Bonne Journ\xe9e \U0002030a\U00020347"
testcases = (
Expand Down
5 changes: 5 additions & 0 deletions ...WS.d/next/Library/2026-07-23-21-00-00.gh-issue-154568.float16-reconstructor.rst
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
Fix ``array._array_reconstructor`` (used internally for unpickling
:class:`array.array`) ignoring the requested byte order and always
decoded as big-endian. This could silently produce wrong values when
unpickling a half precision float array on a machine with different
native endianness than the one that pickled it.
2 changes: 1 addition & 1 deletion Modules/arraymodule.c
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2281,7 +2281,7 @@ array__array_reconstructor_impl(PyObject *module, PyTypeObject *arraytype,
case IEEE_754_FLOAT16_LE:
case IEEE_754_FLOAT16_BE: {
Py_ssize_t i;
int le = (mformat_code == IEEE_754_FLOAT_LE) ? 1 : 0;
int le = (mformat_code == IEEE_754_FLOAT16_LE) ? 1 : 0;
Py_ssize_t itemcount = Py_SIZE(items) / 2;
const char *memstr = PyBytes_AS_STRING(items);

Expand Down
Loading