Skip to content

Clip F to I conversion to the INT32 range - #10373

Open
usmanmateen wants to merge 2 commits into
python-pillow:mainfrom
usmanmateen:fix/f-to-i-overflow
Open

usmanmateen wants to merge 2 commits into
python-pillow:mainfrom
usmanmateen:fix/f-to-i-overflow

Conversation

@usmanmateen

Copy link
Copy Markdown
Contributor

Fixes #10372.

Changes proposed in this pull request:

  • Clip F to I conversion to the INT32 range in f2i, and map NaN to 0. The cast had no range check, so on x86 every value above the range, inf and NaN became -2147483648, and an I image at 2147483647 converted to F and back did too. These casts are undefined behavior in C, so the result also depended on the CPU.
  • Add a test with values at and past both ends of the range, inf, -inf and NaN.
>>> Image.new("F", (1, 1), float("inf")).convert("I").getpixel((0, 0))
-2147483648

With this change, it is 2147483647, the same clipping clip32() in Filter.c does. NaN becomes 0, which is what F to L and F to I;16 give.

f2i cast floats to INT32 with no range check. Out-of-range values and NaN are undefined behavior in C, and on x86 they all became -2147483648. Clip to the INT32 range like clip32() in Filter.c, and map NaN to 0 like the F to L and F to I;16 converters.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Converting F images to I turns large values, inf and NaN into -2147483648

2 participants