Fix chop_threshold crashes in the pandas-free complex repr - #620
Open
eriknw wants to merge 2 commits into
Open
Conversation
eriknw
force-pushed
the
39-repr-chop-complex-overflow
branch
from
August 5, 2026 17:44
7ee751f to
44033e7
Compare
eriknw
force-pushed
the
39-repr-chop-complex-overflow
branch
from
August 5, 2026 18:03
44033e7 to
2fa3cb8
Compare
eriknw
force-pushed
the
39-repr-chop-complex-overflow
branch
from
August 5, 2026 18:05
2fa3cb8 to
376ee94
Compare
eriknw
force-pushed
the
39-repr-chop-complex-overflow
branch
from
August 6, 2026 07:59
376ee94 to
e0eb986
Compare
eriknw
force-pushed
the
39-repr-chop-complex-overflow
branch
from
August 6, 2026 15:39
e0eb986 to
9d99244
Compare
eriknw
force-pushed
the
39-repr-chop-complex-overflow
branch
2 times, most recently
from
August 6, 2026 20:36
25d557b to
dbff5f6
Compare
eriknw
force-pushed
the
39-repr-chop-complex-overflow
branch
from
August 6, 2026 20:41
dbff5f6 to
b022fa4
Compare
eriknw
force-pushed
the
39-repr-chop-complex-overflow
branch
from
August 7, 2026 02:49
b022fa4 to
b66bc7c
Compare
The COO form formats a complex column through _format_float_column, where display.chop_threshold compares each value's magnitude. Two crashes hid there, found by an overnight differential fuzz against the pandas renderer: - abs() of a python complex raises OverflowError for finite components near the float max, where numpy's abs returns inf. pandas therefore never chops such a value and renders the object fine; the pandas-free path crashed. Treat an overflowing magnitude as never chopped. Byte-identity for this case is restored, verified against the pandas renderer at 6f1eb02 (repr and HTML hashes equal). - A value that actually chops was replaced with float 0.0, which puts a j-less string into the complex column, and _trim_zeros_complex cannot parse that (IndexError). pandas 3.0.3 has the same bug in its own chop path (pandas.io.formats.format._trim_zeros_complex raises the same IndexError on the same input), so byte-identity here means identity with a crash. Chop to the value's own type of zero instead: the chopped cell renders 0.000000e+00+0.000000e+00j and the object reprs where pandas cannot. The second point is a deliberate, documented divergence from pandas, reachable only with display.chop_threshold set on a complex dtype.
On Windows with python-suitesparse-graphblas older than 7.4.3.1 the CFFI
bindings are built without complex support (MSVC lacks C99 _Complex), so
dtypes._supports_complex is False, FC64 is never registered, and
Vector.from_coo(..., dtype="FC64") raises ValueError before the test
gets to the formatting code under test. Guard with the same
skipif("not dtypes._supports_complex") pattern test_scalar.py uses;
the module already imports dtypes, which the string condition needs.
Verified the test still runs and passes on a complex-capable host, and
that the skipif string evaluates to True in the module namespace when
_supports_complex is flipped to False.
eriknw
force-pushed
the
39-repr-chop-complex-overflow
branch
from
August 7, 2026 05:09
b66bc7c to
d09186f
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Stacked on #619. An overnight differential fuzz of #605's pandas-free renderer against the real pandas renderer (34,618 repr/HTML comparisons across 17,309 case pairs) found a crash family in the
display.chop_thresholdpath for complex dtypes, reachable in the COO (long) display form:abs()of a python complex raises OverflowError for finite components near the float max, where numpy'sabs(which pandas uses) returns inf and never chops. Fixed by treating an overflowing magnitude as never chopped; the result is byte-identical to the pandas renderer (verified, repr and HTML hashes equal against6f1eb02).0.0, putting a j-less string into the complex column, which_trim_zeros_complexcannot parse (IndexError). pandas 3.0.3 crashes the same way on the same input in its own chop path, so byte-identity would mean reproducing a pandas crash. This fix chops to the value's own type of zero, so the cell renders0.000000e+00+0.000000e+00jand the object reprs where pandas cannot. A deliberate, documented divergence, reachable only withchop_thresholdset on a complex dtype.Gates: full pinned suite 1114 passed / 145 skipped; pre-commit all hooks pass. The broader fuzz findings (all in non-crash territory) are written up in the overnight stack report for the maintainer meeting.