Skip to content

Render rich reprs without pandas, byte-identical to pandas output - #605

Open
eriknw wants to merge 3 commits into
24-operator-resolutionfrom
25-pandas-free-repr
Open

Render rich reprs without pandas, byte-identical to pandas output#605
eriknw wants to merge 3 commits into
24-operator-resolutionfrom
25-pandas-free-repr

Conversation

@eriknw

@eriknw eriknw commented Aug 4, 2026

Copy link
Copy Markdown
Member

The rich repr and repr_html previously built a pandas DataFrame and let
pandas render it. Rendering is now done in-house with a numpy-only frame, a
ported text formatter, and an html builder. Users without pandas now get the
same rich repr they previously needed pandas for; with pandas present the
output is byte-identical, and pandas display options that could affect the
old output are still honored.

  • The renderer (9bf7d91): verified against the pandas render with a
    differential harness totaling about 145,000 checks over shapes, density,
    mask and transpose views, every dtype including UDTs, float and complex
    cell formatting across the precision and scientific-switchover
    boundaries, expression reprs in both autocompute modes, and display
    options crossed with terminal sizes. The only divergences are two obscure
    corners documented in the commit message, one of which is a fix rather
    than a regression. Rendering is also 7-12x faster, since pandas'
    formatter machinery is no longer built per call.
  • Un-gate 29 formatting tests (e9ad1a6): in a pandas-free environment
    test_formatting.py goes from 12 passed / 40 skipped to 41 passed / 11
    skipped, with several un-gated tests asserting the full pandas-style HTML
    verbatim.
  • Un-gate the three autocompute formatting tests (d009b4c): 41 to 44
    passed without pandas. The eight tests that exercise pd.option_context
    stay gated; they genuinely need pandas.

Stack created with GitHub Stacks CLIGive Feedback 💬

@eriknw eriknw changed the title 25 pandas free repr Render rich reprs without pandas, byte-identical to pandas output Aug 4, 2026
@eriknw
eriknw marked this pull request as ready for review August 4, 2026 16:07
@eriknw
eriknw force-pushed the 25-pandas-free-repr branch 2 times, most recently from 637a692 to 4d57cea Compare August 5, 2026 00:06
@eriknw
eriknw force-pushed the 25-pandas-free-repr branch from 4d57cea to 7c3a51f Compare August 5, 2026 03:18
@eriknw
eriknw force-pushed the 25-pandas-free-repr branch from 7c3a51f to e9afc5b Compare August 5, 2026 17:44
@eriknw
eriknw force-pushed the 25-pandas-free-repr branch from e9afc5b to 584bd56 Compare August 5, 2026 18:03
@eriknw
eriknw force-pushed the 25-pandas-free-repr branch 2 times, most recently from edb31bc to e45cfa3 Compare August 6, 2026 07:59
@eriknw
eriknw force-pushed the 25-pandas-free-repr branch from e45cfa3 to 89f123c Compare August 6, 2026 15:39
@eriknw
eriknw force-pushed the 25-pandas-free-repr branch from 89f123c to 6bb0d80 Compare August 6, 2026 15:41
@eriknw
eriknw force-pushed the 25-pandas-free-repr branch 2 times, most recently from 2a64484 to 982da91 Compare August 6, 2026 20:41
@eriknw
eriknw force-pushed the 25-pandas-free-repr branch from 982da91 to 310e8ec Compare August 7, 2026 02:48
eriknw added 3 commits August 7, 2026 00:09
The rich repr and _repr_html_ previously built a pandas DataFrame and
let pandas render it. Rendering is now done in-house: a lightweight
numpy-only frame, a ported text formatter (cell formatting with
precision and scientific switchover, column widths, max_rows and
max_columns truncation, terminal fit, multiline wrap), and an html
builder reproducing the notebook table. With pandas absent, users now
get the same rich repr they previously needed pandas for; with pandas
present, output is byte-identical, and pandas display options that
could affect the old output are still honored (max_rows, min_rows,
max_columns, width, expand_frame_repr, precision, max_colwidth,
chop_threshold, colheader_justify, float_format, html.border,
html.use_mathjax).

Known non-reproduced corners, all obscure: CJK east-asian width
handling; pprint_nest_depth and max_seq_items on huge UDT cells;
notebook_repr_html=False, where pandas' _repr_html_ returns None and
the old code embedded the literal string "None" in its place, so the
new table is a fix rather than a regression; and max_colwidth of 3 or
less, where pandas narrows the column labels too and leaves them
ragged while the hand renderer keeps them padded to a uniform width.
That last case needs a deliberate max_colwidth <= 3 (the default is
50) together with column labels of differing digit counts, and it
affects the text repr only, since the html path strips cell padding.

The no-pandas tests now assert the pandas-absent output equals the
pandas-present output instead of pinning a header-only fallback.

Verified against the previous pandas render with a differential that
swaps which formatting module graphblas.core resolves to and compares
repr() and _repr_html_() end to end: 12376 checks over shapes,
density, mask and transpose views, and every dtype including two UDTs;
82320 checks over float and complex cell formatting (precision 0
through 17, the scientific switchover boundaries, chop_threshold,
float_format); 1024 checks over expression and infix-expression reprs
in both autocompute modes; and 49140 checks over pandas display
options crossed with six terminal sizes. The only divergences are the
two described above.

Rendering is also faster, since pandas' formatter machinery is no
longer built per call: repr of a 4x4 Matrix ~850us -> ~90us, of a
10-element Vector ~1.5ms -> ~124us, of a 1e6-element Vector ~9.6ms ->
~1.1ms, and _repr_html_ of a 4x4 Matrix ~373us -> ~53us. These are
best-of-5 with both paths interleaved in one process; the ratios (7x
to 12x) held across repeated runs on a shared machine, the absolute
figures less so.
The rich repr renders without pandas as of the previous commit, so 29
of the 37 skipif("not pd") gates in test_formatting.py are obsolete;
in a pandas-free environment those tests now run instead of skipping
(12 -> 41 passing). The 8 tests exercising pd.option_context stay
gated, as do the three autocompute tests with their own runtime skip.
Several un-gated tests assert the full pandas-style HTML verbatim and
pass without pandas, confirming the hand renderer is byte-identical.

Verified with pandas installed (unchanged: the removed gates never
fired there) and with pandas blocked at sys.meta_path, where
test_formatting.py goes from 12 passed / 40 skipped to 41 passed / 11
skipped, with no failures.
Since the pandas-free repr, test_autocompute, test_autocompute_html, and
test_index_expr_autocompute only assert repr/_repr_html_ output that renders
byte-identically without pandas, so the runtime "if not pd: skip" guards are
obsolete. Removed them. The eight large-repr tests that call pd.option_context
stay gated (they genuinely need pandas to exercise user display options).

Verified with pandas installed (unchanged: the removed guards never fired
there) and with pandas blocked at sys.meta_path, where test_formatting.py goes
from 41 passed / 11 skipped to 44 passed / 8 skipped; the three moved from skip
to pass, with no failures.
@eriknw
eriknw force-pushed the 25-pandas-free-repr branch from 310e8ec to 730498f Compare August 7, 2026 05:09
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant