Skip to content

BUG: propagate errors in stringdtype comparisons - #33214

Open
ngoldbaum wants to merge 3 commits into
numpy:mainfrom
ngoldbaum:stringdtype-compare-errors
Open

ngoldbaum wants to merge 3 commits into
numpy:mainfrom
ngoldbaum:stringdtype-compare-errors

Conversation

@ngoldbaum

@ngoldbaum ngoldbaum commented Nov 3, 2026 •

Copy link
Copy Markdown
Member

PR summary

While working on #33213 I noticed a similar pattern of writing to the output buffer before checking for errors ufuncs and operations that rely on the internal StringDType comparison API.

Currently StringDType's string comparison helper has no way to report errors. If it sees a null that isn't nan-like or a string, it raises an exception but returns 0, and its callers keep going as if the two strings compared equal:

In [1]: import numpy as np

In [2]: arr = np.array(['d', None, 'a'], dtype=np.dtypes.StringDType(na_object=None))

In [3]: out = np.zeros_like(arr)

In [4]: np.maximum(arr, 'b', out=out)
---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
Cell In[4], line 1
----> 1 np.maximum(arr, 'b', out=out)

ValueError: Cannot compare null that is not a nan-like value

In [5]: out
Out[5]: array(['d', None, 'b'], dtype=StringDType(na_object=None))

In [6]: idx = np.array(-1)

In [7]: arr.argmax(out=idx)
---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
ValueError: Cannot compare null that is not a nan-like value

The above exception was the direct cause of the following exception:

SystemError                               Traceback (most recent call last)
Cell In[7], line 1
----> 1 arr.argmax(out=idx)

SystemError: <method 'argmax' of 'numpy.ndarray' objects> returned a result with an exception set

In [8]: idx
Out[8]: array(0)

Note how np.maximum keeps writing to out after the error, including the element that caused it. Even worse: argmax and argmin with out= raises a SystemError and writes to out too.

The fix is to have the comparison helper return an error code. minimum/maximum and the StringDType argmax/argmin implementations now stop at the first error without writing the failing element, and the shared argmax/argmin loop now stops after the first failing row, including for object arrays. The PyArray_CompareFunc wrappers used for sorting and searchsorted keep their current behavior because that signature can't report errors. I also extended the existing null sorting test to cover argmax, argmin, and maximum with out=, and added tests that reductions stop before writing subsequent rows.

StringDType reductions also skip allocator locking for singleton rows, and comparisons only read missing-value metadata when an operand is null.

It's possible to backport this but it's pretty niche and it would need to be done manually so I'm not marking this for backport.

AI Disclosure

I iterated on this with an AI model.

@ngoldbaum ngoldbaum added this to the 2.6.0 Release milestone Nov 3, 2026
@ngoldbaum ngoldbaum added 00 - Bug component: numpy.strings String dtypes and functions labels Nov 3, 2026

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

00 - Bug component: numpy.strings String dtypes and functions

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant