Skip to content

ENH: note on the error when out may be partially written#31965

Open
Ijtihed wants to merge 1 commit into
numpy:mainfrom
Ijtihed:enh/note-partial-out-write
Open

ENH: note on the error when out may be partially written#31965
Ijtihed wants to merge 1 commit into
numpy:mainfrom
Ijtihed:enh/note-partial-out-write

Conversation

@Ijtihed

@Ijtihed Ijtihed commented Jul 11, 2026

Copy link
Copy Markdown
Contributor

PR summary

follow up to the discussion in #31940

np.put and np.putmask write directly into the caller's array and can raise part-way through (e.g. on an out-of-bounds index) which silently leave it partially modified. this adds a central helper that attaches a PEP 678 note to the error

IndexError: index 99 is out of bounds for axis 0 with size 10
the output array may have been partially modified

the note is only added when no internal copy shielded the caller's array. np.take and np.choose are wired up too but their mode="raise" paths currently copy first so the note only takes effect there if that copy is removed

AI Disclosure

Used Claude for double checking if I missed anything

@Ijtihed

Ijtihed commented Jul 11, 2026

Copy link
Copy Markdown
Contributor Author

cc @ngoldbaum @seberg @ikrommyd

@Ijtihed Ijtihed force-pushed the enh/note-partial-out-write branch from 1762456 to c072052 Compare July 11, 2026 10:35
@ikrommyd

Copy link
Copy Markdown
Member

Could you do a scan if there are any other functions where partial modification can occur on the out array?

@Ijtihed Ijtihed force-pushed the enh/note-partial-out-write branch from c072052 to 4209c33 Compare July 12, 2026 05:49
@Ijtihed

Ijtihed commented Jul 12, 2026

Copy link
Copy Markdown
Contributor Author

Could you do a scan if there are any other functions where partial modification can occur on the out array?

caller's array can be left partially modified on error in these

  • sort / partition (in-place, raising object comparison)
  • a[mask] = vals and a[idx] = vals / put_along_axis (failing value cast mid-write)
  • np.copyto / dst[...] = src / concatenate(out=) when the casting rule allows an element-wise-failing cast
  • np.round(out=, decimals != 0) (leaves intermediate a * 10**decimals values) and np.mean(out=) (leaves undivided sum)
  • ufuncs with a matching-dtype out (elementwise, reduce, accumulate, reduceat, object matmul/dot, einsum) the loop writes straight into out
  • argmax / argmin with out on object dtype

fyi: take/choose weren't fully shielded by their internal copy either so i added to the pr

@ikrommyd

ikrommyd commented Jul 12, 2026

Copy link
Copy Markdown
Member

@seberg do you think one should add the note to all cases where an exception being raised can cause partial modification to out array here in this PR?

@mattip

mattip commented Jul 12, 2026

Copy link
Copy Markdown
Member

This has ballooned into way to much reviewer work for something that is really not a problem. Please don't continue adding on more followup PRs for this.

@Ijtihed

Ijtihed commented Jul 12, 2026

Copy link
Copy Markdown
Contributor Author

This has ballooned into way to much reviewer work for something that is really not a problem. Please don't continue adding on more followup PRs for this.

the scan was a response to @ikrommyd 's question. The other issues are independent of this specific fix :)

I will avoid adding more follow ups from now on. thanks for flagging this!

@ikrommyd

ikrommyd commented Jul 12, 2026

Copy link
Copy Markdown
Member

I was probably not clear enough. I only wanted to identify if there are any other places to put the add_note helper where the user is already visibly experiencing a partial modification in the out array and we can just improve the error they are already getting.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants