Skip to content

coerce dtype=np.int64 for np.prod calls in subsample #806

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
kushalkolar merged 4 commits into from
May 20, 2025
Merged

coerce dtype=np.int64 for np.prod calls in subsample #806

Changes from 2 commits
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
39e01d3
coerce dtype=np.int64 for np.prod calls in subsample
FlynnOConnell Apr 25, 2025
d1bb278
fix: tuple is immutable
FlynnOConnell Apr 27, 2025
4c3556b
Merge branch 'main' into fix-int-overflow
FlynnOConnell May 20, 2025
3a2012d
change arr.shape calls to full_shape as uint64
FlynnOConnell May 20, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 3 additions & 3 deletions fastplotlib/utils/functions.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -455,17 +455,17 @@ def subsample_array(
np.ndarray
subsample of the input array
"""
if np.prod(arr.shape) <= max_size:
if np.prod(arr.shape, dtype=np.int64) <= max_size:
Comment thread
kushalkolar marked this conversation as resolved.
Outdated
return arr[:] # no need to subsample if already below the threshold

# get factor by which to divide all dims
f = np.power((np.prod(arr.shape) / max_size), 1.0 / arr.ndim)
f = np.power((np.prod(arr.shape, dtype=np.int64) / max_size), 1.0 / arr.ndim)

# new shape for subsampled array
ns = np.floor(np.array(arr.shape) / f).clip(min=1)

# get the step size for the slices
slices = tuple(
slices = list(
slice(None, None, int(s)) for s in np.floor(arr.shape / ns).astype(int)
)

Expand Down