Skip to content

ENH: Return numpy scalars directly from the full-reduction fast path - #32269

Open
eendebakpt wants to merge 1 commit into
numpy:mainfrom
eendebakpt:reduce_scalar_fastpath
Open

ENH: Return numpy scalars directly from the full-reduction fast path#32269
eendebakpt wants to merge 1 commit into
numpy:mainfrom
eendebakpt:reduce_scalar_fastpath

Conversation

@eendebakpt

Copy link
Copy Markdown
Contributor

PR summary

When a full reduction (axis=None) hits the contiguous fast path and the caller would immediately convert the 0-d result to a numpy scalar anyway (plain ndarray input, no out=, no __array_wrap__ to apply), accumulate into a stack buffer and create the scalar directly with PyArray_Scalar. This skips allocating the temporary 0-d array and the wrap machinery.

The scalar path is only taken for single-output reductions over dtypes whose scalars are self-contained (elsize <= sizeof(npy_clongdouble), not flexible, no NPY_USE_GETITEM), on top of the existing fast-path preconditions.

Benchmark results:

Benchmark main PR
np.sum(a10) 580 ns 399 ns: 1.45x faster
np.sum(a1000) 802 ns 607 ns: 1.32x faster
np.max(a10) 580 ns 399 ns: 1.46x faster
np.max(a1000) 764 ns 572 ns: 1.34x faster
np.min(i50) 586 ns 407 ns: 1.44x faster
np.add.reduce(a10) 505 ns 347 ns: 1.45x faster
np.multiply.reduce(i10) 519 ns 365 ns: 1.42x faster
a1000.sum() 816 ns 622 ns: 1.31x faster
np.all(b50) 584 ns 381 ns: 1.53x faster
np.any(b50) 569 ns 370 ns: 1.54x faster
np.sum(a100k) [control] not significant
np.sum(a2d, axis=0) [control] 2.92 µs 3.11 µs
Geometric mean (ref) 1.34x faster

The two control cases do not use the scalar path. The small difference on the axis=0 control is binary-layout noise.

Benchmark script
"""pyperf benchmarks for the full-reduction scalar fast path.

Representative cases: full reductions (axis=None) returning a scalar,
where the fast path can construct the numpy scalar directly.
Controls: large arrays (loop-dominated) and axis reductions (fast path
not taken for the wrapping change).
"""
import pyperf
import numpy as np

runner = pyperf.Runner()
runner.metadata['numpy_version'] = np.__version__

setup = """
import numpy as np
a10 = np.arange(10, dtype=np.float64)
a1000 = np.arange(1000, dtype=np.float64)
a100k = np.arange(100_000, dtype=np.float64)
i10 = np.arange(10, dtype=np.int64)
i50 = np.arange(50, dtype=np.int64)
b50 = np.ones(50, dtype=bool)
a2d = np.ones((100, 100))
"""

benchmarks = [
    # small full reductions -- the target of the optimization
    ("np.sum(a10)", "np.sum(a10)"),
    ("np.sum(a1000)", "np.sum(a1000)"),
    ("np.max(a10)", "np.max(a10)"),        # no identity: seeds from arr[0]
    ("np.max(a1000)", "np.max(a1000)"),
    ("np.min(i50)", "np.min(i50)"),
    ("np.add.reduce(a10)", "np.add.reduce(a10)"),
    ("np.multiply.reduce(i10)", "np.multiply.reduce(i10)"),
    ("a1000.sum()", "a1000.sum()"),
    ("np.all(b50)", "np.all(b50)"),    # bool singleton scalars
    ("np.any(b50)", "np.any(b50)"),
    # controls -- should be unchanged
    ("np.sum(a100k) [control]", "np.sum(a100k)"),
    ("np.sum(a2d, axis=0) [control]", "np.sum(a2d, axis=0)"),
]

for name, stmt in benchmarks:
    runner.timeit(name, stmt, setup=setup)

Run with python bench_reduce_scalar.py -o result.json on each build, then python -m pyperf compare_to main.json pr.json --table.

AI Disclosure

Claude Fable was used in prototyping, refining and benchmarking the PR.

@ngoldbaum ngoldbaum left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Neat, just one driveby comment about the unrelated docs change.

You said fable helped you - IMO it produces way too many comments without a lot of prompting to discourage it. Also IMO, this PR as-is probably has too many comments.

Comment thread numpy/_core/src/umath/ufunc_object.c Outdated
@eendebakpt

Copy link
Copy Markdown
Contributor Author

You said fable helped you - IMO it produces way too many comments without a lot of prompting to discourage it. Also IMO, this PR as-is probably has too many comments.

Agreed. I already have statements in the memory files begging to be concise in comments, but it is still quite verbose. I will adapt the comments

@eendebakpt
eendebakpt force-pushed the reduce_scalar_fastpath branch from d654d5a to 9dff34c Compare August 12, 2026 22:21
When the caller would convert the 0-d result to a scalar anyway,
accumulate into a stack buffer and create the scalar directly with
PyArray_Scalar, skipping the temporary 0-d array and the wrap
machinery.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@eendebakpt
eendebakpt force-pushed the reduce_scalar_fastpath branch from 9dff34c to 8e249b8 Compare August 12, 2026 22:24
/* Allocate one 0-d result per output so the loop can write into them.
* With ``return_scalar`` set, accumulate into a stack buffer.
*/
npy_clongdouble buffer; /* aligned scratch large enough for any number */

@ngoldbaum ngoldbaum Aug 13, 2026

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't like that this relies on the platform-dependent size and alignment of this type and a type punning pattern that, per the C standard, is UB. We don't compile with -f-strict-aliasing but it would be nice if we could in the future and I'd rather not add even more mechanisms based on type punning.

Instead, this could be an instance of a union:

typedef union {
    npy_bool boolean;
    npy_byte byte;
    ... # all the other non-flexible non-reference builtin dtypes
    npt timedelta timedelta;
} reduction_scalar_storage

Then below you could explicitly switch on descr->type_num and point accum[0] at the corresponding member of the union like &storage.double_ for double-precision float accumulators.

That also allows you to exclude object, StringDType, flexible legacy string dtypes, and void by construction. You also get a stack-allocated accumulator that is automatically the correct type, size, and alignment for the buffer without relying on the platform-dependent size and alignment of npy_clongdouble.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants