Skip to content

Draft: MAINT, ENH: Tuple-spec array_function_dispatch #31943

Open
eendebakpt wants to merge 4 commits into
numpy:mainfrom
eendebakpt:fast_dispatcher_rebased
Open

Draft: MAINT, ENH: Tuple-spec array_function_dispatch #31943
eendebakpt wants to merge 4 commits into
numpy:mainfrom
eendebakpt:fast_dispatcher_rebased

Conversation

@eendebakpt

@eendebakpt eendebakpt commented Jul 9, 2026

Copy link
Copy Markdown
Contributor

We allow @array_function_dispatch to take a tuple of relevant-argument names instead of a callable dispatcher. The names are resolved to (kwname, position) pairs at decoration time. The tuple approach avoids the overhead of calling the dispatcher and creating an intermediate tuple. Fast paths are added for exact numpy arrays and python basic types.

Fixes #31866.

About 260 trivial dispatchers are migrated (~600 lines removed); the callable form remains for sequence dispatchers

Benchmark results (Python 3.14):

np.sum(array_4): Mean +- std dev: [bench_main_sq] 621 ns +- 5 ns -> [bench_branch_sq] 576 ns +- 4 ns: 1.08x faster
np.sum(array_20): Mean +- std dev: [bench_main_sq] 625 ns +- 4 ns -> [bench_branch_sq] 579 ns +- 3 ns: 1.08x faster
np.prod(array_20): Mean +- std dev: [bench_main_sq] 598 ns +- 3 ns -> [bench_branch_sq] 550 ns +- 7 ns: 1.09x faster
np.max(array_20): Mean +- std dev: [bench_main_sq] 591 ns +- 3 ns -> [bench_branch_sq] 560 ns +- 10 ns: 1.06x faster
np.sum(int_20): Mean +- std dev: [bench_main_sq] 628 ns +- 9 ns -> [bench_branch_sq] 585 ns +- 14 ns: 1.07x faster
np.sum(10x10): Mean +- std dev: [bench_main_sq] 635 ns +- 5 ns -> [bench_branch_sq] 587 ns +- 3 ns: 1.08x faster
np.sum(10x10,axis=0): Mean +- std dev: [bench_main_sq] 1.20 us +- 0.02 us -> [bench_branch_sq] 1.14 us +- 0.02 us: 1.05x faster
np.sum(10x10,axis=1): Mean +- std dev: [bench_main_sq] 1.14 us +- 0.02 us -> [bench_branch_sq] 1.10 us +- 0.02 us: 1.04x faster
np.any(bool_20): Mean +- std dev: [bench_main_sq] 572 ns +- 16 ns -> [bench_branch_sq] 516 ns +- 8 ns: 1.11x faster
np.sum(2x2_float): Mean +- std dev: [bench_main_sq] 628 ns +- 8 ns -> [bench_branch_sq] 575 ns +- 5 ns: 1.09x faster
np.sum(2x2_int): Mean +- std dev: [bench_main_sq] 626 ns +- 7 ns -> [bench_branch_sq] 584 ns +- 9 ns: 1.07x faster
np.sin(array_20): Mean +- std dev: [bench_main_sq] 371 ns +- 6 ns -> [bench_branch_sq] 376 ns +- 4 ns: 1.01x slower
np.sum(subclass_20): Mean +- std dev: [bench_main_sq] 972 ns +- 10 ns -> [bench_branch_sq] 945 ns +- 16 ns: 1.03x faster
np.transpose(10x10): Mean +- std dev: [bench_main_sq] 259 ns +- 2 ns -> [bench_branch_sq] 226 ns +- 1 ns: 1.15x faster
np.ravel(array_20): Mean +- std dev: [bench_main_sq] 233 ns +- 4 ns -> [bench_branch_sq] 196 ns +- 2 ns: 1.19x faster

Geometric mean: 1.08x faster

np.sin is the control (not dispatched through @array_function_dispatch).

Benchmark script
"""Micro-benchmark for np.sum/prod/max/any dispatcher overhead."""
import numpy as np
import pyperf


class MyArr(np.ndarray):
    pass


def make_setup():
    return {
        "array_4":    np.arange(4, dtype=np.float64),
        "array_20":   np.arange(20, dtype=np.float64),
        "int_20":     np.arange(20, dtype=np.int64),
        "bool_20":    np.ones(20, dtype=bool),
        "arr_10x10":  np.arange(100, dtype=np.float64).reshape(10, 10),
        "arr_2x2f":   np.arange(4, dtype=np.float64).reshape(2, 2),
        "arr_2x2i":   np.arange(4, dtype=np.int64).reshape(2, 2),
        "subclass_20":np.arange(20, dtype=np.float64).view(MyArr),
    }


BENCHES = [
    ("np.sum(array_4)",         "np.sum(a['array_4'])"),
    ("np.sum(array_20)",        "np.sum(a['array_20'])"),
    ("np.prod(array_20)",       "np.prod(a['array_20'])"),
    ("np.max(array_20)",        "np.max(a['array_20'])"),
    ("np.sum(int_20)",          "np.sum(a['int_20'])"),
    ("np.sum(10x10)",           "np.sum(a['arr_10x10'])"),
    ("np.sum(10x10,axis=0)",    "np.sum(a['arr_10x10'], axis=0)"),
    ("np.sum(10x10,axis=1)",    "np.sum(a['arr_10x10'], axis=1)"),
    ("np.any(bool_20)",         "np.any(a['bool_20'])"),
    ("np.sum(2x2_float)",       "np.sum(a['arr_2x2f'])"),
    ("np.sum(2x2_int)",         "np.sum(a['arr_2x2i'])"),
    ("np.sin(array_20)",        "np.sin(a['array_20'])"),
    ("np.sum(subclass_20)",     "np.sum(a['subclass_20'])"),
    # cheap view ops migrated in the mass tuple-spec conversion
    ("np.transpose(10x10)",     "np.transpose(a['arr_10x10'])"),
    ("np.ravel(array_20)",      "np.ravel(a['array_20'])"),
]


def main():
    runner = pyperf.Runner()
    runner.metadata['numpy_version'] = np.__version__
    a = make_setup()
    for name, expr in BENCHES:
        runner.timeit(name=name, stmt=expr, globals={'np': np, 'a': a})


if __name__ == '__main__':
    main()

AI Disclosure

Claude Opus 4.x was used to find optimizations for the dispatcher overhead. The corresponding issue contains an alternative. After selecting this option, Claude was used to refine the PR.

Allow @array_function_dispatch to accept a tuple of relevant argument
names instead of a callable dispatcher.  Names are resolved to
(kwname, position) pairs at decoration time; dispatcher_vectorcall
extracts the relevant args directly in C and skips the override
machinery when every one is an exact ndarray or a basic Python type.
Keyword-only args are never matched positionally and positional-only
args never by keyword, so signature-invalid calls still raise
TypeError.

Migrate ~260 trivial dispatchers to the new form (~600 lines removed).
The legacy callable form is unchanged and still used by dispatchers
with sequence arguments (concatenate, stack, ...); it also benefits
from an early return in get_implementing_args_and_methods when all
args are override-free.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@github-actions

This comment has been minimized.

Comment thread numpy/_core/overrides.py
Comment on lines +203 to +205
if is_tuple_spec:
spec = _resolve_relevant_arg_spec(implementation, dispatcher)
public_api = _ArrayFunctionDispatcher(spec, implementation)

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

did you verify that this didn't affect the inspect.signature of the decorated functions?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I added a parameterized test test_inspect_signature_matches_implementation to check for this.

@eendebakpt eendebakpt force-pushed the fast_dispatcher_rebased branch 4 times, most recently from eea450b to 1a938d7 Compare July 12, 2026 19:55
@eendebakpt eendebakpt force-pushed the fast_dispatcher_rebased branch from 1a938d7 to cbd471f Compare July 12, 2026 19:58
eendebakpt and others added 2 commits July 12, 2026 22:14
Strengthen test_inspect_sum (previously only checked that `axis` is a
parameter) and verify inspect.signature equals the implementation's
signature for both dispatcher forms, including positional-only and
keyword-only relevant args and submodule functions.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

ENH: Reduce __array_function__ dispatch overhead

3 participants