bpo-44775: Speed-up typing.cast by implementing it in C by uriyyo · Pull Request #27474 · python/cpython

uriyyo · 2021-07-30T09:37:00Z

https://bugs.python.org/issue44775

uriyyo · 2021-07-30T09:37:14Z

@Fidget-Spinner Could you please review this PR?

pablogsal · 2021-07-30T09:56:42Z

+_typing.cast -> object
+
+    typ: object
+    val: object


If you make the value positional only this couldn't probably be even faster

Agree it will make cast faster, but in such case it won't be compatible with current version of typing.cast that can accept arguments as keywords.

Um, does mypy even accept keyword args? I don't see any reason why anyone should use keyword args for cast().

It doesn't: https://mypy-play.net/?mypy=latest&python=3.10&gist=ecd557de5e17eb73097742a740e77c37.

Okay then I think the concern is moot.

If we drop keyword support and make this only METH_FASTCALL, the specializer may eventually make this 8% faster (it currently only does so for CALL_FUNCTION, not CALL_METHOD).

@uriyyo could you consider removing keyword support please?

For consistency, we can also make the Python typing.cast version keyword-only (we will need to add an entry in What's New and news for that later since it breaks invalid code).

pablogsal · 2021-07-30T09:58:48Z

+_typing_cast_impl(PyObject *module, PyObject *typ, PyObject *val)
+/*[clinic end generated code: output=11224a3fa037a9a1 input=4dce97a747d81e0a]*/
+{
+    Py_INCREF(val);


I am a bit concerned about this new module where all the functions do the same thing (just return the same value). We should speed up the calls in general as implementing these trivial functions in c to avoid the call overhead feels like a maintainance concern to me.

To be clear, in not opposed to this pr but it starts to feel like a anti-pattern smell.

I have the same concern with @pablogsal and I would like to request to typing modules authors to follow conventional CPython accelerated extension module structure which is written based on class, function unit not the partial instance method, etc. (This case looks proper as module function unit)

Those extension modules maintain Python and C versions even though maintain cost exists.
And those decisions are based on that the accelerated gain is worth than maintain cost.
So If we feel that following conventional structure is too complicated as the maintenance view,
IMHO, it means that the accelerated version is not worth writing.
For my example, I love to implement vectorcall but I decided to drop my enumerate vectorcall because the implementation was too complicated. (see #25154)

cc @Fidget-Spinner

Anyway, this PR makes 2x faster on my local machine.

Mean +- std dev: [cast_base] 114 ns +- 5 ns -> [cast_pr] 56.0 ns +- 1.8 ns: 2.03x faster

import pyperf runner = pyperf.Runner() runner.timeit(name='bench typing.cast', stmt='val = typing.cast(int, num)', setup = ''' import typing import random num = random.randint(0, 10000) ''')

@pablogsal @corona10 Thanks for your opinion, and I agree with your points.

Case of typing.cast is super simple like it was with typing.NewType.__call__ and we can get x2 boost with small efforts and in my opinion it will be great to have such boost)

I understand your concerns regarding maintenance costs and it's valid point for me.

And those decisions are based on that the accelerated gain is worth than maintain cost.

@corona10 thank for pinging me. To be honest I don't know enough to comment. Properly benchmarking typing is very hard and still unsolved. IMO, there are 3 aspects to typing performance:

Static type checkers implemented in Python like mypy.

Runtime type checkers/introspection like Pydantic.

Runtime overhead of typed code vs fully untyped code.

All 3 will benefit from general CPython optimizations (eg. Vectorcall, specialization, cache). Although only 3. will benefit the most from speedups to typing module. 2. will benefit a little. 1. will have little benefit.

I'm working on covering these cases in pyperformance (I already submitted 1 open PR for case 1.). And I will send out an email to typing-sig for discussion on 2. and 3. soon.

To be clear, I am not against this PR at all (I'm +0). I just don't have enough data at the moment to decide :(.

The entire reason _typing exists is that the identity function it houses was deemed improper to put inside _functools or _operator. I think the identity functions have a place as they are next to trivial development-wise but provide a significant performance improvement.

I understand @corona10's argument about providing a full symmetrical accelerated C module vs. a Python module but this currently neither necessary (because it's just two identity functions) nor particularly feasible (because it's a lot of duplicate work and typing is still evolving quite rapidly).

IMO it would be best just to take this improvement as is.

but this currently neither necessary (because it's just two identity functions) nor particularly feasible (because it's a lot of duplicate work and typing is still evolving quite rapidly).

That is necessary per PEP399, as these functions are going to be the public API

In particular:

This PEP requires that in these instances that the C code must pass the test suite used for the pure Python code to act as much as a drop-in replacement as reasonably possible.

Given that we are seeking speed, we could drop a bit the requirement and use positional-only, but we are on the line of the interpretation of PEP 399

bedevere-bot · 2021-07-30T10:00:36Z

A Python core developer has requested some changes be made to your pull request before we can consider merging it. If you could please address their requests along with any other requests in other reviews from core developers that would be appreciated.

Once you have made the requested changes, please leave a comment on this pull request containing the phrase I have made the requested changes; please review again. I will then notify any core developers who have left a review that you're ready for them to take another look at this pull request.

uriyyo · 2021-07-30T11:43:21Z

I have made the requested changes; please review again

bedevere-bot · 2021-07-30T11:43:24Z

Thanks for making the requested changes!

@pablogsal: please review the changes made to this pull request.

ambv · 2021-07-30T11:49:50Z

This passes -m test -R: test_typing test_types.

Fidget-Spinner · 2021-07-31T03:14:06Z

A branch from this comment python/pyperformance#105 (comment):

@JelleZijlstra thanks for your insightful thoughts. This PR by Yurii speeds up typing.cast in C . If you think it will make a difference, I can bench mypy (non mypyc-compiled form) in pyperformance and see.

markshannon · 2021-08-09T11:32:05Z

Do we have any evidence that this will be faster for 3.11 in real programs?
The micro benchmark is so small that all the parts of the interpreter that are used and the new code fit in L1 Icache, but in normal programs the new code will evict part of the interpreter. Conversely, you could argue that it reduces the pressure on the Dcache, as there is no need for the Python function and code object. The point is that micro benchmarks like the one given do not represent the real performance impact.

Calls to Python functions will be faster in 3.11. I can't promise how much, probably not enough to close the gap in this case, but it changes the maintenance/performance ratio.
Once we have tracing/function-inlining (3.12/3.13?) then the Python version will be faster.

[The fastest possible implementation would be to remove the call at compile time. PEP 638 anyone? 🙂]

github-actions · 2021-09-09T00:05:21Z

This PR is stale because it has been open for 30 days with no activity.

Fidget-Spinner · 2021-10-20T12:15:44Z

@uriyyo can I trouble you to rebase this against main and try the microbenchmarks again please?

Recently, we've optimized function calling to inline python calls, so the Python versions of typing.cast() shouldn't be as slow now (although we've also optimized C METH_O calling by 20%, so maybe it balanced out 😆 ?).

uriyyo · 2021-10-20T12:53:21Z

@Fidget-Spinner Results of my local run:

+-------------------+-----------+-----------------------+
| Benchmark         | c_version | py_version            |
+===================+===========+=======================+
| bench typing.cast | 29.5 ns   | 54.5 ns: 1.85x slower |
+-------------------+-----------+-----------------------+

Fidget-Spinner · 2021-10-20T13:38:41Z

@Fidget-Spinner Results of my local run:

Thank you! Can I clarify if that's the pyperf script by Dong-hee at #27474 (comment), or is that from timeit script on your bpo?

Either way, the results show that python calls have sped up. It used to be 2x slower 😉 .

After a month, I'm now strongly +1 in favor of this. Since @ambv is also in favor (from what I infer, sorry if I'm mistaken), that makes two typing maintainers (I also strongly agree with Łukasz' comments here) . I would love to have syntactic macros over this, but I don't think that's going to land in 3.11. So unless there's anyone strongly -1, I plan to merge this in a week.

uriyyo · 2021-10-20T15:34:38Z

@Fidget-Spinner You are right, I used pyperf script by Dong-hee.

markshannon · 2021-10-20T17:41:02Z

The Python version should be even quicker now, and will eventually be faster than the C version 🙂

Fidget-Spinner · 2021-10-21T03:04:33Z

The Python version should be even quicker now, and will eventually be faster than the C version 🙂

@markshannon the Python version went from 2x slower to 1.85x slower (a huge improvement!). But currently the C version is still quite far ahead.

I suspect the Python version may become faster in 3.12/13. So we can accept this change for now and revert it whenever that happens. Users will get to enjoy the speed bump from 3.11 onwards.

# Conflicts: # Doc/whatsnew/3.11.rst

uriyyo · 2021-10-28T16:58:25Z

@Fidget-Spinner Could you please review this PR?

markshannon · 2021-10-29T11:01:29Z

I'm still not convinced. Sure, this makes typing.cast faster on some contrived microbenchmarks, but does it make a difference in real code?
If typing.cast is heavily used in real code, then we should discuss the best way to optimize this function in the medium term.
If is not widely used, then why bother?

For example, we might want to specialize for all return self functions, by simply skipping them. Opaque C code is a barrier to such optimizations.

JelleZijlstra · 2022-04-13T03:51:43Z

@Fidget-Spinner do you think this is still worth merging? We should probably make a decision before the feature freeze.

For what it's worth, I think it's probably not worth the additional complexity.

Fidget-Spinner · 2022-04-13T15:55:14Z

@JelleZijlstra I still think it's worth it but with how contentious it is I'd say we shouldn't merge it. Once again, I have no clear insight into how common typing.cast is used in the paths of performance critical code.

JelleZijlstra · 2022-04-30T22:43:37Z

Seems like there's not enough support to merge this. Thanks for your contribution, and sorry it took so long!

insilications · 2026-03-05T13:21:30Z

So no plans to reconsider this, correct?

bpo-44775: Speed-up typing.cast by implementing it in C

9e07b2e

uriyyo requested a review from gvanrossum as a code owner July 30, 2021 09:37

bedevere-bot added the awaiting review label Jul 30, 2021

the-knights-who-say-ni added the CLA signed label Jul 30, 2021

Update _typing.cast signature

79f7bcd

pablogsal requested changes Jul 30, 2021

View reviewed changes

bedevere-bot added awaiting changes and removed awaiting review labels Jul 30, 2021

Fidget-Spinner added the type-feature A feature request or enhancement label Jul 30, 2021

Add docstring for _typing.cast

2f48aff

uriyyo requested a review from pablogsal July 30, 2021 11:43

bedevere-bot added awaiting change review and removed awaiting changes labels Jul 30, 2021

github-actions Bot added the stale Stale PR or inactive for long period of time. label Sep 9, 2021

Merge remote-tracking branch 'upstream/main' into fix-issue-44775

634b506

Remove keyword support for typing.cast

7d78279

Merge remote-tracking branch 'upstream/main' into fix-issue-44775

1477a08

# Conflicts: # Doc/whatsnew/3.11.rst

Fidget-Spinner reviewed Oct 29, 2021

View reviewed changes

Comment thread Doc/whatsnew/3.11.rst Outdated

Comment thread Misc/NEWS.d/next/Library/2021-07-30-12-35-10.bpo-44775.gDLT2g.rst Outdated

Fidget-Spinner added 2 commits October 29, 2021 18:18

Update news

3dcbf9e

Update whatsnew

5cd9a06

Fidget-Spinner approved these changes Oct 29, 2021

View reviewed changes

bedevere-bot added awaiting merge and removed awaiting change review labels Oct 29, 2021

Fidget-Spinner added DO-NOT-MERGE and removed stale Stale PR or inactive for long period of time. labels Oct 29, 2021

AlexWaygood added topic-typing performance Performance or resource usage labels Apr 13, 2022

JelleZijlstra closed this Apr 30, 2022

uriyyo mentioned this pull request Apr 30, 2022

Speed-up typing.cast by implementing it in C #88938

Closed

bdraco mentioned this pull request May 28, 2022

Guard expensive casts in performance sensitive spots with if TYPE_CHECKING home-assistant/core#71960

Merged

22 tasks

coderabbitai Bot mentioned this pull request Mar 31, 2026

Type system UI formatter and payload contracts for MyPy rollout arthexis/arthexis#6642

Merged

Uh oh!

Uh oh!

Conversation

uriyyo commented Jul 30, 2021 • edited by bedevere-bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

uriyyo commented Jul 30, 2021

Uh oh!

Choose a reason for hiding this comment

Uh oh!

uriyyo Jul 30, 2021 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

corona10 Jul 30, 2021 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

corona10 Jul 30, 2021 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Fidget-Spinner Jul 30, 2021 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

pablogsal Jul 30, 2021 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

pablogsal Jul 30, 2021 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

bedevere-bot commented Jul 30, 2021

Uh oh!

uriyyo commented Jul 30, 2021

Uh oh!

bedevere-bot commented Jul 30, 2021

Uh oh!

ambv commented Jul 30, 2021

Uh oh!

Fidget-Spinner commented Jul 31, 2021 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

markshannon commented Aug 9, 2021

Uh oh!

github-actions Bot commented Sep 9, 2021

Uh oh!

Fidget-Spinner commented Oct 20, 2021

Uh oh!

uriyyo commented Oct 20, 2021

Uh oh!

Fidget-Spinner commented Oct 20, 2021

Uh oh!

uriyyo commented Oct 20, 2021

Uh oh!

markshannon commented Oct 20, 2021

Uh oh!

Fidget-Spinner commented Oct 21, 2021 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

uriyyo commented Oct 28, 2021

uriyyo commented Jul 30, 2021 •

edited by bedevere-bot

Loading

uriyyo Jul 30, 2021 •

edited

Loading

corona10 Jul 30, 2021 •

edited

Loading

corona10 Jul 30, 2021 •

edited

Loading

Fidget-Spinner Jul 30, 2021 •

edited

Loading

pablogsal Jul 30, 2021 •

edited

Loading

pablogsal Jul 30, 2021 •

edited

Loading

Fidget-Spinner commented Jul 31, 2021 •

edited

Loading

Fidget-Spinner commented Oct 21, 2021 •

edited

Loading

insilications commented Mar 5, 2026 •

edited

Loading