Skip to content

gh-92810: Reduce memory usage by ABCMeta.__subclasscheck__ (alternative2)#150540

Open
dolfinus wants to merge 4 commits into
python:mainfrom
dolfinus:improvement/ABCMeta_subclasscheck_v3
Open

gh-92810: Reduce memory usage by ABCMeta.__subclasscheck__ (alternative2)#150540
dolfinus wants to merge 4 commits into
python:mainfrom
dolfinus:improvement/ABCMeta_subclasscheck_v3

Conversation

@dolfinus
Copy link
Copy Markdown

@dolfinus dolfinus commented May 27, 2026

Alternative implementation of #131914 which does not check __subclasses__ recursively.
To handle cases like:

class Number(ABC): ...

class Real(Number): ...

class Integral(Real): ...

Integral.register(int)

assert issubclass(int, Number) is True

which previously were implemented via recursive __subclasses__ check, method cls.register(subclass) is calling super(cls).register(subclass) recursively ("bubble-up" registration to all the parents).

For benchmark from #131914:

sudo ./python -m pyperf system tune
taskset -c 0 ./python benchmark.py --metaclass abc.ABCMeta --rounds 3 --classes 5000
taskset -c 0 ./python benchmark.py --metaclass _py_abc.ABCMeta --rounds 3 --classes 5000
Impl Max memory before, MB Max memory after, MB
_abc 6331 50
_py_abc 4422 61
Impl Total time before Total time after
_abc 6m 16s 1m 23s
_py_abc 8m 48s 3m 53s
Check Impl before after Impl before after
isinstance(child, Parent) _abc 0.115us
4MiB...15MiB
0.121us
2MiB...15MiB
_py_abc 0.212us
11MiB...24MiB
0.218us
11MiB...24MiB
issubclass(Child, Parent) _abc 0.105us
0MiB...1MiB
0.111us
0MiB...1MiB
_py_abc 0.203us
6MiB...8MiB
0.208us
5MiB...8MiB
isinstance(child, Grandparent) _abc 0.112us
0MiB...2MiB
0.119us
1MiB...1MiB
_py_abc 0.210us
4MiB...7MiB
0.216us
4MiB...7MiB
issubclass(Child, Grandparent) _abc 0.103us
0MiB
0.108us
0MiB
_py_abc 0.201us
0MiB...1MiB
0.207us
0MiB...1MiB
not isinstance(child, Sibling) _abc 0.113us
4MiB...14MiB
0.121us
3MiB...14MiB
_py_abc 0.348us
13MiB...23MiB
0.354us
13MiB...22MiB
not issubclass(Child, Sibling) _abc 0.105us
1MiB
0.110us
1MiB...2MiB
_py_abc 0.328us
8MiB...10MiB
0.332us
9MiB...11MiB
not isinstance(child, Cousin) _abc 0.115us
1MiB...2MiB
0.121us
1MiB...2MiB
_py_abc 0.350us
7MiB...9MiB
0.354us
7MiB...10MiB
not issubclass(Child, Cousin) _abc 0.104us
0MiB
0.110us
0MiB...1MiB
_py_abc 0.329us
4MiB
0.333us
4MiB...5MiB
not isinstance(child, Uncle) _abc 7.268us
6174MiB...6333MiB
0.125us
0MiB...1MiB
_py_abc 9.957us
4382MiB...4422MiB
0.360us
6MiB
not issubclass(Child, Uncle) _abc 7.099us
6171MiB
0.114us
0MiB
_py_abc 9.936us
4380MiB
0.336us
4MiB...5MiB

Memory increment is measured during isinstance() / issubclass() calls, not during preparation, like class creation or registration where actual registry allocation is performed. So memory usage in tables below is almost always 0.

Timing drop in _py_abc implementation for 2 first rows is due to if subclass in cls._abc_registry: check added to match _abc.c implementation.

Check Impl before after Impl before after
isinstance(child, Parent.register) _abc 0.273us
0MiB
0.272us
0MiB
_py_abc 0.440us
0MiB
0.275us
0MiB
issubclass(Child, Parent.register) _abc 0.154us
0MiB
0.157us
0MiB
_py_abc 0.427us
0MiB
0.261us
0MiB
isinstance(child, Grandparent.register) _abc 0.114us
0MiB
0.120us
0MiB
_py_abc 0.253us
0MiB
0.262us
0MiB
issubclass(Child, Grandparent.register) _abc 0.103us
0MiB
0.109us
0MiB
_py_abc 0.240us
0MiB
0.247us
0MiB
not isinstance(child, Sibling.register) _abc 0.027us
0MiB
0.030us
1MiB
_py_abc 0.028us
0MiB
0.029us
2MiB
not issubclass(Child, Sibling.register) _abc 0.018us
0MiB
0.018us
1MiB
_py_abc 0.018us
0MiB
0.018us
2MiB
not isinstance(child, Cousin.register) _abc 0.028us
0MiB
0.028us
2MiB
_py_abc 0.028us
0MiB
0.029us
3MiB
not issubclass(Child, Cousin.register) _abc 0.018us
0MiB
0.018us
2MiB
_py_abc 0.019us
0MiB
0.018us
3MiB
not isinstance(child, Uncle.register) _abc 0.249us
0MiB
0.233us
2MiB
_py_abc 0.843us
0MiB
0.866us
3MiB
not issubclass(Child, Uncle.register) _abc 0.238us
0MiB
0.227us
2MiB
_py_abc 0.815us
0MiB
0.839us
3MiB

Just to check that nothing is broken:

Check Impl before after Impl before after
not isinstance(child, Unrelated) _abc 0.028us
0MiB
0.028us
0MiB
_py_abc 0.028us
0MiB
0.029us
0MiB
not issubclass(Child, Unrelated) _abc 0.018us
0MiB
0.018us
0MiB
_py_abc 0.018us
0MiB
0.018us
0MiB
not isinstance(child, UnrelatedABC) _abc 0.118us
0MiB
0.121us
0MiB
_py_abc 0.469us
0MiB
0.466us
0MiB
not issubclass(Child, UnrelatedABC) _abc 0.110us
0MiB
0.112us
0MiB
_py_abc 0.442us
0MiB
0.442us
0MiB

Flamegraphs for _py_abc impl and test issubclass_uncle (the most time and memory consuming case on main):
main_vs_pr150540.tar.gz

@dolfinus dolfinus requested a review from AA-Turner as a code owner May 27, 2026 22:47
@bedevere-app
Copy link
Copy Markdown

bedevere-app Bot commented May 27, 2026

Most changes to Python require a NEWS entry. Add one using the blurb_it web app or the blurb command-line tool.

If this change has little impact on Python users, wait for a maintainer to apply the skip news label instead.

@read-the-docs-community
Copy link
Copy Markdown

read-the-docs-community Bot commented May 27, 2026

Documentation build overview

📚 cpython-previews | 🛠️ Build #32877634 | 📁 Comparing 49e8b49 against main (9242700)

  🔍 Preview build  

2 files changed
± whatsnew/3.16.html
± whatsnew/changelog.html

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant