typing caches keep extension types alive past shutdown

Bug report

Bug description:

typing caches every subscripted type in global lru_cache. In real-world applications, this almost unavoidably leads to interpreter-wide leaks across extensions that leak checkers like valgrind and binding tools like nanobind complain about.

The problem has several components:

An expression like typing.List[ext.Widget] creates a cache entry that holds a reference to Widget. In principle this should be cleared up when typing is GCed at interpreter shutdown.
Many of the big ML framework (e.g., torch) hold references to typing that are never released when Python shuts down. Typing is an extremely commonly used module, and many existing extensions do have benign leak issues at shutdown time. This problem is unfixable, correcting every Python extension out there is a "let's boil the ocean" kind of impossible task.
typing now greatly exacerbates this leak because it also keeps an innocent and well-defined extension from shutting down in an orderly manner. This often comes up with the nanobind binding generator, which loudly complains about any leak of types/functions/instances at shutdown time, where the above behavior can generate thousands of warnings about issues that are ultimately caused by the dense web of uncollectable references generated by typing.

A single such reference can keep a whole extension from being torn down, producing a burst of warnings like the following, none of which point to an actual bug in the extension:

nanobind: leaked 287 types!
 - leaked type "mymodule.Widget"
 - leaked type "mymodule.Mesh"
 - leaked type "mymodule.Transform"
 - ... skipped remainder
nanobind: leaked 1844 functions!
 - leaked function "Widget.__init__"
 - leaked function "Mesh.subdivide"
 - ... skipped remainder
nanobind: this is likely caused by a reference counting issue in the binding code.
See https://nanobind.readthedocs.io/en/latest/refleaks.html

This issue already came up once before in #98253, but this fix only broke a cycle inside _tp_cache and left the retention in place. Basically this fix was not enough.

Reproducer

A minimal CPython extension demonstrating the problem is available at https://github.com/wjakob/typing-leak.

ext.c is a tiny extension with just two pieces: Box, an object with no tp_traverse (so a cycle through it is invisible to the GC), and Meta, a metaclass that prints when one of its classes is allocated and deleted.

pip install -e .
python good.py    # Widget allocated / Widget deleted
python bad.py     # Widget allocated   (never deleted: leaked)

good.py defines a class and mentions it in one annotation:

import typing, ext

class Widget(metaclass=ext.Meta):
    pass

def f() -> typing.List[Widget]:
    ...

Widget is freed at shutdown, as it should be. bad.py is the same, plus an uncollectable cycle that holds one unrelated annotation:

box = ext.Box()
box.ref = [box, typing.Dict[str, int]]

Fix

typing cannot rely on being garbage-collected at shutdown, so it should clear its caches explicitly. The cache_clear callables are already gathered in typing._cleanups; they just never run. Registering them with atexit is enough:

# Lib/typing.py
import atexit
atexit.register(lambda: [clear() for clear in _cleanups])

It is also the same fix I suggested the last time this came up, which was not deemed nice. Please consider it, I really do think this needs to be fixed and hope this makes a case for it.

Affected Python versions

I reproduced the issue on Python 3.10 through 3.14, free-threaded Python included.

CPython versions tested on:

CPython main branch

Operating systems tested on:

Linux

Linked PRs

I'd like to work on this issue. I'll investigate the cache cleanup approach and see if I can prepare a PR.

Just to be clear: do the types actually leak in the sense that free() is never called, or just that they are alive for much longer than they need to be?

I want to ask as well, but in the ext.c code, you explicitly didn't add tp_traverse nor tp_clear. Isn't it the reason why you have a leak? the type is also not marked as a GC-type.

The types leak in the sense that they aren't freed by the time that linux shuts down the process.

The point of ext.c is precisely to model a "bad" package that keeps typing alive. The mistake (missing tp_traverse) is intentional. What the reproducer demonstrates is that this causes a leak elsewhere (in this case, for Widget).

So mental model: Widget, the nice package that tries to do everything right (including checking for leaks via a metaclass). ext.Box, the bad guy somewhere out there in the Python ecosystem.

Yeah but why is typing the bad guy then? the problem is in ext, so why should we actually change typing itself?

Yeah but why is typing the bad guy then?

Well.. that's more or less exactly the same answer I got last time. Please consider the role that typing plays in this story (bullet point 3 in the description at the top).

Perhaps it's best to give a human analogy: this is like a hotel where checkout requires all guests to behave perfectly, but one rude person sneaks out with unresolved paperwork. Instead of treating that as that guest's problem, the hotel has a policy of turning that into everyone else's problem by keeping them hostage in the lobby until the end of time. typing is the hotel.

Any of a bazillion packages out there with imperfect reference counting can inflict reference leaks on my package, thanks to typing which acts as a kind of middleman and re-distributor of reference leaks. I don't think this is a good system.

"Fix the other extension" is not a response that one can give with a straight face here. It is a "boil the ocean" kind of impossible task. Major packages have these problems. For example, you can easily reproduce the issue also with import torch.

The technical solution to this problem is a trivial 3-line addition to typing.py:

import atexit
atexit.register(lambda: [clear() for clear in _cleanups])
del attest

Register a cleanup callback that runs when Python shuts down. The callback will then clear the LRU cache, which will clear any types held hostage through it. Every package can shut down orderly and leak-free, which makes focusing on actual leak-related issues possible.

I'm fine with adding an atexit to clear the typing caches, but a couple of questions:

Do other standard library modules with internal caches use atexit to clear them? Or is typing simply unique in how much it caches?
I wonder how much we even gain from these caches. Annotations are no longer evaluated by default in 3.14+, and many typing types are now in C and don't use caches. (I considered this when rewriting Union for 3.14, and it seemed that the overhead of re-adding a cache was usually not worth it.)
We could also switch to weakref-based caches, but that might make performance worse.
What's the concrete bad effect from the "leak" here? In some sense, clearing the caches in atexit isn't too different from leaking them at process shutdown: they stay alive until the end of the useful life of the process in either case. Would fixing this make it easier to find other, less benign leaks in nanobind-using extension?

Do other standard library modules with internal caches use atexit to clear them? Or is typing simply unique in how much it caches?

typing stands out because it is so widespread. Every big project is going to use it somewhere.

I can't comment on your two questions regarding much those caches actually help typing improve performance.

What's the concrete bad effect from the "leak" here? [..]

Python has two very different at-exit callbacks. The one being discussed here is atexit.register(callable). Python is still functional when invoking callables registered that way. The garbage collector runs after this stage and can clean up any remaining disconnected reference cycles.

The second mechanism is Py_AtExit() in the C API, which runs callback when Python has shut down. Python is really completely dead then, even basic operations like Py_DECREF() will segfault the process.

What nanobind does is to register a callback with the latter (Py_AtExit()), which makes an account of all the nanobind-owned types, functions, and instances. If any of them haven't been reclaimed, then it means there is a GC problem. Besides nanobind, this will also make memory leak trackers like valgrind unhappy.

Note that nanobind can't really "clean" up at this stage, as that would probably lead to crashes since it would also need to call C++ destructors that may try to call back into Python. So the only remaining choice is to warn.

So to me there is a huge difference between cleaning up in atexit.register() and not cleaning up at all. One still gives the opportunity of shutting down the process in an orderly manner, while the other one does not.

I wasn't trying to say we don't need to do anything on typing. I wanted to understand why typing would be the only culprit in the story but now I have my answer. To summarize you had in your previous PR is actually not entirely solved because we are still not using those cleanups right? So, considering we don't call them, up until now, it wouldn't have made sense to keep them (they are private variables).

I think we could have a typing.clear_cache() public function (or typing.cache_clear(); I never remember what is our convention and whether we have one) so that users can also decide to call it if they think the typing cache is growing too large because of type annotations for instance, e.g., or when they want to call gc.collect() and be sure there are no leaks due to that.

However, the problem of ext leaking things would still remain right?

I think we could have a typing.clear_cache() public function (or typing.cache_clear(); I never remember what is our convention and whether we have one) so that users can also decide to call it if they think the typing cache is growing too large because of type annotations for instance,

This would cement an implementation detail of typing into a public API surface that will need to be supported in the future, so I am not sure it's a good idea. The reported issue here isn't that the LRU caches are getting too big, but that they redistribute reference leaks to other packages. Adding an API surface still wouldn't fix the issue with the reference leaks—it would push it to users that would have to register their own atexit cleanup callbacks to then call that function.

I think the proposed change is the easiest one: add those lines somewhere in typing.py:

# Clear LRU caches in an exit handler to avoid a ref-leaking extension using typing
# from causing reference leaks in other (innocent) typed extensions. See issue #151728 
import atexit
atexit.register(lambda: [clear() for clear in _cleanups])
del attest

This fixes the issue, does not impose any meaningful cost, and keeps the private details private without future compatibility implications.

pythongh-151728: Clear the typing caches at interpreter shutdown

@JelleZijlstra

@JelleZijlstra I opened a PR with this change.

pythongh-151728: Clear the typing caches at interpreter shutdown

gh-151728: Clear the typing caches at interpreter shutdown (GH-154858)

I don't think this is a bug, so, like in #98253, I'm not backporting the fix.

Keeping open for investigating Jelle's ideas.

pythongh-151728: Revert "Clear the typing caches at interpreter shutd…

gh-151728: Partially revert "Clear the typing caches at interpreter s…

gh-151728: Clear the typing caches at interpreter shutdown (GH-155002)

pythongh-151728: Clear the typing caches at interpreter shutdown (pyt…

pythongh-151728: Partially revert "Clear the typing caches at interpr…

pythongh-151728: Clear the typing caches at interpreter shutdown (pyt…

@encukou

@encukou This issue can be closed, right?

Navigation Menu

Repository navigation

`typing` caches keep extension types alive past shutdown #151728

Bug report

Bug description:

Reproducer

Fix

Affected Python versions

CPython versions tested on:

Operating systems tested on:

Linked PRs

Metadata

Assignees

Labels

Projects

Milestone

Relationships

Development

typing caches keep extension types alive past shutdown #151728

Description

Bug report

Bug description:

Reproducer

Fix

Affected Python versions

CPython versions tested on:

Operating systems tested on:

Linked PRs

Activity

amirhd4 commented on Jun 19, 2026

ZeroIntensity commented on Jun 20, 2026

picnixz commented on Jun 20, 2026

wjakob commented on Jun 20, 2026

picnixz commented on Jun 20, 2026

wjakob commented on Jun 20, 2026

JelleZijlstra commented on Jun 20, 2026

wjakob commented on Jun 20, 2026

picnixz commented on Jun 21, 2026

wjakob commented on Jun 21, 2026

wjakob commented on Jul 29, 2026

encukou commented on Jul 30, 2026

wjakob commented on Sep 23, 2026

Metadata

Metadata

Assignees

Labels

Projects

Milestone

Relationships

Development

Issue actions

`typing` caches keep extension types alive past shutdown #151728