Skip to content

gh-154675: Use lazy imports in pdb, bdb, code - #156567

Open
hugovk wants to merge 1 commit into
python:mainfrom
hugovk:3.16-lazy-pdb
Open

gh-154675: Use lazy imports in pdb, bdb, code#156567
hugovk wants to merge 1 commit into
python:mainfrom
hugovk:3.16-lazy-pdb

Conversation

@hugovk

@hugovk hugovk commented Aug 29, 2026

Copy link
Copy Markdown
Member

The goal here is to the improve import time of pdb, one of the slowest stdlib modules to import.

Before: 52 ms

main on macOS with optimised build:

image

After: 9 ms

We can lazily import several dependencies, and also do the same in its dependencies bdb and code to make them lighter:

image

pdb.set_trace() (or breakpoint() with defaults) is commonly used for this module, and will reify 6 of the 18 lazy imports here, and the other 12 remain deferred.

And pdb is often imported without running set_trace(), so we get the benefit of all 18 lazy imports.

This also speeds up every pytest invocation, not just pytest --pdb: pytest unconditionally imports pdb at startup via its default debugging plugin. The pytest project found deferring import of pdb saved ~7% of pytest startup on Python 3.14 (pytest-dev/pytest#14874), but the change was declined in favour of improving import time in CPython itself.

@hugovk
hugovk requested a review from gaogaotiantian as a code owner August 29, 2026 11:07
@hugovk hugovk added stdlib Standard Library Python modules in the Lib/ directory performance Performance or resource usage labels Aug 29, 2026
Comment thread Lib/pdb.py
import _pyrepl.utils
except ModuleNotFoundError:
_pyrepl = None
lazy import _colorize

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hmm, this is now different to the other files, where we have lazy imports directly after the regular imports.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The isort/ruff rule is:

import a
from b import c
lazy import d
lazy from e import f

And this is following that, except I left the spaces between groups, but can remove those if you like?

@gaogaotiantian

Copy link
Copy Markdown
Member

I think making some import lazy definitely makes sense. Making import pdb faster benefits more than pdb itself - pytest is a great example. But how did we choose the libraries to lazy import? Is it quite arbitrary? Is there a rule to follow in stdlib - like which are the libraries we should lazy import? Or did we leave the ones that will "definitely" be used? I'm not against the current list, but it might be clearer for future developers if we have a guidance to follow.

@hugovk

hugovk commented Aug 29, 2026

Copy link
Copy Markdown
Member Author

I ran:

pip install tuna
./python.exe -X importtime -c "import pdb" 2> import.log
tuna import.log

This gives the blue diagram to visualise the import times.

Then looked at the slowest imports, and made them lazy one by one.

Some would gett reified right away, like bdb (due to class Pdb(bdb.Bdb, cmd.Cmd)) and code (due to class _PdbInteractiveConsole(code.InteractiveConsole)), so I left those as eager imports but also made some of their own imports lazy in the same way to make then lighter.

Others like typing also get reified immediately (due to @typing.override), so that was also left as eager. We've tackled making typing faster before.

The tests ensure that a lazy import doesn't end up getting immediately reified due to some other change. If that happens down the line, we can either do something to ensure it remains lazy, or make a conscious decision to allow it be eager, and update the test.

We also have some deferred imports, like runpy, which are imported inside functions. This was the pre-lazy way of doing it, so we can now hoist them up as explicit lazy imports.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

awaiting core review performance Performance or resource usage stdlib Standard Library Python modules in the Lib/ directory

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants