Skip to content

Deprecation cleanup, decorator-dependency removal, and startup-time fixes - #15310

Merged
Carreau merged 5 commits into
mainfrom
claude/deprecated-modernization-review-frdfap
Jul 21, 2026
Merged

Deprecation cleanup, decorator-dependency removal, and startup-time fixes#15310
Carreau merged 5 commits into
mainfrom
claude/deprecated-modernization-review-frdfap

Conversation

@Carreau

@Carreau Carreau commented Jul 14, 2026

Copy link
Copy Markdown
Member

Deprecation, modernization, and performance cleanup batch for the IPython codebase, in 5 commits (see docs/modernization_triage_2026-07.md, added by the last commit, for a standalone changelog).

Deprecated APIs removed

  • IPCompleter.limit_to__all__ config option (deprecated 5.0)
  • IPCompleter.python_matches method (deprecated 8.27; superseded by python_matcher, wasn't in the active matcher set)
  • OInfo.get() (deprecated 8.13; a transitional shim from when OInfo stopped being a dict)
  • pylabtools' module-level backends/backend2gui __getattr__ shim (deprecated 8.24)
  • InteractiveShell.run_cell_async/should_run_async's implicit transform_cell fallback (deprecated 7.17; now raises TypeError instead of silently transforming; run_cell itself is unaffected)

Completer.greedy (deprecated 8.8) is kept: pyflyby's test suite, exercised by this repo's downstream CI, still sets it via %config, and removing it breaks their expected-output tests on an unrecognized-option warning.

Modernization

  • Drop the decorator runtime dependency (and types-decorator stub package): the three uses in formatters.py/history.py are rewritten as plain functools.wraps closures, typed with ParamSpec so they stay signature-preserving under mypy's disallow_untyped_decorators.
  • Collapse a typing_extensions conditional import in completer.py — everything it imported is in the 3.11 stdlib, which is the minimum supported version.
  • Delete setup.cfg (its [tool.black] section was in INI syntax, which black never reads); move the exclusion into pyproject.toml.

Performance

  • Import jedi (and parso, which compiles grammars on import) lazily on first completion instead of at IPython import time.
  • Hoist regexes that were being compiled inside hot functions (IPCompleter.global_matches, Magics.format_latex) to module level.
  • Fix a memory-retention bug: Pdb._cachable_skip/_cached_one_parent_frame_debuggerskip used a class-level @lru_cache(1024) keyed by frame objects, which kept every Pdb instance and up to 1024 frames (with their locals and back-chains) alive for the process lifetime. Now per-instance, size-bounded, cleared on each interaction.
  • Bound the previously-unbounded count_lines_in_py_file cache in tbtools.
  • Fix a latent import cycle in IPython.terminal (interactiveshell → debugger → embed → interactiveshell): terminal/debugger.py now imports terminal.embed inside do_interact, the only place it's used, instead of at module level.

@Carreau
Carreau force-pushed the claude/deprecated-modernization-review-frdfap branch from c1adcf1 to e1aef2e Compare July 16, 2026 12:46
claude added 5 commits July 17, 2026 06:24
- Import jedi (and parso, which compiles grammars on import) lazily on
  first completion instead of at IPython import time; this removes
  ~50ms (warm) from 'import IPython'. JEDI_INSTALLED now uses
  importlib.util.find_spec, and jedi configuration happens in the new
  cached _get_jedi() helper.
- Hoist regexes compiled inside functions to module level: the
  snake-case matcher in IPCompleter.global_matches (recompiled on every
  completion request), two colder completer regexes, and the five
  format_latex regexes in magic.py.
- Replace the class-level @lru_cache(1024) on Pdb._cachable_skip and
  Pdb._cached_one_parent_frame_debuggerskip with per-instance,
  size-bounded caches cleared on each interaction. The lru_cache kept
  every Pdb instance and up to 1024 frames (with their locals and whole
  back-chains) alive for the lifetime of the process.
- Bound the count_lines_in_py_file cache in tbtools (was unbounded,
  keyed by filename).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01GEyCEHEuB6vN85jweKvQU7
…s branch

- Rewrite catch_format_error (formatters), only_when_enabled and
  catch_corrupt_db (history) as plain functools.wraps closures; remove
  the 'decorator' runtime dependency and the types-decorator stubs from
  the extras and the mypy CI install. The two history.py decorators are
  typed as signature-preserving with ParamSpec, matching what the
  types-decorator stubs gave them, so disallow_untyped_decorators stays
  clean and every decorated HistoryAccessor method keeps its inferred
  type.
- completer.py: import TypedDict/Protocol/NotRequired/TypeAlias/
  TypeGuard from typing unconditionally - all are in the 3.11 stdlib,
  which is the minimum supported version. typing_extensions remains
  only for TypeAliasType in guarded_eval on 3.11.
- Delete setup.cfg: its [tool.black] section was written in INI syntax
  that black never reads; move the exclusion to pyproject.toml.
- splitinput.py: align the LineInfo.ofind() docstring with the warning
  (deprecated in 9.9, per the whatsnew entry for #15057).
- setupbase.py: drop a stale Python-2-era comment.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01GEyCEHEuB6vN85jweKvQU7
- IPCompleter.limit_to__all__ config option (deprecated 5.0): completion
  on object.<tab> now always uses dir()-based discovery.
- IPCompleter.python_matches (deprecated 8.27): superseded by
  python_matcher, and was no longer part of the active matcher set.
- OInfo.get() (deprecated 8.13, transitional helper from when OInfo
  stopped being a dict in 8.12): access dataclass fields directly.
- pylabtools module-level backends/backend2gui __getattr__ shim
  (deprecated 8.24). The underlying dicts remain: they still back the
  find_gui_and_backend fallback for Matplotlib < 3.9.
- run_cell_async/should_run_async now raise TypeError instead of
  transforming the cell themselves when transformed_cell is not passed
  (DeprecationWarning since 7.17; ipykernel >= 6 always passes it).
  run_cell is unaffected.
- test_oinspect no longer needs the decorator package: the find_file
  wrapper-chain tests use functools.wraps, which sets the same
  __wrapped__ chain.

Completer.greedy (deprecated 8.8) was evaluated for removal and kept:
downstream projects (pyflyby's test suite, run in this repo's downstream
CI) still set it via %config, and removing it makes traitlets print
"Config option not recognized", breaking their expected-output tests.

The 8.6-era matcher-v1 methods (magic_config_matches,
python_func_kw_matches, dict_key_matches, dispatch_custom_completer)
were deliberately kept too: they are the implementations their v2
matcher wrappers delegate to.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01GEyCEHEuB6vN85jweKvQU7
While investigating making IPython.embed/InteractiveShell/Application
lazy top-level attributes (reverted - see below), found a latent cycle
inside the terminal package: interactiveshell -> debugger -> embed ->
interactiveshell. It only worked because IPython/__init__.py happened
to import terminal.embed before anything reached terminal.debugger;
removing that ordering constraint made it a real ImportError.

terminal/debugger.py now imports terminal.embed inside do_interact,
the only place it's used, instead of at module level.

The lazy-attribute idea itself doesn't ship here: pyflyby's downstream
CI (green on main) broke because its _interactive.py does `import
IPython` and then accesses attribute chains like
IPython.terminal.ipapp.TerminalIPythonApp and
IPython.core.application.BaseIPythonApplication, which only resolve
through these imports' transitive side effects (its except
AttributeError path turns a missing attribute into a hard RuntimeError,
so `py` dies with EOF in their interactive tests). Left a comment
explaining the constraint; other downstreams likely have the same
pattern. A patch making pyflyby import those submodules explicitly has
been prepared for upstream, after which this is worth revisiting -
warm import time would drop from ~270ms to ~17ms.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01GEyCEHEuB6vN85jweKvQU7
Summary of this cleanup batch: the deprecated APIs removed, the
modernization changes, and the performance fixes.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01GEyCEHEuB6vN85jweKvQU7
@Carreau
Carreau force-pushed the claude/deprecated-modernization-review-frdfap branch from e1aef2e to c694a2d Compare July 17, 2026 06:30
@Carreau Carreau changed the title Performance: lazy jedi import, hoisted regexes, debugger cache leak fix Deprecation cleanup, decorator-dependency removal, and startup-time fixes Jul 17, 2026
@Carreau
Carreau force-pushed the claude/deprecated-modernization-review-frdfap branch from c694a2d to bd96ba1 Compare July 17, 2026 07:50
@Carreau
Carreau merged commit f51c0b1 into main Jul 21, 2026
25 checks passed
@Carreau Carreau added this to the 9.16 milestone Jul 24, 2026
lyskov pushed a commit to RosettaCommons/rosetta that referenced this pull request Aug 4, 2026
This PR patches a nascent bug in the `pyrosetta.distributed` required
dependencies:
`source/src/python/PyRosetta/src/pyrosetta/distributed/utility/log.py`
imports the `decorator` package, although it's not explicitly declared
as a dependency in either the unit tests or the
[`pyrosetta-distributed`](https://www.piwheels.org/project/pyrosetta-distributed/)
meta-package. Currently, the PyRosetta unit tests indirectly rely on the
`decorator` package to be installed via the `jupyter` -> `ipykernel` ->
`ipython` -> `decorator` dependency chain, and `ipython` seems to have
removed it as a dependency in their latest 9.16.0 version (see the
following PR: ipython/ipython#15310).
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants