Skip to content

Make caller locals visible to nested scopes in embedded shells#15289

Merged
Carreau merged 2 commits into
mainfrom
claude/ipython-issue-136-5z1sc9
Jul 7, 2026
Merged

Make caller locals visible to nested scopes in embedded shells#15289
Carreau merged 2 commits into
mainfrom
claude/ipython-issue-136-5z1sc9

Conversation

@Carreau

@Carreau Carreau commented Jul 7, 2026

Copy link
Copy Markdown
Member

Code typed into an embedded IPython shell is compiled as module-level
code, so lambdas, generator expressions, comprehensions and function
bodies defined interactively look their free variables up in globals()
and could not see the local variables of the frame the shell was
embedded in:

def f():
    x = 1
    IPython.embed()   # (lambda: x)() -> NameError

Fix this by giving the embedded shell a globals namespace (a dict
subclass installed via make_main_module_type) that resolves reads
through the caller's local namespace first, then its own snapshot of
the module globals, then the live module dict, mimicking the closure
lookup the code would have had if written in place. Passing a dict
subclass to exec disables the exact-dict LOAD_GLOBAL fast path, which
is what makes the getitem override effective.

STORE_GLOBAL bypasses setitem and mutates the C-level dict storage
directly, so 'global' assignments made by shell-defined functions land
in the snapshot; they are propagated back to the real module when the
shell exits.

Closes gh-136, see Quansight/deshaw#576

@Carreau Carreau force-pushed the claude/ipython-issue-136-5z1sc9 branch from b4bce68 to 0adf9d6 Compare July 7, 2026 08:15
Carreau and others added 2 commits July 7, 2026 08:48
Code typed into an embedded IPython shell is compiled as module-level
code, so lambdas, generator expressions, comprehensions and function
bodies defined interactively look their free variables up in globals()
and could not see the local variables of the frame the shell was
embedded in:

    def f():
        x = 1
        IPython.embed()   # (lambda: x)() -> NameError

Fix this by giving the embedded shell a globals namespace (a dict
subclass installed via make_main_module_type) that resolves reads
through the caller's local namespace first, then its own snapshot of
the module globals, then the live module dict, mimicking the closure
lookup the code would have had if written in place. Passing a dict
subclass to exec disables the exact-dict LOAD_GLOBAL fast path, which
is what makes the __getitem__ override effective.

STORE_GLOBAL bypasses __setitem__ and mutates the C-level dict storage
directly, so 'global' assignments made by shell-defined functions land
in the snapshot; they are propagated back to the real module when the
shell exits.

Closes gh-136
The embed integration tests exercise the new namespace machinery only
in spawned subprocesses, where coverage is not collected. Test the
lookup order, exec/nested-scope resolution, and module write-back of
_EmbedGlobals directly in-process.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PkPMSsnbYhtE281smQ2sfT
@Carreau Carreau force-pushed the claude/ipython-issue-136-5z1sc9 branch from 385bfd8 to 52366fe Compare July 7, 2026 08:48
@Carreau Carreau added this to the 9.16 milestone Jul 7, 2026
@Carreau Carreau merged commit 01b202e into main Jul 7, 2026
27 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

IPythonShellEmbed fails to recognize local variables

2 participants