Make caller locals visible to nested scopes in embedded shells#15289
Merged
Conversation
b4bce68 to
0adf9d6
Compare
Code typed into an embedded IPython shell is compiled as module-level
code, so lambdas, generator expressions, comprehensions and function
bodies defined interactively look their free variables up in globals()
and could not see the local variables of the frame the shell was
embedded in:
def f():
x = 1
IPython.embed() # (lambda: x)() -> NameError
Fix this by giving the embedded shell a globals namespace (a dict
subclass installed via make_main_module_type) that resolves reads
through the caller's local namespace first, then its own snapshot of
the module globals, then the live module dict, mimicking the closure
lookup the code would have had if written in place. Passing a dict
subclass to exec disables the exact-dict LOAD_GLOBAL fast path, which
is what makes the __getitem__ override effective.
STORE_GLOBAL bypasses __setitem__ and mutates the C-level dict storage
directly, so 'global' assignments made by shell-defined functions land
in the snapshot; they are propagated back to the real module when the
shell exits.
Closes gh-136
The embed integration tests exercise the new namespace machinery only in spawned subprocesses, where coverage is not collected. Test the lookup order, exec/nested-scope resolution, and module write-back of _EmbedGlobals directly in-process. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01PkPMSsnbYhtE281smQ2sfT
385bfd8 to
52366fe
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Code typed into an embedded IPython shell is compiled as module-level
code, so lambdas, generator expressions, comprehensions and function
bodies defined interactively look their free variables up in globals()
and could not see the local variables of the frame the shell was
embedded in:
Fix this by giving the embedded shell a globals namespace (a dict
subclass installed via make_main_module_type) that resolves reads
through the caller's local namespace first, then its own snapshot of
the module globals, then the live module dict, mimicking the closure
lookup the code would have had if written in place. Passing a dict
subclass to exec disables the exact-dict LOAD_GLOBAL fast path, which
is what makes the getitem override effective.
STORE_GLOBAL bypasses setitem and mutates the C-level dict storage
directly, so 'global' assignments made by shell-defined functions land
in the snapshot; they are propagated back to the real module when the
shell exits.
Closes gh-136, see Quansight/deshaw#576