-
Notifications
You must be signed in to change notification settings - Fork 1.5k
Fix some pyo3 testcase against the c-api #8349
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
Changes from all commits
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -1,14 +1,15 @@ | ||
| use crate::util::CStrExt; | ||
| use crate::{PyObject, pystate::with_vm}; | ||
| use core::ffi::c_char; | ||
| use rustpython_vm::builtins::{PyCode, PyDict, PyModule, PyStr}; | ||
| use rustpython_vm::builtins::{PyCode, PyDict, PyModule, PyStr, PyTuple}; | ||
| use rustpython_vm::import::import_code_obj; | ||
|
|
||
| #[unsafe(no_mangle)] | ||
| pub unsafe extern "C" fn PyImport_Import(name: *mut PyObject) -> *mut PyObject { | ||
| with_vm(|vm| { | ||
| let name = unsafe { (&*name).try_downcast_ref::<PyStr>(vm)? }; | ||
| vm.import(name, 0) | ||
| let from_list = PyTuple::new_ref_typed(vec![vm.ctx.new_str("*")], &vm.ctx); | ||
| vm.import_from(name, &from_list, 0) | ||
|
Comment on lines
+11
to
+12
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. 🎯 Functional Correctness | 🟠 Major | ⚡ Quick win 🧩 Analysis chain🏁 Script executed: #!/bin/bash
set -euo pipefail
printf 'Repository files matching import.rs:\n'
fd -a 'import\.rs$' . | sed 's#^\./##'
printf '\nFile outline:\n'
ast-grep outline crates/capi/src/import.rs --view compact || true
printf '\nRelevant source:\n'
cat -n crates/capi/src/import.rs
printf '\nSearch for import_from and PyImport_Import references:\n'
rg -n "import_from|PyImport_Import|PyImport_ImportModule|PyImport_Import\\(|new_ref_typed|__all__|__doc__" crates/capi src . --glob '*.rs' | head -200Repository: RustPython/RustPython Length of output: 12300 🌐 Web query:
💡 Result: PyImport_ImportModuleNoBlock was a C API function in CPython that historically provided a way to import modules without blocking on the global import lock [1][2]. Background and Functionality: In older versions of CPython, a global import lock was used to protect the module import process, which could lead to deadlocks in multi-threaded applications [2]. PyImport_ImportModuleNoBlock was introduced as a "clumsy workaround" to allow C extensions to import modules without waiting for this lock, raising an ImportError instead if the lock was held by another thread [3][2]. Changes in Python 3.3+: With the introduction of per-module import locks in Python 3.3, the original purpose of this function became obsolete [1][2]. Consequently, PyImport_ImportModuleNoBlock was refactored to simply act as an alias for PyImport_ImportModule [4][3][5]. Deprecation and Removal: Due to its redundancy, the function was deprecated in Python 3.13 and is scheduled for removal in Python 3.15 [1][5]. Developers are advised to use PyImport_ImportModule instead [1]. Regarding the specific terms in your query ("all", "dummy", "doc"): These are standard Python identifiers and attributes [6]. While they appear throughout the CPython codebase (often used for defining module exports or docstrings), they are not specifically or uniquely associated with the internal logic of PyImport_ImportModuleNoBlock [3][7]. The function's implementation in Python/import.c is straightforward, simply wrapping the standard import call [4][3]. Citations:
🏁 Script executed: #!/bin/bash
set -euo pipefail
printf 'Import implementation around import_from/import_code:\n'
sed -n '320,430p' crates/vm/src/import.rs | cat -n -v
printf '\nFrame import_from and __all__ handling:\n'
sed -n '6920,6985p' crates/vm/src/frame.rs | cat -n -v
printf '\nC API implementation around PyImport_Import calls:\n'
rg -n "PyImport_Import|Module_Lookup|Import_Import|fromlist|__all__" crates/vm/src crates/capi/src crates/stdlib/src --glob '*.rs' | head -200
printf '\nPython runtime behavior for wildcard/non-wildcard imports into a pkg with __all__:\n'
cat > /tmp/rustpython_wildcard_probe.py <<'PY'
import sys
import pathlib
sys.path.insert(0, str(pathlib.Path(__file__).parent))
import importlib.util
spec = importlib.util.find_spec('collections.abc')
target = pathlib.Path(spec.origin).parent
(target / '_wildcardpkg.py').write_text("import sys\n")
(target / '_nonwild.py').write_text("import sys\n")
(target / '_wildcardpkg' / 'mod1.py').write_text("import sys\n")
(target / '_nonwild' / 'mod1.py').write_text("import sys\n")
imp = __import__('_wildcardpkg', fromlist=['*'])
imp = __import__('_nonwild', fromlist=['__doc__'])
PY
python3 /tmp/rustpython_wildcard_probe.py || trueRepository: RustPython/RustPython Length of output: 10480 🌐 Web query:
💡 Result: In RustPython, Citations:
🏁 Script executed: #!/bin/bash
set -euo pipefail
# Use a tmpdir outside site-packages so the probe can write files safely.
tmpdir="$(mktemp -d)"
cd "$tmpdir"
cat > probe.py <<'PY'
import sys
sys.path.insert(0, ".")
pathlib(__import__("pathlib").Path("pkg_star")).mkdir()
pathlib("pkg_star/mod1.py").write_text("")
pathlib("pkg_star").write_bytes(b"from . import mod1")
pkg = __import__("pkg_star", fromlist=["*"])
print("pkg_star.star_imported", "mod1" in sys.modules)
sys.modules.pop("pkg_star.mod1", None)
pathlib("pkg_nostar").mkdir()
pathlib("pkg_nostar/mod1.py").write_text("")
pathlib("pkg_nostar").write_bytes(b"from . import mod1")
pkg = __import__("pkg_nostar", fromlist=["__doc__"])
print("pkg_nostar.star_imported", "pkg_nostar.mod1" in sys.modules)
PY
python3 probe.pyRepository: RustPython/RustPython Length of output: 354 🏁 Script executed: #!/bin/bash
set -euo pipefail
tmpdir="$(mktemp -d)"
cd "$tmpdir"
cat > probe.py <<'PY'
import sys, pathlib
pathlib.Path("pkg_star").mkdir(parents=True)
pathlib.Path("pkg_star/mod1.py").write_text("")
pathlib.Path("pkg_star/__init__.py").write_bytes(b"from . import mod1")
pkg = __import__("pkg_star", fromlist=["*"])
print("pkg_star.star_imported", "pkg_star.mod1" in sys.modules)
del sys.modules["pkg_star.mod1"]
pathlib.Path("pkg_nostar").mkdir(parents=True)
pathlib.Path("pkg_nostar/mod1.py").write_text("")
pathlib.Path("pkg_nostar/__init__.py").write_bytes(b"from . import mod1")
pkg = __import__("pkg_nostar", fromlist=["__doc__"])
print("pkg_nostar.star_imported", "pkg_nostar.mod1" in sys.modules)
PY
python3 probe.pyRepository: RustPython/RustPython Length of output: 389 🏁 Script executed: #!/bin/bash
set -euo pipefail
cat > /tmp/python_import_check.py <<'PY'
import importlib.util
spec = importlib.util.find_spec("collections.abc")
print(spec.origin)
print(spec.submodule_search_locations)
PY
python3 /tmp/python_import_check.py
cat > /tmp/collections_abc_probe.py <<'PY'
import sys
pkg = __import__("collections.abc", fromlist=["__doc__"])
print("seq_loaded", "collections.abc.Sequence" in sys.modules)
print("abc_loaded", pkg is sys.modules["collections.abc"])
print("has Sequence", hasattr(pkg, "Sequence"))
PY
python3 /tmp/collections_abc_probe.pyRepository: RustPython/RustPython Length of output: 255 🌐 Web query:
💡 Result: In RustPython, the VirtualMachine::import_from method is a core function used to implement the Python import mechanism, specifically handling statements like from module_name import item1, item2 [1]. The method signature is defined as follows: pub fn import_from<'a>( &self, module_name: impl AsPyStr<'a>, from_list: &Py, level: usize,) -> PyResult Parameters: 1. module_name: The name of the module to import from [1]. 2. from_list: A tuple of strings (PyTuple) representing the names (or attributes) to be imported from the module [1]. This corresponds to the list of names specified after the import keyword in a Python from... import... statement [2]. 3. level: An integer indicating whether to use absolute or relative imports (0 for absolute, >0 for relative) [1][3]. Functionality: This method is roughly equivalent to the Python statement from module_name import item1, item2 [1]. Internally, it invokes the import system to locate the module and then extracts the requested symbols specified in the from_list [1][4]. The VirtualMachine struct is defined in crates/vm/src/vm/mod.rs [5], and the import logic frequently interacts with the importlib machinery initialized during the VM setup [4]. While VirtualMachine::import handles standard imports (import module_name), import_from provides the necessary mechanism for accessing specific objects from within a module [1]. Citations:
Use a non-wildcard sentinel for
🤖 Prompt for AI Agents |
||
| }) | ||
| } | ||
|
|
||
|
|
@@ -74,4 +75,12 @@ mod tests { | |
| let _module = py.import("types").unwrap(); | ||
| }) | ||
| } | ||
|
|
||
| #[test] | ||
| fn import_sub_module() { | ||
| Python::attach(|py| { | ||
| let module = py.import("collections.abc").unwrap(); | ||
| module.getattr("Sequence").unwrap(); | ||
| }) | ||
| } | ||
| } | ||
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
is this correct change? I didn't check code yet, but looking not matching to documented behavior
https://docs.python.org/3/c-api/import.html#c.PyImport_Import
Uh oh!
There was an error while loading. Please reload this page.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Before this change the testcase failed. It returned the top level module instead of
abc. Current implementation passes all tests so it may be correct.