Add support for creating functions with the c-api#7984
Conversation
|
No actionable comments were generated in the recent review. 🎉 ℹ️ Recent review info⚙️ Run configurationConfiguration used: Path: .coderabbit.yml Review profile: CHILL Plan: Pro Run ID: 📒 Files selected for processing (1)
📝 WalkthroughWalkthroughAdds a new C-API methodobject module with PyMethodDef and calling-convention adapters, exports PyCMethod_New/PyCFunction_New/PyCFunction_NewEx, and updates the VM builder to accept/thread an optional self (zelf) into constructed native function objects. ChangesC-API Method Object Implementation
VM Method Construction Integration
Sequence DiagramsequenceDiagram
participant C_Ext as C extension
participant capi as capi::PyCFunction_New*
participant Orchestrator as build_method_def
participant VM as VirtualMachine
participant Native as PyNativeFunction
C_Ext->>capi: call PyCFunction_New(ml, self)
capi->>Orchestrator: build_method_def(ml, has_self)
Orchestrator->>VM: .build_function(vm, optional_zelf)
VM->>Native: create PyNativeFunction(zelf assigned)
Native-->>C_Ext: callable PyObject pointer
Estimated code review effort🎯 4 (Complex) | ⏱️ ~45 minutes Suggested reviewers
Poem
🚥 Pre-merge checks | ✅ 4 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (4 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. ✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
There was a problem hiding this comment.
Actionable comments posted: 2
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@crates/capi/src/methodobject.rs`:
- Around line 91-114: The VARARGS/VARARGS|KEYWORDS/FASTCALL/FASTCALL|KEYWORDS
branches always call take_self_arg which strips the first positional arg even
when has_self is false; fix by capturing the local has_self (the same way NOARGS
and O do) and either (A) pass has_self into take_self_arg so the adapters
(call_function, call_fast_function, call_function_with_keywords,
call_fast_function_with_keywords) can honor it, or (B) branch each case on
has_self and create different callable signatures (one that calls take_self_arg
when has_self==true and one that forwards args untouched when false) before
calling vm.ctx.new_method_def with the appropriate callable.
- Around line 78-90: The NOARGS branch (PyMethodFlags::NOARGS) incorrectly
creates and passes an empty tuple for args when has_self is true; change the
closure created in that branch (the callable that takes zelf: PyObjectRef) to
call call_function with None for the args parameter (i.e., pass None so it
becomes NULL) instead of Some(vec![zelf]) wrapped as an empty tuple, or
alternatively implement a small helper (e.g., call_function_noargs) that always
forwards NULL for the args parameter; update the callable and keep using
vm.ctx.new_method_def(name, callable, flags, doc) so behavior matches the
has_self=false path.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yml
Review profile: CHILL
Plan: Pro
Run ID: 9893aed3-e881-4cbe-841a-48f858dae982
⛔ Files ignored due to path filters (1)
Cargo.lockis excluded by!**/*.lock
📒 Files selected for processing (5)
crates/capi/Cargo.tomlcrates/capi/src/lib.rscrates/capi/src/methodobject.rscrates/vm/src/function/method.rscrates/vm/src/vm/vm_new.rs
| PyMethodFlags::VARARGS => { | ||
| let callable = move |args: PosArgs, vm: &VirtualMachine| unsafe { | ||
| call_function(vm, method, flags, Some(args)) | ||
| }; | ||
| Ok(vm.ctx.new_method_def(name, callable, flags, doc)) | ||
| }, | ||
| PyMethodFlags::VARARGS | PyMethodFlags::KEYWORDS => { | ||
| let callable = move | args: FuncArgs, vm: &VirtualMachine| unsafe { | ||
| call_function_with_keywords(vm, method, flags, args) | ||
| }; | ||
| Ok(vm.ctx.new_method_def(name, callable, flags, doc)) | ||
| }, | ||
| PyMethodFlags::FASTCALL | PyMethodFlags::KEYWORDS => { | ||
| let callable = move |args: FuncArgs, vm: &VirtualMachine| unsafe { | ||
| call_fast_function_with_keywords(vm, method, flags, args) | ||
| }; | ||
| Ok(vm.ctx.new_method_def(name, callable, flags, doc)) | ||
| }, | ||
| PyMethodFlags::FASTCALL => { | ||
| let callable = move |args: PosArgs, vm: &VirtualMachine| unsafe { | ||
| call_fast_function(vm, method, flags, args) | ||
| }; | ||
| Ok(vm.ctx.new_method_def(name, callable, flags, doc)) | ||
| }, |
There was a problem hiding this comment.
Argument corruption when has_self=false for VARARGS/FASTCALL conventions.
For VARARGS, VARARGS|KEYWORDS, FASTCALL, and FASTCALL|KEYWORDS, the wrapper always calls take_self_arg, which removes the first positional argument unless the STATIC flag is set. However, when has_self=false:
- The VM does not prepend any self (since
zelf=None) - The callable receives the user's original arguments
take_self_argincorrectly removes the first user argument
Compare to NOARGS (lines 78-90) and O (lines 115-130) which correctly branch on has_self to create different callable signatures.
The fix is to capture has_self and pass it to take_self_arg, or branch on has_self for each calling convention like NOARGS and O do.
Proposed fix: Update take_self_arg to consider has_self
One approach - pass has_self to the call adapters:
unsafe fn call_function<A: Into<FuncArgs>>(
vm: &VirtualMachine,
method: PyMethodPointer,
flags: PyMethodFlags,
args: Option<A>,
+ has_self: bool,
) -> PyResult {
let f = unsafe { method.PyCFunction };
let (slf, arg_tuple) = if let Some(mut args) = args.map(Into::into) {
- let slf = take_self_arg(&mut args, flags);
+ let slf = take_self_arg(&mut args, flags, has_self);
let arg_tuple = vm.ctx.new_tuple(args.args);
(slf, Some(arg_tuple))
} else {
(None, None)
};
// ...
}
-fn take_self_arg(args: &mut FuncArgs, flags: PyMethodFlags) -> Option<PyObjectRef> {
- if flags.contains(PyMethodFlags::STATIC) {
+fn take_self_arg(args: &mut FuncArgs, flags: PyMethodFlags, has_self: bool) -> Option<PyObjectRef> {
+ if !has_self || flags.contains(PyMethodFlags::STATIC) {
None
} else {
args.take_positional()
}
}Then update the VARARGS case to capture has_self:
PyMethodFlags::VARARGS => {
- let callable = move |args: PosArgs, vm: &VirtualMachine| unsafe {
- call_function(vm, method, flags, Some(args))
+ let callable = move |args: PosArgs, vm: &VirtualMachine| unsafe {
+ call_function(vm, method, flags, Some(args), has_self)
};
Ok(vm.ctx.new_method_def(name, callable, flags, doc))
},Apply similar changes to VARARGS|KEYWORDS, FASTCALL, and FASTCALL|KEYWORDS branches.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@crates/capi/src/methodobject.rs` around lines 91 - 114, The
VARARGS/VARARGS|KEYWORDS/FASTCALL/FASTCALL|KEYWORDS branches always call
take_self_arg which strips the first positional arg even when has_self is false;
fix by capturing the local has_self (the same way NOARGS and O do) and either
(A) pass has_self into take_self_arg so the adapters (call_function,
call_fast_function, call_function_with_keywords,
call_fast_function_with_keywords) can honor it, or (B) branch each case on
has_self and create different callable signatures (one that calls take_self_arg
when has_self==true and one that forwards args untouched when false) before
calling vm.ctx.new_method_def with the appropriate callable.
b16204f to
cc4b1c7
Compare
JamesClarke7283
left a comment
There was a problem hiding this comment.
Nice groundwork on the C API surface. Build is clean and clippy passes the workspace lint set, which already includes pedantic and a fair bit of nursery. Most of what's left is semantic.
1. VARARGS / FASTCALL paths strip a real user argument as self when the function has no bound self.
PyMethodFlags::VARARGS => {
let callable = move |args: PosArgs, vm: &VirtualMachine| unsafe {
call_function(vm, method, flags, Some(args))
};
...
},fn take_self_arg(args: &mut FuncArgs, flags: PyMethodFlags) -> Option<PyObjectRef> {
if flags.contains(PyMethodFlags::STATIC) { None } else { args.take_positional() }
}Callable::call for PyNativeFunction only prepends zelf when zelf.is_some() && !STATIC. After PyCFunction_New(ml, NULL) with METH_VARARGS, calling fn(a, b) makes take_self_arg consume a as self and the C function sees args = (b,). Silent data corruption with no error. Same shape for the three other branches at lines 95 to 118. NOARGS and O already split on has_self; the VARARGS / FASTCALL family needs the same treatment, or pass has_self into take_self_arg.
2. module is silently dropped, so __module__ is always None for anything built through this API.
pub unsafe extern "C" fn PyCMethod_New(
ml: *mut PyMethodDef,
slf: *mut PyObject,
_module: *mut PyObject,
_cls: *mut PyTypeObject,
) -> *mut PyObject {CPython does op->m_module = Py_XNewRef(module) and exposes it via __module__. Here it is bound to _module and never read. HeapMethodDef::build_function also lacks a module parameter, so even threading it through would not reach PyNativeFunction.module. Breaks inspect.getmodule, pickling and traceback module attribution.
3. assert! and .expect() across an extern \"C\" boundary.
with_vm(|vm| -> PyResult {
assert!(
_cls.is_null(),
\"PyCMethod_New does not support METH_METHOD on abi3\"
);vm.take_raised_exception()
.expect(\"Native function returned NULL, but there was no exception set\")with_vm does not catch panics, and neither function is extern \"C-unwind\", so a panic here aborts the process. Any caller that passes a non-null cls (the PEP 573 path with METH_METHOD | METH_FASTCALL | METH_KEYWORDS) takes the interpreter down. Likewise for any buggy extension that returns NULL without PyErr_SetString. Both should be vm.new_system_error(...) plus a NULL return, matching _Py_CheckFunctionResult in CPython.
4. PyMethodFlags::from_bits rejects valid CPython flag combinations.
let flags = PyMethodFlags::from_bits(ml.ml_flags as u32)
.ok_or_else(|| vm.new_system_error(\"PyMethodDef contains unknown flags\"))?;METH_COEXIST (0x40) and METH_STACKLESS (0x100) are commented out of PyMethodFlags in crates/vm/src/function/method.rs. CPython masks unknown bits and treats them as ignored, so a textbook pattern like {\"__contains__\", impl, METH_O | METH_COEXIST, NULL} (used by several stdlib C modules) fails registration outright. Either switch to from_bits_truncate or declare the bits.
5. CStr::from_ptr(ml.ml_name) with no null check.
let name = unsafe { CStr::from_ptr(ml.ml_name) }
.to_str()
.map_err(|_| vm.new_system_error(\"Method name was not valid UTF-8\"))?;Null ml_name is UB. ml_doc is already guarded with NonNull::new two lines down, so this is also an internal inconsistency. Same guard plus a SystemError.
Minor: the #[cfg(false)] mod tests block at lines 309 to 369 is dead code that references pyo3 types not declared as a dev dependency. Either wire it into CI or drop it.
Summary by CodeRabbit
New Features
Bug Fixes
Chores