ffi: No interior NULs (part 1)#8245
Conversation
|
Important Review skippedDraft detected. Please check the settings in the CodeRabbit UI or the ⚙️ Run configurationConfiguration used: Path: .coderabbit.yml Review profile: CHILL Plan: Pro Run ID: You can disable this status message by setting the Use the checkbox below for a quick retry:
📝 WalkthroughWalkthroughThis PR makes FFI and ctypes string/byte conversions fallible when interior NULs or wide-string encoding errors occur. The new wtf8 iterator and error type are threaded through host_env and vm ctypes helpers, and Windows wide-string conversion paths now return ChangesFallible NUL-terminated string conversions
Estimated code review effort: 3 (Moderate) | ~25 minutes Sequence Diagram(s)sequenceDiagram
participant Python as Python ctypes call
participant ConvParam as vm::_ctypes::function::conv_param
participant Base as vm::_ctypes::base helpers
participant HostEnv as host_env ctypes helpers
participant Wtf8 as wtf8::Wtf8
Python->>ConvParam: pass PyStr / PyBytes
ConvParam->>Base: request null-terminated buffer
Base->>HostEnv: call null_terminated_bytes / wide-string helpers
HostEnv->>Wtf8: encode_wide_ffi() for wide input
Wtf8-->>HostEnv: Ok(units) or Err(InteriorNulError)
HostEnv-->>Base: Result buffer or NulError
Base-->>ConvParam: propagate error via ?
ConvParam-->>Python: return error or continue call setup
Possibly related PRs
🚥 Pre-merge checks | ✅ 5✅ Passed checks (5 passed)
✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
There was a problem hiding this comment.
Actionable comments posted: 5
Caution
Some comments are outside the diff and can’t be posted inline due to platform limitations.
⚠️ Outside diff range comments (2)
crates/host_env/src/fileutils.rs (1)
80-97: 🩺 Stability & Availability | 🔴 Critical | ⚡ Quick winAdd the missing
Ok(())and propagate the result
crates/host_env/src/fileutils.rs:76-97currently ends with(), even though the signature returnsResult<(), io::Error>, so this won’t compile. Thecrates/host_env/src/nt.rs:831,836,857call sites also discard the returnedResult, which dropspath.to_wide()?failures; thread it through with?.🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@crates/host_env/src/fileutils.rs` around lines 80 - 97, The helper in fileutils.rs returns Result<(), io::Error> but currently falls off the end with unit, so add an explicit Ok(()) after the permission update logic. Also make the nt.rs callers that use this helper propagate its Result instead of ignoring it, so failures from path.to_wide()? are not dropped; update the call sites around the file metadata handling to use ? and thread the error upward.crates/vm/src/stdlib/_ctypes/base.rs (1)
1596-1596: 🎯 Functional Correctness | 🔴 Critical | ⚡ Quick winHandle the fallible
str_to_wchar_bytesresult here. This call still destructures aResult;?only works onceInteriorNulErroris mapped intoPyBaseExceptionRef, so either add that conversion or convert the error explicitly at this call site.🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@crates/vm/src/stdlib/_ctypes/base.rs` at line 1596, The call in the wchar conversion path is still destructuring a fallible `str_to_wchar_bytes` result directly, so update the logic around `str_to_wchar_bytes` to properly handle its `Result` before destructuring. In the `_ctypes::base` code path that builds the wide string buffer, either add a conversion from `InteriorNulError` into `PyBaseExceptionRef` so `?` can be used cleanly, or explicitly map the error at the call site before extracting `holder` and `ptr`.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@crates/host_env/src/ctypes.rs`:
- Around line 542-546: The vec_into_bytes helper currently reinterprets the
original allocation with Vec::from_raw_parts, which can deallocate with the
wrong layout for wide-string buffers. Update vec_into_bytes in ctypes.rs to copy
the bytes out of the source Vec<T> instead of casting the allocation; keep the
existing size_of::<T>() guard, but replace the raw-parts reconstruction with a
safe byte copy approach so the returned Vec<u8> owns a correctly laid-out
allocation.
In `@crates/host_env/src/windows.rs`:
- Around line 413-417: The `to_wide` method in `windows.rs` is using `io::Error`
as a direct mapper, which won’t compile in this context. Update the
`WideCString::from_os_str(self)` error handling to use `io::Error::other`,
matching the pattern used by the other conversion methods, while keeping the
rest of `to_wide` unchanged.
- Around line 428-435: The Wtf8 implementation of ToWideString is incomplete and
uses the wrong encoder for the checked Result-based API. In the ToWideString
impl for Wtf8, add the missing to_wide_cstring method alongside to_wide and
to_wide_with_nul, and update all three methods to use encode_wide_ffi() so they
return Result<Vec<u16>, io::Error> correctly and reject interior NULs. Keep the
fix localized to the Wtf8 trait implementation in windows.rs.
In `@crates/vm/src/stdlib/_ctypes/function.rs`:
- Line 158: In the ctypes conversion helpers, the `?` operator is being used on
errors that do not automatically convert into `PyBaseExceptionRef`, so fix the
error handling in the functions that call
`rustpython_host_env::ctypes::utf16z_bytes` and `null_terminated_bytes` by
explicitly mapping those `InteriorNulError` and `NulError` values into the
Python exception type before propagating them. Apply the same change in the
corresponding logic in `function.rs` and `base.rs`, keeping the conversion
localized near the existing `utf16z_bytes` / `null_terminated_bytes` calls.
In `@crates/wtf8/src/lib.rs`:
- Around line 915-916: The doc comment on encode_wide_ffi names the wrong
encoding: it currently says the function converts to potentially ill-formed
UTF-8, but this helper returns potentially ill-formed UTF-16 wide code units
like encode_wide. Update the comment text for encode_wide_ffi to describe UTF-16
instead of UTF-8, keeping the note about checking for interior NULs.
---
Outside diff comments:
In `@crates/host_env/src/fileutils.rs`:
- Around line 80-97: The helper in fileutils.rs returns Result<(), io::Error>
but currently falls off the end with unit, so add an explicit Ok(()) after the
permission update logic. Also make the nt.rs callers that use this helper
propagate its Result instead of ignoring it, so failures from path.to_wide()?
are not dropped; update the call sites around the file metadata handling to use
? and thread the error upward.
In `@crates/vm/src/stdlib/_ctypes/base.rs`:
- Line 1596: The call in the wchar conversion path is still destructuring a
fallible `str_to_wchar_bytes` result directly, so update the logic around
`str_to_wchar_bytes` to properly handle its `Result` before destructuring. In
the `_ctypes::base` code path that builds the wide string buffer, either add a
conversion from `InteriorNulError` into `PyBaseExceptionRef` so `?` can be used
cleanly, or explicitly map the error at the call site before extracting `holder`
and `ptr`.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yml
Review profile: CHILL
Plan: Pro
Run ID: de55e068-6ea7-4cd4-8e9e-98e767127b8a
📒 Files selected for processing (6)
crates/host_env/src/ctypes.rscrates/host_env/src/fileutils.rscrates/host_env/src/windows.rscrates/vm/src/stdlib/_ctypes/base.rscrates/vm/src/stdlib/_ctypes/function.rscrates/wtf8/src/lib.rs
ec9ad96 to
ee369e2
Compare
There was a problem hiding this comment.
Actionable comments posted: 1
🧹 Nitpick comments (1)
crates/host_env/src/ctypes.rs (1)
1095-1097: 📐 Maintainability & Code Quality | 🔵 Trivial | ⚡ Quick winAdd tests for the new fallible
null_terminated_bytes.This is a security-critical function (interior NUL detection for FFI), but the test module has no coverage for it. Consider adding tests for: valid input without NULs, input containing an interior NUL (should return
Err(NulError)), and empty input.🧪 Suggested tests
#[test] fn null_terminated_bytes_valid() { assert_eq!( null_terminated_bytes(b"hello").unwrap(), b"hello\0" ); } #[test] fn null_terminated_bytes_interior_nul_rejected() { assert!(null_terminated_bytes(b"hel\0lo").is_err()); } #[test] fn null_terminated_bytes_empty() { assert_eq!(null_terminated_bytes(b"").unwrap(), b"\0"); }🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@crates/host_env/src/ctypes.rs` around lines 1095 - 1097, Add test coverage for the fallible null_terminated_bytes helper in ctypes.rs, since it now performs FFI-safe interior NUL validation via CString::new. Extend the existing test module with cases for valid non-NUL input, input containing an interior NUL that must return Err(NulError), and empty input; use the null_terminated_bytes function name directly so the tests clearly target the new behavior.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@crates/host_env/src/ctypes.rs`:
- Around line 2-3: The `CString` import in `ctypes.rs` is incorrectly gated with
`#[cfg(unix)]` even though `null_terminated_bytes` uses `CString`
unconditionally and is called from `ensure_z_null_terminated` in `base.rs` and
`conv_param` in `function.rs` on all targets. Remove the unix-only cfg from the
`CString` import so `null_terminated_bytes` can compile on non-unix platforms as
well, and make sure the identifier references in `ctypes.rs` remain valid
without any platform-specific gating.
---
Nitpick comments:
In `@crates/host_env/src/ctypes.rs`:
- Around line 1095-1097: Add test coverage for the fallible
null_terminated_bytes helper in ctypes.rs, since it now performs FFI-safe
interior NUL validation via CString::new. Extend the existing test module with
cases for valid non-NUL input, input containing an interior NUL that must return
Err(NulError), and empty input; use the null_terminated_bytes function name
directly so the tests clearly target the new behavior.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yml
Review profile: CHILL
Plan: Pro
Run ID: 2c9c6278-53c9-4285-ac5d-4a5a4b8a716e
📒 Files selected for processing (6)
crates/host_env/src/ctypes.rscrates/host_env/src/fileutils.rscrates/host_env/src/windows.rscrates/vm/src/stdlib/_ctypes/base.rscrates/vm/src/stdlib/_ctypes/function.rscrates/wtf8/src/lib.rs
💤 Files with no reviewable changes (1)
- crates/wtf8/src/lib.rs
🚧 Files skipped from review as they are similar to previous changes (4)
- crates/host_env/src/fileutils.rs
- crates/vm/src/stdlib/_ctypes/function.rs
- crates/vm/src/stdlib/_ctypes/base.rs
- crates/host_env/src/windows.rs
ee369e2 to
20fd74b
Compare
20fd74b to
d41d6a4
Compare
Interior NULs is a security hazard for C-style strings. A NUL byte truncates a string which can lead the caller and callee to see two different strings. It can cause path traversal attacks where a path in Python looks complete but it is interpreted differently through FFI. RustPython needs to handle this for its C-API as well as raw libc or Windows calls. Both Rust's standard library as well as Rustix handle interior NULs for us. Finally, this PR is non-exhaustive. I will have to rely heavily on CodeRabbit to help lint it to ensure that interior NUL checks are only introduced for FFI and not outside of it. **AI disclosure:** I relied on AI to ensure I'm solving this problem correctly. Mainly, I used it to check if the FFI functions I'm modifying need to handle interior NULs. **Sources:** * https://owasp.org/www-community/attacks/Embedding_Null_Code * python/cpython#11656 Assisted-by: Codex
d41d6a4 to
b3816bb
Compare
(This is WIP; the Windows code doesn't even compile yet 😆 Opening as placeholder + CodeRabbit lints )
Interior NULs is a security hazard for C-style strings. A NUL byte truncates a string which can lead the caller and callee to see two different strings. It can cause path traversal attacks where a path in Python looks complete but it is interpreted differently through FFI.
RustPython needs to handle this for its C-API as well as raw libc or Windows calls. Both Rust's standard library as well as Rustix handle interior NULs for us.
Finally, this PR is non-exhaustive. I will have to rely heavily on CodeRabbit to help lint it to ensure that interior NUL checks are only introduced for FFI and not outside of it.
AI disclosure: I relied on AI to ensure I'm solving this problem correctly. Mainly, I used it to check if the FFI functions I'm modifying need to handle interior NULs.
Sources:
Assisted-by: Codex
Summary
Summary by CodeRabbit
New Features
Bug Fixes