Skip to content

Specialized ops #7301

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
youknowone merged 20 commits into from
Mar 3, 2026
Merged

Specialized ops #7301

Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
20 commits
Select commit Hold shift + click to select a range
687e99f
Add debug_assert to invoke_exact_args, lazy func_version reassignment
youknowone Mar 2, 2026
81d307b
working
youknowone Mar 1, 2026
0176223
Add COMPARE_OP, TO_BOOL, FOR_ITER, LOAD_GLOBAL specialization
youknowone Mar 1, 2026
9bb0c46
Add BINARY_SUBSCR, CONTAINS_OP, UNPACK_SEQUENCE, STORE_ATTR specializ…
youknowone Mar 1, 2026
1c07777
Add STORE_SUBSCR, BinaryOpAddUnicode, ToBoolAlwaysTrue, CallLen, Call…
youknowone Mar 1, 2026
240f3ac
Add BinaryOpSubscrStrInt, CallStr1, CallTuple1 specialization
youknowone Mar 1, 2026
cadb9be
Add BinaryOpInplaceAddUnicode specialization
youknowone Mar 1, 2026
fd098fe
Add LoadAttrModule, CallBuiltinO, CallPyGeneral, CallBoundMethodGener…
youknowone Mar 2, 2026
dd29113
Add LoadAttrNondescriptor*, CallMethodDescriptor* specialization
youknowone Mar 2, 2026
b238a27
Add CallBuiltinFast, CallNonPyGeneral specialization
youknowone Mar 2, 2026
d950035
Add SendGen specialization for generator/coroutine send
youknowone Mar 2, 2026
32376d5
Add LoadAttrSlot, StoreAttrSlot specialization for __slots__ access
youknowone Mar 2, 2026
a7c179c
Add LoadSuperAttrAttr, LoadSuperAttrMethod, CallBuiltinClass, CallBui…
youknowone Mar 2, 2026
e1289f1
Add LoadAttrProperty specialization for property descriptor access
youknowone Mar 2, 2026
2350bc1
Add LoadAttrClass specialization for class attribute access
youknowone Mar 2, 2026
ba9d528
Add BinaryOpSubscrListSlice specialization
youknowone Mar 2, 2026
3c88368
Add CallKwPy, CallKwBoundMethod, CallKwNonPy specialization
youknowone Mar 2, 2026
ab6bbb6
Clean up comments in specialization code
youknowone Mar 2, 2026
48fd5c7
fix check_signals
youknowone Mar 2, 2026
51accdb
fix import
youknowone Mar 3, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
working
  • Loading branch information
@youknowone
youknowone committed Mar 3, 2026
commit 81d307b106c7759130f4aa1b4a8ebcf2d7471f47
17 changes: 17 additions & 0 deletions crates/vm/src/builtins/list.rs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -637,6 +637,23 @@ impl PyListIterator {
}
}

impl PyListIterator {
/// Fast path for FOR_ITER specialization.
pub(crate) fn fast_next(&self) -> Option<PyObjectRef> {
self.internal
.lock()
.next(|list, pos| {
let vec = list.borrow_vec();
Ok(PyIterReturn::from_result(vec.get(pos).cloned().ok_or(None)))
})
.ok()
.and_then(|r| match r {
PyIterReturn::Return(v) => Some(v),
PyIterReturn::StopIteration(_) => None,
})
}
}

impl SelfIter for PyListIterator {}
impl IterNext for PyListIterator {
fn next(zelf: &Py<Self>, _vm: &VirtualMachine) -> PyResult<PyIterReturn> {
Expand Down
13 changes: 13 additions & 0 deletions crates/vm/src/builtins/range.rs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -660,6 +660,19 @@ impl PyRangeIterator {
}
}

impl PyRangeIterator {
/// Fast path for FOR_ITER specialization. Returns the next isize value
/// without allocating PyInt or PyIterReturn.
pub(crate) fn fast_next(&self) -> Option<isize> {
let index = self.index.fetch_add(1);
if index < self.length {
Some(self.start + (index as isize) * self.step)
} else {
Comment thread
coderabbitai[bot] marked this conversation as resolved.
None
}
}
}

impl SelfIter for PyRangeIterator {}
impl IterNext for PyRangeIterator {
fn next(zelf: &Py<Self>, vm: &VirtualMachine) -> PyResult<PyIterReturn> {
Expand Down
18 changes: 18 additions & 0 deletions crates/vm/src/builtins/tuple.rs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -572,6 +572,24 @@ impl PyTupleIterator {
}
}

impl PyTupleIterator {
/// Fast path for FOR_ITER specialization.
pub(crate) fn fast_next(&self) -> Option<PyObjectRef> {
self.internal
.lock()
.next(|tuple, pos| {
Ok(PyIterReturn::from_result(
tuple.get(pos).cloned().ok_or(None),
))
})
.ok()
.and_then(|r| match r {
PyIterReturn::Return(v) => Some(v),
PyIterReturn::StopIteration(_) => None,
})
}
}

impl SelfIter for PyTupleIterator {}
impl IterNext for PyTupleIterator {
fn next(zelf: &Py<Self>, _vm: &VirtualMachine) -> PyResult<PyIterReturn> {
Expand Down
17 changes: 16 additions & 1 deletion crates/vm/src/dict_inner.rs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,9 @@ use crate::{
object::{Traverse, TraverseFn},
};
use alloc::fmt;
use core::{mem::size_of, ops::ControlFlow};
use core::mem::size_of;
use core::ops::ControlFlow;
use core::sync::atomic::{AtomicU64, Ordering::Relaxed};
use num_traits::ToPrimitive;

// HashIndex is intended to be same size with hash::PyHash
Expand All @@ -34,6 +36,7 @@ type EntryIndex = usize;

pub struct Dict<T = PyObjectRef> {
inner: PyRwLock<DictInner<T>>,
version: AtomicU64,
}

unsafe impl<T: Traverse> Traverse for Dict<T> {
Expand Down Expand Up @@ -98,6 +101,7 @@ impl<T: Clone> Clone for Dict<T> {
fn clone(&self) -> Self {
Self {
inner: PyRwLock::new(self.inner.read().clone()),
version: AtomicU64::new(0),
}
}
}
Expand All @@ -111,6 +115,7 @@ impl<T> Default for Dict<T> {
indices: vec![IndexEntry::FREE; 8],
entries: Vec::new(),
}),
version: AtomicU64::new(0),
}
}
}
Expand Down Expand Up @@ -254,6 +259,16 @@ impl<T> DictInner<T> {
type PopInnerResult<T> = ControlFlow<Option<DictEntry<T>>>;

impl<T: Clone> Dict<T> {
/// Monotonically increasing version counter for mutation tracking.
pub fn version(&self) -> u64 {
self.version.load(Relaxed)
}

/// Bump the version counter after any mutation.
fn bump_version(&self) {
self.version.fetch_add(1, Relaxed);
}
Comment on lines +263 to +270
Copy link
Copy Markdown
Contributor

@coderabbitai coderabbitai Bot Mar 2, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟠 Major

🧩 Analysis chain

🏁 Script executed:

# 1) Show where version bumps happen relative to lock scopes.
rg -n -C4 'bump_version\(' crates/vm/src/dict_inner.rs

Repository: RustPython/RustPython

Length of output: 2663

🏁 Script executed:

# 2) Show consumers of dict version values (to validate whether reads are lock-free).
rg -n -C4 '\.version\(\)' crates/vm/src

Repository: RustPython/RustPython

Length of output: 3008

🏁 Script executed:

# 3) Get the full context around lines 263-270 and surrounding code
head -n 300 crates/vm/src/dict_inner.rs | tail -n +250

Repository: RustPython/RustPython

Length of output: 1941

Use Acquire/Release ordering for version field and perform version bumps under the write lock.

The version() and bump_version() methods use Relaxed ordering. Currently, bump_version() is called after the write lock is released at lines 319, 383, 441, 465, 499, 536, 738, and 756. Meanwhile, version() is read lock-free in frame.rs (lines 4644, 4680, 7076) to validate LOAD_GLOBAL specialization caches. With Relaxed ordering, a reader may see stale version values after mutations complete, causing cache validation to incorrectly accept invalidated caches.

🔧 Suggested direction 
-use core::sync::atomic::{AtomicU64, Ordering::Relaxed};
+use core::sync::atomic::{AtomicU64, Ordering::{Acquire, Release}};
...
 pub fn version(&self) -> u64 {
-    self.version.load(Relaxed)
+    self.version.load(Acquire)
 }
...
 fn bump_version(&self) {
-    self.version.fetch_add(1, Relaxed);
+    self.version.fetch_add(1, Release);
 }

Also move each bump_version() call into the same write-lock critical section as the corresponding mutation to ensure version updates are ordered with respect to dict mutations.

🤖 Prompt for AI Agents 
Verify each finding against the current code and only fix it if needed.

In `@crates/vm/src/dict_inner.rs` around lines 263 - 270, The version() accessor
and bump_version() updater must use proper Acquire/Release ordering and
bump_version must be called while holding the dict write lock: change version()
to load with Acquire and bump_version() to fetch_add with Release (function
names: version and bump_version in dict_inner.rs), and move every call to
bump_version so it executes inside the same write-lock critical section where
the dictionary mutation occurs (the callers in this repo include the mutation
sites referenced in frame.rs that read the version lock-free for LOAD_GLOBAL
caching); ensure the write lock remains held across the mutation and the
bump_version call so the Release store synchronizes with readers that use
Acquire.


fn read(&self) -> PyRwLockReadGuard<'_, DictInner<T>> {
self.inner.read()
}
Expand Down
Loading