Skip to content

Bytecode parity #7504

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
youknowone merged 15 commits into from
Mar 25, 2026
Merged

Bytecode parity #7504

Changes from 1 commit
Commits
Show all changes
15 commits
Select commit Hold shift + click to select a range
a6aab76
Match CPython LOAD_SPECIAL stack semantics for with/async-with
youknowone Mar 23, 2026
d015f02
Normalize LOAD_FAST_CHECK and JUMP_BACKWARD_NO_INTERRUPT
youknowone Mar 23, 2026
37d1e03
Add EXTENDED_ARG to SKIP_OPS, normalize LOAD_FAST_CHECK and JUMP_BACK…
youknowone Mar 23, 2026
b559a6e
Remove duplicate return-None when block already has return
youknowone Mar 23, 2026
6b200d9
Improve __static_attributes__ collection accuracy
youknowone Mar 23, 2026
122f76a
Use method mode for function-local import attribute calls
youknowone Mar 25, 2026
d1641dd
Optimize constant iterable before GET_ITER to LOAD_CONST tuple
youknowone Mar 25, 2026
5b19b70
Fix cell variable ordering: parameters first, then alphabetical
youknowone Mar 25, 2026
5b32c68
Emit COPY_FREE_VARS before MAKE_CELL matching CPython order
youknowone Mar 25, 2026
96462fa
Fix RESUME AfterYield encoding to match CPython 3.14 (value 5)
youknowone Mar 25, 2026
296eaa1
Address code review feedback from #7481
youknowone Mar 25, 2026
b2eaa71
Fold constant list/set/tuple literals in compiler
youknowone Mar 23, 2026
c7f505d
Use BUILD_MAP 0 + MAP_ADD for large dicts (>= 16 pairs)
youknowone Mar 23, 2026
a23315a
Fix clippy warnings and cargo fmt
youknowone Mar 25, 2026
cab59e7
fix surrogate
youknowone Mar 25, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
Fix cell variable ordering: parameters first, then alphabetical 
CPython orders cell variables with parameter cells first (in
parameter definition order), then non-parameter cells sorted
alphabetically. Previously all cells were sorted alphabetically.

Also add for-loop iterable optimization: constant BUILD_LIST/SET
before GET_ITER is folded to just LOAD_CONST tuple.
  • Loading branch information
@youknowone
youknowone committed Mar 25, 2026
commit 5b19b7031ef036dd3ece601b0d5202021e37d855
22 changes: 19 additions & 3 deletions crates/codegen/src/compile.rs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -1043,16 +1043,32 @@ impl Compiler {

// Build cellvars using dictbytype (CELL scope or COMP_CELL flag, sorted)
let mut cellvar_cache = IndexSet::default();
let mut cell_names: Vec<_> = ste
// CPython ordering: parameter cells first (in parameter order),
// then non-parameter cells (alphabetically sorted)
let cell_symbols: Vec<_> = ste
.symbols
.iter()
.filter(|(_, s)| {
s.scope == SymbolScope::Cell || s.flags.contains(SymbolFlags::COMP_CELL)
})
.map(|(name, _)| name.clone())
.collect();
cell_names.sort();
for name in cell_names {
let mut param_cells = Vec::new();
let mut nonparam_cells = Vec::new();
for name in cell_symbols {
if varname_cache.contains(&name) {
param_cells.push(name);
} else {
nonparam_cells.push(name);
}
}
// param_cells are already in parameter order (from varname_cache insertion order)
param_cells.sort_by_key(|n| varname_cache.get_index_of(n.as_str()).unwrap_or(usize::MAX));
nonparam_cells.sort();
for name in param_cells {
cellvar_cache.insert(name);
}
for name in nonparam_cells {
cellvar_cache.insert(name);
Comment on lines +1071 to 1095
Copy link
Copy Markdown
Contributor

@coderabbitai coderabbitai Bot Mar 25, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟠 Major

Detect parameter cells from PARAMETER, not from varname_cache.

Line 1082 is using varname_cache.contains(&name), but varname_cache contains every local, not just arguments. That means ordinary captured locals also land in param_cells, so non-parameter cellvars keep local-definition order instead of the advertised alphabetical order. Because cellvars are indexed positionally later, this still shifts co_cellvars/localsplus metadata away from CPython.

Possible fix 
-        let cell_symbols: Vec<_> = ste
+        let cell_symbols: Vec<_> = ste
             .symbols
             .iter()
             .filter(|(_, s)| {
                 s.scope == SymbolScope::Cell || s.flags.contains(SymbolFlags::COMP_CELL)
             })
-            .map(|(name, _)| name.clone())
+            .map(|(name, sym)| (name.clone(), sym.flags))
             .collect();
         let mut param_cells = Vec::new();
         let mut nonparam_cells = Vec::new();
-        for name in cell_symbols {
-            if varname_cache.contains(&name) {
+        for (name, flags) in cell_symbols {
+            if flags.contains(SymbolFlags::PARAMETER) {
                 param_cells.push(name);
             } else {
                 nonparam_cells.push(name);
             }
         }
📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
let cell_symbols: Vec<_> = ste
.symbols
.iter()
.filter(|(_, s)| {
s.scope == SymbolScope::Cell || s.flags.contains(SymbolFlags::COMP_CELL)
})
.map(|(name, _)| name.clone())
.collect();
cell_names.sort();
for name in cell_names {
let mut param_cells = Vec::new();
let mut nonparam_cells = Vec::new();
for name in cell_symbols {
if varname_cache.contains(&name) {
param_cells.push(name);
} else {
nonparam_cells.push(name);
}
}
// param_cells are already in parameter order (from varname_cache insertion order)
param_cells.sort_by_key(|n| varname_cache.get_index_of(n.as_str()).unwrap_or(usize::MAX));
nonparam_cells.sort();
for name in param_cells {
cellvar_cache.insert(name);
}
for name in nonparam_cells {
cellvar_cache.insert(name);
let cell_symbols: Vec<_> = ste
.symbols
.iter()
.filter(|(_, s)| {
s.scope == SymbolScope::Cell || s.flags.contains(SymbolFlags::COMP_CELL)
})
.map(|(name, sym)| (name.clone(), sym.flags))
.collect();
let mut param_cells = Vec::new();
let mut nonparam_cells = Vec::new();
for (name, flags) in cell_symbols {
if flags.contains(SymbolFlags::PARAMETER) {
param_cells.push(name);
} else {
nonparam_cells.push(name);
}
}
// param_cells are already in parameter order (from varname_cache insertion order)
param_cells.sort_by_key(|n| varname_cache.get_index_of(n.as_str()).unwrap_or(usize::MAX));
nonparam_cells.sort();
for name in param_cells {
cellvar_cache.insert(name);
}
for name in nonparam_cells {
cellvar_cache.insert(name);
}
🤖 Prompt for AI Agents 
Verify each finding against the current code and only fix it if needed.

In `@crates/codegen/src/compile.rs` around lines 1071 - 1095, The code is
misclassifying parameter cellvars by checking varname_cache.contains(&name);
instead, classify a cell symbol as a parameter by inspecting its
SymbolFlags::PARAMETER on the original symbol entry (the map iterated as
ste.symbols) and push names into param_cells only when that flag is present,
otherwise push into nonparam_cells; keep the subsequent ordering logic (sort
param_cells by varname_cache.get_index_of(...) and nonparam_cells.sort()) and
then insert into cellvar_cache as before (referencing ste.symbols,
SymbolFlags::PARAMETER, varname_cache, param_cells, nonparam_cells, and
cellvar_cache).

}

Expand Down