Skip to content

Commit 1f1be5e

Browse files
youknowoneyouknowone
authored
Align bytecode codegen structure with CPython 3.14 (#7588)
* Align bytecode codegen structure with CPython 3.14 * Bytecode parity - constant folding, annotation ordering, superinstruction alignment - Add BoolOp constant folding with short-circuit semantics in compile_expression - Add constant truthiness evaluation for assert statement optimization - Disable const collection/boolop folding in starred unpack and assignment contexts - Move annotation block generation after body with AnnotationsPlaceholder splicing - Reorder insert_superinstructions to run before push_cold_blocks (matching flowgraph.c) - Lower LOAD_CLOSURE after superinstructions to avoid false LOAD_FAST_LOAD_FAST - Add ToBool before PopJumpIf in comparisons and chained compare cleanup blocks - Unify annotation dict building to always use incremental BuildMap + StoreSubscr - Add TrueDivide constant folding for integer operands - Fold constant sets to Frozenset (not Tuple) in try_fold_constant_collection - Add PyVmBag for frozenset constant materialization in code objects - Add remove_redundant_const_pop_top_pairs pass and peephole const+branch folding - Emit Nop for skipped constant expressions and constant-true asserts - Preserve comprehension local ordering by source-order bound name collection - Simplify annotation scanning in symboltable (remove simple-name gate) * Fix CI regressions in marshal and fast-local ops * impl more * Align bytecode codegen with CPython structure * Bytecode parity - comprehension/except scope ordering, load_fast_borrow fixes - Reorder comprehension symbol-table walk so the outermost iterator registers its sub_tables in the enclosing scope before the comp scope, and rescan elt/ifs in CPython's order. Codegen peeks past the outermost iterator's nested scopes to find the comprehension table. - For plain try/except, emit handler sub_tables before the else block so codegen's linear sub_table cursor stays aligned. - Rename `collect_simple_annotations` to `collect_annotations` and evaluate non-simple annotations during __annotate__ compilation to preserve source-order side effects while keeping the simple-name index stable. - Dedupe equivalent code constants in `arg_constant` and add a structural equality check on `CodeObject`. - Disable LOAD_FAST_BORROW for the tail end block when a try has a bare `except:` clause, and have `new_block` inherit the flag from the current block. - Remove `cfg!(debug_assertions)` guard around the `optimize_load_fast_borrow` start-depth check so mismatches are handled (return instead of assert) in release builds. - Collapse nop-only blocks that precede a return epilogue and hoist the prior line number into the next real instruction so the line table matches. - Unmark now-passing `test_consts_in_conditionals`, `test_load_fast_unknown_simple`, `test_load_fast_known_because_already_loaded`, and PEP 646 f3/f4 annotation checks. * Bytecode parity - try/except line tracking, assert 0 shape - In `compile_try_except`, drop the leading Nop and set the end block's source range from the last orelse/body statement so line events after the try fall on the right line. - Recognise constant-false asserts as the direct-raise shape (no ToBool/PopJumpIfFalse) and flip the test assertion accordingly. - Extend `remove_redundant_nops_in_blocks` to also look through a trailing nop before a return-epilogue pair (LoadConst/ReturnValue or LoadSmallInt/ReturnValue) so the epilogue keeps the correct line number. - Rename `preds` to `predecessor_blocks` in the LOAD_FAST_BORROW disable pass and add a test-only `debug_late_cfg_trace` helper. - Regenerate the `nested_double_async_with` snapshot: the tail reference to `stop_exc` now emits LOAD_FAST instead of LOAD_FAST_BORROW. * Bytecode parity - iter folding, break/continue line, cold inlining - Fold a constant list iterable into a constant tuple in for-loop iterable position, matching the CPython optimizer, and strip a redundant LIST_TO_TUPLE immediately before GET_ITER in the IR peephole pass. - Emit a Nop at the break/continue source range before unwinding so line events land on the break/continue statement instead of the following instruction. - Drop `propagate_disable_load_fast_borrow`; the forward propagation was over-zealous and the per-block inheritance in `new_block` plus the bare-except marker are enough. - Relax `inline_small_or_no_lineno_blocks` so small exit blocks at the tail of a cold block are always inlined, not just return epilogues. - Add codegen tests covering the LIST_TO_TUPLE/GET_ITER peephole and the late-CFG trace helper for a for-loop list-literal iterable.
1 parent 4f1cf6d commit 1f1be5e

26 files changed

Lines changed: 4770 additions & 1216 deletions

‎Lib/test/test_compile.py‎

Lines changed: 0 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -1084,7 +1084,6 @@ def unused_block_while_else():
10841084
self.assertEqual('RETURN_VALUE', opcodes[-1].opname)
10851085
self.assertEqual(None, opcodes[-1].argval)
10861086

1087-
@unittest.expectedFailure # TODO: RUSTPYTHON; AssertionError: 3 != 8
10881087
def test_false_while_loop(self):
10891088
def break_in_while():
10901089
while False:
@@ -1103,7 +1102,6 @@ def continue_in_while():
11031102
self.assertEqual('RETURN_VALUE', opcodes[-1].opname)
11041103
self.assertEqual(None, opcodes[1].argval)
11051104

1106-
@unittest.expectedFailure # TODO: RUSTPYTHON
11071105
def test_consts_in_conditionals(self):
11081106
def and_true(x):
11091107
return True and x

‎Lib/test/test_grammar.py‎

Lines changed: 0 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -304,7 +304,6 @@ def test_var_annot_syntax_errors(self):
304304
" nonlocal x\n"
305305
" x: int\n")
306306

307-
@unittest.expectedFailure # TODO: RUSTPYTHON
308307
def test_var_annot_basic_semantics(self):
309308
# execution order
310309
with self.assertRaises(ZeroDivisionError):

‎Lib/test/test_patma.py‎

Lines changed: 0 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -3448,7 +3448,6 @@ def f(command): # 0
34483448
self.assertListEqual(self._trace(f, "go x"), [1, 2, 4, 5])
34493449
self.assertListEqual(self._trace(f, "spam"), [1, 2, 4, 6, 7])
34503450

3451-
@unittest.expectedFailure # TODO: RUSTPYTHON
34523451
def test_default_capture(self):
34533452
def f(command): # 0
34543453
match command.split(): # 1
@@ -3463,7 +3462,6 @@ def f(command): # 0
34633462
self.assertListEqual(self._trace(f, "go x"), [1, 2, 4, 5])
34643463
self.assertListEqual(self._trace(f, "spam"), [1, 2, 4, 6, 7])
34653464

3466-
@unittest.expectedFailure # TODO: RUSTPYTHON
34673465
def test_no_default(self):
34683466
def f(command): # 0
34693467
match command.split(): # 1
@@ -3476,7 +3474,6 @@ def f(command): # 0
34763474
self.assertListEqual(self._trace(f, "go x"), [1, 2, 4, 5])
34773475
self.assertListEqual(self._trace(f, "spam"), [1, 2, 4])
34783476

3479-
@unittest.expectedFailure # TODO: RUSTPYTHON
34803477
def test_only_default_wildcard(self):
34813478
def f(command): # 0
34823479
match command.split(): # 1
@@ -3487,7 +3484,6 @@ def f(command): # 0
34873484
self.assertListEqual(self._trace(f, "go x"), [1, 2, 3])
34883485
self.assertListEqual(self._trace(f, "spam"), [1, 2, 3])
34893486

3490-
@unittest.expectedFailure # TODO: RUSTPYTHON
34913487
def test_only_default_capture(self):
34923488
def f(command): # 0
34933489
match command.split(): # 1

‎Lib/test/test_peepholer.py‎

Lines changed: 1 addition & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -441,6 +441,7 @@ def test_constant_folding_binop(self):
441441
self.check_lnotab(code)
442442

443443

444+
@unittest.expectedFailure # TODO: RUSTPYTHON
444445
def test_constant_folding_remove_nop_location(self):
445446
sources = [
446447
"""
@@ -785,15 +786,13 @@ def f(a, b, c):
785786
c, b, a = a, b, c
786787
self.assertNotInBytecode(f, "SWAP")
787788

788-
@unittest.expectedFailure # TODO: RUSTPYTHON
789789
def test_static_swaps_match_mapping(self):
790790
for a, b, c in product("_a", "_b", "_c"):
791791
pattern = f"{{'a': {a}, 'b': {b}, 'c': {c}}}"
792792
with self.subTest(pattern):
793793
code = compile_pattern_with_fast_locals(pattern)
794794
self.assertNotInBytecode(code, "SWAP")
795795

796-
@unittest.expectedFailure # TODO: RUSTPYTHON
797796
def test_static_swaps_match_class(self):
798797
forms = [
799798
"C({}, {}, {})",
@@ -808,7 +807,6 @@ def test_static_swaps_match_class(self):
808807
code = compile_pattern_with_fast_locals(pattern)
809808
self.assertNotInBytecode(code, "SWAP")
810809

811-
@unittest.expectedFailure # TODO: RUSTPYTHON
812810
def test_static_swaps_match_sequence(self):
813811
swaps = {"*_, b, c", "a, *_, c", "a, b, *_"}
814812
forms = ["{}, {}, {}", "{}, {}, *{}", "{}, *{}, {}", "*{}, {}, {}"]
@@ -863,7 +861,6 @@ def f():
863861
y = x + x
864862
self.assertInBytecode(f, 'LOAD_FAST_BORROW_LOAD_FAST_BORROW')
865863

866-
@unittest.expectedFailure # TODO: RUSTPYTHON; RETURN_VALUE
867864
def test_load_fast_unknown_simple(self):
868865
def f():
869866
if condition():
@@ -906,7 +903,6 @@ def f5(x=0):
906903
self.assertInBytecode(f5, 'LOAD_FAST_BORROW')
907904
self.assertNotInBytecode(f5, 'LOAD_FAST_CHECK')
908905

909-
@unittest.expectedFailure # TODO: RUSTPYTHON; RETURN_VALUE
910906
def test_load_fast_known_because_already_loaded(self):
911907
def f():
912908
if condition():

‎Lib/test/test_pep646_syntax.py‎

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -305,11 +305,11 @@
305305
{'args': StarredB}
306306
307307
>>> def f3(*args: *b, arg1: int): pass
308-
>>> f3.__annotations__ # TODO: RUSTPYTHON # doctest: +EXPECTED_FAILURE
308+
>>> f3.__annotations__
309309
{'args': StarredB, 'arg1': <class 'int'>}
310310
311311
>>> def f4(*args: *b, arg1: int = 2): pass
312-
>>> f4.__annotations__ # TODO: RUSTPYTHON # doctest: +EXPECTED_FAILURE
312+
>>> f4.__annotations__
313313
{'args': StarredB, 'arg1': <class 'int'>}
314314
315315
>>> def f5(*args: *b = (1,)): pass # TODO: RUSTPYTHON # doctest: +EXPECTED_FAILURE

‎Lib/test/test_sys_settrace.py‎

Lines changed: 0 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -1957,8 +1957,6 @@ def test_jump_out_of_finally_block(output):
19571957
finally:
19581958
output.append(5)
19591959

1960-
# TODO: RUSTPYTHON
1961-
@unittest.expectedFailure
19621960
@jump_test(1, 5, [], (ValueError, "into an 'except'"))
19631961
def test_no_jump_into_bare_except_block(output):
19641962
output.append(1)
@@ -1967,8 +1965,6 @@ def test_no_jump_into_bare_except_block(output):
19671965
except:
19681966
output.append(5)
19691967

1970-
# TODO: RUSTPYTHON
1971-
@unittest.expectedFailure
19721968
@jump_test(1, 5, [], (ValueError, "into an 'except'"))
19731969
def test_no_jump_into_qualified_except_block(output):
19741970
output.append(1)

0 commit comments

Comments
 (0)