Skip to content

Align compiler and VM bytecode contracts with CPython 3.15 - #9088

Closed
youknowone wants to merge 16 commits into
RustPython:compiler-315-prerequisites-main2a-c78-stw0bafrom
youknowdot:compiler-315-current-integration
Closed

youknowone wants to merge 16 commits into
RustPython:compiler-315-prerequisites-main2a-c78-stw0bafrom
youknowdot:compiler-315-current-integration

Conversation

@youknowone

@youknowone youknowone commented Oct 12, 2026 •

Copy link
Copy Markdown
Member

Summary

Align the compiler and VM bytecode contracts with final CPython 3.15.0, pinned to d220409a13b77de7a6c30abe16ea90db01528edc.

The transition covers opcode IDs/caches and stack contracts, common constants, iterator/generator creation, code-object field ownership, public opcode metadata, and the affected runtime consumers. Subsequent CI repairs cover script validation, exception cleanup, generator frame lifecycle, code-name ownership, warnings/format contracts, constant folding, and local monitoring events.

Current prerequisite stack

This Draft targets compiler-315-prerequisites-main2a-c78-stw0ba at 987188dbeaaac3d155a6c5e48b467db0900d43f3. The prerequisite contains:

#8951 and #9094 remain open at this checkpoint. Both exact prerequisite heads passed their hosted CI. Contributor #9095 remains separate and is not included.

Compiler head: a2f28469a7e91be7da0baa9383be702f63b8fc19, tree 0c1d52af0f5fac8d594c5a39705c4cb96532a399.

The compiler-only diff remains 87 files, 13,311 additions and 2,460 deletions, including 13 Lib files. Its ordered per-file additions/deletions are identical to the preceding green compiler head; no Lib bytes changed in this integration. The seven reconstructed compiler commits and subsequent prerequisite integration commits retain their validation history. No temporary diagnostic workflow/scripts remain. Canonical opcode/dis and affected fixture/helper copies came through update_lib from pinned final CPython. The separately approved monitoring setup-order exception imports optional _testcapi before registering tools, preserving the existing missing-helper skip.

Do not merge into this temporary prerequisite branch. After the remaining prerequisites land, reconcile with their actual main merges, retarget and verify the merge tree and hosted checks.

Validation provenance

The following local build/runtime results belong to the earlier reconciled head dc5ce40943b1980fb15d38cffb14e7bfb7cd1b36 / tree 38272ea96767ab699ff88f599214cafab75e7726:

  • Native build passed with Rust 1.99, jobs1, using that tree and its own Lib.
  • Strict root/VM/compiler-core/codegen/common all-target Clippy passed with warnings denied.
  • All 18 isolated bounded invocations passed: ten ordinary fixture processes, three selector groups (seven generator/coroutine methods, seventeen compiler/disassembly/monitoring/optimizer methods, and four warning-filter methods), and five unchanged snippet processes. The optional C API fixture verified five existing skips and subsequent clean ordinary setup; it did not execute C API test bodies.
  • Exact source, binary and 2,342 validation inputs were bound before execution. Native SHA256: 23fdccfab0bfae348175cd19ac1967af9ed156cd39a41e53acfbbddbd05390d5; its embedded stamp is dc5ce40.
  • Source/ownership integration and the complete source/native recovery archive were independently reviewed. These are focused Linux results for that historical source; they are not relabeled as fresh local runtime validation of the current integration.

The preceding compiler head 89cd7bb8d6220dc244b425dcd8cb18c9ca99fbef passed all 25 hosted CI jobs and CodSpeed. The current prerequisite #9094 head also passed all 25 jobs and CodSpeed, including the four previously stalled interpreter/STW tests. The original Windows manager failure was observed through preserved canonical output; after socket detachment, the original remote-manager method and the full Windows suite passed. Temporary capture code was then removed.

The current integration has normal-hook and independent source/ancestry/diff validation. It introduces current-main changes and the validated STW reentry hardening without changing the compiler-only patch. No new local runtime probe was run for this merge. Hosted CI for the current head passed all 25 jobs, including all three platform Python suites and Rust checks: CI run 38410788160. CodSpeed run 38410788106 also passed. These results establish CI stabilization for this exact integration; the compatibility limits below remain.

Compatibility limits

This is not complete byte-exact CPython compatibility. The historical unchanged 120-case strict corpus had 117 recursive raw-field matches and zero complete byte matches; it has not been rerun or relabeled for this reconciled head. Full marshal parity and PEP 810 runtime support remain incomplete. Private development pyc magic rejects incompatible intermediate formats, while the public cache tag is implementation/version-derived.

AI-assisted with OpenAI Codex; model identifier unavailable.

@coderabbitai

coderabbitai Bot commented Oct 12, 2026 •

Copy link
Copy Markdown
Contributor

Important

Draft PR not reviewed

Draft PRs are not automatically reviewed by default.

  • Trigger a manual review

To automatically review draft PRs, update your CodeRabbit configuration:

reviews:
  auto_review:
    drafts: true
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@github-actions

github-actions Bot commented Oct 12, 2026 •

Copy link
Copy Markdown
Contributor

📦 Library Dependencies

The following Lib/ modules were modified. Here are their dependencies:

:4: SyntaxWarning: invalid decimal literal
:7: SyntaxWarning: "\z" is an invalid escape sequence. Such sequences will not work in the future. Did you mean "\z"? A raw string is also an option.
:10: SyntaxWarning: "\400" is an invalid octal escape sequence. Such sequences will not work in the future. Did you mean "\400"? A raw string is also an option.
:4: SyntaxWarning: invalid decimal literal
:7: SyntaxWarning: "\z" is an invalid escape sequence. Such sequences will not work in the future. Did you mean "\z"? A raw string is also an option.
:10: SyntaxWarning: "\400" is an invalid octal escape sequence. Such sequences will not work in the future. Did you mean "\400"? A raw string is also an option.
:4: SyntaxWarning: invalid decimal literal
:7: SyntaxWarning: "\z" is an invalid escape sequence. Such sequences will not work in the future. Did you mean "\z"? A raw string is also an option.
:10: SyntaxWarning: "\400" is an invalid octal escape sequence. Such sequences will not work in the future. Did you mean "\400"? A raw string is also an option.
[x] test: cpython/Lib/test/test_compile.py
[x] test: cpython/Lib/test/test_compiler_assemble.py
[x] test: cpython/Lib/test/test_compiler_codegen.py
[x] test: cpython/Lib/test/test_peepholer.py

dependencies:

dependent tests: (no tests depend on compile)

[ ] lib: cpython/Lib/code.py
[x] test: cpython/Lib/test/test_code_module.py

dependencies:

  • code (native: builtins, readline, sys)
    • argparse (native: sys)
    • codeop
    • traceback (native: _missing_stdlib_info, _suggestions, collections.abc, importlib.machinery, importlib.resources.readers, importlib.util, itertools, sys, unicodedata)

dependent tests: (2 tests)
- [ ] pdb: test_pdb
- [ ] sqlite3.main: test_sqlite3

[ ] test: cpython/Lib/test/test_monitoring.py (TODO: 3)

dependencies:

dependent tests: (no tests depend on monitoring)

[x] lib: cpython/Lib/warnings.py
[x] lib: cpython/Lib/_py_warnings.py
[ ] test: cpython/Lib/test/test_warnings (TODO: 1)

dependencies:

  • warnings

dependent tests: (55 tests)

  • warnings: test_argparse test_array test_asyncio test_buffer test_builtin test_codecs test_codeop test_compile test_coroutines test_ctypes test_decimal test_descr test_fnmatch test_fstring test_funcattrs test_genericpath test_glob test_global test_grammar test_gzip test_hashlib test_hmac test_importlib test_inspect test_io test_logging test_ntpath test_os test_pickle test_posix test_pty test_pyclbr test_pyrepl test_random test_re test_runpy test_set test_socket test_sqlite3 test_str test_string_literals test_support test_sys test_sys_settrace test_tarfile test_tempfile test_thread test_threading test_type_cache test_typing test_unicode_file_functions test_unittest test_unparse test_xml_etree test_zipimport

[x] lib: cpython/Lib/opcode.py
[x] lib: cpython/Lib/_opcode_metadata.py
[x] test: cpython/Lib/test/test__opcode.py
[x] test: cpython/Lib/test/test_opcodes.py

dependencies:

  • opcode

dependent tests: (50 tests)

  • opcode: test__opcode test_code test_compile test_dis test_peepholer
    • dis: test_ast test_compiler_assemble test_dtrace test_fstring test_inspect test_monitoring test_opcache test_patma test_positional_only_arg test_type_cache
      • bdb: test_bdb test_pdb
      • inspect: test_abc test_argparse test_asyncgen test_buffer test_builtin test_clinic test_collections test_coroutines test_decimal test_enum test_functools test_generators test_grammar test_ntpath test_operator test_posixpath test_pydoc test_signal test_sqlite3 test_traceback test_turtle test_type_annotations test_type_params test_types test_typing test_unittest test_yield_from test_zipimport test_zipimport_support test_zoneinfo
      • modulefinder: test_importlib test_modulefinder
      • trace: test_trace

[ ] test: cpython/Lib/test/test_import (TODO: 3)

dependencies:

dependent tests: (no tests depend on import)

[x] lib: cpython/Lib/dis.py
[x] test: cpython/Lib/test/test_dis.py (TODO: 1)

dependencies:

  • dis

dependent tests: (80 tests)

  • dis: test__opcode test_ast test_code test_compile test_compiler_assemble test_dis test_dtrace test_fstring test_inspect test_monitoring test_opcache test_patma test_peepholer test_positional_only_arg test_type_cache
    • bdb: test_bdb test_pdb
    • inspect: test_abc test_argparse test_asyncgen test_buffer test_builtin test_clinic test_collections test_coroutines test_decimal test_enum test_functools test_generators test_grammar test_ntpath test_operator test_posixpath test_pydoc test_signal test_sqlite3 test_traceback test_turtle test_type_annotations test_type_params test_types test_typing test_unittest test_yield_from test_zipimport test_zipimport_support test_zoneinfo
      • ast: test_codeop test_compiler_codegen test_future_stmt test_peg_generator test_site test_ssl test_type_comments test_ucn test_unparse
      • asyncio: test_asyncio test_concurrent_futures test_external_inspection test_logging test_os test_unittest
      • cmd: test_cmd
      • dataclasses: test__colorize test_copy test_ctypes test_genericalias test_pprint test_regrtest
      • importlib.metadata: test_importlib
      • pkgutil: test_pkgutil test_pyrepl test_runpy
      • rlcompleter: test_pyrepl test_rlcompleter
      • trace: test_trace
      • xmlrpc.server: test_docxmlrpc test_xmlrpc
    • modulefinder: test_importlib test_modulefinder

[x] test: cpython/Lib/test/test_generators.py
[ ] test: cpython/Lib/test/test_genexps.py (TODO: 2)
[x] test: cpython/Lib/test/test_generator_stop.py
[x] test: cpython/Lib/test/test_yield_from.py

dependencies:

dependent tests: (no tests depend on generator)

[ ] test: cpython/Lib/test/test_format.py

dependencies:

dependent tests: (no tests depend on format)

Legend:

  • [+] path exists in CPython
  • [x] up-to-date, [ ] outdated

@codspeed

codspeed Bot commented Oct 13, 2026 •

Copy link
Copy Markdown
Contributor

Merging this PR will not alter performance

✅ 62 untouched benchmarks
⏩ 4 skipped benchmarks1


Comparing youknowdot:compiler-315-current-integration (89cd7bb) with main (e09846b)2

Open in CodSpeed

Footnotes

  1. 4 benchmarks were skipped, so the baseline results were used instead. If they were deleted from the codebase, click here and archive them to remove them from the performance reports. ↩

  2. No successful run was found on compiler-315-prerequisites-e098-c78-a26-stw (f1e4a56) during the generation of this report, so main (e09846b) was used instead as the comparison base. There might be some changes unrelated to this pull request in this report. ↩

Reconstruct the compiler-only baseline from validated487ae47 on the current main and frozen mapping prerequisitec78d811. Preserve the full source delta while dropping obsolete prerequisite ancestry; detailed old-to-new provenance is retained in the recovery artifact.

Assisted-by: OpenAI Codex: model identifier unavailable
…racts

Replay reviewed compiler CI repairs 2191a9d, 1f1fe04, a2dacaf on the frozen current prerequisite. Intermediate compile/diagnostic corrections are combined where they address the same failure.

Assisted-by: OpenAI Codex: model identifier unavailable
Replay reviewed compiler CI repairs 476bb74 on the frozen current prerequisite. Intermediate compile/diagnostic corrections are combined where they address the same failure.

Assisted-by: OpenAI Codex: model identifier unavailable
Replay reviewed compiler CI repairs 6482adf on the frozen current prerequisite. Intermediate compile/diagnostic corrections are combined where they address the same failure.

Assisted-by: OpenAI Codex: model identifier unavailable
Replay reviewed compiler CI repairs 246c309 on the frozen current prerequisite. Intermediate compile/diagnostic corrections are combined where they address the same failure.

Assisted-by: OpenAI Codex: model identifier unavailable
Replay reviewed compiler CI repairs 7ecedcf on the frozen current prerequisite. Intermediate compile/diagnostic corrections are combined where they address the same failure.

Assisted-by: OpenAI Codex: model identifier unavailable
…l 3.15

Replay reviewed compiler CI repairs ec206b1, 191fe58 on the frozen current prerequisite. Intermediate compile/diagnostic corrections are combined where they address the same failure.

Assisted-by: OpenAI Codex: model identifier unavailable
@youknowone
youknowone changed the base branch from compiler-315-current-prerequisites to compiler-315-prerequisites-e098-c78 October 13, 2026 06:27
@youknowdot
youknowdot force-pushed the compiler-315-current-integration branch from c70fd5d to dc5ce40 Compare October 13, 2026 06:28
Preserve canonical tests while recording bounded per-method progress under separate Windows Job Objects.

Assisted-by: OpenAI Codex: model identifier unavailable
Replace successful isolated diagnostics with persistent capture of the existing canonical worker output, without changing test selection or timeouts.

Assisted-by: OpenAI Codex: model identifier unavailable
Snapshot locked send buffers before waiting and preserve the host-captured Winsock error across VM reattachment.

Assisted-by: OpenAI Codex: model identifier unavailable
Combine exact RustPython#8951 mapping head c78d811 and RustPython#9093 socket I/O head a26d161 on main e09846b.

Assisted-by: OpenAI Codex: model identifier unavailable
Keep the compiler-only diff unchanged while incorporating the exact source-reviewed RustPython#9093 repair.

Assisted-by: OpenAI Codex: model identifier unavailable
@youknowone
youknowone changed the base branch from compiler-315-prerequisites-e098-c78 to compiler-315-prerequisites-e098-c78-a26 October 13, 2026 09:52
Assisted-by: OpenAI Codex: model identifier unavailable
Assisted-by: OpenAI Codex: model identifier unavailable
The original test_remote and all 82 manager tests completed in the retained hosted capture. Restore the unchanged canonical invocation and remove temporary capture upload/launcher.

Assisted-by: OpenAI Codex: model identifier unavailable
@youknowone
youknowone changed the base branch from compiler-315-prerequisites-e098-c78-a26 to compiler-315-prerequisites-e098-c78-a26-stw October 13, 2026 11:46
Preserve the compiler-only changes while incorporating current main and the validated mapping/STW prerequisites.

Assisted-by: OpenAI Codex: model identifier unavailable
@youknowone
youknowone changed the base branch from compiler-315-prerequisites-e098-c78-a26-stw to compiler-315-prerequisites-main2a-c78-stw0ba October 13, 2026 14:07
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants