Align code introspection and monitoring with Python 3.15 by youknowdot · Pull Request #9178 · RustPython/RustPython

youknowdot · 2026-10-17T15:59:39Z

Summary

Extract a coherent code-object metadata and monitoring compatibility batch from #8954, based directly on main at fc9f4a9.

Preserve co_qualname object semantics: exact strings are interned, string subclasses retain identity, and marshal round trips preserve WTF8 qualified names. Marshal keeps CPython's exact-string policy.
Apply no_debug_ranges consistently and normalize valid signed location-table deltas without changing bytecode offsets. Unconvertible tables retain existing bytes; this is not a general malformed-table reader hardening change.
Track monitoring DISABLE by event and physical instruction site, preserve CALL/C_RETURN/C_RAISE coupling, and handle local non-instrumented subscriptions.
Match observed legacy branch behavior through LINE/INSTRUCTION wrappers, including dispatch source offsets and the one-invocation LINE removal transition.
Release replaced/cleared monitoring callbacks outside the monitoring lock, retaining CPython's tool-name visibility and nested-free ordering. Restore tracing state before interpreting callback results.
Import the canonical monitoring shutdown helper through scripts/update_lib and retire only its genuinely executed, now-passing expectedFailure marker.

Canonical source: CPython 3.15.0 at d220409a13b77de7a6c30abe16ea90db01528edc. Existing related canonical files were already present; no artificial Lib updates, duplicate snippets, or Python-behavior Rust tests are added. Merged function/type lifetime fixes remain intact.

Validation

Current source: 675826ba7a.

Fresh default-feature release build (filesystem stdlib) and normal commit hooks passed.
Five bounded oracle/candidate observations match all JSON values, including exact-string interning, subclass identity/rejection, surrogate marshal values, valid signed location entries, no-debug ranges, legacy/direct branch disabling, chronological wrapper offsets, and callback-finalizer reentry/tool reuse.
Pinned CPython test_code: 42 total, 41 passed and 1 skipped. Candidate repository test_code: 37 total, 21 passed, 15 skipped, 1 existing expected failure.
Pinned CPython test_monitoring: 98 passed. Candidate repository test_monitoring: 98 total, 91 passed, 5 skipped, 2 existing expected failures. The shutdown helper ran successfully in a child interpreter before its marker was retired.
Workspace native: 1,369 passed. C-API native: 123 passed, 4 existing ignored tests.
Strict workspace/C-API Clippy and compiler-only nonthreading check passed. Workspace docs: 24 passed, 18 ignored; C-API docs: no test cases.
test_marshal: 75 total, 59 passed, 15 skipped, 1 existing expected failure.
test_sys_settrace: 448 total, 371 passed, 77 skipped.
test_dis: 145 total, 119 passed, 26 skipped.
test_types: 136 total, 132 passed, 2 skipped, 2 existing expected failures.

Local gates bind source 675826ba7a to base fc9f4a9. Main advanced independently during validation; hosted merge-ref CI must establish integration with those landed changes before merge readiness. Local old-base test counts are not claimed as merged-tree results.

The repository suites retain genuine unsupported C-extension/resource guards. C-API native coverage does not imply Python _testcapi availability. No CPU/network/bigmem resources were enabled.

Scope limits

This does not claim complete monitoring parity for callback-time instrumentation replacement, cross-tool callback mutation/order, orphan/loop predecessor mapping, or reserved tracing/profiling instrumentation. Those inherited paths are not redesigned here. It also does not claim safe interpretation of arbitrary malformed location tables or lossless unrelated compiler-only string fields.

Baseline mismatches were retained and corrected against pinned source and oracle observations; no new test exclusions or altered assertions were used.

Assisted-by: OpenAI Codex:model version unavailable

Preserve canonical qualified-name interning and subclass values, normalize valid debug-range tables, and correct event-specific monitoring disable and callback ownership boundaries. Assisted-by: OpenAI Codex:model version unavailable

Copy the missing test_monitoring dependency from pinned CPython 3.15.0 through scripts/update_lib, retaining existing suite guards. Assisted-by: OpenAI Codex:model version unavailable

Model active source instrumentation across all subscribed tools instead of unconditional legacy edge pairing. Preserve tool names during unlocked callback finalization before releasing them. Assisted-by: OpenAI Codex:model version unavailable

Retain decoded WTF8 qualnames, use exact-str marshal dispatch, and model wrapped branch dispatch with physical DISABLE sites. Retire the proven monitoring shutdown guard through update_lib. Assisted-by: OpenAI Codex:model version unavailable

coderabbitai · 2026-10-17T15:59:47Z

Note

Currently processing new changes in this PR. This may take a few minutes, please wait...

⚙️ Run configuration

Configuration used: Repository: RustPython/RustPython/.coderabbit.yml
Review profile: CHILL
Plan: Advanced
Run ID: b02f77fd-7476-4676-a0eb-9c61f21c3ab7

📥 Commits

Reviewing files that changed from the base of the PR and between 65bbaca and 675826b.

⛔ Files ignored due to path filters: 2

Lib/test/_test_monitoring_shutdown.py is excluded by !Lib/**
Lib/test/test_monitoring.py is excluded by !Lib/**

📒 Files selected for processing: 7

crates/compiler-core/src/marshal.rs
crates/vm/src/builtins/code.rs
crates/vm/src/builtins/code/const_cache.rs
crates/vm/src/builtins/function.rs
crates/vm/src/frame.rs
crates/vm/src/stdlib/marshal.rs
crates/vm/src/stdlib/sys/monitoring.rs

+--------------------------------------------------------------------------+
| This function returns three different shapes. It's basically modern art. |
+--------------------------------------------------------------------------+
  \
   \     /)/)
        ( '')
        /  |
      o(___)

✨ Finishing Touches

🧪 Generate unit tests (beta)

Create a new PR

Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

_{Comment @coderabbitai help to get the list of available commands.}

github-actions · 2026-10-17T16:05:45Z

📦 Library Dependencies

The following Lib/ modules were modified. Here are their dependencies:

:4: SyntaxWarning: invalid decimal literal
:7: SyntaxWarning: "\z" is an invalid escape sequence. Such sequences will not work in the future. Did you mean "\z"? A raw string is also an option.
:10: SyntaxWarning: "\400" is an invalid octal escape sequence. Such sequences will not work in the future. Did you mean "\400"? A raw string is also an option.
:4: SyntaxWarning: invalid decimal literal
:7: SyntaxWarning: "\z" is an invalid escape sequence. Such sequences will not work in the future. Did you mean "\z"? A raw string is also an option.
:10: SyntaxWarning: "\400" is an invalid octal escape sequence. Such sequences will not work in the future. Did you mean "\400"? A raw string is also an option.
:4: SyntaxWarning: invalid decimal literal
:7: SyntaxWarning: "\z" is an invalid escape sequence. Such sequences will not work in the future. Did you mean "\z"? A raw string is also an option.
:10: SyntaxWarning: "\400" is an invalid octal escape sequence. Such sequences will not work in the future. Did you mean "\400"? A raw string is also an option.
[ ] lib: cpython/Lib/io.py
[x] lib: cpython/Lib/_pyio.py
[ ] test: cpython/Lib/test/test_io (TODO: 10)

dependencies:

io (native: _io, _thread, errno, msvcrt, sys)
- _pyio
- _collections_abc (native: sys)
- abc, codecs, locale, os, stat, warnings

dependent tests: (108 tests)

io: test__colorize test_android test_argparse test_ast test_asyncio test_base64 test_buffer test_bufio test_builtin test_bz2 test_calendar test_cmd test_cmd_line_script test_codecs test_compile test_compileall test_compiler_assemble test_concurrent_futures test_configparser test_contextlib test_csv test_dbm_dumb test_descr test_dis test_email test_enum test_file test_fileinput test_fileio test_ftplib test_generated_cases test_getpass test_gzip test_hashlib test_http_cookiejar test_httplib test_httpservers test_importlib test_inspect test_io test_json test_largefile test_logging test_lzma test_mailbox test_marshal test_memoryio test_memoryview test_mimetypes test_minidom test_multibytecodec test_optparse test_pathlib test_pdb test_peg_generator test_pickle test_pickletools test_platform test_plistlib test_pprint test_print test_profile test_pstats test_pty test_pulldom test_pydoc test_pyexpat test_pyrepl test_quopri test_regrtest test_robotparser test_sax test_shlex test_shutil test_site test_smtplib test_socket test_socketserver test_subprocess test_support test_sys test_tarfile test_tempfile test_threadedtempfile test_timeit test_tokenize test_traceback test_types test_typing test_unittest test_univnewlines test_urllib test_urllib2 test_uuid test_wave test_webbrowser test_winconsoleio test_wsgiref test_xml_dom_xmlbuilder test_xml_etree test_xml_etree_c test_xmlrpc test_xpickle test_zipapp test_zipfile test_zipimport test_zoneinfo test_zstd

[x] test: cpython/Lib/test/test_monitoring.py (TODO: 3)

dependencies:

dependent tests: (no tests depend on monitoring)

[ ] test: cpython/Lib/test/test_finalization.py

dependencies:

dependent tests: (no tests depend on finalization)

(module 'bufio' not found)

(module 'largefile' not found)

[ ] lib: cpython/Lib/weakref.py
[x] lib: cpython/Lib/_weakrefset.py
[ ] test: cpython/Lib/test/test_weakref.py (TODO: 9)
[x] test: cpython/Lib/test/test_weakset.py

dependencies:

weakref (native: _weakref, atexit, gc, itertools, sys)
- _weakrefset
- _collections_abc
- _weakrefset, copy, types

dependent tests: (200 tests)

[ ] test: cpython/Lib/test/test_io (TODO: 10)

dependencies:

dependent tests: (no tests depend on file)

(module 'fileio' not found)

[ ] test: cpython/Lib/test/test_descr.py (TODO: 2)
[ ] test: cpython/Lib/test/test_descrtut.py (TODO: 2)

dependencies:

dependent tests: (no tests depend on descr)

(module 'memoryio' not found)

[ ] test: cpython/Lib/test/test_gc.py (TODO: 4)

dependencies:

dependent tests: (174 tests)

(module 'univnewlines' not found)

Legend:

[+] path exists in CPython
[x] up-to-date, [ ] outdated

chatgpt-codex-connector · 2026-10-18T01:57:26Z

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review	Status	Commit	Review trigger
📝 Code Review	✅ Completed 2026-10-18T02:05:38.560757Z	`675826b`	Draft marked ready

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

chatgpt-codex-connector

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 675826ba7a

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

chatgpt-codex-connector · 2026-10-18T02:05:36Z

+        callbacks
+    };
+    // Callback finalizers may reenter monitoring; release owners after unlocking.
+    drop(callbacks);


Re-clear reentrant event registrations after callback teardown

When a callback is kept alive only by the monitoring table and its __del__ calls set_events or set_local_events (and may re-register itself), this drop runs after clear_tool_id/free_tool_id have already cleared the tool's masks, so the destructor's new masks survive; free_tool_id can then leave a nameless tool with live callbacks/events, and immediate reuse inherits them. Re-clear the tool's masks after finalizers run while still dropping callback owners outside the lock.

Useful? React with 👍 / 👎.

chatgpt-codex-connector · 2026-10-18T02:05:36Z

+
+import sys
+
+class CallableCycle:


Keep executable fixtures out of restricted test changes

Adding this class creates new executable test logic in Lib/test, but the repository's root instructions permit test-file changes only to add or remove @unittest.expectedFailure markers and their associated TODO comments. Restore the shutdown test's marker rather than adding this helper so the change stays within the allowed test-update policy.

AGENTS.md reference: AGENTS.md:L274-L282

Useful? React with 👍 / 👎.

youknowdot added 4 commits October 17, 2026 22:49

Import canonical monitoring shutdown helper

61c51b0

Copy the missing test_monitoring dependency from pinned CPython 3.15.0 through scripts/update_lib, retaining existing suite guards. Assisted-by: OpenAI Codex:model version unavailable

youknowone approved these changes Oct 18, 2026

View reviewed changes

youknowone marked this pull request as ready for review October 18, 2026 01:57

youknowone merged commit efa8c26 into RustPython:main Oct 18, 2026
22 checks passed

chatgpt-codex-connector Bot reviewed Oct 18, 2026

View reviewed changes

Navigation Menu

Repository navigation

Align code introspection and monitoring with Python 3.15 - #9178

Align code introspection and monitoring with Python 3.15#9178
youknowone merged 4 commits into
RustPython:mainfrom
youknowdot:fix/code-introspection-monitoring-315

youknowdot commented Oct 17, 2026

Uh oh!

coderabbitai Bot commented Oct 17, 2026 •

edited

Loading

Uh oh!

github-actions Bot commented Oct 17, 2026

Uh oh!

Uh oh!

chatgpt-codex-connector Bot commented Oct 18, 2026 •

edited

Loading

Uh oh!

chatgpt-codex-connector Bot left a comment

Uh oh!

chatgpt-codex-connector Bot Oct 18, 2026

Uh oh!

chatgpt-codex-connector Bot Oct 18, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Conversation

youknowdot commented Oct 17, 2026

Summary

Validation

Scope limits

Uh oh!

coderabbitai Bot commented Oct 17, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions Bot commented Oct 17, 2026

📦 Library Dependencies

Uh oh!

Uh oh!

chatgpt-codex-connector Bot commented Oct 18, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codex Review Summary

Uh oh!

chatgpt-codex-connector Bot left a comment

Choose a reason for hiding this comment

💡 Codex Review

Uh oh!

chatgpt-codex-connector Bot Oct 18, 2026

Choose a reason for hiding this comment

Uh oh!

chatgpt-codex-connector Bot Oct 18, 2026

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

coderabbitai Bot commented Oct 17, 2026 •

edited

Loading

chatgpt-codex-connector Bot commented Oct 18, 2026 •

edited

Loading