Skip to content

Align mmap operations and capability reporting with Python 3.15 - #9141

Merged
youknowone merged 1 commit into
RustPython:mainfrom
youknowdot:split/mmap-final
Oct 15, 2026
Merged

youknowone merged 1 commit into
RustPython:mainfrom
youknowdot:split/mmap-final

Conversation

@youknowdot

Copy link
Copy Markdown
Contributor

Summary

  • Align native mmap flush flags, default ranges, naming validation, descriptor tracking, and Linux resize behavior with CPython 3.15.0.
  • Inspect raw descriptors through checked syscalls without manufacturing a borrowed Rust descriptor from arbitrary Python integers. Restrict file-length checks to successful regular-file metadata; duplicate descriptors only for owned tracking.
  • Preserve the existing buffer-export locking and mutation/lifetime fixes from Synchronize mmap buffer exports with close and resize #9015 and Validate mmap assignments against current storage #9071. Keep the lock across supported resize operations.
  • Expose resize only where the host implementation exists (Linux and Windows), instead of exposing a method that always raises SystemError on other Unix targets. CPython final omits resize when MS_WINDOWS || HAVE_MREMAP is false; this matches Darwin's unsupported capability without adding test skips.
  • Update Lib/test/test_mmap.py using actual scripts/update_lib migrate against CPython v3.15.0 (d220409a13b77de7a6c30abe16ea90db01528edc), preserving existing RustPython guards. No hand-edited Lib changes or new snippets.

This is a three-file host/native/canonical-test batch. It does not adopt the separate storage/GC redesign in #8939.

Validation

  • Normal commit hooks passed; source hashes and clean head verified before/after validation.
  • Matching narrow Linux release build passed (offline/locked, defaults disabled, threading,stdlib,stdio,importlib,host_env, one build job).
  • Strict release VM/stdlib all-target Clippy passed with the established explicit feature set and -D warnings.
  • Six unchanged canonical tests plus their unchanged setup/teardown and decorators passed on both the matching RustPython binary and pinned CPython 3.15.0 reference: test_trackfd_parameter, test_trackfd_neg1, test_bad_file_desc, test_flush_parameters, test_resize, and test_set_name. No skips or expected failures occurred.
  • Each selected case used a fresh private working/temp directory, verified source/binary/native/support origins and cache-free standard-library-only paths, -S -B, assertions enabled, C locale, a 30-second timeout plus 5-second kill grace, 1-GiB address-space limit, and 256-KiB combined output cap. Maximum selected mapping was three pages, with page size capped at 64 KiB.

Coverage limits

Darwin method availability is source-reviewed, not locally runtime-verified. Windows runtime and cross-target compilation were not performed. Android/NetBSD native remapping backends remain unsupported here; this is not a full cross-platform parity claim. Existing anonymous-map flags/protection limitations and the conservative shared-anonymous growth restriction remain.

The naming test ran in normal release mode, not development mode, and therefore does not establish actual kernel VMA naming. No zero-size resize, forced remap failure, post-failure memory access, reentrancy/stress, full mmap suite, full Rust workspace, or CAPI test run was performed. In particular, truncation followed by failed remapping retains its existing hazard and is not claimed solved by this PR.

The reference's initial isolated support import stopped before tests because generated sysconfig data was absent; rerunning with a hash-verified copy of the actual build-generated literal configuration passed all six cases. The reference build retains its documented out-of-tree -dirty banner metadata caveat; pinned source and library provenance were verified.

Implementation and validation were assisted by OpenAI Codex.

Separate raw descriptor inspection from owned tracking, update flush and naming behavior, and expose resize only on implemented backends. Migrate canonical mmap tests using update_lib while preserving existing RustPython guards.

Assisted-by: OpenAI Codex:model version unavailable
@coderabbitai

coderabbitai Bot commented Oct 15, 2026 •

Copy link
Copy Markdown
Contributor

Review in Change Stack →

Note

Currently processing new changes in this PR. This may take a few minutes, please wait...

⚙️ Run configuration
  • Configuration used: Repository: RustPython/RustPython/.coderabbit.yml
  • Review profile: CHILL
  • Plan: Advanced
  • Run ID: 2d402361-c131-484c-bf14-7b73bda2e4a3
📥 Commits

Reviewing files that changed from the base of the PR and between 498adff and 9df6528.

⛔ Files ignored due to path filters: 1
  • Lib/test/test_mmap.py is excluded by !Lib/**
📒 Files selected for processing: 2
  • crates/host_env/src/mmap.rs
  • crates/stdlib/src/mmap.rs
+---------------------------------------------+
| Nose to the grindstone, eyes on the screen. |
+---------------------------------------------+
  \
   \   (\__/)
       (•ㅅ•)
       /   づ
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create a new PR
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@github-actions

Copy link
Copy Markdown
Contributor

📦 Library Dependencies

The following Lib/ modules were modified. Here are their dependencies:

:4: SyntaxWarning: invalid decimal literal
:7: SyntaxWarning: "\z" is an invalid escape sequence. Such sequences will not work in the future. Did you mean "\z"? A raw string is also an option.
:10: SyntaxWarning: "\400" is an invalid octal escape sequence. Such sequences will not work in the future. Did you mean "\400"? A raw string is also an option.
:4: SyntaxWarning: invalid decimal literal
:7: SyntaxWarning: "\z" is an invalid escape sequence. Such sequences will not work in the future. Did you mean "\z"? A raw string is also an option.
:10: SyntaxWarning: "\400" is an invalid octal escape sequence. Such sequences will not work in the future. Did you mean "\400"? A raw string is also an option.
:4: SyntaxWarning: invalid decimal literal
:7: SyntaxWarning: "\z" is an invalid escape sequence. Such sequences will not work in the future. Did you mean "\z"? A raw string is also an option.
:10: SyntaxWarning: "\400" is an invalid octal escape sequence. Such sequences will not work in the future. Did you mean "\400"? A raw string is also an option.
[ ] lib: cpython/Lib/pprint.py
[ ] test: cpython/Lib/test/test_pprint.py

dependencies:

  • pprint (native: sys)
    • collections (native: _collections, _weakref, itertools, sys)
    • dataclasses (native: itertools, sys)
    • io (native: _io, _thread, errno, msvcrt, sys)
    • re (native: _casefix, _constants, _sre, sys, unicodedata)
    • types (native: _types, sys)

dependent tests: (97 tests)

  • pprint: test_htmlparser test_pprint test_ssl test_sys_setprofile test_unittest
    • pdb: test_pdb
    • pickle: test_annotationlib test_argparse test_array test_ast test_asyncio test_bool test_builtin test_bytes test_bz2 test_codecs test_collections test_concurrent_futures test_configparser test_coroutines test_csv test_ctypes test_decimal test_defaultdict test_deque test_descr test_dict test_dictviews test_email test_enum test_enumerate test_exceptions test_fractions test_functools test_generators test_genericalias test_http_cookies test_importlib test_inspect test_io test_ipaddress test_iter test_itertools test_json test_list test_logging test_lzma test_memoryio test_memoryview test_minidom test_opcache test_operator test_ordered_dict test_os test_pathlib test_pickle test_picklebuffer test_pickletools test_platform test_plistlib test_positional_only_arg test_posix test_random test_range test_re test_set test_shelve test_slice test_socket test_statistics test_str test_string test_structseq test_super test_time test_trace test_tuple test_turtle test_type_aliases test_type_params test_types test_typing test_unittest test_uuid test_xml_dom_minicompat test_xml_etree test_xpickle test_zipfile test_zlib test_zoneinfo
      • logging: test_asyncio test_concurrent_futures test_hashlib test_pkgutil test_support test_urllib2net
      • tracemalloc: test_tracemalloc

[ ] lib: cpython/Lib/json
[ ] test: cpython/Lib/test/test_json (TODO: 2)

dependencies:

  • json (native: _json, decoder, encoder, json.tool, sys)
    • _colorize (native: builtins, collections.abc, nt, sys)
    • argparse (native: sys)
    • re
    • codecs, warnings

dependent tests: (14 tests)

  • json: test_embed test_json test_logging test_plistlib test_pyrepl test_subprocess test_sysconfig test_tomllib test_tools test_traceback test_zoneinfo
    • importlib.metadata: test_importlib
    • multiprocessing.resource_tracker: test_concurrent_futures
    • pdb: test_pdb

[x] test: cpython/Lib/test/test_mmap.py (TODO: 19)

dependencies:

dependent tests: (2 tests)

  • mmap: test_os
    • multiprocessing.shared_memory: test_genericalias

Legend:

  • [+] path exists in CPython
  • [x] up-to-date, [ ] outdated

@codspeed

codspeed Bot commented Oct 15, 2026

Copy link
Copy Markdown
Contributor

Merging this PR will not alter performance

✅ 62 untouched benchmarks
⏩ 4 skipped benchmarks1


Comparing youknowdot:split/mmap-final (9df6528) with main (498adff)

Open in CodSpeed

Footnotes

  1. 4 benchmarks were skipped, so the baseline results were used instead. If they were deleted from the codebase, click here and archive them to remove them from the performance reports. ↩

@youknowone
youknowone marked this pull request as ready for review October 15, 2026 10:53
@youknowone
youknowone merged commit 0384ddc into RustPython:main Oct 15, 2026
32 checks passed
@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Oct 15, 2026 •

Copy link
Copy Markdown

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ✅ Completed 2026-10-15T11:07:40.878483Z 9df6528 Draft marked ready
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 9df652854d

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread Lib/test/test_mmap.py
# Try to seek to negative position...
self.assertRaises(ValueError, m.seek, -len(m)-1, 2)

@unittest.skipUnless(hasattr(mmap.mmap, 'resize'), 'requires mmap.resize')

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Badge Replace the resize skip with an expected failure

On Darwin and other targets where mmap.resize is intentionally absent, this decorator suppresses the test for an ordinary missing-feature failure; running the body would only raise an exception, not terminate or hang the process. Use the repository's conditional expected-failure marker so a later implementation is reported as an unexpected pass, since skips are reserved for process-killing cases.

AGENTS.md reference: AGENTS.md:L98-L101

Useful? React with 👍 / 👎.

Comment thread Lib/test/test_mmap.py
if close_original_fd:
f.close()
self.assertEqual(len(m), size)
with self.assertRaises(ValueError):

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Badge Restore the original trackfd assertion

In the trackfd=False case this changes the test oracle from OSError/EBADF to ValueError, rather than changing only an allowed expected-failure marker. The repository explicitly forbids modifying test assertions, logic, or data, so keep the test body unchanged and make the Rust implementation satisfy the existing assertion instead.

AGENTS.md reference: AGENTS.md:L273-L280

Useful? React with 👍 / 👎.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants