Skip to content

Update Unicode name preparation and codecs to Python 3.15 - #9132

Merged
youknowone merged 1 commit into
RustPython:mainfrom
youknowdot:split/unicode-codecs-315
Oct 15, 2026
Merged

Update Unicode name preparation and codecs to Python 3.15#9132
youknowone merged 1 commit into
RustPython:mainfrom
youknowdot:split/unicode-codecs-315

Conversation

@youknowdot

@youknowdot youknowdot commented Oct 15, 2026

Copy link
Copy Markdown
Contributor

Summary

Continue the Python 3.15 library split from #8954 with Unicode name preparation and its related codecs.

  • Refresh stringprep from CPython 3.15.0, including the Unicode 17 compatibility mappings while retaining Unicode 3.2 normalization/category semantics required by RFC 3454.
  • Refresh IDNA and punycode to use the already-landed bytearray.take_bytes() API, and sync the UU codec cleanup.
  • Import the canonical new IDNA case-folding cases. Preserve the existing Windows test_cp_utf7 expected-failure marker unchanged.

All five files were imported using the repository's actual scripts/update_lib migrate command from CPython commit d220409a13b77de7a6c30abe16ea90db01528edc (v3.15.0). The four runtime files are byte-identical to that source; the test file differs only by its pre-existing RustPython marker. No hand-edited library compatibility patch or new test waiver is included.

Validation

  • Independent source/provenance review of the complete five-file diff and selected-test runner.
  • Fresh release build on main 3ab5c0c2f207e335972cd16b776b9b522ffe7e0f, after cleaning all workspace release packages and parser/XML dependencies to avoid cross-worktree cache reuse.
  • 18 unchanged canonical test methods passed: StringprepTests, PunycodeTest, NameprepTest, IDNACodecTest, and the UU invalid-input method. Each ran in a separate process with a 30-second timeout and 1 GiB memory limit, asserting candidate library origins. Zero unittest skips/expected failures; Nameprep retains its two upstream None fixture placeholders.
  • Native and CPython-final Unicode version checks: 17.0.0 and legacy 3.2.0.
  • Normal commit hooks and git diff --check passed.

No full codec/big-memory suite, full Rust workspace suite, or C-API suite was run locally. Hosted CI remains required.

AI assistance: OpenAI Codex (model version unavailable).

@youknowdot
Update Unicode name preparation and codecs to Python 3.15
1921aa2 
Import stringprep and related codecs from CPython 3.15.0 using scripts/update_lib. Preserve the existing Windows codec test marker.

Assisted-by: OpenAI Codex:model version unavailable
@coderabbitai

coderabbitai Bot commented Oct 15, 2026 •
edited
Loading

Copy link
Copy Markdown
Contributor

Important

Review skipped

Review was skipped due to path filters

⛔ Files ignored due to path filters: 5
  • Lib/encodings/idna.py is excluded by !Lib/**
  • Lib/encodings/punycode.py is excluded by !Lib/**
  • Lib/encodings/uu_codec.py is excluded by !Lib/**
  • Lib/stringprep.py is excluded by !Lib/**
  • Lib/test/test_codecs.py is excluded by !Lib/**

CodeRabbit blocks several paths by default. You can override this behavior by explicitly including those paths in the path filters. For example, including **/dist/** will override the default block on the dist directory, by removing the pattern from both the lists.

⚙️ Run configuration
  • Configuration used: Repository: RustPython/RustPython/.coderabbit.yml
  • Review profile: CHILL
  • Plan: Advanced
  • Run ID: ca50a687-d9ad-49be-9bfe-f1e9105d474c

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

  • 🔍 Trigger review
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@github-actions

github-actions Bot commented Oct 15, 2026

Copy link
Copy Markdown
Contributor

📦 Library Dependencies

The following Lib/ modules were modified. Here are their dependencies:

:4: SyntaxWarning: invalid decimal literal
:7: SyntaxWarning: "\z" is an invalid escape sequence. Such sequences will not work in the future. Did you mean "\z"? A raw string is also an option.
:10: SyntaxWarning: "\400" is an invalid octal escape sequence. Such sequences will not work in the future. Did you mean "\400"? A raw string is also an option.
:4: SyntaxWarning: invalid decimal literal
:7: SyntaxWarning: "\z" is an invalid escape sequence. Such sequences will not work in the future. Did you mean "\z"? A raw string is also an option.
:10: SyntaxWarning: "\400" is an invalid octal escape sequence. Such sequences will not work in the future. Did you mean "\400"? A raw string is also an option.
:4: SyntaxWarning: invalid decimal literal
:7: SyntaxWarning: "\z" is an invalid escape sequence. Such sequences will not work in the future. Did you mean "\z"? A raw string is also an option.
:10: SyntaxWarning: "\400" is an invalid octal escape sequence. Such sequences will not work in the future. Did you mean "\400"? A raw string is also an option.
[x] lib: cpython/Lib/codecs.py
[x] test: cpython/Lib/test/test_charmapcodec.py
[ ] test: cpython/Lib/test/test_codeccallbacks.py (TODO: 3)
[x] test: cpython/Lib/test/test_codecencodings_cn.py
[x] test: cpython/Lib/test/test_codecencodings_hk.py
[x] test: cpython/Lib/test/test_codecencodings_iso2022.py
[x] test: cpython/Lib/test/test_codecencodings_jp.py
[x] test: cpython/Lib/test/test_codecencodings_kr.py
[x] test: cpython/Lib/test/test_codecencodings_tw.py
[x] test: cpython/Lib/test/test_codecmaps_cn.py
[x] test: cpython/Lib/test/test_codecmaps_hk.py
[x] test: cpython/Lib/test/test_codecmaps_jp.py
[x] test: cpython/Lib/test/test_codecmaps_kr.py
[x] test: cpython/Lib/test/test_codecmaps_tw.py
[x] test: cpython/Lib/test/test_codecs.py (TODO: 2)
[x] test: cpython/Lib/test/test_multibytecodec.py
[x] test: cpython/Lib/test/testcodec.py

dependencies:

  • codecs

dependent tests: (175 tests)

  • codecs: test_charmapcodec test_codeccallbacks test_codecs test_eof test_exceptions test_importlib test_inspect test_io test_json test_locale test_logging test_multibytecodec test_os test_pdb test_plistlib test_sax test_str test_sys
    • email.charset: test_email
    • email.utils: test_httpservers test_smtplib test_urllib2
      • http.server: test_robotparser test_urllib2_localnet test_xmlrpc
      • logging.handlers: test_concurrent_futures test_pkgutil
      • smtplib: test_smtpnet
      • urllib.request: test_http_cookiejar test_pathlib test_pydoc test_site test_ssl test_urllib test_urllib2net test_urllibnet
    • encodings: test_email
      • locale: test__locale test_builtin test_c_locale_coercion test_calendar test_decimal test_float test_format test_re test_regrtest test_strftime test_strptime test_types test_utf8_mode
    • json: test_embed test_pyrepl test_subprocess test_sysconfig test_tomllib test_tools test_traceback test_zoneinfo
      • importlib.metadata: test_importlib
    • pickle: test_annotationlib test_argparse test_array test_ast test_asyncio test_bool test_bytes test_bz2 test_collections test_concurrent_futures test_configparser test_coroutines test_csv test_ctypes test_defaultdict test_deque test_descr test_dict test_dictviews test_email test_enum test_enumerate test_fractions test_functools test_generators test_genericalias test_http_cookies test_ipaddress test_iter test_itertools test_list test_lzma test_memoryio test_memoryview test_minidom test_opcache test_operator test_ordered_dict test_pickle test_picklebuffer test_pickletools test_platform test_positional_only_arg test_posix test_random test_range test_set test_shelve test_slice test_socket test_statistics test_string test_structseq test_super test_time test_trace test_tuple test_turtle test_type_aliases test_type_params test_typing test_unittest test_uuid test_xml_dom_minicompat test_xml_etree test_xpickle test_zipfile test_zlib test_zoneinfo
      • tracemalloc: test_tracemalloc
    • plistlib:
      • platform: test__osx_support test_asyncio test_baseexception test_cmath test_ctypes test_fcntl test_math test_mimetypes test_shutil test_winreg test_wsgiref
    • tokenize: test_linecache test_peg_generator test_tabnanny test_tokenize test_unparse
      • inspect: test_abc test_asyncgen test_buffer test_clinic test_code test_grammar test_monitoring test_ntpath test_patma test_posixpath test_signal test_sqlite3 test_type_annotations test_yield_from test_zipimport test_zipimport_support
      • linecache: test_bdb
      • traceback: test_asyncio test_code_module test_contextlib test_contextlib_async test_dictcomps test_importlib test_listcomps test_pyexpat test_setcomps test_threadedtempfile test_threading test_unittest test_with

[x] lib: cpython/Lib/stringprep.py
[x] test: cpython/Lib/test/test_stringprep.py

dependencies:

  • stringprep

dependent tests: (3 tests)

  • stringprep: test_stringprep
    • encodings.idna: test_codecs test_pydoc

[x] lib: cpython/Lib/encodings

dependencies:

  • encodings

dependent tests: (164 tests)

  • encodings: test_codecs test_email test_pydoc test_sys
    • codecs: test_charmapcodec test_codeccallbacks test_eof test_exceptions test_importlib test_inspect test_io test_json test_locale test_logging test_multibytecodec test_os test_pdb test_plistlib test_sax test_str
      • email.charset: test_email
      • email.utils: test_httpservers test_smtplib test_urllib2
      • json: test_embed test_pyrepl test_subprocess test_sysconfig test_tomllib test_tools test_traceback test_zoneinfo
      • pickle: test_annotationlib test_argparse test_array test_ast test_asyncio test_bool test_builtin test_bytes test_bz2 test_collections test_concurrent_futures test_configparser test_coroutines test_csv test_ctypes test_decimal test_defaultdict test_deque test_descr test_dict test_dictviews test_email test_enum test_enumerate test_fractions test_functools test_generators test_genericalias test_http_cookies test_importlib test_ipaddress test_iter test_itertools test_list test_lzma test_memoryio test_memoryview test_minidom test_opcache test_operator test_ordered_dict test_pathlib test_pickle test_picklebuffer test_pickletools test_platform test_positional_only_arg test_posix test_random test_range test_re test_set test_shelve test_slice test_socket test_statistics test_string test_structseq test_super test_time test_trace test_tuple test_turtle test_type_aliases test_type_params test_types test_typing test_unittest test_uuid test_xml_dom_minicompat test_xml_etree test_xpickle test_zipfile test_zlib test_zoneinfo
      • tokenize: test_linecache test_peg_generator test_tabnanny test_tokenize test_unparse
    • locale: test__locale test_c_locale_coercion test_calendar test_float test_format test_regrtest test_strftime test_strptime test_utf8_mode
      • calendar: test_imaplib
      • gettext: test_gettext
      • site: test_site
      • subprocess: test_android test_asyncio test_atexit test_audit test_cmd_line test_cmd_line_script test_concurrent_futures test_ctypes test_dtrace test_external_inspection test_faulthandler test_file_eintr test_gc test_gzip test_json test_launcher test_msvcrt test_ntpath test_osx_env test_peg_generator test_poll test_py_compile test_pyrepl test_quopri test_repl test_runpy test_script_helper test_select test_shutil test_signal test_sqlite3 test_support test_sys_settrace test_tempfile test_threading test_unittest test_venv test_wait3 test_webbrowser test_zipfile

Legend:

  • [+] path exists in CPython
  • [x] up-to-date, [ ] outdated

@codspeed

codspeed Bot commented Oct 15, 2026

Copy link
Copy Markdown
Contributor

Merging this PR will not alter performance

✅ 62 untouched benchmarks
⏩ 4 skipped benchmarks1

Comparing youknowdot:split/unicode-codecs-315 (1921aa2) with main (3ab5c0c)

Open in CodSpeed

Footnotes

  1. 4 benchmarks were skipped, so the baseline results were used instead. If they were deleted from the codebase, click here and archive them to remove them from the performance reports. ↩

@youknowone
youknowone marked this pull request as ready for review October 15, 2026 06:05
@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Oct 15, 2026

Copy link
Copy Markdown

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ⚠️ Failed 2026-10-15T06:05:53.994856Z 1921aa2 Draft marked ready
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Oct 15, 2026

Copy link
Copy Markdown

You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard.

@youknowone
youknowone merged commit 4d54126 into RustPython:main Oct 15, 2026
22 of 23 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@youknowdot @youknowone