Skip to content

Support marshal format 6 and frozendict serialization - #9130

Merged
youknowone merged 1 commit into
RustPython:mainfrom
youknowdot:split/marshal-v6-frozendict
Oct 15, 2026
Merged

Support marshal format 6 and frozendict serialization#9130
youknowone merged 1 commit into
RustPython:mainfrom
youknowdot:split/marshal-v6-frozendict

Conversation

@youknowone

@youknowone youknowone commented Oct 15, 2026 •
edited by coderabbitai Bot
Loading

Copy link
Copy Markdown
Member

Summary

Extract the native marshal format 6 / frozendict changes from #8954 against CPython v3.15.0 (d220409a13b77de7a6c30abe16ea90db01528edc).

  • Add the } frozendict tag and version 6 defaults; reject frozendict dumps using older formats.
  • Decode immutable mappings through a temporary dictionary, publishing their reserved reference only after construction. Reject recursive backreferences to incomplete immutable mappings.
  • Match final mapping termination at a null key or value.
  • Preserve current main's owned writer-reference entries, interned-string reference eligibility, and exact code-object fields / identity handling.

Only two Rust source files change. No Lib/ files or existing tests are modified.

Validation

  • Fresh narrow release build with threading, stdlib, stdio, importlib and host_env: passed after invalidating all workspace-package artifacts while retaining third-party caches.
  • Twelve unchanged canonical CPython final test methods: passed, zero skips. Each ran in a separate process with a 30-second / 1-GiB bound and private temporary directory. Coverage includes frozendict round trips and old-version rejection, small immutable reference-loop rejection, existing containers, code objects, allow_code and string interning.
  • Strict compiler-core / VM / stdlib all-target Clippy with compiler/threading/host_env and -D warnings: passed after workspace-wide debug artifact invalidation.
  • Normal configured commit hooks and git diff --check: passed.

An initial build reused stale common-crate artifacts from another worktree and failed visibility checks; all workspace artifacts were invalidated before the successful definitive build. No source workaround was added for that cache mismatch.

Canonical selectors were copied unchanged from the pinned final source into an external bounded runner. Full workspace/C-API suites, the full marshal suite, large-memory/deep-recursion/concurrency probes and other platforms were not run locally. The two-object reference-loop case is not a general GC safety claim. Hosted CI remains required.

AI-assisted implementation and validation with OpenAI Codex.

Summary by CodeRabbit

  • New Features
    • Marshal format version 6 adds support for serializing and deserializing frozen dictionaries.
    • The default format version for marshal.dump and marshal.dumps is now 6.
    • Frozen dictionaries are supported in version 6 and later; attempting to serialize unsupported frozen-dictionary objects raises a ValueError.

@youknowdot
Support marshal format 6 and frozendict serialization
4a9d177 
Extract the Python 3.15 final mapping format while preserving current code-object field identities and owned writer references.

Assisted-by: OpenAI Codex:model version unavailable
@coderabbitai

coderabbitai Bot commented Oct 15, 2026 •
edited
Loading

Copy link
Copy Markdown
Contributor

Review in Change Stack →

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration
  • Configuration used: Repository: RustPython/RustPython/.coderabbit.yml
  • Review profile: CHILL
  • Plan: Advanced
  • Run ID: 890adcae-d6f5-4592-b302-ef054de5112f
📥 Commits

Reviewing files that changed from the base of the PR and between 3ab5c0c and 4a9d177.

📒 Files selected for processing: 2
  • crates/compiler-core/src/marshal.rs
  • crates/vm/src/stdlib/marshal.rs

Included review availability: This review used your included allowance. Your plan provides up to 10 included reviews per hour; 9 remain after this review.

📝 Walkthrough

Marshal format version 6 adds frozen-dictionary encoding and decoding. The VM serializes and constructs Python frozen dictionaries, and dump and dumps now default to version 6.

Changes: Frozen dictionary marshal support

Layer / File(s) Summary
Marshal format and mapping handling
crates/compiler-core/src/marshal.rs		 The marshal type table recognizes frozen dictionaries. Decoding reads mapping entries and freezes completed mappings through MarshalBag. Encoding writes the frozen-dictionary marker.
Python frozen-dictionary integration
crates/vm/src/stdlib/marshal.rs		 Serialization handles Python dictionaries and frozen dictionaries through a shared path. Frozen-dictionary serialization requires version 6 or later and the exact built-in type. Decoding constructs a Python frozen dictionary. The dump and dumps defaults change to version 6.

Priority: ➖ Normal

Estimated code review effort: 3 (Moderate) | ~20 minutes

Change: Feature

Sequence Diagram(s)

sequenceDiagram
  participant MarshalDecoder
  participant PyMarshalBag
  participant PyFrozenDict
  MarshalDecoder->>PyMarshalBag: freeze_dict(decoded mapping)
  PyMarshalBag->>PyFrozenDict: construct frozen mapping
Loading
Merge Risk: ⚪ Minimal · up to 4a9d1

This change adds marshal format 6 and frozendict serialization. No concrete defects were identified in the supplied context. Hosted CI should still confirm the full marshal suite and other platforms before merge.

Pre-merge checks: Passed 4 · Failed 1

Failed Failed checks: 1 warning

Check name Status Explanation Resolution
Docstring Coverage Warning Docstring coverage is 0.00% which is insufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff. Analyzed 7 functions across 2 files. Write docstrings for the functions missing them to satisfy the coverage threshold.
Passed Passed checks: 4 passed
Check name Status Explanation
Description Check Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check Passed The title clearly and concisely describes the main changes: marshal format 6 support and frozendict serialization.
Linked Issues check Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check Passed Check skipped because no linked issues were found for this pull request.
  • Fix all pre-merge checks with AI
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create a new PR
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@youknowone
youknowone marked this pull request as ready for review October 15, 2026 03:10
@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Oct 15, 2026

Copy link
Copy Markdown

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ⚠️ Failed 2026-10-15T03:11:00.643605Z 4a9d177 Draft marked ready
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Oct 15, 2026

Copy link
Copy Markdown

You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard.

@youknowone
youknowone merged commit 79e7c9d into RustPython:main Oct 15, 2026
21 checks passed
@youknowdot youknowdot mentioned this pull request Oct 15, 2026
Draft
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@youknowone @youknowdot