marshal: round-trip code constants through the runtime bag - #8516
Conversation
Assisted-by: OpenAI Codex:gpt-5.5
Decode code fields through the runtime MarshalBag so co_consts values that do not fit the compiler constant enum remain available to VM code wrappers while the compiler table receives shape placeholders. Assisted-by: OpenAI Codex:GPT-5.4
serialize_code gains a serialize_code_with variant that writes each co_consts entry through a caller-supplied writer; serialize_code keeps the BorrowedConstant writer as its default. The VM writer passes its own write_object_depth, so a code constant that Literal holds but BorrowedConstant cannot describe reaches the stream instead of panicking in borrow_obj_constant, and a constant shared with the enclosing object takes an entry in the writer's reference table. PyMarshalBag implements constant_ref_from_value, bytes_from_value, str_from_value and tuple_elements_from_value, which deserialize_code_value_inner requires to decode code fields through the runtime bag. Assisted-by: Claude
|
Caution Review failedThe pull request is closed. ℹ️ Recent review info⚙️ Run configurationConfiguration used: Path: .coderabbit.yml Review profile: CHILL Plan: Pro Plus Run ID: ⛔ Files ignored due to path filters (1)
📒 Files selected for processing (3)
📝 WalkthroughWalkthroughMarshal now preserves runtime values in code constants during serialization and deserialization. Compiler and VM marshal layers share reference-table handling. Tests cover arbitrary constants and shared object identity. ChangesRuntime Constant Marshal
Estimated code review effort: 4 (Complex) | ~45 minutes Sequence Diagram(s)sequenceDiagram
participant PythonMarshal
participant VMMarshal
participant CompilerMarshal
participant ReferenceTable
PythonMarshal->>VMMarshal: serialize code object
VMMarshal->>CompilerMarshal: call serialize_code_with
CompilerMarshal->>VMMarshal: write each runtime constant
VMMarshal->>ReferenceTable: marshal constant with shared references
ReferenceTable-->>PythonMarshal: return serialized code data
✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
📦 Library DependenciesThe following Lib/ modules were modified. Here are their dependencies: [x] test: cpython/Lib/test/test_itertools.py (TODO: 5) dependencies: dependent tests: (56 tests)
[x] lib: cpython/Lib/code.py dependencies:
dependent tests: (2 tests) [x] test: cpython/Lib/test/test_structseq.py (TODO: 7) dependencies: dependent tests: (no tests depend on structseq) [x] test: cpython/Lib/test/test_marshal.py (TODO: 8) dependencies: dependent tests: (25 tests)
[ ] test: cpython/Lib/test/test_set.py (TODO: 4) dependencies: dependent tests: (no tests depend on set) Legend:
|
| self.assertIs(a[0], a) | ||
| # Direct self-reference which cannot be created in Python. | ||
| # This creates a reference loop which cannot be collected. | ||
| if False: |
marshal.loads(marshal.dumps(code))loses or rejects anyco_constsentry that the compiler constant enum cannot describe, because a code object's constants are read and written throughConstantBag/BorrowedConstantrather than through the runtime object bag.Reader
deserialize_code_value_innerdecodes the code fields through the runtimeMarshalBaginstead ofdeserialize_code_inner, soco_constsvalues survive as the objects they were, and the code fields share the single reference index space CPython'sr_object()uses.MarshalBaggainsmake_code_with_constants,code_constant_from_value,bytes_from_value,str_from_valueandtuple_elements_from_value; the compiler bags keep the previous behaviour through the defaults and the blanket impl.PyMarshalBagimplements those hooks.Literalwraps any object, so a decoded constant is already its own compiler-side constant andmake_code_with_constantskeeps the default.Writer
serialize_codegains aserialize_code_withvariant that writes eachco_constsentry through a caller-supplied writer;serialize_codepasses theBorrowedConstantwriter, so compiler callers are unchanged. The VM writer passes its ownwrite_object_depth, whichBorrowedConstantcannot describe, instead of panicking inborrow_obj_constant, andTests
extra_tests/snippets/stdlib_marshal.pygains a round trip over list/dict/set constants and an identity check for a constant shared with the enclosing tuple. Both pass on CPython 3.14 and on this branch (13/13).Lib/test/test_marshal.pypasses in a release build (75 run, 18 skipped). A debug build segfaults intest_loads_recursion, which nests 2**20 tuples and exhausts the native stack before the 2000 depth limit — unrelated to this change and present on both sides.🤖 Generated with Claude Code
Summary by CodeRabbit
New Features
Bug Fixes
Tests