Skip to content

gh-145261, gh-106939: multiprocessing.shared_memory: fix ShareableList corruption for multi-byte strings and null bytes #145266

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
zetzschest wants to merge 8 commits into
base: main
Choose a base branch
from
Open

gh-145261, gh-106939: multiprocessing.shared_memory: fix ShareableList corruption for multi-byte strings and null bytes #145266

Changes from 1 commit
Commits
Show all changes
8 commits
Select commit Hold shift + click to select a range
6261c86
gh-145261: Fix ShareableList corruption for multi-byte strings and by…
stefanzetzsche Feb 26, 2026
34003c6
📜🤖 Added by blurb_it.
blurb-it[bot] Feb 26, 2026
3f874bc
docs: Update ShareableList documentation for fixed trailing null bug
stefanzetzsche Feb 27, 2026
254230f
fix: Also preserve trailing nulls in strings, not just bytes
stefanzetzsche Feb 27, 2026
2c0935c
test: Update ShareableList tests to reflect actual byte lengths
stefanzetzsche Feb 27, 2026
eb4ce8a
docs: Update ShareableList documentation to reflect trailing null pre…
stefanzetzsche Feb 27, 2026
e3b5fd3
Merge remote-tracking branch 'origin/main' into fix/multiprocessing_s…
stefanzetzsche Mar 11, 2026
feba9b5
test: Add tests for trailing null preservation and multi-byte UTF-8 i…
stefanzetzsche Mar 11, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
test: Add tests for trailing null preservation and multi-byte UTF-8 i… 
…n ShareableList

Add test_shared_memory_ShareableList_trailing_nulls covering gh-106939
(bytes/str values with trailing null bytes preserved correctly) and
test_shared_memory_ShareableList_multibyte_utf8 covering gh-145261
(multi-byte UTF-8 strings stored and retrieved without corruption).
Both tests include cross-process verification via name-based attachment.
  • Loading branch information
@stefanzetzsche
stefanzetzsche committed Mar 11, 2026
commit feba9b5be41f876c838f2814df2d89e79b9287bf
52 changes: 52 additions & 0 deletions Lib/test/_test_multiprocessing.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -4899,6 +4899,58 @@ def test_shared_memory_ShareableList_pickling_dead_object(self):
with self.assertRaises(FileNotFoundError):
pickle.loads(serialized_sl)

def test_shared_memory_ShareableList_trailing_nulls(self):
# gh-106939: ShareableList should preserve trailing null bytes
# in bytes and str values.
sl = shared_memory.ShareableList([
b'\x03\x02\x01\x00\x00\x00',
'?\x00',
b'\x00\x00\x00',
b'',
b'no nulls',
])
self.addCleanup(sl.shm.unlink)
self.addCleanup(sl.shm.close)

self.assertEqual(sl[0], b'\x03\x02\x01\x00\x00\x00')
self.assertEqual(sl[1], '?\x00')
self.assertEqual(sl[2], b'\x00\x00\x00')
self.assertEqual(sl[3], b'')
self.assertEqual(sl[4], b'no nulls')

sl2 = shared_memory.ShareableList(name=sl.shm.name)
self.addCleanup(sl2.shm.close)
self.assertEqual(sl2[0], b'\x03\x02\x01\x00\x00\x00')
self.assertEqual(sl2[1], '?\x00')
self.assertEqual(sl2[2], b'\x00\x00\x00')
self.assertEqual(sl2[3], b'')
self.assertEqual(sl2[4], b'no nulls')

def test_shared_memory_ShareableList_multibyte_utf8(self):
# gh-145261: ShareableList should correctly handle multi-byte
# UTF-8 strings without corruption or spillage.
sl = shared_memory.ShareableList([
'ascii', # 1-byte per char (5 bytes)
'café', # 2-byte char: é (5 bytes)
'中文测试', # 3-byte per char (12 bytes)
'𐀀𐀁', # 4-byte per char (8 bytes)
])
self.addCleanup(sl.shm.unlink)
self.addCleanup(sl.shm.close)

self.assertEqual(sl[0], 'ascii')
self.assertEqual(sl[1], 'café')
self.assertEqual(sl[2], '中文测试')
self.assertEqual(sl[3], '𐀀𐀁')

# Verify cross-process access via name-based attachment.
sl2 = shared_memory.ShareableList(name=sl.shm.name)
self.addCleanup(sl2.shm.close)
self.assertEqual(sl2[0], 'ascii')
self.assertEqual(sl2[1], 'café')
self.assertEqual(sl2[2], '中文测试')
self.assertEqual(sl2[3], '𐀀𐀁')

def test_shared_memory_cleaned_after_process_termination(self):
cmd = '''if 1:
import os, time, sys
Expand Down
Loading