Skip to content
Merged
Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
45 commits
Select commit Hold shift + click to select a range
05ae5ad
Add Ascii85, base85, and Z85 support to binascii
kangtastic Mar 8, 2023
aa06c5d
Restore base64.py
kangtastic Apr 26, 2025
6377440
Create _base64 module with wrappers for accelerated functions
kangtastic Apr 26, 2025
6c0e4a3
Test both Python and C codepaths in base64
kangtastic Apr 26, 2025
ce4773c
Match behavior between Python and C base 85 functions
kangtastic Apr 26, 2025
4072e3b
Add Z85 tests to binascii
kangtastic Apr 27, 2025
bc9217f
Update generated files
kangtastic Apr 27, 2025
2c40ba0
Avoid importing functools
kangtastic Apr 28, 2025
fd9eaf7
Avoid circular import in _base64
kangtastic Apr 28, 2025
4746d18
Do not use a decorator for changing exception type
kangtastic Apr 28, 2025
d075593
Test Python and C codepaths in base64 using mixins
kangtastic Apr 28, 2025
6d65fec
Remove leading underscore from functions in private module
kangtastic Apr 29, 2025
a241356
Merge branch 'main' into gh-101178-rework-base85
serhiy-storchaka Dec 24, 2025
0df9a40
Use more modern C API.
serhiy-storchaka Dec 24, 2025
60fbd7c
Fix tests.
serhiy-storchaka Dec 24, 2025
a070887
Merge branch 'main' into gh-101178-rework-base85
serhiy-storchaka Dec 25, 2025
167e83e
Fix new tests.
serhiy-storchaka Dec 25, 2025
01df442
Optimize binascii.b2a_ascii85().
serhiy-storchaka Dec 26, 2025
7885918
Apply suggestions from code review
serhiy-storchaka Dec 27, 2025
1e928e3
Update C style to more closely adhere to PEP-7
kangtastic Dec 28, 2025
2691a0a
Remove pure-Python base-85-related codepaths in base64
kangtastic Dec 28, 2025
b9d27bd
Remove now-unnecessary _base64 module and fix tests
kangtastic Dec 28, 2025
780517a
Separate Z85 from Base85 on the Python API side
kangtastic Dec 28, 2025
bc9a66d
Fix tests after separating Base85 from Z85
kangtastic Dec 28, 2025
dc1d3fc
Merge branch 'main' into gh-101178-rework-base85
kangtastic Dec 28, 2025
c5de5a1
Update generated files after merging main
kangtastic Dec 28, 2025
3bb3b18
Update Misc/NEWS.d and Misc/ACKS
kangtastic Dec 28, 2025
6f09fa8
Update generated files again
kangtastic Dec 29, 2025
6d8f897
Fix typo in NEWS entry
kangtastic Dec 29, 2025
3582492
Merge branch 'main' into gh-101178-rework-base85
serhiy-storchaka Jan 14, 2026
879dd86
Move the NEWS entry to the correct section.
serhiy-storchaka Jan 14, 2026
3cdc3c5
Minor cleanups, align lookup tables to 64 bytes (NFC)
kangtastic Jan 17, 2026
8adaf2c
Allow up to sys.maxsize output length when encoding base 85
kangtastic Jan 18, 2026
2b2ecc4
Fix Ascii85 test from mainline
kangtastic Jan 18, 2026
da165d1
Allow up to sys.maxsize output length when decoding base 85
kangtastic Jan 18, 2026
bf32f99
Defer base 85 overflow check during decoding
kangtastic Jan 18, 2026
74f6ceb
Merge branch 'main' into gh-101178-rework-base85
kangtastic Jan 18, 2026
4ba3e50
Merge branch 'main' into gh-101178-rework-base85
serhiy-storchaka Feb 6, 2026
99e0bde
Rename parameters to match the base64 module.
serhiy-storchaka Feb 6, 2026
cc6d485
Remove parameters strict_mode and newline.
serhiy-storchaka Feb 6, 2026
30f54a1
Optimize ignorechars cache.
serhiy-storchaka Feb 6, 2026
37df735
Harmonize documentation.
serhiy-storchaka Feb 6, 2026
56a02b2
Add What's New entries.
serhiy-storchaka Feb 6, 2026
adb1922
Polish tests.
serhiy-storchaka Feb 6, 2026
0730fdf
Rename internal Base 85 codec functions to match Base 64 helpers
kangtastic Feb 6, 2026
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
Remove parameters strict_mode and newline.
  • Loading branch information
serhiy-storchaka committed Feb 6, 2026
commit cc6d485e4c2787df094371877ace351ec8479afb
16 changes: 4 additions & 12 deletions Doc/library/binascii.rst
Original file line number Diff line number Diff line change
Expand Up @@ -145,14 +145,12 @@ The :mod:`binascii` module defines the following functions:
.. versionadded:: next


.. function:: a2b_base85(string, /, *, strict_mode=False)
.. function:: a2b_base85(string, /)

Convert Base85 data back to binary and return the binary data.
More than one line may be passed at a time.

If *strict_mode* is true, only valid Base85 data will be converted.
Invalid Base85 data will raise :exc:`binascii.Error`.

Valid Base85 data contains characters from the Base85 alphabet in groups
of five (except for the final group, which may have from two to five
characters). Each group encodes 32 bits of binary data in the range from
Expand All @@ -161,26 +159,22 @@ The :mod:`binascii` module defines the following functions:
.. versionadded:: next


.. function:: b2a_base85(data, /, *, pad=False, newline=True)
.. function:: b2a_base85(data, /, *, pad=False)

Convert binary data to a line of ASCII characters in Base85 coding.
The return value is the converted line.

If *pad* is true, the input is padded to a multiple of 4 before encoding.

If *newline* is true, a newline char is appended to the result.

.. versionadded:: next


.. function:: a2b_z85(string, /, *, strict_mode=False)
.. function:: a2b_z85(string, /)

Convert Z85 data back to binary and return the binary data.
More than one line may be passed at a time.

If *strict_mode* is true, only valid Z85 data will be converted.
Invalid Z85 data will raise :exc:`binascii.Error`.

Valid Z85 data contains characters from the Z85 alphabet in groups
of five (except for the final group, which may have from two to five
characters). Each group encodes 32 bits of binary data in the range from
Expand All @@ -191,15 +185,13 @@ The :mod:`binascii` module defines the following functions:
.. versionadded:: next


.. function:: b2a_z85(data, /, *, pad=False, newline=True)
.. function:: b2a_z85(data, /, *, pad=False)

Convert binary data to a line of ASCII characters in Z85 coding.
The return value is the converted line.

If *pad* is true, the input is padded to a multiple of 4 before encoding.

If *newline* is true, a newline char is appended to the result.

See `Z85 specification <https://rfc.zeromq.org/spec/32/>`_ for more information.
Comment thread
serhiy-storchaka marked this conversation as resolved.

.. versionadded:: next
Expand Down
8 changes: 4 additions & 4 deletions Lib/base64.py
Original file line number Diff line number Diff line change
Expand Up @@ -382,25 +382,25 @@ def b85encode(b, pad=False):
If pad is true, the input is padded with b'\\0' so its length is a multiple of
4 bytes before encoding.
"""
return binascii.b2a_base85(b, pad=pad, newline=False)
return binascii.b2a_base85(b, pad=pad)

def b85decode(b):
"""Decode the base85-encoded bytes-like object or ASCII string b

The result is returned as a bytes object.
"""
return binascii.a2b_base85(b, strict_mode=True)
return binascii.a2b_base85(b)

def z85encode(s, pad=False):
"""Encode bytes-like object b in z85 format and return a bytes object."""
return binascii.b2a_z85(s, pad=pad, newline=False)
return binascii.b2a_z85(s, pad=pad)

def z85decode(s):
"""Decode the z85-encoded bytes-like object or ASCII string b

The result is returned as a bytes object.
"""
return binascii.a2b_z85(s, strict_mode=True)
return binascii.a2b_z85(s)

# Legacy interface. This code could be cleaned up since I don't believe
# binascii has any line length limitations. It just doesn't seem worth it
Expand Down
144 changes: 30 additions & 114 deletions Lib/test/test_binascii.py
Original file line number Diff line number Diff line change
Expand Up @@ -536,49 +536,30 @@ def test_base85_valid(self):
self.assertEqual(res, self.rawdata)

# Test decoding inputs with length 1 mod 5
self.assertEqual(binascii.a2b_base85(self.type2test(b"a")), b"")
self.assertEqual(binascii.a2b_base85(self.type2test(b" b ")), b"")
self.assertEqual(binascii.a2b_base85(self.type2test(b"b/Y\"*,j'Nc")), b"test")

def test_base85_invalid(self):
# Test base85 with invalid characters interleaved
lines, i = [], 0
for k in range(1, len(self.rawdata) + 1):
b = self.type2test(self.rawdata[i:i + k])
a = binascii.b2a_base85(b)
lines.append(a)
i += k
if i >= len(self.rawdata):
break

fillers = bytearray()
valid = b"0123456789ABCDEFGHIJKLMNOPQRSTUVWXYZ" \
b"abcdefghijklmnopqrstuvwxyz!#$%&()*+-;<=>?@^_`{|}~"
for i in range(256):
if i not in valid:
fillers.append(i)
def addnoise(line):
res = bytearray()
for i in range(len(line)):
res.append(line[i])
for j in range(i, len(fillers), len(line)):
res.append(fillers[j])
return res
res = bytearray()
for line in map(addnoise, lines):
a = self.type2test(line)
b = binascii.a2b_base85(a)
res += b
self.assertEqual(res, self.rawdata)
self.assertEqual(binascii.a2b_base85(self.type2test(b'a')), b'')
self.assertEqual(binascii.a2b_base85(self.type2test(b'a')), b'')
self.assertEqual(binascii.a2b_base85(self.type2test(b'ab')), b'q')
self.assertEqual(binascii.a2b_base85(self.type2test(b'abc')), b'qa')
self.assertEqual(binascii.a2b_base85(self.type2test(b'abcd')), b'qa\x9e')
self.assertEqual(binascii.a2b_base85(self.type2test(b'abcde')), b'qa\x9e\xb6')
self.assertEqual(binascii.a2b_base85(self.type2test(b'abcdef')), b'qa\x9e\xb6')

def test_base85_errors(self):
def _assertRegexTemplate(assert_regex, data, **kwargs):
with self.assertRaisesRegex(binascii.Error, assert_regex):
binascii.a2b_base85(self.type2test(data), **kwargs)

def assertNonBase85Data(data):
_assertRegexTemplate(r"(?i)bad base85 character", data)

def assertOverflow(data):
_assertRegexTemplate(r"(?i)base85 overflow", data)

assertNonBase85Data(b"\xda")
assertNonBase85Data(b"00\0\0")
assertNonBase85Data(b"Z )*")
assertNonBase85Data(b"bY*jNb0Hyq\n")

# Test base85 with out-of-range encoded value
assertOverflow(b"}")
assertOverflow(b"|O")
Expand All @@ -599,29 +580,6 @@ def test_base85_pad(self):
b_pad_expected = b + b"\0" * padding
self.assertEqual(b_pad, b_pad_expected)

def test_base85_strict_mode(self):
# Test base85 with strict mode on
def assertNonBase85Data(data, expected):
data = self.type2test(data)
with self.assertRaisesRegex(binascii.Error, r"(?i)bad base85 character"):
binascii.a2b_base85(data, strict_mode=True)
default_res = binascii.a2b_base85(data)
non_strict_res = binascii.a2b_base85(data, strict_mode=False)
self.assertEqual(default_res, non_strict_res)
self.assertEqual(non_strict_res, expected)

assertNonBase85Data(b"\xda", b"")
assertNonBase85Data(b"00\0\0", b"\0")
assertNonBase85Data(b"Z )*", b"ok")
assertNonBase85Data(b"bY*jNb0Hyq\n", b"tests!!~")

def test_base85_newline(self):
# Test base85 newline parameter
b = self.type2test(b"t3s\t ")
self.assertEqual(binascii.b2a_base85(b), b"bTe}aAO\n")
self.assertEqual(binascii.b2a_base85(b, newline=True), b"bTe}aAO\n")
self.assertEqual(binascii.b2a_base85(b, newline=False), b"bTe}aAO")

def test_z85_valid(self):
# Test Z85 with valid data
lines, i = [], 0
Expand All @@ -640,49 +598,30 @@ def test_z85_valid(self):
self.assertEqual(res, self.rawdata)

# Test decoding inputs with length 1 mod 5
self.assertEqual(binascii.a2b_z85(self.type2test(b"a")), b"")
self.assertEqual(binascii.a2b_z85(self.type2test(b" b ")), b"")
self.assertEqual(binascii.a2b_z85(self.type2test(b"B y,/;J_n\\c")), b"test")

def test_z85_invalid(self):
# Test Z85 with invalid characters interleaved
lines, i = [], 0
for k in range(1, len(self.rawdata) + 1):
b = self.type2test(self.rawdata[i:i + k])
a = binascii.b2a_z85(b)
lines.append(a)
i += k
if i >= len(self.rawdata):
break

fillers = bytearray()
valid = b"0123456789abcdefghijklmnopqrstuvwxyz" \
b"ABCDEFGHIJKLMNOPQRSTUVWXYZ.-:+=^!/*?&<>()[]{}@%$#"
for i in range(256):
if i not in valid:
fillers.append(i)
def addnoise(line):
res = bytearray()
for i in range(len(line)):
res.append(line[i])
for j in range(i, len(fillers), len(line)):
res.append(fillers[j])
return res
res = bytearray()
for line in map(addnoise, lines):
a = self.type2test(line)
b = binascii.a2b_z85(a)
res += b
self.assertEqual(res, self.rawdata)
self.assertEqual(binascii.a2b_z85(self.type2test(b'')), b'')
self.assertEqual(binascii.a2b_z85(self.type2test(b'a')), b'')
self.assertEqual(binascii.a2b_z85(self.type2test(b'ab')), b'\x1f')
self.assertEqual(binascii.a2b_z85(self.type2test(b'abc')), b'\x1f\x85')
self.assertEqual(binascii.a2b_z85(self.type2test(b'abcd')), b'\x1f\x85\x9a')
self.assertEqual(binascii.a2b_z85(self.type2test(b'abcde')), b'\x1f\x85\x9a$')
self.assertEqual(binascii.a2b_z85(self.type2test(b'abcdef')), b'\x1f\x85\x9a$')

def test_z85_errors(self):
def _assertRegexTemplate(assert_regex, data, **kwargs):
with self.assertRaisesRegex(binascii.Error, assert_regex):
binascii.a2b_z85(self.type2test(data), **kwargs)

def assertNonZ85Data(data):
_assertRegexTemplate(r"(?i)bad z85 character", data)

def assertOverflow(data):
_assertRegexTemplate(r"(?i)z85 overflow", data)

assertNonZ85Data(b"\xda")
assertNonZ85Data(b"00\0\0")
assertNonZ85Data(b"z !/")
assertNonZ85Data(b"By/JnB0hYQ\n")

# Test Z85 with out-of-range encoded value
assertOverflow(b"%")
assertOverflow(b"%n")
Expand All @@ -703,29 +642,6 @@ def test_z85_pad(self):
b_pad_expected = b + b"\0" * padding
self.assertEqual(b_pad, b_pad_expected)

def test_z85_strict_mode(self):
# Test Z85 with strict mode on
def assertNonZ85Data(data, expected):
data = self.type2test(data)
with self.assertRaisesRegex(binascii.Error, r"(?i)bad z85 character"):
binascii.a2b_z85(data, strict_mode=True)
default_res = binascii.a2b_z85(data)
non_strict_res = binascii.a2b_z85(data, strict_mode=False)
self.assertEqual(default_res, non_strict_res)
self.assertEqual(non_strict_res, expected)

assertNonZ85Data(b"\xda", b"")
assertNonZ85Data(b"00\0\0", b"\0")
assertNonZ85Data(b"z !/", b"ok")
assertNonZ85Data(b"By/JnB0hYQ\n", b"tests!!~")

def test_z85_newline(self):
# Test Z85 newline parameter
b = self.type2test(b"t3s\t ")
self.assertEqual(binascii.b2a_z85(b), b"BtE$Aao\n")
self.assertEqual(binascii.b2a_z85(b, newline=True), b"BtE$Aao\n")
self.assertEqual(binascii.b2a_z85(b, newline=False), b"BtE$Aao")

def test_uu(self):
MAX_UU = 45
for backtick in (True, False):
Expand Down
Loading