Skip to content

gh-124951: Optimize base64 encode & decode for an easy 2-3x speedup [no SIMD] #143262

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
gpshead merged 13 commits into from
Jan 2, 2026
Merged

gh-124951: Optimize base64 encode & decode for an easy 2-3x speedup [no SIMD] #143262

Changes from 1 commit
Commits
Show all changes
13 commits
Select commit Hold shift + click to select a range
2d2be30
Add base64 benchmark tool and optimize encoding/decoding
gpshead Dec 28, 2025
573eaf3
Fix MSVC build: move table_b2a_base64 before inline functions
gpshead Dec 29, 2025
eaac671
NEWS entry
gpshead Dec 29, 2025
060dbf5
Add a whatsnew entry
gpshead Dec 29, 2025
1e12273
expose the benchmark defaults in the help text.
gpshead Dec 29, 2025
ef38895
Align base64 tables to 64-byte cache line boundaries
gpshead Dec 29, 2025
7458c99
Use BASE64_PAD macro instead of literal '=' in fast path
gpshead Dec 29, 2025
1f8ff74
Simplify block comment for base64 helpers
gpshead Dec 29, 2025
4b1245b
Remove binasciibench in favor of pyperformance bm_base64 addition PR.
gpshead Dec 29, 2025
4b50c73
reword L1 cache line comment
gpshead Jan 2, 2026
f7b27ee
doc Contributed by
gpshead Jan 2, 2026
2b13b81
Remove redundant PAD check in base64 decode fast path
gpshead Jan 2, 2026
30976a9
Use pointer increment in base64_encode_fast
gpshead Jan 2, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
Align base64 tables to 64-byte cache line boundaries 
Add Py_ALIGNED(64) to both lookup tables to ensure each fits
within a single L1 cache line, reducing potential cache misses
during encoding/decoding loops.

Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
  • Loading branch information
@gpshead @claude
gpshead and claude committed Dec 29, 2025
commit ef38895ff3208ed439ba1c63e7a26ea1c77a72ca
6 changes: 4 additions & 2 deletions Modules/binascii.c
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -76,7 +76,8 @@ get_binascii_state(PyObject *module)
}


static const unsigned char table_a2b_base64[] = {
/* Align to 64 bytes to ensure table fits in a single L1 cache line */
Comment thread
gpshead marked this conversation as resolved.
Outdated
static const unsigned char table_a2b_base64[] Py_ALIGNED(64) = {
-1,-1,-1,-1, -1,-1,-1,-1, -1,-1,-1,-1, -1,-1,-1,-1,
-1,-1,-1,-1, -1,-1,-1,-1, -1,-1,-1,-1, -1,-1,-1,-1,
-1,-1,-1,-1, -1,-1,-1,-1, -1,-1,-1,62, -1,-1,-1,63,
Expand Down Expand Up @@ -109,7 +110,8 @@ static const unsigned char table_a2b_base64[] = {
* This allows the compiler to better optimize the hot loops.
*/

static const unsigned char table_b2a_base64[] =
/* Align to 64 bytes to ensure table fits in a single L1 cache line */
static const unsigned char table_b2a_base64[] Py_ALIGNED(64) =
"ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz0123456789+/";

/* Encode 3 bytes into 4 base64 characters. */
Expand Down
Loading