Skip to content

lib: optimize TextEncoder.encode for short strings - #66831

Open
mertcanaltin wants to merge 3 commits into
nodejs:mainfrom
mertcanaltin:mert/textencoder-small-string
Open

mertcanaltin wants to merge 3 commits into
nodejs:mainfrom
mertcanaltin:mert/textencoder-small-string

Conversation

@mertcanaltin

Copy link
Copy Markdown
Member

This pr improvement short strings for TextEncoder.encode.

Benchmark results:

(.venv) ➜  node git:(mert/textencoder-small-string) ✗ npx node-benchmark-compare te-clean.csv                                                        
                                                                          confidence improvement accuracy (*)   (**)  (***)
util/text-encoder.js op='encode' type='ascii' n=1000000 len=256                    *      1.67 %       ±1.45% ±1.94% ±2.54%
util/text-encoder.js op='encode' type='ascii' n=1000000 len=32                   ***    127.18 %       ±1.93% ±2.58% ±3.40%
util/text-encoder.js op='encode' type='one-byte-string' n=1000000 len=256                 1.39 %       ±2.07% ±2.79% ±3.68%
util/text-encoder.js op='encode' type='one-byte-string' n=1000000 len=32         ***     85.44 %       ±2.43% ±3.27% ±4.31%
util/text-encoder.js op='encode' type='two-byte-string' n=1000000 len=256                -0.12 %       ±0.86% ±1.15% ±1.49%
util/text-encoder.js op='encode' type='two-byte-string' n=1000000 len=32         ***     74.83 %       ±1.45% ±1.94% ±2.54%

Be aware that when doing many comparisons the risk of a false-positive result increases.
In this case, there are 6 comparisons, you can thus expect the following amount of false-positive results:
  0.30 false positives, when considering a   5% risk acceptance (*, **, ***),
  0.06 false positives, when considering a   1% risk acceptance (**, ***),
  0.01 false positives, when considering a 0.1% risk acceptance (***)

I got help from Claude Opus 5.5. I reviewed the changes and ran the tests and benchmarks myself.

Assisted-by: Claude opus 5.5
Signed-off-by: Mert Can Altin <mertgold60@gmail.com>
@nodejs-github-bot nodejs-github-bot added encoding Issues and PRs related to the TextEncoder and TextDecoder APIs. needs-ci PRs that need a full CI run. labels Oct 18, 2026
Comment thread test/parallel/test-whatwg-encoding-custom-textencoder-short.js Outdated
mertcanaltin and others added 2 commits October 18, 2026 16:51
Co-authored-by: Antoine du Hamel <duhamelantoine1995@gmail.com>
@mertcanaltin
mertcanaltin requested a review from aduh95 October 18, 2026 13:58
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

encoding Issues and PRs related to the TextEncoder and TextDecoder APIs. needs-ci PRs that need a full CI run.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants