Skip to content

improve: more smart conditions of string decoding functions#39

Merged
gfx merged 3 commits intomasterfrom
select_best_function_in_str_decoding
May 29, 2019
Merged

improve: more smart conditions of string decoding functions#39
gfx merged 3 commits intomasterfrom
select_best_function_in_str_decoding

Conversation

@gfx
Copy link
Copy Markdown
Member

@gfx gfx commented May 29, 2019 •
edited
Loading

This is a kind of tragedy that TextDecoder is not always the best way. I wish someday when these complex conditions are completely obsolete, but for now, this is the best.

benchmark results:

$ npx ts-node benchmark/decode-string.ts

## string "A" x 10 (byteLength=10)

utf8DecodeJs x 6,820,213 ops/sec ±5.21% (82 runs sampled)
utf8DecodeWasm x 1,030,221 ops/sec ±3.86% (81 runs sampled)
TextDecoder x 788,986 ops/sec ±2.04% (79 runs sampled)

## string "A" x 100 (byteLength=100)

utf8DecodeJs x 825,191 ops/sec ±1.89% (86 runs sampled)
utf8DecodeWasm x 507,129 ops/sec ±2.98% (81 runs sampled)
TextDecoder x 495,937 ops/sec ±10.66% (66 runs sampled)

## string "A" x 200 (byteLength=200)

utf8DecodeJs x 406,791 ops/sec ±3.24% (80 runs sampled)
utf8DecodeWasm x 339,505 ops/sec ±3.38% (83 runs sampled)
TextDecoder x 485,291 ops/sec ±2.81% (76 runs sampled)

## string "A" x 1000 (byteLength=1000)

utf8DecodeJs x 65,969 ops/sec ±4.95% (69 runs sampled)
utf8DecodeWasm x 83,295 ops/sec ±3.22% (76 runs sampled)
TextDecoder x 260,622 ops/sec ±4.55% (83 runs sampled)

## string "A" x 10000 (byteLength=10000)

utf8DecodeJs x 8,696 ops/sec ±2.89% (81 runs sampled)
utf8DecodeWasm x 9,953 ops/sec ±3.42% (82 runs sampled)
TextDecoder x 45,540 ops/sec ±4.64% (82 runs sampled)

## string "A" x 100000 (byteLength=100000)

utf8DecodeJs x 408 ops/sec ±4.20% (83 runs sampled)
utf8DecodeWasm x 84.42 ops/sec ±3.50% (72 runs sampled)
TextDecoder x 3,443 ops/sec ±4.34% (80 runs sampled)

## string "あ" x 10 (byteLength=30)

utf8DecodeJs x 4,456,073 ops/sec ±4.75% (75 runs sampled)
utf8DecodeWasm x 1,030,237 ops/sec ±3.24% (86 runs sampled)
TextDecoder x 626,129 ops/sec ±5.00% (70 runs sampled)

## string "あ" x 100 (byteLength=300)

utf8DecodeJs x 512,368 ops/sec ±9.75% (75 runs sampled)
utf8DecodeWasm x 423,180 ops/sec ±5.45% (80 runs sampled)
TextDecoder x 390,576 ops/sec ±2.21% (81 runs sampled)

## string "あ" x 200 (byteLength=600)

utf8DecodeJs x 339,130 ops/sec ±0.90% (93 runs sampled)
utf8DecodeWasm x 347,386 ops/sec ±2.08% (90 runs sampled)
TextDecoder x 250,218 ops/sec ±4.65% (77 runs sampled)

## string "あ" x 1000 (byteLength=3000)

utf8DecodeJs x 54,151 ops/sec ±7.31% (75 runs sampled)
utf8DecodeWasm x 71,328 ops/sec ±9.33% (75 runs sampled)
TextDecoder x 62,094 ops/sec ±7.82% (68 runs sampled)

## string "あ" x 10000 (byteLength=30000)

utf8DecodeJs x 6,474 ops/sec ±2.55% (87 runs sampled)
utf8DecodeWasm x 9,338 ops/sec ±1.50% (90 runs sampled)
TextDecoder x 8,195 ops/sec ±2.76% (89 runs sampled)

## string "あ" x 100000 (byteLength=300000)

utf8DecodeJs x 372 ops/sec ±3.82% (80 runs sampled)
utf8DecodeWasm x 85.82 ops/sec ±2.21% (73 runs sampled)
TextDecoder x 895 ops/sec ±1.22% (89 runs sampled)

## string "🌏" x 20 (byteLength=40)

utf8DecodeJs x 2,951,171 ops/sec ±1.22% (91 runs sampled)
utf8DecodeWasm x 732,553 ops/sec ±7.60% (72 runs sampled)
TextDecoder x 511,127 ops/sec ±9.85% (59 runs sampled)

## string "🌏" x 200 (byteLength=400)

utf8DecodeJs x 140,319 ops/sec ±16.40% (39 runs sampled)
utf8DecodeWasm x 255,222 ops/sec ±9.40% (74 runs sampled)
TextDecoder x 320,214 ops/sec ±4.21% (77 runs sampled)

## string "🌏" x 400 (byteLength=800)

utf8DecodeJs x 169,404 ops/sec ±6.21% (83 runs sampled)
utf8DecodeWasm x 176,243 ops/sec ±5.15% (79 runs sampled)
TextDecoder x 177,882 ops/sec ±10.15% (67 runs sampled)

## string "🌏" x 2000 (byteLength=4000)

utf8DecodeJs x 26,572 ops/sec ±6.99% (71 runs sampled)
utf8DecodeWasm x 43,111 ops/sec ±3.76% (80 runs sampled)
TextDecoder x 52,740 ops/sec ±5.80% (75 runs sampled)

## string "🌏" x 20000 (byteLength=40000)

utf8DecodeJs x 3,234 ops/sec ±4.96% (75 runs sampled)
utf8DecodeWasm x 4,254 ops/sec ±4.55% (80 runs sampled)
TextDecoder x 5,502 ops/sec ±3.99% (78 runs sampled)

## string "🌏" x 200000 (byteLength=400000)

utf8DecodeJs x 176 ops/sec ±6.27% (68 runs sampled)
utf8DecodeWasm x 33.61 ops/sec ±6.59% (44 runs sampled)
TextDecoder x 546 ops/sec ±4.38% (79 runs sampled)

@gfx
Copy link
Copy Markdown
Member Author

gfx commented May 29, 2019

cc: @sergeyzenchenko

gfx
gfx commented May 29, 2019
View reviewed changes
Comment thread benchmark/decode-string.ts
@@ -1,50 +1,45 @@
/* eslint-disable no-console */
import { utf8Encode, utf8Count, utf8Decode } from "../src/utils/utf8";
import { utf8Encode, utf8Count, utf8DecodeJs, utf8DecodeTD } from "../src/utils/utf8";
Copy link
Copy Markdown
Member Author

@gfx gfx May 29, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

To benchmark them separately, I have split utf8decode functions.

@codecov-io
Copy link
Copy Markdown

codecov-io commented May 29, 2019 •
edited
Loading

Codecov Report

Merging #39 into master will decrease coverage by 0.21%.
The diff coverage is 83.33%.

Impacted file tree graph

@@            Coverage Diff             @@
##           master      #39      +/-   ##
==========================================
- Coverage   95.65%   95.43%   -0.22%     
==========================================
  Files          14       14              
  Lines         759      767       +8     
  Branches      160      163       +3     
==========================================
+ Hits          726      732       +6     
- Misses         14       15       +1     
- Partials       19       20       +1
Impacted Files Coverage Δ
src/wasmFunctions.ts 93.87% <100%> (+3.4%) ⬆️
src/Decoder.ts 98.55% <66.66%> (-0.72%) ⬇️
src/utils/utf8.ts 88.23% <80.95%> (-1.42%) ⬇️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 7f338b2...46f1ec9. Read the comment docs.

@codecov-io
Copy link
Copy Markdown

codecov-io commented May 29, 2019 •
edited
Loading

Codecov Report

Merging #39 into master will decrease coverage by 0.21%.
The diff coverage is 83.33%.

Impacted file tree graph

@@            Coverage Diff             @@
##           master      #39      +/-   ##
==========================================
- Coverage   95.65%   95.43%   -0.22%     
==========================================
  Files          14       14              
  Lines         759      767       +8     
  Branches      160      163       +3     
==========================================
+ Hits          726      732       +6     
- Misses         14       15       +1     
- Partials       19       20       +1
Impacted Files Coverage Δ
src/wasmFunctions.ts 93.87% <100%> (+3.4%) ⬆️
src/Decoder.ts 98.55% <66.66%> (-0.72%) ⬇️
src/utils/utf8.ts 88.23% <80.95%> (-1.42%) ⬇️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 7f338b2...46f1ec9. Read the comment docs.

@gfx gfx changed the title improve: more smart selection of string decoding functions improve: more smart conditions of string decoding functions May 29, 2019
@gfx gfx merged commit c2f8f6b into master May 29, 2019
@gfx gfx deleted the select_best_function_in_str_decoding branch May 29, 2019 02:11
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@gfx @codecov-io