Optimize string decoding performance#34
Merged
gfx merged 1 commit intomsgpack:masterfrom May 28, 2019
Merged
Conversation
Collaborator
Author
|
I think best way is to remove WASM decoding version in order to simplify the code. |
Member
|
Really fascinating results. Comparison with utf8Decode, utf8DecodeWasm, and TextDecoder like this: /* eslint-disable no-console */
import { utf8Encode, utf8Count, utf8Decode } from "../src/utils/utf8";
import { utf8DecodeWasm } from "../src/wasmFunctions";
// @ts-ignore
import Benchmark from "benchmark";
const textDecoder = new TextDecoder();
const dataSet = [10, 100, 200, 1_000, 10_000, 100_000].map((n) => {
return "a".repeat(n);
});
for (const str of dataSet) {
const byteLength = utf8Count(str);
const bytes = new Uint8Array(new ArrayBuffer(byteLength));
utf8Encode(str, bytes, 0);
console.log(`\n## string length=${str.length} byteLength=${byteLength}\n`);
const suite = new Benchmark.Suite();
const N = Math.round(100_0000 / str.length);
// use the result to avoid void-context optimizations
let count = 0;
suite.add("utf8Decode", () => {
if (utf8Decode(bytes, 0, byteLength) !== str) {
throw new Error("wrong result!");
}
});
suite.add("utf8DecodeWasm", () => {
if (utf8DecodeWasm(bytes, 0, byteLength) !== str) {
throw new Error("wrong result!");
}
});
suite.add("TextDecoder", () => {
if (textDecoder.decode(bytes.subarray(0, byteLength)) !== str) {
throw new Error("wrong result!");
}
});
suite.on("cycle", (event: any) => {
console.log(String(event.target));
});
suite.run();
}The result is already what you know, but the best threshold seems 200 (not 32): And as you suggested, WASM version is redundant if TextDecoder is available. It looks too early. |
Member
|
I'll still keep WASM ver. for a while because it should become faster than the current WASM engine. Especially wasm/reference-types could boost the speed significantly. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
@gfx here is string decoding optimization PR
#33