Skip to content

util: add fast path for Latin1 decoding #55275

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
nodejs-github-bot merged 15 commits into from
Dec 3, 2024
Merged

util: add fast path for Latin1 decoding #55275

Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
15 commits
Select commit Hold shift + click to select a range
b12ba58
util: add fast path for Latin1 decoding
Oct 5, 2024
3c239bd
handle input correctly in decode functions
Oct 5, 2024
17f317e
repair
Oct 5, 2024
e0a40c6
Update lib/internal/encoding.js
mertcanaltin Oct 5, 2024
b7420b5
add ignoreBOM and fatal support to decodeLatin1
Oct 5, 2024
8bc747e
lint
Oct 5, 2024
fced029
update tests to support ignoreBOM and fatal for decodeLatin1
Oct 5, 2024
af7cca5
lint
Oct 5, 2024
659f1b6
correct BOM check in decodeLatin1 for uint8_t comparison
Oct 5, 2024
b92e571
add BOM test case to cover 0xFF in decodeLatin1
Oct 5, 2024
94109bf
Update encoding_binding.cc
mertcanaltin Oct 6, 2024
6dec37f
delete result.resize
Oct 6, 2024
f62f4c4
Update encoding_binding.cc
mertcanaltin Oct 7, 2024
f96e2ef
Update encoding_binding.cc
mertcanaltin Oct 7, 2024
77bc5ee
update text-decoder to use windows-1252 encoding
Oct 8, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
add BOM test case to cover 0xFF in decodeLatin1
  • Loading branch information
Mert Can Altin
Mert Can Altin committed Oct 5, 2024
commit b92e571f04983a7cf029649e9b2fef5db312bb75
2 changes: 1 addition & 1 deletion src/encoding_binding.cc
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -267,7 +267,7 @@ void BindingData::DecodeLatin1(const FunctionCallbackInfo<Value>& args) {
const uint8_t* data = buffer.data();
size_t length = buffer.length();

if (ignore_bom && length > 0 && data[0] == 0xFEFF) {
if (ignore_bom && length > 0 && data[0] == 0xFF) {
data++;
length--;
}
Expand Down
19 changes: 19 additions & 0 deletions test/cctest/test_encoding_binding.cc
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -132,5 +132,24 @@ TEST_F(EncodingBindingTest, DecodeLatin1_IgnoreBOMAndFatal) {
EXPECT_STREQ(*utf8_result, "Áéó");
}

TEST_F(EncodingBindingTest, DecodeLatin1_BOMPresent) {
Environment* env = CreateEnvironment();
Isolate* isolate = env->isolate();
HandleScope handle_scope(isolate);

const uint8_t latin1_data[] = {0xFF, 0xC1, 0xE9, 0xF3};
Local<ArrayBuffer> ab = ArrayBuffer::New(isolate, sizeof(latin1_data));
memcpy(ab->GetBackingStore()->Data(), latin1_data, sizeof(latin1_data));

Local<Uint8Array> array = Uint8Array::New(ab, 0, sizeof(latin1_data));
Local<Value> args[] = {array};

Local<Value> result;
EXPECT_TRUE(RunDecodeLatin1(env, args, true, false, &result));

String::Utf8Value utf8_result(isolate, result);
EXPECT_STREQ(*utf8_result, "Áéó");
}

} // namespace encoding_binding
} // namespace node