Skip to content

Commit b3af954

Browse files
msaboffmsaboff
committed
Implement RegExp Unicode property escapes
https://bugs.webkit.org/show_bug.cgi?id=172069 Reviewed by JF Bastien. JSTests: Enabled Unicode Property tests. * test262.yaml: Source/JavaScriptCore: Added Unicode Properties by extending the existing CharacterClass processing. Introduced a new Python script, generateYarrUnicodePropertyTables.py, that parses Unicode Database files to create character class data. The result is a set of functions that return character classes, one for each of the required Unicode properties. There are many cases where many properties are handled by one function, primarily due to property aliases, but also due to Script_Extension properties that are the same as the Script property for the same script value. Extended the BuiltInCharacterClassID enum so it can be used also for Unicode property character classes. Unicode properties are the enum value BaseUnicodePropertyID plus a zero based value, that value being the index to the corrensponding character class function. The generation script also creates static hashing tables similar to what we use for the generated .lut.h lookup table files. These hashing tables map property names to the function index. Using these hashing tables, we can lookup a property name and if present convert it to a function index. We add that index to BaseUnicodePropertyID to create a BuiltInCharacterClassID. When we do syntax parsing, we convert the property to its corresponding BuiltInCharacterClassID. When doing real parsing we takes the returned BuiltInCharacterClassID and use it to get the actual character class by calling the corresponding generated function. Added a new CharacterClass constructor that can take literal arrays for ranges and matches to make the creation of large static character classes more efficent. Since the Unicode character classes typically have more matches and ranges, the character class matching in the interpreter has been updated to use binary searching for matches and ranges with more than 6 entries. * CMakeLists.txt: * DerivedSources.make: * JavaScriptCore.xcodeproj/project.pbxproj: * Scripts/generateYarrUnicodePropertyTables.py: Added. (openOrExit): (openUCDFileOrExit): (verifyUCDFilesExist): (ceilingToPowerOf2): (Aliases): (Aliases.__init__): (Aliases.parsePropertyAliasesFile): (Aliases.parsePropertyValueAliasesFile): (Aliases.globalAliasesFor): (Aliases.generalCategoryAliasesFor): (Aliases.generalCategoryForAlias): (Aliases.scriptAliasesFor): (Aliases.scriptNameForAlias): (PropertyData): (PropertyData.__init__): (PropertyData.setAliases): (PropertyData.makeCopy): (PropertyData.getIndex): (PropertyData.getCreateFuncName): (PropertyData.addMatch): (PropertyData.addRange): (PropertyData.addMatchUnorderedForMatchesAndRanges): (PropertyData.addRangeUnorderedForMatchesAndRanges): (PropertyData.addMatchUnordered): (PropertyData.addRangeUnordered): (PropertyData.removeMatchFromRanges): (PropertyData.removeMatch): (PropertyData.dumpMatchData): (PropertyData.dump): (PropertyData.dumpAll): (PropertyData.dumpAll.std): (PropertyData.createAndDumpHashTable): (Scripts): (Scripts.__init__): (Scripts.parseScriptsFile): (Scripts.parseScriptExtensionsFile): (Scripts.dump): (GeneralCategory): (GeneralCategory.__init__): (GeneralCategory.createSpecialPropertyData): (GeneralCategory.findPropertyGroupFor): (GeneralCategory.addNextCodePoints): (GeneralCategory.parse): (GeneralCategory.dump): (BinaryProperty): (BinaryProperty.__init__): (BinaryProperty.parsePropertyFile): (BinaryProperty.dump): * Scripts/hasher.py: Added. (stringHash): * Sources.txt: * ucd/DerivedBinaryProperties.txt: Added. * ucd/DerivedCoreProperties.txt: Added. * ucd/DerivedNormalizationProps.txt: Added. * ucd/PropList.txt: Added. * ucd/PropertyAliases.txt: Added. * ucd/PropertyValueAliases.txt: Added. * ucd/ScriptExtensions.txt: Added. * ucd/Scripts.txt: Added. * ucd/UnicodeData.txt: Added. * ucd/emoji-data.txt: Added. * yarr/Yarr.h: * yarr/YarrInterpreter.cpp: (JSC::Yarr::Interpreter::testCharacterClass): * yarr/YarrParser.h: (JSC::Yarr::Parser::parseEscape): (JSC::Yarr::Parser::parseTokens): (JSC::Yarr::Parser::isUnicodePropertyValueExpressionChar): (JSC::Yarr::Parser::tryConsumeUnicodePropertyExpression): * yarr/YarrPattern.cpp: (JSC::Yarr::CharacterClassConstructor::appendInverted): (JSC::Yarr::YarrPatternConstructor::atomBuiltInCharacterClass): (JSC::Yarr::YarrPatternConstructor::atomCharacterClassBuiltIn): (JSC::Yarr::YarrPattern::errorMessage): (JSC::Yarr::PatternTerm::dump): * yarr/YarrPattern.h: (JSC::Yarr::CharacterRange::CharacterRange): (JSC::Yarr::CharacterClass::CharacterClass): (JSC::Yarr::YarrPattern::reset): (JSC::Yarr::YarrPattern::unicodeCharacterClassFor): * yarr/YarrUnicodeProperties.cpp: Added. (JSC::Yarr::HashTable::entry const): (JSC::Yarr::unicodeMatchPropertyValue): (JSC::Yarr::unicodeMatchProperty): (JSC::Yarr::createUnicodeCharacterClassFor): * yarr/YarrUnicodeProperties.h: Added. Source/WebCore: Refactoring change - Added BuiltInCharacterClassID:: prefix to uses of the enum. * contentextensions/URLFilterParser.cpp: (WebCore::ContentExtensions::PatternParser::atomBuiltInCharacterClass): LayoutTests: New test. * js/regexp-unicode-properties-expected.txt: Added. * js/regexp-unicode-properties.html: Added. * js/script-tests/regexp-unicode-properties.js: Added. Canonical link: https://commits.webkit.org/194348@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@223081 268f45cc-cd09-0410-ab3c-d52691b4dbfc
1 parent 480c05f commit b3af954

32 files changed

Lines changed: 62805 additions & 786 deletions

‎JSTests/ChangeLog‎

Lines changed: 11 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,3 +1,14 @@
1+
2017-10-09 Michael Saboff <msaboff@apple.com>
2+
3+
Implement RegExp Unicode property escapes
4+
https://bugs.webkit.org/show_bug.cgi?id=172069
5+
6+
Reviewed by JF Bastien.
7+
8+
Enabled Unicode Property tests.
9+
10+
* test262.yaml:
11+
112
2017-10-09 Commit Queue <commit-queue@webkit.org>
213

314
Unreviewed, rolling out r223015 and r223025.

‎JSTests/test262.yaml‎

Lines changed: 750 additions & 750 deletions
Large diffs are not rendered by default.

‎LayoutTests/ChangeLog‎

Lines changed: 13 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,3 +1,16 @@
1+
2017-10-09 Michael Saboff <msaboff@apple.com>
2+
3+
Implement RegExp Unicode property escapes
4+
https://bugs.webkit.org/show_bug.cgi?id=172069
5+
6+
Reviewed by JF Bastien.
7+
8+
New test.
9+
10+
* js/regexp-unicode-properties-expected.txt: Added.
11+
* js/regexp-unicode-properties.html: Added.
12+
* js/script-tests/regexp-unicode-properties.js: Added.
13+
114
2017-10-09 Andy Estes <aestes@apple.com>
215

316
[Payment Request] Implement PaymentRequest.show() and PaymentRequest.hide()

‎LayoutTests/js/regexp-unicode-properties-expected.txt‎

Lines changed: 41 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,41 @@
1+
Test for of RegExp Unicode properties
2+
3+
On success, you will see a series of "PASS" messages, followed by "TEST COMPLETE".
4+
5+
6+
PASS matchResult1[0] is src1
7+
PASS re2.test(src2) is true
8+
PASS /\p{Any-Old-Property}/u.test(src2) threw exception SyntaxError: Invalid regular expression: invalid property expression.
9+
PASS /\p{Script=Hebrew/u.test("Testing") threw exception SyntaxError: Invalid regular expression: invalid property expression.
10+
PASS /\p{Hebrew/u.test("Testing") threw exception SyntaxError: Invalid regular expression: invalid property expression.
11+
PASS /\p{Hebrew}/u.test("Testing") threw exception SyntaxError: Invalid regular expression: invalid property expression.
12+
PASS /\p{Letter/u.test("Testing") threw exception SyntaxError: Invalid regular expression: invalid property expression.
13+
PASS /\p{Cc/u.test("Testing") threw exception SyntaxError: Invalid regular expression: invalid property expression.
14+
PASS src3.match(re3a)[0] is src3
15+
PASS src3.match(re3b)[0] is src3
16+
PASS src4.match(re4a) is exp4
17+
PASS src4.match(re4c) is exp4
18+
PASS src4.match(re4d) is exp4
19+
PASS src4.match(re4e) is exp4
20+
PASS src4.match(re4f) is exp4
21+
PASS src5.match(re5a)[0] is "abdudtrjſºⅇꜹ𑣄"
22+
PASS src5.match(re5b)[0] is "abdudtrjſ"
23+
PASS src5.match(re5c)[0] is "SOME LOWER CASE: "
24+
PASS re6a.exec(src6)[0] is "λόγος"
25+
PASS re6b.exec(src6)[0] is "The Greek word "
26+
PASS re6c.exec(src6)[0] is "The Greek word λόγος means "
27+
PASS re7a.exec(src7a)[0] is "+"
28+
PASS re7b.exec(src7a)[0] is "+"
29+
PASS re7c.exec(src7a)[0] is "2"
30+
PASS re7d.exec(src7a)[0] is "2 + 2 = 4"
31+
PASS re7d.exec(src7b)[0] is "2 ^ "
32+
PASS re7e.exec(src7b)[0] is "2 ^ .5 = 1.4142135"
33+
PASS re7e.exec(src7c)[0] is "2 ÷ 3 = ~0.666666"
34+
PASS re7f.exec(src7d)[0] is "5³ + 3² = 134"
35+
PASS src7d.match(re7g) is ["+", "="]
36+
PASS re8a.exec(src8)[0] is "வணக்கம்"
37+
PASS re8b.exec(src8)[0] is "Hello is வணக்கம் in Tamil"
38+
PASS successfullyParsed is true
39+
40+
TEST COMPLETE
41+

‎LayoutTests/js/regexp-unicode-properties.html‎

Lines changed: 10 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,10 @@
1+
<!DOCTYPE HTML PUBLIC "-//IETF//DTD HTML//EN">
2+
<html>
3+
<head>
4+
<script src="../resources/js-test-pre.js"></script>
5+
</head>
6+
<body>
7+
<script src="script-tests/regexp-unicode-properties.js"></script>
8+
<script src="../resources/js-test-post.js"></script>
9+
</body>
10+
</html>

‎LayoutTests/js/script-tests/regexp-unicode-properties.js‎

Lines changed: 83 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,83 @@
1+
description(
2+
'Test for of RegExp Unicode properties'
3+
);
4+
5+
let src1 = ""
6+
for (let c = 1; c <= 0x7f; c++)
7+
src1 += String.fromCharCode(c);
8+
let re1 = new RegExp("\\p{ASCII}*", "u")
9+
var matchResult1 = src1.match(re1);
10+
shouldBe('matchResult1[0]', 'src1');
11+
12+
let src2 = "p{Any-Old-Property}";
13+
let re2 = /\p{Any-Old-Property}/;
14+
shouldBeTrue('re2.test(src2)');
15+
16+
shouldThrow('/\\p{Any-Old-Property}/u.test(src2)', '"SyntaxError: Invalid regular expression: invalid property expression"');
17+
shouldThrow('/\\p{Script=Hebrew/u.test("Testing")', '"SyntaxError: Invalid regular expression: invalid property expression"');
18+
shouldThrow('/\\p{Hebrew/u.test("Testing")', '"SyntaxError: Invalid regular expression: invalid property expression"');
19+
shouldThrow('/\\p{Hebrew}/u.test("Testing")', '"SyntaxError: Invalid regular expression: invalid property expression"');
20+
shouldThrow('/\\p{Letter/u.test("Testing")', '"SyntaxError: Invalid regular expression: invalid property expression"');
21+
shouldThrow('/\\p{Cc/u.test("Testing")', '"SyntaxError: Invalid regular expression: invalid property expression"');
22+
23+
let src3 = "DeadBeef4A11";
24+
let re3a = /\p{ASCII_Hex_Digit}*/u;
25+
shouldBe('src3.match(re3a)[0]', 'src3');
26+
let re3b = /\p{AHex}*/u;
27+
shouldBe('src3.match(re3b)[0]', 'src3');
28+
29+
let src4 = "The Rain In Spain Falls Mainly on THE Plains.";
30+
let re4a = /\p{General_Category=Uppercase_Letter}/ug;
31+
let exp4 = ["T", "R", "I", "S", "F", "M", "T", "H", "E", "P"];
32+
shouldBe('src4.match(re4a)', 'exp4');
33+
let re4c = /\p{General_Category=Lu}/ug;
34+
shouldBe('src4.match(re4c)', 'exp4');
35+
let re4d = /\p{gc=Lu}/ug;
36+
shouldBe('src4.match(re4d)', 'exp4');
37+
let re4e = /\p{Uppercase_Letter}/ug;
38+
shouldBe('src4.match(re4e)', 'exp4');
39+
let re4f = /\p{Lu}/ug;
40+
shouldBe('src4.match(re4f)', 'exp4');
41+
42+
let src5 = "SOME LOWER CASE: abdudtrj\u017f\u00ba\u2147\ua739\u{118c4}";
43+
let re5a = /\p{Lowercase}+/u;
44+
shouldBe('src5.match(re5a)[0]', '"abdudtrj\u017f\u00ba\u2147\ua739\u{118c4}"');
45+
let re5b = /\p{Lowercase_Letter}+/u;
46+
shouldBe('src5.match(re5b)[0]', '"abdudtrj\u017f"');
47+
let re5c = /\P{Lowercase_Letter}+/u;
48+
shouldBe('src5.match(re5c)[0]', '"SOME LOWER CASE: "');
49+
50+
let src6 = "The Greek word \u03bb\u03cc\u03b3\u03bf\u03c2 means \"reason\".";
51+
let re6a = /\p{Script_Extensions=Grek}+/u;
52+
shouldBe('re6a.exec(src6)[0]', '"\u03bb\u03cc\u03b3\u03bf\u03c2"');
53+
let re6b = /\P{Script_Extensions=Grek}+/u;
54+
shouldBe('re6b.exec(src6)[0]', '"The Greek word "');
55+
let re6c = /[ \p{Script=Latin}\p{Script_Extensions=Grek}]+/u;
56+
shouldBe('re6c.exec(src6)[0]', '"The Greek word \u03bb\u03cc\u03b3\u03bf\u03c2 means "');
57+
58+
let src7a = "2 + 2 = 4";
59+
let re7a = /\p{Math}/u;
60+
shouldBe('re7a.exec(src7a)[0]', '"+"');
61+
let re7b = /[\p{Math}]+/u;
62+
shouldBe('re7b.exec(src7a)[0]', '"+"');
63+
let re7c = /[\p{Number}\p{Math}]+/u;
64+
shouldBe('re7c.exec(src7a)[0]', '"2"');
65+
let re7d = /[\p{Number}\p{Math}\p{space}]+/u;
66+
shouldBe('re7d.exec(src7a)[0]', '"2 + 2 = 4"');
67+
let src7b = "2 ^ .5 = 1.4142135";
68+
shouldBe('re7d.exec(src7b)[0]', '"2 ^ "');
69+
let re7e = /[\p{Number}\p{Math}\p{space}\.]+/u;
70+
shouldBe('re7e.exec(src7b)[0]', '"2 ^ .5 = 1.4142135"');
71+
let src7c = "2 \u00f7 3 = ~0.666666";
72+
shouldBe('re7e.exec(src7c)[0]', '"2 \u00f7 3 = ~0.666666"');
73+
let re7f = /[\p{N}\{No}\p{Math}\p{space}\.]+/u;
74+
let src7d = "5\u00b3 + 3\u00b2 = 134";
75+
shouldBe('re7f.exec(src7d)[0]', '"5\u00b3 + 3\u00b2 = 134"');
76+
let re7g = /[\p{Math}]+/ug;
77+
shouldBe('src7d.match(re7g)', '["+", "="]');
78+
79+
let src8 = "Hello is \u0bb5\u0ba3\u0b95\u0bcd\u0b95\u0bae\u0bcd in Tamil";
80+
let re8a = /[\p{Script_Extensions=Taml}]+/u;
81+
shouldBe('re8a.exec(src8)[0]', '"\u0bb5\u0ba3\u0b95\u0bcd\u0b95\u0bae\u0bcd"');
82+
let re8b = /[\p{Script_Extensions=Latn}\p{Script_Extensions=Taml} ]+/u;
83+
shouldBe('re8b.exec(src8)[0]', '"Hello is \u0bb5\u0ba3\u0b95\u0bcd\u0b95\u0bae\u0bcd in Tamil"');

‎Source/JavaScriptCore/CMakeLists.txt‎

Lines changed: 9 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -365,6 +365,15 @@ add_custom_command(
365365
list(APPEND JavaScriptCore_HEADERS ${DERIVED_SOURCES_JAVASCRIPTCORE_DIR}/yarr/RegExpJitTables.h)
366366
WEBKIT_ADD_SOURCE_DEPENDENCIES(${CMAKE_CURRENT_SOURCE_DIR}/yarr/YarrPattern.cpp ${DERIVED_SOURCES_JAVASCRIPTCORE_DIR}/yarr/RegExpJitTables.h)
367367

368+
add_custom_command(
369+
OUTPUT ${DERIVED_SOURCES_JAVASCRIPTCORE_DIR}/yarr/UnicodePatternTables.h
370+
MAIN_DEPENDENCY ${JAVASCRIPTCORE_DIR}/Scripts/generateYarrUnicodePropertyTables.py
371+
DEPENDS ${JAVASCRIPTCORE_DIR}/Scripts/hasher.py ${JAVASCRIPTCORE_DIR}/ucd/DerivedBinaryProperties.txt ${JAVASCRIPTCORE_DIR}/ucd/DerivedCoreProperties.txt ${JAVASCRIPTCORE_DIR}/ucd/DerivedNormalizationProps.txt ${JAVASCRIPTCORE_DIR}/ucd/PropList.txt ${JAVASCRIPTCORE_DIR}/ucd/PropertyAliases.txt ${JAVASCRIPTCORE_DIR}/ucd/PropertyValueAliases.txt ${JAVASCRIPTCORE_DIR}/ucd/ScriptExtensions.txt ${JAVASCRIPTCORE_DIR}/ucd/Scripts.txt ${JAVASCRIPTCORE_DIR}/ucd/UnicodeData.txt ${JAVASCRIPTCORE_DIR}/ucd/emoji-data.txt
372+
COMMAND ${PYTHON_EXECUTABLE} ${JAVASCRIPTCORE_DIR}/Scripts/generateYarrUnicodePropertyTables.py ${JAVASCRIPTCORE_DIR}/ucd ${DERIVED_SOURCES_JAVASCRIPTCORE_DIR}/yarr/UnicodePatternTables.h
373+
VERBATIM)
374+
list(APPEND JavaScriptCore_HEADERS ${DERIVED_SOURCES_JAVASCRIPTCORE_DIR}/yarr/UnicodePatternTables.h)
375+
WEBKIT_ADD_SOURCE_DEPENDENCIES(${CMAKE_CURRENT_SOURCE_DIR}/yarr/UnicodePatternTables.cpp ${DERIVED_SOURCES_JAVASCRIPTCORE_DIR}/yarr/UnicodePatternTables.h)
376+
368377
add_custom_command(
369378
OUTPUT ${DERIVED_SOURCES_JAVASCRIPTCORE_DIR}/yarr/YarrCanonicalizeUnicode.cpp
370379
MAIN_DEPENDENCY ${JAVASCRIPTCORE_DIR}/yarr/generateYarrCanonicalizeUnicode

‎Source/JavaScriptCore/ChangeLog‎

Lines changed: 127 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,3 +1,130 @@
1+
2017-10-09 Michael Saboff <msaboff@apple.com>
2+
3+
Implement RegExp Unicode property escapes
4+
https://bugs.webkit.org/show_bug.cgi?id=172069
5+
6+
Reviewed by JF Bastien.
7+
8+
Added Unicode Properties by extending the existing CharacterClass processing.
9+
10+
Introduced a new Python script, generateYarrUnicodePropertyTables.py, that parses
11+
Unicode Database files to create character class data. The result is a set of functions
12+
that return character classes, one for each of the required Unicode properties.
13+
There are many cases where many properties are handled by one function, primarily due to
14+
property aliases, but also due to Script_Extension properties that are the same as the
15+
Script property for the same script value.
16+
17+
Extended the BuiltInCharacterClassID enum so it can be used also for Unicode property
18+
character classes. Unicode properties are the enum value BaseUnicodePropertyID plus a
19+
zero based value, that value being the index to the corrensponding character class
20+
function. The generation script also creates static hashing tables similar to what we
21+
use for the generated .lut.h lookup table files. These hashing tables map property
22+
names to the function index. Using these hashing tables, we can lookup a property
23+
name and if present convert it to a function index. We add that index to
24+
BaseUnicodePropertyID to create a BuiltInCharacterClassID.
25+
26+
When we do syntax parsing, we convert the property to its corresponding BuiltInCharacterClassID.
27+
When doing real parsing we takes the returned BuiltInCharacterClassID and use it to get
28+
the actual character class by calling the corresponding generated function.
29+
30+
Added a new CharacterClass constructor that can take literal arrays for ranges and matches
31+
to make the creation of large static character classes more efficent.
32+
33+
Since the Unicode character classes typically have more matches and ranges, the character
34+
class matching in the interpreter has been updated to use binary searching for matches and
35+
ranges with more than 6 entries.
36+
37+
* CMakeLists.txt:
38+
* DerivedSources.make:
39+
* JavaScriptCore.xcodeproj/project.pbxproj:
40+
* Scripts/generateYarrUnicodePropertyTables.py: Added.
41+
(openOrExit):
42+
(openUCDFileOrExit):
43+
(verifyUCDFilesExist):
44+
(ceilingToPowerOf2):
45+
(Aliases):
46+
(Aliases.__init__):
47+
(Aliases.parsePropertyAliasesFile):
48+
(Aliases.parsePropertyValueAliasesFile):
49+
(Aliases.globalAliasesFor):
50+
(Aliases.generalCategoryAliasesFor):
51+
(Aliases.generalCategoryForAlias):
52+
(Aliases.scriptAliasesFor):
53+
(Aliases.scriptNameForAlias):
54+
(PropertyData):
55+
(PropertyData.__init__):
56+
(PropertyData.setAliases):
57+
(PropertyData.makeCopy):
58+
(PropertyData.getIndex):
59+
(PropertyData.getCreateFuncName):
60+
(PropertyData.addMatch):
61+
(PropertyData.addRange):
62+
(PropertyData.addMatchUnorderedForMatchesAndRanges):
63+
(PropertyData.addRangeUnorderedForMatchesAndRanges):
64+
(PropertyData.addMatchUnordered):
65+
(PropertyData.addRangeUnordered):
66+
(PropertyData.removeMatchFromRanges):
67+
(PropertyData.removeMatch):
68+
(PropertyData.dumpMatchData):
69+
(PropertyData.dump):
70+
(PropertyData.dumpAll):
71+
(PropertyData.dumpAll.std):
72+
(PropertyData.createAndDumpHashTable):
73+
(Scripts):
74+
(Scripts.__init__):
75+
(Scripts.parseScriptsFile):
76+
(Scripts.parseScriptExtensionsFile):
77+
(Scripts.dump):
78+
(GeneralCategory):
79+
(GeneralCategory.__init__):
80+
(GeneralCategory.createSpecialPropertyData):
81+
(GeneralCategory.findPropertyGroupFor):
82+
(GeneralCategory.addNextCodePoints):
83+
(GeneralCategory.parse):
84+
(GeneralCategory.dump):
85+
(BinaryProperty):
86+
(BinaryProperty.__init__):
87+
(BinaryProperty.parsePropertyFile):
88+
(BinaryProperty.dump):
89+
* Scripts/hasher.py: Added.
90+
(stringHash):
91+
* Sources.txt:
92+
* ucd/DerivedBinaryProperties.txt: Added.
93+
* ucd/DerivedCoreProperties.txt: Added.
94+
* ucd/DerivedNormalizationProps.txt: Added.
95+
* ucd/PropList.txt: Added.
96+
* ucd/PropertyAliases.txt: Added.
97+
* ucd/PropertyValueAliases.txt: Added.
98+
* ucd/ScriptExtensions.txt: Added.
99+
* ucd/Scripts.txt: Added.
100+
* ucd/UnicodeData.txt: Added.
101+
* ucd/emoji-data.txt: Added.
102+
* yarr/Yarr.h:
103+
* yarr/YarrInterpreter.cpp:
104+
(JSC::Yarr::Interpreter::testCharacterClass):
105+
* yarr/YarrParser.h:
106+
(JSC::Yarr::Parser::parseEscape):
107+
(JSC::Yarr::Parser::parseTokens):
108+
(JSC::Yarr::Parser::isUnicodePropertyValueExpressionChar):
109+
(JSC::Yarr::Parser::tryConsumeUnicodePropertyExpression):
110+
* yarr/YarrPattern.cpp:
111+
(JSC::Yarr::CharacterClassConstructor::appendInverted):
112+
(JSC::Yarr::YarrPatternConstructor::atomBuiltInCharacterClass):
113+
(JSC::Yarr::YarrPatternConstructor::atomCharacterClassBuiltIn):
114+
(JSC::Yarr::YarrPattern::errorMessage):
115+
(JSC::Yarr::PatternTerm::dump):
116+
* yarr/YarrPattern.h:
117+
(JSC::Yarr::CharacterRange::CharacterRange):
118+
(JSC::Yarr::CharacterClass::CharacterClass):
119+
(JSC::Yarr::YarrPattern::reset):
120+
(JSC::Yarr::YarrPattern::unicodeCharacterClassFor):
121+
* yarr/YarrUnicodeProperties.cpp: Added.
122+
(JSC::Yarr::HashTable::entry const):
123+
(JSC::Yarr::unicodeMatchPropertyValue):
124+
(JSC::Yarr::unicodeMatchProperty):
125+
(JSC::Yarr::createUnicodeCharacterClassFor):
126+
* yarr/YarrUnicodeProperties.h: Added.
127+
1128
2017-10-09 Commit Queue <commit-queue@webkit.org>
2129

3130
Unreviewed, rolling out r223015 and r223025.

‎Source/JavaScriptCore/DerivedSources.make‎

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -62,6 +62,7 @@ all : \
6262
KeywordLookup.h \
6363
RegExpJitTables.h \
6464
AirOpcode.h \
65+
UnicodePatternTables.h \
6566
yarr/YarrCanonicalizeUnicode.cpp \
6667
WasmOps.h \
6768
WasmValidateInlines.h \
@@ -293,6 +294,9 @@ InjectedScriptSource.h : inspector/InjectedScriptSource.js $(JavaScriptCore_SCRI
293294
AirOpcode.h: $(JavaScriptCore)/b3/air/opcode_generator.rb $(JavaScriptCore)/b3/air/AirOpcode.opcodes
294295
$(RUBY) $^
295296

297+
UnicodePatternTables.h: $(JavaScriptCore)/Scripts/generateYarrUnicodePropertyTables.py $(JavaScriptCore)/ucd/DerivedBinaryProperties.txt $(JavaScriptCore)/ucd/DerivedCoreProperties.txt $(JavaScriptCore)/ucd/DerivedNormalizationProps.txt $(JavaScriptCore)/ucd/PropList.txt $(JavaScriptCore)/ucd/PropertyAliases.txt $(JavaScriptCore)/ucd/PropertyValueAliases.txt $(JavaScriptCore)/ucd/ScriptExtensions.txt $(JavaScriptCore)/ucd/Scripts.txt $(JavaScriptCore)/ucd/UnicodeData.txt $(JavaScriptCore)/ucd/emoji-data.txt
298+
$(PYTHON) $(JavaScriptCore)/Scripts/generateYarrUnicodePropertyTables.py $(JavaScriptCore)/ucd ./UnicodePatternTables.h
299+
296300
yarr/YarrCanonicalizeUnicode.cpp: $(JavaScriptCore)/yarr/generateYarrCanonicalizeUnicode $(JavaScriptCore)/ucd/CaseFolding.txt
297301
$(PYTHON) $(JavaScriptCore)/yarr/generateYarrCanonicalizeUnicode $(JavaScriptCore)/ucd/CaseFolding.txt ./yarr/YarrCanonicalizeUnicode.cpp
298302

0 commit comments

Comments
 (0)