Skip to content

Correct regular expression flags scanning for non-BMP characters #58612

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
rbuckton merged 5 commits into from
Jun 4, 2024
Merged

Correct regular expression flags scanning for non-BMP characters #58612

Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
5 commits
Select commit Hold shift + click to select a range
0725a3a
Correct flags scanning for non-BMP characters
graphemecluster May 22, 2024
f91f83f
Merge branch 'main' into regex-non-bmp-flags
graphemecluster May 30, 2024
406468c
Optimization: Lookup by `CharacterCodes`
graphemecluster May 30, 2024
723b1d6
Add remarks to test case file
rbuckton May 31, 2024
15783d6
Merge branch 'main' into regex-non-bmp-flags
graphemecluster May 31, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
Add remarks to test case file
  • Loading branch information
@rbuckton @graphemecluster
rbuckton authored and graphemecluster committed May 31, 2024
commit 723b1d68c6c7409ddac5a9267efee9089324d283
18 changes: 12 additions & 6 deletions tests/baselines/reference/regularExpressionWithNonBMPFlags.errors.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,12 +1,18 @@
regularExpressionWithNonBMPFlags.ts(1,23): error TS1499: Unknown regular expression flag.
regularExpressionWithNonBMPFlags.ts(1,25): error TS1499: Unknown regular expression flag.
regularExpressionWithNonBMPFlags.ts(1,28): error TS1499: Unknown regular expression flag.
regularExpressionWithNonBMPFlags.ts(1,41): error TS1499: Unknown regular expression flag.
regularExpressionWithNonBMPFlags.ts(1,43): error TS1499: Unknown regular expression flag.
regularExpressionWithNonBMPFlags.ts(1,45): error TS1499: Unknown regular expression flag.
regularExpressionWithNonBMPFlags.ts(7,23): error TS1499: Unknown regular expression flag.
regularExpressionWithNonBMPFlags.ts(7,25): error TS1499: Unknown regular expression flag.
regularExpressionWithNonBMPFlags.ts(7,28): error TS1499: Unknown regular expression flag.
regularExpressionWithNonBMPFlags.ts(7,41): error TS1499: Unknown regular expression flag.
regularExpressionWithNonBMPFlags.ts(7,43): error TS1499: Unknown regular expression flag.
regularExpressionWithNonBMPFlags.ts(7,45): error TS1499: Unknown regular expression flag.


==== regularExpressionWithNonBMPFlags.ts (6 errors) ====
// The characters in the following regular expression are ASCII-lookalike characters found in Unicode, including:
// - 𝘴 (U+1D634 Mathematical Sans-Serif Italic Small S)
// - 𝘪 (U+1D62A Mathematical Sans-Serif Italic Small I)
// - 𝘮 (U+1D62E Mathematical Sans-Serif Italic Small M)
//
// See https://en.wikipedia.org/wiki/Mathematical_Alphanumeric_Symbols
const 𝘳𝘦𝘨𝘦𝘹 = /(?𝘴𝘪-𝘮:^𝘧𝘰𝘰.)/𝘨𝘮𝘶;
~~
!!! error TS1499: Unknown regular expression flag.
Expand Down
12 changes: 12 additions & 0 deletions tests/baselines/reference/regularExpressionWithNonBMPFlags.js
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,8 +1,20 @@
//// [tests/cases/compiler/regularExpressionWithNonBMPFlags.ts] ////

//// [regularExpressionWithNonBMPFlags.ts]
// The characters in the following regular expression are ASCII-lookalike characters found in Unicode, including:
// - 𝘴 (U+1D634 Mathematical Sans-Serif Italic Small S)
// - 𝘪 (U+1D62A Mathematical Sans-Serif Italic Small I)
// - 𝘮 (U+1D62E Mathematical Sans-Serif Italic Small M)
//
// See https://en.wikipedia.org/wiki/Mathematical_Alphanumeric_Symbols
const 𝘳𝘦𝘨𝘦𝘹 = /(?𝘴𝘪-𝘮:^𝘧𝘰𝘰.)/𝘨𝘮𝘶;


//// [regularExpressionWithNonBMPFlags.js]
// The characters in the following regular expression are ASCII-lookalike characters found in Unicode, including:
// - 𝘴 (U+1D634 Mathematical Sans-Serif Italic Small S)
// - 𝘪 (U+1D62A Mathematical Sans-Serif Italic Small I)
// - 𝘮 (U+1D62E Mathematical Sans-Serif Italic Small M)
//
// See https://en.wikipedia.org/wiki/Mathematical_Alphanumeric_Symbols
const 𝘳𝘦𝘨𝘦𝘹 = /(?𝘴𝘪-𝘮:^𝘧𝘰𝘰.)/𝘨𝘮𝘶;
8 changes: 7 additions & 1 deletion tests/baselines/reference/regularExpressionWithNonBMPFlags.symbols
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,6 +1,12 @@
//// [tests/cases/compiler/regularExpressionWithNonBMPFlags.ts] ////

=== regularExpressionWithNonBMPFlags.ts ===
// The characters in the following regular expression are ASCII-lookalike characters found in Unicode, including:
// - 𝘴 (U+1D634 Mathematical Sans-Serif Italic Small S)
// - 𝘪 (U+1D62A Mathematical Sans-Serif Italic Small I)
// - 𝘮 (U+1D62E Mathematical Sans-Serif Italic Small M)
//
// See https://en.wikipedia.org/wiki/Mathematical_Alphanumeric_Symbols
const 𝘳𝘦𝘨𝘦𝘹 = /(?𝘴𝘪-𝘮:^𝘧𝘰𝘰.)/𝘨𝘮𝘶;
>𝘳𝘦𝘨𝘦𝘹 : Symbol(𝘳𝘦𝘨𝘦𝘹, Decl(regularExpressionWithNonBMPFlags.ts, 0, 5))
>𝘳𝘦𝘨𝘦𝘹 : Symbol(𝘳𝘦𝘨𝘦𝘹, Decl(regularExpressionWithNonBMPFlags.ts, 6, 5))

6 changes: 6 additions & 0 deletions tests/baselines/reference/regularExpressionWithNonBMPFlags.types
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,6 +1,12 @@
//// [tests/cases/compiler/regularExpressionWithNonBMPFlags.ts] ////

=== regularExpressionWithNonBMPFlags.ts ===
// The characters in the following regular expression are ASCII-lookalike characters found in Unicode, including:
// - 𝘴 (U+1D634 Mathematical Sans-Serif Italic Small S)
// - 𝘪 (U+1D62A Mathematical Sans-Serif Italic Small I)
// - 𝘮 (U+1D62E Mathematical Sans-Serif Italic Small M)
//
// See https://en.wikipedia.org/wiki/Mathematical_Alphanumeric_Symbols
const 𝘳𝘦𝘨𝘦𝘹 = /(?𝘴𝘪-𝘮:^𝘧𝘰𝘰.)/𝘨𝘮𝘶;
>𝘳𝘦𝘨𝘦𝘹 : RegExp
> : ^^^^^^
Expand Down
6 changes: 6 additions & 0 deletions tests/cases/compiler/regularExpressionWithNonBMPFlags.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,3 +1,9 @@
// @target: esnext

// The characters in the following regular expression are ASCII-lookalike characters found in Unicode, including:
// - 𝘴 (U+1D634 Mathematical Sans-Serif Italic Small S)
// - 𝘪 (U+1D62A Mathematical Sans-Serif Italic Small I)
// - 𝘮 (U+1D62E Mathematical Sans-Serif Italic Small M)
//
// See https://en.wikipedia.org/wiki/Mathematical_Alphanumeric_Symbols
const 𝘳𝘦𝘨𝘦𝘹 = /(?𝘴𝘪-𝘮:^𝘧𝘰𝘰.)/𝘨𝘮𝘶;
Comment thread
graphemecluster marked this conversation as resolved.