Fix unicode escapes in jsx identifiers and extended unicode characters in jsdoc#32716
Conversation
| while (isIdentifierPart(text.charCodeAt(pos), ScriptTarget.Latest) && pos < end) { | ||
| pos++; | ||
| let nextChar: number; | ||
| while (isIdentifierPart(nextChar = codePointAt(text, pos), ScriptTarget.Latest) && pos < end) { |
There was a problem hiding this comment.
is it intentional not to allow unicode escape sequences in JSDoc identifiers?
There was a problem hiding this comment.
I believe so, but I dunno. Given that it's a comment, it just means that we'll interpret the \u0061 verbatim. @sandersn would be the authority here.
There was a problem hiding this comment.
I'm confused. Doesn't this change cause us to allow escape sequences in JSDoc identifiers?
jsdoc.app doesn't mention unicode as far as google can tell, so we are free to do whatever we want. I vote that JSDoc identifiers work the same as normal ones, which should mean allowing unicode escape sequences.
| pos++; | ||
| let nextChar: number; | ||
| while (isIdentifierPart(nextChar = codePointAt(text, pos), ScriptTarget.Latest) && pos < end) { | ||
| pos += charSize(nextChar); |
There was a problem hiding this comment.
stupid question: why isn't charSize(c) === 2 when c >= 0x10000 instead of c > 0x10000?
| while (isIdentifierPart(text.charCodeAt(pos), ScriptTarget.Latest) && pos < end) { | ||
| pos++; | ||
| let nextChar: number; | ||
| while (isIdentifierPart(nextChar = codePointAt(text, pos), ScriptTarget.Latest) && pos < end) { |
There was a problem hiding this comment.
I'm confused. Doesn't this change cause us to allow escape sequences in JSDoc identifiers?
jsdoc.app doesn't mention unicode as far as google can tell, so we are free to do whatever we want. I vote that JSDoc identifiers work the same as normal ones, which should mean allowing unicode escape sequences.
Unicode characters now scan as identifiers in jsdoc, escapes are still not parsed. |
No. |
|
That makes sense. It would be nice to have JSDoc support escapes as well, but I think the right way to do that is to make JSDoc use the normal scanner. That's a complete rewrite of the low-level parsing code too, so I'm not sure it's worth it to us. |
|
@sandersn Oh, too late. I already pushed a commit that adds unicode escape support into jsdoc. |
|
I'm pretty sure this deprecates #32631 - can someone double check me? |
|
I don't think so - is jsdoc, at least, we're still going to stop parsing before the hyphen. |
I'm a bit confused by this reply. In JSDoc you shouldn't stop parsing before the hyphen if there's no space. The JSDoc documentation says that it should be able to handle dashes in its member names. For example: That does work in JSDoc from the commandline, but not in VSCode. (See the comments here.) In VSCode, if you hover the mouse over the "node" on the "@typedef" line, it should show: However, instead you get: with the "-activedescendant" part missing. If you put quotes around "aria-activedescendant", then you get this instead: |
Which are the two other places we used
isIdentifierStartin the scanner.Fixes #32701