-
-
Notifications
You must be signed in to change notification settings - Fork 34.7k
bpo-32940: IDLE: Simplify StringTranslatePseudoMapping in pyparse #5862
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Changes from all commits
0d46d85
9472ee9
6e41844
47cd451
437ed57
ef77d35
272f732
909989c
7ad8392
968c4a8
d4e350a
74a9796
405c5f7
867e897
5c7ba8b
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -1,6 +1,6 @@ | ||
| """Define partial Python code Parser used by editor and hyperparser. | ||
|
|
||
| Instances of StringTranslatePseudoMapping are used with str.translate. | ||
| Instances of ParseMap are used with str.translate. | ||
|
|
||
| The following bound search and match functions are defined: | ||
| _synchre - start of popular statement; | ||
|
|
@@ -10,7 +10,6 @@ | |
| _closere - line that must be followed by dedent. | ||
| _chew_ordinaryre - non-special characters. | ||
| """ | ||
| from collections.abc import Mapping | ||
| import re | ||
| import sys | ||
|
|
||
|
|
@@ -101,46 +100,27 @@ | |
| """, re.VERBOSE).match | ||
|
|
||
|
|
||
| class StringTranslatePseudoMapping(Mapping): | ||
| r"""Utility class to be used with str.translate() | ||
| class ParseMap(dict): | ||
| r"""Dict subclass that maps anything not in dict to 'x'. | ||
|
Contributor
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Thought of this while I was sleeping. Should be
Member
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. No. PEP434 declares nearly all of idlelib 'private', and it became such in practice nearly 3 years later, for in 3.6. There is no point to further privatization.
Contributor
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Thanks. |
||
|
|
||
| This Mapping class wraps a given dict. When a value for a key is | ||
| requested via __getitem__() or get(), the key is looked up in the | ||
| given dict. If found there, the value from the dict is returned. | ||
| Otherwise, the default value given upon initialization is returned. | ||
| This is designed to be used with str.translate in study1. | ||
| Anything not specifically mapped otherwise becomes 'x'. | ||
| Example: replace everything except whitespace with 'x'. | ||
|
|
||
| This allows using str.translate() to make some replacements, and to | ||
| replace all characters for which no replacement was specified with | ||
| a given character instead of leaving them as-is. | ||
|
|
||
| For example, to replace everything except whitespace with 'x': | ||
|
|
||
| >>> whitespace_chars = ' \t\n\r' | ||
| >>> preserve_dict = {ord(c): ord(c) for c in whitespace_chars} | ||
| >>> mapping = StringTranslatePseudoMapping(preserve_dict, ord('x')) | ||
| >>> text = "a + b\tc\nd" | ||
| >>> text.translate(mapping) | ||
| >>> keepwhite = ParseMap((ord(c), ord(c)) for c in ' \t\n\r') | ||
| >>> "a + b\tc\nd".translate(keepwhite) | ||
| 'x x x\tx\nx' | ||
| """ | ||
| def __init__(self, non_defaults, default_value): | ||
| self._non_defaults = non_defaults | ||
| self._default_value = default_value | ||
|
|
||
| def _get(key, _get=non_defaults.get, _default=default_value): | ||
| return _get(key, _default) | ||
| self._get = _get | ||
| # Calling this triples access time; see bpo-32940 | ||
| def __missing__(self, key): | ||
| return 120 # ord('x') | ||
|
Member
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Return just It is worth to remove
Member
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. My tests, on the issue, show that ints slightly win. I see no need to change. |
||
|
|
||
| def __getitem__(self, item): | ||
| return self._get(item) | ||
|
|
||
| def __len__(self): | ||
| return len(self._non_defaults) | ||
|
|
||
| def __iter__(self): | ||
| return iter(self._non_defaults) | ||
|
|
||
| def get(self, key, default=None): | ||
| return self._get(key) | ||
| # Map all ascii to 120 to avoid __missing__ call, then replace some. | ||
| trans = ParseMap.fromkeys(range(128), 120) | ||
| trans.update((ord(c), ord('(')) for c in "({[") # open brackets => '('; | ||
| trans.update((ord(c), ord(')')) for c in ")}]") # close brackets => ')'. | ||
| trans.update((ord(c), ord(c)) for c in "\"'\\\n#") # Keep these. | ||
|
|
||
|
|
||
| class Parser: | ||
|
|
@@ -224,16 +204,6 @@ def set_lo(self, lo): | |
| if lo > 0: | ||
| self.code = self.code[lo:] | ||
|
|
||
| # Build a translation table to map uninteresting chars to 'x', open | ||
| # brackets to '(', close brackets to ')' while preserving quotes, | ||
| # backslashes, newlines and hashes. This is to be passed to | ||
| # str.translate() in _study1(). | ||
| _tran = {} | ||
| _tran.update((ord(c), ord('(')) for c in "({[") | ||
| _tran.update((ord(c), ord(')')) for c in ")}]") | ||
| _tran.update((ord(c), ord(c)) for c in "\"'\\\n#") | ||
| _tran = StringTranslatePseudoMapping(_tran, default_value=ord('x')) | ||
|
|
||
| def _study1(self): | ||
| """Find the line numbers of non-continuation lines. | ||
|
|
||
|
|
@@ -250,7 +220,7 @@ def _study1(self): | |
| # uninteresting characters. This can cut the number of chars | ||
| # by a factor of 10-40, and so greatly speed the following loop. | ||
| code = self.code | ||
| code = code.translate(self._tran) | ||
| code = code.translate(trans) | ||
| code = code.replace('xxxxxxxx', 'x') | ||
| code = code.replace('xxxx', 'x') | ||
| code = code.replace('xx', 'x') | ||
|
|
||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1 @@ | ||
| Simplify and rename StringTranslatePseudoMapping in pyparse. |
Uh oh!
There was an error while loading. Please reload this page.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Isn't this class equivalent to
defaultdict(lambda:"x")There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This class doesn't add the keys to the dictionary when they haven't been found. It will return the default value without growing the dictionary.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Also, defaultdict calls the function to get 'x', instead of just returning it.