Skip to content

gh-104719: IDLE - test existence of all tokenize references. #104767

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
terryjreedy merged 14 commits into from
May 24, 2023
Merged

gh-104719: IDLE - test existence of all tokenize references. #104767

Changes from 1 commit
Commits
Show all changes
14 commits
Select commit Hold shift + click to select a range
446ddc4
gh-104719: IDLE - test editor.IndentSearcher
terryjreedy May 22, 2023
cb348d8
Add IndentSearcher test.
terryjreedy May 22, 2023
1338a2a
Add new coverage and generate TokenError.
terryjreedy May 22, 2023
d3bac6c
Add minimal test for other tokenize names.
terryjreedy May 22, 2023
7828a83
Merge branch 'main' into toktest
Yhg1s May 22, 2023
c6f59fb
Update Lib/idlelib/editor.py
terryjreedy May 23, 2023
0a99cf0
Update Lib/idlelib/idle_test/test_iomenu.py
terryjreedy May 23, 2023
ec7388e
Add blurb and IDLE news.
terryjreedy May 23, 2023
77f66be
Merge remote-tracking branch 'upstream/main' into toktest
terryjreedy May 23, 2023
7eb9e3c
Update merge.
terryjreedy May 23, 2023
698d55d
Merge branch 'toktest' of https://github.com/terryjreedy/cpython into…
terryjreedy May 23, 2023
bc197be
Revert "Update merge."
terryjreedy May 23, 2023
1d616cc
Merge branch 'main' into toktest
terryjreedy May 23, 2023
d13b811
Merge branch 'main' into toktest
AlexWaygood May 24, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Next Next commit
gh-104719: IDLE - test editor.IndentSearcher 
This class contains all editor references to tokenize module.
  • Loading branch information
@terryjreedy
terryjreedy committed May 22, 2023
commit 446ddc461f99c9fb6dad986a0288226c1693e01e
18 changes: 10 additions & 8 deletions Lib/idlelib/editor.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -1571,7 +1571,7 @@ def reindent_to(self, column):
# blocks are found).

def guess_indent(self):
opener, indented = IndentSearcher(self.text, self.tabwidth).run()
opener, indented = IndentSearcher(self.text).run()
if opener and indented:
raw, indentsmall = get_line_indent(opener, self.tabwidth)
raw, indentlarge = get_line_indent(indented, self.tabwidth)
Expand Down Expand Up @@ -1609,13 +1609,9 @@ def get_line_indent(line, tabwidth):


class IndentSearcher:
"Manage initial indent guess, returned by run method."

# .run() chews over the Text widget, looking for a block opener
# and the stmt following it. Returns a pair,
# (line containing block opener, line containing stmt)
# Either or both may be None.

def __init__(self, text, tabwidth):
def __init__(self, text):
self.text = text
self.tabwidth = tabwidth
self.i = self.finished = 0
Expand All @@ -1633,7 +1629,8 @@ def readline(self):
def tokeneater(self, type, token, start, end, line,
INDENT=tokenize.INDENT,
NAME=tokenize.NAME,
OPENERS=('class', 'def', 'for', 'if', 'try', 'while')):
OPENERS=('class', 'def', 'for', 'if', 'match','try',
Comment thread
terryjreedy marked this conversation as resolved.
Outdated
'while', 'with')):
if self.finished:
pass
elif type == NAME and token in OPENERS:
Expand All @@ -1643,6 +1640,10 @@ def tokeneater(self, type, token, start, end, line,
self.finished = 1

def run(self):
"""Return 2 lines containing block opener and and indent.

Either or both may instead be None.
"""
try:
tokens = tokenize.generate_tokens(self.readline)
for token in tokens:
Expand All @@ -1654,6 +1655,7 @@ def run(self):

### end autoindent code ###


def prepstr(s):
"""Extract the underscore from a string.

Expand Down