Skip to content

bpo-30377: Simplify handling of COMMENT and NL in tokenize.py #1607

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
serhiy-storchaka merged 4 commits into from
May 24, 2017
Merged

bpo-30377: Simplify handling of COMMENT and NL in tokenize.py #1607

Changes from 1 commit
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
a6f464a
add test to explicitly excerisce branches
albertjan May 16, 2017
dabd33f
simplify check for newline or comment
albertjan May 16, 2017
6212509
changes due to comments by @serhiy-storchaka
albertjan May 16, 2017
feaed9d
fix comment, cleaner diff
albertjan May 16, 2017
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
fix comment, cleaner diff
  • Loading branch information
@albertjan
albertjan committed May 16, 2017
commit feaed9d51217ba6406e0ef3fa7791cb1ea6c38c5
2 changes: 1 addition & 1 deletion Lib/tokenize.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -557,7 +557,7 @@ def _tokenize(readline, encoding):
if pos == max:
break

if line[pos] in '#\r\n': # skip comment and blank lines and
if line[pos] in '#\r\n': # skip comments or blank lines
if line[pos] == '#':
comment_token = line[pos:].rstrip('\r\n')
yield TokenInfo(COMMENT, comment_token,
Expand Down