Skip to content

gh-89727: Fix pathlib.Path.walk RecursionError on deep trees #100282

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
barneygale merged 20 commits into from
Mar 22, 2023
Merged

gh-89727: Fix pathlib.Path.walk RecursionError on deep trees #100282

Changes from 1 commit
Commits
Show all changes
20 commits
Select commit Hold shift + click to select a range
037a717
Replace recursion with iteration
zmievsa Dec 15, 2022
f8968b6
Unify walk and _walk methods
zmievsa Dec 15, 2022
ecabf8c
Remove empty typehint
zmievsa Dec 15, 2022
6ec3c3e
Fix all tests
zmievsa Dec 16, 2022
fb5a4c7
Add test for recursion limit
zmievsa Dec 16, 2022
55beb6e
📜🤖 Added by blurb_it.
blurb-it[bot] Dec 16, 2022
7f36140
Merge branch 'main' into gh-89727/fix-pathlib.Path.walk-recursion-depth
zmievsa Dec 16, 2022
508bd4d
Fix recursion limit issue in different envs
zmievsa Dec 16, 2022
2bfc47d
Merge branch 'gh-89727/fix-pathlib.Path.walk-recursion-depth' of gith…
zmievsa Dec 16, 2022
20476ff
Make everything consistent with os.walk
zmievsa Dec 17, 2022
4f2de62
Remove unused import
zmievsa Dec 20, 2022
6313b6a
Merge branch 'main' into gh-89727/fix-pathlib.Path.walk-recursion-depth
zmievsa Dec 20, 2022
17497f0
Replace the usage of infinite_recursion with set_recursion_limit
zmievsa Dec 20, 2022
2543448
Merge branch 'main' into gh-89727/fix-pathlib.Path.walk-recursion-depth
zmievsa Jan 1, 2023
5609e4f
Code review fixes
zmievsa Jan 1, 2023
b7d36de
Update Lib/pathlib.py
Ovsyanka83 Jan 2, 2023
d428a89
Code review fixes
zmievsa Jan 4, 2023
4ce5291
Merge branch 'main' into gh-89727/fix-pathlib.Path.walk-recursion-depth
zmievsa Jan 11, 2023
49d4237
Apply suggestions from code review
Ovsyanka83 Jan 21, 2023
9bc2500
Rename stack to paths
zmievsa Mar 19, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
Fix all tests
  • Loading branch information
@zmievsa
zmievsa committed Dec 16, 2022
commit 6ec3c3e084151d6923cb45fca401bbd53b07c656
15 changes: 7 additions & 8 deletions Lib/pathlib.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -1283,8 +1283,8 @@ def walk(self, top_down=True, on_error=None, follow_symlinks=False):
stack = deque(((False, self),))

while stack:
is_result, top = stack.pop()
if is_result:
must_be_yielded_immediately, top = stack.pop()
if must_be_yielded_immediately:
yield top
continue

Expand All @@ -1294,7 +1294,7 @@ def walk(self, top_down=True, on_error=None, follow_symlinks=False):
# minor reason when (say) a thousand readable directories are still
# left to visit. That logic is copied here.
try:
scandir_it = self._scandir()
scandir_it = top._scandir()
except OSError as error:
if on_error is not None:
on_error(error)
Expand All @@ -1316,15 +1316,14 @@ def walk(self, top_down=True, on_error=None, follow_symlinks=False):
filenames.append(entry.name)

if top_down:
yield top, dirnames, filenames
else:
stack.append((True, (top, dirnames, filenames)))

for dirname in dirnames:
dirpath = self._make_child_relpath(dirname)
for dirname in reversed(dirnames):
Comment thread
barneygale marked this conversation as resolved.
Outdated
dirpath = top._make_child_relpath(dirname)
stack.append((False, dirpath))

if not top_down:
stack.append((True, (top, dirnames, filenames)))


class PosixPath(Path, PurePosixPath):
"""Path subclass for non-Windows systems.
Expand Down