Skip to content

GH-119169: Speed up os.walk(topdown=False) #119186

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
barneygale wants to merge 12 commits into from
Closed

GH-119169: Speed up os.walk(topdown=False) #119186

Changes from 1 commit
Commits
Show all changes
12 commits
Select commit Hold shift + click to select a range
74a495c
GH-119169: Speed up `os.walk()`
barneygale May 19, 2024
eb4b6ff
Merge branch 'main' into os-walk-speedup
barneygale May 23, 2024
46a3ba5
Simplify bottom-up implementation.
barneygale May 23, 2024
2d957a8
Merge branch 'main' into os-walk-speedup
barneygale May 30, 2024
a638202
Apply the same changes to fwalk()
barneygale May 30, 2024
91b5a78
Simplify code slightly
barneygale May 30, 2024
b32c1c0
Simplify exception handling in fwalk()
barneygale Jul 6, 2024
8c7cf04
Merge branch 'main' into os-walk-speedup
barneygale Jul 6, 2024
efd31e2
Undo trailing slash optimisation (moved to another PR)
barneygale Jul 6, 2024
5b89c57
Undo fwalk() changes.
barneygale Jul 6, 2024
525618c
Fix news entry.
barneygale Jul 6, 2024
d3af655
Merge branch 'main' into os-walk-speedup
barneygale Jul 8, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
Undo trailing slash optimisation (moved to another PR)
  • Loading branch information
@barneygale
barneygale committed Jul 6, 2024
commit efd31e2ef5ce2c8d5132d165bef861b8dca6ae90
18 changes: 8 additions & 10 deletions Lib/os.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -410,16 +410,14 @@ def walk(top, topdown=True, onerror=None, followlinks=False):
# Yield before sub-directory traversal if going top down
yield top, dirs, nondirs
# Traverse into sub-directories
if dirs:
prefix = join(top, top[:0])
for dirname in reversed(dirs):
new_path = prefix + dirname
# bpo-23605: os.path.islink() is used instead of caching
# entry.is_symlink() result during the loop on os.scandir() because
# the caller can replace the directory entry during the "yield"
# above.
if followlinks or not islink(new_path):
stack.append(new_path)
for dirname in reversed(dirs):
new_path = join(top, dirname)
# bpo-23605: os.path.islink() is used instead of caching
# entry.is_symlink() result during the loop on os.scandir() because
# the caller can replace the directory entry during the "yield"
# above.
if followlinks or not islink(new_path):
stack.append(new_path)

__all__.append("walk")

Expand Down