Skip to content

GH-89727: Fix shutil.rmtree() recursion error on deep trees #119808

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
barneygale merged 13 commits into from
Jun 1, 2024
Merged

GH-89727: Fix shutil.rmtree() recursion error on deep trees #119808

Changes from 1 commit
Commits
Show all changes
13 commits
Select commit Hold shift + click to select a range
417b829
Tidy up `_rmtree_safe_fd()` implementation.
barneygale May 30, 2024
8d42ea8
Rewrite using list as stack.
barneygale May 30, 2024
7018370
Add NEWS
barneygale May 30, 2024
9f13194
Enable test
barneygale May 30, 2024
aee87db
Use DirEntry.stat() where possible.
barneygale May 30, 2024
d9d0386
Comments, naming, line limit.
barneygale May 30, 2024
4daf2f2
Restore comment.
barneygale May 30, 2024
a335a63
Simplify overall diff
barneygale May 30, 2024
1d4772a
Address some review feedback
barneygale May 31, 2024
462cfc1
Handle further errors in finally: block
barneygale May 31, 2024
8d7ba83
Comments.
barneygale May 31, 2024
11deeab
Handle os.close explicitly
barneygale May 31, 2024
a53e473
Skip os.unlink() if os.DirEntry.is_dir() raises FileNotFoundError
barneygale May 31, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
Handle os.close explicitly
  • Loading branch information
@barneygale
barneygale committed May 31, 2024
commit 11deeabb9dfa27b64d00dcfaca610bcb36f5cff4
15 changes: 4 additions & 11 deletions Lib/shutil.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -646,9 +646,8 @@ def _rmtree_safe_fd(stack, onexc):
# * path: Path of file to operate upon. This is passed to onexc() if an
# error occurs.
# * orig_entry: os.DirEntry, or None if we're processing the top-level
# directory given to rmtree(), or the func is os.close. We used the
# cached stat() of the entry to save a call to os.lstat() when walking
# interior directories.
# directory given to rmtree(). We used the cached stat() of the entry to
# save a call to os.lstat() when walking subdirectories.
func, dirfd, path, orig_entry = stack.pop()
Comment thread
ambv marked this conversation as resolved.
name = path if orig_entry is None else orig_entry.name
try:
Expand All @@ -675,15 +674,9 @@ def _rmtree_safe_fd(stack, onexc):
if not os.path.samestat(orig_st, os.fstat(topfd)):
# Symlinks to directories are forbidden, see GH-46010.
raise OSError("Cannot call rmtree on a symbolic link")
# Schedule the directory to be removed, now that we've established
# it isn't a symlink.
stack.append((os.rmdir, dirfd, path, orig_entry))
finally:
# Schedule the file descriptor to be closed. We omit orig_entry so
# that any FileNotFoundError from os.close() is reported to
# onexc(); normally FileNotFoundError is suppressed for operations
# on subdirectories, but os.close() is an exception.
stack.append((os.close, topfd, path, None))
stack.append((os.close, topfd, path, orig_entry))

func = os.scandir # For error reporting.
with os.scandir(topfd) as scandir_it:
Expand All @@ -704,7 +697,7 @@ def _rmtree_safe_fd(stack, onexc):
except OSError as err:
onexc(os.unlink, fullname, err)
except FileNotFoundError as err:
if orig_entry is None:
if orig_entry is None or func is os.close:
err.filename = path
onexc(func, path, err)
except OSError as err:
Expand Down