Skip to content

gh-93180: Update documentation of os.copy_file_range #93182

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
vstinner merged 10 commits into from
Jun 8, 2022
Merged

gh-93180: Update documentation of os.copy_file_range #93182

Changes from 1 commit
Commits
Show all changes
10 commits
Select commit Hold shift + click to select a range
c57e76f
Update docs about an old constraint of `os.copy_file_range`
illia-v May 24, 2022
23bbe3a
Add a note about a known issue of `os.copy_file_range`
illia-v May 24, 2022
1e38126
Document some of optimizations `os.copy_file_range` can use implicitly
illia-v May 24, 2022
caeb624
Add a news entry
illia-v May 24, 2022
59561bf
Update the note
illia-v May 30, 2022
bcb339d
Merge branch 'main' into fix-issue-93180
illia-v May 30, 2022
069651d
Explain what copying as if files are opened as binary means
illia-v Jun 6, 2022
f6152e4
Merge branch 'main' into fix-issue-93180
illia-v Jun 7, 2022
94ef9cc
Remove the news entry
illia-v Jun 7, 2022
7ab57e1
Apply Victor's suggestions
illia-v Jun 7, 2022
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
Document some of optimizations os.copy_file_range can use implicitly 
This code is responsible for the documented features in the kernel: https://github.com/torvalds/linux/blob/1e57930e9f4083ad5854ab6eadffe790a8167fb4/fs/read_write.c#L1491-L1507
  • Loading branch information
@illia-v
illia-v committed May 24, 2022
commit 1e381265c3513fa1ab469f09fbadb2090c3c6699
5 changes: 4 additions & 1 deletion Doc/library/os.rst
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -806,7 +806,10 @@ as internal buffering of data.

This copy is done without the additional cost of transferring data
from the kernel to user space and then back into the kernel. Additionally,
some filesystems could implement extra optimizations. The copy is done as if
some filesystems could implement extra optimizations, such as the use of
reflinks (i.e., two or more inodes that share pointers to the same
copy-on-write disk blocks; supported file systems include btrfs and XFS)
and server-side copy (in the case of NFS). The copy is done as if
both files are opened as binary.
Copy link
Copy Markdown
Member

@vstinner vstinner May 28, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The function arguments are file descriptors. What does "binary" mean for file descriptors?

Copy link
Copy Markdown
Contributor Author

@illia-v illia-v May 28, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This sentence is not new. I guess it means that bytes written to a destination file will be equal to ones read from the source file.

In contrary, if a file range is read in the text mode in Python and it is written in the text mode to a different file, bytes may differ. For example, line endings may be changed. See https://docs.python.org/3/tutorial/inputoutput.html#reading-and-writing-files.

Copy link
Copy Markdown
Member

@vstinner vstinner Jun 6, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Would you mind to rephase the sentence to explain that? Explain that the copy does not change the line endings, for example?

Copy link
Copy Markdown
Contributor Author

@illia-v illia-v Jun 6, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done 069651d.

BTW, shutil.copyfile looks to behave the same, but shutil.copyfileobj seems to respect the text mode.


The return value is the amount of bytes copied. This could be less than the
Expand Down