Skip to content

Fully implement buffered IO #2323

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
coolreader18 merged 20 commits into from
Nov 7, 2020
Merged

Fully implement buffered IO #2323

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
20 commits
Select commit Hold shift + click to select a range
6ba5880
Add a PyThreadMutex lock type
coolreader18 Oct 12, 2020
7beb932
Distinguish between to_index and to_index_opt
coolreader18 Oct 12, 2020
0745725
Implement buffered io sans buffering
coolreader18 Oct 12, 2020
a1ca535
Make bufferoptions.format a Cow<'static, str>
coolreader18 Oct 13, 2020
5f07081
Make PyType.tp_name() more efficient
coolreader18 Oct 13, 2020
473d538
Make Buffered{Reader,Writer} actually buffered
coolreader18 Oct 13, 2020
ff55a2e
WIP seek
coolreader18 Oct 14, 2020
3d19d97
Expose classes as attributes
coolreader18 Oct 28, 2020
4db7a23
Add/rename vm.obj_len, obj_len_opt
coolreader18 Nov 6, 2020
374a956
Misc changes; exception chaining, PyBytesLike::into_buffer
coolreader18 Nov 6, 2020
5ddb678
Add memory::RcBuffer
coolreader18 Nov 6, 2020
0dc8154
Finish up buffered io
coolreader18 Nov 6, 2020
402594f
Update io.py to CPython 3.8.3
coolreader18 Nov 6, 2020
3f6504c
Add _pyio, test.test_io from CPython 3.8.3
coolreader18 Nov 6, 2020
d2a302c
Skip tests in test_io
coolreader18 Nov 6, 2020
1d6b32c
Fix clippy/wasm errors
coolreader18 Nov 6, 2020
60d917c
Don't lock textio while calling buffer methods
coolreader18 Nov 6, 2020
76fa447
Only define buffered.flush for writer/random
coolreader18 Nov 6, 2020
916bb4c
Fix some other tests
coolreader18 Nov 6, 2020
9433126
Skip test_io on windows
coolreader18 Nov 7, 2020
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .github/workflows/ci.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -123,7 +123,7 @@ jobs:
test_argparse test_json test_bytes test_long test_pwd test_bool test_cgi test_complex
test_exception_hierarchy test_glob test_iter test_list test_os test_pathlib
test_py_compile test_set test_shutil test_sys test_unicode test_unittest test_venv
test_zipimport test_importlib
test_zipimport test_importlib test_io
env:
RUSTPYTHONPATH: ${{ github.workspace }}/Lib
if: runner.os == 'Windows'
Expand Down
2,685 changes: 2,685 additions & 0 deletions Lib/_pyio.py
View file
Open in desktop

Large diffs are not rendered by default.

17 changes: 11 additions & 6 deletions Lib/io.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -41,8 +41,8 @@
"Amaury Forgeot d'Arc <amauryfa@gmail.com>, "
"Benjamin Peterson <benjamin@python.org>")

__all__ = ["BlockingIOError", "open", "IOBase", "RawIOBase", "FileIO",
"BytesIO", "StringIO", "BufferedIOBase",
__all__ = ["BlockingIOError", "open", "open_code", "IOBase", "RawIOBase",
"FileIO", "BytesIO", "StringIO", "BufferedIOBase",
"BufferedReader", "BufferedWriter", "BufferedRWPair",
"BufferedRandom", "TextIOBase", "TextIOWrapper",
"UnsupportedOperation", "SEEK_SET", "SEEK_CUR", "SEEK_END"]
Expand All @@ -51,12 +51,17 @@
import _io
import abc

from _io import *
from _io import (DEFAULT_BUFFER_SIZE, BlockingIOError, UnsupportedOperation,
open, open_code, FileIO, BytesIO, StringIO, BufferedReader,
BufferedWriter, BufferedRWPair, BufferedRandom,
# XXX RUSTPYTHON TODO: IncrementalNewlineDecoder
# IncrementalNewlineDecoder, TextIOWrapper)
TextIOWrapper)

OpenWrapper = _io.open # for compatibility with _pyio

# Pretend this exception was created here.
#UnsupportedOperation.__module__ = "io"
UnsupportedOperation.__module__ = "io"

# for seek()
SEEK_SET = 0
Expand All @@ -83,8 +88,8 @@ class TextIOBase(_io._TextIOBase, IOBase):
except NameError:
pass

for klass in (BytesIO, BufferedReader, BufferedWriter):#, BufferedRandom,
#BufferedRWPair):
for klass in (BytesIO, BufferedReader, BufferedWriter, BufferedRandom,
BufferedRWPair):
BufferedIOBase.register(klass)

for klass in (StringIO, TextIOWrapper):
Expand Down
4 changes: 0 additions & 4 deletions Lib/test/test_cgi.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -203,8 +203,6 @@ def test_log(self):
self.addCleanup(cgi.closelog)
cgi.log("Testing log 4")

# TODO RUSTPYTHON
@unittest.expectedFailure
def test_fieldstorage_readline(self):
# FieldStorage uses readline, which has the capacity to read all
# contents of the input file into memory; we use readline's size argument
Expand Down Expand Up @@ -294,8 +292,6 @@ def test_fieldstorage_multipart_non_ascii(self):
got = getattr(fs.list[x], k)
self.assertEqual(got, exp)

# TODO RUSTPYTHON
@unittest.expectedFailure
def test_fieldstorage_multipart_maxline(self):
# Issue #18167
maxline = 1 << 16
Expand Down
Loading