Skip to content

IteratorByteStream: split large iterator chunks into 64 KB pieces #3752

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
Treadgold wants to merge 2 commits into
base: master
Choose a base branch
from
+32 −2
Open

IteratorByteStream: split large iterator chunks into 64 KB pieces #3752

Changes from 1 commit
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
a48f2ab
IteratorByteStream: split large iterator chunks into 64 KB pieces
Treadgold Jan 31, 2026
fc2d503
IteratorByteStream: split large iterator chunks into 64KB pieces
Treadgold Jan 31, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Next Next commit
IteratorByteStream: split large iterator chunks into 64 KB pieces
  • Loading branch information
@Treadgold
Treadgold committed Jan 31, 2026
commit a48f2abda079afb3deda64340715864bb4119f90
10 changes: 7 additions & 3 deletions httpx/_content.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -59,10 +59,14 @@ def __iter__(self) -> Iterator[bytes]:
yield chunk
chunk = self._stream.read(self.CHUNK_SIZE)
else:
# Otherwise iterate.
# Otherwise iterate, splitting large chunks.
for part in self._stream:
yield part

# Split large chunks into CHUNK_SIZE pieces
offset = 0
while offset < len(part):
chunk_size = min(self.CHUNK_SIZE, len(part) - offset)
yield part[offset : offset + chunk_size]
offset += chunk_size

class AsyncIteratorByteStream(AsyncByteStream):
CHUNK_SIZE = 65_536
Expand Down