Skip to content

GH-98363: Add itertools.batched() #98364

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
rhettinger merged 24 commits into from
Oct 17, 2022
Merged

GH-98363: Add itertools.batched() #98364

Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
24 commits
Select commit Hold shift + click to select a range
057390d
First draft
rhettinger Oct 17, 2022
047c441
.
rhettinger Oct 17, 2022
9061042
Fix NULL test
rhettinger Oct 17, 2022
72a03c6
Improve the docstring and add the regular docs
rhettinger Oct 17, 2022
21a6f67
.
rhettinger Oct 17, 2022
61b83df
Add tests
rhettinger Oct 17, 2022
93bf013
Add entry to toplevel table in the docs
rhettinger Oct 17, 2022
eb6949c
Remove bogus decref
rhettinger Oct 17, 2022
7fd4016
Flatten before joining
rhettinger Oct 17, 2022
8fb8bdf
Add blurb
rhettinger Oct 17, 2022
2b994a3
Add motivating use case and make equivalent code more accurate
rhettinger Oct 17, 2022
fa9c306
Remove TODO comment
rhettinger Oct 17, 2022
c49fb17
Update Lib/test/test_itertools.py
rhettinger Oct 17, 2022
3fbc9c5
Merge branch 'main' into itertools_batched
rhettinger Oct 17, 2022
4c25341
Neaten-up
rhettinger Oct 17, 2022
41acb45
Make the example more self-explanatory
rhettinger Oct 17, 2022
77de77c
Test pure python equivalent
rhettinger Oct 17, 2022
7390670
Test pure python equivalent
rhettinger Oct 17, 2022
c836bc7
Add GC test
rhettinger Oct 17, 2022
c0ca79d
Add "TestVariousIteratorArgs" tests
rhettinger Oct 17, 2022
8238c28
Fix test case I()
rhettinger Oct 17, 2022
b9cbb46
Remove development note
rhettinger Oct 17, 2022
4799e6a
Remove docstring from pure Python equivalent.
rhettinger Oct 17, 2022
bac2f6c
Fix missing argument in the docs table entry.
rhettinger Oct 17, 2022
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
Neaten-up
  • Loading branch information
@rhettinger
rhettinger committed Oct 17, 2022
commit 4c25341c56b8f3a9eb7df55e1f4964e5b6328522
2 changes: 1 addition & 1 deletion Doc/library/itertools.rst
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -201,7 +201,7 @@ loops that truncate the stream.
"Batch data into lists of length n. The last batch may be shorter."
# batched('ABCDEFG', 3) --> ABC DEF G
Copy link
Copy Markdown
Member

@gvanrossum gvanrossum Oct 23, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe tweak the example so it cannot be misunderstood as "return 3 batches"?

Copy link
Copy Markdown

@anderssonjohan anderssonjohan Oct 25, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
# batched('ABCDEFG', 3) --> ABC DEF G
# list(batched('ABCDEFG', 3)) --> [['A', 'B', 'C'], ['D', 'E', 'F'], ['G']]

if n < 1:
raise ValueError('n must be >= 1')
raise ValueError('n must be at least one')
it = iter(iterable)
while (batch := list(islice(it, n))):
yield batch
Expand Down
8 changes: 6 additions & 2 deletions Modules/itertoolsmodule.c
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -65,8 +65,8 @@ static PyTypeObject pairwise_type;

typedef struct {
PyObject_HEAD
Py_ssize_t batch_size;
PyObject *it;
Py_ssize_t batch_size;
} batchedobject;

/*[clinic input]
Expand Down Expand Up @@ -98,7 +98,11 @@ batched_new_impl(PyTypeObject *type, PyObject *iterable, Py_ssize_t n)
batchedobject *bo;

if (n < 1) {
PyErr_SetString(PyExc_ValueError, "n must be >= 1");
/* We could define the n==0 case to return an empty iterator
but that is add odds with the idea that batching should
never throw-away input data.
*/
PyErr_SetString(PyExc_ValueError, "n must be at least one");
return NULL;
}
it = PyObject_GetIter(iterable);
Expand Down