New: AbstractQuerySet.__Iter__ is no more greedy#389
Conversation
The __iter__ now yields until the end of actual page, than start to fetch the next page.
|
Hi @alanjds, thanks for your contribution! In order for us to evaluate and accept your PR, we ask that you sign a contribution license agreement. It's all electronic and will take just minutes. Sincerely, |
|
Thank you @alanjds for signing the Contribution License Agreement. Cheers, |
|
Thanks a lot for the contribution! We will have a cqlengine-focused release next, so I'll be looking at this then. |
|
Thanks for the information, @aholmberg. However, I still think that this is not an "improvement", but a "bugfix". I really would expect __ iter __ to fetch on demand. Otherwise it leads to the pointed annoyances: Timeouts and memory inflation. |
On https://datastax.github.io/python-driver/query_paging.html#handling-paged-results is told how to iterate ove large querysets since 2.6.0 .
However, the cqlengine querysets are fetching all pages before yielding the first result. This leads to timeouts or memory inflation on clients.
This PR makes __ iter__ become a non-greedy generator, yielding until the actual page is exhausted, and then fetching the next one.