Add queue_len() option to builder, and adopt crossbeam_channel#104
Add queue_len() option to builder, and adopt crossbeam_channel#104tarka wants to merge 2 commits into
Conversation
|
Thank you for the PR. I will look at it and see how it fits into the the new architecture. |
|
The reason the test fail is because the oldest rustc we support is 1.13 currently. |
|
Thanks for the heads-up, I missed that. Crossbeam Channel requires 1.28.0. I've tried testing with that, but due to a transitive dependency on cfg-if it requires enabling some 2018 features ( If the team are OK with this I'm happy to do the conversion work. |
|
I guess we will switch the crate to For 2.0 I would rise the minimal level to 1.34.2 because it is shipped with debian Would you be interested in taking a look at the 2.0 design? |
|
Sure, I'll take a look this weekend. |
|
I've had a look at it, and it all looks reasonable. Some parts may have a small user-base, so it might be worth making them enabled via feature-flag to avoid blowing out the dependency size for the default usage. Would it be worth converting it into a PR (Rust RFC-style) and pushing to Reddit and the forum for more input? Or would it be better to just start work on a 2.0 branch and get feedback there? |
This can be considered an RFC patch to gauge interest/acceptance.
This adds a
queue_len()option to the builder. This will limit the number of concurrent pending jobs that can be queued; attempts to queue a job when the queue is full will block until a slot becomes free. The main purpose of this feature is to allow back-pressure in the case where pending jobs use up limited resources (memory, file-handles, etc.).In order to do this this in the simplest manner I've moved the pool to use
crossbeam_channel, which has a drop-in bounded channel implementation.There is a probably a strong case for also providing a
try_execute()function that will return an error orfalseif the queue is full, but I've skipped that until this feature has been discussed.