DPL Analysis: preliminary work for multiple process() functions - #6019
Conversation
6de3b63 to
020c84d
Compare
|
@ktf @jgrosseo this is a working example of multiple process functions in an analysis task. Process functions can have different signatures, only the selected functions will affect what the task subscribes to. The only caveat currently is that filters are applied for each of the process functions, so I'll have to add some caching behavior for selections. |
ktf
left a comment
There was a problem hiding this comment.
Beautiful. Apart from the capitalisation, I have nothing else to add.
|
Excellent! This was very fast. I am in favour of merging this. Next, we can look at how to make this more than trivially configurable. For instance: |
|
It still needs filter caching, otherwise it would be a performance problem. Related to that is the question whether we also want to have optionally different filters for different processes. Configuration like that is not trivial, we'll have to discuss what we can do. |
|
why performance problem? doing the filters twice is what would happen right now with two devices, no? |
|
It seems adding proper caching also requires also changing how |
|
@aalkin I think the filter caching should anyway go beyond the same process. Our idea is to pull this into a common process if there are several tasks (in independent processes) having the same filter, right? |
At a baseline we need to cache the filter result (row list) in a task, so that it can be supplied to each process() in the task that requires same-ish argument. For example if all process() functions require some sort of join based on |
@ktf