It is very hard to investigate the performance improvements or degradation without a good benchmark, that is integrated to the CI.
How I see it
- It should be based on JMH; even if it is a Java tool, it should be an easy task to integrate it to the sbt: there is plugin for it;
- It should be possible to run the whole benchmark less that in a 30 minutes, otherwise we may face problems with GitHub limits;
- I don't think that we should run it on each commit, but it is still nice to have and we should at least run it for any commit to the main!
I think we can start from the small-medium sized dataset from LDBC. There are some datasets of the size about 100-500 Mb that sounds fine for me.
@SauronShepherd you mentioned in the mailing list that you are interesting in the performance topic. What do you think about it?
cc: @rjurney
It is very hard to investigate the performance improvements or degradation without a good benchmark, that is integrated to the CI.
How I see it
I think we can start from the small-medium sized dataset from LDBC. There are some datasets of the size about 100-500 Mb that sounds fine for me.
@SauronShepherd you mentioned in the mailing list that you are interesting in the performance topic. What do you think about it?
cc: @rjurney