You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: docs/ops/doc/Benchmarks.rst
+10-15Lines changed: 10 additions & 15 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -1,7 +1,6 @@
1
1
SciJava Ops Benchmarks
2
2
======================
3
3
4
-
5
4
This page describes a quantitative analysis of the SciJava Ops framework, and is heavily inspired by a similar comparison of `ImgLib2 <https://imagej.net/libs/imglib2/benchmarks>`_.
6
5
7
6
Hardware and Software
@@ -17,7 +16,7 @@ The following software components were used:
17
16
18
17
* Ubuntu 22.04.3 LTS
19
18
* OpenJDK Runtime Environment (build 11.0.21) with OpenJDK 64-Bit Server VM (build 11.0.21, mixed mode, sharing)
All benchmarks are executed using the `Java Microbenchmark Harness <https://github.com/openjdk/jmh>`_, using the following parameters:
@@ -57,31 +56,29 @@ We first benchmark the base penalty of executing this method using SciJava Ops,
57
56
* Output buffer creation + SciJava Ops invocation
58
57
* SciJava Ops invocation using Op adaptation
59
58
60
-
The results are shown in the chart below:
59
+
The results are shown in **Figure 1**. We find Op execution through the SciJava Ops framework adds a few milliseconds of additional overhead. A few additional milliseconds of overhead are observed when SciJava Ops is additionally tasked with creating an output buffer.
61
60
62
61
.. chart:: ../images/BenchmarkMatching.json
63
62
64
-
Algorithm execution performance (lower is better)
63
+
**Figure 1:** Algorithm execution performance (lower is better)
65
64
66
-
Note that the avove requests are benchmarked without assistance from the Op cache, i.e. they are designed to model the full matching process. As repeated Op requests will utilize the Op cache, we benchmark cached Op retrieval separately, with these results shown below:
65
+
Note that the avove requests are benchmarked without assistance from the Op cache, i.e. they are designed to model the full matching process. As repeated Op requests will utilize the Op cache, we benchmark cached Op retrieval separately, with results shown in **Figure 2**. These benchmarks suggest Op caching helps avoid the additional overhead of Op adaptation as its performance approaches that of normal Op execution.
67
66
68
67
.. chart:: ../images/BenchmarkCaching.json
69
68
70
-
Algorithm execution performance with Op caching (lower is better)
71
-
72
-
We can see that with Op caching, TODO
69
+
**Figure 2:** Algorithm execution performance with Op caching (lower is better)
73
70
74
-
Finally, we benchmark the overhead of SciJava Ops parameter conversion, by considering the situation where we initially have a ``RandomAccessibleInterval<ByteType>``, which we must convert to call our method. In such a situation, we consider the following procedures:
71
+
Finally, we benchmark the overhead of SciJava Ops parameter conversion. Suppose we instead wish to operate upon a ``RandomAccessibleInterval<ByteType>`` - we must convert it to call our Op. We consider the following procedures:
* output buffer creation + SciJava Ops invocation using Op conversion
78
75
* SciJava Ops invocation using Op conversion and Op adaptation
79
76
80
-
The results are shown in the chart below:
77
+
The results are shown in **Figure 3**; note the Op cache is **not** enabled. We observe overheads on the order of 10 milliseconds to perform Op conversion with and without Op adaptation.
81
78
82
79
.. chart:: ../images/BenchmarkConversion.json
83
80
84
-
Algorithm execution performance with Op conversion (lower is better)
81
+
**Figure 3:** Algorithm execution performance with Op conversion (lower is better)
85
82
86
83
Framework Comparison
87
84
--------------------
@@ -108,13 +105,11 @@ We then benchmark the performance of executing this code using the following pat
108
105
* SciJava Ops invocation
109
106
* ImageJ Ops invocation (using a ``Class`` wrapper to make the method discoverable within ImageJ Ops)
110
107
111
-
The results are shown in the chart below:
108
+
The results are shown in **Figure 4**. When algorithm matching dominates execution time, the SciJava Ops matching framework provides significant improvement in matching performance in comparison with the original ImageJ Ops framework.
112
109
113
110
.. chart:: ../images/BenchmarkFrameworks.json
114
111
115
-
Algorithm execution performance by Framework (lower is better)
116
-
117
-
From these results, we can see that, when algorithm matching dominates execution time, the SciJava Ops matching framework provides significant improvement in matching performance in comparison with the original ImageJ Ops framework.
112
+
**Figure 4:** Algorithm execution performance by Framework (lower is better)
0 commit comments