Skip to content

Commit b0c337b

Browse files
asafmjack-berg
asafm
and
jack-berg
authored
Memory Mode support: Adding memory mode, and implementing it for Asynchronous Instruments (open-telemetry#5709)
Co-authored-by: jack-berg <34418638+jack-berg@users.noreply.github.com>
1 parent 9a93155 commit b0c337b

47 files changed

Lines changed: 2620 additions & 156 deletions

File tree

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

‎CONTRIBUTING.md‎

Lines changed: 3 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -8,7 +8,7 @@ requirements and recommendations.
88

99
If you want to add new features or change behavior, please make sure your changes follow the
1010
[OpenTelemetry Specification](https://github.com/open-telemetry/opentelemetry-specification).
11-
Otherwise file an issue or submit a PR to the specification repo first.
11+
Otherwise, file an issue or submit a PR to the specification repo first.
1212

1313
Make sure to review the projects [license](LICENSE) and sign the
1414
[CNCF CLA](https://identity.linuxfoundation.org/projects/cncf). A signed CLA will be enforced by an
@@ -52,7 +52,8 @@ $ ./gradlew check
5252

5353
Note: this gradle task will potentially generate changes to files in
5454
the `docs/apidiffs/current_vs_latest`
55-
directory. Please make sure to include any changes to these files in your pull request.
55+
directory. Please make sure to include any changes to these files in your pull request (i.e.
56+
add those files to your commits in the PR).
5657

5758
## PR Review
5859

‎dependencyManagement/build.gradle.kts‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -45,6 +45,7 @@ val DEPENDENCIES = listOf(
4545
"io.opencensus:opencensus-contrib-exemplar-util:${opencensusVersion}",
4646
"org.openjdk.jmh:jmh-core:${jmhVersion}",
4747
"org.openjdk.jmh:jmh-generator-bytecode:${jmhVersion}",
48+
"org.openjdk.jmh:jmh-generator-annprocess:${jmhVersion}",
4849
"org.mockito:mockito-core:${mockitoVersion}",
4950
"org.mockito:mockito-junit-jupiter:${mockitoVersion}",
5051
"org.slf4j:slf4j-simple:${slf4jVersion}",

‎docs/apidiffs/current_vs_latest/opentelemetry-sdk-common.txt‎

Lines changed: 10 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,2 +1,11 @@
11
Comparing source compatibility of against
2-
No changes.
2+
+++ NEW ENUM: PUBLIC(+) FINAL(+) io.opentelemetry.sdk.common.export.MemoryMode (compatible)
3+
+++ CLASS FILE FORMAT VERSION: 52.0 <- n.a.
4+
+++ NEW INTERFACE: java.lang.constant.Constable
5+
+++ NEW INTERFACE: java.lang.Comparable
6+
+++ NEW INTERFACE: java.io.Serializable
7+
+++ NEW SUPERCLASS: java.lang.Enum
8+
+++ NEW FIELD: PUBLIC(+) STATIC(+) FINAL(+) io.opentelemetry.sdk.common.export.MemoryMode REUSABLE_DATA
9+
+++ NEW FIELD: PUBLIC(+) STATIC(+) FINAL(+) io.opentelemetry.sdk.common.export.MemoryMode IMMUTABLE_DATA
10+
+++ NEW METHOD: PUBLIC(+) STATIC(+) io.opentelemetry.sdk.common.export.MemoryMode valueOf(java.lang.String)
11+
+++ NEW METHOD: PUBLIC(+) STATIC(+) io.opentelemetry.sdk.common.export.MemoryMode[] values()

‎docs/apidiffs/current_vs_latest/opentelemetry-sdk-metrics.txt‎

Lines changed: 9 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,2 +1,10 @@
11
Comparing source compatibility of against
2-
No changes.
2+
*** MODIFIED INTERFACE: PUBLIC ABSTRACT io.opentelemetry.sdk.metrics.export.MetricExporter (not serializable)
3+
=== CLASS FILE FORMAT VERSION: 52.0 <- 52.0
4+
+++ NEW METHOD: PUBLIC(+) io.opentelemetry.sdk.common.export.MemoryMode getMemoryMode()
5+
*** MODIFIED INTERFACE: PUBLIC ABSTRACT io.opentelemetry.sdk.metrics.export.MetricReader (not serializable)
6+
=== CLASS FILE FORMAT VERSION: 52.0 <- 52.0
7+
+++ NEW METHOD: PUBLIC(+) io.opentelemetry.sdk.common.export.MemoryMode getMemoryMode()
8+
*** MODIFIED CLASS: PUBLIC FINAL io.opentelemetry.sdk.metrics.export.PeriodicMetricReader (not serializable)
9+
=== CLASS FILE FORMAT VERSION: 52.0 <- 52.0
10+
+++ NEW METHOD: PUBLIC(+) io.opentelemetry.sdk.common.export.MemoryMode getMemoryMode()

‎docs/apidiffs/current_vs_latest/opentelemetry-sdk-testing.txt‎

Lines changed: 11 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,2 +1,12 @@
11
Comparing source compatibility of against
2-
No changes.
2+
*** MODIFIED CLASS: PUBLIC io.opentelemetry.sdk.testing.exporter.InMemoryMetricReader (not serializable)
3+
=== CLASS FILE FORMAT VERSION: 52.0 <- 52.0
4+
+++ NEW METHOD: PUBLIC(+) STATIC(+) io.opentelemetry.sdk.testing.exporter.InMemoryMetricReaderBuilder builder()
5+
+++ NEW METHOD: PUBLIC(+) io.opentelemetry.sdk.common.export.MemoryMode getMemoryMode()
6+
+++ NEW CLASS: PUBLIC(+) FINAL(+) io.opentelemetry.sdk.testing.exporter.InMemoryMetricReaderBuilder (not serializable)
7+
+++ CLASS FILE FORMAT VERSION: 52.0 <- n.a.
8+
+++ NEW SUPERCLASS: java.lang.Object
9+
+++ NEW METHOD: PUBLIC(+) io.opentelemetry.sdk.testing.exporter.InMemoryMetricReader build()
10+
+++ NEW METHOD: PUBLIC(+) io.opentelemetry.sdk.testing.exporter.InMemoryMetricReaderBuilder setAggregationTemporalitySelector(io.opentelemetry.sdk.metrics.export.AggregationTemporalitySelector)
11+
+++ NEW METHOD: PUBLIC(+) io.opentelemetry.sdk.testing.exporter.InMemoryMetricReaderBuilder setDefaultAggregationSelector(io.opentelemetry.sdk.metrics.export.DefaultAggregationSelector)
12+
+++ NEW METHOD: PUBLIC(+) io.opentelemetry.sdk.testing.exporter.InMemoryMetricReaderBuilder setMemoryMode(io.opentelemetry.sdk.common.export.MemoryMode)

‎sdk/common/src/main/java/io/opentelemetry/sdk/common/export/MemoryMode.java‎

Lines changed: 26 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,26 @@
1+
/*
2+
* Copyright The OpenTelemetry Authors
3+
* SPDX-License-Identifier: Apache-2.0
4+
*/
5+
6+
package io.opentelemetry.sdk.common.export;
7+
8+
/** The memory semantics of the SDK. */
9+
public enum MemoryMode {
10+
11+
/**
12+
* Reuses objects to reduce allocations.
13+
*
14+
* <p>In this mode, the SDK reuses objects to reduce allocations, at the expense of disallowing
15+
* concurrent collections / exports.
16+
*/
17+
REUSABLE_DATA,
18+
19+
/**
20+
* Uses immutable data structures.
21+
*
22+
* <p>In this mode, the SDK passes immutable objects to exporters / readers, increasing
23+
* allocations but ensuring safe concurrent exports.
24+
*/
25+
IMMUTABLE_DATA
26+
}

‎sdk/metrics/build.gradle.kts‎

Lines changed: 7 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -41,6 +41,13 @@ testing {
4141
}
4242
}
4343
}
44+
register<JvmTestSuite>("jmhBasedTest") {
45+
dependencies {
46+
implementation("org.openjdk.jmh:jmh-core")
47+
implementation("org.openjdk.jmh:jmh-generator-bytecode")
48+
annotationProcessor("org.openjdk.jmh:jmh-generator-annprocess")
49+
}
50+
}
4451
}
4552
}
4653

‎sdk/metrics/src/jmhBasedTest/java/io/opentelemetry/sdk/metrics/internal/state/AsynchronousMetricStorageGarbageCollectionBenchmark.java‎

Lines changed: 128 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,128 @@
1+
/*
2+
* Copyright The OpenTelemetry Authors
3+
* SPDX-License-Identifier: Apache-2.0
4+
*/
5+
6+
package io.opentelemetry.sdk.metrics.internal.state;
7+
8+
import io.opentelemetry.api.common.Attributes;
9+
import io.opentelemetry.sdk.common.export.MemoryMode;
10+
import io.opentelemetry.sdk.metrics.Aggregation;
11+
import io.opentelemetry.sdk.metrics.SdkMeterProvider;
12+
import io.opentelemetry.sdk.metrics.SdkMeterProviderBuilder;
13+
import io.opentelemetry.sdk.metrics.data.AggregationTemporality;
14+
import io.opentelemetry.sdk.metrics.export.PeriodicMetricReader;
15+
import io.opentelemetry.sdk.metrics.internal.SdkMeterProviderUtil;
16+
import io.opentelemetry.sdk.metrics.internal.exemplar.ExemplarFilter;
17+
import java.time.Duration;
18+
import java.util.List;
19+
import java.util.Random;
20+
import java.util.concurrent.TimeUnit;
21+
import org.openjdk.jmh.annotations.Benchmark;
22+
import org.openjdk.jmh.annotations.BenchmarkMode;
23+
import org.openjdk.jmh.annotations.Fork;
24+
import org.openjdk.jmh.annotations.Measurement;
25+
import org.openjdk.jmh.annotations.Mode;
26+
import org.openjdk.jmh.annotations.OutputTimeUnit;
27+
import org.openjdk.jmh.annotations.Param;
28+
import org.openjdk.jmh.annotations.Scope;
29+
import org.openjdk.jmh.annotations.Setup;
30+
import org.openjdk.jmh.annotations.State;
31+
import org.openjdk.jmh.annotations.TearDown;
32+
import org.openjdk.jmh.annotations.Threads;
33+
import org.openjdk.jmh.annotations.Warmup;
34+
35+
/**
36+
* Run this through {@link AsynchronousMetricStorageGarbageCollectionBenchmarkTest}, as it runs it
37+
* embedded with the GC profiler which what this test designed for (No need for command line run)
38+
*
39+
* <p>This test creates 10 asynchronous counters (any asynchronous instrument will do as the code
40+
* path is almost the same for all async instrument types), and 1000 attribute sets. Each time the
41+
* test runs, it calls `flush` which effectively calls the callback for each counter. Each such
42+
* callback records a random number for each of the 1000 attribute sets. The result list ends up in
43+
* {@link NoopMetricExporter} which does nothing with it.
44+
*
45+
* <p>This is repeated 100 times, collectively called Operation in the statistics and each such
46+
* operation is repeated 20 times - known as Iteration.
47+
*
48+
* <p>Each such test is repeated, with a brand new JVM, for all combinations of {@link MemoryMode}
49+
* and {@link AggregationTemporality}. This is done since each combination has a different code
50+
* path.
51+
*/
52+
@BenchmarkMode(Mode.SingleShotTime)
53+
@OutputTimeUnit(TimeUnit.NANOSECONDS)
54+
@Measurement(iterations = 20, batchSize = 100)
55+
@Warmup(iterations = 10, batchSize = 10)
56+
@Fork(1)
57+
public class AsynchronousMetricStorageGarbageCollectionBenchmark {
58+
59+
@State(value = Scope.Benchmark)
60+
@SuppressWarnings("SystemOut")
61+
public static class ThreadState {
62+
private final int cardinality;
63+
private final int countersCount;
64+
@Param public AggregationTemporality aggregationTemporality;
65+
@Param public MemoryMode memoryMode;
66+
SdkMeterProvider sdkMeterProvider;
67+
private final Random random = new Random();
68+
List<Attributes> attributesList;
69+
70+
/** Creates a ThreadState. */
71+
@SuppressWarnings("unused")
72+
public ThreadState() {
73+
cardinality = 1000;
74+
countersCount = 10;
75+
}
76+
77+
@SuppressWarnings("SpellCheckingInspection")
78+
@Setup
79+
public void setup() {
80+
PeriodicMetricReader metricReader =
81+
PeriodicMetricReader.builder(
82+
// Configure an exporter that configures the temporality and aggregation
83+
// for the test case, but otherwise drops the data on export
84+
new NoopMetricExporter(aggregationTemporality, Aggregation.sum(), memoryMode))
85+
// Effectively disable periodic reading so reading is only done on #flush()
86+
.setInterval(Duration.ofSeconds(Integer.MAX_VALUE))
87+
.build();
88+
SdkMeterProviderBuilder builder = SdkMeterProvider.builder();
89+
SdkMeterProviderUtil.registerMetricReaderWithCardinalitySelector(
90+
builder, metricReader, unused -> cardinality + 1);
91+
92+
attributesList = AttributesGenerator.generate(cardinality);
93+
94+
// Disable examplars
95+
SdkMeterProviderUtil.setExemplarFilter(builder, ExemplarFilter.alwaysOff());
96+
97+
sdkMeterProvider = builder.build();
98+
for (int i = 0; i < countersCount; i++) {
99+
sdkMeterProvider
100+
.get("meter")
101+
.counterBuilder("counter" + i)
102+
.buildWithCallback(
103+
observableLongMeasurement -> {
104+
for (int j = 0; j < attributesList.size(); j++) {
105+
Attributes attributes = attributesList.get(j);
106+
observableLongMeasurement.record(random.nextInt(10_000), attributes);
107+
}
108+
});
109+
}
110+
}
111+
112+
@TearDown
113+
public void tearDown() {
114+
sdkMeterProvider.shutdown().join(10, TimeUnit.SECONDS);
115+
}
116+
}
117+
118+
/**
119+
* Collects all asynchronous instruments metric data.
120+
*
121+
* @param threadState thread-state
122+
*/
123+
@Benchmark
124+
@Threads(value = 1)
125+
public void recordAndCollect(ThreadState threadState) {
126+
threadState.sdkMeterProvider.forceFlush().join(10, TimeUnit.SECONDS);
127+
}
128+
}

‎sdk/metrics/src/jmhBasedTest/java/io/opentelemetry/sdk/metrics/internal/state/AsynchronousMetricStorageGarbageCollectionBenchmarkTest.java‎

Lines changed: 106 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,106 @@
1+
/*
2+
* Copyright The OpenTelemetry Authors
3+
* SPDX-License-Identifier: Apache-2.0
4+
*/
5+
6+
package io.opentelemetry.sdk.metrics.internal.state;
7+
8+
import static org.assertj.core.api.Assertions.assertThat;
9+
10+
import io.opentelemetry.sdk.common.InstrumentationScopeInfo;
11+
import io.opentelemetry.sdk.common.export.MemoryMode;
12+
import io.opentelemetry.sdk.metrics.data.AggregationTemporality;
13+
import io.opentelemetry.sdk.resources.Resource;
14+
import java.util.Collection;
15+
import java.util.HashMap;
16+
import java.util.Map;
17+
import org.assertj.core.data.Offset;
18+
import org.junit.jupiter.api.Assumptions;
19+
import org.junit.jupiter.api.Test;
20+
import org.openjdk.jmh.infra.BenchmarkParams;
21+
import org.openjdk.jmh.results.BenchmarkResult;
22+
import org.openjdk.jmh.results.Result;
23+
import org.openjdk.jmh.results.RunResult;
24+
import org.openjdk.jmh.runner.Runner;
25+
import org.openjdk.jmh.runner.RunnerException;
26+
import org.openjdk.jmh.runner.options.Options;
27+
import org.openjdk.jmh.runner.options.OptionsBuilder;
28+
29+
public class AsynchronousMetricStorageGarbageCollectionBenchmarkTest {
30+
31+
/**
32+
* This test validates that in {@link MemoryMode#REUSABLE_DATA}, {@link
33+
* AsynchronousMetricStorage#collect(Resource, InstrumentationScopeInfo, long, long)} barely
34+
* allocates memory which is then subsequently garbage collected. It is done so comparatively to
35+
* {@link MemoryMode#IMMUTABLE_DATA},
36+
*
37+
* <p>It runs the JMH test {@link AsynchronousMetricStorageGarbageCollectionBenchmark} with GC
38+
* profiler, and measures for each parameter combination the garbage collector normalized rate
39+
* (bytes allocated per Operation).
40+
*
41+
* <p>Memory allocations can be hidden even at an innocent foreach loop on a collection, which
42+
* under the hood allocates an internal object O(N) times. Someone can accidentally refactor such
43+
* loop, resulting in 30% increase of garbage collected objects during a single collect() run.
44+
*/
45+
@SuppressWarnings("rawtypes")
46+
@Test
47+
public void normalizedAllocationRateTest() throws RunnerException {
48+
// GitHub CI has an environment variable (CI=true). We can use it to skip
49+
// this test since it's a lengthy one (roughly 10 seconds) and have it running
50+
// only in GitHub CI
51+
Assumptions.assumeTrue(
52+
"true".equals(System.getenv("CI")),
53+
"This test should only run in GitHub CI since it's long");
54+
55+
// Runs AsynchronousMetricStorageMemoryProfilingBenchmark
56+
// with garbage collection profiler
57+
Options opt =
58+
new OptionsBuilder()
59+
.include(AsynchronousMetricStorageGarbageCollectionBenchmark.class.getSimpleName())
60+
.addProfiler("gc")
61+
.shouldFailOnError(true)
62+
.jvmArgs("-Xmx1500m")
63+
.build();
64+
Collection<RunResult> results = new Runner(opt).run();
65+
66+
// Collect the normalized GC allocation rate per parameters combination
67+
Map<String, Map<String, Double>> resultMap = new HashMap<>();
68+
for (RunResult result : results) {
69+
for (BenchmarkResult benchmarkResult : result.getBenchmarkResults()) {
70+
BenchmarkParams benchmarkParams = benchmarkResult.getParams();
71+
72+
String memoryMode = benchmarkParams.getParam("memoryMode");
73+
String aggregationTemporality = benchmarkParams.getParam("aggregationTemporality");
74+
assertThat(memoryMode).isNotNull();
75+
assertThat(aggregationTemporality).isNotNull();
76+
77+
Map<String, Result> secondaryResults = benchmarkResult.getSecondaryResults();
78+
Result allocRateNorm = secondaryResults.get("gc.alloc.rate.norm");
79+
assertThat(allocRateNorm)
80+
.describedAs("Allocation rate in secondary results: %s", secondaryResults)
81+
.isNotNull();
82+
83+
resultMap
84+
.computeIfAbsent(aggregationTemporality, k -> new HashMap<>())
85+
.put(memoryMode, allocRateNorm.getScore());
86+
}
87+
}
88+
89+
assertThat(resultMap).hasSameSizeAs(AggregationTemporality.values());
90+
91+
// Asserts that reusable data GC allocation rate is a tiny fraction of immutable data
92+
// GC allocation rate
93+
resultMap.forEach(
94+
(aggregationTemporality, memoryModeToAllocRateMap) -> {
95+
Double immutableDataAllocRate =
96+
memoryModeToAllocRateMap.get(MemoryMode.IMMUTABLE_DATA.toString());
97+
Double reusableDataAllocRate =
98+
memoryModeToAllocRateMap.get(MemoryMode.REUSABLE_DATA.toString());
99+
100+
assertThat(immutableDataAllocRate).isNotNull().isNotZero();
101+
assertThat(reusableDataAllocRate).isNotNull().isNotZero();
102+
assertThat(100 - (reusableDataAllocRate / immutableDataAllocRate) * 100)
103+
.isCloseTo(99.8, Offset.offset(2.0));
104+
});
105+
}
106+
}

‎sdk/metrics/src/jmhBasedTest/java/io/opentelemetry/sdk/metrics/internal/state/AttributesGenerator.java‎

Lines changed: 45 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,45 @@
1+
/*
2+
* Copyright The OpenTelemetry Authors
3+
* SPDX-License-Identifier: Apache-2.0
4+
*/
5+
6+
package io.opentelemetry.sdk.metrics.internal.state;
7+
8+
import io.opentelemetry.api.common.Attributes;
9+
import java.util.ArrayList;
10+
import java.util.HashSet;
11+
import java.util.List;
12+
import java.util.Random;
13+
14+
public class AttributesGenerator {
15+
16+
private AttributesGenerator() {}
17+
18+
/**
19+
* Generates a list of unique attributes, with a single attribute key, and random value.
20+
*
21+
* @param uniqueAttributesCount The amount of unique attribute sets to generate
22+
* @return The list of generates {@link Attributes}
23+
*/
24+
public static List<Attributes> generate(int uniqueAttributesCount) {
25+
Random random = new Random();
26+
HashSet<String> attributeSet = new HashSet<>();
27+
ArrayList<Attributes> attributesList = new ArrayList<>(uniqueAttributesCount);
28+
String last = "aaaaaaaaaaaaaaaaaaaaaaaaaa";
29+
for (int i = 0; i < uniqueAttributesCount; i++) {
30+
char[] chars = last.toCharArray();
31+
int attempts = 0;
32+
do {
33+
chars[random.nextInt(last.length())] = (char) (random.nextInt(26) + 'a');
34+
} while (attributeSet.contains(new String(chars)) && ++attempts < 1000);
35+
if (attributeSet.contains(new String(chars))) {
36+
throw new IllegalStateException("Couldn't create new random attributes");
37+
}
38+
last = new String(chars);
39+
attributesList.add(Attributes.builder().put("key", last).build());
40+
attributeSet.add(last);
41+
}
42+
43+
return attributesList;
44+
}
45+
}

0 commit comments

Comments
 (0)