Skip to content

Commit 1add8f1

Browse files
rahul2393rahul2393
committed
feat(spanner): support user-provided OpenTelemetry for built-in client metrics and endpoint attribute for location-aware routing
1 parent b7a8504 commit 1add8f1

22 files changed

Lines changed: 2041 additions & 104 deletions

‎java-spanner/.readme-partials.yaml‎

Lines changed: 65 additions & 24 deletions
Original file line numberDiff line numberDiff line change
@@ -53,29 +53,70 @@ custom_content: |
5353
## Metrics
5454
5555
Cloud Spanner client supports [client-side metrics](https://cloud.google.com/spanner/docs/view-manage-client-side-metrics) that you can use along with server-side metrics to optimize performance and troubleshoot performance issues if they occur.
56-
57-
Client-side metrics are measured from the time a request leaves your application to the time your application receives the response.
56+
57+
Client-side metrics are measured from the time a request leaves your application to the time your application receives the response.
5858
In contrast, server-side metrics are measured from the time Spanner receives a request until the last byte of data is sent to the client.
59-
60-
These metrics are enabled by default. You can opt out of using client-side metrics with the following code:
61-
59+
60+
The default Cloud Monitoring export for these metrics is enabled by default. You can opt out of the default Cloud Monitoring export with the following code:
61+
6262
```
6363
SpannerOptions options = SpannerOptions.newBuilder()
6464
.setBuiltInMetricsEnabled(false)
6565
.build();
6666
```
67-
68-
You can also disable these metrics by setting `SPANNER_DISABLE_BUILTIN_METRICS` to `true`.
69-
70-
> Note: Client-side metrics needs `monitoring.timeSeries.create` IAM permission to export metrics data. Ask your administrator to grant your service account the [Monitoring Metric Writer](https://cloud.google.com/iam/docs/roles-permissions/monitoring#monitoring.metricWriter) (roles/monitoring.metricWriter) IAM role on the project.
67+
68+
You can also disable the default Cloud Monitoring export by setting `SPANNER_DISABLE_BUILTIN_METRICS` to `true`. These controls affect only the Cloud Monitoring export. They do not affect a caller-owned client-metrics export configured with `CustomOpenTelemetryMetricsProvider`.
69+
70+
> Note: Client-side metrics needs `monitoring.timeSeries.create` IAM permission to export metrics data to Cloud Monitoring. Ask your administrator to grant your service account the [Monitoring Metric Writer](https://cloud.google.com/iam/docs/roles-permissions/monitoring#monitoring.metricWriter) (roles/monitoring.metricWriter) IAM role on the project.
71+
72+
#### Exporting client metrics to OpenTelemetry
73+
74+
Client metrics export to a caller-owned OpenTelemetry destination is controlled by a `MetricsProvider`,
75+
set with `SpannerOptions.Builder.setClientMetricsProvider(MetricsProvider)`. The available
76+
providers are:
77+
78+
* `DefaultMetricsProvider` (the default): no caller-owned client-metrics destination is configured.
79+
The built-in Cloud Monitoring export follows `setBuiltInMetricsEnabled` and the
80+
`SPANNER_DISABLE_BUILTIN_METRICS` environment variable. On Spanner Omni, where the Cloud Monitoring
81+
export is not available, the default provider results in no client-metrics export.
82+
* `NoopMetricsProvider`: caller-owned client metrics are explicitly disabled. The Cloud Monitoring
83+
export is controlled separately.
84+
* `CustomOpenTelemetryMetricsProvider`: the same Spanner client instruments are additionally recorded
85+
on an `OpenTelemetry` instance that you provide. You own the metrics pipeline (readers, exporters
86+
and resource). This custom destination is independent of the built-in Cloud Monitoring export on all
87+
instance types, including Spanner Omni (`InstanceType.OMNI`), for which Cloud Monitoring export is
88+
not available.
89+
90+
Client metrics are not recorded when the client runs against the Spanner emulator, regardless of the
91+
configured `MetricsProvider`. gRPC-layer metrics are recorded on a custom destination only when the
92+
provided `OpenTelemetry` instance is an `OpenTelemetrySdk`.
93+
94+
When using `CustomOpenTelemetryMetricsProvider`, it is recommended to register the Spanner
95+
client-metrics views on a dedicated `SdkMeterProviderBuilder` with
96+
`SpannerMetrics.configureMeterProviderBuilder(SdkMeterProviderBuilder)` before creating the
97+
`OpenTelemetry` instance. The views rename the raw instruments, apply the Spanner latency
98+
histogram buckets and restrict the recorded attributes to the supported client-metric labels. Attempt
99+
metrics include the routed `endpoint` label when Spanner Omni location-aware routing supplies it.
100+
101+
```java
102+
SdkMeterProviderBuilder meterProviderBuilder =
103+
SdkMeterProvider.builder().registerMetricReader(PeriodicMetricReader.create(myExporter));
104+
SpannerMetrics.configureMeterProviderBuilder(meterProviderBuilder);
105+
OpenTelemetry openTelemetry =
106+
OpenTelemetrySdk.builder().setMeterProvider(meterProviderBuilder.build()).build();
107+
SpannerOptions options =
108+
SpannerOptions.newBuilder()
109+
.setClientMetricsProvider(CustomOpenTelemetryMetricsProvider.create(openTelemetry))
110+
.build();
111+
```
71112
72113
## Traces
73-
Cloud Spanner client supports OpenTelemetry Traces, which gives insight into the client internals and aids in debugging/troubleshooting production issues.
114+
Cloud Spanner client supports OpenTelemetry Traces, which gives insight into the client internals and aids in debugging/troubleshooting production issues.
74115
75116
By default, the functionality is disabled. You need to add OpenTelemetry dependencies, enable OpenTelemetry traces and must configure the OpenTelemetry with appropriate exporters at the startup of your application.
76117
77118
See [Configure client-side tracing](https://cloud.google.com/spanner/docs/set-up-tracing#configure-client-side-tracing) for more details on configuring traces.
78-
119+
79120
#### OpenTelemetry Dependencies
80121
81122
If you are using Maven, add this to your pom.xml file
@@ -129,7 +170,7 @@ custom_content: |
129170
130171
Spanner spanner = options.getService();
131172
```
132-
173+
133174
#### OpenTelemetry SQL Statement Tracing
134175
The OpenTelemetry traces that are generated by the Java client include any request and transaction
135176
tags that have been set. The traces can also include the SQL statements that are executed and the
@@ -149,42 +190,42 @@ custom_content: |
149190
#### OpenTelemetry API Tracing
150191
You can enable tracing of each API call that the Spanner client executes with the `enableApiTracing`
151192
option. These traces also include any retry attempts for an API call:
152-
193+
153194
```
154195
SpannerOptions options = SpannerOptions.newBuilder()
155196
.setOpenTelemetry(openTelemetry)
156197
.setEnableApiTracing(true)
157198
.build();
158199
```
159-
200+
160201
This option can also be enabled by setting the environment variable
161202
`SPANNER_ENABLE_API_TRACING=true`.
162203
163204
> Note: The attribute keys that are used for additional information about retry attempts and the number of requests might change in a future release.
164-
165-
#### End-to-end Tracing
166-
205+
206+
#### End-to-end Tracing
207+
167208
In addition to client-side tracing, you can opt in for [end-to-end tracing](https://cloud.google.com/spanner/docs/tracing-overview#end-to-end-side-tracing). End-to-end tracing helps you understand and debug latency issues that are specific to Spanner such as the following:
168209
* Identify whether the latency is due to network latency between your application and Spanner, or if the latency is occurring within Spanner.
169210
* Identify the Google Cloud regions that your application requests are being routed through and if there is a cross-region request. A cross-region request usually means higher latencies between your application and Spanner.
170-
211+
171212
```
172213
SpannerOptions options = SpannerOptions.newBuilder()
173214
.setOpenTelemetry(openTelemetry)
174215
.setEnableEndToEndTracing(true)
175216
.build();
176217
```
177-
218+
178219
Refer to [Configure end-to-end tracing](https://cloud.google.com/spanner/docs/set-up-tracing#configure-end-to-end-tracing) to configure end-to-end tracing and to understand its attributes.
179-
220+
180221
> Note: End-to-end traces can only be exported to [Cloud Trace](https://cloud.google.com/trace/docs).
181-
182-
222+
223+
183224
## Instrument with OpenCensus
184225
185226
> Note: OpenCensus project is deprecated. See [Sunsetting OpenCensus](https://opentelemetry.io/blog/2023/sunsetting-opencensus/).
186227
We recommend migrating to OpenTelemetry, the successor project.
187-
228+
188229
## Migrate from OpenCensus to OpenTelemetry
189230
190231
> Using the [OpenTelemetry OpenCensus Bridge](https://mvnrepository.com/artifact/io.opentelemetry/opentelemetry-opencensus-shim), you can immediately begin exporting your metrics and traces with OpenTelemetry.
@@ -214,4 +255,4 @@ custom_content: |
214255
215256
Update your dashboards and alerts to reflect below changes
216257
* **Metrics name** : `cloud.google.com/java` prefix has been removed from OpenTelemery metrics and instead has been added as Instrumenation Scope.
217-
* **Metrics namespace** : OpenTelmetry exporters uses `workload.googleapis.com` namespace opposed to `custom.googleapis.com` with OpenCensus.
258+
* **Metrics namespace** : OpenTelmetry exporters uses `workload.googleapis.com` namespace opposed to `custom.googleapis.com` with OpenCensus.

‎java-spanner/README.md‎

Lines changed: 44 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -155,17 +155,58 @@ Cloud Spanner client supports [client-side metrics](https://cloud.google.com/spa
155155
Client-side metrics are measured from the time a request leaves your application to the time your application receives the response.
156156
In contrast, server-side metrics are measured from the time Spanner receives a request until the last byte of data is sent to the client.
157157

158-
These metrics are enabled by default. You can opt out of using client-side metrics with the following code:
158+
The default Cloud Monitoring export for these metrics is enabled by default. You can opt out of the default Cloud Monitoring export with the following code:
159159

160160
```
161161
SpannerOptions options = SpannerOptions.newBuilder()
162162
.setBuiltInMetricsEnabled(false)
163163
.build();
164164
```
165165

166-
You can also disable these metrics by setting `SPANNER_DISABLE_BUILTIN_METRICS` to `true`.
166+
You can also disable the default Cloud Monitoring export by setting `SPANNER_DISABLE_BUILTIN_METRICS` to `true`. These controls affect only the Cloud Monitoring export. They do not affect a caller-owned client-metrics export configured with `CustomOpenTelemetryMetricsProvider`.
167167

168-
> Note: Client-side metrics needs `monitoring.timeSeries.create` IAM permission to export metrics data. Ask your administrator to grant your service account the [Monitoring Metric Writer](https://cloud.google.com/iam/docs/roles-permissions/monitoring#monitoring.metricWriter) (roles/monitoring.metricWriter) IAM role on the project.
168+
> Note: Client-side metrics needs `monitoring.timeSeries.create` IAM permission to export metrics data to Cloud Monitoring. Ask your administrator to grant your service account the [Monitoring Metric Writer](https://cloud.google.com/iam/docs/roles-permissions/monitoring#monitoring.metricWriter) (roles/monitoring.metricWriter) IAM role on the project.
169+
170+
#### Exporting client metrics to OpenTelemetry
171+
172+
Client metrics export to a caller-owned OpenTelemetry destination is controlled by a `MetricsProvider`,
173+
set with `SpannerOptions.Builder.setClientMetricsProvider(MetricsProvider)`. The available
174+
providers are:
175+
176+
* `DefaultMetricsProvider` (the default): no caller-owned client-metrics destination is configured.
177+
The built-in Cloud Monitoring export follows `setBuiltInMetricsEnabled` and the
178+
`SPANNER_DISABLE_BUILTIN_METRICS` environment variable. On Spanner Omni, where the Cloud Monitoring
179+
export is not available, the default provider results in no client-metrics export.
180+
* `NoopMetricsProvider`: caller-owned client metrics are explicitly disabled. The Cloud Monitoring
181+
export is controlled separately.
182+
* `CustomOpenTelemetryMetricsProvider`: the same Spanner client instruments are additionally recorded
183+
on an `OpenTelemetry` instance that you provide. You own the metrics pipeline (readers, exporters
184+
and resource). This custom destination is independent of the built-in Cloud Monitoring export on all
185+
instance types, including Spanner Omni (`InstanceType.OMNI`), for which Cloud Monitoring export is
186+
not available.
187+
188+
Client metrics are not recorded when the client runs against the Spanner emulator, regardless of the
189+
configured `MetricsProvider`. gRPC-layer metrics are recorded on a custom destination only when the
190+
provided `OpenTelemetry` instance is an `OpenTelemetrySdk`.
191+
192+
When using `CustomOpenTelemetryMetricsProvider`, it is recommended to register the Spanner
193+
client-metrics views on a dedicated `SdkMeterProviderBuilder` with
194+
`SpannerMetrics.configureMeterProviderBuilder(SdkMeterProviderBuilder)` before creating the
195+
`OpenTelemetry` instance. The views rename the raw instruments, apply the Spanner latency
196+
histogram buckets and restrict the recorded attributes to the supported client-metric labels. Attempt
197+
metrics include the routed `endpoint` label when Spanner Omni location-aware routing supplies it.
198+
199+
```java
200+
SdkMeterProviderBuilder meterProviderBuilder =
201+
SdkMeterProvider.builder().registerMetricReader(PeriodicMetricReader.create(myExporter));
202+
SpannerMetrics.configureMeterProviderBuilder(meterProviderBuilder);
203+
OpenTelemetry openTelemetry =
204+
OpenTelemetrySdk.builder().setMeterProvider(meterProviderBuilder.build()).build();
205+
SpannerOptions options =
206+
SpannerOptions.newBuilder()
207+
.setClientMetricsProvider(CustomOpenTelemetryMetricsProvider.create(openTelemetry))
208+
.build();
209+
```
169210

170211
## Traces
171212
Cloud Spanner client supports OpenTelemetry Traces, which gives insight into the client internals and aids in debugging/troubleshooting production issues.

0 commit comments

Comments
 (0)