chore(spanner): migrate grpc-gcp into google-cloud-java#13184
chore(spanner): migrate grpc-gcp into google-cloud-java#13184rahul2393 wants to merge 193 commits into
Conversation
There was a problem hiding this comment.
Code Review
This pull request introduces the grpc-gcp library into the google-cloud-java monorepo, including its core implementation for gRPC channel pooling, affinity management, and multi-endpoint support, along with necessary configuration updates in pom.xml and versions.txt. My review identified two potential resource management issues: the use of an unbounded CachedThreadPool for state notifications, which could lead to thread exhaustion, and the static SHARED_BACKGROUND_SERVICE executor, which lacks a proper lifecycle management, potentially causing resource leaks.
| private final ExecutorService stateNotificationExecutor = | ||
| Executors.newCachedThreadPool( | ||
| GcpThreadFactory.newThreadFactory("gcp-mc-state-notifications-%d")); |
There was a problem hiding this comment.
Using Executors.newCachedThreadPool() can lead to thread exhaustion if tasks are submitted faster than they can be processed, as it creates an unbounded number of threads. Consider using a ThreadPoolExecutor with a bounded queue and a fixed or limited number of threads to prevent potential resource exhaustion.
References
- For safety, use a bounded thread pool (e.g., ThreadPoolExecutor with a defined maximum size) instead of an unbounded one (e.g., Executors.newCachedThreadPool()), even if the current logic seems to limit concurrent tasks.
| private static final ScheduledThreadPoolExecutor SHARED_BACKGROUND_SERVICE = | ||
| createSharedBackgroundService(); |
There was a problem hiding this comment.
The SHARED_BACKGROUND_SERVICE is a static ScheduledThreadPoolExecutor that is never shut down. This can lead to a resource leak, especially in environments where the classloader might be reused or multiple instances of GcpManagedChannel are created and destroyed. Consider managing the lifecycle of this executor or using a shared, managed executor service.
5e64f56 to
1409dc3
Compare
| <dependency> | ||
| <groupId>com.google.cloud</groupId> | ||
| <artifactId>google-cloud-shared-dependencies</artifactId> | ||
| <version>${google-cloud-shared-dependencies.version}</version> |
There was a problem hiding this comment.
This would cause circular dependency issues that google-cloud-shared-dependencies manages grpc-gcp version and grpc-gcp also imports google-cloud-shared-dependencies.
Maven build does not error out because it does not validate declared versions in dependencyManagement sections. But it will cause issues in releasing and confusion in dependency management.
I think we have two options:
- Treat
grpc-gcpa top-level base library like auth. It manages its own dependencies (for now). Its version is managed by first-party-dependencies. It is imported through first-party-dependencies by downstream libraries such as Spanner. - Treat is as a sub module of Spanner and remove it from first-party-dependencies. This is a cleaner solution but it would introduce breaking changes to customers. Now my question is: Is there any
grpc-gcpclasses exposed on the surface and used by customers directly? If not, there are probably minimum risks.
|
Can you update CODEOWNERS so that grpc-gcp is owned by the Spanner team? |
There was a problem hiding this comment.
How is this file used? Are the gen code of the this proto file generated during compilation phase and shipped with the jar?
There was a problem hiding this comment.
Yes. grpc_gcp.proto defines ApiConfig, MethodConfig, AffinityConfig, and ChannelPoolConfig. Those generated classes are used directly by the main code. The protobuf-maven-plugin runs during compile and it includes in the shipped jar.
| <parent> | ||
| <groupId>com.google.cloud</groupId> | ||
| <artifactId>google-cloud-shared-config</artifactId> | ||
| <!-- Do not depend on the version in this monorepo. It causes publication order problems. --> |
There was a problem hiding this comment.
This is an issue for auth but not an issue for grpc-grpc. Because the groupId of grpc-gcp is also com.google.cloud
| <junit.version>4.13.2</junit.version> | ||
| <opencensus.version>0.31.1</opencensus.version> | ||
| <opentelemetry.version>1.57.0</opentelemetry.version> | ||
| <spanner.version>6.118.0-SNAPSHOT</spanner.version><!-- {x-version-update:google-cloud-spanner:current} --> |
There was a problem hiding this comment.
Reminder: the Spanner test should be moved to Spanner module right after this PR is merged.
| @@ -0,0 +1,116 @@ | |||
| # gRPC-GCP for Java | |||
|
|
|||
| grpc-gcp provides gRPC support for Google Cloud Clients. | |||
There was a problem hiding this comment.
Add a blurb that this module is only used by Spanner?
| <module>java-gkehub</module> | ||
| <module>java-gkerecommender</module> | ||
| <module>java-grafeas</module> | ||
| <module>java-grpc-gcp</module> |
There was a problem hiding this comment.
I think this module name can stay as grpc-gcp-java. All other java-* are a client library but this is more like an implementation details of the client libraries.
There was a problem hiding this comment.
I can rename it to grpc-gcp-java if preferred, but we would also need to update the root POM generator/metadata flow so root-pom CI keeps it. Otherwise, the generated root POM will drop the module, root-pom CI regenerates pom.xml, and removes non-java-* modules.
There was a problem hiding this comment.
I tested grpc-gcp-java against the current root POM generator image, and it has been dropped from the generated pom.xml. So using that path requires a generator change/release, not just this PR.
There was a problem hiding this comment.
It should be fixed in #13211 now. I would prefer it to stay as grpc-gcp-java, similar to google-auth-library-java
Summary
Migrate grpc-gcp from GoogleCloudPlatform/grpc-gcp-java/grpc-gcp into google-cloud-java/grpc-gcp as a top-level base Maven module with existing com.google.cloud:grpc-gcp coordinates.
What changed
Validation