feat(grpc-gcp): move scale-up to background worker by rahul2393 · Pull Request #14206 · googleapis/google-cloud-java

rahul2393 · 2026-08-28T06:31:46Z

With dynamic scaling enabled, maybeDynamicUpscale() ran synchronously on the caller thread during channel selection, before the stream was counted. It scaled on the pool-wide average only, one channel per trigger, with no cooldown — so a burst could add channels one RPC at a time while the caller waited on channel construction, and a single hot channel never triggered growth if the average stayed low.

Change

Signal after accounting. activeStreamsCountIncr() evaluates the trigger once the stream is reserved: the selected channel above maxRpcPerChannel, or the pool average above it, signals scale-up. Channel selection itself no longer scales.
Single background worker. A one-slot signal (AtomicBoolean pending/running) coalesces bursts; the worker runs on the shared background executor, loops while signals arrive, and catches Throwable so one failure can't stop future scale-ups. The scale-down timer body gets the same guard.
Bounded, load-sized growth. Desired size = ceil(totalActiveStreams / midpoint(minRpc, maxRpc)), added channels capped at maxScaleUpPercent of the active pool (floor 2) and by maxSize. A scaleUpCooldown (default 10s) is claimed before channel construction so signals arriving mid-build are throttled. READY channels awaiting shutdown are reused before new ones are built.
Construction outside the lock. Delegate channels are built without holding the pool monitor; publication rechecks shutdown and headroom under the lock and closes surplus channels.
Inactive channels excluded from round-robin, fallback, extrema (getMin/MaxActiveStreams) and picker scans; the legacy watermark scale-up no longer computes getMaxActiveStreams() on the hot path when dynamic scaling is enabled.
Shutdown ordering: shuttingDown is set under the pool lock as the first step of shutdown()/shutdownNow(), so a channel cannot be published after shutdown begins. removedChannelRefs is now a concurrent set, safe for the worker and the unsynchronized readers (isShutdown, isTerminated, awaitTermination, stats logging).

gemini-code-assist

Code Review

This pull request introduces asynchronous background scale-up logic for GcpManagedChannel to prevent blocking caller threads during channel creation, adding new configuration options like scaleUpCooldown and maxScaleUpPercent along with comprehensive tests. The review feedback highlights critical concurrency issues that need to be addressed: first, recursive retries in submitScaleUpWorker upon executor rejection could block gRPC transport threads or cause a StackOverflowError; second, race conditions in shutdownNow() and shutdown() could allow background channels to be published after shutdown has initiated, leading to silent resource leaks. These state updates should be properly synchronized.

rahul2393 · 2026-08-28T12:30:15Z

/gemini review

gemini-code-assist

Code Review

This pull request introduces an asynchronous background scale-up worker for dynamic scaling in GcpManagedChannel, moving the scale-up logic off the caller thread. It adds new configuration options such as scaleUpCooldown and maxScaleUpPercent to control scaling behavior, ensures inactive channels are skipped during channel selection, and adds comprehensive unit tests. The review feedback identifies critical issues, including a potential ArithmeticException when the channel pool is empty, thread safety concerns in shutdown() and shutdownNow() due to unsynchronized snapshot creation, and exception-handling robustness during channel building. Additionally, it suggests using Google Truth assertions consistently across the new test file for better readability.

rahul2393 · 2026-08-28T13:16:35Z

/gemini review

gemini-code-assist

Code Review

This pull request introduces an asynchronous background scale-up worker to GcpManagedChannel to handle dynamic scaling off the caller thread, along with new configuration options (scaleUpCooldown, maxScaleUpPercent) and robust shutdown handling. It also updates channel selection strategies to skip inactive channels and adds comprehensive unit tests. The review feedback suggests optimizing the performance of monitoring and routing methods (getMinActiveStreams, getMaxActiveStreams, and pickLeastBusyWithFallback) to avoid potential lock contention on the critical path by utilizing atomic variables or thread-safe collections.

olavloite · 2026-08-28T14:15:11Z

 import javax.annotation.Nullable;

 /** A channel management factory that implements grpc.Channel APIs. */
 public class GcpManagedChannel extends ManagedChannel {


For a follow-up pull request: I think that this class needs to be split. It does more than 'just being a managed channel' and is almost 3,000 LoC, which makes it hard to read.

Yes, will do once all changes are queued lands here

olavloite · 2026-08-28T14:22:10Z

  public int getMaxActiveStreams() {
-    return channelRefs.stream().mapToInt(ChannelRef::getActiveStreamsCount).max().orElse(0);
+    return channelRefs.stream()
+        .filter(ChannelRef::isActive)
+        .mapToInt(ChannelRef::getActiveStreamsCount)
+        .max()
+        .orElse(0);
  }


This method allocates a number of different (small) objects on every invocation, so it should preferably only be called on the hot path when it is really necessary. It is currently always called in pickLeastBusyNoFallback() when p2c is used for channel selection, including when dynamic scaling is disabled. Meaning that it is potentially called on every RPC for no good reason. Could we fix that?

Updated it to lazy be called lazily only when DCP is enabled.

rahul2393 · 2026-08-28T18:08:16Z

/gemini review

gemini-code-assist

Code Review

This pull request introduces asynchronous background scale-up logic to GcpManagedChannel to prevent caller threads from blocking during channel creation, alongside new configuration options like scaleUpCooldown and maxScaleUpPercent to control scaling behavior. It also updates channel selection methods to skip inactive channels and improves channel reuse and shutdown handling. The review feedback highlights two critical issues where calling getChannelRef or getChannelRefByAffinityRef directly when the pool is initialized with zero channels (initSize(0)) will fail because channelRefs is empty. It is recommended to call createFirstChannel() at the entry of both methods to ensure the first channel is properly initialized.

olavloite · 2026-08-29T13:25:33Z

+  private Duration scaleUpCooldown = Duration.ofSeconds(10);
  private int scaleDownConsecutiveLowLoadChecks = 3;
-  private int consecutiveLowLoadChecks;
+  private int maxScaleUpPercent = 30;


(Not related to this PR, and not something we should do in this PR, but it came up in my head while reading this): Should we consider an 'emergency load level' where this maxScaleUpPercent is ignored. Meaning: if the system has been idle for a while and has scaled down to its minChannels (e.g. 2), and then gets a sudden burst of traffic, this limit slows down the scale-up. And while that is normally reasonable, would it maybe make sense to say something like 'if the overall load over all channels is >75 streams per channel, then we ignore this cooldown'?

rahul2393 requested review from a team as code owners August 28, 2026 06:31

rahul2393 requested a review from olavloite August 28, 2026 06:31

gemini-code-assist Bot reviewed Aug 28, 2026

View reviewed changes

Comment thread grpc-gcp-java/src/main/java/com/google/cloud/grpc/GcpManagedChannel.java Outdated

Comment thread grpc-gcp-java/src/main/java/com/google/cloud/grpc/GcpManagedChannel.java Outdated

Comment thread grpc-gcp-java/src/main/java/com/google/cloud/grpc/GcpManagedChannel.java Outdated

rahul2393 removed the request for review from olavloite August 28, 2026 06:43

rahul2393 force-pushed the fm/dcp-split-2-scaleup-worker branch 2 times, most recently from f100bbd to 441212f Compare August 28, 2026 12:29

gemini-code-assist Bot reviewed Aug 28, 2026

View reviewed changes

rahul2393 force-pushed the fm/dcp-split-2-scaleup-worker branch from 441212f to d41c30a Compare August 28, 2026 13:15

gemini-code-assist Bot reviewed Aug 28, 2026

View reviewed changes

Comment thread grpc-gcp-java/src/main/java/com/google/cloud/grpc/GcpManagedChannel.java

Comment thread grpc-gcp-java/src/main/java/com/google/cloud/grpc/GcpManagedChannel.java

Comment thread grpc-gcp-java/src/main/java/com/google/cloud/grpc/GcpManagedChannel.java

olavloite reviewed Aug 28, 2026

View reviewed changes

feat(grpc-gcp): move scale-up to background worker

da55725

rahul2393 force-pushed the fm/dcp-split-2-scaleup-worker branch from d41c30a to da55725 Compare August 28, 2026 17:47

rahul2393 requested a review from olavloite August 28, 2026 18:07

rahul2393 mentioned this pull request Aug 28, 2026

fix(grpc-gcp): align the dynamic channel pool with Go DCP to remove one-hot-channel skew #14191

Open

gemini-code-assist Bot reviewed Aug 28, 2026

View reviewed changes

Comment thread grpc-gcp-java/src/main/java/com/google/cloud/grpc/GcpManagedChannel.java

Comment thread grpc-gcp-java/src/main/java/com/google/cloud/grpc/GcpManagedChannel.java

olavloite approved these changes Aug 29, 2026

View reviewed changes

Uh oh!

Conversation

rahul2393 commented Aug 28, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

gemini-code-assist Bot left a comment

Choose a reason for hiding this comment

Code Review

Uh oh!

Uh oh!

Uh oh!

Uh oh!

rahul2393 commented Aug 28, 2026

Uh oh!

gemini-code-assist Bot left a comment

Choose a reason for hiding this comment

Code Review

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

rahul2393 commented Aug 28, 2026

Uh oh!

gemini-code-assist Bot left a comment

Choose a reason for hiding this comment

Code Review

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

olavloite Aug 28, 2026

Choose a reason for hiding this comment

Uh oh!

rahul2393 Aug 28, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

olavloite Aug 28, 2026

Choose a reason for hiding this comment

Uh oh!

rahul2393 Aug 28, 2026

Choose a reason for hiding this comment

Uh oh!

rahul2393 commented Aug 28, 2026

Uh oh!

gemini-code-assist Bot left a comment

Choose a reason for hiding this comment

Code Review

Uh oh!

Uh oh!

Uh oh!

olavloite Aug 29, 2026

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

rahul2393 commented Aug 28, 2026 •

edited

Loading