tranport: Reduce heap allocs by setting BufferSlice capacity#8797
tranport: Reduce heap allocs by setting BufferSlice capacity#8797arjan-bal merged 3 commits intogrpc:masterfrom
Conversation
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## master #8797 +/- ##
==========================================
+ Coverage 83.30% 83.36% +0.06%
==========================================
Files 418 418
Lines 32897 33010 +113
==========================================
+ Hits 27404 27520 +116
+ Misses 4093 4088 -5
- Partials 1400 1402 +2
🚀 New features to boost your workflow:
|
|
The 50th percentile latency has gone up in the benchmarks: What was the motivation behind this change? |
The minor regression (-0.3% QPS, +0.73% latency) is statistically insignificant. I verified across multiple runs that results vary by less than 1%. Technically, pre-allocating the BufferSlice is more efficient than initializing with zero capacity and growing it incrementally, so the performance impact should be neutral or positive. This should be true until gRPC supports larger data frames.
While working on replacing the |
The change sets an initial estimate for the `mem.BufferSlice` capacity
while reading messages to avoid reallocations for messages larger than
16KB.
## Benchmarks
```sh
$ go run benchmark/benchresult/main.go streaming-before streaming-after
streaming-networkMode_Local-bufConn_true-keepalive_false-benchTime_1m0s-trace_false-latency_0s-kbps_0-MTU_0-maxConcurrentC
alls_120-reqSize_16500B-respSize_16500B-compressor_off-channelz_false-preloader_false-clientReadBufferSize_-1-clientWriteB
ufferSize_-1-serverReadBufferSize_-1-serverWriteBufferSize_-1-sleepBetweenRPCs_0s-connections_1-recvBufferPool_simple-shar
edWriteBuffer_false
Title Before After Percentage
TotalOps 5040044 5024089 -0.32%
SendOps 0 0 NaN%
RecvOps 0 0 NaN%
Bytes/op 75499.82 75446.71 -0.07%
Allocs/op 26.24 24.22 -7.62%
ReqT/op 11088096800.00 11052995800.00 -0.32%
RespT/op 11088096800.00 11052995800.00 -0.32%
50th-Lat 1.215848ms 1.22471ms 0.73%
90th-Lat 2.258081ms 2.25736ms -0.03%
99th-Lat 2.78024ms 2.756574ms -0.85%
Avg-Lat 1.427389ms 1.431989ms 0.32%
GoVersion go1.24.8 go1.24.8
GrpcVersion 1.79.0-dev 1.79.0-dev
```
RELEASE NOTES:
* transport: Reduce slice re-allocations by reserving slice capacity.
The change sets an initial estimate for the
mem.BufferSlicecapacity while reading messages to avoid reallocations for messages larger than 16KB.Benchmarks
$ go run benchmark/benchresult/main.go streaming-before streaming-after streaming-networkMode_Local-bufConn_true-keepalive_false-benchTime_1m0s-trace_false-latency_0s-kbps_0-MTU_0-maxConcurrentC alls_120-reqSize_16500B-respSize_16500B-compressor_off-channelz_false-preloader_false-clientReadBufferSize_-1-clientWriteB ufferSize_-1-serverReadBufferSize_-1-serverWriteBufferSize_-1-sleepBetweenRPCs_0s-connections_1-recvBufferPool_simple-shar edWriteBuffer_false Title Before After Percentage TotalOps 5040044 5024089 -0.32% SendOps 0 0 NaN% RecvOps 0 0 NaN% Bytes/op 75499.82 75446.71 -0.07% Allocs/op 26.24 24.22 -7.62% ReqT/op 11088096800.00 11052995800.00 -0.32% RespT/op 11088096800.00 11052995800.00 -0.32% 50th-Lat 1.215848ms 1.22471ms 0.73% 90th-Lat 2.258081ms 2.25736ms -0.03% 99th-Lat 2.78024ms 2.756574ms -0.85% Avg-Lat 1.427389ms 1.431989ms 0.32% GoVersion go1.24.8 go1.24.8 GrpcVersion 1.79.0-dev 1.79.0-devRELEASE NOTES: