Repository navigation
migrate estimator protobuf to use standard protoc-gen-go - #7298
Conversation
Summary of ChangesHello, I'm Gemini Code Assist1! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed! This pull request modernizes the scheduler-estimator's protobuf implementation to ensure its long-term compatibility with evolving Kubernetes APIs, specifically addressing changes in Kubernetes 1.35 and beyond. The core change involves transitioning to standard protobuf generation tools and introducing a dual-field strategy for Kubernetes object serialization. This approach allows for direct binary handling of Kubernetes types while preserving backward compatibility for existing components, facilitating a smooth upgrade path. Highlights
Using Gemini Code AssistThe full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips. Invoking Gemini You can request assistance from Gemini at any point by creating a comment using either
Customization To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a Limitations & Feedback Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for GitHub and other Google products, sign up here. Footnotes
|
There was a problem hiding this comment.
Code Review
This pull request successfully migrates the scheduler-estimator protobuf implementation from the legacy go-to-protobuf generator to the standard protoc-gen-go toolchain, addressing compatibility concerns with Kubernetes 1.35+. The adoption of a "Peer Fields" strategy, introducing new bytes fields alongside deprecated strongly-typed fields, is a robust approach to ensure graceful migration and wire compatibility. The addition of helper methods (Set..., Unmarshal..., MustSet...) simplifies the conversion between Kubernetes Go types and the new protobuf structure. The changes are comprehensive, affecting protobuf definitions, generation scripts, and Go code across various components to align with the new protobuf standard. However, several instances of ignored errors from Set and Unmarshal helper methods have been identified, which could lead to silent failures or incorrect data processing. These should be addressed to ensure the reliability of the new protobuf serialization/deserialization logic.
d7bbc4f to
a3c449b
Compare
6270ec0 to
9bdd629
Compare
|
Codecov Report❌ Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## master #7298 +/- ##
==========================================
- Coverage 42.15% 41.86% -0.29%
==========================================
Files 876 879 +3
Lines 53645 54224 +579
==========================================
+ Hits 22614 22703 +89
- Misses 29334 29806 +472
- Partials 1697 1715 +18
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Sentry. 🚀 New features to boost your workflow:
|
|
/gemini review |
There was a problem hiding this comment.
Code Review
This pull request migrates the estimator's protobuf implementation from gogo/protobuf to the standard google.golang.org/protobuf library and transitions the schema to proto3. Key updates include pinning the protoc version in the generation script and implementing helper methods for serializing Kubernetes types as byte fields to improve compatibility. Feedback indicates that several exported methods lack mandatory documentation comments, violating the project's style guide.
|
Test report:
$ kubectl logs -f --namespace karmada-system karmada-scheduler-xxx
I0416 09:00:02.464892 1 util.go:91] Invoked MaxAvailableReplicas of estimator general-estimator for workload(apps/v1, kind=Deployment, default/nginx): [{member1 100}]
I0416 09:00:02.469442 1 util.go:91] Invoked MaxAvailableReplicas of estimator scheduler-estimator for workload(apps/v1, kind=Deployment, default/nginx): [{member1 100}]
I0416 09:00:02.469468 1 util.go:112] Target cluster calculated by estimators (available cluster && maxAvailableReplicas): [{member1 100}]
$ kubectl set image --namespace karmada-system deployments/karmada-scheduler karmada-scheduler=docker.io/karmada/karmada-scheduler:v1.17.0
$ kubectl logs -f --namespace karmada-system karmada-scheduler-xxx
I0416 11:47:33.862768 1 util.go:91] Invoked MaxAvailableReplicas of estimator general-estimator for workload(apps/v1, kind=Deployment, default/nginx): [{member1 100}]
I0416 11:47:33.863312 1 event_handler.go:181] Ignore update event of resourceBinding default/nginx-deployment as specification no change
I0416 11:47:33.864241 1 util.go:91] Invoked MaxAvailableReplicas of estimator scheduler-estimator for workload(apps/v1, kind=Deployment, default/nginx): [{member1 100}]
$ kubectl set image --namespace karmada-system deployments/karmada-scheduler karmada-scheduler=docker.io/karmada/karmada-scheduler:latest
$ kubectl set image --namespace karmada-system deployments/karmada-scheduler-estimator-member1 karmada-scheduler-estimator=docker.io/karmada/karmada-scheduler-estimator:v1.17.0
$ kubectl logs -f --namespace karmada-system karmada-scheduler-xxx
I0416 11:52:21.105426 1 util.go:91] Invoked MaxAvailableReplicas of estimator general-estimator for workload(apps/v1, kind=Deployment, default/nginx): [{member1 100}]
I0416 11:52:21.108424 1 util.go:91] Invoked MaxAvailableReplicas of estimator scheduler-estimator for workload(apps/v1, kind=Deployment, default/nginx): [{member1 100}]
I0416 11:52:21.110609 1 util.go:112] Target cluster calculated by estimators (available cluster && maxAvailableReplicas): [{member1 100}]From the test report, this update is backward compatible. |
|
Hi @liggitt, hope you’re doing well. This PR is a follow-up to the earlier discussion around Karmada’s Kubernetes dependency upgrade to v1.35.0. Based on your suggestion in that discussion, this PR:
The PR is now ready for review, and I’d really appreciate your feedback if you have time.
|
70b4137 to
5c17fb6
Compare
There was a problem hiding this comment.
Pull request overview
Note
Copilot was unable to run its full agentic suite in this review.
Migrates the scheduler-estimator protobuf/gRPC API generation from legacy go-to-protobuf/gogo to the standard protoc-gen-go + protoc-gen-go-grpc toolchain, adding “peer bytes” fields to carry Kubernetes types without depending on Kubernetes’ ProtoMessage() marker compatibility layer.
Changes:
- Switched estimator proto definitions to proto3 and regenerated Go types + gRPC stubs with standard generators.
- Introduced helper methods to marshal/unmarshal peer
bytesfields (resource requests, node affinity, tolerations) and updated server/client/plugin code to use them. - Updated build/generation tooling (protoc pinning script,
go:generate, tools.go) and adapted tests to new pointer-based protobuf types.
Reviewed changes
Copilot reviewed 29 out of 90 changed files in this pull request and generated 11 comments.
Show a summary per file
| File | Description |
|---|---|
| pkg/estimator/service/service_grpc.pb.go | New grpc-go stubs generated via protoc-gen-go-grpc. |
| pkg/estimator/service/service.proto | Updates proto import + go_package for standard toolchain. |
| pkg/estimator/service/service.pb.go | Regenerated descriptor-only file via protoc-gen-go. |
| pkg/estimator/service/doc.go | Updates go:generate to standard protoc plugins. |
| pkg/estimator/server/server_test.go | Updates tests to use new helper setters for peer-bytes fields. |
| pkg/estimator/server/server.go | Embeds unimplemented server; logging changes; fixes proto field name + duration handling. |
| pkg/estimator/server/nodes/filter.go | Uses peer-bytes node affinity unmarshal and returns errors. |
| pkg/estimator/server/framework/runtime/framework_test.go | Updates component slice types to pointer-based protobuf messages. |
| pkg/estimator/server/framework/runtime/framework.go | Updates framework APIs to accept []*pb.Component. |
| pkg/estimator/server/framework/plugins/resourcequota/resourcequota_test.go | Updates tests to pointer-based messages + helper setters. |
| pkg/estimator/server/framework/plugins/resourcequota/resourcequota.go | Uses peer-bytes resource requests; propagates errors; updates component APIs. |
| pkg/estimator/server/framework/plugins/noderesource/noderesource_test.go | Updates tests to pointer-based messages + helper setters. |
| pkg/estimator/server/framework/plugins/noderesource/noderesource.go | Uses peer-bytes unmarshalling; simulator now returns error. |
| pkg/estimator/server/framework/interface.go | Interface changes for pointer-based components. |
| pkg/estimator/server/estimate.go | Switches to pointer-based ReplicaRequirements and components. |
| pkg/estimator/scheduling_simulator_components_test.go | Updates simulator tests for pointer-based components + error return. |
| pkg/estimator/scheduling_simulator_components.go | Simulator now handles peer-bytes; returns errors; affinity/tolerations extraction returns error. |
| pkg/estimator/pb/types.go | Removes legacy hand-written types (replaced by generated protos). |
| pkg/estimator/pb/helpers.go | Adds peer-bytes helpers, MustSet..., and formatted logging helpers. |
| pkg/estimator/pb/generated.protomessage.pb.go | Removes temporary ProtoMessage marker file. |
| pkg/estimator/pb/generated.pb.go | Removes gogo-generated protobuf output. |
| pkg/estimator/pb/estimator.proto | Migrates estimator proto to proto3; adds peer bytes fields; deprecates typed fields. |
| pkg/estimator/pb/estimator.pb.go | New generated proto3 Go types via protoc-gen-go. |
| pkg/estimator/client/general.go | Updates scheduling simulation path for pointer-based components and error return. |
| pkg/estimator/client/accurate.go | Updates gRPC request building for pointer-based messages + peer-bytes setters. |
| hack/update-estimator-protobuf.sh | Replaces go-to-protobuf with protoc + std plugins; pins protoc downloader. |
| hack/tools/tools.go | Updates tool imports for standard protoc plugins. |
| go.mod | Removes direct gogo dependency; adds protobuf/grpc tool requirements (see comments). |
| Makefile | Adds a kubernetes build tag to default make test invocation (see comments). |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
| func (pl *resourceQuotaEstimator) aggregateComponentRequirements(components []*pb.Component) (corev1.ResourceList, error) { | ||
| resourceRequirements := map[corev1.ResourceName]int64{} | ||
|
|
||
| for _, component := range components { | ||
| if component.ReplicaRequirements.ResourceRequest == nil { | ||
| if component == nil { | ||
| continue | ||
| } | ||
|
|
||
| requirements, err := component.ReplicaRequirements.UnmarshalResourceRequest() | ||
| if err != nil { | ||
| return nil, err | ||
| } |
There was a problem hiding this comment.
component.ReplicaRequirements can be nil (now that Component and its ReplicaRequirements are pointers), which will panic when calling UnmarshalResourceRequest(). Fix by guarding for component.ReplicaRequirements == nil (either continue as “no requirements” or return a validation error), and ensure evaluateComponentsAgainstQuota() does not pass components with nil requirements when selectors are empty.
There was a problem hiding this comment.
component.ReplicaRequirements can be nil, but calling UnmarshalResourceRequest() on it does not necessarily panic. In Go, a method call on a nil receiver is allowed as long as the method implementation itself does not dereference the nil receiver without checking it first.
There was a problem hiding this comment.
I suggest adding some safeguards to exit the loop promptly.
RainbowMango
left a comment
There was a problem hiding this comment.
Generally looks good, just some nits.
| func (pl *resourceQuotaEstimator) aggregateComponentRequirements(components []*pb.Component) (corev1.ResourceList, error) { | ||
| resourceRequirements := map[corev1.ResourceName]int64{} | ||
|
|
||
| for _, component := range components { | ||
| if component.ReplicaRequirements.ResourceRequest == nil { | ||
| if component == nil { | ||
| continue | ||
| } | ||
|
|
||
| requirements, err := component.ReplicaRequirements.UnmarshalResourceRequest() | ||
| if err != nil { | ||
| return nil, err | ||
| } |
aa217fc to
6b6fb09
Compare
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 30 out of 91 changed files in this pull request and generated 7 comments.
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 30 out of 91 changed files in this pull request and generated 3 comments.
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Signed-off-by: zhzhuang-zju <m17799853869@163.com>
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 30 out of 91 changed files in this pull request and generated 4 comments.
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
|
/cc @RainbowMango |
|
[APPROVALNOTIFIER] This PR is APPROVED This pull-request has been approved by: RainbowMango The full list of commands accepted by this bot can be found here. The pull request process is described here DetailsNeeds approval from an approver in each of these files:
Approvers can indicate their approval by writing |
What type of PR is this?
/kind api-change
What this PR does / why we need it:
This PR migrates the scheduler-estimator protobuf implementation to use the standard
protoc-gen-gotoolchain, replacing the legacygo-to-protobufgenerator. This change is a proactive adaptation for Kubernetes 1.35+ dependencies.Background:
Kubernetes v1.35 has removed the
ProtoMessage()marker method from core REST API types (e.g., Pod, Node). To provide a buffer for downstream projects like Karmada, upstream Kubernetes introduced a temporary build tag,kubernetes_protomessage_one_more_release, which forces these types to implement an emptyProtoMessage()method. However, this compatibility layer is slated for complete removal in Kubernetes v1.36. This PR ensures Karmada is fully decoupled from this dependency ahead of time.Solution:
Adopted a "Peer Fields" strategy:
bytesfields (e.g.,NodeAffinityBytes) alongside existing fields to support direct binary transmission of K8s objects.protoc-gen-goandprotoc-gen-go-grpc.MustSet...,Unmarshal...) to handle the conversion between K8s Go types and the new protobuf structure seamlessly.Which issue(s) this PR fixes:
Parts of #7140
Special notes for your reviewer:
Does this PR introduce a user-facing change?: