JobCoordinator may be turned off by configuration - #829
Conversation
|
[APPROVALNOTIFIER] This PR is APPROVED This pull-request has been approved by: pyalex, woop The full list of commands accepted by this bot can be found here. The pull request process is described here DetailsNeeds approval from an approver in each of these files:
Approvers can indicate their approval by writing |
|
/lgtm |
|
/kind feature |
|
We pretty much coincidentally implemented the same thing at the same time. Illuminating a use case for anyone following along: you probably want to deploy redundant Core instances for availability. They will not play well together if they all try to manage jobs, duplicating things and stomping on each other. A boolean flag serves as poor man's leader election, enable it on only one instance in a deployment. If the active coordinator fails, remediate by bringing it back to life or deploying a new instance with this flag enabled. Although not specific to the job coordination, a related concern is that Serving instances talk to Core to load and cache registered feature set specifications. The caching provides a layer of resilience if Core instances fail. Enabling a means of load balancing / round-robin for the gRPC connections from Serving to Core instances may be desirable however. Some of this may have a home in Feast operations guide documentation. |
What this PR does / why we need it:
We need option to deploy core without job management
Which issue(s) this PR fixes:
Fixes #
Does this PR introduce a user-facing change?: