Skip to content

JobCoordinator may be turned off by configuration - #829

Merged
feast-ci-bot merged 1 commit into
feast-dev:masterfrom
pyalex:optionable-disable-job-coordinator
Jun 25, 2020
Merged

JobCoordinator may be turned off by configuration#829
feast-ci-bot merged 1 commit into
feast-dev:masterfrom
pyalex:optionable-disable-job-coordinator

Conversation

@pyalex

@pyalex pyalex commented Jun 25, 2020

Copy link
Copy Markdown
Collaborator

What this PR does / why we need it:

We need option to deploy core without job management

Which issue(s) this PR fixes:

Fixes #

Does this PR introduce a user-facing change?:

Added the option of turning ingestion job management on or off.

@feast-ci-bot

Copy link
Copy Markdown
Collaborator

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: pyalex, woop

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Details Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@woop

woop commented Jun 25, 2020

Copy link
Copy Markdown
Member

/lgtm

@woop

woop commented Jun 25, 2020

Copy link
Copy Markdown
Member

/kind feature

@feast-ci-bot feast-ci-bot added kind/feature New feature or request and removed needs-kind labels Jun 25, 2020
@feast-ci-bot
feast-ci-bot merged commit 197a701 into feast-dev:master Jun 25, 2020
@ches

ches commented Jul 4, 2020

Copy link
Copy Markdown
Member

We pretty much coincidentally implemented the same thing at the same time.

Illuminating a use case for anyone following along: you probably want to deploy redundant Core instances for availability. They will not play well together if they all try to manage jobs, duplicating things and stomping on each other.

A boolean flag serves as poor man's leader election, enable it on only one instance in a deployment. If the active coordinator fails, remediate by bringing it back to life or deploying a new instance with this flag enabled.

Although not specific to the job coordination, a related concern is that Serving instances talk to Core to load and cache registered feature set specifications. The caching provides a layer of resilience if Core instances fail. Enabling a means of load balancing / round-robin for the gRPC connections from Serving to Core instances may be desirable however.

Some of this may have a home in Feast operations guide documentation.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants