Skip to content

Add historical retrieval via job service - #1107

Merged
pyalex merged 3 commits into
feast-dev:masterfrom
oavdeev:remote-historical-retrieval
Oct 29, 2020
Merged

Add historical retrieval via job service#1107
pyalex merged 3 commits into
feast-dev:masterfrom
oavdeev:remote-historical-retrieval

Conversation

@oavdeev

@oavdeev oavdeev commented Oct 28, 2020

Copy link
Copy Markdown
Collaborator

What this PR does / why we need it:
This implements end-to-end flow for historical retrieval via job service. Tested only manually for now, until we include job service setup in e2e tests.

Which issue(s) this PR fixes:

Fixes #

Does this PR introduce a user-facing change?:

NONE

@oavdeev

oavdeev commented Oct 28, 2020

Copy link
Copy Markdown
Collaborator Author

/kind feature

@feast-ci-bot feast-ci-bot added kind/feature New feature or request and removed needs-kind labels Oct 28, 2020
Comment thread protos/feast/core/JobService.proto Outdated
Comment thread protos/feast/core/JobService.proto Outdated

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

renamed this for consistency with the existing Job.cancel method

Comment thread protos/feast/core/JobService.proto Outdated

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Removed a bunch of unused (or not yet used) parameters here, we can always add them back later.

Comment thread protos/feast/core/JobService.proto Outdated

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

changed this for consistency with client API

Comment thread protos/feast/core/JobService.proto Outdated

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

another change for consistency with client API

Comment thread sdk/python/feast/cli.py Outdated

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't expect people to use this in production, but it is handy for debugging.

Comment thread sdk/python/feast/constants.py Outdated

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Had to bump this since some job service methods in turn make calls to AWS/GCP APIs. Sometimes those take a while especially in development env where your job service is not running in the cloud itself.

Comment thread sdk/python/feast/constants.py Outdated

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We should prob unify those params with core service in another PR.

@oavdeev

oavdeev commented Oct 28, 2020

Copy link
Copy Markdown
Collaborator Author

/kind feature

@pyalex pyalex left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This PR needs tests, both unit & e2e

@pyalex

pyalex commented Oct 29, 2020

Copy link
Copy Markdown
Collaborator

/retest

Comment thread protos/feast/core/JobService.proto Outdated
Comment thread sdk/python/feast/job_service.py Outdated
Signed-off-by: Oleg Avdeev <oleg.v.avdeev@gmail.com>
@oavdeev
oavdeev force-pushed the remote-historical-retrieval branch from e7394a3 to 0b4510d Compare October 29, 2020 09:18
@oavdeev

oavdeev commented Oct 29, 2020

Copy link
Copy Markdown
Collaborator Author

/retest

Signed-off-by: Oleg Avdeev <oleg.v.avdeev@gmail.com>
@oavdeev
oavdeev force-pushed the remote-historical-retrieval branch from 0b4510d to 278f022 Compare October 29, 2020 09:48
@oavdeev

oavdeev commented Oct 29, 2020

Copy link
Copy Markdown
Collaborator Author

/test test-end-to-end

Signed-off-by: Oleg Avdeev <oleg.v.avdeev@gmail.com>
@oavdeev
oavdeev force-pushed the remote-historical-retrieval branch from 278f022 to 68fb26b Compare October 29, 2020 09:55
@feast-ci-bot

feast-ci-bot commented Oct 29, 2020

Copy link
Copy Markdown
Collaborator

@oavdeev: The following test failed, say /retest to rerun them all:

Test name Commit Details Rerun command
python-sdk-integration-test 68fb26b link /test python-sdk-integration-test

Full PR test history

Details

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

@feast-ci-bot

Copy link
Copy Markdown
Collaborator

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: oavdeev, pyalex

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Details Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@pyalex

pyalex commented Oct 29, 2020

Copy link
Copy Markdown
Collaborator

/lgtm

@pyalex
pyalex merged commit de4313b into feast-dev:master Oct 29, 2020
@oavdeev
oavdeev deleted the remote-historical-retrieval branch October 29, 2020 10:26
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants