Implement JobService API calls & connect it to SDK - #1111
Conversation
|
[APPROVALNOTIFIER] This PR is NOT APPROVED This pull-request has been approved by: tsotnet The full list of commands accepted by this bot can be found here. DetailsNeeds approval from an approver in each of these files:Approvers can indicate their approval by writing |
Signed-off-by: Tsotne Tabidze <tsotnet@gmail.com>
1ca0a75 to
221badc
Compare
Signed-off-by: Tsotne Tabidze <tsotnet@gmail.com>
Signed-off-by: Tsotne Tabidze <tsotnet@gmail.com>
…ne mode * New API calls (start_offline_to_online_ingestion, start_stream_to_online_ingestion) now return Remote Jobs instead of job ids * Implement list_jobs & get_job for standalone mode (looks like Spark is running in local mode and we can't get job statuses so we have to keep cache in memory) * Wire up list_jobs & get_job on client side with job service * Tested locally on Feast 101 notebook, everything works Signed-off-by: Tsotne Tabidze <tsotnet@gmail.com>
Signed-off-by: Tsotne Tabidze <tsotnet@gmail.com>
|
/test python-sdk-integration-test |
Signed-off-by: Tsotne Tabidze <tsotnet@gmail.com>
Signed-off-by: Tsotne Tabidze <tsotnet@gmail.com>
Signed-off-by: Tsotne Tabidze <tsotnet@gmail.com>
Signed-off-by: Tsotne Tabidze <tsotnet@gmail.com>
|
/test test-end-to-end |
…mode Signed-off-by: Tsotne Tabidze <tsotnet@gmail.com>
1c89ea2 to
3335ae4
Compare
|
/test test-end-to-end-aws |
Signed-off-by: Tsotne Tabidze <tsotnet@gmail.com>
| feature_table = self.client.get_feature_table( | ||
| request.table_name, request.project | ||
| ) | ||
| job = self.client.start_offline_to_online_ingestion( |
There was a problem hiding this comment.
can we use launcher.start_offline_to_online instead of client here?
There was a problem hiding this comment.
the reason is: client may have different (user-oriented) API (like accepting strings instead of feature table, dataframes instead of source) and it's not really idempotent (and being already called once)
There was a problem hiding this comment.
Done in the fork: #1129. Please comment there if anything else is needed.
|
Closing this and instead opening a PR in a fork: #1129 |
What this PR does / why we need it: