Skip to content

Multiple tiny AWS related fixes - #1083

Merged
khorshuheng merged 2 commits into
feast-dev:masterfrom
oavdeev:aws-more-fixes
Oct 23, 2020
Merged

Multiple tiny AWS related fixes#1083
khorshuheng merged 2 commits into
feast-dev:masterfrom
oavdeev:aws-more-fixes

Conversation

@oavdeev

@oavdeev oavdeev commented Oct 21, 2020

Copy link
Copy Markdown
Collaborator
  • Fix object path math in client staging uploader
  • Add random suffix to the historical retrieval output location
  • EMR job now actually checks for status
  • add boto3 dep

What this PR does / why we need it:

Which issue(s) this PR fixes:

Fixes #

Does this PR introduce a user-facing change?:

NONE

@oavdeev

oavdeev commented Oct 21, 2020

Copy link
Copy Markdown
Collaborator Author

/kind bug

Comment thread sdk/python/setup.py Outdated
Comment thread sdk/python/feast/client.py Outdated

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

shouldn't that strip be part of implementation (GS, s3)?

@oavdeev oavdeev Oct 22, 2020

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We can put strip into both implementations too (gcp actually has it already). It just goes a bit against S3 (and i suspect GCP) semantics where /a and a are separate objects

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I will look into the GCP part again.

@oavdeev
oavdeev force-pushed the aws-more-fixes branch 2 times, most recently from a8a4eff to 2b79209 Compare October 22, 2020 07:02
@oavdeev

oavdeev commented Oct 22, 2020

Copy link
Copy Markdown
Collaborator Author

/retest

@oavdeev

oavdeev commented Oct 22, 2020

Copy link
Copy Markdown
Collaborator Author

/retest

@khorshuheng

Copy link
Copy Markdown
Collaborator

/lgtm

@feast-ci-bot

Copy link
Copy Markdown
Collaborator

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: khorshuheng, oavdeev

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Details Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

* Fix object path math in client staging uploader
* Add random suffix to the historical retrieval output location
* EMR job now actually checks for status
* add boto3 dep

Signed-off-by: Oleg Avdeev <oleg.v.avdeev@gmail.com>
Signed-off-by: Oleg Avdeev <oleg.v.avdeev@gmail.com>
@khorshuheng

Copy link
Copy Markdown
Collaborator

/lgtm

@khorshuheng

Copy link
Copy Markdown
Collaborator

/test test-end-to-end-auth

@khorshuheng
khorshuheng merged commit d779567 into feast-dev:master Oct 23, 2020
@oavdeev
oavdeev deleted the aws-more-fixes branch October 23, 2020 08:56
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants