Support TFRecord as one of the output formats for historical feature retrieval - #1222
Conversation
|
[APPROVALNOTIFIER] This PR is APPROVED This pull-request has been approved by: khorshuheng The full list of commands accepted by this bot can be found here. The pull request process is described here DetailsNeeds approval from an approver in each of these files:
Approvers can indicate their approval by writing |
1ca8d28 to
7d62dec
Compare
There was a problem hiding this comment.
What is with all these noqa: F811? Can it be fixed please?
There was a problem hiding this comment.
That's because we are importing the fixtures from another module, and flake8 doesn't understand pytest fixtures.
One way to fix it would be not to import the fixture and include it directly in test_launchers, though that would mean it's harder for us to reuse the fixture across different test files.
I will research for way on how this can be circumvented without explicitly adding # noqa: F811, but i am not certain if it is possible.
7bb80e3 to
2d019f1
Compare
There was a problem hiding this comment.
| "Args": ["spark-submit", pyspark_script_path] | |
| + args | |
| + ["--packages", ",".join(packages)] | |
| if packages | |
| else [], | |
| "Args": ["spark-submit", pyspark_script_path] | |
| + args | |
| + (["--packages", ",".join(packages)] | |
| if packages | |
| else []), |
1238e6c to
0937a1b
Compare
|
/test test-end-to-end-sparkop |
3 similar comments
|
/test test-end-to-end-sparkop |
|
/test test-end-to-end-sparkop |
|
/test test-end-to-end-sparkop |
|
/test test-end-to-end-azure |
…retrieval Signed-off-by: Khor Shu Heng <khor.heng@gojek.com>
Signed-off-by: Khor Shu Heng <khor.heng@gojek.com>
Signed-off-by: Khor Shu Heng <khor.heng@gojek.com>
Signed-off-by: Khor Shu Heng <khor.heng@gojek.com>
Signed-off-by: Khor Shu Heng <khor.heng@gojek.com>
Signed-off-by: Khor Shu Heng <khor.heng@gojek.com>
Signed-off-by: Khor Shu Heng <khor.heng@gojek.com>
Signed-off-by: Khor Shu Heng <khor.heng@gojek.com>
Signed-off-by: Khor Shu Heng <khor.heng@gojek.com>
Signed-off-by: Khor Shu Heng <khor.heng@gojek.com>
Signed-off-by: Khor Shu Heng <khor.heng@gojek.com>
39840ec to
5d80975
Compare
Signed-off-by: Khor Shu Heng <khor.heng@gojek.com>
|
@khorshuheng: The following test failed, say
Full PR test history. Your PR dashboard. DetailsInstructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here. |
|
/test test-end-to-end |
What this PR does / why we need it:
This PR allows user to specify
tfrecordas the output format of historical feature retrieval, which is useful in cases where the users wish to generate statistics from the retrieved dataset using tfdv, or if the user's machine learning model is based on tensorflow.Which issue(s) this PR fixes:
Fixes #
Does this PR introduce a user-facing change?: