Skip to content

fix: Added Redshift and Spark typecheck to data_source event_timestamp_col inference #2389

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
feast-ci-bot merged 8 commits into from
Mar 17, 2022
+30 −1
Merged

fix: Added Redshift and Spark typecheck to data_source event_timestamp_col inference #2389

Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
8 commits
Select commit Hold shift + click to select a range
68f6e5e
added Redshift typecheck to data_source event_timestamp_col inference
shaibruhis Mar 9, 2022
7f1f15f
address comments
shaibruhis Mar 9, 2022
f6e5907
moved non file data sources into their own test
shaibruhis Mar 9, 2022
ca788ce
addressed comments
shaibruhis Mar 9, 2022
d6703b9
fixed texts
shaibruhis Mar 10, 2022
ef17d06
remove previously defined event_timestamp_column from data_source to …
shaibruhis Mar 10, 2022
87981e8
made a deepcopy of data_sources to not affect other tests
shaibruhis Mar 12, 2022
302e295
linter
shaibruhis Mar 15, 2022
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
addressed comments 
Signed-off-by: Shai Bruhis <shaibruhis@gmail.com>
  • Loading branch information
@shaibruhis
shaibruhis committed Mar 12, 2022
commit ca788ced1f9fbe3254d0f514303cdafe487e13ff
6 changes: 3 additions & 3 deletions sdk/python/feast/inference.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -45,7 +45,7 @@ def update_entities_with_inferred_types_from_feature_views(

# get entity information from information extracted from the view batch source
extracted_entity_name_type_pairs = list(
filter(lambda tup: tup[0] == entity.join_key,col_names_and_types,)
filter(lambda tup: tup[0] == entity.join_key, col_names_and_types,)
)
if len(extracted_entity_name_type_pairs) == 0:
# Doesn't mention inference error because would also be an error without inferencing
Expand All @@ -54,8 +54,8 @@ def update_entities_with_inferred_types_from_feature_views(
its entity's name."""
)

inferred_value_type = (
view.batch_source.source_datatype_to_feast_value_type()(extracted_entity_name_type_pairs[0][1])
inferred_value_type = view.batch_source.source_datatype_to_feast_value_type()(
extracted_entity_name_type_pairs[0][1]
)

if (
Expand Down
6 changes: 3 additions & 3 deletions sdk/python/tests/integration/registration/test_inference.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -111,9 +111,7 @@ def test_infer_datasource_names_dwh():


@pytest.mark.integration
def test_update_file_data_source_with_inferred_event_timestamp_col(
simple_dataset_1
):
def test_update_file_data_source_with_inferred_event_timestamp_col(simple_dataset_1):
df_with_two_viable_timestamp_cols = simple_dataset_1.copy(deep=True)
df_with_two_viable_timestamp_cols["ts_2"] = simple_dataset_1["ts_1"]

Expand All @@ -139,6 +137,8 @@ def test_update_file_data_source_with_inferred_event_timestamp_col(
[file_source], RepoConfig(provider="local", project="test")
)


@pytest.mark.integration
@pytest.mark.universal
Comment thread
adchia marked this conversation as resolved.
Outdated
def test_update_data_sources_with_inferred_event_timestamp_col(universal_data_sources):
(_, _, data_sources) = universal_data_sources
Comment thread
adchia marked this conversation as resolved.
Outdated
Expand Down