Skip to content

feat: Add column reordering to write_to_offline_store #2876

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
feast-ci-bot merged 11 commits into from
Jun 30, 2022
Merged

feat: Add column reordering to write_to_offline_store #2876

Changes from 1 commit
Commits
Show all changes
11 commits
Select commit Hold shift + click to select a range
0a152ee
Add feature extraction logic to batch writer
felixwang9817 Jun 23, 2022
0ce6433
Enable StreamProcessor to write to both online and offline stores
felixwang9817 Jun 27, 2022
a47b39b
Fix incorrect columns error message
felixwang9817 Jun 28, 2022
791322b
Reorder columns in _write_to_offline_store
felixwang9817 Jun 28, 2022
9970032
Make _write_to_offline_store a public method
felixwang9817 Jun 28, 2022
d1ce38f
Import FeatureStore correctly
felixwang9817 Jun 29, 2022
5ff276c
Remove defaults for `processing_time` and `query_timeout`
felixwang9817 Jun 29, 2022
838363a
Clean up `test_offline_write.py`
felixwang9817 Jun 29, 2022
b090a1d
Do not do any custom logic for double underscore columns
felixwang9817 Jun 29, 2022
b75883a
Lint
felixwang9817 Jun 29, 2022
e54ea6e
Switch entity values for all tests using push sources to not affect o…
felixwang9817 Jun 30, 2022
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
Do not do any custom logic for double underscore columns 
Signed-off-by: Felix Wang <wangfelix98@gmail.com>
  • Loading branch information
@felixwang9817
felixwang9817 committed Jun 30, 2022
commit b090a1dd5613e2c4dc703de11eda4f7b46b75f2a
9 changes: 1 addition & 8 deletions sdk/python/feast/feature_store.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -1439,19 +1439,12 @@ def write_to_offline_store(
feature_view_name, allow_registry_cache=allow_registry_cache
)

# Get columns of the batch source and the input dataframe. Ignore columns with double
# underscores, which often signal an internal-use column.
# Get columns of the batch source and the input dataframe.
column_names_and_types = feature_view.batch_source.get_table_column_names_and_types(
self.config
)
source_columns = [column for column, _ in column_names_and_types]
source_columns = [
column for column in source_columns if not re.match("__|__$", column)
]
input_columns = df.columns.values.tolist()
input_columns = [
column for column in input_columns if not re.match("__|__$", column)
]

if set(input_columns) != set(source_columns):
raise ValueError(
Expand Down