Skip to content
Merged
Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
66 commits
Select commit Hold shift + click to select a range
508bad2
merged changes
franciscojavierarceo Sep 24, 2024
899f7f4
saving progress
franciscojavierarceo Aug 17, 2024
5b69c71
merged changes to odfv
franciscojavierarceo Sep 24, 2024
d107354
linted
franciscojavierarceo Aug 18, 2024
a34ec4d
adding the test needed to show the expected behavior
franciscojavierarceo Aug 18, 2024
b95d2a2
updated test case
franciscojavierarceo Aug 21, 2024
47974c2
saving progress
franciscojavierarceo Aug 21, 2024
ceb75a2
merging
franciscojavierarceo Sep 24, 2024
6688933
merged
franciscojavierarceo Sep 24, 2024
4c28acc
merged
franciscojavierarceo Sep 24, 2024
fd577dc
merging
franciscojavierarceo Sep 24, 2024
167fe6c
adding the entity keys for now to do retrieval
franciscojavierarceo Aug 29, 2024
54811d7
adding entity to odfv
franciscojavierarceo Aug 29, 2024
f0d87fc
checking in progress...getting closer
franciscojavierarceo Aug 29, 2024
b7091ca
may have to revert some of this...looks like the challenge is getting…
franciscojavierarceo Aug 31, 2024
d2a12f8
moving things around to make it easier to debug
franciscojavierarceo Sep 1, 2024
9d496ba
debugging
franciscojavierarceo Sep 1, 2024
58280aa
merged
franciscojavierarceo Sep 24, 2024
c42be75
merging
franciscojavierarceo Sep 24, 2024
fb3b315
Rebasing and merging changes from other PR
franciscojavierarceo Sep 6, 2024
82f3f8b
Merging changes continued
franciscojavierarceo Sep 7, 2024
172693d
update the _make_inference to include odfvs with writes in the update…
franciscojavierarceo Sep 7, 2024
66c5b57
have the table being written now...the create table happens in the Sq…
franciscojavierarceo Sep 8, 2024
ea3b98a
checking in progress
franciscojavierarceo Sep 9, 2024
905912b
adding logs
franciscojavierarceo Sep 10, 2024
25d42dd
updating permissions
franciscojavierarceo Sep 10, 2024
03d6116
going to error out on purpose
franciscojavierarceo Sep 10, 2024
9b16615
adding unit test and merging changes
franciscojavierarceo Sep 18, 2024
adbaeb6
almost got everything working and type validation behaving
franciscojavierarceo Sep 18, 2024
11d2914
cleaned up and have tests behaving
franciscojavierarceo Sep 18, 2024
64375ee
adding print
franciscojavierarceo Sep 21, 2024
3e6912a
removing print
franciscojavierarceo Sep 21, 2024
5751a72
checking in progress
franciscojavierarceo Sep 23, 2024
b0208e1
updating test
franciscojavierarceo Sep 25, 2024
463d8bb
adding test
franciscojavierarceo Sep 25, 2024
2981817
linted and updated
franciscojavierarceo Sep 25, 2024
3a33368
removed print
franciscojavierarceo Sep 25, 2024
22bf637
updated tests to test actual behavior
franciscojavierarceo Sep 25, 2024
0d0d117
checking in progress
franciscojavierarceo Sep 28, 2024
5bff836
changing typo
franciscojavierarceo Sep 28, 2024
271f814
updating test
franciscojavierarceo Sep 28, 2024
754b0e8
testing changes
franciscojavierarceo Sep 28, 2024
25c7181
checking to see if thing still working
franciscojavierarceo Sep 29, 2024
1d4023f
removed print
franciscojavierarceo Sep 29, 2024
3662102
undo change for odfv file
franciscojavierarceo Sep 29, 2024
74e7ede
updated tests
franciscojavierarceo Sep 30, 2024
59940cf
okay well have the unit test working
franciscojavierarceo Oct 1, 2024
b223feb
type changes, hope i dont regret them
franciscojavierarceo Oct 1, 2024
01770e2
updated stream feature view piece
franciscojavierarceo Oct 2, 2024
7606481
updated sfv ifelse
franciscojavierarceo Oct 2, 2024
c4ebf18
removing print
franciscojavierarceo Oct 2, 2024
72add32
formatted and updated test
franciscojavierarceo Oct 2, 2024
24e0a84
resolving some linter errors
franciscojavierarceo Oct 3, 2024
b92bf32
fixed linter and formatting
franciscojavierarceo Oct 3, 2024
934c1e9
okay think it is working
franciscojavierarceo Oct 3, 2024
0ce93f2
linter
franciscojavierarceo Oct 3, 2024
bf31d59
updated type map for integration tests
franciscojavierarceo Oct 4, 2024
9aff889
Merge branch 'master' into podfv2
franciscojavierarceo Oct 5, 2024
acc7b7f
updated local feature store test
franciscojavierarceo Oct 5, 2024
12d9100
fixed local fs test
franciscojavierarceo Oct 5, 2024
3c94614
chore: Updated snowflake test to be more explicit about post apply en…
franciscojavierarceo Oct 5, 2024
f01b691
merging
franciscojavierarceo Oct 5, 2024
c26ae75
fixed test to entity_rows_to_read
franciscojavierarceo Oct 5, 2024
c6d55c7
resolved inf conflicts
franciscojavierarceo Oct 5, 2024
52b8c4d
lint
franciscojavierarceo Oct 5, 2024
ca0971a
Updated tests and lint, think I have everything working
franciscojavierarceo Oct 6, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
saving progress
Signed-off-by: Francisco Javier Arceo <farceo@redhat.com>
  • Loading branch information
franciscojavierarceo committed Sep 30, 2024
commit 47974c2cc92d79e3ce3b270d5a74d1ef99c47de5
22 changes: 14 additions & 8 deletions sdk/python/feast/feature_store.py
Original file line number Diff line number Diff line change
Expand Up @@ -1444,19 +1444,18 @@ def write_to_online_store(
inputs: Optional the dictionary object to be written
allow_registry_cache (optional): Whether to allow retrieving feature views from a cached registry.
"""
# TODO: restrict this to work with online StreamFeatureViews and validate the FeatureView type
feature_view_dict = {
fv_proto.name: fv_proto
for fv_proto in self.list_all_feature_views(allow_registry_cache)
}
try:
feature_view: FeatureView = self.get_stream_feature_view(
feature_view_name, allow_registry_cache=allow_registry_cache
)
feature_view = feature_view_dict[feature_view_name]
except FeatureViewNotFoundException:
feature_view = self.get_feature_view(
feature_view_name, allow_registry_cache=allow_registry_cache
)
raise FeatureViewNotFoundException(feature_view_name, self.project)
if df is not None and inputs is not None:
raise ValueError("Both df and inputs cannot be provided at the same time.")
if df is None and inputs is not None:
if isinstance(inputs, dict):
if isinstance(inputs, dict) or isinstance(inputs, List):
try:
df = pd.DataFrame(inputs)
except Exception as _:
Expand All @@ -1465,6 +1464,13 @@ def write_to_online_store(
pass
else:
raise ValueError("inputs must be a dictionary or a pandas DataFrame.")
if df is not None and inputs is None:
if isinstance(df, dict) or isinstance(df, List):
try:
df = pd.DataFrame(df)
except Exception as _:
raise DataFrameSerializationError

provider = self._get_provider()
provider.ingest_df(feature_view, df)

Expand Down
4 changes: 4 additions & 0 deletions sdk/python/feast/infra/passthrough_provider.py
Original file line number Diff line number Diff line change
Expand Up @@ -279,6 +279,10 @@ def ingest_df(
):
table = pa.Table.from_pandas(df)

# TODO: Update this to support On Demand Feature Views.
# Note: A dictionary mapping of column names in this data
# source to feature names in a feature table or view. Only used for feature
# columns, not entity or timestamp columns.
if feature_view.batch_source.field_mapping is not None:
table = _run_pyarrow_field_mapping(
table, feature_view.batch_source.field_mapping
Expand Down
Original file line number Diff line number Diff line change
@@ -1,6 +1,7 @@
import os
import re
import tempfile
import time
import unittest
from datetime import datetime, timedelta
from typing import Any
Expand Down Expand Up @@ -285,9 +286,16 @@ def test_stored_writes(self):
entity_rows = [
{
"driver_id": 1001,
"conv_rate": 0.25,
"acc_rate": 0.25,
}
]

self.store.write_to_online_store(
feature_view_name="python_stored_writes_feature_view",
df=entity_rows,
)
time.sleep(1)
online_python_response = self.store.get_online_features(
entity_rows=entity_rows,
features=[
Expand Down