Expected Behavior
Registering ODFV UDFs that operate on lists of numbers (e.g., cosine similarity of embeddings/vectors) should not
throw errors.
Current Behavior
Defining a ODFV UDF such as cosine similarity and then running feast apply will result in the following error:
def feast_value_type_to_pandas_type(value_type: ValueType) -> Any:
value_type_to_pandas_type: Dict[ValueType, str] = {
ValueType.FLOAT: "float",
ValueType.INT32: "int",
ValueType.INT64: "int",
ValueType.STRING: "str",
ValueType.DOUBLE: "float",
ValueType.BYTES: "bytes",
ValueType.BOOL: "bool",
ValueType.UNIX_TIMESTAMP: "datetime",
}
if value_type in value_type_to_pandas_type:
return value_type_to_pandas_type[value_type]
raise TypeError(
> f"Casting to pandas type for type {value_type} failed. "
f"Type {value_type} not found"
)
E TypeError: Casting to pandas type for type ValueType.DOUBLE_LIST failed. Type ValueType.DOUBLE_LIST not found
Steps to reproduce
Define an ODFV UDF for cosine similarity and try to register it:
from feast import Entity, Feature, FeatureView, ValueType
from feast.data_source import RequestDataSource
from feast.infra.offline_stores.file_source import FileSource
from feast.on_demand_feature_view import on_demand_feature_view
from google.protobuf.duration_pb2 import Duration
import numpy as np
import pandas as pd
item = Entity(
name="item_id",
value_type=ValueType.INT64,
description="item ID",
)
items_fv = FeatureView(
name="items",
entities=["item"],
features=[
Feature(name="embedding", dtype=ValueType.DOUBLE_LIST),
],
batch_source=FileSource(
path="YOUR_PATH",
event_timestamp_column="event_timestamp",
created_timestamp_column="created",
),
online=True,
ttl=Duration(),
tags={},
)
similarity_req = RequestDataSource(
name="similarity_input",
schema={
"vector": ValueType.DOUBLE_LIST,
},
)
@on_demand_feature_view(
inputs={
"items": items_fv,
"similarity_req": similarity_req,
},
features=[
Feature(name="cos", dtype=ValueType.DOUBLE),
],
)
def similarity(features_df: pd.DataFrame) -> pd.DataFrame:
if features_df.size == 0:
return pd.DataFrame({"cos": [0.0]}) # give hint to Feast about return type
vectors_a = features_df["embedding"].apply(np.array)
vectors_b = features_df["vector"].apply(np.array)
dot_products = vectors_a.mul(vectors_b).apply(sum)
norms_q = vectors_a.apply(np.linalg.norm)
norms_doc = vectors_b.apply(np.linalg.norm)
df = pd.DataFrame()
df["cos"] = dot_products / (norms_q * norms_doc)
return dfSpecifications
- Version: 0.14.0
- Platform: all
- Subsystem: Python SDK
Possible Solution
Add the following 2 lines to feast_value_type_to_pandas_type() in type_map.py:
ValueType.FLOAT_LIST: "object",
ValueType.DOUBLE_LIST: "object",
Expected Behavior
Registering ODFV UDFs that operate on lists of numbers (e.g., cosine similarity of embeddings/vectors) should not
throw errors.
Current Behavior
Defining a ODFV UDF such as cosine similarity and then running
feast applywill result in the following error:Steps to reproduce
Define an ODFV UDF for cosine similarity and try to register it:
Specifications
Possible Solution
Add the following 2 lines to
feast_value_type_to_pandas_type()intype_map.py: