Is your feature request related to a problem? Please describe.
We are seeing that ML feature teams want to own their feature projections and not rely on a single global online feature store with the most common reason being reduction of blast radius.
Describe the solution you'd like
A way to create projections of a subset of feature views to specific online stores.
Describe alternatives you've considered
I was pointed towards attempting to implement the requested feature using a custom provider which I managed to get working, albeit rather hacky. I've only focused on fanning out the writing of features.
class ManyOnlineStoresPassthroughConfig(RepoConfig):
online_stores: List[dict]
class ManyOnlineStoresPassthroughProvider(PassthroughProvider):
def online_write_batch(self, config: ManyOnlineStoresPassthroughConfig, table: FeatureView, *args, **kwargs) -> None:
"""
Writes feature view to all subscribers
"""
feature_view_subscribers = [store for store in config.online_stores if table.name in store["view_subscriptions"]]
for store in feature_view_subscribers:
print(f"{table.name} online write to subscriber {store['name']}")
config.online_store = get_online_config_from_type(store["type"])(**store)
self.online_store = get_online_store_from_config(config.online_store)
return super().online_write_batch(config, table, *args, **kwargs)With the config update of
project: aaaa
registry: gs://temp-feast-dev/dev/registry.db
provider: providers.custom_provider.ManyOnlineStoresPassthroughProvider
offline_store:
type: bigquery
dataset: temp_feast-dev
project_id: abc
location: EU
online_stores:
- name: "lovely"
type: redis
redis_type: redis
connection_string: "blah"
view_subscriptions: ["feature_view_1"]
- name: "lovely"
type: redis
redis_type: redis
connection_string: "abc"
view_subscriptions: ["feature_view_1", "feature_view_2"]There's still some unknowns around retrieving the online features as we'd need a way of knowing what projection should be used.
Happy to help wherever I can!
Is your feature request related to a problem? Please describe.
We are seeing that ML feature teams want to own their feature projections and not rely on a single global online feature store with the most common reason being reduction of blast radius.
Describe the solution you'd like
A way to create projections of a subset of feature views to specific online stores.
Describe alternatives you've considered
I was pointed towards attempting to implement the requested feature using a custom provider which I managed to get working, albeit rather hacky. I've only focused on fanning out the writing of features.
With the config update of
There's still some unknowns around retrieving the online features as we'd need a way of knowing what projection should be used.
Happy to help wherever I can!