Skip to content

Commit 73601e4

Browse files
pushkarmoipushkarmoi
authored
feat: Feast/IKV online store documentation (#4146)
* feat: Feast/IKV online store documentation Signed-off-by: Pushkar Gupta <pushkar.moi@gmail.com> * functionality matric Signed-off-by: Pushkar Gupta <pushkar.moi@gmail.com> * more changes Signed-off-by: Pushkar Gupta <pushkar.moi@gmail.com> * mount dir Signed-off-by: Pushkar Gupta <pushkar.moi@gmail.com> --------- Signed-off-by: Pushkar Gupta <pushkar.moi@gmail.com>
1 parent 3fdb716 commit 73601e4

File tree

6 files changed

+99
-19
lines changed

6 files changed

+99
-19
lines changed

‎README.md‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -175,6 +175,7 @@ The list below contains the functionality that contributors are planning to deve
175175
* [x] [Bigtable](https://docs.feast.dev/reference/online-stores/bigtable)
176176
* [x] [SQLite](https://docs.feast.dev/reference/online-stores/sqlite)
177177
* [x] [Dragonfly](https://docs.feast.dev/reference/online-stores/dragonfly)
178+
* [x] [IKV - Inlined Key Value Store](https://docs.feast.dev/reference/online-stores/ikv)
178179
* [x] [Azure Cache for Redis (community plugin)](https://github.com/Azure/feast-azure)
179180
* [x] [Postgres (contrib plugin)](https://docs.feast.dev/reference/online-stores/postgres)
180181
* [x] [Cassandra / AstraDB (contrib plugin)](https://docs.feast.dev/reference/online-stores/cassandra)

‎docs/SUMMARY.md‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -91,6 +91,7 @@
9191
* [Snowflake](reference/online-stores/snowflake.md)
9292
* [Redis](reference/online-stores/redis.md)
9393
* [Dragonfly](reference/online-stores/dragonfly.md)
94+
* [IKV](reference/online-stores/ikv.md)
9495
* [Datastore](reference/online-stores/datastore.md)
9596
* [DynamoDB](reference/online-stores/dynamodb.md)
9697
* [Bigtable](reference/online-stores/bigtable.md)

‎docs/reference/online-stores/README.md‎

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -22,6 +22,10 @@ Please see [Online Store](../../getting-started/architecture-and-components/onli
2222
[dragonfly.md](dragonfly.md)
2323
{% endcontent-ref %}
2424

25+
{% content-ref url="ikv.md" %}
26+
[ikv.md](ikv.md)
27+
{% endcontent-ref %}
28+
2529
{% content-ref url="datastore.md" %}
2630
[datastore.md](datastore.md)
2731
{% endcontent-ref %}

‎docs/reference/online-stores/ikv.md‎

Lines changed: 73 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,73 @@
1+
# IKV (Inlined Key-Value Store) online store
2+
3+
## Description
4+
5+
[IKV](https://github.com/inlinedio/ikv-store) is a fully-managed embedded key-value store, primarily designed for storing ML features. Most key-value stores (think Redis or Cassandra) need a remote database cluster, whereas IKV allows you to utilize your existing application infrastructure to store data (cost efficient) and access it without any network calls (better performance).
6+
7+
For provisioning API keys for using it as an online-store in Feast, go to [https://inlined.io](https://inlined.io) or email onboarding[at]inlined.io
8+
9+
## Getting started
10+
Make sure you have Python and `pip` installed.
11+
12+
Install the Feast SDK and CLI: `pip install feast`
13+
14+
In order to use this online store, you'll need to install the IKV extra (along with the dependency needed for the offline store of choice). E.g.
15+
- `pip install 'feast[gcp, ikv]'`
16+
- `pip install 'feast[snowflake, ikv]'`
17+
- `pip install 'feast[aws, ikv]'`
18+
- `pip install 'feast[azure, ikv]'`
19+
20+
You can get started by using any of the other templates (e.g. `feast init -t gcp` or `feast init -t snowflake` or `feast init -t aws`), and then swapping in IKV as the online store as seen below in the examples.
21+
22+
### 1. Provision an IKV store
23+
Go to [https://inlined.io](https://inlined.io) or email onboarding[at]inlined.io
24+
25+
### 2. Configure
26+
27+
Update `my_feature_repo/feature_store.yaml` with the below contents:
28+
29+
{% code title="feature_store.yaml" %}
30+
```yaml
31+
project: my_feature_repo
32+
registry: data/registry.db
33+
provider: local
34+
online_store:
35+
type: ikv
36+
account_id: secret
37+
account_passkey: secret
38+
store_name: your-store-name
39+
mount_directory: /absolute/path/on/disk/for/ikv/embedded/index
40+
```
41+
{% endcode %}
42+
43+
After provision an IKV account/store, you should the required id, passkey and store-name.
44+
45+
Additionally you must specify a mount-directory - where IKV will pull/update (maintain) a copy of the index for online reads (IKV is an embedded database). It can be skipped only if you don't plan to read any data from this container. The mount directory path usually points to a location on local/remote disk.
46+
47+
The full set of configuration options is available in IKVOnlineStoreConfig at `sdk/python/feast/infra/online_stores/contrib/ikv_online_store/ikv.py`
48+
49+
## Functionality Matrix
50+
51+
The set of functionality supported by online stores is described in detail [here](overview.md#functionality).
52+
Below is a matrix indicating which functionality is supported by the IKV online store.
53+
54+
| | IKV |
55+
| :-------------------------------------------------------- | :---- |
56+
| write feature values to the online store | yes |
57+
| read feature values from the online store | yes |
58+
| update infrastructure (e.g. tables) in the online store | yes |
59+
| teardown infrastructure (e.g. tables) in the online store | yes |
60+
| generate a plan of infrastructure changes | no |
61+
| support for on-demand transforms | yes |
62+
| readable by Python SDK | yes |
63+
| readable by Java | no |
64+
| readable by Go | no |
65+
| support for entityless feature views | yes |
66+
| support for concurrent writing to the same key | yes |
67+
| support for ttl (time to live) at retrieval | no |
68+
| support for deleting expired data | no |
69+
| collocated by feature view | no |
70+
| collocated by feature service | no |
71+
| collocated by entity key | yes |
72+
73+
To compare this set of functionality against other online stores, please see the full [functionality matrix](overview.md#functionality-matrix).

‎docs/reference/online-stores/overview.md‎

Lines changed: 19 additions & 19 deletions
Original file line numberDiff line numberDiff line change
@@ -29,26 +29,26 @@ See this [issue](https://github.com/feast-dev/feast/issues/2254) for a discussio
2929
## Functionality Matrix
3030

3131
There are currently five core online store implementations: `SqliteOnlineStore`, `RedisOnlineStore`, `DynamoDBOnlineStore`, `SnowflakeOnlineStore`, and `DatastoreOnlineStore`.
32-
There are several additional implementations contributed by the Feast community (`PostgreSQLOnlineStore`, `HbaseOnlineStore`, and `CassandraOnlineStore`), which are not guaranteed to be stable or to match the functionality of the core implementations.
32+
There are several additional implementations contributed by the Feast community (`PostgreSQLOnlineStore`, `HbaseOnlineStore`, `CassandraOnlineStore` and `IKVOnlineStore`), which are not guaranteed to be stable or to match the functionality of the core implementations.
3333
Details for each specific online store, such as how to configure it in a `feature_store.yaml`, can be found [here](README.md).
3434

3535
Below is a matrix indicating which online stores support what functionality.
3636

37-
| | Sqlite | Redis | DynamoDB | Snowflake | Datastore | Postgres | Hbase | [[Cassandra](https://cassandra.apache.org/_/index.html) / [Astra DB](https://www.datastax.com/products/datastax-astra?utm_source=feast)] |
38-
| :-------------------------------------------------------- | :-- | :-- | :-- | :-- | :-- | :-- | :-- | :-- |
39-
| write feature values to the online store | yes | yes | yes | yes | yes | yes | yes | yes |
40-
| read feature values from the online store | yes | yes | yes | yes | yes | yes | yes | yes |
41-
| update infrastructure (e.g. tables) in the online store | yes | yes | yes | yes | yes | yes | yes | yes |
42-
| teardown infrastructure (e.g. tables) in the online store | yes | yes | yes | yes | yes | yes | yes | yes |
43-
| generate a plan of infrastructure changes | yes | no | no | no | no | no | no | yes |
44-
| support for on-demand transforms | yes | yes | yes | yes | yes | yes | yes | yes |
45-
| readable by Python SDK | yes | yes | yes | yes | yes | yes | yes | yes |
46-
| readable by Java | no | yes | no | no | no | no | no | no |
47-
| readable by Go | yes | yes | no | no | no | no | no | no |
48-
| support for entityless feature views | yes | yes | yes | yes | yes | yes | yes | yes |
49-
| support for concurrent writing to the same key | no | yes | no | no | no | no | no | no |
50-
| support for ttl (time to live) at retrieval | no | yes | no | no | no | no | no | no |
51-
| support for deleting expired data | no | yes | no | no | no | no | no | no |
52-
| collocated by feature view | yes | no | yes | yes | yes | yes | yes | yes |
53-
| collocated by feature service | no | no | no | no | no | no | no | no |
54-
| collocated by entity key | no | yes | no | no | no | no | no | no |
37+
| | Sqlite | Redis | DynamoDB | Snowflake | Datastore | Postgres | Hbase | [[Cassandra](https://cassandra.apache.org/_/index.html) / [Astra DB](https://www.datastax.com/products/datastax-astra?utm_source=feast)] | [IKV](https://inlined.io) |
38+
| :-------------------------------------------------------- | :-- | :-- | :-- | :-- | :-- | :-- | :-- | :-- | :-- |
39+
| write feature values to the online store | yes | yes | yes | yes | yes | yes | yes | yes | yes |
40+
| read feature values from the online store | yes | yes | yes | yes | yes | yes | yes | yes | yes |
41+
| update infrastructure (e.g. tables) in the online store | yes | yes | yes | yes | yes | yes | yes | yes | yes |
42+
| teardown infrastructure (e.g. tables) in the online store | yes | yes | yes | yes | yes | yes | yes | yes | yes |
43+
| generate a plan of infrastructure changes | yes | no | no | no | no | no | no | yes | no |
44+
| support for on-demand transforms | yes | yes | yes | yes | yes | yes | yes | yes | yes |
45+
| readable by Python SDK | yes | yes | yes | yes | yes | yes | yes | yes | yes |
46+
| readable by Java | no | yes | no | no | no | no | no | no | no |
47+
| readable by Go | yes | yes | no | no | no | no | no | no | no |
48+
| support for entityless feature views | yes | yes | yes | yes | yes | yes | yes | yes | yes |
49+
| support for concurrent writing to the same key | no | yes | no | no | no | no | no | no | yes |
50+
| support for ttl (time to live) at retrieval | no | yes | no | no | no | no | no | no | no |
51+
| support for deleting expired data | no | yes | no | no | no | no | no | no | no |
52+
| collocated by feature view | yes | no | yes | yes | yes | yes | yes | yes | no |
53+
| collocated by feature service | no | no | no | no | no | no | no | no | no |
54+
| collocated by entity key | no | yes | no | no | no | no | no | no | yes |

‎docs/roadmap.md‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -33,6 +33,7 @@ The list below contains the functionality that contributors are planning to deve
3333
* [x] [Bigtable](https://docs.feast.dev/reference/online-stores/bigtable)
3434
* [x] [SQLite](https://docs.feast.dev/reference/online-stores/sqlite)
3535
* [x] [Dragonfly](https://docs.feast.dev/reference/online-stores/dragonfly)
36+
* [x] [IKV - Inlined Key Value Store](https://docs.feast.dev/reference/online-stores/ikv)
3637
* [x] [Azure Cache for Redis (community plugin)](https://github.com/Azure/feast-azure)
3738
* [x] [Postgres (contrib plugin)](https://docs.feast.dev/reference/online-stores/postgres)
3839
* [x] [Cassandra / AstraDB (contrib plugin)](https://docs.feast.dev/reference/online-stores/cassandra)

0 commit comments

Comments
 (0)