Skip to content

Commit 824859b

Browse files
ejscribnerejscribner
authored
feat: Add Couchbase as an online store (#4637)
1 parent ca8a7ab commit 824859b

23 files changed

+827
-50
lines changed

‎Makefile‎

Lines changed: 19 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -359,6 +359,25 @@ test-python-universal-singlestore-online:
359359
-k "test_retrieve_online_documents" \
360360
sdk/python/tests/integration/online_store/test_universal_online.py
361361

362+
test-python-universal-couchbase-online:
363+
PYTHONPATH='.' \
364+
FULL_REPO_CONFIGS_MODULE=sdk.python.feast.infra.online_stores.contrib.couchbase_repo_configuration \
365+
PYTEST_PLUGINS=sdk.python.tests.integration.feature_repos.universal.online_store.couchbase \
366+
python -m pytest -n 8 --integration \
367+
-k "not test_universal_cli and \
368+
not test_go_feature_server and \
369+
not test_feature_logging and \
370+
not test_reorder_columns and \
371+
not test_logged_features_validation and \
372+
not test_lambda_materialization_consistency and \
373+
not test_offline_write and \
374+
not test_push_features_to_offline_store and \
375+
not gcs_registry and \
376+
not s3_registry and \
377+
not test_universal_types and \
378+
not test_snowflake" \
379+
sdk/python/tests
380+
362381
test-python-universal:
363382
python -m pytest -n 8 --integration sdk/python/tests
364383

‎docs/SUMMARY.md‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -107,6 +107,7 @@
107107
* [Remote](reference/online-stores/remote.md)
108108
* [PostgreSQL (contrib)](reference/online-stores/postgres.md)
109109
* [Cassandra + Astra DB (contrib)](reference/online-stores/cassandra.md)
110+
* [Couchbase (contrib)](reference/online-stores/couchbase.md)
110111
* [MySQL (contrib)](reference/online-stores/mysql.md)
111112
* [Hazelcast (contrib)](reference/online-stores/hazelcast.md)
112113
* [ScyllaDB (contrib)](reference/online-stores/scylladb.md)

‎docs/reference/online-stores/README.md‎

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -46,6 +46,10 @@ Please see [Online Store](../../getting-started/components/online-store.md) for
4646
[cassandra.md](cassandra.md)
4747
{% endcontent-ref %}
4848

49+
{% content-ref url="couchbase.md" %}
50+
[couchbase.md](couchbase.md)
51+
{% endcontent-ref %}
52+
4953
{% content-ref url="mysql.md" %}
5054
[mysql.md](mysql.md)
5155
{% endcontent-ref %}
@@ -60,6 +64,7 @@ Please see [Online Store](../../getting-started/components/online-store.md) for
6064

6165
{% content-ref url="remote.md" %}
6266
[remote.md](remote.md)
67+
{% endcontent-ref %}
6368

6469
{% content-ref url="singlestore.md" %}
6570
[singlestore.md](singlestore.md)

‎docs/reference/online-stores/couchbase.md‎

Lines changed: 78 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,78 @@
1+
# Couchbase Online Store
2+
> NOTE:
3+
> This is a community-contributed online store that is in alpha development. It is not officially supported by the Feast project.
4+
5+
## Description
6+
The [Couchbase](https://www.couchbase.com/) online store provides support for materializing feature values into a Couchbase Operational cluster for serving online features in real-time.
7+
8+
* Only the latest feature values are persisted
9+
* Features are stored in a document-oriented format
10+
11+
The data model for using Couchbase as an online store follows a document format:
12+
* Document ID: `{project}:{table_name}:{entity_key_hex}:{feature_name}`
13+
* Document Content:
14+
* `metadata`:
15+
* `event_ts` (ISO formatted timestamp)
16+
* `created_ts` (ISO formatted timestamp)
17+
* `feature_name` (String)
18+
* `value` (Base64 encoded protobuf binary)
19+
20+
21+
## Getting started
22+
In order to use this online store, you'll need to run `pip install 'feast[couchbase]'`. You can then get started with the command `feast init REPO_NAME -t couchbase`.
23+
24+
To get started with Couchbase Capella Operational:
25+
1. [Sign up for a Couchbase Capella account](https://docs.couchbase.com/cloud/get-started/create-account.html#sign-up-free-tier)
26+
2. [Deploy an Operational cluster](https://docs.couchbase.com/cloud/get-started/create-account.html#getting-started)
27+
3. [Create a bucket](https://docs.couchbase.com/cloud/clusters/data-service/manage-buckets.html#add-bucket)
28+
- This can be named anything, but must correspond to the bucket described in the `feature_store.yaml` configuration file.
29+
4. [Create cluster access credentials](https://docs.couchbase.com/cloud/clusters/manage-database-users.html#create-database-credentials)
30+
- These credentials should have full access to the bucket created in step 3.
31+
5. [Configure allowed IP addresses](https://docs.couchbase.com/cloud/clusters/allow-ip-address.html)
32+
- You must allow the IP address of the machine running Feast.
33+
34+
## Example
35+
{% code title="feature_store.yaml" %}
36+
```yaml
37+
project: my_feature_repo
38+
registry: data/registry.db
39+
provider: local
40+
online_store:
41+
type: couchbase
42+
connection_string: couchbase://127.0.0.1 # Couchbase connection string, copied from 'Connect' page in Couchbase Capella console
43+
user: Administrator # Couchbase username from access credentials
44+
password: password # Couchbase password from access credentials
45+
bucket_name: feast # Couchbase bucket name, defaults to feast
46+
kv_port: 11210 # Couchbase key-value port, defaults to 11210. Required if custom ports are used.
47+
entity_key_serialization_version: 2
48+
```
49+
{% endcode %}
50+
51+
The full set of configuration options is available in `CouchbaseOnlineStoreConfig`.
52+
53+
54+
## Functionality Matrix
55+
The set of functionality supported by online stores is described in detail [here](overview.md#functionality).
56+
Below is a matrix indicating which functionality is supported by the Couchbase online store.
57+
58+
| | Couchbase |
59+
| :-------------------------------------------------------- | :-------- |
60+
| write feature values to the online store | yes |
61+
| read feature values from the online store | yes |
62+
| update infrastructure (e.g. tables) in the online store | yes |
63+
| teardown infrastructure (e.g. tables) in the online store | yes |
64+
| generate a plan of infrastructure changes | no |
65+
| support for on-demand transforms | yes |
66+
| readable by Python SDK | yes |
67+
| readable by Java | no |
68+
| readable by Go | no |
69+
| support for entityless feature views | yes |
70+
| support for concurrent writing to the same key | yes |
71+
| support for ttl (time to live) at retrieval | no |
72+
| support for deleting expired data | no |
73+
| collocated by feature view | yes |
74+
| collocated by feature service | no |
75+
| collocated by entity key | no |
76+
77+
To compare this set of functionality against other online stores, please see the full [functionality matrix](overview.md#functionality-matrix).
78+

‎sdk/python/docs/source/feast.infra.online_stores.contrib.couchbase_online_store.rst‎

Lines changed: 21 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,21 @@
1+
feast.infra.online\_stores.contrib.couchbase\_online\_store package
2+
===================================================================
3+
4+
Submodules
5+
----------
6+
7+
feast.infra.online\_stores.contrib.couchbase\_online\_store.couchbase module
8+
----------------------------------------------------------------------------
9+
10+
.. automodule:: feast.infra.online_stores.contrib.couchbase_online_store.couchbase
11+
:members:
12+
:undoc-members:
13+
:show-inheritance:
14+
15+
Module contents
16+
---------------
17+
18+
.. automodule:: feast.infra.online_stores.contrib.couchbase_online_store
19+
:members:
20+
:undoc-members:
21+
:show-inheritance:

‎sdk/python/docs/source/feast.infra.online_stores.contrib.rst‎

Lines changed: 9 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -8,6 +8,7 @@ Subpackages
88
:maxdepth: 4
99

1010
feast.infra.online_stores.contrib.cassandra_online_store
11+
feast.infra.online_stores.contrib.couchbase_online_store
1112
feast.infra.online_stores.contrib.hazelcast_online_store
1213
feast.infra.online_stores.contrib.hbase_online_store
1314
feast.infra.online_stores.contrib.ikv_online_store
@@ -24,6 +25,14 @@ feast.infra.online\_stores.contrib.cassandra\_repo\_configuration module
2425
:undoc-members:
2526
:show-inheritance:
2627

28+
feast.infra.online\_stores.contrib.couchbase\_repo\_configuration module
29+
------------------------------------------------------------------------
30+
31+
.. automodule:: feast.infra.online_stores.contrib.couchbase_repo_configuration
32+
:members:
33+
:undoc-members:
34+
:show-inheritance:
35+
2736
feast.infra.online\_stores.contrib.elasticsearch module
2837
-------------------------------------------------------
2938

‎sdk/python/docs/source/feast.infra.online_stores.rst‎

Lines changed: 8 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -84,6 +84,14 @@ feast.infra.online\_stores.sqlite module
8484
:undoc-members:
8585
:show-inheritance:
8686

87+
feast.infra.online\_stores.vector\_store module
88+
-----------------------------------------------
89+
90+
.. automodule:: feast.infra.online_stores.vector_store
91+
:members:
92+
:undoc-members:
93+
:show-inheritance:
94+
8795
Module contents
8896
---------------
8997

‎sdk/python/docs/source/feast.permissions.client.rst‎

Lines changed: 11 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -20,10 +20,10 @@ feast.permissions.client.auth\_client\_manager module
2020
:undoc-members:
2121
:show-inheritance:
2222

23-
feast.permissions.client.auth\_client\_manager\_factory module
24-
--------------------------------------------------------------
23+
feast.permissions.client.client\_auth\_token module
24+
---------------------------------------------------
2525

26-
.. automodule:: feast.permissions.client.auth_client_manager_factory
26+
.. automodule:: feast.permissions.client.client_auth_token
2727
:members:
2828
:undoc-members:
2929
:show-inheritance:
@@ -44,6 +44,14 @@ feast.permissions.client.http\_auth\_requests\_wrapper module
4444
:undoc-members:
4545
:show-inheritance:
4646

47+
feast.permissions.client.intra\_comm\_authentication\_client\_manager module
48+
----------------------------------------------------------------------------
49+
50+
.. automodule:: feast.permissions.client.intra_comm_authentication_client_manager
51+
:members:
52+
:undoc-members:
53+
:show-inheritance:
54+
4755
feast.permissions.client.kubernetes\_auth\_client\_manager module
4856
-----------------------------------------------------------------
4957

‎sdk/python/docs/source/feast.protos.feast.core.rst‎

Lines changed: 16 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -244,6 +244,22 @@ feast.protos.feast.core.Policy\_pb2\_grpc module
244244
:undoc-members:
245245
:show-inheritance:
246246

247+
feast.protos.feast.core.Project\_pb2 module
248+
-------------------------------------------
249+
250+
.. automodule:: feast.protos.feast.core.Project_pb2
251+
:members:
252+
:undoc-members:
253+
:show-inheritance:
254+
255+
feast.protos.feast.core.Project\_pb2\_grpc module
256+
-------------------------------------------------
257+
258+
.. automodule:: feast.protos.feast.core.Project_pb2_grpc
259+
:members:
260+
:undoc-members:
261+
:show-inheritance:
262+
247263
feast.protos.feast.core.Registry\_pb2 module
248264
--------------------------------------------
249265

‎sdk/python/docs/source/feast.rst‎

Lines changed: 32 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -28,6 +28,14 @@ feast.aggregation module
2828
:undoc-members:
2929
:show-inheritance:
3030

31+
feast.arrow\_error\_handler module
32+
----------------------------------
33+
34+
.. automodule:: feast.arrow_error_handler
35+
:members:
36+
:undoc-members:
37+
:show-inheritance:
38+
3139
feast.base\_feature\_view module
3240
--------------------------------
3341

@@ -196,6 +204,14 @@ feast.flags\_helper module
196204
:undoc-members:
197205
:show-inheritance:
198206

207+
feast.grpc\_error\_interceptor module
208+
-------------------------------------
209+
210+
.. automodule:: feast.grpc_error_interceptor
211+
:members:
212+
:undoc-members:
213+
:show-inheritance:
214+
199215
feast.importer module
200216
---------------------
201217

@@ -244,6 +260,14 @@ feast.online\_response module
244260
:undoc-members:
245261
:show-inheritance:
246262

263+
feast.project module
264+
--------------------
265+
266+
.. automodule:: feast.project
267+
:members:
268+
:undoc-members:
269+
:show-inheritance:
270+
247271
feast.project\_metadata module
248272
------------------------------
249273

@@ -292,6 +316,14 @@ feast.repo\_operations module
292316
:undoc-members:
293317
:show-inheritance:
294318

319+
feast.rest\_error\_handler module
320+
---------------------------------
321+
322+
.. automodule:: feast.rest_error_handler
323+
:members:
324+
:undoc-members:
325+
:show-inheritance:
326+
295327
feast.saved\_dataset module
296328
---------------------------
297329

0 commit comments

Comments
 (0)