Skip to content

Commit 5084c1a

Browse files
tokokotqtensor
tokoko
authored and
tqtensor
committed
feat: Add duckdb offline store (feast-dev#3981)
* add ibis and duckdb offline stores Signed-off-by: tokoko <togurg14@freeuni.edu.ge> * add linter ignore rule in ibis Signed-off-by: tokoko <togurg14@freeuni.edu.ge> * add linter ignore rule in ibis Signed-off-by: tokoko <togurg14@freeuni.edu.ge> --------- Signed-off-by: tokoko <togurg14@freeuni.edu.ge>
1 parent e86bbda commit 5084c1a

8 files changed

Lines changed: 463 additions & 1 deletion

File tree

‎Makefile‎

Lines changed: 13 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -186,6 +186,19 @@ test-python-universal-athena:
186186
not test_snowflake" \
187187
sdk/python/tests
188188

189+
test-python-universal-duckdb:
190+
PYTHONPATH='.' \
191+
FULL_REPO_CONFIGS_MODULE=sdk.python.feast.infra.offline_stores.contrib.duckdb_repo_configuration \
192+
FEAST_USAGE=False IS_TEST=True \
193+
python -m pytest -n 8 --integration \
194+
-k "not test_nullable_online_store and \
195+
not gcs_registry and \
196+
not s3_registry and \
197+
not test_snowflake and \
198+
not bigquery and \
199+
not test_spark_materialization_consistency" \
200+
sdk/python/tests
201+
189202
test-python-universal-postgres-offline:
190203
PYTHONPATH='.' \
191204
FULL_REPO_CONFIGS_MODULE=sdk.python.feast.infra.offline_stores.contrib.postgres_repo_configuration \

‎sdk/python/feast/infra/offline_stores/contrib/duckdb_offline_store/__init__.py‎

Whitespace-only changes.

‎sdk/python/feast/infra/offline_stores/contrib/duckdb_offline_store/duckdb.py‎

Lines changed: 17 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,17 @@
1+
import ibis
2+
from pydantic import StrictStr
3+
4+
from feast.infra.offline_stores.contrib.ibis_offline_store.ibis import IbisOfflineStore
5+
from feast.repo_config import FeastConfigBaseModel
6+
7+
8+
class DuckDBOfflineStoreConfig(FeastConfigBaseModel):
9+
type: StrictStr = "duckdb"
10+
# """ Offline store type selector"""
11+
12+
13+
class DuckDBOfflineStore(IbisOfflineStore):
14+
@staticmethod
15+
def setup_ibis_backend():
16+
# there's no need to call setup as duckdb is default ibis backend
17+
ibis.set_backend("duckdb")

‎sdk/python/feast/infra/offline_stores/contrib/duckdb_repo_configuration.py‎

Lines changed: 19 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,19 @@
1+
from feast.infra.offline_stores.contrib.duckdb_offline_store.duckdb import (
2+
DuckDBOfflineStoreConfig,
3+
)
4+
from tests.integration.feature_repos.universal.data_sources.file import ( # noqa: E402
5+
FileDataSourceCreator,
6+
)
7+
8+
9+
class DuckDBDataSourceCreator(FileDataSourceCreator):
10+
def create_offline_store_config(self):
11+
self.duckdb_offline_store_config = DuckDBOfflineStoreConfig()
12+
return self.duckdb_offline_store_config
13+
14+
15+
AVAILABLE_OFFLINE_STORES = [
16+
("local", DuckDBDataSourceCreator),
17+
]
18+
19+
AVAILABLE_ONLINE_STORES = {"sqlite": ({"type": "sqlite"}, None)}

‎sdk/python/feast/infra/offline_stores/contrib/ibis_offline_store/__init__.py‎

Whitespace-only changes.

0 commit comments

Comments
 (0)