Skip to content

Commit 632f88b

Browse files
author
Tsotne Tabidze
committed
Add datastore benchmarks
1 parent 4679831 commit 632f88b

File tree

9 files changed

+875
-8
lines changed

9 files changed

+875
-8
lines changed

‎python/docker/datastore/Dockerfile‎

Lines changed: 14 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,14 @@
1+
FROM python:3.9
2+
3+
RUN pip3 install 'feast[gcp]==0.17.0'
4+
# RUN pip3 install 'git+https://github.com/feast-dev/feast.git@1b98ec94e3573991627d561d6d207126a40a21cf#subdirectory=sdk/python'
5+
RUN pip3 install 'git+https://github.com/feast-dev/feast.git@master#subdirectory=sdk/python'
6+
7+
8+
COPY feature_repos/datastore feature_repo
9+
10+
WORKDIR feature_repo
11+
12+
ENV FEAST_USAGE=False
13+
14+
CMD feast serve --host "0.0.0.0" --port 6566

‎python/docker/datastore/docker-compose.yml‎

Lines changed: 9 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,9 @@
1+
services:
2+
feast:
3+
build:
4+
context: ../..
5+
dockerfile: docker/datastore/Dockerfile
6+
ports:
7+
- "6566-6581:6566"
8+
deploy:
9+
replicas: 16

‎python/docker/dynamo/docker-compose.yml‎

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -1,8 +1,8 @@
11
services:
22
feast:
33
build:
4-
context: ..
5-
dockerfile: docker/Dockerfile
4+
context: ../..
5+
dockerfile: docker/dynamo/Dockerfile
66
ports:
77
- "6566-6581:6566"
88
deploy:

‎python/docker/redis/docker-compose.yml‎

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -1,8 +1,8 @@
11
services:
22
feast:
33
build:
4-
context: ..
5-
dockerfile: docker/Dockerfile
4+
context: ../..
5+
dockerfile: docker/redis/Dockerfile
66
ports:
77
- "6566-6581:6566"
88
deploy:

‎python/feature_repos/datastore/example.py‎

Lines changed: 44 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,44 @@
1+
from google.protobuf.duration_pb2 import Duration
2+
3+
from feast import Entity, Feature, FeatureView, FileSource, ValueType, FeatureService
4+
5+
generated_data_source = FileSource(
6+
path="../../../generated_data.parquet",
7+
event_timestamp_column="event_timestamp",
8+
)
9+
10+
entity = Entity(
11+
name="entity",
12+
value_type=ValueType.INT64,
13+
)
14+
15+
feature_views = [
16+
FeatureView(
17+
name=f"feature_view_{i}",
18+
entities=["entity"],
19+
ttl=Duration(seconds=86400),
20+
features=[
21+
Feature(name=f"feature_{10 * i + j}", dtype=ValueType.INT64)
22+
for j in range(10)
23+
],
24+
online=True,
25+
batch_source=generated_data_source,
26+
)
27+
for i in range(25)
28+
]
29+
30+
feature_services = [
31+
FeatureService(
32+
name=f"feature_service_{i}",
33+
features=feature_views[:5*(i + 1)],
34+
)
35+
for i in range(5)
36+
]
37+
38+
def add_definitions_in_globals():
39+
for i, fv in enumerate(feature_views):
40+
globals()[f"feature_view_{i}"] = fv
41+
for i, fs in enumerate(feature_services):
42+
globals()[f"feature_service_{i}"] = fs
43+
44+
add_definitions_in_globals()

‎python/feature_repos/datastore/feature_store.yaml‎

Lines changed: 10 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,10 @@
1+
registry: data/registry.db
2+
project: feature_repo
3+
provider: gcp
4+
online_store:
5+
type: datastore
6+
offline_store:
7+
type: file
8+
flags:
9+
alpha_features: true
10+
python_feature_server: true

‎python/feature_repos/dynamo/example.py‎

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -3,7 +3,7 @@
33
from feast import Entity, Feature, FeatureView, FileSource, ValueType, FeatureService
44

55
generated_data_source = FileSource(
6-
path="../generated_data.parquet",
6+
path="../../../generated_data.parquet",
77
event_timestamp_column="event_timestamp",
88
)
99

@@ -41,4 +41,4 @@ def add_definitions_in_globals():
4141
for i, fs in enumerate(feature_services):
4242
globals()[f"feature_service_{i}"] = fs
4343

44-
add_definitions_in_globals()
44+
add_definitions_in_globals()

‎python/feature_repos/redis/example.py‎

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -3,7 +3,7 @@
33
from feast import Entity, Feature, FeatureView, FileSource, ValueType, FeatureService
44

55
generated_data_source = FileSource(
6-
path="../generated_data.parquet",
6+
path="../../../generated_data.parquet",
77
event_timestamp_column="event_timestamp",
88
)
99

@@ -41,4 +41,4 @@ def add_definitions_in_globals():
4141
for i, fs in enumerate(feature_services):
4242
globals()[f"feature_service_{i}"] = fs
4343

44-
add_definitions_in_globals()
44+
add_definitions_in_globals()

‎python/full-report-datastore.log‎

Lines changed: 790 additions & 0 deletions
Large diffs are not rendered by default.

0 commit comments

Comments
 (0)