Skip to content

Commit 27e12fe

Browse files
author
Tsotne Tabidze
authored
Update README.md
1 parent 27e1262 commit 27e12fe

File tree

1 file changed

+119
-3
lines changed

1 file changed

+119
-3
lines changed

‎python/README.md‎

Lines changed: 119 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -56,15 +56,131 @@ feast materialize-incremental $(date -u +"%Y-%m-%dT%H:%M:%S")
5656
sed -i 's/localhost:6379/redis:6379/g' feature_store.yaml # make sure to change this back, since it can mess up with feature servers if you run another docker-compose command later.
5757
```
5858

59-
4. Run Benchmarks
59+
4. Check that feature servers are working & they have materialized data
6060
```
61-
cd ../..
61+
cd ../../..
62+
parquet-tools show --columns entity generated_data.parquet 2>/dev/null | head -n 6
63+
```
64+
This should return something like this:
65+
```
66+
+----------+
67+
| entity |
68+
|----------|
69+
| 94 |
70+
| 1992 |
71+
| 4475 |
72+
```
73+
Take your 3 numbers and replace in this query:
74+
```
75+
curl -X POST \
76+
"http://127.0.0.1:6566/get-online-features" \
77+
-H "accept: application/json" \
78+
-d '{
79+
"feature_service": "feature_service_0",
80+
"entities": {
81+
"entity": [94, 1992, 4475]
82+
}
83+
}' | jq
84+
```
85+
86+
In the output, make sure that `"values"` field contains none of the null values. It should look something like this:
87+
```
88+
{
89+
"values": [
90+
4475,
91+
1551,
92+
9889,
93+
```
94+
95+
5. Run Benchmarks
96+
```
97+
cd python
6298
./run-benchmark.sh
6399
```
64100

65101
## AWS DynamoDB
66102

67-
TODO
103+
For this benchmark, you'll need to have AWS credentials configured in `~/.aws/credentials`.
104+
105+
1. Apply feature definitions to create a Feast repo.
106+
```
107+
cd feature_repos/dynamo
108+
feast apply
109+
```
110+
111+
2. Deploy feature servers using docker-compose
112+
```
113+
cd ../../docker/dynamo
114+
docker-compose up -d
115+
```
116+
If everything goes well, you should see an output like this:
117+
```
118+
Creating dynamo_feast_1 ... done
119+
Creating dynamo_feast_2 ... done
120+
Creating dynamo_feast_3 ... done
121+
Creating dynamo_feast_4 ... done
122+
Creating dynamo_feast_5 ... done
123+
Creating dynamo_feast_6 ... done
124+
Creating dynamo_feast_7 ... done
125+
Creating dynamo_feast_8 ... done
126+
Creating dynamo_feast_9 ... done
127+
Creating dynamo_feast_10 ... done
128+
Creating dynamo_feast_11 ... done
129+
Creating dynamo_feast_12 ... done
130+
Creating dynamo_feast_13 ... done
131+
Creating dynamo_feast_14 ... done
132+
Creating dynamo_feast_15 ... done
133+
Creating dynamo_feast_16 ... done
134+
```
135+
136+
3. Materialize data to DynamoDB
137+
```
138+
cd ../../feature_repos/dynamo
139+
feast materialize-incremental $(date -u +"%Y-%m-%dT%H:%M:%S")
140+
```
141+
142+
4. Check that feature servers are working & they have materialized data
143+
```
144+
cd ../../..
145+
parquet-tools show --columns entity generated_data.parquet 2>/dev/null | head -n 6
146+
```
147+
This should return something like this:
148+
```
149+
+----------+
150+
| entity |
151+
|----------|
152+
| 94 |
153+
| 1992 |
154+
| 4475 |
155+
```
156+
Take your 3 numbers and replace in this query:
157+
```
158+
curl -X POST \
159+
"http://127.0.0.1:6566/get-online-features" \
160+
-H "accept: application/json" \
161+
-d '{
162+
"feature_service": "feature_service_0",
163+
"entities": {
164+
"entity": [94, 1992, 4475]
165+
}
166+
}' | jq
167+
```
168+
169+
In the output, make sure that `"values"` field contains none of the null values. It should look something like this:
170+
```
171+
{
172+
"values": [
173+
4475,
174+
1551,
175+
9889,
176+
```
177+
178+
5. Run Benchmarks
179+
```
180+
cd python
181+
./run-benchmark.sh
182+
```
183+
68184

69185
## GCP Datastore
70186

0 commit comments

Comments
 (0)