Skip to content

Commit b90facd

Browse files
woopgitbook-bot
woop
authored and
gitbook-bot
committed
GitBook: [0.3-dev] one page modified
1 parent 6a0a3b5 commit b90facd

1 file changed

Lines changed: 258 additions & 23 deletions

File tree

‎docs/getting-started/install-feast.md‎

Lines changed: 258 additions & 23 deletions
Original file line numberDiff line numberDiff line change
@@ -2,7 +2,10 @@
22

33
## Overview
44

5-
This guide will demonstrate installing Feast using [**Minikube**](install-feast.md#minikube)**.** This installation has no external dependencies. One downside is that this installation does not include a historical feature store. It is meant to get users up and running quickly.
5+
This installation guide will demonstrate two ways of installing Feast:
6+
7+
* [**Minikube \(Minimal\)**](install-feast.md#minikube)**:** This installation has no external dependencies, but does not have a historical feature store installed. It allows users to quickly get a feel for Feast.
8+
* \*\*\*\*[**Google Kubernetes Engine \(Recommended\):**](install-feast.md#google-kubernetes-engine) ****The guide that follows is a single cluster Feast installation on Google's GKE. It has Google Cloud specific dependencies like BigQuery, Dataflow, and Google Cloud Storage.
69

710
## Minikube
811

@@ -28,55 +31,55 @@ The following software should be installed prior to starting:
2831

2932
### 1. Set up Minikube
3033

31-
a. Start Minikube. Note the minimum cpu and memory below:
34+
Start Minikube. Note the minimum cpu and memory below:
3235

33-
```text
36+
```bash
3437
minikube start --cpus=3 --memory=4096 --kubernetes-version='v1.15.5'
3538
```
3639

37-
b. Set up your Feast environmental variables
40+
Set up your Feast environmental variables
3841

39-
```text
42+
```bash
4043
export FEAST_IP=$(minikube ip)
4144
export FEAST_CORE_URL=${FEAST_IP}:32090
4245
export FEAST_SERVING_URL=${FEAST_IP}:32091
4346
```
4447

4548
### 2. Install Feast with Helm
4649

47-
a. Clone the [Feast repository](https://github.com/gojek/feast/) and navigate to the `charts` sub-directory:
50+
Clone the [Feast repository](https://github.com/gojek/feast/) and navigate to the `charts` sub-directory:
4851

49-
```text
52+
```bash
5053
git clone https://github.com/gojek/feast.git && \
5154
cd feast && export FEAST_HOME_DIR=$(pwd) && \
5255
cd infra/charts/feast
5356
```
5457

55-
b. Copy the `values-demo.yaml` file for your installation:
58+
Copy the `values-demo.yaml` file for your installation:
5659

57-
```text
60+
```bash
5861
cp values-demo.yaml my-feast-values.yaml
5962
```
6063

61-
c. Update all occurrences of the domain `feast.example.com` inside of `my-feast-values.yaml` with your Minikube IP. This is to allow external access to the services in the cluster. You can find your Minikube IP by running the following command `minikube ip`, or simply replace the text from the command line:
64+
Update all occurrences of the domain `feast.example.com` inside of `my-feast-values.yaml` with your Minikube IP. This is to allow external access to the services in the cluster. You can find your Minikube IP by running the following command `minikube ip`, or simply replace the text from the command line:
6265

63-
```text
66+
```bash
6467
sed -i "s/feast.example.com/${FEAST_IP}/g" my-feast-values.yaml
6568
```
6669

67-
d. Install the Feast Helm chart:
70+
Install the Feast Helm chart:
6871

69-
```text
72+
```bash
7073
helm install --name feast -f my-feast-values.yaml .
7174
```
7275

73-
e. Ensure that the system comes online. This will take a few minutes
76+
Ensure that the system comes online. This will take a few minutes
7477

75-
```text
78+
```bash
7679
watch kubectl get pods
7780
```
7881

79-
```text
82+
```bash
8083
NAME READY STATUS RESTARTS AGE
8184
pod/feast-feast-core-666fd46db4-l58l6 1/1 Running 0 5m
8285
pod/feast-feast-serving-online-84d99ddcbd 1/1 Running 0 6m
@@ -92,26 +95,258 @@ pod/feast-zookeeper-2 1/1 Running 0 5m
9295

9396
### 3. Connect to Feast with the Python SDK
9497

95-
a. Install the Python SDK using pip:
98+
Install the Python SDK using pip:
9699

97-
```text
100+
```bash
98101
pip install -e ${FEAST_HOME_DIR}/sdk/python
99102
```
100103

101-
b. Configure the Feast Python SDK:
104+
Configure the Feast Python SDK:
102105

103-
```text
106+
```bash
104107
feast config set core_url ${FEAST_CORE_URL}
105108
feast config set serving_url ${FEAST_SERVING_URL}
106109
```
107110

108-
c. Make sure that both Feast Core and Feast Serving are connected:
111+
Make sure that both Feast Core and Feast Serving are connected:
112+
113+
```bash
114+
feast version
115+
```
116+
117+
```javascript
118+
{
119+
"sdk": {
120+
"version": "feast 0.3.0"
121+
},
122+
"core": {
123+
"url": "192.168.99.100:32090",
124+
"version": "0.3",
125+
"status": "connected"
126+
},
127+
"serving": {
128+
"url": "192.168.99.100:32091",
129+
"version": "0.3",
130+
"status": "connected"
131+
}
132+
}
133+
```
134+
135+
That's it! You can now start to use Feast!
136+
137+
## Google Kubernetes Engine
138+
139+
### Overview <a id="m_5245424069798496115gmail-overview-1"></a>
140+
141+
This guide will install Feast into a Kubernetes cluster on GCP. It assumes that all of your services will run within a single K8s cluster. Once Feast is installed you will be able to:
142+
143+
* Define and register features.
144+
* Load feature data from both batch and streaming sources.
145+
* Retrieve features for model training.
146+
* Retrieve features for online serving.
147+
148+
{% hint style="info" %}
149+
This guide requires [Google Cloud Platform](https://cloud.google.com/) for installation.
150+
151+
* [BigQuery](https://cloud.google.com/bigquery/) is used for storing historical features.
152+
* [Cloud Dataflow](https://cloud.google.com/dataflow/) is used for running data ingestion jobs.
153+
* [Google Cloud Storage](https://cloud.google.com/storage/) is used for intermediate data storage.
154+
{% endhint %}
155+
156+
### 0. Requirements <a id="m_5245424069798496115gmail-requirements"></a>
157+
158+
1. [Google Cloud SDK ](https://cloud.google.com/sdk/install)installed, authenticated, and configured to the project you will use.
159+
2. [Kubectl](https://kubernetes.io/docs/tasks/tools/install-kubectl/) installed.
160+
3. [Helm](https://helm.sh/3) \(2.16.0 or greater\) installed on your local machine with Tiller installed in your cluster.
161+
162+
### 1. Set up GCP
163+
164+
First define the environmental variables that we will use throughout this installation. Please customize these to reflect your environment.
165+
166+
```bash
167+
export FEAST_GCP_PROJECT_ID=my-gcp-project
168+
export FEAST_GCP_REGION=us-central1
169+
export FEAST_GCP_ZONE=us-central1-a
170+
export FEAST_BIGQUERY_DATASET_ID=feast
171+
export FEAST_GCS_BUCKET=${FEAST_GCP_PROJECT_ID}_feast_bucket
172+
export FEAST_GKE_CLUSTER_NAME=feast
173+
export FEAST_S_ACCOUNT_NAME=feast-sa
174+
```
175+
176+
Create a Google Cloud Storage bucket for Feast to stage data during exports:
177+
178+
```bash
179+
gsutil mb gs://${FEAST_GCS_BUCKET}
180+
```
181+
182+
Create a BigQuery dataset for storing historical features:
183+
184+
```bash
185+
bq mk ${FEAST_BIGQUERY_DATASET_ID}
186+
```
187+
188+
Create the service account that Feast will run as:
189+
190+
```bash
191+
gcloud iam service-accounts create ${FEAST_SERVICE_ACCOUNT_NAME}
192+
193+
gcloud projects add-iam-policy-binding ${FEAST_GCP_PROJECT_ID} \
194+
--member serviceAccount:${FEAST_S_ACCOUNT_NAME}@${FEAST_GCP_PROJECT_ID}.iam.gserviceaccount.com \
195+
--role roles/editor
196+
197+
gcloud iam service-accounts keys create key.json --iam-account \
198+
${FEAST_S_ACCOUNT_NAME}@${FEAST_GCP_PROJECT_ID}.iam.gserviceaccount.com
199+
```
200+
201+
Ensure that [Dataflow API is enabled](https://console.cloud.google.com/apis/api/dataflow.googleapis.com/overview):
202+
203+
```bash
204+
gcloud services enable dataflow.googleapis.com
205+
```
206+
207+
### 2. Set up a Kubernetes \(GKE\) cluster
208+
209+
{% hint style="warning" %}
210+
Provisioning a GKE cluster can expose your services publicly. This guide does not cover securing access to the cluster.
211+
{% endhint %}
212+
213+
Create a GKE cluster:
214+
215+
```bash
216+
gcloud container clusters create ${FEAST_GKE_CLUSTER_NAME} \
217+
--machine-type n1-standard-4
218+
```
219+
220+
Create a secret in the GKE cluster based on your local key `key.json`:
221+
222+
```bash
223+
kubectl create secret generic feast-gcp-service-account --from-file=key.json
224+
```
225+
226+
For this guide we will use `NodePort` for exposing Feast services. In order to do so, we must find an internal IP of at least one GKE node.
227+
228+
```bash
229+
export FEAST_IP=$(kubectl describe nodes | grep InternalIP | awk '{print $2}' | head -n 1)
230+
export FEAST_CORE_URL=${FEAST_IP}:32090
231+
export FEAST_ONLINE_SERVING_URL=${FEAST_IP}:32091
232+
export FEAST_BATCH_SERVING_URL=${FEAST_IP}:32092
233+
```
234+
235+
Confirm that you are able to access this node:
236+
237+
```bash
238+
ping $FEAST_IP
239+
```
240+
241+
```bash
242+
PING 10.123.114.11 (10.203.164.22) 56(84) bytes of data.
243+
64 bytes from 10.123.114.11: icmp_seq=1 ttl=63 time=54.2 ms
244+
64 bytes from 10.123.114.11: icmp_seq=2 ttl=63 time=51.2 ms
245+
```
246+
247+
### 3. Set up Helm
248+
249+
Run the following command to provide Tiller with authorization to install Feast:
250+
251+
```bash
252+
kubectl apply -f - <<EOF
253+
apiVersion: v1
254+
kind: ServiceAccount
255+
metadata:
256+
name: tiller
257+
namespace: kube-system
258+
---
259+
apiVersion: rbac.authorization.k8s.io/v1
260+
kind: ClusterRoleBinding
261+
metadata:
262+
name: tiller
263+
roleRef:
264+
apiGroup: rbac.authorization.k8s.io
265+
kind: ClusterRole
266+
name: cluster-admin
267+
subjects:
268+
- kind: ServiceAccount
269+
name: tiller
270+
namespace: kube-system
271+
EOF
272+
```
273+
274+
Install Tiller:
275+
276+
```bash
277+
helm init --service-account tiller
278+
```
279+
280+
### 4. Install Feast with Helm
281+
282+
Clone the [Feast repository](https://github.com/gojek/feast/) and navigate to the `charts` sub-directory:
283+
284+
```bash
285+
git clone https://github.com/gojek/feast.git && cd feast && \
286+
git checkout 0.3-dev && \
287+
export FEAST_HOME_DIR=$(pwd) && \
288+
cd infra/charts/feast
289+
```
290+
291+
Make a copy of the Helm `values.yaml` so that it can be customized for your Feast deployment:
292+
293+
```bash
294+
cp values.yaml my-feast-values.yaml
295+
```
296+
297+
Update `my-feast-values.yaml` based on your GCP and GKE environment.
298+
299+
* Required fields are paired with comments which indicate whether they need to be replaced.
300+
* All occurrences of `feast.example.com` should be replaced with either your domain name or the IP stored in `$FEAST_IP`.
301+
302+
Install the Feast Helm chart:
303+
304+
```bash
305+
helm install --name feast -f my-feast-values.yaml .
306+
```
307+
308+
Ensure that the system comes online. This will take a few minutes
309+
310+
```bash
311+
watch kubectl get pods
312+
```
313+
314+
```bash
315+
NAME READY STATUS RESTARTS AGE
316+
pod/feast-feast-core-666fd46db4-l58l6 1/1 Running 0 5m
317+
pod/feast-feast-serving-online-84d99ddcbd 1/1 Running 0 6m
318+
pod/feast-kafka-0 1/1 Running 0 3m
319+
pod/feast-kafka-1 1/1 Running 0 4m
320+
pod/feast-kafka-2 1/1 Running 0 4m
321+
pod/feast-postgresql-0 1/1 Running 0 5m
322+
pod/feast-redis-master-0 1/1 Running 0 5m
323+
pod/feast-zookeeper-0 1/1 Running 0 5m
324+
pod/feast-zookeeper-1 1/1 Running 0 5m
325+
pod/feast-zookeeper-2 1/1 Running 0 5m
326+
```
327+
328+
### 5. Connect to Feast with the Python SDK
329+
330+
Install the Python SDK using pip:
331+
332+
```bash
333+
pip install -e ${FEAST_HOME_DIR}/sdk/python
334+
```
335+
336+
Configure the Feast Python SDK:
337+
338+
```bash
339+
feast config set core_url ${FEAST_CORE_URL}
340+
feast config set serving_url ${FEAST_ONLINE_SERVING_URL}
341+
```
342+
343+
Make sure that both Feast Core and Feast Serving are connected:
109344

110-
```text
345+
```bash
111346
feast version
112347
```
113348

114-
```text
349+
```javascript
115350
{
116351
"sdk": {
117352
"version": "feast 0.3.0"

0 commit comments

Comments
 (0)