Skip to content

Commit f669e51

Browse files
woopgitbook-bot
woop
authored and
gitbook-bot
committed
GitBook: [master] 2 pages modified
1 parent 01c4b44 commit f669e51

2 files changed

Lines changed: 85 additions & 95 deletions

File tree

‎docs/README.md‎

Lines changed: 0 additions & 20 deletions
Original file line numberDiff line numberDiff line change
@@ -17,23 +17,3 @@ Feast aims to:
1717

1818
This means that new ML projects start with a process of feature selection from a catalog instead of having to do feature engineering from scratch.
1919

20-
```python
21-
# Setting things up
22-
fs = feast.Client('feast.example.com')
23-
customer_ids = ['1001', '1002', '1003']
24-
customer_features = ['CreditScore', 'Balance', 'Age', 'NumOfProducts', 'IsActive']
25-
from_date = '2019-01-01'
26-
to_date = '2019-12-31'
27-
28-
# Training your model (typically from a notebook or pipeline)
29-
data = fs.get_batch_features(customer_features, customer_ids, from_date, to_date)
30-
my_model = ml.fit(data.to_train(), data.to_train())
31-
32-
# Serving predictions (when serving the model in production)
33-
prediction = my_model.predict(fs.get_online_features(customer_features, customer_ids))
34-
```
35-
36-
The code above is for illustrative purposes. Please see our getting started guide for more realistic examples.
37-
38-
For more reasons to use Feast, please see [Why Feast?](why-feast.md#why-feast)
39-

‎docs/getting-started/installing-feast.md‎

Lines changed: 85 additions & 75 deletions
Original file line numberDiff line numberDiff line change
@@ -8,44 +8,102 @@ This installation guide will demonstrate three ways of installing Feast:
88
* [**Minikube**](installing-feast.md#minikube)**:** This installation has no external dependencies, but does not have a historical feature store installed. It allows users to quickly get a feel for Feast.
99
* [**Google Kubernetes Engine:**](installing-feast.md#google-kubernetes-engine) This guide installs a single cluster Feast installation on Google's GKE. It has Google Cloud specific dependencies like BigQuery, Dataflow, and Google Cloud Storage.
1010

11-
## Docker Compose
11+
## Docker Compose \(Quickstart\)
1212

1313
### Overview
1414

15-
A docker compose file is provided to quickly test Feast with the official docker images. There is no hard dependency on GCP, unless batch serving is required.
15+
A docker compose file is provided to quickly test Feast with the official docker images. There is no hard dependency on GCP, unless batch serving is required. Once you have set up Feast using Docker Compose, you will be able to:
1616

17-
* Define and register feature set
18-
* Feature ingestion
17+
* Create, register, and manage feature sets
18+
* Ingest feature data into Feast
1919
* Retrieve features for online serving
20-
* Updating the feature set
2120

22-
The docker compose setup uses Direct Runner for the Apache Beam jobs.
21+
{% hint style="info" %}
22+
The docker compose setup uses Direct Runner for the Apache Beam jobs. Running Beam with the Direct Runner means it does not need a dedicated runner like Flink or Dataflow, but this comes at the cost of performance. We recommend the use of a full runner when running Feast with very large workloads.
23+
{% endhint %}
2324

2425
### 0. Requirements
2526

26-
1. [Docker compose](https://docs.docker.com/compose/install/) should be installed.
27-
2. TCP ports 6565, 6566, 8888, and 9094 are not in use. Otherwise, modify the port mappings in `infra/docker-compose/docker-compose.yml` to use unoccupied ports.
28-
3. \(optional\) For batch serving you will also need a [GCP service account key](https://cloud.google.com/iam/docs/creating-managing-service-account-keys) that has access to GCS and BigQuery. Port 6567 will be used for the batch serving endpoint.
27+
* [Docker compose](https://docs.docker.com/compose/install/) should be installed.
28+
* TCP ports 6565, 6566, 8888, and 9094 should not be in use. Otherwise, modify the port mappings in `infra/docker-compose/docker-compose.yml` to use unoccupied ports.
29+
* \(for batch serving only\) For batch serving you will also need a [GCP service account key](https://cloud.google.com/iam/docs/creating-managing-service-account-keys) that has access to GCS and BigQuery. Port 6567 will be used for the batch serving endpoint.
30+
* \(for batch serving only\) [Google Cloud SDK ](https://cloud.google.com/sdk/install)installed, authenticated, and configured to the project you will use.
31+
32+
### 1. Step-by-step guide \(Online serving only\)
33+
34+
Clone the [Feast repository](https://github.com/gojek/feast/) and navigate to the `docker-compose` sub-directory:
35+
36+
```bash
37+
git clone https://github.com/gojek/feast.git && \
38+
cd feast && export FEAST_HOME_DIR=$(pwd) && \
39+
cd infra/docker-compose
40+
```
41+
42+
Make a copy of the `.env.sample` file:
43+
44+
```bash
45+
cp .env.sample .env
46+
```
47+
48+
Start Feast:
49+
50+
```javascript
51+
docker-compose up -d
52+
```
53+
54+
A Jupyter notebook is now available to use Feast:
55+
56+
[http://localhost:8888/notebooks/feast-notebooks/feast-quickstart.ipynb](http://localhost:8888/notebooks/feast-notebooks/feast-quickstart.ipynb)
57+
58+
### 2. Step-by-step guide \(Batch and online serving\)
59+
60+
Clone the [Feast repository](https://github.com/gojek/feast/) and navigate to the `docker-compose` sub-directory:
61+
62+
```bash
63+
git clone https://github.com/gojek/feast.git && \
64+
cd feast && export FEAST_HOME_DIR=$(pwd) && \
65+
cd infra/docker-compose
66+
```
67+
68+
Create a [service account ](https://cloud.google.com/iam/docs/creating-managing-service-accounts)from the GCP console and copy it to the `gcp-service-accounts` folder:
69+
70+
```javascript
71+
cp my-service-account.json ${FEAST_HOME_DIR}/infra/docker-compose/gcp-service-accounts
72+
```
73+
74+
Create a Google Cloud Storage bucket. Make sure that your service account above has read/write permissions to this bucket:
75+
76+
```bash
77+
gsutil mb gs://my-feast-staging-bucket
78+
```
79+
80+
Make a copy of the `.env.sample` file:
2981

30-
### 1. Step-by-step guide \(Online serving\)
82+
```bash
83+
cp .env.sample .env
84+
```
3185

32-
1. Navigate to `infra/docker-compose`.
33-
2. Copy `.env.sample` to `.env`.
34-
3. `docker-compose up -d`
35-
4. A Jupyter Notebook server should be accessible via `localhost:8888`
36-
5. Please wait a minute or two for the Feast services to be ready before running the notebook. You will know that the services are ready when port `6565` and `6566` starts listening.
86+
Customize the `.env` file based on your environment. At the very least you have to modify:
87+
88+
* **FEAST\_CORE\_GCP\_SERVICE\_ACCOUNT\_KEY:** This should be your service account file name without the .json extension.
89+
* **FEAST\_BATCH\_SERVING\_GCP\_SERVICE\_ACCOUNT\_KEY:** This should be your service account file name without the .json extension.
90+
* **FEAST\_JUPYTER\_GCP\_SERVICE\_ACCOUNT\_KEY:** This should be your service account file name without the .json extension.
91+
* **FEAST\_JOB\_STAGING\_LOCATION:** Google Cloud Storage bucket that Feast will use to stage data exports and batch retrieval requests.
92+
93+
We will also need to customize the `bq-store.yml` file inside `infra/docker-compose/serving/` to configure the BigQuery storage configuration as well as the feature sets that the store subscribes to. At a minimum you will need to set:
94+
95+
* **project\_id:** This is you GCP project id.
96+
* **dataset\_id:** This is the name of the BigQuery dataset that tables will be created in. Each feature set will have one table in BigQuery.
97+
98+
Start Feast:
99+
100+
```javascript
101+
docker-compose -f docker-compose.yml -f docker-compose.batch.yml up -d
102+
```
37103

38-
### 2. Step-by-step guide \(Batch serving\)
104+
A Jupyter notebook is now available to use Feast:
39105

40-
1. Navigate to `infra/docker-compose`.
41-
2. Copy `.env.sample` to `.env`.
42-
3. Copy your GCP account service key\(s\) to `infra/docker-compose/gcp-service-accounts`.
43-
4. Modify the value of `FEAST_<SERVICE_NAME>_GCP_SERVICE_ACCOUNT_KEY` in your `.env` file. It should be the json file name without extension.
44-
5. Modify the value of `infra/docker-compose/serving/bq-store.yml`. Alternatively, you can also point to a different store configuration file by modifying `FEAST_BATCH_STORE_CONFIG` in your `.env` file.
45-
6. `docker-compose -f docker-compose.yml -f docker-compose.batch.yml up -d`
46-
7. A jupyter notebook server should be accessible via `localhost:8888`
47-
8. Please wait a minute or two for the Feast services to be ready before running the notebook. You will know that the services are ready when port `6565` and `6567` starts listening.
48-
9. When you are done, run `docker-compose -f docker-compose.yml -f docker-compose.batch.yml down` to shutdown the services.
106+
[http://localhost:8888/notebooks/feast-notebooks](http://localhost:8888/tree/feast-notebooks)
49107

50108
## Minikube
51109

@@ -67,7 +125,7 @@ The following software should be installed prior to starting:
67125

68126
1. [Minikube](https://kubernetes.io/docs/tasks/tools/install-minikube/) should be installed.
69127
2. [Kubectl](https://kubernetes.io/docs/tasks/tools/install-kubectl/) installed and configured to work with Minikube.
70-
3. [Helm](https://helm.sh/3) \(2.16.0 or greater\).
128+
3. [Helm](https://helm.sh/3) \(2.16.0 or greater\). Helm 3 has not been tested yet.
71129

72130
### 1. Set up Minikube
73131

@@ -154,30 +212,6 @@ feast config set core_url ${FEAST_CORE_URL}
154212
feast config set serving_url ${FEAST_SERVING_URL}
155213
```
156214

157-
Make sure that both Feast Core and Feast Serving are connected:
158-
159-
```bash
160-
feast version
161-
```
162-
163-
```javascript
164-
{
165-
"sdk": {
166-
"version": "feast 0.3.2"
167-
},
168-
"core": {
169-
"url": "192.168.99.100:32090",
170-
"version": "0.3",
171-
"status": "connected"
172-
},
173-
"serving": {
174-
"url": "192.168.99.100:32091",
175-
"version": "0.3",
176-
"status": "connected"
177-
}
178-
}
179-
```
180-
181215
That's it! You can now start to use Feast!
182216

183217
## Google Kubernetes Engine
@@ -203,7 +237,7 @@ This guide requires [Google Cloud Platform](https://cloud.google.com/) for insta
203237

204238
1. [Google Cloud SDK ](https://cloud.google.com/sdk/install)installed, authenticated, and configured to the project you will use.
205239
2. [Kubectl](https://kubernetes.io/docs/tasks/tools/install-kubectl/) installed.
206-
3. [Helm](https://helm.sh/3) \(2.16.0 or greater\) installed on your local machine with Tiller installed in your cluster.
240+
3. [Helm](https://helm.sh/3) \(2.16.0 or greater\) installed on your local machine with Tiller installed in your cluster. Helm 3 has not been tested yet.
207241

208242
### 1. Set up GCP
209243

@@ -386,29 +420,5 @@ feast config set core_url ${FEAST_CORE_URL}
386420
feast config set serving_url ${FEAST_ONLINE_SERVING_URL}
387421
```
388422

389-
Make sure that both Feast Core and Feast Serving are connected:
390-
391-
```bash
392-
feast version
393-
```
394-
395-
```javascript
396-
{
397-
"sdk": {
398-
"version": "feast 0.3.2"
399-
},
400-
"core": {
401-
"url": "192.168.99.100:32090",
402-
"version": "0.3",
403-
"status": "connected"
404-
},
405-
"serving": {
406-
"url": "192.168.99.100:32091",
407-
"version": "0.3",
408-
"status": "connected"
409-
}
410-
}
411-
```
412-
413423
That's it! You can now start to use Feast!
414424

0 commit comments

Comments
 (0)