Skip to content

Commit 5fcc30f

Browse files
lgvitalfeast-ci-bot
lgvital
authored and
feast-ci-bot
committed
Update GKE installation and chart values to work with 0.4.3 (feast-dev#434)
1 parent 833d495 commit 5fcc30f

2 files changed

Lines changed: 105 additions & 41 deletions

File tree

‎docs/getting-started/installing-feast.md‎

Lines changed: 12 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -268,7 +268,7 @@ bq mk ${FEAST_BIGQUERY_DATASET_ID}
268268
Create the service account that Feast will run as:
269269

270270
```bash
271-
gcloud iam service-accounts create ${FEAST_SERVICE_ACCOUNT_NAME}
271+
gcloud iam service-accounts create ${FEAST_S_ACCOUNT_NAME}
272272

273273
gcloud projects add-iam-policy-binding ${FEAST_GCP_PROJECT_ID} \
274274
--member serviceAccount:${FEAST_S_ACCOUNT_NAME}@${FEAST_GCP_PROJECT_ID}.iam.gserviceaccount.com \
@@ -324,6 +324,15 @@ PING 10.123.114.11 (10.203.164.22) 56(84) bytes of data.
324324
64 bytes from 10.123.114.11: icmp_seq=2 ttl=63 time=51.2 ms
325325
```
326326

327+
Add firewall rules in gcloud to open up ports:
328+
```bash
329+
gcloud compute firewall-rules create feast-core-port --allow tcp:32090
330+
gcloud compute firewall-rules create feast-online-port --allow tcp:32091
331+
gcloud compute firewall-rules create feast-batch-port --allow tcp:32092
332+
gcloud compute firewall-rules create feast-redis-port --allow tcp:32101
333+
gcloud compute firewall-rules create feast-kafka-ports --allow tcp:31090-31095
334+
```
335+
327336
### 3. Set up Helm
328337

329338
Run the following command to provide Tiller with authorization to install Feast:
@@ -377,7 +386,8 @@ cp values.yaml my-feast-values.yaml
377386
Update `my-feast-values.yaml` based on your GCP and GKE environment.
378387

379388
* Required fields are paired with comments which indicate whether they need to be replaced.
380-
* All occurrences of `feast.example.com` should be replaced with either your domain name or the IP stored in `$FEAST_IP`.
389+
* All occurrences of `EXTERNAL_IP` should be replaced with either your domain name or the IP stored in `$FEAST_IP`.
390+
* Replace all occurrences of `YOUR_BUCKET_NAME` with your bucket name stored in `$FEAST_GCS_BUCKET`
381391

382392
Install the Feast Helm chart:
383393

@@ -421,4 +431,3 @@ feast config set serving_url ${FEAST_ONLINE_SERVING_URL}
421431
```
422432

423433
That's it! You can now start to use Feast!
424-

‎infra/charts/feast/values.yaml‎

Lines changed: 93 additions & 38 deletions
Original file line numberDiff line numberDiff line change
@@ -2,42 +2,43 @@
22
# - Feast Core
33
# - Feast Serving Online
44
# - Feast Serving Batch
5-
#
5+
#
66
# The configuration for different components can be referenced from:
77
# - charts/feast-core/values.yaml
88
# - charts/feast-serving/values.yaml
99
#
1010
# Note that "feast-serving-online" and "feast-serving-batch" are
1111
# aliases to "feast-serving" chart since in typical scenario two instances
12-
# of Feast Serving: online and batch will be deployed. Both described
12+
# of Feast Serving: online and batch will be deployed. Both described
1313
# using the same chart "feast-serving".
1414
#
1515
# The following are default values for typical Feast deployment, but not
1616
# for production setting. Refer to "values-production.yaml" for recommended
1717
# values in production environment.
18-
#
19-
# Note that the import job by default uses DirectRunner
18+
#
19+
# Note that the import job by default uses DirectRunner
2020
# https://beam.apache.org/documentation/runners/direct/
2121
# in this configuration since it allows Feast to run in more environments
2222
# (unlike DataflowRunner which requires Google Cloud services).
23-
#
24-
# A secret containing Google Cloud service account JSON key is required
25-
# in this configuration.
23+
#
24+
# A secret containing Google Cloud service account JSON key is required
25+
# in this configuration.
2626
# https://cloud.google.com/iam/docs/creating-managing-service-accounts
27-
#
27+
#
2828
# The Google Cloud service account must have the following roles:
2929
# - bigquery.dataEditor
3030
# - bigquery.jobUser
3131
#
3232
# Assuming a service account JSON key file has been downloaded to
3333
# (please name the file key.json):
3434
# /home/user/key.json
35-
#
35+
#
3636
# Run the following command to create the secret in your Kubernetes cluster:
3737
#
3838
# kubectl create secret generic feast-gcp-service-account \
3939
# --from-file=/home/user/key.json
4040
#
41+
# Replace every instance of EXTERNAL_IP with the external IP of your GKE cluster
4142

4243
# ============================================================
4344
# Feast Core
@@ -51,12 +52,15 @@ feast-core:
5152
# to the client. These instances of Feast Serving however can still use
5253
# the same shared Feast Core.
5354
enabled: true
54-
# jvmOptions are options that will be passed to the Java Virtual Machine (JVM)
55+
# Specify what image tag to use. Keep this consistent for all components
56+
image:
57+
tag: "0.4.3"
58+
# jvmOptions are options that will be passed to the Java Virtual Machine (JVM)
5559
# running Feast Core.
5660
#
5761
# For example, it is good practice to set min and max heap size in JVM.
5862
# https://stackoverflow.com/questions/6902135/side-effect-for-increasing-maxpermsize-and-max-heap-size
59-
jvmOptions:
63+
jvmOptions:
6064
- -Xms1024m
6165
- -Xmx1024m
6266
# resources that should be allocated to Feast Core.
@@ -68,18 +72,43 @@ feast-core:
6872
memory: 2048Mi
6973
# gcpServiceAccount is the Google service account that Feast Core will use.
7074
gcpServiceAccount:
71-
# useExistingSecret specifies Feast to use an existing secret containing
75+
# useExistingSecret specifies Feast to use an existing secret containing
7276
# Google Cloud service account JSON key file.
73-
#
77+
#
7478
# This is the only supported option for now to use a service account JSON.
7579
# Feast admin is expected to create this secret before deploying Feast.
7680
useExistingSecret: true
7781
existingSecret:
7882
# name is the secret name of the existing secret for the service account.
79-
name: feast-gcp-service-account
83+
name: feast-gcp-service-account
8084
# key is the secret key of the existing secret for the service account.
8185
# key is normally derived from the file name of the JSON key file.
8286
key: key.json
87+
# Setting service.type to NodePort exposes feast-core service at a static port
88+
service:
89+
type: NodePort
90+
grpc:
91+
# this is the port that is exposed outside of the cluster
92+
nodePort: 32090
93+
# Make kafka externally accessible using NodePort
94+
# Please set EXTERNAL_IP to your cluster's external IP
95+
kafka:
96+
external:
97+
enabled: true
98+
type: NodePort
99+
domain: EXTERNAL_IP
100+
configurationOverrides:
101+
"advertised.listeners": |-
102+
EXTERNAL://EXTERNAL_IP:$((31090 + ${KAFKA_BROKER_ID}))
103+
"listener.security.protocol.map": |-
104+
PLAINTEXT:PLAINTEXT,EXTERNAL:PLAINTEXT
105+
application.yaml:
106+
feast:
107+
stream:
108+
options:
109+
# Point to one of your Kafka brokers
110+
# Please set EXTERNAL_IP to your cluster's external IP
111+
bootstrapServers: EXTERNAL_IP:31090
83112

84113
# ============================================================
85114
# Feast Serving Online
@@ -88,14 +117,22 @@ feast-core:
88117
feast-serving-online:
89118
# enabled specifies whether to install Feast Serving Online component.
90119
enabled: true
120+
# Specify what image tag to use. Keep this consistent for all components
121+
image:
122+
tag: "0.4.3"
91123
# redis.enabled specifies whether Redis should be installed as part of Feast Serving.
92-
#
124+
#
93125
# If enabled is set to "false", Feast admin has to ensure there is an
94126
# existing Redis running outside Feast, that Feast Serving can connect to.
127+
# master.service.type set to NodePort exposes Redis to outside of the cluster
95128
redis:
96129
enabled: true
130+
master:
131+
service:
132+
nodePort: 32101
133+
type: NodePort
97134
# jvmOptions are options that will be passed to the Feast Serving JVM.
98-
jvmOptions:
135+
jvmOptions:
99136
- -Xms1024m
100137
- -Xmx1024m
101138
# resources that should be allocated to Feast Serving.
@@ -105,23 +142,28 @@ feast-serving-online:
105142
memory: 1024Mi
106143
limits:
107144
memory: 2048Mi
145+
# Make service accessible to outside of cluster using NodePort
146+
service:
147+
type: NodePort
148+
grpc:
149+
nodePort: 32091
108150
# store.yaml is the configuration for Feast Store.
109-
#
151+
#
110152
# Refer to this link for more description:
111153
# https://github.com/gojek/feast/blob/79eb4ab5fa3d37102c1dca9968162a98690526ba/protos/feast/core/Store.proto
112154
store.yaml:
113155
name: redis
114156
type: REDIS
115157
redis_config:
116-
# If redis.enabled is set to false, Feast admin should uncomment and
117-
# set the host value to an "existing" Redis instance Feast will use as
118-
# online Store.
119-
#
120-
# Else, if redis.enabled is set to true, no additional configuration is
121-
# required.
158+
# If redis.enabled is set to false, Feast admin should uncomment and
159+
# set the host value to an "existing" Redis instance Feast will use as
160+
# online Store. Also use the correct port for that existing instance.
122161
#
162+
# Else, if redis.enabled is set to true, replace EXTERNAL_IP with your
163+
# cluster's external IP.
123164
# host: redis-host
124-
port: 6379
165+
host: EXTERNAL_IP
166+
port: 32101
125167
subscriptions:
126168
- name: "*"
127169
project: "*"
@@ -134,14 +176,17 @@ feast-serving-online:
134176
feast-serving-batch:
135177
# enabled specifies whether to install Feast Serving Batch component.
136178
enabled: true
179+
# Specify what image tag to use. Keep this consistent for all components
180+
image:
181+
tag: "0.4.3"
137182
# redis.enabled specifies whether Redis should be installed as part of Feast Serving.
138-
#
183+
#
139184
# This is usually set to "false" for Feast Serving Batch because the default
140185
# store is BigQuery.
141186
redis:
142187
enabled: false
143188
# jvmOptions are options that will be passed to the Feast Serving JVM.
144-
jvmOptions:
189+
jvmOptions:
145190
- -Xms1024m
146191
- -Xmx1024m
147192
# resources that should be allocated to Feast Serving.
@@ -151,17 +196,22 @@ feast-serving-batch:
151196
memory: 1024Mi
152197
limits:
153198
memory: 2048Mi
199+
# Make service accessible to outside of cluster using NodePort
200+
service:
201+
type: NodePort
202+
grpc:
203+
nodePort: 32092
154204
# gcpServiceAccount is the service account that Feast Serving will use.
155205
gcpServiceAccount:
156-
# useExistingSecret specifies Feast to use an existing secret containing
206+
# useExistingSecret specifies Feast to use an existing secret containing
157207
# Google Cloud service account JSON key file.
158-
#
208+
#
159209
# This is the only supported option for now to use a service account JSON.
160210
# Feast admin is expected to create this secret before deploying Feast.
161211
useExistingSecret: true
162212
existingSecret:
163213
# name is the secret name of the existing secret for the service account.
164-
name: feast-gcp-service-account
214+
name: feast-gcp-service-account
165215
# key is the secret key of the existing secret for the service account.
166216
# key is normally derived from the file name of the JSON key file.
167217
key: key.json
@@ -172,28 +222,33 @@ feast-serving-batch:
172222
# for a complete list and description of the configuration.
173223
application.yaml:
174224
feast:
175-
jobs:
176-
# staging-location specifies the URI to store intermediate files for
225+
jobs:
226+
# staging-location specifies the URI to store intermediate files for
177227
# batch serving (required if using BigQuery as Store).
178-
#
179-
# Please set the value to an "existing" Google Cloud Storage URI that
228+
#
229+
# Please set the value to an "existing" Google Cloud Storage URI that
180230
# Feast serving has write access to.
181-
staging-location: gs://bucket/path
182-
# Type of store to store job metadata.
231+
staging-location: gs://YOUR_BUCKET_NAME/serving/batch
232+
# Type of store to store job metadata.
183233
#
184-
# This default configuration assumes that Feast Serving Online is
234+
# This default configuration assumes that Feast Serving Online is
185235
# enabled as well. So Feast Serving Batch will share the same
186236
# Redis instance to store job statuses.
187237
store-type: REDIS
238+
store-options:
239+
# Use the externally exposed redis instance deployed by Online service
240+
# Please set EXTERNAL_IP to your cluster's external IP
241+
host: EXTERNAL_IP
242+
port: 32101
188243
# store.yaml is the configuration for Feast Store.
189-
#
244+
#
190245
# Refer to this link for more description:
191246
# https://github.com/gojek/feast/blob/79eb4ab5fa3d37102c1dca9968162a98690526ba/protos/feast/core/Store.proto
192247
store.yaml:
193248
name: bigquery
194249
type: BIGQUERY
195250
bigquery_config:
196-
# project_id specifies the Google Cloud Project. Please set this to the
251+
# project_id specifies the Google Cloud Project. Please set this to the
197252
# project id you are using BigQuery in.
198253
project_id: PROJECT_ID
199254
# dataset_id specifies an "existing" BigQuery dataset Feast Serving Batch

0 commit comments

Comments
 (0)