Skip to content

Commit ac0d40c

Browse files
authored
Add Helm charts to Feast Java (feast-dev#2)
* Add Helm charts for Feast Core and Feast Serving Signed-off-by: Willem Pienaar <git@willem.co> * Add validation step for chart versions Signed-off-by: Willem Pienaar <git@willem.co> * Add updated helm docs * Update versions for Java * Fix path problem with script folders Signed-off-by: Willem Pienaar <git@willem.co> * Fix typo in install helm CI Signed-off-by: Willem Pienaar <git@willem.co> * Fix Chart versions
1 parent ee23a9c commit ac0d40c

26 files changed

Lines changed: 1256 additions & 2 deletions

.github/workflows/release.yml

Lines changed: 25 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -132,3 +132,28 @@ jobs:
132132
echo -n ${{ secrets.GPG_PRIVATE_KEY }} > /etc/gpg/private-key
133133
echo -n ${{ secrets.MAVEN_SETTINGS }} > /root/.m2/settings.xml
134134
infra/scripts/publish-java-sdk.sh --revision ${VERSION_WITHOUT_PREFIX}
135+
136+
137+
publish-helm-charts:
138+
runs-on: ubuntu-latest
139+
needs: get-version
140+
env:
141+
HELM_VERSION: v2.17.0
142+
VERSION_WITHOUT_PREFIX: ${{ needs.get-version.outputs.version_without_prefix }}
143+
steps:
144+
- uses: actions/checkout@v2
145+
- uses: GoogleCloudPlatform/github-actions/setup-gcloud@master
146+
with:
147+
version: '290.0.1'
148+
export_default_credentials: true
149+
project_id: ${{ secrets.GCP_PROJECT_ID }}
150+
service_account_key: ${{ secrets.GCP_SA_KEY }}
151+
- run: gcloud auth configure-docker --quiet
152+
- name: Remove previous Helm
153+
run: sudo rm -rf $(which helm)
154+
- name: Install Helm
155+
run: ./infra/scripts/install-helm.sh
156+
- name: Validate all version consistency
157+
run: ./infra/scripts/validate-helm-chart-versions.sh $VERSION_WITHOUT_PREFIX
158+
- name: Publish Helm charts
159+
run: ./infra/scripts/push-helm-charts.sh $VERSION_WITHOUT_PREFIX

datatypes/java/README.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -16,7 +16,7 @@ Dependency Coordinates
1616
<dependency>
1717
<groupId>dev.feast</groupId>
1818
<artifactId>datatypes-java</artifactId>
19-
<version>0.10.0-SNAPSHOT</version>
19+
<version>0.25.0-SNAPSHOT</version>
2020
</dependency>
2121
```
2222

infra/charts/feast-core/Chart.yaml

Lines changed: 10 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,10 @@
1+
apiVersion: v1
2+
description: "Feast Core: Feature registry for Feast."
3+
name: feast-core
4+
version: 0.25.0
5+
appVersion: 0.25.0
6+
keywords:
7+
- machine learning
8+
- big data
9+
- mlops
10+
home: https://github.com/feast-dev/feast-java

infra/charts/feast-core/README.md

Lines changed: 68 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,68 @@
1+
feast-core
2+
==========
3+
Feast Core: Feature registry for Feast.
4+
5+
Current chart version is `0.25.0`
6+
7+
Source code can be found [here](https://github.com/feast-dev/feast-java)
8+
9+
10+
11+
## Chart Values
12+
13+
| Key | Type | Default | Description |
14+
|-----|------|---------|-------------|
15+
| "application-generated.yaml".enabled | bool | `true` | Flag to include Helm generated configuration for http port, Feast database URL, Kafka bootstrap servers and jobs metrics host. This is useful for deployment that uses default configuration for Kafka, Postgres and StatsD exporter. Please set `application-override.yaml` to override this configuration. |
16+
| "application-override.yaml" | object | `{"enabled":true}` | Configuration to override the default [application.yaml](https://github.com/feast-dev/feast/blob/master/core/src/main/resources/application.yml). Will be created as a ConfigMap. `application-override.yaml` has a higher precedence than `application-secret.yaml` |
17+
| "application-secret.yaml" | object | `{"enabled":true}` | Configuration to override the default [application.yaml](https://github.com/feast-dev/feast/blob/master/core/src/main/resources/application.yml). Will be created as a Secret. `application-override.yaml` has a higher precedence than `application-secret.yaml`. It is recommended to either set `application-override.yaml` or `application-secret.yaml` only to simplify config management. |
18+
| "application.yaml".enabled | bool | `true` | Flag to include the default [configuration](https://github.com/feast-dev/feast/blob/master/core/src/main/resources/application.yml). Please set `application-override.yaml` to override this configuration. |
19+
| envOverrides | object | `{}` | Extra environment variables to set |
20+
| image.pullPolicy | string | `"IfNotPresent"` | Image pull policy |
21+
| image.repository | string | `"gcr.io/kf-feast/feast-core"` | Docker image repository |
22+
| image.tag | string | `"develop"` | Image tag |
23+
| ingress.grpc.annotations | object | `{}` | Extra annotations for the ingress |
24+
| ingress.grpc.auth.enabled | bool | `false` | Flag to enable auth |
25+
| ingress.grpc.class | string | `"nginx"` | Which ingress controller to use |
26+
| ingress.grpc.enabled | bool | `false` | Flag to create an ingress resource for the service |
27+
| ingress.grpc.hosts | list | `[]` | List of hostnames to match when routing requests |
28+
| ingress.grpc.https.enabled | bool | `true` | Flag to enable HTTPS |
29+
| ingress.grpc.https.secretNames | object | `{}` | Map of hostname to TLS secret name |
30+
| ingress.grpc.whitelist | string | `""` | Allowed client IP source ranges |
31+
| ingress.http.annotations | object | `{}` | Extra annotations for the ingress |
32+
| ingress.http.auth.authUrl | string | `"http://auth-server.auth-ns.svc.cluster.local/auth"` | URL to an existing authentication service |
33+
| ingress.http.auth.enabled | bool | `false` | Flag to enable auth |
34+
| ingress.http.class | string | `"nginx"` | Which ingress controller to use |
35+
| ingress.http.enabled | bool | `false` | Flag to create an ingress resource for the service |
36+
| ingress.http.hosts | list | `[]` | List of hostnames to match when routing requests |
37+
| ingress.http.https.enabled | bool | `true` | Flag to enable HTTPS |
38+
| ingress.http.https.secretNames | object | `{}` | Map of hostname to TLS secret name |
39+
| ingress.http.whitelist | string | `""` | Allowed client IP source ranges |
40+
| javaOpts | string | `nil` | [JVM options](https://docs.oracle.com/cd/E22289_01/html/821-1274/configuring-the-default-jvm-and-java-arguments.html). For better performance, it is advised to set the min and max heap: <br> `-Xms2048m -Xmx2048m` |
41+
| livenessProbe.enabled | bool | `false` | Flag to enabled the probe |
42+
| livenessProbe.failureThreshold | int | `5` | Min consecutive failures for the probe to be considered failed |
43+
| livenessProbe.initialDelaySeconds | int | `60` | Delay before the probe is initiated |
44+
| livenessProbe.periodSeconds | int | `10` | How often to perform the probe |
45+
| livenessProbe.successThreshold | int | `1` | Min consecutive success for the probe to be considered successful |
46+
| livenessProbe.timeoutSeconds | int | `5` | When the probe times out |
47+
| logLevel | string | `"WARN"` | Default log level, use either one of `DEBUG`, `INFO`, `WARN` or `ERROR` |
48+
| logType | string | `"Console"` | Log format, either `JSON` or `Console` |
49+
| nodeSelector | object | `{}` | Node labels for pod assignment |
50+
| podLabels | object | `{}` | Labels to be added to Feast Core pods |
51+
| postgresql.existingSecret | string | `""` | Existing secret to use for authenticating to Postgres |
52+
| prometheus.enabled | bool | `true` | Flag to enable scraping of Feast Core metrics |
53+
| readinessProbe.enabled | bool | `true` | Flag to enabled the probe |
54+
| readinessProbe.failureThreshold | int | `5` | Min consecutive failures for the probe to be considered failed |
55+
| readinessProbe.initialDelaySeconds | int | `20` | Delay before the probe is initiated |
56+
| readinessProbe.periodSeconds | int | `10` | How often to perform the probe |
57+
| readinessProbe.successThreshold | int | `1` | Min consecutive success for the probe to be considered successful |
58+
| readinessProbe.timeoutSeconds | int | `10` | When the probe times out |
59+
| replicaCount | int | `1` | Number of pods that will be created |
60+
| resources | object | `{}` | CPU/memory [resource requests/limit](https://kubernetes.io/docs/concepts/configuration/manage-compute-resources-container/#resource-requests-and-limits-of-pod-and-container) |
61+
| secrets | list | `[]` | List of Kubernetes secrets to be mounted on Feast Core pods. These secrets will be mounted on /etc/secrets/<secret name>. |
62+
| service.grpc.nodePort | string | `nil` | Port number that each cluster node will listen to |
63+
| service.grpc.port | int | `6565` | Service port for GRPC requests |
64+
| service.grpc.targetPort | int | `6565` | Container port serving GRPC requests |
65+
| service.http.nodePort | string | `nil` | Port number that each cluster node will listen to |
66+
| service.http.port | int | `80` | Service port for HTTP requests |
67+
| service.http.targetPort | int | `8080` | Container port serving HTTP requests and Prometheus metrics |
68+
| service.type | string | `"ClusterIP"` | Kubernetes service type |
Lines changed: 45 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,45 @@
1+
{{/* vim: set filetype=mustache: */}}
2+
{{/*
3+
Expand the name of the chart.
4+
*/}}
5+
{{- define "feast-core.name" -}}
6+
{{- default .Chart.Name .Values.nameOverride | trunc 63 | trimSuffix "-" -}}
7+
{{- end -}}
8+
9+
{{/*
10+
Create a default fully qualified app name.
11+
We truncate at 63 chars because some Kubernetes name fields are limited to this (by the DNS naming spec).
12+
If release name contains chart name it will be used as a full name.
13+
*/}}
14+
{{- define "feast-core.fullname" -}}
15+
{{- if .Values.fullnameOverride -}}
16+
{{- .Values.fullnameOverride | trunc 63 | trimSuffix "-" -}}
17+
{{- else -}}
18+
{{- $name := default .Chart.Name .Values.nameOverride -}}
19+
{{- if contains $name .Release.Name -}}
20+
{{- .Release.Name | trunc 63 | trimSuffix "-" -}}
21+
{{- else -}}
22+
{{- printf "%s-%s" .Release.Name $name | trunc 63 | trimSuffix "-" -}}
23+
{{- end -}}
24+
{{- end -}}
25+
{{- end -}}
26+
27+
{{/*
28+
Create chart name and version as used by the chart label.
29+
*/}}
30+
{{- define "feast-core.chart" -}}
31+
{{- printf "%s-%s" .Chart.Name .Chart.Version | replace "+" "_" | trunc 63 | trimSuffix "-" -}}
32+
{{- end -}}
33+
34+
{{/*
35+
Common labels
36+
*/}}
37+
{{- define "feast-core.labels" -}}
38+
app.kubernetes.io/name: {{ include "feast-core.name" . }}
39+
helm.sh/chart: {{ include "feast-core.chart" . }}
40+
app.kubernetes.io/instance: {{ .Release.Name }}
41+
{{- if .Chart.AppVersion }}
42+
app.kubernetes.io/version: {{ .Chart.AppVersion | quote }}
43+
{{- end }}
44+
app.kubernetes.io/managed-by: {{ .Release.Service }}
45+
{{- end -}}
Lines changed: 68 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,68 @@
1+
{{- /*
2+
This takes an array of three values:
3+
- the top context
4+
- the feast component
5+
- the service protocol
6+
- the ingress context
7+
*/ -}}
8+
{{- define "feast.ingress" -}}
9+
{{- $top := (index . 0) -}}
10+
{{- $component := (index . 1) -}}
11+
{{- $protocol := (index . 2) -}}
12+
{{- $ingressValues := (index . 3) -}}
13+
apiVersion: extensions/v1beta1
14+
kind: Ingress
15+
{{ include "feast.ingress.metadata" . }}
16+
spec:
17+
rules:
18+
{{- range $host := $ingressValues.hosts }}
19+
- host: {{ $host }}
20+
http:
21+
paths:
22+
- path: /
23+
backend:
24+
serviceName: {{ include (printf "feast-%s.fullname" $component) $top }}
25+
servicePort: {{ index $top.Values "service" $protocol "port" }}
26+
{{- end }}
27+
{{- if $ingressValues.https.enabled }}
28+
tls:
29+
{{- range $host := $ingressValues.hosts }}
30+
- secretName: {{ index $ingressValues.https.secretNames $host | default (splitList "." $host | rest | join "-" | printf "%s-tls") }}
31+
hosts:
32+
- {{ $host }}
33+
{{- end }}
34+
{{- end -}}
35+
{{- end -}}
36+
37+
{{- define "feast.ingress.metadata" -}}
38+
{{- $commonMetadata := fromYaml (include "common.metadata" (first .)) }}
39+
{{- $overrides := fromYaml (include "feast.ingress.metadata-overrides" .) -}}
40+
{{- toYaml (merge $overrides $commonMetadata) -}}
41+
{{- end -}}
42+
43+
{{- define "feast.ingress.metadata-overrides" -}}
44+
{{- $top := (index . 0) -}}
45+
{{- $component := (index . 1) -}}
46+
{{- $protocol := (index . 2) -}}
47+
{{- $ingressValues := (index . 3) -}}
48+
{{- $commonFullname := include "common.fullname" $top }}
49+
metadata:
50+
name: {{ $commonFullname }}-{{ $component }}-{{ $protocol }}
51+
annotations:
52+
kubernetes.io/ingress.class: {{ $ingressValues.class | quote }}
53+
{{- if (and (eq $ingressValues.class "nginx") $ingressValues.auth.enabled) }}
54+
nginx.ingress.kubernetes.io/auth-url: {{ $ingressValues.auth.authUrl | quote }}
55+
nginx.ingress.kubernetes.io/auth-response-headers: "x-auth-request-email, x-auth-request-user"
56+
nginx.ingress.kubernetes.io/auth-signin: "https://{{ $ingressValues.auth.signinHost | default (splitList "." (index $ingressValues.hosts 0) | rest | join "." | printf "auth.%s")}}/oauth2/start?rd=/r/$host/$request_uri"
57+
{{- end }}
58+
{{- if (and (eq $ingressValues.class "nginx") $ingressValues.whitelist) }}
59+
nginx.ingress.kubernetes.io/whitelist-source-range: {{ $ingressValues.whitelist | quote -}}
60+
{{- end }}
61+
{{- if (and (eq $ingressValues.class "nginx") (eq $protocol "grpc") ) }}
62+
# TODO: Allow choice of GRPC/GRPCS
63+
nginx.ingress.kubernetes.io/backend-protocol: "GRPC"
64+
{{- end }}
65+
{{- if $ingressValues.annotations -}}
66+
{{ include "common.annote" $ingressValues.annotations | indent 4 }}
67+
{{- end }}
68+
{{- end -}}
Lines changed: 26 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,26 @@
1+
apiVersion: v1
2+
kind: ConfigMap
3+
metadata:
4+
name: {{ template "feast-core.fullname" . }}
5+
namespace: {{ .Release.Namespace }}
6+
labels:
7+
app: {{ template "feast-core.name" . }}
8+
component: core
9+
chart: {{ .Chart.Name }}-{{ .Chart.Version | replace "+" "_" }}
10+
release: {{ .Release.Name }}
11+
heritage: {{ .Release.Service }}
12+
data:
13+
application-generated.yaml: |
14+
{{- if index .Values "application-generated.yaml" "enabled" }}
15+
spring:
16+
datasource:
17+
url: jdbc:postgresql://{{ .Release.Name }}-postgresql:5432/postgres
18+
19+
server:
20+
port: {{ .Values.service.http.targetPort }}
21+
{{- end }}
22+
23+
application-override.yaml: |
24+
{{- if index .Values "application-override.yaml" "enabled" }}
25+
{{- toYaml (index .Values "application-override.yaml") | nindent 4 }}
26+
{{- end }}

0 commit comments

Comments
 (0)