Skip to content

Commit 4a696dc

Browse files
tchughesivtchughesiv
authored
feat: Add a Kubernetes Operator for the Feast Feature Server (#4145)
* add a kubernetes operator for the feast-feature-server Signed-off-by: Tommy Hughes <tohughes@redhat.com> * operator makefile fixes Signed-off-by: Tommy Hughes <tohughes@redhat.com> * addt'l operator documentation Signed-off-by: Tommy Hughes <tohughes@redhat.com> --------- Signed-off-by: Tommy Hughes <tohughes@redhat.com>
1 parent 690a621 commit 4a696dc

32 files changed

+833
-1
lines changed

‎.github/workflows/publish.yml‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -49,7 +49,7 @@ jobs:
4949
needs: [get-version, publish-python-sdk]
5050
strategy:
5151
matrix:
52-
component: [feature-server, feature-server-python-aws, feature-server-java, feature-transformation-server]
52+
component: [feature-server, feature-server-python-aws, feature-server-java, feature-transformation-server, feast-operator]
5353
env:
5454
MAVEN_CACHE: gs://feast-templocation-kf-feast/.m2.2020-08-19.tar
5555
REGISTRY: feastdev

‎Makefile‎

Lines changed: 12 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -401,6 +401,18 @@ build-feature-server-java-docker:
401401
-t $(REGISTRY)/feature-server-java:$(VERSION) \
402402
-f java/infra/docker/feature-server/Dockerfile --load .
403403

404+
push-feast-operator-docker:
405+
cd infra/feast-operator && \
406+
IMAGE_TAG_BASE=$(REGISTRY)/feast-operator \
407+
VERSION=$(VERSION) \
408+
$(MAKE) docker-push
409+
410+
build-feast-operator-docker:
411+
cd infra/feast-operator && \
412+
IMAGE_TAG_BASE=$(REGISTRY)/feast-operator \
413+
VERSION=$(VERSION) \
414+
$(MAKE) docker-build
415+
404416
# Dev images
405417

406418
build-feature-server-dev:

‎docs/how-to-guides/running-feast-in-production.md‎

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -225,6 +225,8 @@ helm install feast-release feast-charts/feast-feature-server \
225225

226226
This will deploy a single service. The service must have read access to the registry file on cloud storage and to the online store (e.g. via [podAnnotations](https://kubernetes-on-aws.readthedocs.io/en/latest/user-guide/iam-roles.html)). It will keep a copy of the registry in their memory and periodically refresh it, so expect some delays in update propagation in exchange for better performance.
227227

228+
> Alternatively, deploy the same helm chart with a [Kubernetes Operator](/infra/feast-operator).
229+
228230
## 5. Using environment variables in your yaml configuration
229231

230232
You might want to dynamically set parts of your configuration from your environment. For instance to deploy Feast to production and development with the same configuration, but a different server. Or to inject secrets without exposing them in your git repo. To do this, it is possible to use the `${ENV_VAR}` syntax in your `feature_store.yaml` file. For instance:

‎infra/charts/feast-feature-server/README.md‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -17,6 +17,7 @@ A base64 encoded version of the `feature_store.yaml` file is needed. Helm instal
1717
```
1818
helm install feast-feature-server feast-charts/feast-feature-server --set feature_store_yaml_base64=$(base64 feature_store.yaml)
1919
```
20+
> Alternatively, deploy this helm chart with a [Kubernetes Operator](/infra/feast-operator).
2021
2122
## Tutorial
2223
See [here](https://github.com/feast-dev/feast/tree/master/examples/python-helm-demo) for a sample tutorial on testing this helm chart with a demo feature repository and a local Redis instance.

‎infra/feast-operator/.gitignore‎

Lines changed: 14 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,14 @@
1+
2+
# Binaries for programs and plugins
3+
*.exe
4+
*.exe~
5+
*.dll
6+
*.so
7+
*.dylib
8+
bin
9+
10+
# editor and IDE paraphernalia
11+
.idea
12+
*.swp
13+
*.swo
14+
*~

‎infra/feast-operator/Dockerfile‎

Lines changed: 7 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,7 @@
1+
# Build the manager binary
2+
FROM quay.io/operator-framework/helm-operator:v1.34.1
3+
4+
ENV HOME=/opt/helm
5+
COPY watches.yaml ${HOME}/watches.yaml
6+
COPY --from=helmcharts feast-feature-server ${HOME}/helm-charts/feast-feature-server
7+
WORKDIR ${HOME}

‎infra/feast-operator/Makefile‎

Lines changed: 231 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,231 @@
1+
# VERSION defines the project version for the bundle.
2+
# Update this value when you upgrade the version of your project.
3+
# To re-generate a bundle for another specific version without changing the standard setup, you can:
4+
# - use the VERSION as arg of the bundle target (e.g make bundle VERSION=0.0.2)
5+
# - use environment variables to overwrite this value (e.g export VERSION=0.0.2)
6+
VERSION ?= 0.37.1
7+
8+
# CHANNELS define the bundle channels used in the bundle.
9+
# Add a new line here if you would like to change its default config. (E.g CHANNELS = "candidate,fast,stable")
10+
# To re-generate a bundle for other specific channels without changing the standard setup, you can:
11+
# - use the CHANNELS as arg of the bundle target (e.g make bundle CHANNELS=candidate,fast,stable)
12+
# - use environment variables to overwrite this value (e.g export CHANNELS="candidate,fast,stable")
13+
ifneq ($(origin CHANNELS), undefined)
14+
BUNDLE_CHANNELS := --channels=$(CHANNELS)
15+
endif
16+
17+
# DEFAULT_CHANNEL defines the default channel used in the bundle.
18+
# Add a new line here if you would like to change its default config. (E.g DEFAULT_CHANNEL = "stable")
19+
# To re-generate a bundle for any other default channel without changing the default setup, you can:
20+
# - use the DEFAULT_CHANNEL as arg of the bundle target (e.g make bundle DEFAULT_CHANNEL=stable)
21+
# - use environment variables to overwrite this value (e.g export DEFAULT_CHANNEL="stable")
22+
ifneq ($(origin DEFAULT_CHANNEL), undefined)
23+
BUNDLE_DEFAULT_CHANNEL := --default-channel=$(DEFAULT_CHANNEL)
24+
endif
25+
BUNDLE_METADATA_OPTS ?= $(BUNDLE_CHANNELS) $(BUNDLE_DEFAULT_CHANNEL)
26+
27+
# IMAGE_TAG_BASE defines the docker.io namespace and part of the image name for remote images.
28+
# This variable is used to construct full image tags for bundle and catalog images.
29+
#
30+
# For example, running 'make bundle-build bundle-push catalog-build catalog-push' will build and push both
31+
# feastdev/feast-operator-bundle:$VERSION and feastdev/feast-operator-catalog:$VERSION.
32+
IMAGE_TAG_BASE ?= feastdev/feast-operator
33+
34+
# BUNDLE_IMG defines the image:tag used for the bundle.
35+
# You can use it as an arg. (E.g make bundle-build BUNDLE_IMG=<some-registry>/<project-name-bundle>:<tag>)
36+
BUNDLE_IMG ?= $(IMAGE_TAG_BASE)-bundle:v$(VERSION)
37+
38+
# BUNDLE_GEN_FLAGS are the flags passed to the operator-sdk generate bundle command
39+
BUNDLE_GEN_FLAGS ?= -q --overwrite --version $(VERSION) $(BUNDLE_METADATA_OPTS)
40+
41+
# USE_IMAGE_DIGESTS defines if images are resolved via tags or digests
42+
# You can enable this value if you would like to use SHA Based Digests
43+
# To enable set flag to true
44+
USE_IMAGE_DIGESTS ?= false
45+
ifeq ($(USE_IMAGE_DIGESTS), true)
46+
BUNDLE_GEN_FLAGS += --use-image-digests
47+
endif
48+
49+
# Set the Operator SDK version to use. By default, what is installed on the system is used.
50+
# This is useful for CI or a project to utilize a specific version of the operator-sdk toolkit.
51+
OPERATOR_SDK_VERSION ?= v1.34.1
52+
53+
KUSTOMIZE_VERSION ?= v5.2.1
54+
HELM_VERSION ?= v1.34.1
55+
OPM_VERSION ?= v1.23.0
56+
57+
# Image URL to use all building/pushing image targets
58+
IMG ?= $(IMAGE_TAG_BASE):$(VERSION)
59+
60+
.PHONY: all
61+
all: docker-build
62+
63+
##@ General
64+
65+
# The help target prints out all targets with their descriptions organized
66+
# beneath their categories. The categories are represented by '##@' and the
67+
# target descriptions by '##'. The awk commands is responsible for reading the
68+
# entire set of makefiles included in this invocation, looking for lines of the
69+
# file as xyz: ## something, and then pretty-format the target and help. Then,
70+
# if there's a line with ##@ something, that gets pretty-printed as a category.
71+
# More info on the usage of ANSI control characters for terminal formatting:
72+
# https://en.wikipedia.org/wiki/ANSI_escape_code#SGR_parameters
73+
# More info on the awk command:
74+
# http://linuxcommand.org/lc3_adv_awk.php
75+
76+
.PHONY: help
77+
help: ## Display this help.
78+
@awk 'BEGIN {FS = ":.*##"; printf "\nUsage:\n make \033[36m<target>\033[0m\n"} /^[a-zA-Z_0-9-]+:.*?##/ { printf " \033[36m%-15s\033[0m %s\n", $$1, $$2 } /^##@/ { printf "\n\033[1m%s\033[0m\n", substr($$0, 5) } ' $(MAKEFILE_LIST)
79+
80+
##@ Build
81+
82+
.PHONY: run
83+
run: helm-operator ## Run against the configured Kubernetes cluster in ~/.kube/config
84+
$(HELM_OPERATOR) run
85+
86+
.PHONY: docker-build
87+
docker-build: ## Build docker image with the manager.
88+
docker build --build-context helmcharts=../charts/ -t ${IMG} .
89+
90+
.PHONY: docker-push
91+
docker-push: ## Push docker image with the manager.
92+
docker push ${IMG}
93+
94+
# PLATFORMS defines the target platforms for the manager image be build to provide support to multiple
95+
# architectures. (i.e. make docker-buildx IMG=myregistry/mypoperator:0.0.1). To use this option you need to:
96+
# - able to use docker buildx . More info: https://docs.docker.com/build/buildx/
97+
# - have enable BuildKit, More info: https://docs.docker.com/develop/develop-images/build_enhancements/
98+
# - be able to push the image for your registry (i.e. if you do not inform a valid value via IMG=<myregistry/image:<tag>> than the export will fail)
99+
# To properly provided solutions that supports more than one platform you should use this option.
100+
PLATFORMS ?= linux/arm64,linux/amd64,linux/s390x,linux/ppc64le
101+
.PHONY: docker-buildx
102+
docker-buildx: ## Build and push docker image for the manager for cross-platform support
103+
- docker buildx create --name project-v3-builder
104+
- docker buildx use project-v3-builder
105+
- docker buildx build --push --platform=$(PLATFORMS) --build-context helmcharts=../charts/ --tag ${IMG} -f Dockerfile .
106+
- docker buildx rm project-v3-builder
107+
108+
##@ Deployment
109+
110+
.PHONY: install
111+
install: kustomize ## Install CRDs into the K8s cluster specified in ~/.kube/config.
112+
$(KUSTOMIZE) build config/crd | kubectl apply -f -
113+
114+
.PHONY: uninstall
115+
uninstall: kustomize ## Uninstall CRDs from the K8s cluster specified in ~/.kube/config.
116+
$(KUSTOMIZE) build config/crd | kubectl delete -f -
117+
118+
.PHONY: deploy
119+
deploy: kustomize ## Deploy controller to the K8s cluster specified in ~/.kube/config.
120+
cd config/manager && $(KUSTOMIZE) edit set image controller=${IMG}
121+
$(KUSTOMIZE) build config/default | kubectl apply -f -
122+
123+
.PHONY: undeploy
124+
undeploy: kustomize ## Undeploy controller from the K8s cluster specified in ~/.kube/config.
125+
$(KUSTOMIZE) build config/default | kubectl delete -f -
126+
127+
OS := $(shell uname -s | tr '[:upper:]' '[:lower:]')
128+
ARCH := $(shell uname -m | sed 's/x86_64/amd64/' | sed 's/aarch64/arm64/')
129+
130+
.PHONY: kustomize
131+
KUSTOMIZE = $(shell pwd)/bin/kustomize
132+
kustomize: ## Download kustomize locally if necessary.
133+
ifeq (,$(wildcard $(KUSTOMIZE)))
134+
ifeq (,$(shell which kustomize 2>/dev/null))
135+
@{ \
136+
set -e ;\
137+
mkdir -p $(dir $(KUSTOMIZE)) ;\
138+
curl -sSLo - https://github.com/kubernetes-sigs/kustomize/releases/download/kustomize/$(KUSTOMIZE_VERSION)/kustomize_$(KUSTOMIZE_VERSION)_$(OS)_$(ARCH).tar.gz | \
139+
tar xzf - -C bin/ ;\
140+
}
141+
else
142+
KUSTOMIZE = $(shell which kustomize)
143+
endif
144+
endif
145+
146+
.PHONY: helm-operator
147+
HELM_OPERATOR = $(shell pwd)/bin/helm-operator
148+
helm-operator: ## Download helm-operator locally if necessary, preferring the $(pwd)/bin path over global if both exist.
149+
ifeq (,$(wildcard $(HELM_OPERATOR)))
150+
ifeq (,$(shell which helm-operator 2>/dev/null))
151+
@{ \
152+
set -e ;\
153+
mkdir -p $(dir $(HELM_OPERATOR)) ;\
154+
curl -sSLo $(HELM_OPERATOR) https://github.com/operator-framework/operator-sdk/releases/download/$(HELM_VERSION)/helm-operator_$(OS)_$(ARCH) ;\
155+
chmod +x $(HELM_OPERATOR) ;\
156+
}
157+
else
158+
HELM_OPERATOR = $(shell which helm-operator)
159+
endif
160+
endif
161+
162+
.PHONY: operator-sdk
163+
OPERATOR_SDK ?= $(shell pwd)/bin/operator-sdk
164+
operator-sdk: ## Download operator-sdk locally if necessary.
165+
ifeq (,$(wildcard $(OPERATOR_SDK)))
166+
ifeq (, $(shell which operator-sdk 2>/dev/null))
167+
@{ \
168+
set -e ;\
169+
mkdir -p $(dir $(OPERATOR_SDK)) ;\
170+
curl -sSLo $(OPERATOR_SDK) https://github.com/operator-framework/operator-sdk/releases/download/$(OPERATOR_SDK_VERSION)/operator-sdk_$(OS)_$(ARCH) ;\
171+
chmod +x $(OPERATOR_SDK) ;\
172+
}
173+
else
174+
OPERATOR_SDK = $(shell which operator-sdk)
175+
endif
176+
endif
177+
178+
.PHONY: bundle
179+
bundle: kustomize operator-sdk ## Generate bundle manifests and metadata, then validate generated files.
180+
$(OPERATOR_SDK) generate kustomize manifests -q
181+
cd config/manager && $(KUSTOMIZE) edit set image controller=$(IMG)
182+
$(KUSTOMIZE) build config/manifests | $(OPERATOR_SDK) generate bundle $(BUNDLE_GEN_FLAGS)
183+
$(OPERATOR_SDK) bundle validate ./bundle
184+
185+
.PHONY: bundle-build
186+
bundle-build: ## Build the bundle image.
187+
docker build -f bundle.Dockerfile -t $(BUNDLE_IMG) .
188+
189+
.PHONY: bundle-push
190+
bundle-push: ## Push the bundle image.
191+
$(MAKE) docker-push IMG=$(BUNDLE_IMG)
192+
193+
.PHONY: opm
194+
OPM = $(shell pwd)/bin/opm
195+
opm: ## Download opm locally if necessary.
196+
ifeq (,$(wildcard $(OPM)))
197+
ifeq (,$(shell which opm 2>/dev/null))
198+
@{ \
199+
set -e ;\
200+
mkdir -p $(dir $(OPM)) ;\
201+
curl -sSLo $(OPM) https://github.com/operator-framework/operator-registry/releases/download/$(OPM_VERSION)/$(OS)-$(ARCH)-opm ;\
202+
chmod +x $(OPM) ;\
203+
}
204+
else
205+
OPM = $(shell which opm)
206+
endif
207+
endif
208+
209+
# A comma-separated list of bundle images (e.g. make catalog-build BUNDLE_IMGS=example.com/operator-bundle:v0.1.0,example.com/operator-bundle:v0.2.0).
210+
# These images MUST exist in a registry and be pull-able.
211+
BUNDLE_IMGS ?= $(BUNDLE_IMG)
212+
213+
# The image tag given to the resulting catalog image (e.g. make catalog-build CATALOG_IMG=example.com/operator-catalog:v0.2.0).
214+
CATALOG_IMG ?= $(IMAGE_TAG_BASE)-catalog:v$(VERSION)
215+
216+
# Set CATALOG_BASE_IMG to an existing catalog image tag to add $BUNDLE_IMGS to that image.
217+
ifneq ($(origin CATALOG_BASE_IMG), undefined)
218+
FROM_INDEX_OPT := --from-index $(CATALOG_BASE_IMG)
219+
endif
220+
221+
# Build a catalog image by adding bundle images to an empty catalog using the operator package manager tool, 'opm'.
222+
# This recipe invokes 'opm' in 'semver' bundle add mode. For more information on add modes, see:
223+
# https://github.com/operator-framework/community-operators/blob/7f1438c/docs/packaging-operator.md#updating-your-existing-operator
224+
.PHONY: catalog-build
225+
catalog-build: opm ## Build a catalog image.
226+
$(OPM) index add --container-tool docker --mode semver --tag $(CATALOG_IMG) --bundles $(BUNDLE_IMGS) $(FROM_INDEX_OPT)
227+
228+
# Push the catalog image.
229+
.PHONY: catalog-push
230+
catalog-push: ## Push a catalog image.
231+
$(MAKE) docker-push IMG=$(CATALOG_IMG)

‎infra/feast-operator/PROJECT‎

Lines changed: 20 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,20 @@
1+
# Code generated by tool. DO NOT EDIT.
2+
# This file is used to track the info used to scaffold your project
3+
# and allow the plugins properly work.
4+
# More info: https://book.kubebuilder.io/reference/project-config.html
5+
domain: feast.dev
6+
layout:
7+
- helm.sdk.operatorframework.io/v1
8+
plugins:
9+
manifests.sdk.operatorframework.io/v2: {}
10+
scorecard.sdk.operatorframework.io/v2: {}
11+
projectName: feast-operator
12+
resources:
13+
- api:
14+
crdVersion: v1
15+
namespaced: true
16+
domain: feast.dev
17+
group: charts
18+
kind: FeastFeatureServer
19+
version: v1alpha1
20+
version: "3"

‎infra/feast-operator/README.md‎

Lines changed: 39 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,39 @@
1+
# Feast Feature Server Helm-based Operator
2+
3+
This Operator was built with the [operator-sdk](https://github.com/operator-framework/operator-sdk) and leverages the [feast-feature-server helm chart](/infra/charts/feast-feature-server).
4+
5+
## Installation
6+
7+
1. __Install [kubectl](https://kubernetes.io/docs/tasks/tools/install-kubectl/)__
8+
2. __Install the Operator on a Kubernetes cluster__
9+
10+
```bash
11+
make deploy
12+
```
13+
14+
3. __Install a Feast Feature Server on Kubernetes__
15+
16+
A base64 encoded version of the `feature_store.yaml` file is required. FeastFeatureServer CR install example:
17+
```bash
18+
cat <<EOF | kubectl create -f -
19+
apiVersion: charts.feast.dev/v1alpha1
20+
kind: FeastFeatureServer
21+
metadata:
22+
name: example
23+
spec:
24+
feature_store_yaml_base64: $(cat feature_store.yaml | base64)
25+
EOF
26+
```
27+
Ensure it was successfully created on the cluster and that the `feature_store_yaml_base64` field was properly set. The following command should return output which is identical to your `feature_store.yaml`:
28+
```bash
29+
kubectl get feastfeatureserver example -o jsonpath={.spec.feature_store_yaml_base64} | base64 -d
30+
```
31+
Watch as the operator creates a running feast feature server:
32+
```bash
33+
kubectl get all
34+
kubectl logs deploy/example-feast-feature-server
35+
```
36+
37+
The `FeastFeatureServer.spec` allows one to configure the [same parameters](https://github.com/feast-dev/feast/tree/master/infra/charts/feast-feature-server#values) as the feast-feature-server helm chart. An example of this can be seen with the included sample [FeastFeatureServer](config/samples/charts_v1alpha1_feastfeatureserver.yaml).
38+
39+
> To install the aforementioned sample FeastFeatureServer, run this command - `kubectl create -f config/samples/charts_v1alpha1_feastfeatureserver.yaml`

0 commit comments

Comments
 (0)