|
| 1 | + |
| 2 | +# Running Feast Python / Go Feature Server with Redis on Kubernetes |
| 3 | + |
| 4 | +For this tutorial, we set up Feast with Redis. |
| 5 | + |
| 6 | +We use the Feast CLI to register and materialize features, and then retrieving via a Feast Python feature server deployed in Kubernetes |
| 7 | + |
| 8 | +## First, let's set up a Redis cluster |
| 9 | +1. Start minikube (`minikube start`) |
| 10 | +2. Use helm to install a default Redis cluster |
| 11 | + ```bash |
| 12 | + helm repo add bitnami https://charts.bitnami.com/bitnami |
| 13 | + helm repo update |
| 14 | + helm install my-redis bitnami/redis |
| 15 | + ``` |
| 16 | +  |
| 17 | +3. Port forward Redis so we can materialize features to it |
| 18 | + |
| 19 | + ```bash |
| 20 | + kubectl port-forward --namespace default svc/my-redis-master 6379:6379 |
| 21 | + ``` |
| 22 | +4. Get your Redis password using the command (pasted below for convenience). We'll need this to tell Feast how to communicate with the cluster. |
| 23 | +
|
| 24 | + ```bash |
| 25 | + export REDIS_PASSWORD=$(kubectl get secret --namespace default my-redis -o jsonpath="{.data.redis-password}" | base64 --decode) |
| 26 | + echo $REDIS_PASSWORD |
| 27 | + ``` |
| 28 | +
|
| 29 | +## Next, we setup a local Feast repo |
| 30 | +1. Install Feast with Redis dependencies `pip install "feast[redis]"` |
| 31 | +2. Make a bucket in GCS (or S3) |
| 32 | +3. The feature repo is already setup here, so you just need to swap in your GCS bucket and Redis credentials. |
| 33 | + We need to modify the `feature_store.yaml`, which has two fields for you to replace: |
| 34 | + ```yaml |
| 35 | + registry: gs://[YOUR GCS BUCKET]/demo-repo/registry.db |
| 36 | + project: feast_python_demo |
| 37 | + provider: gcp |
| 38 | + online_store: |
| 39 | + type: redis |
| 40 | + # Note: this would normally be using instance URL's to access Redis |
| 41 | + connection_string: localhost:6379,password=[YOUR PASSWORD] |
| 42 | + offline_store: |
| 43 | + type: file |
| 44 | + entity_key_serialization_version: 2 |
| 45 | + ``` |
| 46 | +4. Run `feast apply` from within the `feature_repo` directory to apply your local features to the remote registry |
| 47 | + - Note: you may need to authenticate to gcloud first with `gcloud auth login` |
| 48 | +5. Materialize features to the online store: |
| 49 | + ```bash |
| 50 | + CURRENT_TIME=$(date -u +"%Y-%m-%dT%H:%M:%S") |
| 51 | + feast materialize-incremental $CURRENT_TIME |
| 52 | + ``` |
| 53 | + |
| 54 | +## Now let's setup the Feast Server |
| 55 | +1. Add the gcp-auth addon to mount GCP credentials: |
| 56 | + ```bash |
| 57 | + minikube addons enable gcp-auth |
| 58 | + ``` |
| 59 | +2. Add Feast's Python/Go feature server chart repo |
| 60 | + ```bash |
| 61 | + helm repo add feast-charts https://feast-helm-charts.storage.googleapis.com |
| 62 | + helm repo update |
| 63 | + ``` |
| 64 | +3. For this tutorial, because we don't have a direct hosted endpoint into Redis, we need to change `feature_store.yaml` to talk to the Kubernetes Redis service |
| 65 | + ```bash |
| 66 | + sed -i '' 's/localhost:6379/my-redis-master:6379/g' feature_store.yaml |
| 67 | + ``` |
| 68 | +4. Install the Feast helm chart: `helm install feast-release feast-charts/feast-feature-server --set feature_store_yaml_base64=$(base64 feature_store.yaml)` |
| 69 | + > **Dev instructions**: if you're changing the java logic or chart, you can do |
| 70 | + 1. `eval $(minikube docker-env)` |
| 71 | + 2. `make build-feature-server-dev` |
| 72 | + 3. `helm install feast-release ../../../infra/charts/feast-feature-server --set image.tag=dev --set feature_store_yaml_base64=$(base64 feature_store.yaml)` |
| 73 | +5. (Optional): check logs of the server to make sure it’s working |
| 74 | + ```bash |
| 75 | + kubectl logs svc/feast-feature-server |
| 76 | + ``` |
| 77 | +6. Port forward to expose the grpc endpoint: |
| 78 | + ```bash |
| 79 | + kubectl port-forward svc/feast-feature-server 6566:80 |
| 80 | + ``` |
| 81 | +7. Run test fetches for online features:8. |
| 82 | + - First: change back the Redis connection string to allow localhost connections to Redis |
| 83 | + ```bash |
| 84 | + sed -i '' 's/my-redis-master:6379/localhost:6379/g' feature_store.yaml |
| 85 | + ``` |
| 86 | + - Then run the included fetch script, which fetches both via the HTTP endpoint and for comparison, via the Python SDK |
| 87 | + ```bash |
| 88 | + python test_python_fetch.py |
| 89 | + ``` |
0 commit comments