diff --git a/docs/how-to-guides/feast-operator/02-persistence.md b/docs/how-to-guides/feast-operator/02-persistence.md index fe8c445e77e..86f037a3b1d 100644 --- a/docs/how-to-guides/feast-operator/02-persistence.md +++ b/docs/how-to-guides/feast-operator/02-persistence.md @@ -217,6 +217,14 @@ For external DB-backed offline stores (BigQuery, Snowflake, Spark, Trino, etc.), `persistence.store.type` and a Secret with the matching key. See [Offline Stores](../reference/offline-stores/) in the SDK docs. +> **Important — contrib store drivers require a custom image.** +> The published `quay.io/feastdev/feature-server` image ships with `feast[minimal]` +> (aws, gcp, snowflake, redis, go, mysql, postgres-c, opentelemetry, +> grpcio, k8s, duckdb, mcp, milvus). Contrib offline stores such as **Trino, Iceberg, +> Spark, Athena, ClickHouse**, and others are **not** included in the base image. To use +> them, build a custom feature-server image that adds the required extras. See +> [Building a custom feature-server image](#building-a-custom-feature-server-image) below. + ### Registry | `type` | Secret key | Notes | @@ -985,6 +993,200 @@ See [Guide 6 — Batch & Jobs](06-batch-and-jobs.md) for full details. --- +## Building a custom feature-server image + +The published `quay.io/feastdev/feature-server` image includes a curated subset of Feast +extras. Contrib store drivers (Trino, Iceberg, Spark, Athena, ClickHouse, etc.) and any +additional Python packages your feature transformations depend on are **not** included. To +use them, build a custom image that extends the base image with the packages you need. + +> **You do not need to clone the Feast repository.** Extend the published base image and +> install extra packages on top. + +### Writing the Dockerfile + +Create a `Dockerfile` in your own infrastructure repository (or the repository that holds +your Feast feature definitions): + +```dockerfile +FROM quay.io/feastdev/feature-server:0.65.0 + +RUN uv pip install --no-cache-dir \ + "feast[trino,redis,iceberg]==0.65.0" +``` + +Pin the Feast version in both the base image tag and the `pip install` command so the +server and client libraries stay in sync. Add any other Python packages your feature +transformations need (ML libraries, internal SDKs, etc.): + +```dockerfile +FROM quay.io/feastdev/feature-server:0.65.0 + +RUN uv pip install --no-cache-dir \ + "feast[trino,redis,iceberg,mlflow]==0.65.0" \ + "scikit-learn>=1.3,<2" \ + "my-internal-sdk==1.2.3" +``` + +### Building and pushing via CI/CD + +The image build and push should run entirely in CI/CD — never from a developer laptop in +a production workflow. + +```yaml +# Example: GitHub Actions +name: Build Feast Feature Server +on: + push: + branches: [main] + paths: + - 'feast/Dockerfile' + +jobs: + build: + runs-on: ubuntu-latest + steps: + - uses: actions/checkout@v4 + - uses: docker/login-action@v3 + with: + registry: registry.example.com + username: ${{ secrets.REGISTRY_USER }} + password: ${{ secrets.REGISTRY_PASSWORD }} + - uses: docker/build-push-action@v5 + with: + context: feast/ + push: true + tags: registry.example.com/feast/feature-server:0.65.0-custom +``` + +### Referencing the custom image in the FeatureStore CR + +Point the operator at the custom image using `server.image` on each service that needs it. +Also set `services.initImage` so the init containers (`feast-init` for git clone/staging +and `feast-apply` for registry updates) use the same custom image — otherwise they run +with the default image which lacks the contrib drivers: + +```yaml +apiVersion: feast.dev/v1 +kind: FeatureStore +metadata: + name: my-feature-store +spec: + feastProject: my_project + feastProjectDir: + git: + url: https://github.com/my-org/feast-feature-repo + ref: + services: + initImage: registry.example.com/feast/feature-server:0.65.0-custom + offlineStore: + server: + image: registry.example.com/feast/feature-server:0.65.0-custom + persistence: + store: + type: trino + secretRef: + name: feast-offline-store + onlineStore: + server: + image: registry.example.com/feast/feature-server:0.65.0-custom + persistence: + store: + type: redis + secretRef: + name: feast-online-store + registry: + local: + server: + image: registry.example.com/feast/feature-server:0.65.0-custom + persistence: + store: + type: sql + secretRef: + name: feast-data-stores +``` + +The `git` section above points to your **feature-definition repository** (not Feast's +GitHub repository). The operator clones that repository at deployment time to load +`feature_store.yaml` and feature definitions. + +Alternatively, set the image once for all services cluster-wide via the operator +environment variable: + +```sh +kubectl set env deployment/feast-operator-controller-manager \ + RELATED_IMAGE_FEATURE_SERVER=registry.example.com/feast/feature-server:0.65.0-custom \ + -n feast-operator-system +``` + +See [Guide 3 — Serving & Observability](03-serving-and-observability.md#container-image-and-resources) +for the full image resolution priority chain. + +### Baking the feature repo into the custom image + +If the cluster cannot clone a Git repository at runtime (air-gapped environments), combine +the custom dependencies with the feature repository in one image: + +```dockerfile +FROM quay.io/feastdev/feature-server:0.65.0 + +RUN uv pip install --no-cache-dir \ + "feast[trino,redis,iceberg]==0.65.0" + +COPY feature_repo/ /opt/feast/feature_repo/ +``` + +Then use the `packaged` provisioning mode so the operator reads the baked-in repo: + +```yaml +spec: + feastProject: my_project + feastProjectDir: + packaged: + image: registry.example.com/feast/feature-server:0.65.0-custom + featureRepoPath: /opt/feast/feature_repo + services: + runFeastApplyOnInit: false # apply is handled separately + offlineStore: + server: {} + persistence: + store: + type: trino + secretRef: + name: feast-offline-store +``` + +See [Guide 1 — Project Provisioning](01-project-provisioning.md#option-c--use-a-repository-packaged-in-an-image-feastprojectdirpackaged) +for full `packaged` details. + +### Included extras in the base image + +For reference, the base `quay.io/feastdev/feature-server` image installs `feast[minimal]`, +which expands to: + +``` +feast[aws, gcp, snowflake, redis, go, mysql, postgres-c, opentelemetry, grpcio, k8s, duckdb, mcp, milvus] +``` + +The `minimal` extra is defined in the Feast +[`pyproject.toml`](https://github.com/feast-dev/feast/blob/stable/pyproject.toml). + +Extras **not** included in `minimal` (require a custom image): + +| Extra | Offline store | +|-------|---------------| +| `trino` | [Trino](../reference/offline-stores/trino.md) | +| `spark` | [Spark](../reference/offline-stores/spark.md) | +| `athena` | [Athena](../reference/offline-stores/athena.md) | +| `mssql` | [MSSQL](../reference/offline-stores/mssql.md) | +| `clickhouse` | [ClickHouse](../reference/offline-stores/clickhouse.md) | +| `oracle` | [Oracle](../reference/offline-stores/oracle.md) | +| `couchbase` | [Couchbase](../reference/offline-stores/couchbase.md) | +| `iceberg` | Apache Iceberg | +| `mlflow` | MLflow model registry | + +--- + ## Troubleshooting | Symptom | Likely cause | Fix | diff --git a/docs/how-to-guides/feast-operator/04-registry-topology.md b/docs/how-to-guides/feast-operator/04-registry-topology.md index a74f8432fe4..99cd10da988 100644 --- a/docs/how-to-guides/feast-operator/04-registry-topology.md +++ b/docs/how-to-guides/feast-operator/04-registry-topology.md @@ -207,6 +207,83 @@ spec: --- +## Client-side configuration (auto-generated ConfigMap) + +When the operator deploys a `FeatureStore` CR, it automatically creates a ConfigMap named +`feast--client` in the same namespace. This ConfigMap contains a ready-to-use +`feature_store.yaml` that points at the deployed remote services (online store, offline +store, and registry). + +For example, a CR named `testing` produces a ConfigMap `feast-testing-client`: + +```yaml +# ConfigMap: feast-testing-client (auto-generated by the operator) +apiVersion: v1 +kind: ConfigMap +metadata: + name: feast-testing-client +data: + feature_store.yaml: | + project: testing + provider: local + online_store: + path: https://feast-testing-online.feast.svc.cluster.local:443 + type: remote + cert: /etc/pki/tls/custom-certs/ca-bundle.crt + registry: + path: feast-testing-registry.feast.svc.cluster.local:443 + registry_type: remote + cert: /etc/pki/tls/custom-certs/ca-bundle.crt + auth: + type: no_auth + entity_key_serialization_version: 3 +``` + +### Using the client ConfigMap in-cluster + +Mount the ConfigMap into your application pod so the Feast SDK discovers the configuration +automatically: + +```yaml +apiVersion: apps/v1 +kind: Deployment +metadata: + name: ml-serving-app +spec: + template: + spec: + containers: + - name: app + volumeMounts: + - name: feast-client-config + mountPath: /opt/feast + readOnly: true + volumes: + - name: feast-client-config + configMap: + name: feast-testing-client +``` + +Then initialize the Feast store from the mounted path: + +```python +from feast import FeatureStore + +store = FeatureStore(repo_path="/opt/feast") +``` + +### Using the client ConfigMap outside the cluster + +Copy the ConfigMap content to your local `feature_store.yaml` for development or +testing outside the cluster. Adjust hostnames and TLS paths as needed (e.g. use +`kubectl port-forward` or an Ingress endpoint): + +```sh +kubectl get configmap feast-testing-client -o jsonpath='{.data.feature_store\.yaml}' > feature_store.yaml +``` + +--- + ## See also - [API reference — `RegistryConfig`](https://github.com/feast-dev/feast/blob/stable/infra/feast-operator/docs/api/markdown/ref.md#registryconfig) diff --git a/docs/how-to-guides/feast-operator/README.md b/docs/how-to-guides/feast-operator/README.md index 25d71c130c7..5a79c3ac960 100644 --- a/docs/how-to-guides/feast-operator/README.md +++ b/docs/how-to-guides/feast-operator/README.md @@ -39,6 +39,8 @@ look for store-specific YAML options in the Feast SDK docs. - **"How do I wire Postgres/Redis/DuckDB as my store?"** → [Guide 2](02-persistence.md) - **"How do I enable Prometheus scraping for the feature server?"** → [Guide 3](03-serving-and-observability.md) - **"How do I make all services share a remote registry?"** → [Guide 4](04-registry-topology.md) +- **"How do I get the client `feature_store.yaml` for connecting to my Feast services?"** → [Guide 4 — Client ConfigMap](04-registry-topology.md#client-side-configuration-auto-generated-configmap) +- **"How do I use a contrib offline store like Trino or Iceberg?"** → [Guide 2 — Custom Image](02-persistence.md#building-a-custom-feature-server-image) - **"How do I enable Kubernetes RBAC or OIDC auth?"** → [Guide 5](05-security.md) - **"How do I schedule nightly materialization?"** → [Guide 6](06-batch-and-jobs.md) - **"How do I send lineage events to Marquez?"** → [Guide 7](07-openlineage-and-materialization.md)