Skip to content

Commit ce35ce6

Browse files
authored
feat: Added batching to feature server /push to offline store (feast-dev#5683) (feast-dev#5729)
* feat: added batching to feature server /push to offline store (feast-dev#5683) Signed-off-by: Jacob Weinhold <29459386+jfw-ppi@users.noreply.github.com> fix: formatting,l int errors (feast-dev#5683) Signed-off-by: Jacob Weinhold <29459386+jfw-ppi@users.noreply.github.com> * Merge branch 'master' into 5683-configurable-batching-feature-server-push Signed-off-by: Jacob Weinhold <29459386+jfw-ppi@users.noreply.github.com> * feat: added batching to feature server /push to offline store ([feast-dev#5683](feast-dev#5683)) Signed-off-by: Jacob Weinhold <29459386+jfw-ppi@users.noreply.github.com> --------- Signed-off-by: Jacob Weinhold <29459386+jfw-ppi@users.noreply.github.com>
1 parent b16fa09 commit ce35ce6

5 files changed

Lines changed: 636 additions & 19 deletions

File tree

docs/reference/feature-servers/python-feature-server.md

Lines changed: 20 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -199,6 +199,26 @@ requests.post(
199199
"http://localhost:6566/push",
200200
data=json.dumps(push_data))
201201
```
202+
#### Offline write batching for `/push`
203+
204+
The Python feature server supports configurable batching for the **offline**
205+
portion of writes executed via the `/push` endpoint.
206+
207+
Only the offline part of a push is affected:
208+
209+
- `to: "offline"`**fully batched**
210+
- `to: "online_and_offline"`**online written immediately**, **offline batched**
211+
- `to: "online"` → unaffected, always immediate
212+
213+
Enable batching in your `feature_store.yaml`:
214+
215+
```yaml
216+
feature_server:
217+
type: local
218+
offline_push_batching_enabled: true
219+
offline_push_batching_batch_size: 1000
220+
offline_push_batching_batch_interval_seconds: 10
221+
```
202222
203223
### Materializing features
204224

docs/reference/feature-store-yaml.md

Lines changed: 16 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -25,6 +25,22 @@ online_store:
2525
* **project\_id** — Optional parameter for the datastore online store. Sets the GCP project id used by Feast, if not set Feast will use the default GCP project id in the local environment.
2626
* **project** — Defines a namespace for the entire feature store. Can be used to isolate multiple deployments in a single installation of Feast.
2727
28+
### feature_server
29+
30+
The `feature_server` block configures the Python Feature Server when it is used
31+
to serve online features and handle `/push` requests. This section is optional
32+
and only applies when running the Python feature server.
33+
34+
An example configuration:
35+
36+
```yaml
37+
feature_server:
38+
type: local
39+
offline_push_batching_enabled: true # Enables batching of offline writes processed by /push. Online writes are unaffected.
40+
offline_push_batching_batch_size: 100 # Maximum number of buffered rows before writing to the offline store.
41+
offline_push_batching_batch_interval_seconds: 5 # Maximum time rows may remain buffered before a forced flush.
42+
```
43+
2844
## Providers
2945

3046
The `provider` field defines the environment in which Feast will execute data flows. As a result, it also determines the default values for other fields.

0 commit comments

Comments
 (0)