Skip to content

feat: Make batch_source optional in PushSource (#5440) #5454

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
franciscojavierarceo merged 4 commits into from
Jun 17, 2025
Merged

feat: Make batch_source optional in PushSource (#5440) #5454

Changes from 1 commit
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
a512c29
Make batch_source optional in PushSource (#5440)
snehsuresh Jun 14, 2025
2f1d015
docs: make batch_source optional in PushSource
snehsuresh Jun 16, 2025
7cc422c
refactor: remove comment
snehsuresh Jun 16, 2025
2bc0b4a
fix: ensure batch_source is valid DataSource after null check
snehsuresh Jun 16, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
docs: make batch_source optional in PushSource 
Signed-off-by: snehsuresh <snehsuresh02@gmail.com>
  • Loading branch information
@snehsuresh
snehsuresh committed Jun 16, 2025
commit 2f1d015bfce1c302c597f19ee62a617556250ac0
9 changes: 8 additions & 1 deletion docs/reference/data-sources/push.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -6,9 +6,12 @@ Push sources allow feature values to be pushed to the online store and offline s

Push sources can be used by multiple feature views. When data is pushed to a push source, Feast propagates the feature values to all the consuming feature views.

Push sources must have a batch source specified. The batch source will be used for retrieving historical features. Thus users are also responsible for pushing data to a batch data source such as a data warehouse table. When using a push source as a stream source in the definition of a feature view, a batch source doesn't need to be specified in the feature view definition explicitly.
Push sources can optionally have a batch_source specified. If provided, it enables retrieval of historical features and supports materialization from the offline store to the online store. However, if your features are generated post-training or are only needed online (e.g., embeddings), you can omit the batch_source.

When a batch_source is used, users are responsible for ensuring that data is also pushed to a batch data source, such as a data warehouse. Note that when a push source is used as a stream source in a feature view definition, a batch_source does not need to be explicitly specified in the feature view itself.

## Stream sources

Streaming data sources are important sources of feature values. A typical setup with streaming data looks like:

1. Raw events come in (stream 1)
Expand All @@ -20,7 +23,9 @@ Streaming data sources are important sources of feature values. A typical setup
Feast allows users to push features previously registered in a feature view to the online store for fresher features. It also allows users to push batches of stream data to the offline store by specifying that the push be directed to the offline store. This will push the data to the offline store declared in the repository configuration used to initialize the feature store.

## Example (basic)

### Defining a push source

Note that the push schema needs to also include the entity.

```python
Expand All @@ -43,7 +48,9 @@ fv = FeatureView(
```

### Pushing data

Note that the `to` parameter is optional and defaults to online but we can specify these options: `PushMode.ONLINE`, `PushMode.OFFLINE`, or `PushMode.ONLINE_AND_OFFLINE`.

```python
from feast import FeatureStore
import pandas as pd
Expand Down