You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Is your feature request related to a problem? Please describe.
Picking this back up from a Slack thread a while ago, where the dedup-only handling of created_timestamp_column was confirmed.
get_historical_features enforces point-in-time correctness on the event timestamp only. When a feature view has a created_timestamp_column, it is used purely as a dedup tiebreaker (MAX(created_timestamp) per event timestamp), never as a filter. There is no created_timestamp <= entity_timestamp predicate anywhere in the join templates (verified on current master, BigQuery template as reference; the other offline stores follow the same pattern).
As a result, a training row at entity timestamp T can be served a value that did not exist yet at T: its event_timestamp <= T but its created_timestamp > T (backfills, late corrections, delayed batch writes). This causes:
Feature leakage: training rows see values that were written after the event.
Non-reproducible training sets: re-running after a backfill silently changes past feature vectors.
Offline/online skew: the online model at T served the older value, offline retrieval returns the newer one.
Concrete example: two rows for the same entity and event timestamp, value 4000 created before the entity timestamp and 6000 created after it. get_historical_features returns 6000.
Describe the solution you'd like
An opt-in, per-query flag, default off:
store.get_historical_features(
entity_df=...,
features=...,
filter_by_created_timestamp=True, # only use feature versions known as of each entity row's timestamp
)
When enabled and the feature view has a created_timestamp_column, add a single predicate to the point-in-time join:
Default False produces identical SQL and results to today. The flag can be threaded through the same optional-kwargs mechanism as the recently added start_date/end_date, so offline stores that have not implemented it are unaffected unless a user explicitly opts in.
Describe alternatives you've considered
A per-FeatureView or config-level setting. Per-query is preferred since the same view may want both leak-free training and "latest known" analysis.
reproducible historical feature queries #2980 proposed a global max_created_time fixed at query time (closed by the stale bot). This proposal is the per-row variant: each entity row is cut off at its own timestamp, which is what you need to emulate what the online store would have served at each event.
We currently run a patched BigQuery join template with this predicate in production. It works but has to be re-synced on every Feast upgrade.
Additional context
Rows with a NULL created_timestamp would be excluded by the predicate; suggest documenting the column must be non-null when the mode is enabled.
Orthogonal to ttl: TTL bounds by event time, this bounds by known time.
I have a PR ready implementing this across the offline stores (SQL templates, ibis-based stores, dask) with tests. Happy to iterate on naming and API surface.
Is your feature request related to a problem? Please describe.
Picking this back up from a Slack thread a while ago, where the dedup-only handling of
created_timestamp_columnwas confirmed.get_historical_featuresenforces point-in-time correctness on the event timestamp only. When a feature view has acreated_timestamp_column, it is used purely as a dedup tiebreaker (MAX(created_timestamp)per event timestamp), never as a filter. There is nocreated_timestamp <= entity_timestamppredicate anywhere in the join templates (verified on current master, BigQuery template as reference; the other offline stores follow the same pattern).As a result, a training row at entity timestamp
Tcan be served a value that did not exist yet atT: itsevent_timestamp <= Tbut itscreated_timestamp > T(backfills, late corrections, delayed batch writes). This causes:Tserved the older value, offline retrieval returns the newer one.Concrete example: two rows for the same entity and event timestamp, value
4000created before the entity timestamp and6000created after it.get_historical_featuresreturns6000.Describe the solution you'd like
An opt-in, per-query flag, default off:
When enabled and the feature view has a
created_timestamp_column, add a single predicate to the point-in-time join:Default
Falseproduces identical SQL and results to today. The flag can be threaded through the same optional-kwargs mechanism as the recently addedstart_date/end_date, so offline stores that have not implemented it are unaffected unless a user explicitly opts in.Describe alternatives you've considered
FeatureViewor config-level setting. Per-query is preferred since the same view may want both leak-free training and "latest known" analysis.max_created_timefixed at query time (closed by the stale bot). This proposal is the per-row variant: each entity row is cut off at its own timestamp, which is what you need to emulate what the online store would have served at each event.Additional context
created_timestampwould be excluded by the predicate; suggest documenting the column must be non-null when the mode is enabled.ttl: TTL bounds by event time, this bounds by known time.