Expected Behavior
Remote write_to_online_store should round-trip regardless of the feature view's
timestamp column names. Writing works for a local provider (the full dataframe is
converted in-process using the source's configured column names) and should behave the
same when the online store is remote (online_store: { type: remote } / the feature
server /write-to-online-store endpoint).
Current Behavior
Writing to the online store through the remote path fails on the server with:
KeyError: 'Field "created_at" does not exist in schema'
whenever the feature view's batch source sets created_timestamp_column (or
timestamp_field) to anything other than the literals "created" /
"event_timestamp".
Root cause — RemoteOnlineStore.online_write_batch serializes the timestamps into
hardcoded column names when building the request body:
# sdk/python/feast/infra/online_stores/remote.py (online_write_batch)
columnar_data["event_timestamp"].append(_to_naive_utc(event_ts).isoformat())
columnar_data["created"].append(
_to_naive_utc(created_ts).isoformat() if created_ts else None
)
req_body = {"feature_view_name": table.name, "df": columnar_data, ...}
post_remote_online_write(config=config, req_body=req_body)
The server rebuilds the dataframe verbatim and runs the standard conversion:
# sdk/python/feast/feature_server.py
df = pd.DataFrame(request.df) # columns: entities, features, event_timestamp, created
store.write_to_online_store(..., df=df)
# -> _prep_rows_to_write_for_ingestion -> _convert_arrow_to_proto
but _convert_arrow_fv_to_proto (in utils.py) resolves the created-timestamp column
by the feature-view-configured name, not the hardcoded "created":
if feature_view.batch_source.created_timestamp_column:
created_timestamps = [
... table.column(
feature_view.batch_source.created_timestamp_column # e.g. "created_at"
) ...
]
So the client emits the column as created and the server looks it up as created_at
→ KeyError. The timestamp value is not lost — it is on the wire under created;
it is simply labeled with a name the server does not look up. The client write and
the server-side conversion disagree on the timestamp column names.
Server-side traceback:
File ".../feast/feature_server.py", in write_to_online_store
File ".../feast/feature_store.py", in write_to_online_store
File ".../feast/infra/passthrough_provider.py", in _prep_rows_to_write_for_ingestion
File ".../feast/utils.py", in _convert_arrow_fv_to_proto
KeyError: 'Field "created_at" does not exist in schema'
Steps to reproduce
- Define a feature view whose batch source uses non-default timestamp column names,
e.g. PostgreSQLSource(timestamp_field="event_timestamp", created_timestamp_column="created_at", ...).
- Point a client at a remote online store:
online_store: { type: remote, path: <feature-server-url> } (or call the feature server /write-to-online-store).
store.write_to_online_store(fv_name, df=df) where df includes the created_at
column. The client-side conversion succeeds; the server returns HTTP 500 with the
KeyError above. The same write against a local provider succeeds.
Specifications
- Version: 0.64.0 (client and feature server)
- Platform: Linux, Python 3.13
- Subsystem: remote online store (
feast/infra/online_stores/remote.py) /
Python feature server /write-to-online-store (feast/feature_server.py) /
_convert_arrow_fv_to_proto (feast/utils.py)
Possible Solution
In RemoteOnlineStore.online_write_batch, label the timestamp columns using the
feature view's configured fields instead of hardcoded literals:
event_col = table.batch_source.timestamp_field or "event_timestamp"
created_col = table.batch_source.created_timestamp_column or "created"
columnar_data[event_col].append(...)
if created_col:
columnar_data[created_col].append(...)
Alternatively, have the server map the well-known event_timestamp / created
request columns onto the FV's configured names before conversion. Fixing the client to
emit the configured names is the smaller change and keeps the request self-describing.
Expected Behavior
Remote
write_to_online_storeshould round-trip regardless of the feature view'stimestamp column names. Writing works for a local provider (the full dataframe is
converted in-process using the source's configured column names) and should behave the
same when the online store is remote (
online_store: { type: remote }/ the featureserver
/write-to-online-storeendpoint).Current Behavior
Writing to the online store through the remote path fails on the server with:
whenever the feature view's batch source sets
created_timestamp_column(ortimestamp_field) to anything other than the literals"created"/"event_timestamp".Root cause —
RemoteOnlineStore.online_write_batchserializes the timestamps intohardcoded column names when building the request body:
The server rebuilds the dataframe verbatim and runs the standard conversion:
but
_convert_arrow_fv_to_proto(inutils.py) resolves the created-timestamp columnby the feature-view-configured name, not the hardcoded
"created":So the client emits the column as
createdand the server looks it up ascreated_at→
KeyError. The timestamp value is not lost — it is on the wire undercreated;it is simply labeled with a name the server does not look up. The client write and
the server-side conversion disagree on the timestamp column names.
Server-side traceback:
Steps to reproduce
e.g.
PostgreSQLSource(timestamp_field="event_timestamp", created_timestamp_column="created_at", ...).online_store: { type: remote, path: <feature-server-url> }(or call the feature server/write-to-online-store).store.write_to_online_store(fv_name, df=df)wheredfincludes thecreated_atcolumn. The client-side conversion succeeds; the server returns HTTP 500 with the
KeyErrorabove. The same write against a local provider succeeds.Specifications
feast/infra/online_stores/remote.py) /Python feature server
/write-to-online-store(feast/feature_server.py) /_convert_arrow_fv_to_proto(feast/utils.py)Possible Solution
In
RemoteOnlineStore.online_write_batch, label the timestamp columns using thefeature view's configured fields instead of hardcoded literals:
Alternatively, have the server map the well-known
event_timestamp/createdrequest columns onto the FV's configured names before conversion. Fixing the client to
emit the configured names is the smaller change and keeps the request self-describing.