Skip to content

feat: Support typed default values for missing feature values - #6716

Draft
addenergyx wants to merge 12 commits into
feast-dev:masterfrom
addenergyx:feat/typed-feature-default-values
Draft

feat: Support typed default values for missing feature values#6716
addenergyx wants to merge 12 commits into
feast-dev:masterfrom
addenergyx:feat/typed-feature-default-values

Conversation

@addenergyx

@addenergyx addenergyx commented Aug 9, 2026

Copy link
Copy Markdown
Contributor

What this PR does / why we need it:

Implements #6715. A field can declare a typed default that Feast substitutes when the feature is missing or null, so every consumer stops imputing it separately and training and serving agree.

Field(name="transaction_count", dtype=Int64, default_value=0)

FeatureSpecV2 gains feast.types.Value default_value = 8. Message presence separates "no default" from a configured 0, 0.0, false or "", so HasField is used rather than truthiness.

Applied in Python online retrieval, Python historical retrieval, and Go serving — in both Python paths before ODFVs run, so a transformation sees the same source value online and offline. Scalar defaults are pushed into the point-in-time query as COALESCE, so warehouse exports stay server-side instead of dragging the result through pandas; anything not expressible as a portable SQL literal falls back to filling in Python. With no defaults declared the generated SQL is byte-identical to before.

Note on string literals: no escaping is portable across the dialects that share this template. BigQuery rejects the SQL-standard '' form, parsing 'O''Brien' as two adjacent literals, while backslash escapes are literal in Trino. A string default containing a quote, backslash or newline is therefore filled in Python rather than pushed down. Plain strings push down normally.

Statuses are unchanged: a defaulted feature still reports NOT_FOUND. #6270 added a Prometheus counter read from that same status vector, so marking defaults PRESENT would blind it exactly when an upstream materialization job breaks. Values outside TTL keep their real value and OUTSIDE_MAX_AGE.

Validation rejects a float an integer dtype would truncate (Int64 with 1.5) and values the type cannot represent, but does not compare round-trips by value — that would reject 0.1 on Float32. Mutable defaults are deep-copied. Two views exposing the same feature name with different defaults raise under full_feature_names=False rather than silently keeping one.

Known gaps

  • Jobs overriding the public methods bypass the fill. Ray, the Ray compute engine and Spark's native path override to_arrow/to_df and short-circuit when there are no ODFVs — exactly the defaults-only case. Ibis and Trino persist write server-side without it. The deeper fix is a non-overridable base hook so a native export path must declare itself faithful; that touches ~15 job classes and belongs in its own PR.
  • The remote offline server persists without defaults, since it builds a job directly rather than through FeatureStore.get_historical_features.
  • ODFV output defaults are not applied. Related: Python marks every transformed output PRESENT regardless of value, which looks like a separate pre-existing bug.
  • RequestSource defaults are not included — synthesizing an omitted request field would turn a caller bug into zeros that look deliberate.
  • Struct/Map defaults are validated only against the coarse ValueType, not the field schema.
  • Go cannot serve every default Python accepts (JSON, Map, Struct, Decimal), and Java serving is not updated.

Separately, IbisRetrievalJob.persist() never consults on_demand_feature_views, so ibis saved datasets appear to drop ODFV columns today, independent of this change. Happy to raise that as its own issue.

Which issue(s) this PR fixes:

Fixes #6715

Checks

  • I've made sure the tests are passing.
  • My commits are signed off (git commit -s)
  • My PR title follows conventional commits format

Testing Strategy

  • Unit tests
  • Integration tests
  • Manual tests
  • Testing is not required for this change

Unit tests cover the schema (round-trip including zero-like and array defaults, old registries without field 8, rejection rules), Python retrieval (missing row, unset value and explicit null_val; statuses preserved; mixed batches; full_feature_names; mismatched Arrow column types; the ODFV ordering guarantee; SQL literal escaping), and the Go path.

Two caveats on how much the green tick is worth:

BigQuery was verified end-to-end against a real dataset — present values untouched, a null on an existing row and a missing entity row both defaulted, plain string defaults pushed down, and the no-defaults baseline still returning nulls. That run is what caught the string-escaping problem above; generated-SQL assertions alone would not have.

Snowflake, Redshift, Athena, Trino, Spark and Clickhouse remain unexecuted — I have no instance of them. Their pushdown is verified as generated SQL only. COALESCE and the remaining literal forms are standard across them, and the no-defaults output is unchanged so existing queries cannot regress, but integration tests are the first real check for the defaulted path.

Regressions were checked differentially — sdk/python/tests/unit run on this branch and on master with the same command, identical failure sets. My environment has pre-existing failures from missing optional extras, so that is not a green suite. go build, go vet and go test ./go/internal/feast/onlineserving/... pass.

Misc

The regenerated _pb2 files use grpcio-tools==1.60.1 / mypy-protobuf==3.3.0 to match the tree, which is all gencode 4.25.1 with no runtime_version validation. A current grpcio-tools emits a runtime check requiring protobuf >= 7, which would break CI (protobuf==6.33.6) and users on the declared >=4.24.0 floor. Only incidental diff is the header moving to 4.25.0. Happy to redo this if there is a preferred pinned toolchain.

Release note:

Feature fields can declare a typed `default_value` that Feast substitutes for missing or null values during online retrieval, historical retrieval and Go serving, applied before on demand feature views run. Scalar defaults are pushed into the point-in-time query so warehouse exports stay server-side. Opt-in; fields without a default are unaffected. Feature statuses are unchanged, so a defaulted feature still reports NOT_FOUND.

Signed-off-by: David <david-adeniji@hotmail.co.uk>
@codecov-commenter

codecov-commenter commented Aug 9, 2026

Copy link
Copy Markdown

⚠️ Please install the 'codecov app svg image' to ensure uploads and comments are reliably processed by Codecov.

Codecov Report

❌ Patch coverage is 76.88172% with 43 lines in your changes missing coverage. Please review.
✅ Project coverage is 46.94%. Comparing base (b7ae488) to head (180a446).
⚠️ Report is 1 commits behind head on master.

Files with missing lines Patch % Lines
go/internal/feast/model/basefeatureview.go 0.00% 14 Missing ⚠️
...python/feast/infra/offline_stores/offline_utils.py 66.66% 5 Missing and 1 partial ⚠️
go/internal/feast/onlineserving/serving.go 66.66% 2 Missing and 2 partials ⚠️
...k/python/feast/infra/online_stores/online_store.py 55.55% 3 Missing and 1 partial ⚠️
go/internal/feast/model/field.go 0.00% 3 Missing ⚠️
sdk/python/feast/feature_store.py 33.33% 1 Missing and 1 partial ⚠️
...python/feast/infra/offline_stores/offline_store.py 95.83% 1 Missing and 1 partial ⚠️
sdk/python/feast/infra/offline_stores/redshift.py 33.33% 2 Missing ⚠️
sdk/python/feast/utils.py 93.10% 1 Missing and 1 partial ⚠️
sdk/python/feast/field.py 97.22% 0 Missing and 1 partial ⚠️
... and 3 more
❗ Your organization needs to install the Codecov GitHub app to enable full functionality.
Additional details and impacted files

Impacted file tree graph

@@            Coverage Diff             @@
##           master    #6716      +/-   ##
==========================================
+ Coverage   46.75%   46.94%   +0.19%     
==========================================
  Files         414      414              
  Lines       50348    50518     +170     
  Branches     7208     7242      +34     
==========================================
+ Hits        23539    23716     +177     
+ Misses      25157    25136      -21     
- Partials     1652     1666      +14     
Flag Coverage Δ *Carryforward flag
go-feature-server 31.68% <27.58%> (+1.09%) ⬆️
python-unit 48.20% <85.98%> (+0.12%) ⬆️ Carriedforward from 8fa3a09

*This pull request uses carry forward flags. Click here to find out more.

Files with missing lines Coverage Δ
sdk/python/feast/feature_view_projection.py 84.37% <100.00%> (+0.76%) ⬆️
...ra/offline_stores/contrib/ray_offline_store/ray.py 0.00% <ø> (ø)
...ffline_stores/contrib/spark_offline_store/spark.py 36.25% <100.00%> (ø)
...ffline_stores/contrib/trino_offline_store/trino.py 50.67% <ø> (ø)
sdk/python/feast/on_demand_feature_view.py 71.58% <ø> (ø)
sdk/python/feast/field.py 84.96% <97.22%> (+3.45%) ⬆️
sdk/python/feast/infra/offline_stores/bigquery.py 49.67% <50.00%> (+0.08%) ⬆️
...line_stores/contrib/athena_offline_store/athena.py 39.47% <50.00%> (+0.32%) ⬆️
sdk/python/feast/infra/offline_stores/snowflake.py 39.48% <50.00%> (+0.12%) ⬆️
sdk/python/feast/feature_store.py 42.94% <33.33%> (-0.02%) ⬇️
... and 8 more

... and 1 file with indirect coverage changes


Continue to review full report in Codecov by Harness.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update b7ae488...180a446. Read the comment docs.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

Signed-off-by: David <david-adeniji@hotmail.co.uk>
@addenergyx addenergyx changed the title feat: Add typed default_value to Field and FeatureSpecV2 feat: Support typed default values for missing feature values Aug 9, 2026
Signed-off-by: David <david-adeniji@hotmail.co.uk>
Signed-off-by: David <david-adeniji@hotmail.co.uk>
Signed-off-by: David <david-adeniji@hotmail.co.uk>
Signed-off-by: David <david-adeniji@hotmail.co.uk>
Signed-off-by: David <david-adeniji@hotmail.co.uk>
Signed-off-by: David <david-adeniji@hotmail.co.uk>
Signed-off-by: David <david-adeniji@hotmail.co.uk>
Signed-off-by: David <david-adeniji@hotmail.co.uk>
…projections

Signed-off-by: David <david-adeniji@hotmail.co.uk>
Signed-off-by: David <david-adeniji@hotmail.co.uk>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Support typed default values for missing features

2 participants