Skip to content

when calling for several ODFVs at once, only fetches one ODFV's column-list per shared source view (last request wins), #6621

Description

@addenergyx

Expected Behavior

When several on demand feature views share a source feature view but use different input columns, feature resolution should produce one entry for the shared source view whose projection contains the union of the required input columns.

Current Behavior

_get_feature_views_to_use appends the shared source view once per ODFV, each copy carrying only that ODFV's projection. The membership check if source_fv not in fvs_to_use never matches because BaseFeatureView.eq compares projections, and the freshly fetched view (full projection) never equals a stored narrowed copy. Downstream, _group_feature_refs keys its view index by projection.name_to_use(), so the last duplicate wins and the other ODFVs' source projections are silently dropped.

This corrupts get_historical_features output: with a request containing only ODFV refs, the losing ODFVs' input columns are never fetched, so transforms receive missing inputs (NaN in pandas mode). We hit this in production on 0.46, where a downstream fillna(0) turned the missing inputs into silently wrong feature values on about 10% of rows.

Since 0.62 the retrieval path is masked by the ref injection loop added in #6140, but the resolution helper still returns duplicate entries with conflicting projections to its other callers (_get_online_request_context, retrieve_online_documents, retrieve_online_documents_v2).

Steps to reproduce

  src_fv = FeatureView(name="src_fv", entities=[driver], schema=[Field(name="a", dtype=Float64), Field(name="b", dtype=Float64)], source=...)

  @on_demand_feature_view(sources=[src_fv[["a"]]], schema=[Field(name="a_out", dtype=Float64)], mode="pandas")
  def odfv_a(inputs):
  @on_demand_feature_view(sources=[src_fv[["b"]]], schema=[Field(name="b_out", dtype=Float64)], mode="pandas")
  def odfv_b(inputs):
      return pd.DataFrame({"b_out": inputs["b"] + 100})

  fvs, _ = _get_feature_views_to_use(store.registry, store.project, ["odfv_a:a_out", "odfv_b:b_out"])
  # Actual: two src_fv entries with projections ['a'] and ['b']
  # Expected: one src_fv entry with projection ['a', 'b']

Specifications

  • Version: master (also reproduced on 0.46; wrong retrieval values on 0.61 and earlier)
  • Platform: all
  • Subsystem: historical retrieval

Possible Solution

In _get_feature_views_to_use, match existing entries by projection.name_to_use() and merge projections (union of features, deduplicated by name) instead of appending a duplicate.

Related: #6099, #6140

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions