Skip to content

BigQueryOfflineStore facing problems with complex feature views #3426

@sudohainguyen

Description

@sudohainguyen

Expected Behavior

I expect BigQueryOffline store would be able to have freedom in joining multiple feature views at large scale, including table-based feature views and SQL query-based feature views

Current Behavior

When I conducted point-in-time joins across like 30 feature views, BQ client thrown this error

400 Resources exceeded during query execution: Not enough resources for query planning - too many subqueries or query is too complex.

Steps to reproduce

  1. Prepare 2 table-based feature views
  2. Prepare 2 query-based feature views (with pivot statement on 10 values)
  3. Conduct point-in-time joining on all feature views

Specifications

  • Version: 0.27.1
  • Platform: Linux

Possible Solution

Convert BQ template from using WITH to using TEMP TABLE

Metadata

Metadata

Assignees

No one assigned

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions