feat: Add SingleStore online store support for feature view versioning#6195
feat: Add SingleStore online store support for feature view versioning#6195antznette1 wants to merge 18 commits intofeast-dev:masterfrom
Conversation
88a6e5b to
0de8845
Compare
|
Good day @franciscojavierarceo |
ef34979 to
857d237
Compare
a197d10 to
6cd6eb7
Compare
|
@antznette1 Can you please take a look at #6193, it has shared helper added now for table name and pattern for deleting all versions in teardown ? |
c646964 to
b06b1ce
Compare
dbab47b to
f97a997
Compare
|
@ntkathole , |
@antznette1 Sure, please fix the conflicts and the tests |
Signed-off-by: antznette1 <ochiezeanthonette@gmail.com>
Signed-off-by: antznette1 <ochiezeanthonette@gmail.com>
Signed-off-by: antznette1 <ochiezeanthonette@gmail.com>
Signed-off-by: antznette1 <ochiezeanthonette@gmail.com>
Signed-off-by: antznette1 <ochiezeanthonette@gmail.com>
Signed-off-by: antznette1 <ochiezeanthonette@gmail.com>
Signed-off-by: antznette1 <ochiezeanthonette@gmail.com>
Signed-off-by: antznette1 <ochiezeanthonette@gmail.com>
Signed-off-by: antznette1 <ochiezeanthonette@gmail.com>
Signed-off-by: antznette1 <ochiezeanthonette@gmail.com>
…singlestore.py Co-authored-by: devin-ai-integration[bot] <158243242+devin-ai-integration[bot]@users.noreply.github.com> Signed-off-by: antznette1 <ochiezeanthonette@gmail.com>
…singlestore.py Co-authored-by: devin-ai-integration[bot] <158243242+devin-ai-integration[bot]@users.noreply.github.com> Signed-off-by: antznette1 <ochiezeanthonette@gmail.com>
Signed-off-by: antznette1 <ochiezeanthonette@gmail.com>
Signed-off-by: antznette1 <ochiezeanthonette@gmail.com>
Signed-off-by: antznette1 <ochiezeanthonette@gmail.com>
Signed-off-by: antznette1 <ochiezeanthonette@gmail.com>
Signed-off-by: antznette1 <ochiezeanthonette@gmail.com>
Signed-off-by: antznette1 <ochiezeanthonette@gmail.com>
f97a997 to
8a64179
Compare
| def online_store_table_id( | ||
| project: str, | ||
| table: FeatureView, | ||
| enable_versioning: bool = False, | ||
| version: Optional[int] = None, | ||
| ) -> str: | ||
| name = table.name | ||
| if enable_versioning: | ||
| version = getattr(table.projection, "version_tag", None) | ||
| if version is None: | ||
| version = getattr(table, "current_version_number", None) | ||
| if version is not None and version > 0: | ||
| name = f"{table.name}_v{version}" | ||
| return name | ||
| resolved_version = version | ||
| if resolved_version is None: | ||
| resolved_version = getattr(table.projection, "version_tag", None) | ||
| if resolved_version is None: | ||
| resolved_version = getattr(table, "current_version_number", None) | ||
| if resolved_version is not None and resolved_version > 0: | ||
| name = f"{table.name}_v{resolved_version}" | ||
| return f"{project}_{name}" | ||
|
|
||
|
|
||
| def compute_table_id(project: str, table: Any, enable_versioning: bool = False) -> str: | ||
| """Build the online-store table name, appending a version suffix when versioning is enabled.""" | ||
| return f"{project}_{compute_versioned_name(table, enable_versioning)}" | ||
| return online_store_table_id(project, table, enable_versioning) |
There was a problem hiding this comment.
🔴 Removed compute_versioned_name from helpers.py but redis.py and dynamodb.py still import it
The PR refactored helpers.py by replacing compute_versioned_name with online_store_table_id, but two online store modules still import the removed function. redis.py:39 imports compute_versioned_name and uses it at redis.py:62, and dynamodb.py:27 imports it and uses it at dynamodb.py:1158. Since the PR explicitly modified both of these files (to add the registry parameter to teardown), this is an incomplete refactoring. Any user of the Redis or DynamoDB online store will get an ImportError the moment the module is loaded.
Prompt for agents
The function compute_versioned_name was removed from helpers.py and replaced with online_store_table_id, but two callers were not migrated:
1. sdk/python/feast/infra/online_stores/redis.py line 39 imports compute_versioned_name from helpers. It is used at line 62 in _versioned_fv_name(). This function returns just the versioned name (without project prefix) for Redis hash key construction. You need to either: (a) re-export a backward-compatible compute_versioned_name wrapper in helpers.py that extracts the name portion from online_store_table_id, or (b) update redis.py to use online_store_table_id and strip the project prefix, or (c) add a simpler helper that returns just the versioned name without the project prefix. Also add supports_versioned_online_reads = True to RedisOnlineStore if Redis should continue supporting versioned reads.
2. sdk/python/feast/infra/online_stores/dynamodb.py line 27 imports compute_versioned_name from helpers. It is used at line 1158 in _get_table_name(). DynamoDB uses it to get the versioned name which is then inserted into a template. Same approach needed here. Also add supports_versioned_online_reads = True to DynamoDBOnlineStore if DynamoDB should continue supporting versioned reads.
Note that the old compute_versioned_name returned just the name (e.g. 'my_view_v2') without a project prefix, while online_store_table_id returns 'project_my_view_v2'. The callers in redis.py and dynamodb.py rely on the old behavior (no project prefix), so a simple rename won't work.
Was this helpful? React with 👍 or 👎 to provide feedback.
| super().__init__( | ||
| f"Versioned feature reads (@v{version}) are not yet supported by {store_name}. " | ||
| f"Currently only SQLite, PostgreSQL, MySQL, FAISS, Redis, and DynamoDB support version-qualified feature references. " | ||
| f"Currently only SQLite, PostgreSQL, MySQL, FAISS, Redis, DynamoDB, and SingleStore support version-qualified feature references. " |
There was a problem hiding this comment.
🟡 VersionedOnlineReadNotSupported error message lists FAISS, Redis, DynamoDB as supported but they are no longer recognized
The error message at errors.py:145 was updated to include SingleStore but still lists "FAISS, Redis, DynamoDB" as supporting versioned reads. However, the new _check_versioned_read_support logic at online_store.py:280 only allows stores that set supports_versioned_online_reads = True (or are SqliteOnlineStore). FaissOnlineStore does not set this property, and RedisOnlineStore/DynamoDBOnlineStore don't either. So when users encounter this error on these stores, the message incorrectly tells them the store should work.
| f"Currently only SQLite, PostgreSQL, MySQL, FAISS, Redis, DynamoDB, and SingleStore support version-qualified feature references. " | |
| f"Currently only SQLite, PostgreSQL, MySQL, and SingleStore support version-qualified feature references. " |
Was this helpful? React with 👍 or 👎 to provide feedback.
What this PR does / why we need it:
This PR adds online feature view versioning support for the SingleStore online store when
registry.enable_online_feature_view_versioningis enabled.It updates SingleStore’s table routing to read/write from versioned namespaces (e.g.
{project}_{feature_view}_v{N}) so version-qualified refs likedriver_stats@v1:avg_daily_tripscan be served from the correct underlying online table. It also updates the global online-store guard to allow version-qualified reads for SingleStore (and still reject unsupported stores) when versioning is enabled.Additionally, it adds an integration/universal test covering SingleStore version-qualified reads.
Which issue(s) this PR fixes:
Fixes #6181
Checks
git commit -s)Testing Strategy
Misc
Notes:
registry.enable_online_feature_view_versioningis enabled; otherwiseVersionedOnlineReadNotSupportedis raised.