feat: Implement RegistryServer.Proto RPC with RBAC-filtered response (#6558) - #6552
Conversation
The remote registry client (RemoteRegistry.proto) calls stub.Proto(Empty()) to fetch the full registry, but the RegistryServer servicer never implemented the Proto RPC, so remote clients received UNIMPLEMENTED. Return proxied_registry.proto(). Signed-off-by: Nick Quinn <nicholas_quinn@apple.com>
| self.proxied_registry = registry | ||
|
|
||
| def Proto(self, request: Empty, context) -> RegistryProto: | ||
| return self.proxied_registry.proto() |
There was a problem hiding this comment.
@nquinn408 it was intentionally removed, so that it won't return all information to the users not having access. Probably proper rbac needs to be added if we need it for any purpose.
There was a problem hiding this comment.
I understand @ntkathole but then it breaks RemoteRegistry.proto, so I don't know what the solution is. I guess a different implementation for RemoteRegistry.proto?
There was a problem hiding this comment.
There was a problem hiding this comment.
I think we can construct the response by calling each typed List* method (have RBAC checks) rather than forwarding the raw proto().
There was a problem hiding this comment.
Is there an open issue to address? I will give it a shot.
There was a problem hiding this comment.
@ntkathole I restructured this PR to address your issue: #6558. Can you kick off LLM review?
The Proto RPC existed in the RegistryServer proto contract but had no real server-side implementation, so RemoteRegistry.proto() failed with UNIMPLEMENTED (breaking CachingRegistry init, registry refresh, and metrics over a remote registry). The naive fix (return proxied_registry.proto()) would bypass RBAC and return every object regardless of authorization. Build the RegistryProto from individually RBAC-filtered list_* calls, each passed through permitted_resources(..., AuthzedAction.DESCRIBE), iterating every project. Under NoAuthConfig this is a no-op so the full registry is returned (remote registries keep working); with auth enabled callers only see objects they may DESCRIBE. Adds unit tests covering the no-auth full-return path, RBAC filtering, and the empty-registry case. Closes feast-dev#6558 Signed-off-by: Nick Quinn <nicholas_quinn@apple.com>
ntkathole
left a comment
There was a problem hiding this comment.
looks good, pending comment
…Proto Addresses review feedback on PR feast-dev#6552: RegistryServer.Proto rebuilds the RegistryProto from individual RBAC-filtered list calls and was stamping last_updated with datetime.now(), making the registry look freshly committed on every call. Copy the authentic last_updated and version_id from the source registry proto instead, so cache consumers (e.g. the remote feature server) see the registry's real freshness metadata. Only these two scalar fields are read from the source proto — no objects — so RBAC filtering is unaffected. Update the unit test to give the fake registry a proto() returning known metadata and assert it is carried through (rather than asserting "now"). Signed-off-by: Nick Quinn <nicholas_quinn@apple.com>
…east-dev#6558) (feast-dev#6552) Signed-off-by: Himanshu Singh <himanshu.singh@walmart.com>
…6569) * retrieval from bigquery without entity df Signed-off-by: Himanshu Singh <himanshu.singh@walmart.com> * feat: Implement RegistryServer.Proto RPC with RBAC-filtered response (#6558) (#6552) Signed-off-by: Himanshu Singh <himanshu.singh@walmart.com> * feat(operator): integrate cluster TLS profile for OCP 5.0 compliance Read the cluster TLS profile from apiservers.config.openshift.io/cluster at startup. Apply MinVersion, CipherSuites, and NextProtos to webhook and metrics server TLS configs. Fail closed on unexpected errors. Use Intermediate defaults on non-OpenShift clusters. This ensures the feast-operator honors the cluster-wide TLS security profile, which is required for OCP 5.0. The implementation gracefully falls back to hardened defaults (TLS 1.2, ECDHE ciphers) on non-OpenShift clusters, so it does not break vanilla Kubernetes deployments. Signed-off-by: Ugo Giordano <ugiordan@redhat.com> Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> Signed-off-by: Ugo Giordano <ugiordan@redhat.com> Signed-off-by: Himanshu Singh <himanshu.singh@walmart.com> * fixing linter error Signed-off-by: Himanshu Singh <himanshu.singh@walmart.com> * fixing unit test error Signed-off-by: Himanshu Singh <himanshu.singh@walmart.com> * linter errors Signed-off-by: Himanshu Singh <himanshu.singh@walmart.com> * applied recos Signed-off-by: Himanshu Singh <himanshu.singh@walmart.com> * unit tests and linter error Signed-off-by: Himanshu Singh <himanshu.singh@walmart.com> * pre commit checks Signed-off-by: Himanshu Singh <himanshu.singh@walmart.com> * pre commit checks Signed-off-by: Himanshu Singh <himanshu.singh@walmart.com> * fixing errors Signed-off-by: Himanshu Singh <himanshu.singh@walmart.com> * casting null as string Signed-off-by: Himanshu Singh <himanshu.singh@walmart.com> * fixing test cases Signed-off-by: Himanshu Singh <himanshu.singh@walmart.com> --------- Signed-off-by: Himanshu Singh <himanshu.singh@walmart.com> Signed-off-by: Ugo Giordano <ugiordan@redhat.com> Co-authored-by: Himanshu Singh <himanshu.singh@walmart.com> Co-authored-by: nquinn408 <57655411+nquinn408@users.noreply.github.com> Co-authored-by: Ugo Giordano <ugiordan@redhat.com> Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
# [0.65.0](v0.64.0...v0.65.0) (2026-07-20) ### Bug Fixes * add debug logging for FIPS mode detection fallback ([6c1b24e](6c1b24e)) * Build embedded UI from local source ([#6525](#6525)) ([3500349](3500349)) * Bump decommissioned Snowflake Python UDF runtime from 3.9 to 3.10 ([#6606](#6606)) ([#6608](#6608)) ([10341e4](10341e4)) * configure FIPS-compliant gRPC cipher suites for offline server ([6bc80a2](6bc80a2)) * Correct Flink PyArrow dependency constraints ([#6604](#6604)) ([70a9751](70a9751)) * Fix ValueError in signal handling for Trino worker threads ([#6428](#6428)) ([506d919](506d919)) * Fixed monitoring page issues ([7946018](7946018)) * Make pytest config compatible with newer pytest ([#5779](#5779)) ([a57ea33](a57ea33)) * Replace comma with space in DynamoDB-incompatible label tag value ([51e3a16](51e3a16)) * Resolve UI build warnings ([#6529](#6529)) ([abe92af](abe92af)) * Unblock nightly UI build ([#6570](#6570)) ([f296d4b](f296d4b)) * Use LONGBLOB for SQL registry proto columns on MySQL ([#6566](#6566)) ([7e4beb2](7e4beb2)) ### Features * Add click-to-zoom lightbox for blog post images ([#6575](#6575)) ([1cb23fd](1cb23fd)) * Add dark mode support to website and blog ([#6589](#6589)) ([7358fb8](7358fb8)) * Add OnlineStore for Aerospike ([#6532](#6532)) ([9cd35e1](9cd35e1)) * Add OpenLineage Consumer to Feast - receive, store, and visualize cross-producer lineage ([#6549](#6549)) ([a834126](a834126)) * Add registry list feature views by updated since ([#6092](#6092)) ([#6093](#6093)) ([006c606](006c606)) * Add ScyllaDB online store with vector search ([#6508](#6508)) ([1669661](1669661)) * Added compute and jobs UI ([ba2c05c](ba2c05c)) * Added Iceberg REST Catalog data source support ([e0a8573](e0a8573)) * Bring Your Own Spark - SparkApplication ([#6550](#6550)) ([dcd496f](dcd496f)) * **cassandra:** Add multi-DC support via per-datacenter execution profiles ([#6434](#6434)) ([0de9196](0de9196)) * Enhanced data source creation as a visual catalog with type-specific forms ([#6557](#6557)) ([d6acbba](d6acbba)) * Enhanced datasets UI functionality ([de11152](de11152)) * Implement RegistryServer.Proto RPC with RBAC-filtered response ([#6558](#6558)) ([#6552](#6552)) ([0d02614](0d02614)) * New zoned timestamp feature type ([#6536](#6536)) ([#6537](#6537)) ([eb042f0](eb042f0)) * **operator:** Auto-create RBAC for spark_application batch engine ([#6597](#6597)) ([f487b37](f487b37)) * **operator:** integrate cluster TLS profile for OCP 5.0 compliance ([43263a6](43263a6)) * Permissions CRUD UI and OIDC auth integration in UI ([6511da1](6511da1)) * Retrieve historical features from BigQuery without entity_df ([#6569](#6569)) ([cd5f6bb](cd5f6bb)), closes [#6558](#6558) [#6552](#6552) * **spark:** SparkSource query+path and pre-computed offline read for BatchFeatureView ([#6440](#6440)) ([4dc8757](4dc8757)) ### BREAKING CHANGES * total_timeout_ms is renamed to batch_total_timeout_ms. Config files using the old name must be updated. No default value change. Docs updated (reference + perf-tuning guide) with a short explainer on the per-attempt vs total deadline distinction. Two new unit tests pin the policy wiring: socket_timeout_ms propagates to all three scopes, and is omitted (not injected as None) when unset. Signed-off-by: Valentyn Kahamlyk <valentin.kagamlyk@gmail.com> * refactor(aerospike): use MAP_KEY_ORDERED, KEY_DIGEST, and instance-scoped client Cheap-win cleanups flagged in review, all touching the same small patch of write-path and lifecycle code. * Map CDTs are now created with MAP_KEY_ORDERED. map_get_by_key / map_remove_by_key on an ordered map are O(log N) in the map size instead of O(N); matters on reads of wide feature views and on the update() background scan (which walks every record in the project's set). * Writes drop POLICY_KEY_SEND and rely on the client default (POLICY_KEY_DIGEST). The serialized entity key is no longer stored alongside each record, saving per-record storage the read path never consumes (batch_operate preserves request order; results are paired back by zip in online_read). * _client moves from a class attribute to an instance attribute (set in __init__). Previously two AerospikeOnlineStore instances could share the cached client through class state until one wrote self._client. With the instance attribute the state is always per-instance from construction. * Drop MongoDB references from class docstrings and comments (they referred to how the storage layout was derived rather than documenting current behavior). Also rewrite the _build_batch_writes docstring to describe the policies applied on the write path. Unit test assertions for the write-path record are updated: bw.policy is now None (client default applies) and map ops carry map_policy={'map_order': MAP_KEY_ORDERED}. All three docker-backed integration tests still pass end-to-end (cross-FV upsert, update() background scan, full feature-store round-trip), so the read/write shape survives the ordering and policy changes against a real server. Signed-off-by: Valentyn Kahamlyk <valentin.kagamlyk@gmail.com> * feat(aerospike): add per-FV namespace/set overrides and prewriting hook Adds three configuration knobs to AerospikeOnlineStoreConfig: - namespace_overrides: pin individual feature views to a different Aerospike namespace (e.g. RAM-only vs. SSD-backed) without splitting the project across stores. - set_overrides: place a feature view in its own set so admin ops on it (truncate, scan-based deletes during `feast apply`) do not touch records of other views. - prewriting_hook: import-string-resolved callable invoked once per online_write_batch with the rows about to be written, returning the rows that actually go on the wire. Resolved and cached on first use; returning [] short-circuits the wire call. Read, write, update and teardown paths all honour the per-FV ns/set resolution. update() groups dropped feature views by their resolved (ns, set) pair and issues one background scan per group. teardown() truncates every unique (ns, set) pair the project may have written to, including the store-level default. Adds 22 unit tests for the new behaviour and updates 3 existing call sites of _build_batch_writes for the new namespace= parameter. Adds a sample hook module under examples/online_store/aerospike_overrides_and_hooks/ and corresponding sections in docs/reference/online-stores/aerospike.md. Signed-off-by: Valentyn Kahamlyk <valentin.kagamlyk@gmail.com> * test: update aerospike image tag Signed-off-by: Valentyn Kahamlyk <valentin.kagamlyk@gmail.com> * chore: sync README template and secrets baseline after master merge Signed-off-by: Valentyn Kahamlyk <valentin.kagamlyk@gmail.com> * chore: fix secrets baseline line number for v1 operator types Adding aerospike to the feast-operator enum shifted the allowlisted SecretRef entry in api/v1/featurestore_types.go by one line. Signed-off-by: Valentyn Kahamlyk <valentin.kagamlyk@gmail.com> * docs: update aerospike docs Signed-off-by: Valentyn Kahamlyk <valentin.kagamlyk@gmail.com> * fix(aerospike): wire batch max_retries and fix empty projection handling Copilot review feedback on PR #6532: - Add max_retries to the batch client policy (batch_operate/batch_write path) - Treat empty projected feature maps as present FV slots (is not None) - Return {} from _normalize_projected_features([]) instead of None - Fix projection unit test mock/assertions - Correct prewriting_hook config docstring Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com> Signed-off-by: Valentyn Kahamlyk <valentin.kagamlyk@gmail.com> * style(aerospike): format online_read docs assignment for ruff Signed-off-by: Valentyn Kahamlyk <valentin.kagamlyk@gmail.com> * chore: update pixi.lock for aerospike optional extra Regenerate the v6 lockfile with Pixi v0.63.1 after adding the aerospike extra to pyproject.toml. Signed-off-by: Valentyn Kahamlyk <valentin.kagamlyk@gmail.com> * fix(aerospike): add client init lock and batch chunking Guard lazy client creation with a lock to avoid connection leaks under concurrent first use, and chunk batch reads/writes by batch_max_records so large materializations stay under Aerospike server batch limits. Signed-off-by: Valentyn Kahamlyk <valentin.kagamlyk@gmail.com>
# [0.65.0](feast-dev/feast@v0.64.0...v0.65.0) (2026-07-20) ### Bug Fixes * add debug logging for FIPS mode detection fallback ([6c1b24e](feast-dev@6c1b24e)) * Build embedded UI from local source ([feast-dev#6525](feast-dev#6525)) ([3500349](feast-dev@3500349)) * Bump decommissioned Snowflake Python UDF runtime from 3.9 to 3.10 ([feast-dev#6606](feast-dev#6606)) ([feast-dev#6608](feast-dev#6608)) ([10341e4](feast-dev@10341e4)) * configure FIPS-compliant gRPC cipher suites for offline server ([6bc80a2](feast-dev@6bc80a2)) * Correct Flink PyArrow dependency constraints ([feast-dev#6604](feast-dev#6604)) ([70a9751](feast-dev@70a9751)) * Fix ValueError in signal handling for Trino worker threads ([feast-dev#6428](feast-dev#6428)) ([506d919](feast-dev@506d919)) * Fixed monitoring page issues ([7946018](feast-dev@7946018)) * Make pytest config compatible with newer pytest ([feast-dev#5779](feast-dev#5779)) ([a57ea33](feast-dev@a57ea33)) * Replace comma with space in DynamoDB-incompatible label tag value ([51e3a16](feast-dev@51e3a16)) * Resolve UI build warnings ([feast-dev#6529](feast-dev#6529)) ([abe92af](feast-dev@abe92af)) * Unblock nightly UI build ([feast-dev#6570](feast-dev#6570)) ([f296d4b](feast-dev@f296d4b)) * Use LONGBLOB for SQL registry proto columns on MySQL ([feast-dev#6566](feast-dev#6566)) ([7e4beb2](feast-dev@7e4beb2)) ### Features * Add click-to-zoom lightbox for blog post images ([feast-dev#6575](feast-dev#6575)) ([1cb23fd](feast-dev@1cb23fd)) * Add dark mode support to website and blog ([feast-dev#6589](feast-dev#6589)) ([7358fb8](feast-dev@7358fb8)) * Add OnlineStore for Aerospike ([feast-dev#6532](feast-dev#6532)) ([9cd35e1](feast-dev@9cd35e1)) * Add OpenLineage Consumer to Feast - receive, store, and visualize cross-producer lineage ([feast-dev#6549](feast-dev#6549)) ([a834126](feast-dev@a834126)) * Add registry list feature views by updated since ([feast-dev#6092](feast-dev#6092)) ([feast-dev#6093](feast-dev#6093)) ([006c606](feast-dev@006c606)) * Add ScyllaDB online store with vector search ([feast-dev#6508](feast-dev#6508)) ([1669661](feast-dev@1669661)) * Added compute and jobs UI ([ba2c05c](feast-dev@ba2c05c)) * Added Iceberg REST Catalog data source support ([e0a8573](feast-dev@e0a8573)) * Bring Your Own Spark - SparkApplication ([feast-dev#6550](feast-dev#6550)) ([dcd496f](feast-dev@dcd496f)) * **cassandra:** Add multi-DC support via per-datacenter execution profiles ([feast-dev#6434](feast-dev#6434)) ([0de9196](feast-dev@0de9196)) * Enhanced data source creation as a visual catalog with type-specific forms ([feast-dev#6557](feast-dev#6557)) ([d6acbba](feast-dev@d6acbba)) * Enhanced datasets UI functionality ([de11152](feast-dev@de11152)) * Implement RegistryServer.Proto RPC with RBAC-filtered response ([feast-dev#6558](feast-dev#6558)) ([feast-dev#6552](feast-dev#6552)) ([0d02614](feast-dev@0d02614)) * New zoned timestamp feature type ([feast-dev#6536](feast-dev#6536)) ([feast-dev#6537](feast-dev#6537)) ([eb042f0](feast-dev@eb042f0)) * **operator:** Auto-create RBAC for spark_application batch engine ([feast-dev#6597](feast-dev#6597)) ([f487b37](feast-dev@f487b37)) * **operator:** integrate cluster TLS profile for OCP 5.0 compliance ([43263a6](feast-dev@43263a6)) * Permissions CRUD UI and OIDC auth integration in UI ([6511da1](feast-dev@6511da1)) * Retrieve historical features from BigQuery without entity_df ([feast-dev#6569](feast-dev#6569)) ([cd5f6bb](feast-dev@cd5f6bb)), closes [feast-dev#6558](feast-dev#6558) [feast-dev#6552](feast-dev#6552) * **spark:** SparkSource query+path and pre-computed offline read for BatchFeatureView ([feast-dev#6440](feast-dev#6440)) ([4dc8757](feast-dev@4dc8757)) ### BREAKING CHANGES * total_timeout_ms is renamed to batch_total_timeout_ms. Config files using the old name must be updated. No default value change. Docs updated (reference + perf-tuning guide) with a short explainer on the per-attempt vs total deadline distinction. Two new unit tests pin the policy wiring: socket_timeout_ms propagates to all three scopes, and is omitted (not injected as None) when unset. Signed-off-by: Valentyn Kahamlyk <valentin.kagamlyk@gmail.com> * refactor(aerospike): use MAP_KEY_ORDERED, KEY_DIGEST, and instance-scoped client Cheap-win cleanups flagged in review, all touching the same small patch of write-path and lifecycle code. * Map CDTs are now created with MAP_KEY_ORDERED. map_get_by_key / map_remove_by_key on an ordered map are O(log N) in the map size instead of O(N); matters on reads of wide feature views and on the update() background scan (which walks every record in the project's set). * Writes drop POLICY_KEY_SEND and rely on the client default (POLICY_KEY_DIGEST). The serialized entity key is no longer stored alongside each record, saving per-record storage the read path never consumes (batch_operate preserves request order; results are paired back by zip in online_read). * _client moves from a class attribute to an instance attribute (set in __init__). Previously two AerospikeOnlineStore instances could share the cached client through class state until one wrote self._client. With the instance attribute the state is always per-instance from construction. * Drop MongoDB references from class docstrings and comments (they referred to how the storage layout was derived rather than documenting current behavior). Also rewrite the _build_batch_writes docstring to describe the policies applied on the write path. Unit test assertions for the write-path record are updated: bw.policy is now None (client default applies) and map ops carry map_policy={'map_order': MAP_KEY_ORDERED}. All three docker-backed integration tests still pass end-to-end (cross-FV upsert, update() background scan, full feature-store round-trip), so the read/write shape survives the ordering and policy changes against a real server. Signed-off-by: Valentyn Kahamlyk <valentin.kagamlyk@gmail.com> * feat(aerospike): add per-FV namespace/set overrides and prewriting hook Adds three configuration knobs to AerospikeOnlineStoreConfig: - namespace_overrides: pin individual feature views to a different Aerospike namespace (e.g. RAM-only vs. SSD-backed) without splitting the project across stores. - set_overrides: place a feature view in its own set so admin ops on it (truncate, scan-based deletes during `feast apply`) do not touch records of other views. - prewriting_hook: import-string-resolved callable invoked once per online_write_batch with the rows about to be written, returning the rows that actually go on the wire. Resolved and cached on first use; returning [] short-circuits the wire call. Read, write, update and teardown paths all honour the per-FV ns/set resolution. update() groups dropped feature views by their resolved (ns, set) pair and issues one background scan per group. teardown() truncates every unique (ns, set) pair the project may have written to, including the store-level default. Adds 22 unit tests for the new behaviour and updates 3 existing call sites of _build_batch_writes for the new namespace= parameter. Adds a sample hook module under examples/online_store/aerospike_overrides_and_hooks/ and corresponding sections in docs/reference/online-stores/aerospike.md. Signed-off-by: Valentyn Kahamlyk <valentin.kagamlyk@gmail.com> * test: update aerospike image tag Signed-off-by: Valentyn Kahamlyk <valentin.kagamlyk@gmail.com> * chore: sync README template and secrets baseline after master merge Signed-off-by: Valentyn Kahamlyk <valentin.kagamlyk@gmail.com> * chore: fix secrets baseline line number for v1 operator types Adding aerospike to the feast-operator enum shifted the allowlisted SecretRef entry in api/v1/featurestore_types.go by one line. Signed-off-by: Valentyn Kahamlyk <valentin.kagamlyk@gmail.com> * docs: update aerospike docs Signed-off-by: Valentyn Kahamlyk <valentin.kagamlyk@gmail.com> * fix(aerospike): wire batch max_retries and fix empty projection handling Copilot review feedback on PR feast-dev#6532: - Add max_retries to the batch client policy (batch_operate/batch_write path) - Treat empty projected feature maps as present FV slots (is not None) - Return {} from _normalize_projected_features([]) instead of None - Fix projection unit test mock/assertions - Correct prewriting_hook config docstring Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com> Signed-off-by: Valentyn Kahamlyk <valentin.kagamlyk@gmail.com> * style(aerospike): format online_read docs assignment for ruff Signed-off-by: Valentyn Kahamlyk <valentin.kagamlyk@gmail.com> * chore: update pixi.lock for aerospike optional extra Regenerate the v6 lockfile with Pixi v0.63.1 after adding the aerospike extra to pyproject.toml. Signed-off-by: Valentyn Kahamlyk <valentin.kagamlyk@gmail.com> * fix(aerospike): add client init lock and batch chunking Guard lazy client creation with a lock to avoid connection leaks under concurrent first use, and chunk batch reads/writes by batch_max_records so large materializations stay under Aerospike server batch limits. Signed-off-by: Valentyn Kahamlyk <valentin.kagamlyk@gmail.com>
The remote registry client (RemoteRegistry.proto) calls stub.Proto(Empty()) to fetch the full registry, but the RegistryServer service never implemented the Proto RPC, so remote clients received UNIMPLEMENTED. Return proxied_registry.proto().