Redis can delete incorrect keys

I haven't tested this explicitly but from the data model + this function it looks to be as if two FeatureViews have the same Entities (but are different tables) and one view is deleted the online keys for the other view will also be deleted... This seems like a bit of a flaw either in the Redis data model or in the deletion mechanism.

feast/sdk/python/feast/infra/online_stores/redis.py

Lines 75 to 88 in ec41653

    
           def delete_table_values(self, config: RepoConfig, table: FeatureView): 
        
               client = self._get_client(config.online_store) 
        
               deleted_count = 0 
        
               pipeline = client.pipeline() 
        
               prefix = _redis_key_prefix(table.entities) 
        
               for _k in client.scan_iter( 
        
                   b"".join([prefix, b"*", config.project.encode("utf8")]) 
        
               ): 
        
                   pipeline.delete(_k) 
        
                   deleted_count += 1 
        
               pipeline.execute() 
        
               logger.debug(f"Deleted {deleted_count} keys for {table.name}")

good catch - briefly looking at the code, I'm not even convinced that the key we're using to delete matches the key for insertion (e.g. we might be deleting nothing at all)

Navigation Menu

Repository navigation

Redis can delete incorrect keys #2150

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

	def delete_table_values(self, config: RepoConfig, table: FeatureView):
	client = self._get_client(config.online_store)
	deleted_count = 0
	pipeline = client.pipeline()
	prefix = _redis_key_prefix(table.entities)

	for _k in client.scan_iter(
	b"".join([prefix, b"*", config.project.encode("utf8")])
	):
	pipeline.delete(_k)
	deleted_count += 1
	pipeline.execute()

	logger.debug(f"Deleted {deleted_count} keys for {table.name}")

Redis can delete incorrect keys #2150

Description

Activity

felixwang9817 commented on Dec 17, 2021

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions