Expected Behavior
Should only connect to the relevant stores specified in the FeatureSpec.
Current Behavior
Will pull and attempt to connect to all the stores once the job is sent to the remote Flink cluster. If there are stores it cannot connect to, the entire job will fail.
Steps to reproduce
- Register a storage that cannot be reached (e.g. 1.2.3.4:6379)
- Run a job (not using that store) on a remote flink cluster
Possible Solution
Since we only pull specs at the graph building stage now, ingestion shouldn't have to retrieve and cache all available stores, only the stores it needs.
Expected Behavior
Should only connect to the relevant stores specified in the FeatureSpec.
Current Behavior
Will pull and attempt to connect to all the stores once the job is sent to the remote Flink cluster. If there are stores it cannot connect to, the entire job will fail.
Steps to reproduce
Possible Solution
Since we only pull specs at the graph building stage now, ingestion shouldn't have to retrieve and cache all available stores, only the stores it needs.