Port feast 0.10+ data model to feast-serving by achals · Pull Request #37 · feast-dev/feast-java-old

achals · 2021-09-22T06:10:23Z

This PR the Feast data model introduced in feast 0.10+ to feast-serving in a backwards compatible way.

If a registry location is specified in the application.yml that implies that feast-serving should use the new data model (feature views and the registry proto instead of the SpecService).

Currently, this has only been tested on the Redis online store. Also, in this version of the PR only local registry files are supported. However, interfaces are defined to allow reading the registry from other locations as needed. Similarly, other online stores may be used (but as of feast 0.13 BigTable and Cassandra are not supported for materializations).

Signed-off-by: Achal Shah <achals@gmail.com>

adchia · 2021-10-06T12:43:11Z

+import org.springframework.core.env.Environment;
+import org.springframework.core.type.AnnotatedTypeMetadata;
+
+public class CoreCondition implements Condition {


Add javadocs?

adchia · 2021-10-06T12:48:23Z

+  @Override
+  public RegistryProto.Registry getRegistry() {
+    try {
+


nit: remove extra lines in the beginning / end of this try?

adchia · 2021-10-06T13:06:19Z

+  }
+
+  @Override
+  public RegistryProto.Registry getRegistry() {


do you want to cache this file read?

Yeah good idea

adchia · 2021-10-06T13:25:47Z

+import org.apache.commons.lang3.tuple.Pair;
+
+// This is derived from
+// https://github.com/feast-dev/feast/blob/b1ccf8dd1535f721aee8bea937ee38feff80bec5/sdk/python/feast/infra/key_encoding_utils.py#L22


adchia · 2021-10-06T13:28:43Z

+  @Override
+  public List<String> getEntitiesList(
+      String projectName, ServingAPIProto.FeatureReferenceV2 featureReference) {
+    final RegistryProto.Registry registry = this.registryRepository.getRegistry();


any reason why we wouldn't push these methods all into the RegistryRepo?

I want to keep implementing custom registries (s3, GCS) easy so I decoupled the two layers.

i see. could see these being methods in a base class (or helper methods). These logically seem to fit better within a registry than here

Signed-off-by: Achal Shah <achals@gmail.com>

* Update feast dep to 0.12 Signed-off-by: Achal Shah <achals@gmail.com> * Port feast 0.10+ data model to feast-serving Signed-off-by: Achal Shah <achals@gmail.com> * Fix tests Signed-off-by: Achal Shah <achals@gmail.com> * Fix integ tests Signed-off-by: Achal Shah <achals@gmail.com> * Fix integ tests Signed-off-by: Achal Shah <achals@gmail.com> * remove logging Signed-off-by: Achal Shah <achals@gmail.com> * Fix ilnt Signed-off-by: Achal Shah <achals@gmail.com> * Fix serialization Signed-off-by: Achal Shah <achals@gmail.com> * Implement EntityKeySerialization correctly Signed-off-by: Achal Shah <achals@gmail.com> * Update workflows Signed-off-by: Achal Shah <achals@gmail.com> * Update python version Signed-off-by: Achal Shah <achals@gmail.com> * Change redis ports Signed-off-by: Achal Shah <achals@gmail.com> * materialize into redis Signed-off-by: Achal Shah <achals@gmail.com> * fix path Signed-off-by: Achal Shah <achals@gmail.com> * Install redis vairant Signed-off-by: Achal Shah <achals@gmail.com> * Remove odfv Signed-off-by: Achal Shah <achals@gmail.com> * Include test file Signed-off-by: Achal Shah <achals@gmail.com> * update source Signed-off-by: Achal Shah <achals@gmail.com> * update source Signed-off-by: Achal Shah <achals@gmail.com> * update source Signed-off-by: Achal Shah <achals@gmail.com> * update source Signed-off-by: Achal Shah <achals@gmail.com> * Wrestling with spring Signed-off-by: Achal Shah <achals@gmail.com> * Tests Signed-off-by: Achal Shah <achals@gmail.com> * Remove github action Signed-off-by: Achal Shah <achals@gmail.com> * Add registry Signed-off-by: Achal Shah <achals@gmail.com> * Remove redundant stuff Signed-off-by: Achal Shah <achals@gmail.com> * Rename test Signed-off-by: Achal Shah <achals@gmail.com> * awaitTermination Signed-off-by: Achal Shah <achals@gmail.com> * lint Signed-off-by: Achal Shah <achals@gmail.com> * lint Signed-off-by: Achal Shah <achals@gmail.com> * dynamic properties instead Signed-off-by: Achal Shah <achals@gmail.com> * dirtiescontext Signed-off-by: Achal Shah <achals@gmail.com> * python 3.7 Signed-off-by: Achal Shah <achals@gmail.com> * spotless Signed-off-by: Achal Shah <achals@gmail.com> * Dirty Context after test method as well Signed-off-by: Achal Shah <achals@gmail.com> * Cleanup Signed-off-by: Achal Shah <achals@gmail.com> * Cleanup Signed-off-by: Achal Shah <achals@gmail.com> * cr Signed-off-by: Achal Shah <achals@gmail.com> * spotless Signed-off-by: Achal Shah <achals@gmail.com>

achals added 2 commits September 20, 2021 09:38

Update feast dep to 0.12

7534bfb

Signed-off-by: Achal Shah <achals@gmail.com>

Port feast 0.10+ data model to feast-serving

cd0f2b9

Signed-off-by: Achal Shah <achals@gmail.com>

achals requested review from adchia and woop September 22, 2021 06:10

achals assigned woop Sep 22, 2021

achals added 6 commits September 21, 2021 23:11

Fix tests

3bef6ec

Signed-off-by: Achal Shah <achals@gmail.com>

Fix integ tests

c68fb3d

Signed-off-by: Achal Shah <achals@gmail.com>

Fix integ tests

42efa34

Signed-off-by: Achal Shah <achals@gmail.com>

remove logging

265b984

Signed-off-by: Achal Shah <achals@gmail.com>

Fix ilnt

7f779ce

Signed-off-by: Achal Shah <achals@gmail.com>

Fix serialization

f4d68fb

Signed-off-by: Achal Shah <achals@gmail.com>

achals mentioned this pull request Sep 23, 2021

Float features are materialized into online stores as doubles feast-dev/feast#1904

Closed

achals added 18 commits September 24, 2021 17:09

Implement EntityKeySerialization correctly

34ec33c

Signed-off-by: Achal Shah <achals@gmail.com>

Update workflows

676d57e

Signed-off-by: Achal Shah <achals@gmail.com>

Update python version

e0edb68

Signed-off-by: Achal Shah <achals@gmail.com>

Change redis ports

771df7b

Signed-off-by: Achal Shah <achals@gmail.com>

materialize into redis

42d7147

Signed-off-by: Achal Shah <achals@gmail.com>

fix path

087f4e6

Signed-off-by: Achal Shah <achals@gmail.com>

Install redis vairant

702094e

Signed-off-by: Achal Shah <achals@gmail.com>

Remove odfv

83039f8

Signed-off-by: Achal Shah <achals@gmail.com>

Include test file

3f8d26e

Signed-off-by: Achal Shah <achals@gmail.com>

update source

76ce225

Signed-off-by: Achal Shah <achals@gmail.com>

update source

66c8d7e

Signed-off-by: Achal Shah <achals@gmail.com>

update source

100e37f

Signed-off-by: Achal Shah <achals@gmail.com>

update source

e1e1f36

Signed-off-by: Achal Shah <achals@gmail.com>

Wrestling with spring

7ae280e

Signed-off-by: Achal Shah <achals@gmail.com>

Tests

efc13b9

Signed-off-by: Achal Shah <achals@gmail.com>

Remove github action

dfff7b9

Signed-off-by: Achal Shah <achals@gmail.com>

Add registry

c544ea1

Signed-off-by: Achal Shah <achals@gmail.com>

Remove redundant stuff

2e1a447

Signed-off-by: Achal Shah <achals@gmail.com>

achals added 11 commits October 5, 2021 16:16

Rename test

9de9d01

Signed-off-by: Achal Shah <achals@gmail.com>

awaitTermination

f303101

Signed-off-by: Achal Shah <achals@gmail.com>

lint

588f149

Signed-off-by: Achal Shah <achals@gmail.com>

lint

2bbac62

Signed-off-by: Achal Shah <achals@gmail.com>

dynamic properties instead

79757e4

Signed-off-by: Achal Shah <achals@gmail.com>

dirtiescontext

749c4d0

Signed-off-by: Achal Shah <achals@gmail.com>

python 3.7

ebce9ac

Signed-off-by: Achal Shah <achals@gmail.com>

spotless

174d757

Signed-off-by: Achal Shah <achals@gmail.com>

Dirty Context after test method as well

8e7ed44

Signed-off-by: Achal Shah <achals@gmail.com>

Cleanup

cd011c3

Signed-off-by: Achal Shah <achals@gmail.com>

Cleanup

28a2b2b

Signed-off-by: Achal Shah <achals@gmail.com>

adchia self-assigned this Oct 6, 2021

adchia reviewed Oct 6, 2021

View reviewed changes

achals added 2 commits October 6, 2021 08:31

cr

f8f1c0a

Signed-off-by: Achal Shah <achals@gmail.com>

spotless

ba1420a

Signed-off-by: Achal Shah <achals@gmail.com>

adchia approved these changes Oct 6, 2021

View reviewed changes

woop reviewed Oct 7, 2021

View reviewed changes

Comment thread serving/src/main/java/feast/serving/config/ContextClosedHandler.java

woop merged commit e622a88 into feast-dev:master Oct 7, 2021

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Port feast 0.10+ data model to feast-serving#37

Port feast 0.10+ data model to feast-serving#37
woop merged 39 commits into
feast-dev:masterfrom
achals:achal/registry-backed-serving

achals commented Sep 22, 2021 •

edited

Loading

Uh oh!

adchia Oct 6, 2021

Uh oh!

adchia Oct 6, 2021

Uh oh!

Uh oh!

Uh oh!

Uh oh!

adchia Oct 6, 2021

Uh oh!

achals Oct 6, 2021

Uh oh!

Uh oh!

adchia Oct 6, 2021

Uh oh!

adchia Oct 6, 2021

Uh oh!

achals Oct 6, 2021

Uh oh!

adchia Oct 6, 2021

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Conversation

achals commented Sep 22, 2021 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

achals commented Sep 22, 2021 •

edited

Loading