Skip to content

Commit eebf458

Browse files
mrzzyZhu Zhanyan
mrzzy
and
Zhu Zhanyan
authored
Add Field Status metadata to Online Serving (#658)
* Update ServingService protobuf with additions to support online metadata retrieval. Additions are outlined in detail in this RFC: https://docs.google.com/document/d/1VQngwBcx-yWgGpAbsFVdth9GnjL8q-ZgUNBGv57R0Fk/ * Change error thrown by sendMultiGet() to Unknown as it describes the fault more accurately * Return boolean primitive instead of Boolean object type in FeatureRowDecoder This removes the unneeded possiblity of a null being returning from the the decoder. * Config sendAndProcessMultiGet() to throw an exception on feature row decoding failure. Instead of the current behaviour to silently fail with and return an empty feature row. * Refactor sendMultiGetAndProcess() to getFeaturesForFeatureSet(). Rename as the original name described the implementation instead what the goal of the method is. Construct and pass the decoder outside method in getFeaturesForFeatureSet() instead of inside method in sendMultiGetAndProcess() as constructing it inside added uncessary arguments and code (ie getting the string ref out a redis key seemed out of place.) Changed getFeaturesForFeatureSet() to return null instead of a nullFeatureRow. when retrieving data from redis using sendMultiGet() using a key missing in Redis. This makes the missing key condition easier to detect. Removed unncessary arguments passed to sendMultiGetAndProcess() - featureReference used to contruct the nullFeatureRow - featureSetSpec used to contruct the decoder. * Update ResponseJSONMapper to use new GetOnlineFeatures protobuf * Remove references to FieldValues in OnlineServingService & Tests * Update java sdk's FieldClient to use the new GetOnlineFeatures protobuf * Update OnlineRetriever to return null on missing retrieval (such as missing key for redis). * Refactor out unnecessary nesting of returned list in OnlineRetriever's getOnlineFeatures() This refactor is done to improve code readablity as it is difficult to conceptualise what each nesting of the list corresponds to: - developers have to keep in mind that the outer nest corresponds to faetures requests and the inner nesting corresponds to entity rows. The nesting is unnecessary as: - inside RedisOnlineRetriever we have a loop over feature set requests. - outside in OnlineServingService where this is used, we also loop over feature set requests. * Add online feature metadata to OnlineServingService's getOnlineFeatures() Updated getOnlineFeatures() to return metadata on each response Field. - Added sanity check with exception to ensure that the feature rows retrieved from the OnlineRetriever matches the no. of entity rows pass to it. Refactor code out of getOnlineFeatures() to smaller methods to reduce method size and improve readabilty: - added unpackFields(entityRow) to unpack EntityRows into response Fields - added unpackFields(featureRow) to unpack FeatureRows into response Fields - added attachMetadata(field) to add metadata to response Fields. - moved loop over feature references into populateStaleKeyCountMetrics() * Update OnlineServingServiceTest to check that online metadata is set correctly. * Added online metadata support java sdk's FeastClient. * Changed go sdk's Row to store a map of fields instead of just Values * Update response.go to use updated Row and take into account FieldStatus * Added Field() to go sdk's types.go to construct fields from a Value. * Update go sdk's request.go to use updated Row and Add GetOnlineFeatures flags Added missing boolean flags for GetOnlineFeatures's IncludeMetadataInResponse and OmitEntitiesInResponse flags. * Update go protobuf generated code for updated protobuf defintion * Update python sdk's client to document new fieldstatus metadata, added flags Added missing boolean flags for GetOnlineFeaturesRequest's omit_entities_in_response & include_metadata_in_response flags. Update unit tests with changes to GetOnlineFeaturesResponse protobuff * Fixed issue where getOnlineFeatures() returned Record in a non deterministic order. Enforce that getOnlineFeatures() returns Records in the same order as the user provides the EntityRow * Update e2e ingestion tests to support new get_online_features api and check for metadata * Fixed NullPointerException when feature row is null. * Fixed issue in which unpackFields(entityRow) was not responding to includeMetadata flag * Check FieldStatus in end to end tests to wait for ingestion of values to complete. * Moved test checks out of polling loop in e2e online ingestion tests * Apply spotless java formatting * Fixed misplaced break in end to end tests * Fixed typos in e2e ingestion tests. * Apply changes on RedisOnlineRetriever to RedisClusterOnlineRetriever Changes * Removed unnessary nesting of list in FeaturRows returned * Refactored sendMultiGetAndProcess() to getOnlineFeaturesForFeatureSet() Necessary due to code duplciation between RedisOnlineRetriever and RedisOnlineRetriever. * Rename getFeaturesForFeatureSet() to getFeaturesFromRedis() for better sementics * Update Serving application.yml to include example on how to config RedisCluster * Correct missing include_meta flag in e2e tests get_online_features() calls * Correct minor comment typo in serving's application.yml * Correct another minor typo in e2e test: missing .status * Fix Feast core and serving compilation/test failures after rebase * Fix go SDK tests after rebase * Update ServingService protobuf to include metadata without breaking changes. * Revert Feast Serving's ServingService & ResponseJSONMapper to FieldValues * Revert serving's OnlineServingService to backwards compatible FieldValues API * Revert Java SDK's FeastClient to backwards compatible FieldStatus API. * Revert Go SDK's client.go to backwards compatible FieldStatus API. * Fix Java SDK unit tests * Added .Statuses() method to GO Sdk's OnlineFeaturesResponse to get metadata * Fixed Java SDK not stripping project from field values and statuses * Fixed Go SDK's client.go not stripping project from field values and statuses * Fixed Go SDK's Unit tests * Revert Python SDK to backward compatible FieldValues API & fix tests * Move hardcoded constants from job.py to constants.py in Python SDK to consolidate constants * Added wait backoff implementation in Python SDK: wait_retry_backoff() * Update Python SDK's job.py to use shared backoff wait implemetation * Clarify that maximum range in ServingService proto to max age * Fixed online serving e2e tests * Fix python lint. * Collect entity refs in Python SDK as a single line. * Use guard style if statements in Go Sdk's types.go * Make strip field values part of Python SDK's get_online_features() more functional * Fix go unit tests * Remove include_metadata_in_response flag from ServingService proto as its redundant. * Readd missing span log call to log retrieved feature rows. * Use more modern Streams.zip() to iterate entity rows and feature rows in OnlineServingService * Change utility methods in OnlineServingService to static. * Use "outside max age" instead "stale" term in OnlineServingService as "stale" is a overloaded term. * Refactor serving's OnlineRetriever to return a empty Optional<FeatureRow> instead of returning null. This allows the code to more semenatic and readable compared to dealing with nullable FeatureRow values * Fixed typo in basic redis e2e tests * Fixed typo OnlineServingService's logFeatureRowsTrace() Co-authored-by: Zhu Zhanyan <zhu.zhanyan@gojek.com>
1 parent fbb7a02 commit eebf458

File tree

33 files changed

+1825
-974
lines changed

33 files changed

+1825
-974
lines changed

‎go.mod‎

Lines changed: 1 addition & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -25,8 +25,7 @@ require (
2525
golang.org/x/lint v0.0.0-20200302205851-738671d3881b // indirect
2626
golang.org/x/net v0.0.0-20200513185701-a91f0712d120
2727
golang.org/x/sys v0.0.0-20200515095857-1151b9dac4a9 // indirect
28-
golang.org/x/tools v0.0.0-20200521211927-2b542361a4fc // indirect
29-
google.golang.org/genproto v0.0.0-20200515170657-fc4c6c6a6587 // indirect
28+
golang.org/x/tools v0.0.0-20200604042327-9b20fe4cabe8 // indirect
3029
google.golang.org/grpc v1.29.1
3130
google.golang.org/protobuf v1.24.0 // indirect
3231
gopkg.in/russross/blackfriday.v2 v2.0.0 // indirect

‎go.sum‎

Lines changed: 8 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -162,6 +162,7 @@ github.com/google/go-cmp v0.3.0 h1:crn/baboCvb5fXaQ0IJ1SGTsTVrWpDsCWC8EGETZijY=
162162
github.com/google/go-cmp v0.3.0/go.mod h1:8QqcDgzrUqlUb/G2PQTWiueGozuR1884gddMywk6iLU=
163163
github.com/google/go-cmp v0.3.1 h1:Xye71clBPdm5HgqGwUkwhbynsUJZhDbS20FvLhQ2izg=
164164
github.com/google/go-cmp v0.3.1/go.mod h1:8QqcDgzrUqlUb/G2PQTWiueGozuR1884gddMywk6iLU=
165+
github.com/google/go-cmp v0.4.0 h1:xsAVV57WRhGj6kEIi8ReJzQlHHqcBYCElAvkovg3B/4=
165166
github.com/google/go-cmp v0.4.0/go.mod h1:v8dTdLbMG2kIc/vJvl+f65V22dbkXbowE6jgT/gNBxE=
166167
github.com/google/gofuzz v0.0.0-20161122191042-44d81051d367/go.mod h1:HP5RmnzzSNb993RKQDq4+1A4ia9nllfqcQFTQJedwGI=
167168
github.com/google/gofuzz v0.0.0-20170612174753-24818f796faf/go.mod h1:HP5RmnzzSNb993RKQDq4+1A4ia9nllfqcQFTQJedwGI=
@@ -319,6 +320,7 @@ github.com/stretchr/testify v1.2.0/go.mod h1:a8OnRcib4nhh0OaRAV+Yts87kKdq0PP7pXf
319320
github.com/stretchr/testify v1.2.2 h1:bSDNvY7ZPG5RlJ8otE/7V6gMiyenm9RtJ7IUVIAoJ1w=
320321
github.com/stretchr/testify v1.2.2/go.mod h1:a8OnRcib4nhh0OaRAV+Yts87kKdq0PP7pXfy6kDkUVs=
321322
github.com/stretchr/testify v1.3.0/go.mod h1:M5WIy9Dh21IEIfnGCwXGc5bZfKNJtfHm1UVUgZn+9EI=
323+
github.com/stretchr/testify v1.4.0 h1:2E4SXV/wtOkTonXsotYi4li6zVWxYlZuYNCXe9XRJyk=
322324
github.com/stretchr/testify v1.4.0/go.mod h1:j7eGeouHqKxXV5pUuKE4zz7dFj8WfuZ+81PSLYec5m4=
323325
github.com/tmc/grpc-websocket-proxy v0.0.0-20170815181823-89b8d40f7ca8/go.mod h1:ncp9v5uamzpCO7NfCPTXjqaC+bZgJeR0sMTm6dMHP7U=
324326
github.com/tmc/grpc-websocket-proxy v0.0.0-20190109142713-0ad062ec5ee5/go.mod h1:ncp9v5uamzpCO7NfCPTXjqaC+bZgJeR0sMTm6dMHP7U=
@@ -466,8 +468,12 @@ golang.org/x/tools v0.0.0-20200519205726-57a9e4404bf7 h1:nm4zDh9WvH4jiuUpMY5RUsv
466468
golang.org/x/tools v0.0.0-20200519205726-57a9e4404bf7/go.mod h1:EkVYQZoAsY45+roYkvgYkIh4xh/qjgUK9TdY2XT94GE=
467469
golang.org/x/tools v0.0.0-20200521211927-2b542361a4fc h1:6m2YO+AmBApbUOmhsghW+IfRyZOY4My4UYvQQrEpHfY=
468470
golang.org/x/tools v0.0.0-20200521211927-2b542361a4fc/go.mod h1:EkVYQZoAsY45+roYkvgYkIh4xh/qjgUK9TdY2XT94GE=
471+
golang.org/x/tools v0.0.0-20200601175630-2caf76543d99/go.mod h1:EkVYQZoAsY45+roYkvgYkIh4xh/qjgUK9TdY2XT94GE=
472+
golang.org/x/tools v0.0.0-20200604042327-9b20fe4cabe8 h1:8Xr1qwxn90MXYKftwNxIO2g4J+26naghxFS5rYiTZww=
473+
golang.org/x/tools v0.0.0-20200604042327-9b20fe4cabe8/go.mod h1:EkVYQZoAsY45+roYkvgYkIh4xh/qjgUK9TdY2XT94GE=
469474
golang.org/x/xerrors v0.0.0-20190717185122-a985d3407aa7/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0=
470475
golang.org/x/xerrors v0.0.0-20191011141410-1b5146add898/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0=
476+
golang.org/x/xerrors v0.0.0-20191204190536-9bdfabe68543 h1:E7g+9GITq07hpfrRu66IVDexMakfv52eLZ2CXBWiKr4=
471477
golang.org/x/xerrors v0.0.0-20191204190536-9bdfabe68543/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0=
472478
gonum.org/v1/gonum v0.0.0-20190331200053-3d26580ed485/go.mod h1:2ltnJ7xHfj0zHS40VVPYEAAMTa3ZGguvHGBSJeRWqE0=
473479
gonum.org/v1/netlib v0.0.0-20190313105609-8cb42192e0e0/go.mod h1:wa6Ws7BG/ESfp6dHfk7C6KdzKA7wR7u/rKwOGE66zvw=
@@ -491,6 +497,7 @@ google.golang.org/genproto v0.0.0-20200319113533-08878b785e9c h1:5aI3/f/3eCZps9x
491497
google.golang.org/genproto v0.0.0-20200319113533-08878b785e9c/go.mod h1:55QSHmfGQM9UVYDPBsyGGes0y52j32PQ3BqQfXhyH3c=
492498
google.golang.org/genproto v0.0.0-20200515170657-fc4c6c6a6587 h1:1Ym+vvUpq1ZHvxzn34gENJX8U4aKO+vhy2P/2+Xl6qQ=
493499
google.golang.org/genproto v0.0.0-20200515170657-fc4c6c6a6587/go.mod h1:YsZOwe1myG/8QRHRsmBRE1LrgQY60beZKjly0O1fX9U=
500+
google.golang.org/genproto v0.0.0-20200526211855-cb27e3aa2013 h1:+kGHl1aib/qcwaRi1CbqBZ1rk19r85MNUf8HaBghugY=
494501
google.golang.org/genproto v0.0.0-20200526211855-cb27e3aa2013/go.mod h1:NbSheEEYHJ7i3ixzK3sjbqSGDJWnxyFXZblF3eUsNvo=
495502
google.golang.org/grpc v1.13.0/go.mod h1:yo6s7OP7yaDglbqo1J04qKzAhqBH6lvTonzMVmEdcZw=
496503
google.golang.org/grpc v1.19.0/go.mod h1:mqu4LbDTu4XGKhr4mRzUsmM4RtVoemTSY81AxZiDr8c=
@@ -540,6 +547,7 @@ gopkg.in/yaml.v2 v2.0.0-20170812160011-eb3733d160e7/go.mod h1:JAlM8MvJe8wmxCU4Bl
540547
gopkg.in/yaml.v2 v2.2.1/go.mod h1:hI93XBmqTisBFMUTm0b8Fm+jr3Dg1NNxqwp+5A1VGuI=
541548
gopkg.in/yaml.v2 v2.2.2 h1:ZCJp+EgiOT7lHqUV2J862kp8Qj64Jo6az82+3Td9dZw=
542549
gopkg.in/yaml.v2 v2.2.2/go.mod h1:hI93XBmqTisBFMUTm0b8Fm+jr3Dg1NNxqwp+5A1VGuI=
550+
gopkg.in/yaml.v2 v2.2.4 h1:/eiJrUcujPVeJ3xlSWaiNi3uSVmDGBK1pDHUHAnao1I=
543551
gopkg.in/yaml.v2 v2.2.4/go.mod h1:hI93XBmqTisBFMUTm0b8Fm+jr3Dg1NNxqwp+5A1VGuI=
544552
gotest.tools v2.2.0+incompatible/go.mod h1:DsYFclhRJ6vuDpmuTbkuFWG+y2sxOXAzmJt81HFBacw=
545553
honnef.co/go/tools v0.0.0-20190102054323-c2f93a96b099/go.mod h1:rf3lG4BRIbNafJWhAfAdb/ePZxsR/4RtNHQocxwk9r4=

‎protos/feast/serving/ServingService.proto‎

Lines changed: 35 additions & 12 deletions
Original file line numberDiff line numberDiff line change
@@ -91,35 +91,58 @@ message GetOnlineFeaturesRequest {
9191
bool omit_entities_in_response = 3;
9292

9393
message EntityRow {
94-
// Request timestamp of this row. This value will be used, together with maxAge,
95-
// to determine feature staleness.
94+
// Request timestamp of this row. This value will be used,
95+
// together with maxAge, to determine feature staleness.
9696
google.protobuf.Timestamp entity_timestamp = 1;
9797

9898
// Map containing mapping of entity name to entity value.
9999
map<string,feast.types.Value> fields = 2;
100100
}
101101
}
102102

103-
message GetBatchFeaturesRequest {
104-
// List of features that are being retrieved
105-
repeated FeatureReference features = 3;
106-
107-
// Source of the entity dataset containing the timestamps and entity keys to retrieve
108-
// features for.
109-
DatasetSource dataset_source = 2;
110-
}
111-
112103
message GetOnlineFeaturesResponse {
113104
// Feature values retrieved from feast.
114105
repeated FieldValues field_values = 1;
115106

116107
message FieldValues {
117108
// Map of feature or entity name to feature/entity values.
118109
// Timestamps are not returned in this response.
119-
map<string,feast.types.Value> fields = 1;
110+
map<string, feast.types.Value> fields = 1;
111+
// Map of feature or entity name to feature/entity statuses/metadata.
112+
map<string, FieldStatus> statuses = 2;
113+
}
114+
115+
enum FieldStatus {
116+
// Status is unset for this field.
117+
INVALID = 0;
118+
119+
// Field value is present for this field and age is within max age.
120+
PRESENT = 1;
121+
122+
// Values could be found for entity key and age is within max age, but
123+
// this field value is assigned a value on ingestion into feast.
124+
NULL_VALUE = 2;
125+
126+
// Entity key did not return any values as they do not exist in Feast.
127+
// This could suggest that the feature values have not yet been ingested
128+
// into feast or the ingestion failed.
129+
NOT_FOUND = 3;
130+
131+
// Values could be found for entity key, but field values are outside the maximum
132+
// allowable range.
133+
OUTSIDE_MAX_AGE = 4;
120134
}
121135
}
122136

137+
message GetBatchFeaturesRequest {
138+
// List of features that are being retrieved
139+
repeated FeatureReference features = 3;
140+
141+
// Source of the entity dataset containing the timestamps and entity keys to retrieve
142+
// features for.
143+
DatasetSource dataset_source = 2;
144+
}
145+
123146
message GetBatchFeaturesResponse {
124147
Job job = 1;
125148
}

‎sdk/go/client.go‎

Lines changed: 11 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -59,21 +59,25 @@ func (fc *GrpcClient) GetOnlineFeatures(ctx context.Context, req *OnlineFeatures
5959
}
6060
}
6161

62-
// strip projects from to projects
63-
for _, fieldValue := range resp.GetFieldValues() {
64-
stripFields := make(map[string]*types.Value, len(fieldValue.Fields))
65-
for refStr, value := range fieldValue.Fields {
62+
// strip projects from feature refs recieved
63+
for _, fieldValues := range resp.GetFieldValues() {
64+
stripFields := make(map[string]*types.Value, len(fieldValues.Fields))
65+
stripStatuses := make(map[string]serving.GetOnlineFeaturesResponse_FieldStatus, len(fieldValues.Statuses))
66+
for refStr, _ := range fieldValues.Fields {
6667
_, isEntity := entityRefs[refStr]
68+
stripRefStr := refStr
6769
if !isEntity { // is feature ref
6870
featureRef, err := parseFeatureRef(refStr, true)
6971
if err != nil {
7072
return nil, err
7173
}
72-
refStr = toFeatureRefStr(featureRef)
74+
stripRefStr = toFeatureRefStr(featureRef)
7375
}
74-
stripFields[refStr] = value
76+
77+
stripFields[stripRefStr] = fieldValues.Fields[refStr]
78+
stripStatuses[stripRefStr] = fieldValues.Statuses[refStr]
7579
}
76-
fieldValue.Fields = stripFields
80+
fieldValues.Fields, fieldValues.Statuses = stripFields, stripStatuses
7781
}
7882

7983
return &OnlineFeaturesResponse{RawResponse: resp}, err

‎sdk/go/client_test.go‎

Lines changed: 13 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -27,6 +27,7 @@ func TestGetOnlineFeatures(t *testing.T) {
2727
Features: []string{
2828
"driver:rating",
2929
"rating",
30+
"null_value",
3031
},
3132
Entities: []Row{
3233
{"driver_id": Int64Val(1)},
@@ -41,6 +42,12 @@ func TestGetOnlineFeatures(t *testing.T) {
4142
Fields: map[string]*types.Value{
4243
"driver_project/driver:rating": Int64Val(1),
4344
"driver_project/rating": Int64Val(1),
45+
"driver_project/null_value": {},
46+
},
47+
Statuses: map[string]serving.GetOnlineFeaturesResponse_FieldStatus{
48+
"driver_project/driver:rating": serving.GetOnlineFeaturesResponse_PRESENT,
49+
"driver_project/rating": serving.GetOnlineFeaturesResponse_PRESENT,
50+
"driver_project/null_value": serving.GetOnlineFeaturesResponse_NULL_VALUE,
4451
},
4552
},
4653
},
@@ -53,6 +60,12 @@ func TestGetOnlineFeatures(t *testing.T) {
5360
Fields: map[string]*types.Value{
5461
"driver:rating": Int64Val(1),
5562
"rating": Int64Val(1),
63+
"null_value": {},
64+
},
65+
Statuses: map[string]serving.GetOnlineFeaturesResponse_FieldStatus{
66+
"driver:rating": serving.GetOnlineFeaturesResponse_PRESENT,
67+
"rating": serving.GetOnlineFeaturesResponse_PRESENT,
68+
"null_value": serving.GetOnlineFeaturesResponse_NULL_VALUE,
5669
},
5770
},
5871
},

0 commit comments

Comments
 (0)