Skip to content

Add Discovery API for listing Features - #797

Merged
feast-ci-bot merged 16 commits into
feast-dev:masterfrom
terryyylim:discovery-api
Jun 18, 2020
Merged

Add Discovery API for listing Features#797
feast-ci-bot merged 16 commits into
feast-dev:masterfrom
terryyylim:discovery-api

Conversation

@terryyylim

@terryyylim terryyylim commented Jun 13, 2020

Copy link
Copy Markdown
Member

What this PR does / why we need it:
Adds listing functionality for Features.
Features

  • Can list by project, entities, labels

Which issue(s) this PR fixes:

Fixes #

Does this PR introduce a user-facing change?:

User can now list Features.

@terryyylim terryyylim added the kind/feature New feature or request label Jun 13, 2020
@terryyylim terryyylim changed the title Add Discovery API for listing Features and Entities [WIP] Add Discovery API for listing Features and Entities Jun 13, 2020
@terryyylim terryyylim changed the title [WIP] Add Discovery API for listing Features and Entities Add Discovery API for listing Features and Entities Jun 13, 2020
Comment thread core/src/main/java/feast/core/grpc/CoreServiceImpl.java
ListEntitiesResponse response = specService.listEntities(request.getFilter());
responseObserver.onNext(response);
responseObserver.onCompleted();
} catch (RetrievalException | IllegalArgumentException | InvalidProtocolBufferException e) {

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is it possible for you to return different gRPC exceptions here instead of INTERNAL for everything? Also, we should probably also catch and return unknown exceptions for the time being, because it can be hard to debug otherwise.

public boolean hasAllEntities(List<String> entitiesFilter) {
List<String> allEntitiesName =
this.entities.stream()
.map(entitySpec -> entitySpec.toProto().getName())

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why are you converting it to a proto?

this.entities.stream()
.map(entitySpec -> entitySpec.toProto().getName())
.collect(Collectors.toList());
for (String entity : entitiesFilter) {

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think it would be easier to sort and compare the lists for equality.

.collect(Collectors.toList());
for (Feature feature : validFeatures) {
validFeaturesMap.put(
this.getProject().getName() + "/" + this.getName() + ":" + feature.getName(), feature);

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Isn't this feature reference building available elsewhere in the code base?

* @param labelsFilter contain labels that should be attached to FeatureSet's features
* @return Map of Feature references and Features
*/
public Map<String, Feature> getFeaturesByAllLabels(Map<String, String> labelsFilter) {

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It seems strange that filtering is a part of this method. Can that be done externally with a static method? I would rather this be getFeaturesByRef and then have a filtering step outside.

* filter
*/
public ListFeaturesResponse listFeatures(ListFeaturesRequest.Filter filter)
throws InvalidProtocolBufferException {

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Where is InvalidProtocolBufferException being thrown and why do you leave it unhandled? It seems like we can add more context there.

// Matching a specific project
featureSets =
featureSetRepository.findAllByNameLikeAndProject_NameOrderByNameAsc("%", project);
} else {

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In what scenario would this else be reached?

}

ListFeaturesResponse.Builder response = ListFeaturesResponse.newBuilder();
if (featureSets.size() > 0) {

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think we need this conditional for size (I know it was in the other method)

Comment thread sdk/python/feast/client.py Outdated
return FeatureSet.from_proto(get_feature_set_response.feature_set)

def list_entities(self) -> Dict[str, Entity]:
def list_features(

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you please add minimal examples to these two methods?

https://sphinxcontrib-napoleon.readthedocs.io/en/latest/example_google.html

Comment thread sdk/python/feast/client.py Outdated
) -> Dict[FeatureRef, Feature]:
"""
Returns a dictionary of entities across all feature sets
Returns a list of features based on filters provided

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You are missing arguments in your docstring.

Comment thread sdk/python/feast/client.py Outdated

return features_dict

def list_entities(self, project: str = None) -> Dict[str, Entity]:

@woop woop Jun 15, 2020

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Missing argument in docstring.

Comment thread sdk/python/tests/test_client.py Outdated
"mocked_client",
[pytest.lazy_fixture("mock_client"), pytest.lazy_fixture("secure_mock_client")],
)
def test_list_entities(self, mocked_client, mocker):

@woop woop Jun 15, 2020

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What part of the code are you testing here? Seems like you are testing almost no code.

("driver_id", driver_entity)
])

filter_by_project_entity_labels_expected = OrderedDict([

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I dont see a good reason for using OrderedDicts here.

@woop woop left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not too far off the mark, but needs some changes. I would also recommend that you update the relevant documentation for these methods in our documentation.

@terryyylim terryyylim changed the title Add Discovery API for listing Features and Entities Add Discovery API for listing Features Jun 15, 2020
@terryyylim

Copy link
Copy Markdown
Member Author

/test test-end-to-end-batch

@ches ches added this to the v0.6.0 milestone Jun 16, 2020
@feast-ci-bot

Copy link
Copy Markdown
Collaborator

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: terryyylim, woop

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Details Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@woop

woop commented Jun 17, 2020

Copy link
Copy Markdown
Member

/lgtm

@feast-ci-bot feast-ci-bot removed the lgtm label Jun 17, 2020
@terryyylim

Copy link
Copy Markdown
Member Author

/test test-end-to-end-batch

@woop

woop commented Jun 18, 2020

Copy link
Copy Markdown
Member

/lgtm

@feast-ci-bot
feast-ci-bot merged commit e343c2f into feast-dev:master Jun 18, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants