Skip to content

feat: add create in Featurestore, EntityType, Feature; add create_entity_type in Featurestore; add create_feature, batch_create_features in EntityType; add ingest_from_* for bq and gcs in EntityType; add and update delete with force delete nested resources#872

Merged
gcf-merge-on-green[bot] merged 11 commits intogoogleapis:mainfrom
morgandu:mor--feature-store-create-and-ingest
Dec 17, 2021
Merged

Conversation

@morgandu
Copy link
Copy Markdown
Contributor

@morgandu morgandu commented Dec 1, 2021 •
edited
Loading

  • add create method in Featurestore, EntityType, Feature classes;
  • add create_entity_type in Featurestore;
  • add create_feature, batch_create_features in EntityType;
  • add ingest_from_bq and ingest_from_gcs methods in EntityType class
  • add and update delete methods with force delete nested resources
  • add unit tests
  • update integration test

@morgandu morgandu requested review from lclc19 and sasha-gitg December 1, 2021 00:10
@google-cla google-cla Bot added the cla: yes This human has signed the Contributor License Agreement. label Dec 1, 2021
@product-auto-label product-auto-label Bot added the api: aiplatform Issues related to the AI Platform API. label Dec 1, 2021
@morgandu morgandu self-assigned this Dec 1, 2021
@morgandu morgandu force-pushed the mor--feature-store-create-and-ingest branch 2 times, most recently from b9d6932 to 8af8486 Compare December 2, 2021 00:39
@morgandu morgandu changed the title feat: add create method in Featurestore, EntityType, Feature classes; add create_entity_type in Featurestore; add create_feature, batch_create_features in EntityType; add ingest_from_bq and ingest_from_gcs methods in EntityType class feat: add create method in Featurestore, EntityType, Feature classes; add create_entity_type in Featurestore; add create_feature, batch_create_features in EntityType; add ingest_from_bq and ingest_from_gcs methods in EntityType class; add and update delete methods with force delete nested resources Dec 2, 2021
@morgandu morgandu changed the title feat: add create method in Featurestore, EntityType, Feature classes; add create_entity_type in Featurestore; add create_feature, batch_create_features in EntityType; add ingest_from_bq and ingest_from_gcs methods in EntityType class; add and update delete methods with force delete nested resources feat: add create in Featurestore, EntityType, Feature; add create_entity_type in Featurestore; add create_feature, batch_create_features in EntityType; add ingest_from_* for bq and gcs in EntityType; add and update delete with force delete nested resources Dec 2, 2021
@morgandu morgandu force-pushed the mor--feature-store-create-and-ingest branch 4 times, most recently from f38cd2d to 56d0db5 Compare December 2, 2021 19:50
diemtvu
diemtvu reviewed Dec 2, 2021
View reviewed changes
Comment thread google/cloud/aiplatform/featurestore/feature.py Outdated
Comment thread google/cloud/aiplatform/featurestore/feature.py
Comment thread google/cloud/aiplatform/featurestore/featurestore.py Outdated
Comment thread google/cloud/aiplatform/featurestore/featurestore.py Outdated
Comment thread google/cloud/aiplatform/featurestore/featurestore.py Outdated
Comment thread google/cloud/aiplatform/featurestore/featurestore.py Outdated
Comment thread google/cloud/aiplatform/featurestore/featurestore.py Outdated
Comment thread tests/system/aiplatform/test_featurestore.py Outdated
@morgandu morgandu force-pushed the mor--feature-store-create-and-ingest branch from 56d0db5 to 9b1b72a Compare December 4, 2021 00:53
sasha-gitg
sasha-gitg requested changes Dec 6, 2021
View reviewed changes
Comment thread google/cloud/aiplatform/featurestore/entity_type.py Outdated
Comment thread google/cloud/aiplatform/featurestore/entity_type.py Outdated
Comment thread google/cloud/aiplatform/featurestore/entity_type.py Outdated
Comment thread google/cloud/aiplatform/featurestore/entity_type.py Outdated
Comment thread google/cloud/aiplatform/featurestore/entity_type.py Outdated
Comment thread google/cloud/aiplatform/utils/featurestore_utils.py Outdated
Comment thread google/cloud/aiplatform/utils/featurestore_utils.py Outdated
Comment thread google/cloud/aiplatform/utils/featurestore_utils.py
Comment thread google/cloud/aiplatform/utils/featurestore_utils.py Outdated
Comment thread tests/system/aiplatform/e2e_base.py Outdated
@morgandu morgandu force-pushed the mor--feature-store-create-and-ingest branch from 9b1b72a to 75ac97e Compare December 10, 2021 06:18
@morgandu morgandu requested a review from sasha-gitg December 10, 2021 06:18
diemtvu
diemtvu reviewed Dec 10, 2021
View reviewed changes
Comment thread google/cloud/aiplatform/featurestore/entity_type.py
Comment thread google/cloud/aiplatform/featurestore/entity_type.py
Comment thread google/cloud/aiplatform/featurestore/entity_type.py Outdated
Comment thread tests/unit/aiplatform/test_utils.py Outdated
@morgandu morgandu force-pushed the mor--feature-store-create-and-ingest branch 2 times, most recently from 41580d9 to 03aa950 Compare December 11, 2021 01:04
sasha-gitg
sasha-gitg requested changes Dec 13, 2021
View reviewed changes
Copy link
Copy Markdown
Member

@sasha-gitg sasha-gitg left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

One high level comment: if possible, please avoid force updating the commit history as some of the previous commits are no longer accessible and it becomes difficult to follow previous comments. It also seems to break "Show changes since last review" view. Thanks!

Comment thread google/cloud/aiplatform/featurestore/entity_type.py
Comment thread google/cloud/aiplatform/featurestore/entity_type.py Outdated
Comment thread google/cloud/aiplatform/featurestore/entity_type.py Outdated
Comment thread google/cloud/aiplatform/featurestore/entity_type.py Outdated
Comment thread google/cloud/aiplatform/featurestore/entity_type.py Outdated
Comment thread google/cloud/aiplatform/featurestore/entity_type.py Outdated
Comment thread google/cloud/aiplatform/featurestore/entity_type.py
Comment thread google/cloud/aiplatform/featurestore/featurestore.py Outdated
diemtvu
diemtvu reviewed Dec 13, 2021
View reviewed changes
Comment thread tests/unit/aiplatform/test_utils.py Outdated
Comment thread google/cloud/aiplatform/featurestore/entity_type.py Outdated
Comment thread google/cloud/aiplatform/featurestore/entity_type.py
Comment thread google/cloud/aiplatform/featurestore/entity_type.py
@morgandu morgandu requested a review from sasha-gitg December 13, 2021 21:26
sasha-gitg
sasha-gitg approved these changes Dec 16, 2021
View reviewed changes
Copy link
Copy Markdown
Member

@sasha-gitg sasha-gitg left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM! Thanks!

@morgandu
feat: add create method in Featurestore, EntityType, Feature classes;…
95f7584 
… add create_entity_type in Featurestore; add create_feature, batch_create_features in EntityType; add ingest_from_bq and ingest_from_gcs methods in EntityType class
@morgandu
feat: enable force on resource deletion
0b494c7
@morgandu
fix: change gcs to str and list[str]; batch_create_request error by c…
f4c5df8 
…hanging feature_config.request_dict to get_create_feature_request() with parent option
@morgandu
fix: change request dict to request proto, or pass in (with parent) d…
06fb2e1 
…irectly to api method; update docstring for feature_source_fields and node_count
@morgandu
fix: updated location in create methods, changed args to data_source …
9198446 
…and feature_time for oneof, changed method validate_id to _is_resource_id, changed method validate_value_type to return None
@morgandu
update: integration tests
5f8492f
@morgandu
update docstring for requiring existing parent resources; update prot…
f2a0956 
…o setting for values if None; fix potential delete breaking change; change request_metadata to None and tests; update unit test for get_timestamp_proto; refactor feature_source_fields; remove optional batch_create from ingest and tests;
@morgandu
update test_get_timestamp_proto using utc timezone info for seconds c…
f24bdbd 
…onversion
@morgandu
revert request_metadata to ()
e6b34c2
@morgandu
fix: async test for ingestion
e26a1cb
@morgandu
include nested resorce refactor
1e1c2fb
@morgandu morgandu force-pushed the mor--feature-store-create-and-ingest branch from a5ab277 to 1e1c2fb Compare December 17, 2021 02:35
@morgandu morgandu added the automerge Merge the pull request once unit tests and other checks pass. label Dec 17, 2021
@gcf-merge-on-green gcf-merge-on-green Bot merged commit ba11c3d into googleapis:main Dec 17, 2021
@gcf-merge-on-green gcf-merge-on-green Bot removed the automerge Merge the pull request once unit tests and other checks pass. label Dec 17, 2021
@morgandu morgandu deleted the mor--feature-store-create-and-ingest branch December 17, 2021 03:40
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@sasha-gitg sasha-gitg sasha-gitg approved these changes

@lclc19 lclc19 Awaiting requested review from lclc19

+1 more reviewer

@diemtvu diemtvu diemtvu approved these changes

Reviewers whose approvals may not affect merge requirements

Assignees

@morgandu morgandu

Labels

api: aiplatform Issues related to the AI Platform API. cla: yes This human has signed the Contributor License Agreement.

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

3 participants

@morgandu @diemtvu @sasha-gitg