Skip to content

Commit a229450

Browse files
jklegarjklegar
committed
Add option to use object store metadata registry (feast-dev#1336)
* Add option to use object store metadata registry Signed-off-by: Jacob Klegar <jacob@tecton.ai> * address comments Signed-off-by: Jacob Klegar <jacob@tecton.ai> * bugfixes and add tests Signed-off-by: Jacob Klegar <jacob@tecton.ai> * add projects Signed-off-by: Jacob Klegar <jacob@tecton.ai> * get gcs test working Signed-off-by: Jacob Klegar <jacob@tecton.ai> * add gcs tests to GH actions in master_only Signed-off-by: Jacob Klegar <jacob@tecton.ai> * lint Signed-off-by: Jacob Klegar <jacob@tecton.ai> * missed an import Signed-off-by: Jacob Klegar <jacob@tecton.ai> * fix protobuf assignment Signed-off-by: Jacob Klegar <jacob@tecton.ai> * addressing comments Signed-off-by: Jacob Klegar <jacob@tecton.ai> * lint Signed-off-by: Jacob Klegar <jacob@tecton.ai> * update docstrings per comments Signed-off-by: Jacob Klegar <jacob@tecton.ai>
1 parent 7a804cf commit a229450

File tree

9 files changed

+725
-124
lines changed

9 files changed

+725
-124
lines changed

‎.github/workflows/master_only.yml‎

Lines changed: 23 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,23 @@
1+
name: master_only
2+
3+
on:
4+
push:
5+
branches:
6+
- master
7+
8+
jobs:
9+
integration-test-python:
10+
runs-on: ubuntu-latest
11+
container: gcr.io/kf-feast/feast-ci:latest
12+
steps:
13+
- uses: actions/checkout@v2
14+
- uses: GoogleCloudPlatform/github-actions/setup-gcloud@master
15+
with:
16+
version: '290.0.1'
17+
export_default_credentials: true
18+
project_id: ${{ secrets.GCP_PROJECT_ID }}
19+
service_account_key: ${{ secrets.GCP_SA_KEY }}
20+
- name: Install python
21+
run: make install-python
22+
- name: Test python
23+
run: FEAST_TELEMETRY=False pytest --verbose --color=yes sdk/python/tests --integration

‎protos/feast/core/Entity.proto‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -29,6 +29,7 @@ message Entity {
2929
EntitySpecV2 spec = 1;
3030
// System-populated metadata for this entity.
3131
EntityMeta meta = 2;
32+
string project = 3;
3233
}
3334

3435
message EntitySpecV2 {

‎protos/feast/core/FeatureTable.proto‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -34,6 +34,7 @@ message FeatureTable {
3434

3535
// System-populated metadata for this feature table.
3636
FeatureTableMeta meta = 2;
37+
string project = 3;
3738
}
3839

3940
message FeatureTableSpec {

‎protos/feast/core/Registry.proto‎

Lines changed: 35 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,35 @@
1+
//
2+
// * Copyright 2020 The Feast Authors
3+
// *
4+
// * Licensed under the Apache License, Version 2.0 (the "License");
5+
// * you may not use this file except in compliance with the License.
6+
// * You may obtain a copy of the License at
7+
// *
8+
// * https://www.apache.org/licenses/LICENSE-2.0
9+
// *
10+
// * Unless required by applicable law or agreed to in writing, software
11+
// * distributed under the License is distributed on an "AS IS" BASIS,
12+
// * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
13+
// * See the License for the specific language governing permissions and
14+
// * limitations under the License.
15+
//
16+
17+
syntax = "proto3";
18+
19+
package feast.core;
20+
option java_package = "feast.proto.core";
21+
option java_outer_classname = "RegistryProto";
22+
option go_package = "github.com/feast-dev/feast/sdk/go/protos/feast/core";
23+
24+
import "feast/core/Entity.proto";
25+
import "feast/core/FeatureTable.proto";
26+
import "google/protobuf/timestamp.proto";
27+
28+
message Registry {
29+
repeated Entity entities = 1;
30+
repeated FeatureTable feature_tables = 2;
31+
32+
string registry_schema_version = 3; // to support migrations; incremented when schema is changed
33+
string version_id = 4; // version id, random string generated on each update of the data; now used only for debugging purposes
34+
google.protobuf.Timestamp last_updated = 5;
35+
}

0 commit comments

Comments
 (0)