From ff4c9af2440635d4ad42552c1c5003a04b044647 Mon Sep 17 00:00:00 2001 From: Willem Pienaar <6728866+woop@users.noreply.github.com> Date: Tue, 19 May 2020 17:39:55 +0800 Subject: [PATCH 001/133] Release v0.5 (#717) --- CHANGELOG.md | 75 ++++++++++++++++++++++++++++++++++++++++++++++++++++ pom.xml | 2 +- 2 files changed, 76 insertions(+), 1 deletion(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 5161174e48b..5339b8c5f57 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -1,5 +1,80 @@ # Changelog +## [v0.5.0](https://github.com/feast-dev/feast/tree/v0.5.0) (2020-05-19) + +[Full Changelog](https://github.com/feast-dev/feast/compare/v0.4.7...v0.5.0) + +**Breaking changes:** + +- Add .proto to packages of Protobuf generated Java classes [\#700](https://github.com/feast-dev/feast/pull/700) ([woop](https://github.com/woop)) +- Add support for feature set updates and remove versions [\#676](https://github.com/feast-dev/feast/pull/676) ([zhilingc](https://github.com/zhilingc)) +- Feast configuration files refactored [\#611](https://github.com/feast-dev/feast/pull/611) ([woop](https://github.com/woop)) + +See [Feast 0.5 Release Issue](https://github.com/feast-dev/feast/issues/527) for more details. + +**Implemented enhancements:** + +- Add general storage API and refactor existing store implementations [\#567](https://github.com/feast-dev/feast/pull/567) ([zhilingc](https://github.com/zhilingc)) +- Add support for feature set updates and remove versions [\#676](https://github.com/feast-dev/feast/pull/676) ([zhilingc](https://github.com/zhilingc)) +- Add unique ingestion id for all batch ingestions [\#656](https://github.com/feast-dev/feast/pull/656) ([zhilingc](https://github.com/zhilingc)) +- Add storage interfaces [\#529](https://github.com/feast-dev/feast/pull/529) ([zhilingc](https://github.com/zhilingc)) +- Add BigQuery storage implementation [\#546](https://github.com/feast-dev/feast/pull/546) ([zhilingc](https://github.com/zhilingc)) +- Add Redis storage implementation [\#547](https://github.com/feast-dev/feast/pull/547) ([zhilingc](https://github.com/zhilingc)) +- Add Support for Redis Cluster [\#502](https://github.com/feast-dev/feast/pull/502) ([lavkesh](https://github.com/lavkesh)) +- Add Ingestion Job management API for Feast Core [\#548](https://github.com/feast-dev/feast/pull/548) ([mrzzy](https://github.com/mrzzy)) +- Add feature and feature set labels for metadata [\#536](https://github.com/feast-dev/feast/pull/536) ([imjuanleonard](https://github.com/imjuanleonard)) +- Update Python SDK so FeatureSet can import Schema from Tensorflow metadata [\#450](https://github.com/feast-dev/feast/pull/450) ([davidheryanto](https://github.com/davidheryanto)) + +**Fixed bugs:** + +- Add feature set status JOB\_STARTING to denote feature sets waiting for job to get to RUNNING state [\#714](https://github.com/feast-dev/feast/pull/714) ([zhilingc](https://github.com/zhilingc)) +- Remove feature set status check for job update requirement [\#708](https://github.com/feast-dev/feast/pull/708) ([khorshuheng](https://github.com/khorshuheng)) +- Fix Feast Core docker image [\#703](https://github.com/feast-dev/feast/pull/703) ([khorshuheng](https://github.com/khorshuheng)) +- Include server port config on the generated application.yml [\#696](https://github.com/feast-dev/feast/pull/696) ([khorshuheng](https://github.com/khorshuheng)) +- Fix typo in all types parquet yml file \(e2e test\) [\#683](https://github.com/feast-dev/feast/pull/683) ([khorshuheng](https://github.com/khorshuheng)) +- Add grpc health probe implementation to core [\#680](https://github.com/feast-dev/feast/pull/680) ([zhilingc](https://github.com/zhilingc)) +- Ensure that generated python code are considered as module [\#679](https://github.com/feast-dev/feast/pull/679) ([khorshuheng](https://github.com/khorshuheng)) +- Fix DataflowJobManager to update existing job instance instead of creating new one [\#678](https://github.com/feast-dev/feast/pull/678) ([zhilingc](https://github.com/zhilingc)) +- Fix config validation for feast.jobs.metrics.host [\#662](https://github.com/feast-dev/feast/pull/662) ([davidheryanto](https://github.com/davidheryanto)) +- Docker compose bug fix [\#661](https://github.com/feast-dev/feast/pull/661) ([woop](https://github.com/woop)) +- Swap join columns [\#647](https://github.com/feast-dev/feast/pull/647) ([zhilingc](https://github.com/zhilingc)) +- Fix Feast Serving not registering its store in Feast Core [\#641](https://github.com/feast-dev/feast/pull/641) ([mrzzy](https://github.com/mrzzy)) +- Kafka producer should raise an exception when it fails to connect to broker [\#636](https://github.com/feast-dev/feast/pull/636) ([junhui096](https://github.com/junhui096)) + +**Merged pull requests:** + +- Change organization from gojek to feast-dev [\#712](https://github.com/feast-dev/feast/pull/712) ([woop](https://github.com/woop)) +- Extract feature set update tests so CI doesn't run it [\#709](https://github.com/feast-dev/feast/pull/709) ([zhilingc](https://github.com/zhilingc)) +- Ensure that batch retrieval tests clean up after themselves [\#704](https://github.com/feast-dev/feast/pull/704) ([zhilingc](https://github.com/zhilingc)) +- Apply default project to rows without project during ingestion [\#701](https://github.com/feast-dev/feast/pull/701) ([zhilingc](https://github.com/zhilingc)) +- Update tests to correct compute region [\#699](https://github.com/feast-dev/feast/pull/699) ([terryyylim](https://github.com/terryyylim)) +- Make Projects optional & Update Feature References [\#693](https://github.com/feast-dev/feast/pull/693) ([mrzzy](https://github.com/mrzzy)) +- Add Java code coverage reporting [\#686](https://github.com/feast-dev/feast/pull/686) ([ches](https://github.com/ches)) +- Update e2e tests to allow non-SNAPSHOT testing [\#672](https://github.com/feast-dev/feast/pull/672) ([woop](https://github.com/woop)) +- Move TFDV stats to higher-numbered protobuf fields [\#669](https://github.com/feast-dev/feast/pull/669) ([ches](https://github.com/ches)) +- Clean up Docker Compose and add test [\#668](https://github.com/feast-dev/feast/pull/668) ([woop](https://github.com/woop)) +- Enable Prow e2e tests by default [\#666](https://github.com/feast-dev/feast/pull/666) ([woop](https://github.com/woop)) +- Add label checking to Prow [\#665](https://github.com/feast-dev/feast/pull/665) ([woop](https://github.com/woop)) +- Upgrade Github Checkout action to v2 [\#660](https://github.com/feast-dev/feast/pull/660) ([khorshuheng](https://github.com/khorshuheng)) +- Fix Redis cluster e2e [\#659](https://github.com/feast-dev/feast/pull/659) ([terryyylim](https://github.com/terryyylim)) +- Split Field model into distinct Feature and Entity objects [\#655](https://github.com/feast-dev/feast/pull/655) ([zhilingc](https://github.com/zhilingc)) +- Use Runner enum type instead of string for Job model [\#651](https://github.com/feast-dev/feast/pull/651) ([ches](https://github.com/ches)) +- JobUpdateTask cleanups [\#650](https://github.com/feast-dev/feast/pull/650) ([ches](https://github.com/ches)) +- Update approvers list [\#648](https://github.com/feast-dev/feast/pull/648) ([khorshuheng](https://github.com/khorshuheng)) +- Update end-to-end test config [\#645](https://github.com/feast-dev/feast/pull/645) ([zhilingc](https://github.com/zhilingc)) +- Fix bigquery config for serving store [\#644](https://github.com/feast-dev/feast/pull/644) ([zhilingc](https://github.com/zhilingc)) +- Fix Dataflow translator bug [\#643](https://github.com/feast-dev/feast/pull/643) ([zhilingc](https://github.com/zhilingc)) +- Fix subscription config and doctests [\#634](https://github.com/feast-dev/feast/pull/634) ([woop](https://github.com/woop)) +- Correct links to why-feast and concepts doc site [\#627](https://github.com/feast-dev/feast/pull/627) ([anderseriksson](https://github.com/anderseriksson)) +- Make error on retrieval of nonexistent feature humanly readable [\#625](https://github.com/feast-dev/feast/pull/625) ([mrzzy](https://github.com/mrzzy)) +- Generate golang code for non-serving protos [\#618](https://github.com/feast-dev/feast/pull/618) ([zhilingc](https://github.com/zhilingc)) +- Regenerate golang code, fix proto comparisons [\#616](https://github.com/feast-dev/feast/pull/616) ([zhilingc](https://github.com/zhilingc)) +- Fix doc building [\#603](https://github.com/feast-dev/feast/pull/603) ([woop](https://github.com/woop)) +- Pin Jupyter Notebook version [\#597](https://github.com/feast-dev/feast/pull/597) ([imjuanleonard](https://github.com/imjuanleonard)) +- Create project if not exists on applyFeatureSet [\#596](https://github.com/feast-dev/feast/pull/596) ([Joostrothweiler](https://github.com/Joostrothweiler)) +- Apply a fixed window before writing row metrics [\#590](https://github.com/feast-dev/feast/pull/590) ([davidheryanto](https://github.com/davidheryanto)) +- Allow tests to run on non-master branches [\#588](https://github.com/feast-dev/feast/pull/588) ([woop](https://github.com/woop)) + ## [v0.4.7](https://github.com/feast-dev/feast/tree/v0.4.7) (2020-03-17) [Full Changelog](https://github.com/feast-dev/feast/compare/v0.4.6...v0.4.7) diff --git a/pom.xml b/pom.xml index 19154b9cf13..fe849fcf8f2 100644 --- a/pom.xml +++ b/pom.xml @@ -39,7 +39,7 @@ - 0.5-SNAPSHOT + 0.5 https://github.com/feast-dev/feast UTF-8 From 426d6dbd35c7e9fe8a9084722af26470f87ec3f9 Mon Sep 17 00:00:00 2001 From: Willem Pienaar Date: Tue, 19 May 2020 17:47:43 +0800 Subject: [PATCH 002/133] Bump version and add SNAPSHOT to pom.xml --- pom.xml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/pom.xml b/pom.xml index fe849fcf8f2..018a101adb9 100644 --- a/pom.xml +++ b/pom.xml @@ -39,7 +39,7 @@ - 0.5 + 0.6-SNAPSHOT https://github.com/feast-dev/feast UTF-8 From 2d8d7ac586a39688ab2392201c76727edf175e19 Mon Sep 17 00:00:00 2001 From: zhilingc Date: Thu, 21 May 2020 17:36:05 +0800 Subject: [PATCH 003/133] Python sdk fixes --- sdk/python/feast/cli.py | 2 +- sdk/python/feast/client.py | 8 +------- 2 files changed, 2 insertions(+), 8 deletions(-) diff --git a/sdk/python/feast/cli.py b/sdk/python/feast/cli.py index 87f12301f86..36d1d489e41 100644 --- a/sdk/python/feast/cli.py +++ b/sdk/python/feast/cli.py @@ -128,7 +128,7 @@ def feature_set_list(): feast_client = Client() # type: Client table = [] - for fs in feast_client.list_feature_sets(): + for fs in feast_client.list_feature_sets(project="*", name="*"): table.append([fs.name, repr(fs)]) from tabulate import tabulate diff --git a/sdk/python/feast/client.py b/sdk/python/feast/client.py index 50e3a65bad2..5e4baf9d598 100644 --- a/sdk/python/feast/client.py +++ b/sdk/python/feast/client.py @@ -385,13 +385,7 @@ def _apply_feature_set(self, feature_set: FeatureSet): feature_set.is_valid() feature_set_proto = feature_set.to_proto() if len(feature_set_proto.spec.project) == 0: - if self.project is None: - raise ValueError( - f"No project found in feature set {feature_set.name}. " - f"Please set the project within the feature set or within " - f"your Feast Client." - ) - else: + if self.project is not None: feature_set_proto.spec.project = self.project # Convert the feature set to a request and send to Feast Core From 8b04cf1fa4c3c2abc1f6419182521adcef37397c Mon Sep 17 00:00:00 2001 From: Willem Pienaar <6728866+woop@users.noreply.github.com> Date: Fri, 22 May 2020 00:00:29 +0800 Subject: [PATCH 004/133] Fix Docker Compose tests for Feast 0.5 (#725) * Update Docker Compose tests to Feast 0.5 * Set lower job polling rate for Docker Compose --- infra/docker-compose/.env.sample | 4 +++- infra/docker-compose/core/direct-runner.yml | 3 --- infra/docker-compose/docker-compose.yml | 2 ++ infra/docker-compose/jupyter/startup.sh | 3 +++ .../docker-compose/serving/online-serving.yml | 21 +++++++++---------- infra/scripts/test-docker-compose.sh | 17 ++++++++++++--- 6 files changed, 32 insertions(+), 18 deletions(-) delete mode 100644 infra/docker-compose/core/direct-runner.yml diff --git a/infra/docker-compose/.env.sample b/infra/docker-compose/.env.sample index 8ca7ca008fb..f3de7330630 100644 --- a/infra/docker-compose/.env.sample +++ b/infra/docker-compose/.env.sample @@ -1,12 +1,14 @@ # General COMPOSE_PROJECT_NAME=feast FEAST_VERSION=latest -FEAST_REPOSITORY_VERSION=v0.4.7 +FEAST_REPOSITORY_VERSION=v0.5-branch # Feast Core FEAST_CORE_IMAGE=gcr.io/kf-feast/feast-core FEAST_CORE_CONFIG=direct-runner.yml FEAST_CORE_GCP_SERVICE_ACCOUNT_KEY=placeholder.json +FEAST_STREAM_OPTIONS_BOOTSTRAPSERVERS=kafka:9092 +FEAST_JOBS_POLLING_INTERVAL_MILLISECONDS=10 # Feast Serving Batch (BigQuery) FEAST_BATCH_SERVING_CONFIG=batch-serving.yml diff --git a/infra/docker-compose/core/direct-runner.yml b/infra/docker-compose/core/direct-runner.yml deleted file mode 100644 index f9123480bb4..00000000000 --- a/infra/docker-compose/core/direct-runner.yml +++ /dev/null @@ -1,3 +0,0 @@ -feast: - jobs: - runner: DirectRunner diff --git a/infra/docker-compose/docker-compose.yml b/infra/docker-compose/docker-compose.yml index 1dda766608e..d327b8e3c80 100644 --- a/infra/docker-compose/docker-compose.yml +++ b/infra/docker-compose/docker-compose.yml @@ -9,6 +9,8 @@ services: environment: DB_HOST: db GOOGLE_APPLICATION_CREDENTIALS: /etc/gcloud/service-accounts/key.json + FEAST_STREAM_OPTIONS_BOOTSTRAPSERVERS: ${FEAST_STREAM_OPTIONS_BOOTSTRAPSERVERS} + FEAST_JOBS_POLLING_INTERVAL_MILLISECONDS: ${FEAST_JOBS_POLLING_INTERVAL_MILLISECONDS} restart: on-failure depends_on: - db diff --git a/infra/docker-compose/jupyter/startup.sh b/infra/docker-compose/jupyter/startup.sh index f9a17984536..5279323cf89 100755 --- a/infra/docker-compose/jupyter/startup.sh +++ b/infra/docker-compose/jupyter/startup.sh @@ -5,6 +5,9 @@ set -ex # Clone Feast repository into Jupyter container git clone -b ${FEAST_REPOSITORY_VERSION} --single-branch https://github.com/feast-dev/feast.git || true +# Install Python dependencies +make -C feast/ compile-protos-python + # Install CI requirements (only needed for running tests) pip install -r feast/sdk/python/requirements-ci.txt diff --git a/infra/docker-compose/serving/online-serving.yml b/infra/docker-compose/serving/online-serving.yml index 089af32dd45..7ca1e7113f3 100644 --- a/infra/docker-compose/serving/online-serving.yml +++ b/infra/docker-compose/serving/online-serving.yml @@ -1,13 +1,12 @@ feast: core-host: core - core-grpc-port: 6565 - store: - config-path: /etc/feast/store.yml - jobs: - store-type: REDIS - store-options: - host: redis - port: 6379 - -grpc: - port: 6566 \ No newline at end of file + active_store: online + stores: + - name: online + type: REDIS + config: + host: redis + port: 6379 + subscriptions: + - name: "*" + project: "*" \ No newline at end of file diff --git a/infra/scripts/test-docker-compose.sh b/infra/scripts/test-docker-compose.sh index 40a85900c2f..f3aa9289e20 100755 --- a/infra/scripts/test-docker-compose.sh +++ b/infra/scripts/test-docker-compose.sh @@ -8,6 +8,20 @@ Running Docker Compose tests with pytest at 'tests/e2e' ============================================================ " +clean_up () { + ARG=$? + + # Shut down docker-compose images + docker-compose -f docker-compose.yml -f docker-compose.online.yml down + + # Remove configuration file + rm .env + + exit $ARG +} + +trap clean_up EXIT + export PROJECT_ROOT_DIR=$(git rev-parse --show-toplevel) export COMPOSE_INTERACTIVE_NO_CLI=1 @@ -30,6 +44,3 @@ ${PROJECT_ROOT_DIR}/infra/scripts/wait-for-it.sh ${JUPYTER_DOCKER_CONTAINER_IP_A # Run e2e tests for Redis docker exec feast_jupyter_1 bash -c 'cd feast/tests/e2e/ && pytest -s basic-ingest-redis-serving.py --core_url core:6565 --serving_url=online-serving:6566' - -# Shut down docker-compose images -docker-compose -f docker-compose.yml -f docker-compose.online.yml down \ No newline at end of file From de6d868938334262410aa1dca531b2a7b1ac31e9 Mon Sep 17 00:00:00 2001 From: Tan Yee Jian Date: Fri, 22 May 2020 00:05:00 +0800 Subject: [PATCH 005/133] Increase Docker Compose test timeout (#715) --- tests/e2e/basic-ingest-redis-serving.py | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/tests/e2e/basic-ingest-redis-serving.py b/tests/e2e/basic-ingest-redis-serving.py index 311f1282477..619b8ca6ce2 100644 --- a/tests/e2e/basic-ingest-redis-serving.py +++ b/tests/e2e/basic-ingest-redis-serving.py @@ -129,7 +129,7 @@ def test_basic_ingest_success(client, cust_trans_df, driver_df): time.sleep(5) -@pytest.mark.timeout(45) +@pytest.mark.timeout(60) @pytest.mark.run(order=12) def test_basic_retrieve_online_success(client, cust_trans_df): # Poll serving for feature values until the correct values are returned From b8456c1bdc1a46fa81d4e25c4ccef7d994c0236f Mon Sep 17 00:00:00 2001 From: Willem Pienaar Date: Fri, 22 May 2020 10:09:53 +0800 Subject: [PATCH 006/133] Remove Docker Compose test from PRs --- .github/workflows/docker_compose_tests.yml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/.github/workflows/docker_compose_tests.yml b/.github/workflows/docker_compose_tests.yml index 28411b88802..301ba116675 100644 --- a/.github/workflows/docker_compose_tests.yml +++ b/.github/workflows/docker_compose_tests.yml @@ -1,6 +1,6 @@ name: docker compose tests -on: [push, pull_request] +on: [push] jobs: basic-redis-e2e-tests-docker-compose: From aeacb0027a5f47042ac1cce86d019666272b0c6b Mon Sep 17 00:00:00 2001 From: Oleksii Moskalenko Date: Fri, 22 May 2020 08:34:56 +0300 Subject: [PATCH 007/133] Rework test-runner for e2e on dataflow w/ reusing existing infrastructure (installed kafka, redis, etc) (#716) * fix e2e-dataflow test runner * e2e tests reusing existing cluster & preinstalled stateful services * explicit tag generation * [DATA-2018] last fixes * Revert "[DATA-2018] last fixes" This reverts commit 6f2af6d25fcd843f57b46e80b33cd7e3564ecf75. * [DATA-2018] last fixes --- .prow/config.yaml | 30 +- .../scripts/test-end-to-end-batch-dataflow.sh | 305 ++++++++++-------- .../values-end-to-end-batch-dataflow.yaml | 12 +- 3 files changed, 190 insertions(+), 157 deletions(-) diff --git a/.prow/config.yaml b/.prow/config.yaml index 9011ef801b4..f970ded0814 100644 --- a/.prow/config.yaml +++ b/.prow/config.yaml @@ -254,8 +254,11 @@ presubmits: - name: service-account-df secret: secretName: feast-e2e-service-account + - name: docker-socket + hostPath: + path: /var/run/docker.sock containers: - - image: maven:3.6-jdk-11 + - image: google/cloud-sdk:273.0.0 command: ["infra/scripts/test-end-to-end-batch-dataflow.sh"] resources: requests: @@ -264,30 +267,13 @@ presubmits: volumeMounts: - name: service-account-df mountPath: "/etc/service-account-df" + - name: docker-socket + mountPath: /var/run/docker.sock + securityContext: + privileged: true skip_branches: - ^v0\.(3|4)-branch$ - - name: test-end-to-end-batch-dataflow-java-8 - decorate: true - always_run: false - spec: - volumes: - - name: service-account-df - secret: - secretName: feast-e2e-service-account - containers: - - image: maven:3.6-jdk-8 - command: ["infra/scripts/test-end-to-end-batch-dataflow.sh"] - resources: - requests: - cpu: "6" - memory: "6144Mi" - volumeMounts: - - name: service-account-df - mountPath: "/etc/service-account-df" - branches: - - ^v0\.(3|4)-branch$ - - name: publish-docker-images decorate: true always_run: true diff --git a/infra/scripts/test-end-to-end-batch-dataflow.sh b/infra/scripts/test-end-to-end-batch-dataflow.sh index 9b3c8b5a83b..e8fb3b95653 100755 --- a/infra/scripts/test-end-to-end-batch-dataflow.sh +++ b/infra/scripts/test-end-to-end-batch-dataflow.sh @@ -4,178 +4,249 @@ echo "Preparing environment variables..." set -e set -o pipefail -test -z ${GOOGLE_APPLICATION_CREDENTIALS} && GOOGLE_APPLICATION_CREDENTIALS="/etc/service-account/service-account-df.json" +test -z ${GOOGLE_APPLICATION_CREDENTIALS} && GOOGLE_APPLICATION_CREDENTIALS="/etc/service-account-df/service-account-df.json" test -z ${GCLOUD_PROJECT} && GCLOUD_PROJECT="kf-feast" test -z ${GCLOUD_REGION} && GCLOUD_REGION="us-central1" test -z ${GCLOUD_NETWORK} && GCLOUD_NETWORK="default" test -z ${GCLOUD_SUBNET} && GCLOUD_SUBNET="default" test -z ${TEMP_BUCKET} && TEMP_BUCKET="feast-templocation-kf-feast" test -z ${K8_CLUSTER_NAME} && K8_CLUSTER_NAME="feast-e2e-dataflow" -test -z ${HELM_RELEASE_NAME} && HELM_RELEASE_NAME="feast-e2e-release" +test -z ${HELM_RELEASE_NAME} && HELM_RELEASE_NAME="pr-$PULL_NUMBER" +test -z ${HELM_COMMON_NAME} && HELM_COMMON_NAME="deps" +test -z ${DATASET_NAME} && DATASET_NAME=feast_e2e_$(date +%s) + +feast_kafka_1_ip_name="feast-kafka-1" +feast_kafka_2_ip_name="feast-kafka-2" +feast_kafka_3_ip_name="feast-kafka-3" +feast_redis_ip_name="feast-redis" +feast_statsd_ip_name="feast-statsd" echo " This script will run end-to-end tests for Feast Core and Batch Serving using Dataflow Runner. -1. Install gcloud SDK and required packages. -2. Create temporary BQ table for Feast Serving. -3. Generate valid names for IP addresses and k8s cluster, then store as environment variables. -4. Create GKE nodepool for Feast e2e test with DataflowRunner. -5. Setup Feast Core, Feast Serving and dependencies using Helm. - - Redis as the job store for Feast Batch Serving. - - Postgres for persisting Feast metadata. - - Kafka and Zookeeper as the Source in Feast. -6. Install Python 3.7.4, Feast Python SDK and run end-to-end tests from +1. Setup K8s cluster (optional, if it was not created before) +2. Reuse existing IP addresses or generate new ones for stateful services +3. Install stateful services (kafka, redis, postgres, etc) (optional) +4. Build core & serving docker images (optional) +5. Create temporary BQ table for Feast Serving. +6. Rollout target images to cluster via helm in dedicated namespace (pr-{number}) +7. Install Python 3.7.4, Feast Python SDK and run end-to-end tests from tests/e2e via pytest. -7. Tear down infrastructure, including Dataflow jobs. +8. Tear down feast services, keep stateful services. " ORIGINAL_DIR=$(pwd) echo $ORIGINAL_DIR -echo " -============================================================ -Installing gcloud SDK and required packages -============================================================ -" -apt-get -qq update -apt-get -y install wget netcat kafkacat build-essential gettext-base curl kubectl -curl -fsSL -o get_helm.sh https://raw.githubusercontent.com/helm/helm/master/scripts/get-helm-3 -chmod 700 $ORIGINAL_DIR/get_helm.sh -$ORIGINAL_DIR/get_helm.sh - -if [[ ! $(command -v gsutil) ]]; then - CURRENT_DIR=$(dirname "$BASH_SOURCE") - . "${CURRENT_DIR}"/install-google-cloud-sdk.sh -fi +echo "Environment:" +printenv export GOOGLE_APPLICATION_CREDENTIALS gcloud auth activate-service-account --key-file ${GOOGLE_APPLICATION_CREDENTIALS} +gcloud -q auth configure-docker gcloud config set project ${GCLOUD_PROJECT} gcloud config set compute/region ${GCLOUD_REGION} gcloud config list -echo " -============================================================ -Creating temp BQ table for Feast Serving -============================================================ -" -DATASET_NAME=feast_e2e_$(date +%s) +apt-get -qq update +apt-get -y install wget build-essential gettext-base curl -bq --location=US --project_id=${GCLOUD_PROJECT} mk \ - --dataset \ - --default_table_expiration 86400 \ - ${GCLOUD_PROJECT}:${DATASET_NAME} +curl -fsSL -o get_helm.sh https://raw.githubusercontent.com/helm/helm/master/scripts/get-helm-3 +chmod 700 $ORIGINAL_DIR/get_helm.sh +$ORIGINAL_DIR/get_helm.sh -echo " -============================================================ -Check and generate valid k8s cluster name -============================================================ -" -count=0 -for cluster_name in $(gcloud container clusters list --format "list(name)") -do - if [[ $cluster_name == "feast-e2e-dataflow"* ]]; then - count=$((count + 1)) - fi -done -temp="$K8_CLUSTER_NAME-$count" -export K8_CLUSTER_NAME=$temp -echo "Cluster name is $K8_CLUSTER_NAME" -echo " +function getPublicAddresses() { + existing_addresses=$(gcloud compute addresses list --filter="region:($GCLOUD_REGION) name:kafka" --format "list(name)") + if [[ -z "$existing_addresses" ]]; then + echo " ============================================================ Reserving IP addresses for Feast dependencies ============================================================ " -feast_kafka_1_ip_name="feast-kafka-$((count*3 + 1))" -feast_kafka_2_ip_name="feast-kafka-$((count*3 + 2))" -feast_kafka_3_ip_name="feast-kafka-$((count*3 + 3))" -feast_redis_ip_name="feast-redis-$((count + 1))" -feast_statsd_ip_name="feast-statsd-$((count + 1))" -gcloud compute addresses create \ - $feast_kafka_1_ip_name $feast_kafka_2_ip_name $feast_kafka_3_ip_name $feast_redis_ip_name $feast_statsd_ip_name \ - --region ${GCLOUD_REGION} --subnet ${GCLOUD_SUBNET} - -ip_count=0 -for ip_addr_name in $feast_kafka_1_ip_name $feast_kafka_2_ip_name $feast_kafka_3_ip_name $feast_redis_ip_name $feast_statsd_ip_name -do - if [[ "$ip_count" == 0 ]]; then - export feast_kafka_1_ip=$(gcloud compute addresses describe ${ip_addr_name} --region=${GCLOUD_REGION} --format "value(address)") - elif [[ "$ip_count" == 1 ]]; then - export feast_kafka_2_ip=$(gcloud compute addresses describe ${ip_addr_name} --region=${GCLOUD_REGION} --format "value(address)") - elif [[ "$ip_count" == 2 ]]; then - export feast_kafka_3_ip=$(gcloud compute addresses describe ${ip_addr_name} --region=${GCLOUD_REGION} --format "value(address)") - elif [[ "$ip_count" == 3 ]]; then - export feast_redis_ip=$(gcloud compute addresses describe ${ip_addr_name} --region=${GCLOUD_REGION} --format "value(address)") - elif [[ "$ip_count" == 4 ]]; then - export feast_statsd_ip=$(gcloud compute addresses describe ${ip_addr_name} --region=${GCLOUD_REGION} --format "value(address)") + + gcloud compute addresses create \ + $feast_kafka_1_ip_name $feast_kafka_2_ip_name $feast_kafka_3_ip_name $feast_redis_ip_name $feast_statsd_ip_name \ + --region ${GCLOUD_REGION} --subnet ${GCLOUD_SUBNET} fi - ip_count=$((ip_count + 1)) - export "$(echo $ip_addr_name | tr '-' '_')=$(gcloud compute addresses describe ${ip_addr_name} --region=${GCLOUD_REGION} --format "value(address)")" -done -echo " + + export feast_kafka_1_ip=$(gcloud compute addresses describe $feast_kafka_1_ip_name --region=${GCLOUD_REGION} --format "value(address)") + export feast_kafka_2_ip=$(gcloud compute addresses describe $feast_kafka_2_ip_name --region=${GCLOUD_REGION} --format "value(address)") + export feast_kafka_3_ip=$(gcloud compute addresses describe $feast_kafka_3_ip_name --region=${GCLOUD_REGION} --format "value(address)") + export feast_redis_ip=$(gcloud compute addresses describe $feast_redis_ip_name --region=${GCLOUD_REGION} --format "value(address)") + export feast_statsd_ip=$(gcloud compute addresses describe $feast_statsd_ip_name --region=${GCLOUD_REGION} --format "value(address)") +} + +function createKubeCluster() { + echo " ============================================================ Creating GKE nodepool for Feast e2e test with DataflowRunner ============================================================ " -gcloud container clusters create ${K8_CLUSTER_NAME} --region ${GCLOUD_REGION} \ - --enable-cloud-logging \ - --enable-cloud-monitoring \ - --network ${GCLOUD_NETWORK} \ - --subnetwork ${GCLOUD_SUBNET} \ - --machine-type n1-standard-2 -sleep 120 - -echo " + gcloud container clusters create ${K8_CLUSTER_NAME} --region ${GCLOUD_REGION} \ + --enable-cloud-logging \ + --enable-cloud-monitoring \ + --network ${GCLOUD_NETWORK} \ + --subnetwork ${GCLOUD_SUBNET} \ + --scopes https://www.googleapis.com/auth/devstorage.read_only,https://www.googleapis.com/auth/logging.write,\ +https://www.googleapis.com/auth/monitoring,https://www.googleapis.com/auth/service.management.readonly,\ +https://www.googleapis.com/auth/servicecontrol,https://www.googleapis.com/auth/trace.append,\ +https://www.googleapis.com/auth/bigquery \ + --machine-type n1-standard-2 + + echo " ============================================================ Create feast-postgres-database Secret in GKE nodepool ============================================================ " -kubectl create secret generic feast-postgresql --from-literal=postgresql-password=password + kubectl create secret generic feast-postgresql --from-literal=postgresql-password=password -echo " + echo " ============================================================ Create feast-gcp-service-account Secret in GKE nodepool ============================================================ " -cd $ORIGINAL_DIR/infra/scripts -kubectl create secret generic feast-gcp-service-account --from-file=${GOOGLE_APPLICATION_CREDENTIALS} + cd $ORIGINAL_DIR/infra/scripts + kubectl create secret generic feast-gcp-service-account --from-file=credentials.json=${GOOGLE_APPLICATION_CREDENTIALS} +} + +function installDependencies() { + echo " +============================================================ +Helm install common parts (kafka, redis, etc) +============================================================ +" + cd $ORIGINAL_DIR/infra/charts/feast + + helm install --wait --debug --values="values-end-to-end-batch-dataflow-updated.yaml" \ + --set "feast-core.enabled=false" \ + --set "feast-online-serving.enabled=false" \ + --set "feast-batch-serving.enabled=false" \ + "$HELM_COMMON_NAME" . + +} + +function buildAndPushImage() +{ + echo docker build -t $1:$2 --build-arg REVISION=$2 -f $3 $ORIGINAL_DIR + docker build -t $1:$2 --build-arg REVISION=$2 -f $3 $ORIGINAL_DIR + docker push $1:$2 +} + +function buildTarget() { + buildAndPushImage "gcr.io/kf-feast/feast-core" "$PULL_NUMBER" "$ORIGINAL_DIR/infra/docker/core/Dockerfile" + buildAndPushImage "gcr.io/kf-feast/feast-serving" "$PULL_NUMBER" "$ORIGINAL_DIR/infra/docker/serving/Dockerfile" +} + +function installTarget() { + echo " +============================================================ +Helm install feast +============================================================ +" + cd $ORIGINAL_DIR/infra/charts/feast + + helm install --wait --timeout 300s --debug --values="values-end-to-end-batch-dataflow-updated.yaml" \ + --set "postgresql.enabled=false" \ + --set "kafka.enabled=false" \ + --set "redis.enabled=false" \ + --set "prometheus-statsd-exporter.enabled=false" \ + --set "prometheus.enabled=false" \ + "$HELM_RELEASE_NAME" . + +} + +function clean() { + echo " + ============================================================ + Cleaning up + ============================================================ + " + cd $ORIGINAL_DIR/tests/e2e + + # Remove BQ Dataset + bq rm -r -f ${GCLOUD_PROJECT}:${DATASET_NAME} + + # Uninstall helm release before clearing PVCs + helm uninstall ${HELM_RELEASE_NAME} + + # Stop Dataflow jobs from retrieved Dataflow job ids in ingesting_jobs.txt + if [ -f ingesting_jobs.txt ]; then + while read line + do + echo $line + gcloud dataflow jobs cancel $line --region=${GCLOUD_REGION} + done < ingesting_jobs.txt + fi +} + +# 1. +existing_cluster=$(gcloud container clusters list --format "list(name)" --filter "name:$K8_CLUSTER_NAME") +if [[ -z $existing_cluster ]]; then + createKubeCluster "$@" +else + gcloud container clusters get-credentials $K8_CLUSTER_NAME --region $GCLOUD_REGION --project $GCLOUD_PROJECT +fi + +# 2. +getPublicAddresses "$@" echo " ============================================================ Export required environment variables ============================================================ " + export TEMP_BUCKET=$TEMP_BUCKET export DATASET_NAME=$DATASET_NAME export GCLOUD_PROJECT=$GCLOUD_PROJECT export GCLOUD_NETWORK=$GCLOUD_NETWORK export GCLOUD_SUBNET=$GCLOUD_SUBNET export GCLOUD_REGION=$GCLOUD_REGION +export HELM_COMMON_NAME=$HELM_COMMON_NAME +export IMAGE_TAG=${PULL_PULL_SHA:1} +envsubst $'$TEMP_BUCKET $DATASET_NAME $GCLOUD_PROJECT $GCLOUD_NETWORK \ + $GCLOUD_SUBNET $GCLOUD_REGION $IMAGE_TAG $HELM_COMMON_NAME $feast_kafka_1_ip + $feast_kafka_2_ip $feast_kafka_3_ip $feast_redis_ip $feast_statsd_ip' < $ORIGINAL_DIR/infra/scripts/test-templates/values-end-to-end-batch-dataflow.yaml > $ORIGINAL_DIR/infra/charts/feast/values-end-to-end-batch-dataflow-updated.yaml + + +# 3. +existing_deps=$(helm list --filter deps -q) +if [[ -z $existing_deps ]]; then + installDependencies "$@" +fi + +# 4. +# buildTarget "$@" + +# 5. echo " ============================================================ -Helm install Feast and its dependencies +Creating temp BQ table for Feast Serving ============================================================ " -cd $ORIGINAL_DIR/infra/scripts/test-templates -envsubst $'$TEMP_BUCKET $DATASET_NAME $GCLOUD_PROJECT $GCLOUD_NETWORK \ - $GCLOUD_SUBNET $GCLOUD_REGION $feast_kafka_1_ip - $feast_kafka_2_ip $feast_kafka_3_ip $feast_redis_ip $feast_statsd_ip' < values-end-to-end-batch-dataflow.yaml > $ORIGINAL_DIR/infra/charts/feast/values-end-to-end-batch-dataflow-updated.yaml -cd $ORIGINAL_DIR/infra/charts -helm template feast +bq --location=US --project_id=${GCLOUD_PROJECT} mk \ + --dataset \ + --default_table_expiration 86400 \ + ${GCLOUD_PROJECT}:${DATASET_NAME} + + +# 6. -cd $ORIGINAL_DIR/infra/charts/feast -helm install --wait --timeout 600s --values="values-end-to-end-batch-dataflow-updated.yaml" ${HELM_RELEASE_NAME} . +set +e +installTarget "$@" +# 7. echo " ============================================================ Installing Python 3.7 with Miniconda and Feast SDK ============================================================ " +cd $ORIGINAL_DIR # Install Python 3.7 with Miniconda wget -q https://repo.continuum.io/miniconda/Miniconda3-4.7.12-Linux-x86_64.sh \ -O /tmp/miniconda.sh @@ -203,7 +274,7 @@ core_ip=$(kubectl get -o jsonpath="{.spec.clusterIP}" service ${HELM_RELEASE_NAM serving_ip=$(kubectl get -o jsonpath="{.spec.clusterIP}" service ${HELM_RELEASE_NAME}-feast-batch-serving) set +e -pytest bq-batch-retrieval.py -m dataflow_runner --core_url "$core_ip:6565" --serving_url "$serving_ip:6566" --gcs_path "gs://${TEMP_BUCKET}/" --junitxml=${LOGS_ARTIFACT_PATH}/python-sdk-test-report.xml +pytest -v bq-batch-retrieval.py -m dataflow_runner --core_url "$core_ip:6565" --serving_url "$serving_ip:6566" --gcs_path "gs://${TEMP_BUCKET}/" --junitxml=${LOGS_ARTIFACT_PATH}/python-sdk-test-report.xml TEST_EXIT_CODE=$? if [[ ${TEST_EXIT_CODE} != 0 ]]; then @@ -215,33 +286,5 @@ if [[ ${TEST_EXIT_CODE} != 0 ]]; then pip list fi -cd ${ORIGINAL_DIR} - -echo " -============================================================ -Cleaning up -============================================================ -" -cd $ORIGINAL_DIR/tests/e2e - -# Remove BQ Dataset -bq rm -r -f ${GCLOUD_PROJECT}:${DATASET_NAME} - -# Uninstall helm release before clearing PVCs -helm uninstall ${HELM_RELEASE_NAME} -kubectl delete pvc --all - -# Release IP addresses -yes | gcloud compute addresses delete $feast_kafka_1_ip_name $feast_kafka_2_ip_name $feast_kafka_3_ip_name $feast_redis_ip_name $feast_statsd_ip_name --region=${GCLOUD_REGION} - -# Tear down GKE infrastructure -gcloud container clusters delete --region=${GCLOUD_REGION} ${K8_CLUSTER_NAME} - -# Stop Dataflow jobs from retrieved Dataflow job ids in ingesting_jobs.txt -while read line -do - echo $line - gcloud dataflow jobs cancel $line --region=${GCLOUD_REGION} -done < ingesting_jobs.txt - +clean "$@" exit ${TEST_EXIT_CODE} diff --git a/infra/scripts/test-templates/values-end-to-end-batch-dataflow.yaml b/infra/scripts/test-templates/values-end-to-end-batch-dataflow.yaml index aff9b0d6b71..44a377db5fa 100644 --- a/infra/scripts/test-templates/values-end-to-end-batch-dataflow.yaml +++ b/infra/scripts/test-templates/values-end-to-end-batch-dataflow.yaml @@ -6,8 +6,11 @@ feast-core: postgresql: existingSecret: feast-postgresql image: - tag: $PULL_PULL_SHA:1 + tag: $IMAGE_TAG application-override.yaml: + spring: + datasource: + url: jdbc:postgresql://$HELM_COMMON_NAME-postgresql:5432/postgres feast: stream: options: @@ -39,7 +42,7 @@ feast-online-serving: # feast-online-serving.enabled -- Flag to install Feast Online Serving enabled: true image: - tag: $PULL_PULL_SHA:1 + tag: $IMAGE_TAG application-override.yaml: feast: active_store: online @@ -60,10 +63,9 @@ feast-batch-serving: # feast-batch-serving.enabled -- Flag to install Feast Batch Serving enabled: true image: - tag: $PULL_PULL_SHA:1 + tag: $IMAGE_TAG gcpServiceAccount: enabled: true - application-override.yaml: feast: active_store: historical @@ -82,6 +84,8 @@ feast-batch-serving: - name: "*" project: "*" version: "*" + job_store: + redis_host: $HELM_COMMON_NAME-redis-master postgresql: # postgresql.enabled -- Flag to install Postgresql From 615423a55da62fc850057f175c9adc9926338cda Mon Sep 17 00:00:00 2001 From: Zhu Zhan Yan Date: Fri, 22 May 2020 15:26:56 +0800 Subject: [PATCH 008/133] Fix Docker Compose Test timeouts (#728) * Fix poll interface of docker compose interval causing timeout errors in test The poll interval of Feast core was set at 10ms. As a result, feast core hogged CPU time from the rest of test code/serving, resulting in nodeterministic timeouts. This changes the poll interval of Feast Core in the Docker Compose test to 30s * Remove old v0.4 unused config from docker-compose setup * Update batch-serving.yml serving config to v0.5 format * Split out core specific config in docker-compose's .env.sample into core.yml * Add github badge for docker-compose tests status in README --- README.md | 2 ++ infra/docker-compose/.env.sample | 16 ++++------ infra/docker-compose/core/core.yml | 14 +++++++++ infra/docker-compose/docker-compose.batch.yml | 3 +- .../docker-compose/docker-compose.online.yml | 3 +- infra/docker-compose/docker-compose.yml | 4 +-- .../docker-compose/serving/batch-serving.yml | 30 +++++++++++-------- infra/docker-compose/serving/bq-store.yml | 9 ------ infra/docker-compose/serving/redis-store.yml | 9 ------ 9 files changed, 43 insertions(+), 47 deletions(-) create mode 100644 infra/docker-compose/core/core.yml delete mode 100644 infra/docker-compose/serving/bq-store.yml delete mode 100644 infra/docker-compose/serving/redis-store.yml diff --git a/README.md b/README.md index 3da4109f9b8..12f8a5c9221 100644 --- a/README.md +++ b/README.md @@ -1,10 +1,12 @@ # Feast - Feature Store for Machine Learning [![Unit Tests](https://github.com/feast-dev/feast/workflows/unit%20tests/badge.svg?branch=master)](https://github.com/feast-dev/feast/actions?query=workflow%3A%22unit+tests%22+branch%3Amaster) +![Docker Compose Tests](https://github.com/feast-dev/feast/workflows/docker%20compose%20tests/badge.svg?branch=master) [![Code Standards](https://github.com/feast-dev/feast/workflows/code%20standards/badge.svg?branch=master)](https://github.com/feast-dev/feast/actions?query=workflow%3A%22code+standards%22+branch%3Amaster) [![Docs latest](https://img.shields.io/badge/Docs-latest-blue.svg)](https://docs.feast.dev/) [![GitHub Release](https://img.shields.io/github/release/feast-dev/feast.svg?style=flat)](https://github.com/feast-dev/feast/releases) + ## Overview Feast (Feature Store) is a tool for managing and serving machine learning features. Feast is the bridge between models and data. diff --git a/infra/docker-compose/.env.sample b/infra/docker-compose/.env.sample index f3de7330630..a267f5e8577 100644 --- a/infra/docker-compose/.env.sample +++ b/infra/docker-compose/.env.sample @@ -5,21 +5,17 @@ FEAST_REPOSITORY_VERSION=v0.5-branch # Feast Core FEAST_CORE_IMAGE=gcr.io/kf-feast/feast-core -FEAST_CORE_CONFIG=direct-runner.yml +FEAST_CORE_CONFIG=core.yml FEAST_CORE_GCP_SERVICE_ACCOUNT_KEY=placeholder.json -FEAST_STREAM_OPTIONS_BOOTSTRAPSERVERS=kafka:9092 -FEAST_JOBS_POLLING_INTERVAL_MILLISECONDS=10 -# Feast Serving Batch (BigQuery) +# Feast Serving +FEAST_SERVING_IMAGE=gcr.io/kf-feast/feast-serving +# Feast Serving - Batch (BigQuery) FEAST_BATCH_SERVING_CONFIG=batch-serving.yml -FEAST_BATCH_STORE_CONFIG=bq-store.yml FEAST_BATCH_SERVING_GCP_SERVICE_ACCOUNT_KEY=placeholder.json FEAST_BATCH_JOB_STAGING_LOCATION=gs://your-gcs-bucket/staging - -# Feast Serving Online (Redis) -FEAST_SERVING_IMAGE=gcr.io/kf-feast/feast-serving +# Feast Serving - Online (Redis) FEAST_ONLINE_SERVING_CONFIG=online-serving.yml -FEAST_ONLINE_STORE_CONFIG=redis-store.yml # Jupyter -FEAST_JUPYTER_GCP_SERVICE_ACCOUNT_KEY=placeholder.json \ No newline at end of file +FEAST_JUPYTER_GCP_SERVICE_ACCOUNT_KEY=placeholder.json diff --git a/infra/docker-compose/core/core.yml b/infra/docker-compose/core/core.yml new file mode 100644 index 00000000000..99d8d6c7ddb --- /dev/null +++ b/infra/docker-compose/core/core.yml @@ -0,0 +1,14 @@ +feast: + jobs: + polling_interval_milliseconds: 30000 + job_update_timeout_seconds: 240 + active_runner: direct + runners: + - name: direct + type: DirectRunner + options: {} + stream: + type: kafka + options: + topic: feast-features + bootstrapServers: kafka:9092 diff --git a/infra/docker-compose/docker-compose.batch.yml b/infra/docker-compose/docker-compose.batch.yml index 247dd0b6719..71f61d634a5 100644 --- a/infra/docker-compose/docker-compose.batch.yml +++ b/infra/docker-compose/docker-compose.batch.yml @@ -5,7 +5,6 @@ services: image: ${FEAST_SERVING_IMAGE}:${FEAST_VERSION} volumes: - ./serving/${FEAST_BATCH_SERVING_CONFIG}:/etc/feast/application.yml - - ./serving/${FEAST_BATCH_STORE_CONFIG}:/etc/feast/store.yml - ./gcp-service-accounts/${FEAST_BATCH_SERVING_GCP_SERVICE_ACCOUNT_KEY}:/etc/gcloud/service-accounts/key.json depends_on: - redis @@ -26,4 +25,4 @@ services: redis: image: redis:5-alpine ports: - - "6379:6379" \ No newline at end of file + - "6379:6379" diff --git a/infra/docker-compose/docker-compose.online.yml b/infra/docker-compose/docker-compose.online.yml index ed96f0e0963..b01d0882fb4 100644 --- a/infra/docker-compose/docker-compose.online.yml +++ b/infra/docker-compose/docker-compose.online.yml @@ -5,7 +5,6 @@ services: image: ${FEAST_SERVING_IMAGE}:${FEAST_VERSION} volumes: - ./serving/${FEAST_ONLINE_SERVING_CONFIG}:/etc/feast/application.yml - - ./serving/${FEAST_ONLINE_STORE_CONFIG}:/etc/feast/store.yml depends_on: - redis ports: @@ -20,4 +19,4 @@ services: redis: image: redis:5-alpine ports: - - "6379:6379" \ No newline at end of file + - "6379:6379" diff --git a/infra/docker-compose/docker-compose.yml b/infra/docker-compose/docker-compose.yml index d327b8e3c80..ab4cf3858a8 100644 --- a/infra/docker-compose/docker-compose.yml +++ b/infra/docker-compose/docker-compose.yml @@ -9,8 +9,6 @@ services: environment: DB_HOST: db GOOGLE_APPLICATION_CREDENTIALS: /etc/gcloud/service-accounts/key.json - FEAST_STREAM_OPTIONS_BOOTSTRAPSERVERS: ${FEAST_STREAM_OPTIONS_BOOTSTRAPSERVERS} - FEAST_JOBS_POLLING_INTERVAL_MILLISECONDS: ${FEAST_JOBS_POLLING_INTERVAL_MILLISECONDS} restart: on-failure depends_on: - db @@ -66,4 +64,4 @@ services: environment: POSTGRES_PASSWORD: password ports: - - "5432:5432" \ No newline at end of file + - "5432:5432" diff --git a/infra/docker-compose/serving/batch-serving.yml b/infra/docker-compose/serving/batch-serving.yml index 0b1c3fbf96c..375380bcee3 100644 --- a/infra/docker-compose/serving/batch-serving.yml +++ b/infra/docker-compose/serving/batch-serving.yml @@ -1,14 +1,20 @@ feast: core-host: core - core-grpc-port: 6565 - store: - config-path: /etc/feast/store.yml - - jobs: - store-type: REDIS - store-options: - host: redis - port: 6379 - -grpc: - port: 6567 + active-store: batch + stores: + - name: historical + type: BIGQUERY + # Changes required for batch serving to work + # Please see https://api.docs.feast.dev/grpc/feast.core.pb.html#Store for configuration options + config: + project_id: my_project + dataset_id: my_dataset + staging_location: gs://mybucket/myprefix + initial_retry_delay_seconds: 1 + total_timeout_seconds: 21600 + subscriptions: + - name: "*" + project: "*" + job_store: + redis_host: redis + redis_port: 6379 diff --git a/infra/docker-compose/serving/bq-store.yml b/infra/docker-compose/serving/bq-store.yml deleted file mode 100644 index cdebee3497f..00000000000 --- a/infra/docker-compose/serving/bq-store.yml +++ /dev/null @@ -1,9 +0,0 @@ -name: warehouse -type: BIGQUERY -bigquery_config: - project_id: - dataset_id: -subscriptions: -- name: "*" - project: "*" - version: "*" \ No newline at end of file diff --git a/infra/docker-compose/serving/redis-store.yml b/infra/docker-compose/serving/redis-store.yml deleted file mode 100644 index 16870be94d7..00000000000 --- a/infra/docker-compose/serving/redis-store.yml +++ /dev/null @@ -1,9 +0,0 @@ -name: serving -type: REDIS -redis_config: - host: redis - port: 6379 -subscriptions: -- name: "*" - project: "*" - version: "*" From ec7d413631ad25296872ae0ebc0bb93bc127096e Mon Sep 17 00:00:00 2001 From: Ches Martin Date: Fri, 22 May 2020 21:49:55 +0700 Subject: [PATCH 009/133] Remove unused Hibernate dep from Serving (#721) --- core/pom.xml | 2 -- pom.xml | 6 ++++++ serving/pom.xml | 7 ------- 3 files changed, 6 insertions(+), 9 deletions(-) diff --git a/core/pom.xml b/core/pom.xml index 2f616d5ed41..23cccb6b193 100644 --- a/core/pom.xml +++ b/core/pom.xml @@ -149,11 +149,9 @@ v1b3-rev266-1.25.0 - org.hibernate hibernate-core - 5.3.6.Final diff --git a/pom.xml b/pom.xml index 018a101adb9..b1106136e46 100644 --- a/pom.xml +++ b/pom.xml @@ -54,6 +54,7 @@ 0.8.0 1.9.10 1.3 + 5.3.6.Final 2.3.0 2.28.2 @@ -249,6 +250,11 @@ kafka-clients ${kafka.version} + + org.hibernate + hibernate-core + ${hibernate.version} + net.bytebuddy byte-buddy diff --git a/serving/pom.xml b/serving/pom.xml index 3754777da07..2d0e06aa13c 100644 --- a/serving/pom.xml +++ b/serving/pom.xml @@ -259,13 +259,6 @@ test - - - org.hibernate - hibernate-core - 5.4.5.Final - - com.fasterxml.jackson.dataformat From 8538769a699b36019c9eab9a7f602ca17f350e26 Mon Sep 17 00:00:00 2001 From: Ches Martin Date: Fri, 22 May 2020 21:55:31 +0700 Subject: [PATCH 010/133] Restore Feast Java SDK and Ingestion compatibility with Java 8 runtimes (#722) * Make storage modules target Java 8, since Ingestion must With the addition of the Storage APIs and connector implementations, we did not specify that their build should target Java 8 bytecode. Because Ingestion depends on them, we regressed on keeping Ingestion compatible with Java 8 runtimes according to #518. This could be confirmed prior to this change with, for example: $ mvn -pl ingestion clean compile $ javap -v storage/connectors/redis/target/classes/feast/storage/connectors/redis/writer/RedisCustomIO.class | grep major major version: 55 I take this as proof that a blacklist approach with 11 as the default in our root POM is error prone, so this change flips it so that 8 is default and our leaf applications that are able use 11 opt into it in whitelist fashion. * Target Java 8 compatibility for Java SDK --- core/pom.xml | 8 +++++ datatypes/java/pom.xml | 9 ----- ingestion/pom.xml | 34 ++++++++++++++----- pom.xml | 3 +- .../java/com/gojek/feast/RequestUtilTest.java | 3 +- serving/pom.xml | 8 +++++ .../redis/writer/RedisCustomIO.java | 6 ++-- .../retriever/RedisOnlineRetrieverTest.java | 8 ++--- .../writer/RedisClusterCustomIO.java | 6 ++-- .../RedisClusterOnlineRetrieverTest.java | 10 +++--- 10 files changed, 63 insertions(+), 32 deletions(-) diff --git a/core/pom.xml b/core/pom.xml index 23cccb6b193..f28a1fa28ba 100644 --- a/core/pom.xml +++ b/core/pom.xml @@ -32,6 +32,14 @@ + + org.apache.maven.plugins + maven-compiler-plugin + + 11 + + + org.jacoco jacoco-maven-plugin diff --git a/datatypes/java/pom.xml b/datatypes/java/pom.xml index a127853258e..5810a6db96a 100644 --- a/datatypes/java/pom.xml +++ b/datatypes/java/pom.xml @@ -37,15 +37,6 @@ - - org.apache.maven.plugins - maven-compiler-plugin - - - 8 - - - org.apache.maven.plugins maven-dependency-plugin diff --git a/ingestion/pom.xml b/ingestion/pom.xml index a62d4202ee0..a004c6d9df6 100644 --- a/ingestion/pom.xml +++ b/ingestion/pom.xml @@ -31,15 +31,6 @@ - - org.apache.maven.plugins - maven-compiler-plugin - - - 8 - - - org.apache.maven.plugins maven-shade-plugin @@ -84,6 +75,31 @@ + + org.apache.maven.plugins + maven-enforcer-plugin + + + + enforce-bytecode-version + + enforce + + + + + 1.8 + + + + + + + org.jacoco jacoco-maven-plugin diff --git a/pom.xml b/pom.xml index b1106136e46..77db4d21373 100644 --- a/pom.xml +++ b/pom.xml @@ -417,7 +417,8 @@ org.apache.maven.plugins maven-compiler-plugin - 11 + + 8 diff --git a/sdk/java/src/test/java/com/gojek/feast/RequestUtilTest.java b/sdk/java/src/test/java/com/gojek/feast/RequestUtilTest.java index f10d82b8726..60643115558 100644 --- a/sdk/java/src/test/java/com/gojek/feast/RequestUtilTest.java +++ b/sdk/java/src/test/java/com/gojek/feast/RequestUtilTest.java @@ -19,6 +19,7 @@ import static org.junit.jupiter.api.Assertions.assertEquals; import static org.junit.jupiter.api.Assertions.assertThrows; +import com.google.common.collect.ImmutableList; import com.google.protobuf.TextFormat; import feast.proto.serving.ServingAPIProto.FeatureReference; import java.util.Arrays; @@ -82,7 +83,7 @@ void renderFeatureRef_ShouldReturnFeatureRefString( } private static Stream provideInvalidFeatureRefs() { - return Stream.of(Arguments.of(List.of("project/feature", ""))); + return Stream.of(Arguments.of(ImmutableList.of("project/feature", ""))); } @ParameterizedTest diff --git a/serving/pom.xml b/serving/pom.xml index 2d0e06aa13c..c7a30166f6b 100644 --- a/serving/pom.xml +++ b/serving/pom.xml @@ -40,6 +40,14 @@ + + org.apache.maven.plugins + maven-compiler-plugin + + 11 + + + org.jacoco jacoco-maven-plugin diff --git a/storage/connectors/redis/src/main/java/feast/storage/connectors/redis/writer/RedisCustomIO.java b/storage/connectors/redis/src/main/java/feast/storage/connectors/redis/writer/RedisCustomIO.java index 3fe9148eed5..eeac5b3d964 100644 --- a/storage/connectors/redis/src/main/java/feast/storage/connectors/redis/writer/RedisCustomIO.java +++ b/storage/connectors/redis/src/main/java/feast/storage/connectors/redis/writer/RedisCustomIO.java @@ -54,8 +54,10 @@ public class RedisCustomIO { private static final int DEFAULT_BATCH_SIZE = 1000; private static final int DEFAULT_TIMEOUT = 2000; - private static TupleTag successfulInsertsTag = new TupleTag<>("successfulInserts") {}; - private static TupleTag failedInsertsTupleTag = new TupleTag<>("failedInserts") {}; + private static TupleTag successfulInsertsTag = + new TupleTag("successfulInserts") {}; + private static TupleTag failedInsertsTupleTag = + new TupleTag("failedInserts") {}; private static final Logger log = LoggerFactory.getLogger(RedisCustomIO.class); diff --git a/storage/connectors/redis/src/test/java/feast/storage/connectors/redis/retriever/RedisOnlineRetrieverTest.java b/storage/connectors/redis/src/test/java/feast/storage/connectors/redis/retriever/RedisOnlineRetrieverTest.java index 4ab2ac9e130..adacacb941e 100644 --- a/storage/connectors/redis/src/test/java/feast/storage/connectors/redis/retriever/RedisOnlineRetrieverTest.java +++ b/storage/connectors/redis/src/test/java/feast/storage/connectors/redis/retriever/RedisOnlineRetrieverTest.java @@ -133,7 +133,7 @@ public void shouldReturnResponseWithValuesIfKeysPresent() { when(syncCommands.mget(redisKeyList)).thenReturn(featureRowBytes); List> expected = - List.of( + ImmutableList.of( Lists.newArrayList( FeatureRow.newBuilder() .setEventTimestamp(Timestamp.newBuilder().setSeconds(100)) @@ -153,7 +153,7 @@ public void shouldReturnResponseWithValuesIfKeysPresent() { .build())); List> actual = - redisOnlineRetriever.getOnlineFeatures(entityRows, List.of(featureSetRequest)); + redisOnlineRetriever.getOnlineFeatures(entityRows, ImmutableList.of(featureSetRequest)); assertThat(actual, equalTo(expected)); } @@ -201,7 +201,7 @@ public void shouldReturnResponseWithUnsetValuesIfKeysNotPresent() { when(syncCommands.mget(redisKeyList)).thenReturn(featureRowBytes); List> expected = - List.of( + ImmutableList.of( Lists.newArrayList( FeatureRow.newBuilder() .setEventTimestamp(Timestamp.newBuilder().setSeconds(100)) @@ -220,7 +220,7 @@ public void shouldReturnResponseWithUnsetValuesIfKeysNotPresent() { .build())); List> actual = - redisOnlineRetriever.getOnlineFeatures(entityRows, List.of(featureSetRequest)); + redisOnlineRetriever.getOnlineFeatures(entityRows, ImmutableList.of(featureSetRequest)); assertThat(actual, equalTo(expected)); } diff --git a/storage/connectors/rediscluster/src/main/java/feast/storage/connectors/rediscluster/writer/RedisClusterCustomIO.java b/storage/connectors/rediscluster/src/main/java/feast/storage/connectors/rediscluster/writer/RedisClusterCustomIO.java index 8984dc877cd..712012a2fb7 100644 --- a/storage/connectors/rediscluster/src/main/java/feast/storage/connectors/rediscluster/writer/RedisClusterCustomIO.java +++ b/storage/connectors/rediscluster/src/main/java/feast/storage/connectors/rediscluster/writer/RedisClusterCustomIO.java @@ -54,8 +54,10 @@ public class RedisClusterCustomIO { private static final int DEFAULT_BATCH_SIZE = 1000; private static final int DEFAULT_TIMEOUT = 2000; - private static TupleTag successfulInsertsTag = new TupleTag<>("successfulInserts") {}; - private static TupleTag failedInsertsTupleTag = new TupleTag<>("failedInserts") {}; + private static TupleTag successfulInsertsTag = + new TupleTag("successfulInserts") {}; + private static TupleTag failedInsertsTupleTag = + new TupleTag("failedInserts") {}; private static final Logger log = LoggerFactory.getLogger(RedisClusterCustomIO.class); diff --git a/storage/connectors/rediscluster/src/test/java/feast/storage/connectors/rediscluster/retriever/RedisClusterOnlineRetrieverTest.java b/storage/connectors/rediscluster/src/test/java/feast/storage/connectors/rediscluster/retriever/RedisClusterOnlineRetrieverTest.java index 16ab7962d60..deed53e1ae8 100644 --- a/storage/connectors/rediscluster/src/test/java/feast/storage/connectors/rediscluster/retriever/RedisClusterOnlineRetrieverTest.java +++ b/storage/connectors/rediscluster/src/test/java/feast/storage/connectors/rediscluster/retriever/RedisClusterOnlineRetrieverTest.java @@ -133,7 +133,7 @@ public void shouldReturnResponseWithValuesIfKeysPresent() { when(syncCommands.mget(redisKeyList)).thenReturn(featureRowBytes); List> expected = - List.of( + ImmutableList.of( Lists.newArrayList( FeatureRow.newBuilder() .setEventTimestamp(Timestamp.newBuilder().setSeconds(100)) @@ -153,7 +153,8 @@ public void shouldReturnResponseWithValuesIfKeysPresent() { .build())); List> actual = - redisClusterOnlineRetriever.getOnlineFeatures(entityRows, List.of(featureSetRequest)); + redisClusterOnlineRetriever.getOnlineFeatures( + entityRows, ImmutableList.of(featureSetRequest)); assertThat(actual, equalTo(expected)); } @@ -201,7 +202,7 @@ public void shouldReturnResponseWithUnsetValuesIfKeysNotPresent() { when(syncCommands.mget(redisKeyList)).thenReturn(featureRowBytes); List> expected = - List.of( + ImmutableList.of( Lists.newArrayList( FeatureRow.newBuilder() .setEventTimestamp(Timestamp.newBuilder().setSeconds(100)) @@ -220,7 +221,8 @@ public void shouldReturnResponseWithUnsetValuesIfKeysNotPresent() { .build())); List> actual = - redisClusterOnlineRetriever.getOnlineFeatures(entityRows, List.of(featureSetRequest)); + redisClusterOnlineRetriever.getOnlineFeatures( + entityRows, ImmutableList.of(featureSetRequest)); assertThat(actual, equalTo(expected)); } From 9293b63bc324120a483f3715259da54995acff11 Mon Sep 17 00:00:00 2001 From: Ches Martin Date: Sat, 23 May 2020 15:45:56 +0700 Subject: [PATCH 011/133] Update owners for Prow / GitHub (#736) - Remove inactive maintainers - Add myself to reviewer rotation --- .github/CODEOWNERS | 10 +++++----- OWNERS | 14 +++++++------- 2 files changed, 12 insertions(+), 12 deletions(-) diff --git a/.github/CODEOWNERS b/.github/CODEOWNERS index a30c747b135..c02551b0147 100644 --- a/.github/CODEOWNERS +++ b/.github/CODEOWNERS @@ -1,8 +1,8 @@ # Each line is a file pattern followed by one or more owners. # https://help.github.com/en/articles/about-code-owners -* @zhilingc @pradithya @woop @davidheryanto @khorshuheng -/core/ @zhilingc @pradithya -/ingestion/ @zhilingc @pradithya -/serving/ @zhilingc @pradithya -/cli/ @zhilingc @pradithya +* @zhilingc @woop @davidheryanto @khorshuheng @pyalex +/core/ @zhilingc +/ingestion/ @zhilingc +/serving/ @zhilingc +/cli/ @zhilingc diff --git a/OWNERS b/OWNERS index 408d4a0d51a..c1595879fa7 100644 --- a/OWNERS +++ b/OWNERS @@ -1,13 +1,13 @@ approvers: - - zhilingc - - pradithya - - woop - - thirteen37 - davidheryanto - khorshuheng -reviewers: - - zhilingc + - pyalex - woop - - thirteen37 + - zhilingc +reviewers: + - ches - davidheryanto - khorshuheng + - pyalex + - woop + - zhilingc From f81864a62aa80ed3163ab47c5c7dd338feee9096 Mon Sep 17 00:00:00 2001 From: Willem Pienaar Date: Sat, 23 May 2020 21:28:41 +0800 Subject: [PATCH 012/133] Add feast logo and fix docker compose badge --- README.md | 9 +++++++-- docs/.gitbook/assets/feast_logo.png | Bin 0 -> 81755 bytes 2 files changed, 7 insertions(+), 2 deletions(-) create mode 100644 docs/.gitbook/assets/feast_logo.png diff --git a/README.md b/README.md index 12f8a5c9221..533c123874d 100644 --- a/README.md +++ b/README.md @@ -1,7 +1,12 @@ -# Feast - Feature Store for Machine Learning +

+ + + +

+
[![Unit Tests](https://github.com/feast-dev/feast/workflows/unit%20tests/badge.svg?branch=master)](https://github.com/feast-dev/feast/actions?query=workflow%3A%22unit+tests%22+branch%3Amaster) -![Docker Compose Tests](https://github.com/feast-dev/feast/workflows/docker%20compose%20tests/badge.svg?branch=master) +[![Docker Compose Tests](https://github.com/feast-dev/feast/workflows/docker%20compose%20tests/badge.svg?branch=master)](https://github.com/feast-dev/feast/actions?query=workflow%3A%22docker+compose+tests%22+branch%3Amaster) [![Code Standards](https://github.com/feast-dev/feast/workflows/code%20standards/badge.svg?branch=master)](https://github.com/feast-dev/feast/actions?query=workflow%3A%22code+standards%22+branch%3Amaster) [![Docs latest](https://img.shields.io/badge/Docs-latest-blue.svg)](https://docs.feast.dev/) [![GitHub Release](https://img.shields.io/github/release/feast-dev/feast.svg?style=flat)](https://github.com/feast-dev/feast/releases) diff --git a/docs/.gitbook/assets/feast_logo.png b/docs/.gitbook/assets/feast_logo.png new file mode 100644 index 0000000000000000000000000000000000000000..cdd97770ed8cbc5b5bc67330ff43869f1de7193e GIT binary patch literal 81755 zcmaG}2|QJ6*WXekp+lt*MUe(W2xaP~feb|@nNuh;Wyo}-qFYiqLO4{0lnf~{pO7w@ z%2dW9Q)N1l%zW!P=yuQXe&_eQ_rCYG_p_fh{nvl3_3ZuLII4AsZ3WK?48z#e4y)>5 z*lHq%EnB~w3I5A15sne~2aCgDeP;|46GDG6xWy^EVAw`XO?BUKx5&QcJV)Nojmhmj z2A4cZ=OfBh%JAr~m!VSss2pXwQ?onVm%xDjIxo(AIHqTHeoIenD=$xAY&%D_m(huf z@#nF@qI$Q{*{SiKXa-E~L|n%E&AprLgKe(o8}M?$Z zqgZhBW_b_asu9Uji?2ETZ|Jp!XIVvDirxMzvENg`S7T`4nsSv`>4mR35q=_c%VHu( zE;lehm(o;NXVG>a!X(ncz+$3BZbf^jBQz!l+uYl6+v-}Aq{{eWiX|5>2P23l*Sg3v ziL~*38*%OEXFx*388Wd;1R9i?T!A z7QFACp+;X}&h#kHgq_zS<|6-t#L|9Ro5AN7tjAT*XMXR0sfv#Usj8IFSg&?cZLL)H z_vv*po^ozW=r&3zL*n}+)&5c{6)ydJD`i}?^z#3(tDYc;Kb4`=fs47P zlk;L5U4u4SkW`z|%)B;us{L}?$WG(|IfIJyQQ#(87>tp`MPEHH`Py07HB%potlAk( z?^t0}`zIhDw7Hk-m}c_J(I*GuMdVi&ox|d}M9^lBf)lRP%Afo&_#=})ILSXh$8j5- z4#;t6yKN^f7zpcc8K>m+Ol5#w-FvRmT}g6AW@VrEKz9D2<96?`F29@fdaXJDemyp; zZW}te?#w+_J$GD=7uor1&dAFA?R3jhRv;sIISj#Gjk8Lgxy`xY%7VT0m)?R)O-oRC zQyixB#|+#qjhf?!ddmmtF2xjZ-~?fbI!%+CBQ{&kFx{@~Ul-F{IlYSRQj)R|XiG}> zg2!Ga5pf%zS{JqY{cX10-plDOC65BYku(U6_W_8?x<$GqOGm0*U7eNbElcS|du39$ z?cMxrGpk2*5{edy(fxF5V#?Z}E#ahEm+(&E((ove4&8gOXj7H|y<*2az+EKv#bpdj z%DJ1hsWol^BzBcrbX8#xic1HXRb;#-Bw9c;M63R zt*?x3zbR$)_`&O<1{;08frIYp-N;pMpQ1*J@#h;Nl(4fhKU*x{(HRu)4#18VI&qOBbPqTkc}+;;Nws>lwJzkt&9Fyi zKf3Ek-yoFndPgthXgCFTmQTy;2%z-e#6$1$TA{GV+!&pv@pX~84z5}xYC7XJr`P#; zPFR!sDDVVEZ#AXYvLaub*2UyX@zXs$axM6gn?+nkFN7v*W_NLmus*N^)5COE%CUjJ zQ#1fbAl?Fbr8edK7`!a?v!yD~Ha1w2MGtU*yIh^D!x-0p_(c=Zp9LOXBdeVaY*i9_2MIbUkcO8cB*aSwORY~l30u*~K zbT;%XY_n06c0r6Rhdk4M>#LokzImJV2AG*)pV#vzJr~I&ZpV=HHvvMtj*FKT_LKbw zsC0_**n~bhh(Tbdsn~hEvtuggq{4`SCEaH6p<7_DK96gExxikh#$ntu!2Z;@E)L5MpW zKM#31q?==5NMH5zsf_#$A#TBy;GW4bqkFCc4B~d7=<$IybaIhuzyVF`so6oS)IY0j z$!(Gsf>PHKIy2xCz+NPqkU2IrwT5&oYXc*J!-7sGyw5>5PW)$4Dw4?7`#*lg+g4Li z?P?%JcPVKQoW*pvx@`{xUvd(=NQWfK`&GhpJH?woekOH>fL0w^#?n*eB4bf@sPm-T zCuJD)ANEqG>C(MR_oR*#ZEn8JQ1)D&?%Cu)P?LljFl4peYGON92B$X_v;pF2(!-N7 zz!NEUK83xIa1T;zh2X2JrGK_pSI+DjOzUW(ZB*yJi<}l_p`8aWa?)Mr-8VZ$5yc|$ z^S<}%ivDX&ppkagc68T{NkZ&tolNXkDAl~jy1j5mbI_a8lx|e}{O5;{1`i>XioV(z zoD@7|rWkhZ9ps$P&HuRw{R_n!6eK4amr=Ja%J3Asuzl#+^x5@`Kl>7dQ{+{hrml4{ z9_Er^*P6V}x_r%HcBZorX%HNl`~+bMxpVK4H`HD3UF14tWXEIM&@S(JrmiY4w41Gj za>JZnaJN11aK0#HNX>iKE}5=X8fY5_`nHJZ2@sy750JYcb7!TlC@&1%{OsW%e>oeS z@KyW4H}&)`>=gh?T#wfu+s!00xY%_^&~>_i%pJ}hRfU`q?m1M^0~6^T=^Wq_Wt+O~ zU~pfDv*lV?RQjMu<|5k+MHafb?N#0ezhG`Lj&z6UPTgCSaS7s?Bl?kO9tv1oVvU)GeDhBdg0<})WQH6mSvtE%vO^Skt0mfI1yyYs zCp$^7<^X|M#OvgcLZ=qsL(flt9%mQX<^!}%11h(3Z$B-;YJwi_A+gBAcf-SHbmkb- zblo5pJQRj3zc-yODUqa+Ga1EYcsV$Qoc)p4LyGPDb@%hb!QB?iXobR`2lON>6B=7Z zKELa!G-R@9uUQw>{D7DCLJYAX{OpN2x^B4CqFo7*aE?V>^cy6EFfcsI$raOUCAfl6 ziSM;rj&4njXv${Nt$E%w!)`j{GWkzmwX3x*JvMlPOyy8o?E(aonk+*0Wb?|Dp;uJ- zO|&L~3L@89%+?N2ORz=ESq?Oiv=ZV2%2lOLyx?~$wOYgI1@G2$<23!EZrcDDzc%A~ zX;a|fB(x!((`QYrT@8O@8{{m8Y}wt{^~dg^l<7x*9ex&cX?g}y7VxVmTeG$Kfp1C| z0Xl3JJz5Z@R!_Sr{ZDIz?PZxn>^0~iX=)8T8WC?1eG&O|k@CWKn1;C1^Yn80ao2(nF?A*Yd+1QHuy;0y*6QgrQbkO>a8;sk}Rk zhtAq#8z3x19wCOVq~vNL6?RSA{G!t$>5A+MX_XH=h)U4619TuL&R@hS#}XiZZXZWw zOjIWa@0%S8PPf9j6zIG>%51Gt|NqFT*lk}>rqtF~d73ZOoZ9mxSVz^8{TGEd2My2>5jYR3wVF{9 zfQN_E2cz6Uc(`_KZHCqh^%oAhVYx<#YVy(p2KG(^tRn!?hz!|%UxGE0iY*58d;)t1 z4MDQ(9PhZs`#;$crgaU(5vTGF=rFlRum(Z=rY~YhT}aXtR(0EFl6$foWYH!oW-J$l z47~1UxRh%?Pk8|684yZ`e&`}NBzF!(S3XLm9fLCynRu2gARw*>=`bCOhvJ^R7fcB~ z5CQh+C0s9?3WoQ$wXEZ&i^(ufaF1CXRB5TbkaJtE@!{R{UZ~3Iv*|7+)dC>gf}H}- z7HftCaO7Uyq6q+tS9jwK%bUym4<>h-3RCfozRtsRfykAgq$borWmX@S|A!&rv!D@R zkvIx=xnmcxP*6U%h*6$7+s5koB}`@EXkBuuQrpqt-P>f zUDTwFT^O`W5kq?Gn*TNPN86R0MVjDX${LF7zHO$bT{{kLw!CFBrceK7MZH zQKdy%@?lU%kYlIM+*u*#m8n)==pA1HABx5w? zK<%~2ecQ-xp6eMMId=PggFpw5sAKgiZCRIYBtSo*2$O(4TFv3ck!6YahLJf+wwLrW zr196RkI5GO+(K?IzK)D&4kE2?Fb(L!I}nWJ4=>A2nj_pLEoP0>5}<7)U1;nPwaH39 z6Tq{8Qg4^tg$MSGp1-AK*TMq|bs<6<^DWL>ZI+!*!eFR9`L4~S(_0|nw-%8s{)vG% zw#YNzd(}#o3LKi*;<|Weom>n)mh%1TmE)wExh8SCMhos(a&oUXFC)QqkzL+~i5DfwVr}e|ta&=v4?REK zlNWNDnJt11%N@RoLx;S5k*DE+924I{L@A3)jNJj`unjHYjX$8`b0WD_g~6osRk2qF`m7yvY}j~0oyJc+{%DyoqZ=`y;W?JB zOtW|B5}X8pZM*N3bvqz#d7PqT4J1A$Jtn|vdyx6pi!^Ztv>ae&TA=*0~B;ZC9 zxH0LC+P-aHi-ZM}6C6J^@-~Jz$v=Nnx?O7ao>$(swg!}6Jz$~@I)F$ z@HgIf3OB!TGeAyq#6BWA>gIXAfuJdv^3wbc0Rh%d#7lKYg5#<14RB-Dt$?Ng zIjNEKrdqebjI=7cOp8WdWdB%0rGv%lcUENoh&w0}t-;=}o)y^uxhLEtvWIT+LxBOT zZ^7on%UV9G7N6nqXRe{@-+8H|pi=nwiGFKOr@9Z8biA|`Ih+yfFUp8btCcF2ey!9@ zajm_Dcek;2N9w@OVY;Fzd#2i5So&#r9gPApH-O;G@fV?baHtIQ^1C0q>M;_UTE*$+2o#U2 zA`hix4$Cl$w0-8!rpO4cEj(A9L3r;Dei)uDIyuoQ%ttp*ncCH^_VwzJR3pP4SPq@4y7P*auH!z$F(_+=WTJ}c5OVCx!7|gP)jMCpBfKUqpVUj%X{c*8)>M5 zjPJt5fJhBBayFVA2$(A!^8?1tTJXF~`B56;NUH;~9Oz*ClYLL+>z%tU@8@=rk;~dG z3z~TGkE}(+0U4nK(U&?DQTIezo42oz~Kyq9~ zyvi*Ip`CRz;H@;fc05AMN`O8yi#@Ko=I<*NX_YPKP#8=pM|mxRC(=`b>NWDSq(*6$MaxQXJtjD@0JPPR0 zvK82r0waWJS$^vviUbX$ySYec_xF4b!;trM$%e>7M$p+d5d4mV6a2ebwfF+$4JtSe_IM}?ij}Vv z|DIeD8TIVH4+07UXI0wmV-3ZC^8kcXUR_Vi_;nqnHfuPFP)@n_ zsbop;%Vi$_SKl7m0%Bgy3<%)TEsFS5_s?+Nd%Cf>IJkHIejpMrf&#h#Q`4!r!ut3` zd9A~$-7*uoGIZr*E+41@S*kLWC(3gd)h+mOnK(x?3%SFU=CtCt?E^2W+E|Bl7JEg> zM(~VeV3A)Q!iE}o3vOI=`ATMclaAo+y0j9JhB=9`){77jhvVJp6G+6*v=1Y@FW3IK z6M2!6;K>@f9Tn`hN3skL76Y#c=egb3z;m~hq~ZzXlKLn61+Hp z@|Budi9f7*=(@;Lg6(SA$H0*sh6=w|U(=-Ww+Jd%jOuuS0;D?0EAVjJ13Jp$4FzKb z*IeGyiCh7Ub}mnR;sI*RNO<&-Rw4@5Rgv=#5TP;|9Iac3Td^LJjjCbT`s|B(LNx|- zG>p{7MT}KL)Z=Tb{yHl4SaSR~_29`Ck)!M9Dj#;c7Y~8U7c*|E|FzJZO)I$q<)X;S z(aY2Fk}?$LoCz_9!AG?FBf+7Pk4up^L^M0<-$ z^i>i0%RsvVP63F^+I267Y_2Vg@TiqaVtu0>fK#>79gT|NBz9U+QyI=tF|eqfn#s>A zwxyJ>oO!GzfIN=8C(Y5qkYg$$j-GaR;C0 znb1D18XD;A$g-BYe!(16IM(VJGf?&1B}FOwC8Lz()J_`_U!fbwtOn7V{_IVuM-7VA zo0m(X!+6~G2aU9t+tQ)xX%U6eH}e9?1Qvay3e@FUaHx2%E+ccNHQQ0UKw37g@(cu8 zCv*4`@T!x(Vjz(ILlKlBU} zKh?Ps4CoH;R6`@M^%f}LP&vz38O?b&n&1h`d+?% zEsooQi3d50fjO-h4&_%x+)p6%NWrD8vbh4}8D+b37h)tQd8&+g*3#GmZ@xvL^cliW zg?e&l<4|H06qeBvNTv&RLwz6b52c<*(udDkDJS;U@ zC>;SP+aCZ*wzmb6-K#?@jl&cmYy*V|{-_vq2k^$^60TEvpnFkl`QPI$~NAM(yX1wL_skXR! zF}h_;Ob9OQ926JKhIddOz3%lFpD}?qzto4j|&YE7p>uu_z zg-=oh0F5>s%MKW`vB1lwJsUL46r<}^r0jFMXOE=$HLNJ^y|f<^RaII#6#ufM2(D?` za-4huVnBcz1GuyY3uVkXRK~1ya2U9-w0V*)*@*w;cF%{(uAlopemMn11V( zp67*WWhO>Y&i0^Z1R)Yih0Ewc)X0z_eSjE#Pke)i>weIT?&FVBSaYq6N%+8ViLUOpiFP;hnLNMv}B`Rv$_ zS9HfB#?}`G2Lg&E_x1eKC(V9@%7^A_Os21u4Ls}H9?+h1!wbOEwxZZHVq2w1EfM5X zdM2-hGi1B!KJI9prIml|2s{J@CP3xT8)_E&hG?s$6eERsazf0lmoCjxEU$|2ZbDi( zP^@!ydE)|a2;;>o{{Z69J2*=_Hp3HHw0El@YL_xP`I~%(j(Gc6T43JQMuEZhMiR7J zn^A5r6#{#<;5e_a$0Ed>dDDtbEqUlfccmYt6Njj+!a>wj0=kJ9-MDdzvPe7=S+@Ve z%c=Tnv>@{}0&b#05|G;+%Mtw{c9`NEEvzx(=lFbGrPp-l{k%Ad3Y-XAA_7jKRvxt) ze?i4c@-@`U8$Cj^HGJiAZGJgqj|QHljdMteXqn?V3(WXKen~~B*dCf&RFtoZa1c-~ z=0=5E5|zi4r==W#3Z4fq`=y1Ji^K>O1~VT<%BADY7+-)Ugj5Y)&xjm(T<3A{ zeEB8xWjCq>NWMeHv<-r43Ap?YwfO?l`)jHiLd)$HwJEmeP_E#Y;BW@HCCK~N(=w8n zorS^nXm9D%N^g|fI9fY$OIcC|JJg+)pPtM{OX{nc+Dm0&Z$_UFp*kh*GpZfl2fPjp zfE?z~nGZ!<{T&*eVed5EhC)yn4_rv?nQNCoQ8x4XU65_P@Ylirm6*Z1+Wsm}(BN5yE(LNenddNQHMcm?)0gIhwk7bp)X zm}Jse>)PpGFtMNZF}X_ZrETajSvLZHlmFJL(aVVZW8P(`C1*mLDp1?JfNK960lnw> zGUliy;yxDG*tyAZ0jsbQvo20r|IAfvFO8-)^6D1&&n_TvkX`5pmcj=VJG9{(i~aYse7}?mT1``4 zToswU2?(%A_FS9WvKztFzanH73O~n@tK%Y<1K5?xk1Aq+gWO^b&K?r6)AKA73P>#VapW9LAM}UJ&h?ET@zUb8Q;YEiG?*4 zTXTtndKTBk0M@5HOIS^+ZbfCa8cq#q$9Gco8?ochUTI)lO)wy;UBZ zt(mwzTWZ4awJi&IrVwW`D1-Pto^_Blx#21!BEHRsOjo#hmqz{C8 z_8~{>EUgQBA9bFy1_yH@zGI$GuYbsz6L3nS!(ldEV0YJx$XS@jjCY5sSfi5c#SR*+ zlcKGKscHalnlBpo?il?2&pz(%fM!6_u0hXJZEP`n^J(SBrmhJW27~W}ShDCa_(Okn zJ?f!YhByhJ)aaw(V{!(T=t^bW>K4zyn2T>6w&_rrm``VWn zj)=?o%DJ@(npuVoNAm{Rs{JE)o{i?wivQJ^&zkBn!(U)R+m9Md>jr}5V_0*>iP0tP zAg_&A$&T+;rLsPlRYktHzgvhUZF`-xY41}CG_WsP>X}Qbhy>A<74|=Gi+@>ng$h~Ic8p>s8>H|sXd9j~+p!;iwG<#F# zOe#?s>Npvd@$M=C8VleXTqt10bzdvrEKS6R20J0{#T>dGjT-HH_fMuS8LeL9OIuLI z9AV}qLO+U1!N@R(9ft03<&Ug@vA!37fyc|ad;B9%pUwi6cAFym#AjL}Los9*QxKh4 z(jxyiY#B8;ojXMZD-E^27FO`x1MPh~rIa+Lc=6B_EnECrxNk1v4spZ4*|@9=aTzgz zi|y8d>h?h+MfE-P&j+_u2j)yP59QH5*6QQFt5kPCBoMn~25!E1`~AV^p zxc8)zP1(6)1qqR!plz8+p0_5aX_K&3MZ|RxU9ldSPDuD(Zjiq$K>ktcY-v+HlTnzt zElkr+?)3h|^C2qwe02W!mg+}F)0Iy4w4>sfHTG+f5|gDjz;1SS4M_@idrX&~^A+<< zw4BoIh4>lDS85ZZt}stv#AwXt&}Xbj4WMt3;uX2h`VExt%Ye2lN1m5MnvQ-$rda<{qmXP^s7rW(StW1dPD^eu>VED zh3IvL=nYg?4xgO-?(FEzoIN%mfSn$GrNo^{|NJ*M=S&ID_nnK}I*45Pc)7r$##Uuv zU^$G0eCB-ub{9O`MmN%EfgGba7rTFy&%=5AD3cg{0w{W(<Ng^htilTDs9+}UO(HHF5)Z@N_amxk(>W@`m~D3 z?hyfoPdt1ya=VGnuApWZDAfVxrFV2WL9c&_MRlMio0d}8)a!9?rj{Smv}wUja&}mG zcG3r!h4`F*f8!H}0R064`hDEnejo!HO_t+=f?Agg)>iv^1!^bXqV1_DB*-gdt&O@W z20aoHXms*KtrqqBRdM$P5aA;RndoppbnjewKO0_RW^423*739NTBkcR?#8kRc;9`J z_p<$Tq!(7tR;=VJBeKdeg4bEgbYkgFdnbb-woa~>7a7+FOp82s7Ba$*tukuoxiqf* z`Nhng^;L|k>iV6`y@si9=0=4(J<)(*QP zQR9z#-<@MSx`ORb|9{BL7rJm3fGQ-KxyM5vn5TNA!}|tg0E6M{!p*hQ)8((5PPQ@{ zDHEIiyq|gx>Qb=OK%fSGop4-_OTeXBc2{3@?RwC*GTopur6*Jv0v1l{E1xfvjoeqGq}RshIj#4Br}+s6CulF!y?W)rWjOgib= zK=>As!^_y<>2Rn+q?=81?D*&?Ye)B?+R-wzhM@OAu+ts>v?2Q6D)3k>P&X5(+a!5p zf)nTWEob}P&lM7TWvuznNR?M+dn=@*eUV*DH`al9XN$9*U$CVGb4~q7hodTCtl9V2 zey-Mrubs~whqr(Cu&Ns||GdAly=ikQiQ%Q9wrH>bQQ<(jul;cIvG5yjnu6OJ2j&jLQBlW8TbzFnDQ z_AF#$o$~G?n@xLIYEGJ)HarThzM2!@mbu=-`GXp<#v&{v&w?K(u!C-9|FT0i{rdCc zSC020&G3h1Bn!Ymd@YXoAe-jaHwDP$cx1&N=8xVtQodgm-gW8hD%uT5z#;2YFMt>Z zBjrkH&rnz!9rQ;>w+~zJ){cGwc{_MwR;$p~C-@tnsD7xlkYSYt>v2Cyav!qDUh+|v zwaYl>ZtpVfcC0mFKORRUs+pM|o#RR`GcwN%uC!#NH<|GY( z)ID1kq&|uJQ4dnj1(Q^Vg?gC>M3S=yZA%X8%cz_p>k~HC&Dvc5{MSX!_~#Jrs)j5V z_er%LC^o!!FR^iDdQ*jj!J%TG1I35@=(OE391;E|7?f7S*`HP`VFV%zKDwH9`xc12 z#Pb1RXHY(JtqG-~h*`;6@3%)5qH;hm!^L zF3mO#AEeb3zP~=4Lm^um_ag{g46Jm%cOq?RX0oaX2#)(v5Mj2PJL;+r^H%L6_+5gn z2JEy8e6lNm1s-t5o^Du9H5UIN)vS6BFcu|$;74W&+iZd{*Ry1S_e8g~xI_18caCMi zm{x&(-!?0ih9G|#InktLwy$pmbOHE8$a)&&9b8*jPW#BzC>W(TECl1G=hR??Nj$(t zAg{M!o<(Jjw_&LDR>5C8Xv+#LH@9*nd7t{7?*b8K8LQ^Ry_%kUU`|{+sq*ee!4!uG zvy+9v!vSNa>uC{>+%;{%MA3x>UY1lLdM};yWO5(&S`_dr`-;hI*7KRFjrP(LHz^4wSV0 zip*G@i|fc`q>E$)IOaX3y&&qvkwyroXu&hD)?09wur)n44)zC6@6FG(q(eT+ z7}%I&?#|KouZ&H;oRsi#1^kh+{$$+6(?9Vecc98XkhZ;w@x$xx&*u~Qt$<6ad|~p^ z?MAARc2vv!de-nALU<@EH}^IjGr{;lOFQQ^{N#-q!*;8pbE-RJi~1UK%|>gjU5||p zO1-h7HRl+-miqFmxoXnVMAJsd4&siVVYG6=su0v|yiT?+|MJc%@W(p*PhuJ!;1pe~ z54;Y*0okGN)EsbdKFOX3%URg8A^b;6sd9DI%lx>(K-!52#@{X5dyrB6&IO~#GNh}Q zrrr2(U8j{LEUgMBv){Gx$Z9lttMc*C9n@2Rv(dWqX`3JU?^*Dr_W`HCe?uILE?$~G zbtSkrz|EX4^eAw6NaF;H&Q7G2t7h2%vGTG|R@?oQur*Rn{?1=uN%yWNdc`-K2O=EK zKT3Cr32Mm}+1U1>>*sL&)}MHaPf-)C5IQpxQZVxQQswx>O%ufWt>DnAE@Y!XI8e1f z&UixdW#G*QY7C7xhW!~YE>(JG?wH)%&wsFZMsKFQEM?H^$FqWsaX+PIJKl?XcX4q0 zj1M-9jP&?$zecsV3rsC_Hv;TK{o+$H=gC&L>k}fQwNF-od2vnKJojo zz)RN$#RjI8-KFo|_3Dl8FesQwC|D{s$h>ZWMrRW*gbnEi9y4yfsbky~ z+2W5*D0zm!mSG6Z1DdWFM=bHN|F`d;VbfD}4a8%AQm(07!pAS-zNPy|FG%;FnD$rB zE?6$3?!F$^Fmhs`EvHgGM;jqK51fO@KmiN6zNjJE~V4fzihCnlWzGuEcU2p_!z z)x`#u3AaSNsgld)j`if7T_2nKbYI!C$aIe}@Sd1Wg~2w^zoxorkC$o^xOn|nvut3Z zfX;U%|J zS?ett!0vTG>|5HjZ>|4Be`Mu2Liea(a92u7{+K2um;HP1CDq(O<<^8V$U7w9pp^~R zAO6EV=Rezkl%=6yQ#qoeo-ndX4pYcHQ8`}rlu`Rlk0EA!rz#7+yrHAsRo;*`V)rr4 zbR|X_&fHSG?l>Etavjr#2f{%xD^)i*fAWmNp;ipb#e3akSJ)Ko%-|-f)U$*%tS>)W zmsG{^fC(eBEZ5v+)|Ka}W73spJmcgpe$aiB69+}J(iFo|*T7K@HGuf#@UTjMr;WB! z<(_Iwxga=l2L|&gzjgF!lW1IAdW)vP3i#5ILNejwkG$&6?eCB5zdg`qK+u`^DgT%L zenLYlDpyk#VNuaR>xuWY&kGmw=Q+CvP5{ z$HCuqZmNfl`b{2&hqhakw!v32C|o+~{%exx>BJ{}j81$lp5LJiV z?)A)9Zzmo@Cj^q*78NKAtkbUp6LUP~7tH=IXY0!d89!FZcz^l%%JIT85K#u|-{b^u zTiXngBMNY1$42)P3K9rPEjN!&j3Znl0&HrdEdxkYj2{(DN9p<}qL-DuGc$i+rVIG< zfWo@bqzmL_7~W&xt?+y6szy@qFm0)YZk?FY`fqoY!f{gLeY9NYryWZ(kok0c#S_+_oiJg~iIS?I~7 znANcx4sACzv;CK3XlIGF5%ko*B`@7-9WbQO<7sUSx1djNxNCrW1@Oe~8R-xG zu)iKIZf;tb?QK}g7<;H2c?|U@A-}z+VEpf3!bkO&%P_0Ac^f^(J*UdZW|o9#OOu9% zLD9JP9=I}Vb5JW@0M102!?>FYJ4*M(kI5F5bBm)w1=Q+&rgC9XEcli5>3p*jRk$UX z#!2<_$CSBpT$jYRKS4T}SdFt3L;+42g=048L=%R>AR)WJWSt_4Gi&X%-()!~gP_CG z$G1+Z4SmE7qWXM6iLjm$M^S9vHU@Q^IxK@eEu{dnZfQW+2TppRMRTOZA98|P8zO)2 z$f~d`Z2Wc}r$~kT-u){vjla~trFt_dNglJhy?hH?`N+J-xSJ4<=dQQNoQoc_ zmbRdOfy;CPd3h^-P`_a#D@Ll^CfOfwD%*0&lYv89#Rm^;ZP^d!?^V)4wDt3%X^j`2 z1sEiLT81JVU{LY-@9}%g&h5k9%zT8yRL5q(ayG}kG*KSDPww<>XJ~UcZi3wU{_=B` z<5Rr_Yha)8N8kVuDo;_yh89UnyF}P)sBuQrIz23Yhhp_@!bb%`V_tdXCy_FSr*>eb z+_hEN(SE@Lp_uoI`eel@-(e z-L7Wk%^|7V>Lk$~Pun^SZ+pToxXXFq#wLb>C(XLk@wqzc(Pdt6zJ%zU|0)%>s{yIc z1e%0_XuFE1 z)S7l$ux67PTs8OmJMIM8_x?yeBZlV|l4G&)^$QrY9CtkiJACZ^z_}g}3NuY$(MI?N z`-^X`>z%$RRj#1MiV)*KUaLvC)WTZWk3+GR`i$!-tDb?YKb7-ei7D)Tqh`t)GWkNg zYQo&ZraPi&)ZZC}VpdPgI!kLy)aSbS~85Jl88W`U^ zyPW#l^w9LUQ!Var=t34;=K8$15b9Q(1+Rp~_R#a5Sx7*em8)G|^kIQn_nm+{H*|rA!-) zq?x&=nmUAWdb%-re#&-=S(>)jNd{@87`QV3`AQY}t zPMz(&!$fXIY|roaUr3p;gl?SU&%gpUW$SZqF?_-uoo%E0j2k959uVK8yeYl`=0ZRM z{ZPttgvy~exKJ^RD}FGUZ}X>>>AL9)EfSQJCo4mSmzL?5#9dp4c^^B((($^T4Er(x0Q_G;^ zI?&c1sRM(DP>*-6S;6~zk{+b$b}vDvaI5t*R&44P;iI5nWivAiB>@&58m>c1I1g*P z*}*&bdl6OxETAS^0e^|7IjZQLNIuj?#{4ev@9ybrGb1UomT~FF5y1NVu^iacbKtXi z97Dwy7(7F$@{7Y~u1w!UUUo**omz?wriyE{8ObX^wN0Er1RVWHx(;I35{_5Oi&?9z zz{n9g>hExl-AXKYfgyPn{Q*gXu1N76Xi}k#z8G>77Sx?vQ0)7@zZs?cg*~4fJFh*t zkeSFI;s|RiAOBbh{Spj0+OJ+O&Xj^52H02zAW#b70zuHX#mM=DkB)iVPF022)V;?%F;k}o zWGm!N476aA{1NaSg|&%L6i)d2k$xJ%5#JDf8h_=<6fqcF%|W1uUof6SKp z0flD9Rz(Rskw4%5ele)Z_S{={1`*M?OO>C)tuQMM^=}g3vm7iGL3Y6}CZcgcT~&-z z8n6k{et3ghK?)YV4MZabvAzJ;6VaQQHJXedNt7>?HF$18h0-JX?z0Rk20H5J24@q} z7;|4YbcN_#=Efxe_Z0%zAX{EM$x3!7Y&-#VhMfDBJuI4Ap=);qt zC@JPbefSM{P-^cx&im7fg6y~Wga>yC&R9#8pNuf;zHx`TD*P=s_ElZavJel!lOSo$ zeVskG0-UOftsgUeQt$H4@TpUgfQzQ?wC&&`)rjtU8Xy-KAoPBv=GZCrV+bFqZIjm~ zqoZ8rFtj0v{ItLR`I_SbA*OuPQD@~iV!^s&*--}5Y#sIYm)nYol8iQa`-LE-kq;ro zGR!L$T7Z|l)H92?pcv>f{Rx%VgWn!(pxPH?>u~hSaY%+W82-GPXvlDl{jD{omSb6v z`0xVohCbyVh%)pFazL^`koECK-7gQ>1j;NYpc^b>zaVY~04YE25L4m(Q!J4f=_1v$ zntL?yQ5`+I;399R9Y{jJa=UX=KEUB7w3?o!7`4M_smI1ATJ!RK%BCeT2vK*Sl1eK@}6|I04sm%`f;bhh=> zYp?X*nf~{3_)gU)vn`u9;JWxmi1JJNBBjtNLh9#)6kuQ_ji}m(k&dSWKiD-%KU)qI zLl5M6#X>g;zcW*MV@7o^=K(+qUUsaHNYEL-HJ4Iq7^bmGQ_so&!*N!omj>Zzo_-E?BVlMTG z=tTeua9sxPzu0u54ix)d@)L%p9yFxwfH*ztedj*?sQ;BD%pl-hufo^Am&Cn;68@A; zVh86T;N#A0*j~Z7;{$gUB*pPjE*7Z9p4tIw6F?1e=j6s z&1HVGziv3JH33WQQs0r?Dk8dvCACQ@U=}n+fs=z6IyQ$EK@-j-|1Am({Ed`k!(j-4 zBbVm2kGZI$dXiluaA)Rf;!=G7$;ZLmxF2OtFc+eSDLl*#zN3w|pQmz0P+$X%IiUE- z?QA_@)t1cw;|(PXc)&B8X=}kYW z{2|NWk;C3p!_BXOHwx8xaH#mkz8B`uagmV5+}s}pW5GW+_4MT4O1Jz#3X2xgLELT6Cp=U<&%-N6>T;o|W}iSm{`Sr~cQ4Xq^_;66wp+u?0~;XehM zZsBTu$Oa>~0*T*28y|c9%E{|xw@bLQt|a z?9cG9V28}2R0B}d_+-IZ!k(D-w0|-KyoYvC{VE2sJK9BK;t#Y7cuC0D0v3P)$A!H0 z7Bd?+P;IOl2~MpK<{?!51dmZc073ofc0Ww&%MKZp#1zDlYPXM;6M0d)xd45+N8#U0 z;SSX!8cLzj`=Qf-mVf&65s*!_bi< zFBGNxOLOKrrtX45R59RhnukR<94x+jUJBQhk3P@mc1AP~2q7uft4!WxJrXGhUZ4&m zpd!-&x)ngJ0*?XOpt|S)TXo(p==l2&g@fxP2QHlSz?USgjAW6Am3tzq*w$dggX-tk z<7QvxV!34i*M-*x*6P(HQ%7(O5e*HC{u4IU_Qfjgc8A1Ydg zf2aGnYanGwn9^t?{4wPT_(Bvy?5xB$YCebAxf({y<>hj22Qb0`f!ytzpp?mxv728C zI~}4k;i<$QZ?z?91s0Ux?V=DAp!j};gnPn4h&GZre67R=_*vIK|Ed z-udcxqEz8yWtK$F31q}w5_99*^4G7j(@Qj3wgHjE<}6@bb+1yr~OoGQU{!Hm$sGJWyZprMTw}1Kv@kodk*#Xe7d0W z5fn7k>8Qdn;I;F6u^(?KV6AbwqjS1D@-2KKSR|HEKajc9s_Y%K!|jlsI6>1Vk*1q# z;S(g(Ugztl+E0F+)eOtep*niK#RAx4ZGe$`m9_oUZB77CdmmLlkHgtN=T_03DyK>Y z9~v@_1j!D}rR+!ErI$?MoC*h(;2Qh}1iZkD7aG!EpDcap1XV5vTDFl}fYGW_vvR!o z6WY}_*wxK>l!LPv$o~I;{edeM?E3ACW0Eq0FJBr;Kg}Jz2C@xCWsh<~*--?tE5J+k zAhJQIX@_g!#GOX@qUzPpg$y~#vA4G*g#~v|{ZoOGyOt)JJ4Oxi< z_?U_fd`Dyh&OT|6bUU2?UTBqg-HYF!3IuL`z&gCZ;qlD+`n!2Doi<>khB(5nYwxfJ zKaa)0Gy|Ui2}yw!&<@TXN~41)s6hhL%l#~+sBM|H0N`X_Xvx}4`;I(ze2lK1!TubSCVb9%)yjO3#(x*R z-g2}bmZ5ZlGl`%ajlQ1algdWz+uf+D3r(;r{MiD>5ySPt=cqsGfJyMnKRSg4&w&S# zXMs-k2VxaNgFf)*M3}Ep(94tv+5m;$>&7*}%$#~q=}7N?V8l)O$PpHwJ%>eiI#*Uc zVk8MPr!!QGDRYDHlpV02Lr^X+IQNsu9Fe5`ykhU$f595Ne`iTI1<^Qn)yPviV;o0( z*I;9#mT!w-m?;XHhNP{8&bv}~^d9X9zMumAGc=KdBUgJmS5&-)Hy2rj7OGLaNZN}e z%C*N-2GPfZJ?A>ZWEF~PC%h$mn@=HZK3su^X7f9=NI8e)=5XGH!&kbGe_K4^uFf{g z!O;8M`A;MDFCE-~w}>N@8IP6i#X#AAO~G0U@MV7TKG@E`tbKyYMSA&ttpCNd*!z{O zMY4HXq7}}S4Q&%_n8XR0qxri{@fKP35ZZW4AVpKF|4q6@*#!dOveNA@R6C&?Qudp8 zlJ%2zNO?A7Go5CLk+Jr5!dn1&4*!7w1imooD}Vq5N*p+Z;eKHU8u~>?-x@E8q*l}J zcK_~2K}UT{ysK$ed(LfL^-1}jrom}pEbJcPWAij0rnz@i3OZm|6*LmY#O5~zZXF_G z{+Co4*D7r^2JVCls6R^I{rh|~C*`Xq#%;D^FTa$>GCXi;oB&P&%mDUe4|Ayad1g~v z#6l#p+m0Q5{E>+N{W@vYj=1BMeH-*$$2J{-{C&FJSx{wI3HorSO#p&0B77m10xz5_ zl&#oyh6ff9p&0o9a4esv`<_Z^ zmtC@)=XfB3E7oxMv zAZqKt`=FdWFw93|;GFq$e3$b)mI%LWH%HI`y}RFG`u>P$|DocvV(ZbvFit3w-@BceL`c#MF7bSt_Rp6`Gb1TK{&{9QQ68$3 zEs)rp@}aC?rY9?U+$|;tQlK|VHt_xa>if`*uquReYinY!z&s_buQ!31#9Z-uhqgUL zxiu9iW|Tc#gIB*x%jDSm_%G)%j>{SzMBJAomHssW=}Q`G9z3a9arLCBLSo{F6}L{B zo_eznCog-;nnGgw?j}zNGF>DmfXVvVSN8e z>6GL6-76*Cv4k1j4ENrKSsh~`@2yXo#S8S*KPPw8BXlIy=)x>&IAJcMAJex<6>@&k zs%b2-myOka+i#Ea+`9V8WIC6*VFE#`d^KKPSF5r-x8Q_?`&}KUiZ8COWeRGVe2(05 zT>sZU0?&qebvB(Fsnk^ueU45!;1d#nO>`gGM9qhOFCKp;LV;#KyyWNkQ-(KxJ5l$; z)&9AwxtpKmp3ecHy9W+mu}K?VRj5Dy&w=1%bGM~&c*=|NY2`DwCgZL;6%cf~O=-wLI!`vma2%RCAO+5QcT^8en_DI1hvW+sW^koqfcD#=!M zlMr`G(^CId7{Cz5)zztx<{I}zB2?3u^|4r2^O$naJIRtRlfSn{XDU73>HP7J)pJDW z+6LQlLfy?d8Y#O&%e9IM?H(?yF5zRnzR<7HEs{ZEzcnoUVl<9Cp!)eN^!`FK-?Q1T zX-h79XWT1w`L98BCAcYd&1(4ng!ols2 zoF8jzc>Z2TW&GhS9npaf`Oo*MpK!YZe(_0t1@#o{!2JVDQ@YqRUyEM**ebDL0DLw$ z>btu~jDyN~@bUqY$%NoxQnJ(MvtqBdZ}Fq3#gV36a9!S_56cCrlt+5T?DW9u=1c#U zWqNa~t3g(L>8GS-pGT8RV+4^wJPbn$)vg^snw2cj&{biSQ?!qFDB`boHL0m#3z30s z(^+wiaFNvUsd!z`n(Yh{T-N)3sLbRtNyc^ZsOIade-pfVNBQ=dTREBPTlJD%bM*KV zNItOyoA@_2Z$Sy#yZg(lBw*X9Qxh}iT2K34_9-{Suni9V)c_$<_B%wLzQk=;co3y? z{c-Lx{PUCQ&Fpc-in?;8ep}xThzL}so8QVjvUOBrI>#*+Qf}%cWS|=`W(|4C(?u!~lUrzlVoFB~-!WBJkx+%#|*u{r^ zI>}q=z%ew!GX8&LeF;3&+x!2AZlxqdDBDG0WQ{BlLb**@8cHEcB8jYpi0MWYam&cQ zw2Wx6X3a8{C5*J#vM=FAF-Vpy|L2TM_xt;w*UNXgcjnAF&v};j^M0Ps`LqWOkUr#J zo#ZCqd4)4?ndrcPDBuVO^(xmb+L*Ejshm0^}%CO2I%n~tYDUwIeR_UPhv?(A^R zk0cpiz;7V8)-I3EzLkNcWbZ@^er>WM;KEXzmn5Vh$9w|{*NFUDEB5@Xn5Peczsd@8 z&RFx31|3BoYR|7ZV~F1{m@+)1P+I1k%bD5S4JX1Gi=xATkL@DBD(xIvKMn3_Yi4*uOuCDi)REp=Hsv0 z1ror2bd(;RSCTGeVpL$i?Y*dLI1Ey$Zd*1IFt_c9B8YS(_yKQg5LU4nZ|K_yacE$eGn}oHmxamYlVBvM0`|m z2MZiH_+>|Dv@jQryrB3!#KWMmA;~B{xa=T9}c&{iOf(4~J$q;>e?fejfol4auw;Yv;L5J$~WfQssG* z9%8+&gDTzu@z<;HE9z6q-yZzokYhDS>_6I1Q{gsxb;+WBJyGJJ&z;7jdpGn*lD*;y zby{}IoH{-)sysEySJV+#1-zv1it%Tb1W>DTE3R|=IJ3Fgk5^~bb~}&$$Lt~=2120W zV6l#cQ#W2*{pQAJXIaNgl<=^j`M)y4OZ5F5tT~FuRVXiu1knz&^f`o8$5-6_s9p%( zi*om}I3G5fZuwU}nCW26*dHVBrtPe3E|GW>Zg9uDMSJf_xn32=NC${=UT2KuS?6*} zpw1qd!7V(#ZLjd{!xBbg-JATOd!oGCo6C+myp-UvwA`e1&Bz4b>U{yAJyC`XvR~@> zNl?xBXCv~GAu6$7bv94ezgRD10RETJj6jLGsgTLN;ia=cC7*Y6IYxVPeBE39(qnr! zus=$i8a}@!@^tv<(-nbuHzE2V0ETEQ6ZoY!*sK&Bg5|`kCY$3V4JKOK z4}(MtfC=l@F-MLcXByP1-*X_cPY-zlYJ=hprgzy3DK(=o!%-Fl{VsfRUxBg;=dN&I8l`-8(3?{6m!e(#KeB^`B3?OzQR<){SR;oFd4s!Ps4Tf^Zk` zIfB{t9!&P2pqjG^L2dz&)?E`x7VssTb;%nx5D&r+G$=`DuH8I60)2TrYr1DPpP^jZ z3y6Ssa@FNvW4HTj_a>#0mMo2g>@?U!1PV0&xj9%vJ2|Qgp$WHl=-PD1)Dau?LFT#z zPxWdaW4zc*`2g!aBiyO(V${^-kjk|Mi>?ng{pIMJpmNrgDLnO-7$^xY&N7(emu;_X z2gBS_Ko2=yMch07WwvF%po&+g%;jx%mIa!BU_zbcoF*IpHDqHR64A0U!O!KtG8&wJ z=;|(t4s{4Wu>;Eod;^6a1vwg3J*e!in(4TB5KUa}UEK9xm2<1RB8Iq~d3@}03Sqvn zWODN7%rkbfhfqDH5le7;`ZUqmZlk;wjvVDGomyHK_~Y#}m|LC;JbL=Al@I_Pdg^|K zb=gUF10S>Gp2##?cNve><@c6X@v-Z$_^q=b4@3$y)m(TWv|ge5lrJ+)`v==yj#nKW zgF}=B(w5pqk5iKkyaL40yRfX-J)liZpP%VSrCWoS&J;G%WvK|^NDdQt9OF$|cHhpk z9^4Z5Z?{BLzHUKIAc#*|*SS~|q+e-#^EtAHM&vfSBN)d_1%$MhOcHKHm|4x+$^tp0 zM1X0j>x4IEw1Drkg6i6(#n>K7tvXXV_a0L@xuL~#+j)vj=Z`R$Ad2oyg`vq*-s_g+ z#;w=<$25nGK)~z=CU%Sy@R{~OH#?6)f83?Ml{S39zcwe#SQS zg0DAaP)S<)U&KSuhfvtxC6@&Qhu&y<3r`h75?RklT%WDfG^7?8gdg;%B@qGyC+t}I z7Fv#$pbLMApZvsiFb64Kf}qjsesiL67I=(${+vr3A{yiIciEWh{v(vhD+<+sA7Z@69`fjKJdOEajcobnz&o9VY}Cn2X!r@=!2a@X9NA0C z3uYKHggtsj@lHF#KDrQ$2#(@$*Wu5^_dWcbV^d5{BwWrff9OWjn>f?_<8=70L1U~l zhHs$UR@o?HkJ3YKL%NBkZUnW7Wd$z>1nE@fD~MSU)O(jzd?hB%oe&7y{%g^g&qjsn zuG|Z^=PjfE{-?1n1zi1T^Ni+OY`2jn``P*eoiG>WE={3 znzeHyWnbH#%Cw{clna6wpF{~_7LpJ=?6&jplcUg?d4rdKp}Tw!f`{8yo!Ma}>Er*w zHZB}?8Z07TH$1?*n&7QVdCL69I@G<5qh9kpxG#^5y)MToe|tJjw_the#xv_FJhj25 z#m*lxtx-OvV&B>;cv5#7q5|Y&s#_SX71{R6%o#1UDgq~#6hpi(ki~^t~+LRTl7dc+BzBI-0GACX6b~RdS zYU!uOvg7 z>M!#v$@r7e&e&$I01tNlxEJ);i%Um!F=CCn%wUwd_t-rieNe~Q;|kT$eS5y$|0PV< zqy^09Oz}S>kItQVN;1>|rMC>>kOfsz><;4i@gG%E%;q8F8xruXJ(t}E8_n076_$bR zKp?=%t?*wypf*4(7NGc=-rJp|Bx_(RCK_0MiQfQ}Jl5z=31f|?$p&SECtk}<-!1$H zr6o*ZY-o?soyRQ``{0b!t-6Y%<3Fb`J=(IimtUIg*~#H&hz`Ywb7!^}|1;64OpvA>-i}GUYMS=vc9{Zv}{@!)IWvYkMj#{N7H`aMb#;THU#stp`)9%70 zJI9t%gIc`z;KK{X1AHGq-f*wN#65URzOnsy<@+V8+m;=m@#)FpJOF> zTvtbX!KC-8uHtL)0@ZvmXkp*BpU8t85`I_SuThVMvJdnI!BOQC2)q9j39?gH(^Y&s zmDvqG&zNE2v3a+?Kc42`mr5Zue9CCuYrXuZHCFzr+;&g>G+olvWJq|&!u~(nFyJ@- zw|NUhnsR4meE*^HD)DNcL_&!&Du}>vYCXh994E161SB#*_=8;a6i8Qk3|RWC=^%xH zN2GE%&*XQfk;bmb_|2*Jbi?+(er7A#i6A93kFzEp+D3GR$s!2T0M02<&@a667EtRlDwE^nT!9a-*^C+g_U;%jHH0bAkzd~7wi4K9HsL` z8$?R~H=Q@mv6L(n>y_%#Ae`(yG*_738pbz(FGt(Pu%;srrBWUM<$_wJMr*=y`SYrx zjVlU8f)*jL)G`7~dgnn(^IikoDXs4>qXS=Go}Yha0kM*H4roD%adc?JIG`~pk0e(b zxV{2%3i`P(h-NutTm~YHgq1?jB``pNb_2xMzb+WPY2F)9xyFwg81~3L-FPE5yJ;9g zHAIqbMKB0na3_x2dCtPCVIRW^FMN9WGv5>{O>+ubY++o#LkZhS))(f(IC5Cc-l7Fp zAfL>C@zLl%=NtG?Qu|m?tMBFDWUU#BL)eU2yMCko?S<#AVA1NnuYhhkQ2}naWLgBnyubDo)Ai2b z;KUAXixniM0ce=FzI?OD*EfQXXCP}+V`F7Crw3B_Sz=lhpSLaxeqdx;mh3fWs&BUR z&D(n{Rk`jw6S0C$bguwA$6r;4pQp5jPJil|yzl}z>7)?4m;c4ge(V8oqt9hb-8PsjA`r3LmUrr#j z$M5kPJn!oVKFX8Bt}|K+GyYn|_iP3^9-9IbBdB_~hkJp_v$fr!)F%~0YlP?xE>*+> zg$l@@>{iMjw#30BnFc z^d1*!-sQU=0V!7ElBUgy}{kZBHJ+M@8P=UjYu zPw};O$rxtrg1S82kmxBHg$F3n249Pf67)+ng&)o(_EMt z3v6$n+9K4!P+BovWY-Ag06z!cZnyw8?L8_4U%mXwcKb(HcwXpW~$uo+YkY^_X?I zyUREGW>j?R2p|rg*QoGZpYO|QXNlj6Nlq5OtV4azSZvhU56qa)_TqmYe|7VhWCpgH z#TGPF(Onl%AoWEnXo33u)iwGU^AErp!#9t$t;;ufbN+P;s^`r{`fo0BMHYxr4@lsk z3*KyF211Wq@a6x_P^PASp8xH!*@5ZZD59Yxw@a?(Qi@m#m7(o$B^ruWv_SpdRK{O(DULNi;bnx-= zT!^2$9Pk6f3I(h9V#E<(q6lb2ZNVt70W4?*{S!6}5A!SaX#`~_U#}t;-Gj7vC}}hj zLVe{b-A(hs=hPn-%W4-;@z=y9d^9OEIH|HuJ0WxMt@T&Z6s0-I5B;W1y%#eR=A#Ot z9IYv(ed|G5WCk{G`1&;2YYPxWc2*kxzq5$mLy}jXbaaqDe8ndb@V@)rqhu!mJf!(! z%mtp@2~Cdl;k5g{9l7%r0=oAq#OSonAdYTuBWi46Chv920ICCqNa+cbeerP>4$?xQ zx+D^`Mlq|sHk|#!TWfV|!yImmW|kFAe-CE%H{?D#6#?`nma6;8CO!}4Kr|U#ZUJiK z$6{;5pbU*1?5;R8qr!>jKFVaiHDJ|ta6?;rnV*kZ{_ub09`3$8;Ej}y^3hKmo*4Ui zn>AySKM2z8gBuOrgdI=@@XA5=A6*_fug)v1`a0*}e7?uWUyHw+;|nPB+?8KDT4zY! zh;`m!ooqZE{)BYrya5tw+uo$VAh`gOAAwWm_^JB^1w*~1kMz_L5t2h;+@Nc#TXzoO zv$JZ0YlP;dm#nba%|L|Ud^!JxAhDz* zZK%hNh%iXd07ACrG9Fzae#^eC+iVDgtEU#lW78gP$H|$6Bg~E;zc!lrxPgyC3&kVn zS-v0lumm!c1|KuJud}_)VhY*-^$u{IGo}1x4Erq54+l`=BYWqu;4v>ZVA1a#3{j{i zpJ9sk6!$16t~k7c2&D5_v=|+!Fy4Cs>Ur{EGJDmku4YFo zpU!Ca#b?A{yo@84@_boUOct-# zvU(6rhl)OrM#nSmJjshsI^3)D&8BiD0!ahR&xZ*^{?}sW{3HO6XrK=zqY5&y0v%bc z`mH5Zwy}5i`>#EjYn#>bQ#iP`7;{mJQt6Sb zD45$}q6jI4@T67$s3oT5mkU z&Crt1XD~k(A$y59jUBa`TG9`n&|}((w#Sav{SGD9iKRAEDCaSF{fe1A9a_&4a<9>L zF&>bmnypyFtH*-s$E7^er8lPHhWm$-1+R0+o(_kc`P6IV27Aj<9E^voC*Ffg#|KC$ z;1vxI{IXg**}IDft2xmn3r_Qs`WD-1LxEje+gY}zZsWZ_#Pm6zmIVbzE<25*P6(b* zt{{A?qzh*??80Ij2vm%)E|d-}ff)r|Wf#$iVTg4h+9R6x{?m5w*cW%Lw}_bjmZYBk z9u+B162&ExKi`AJE=jH>?fQtqWv|~p#r;_7rQ(02R>&S_MD~0MK&bXQIyj3<*8^o7 zR+l=`=2jF?b4zHd4>+F&L)jGP?|uTQdooY<=6v>i^BUD7h5&Ayoa`Y7He8uHELsM?0}aXVcc4kt1o;j6H~Xa!N*uqX)+HS=VRMeytsA z(zmr(j1W|hMi?k~)4L!=g{eP&)a~&MtmHF-oJ<6LkhU{9ZVP4{W%<`u9aSmyBBp7- z7~X9tv1~!$EoN|T(U@Vn%_bTr3lm6$yo4jec99A~9X^E{ihVv@AHk;UtD@uvsa$pE zzp~qfn+sM7FMYKIN45TEtcvP2Y}K93+~Fzo37T; zg`U8GQjp30!|8Cp9#BpLx!5X7A}R~i7GJJ;#-2*;ZHD7~e^6wbb{AAo6{`7Lm2&K@ zh9ddwG{-d>hEwMJjzM_CFs}xPO0mxfe_cF!$!=<$Ow)J_y|#&p<%@90IvqVEMvQwd z1_g4fhnInahoti!wHNt#NnTBvovg-XCY*WX$>~cWfIDjVbd&zZoAwX2{8((5lX$hEOyC75~VflN8RjCc&+#5xLpsfo8X5dS|V_SO= zA(lL7hxw1;zh#7L`;*Uef1AC7;EnRj63^Is#i$S2d|f)z;9Q%e$zFmwK5}Oq@Eee8 z($+;2S{SStKA=f+y-kSn{sb8xJWd#tEpeW{ToBzQNRM5PbO%9u9F5cAL&U|=HJBq& z=bRI*bxYj<5dLS7LXAuuxeHHOL-sys(8|mFH*}M9B|?2&fJ2 zCIzTP&5Z2!fxnU7iDVzLB*ioh84u!J0DV?ok+Oiz~=4o(3W`oE0`tK1&y5onLUGg4-ROUl;_ zGQ#B*&sTOe4f=+=?=&GkE>`}V=X0y8T>Uf?_*g3W2>^xi??^laJ&N#fkC6;7A($g# z>hyM7ZLV%$)F55O(FIgLxv)fFh{ylc`x@6Y(VABa7W(9(ka-s!mYXM-MlNpA@!4s; zJt}Mcg4)7(HM7~Er9@12GMSWlMG;BInHNw3B?_4F1~oY-+EDk?yFkM1tm{|9IML{a z!qBby8BKSUi$%~p;@^Glq)FM-r^fs1{NLSQm>~~MM#cZeC_cFaM6$PqFtc`*VKA$3J_N~= znQF6Q)jr$oG;Syesj8q2_-Hgzx5K0Skf}(A=KjUe{)?0sWI;G>XsuAqrS?_y2VD85 zCq2IS+U>98NN2a&X%x1UY~$Bj&opA1{IMtc?|r==f7D%M3A761u=5wzQT<7~O(hecpEkm#7h?g1p*;+}Vrzs9sMW{uz7&nHR|hs=#UW^g=~V z5zMpco9V<3|NP;>fn&ych+3rC?0o=F#^3>CXRC6+O^759Fv^cBP>TLN$Es zbq;hV?tks<^>tWhAw5_DZMIETsV}BZ5iLQi=I^0GrMqfS` zs^b!vO)A79w-O%<`Wc4!O)nhZW*^`(ry1n-qfdXW`*j7dSBfAQb~JKYd>-o<_|zPv*JiNzK6=gB2qAS>Lmnf zvY=x(-qUKMM42yr=0fSOfih>&w$K=}2zT#k|0pg{s7drAot^XCqyKss%nGg+iGdy{ zY-qI2SKj%V<@bOSnvCuo&^o9XMANqXiC-QBgml4xEnwxs=x2kLW zbL-{t@igZPjf-#cG;%YaY#Y6!f<;5$Sv6l4{-t1c*JSZb{p9U1$tk(v!kdzfpOGn2 zPf6^@ddFA2bSZ5LQY-5fju?^(-}P^;WeisVNbkK$z8H}p-8Grr=kQ-ZAe}M0>p$*| zxGtA{Lt$(?w{|{>^|gPQWUJ|tkln)a>F~33+=owtAEF5Ci`tISaI3g=>4p*0$fUBI zBhceyOGJ;z)=&LBypqS$mh-%8{&Kf;=^&<5)T^n+@+GE7^=*|mze?N4M^1vK(|cR_ zB%(Gxqd2bB*(Lw@vWITxOIH5a`O#UIiWKMC+%v4PUp!A8Gr7dH(Pzw|>N~T{6jfAh;Z~^h7x(09S|gsWBEO>ZWKvJ-Q+XHH-BW~W>*Go1 zaOBJhe3+v~D~c=2Xxh#Qh$|T{D=#i)^YN0~ccNcK zgZR_yjDZu^Q{7VaN1PLZqP}OtH!HOICPaMV=<6k7T;0v^k=6_Mizh zvcfo9pM1JSzuzY|a@d=dv@#>OX%>kzP2a+}44=t}rcZYXhFgo2?Y42kSJR$Y@mJiH zQyRRljSz^uvP%}@ zu?UNtGM??aye9#)UcD7PzGTe1{nTQ)Cp{BKA9emGSU%pZeI65{ob>E*e{rcahWSXx z38_Uqm7?`r@xm%nW{G+8X%b+>}I$!a- zh)Y?go*R(AK~8ZBaD8W?`;c76d}cy{8UEyR%|D1Z{`y`GR%ZWm)83O3oD-s=0WxGu zFrC9V&nV;`%bfxM0c9t3H?3Va+Wb`~0NtARHA4MUJDH)!E~%=I*v^9cVQT&)tn#y~ z@NC~On@#)fyt^-(Cx7T3UE}@v{p@@>$%CJB`sDMe`CPjBJ%1FX1~QI@`&CP<8iw!^ z=z_6y)`MBU6Sr(OJnmvyeR8(=xQl~%#jJMqrwW(J1lNzYID0sGY`bxogzH+i&TC1b zK3h29WtMP-WIdq{+NmihB~N7o)5mPon-2WZDAsRN7XPW34{qEF@oi;~8Mw9ZI|azz z$@90^szTBLRqWv+1x?{M(rzDRZn?NG5vS8U*_MFwN z;3x?Y$w4c5gB7ud*;_}djIa7kn_cjL4X3;0v)U&Kj_@!^5SyE(!+!pV!?>Th>tYM2 zPlsnxDmQ7QmR1SCX)`}?9C8hkQbz{-LvdgXbt>`V_E#E+6`$Lqan}; zXeiq6QF@5p;{l(kW+&^W7vp?z?vxoBCe&`U+zNh35n=r|Hfcc3s2-;#o)h=BZ zeLzE*1mL2Hkh!9~(yB5v9~ws*2=@()y$Xw+lpD>3b#(6r2rohd4u6W=cHv3P2h$L|?(Ze-rJa5ThR`saB) zftw7BdBdEg067{UhMXTOB*Z*xyr)nP56Qi|ljEOj<=wMUD3+)dV=N0M@Hfzt#GeSc zVq@JG_52IL1%RG^C`A{|rnBzNs^;%Qdq+e55SdhLa5BNYWgaUahPO}Zsj+KY?9S5+ z*=?q|;m8r0dkYG5zZJ2Q^g;)`7V!m2`>`>@vW#W_viNUVW_g~T=&~}~?&;TqJ|F#l zqst?mIuwJVx74tfknAtj;h7dGx^{-d8Z`y%f`Z+f!=hz;a4S{fo$O#?X^6 zSP{P*-J_hO-{SZ>pp`xU#E)s82XtZ$ty%mTDHkp1w9KR8 zE_ThEyj;UIvqRL1gzWw-nY89As3;|dE|Bu4AZ$)Uw>itX70WqhdyalUE(TO2Fj7JaFbBa@yG>IidtVL1za$er}o zr-w}^LvWqDyyu0y35u^goJYg#Q*JPh@NW1{cGi0;eHx0ckSG-JXW03S=u?3O>e z;$wM-N+Uvb89eyAY%(o@Cax%F0gK(H==X76U_1SpJ@dP9)9nl7YnWq`mEHm$XFhTA zIVpEs8|Eig(7Qr4dpbgWO060qkd)_M!ewd3y!KWR>Y|KA=izZ-Ic-_5_}F!@k{u)g(dJYvKU_o2t`AZA^ry1} z29EG4`f%)mMna{2WyNUT3mFP8I>7D4MjibpG+^izf} zTBh5(XF0f71ydhUoVw+5pcEmGD{{xzl- z82n?A^3E24el>tGgC-Bl_|}PU%+;OgkxCXE~S&FKo2gQW9|Dpx&aI)?F0I=wkt# zO;3BgLuuWS^Wue_v3*a_rUwXh7f@4t;V4lNwmyDT8=*Y(lmJH4FXWu~WxHWBnyg0- z0(t_F$lL)J73k;wniiw&mk^%+Gs;Vfumt0_6Cm7iTh86S*_ z6zgwM!n&vYm<9o7E|)cenoxoeUqcN2k8hkJ{_>3_V#`?r2B%jS+BqiXV_B>lLRj(ZsIQQ(( z&?lj@Rt(aQR?Bm+IXtgrz5M>&E{(t$V7-yx6nA-YaJu&V+<{g29Zvx~Kg-VolSUz( zC-O9RZBJ=U7zPFH;o!fJjIiTh>?#Q1pet@-VkP?Q%6Prv5MLkCvpiVI#VC+dhIf&vF0<|7g>k;0<~h=ws6SjrJsqjG zxWQDQIB2$U#i60XnzM(n?&&L6lru;kXAr}2vAusi&KzGz2cc*np#0b&ISTukr8Y$+QS=mMTmmYf(vxU@X^1ZIWk4w_+8m#km8 z1n56%H}V9azaWXQi8R?&LD1Nv32>3&jk!I2%8Ji&63acS2n}jNJ@rhz*LLS+R;1`x zcCcR&B5I`via}5opuY^a1NGZuCf^zE@$11?bO?G0JlMVpaF4%b!s~uIyuoG?q{3i$XD>#Tw606HLKv={Dg-s$3kk2aSJj4UnwuO`wl`SO-Gun=@ z^6(suzBj4~?vNX87ajc}y|o{sc{}aD=+KheFT6B#m}xdkV%j&2aI*qYZ1l%cG5NKH z)8aFi_jG`p;N21o(YP-iAOxVzP8mjdch=)>%b?PLfkXS}7o2u2uI3PqFL7^JL(70{ z!ZeVdv-SEqTH#3kk4Esj_TwO+E z<@&{#Z?}E@P<+QyWECNV|0!y>#eYYt4QYx3{KI0tR$d$yS~r^kZVax%G5ZcIRW!FJfx!zjGgxD zGd)7EPk%@oS~snSD~-QnnzfN+5`b2^on+V2S1mJQc%u>65zx83dS$q=gsyXapWy;c zvDNwvRs?rA`RZxx9&$k3!*yUs8~bL3*s+wIidi6KBTW@GqS53IQGjXds{y7_(ooHp zqU?iw=%k)GJp=S!bmQbiK9oU8ySd&y*cBcV=lgM^)5b}E zeoj|5&R*B%;d`=pTqb&dSleI7w%g9Qjx>Ckb}BfL<-nu%O79og37|vrGI_=v(^&>cPIkf{8KbJ;ZI6W`;AauGIxa^Iz)H6IBsN$^?#vf z_<4sn&;{&Gd!nAM#YL}vpt&_dx$Iow_^h78@!hJSTm+Bt4rNcYaeD{2Ko1~-T_P6v zgok2%mv9NaW`$^Q4;hJphx%B652Q_B|3&wBBR>jg_;hD2BXKG^Yd81ucCm_Xtz?C# z^z*a#XzPe_BCuXhAx^FEMkXd&tIz>GnG}uL`Yc!p(GNWlN;H^ol1|hB*rP(YiXuq` zecsEZzVOp4LnD!V%P!&Q;j3?rXXLeoy zX;7}A8!9Nee*Bz~pd9Z(rSGg;RcXNwU?rFzrUx#+S#Gde)zePX%@M&D% z8}_)BJMLuIE#b(<^-MI|e3qqQ!p+@2upl84Qe`20*UmYRv^pFjFM)`VwgQi#Cci6b z6-;65&eH1H$&Un~&Pbd|ET7>_yI}O54f6(x?4C;1lu*)?`-2h!@g(sI4IPggJ32xd z$BGB_jQZoZVTYBRILc_a%09}?SpW{hDqr)DB}v86>mOXZfBs=oNRr3AcPfs0>iit= z)W!VUa_AjGLO0QJ$4pKo{r~`^j^r%P{;_HL^@UdU>2?3gLB8(rOU_uEG^8C3XAQ;k#q)fampEN>%HbQ_t_q2KckE4H^dh}g+f;2CPfDIGql zhj1onduVgaAp>0u&z~@}b#kmI_ zVGFEL_#i&ajEx<0usSnj$+VN=7a*6S;WLhoixx~@ce-YfCv7N$PxYM zHYYc@SG3$B%V7#8Kld0(Kw9j+tkU9dNR)l-Zog9<0E^zJq;1DMFC}teK7Zve>{a9) z=Qi%%_=jU>1YHfrP}dc<^z5?T45UIRA`Pc)_yvc+j0b!axfog?i?42nv*LZme$^-{ zZg@caJKe1bwX3fs{hXfL(Yix>fCQ?Fxjmw5tZS3tT6V}OZs1-a(AI-8xB0!ILN`{s z1)w1p^6=N6(!l*wpnID5d>iW!AIwS&f1 z;rBif81YtAoLSt-6qmNFM#Te_{5U4IjKL_#XHVUOe3nK=D5%$DUt}l%D+d)j1iy#q z`4H~_8-7*9Hb0ZzWaFTIK@6nkd=wvM!IaHq-n^yeQJ>n^_JlX8o;u9tNO=%iWURRX zCCPLvtC)J+O#)3r>ET%GYI+E1r+(!e3T zZpE@Ec27wu8tu+Yeo=la6kbxWOU>pLUF5VXe|sR97J^G0or{4O(_A=bvk_Pc6~UN+ z^ZMzl%a&J@5Otv|oAr~LcdT~KhWpH!FUw}oSPwe8ExH~6k@n1ANKTXb^} zT{V_o&t!LEx~9$6ILcv>!5%`5<&yvf@UluxS6YGgoh@9;b`Df^K^w z24!)u;Jc7HRO{KquOy7!mJ2&YH~)q!QIGVV%Aj+UtIlZ>hn0#WNHL|P$WAh5<|5Qf z@G=9Q=uJTh&btCg2_XGOtk2_VTM7vh+bx{a!wVlFOyo^Xsn0U-H3S4!0 z_{53Jc8Lw2j@qoDnBwmJAwC3bsjMk{QZokZnW)^=!vAoz**OqwM0K6bFD8PQsF@c^ zZ?;Br#(Rj4r6uNTV&z39VlH<)*F$C$v?VqafYeoaFpF6MWESmHyG5P)9#+{P@gYW? zu2;#7dRVF;gocZ{2E}iwKnYM{_6V7@R<|dXO$ab!2D4#X>Qp|gFQ6E91p(i>OByW@ zl{%rAl8}sQ8&wEYs{fA>|L)~__mC?`rOM%RXS8AzERY2=xcn$6_pW@PAg7U7fAj;U zHk-%=1DKQ8h)%|4e#G$gYNAL|L3u`4Xwl8k|DZ@0$Q;_97pT=S74O=3q}@w3`7oAx zGR`+D?rfZ|(p}>KUQWOOEk~K0QASQcNxT0>6dFDrb3LO;x7|y!XHio0%XR@xI>CqN z!!bVhYC-8!vMTt;7Yafaa<>W(z1@k)Z$Au`HqYB}FYGo98gaV29Hd;IJ?e{kNK;V3 zouX7Gz$#(X@+7V*=zok?jKS?_cOcB?-iat3VE&3cBXX1lBZs-MUZd(Qb@mP|o`b*< z=<1bT8E0i0Yem%`DmpVJeSfdVXj^cqEoo*piX+bh0E%CN)sD~iNsC&6k9@j1C$Yk9vX5Qy zDCmNi<=AVVr?|Z?QE=~vKNCVDAO%v~vYcpnSZIo9f4l!xIp@ZU=oD8R4dwdC{Jrka zFTz9Z7?aJT*SjD5fRgOS0uF}B&T=0%ajHw8mO5Qw>CC!JjV5$M682u@QL-I^ zzyM$-+Lyl1c+BNUXKI=emVdtKy&9Na{z!liJS{dpas9Nk;1s~)W#6;b248KV(* zPwiGTQk;@hT(;KJhzK3#oH#B^S^srb5J+mGHv(hu-2N#oDv({lQCbly6O0KpTVGYi zKk&H|GQ{WHigF!FS}iqaY_tpTyf|KSx$~kIn1vtk zobtBv`}>+h(yeib1eg+?%AoA~cfN1tKzAdR!EVnogQC~`ncxA2-^TX{T478Z)xs14 zF`L)7$kS&lmWA$xZi&Bi0w32wL>tbWLkH6d?Y1MLFLsfUEa`*7>vcNTp!hh)%QU{o zs`FQs&V8B(ZLnw-t)ptxQx7<_Y^eitIj2|Z<+9#@<_)ov-l_viMK?fTn8Ms+7{nDZ z@EOU2o6}GH6Wyq*-H#NU%Od4ELIp{F?M=yhY9V08ZE&9LKaK$(2hQjN*qku0siuPE z%7Q?{H%qhS<5oago9c9|Mvj7a6i;^)W^>tzbI>PZ`J09{KBKXk+T;6a0r~)IXBD9U zrTt*>92_i;5(D;ex^uc=bHwUWDW}ZiFL*-b595C#lu*-uj&MrF4`rIE zJM%+#i@vQk>*BUm5gi&Y6;k`tW>;dkRv~;iUR&#}7+*Zz`h$+-FDYB`SE$ zJV(^v0(3});n8d*Bud$u(VCK%uLAU_P*z$P7kd!omq+tn51Rc0L1+j`aPE;R`RrOt z(W?_<-VW!U(|wooEfeTnHAkNWwx#s)%uF(&ZtR4%nwj1P?+egQ0=4jcNhmmKLN*E& zdi&+;3c_-?mu(Py?HfXe7%c+qZ=k<$ALOpThLAZl9i7|D zt!DydF9)hnK~;m#*LK>AeUT$Sk2y#II2&uyzbP46y?mL6MYGWzXRo&84Bvi-(S7T( zOd`ZalNTKb^rO^NU1&5Vs1CIie7YQZI;-~MyASoY4le>2e6Uo`V-Q80*yPQP+Zg$-7wBLVsP`amcXOo;!QdJ)Q&h zLLbBpCpK~tZ>Zh~4VGsjwlr<2oA7;<2TsDrz|=ZYV3V^d-A1@tH$W3Xn=*ahZreHG zNAJDs(cu}L)r5y~dRs^AS3+ zi{QAiYXF!@4y=7;5UUee1c(Qv*Ro->#_+YI>B}PtK$C}eDy--bz4yY0gU(O7ZG{G^ z)A|hZTKh1-MNan}Uf6bbOZ+EZWxxqBz#yqbhQj;TTu7a__16JSB5y#~AwMWk?r&v> z?Rx5cjRP3HGTx<=4g8oKsXD!&L{<(hXY=6wT*dAlykMBuz%$39QVC`G?D*z8?W(N> zOj-x*H%AFQpr%MgZ6%u7g?g${IjoSR_uMW zD7=M>-+9BKMJu!Dx*K{@LjkGHT_fezMBbc>{x9+_`ulC?T)q){C4+GS(I(|uVm+(*#A(vO)nMJJwR{mI8pL!pd-M3X*5*zj`GTGFwFP`^(#Px z^Nmcz5$9c2>#!3~QOB9RoM`uryKCteO33Tb1w#I`w|rmQ-tup(5`o)fxfwtI=V^G9 zSZZ(|2WGPYx)e})rTHditty8on7}L@LGWA{oxPL`OXuXQu!u&0k3M<>cjr~OFGv4{ zr8ix0Fk}wP`U!D6xot(FlzijgIOCVaIULgu7fY08J#V#o*SD9_1Sq~b@jx+yK##R^C8Q#Yhe)& zsEM&^7{w!+Fowr1&V^bZXi)ryJdlkg-JBXhgh&HAvPzcq^-6D}6q7zEQB={>7N!_& zxdFCE+YR3?Z#C*4+IA@jR8n4+etr@id&`p!RvyzUjb^aHl)NKVYq3Cuh9mZ%5poYn z9*_O+oNa@!$5^`K=Zn8|oXgd@Y zn6Rf*TfNS(6n(l%@o}PGHqN2z<};+5DU9zBt7@r9p{iSOm_}^^)zoPXmd#8iEiL6{ ze1+e)=|pnU4u~t%db!FT|^pZxSt%Fnn6LunL$+Eg^!PmwJLNhk7vd!$(ZDI>x ziV13~-=s{Q8B!80IIsz8%zak04YRq3$F*|0pGKx4Z-`FE<}+F?YwS<<#qQs))1!pN zLV3Tu&eR#hb~W=Fb@QXegh!$>310?C=NkM$_en$D-0DwZa$<_?qNmL|G1~tl?9BtA z-oO9xmn2G(Hi{^Xt(2{7iEfHUwvw_N5tTLjZn_l;sToA1?P?*}E|P7cXe_xUOO0h@ z%f4mH_C2qc#=ZEwf4}D+VS2uvujg{k<2=qe&*yXW-DUJ9As}`$;#7YimSCBeBFYE5 z?akWoKA=UQbLR{GefH8m2RO6Rbb<*vlOz$JC&vs{0<~K6<5N2aU}Xc!TKJ(Gf*}=~ z84uFWpYOm4j=f)pG+W8Za2{ntQnw{Upx;WM0@;2TC2vUYm}i5!hP7 z{)iJpu+S7wRECI^wdR_cj)9CWB75XyRv-q?bL8PI-KVF~x1g8PFZ@4js=g2NYVh(& zMZ+C2gi>Y_jG{;hw$XB-7Detu#l@p`HN{W{%_XbGAr++ zqr!@86pK0&{d`5+S|nBa?qk#euAINa(d?tv(djRlm4DR9ayv78gnE;hoQ+)SYHWza zITe}lqZ~-;29I<9c`9l^S+MEYJF`03XTzX)SmC)I14t4w+{6yTa~5rj@s6mT8C`jW zfo~;i!hhFAcHNFSSeAm&-)}$PakO{-)2i|N74REDO!=oAx2wI&PP5;&iJufggY7L@ zh}?2v!Dt+ae#`{jHA7I!sz9f73rwkh_cv>uf9L6fSl$X^P4xMWTcG}CUm{;8VbDN0 zI={rPvyoa z99Z#Cb|yD z^|m!bOlG7$?feEg)I6YH2><~wuaAO{CE}FsFrs~j4@tXhfK^aEoKWd>bMLaRb85!Jv7vlYOvemEb`>hzOseAv$9eC?d)7sN50vdzO)*bH$G22Sv z#;8BBDi<y6^iKj4(+2An%qcj*`1;T4>ybiOvplLdbiB>B&0k` zq(w3E)V@nqP?w-NzZ9+gyS|%ZL;ff*!&naF(q$el+EWh1R{iTX$xEuMXIpFvN@wNU z?^Y-Y6l@`EwKGSXSUKOsvyZ=OcyhBQpGAV7KVP1Q^c+VBhqN{)GwqV|Ik$Q?HW+pz zS4Ey);hTJjs^3*GhSn=V4FY5&s}@uy6U@qd;kjw-o&N;Bal$=uy?7?tjz3U z2>xJn1;heB?UG`V{JDwREHiDafXRvtyod9J%d@c z{^#|%?i9iA;~=>j5=^uEBR=8 z;8oim6tpg3wR`+X-b0_3D#vv>GRB3afI?3QaWa%R+X1s6sf+;tPe?}a4|b7GVTKk_ zi z%tNFIPe4v7H!H^7-o^gh{L&zZ_-EahlACuUp=AZ<_7ha?jHt9bD>QhL3ts!sJd-fX zO85(t{nU?uZ_i%YkG;vtkrZP*A7zzjz=Wt>ba+Wo~#Cf%Z7s}9&yU#@wB zZQKdfK`c~&)Yjw%M}s-MbcFG=_*YE|BB`McZt>rgZM>>~yn=VLjt~5kE0JJw0w$3V zH~(&jCiQpYrnHLOtQ;AJ-XT$9GS9fu_E;f2ds(U}M%!*)4p-a1oh2gmk#vT{Tfg)E zN8epXH;$HIaTTQJyYO!a2#Z`uc6o~F;h8DVVUlZW46=0u*wN#kbn@9NnbOno1qU`` zw>lwe=-r?|?c4>pQ~-*)zLf+mSj4>OMG6U#4y@GB=*XPKqagWkc)pZ!!{2x+A~Oq z`N{Nf_kpN9CRSwr{jvsw`qv%EqU8~KVWz*JoNJPN6+BIt8E=Dy<(D)K&PmR=ZK5!ien zfvnW={Z1!5Gw*gG;xxmfKd+O|++ZLQ!StsPJMAeGQgivxp43w@Id$AE_liV*Y5Ic} zSH$gE6WGq9Ari1rA{_3bFbVhtF>J(g_&y2(skpl{@4ZZE27L<}|Vxphq0QBJUFEWE;3fa@7-rLCIfrjUP59WEZ=Rw~H? zpYq`f<3JcV<)asTu(nRV=F@aDul zGa^14x#m7r^b|-se&C!~6XTWWN|L}F6Ww_NC0+(d8WIeK#v~3*239Cy5J~FpESJk3P08Q}+@lG6~X#AW~ni&ZDY+OM17wrbqKxiwF zAwV-*=zalQnsu^uMGU)4Fg*|Xv|e7_Ol`1r!6O#Nse5lGj*bkRC>Q^G9fmbB|8)Q=lKs6)YQ2t4T|sE$xd@vAA)$6Cc6BNgY%a5`9kJ_$no$; zf|>NSUkj*J@4?h%|tH;5_lYK#%*_U2tqa3zU2heFqMS7R2% zH_YGdp9gQUzV>|UIP&NGJb<5;`{-KRhlh!7r+m;jp3h-N@b!e4@^%PKlvi{2%xvHa zfb$vF+A=OSFPp`E@z!H3w8A~P?Kk>Qh+&3eH|8Z1JH5Fv_AInuJ6FZuw5HZC7CGsR<~l#yL8YrBLeRa< z*iH&C!Q6)a&6$;Xwy_6aFwA?Zn6!(C;HFz&dj5|1eoutHGsLl)R#kBbk4PN;)OU&R z*gK$P+N*^u@5aoyKb5%ed%j)?J5@-$iNzx5CmbM1TPMqEN;3&)v1K$%*gnd#mL{y1 zOa_BKn;7vKyz_wJ6$ehOVigB_FZ!o(j=fMssJ{7Q*rdoc6f(l^dwuE@w$GJ#)7PdR zdOdx0_Rlh}I@x`178I>oODomrZo($^{uSX}9`|ytTPvj_vIl zXO2ySI(7mR7lc<6wxR@os>sw>;DqShcCNX*F`Y9*t&(yE#>HN@;Dn+$5hazxviW&< zR31F>$Tk*rVtN*&cx?o}rsrvdvzZSYrBy@B&#-Dr-TW_EKWhE07tALlxx3Z80F{ru z109Ks8L4PUI<%>_R5#T7&~%&wFGuqaRk3!BV3-TH$&YBZ@`prfzeFmK&SSLjY}kxl}Y@g?gby*=djUBo_UoOv;H9=!9NDFUyCL* z)bOv9NRo)-yJ_cftZIdV4S+rnJu<*MD zsQAC%qQn!6GF{%Q$+d=J?=w$Q^nRYdT6jFs8+^FE38HkSg0H$f&pe&;#)3+L`wD1? z&cPKWZN}$2no?)v!DYRYj+;>+A4HR9U$9v3VZVJrMO5~TdBve$xoFMLwz3|X(VA8s zSrvb7UqYhTI_CIpxNqj_i*ZiEADc>QEPV_H=~c%#rAIpc;A)Q))QCSPnsk3%_}L?^ zEcKn>{yoQN0@1r+?RBy;-~7VTrPRZfSW{mC4R~-MLYB|J-%32PsG(dzVM!#(z6Ps;F`A{*%dMtwbrs(94w_YK|;qMa$k0{qSqwhdB0wP33 zz*g{0`xiEp5Z6F-mi90SLzImBklvTncjfdAsfb(Gesz8E<*wN9_kYHGox7}%S7*X; zKB9l*`L%R>?KKx+Sh>yU_IX6oVPGoc2QU;_8^&txmOjvXsgaY#8PPeT0q2Y%-mh_~ z=3rTc7py`yrOsKr=x#4(xXSsCO5Gz~qiMHKZEg3o0yO4Zd!9UYz%VoR)Xo3CL0cfO z2)_ruj%L2-MppEOn7M1^8+B?J5mQ_@a22i&502h*i@BggS29~dDgL5|XWotA>>B8^$Gl#K4$uCvwSBZSV9Y;LF-E~lS2E|BXVd|YfjF221jE;o);j;| z?O?o61B{i!kwD6PJkS5Cd4mb>ib1zupPyFra?j4*WlPFFw8^l_mUD&Yn@7a{<5uFI z#&qo_?m8El=i}X5>r~KU`KhfmsK{BC`r@$>G>ef#yMl86^Iu-szp6w- zC}<&;RbJwJ==>~`74gl0n{n1Yy(x0VCcToNKWlb;Qg+rjqJHl3$iv&i65*$A`smII z%S=gAA(G28+rV2U?4dGwu3kw5y|`!|bt>ItHffth@0@QOK{CrL3f6wt>Qgw~TMz-9 zXF}MINzd9$S=8aN87S0f(+Z_yU~$}niq3KLtqa!~vkyF1HU+_s`0lwpUVecGumQ=$ z{ud#r`BQ$2xk&M@z(1)xX0+Khm1{%tda{t-I*j35{qY&rDG^{STnpjN+0xPdZhC81 ziI;U*wrifg*fGQ6`8ejYqU@Kkt26QH;{kzxmWA&Vm>wzqv#jdScyv#x#^CIk^^*$W zevQo!V%H|b5?!oe9Bb!iE-GX8aRjOc42prw+KN7t-=toHQUQa|Q)L&eKW^R(2YKKK zMuhXuwRHBU4+6HTwGOmLI2MfYP8mNEN>#E6-<*$P+Oxf|TNbHjn@PN_;%YDR@0A?L zgXm-L#zd-4g^X)#BC;IZEP6F&2eFD;@gn^kQd{4ECV?g}*Nx)+Z!o3|T)z22KUrNG z|JSxP--*XEOk19Sb9`sj;MO(yh{0K#E!|LuJ`bloU5sE}r}b~UP#)w!VeAEfTa01l zT%E^1ald%0CHDTn8OjlkcJYCPvcoU<2A@|bIf<{J-0L3DirtD*L{1D$4ODDYTiQ@N zKEg1##Hv%P^s`ZG8&xuPJ$#*vi~2UoF1`>X^_4a_e}9c&#`X9zj8eYbTD{slU@)Pf?A@;pUn;!bIR`-18aclP-m6 z)R~ky$pLV`)}ihf8j`?@XL`=_M0dcJq(dtSFCsDr;d*TPfE_o?j7;YVsN#u*KZrH+ zMCaw3g(s^}7qtGaVU39gi#V+-l_GCv)A9GOO+*!rQbG042!X>-G(p}R)|K*Pek>&r z`zWxD1)6Z?HUoi(ow4OFFZtamI@Y%nDTmwSQ-+kdWtgYNXNvSkPAjP5N@6eBX5ofd1~FW8^(k+6^Ld9{;xy6rUbGKMo@ z)6{cl#TUK893j46kn|Fb%kVK)QVksbN}-(>d+%mQ*&U~G|LsZZem3UB{qS}5wf|J; zLLE#bvBmD`C2 zM&J&gqW?B=c?*|cOPfjj@24R-_^Jvn4;F(H0;wAkB6@;fM<`!4#5(ZUX$#TKrPTzT zL~K_%LZ|)YUOx-*L6Gf(h78L=akj$JFUH`@Q4;S*BUk%^KWVH`idYp&a}xjV0S!4n z*#tu&reJOWz3XFO5f$Xx!Rst*~DRYOK!LrwW0P@}>3w?-3 zZ~BE@k$%C!b*41U+^n>7%8U`@DJ5vT<$~ON;a`!;ANc28nN~vTgBYJ8;z>-TCTkQ> zV^t7^%|ZYgS8%h>sLKrFY-2P*V3Bs&#zjj{`GyC!Y1{ebxaVQ9?V%BtTu5z2(5ah# zI%osfIVECgeh4bi`_}$OJ%m_uue|Wm_?jl7o1R7&_x?gDo>l5G3S6r~N$B&(0{pEm zp+Rt8cFfBP7*!T582D!0lv&v*lEPR_G+2Np@V@IGlt_3T#crTp_QCV`>h+pEyYlUNNACWYQ_9Y!cG!b<9v2=B|M{8mujD2507R9i@vS8TVS zJg*4dNIY=)jE2GiRI>`u;}39|+zd;=e1N6n@tEZA1`vWN8cyovJzk>>*IiwL71e>1 z-Xq@)-ZU<$$d#d2(IFiVaCqmBzLXtX)8}8Y5Q`Dl!bXPLstV1-&Tsn&jg?x#ti8Du z&k4U(gzbPQkKolvswfJx#BHW(SfB||b_5vp@6scLMpmrE^&u_=5$zSJO^`J$c6;#; zt+L_;j*y{%&IaGQQa-ey^;kpq8jijThnN5w1z9*cZ(yzwtE;%$ZVhb@e>x{~tfO@x z^76`q6Jp$}04FJen8mrY5I=Eo_|E7-YJs$i`d41 zekcpB10W8Brzq)|;V5}L=hSC<6fobi=E6c_ni}_~KpdDftr2*O3oZdnfci^j!y*jwr^EJOWa%*I-u%}0&6te#lAlAuFq_*-{#WwEvO zYtfCYDWUAd&2USG$!ZWF;(3hV+oc14hjdsmMB9HbOa~HcZ|z@ma8H7PFHgNynTywl z0uH-dIfhKBf+~<9x&{W9aspEiyT?zM41w6f2rg8HWqEgkQbQYR?eFBh=vvm= zD_8Vf6kXdC5d^dvsVhnhu(B&E20CI}QlqE5CbV)gmF=K! zTmHIQ1M@%OThzj}{Ih0O5%Ablq&j2T;pPl?Y5_?F=ev89(Tb$MjLfU1^J499#?})r zcd=Ip9(l2rbGvoNt)qneNOU*K4nbv}%?SHnXl#p^iRGycSmn5eEay#F3>%}VP;=}4 zWA6%1p4AyzA$dkbxSW}`Cfh;=@C)|+q?M(pt)i^Jf+7TaOvcZk)YT0QEq65F7>zvK zU3T~N*#aaKZWmJ&+{Y2k`m~5xc@jo+7Eb7dr7;a|{x#L%@7e@ri|B@N8-vK-sS~t~ zX_c}59XU5#`lbfwd~W8W>pilffcbk z6zP45Wap#X6*>h2k)OLhs<7)Qp|ao8NU^kb z3)ZrN^kc&lJv}7r>!y1-=~fO-pZ)74Gi!T{dj22<=ys)P=;?!$Zd61E(5&*{2M%Q~^~N~O&EYutD< zLjgo!2YO#1<*G31xchQQkb9GVm0WRw*0DmdAl?6}bA|j>2Z?4ct#YOfeLFx=d!A`l zXH;4zqeC|Nx($)byZ4w}7l@iaRUx#QhG*tv-|X(#R{iFQM<`Lkiln0S^{=^!nekPA zY!sz0MiS51IPx-@v-*nm1M*g#A1CGgt%j5oa+XM@{Xkcq^FBH}CjozfDtb{Xbl zdkdxzk_=;#RLkP%d~yya5`AB9PiItXrY$i{y5#+(QhSAqX!Q9%c}5Ok_v{37mNczidOo0URY!+@ z^60nPD_a%QTpbjQa#z(GWY!K{_Lov_&OAm;9v7xI@d#vHg5Jr_i$y|-K!hJm-4p$0 zc^)(`ft88d#hfNpd2A_6E(hWpW|I3{t`6etVx&^dY`2?_Puu=_$%4>2&nLTe{e_B{ zonQ+Xa=MGDkQYc8h{ZlFuHP*o6SuSMx$*GGwQ~3NcZ?wu0|=qo{XF%(_iuf=pTo&S@KxHs(m}xt zRzp#b=4$-f_1x0$uJNob2Sti|m7ZB-w#K;1Hp8Lc6+F{&ei)EhgIY3}flhOs<*+WT*r zl-gXbhtnRUJ-n*zaA$_I^bT4>_6j6Lnvqlix7o2lC_S$`TwNqzF zXUp8H`A)%J0b&litVFMERV8DMiH^6<`n3jGVxxqAD5eLcPm=NyGtIWGq_Ex?nTNWU zHAYo1^Zj4SaXxx%FfhdW7}lfq@gV{r65q>TC2#W<+PH5^bC;t&q6$8TyLYrhgN^yK z?R`?+_e%Q6t%zt2yE2|r#Y~q89cl`wF$%X$u3H6WlER1L5=ALbYq&MhaqNtri$mu^ zgNL>~As<|&Y7Z;*z=NmWeA36?T zuk0xS4Ma1cC{XX;dSxZZn*6Q>cp5wqEF`5U%Rq2JA^0pj&jk zFGf|gteHd9jm)iqLkcxHSCD{bwchvJtn&W0_mRBMMwoM)JwUV{@0swFSM#<<@oH8_ zD&(tCghPGo6nMT_Up%AAo+T9+-qbGY5Nk}0ZKI%(Ewm6h+^jB#exgETf$~g2O2omd zDXZ@sbUyrf=B=vYyZC|=!S5En(pGt{`ZL;!N6dVM0C{agncN2K0*RMec8DTb$`L2>n$Jk<@ zxfYvs1rD*&=3=xr{eB0MdV}xo#~)hBqeVDG*))07`=(BjRuCw|J+{G|l0x%K$s(Dx zQims9fAy}W$a)=Q5#i3P)r*A?2QhqOyw%_2wr8I?EQ!t+1z|@?yEL^BB- zV*FDLv%lX$+`A*{{f}fiGo|xI=B?6A1yahG;iIzyGT5R+8ghB3L(aRm!U2h;TRRPV z`f6qzKwslpM9!=7C#h)2#epdJRrSG#zE3}Mlz--Eefvn_smw`+v9bF*%f7VrBd6UR zoJb%tc_IKJYlFsT)aL{eHwPrWh8F{dv9tXXjO=tK; zJkHDXP0LHy;-W9m)GUb76I|=tpDH|nX;?`OgQa~!4&TfDXg~2_>rlkFf_If0GMmj~kNBr!eV}N4S1A0!*Cv8XbPzCOf9>mRk$-o7`JnU;swe^P2dNkz3Pso|zCdH^9 zzb#P7tRS`xPm+v!V(^G>j!}ZaPi?+(>`;AnrlWF%ry9kT8JV;N8z0@OvxM&lIwf6zm+p3LzHSat&Qc-gp&x zt;c`X>p|`|qEFc#lGBC9UYi$@eQ%WF{lyxc z%-&dBB!S+VM50qK$zL&E>Zx|Inc;*`H2Dnoll7qSgLc9|G0Lv}G#b;?khY zgjKl&Xo5B52khiPdtGo#y`y4#z0U9(*J4k#JHTT-(SUw*7>uHN>t6yi#rb|ClH9;d z;vSNzNmBBbJcvg>j;d2q#&J+@Vya%37-a3OCAq3g=h^Xoo+l>Sa}8;dgf-Q-(!R+Q z!}CN(RZmjxPhVf=*RI9XM91kt5X|~Z3A|C&SA3>q8K$n{wq=P|=NBB(|Ng}wZs76{ zFP_1F8bftEQZ{n)HL#sj&N>_&ndA}9npt}y2|7u1JORzh#=HGEhC+^kdQ!pV5}<#C zqzDHCP1fE+*@nM6Pq_6htMUHhor6BfqrPK$$ZJ;1zmx1~XK$Ttivq?4vvK{wm9Uex z+f{6*J3GT@S&gw~I=AkMWU(3B`ttk8&P<}9$IsS+?=SZ(c{CT{;3>l*DxnFGUhkT~ zTS9O9vk}-;Eo4j@gSE`Q zp+1U|9f;Fjl~7jhNC%!)^2ZCX8g|z{bNEzqQpW93E`k<%)%p*Q5H@nPQBS4?(vUkO zUt%5_V|^ro|3PgswA~et>CVO!631);wcw>^~M3Y zTRAh=!Dv{4vr(>|eq-UGs5_Fw!Ju8!cXzcCoX@RC#=-sgmeqHIz9n0U_lX^_FJO6D zl%Essil!yx9ledgy8SdMU$=8)Xg`(Zn#kmqk_=kGzQ#f8HM+lAK*I(JnrYLPtQXJ@ zpk+E6=7v&A&g6}LB6s90VqU}3`0MS5obyUeqfTH;_wK*mKP%@hm#^%{{rIQi=Kj)w z>XCnBu-m#R?(5^*jJH*Z{#5S@yxSV1eb&##9yTXavccpi%p9Ih6PkYsg5=eGntEjw zrJcN;o}|bX-*zY-sL*NK(Htd$`=#Vgd`UUaqlWt{B3jQo{+gB^tjdSO8~3%+@KsXK z0-+pmZ+#vQ=i%LrPJegvvOa(h&Gowi-@ER}yNafBu%ZlVJj4@XGf)^AlarXKI3_^f z0|YM1eySwxR$eL3DiyQBZ3N|~@=ae|C-rDq<-AfF0RcrpGqd|-6aH8;G&$`9oUw(AVrBzuAtY$7pX_D<*HgAXooMn@@EH#QQ1;lR zmLsA@4SaJ?kQ`tJZy`$P+aA((w!01YS-{&qxRdgZI~%-d@dS4o-817G1pqH5KUuMo zA<8gBV(g`4Zu~w6v+;PI_NJecHlCtQns=xnV*9=9W#IoQL2NqdtNBGH6I_ceNZ|XL z)B}hNifPF8-b&f^G~_Bemy7~8E?(hj7o>U`YVRx?3Q&8Pl&m^_&^n^m~)Wd`lC8piH>d_UVq4>a4z8w)XfPLgqiZ zOUCPY?&fWZ0aq94|8o!*`@@TL^)rV^Qh`F&AJ7uYNW|oG92X)HP>{jq)>G$3LnK-22h1hWg#lo4X3p96yV`AeDGb1;W?IY#zuN+qjg9B{ajD;}9 zwLD2$L$9WX`$HV z0f-xRaR!QwoQ-q7VXOYXbX=hvPvv!mCc9jO4~K?5=t z1U6!#?FMWIrMT~IOqF91xu1Y8AfrUzH|oh{JLhnXtGVaQU(RyWrm(5TyuQXhou=^~ zyXQ%~E@hj@d0WHsUUNm~#dQE)BdKoJM82vW#QOtm>{k;#>o@XAP6Gl1l_%d=D0#Mc z-u(FAo)5}79NVpKBQPg7YfQ_^wv3c(Tz9r9o7VCn_S!CV6_gKqlI+LisXd=L3WXJn zM#JP*!>nG_x;)t`Cc6+jf{K*0y0+Ub!#k}}xJb>pCK7YirWJ&_90zifljQ~$kq-<8z$>4{;Zg^h!Zx`;CHDVGd@tIr(V&|nsH zf>Q^u>N7_ve)jlq#m$A9zki{Rtk#h}t3%4$*XU&Q#-b$`KIhNbdLF!j`FjBZ0gdqc z$a$$K@mW+RoWm<_$N0pzJ#;O`lgOZzRen35e|v#Q+^V8JJHBivPU<)RID#R<$u}(& zd64xUaNtB6o$j?|Vj;an?KkABTd$exxlBTt1G()&jU2ZcEYJ88*W&((JfAncJSa9S zDh=W1UZq;AP#qI^wV&}oFv%$hL|k~Q>VU*ilrJwTbpO>lML~yjoDJW!Ok5p@|5G;X z3e5k^(TVrR57)OXstkY7XQ6&+AI=MeWv%W$6>lB*EFUl^%0XUh*qAgkzQAS{9(^rP zL>|1zFM<;EJRs;5@JKc2wYGCiTh5)OJ)+c&f2e2?_^LAu=JNrWF9pI>6M);MU4d6| zB6{x@7&oK+3ALMVi)|)%QqOk;zwWM&}|s_7vSFsdEIZjA@pBWM4zA( zL-csuQiLio?`c-@Kp06Eb7e1+R7gr0Xuf+<@tn9O(&FET&Hmpo`!C^=+|voX>&OuQ zvAD6QD*TbE0JVEO%gU1qq@w zz&CJznD4)^*L|zzbVg0aPlC*$bh(LcFk_&Ra`;=|k6VDfSK5@o>t5RVqWXO4IYw2+ zPeR7#sjH3*fqo6z?5hX;Cxu%4oP6U81-O+QQ=I>u5 zC5p(Y3jeywO%PHzI4=v6Esp)Dby!sFUZ`(j=j}8kfohTGVfqMyIDjGId$31)$azs^ z_=8Y#v%OgD8-7@j3XYZPo|VXs00_x|sFL!wpeeb<^$_AD2wrrXAeG&AsU{*0py_?XN+>K?!m0Lp2T*tD z33-~-mgcrpuaJRVN~>u$B2;j^Es4VveKb03f%QYeUTbi$Krquexwm}RFGB*kt z4vXIj_XsP!nY0BKr9=f)<9P%sQU3+r-fxb|wq8mxLDo!2VVlmXbL~r>r1*LNEHL+} zk$VQqQGyh?el^TRIgL3)Z_dFmV})UuV)9-Vpj(RyrEaik&}#Okj+uzUK8Q!^YLk?V zu}2>OkLcg@E5PDUN%pg?Q{t^u<(EsvXX0L2^Gr4X4w@&!>5!)j+)PUR{$_WAwS-G_|X(%+^$b*9r}yEcBu~e z?rDE8>j5s{J*zuy?cc)SjW6fzeH3c%D~bm2@i2+H%imhd@Aokvphm6^f9r%?C|B7~ zdZD`r_z5w4(0K2KH@y{p{F2O$r78o}`2TW$Y|8kr4GR_C06GFms2B*FC1s*jDo%F= z7ULkP84PlfrKs7b>(2qKpOA|ap44*PVBNBtil$D2wU?qW=hypz zA|q#ghrMg$)bX>CZIxJDElU&PrZ`K_{+z}lR1!<|o5j({|1z;q9sg0_9{>V=JRSJq z?KT{>HS-{dqMc-l2f^K6+gT~zwRn-s039c4U3P4tdO(9wZ6J7;Nl$PGZ9SlC^*g}I z2xeEJ^_k&i}0rGIk?Uza&}ALH1rYLO7Ye z4v4>Yib{O+o{FLr{H+o)!2sOyMJ2VbOm*fLEG(*Wotfy}6a9xVEFa1&wf%&TzTQk$ zG99hFvW#v^a)EzkM!GM+71Ri%WE5 z`f76!ihYnZLE*FplVatQC~^I*bJKBFARUVzLn>;XX#IKv5TRTmh+?n8mK)GzWeBh2 z<3#Xd930jyTBPD3t1Y!lfxp7=>$?=c8aYdG>pN{V7`VvI2iEVk$)N*H!s$R~HZ4>0 z`hVte6v(wCwUCr&h~Cj8>65^FiyYdL)9LbnrGi4kF*4ZfOC@Qd{E5~BkfePih=hX~ z!w-9+0zr8REJRpg^cP~?s!w_{gxh>(DZv?RmQ_}o{0O@+!03lRy3v)Mfr1(#36@p! zE!8hJ&=3NoA`;8cn?92ztQ1rDojebRk78PJRdxMZ8}BY!aKG54T9f;D+MFY){ZCHV7SEp5(bLDMaoMRf`uJ&n|766DoKd4UJQ*sKhS?o9Q`ds z|D)TE00g(5CmmY;Mv3tYuVf&9KgpAZVZJ;hHULwIj%ZBGyP&{vqa-`{_SKSe5T5I?J7KZXeknTH;0;X*scd;ovxpa ziJFKQa+rT4ER(DmnKWwai>6m3O|l7Jx$O$j0~qvEa(llpRRuJ6ld@3ESx5HeYBb|^ zF1A!Zzfs6gD=K&A5wDmwmGG`nxDZ(Y{x&DYsZ7WDk0DNW{Xqc(oaP0LnPvQ>#K-dd zxOmzzyM`8v8^O^meLA7%{hT0%VXiysPHwYKZ%8J^ zi70IEgA_>pL0FXDo$`}OZiG#;_$@WGWOOUWqH?SBU6e$MH*XQS8dqmo2xL~mF}-01 zlr50nR(KSaGYj8z#!+VZa<=YL$`MJ-&h56pV-;^DGv))q1ZkKYFVaq>adTnf6ORXN zkG*5hI4&1kd2JE*Mu{4l;=hU(HUoaYQvPo}6Zx%Y>zC=7Q^Pfp@zcNcZ2bq&vwBAe z{GY>{_wA#CkaO&OSdN=h7>k8I%R`?fIIoNZ z$cu#*xlhcMx13n_?-8!wlO?%tqt8Ggs%SoBtN6C+dPrqassN5caM_Q^l558C`$%y& zmNECx>Kl!m>Z4iw=XP>Ff>%DXECid>?0~76`fO0fTT$QR&dRLneTVA(@W=g72J$zwe(ydnx zu0|S1M~VL^;IFOXgUHK{7Z;8PN$go93h)OpAXdKEBB{wCDcg|bMe4(b831z|f)gN5 ztGaGbYZ0WLUgnDtRTL+C<@o;Uwm)=K4HJRL*Xt1qsxBG0yLkW6EyR=cIK#E(8q44W zDX<#8-h7#on7vbazm~JTI@~E1E$&eW*q4Nx+W_ON7X~<4E`LaYn&&VZrezq?r!~Vj z^3}r|^-=lruq^(R3mM;0+2p3$Ww1$4VbkFUS+E?HJuxgxb?+TV5P3D2-if^NaNM(1 zfltPjsq5qN_M@9h7L{fVPv@11*|FXB_aLe=gU%1)M3BXsoptQ2q1^{;SzIRgi^#i9 z<%3QN31RniP`$3vv}p#=pM6-ufkqu^D7gxGc%TALl_M>CMR3)m&U6>Zn zQ{4EEiP?K>mKr?zi@4X~0jKkmgC?j}ImUo-&cD8@LJy{&bK*Y+O%$>(bqWW+2=A+S zFmH46=E&+$cf$Pv>HkJwCGHk{0UX9oYWfJfJ<=z?Wk7Y`Pkto8W5dy`*+q#bY}|vV zq4&aQ?`2O&>!iEEsuLZDm8mf2aPRGsVpen`Y2*-UI<-LklW?7Q?d`BIbeULO$jmO;>n992Xy_9I}bqUnR15) zyMprfCT1VBT)LvE;gFdh-J8S5_-Bpx@r*2@Lt8MP#Me?!H&v#zJ0^ZCA^ovJXrln5i@Q$sAzc z78S~)4e@PX^%s zi?z7idjAggqj&c?v*`+ZIYM)E7GDjZtxd~AJWh6ybYut69(9jz(z(N8v6Yw@eQugrL%7UP%M!j0K<9ftt>4-b#{;AoAe!R zz)AU=(FMlOK{u$Lj53i}p%{^`MzG0YUDGkr#ZmY38jL!OpA)ho{qNF}fc(Y*F$D4} zq;LSXo$*DWIQu&9E2AFcCw)q@Atdd2)4m|&JZW5w6(L^5{H*^&U@75T@m8l%`q!rf zjutO`{|3eE4a4FJgVZc4L;(>cs;IuB07@*f@mWoa#M2k@Q{ZhSHEqFn z;AQ)vqdMKn)MF@7aEOD+TwLM<2XWJXjAp%Rvc@+FeFgsn8qat|Ui+3BiS2UuZ*@xb zhWKkLCmA9;Tdo5mYqr3U3NSWksbUhJi72f$mwM-vm|dev94>yK0_4#5z3gwWnwtiF zT3qO9kY1L9IRL){y04rAIO|IWf}r{#AB>gs;@V(?{Y4zsH3JrMeu@MC<2dAGzf6MG zg(%|`1O)rdf9+cO#GsOJ|Jvph?_3ROt>GrLt_LQ#j?1Bz-VUANOl;~;v@5X#6UhGv zfAo*j*(10Vk{x{CG4T!%uvgv(LHsP2Yj-bu3w{Fsug%o~OZQ_Qca@j+srT+iA4=&X5aadniI1xj%La3Ue*&8NQWNz*Wem6BTf`d8cyt5D)N*TNQ~R)<~N>p ztcLhzemA(tTCW!E6EE_4Py0A;+EszV1u!sSDyu`ysChQe45n zFRRvBs2;Px6Ty-$E_|}k?VUBWHz@FquOElqEO*`Sw}E9@IBUM}z%4iyl$ZM3)B<7k z+zH-xxBbcd#%)XW6uAXk(|WHkpsVew7E~9e;ea&)(Q;pw8pm^;E_C3! zW%_DUoC1(iJ`m)^~a#!p(?Egr9Ns7C-8X>9=@1#zyKjg8DQpo#nu$rRv@9Zex#6Yd!zj1e2#?%fB!<9gXFjR zL+>er{7O=uR(d8}ef%7=wR@JacO(D@b^r3t#XCoVSCenggxvZjOzpu|J!YADO>BVq zkXCgCSw*Q~A3NgYiotD0ci$7Qr9whXfjtnrAnbF-F$BUk4|>-kPJ^hay)x9@R}#0h z-;`|m%R=%FH8k(wjo{2-VWKZ>K=?Wpsh5I7VwcJG1pmgmKQBL@k4HuGA#{Qr>KI&K z770QIcU=>)IEqCqB7tLZX0amm#+aea=(e|CMuX_;0B_)6f)1kbCS!2*i21NL%_!eA zW-0kVCJaUPv9;>Mx=0CH8hpA0P;5ypq}{ z^%CzgP?6uiF@&iq!8@04MFhU-pd#XgWZ^q%XqwA0R{k_}BLgJ|C?SBQ7eMM&7s8&0 zm-d9Rb7TBUv!lO#gIFtD7KEUarz5>mM1o_fVMX|uTk!N_`i^onwEfg$Bp+Mw!K}z_ zZlmsI`HfqbNtOH-=!)6%rZf1DBopJ6h^N|uAKS9DGSL$Dx%7si>yAd$zZ$GUJdc1M zi}C^Y)_<0DI}OgWj%hFOLnAiC(`r8cvT)}#;~AnQu)SVj?z^dmB=$W_NH`b;@L_=& zvM$4ulEa=G>TOjwCH{6}Ga&(zss(f^R^JheAuw8<|5bAvn^YNF=jFL1K6nGUR$ z1dv?zc5z)G(o%|PicZH-VGU;pD+e3m`!)#Isham|L}lr$Afcu2YAHJtF_6uZ+kp%Q z{4YlqfkOGs z$Iz(#nN`Fmm;;X5SFNSnR*=CufVL+P#e0c*2Et#Bc#(CrC{%e}1be@@5FKIVLyw^T zj5sQh<1^$)U zY$l?J%N-!4xz6VnR|ScE=JGuYd4PJdYi)nK05iQi5xP1WK7SW)x-=t#(7Uu6c<5F7 zj(71-@lA7Gs&KyK|Bjdw=juLg=goFIf0=J~fVbVs+49q7gI9-7vqbG#A6a14wPs5% zQNuygeTUm&_ro5C-$^iSd)?%f8@=6Ur(rsdM0r16xufL|r_1YF!7o+fZ^09*E@Zwm z+6y~`k2_D*HLpTus}!jz$x+0vl z>O|5q==0oG)w`WOa51x?f-Bh~ZVUb+na>bTXj9?ntim_Dy~|qHu}uwti=i&X6q7~p zuaegzvz9S!`bq7}gZKzR4s_S|>Y4Jy8sLG?aM|vwDOZd{axP@X{cYqhjc!3(Tzc2R zUc^u(Z-SbC`zkv1sAtvz{Wan(i>&2Wv!XT!_Zi*If2VrrCc?f~ zVHsN`3)68W>kCfv84f=TyD*$BrOOJ;B7NDK{VmK6?{*U{D%P|4yq;Ts_Um>}*NzUN zPOxOsJ&UR-RYT87a1B)!cBCB1k>uzThJ$9aFCaNW(LYk*b=0d_&ZB4diCAK7KXb)@ zg$Ncr4~IyWyQXv|)-(#ibI<@G^MjPvH)hGuT32Rb#!Yz@(-a5iIzG*Cx z664#z(X1T&8FP<`+r>XH+|DPcha{emg|t%%c72j!pQiLaUbiIXU@PhI)|A4$*hVi- z>cC?;4)$VL@&I_xdMEJ13IPD8$SC8%MyjgmTD&W>>}Hr$ARD+R%C5A{ zt>YF#;#{tJS(cVjZcC9<`Iv9goao!XN~qKFejm4S6v@@@)tDrvL`*GpD(-Jx0!(wOz@s8^ zE}A8$ZSWE*EjiS~>XoXTy(eNNYioiq&9&CS@{e6&1$2wzQozyd_qJLY$|kW^)xdk~ znTG|s#OxWcjPuFbejP6X;!#`Zqz99!rmd8oXao(Ib-@m0k~^{e*+iJ#ir>L>Pph2<=*hv~Zdpr9oVY zxvR6^uSx)mOo4uOIgwxI)6EIyGYUz0^U!rr zQrVLfPyYZ`eGy$678Su~Oj7=u!A@}*1#)D>Y}-0+8-yXVrXBv~M*+-iX@mjyak6M& zFU^&&3}z@mww)nQs}S71Z3bKlSqBhn)IV_Nr>?LpwgV5v>ZGJ1TwsytK%c+6j%(+< z#*xqek&{~jrA@%wjb&6&{|kq(UOj$f^TQwMFAnnsL%NvsCx^T&xX(Pc9Ex}%~4iQuSnnjsSfu-{G1pXV1emn2fyePJCqN_7nJJVd6>Wm>sQE)fqTNLw?h>^ zs$nWz$AR!?l8?nFQ0PFHU*#$jRnYDmkS(>A$EQLEC>bXm3P(il$`tLFh#~SK7S`Y-;K<$RK({lyBu5&4+50DjhGDY*;_Ey1AJxOux=|TRZF*Tb5Y> zK_)3HLcl(osNyPu6ip)^m`otqxo%t)g%l(JuTxKhiyZ{7K^}eAgq|z4wf6hZqmpkkb^D*%US`0qTb4< z^8LcFg7tjHKC)?rA#-G3f@vv#DQUBBN&mc%b-Y&;Cyv3vK0gU&Ugran-Ce&^GU4Hv zK51(*OAPRiqAT?rL<)Y(w{;}Tr*hW5=EmVNTa8(Am4tOzhDMbhKXR-D%*I4H)_2S^ zzJV94BUxKN*bs=L-Rroup>FMV%b!J#kUK^*$s$y3bpHP8*25)`?!Ob-^en^}rylwU zQrxcOJm-b)fZ3ugvD`5IaEPi&(KcDmxpERj1{WmT69|$Gh@)jsmu1%4_av#4u8>TN zxf~%#fbccNrz&^8WK!VV6Uo+#0+~%H(;zMU*w|rTYeZX&{X6{UVZ;auh@T^QAks^q foa{ofBHD{3dzMS9iwr4j^c`nAciZKy!cYDN>lo9D literal 0 HcmV?d00001 From 6fb0e7dfe566541c69e02a5124bfbffa1eaaf301 Mon Sep 17 00:00:00 2001 From: Willem Pienaar <6728866+woop@users.noreply.github.com> Date: Sat, 23 May 2020 21:35:22 +0800 Subject: [PATCH 013/133] Restrict Docker Compose tests to master branch --- .github/workflows/docker_compose_tests.yml | 7 +++++-- 1 file changed, 5 insertions(+), 2 deletions(-) diff --git a/.github/workflows/docker_compose_tests.yml b/.github/workflows/docker_compose_tests.yml index 301ba116675..2984ebc37ba 100644 --- a/.github/workflows/docker_compose_tests.yml +++ b/.github/workflows/docker_compose_tests.yml @@ -1,6 +1,9 @@ name: docker compose tests -on: [push] +on: + push: + branches: + - master jobs: basic-redis-e2e-tests-docker-compose: @@ -9,4 +12,4 @@ jobs: steps: - uses: actions/checkout@v2 - name: test docker compose - run: ./infra/scripts/test-docker-compose.sh \ No newline at end of file + run: ./infra/scripts/test-docker-compose.sh From 7c46dc456856b0ced80e2494541a717b2ede0ba0 Mon Sep 17 00:00:00 2001 From: Willem Pienaar Date: Sun, 24 May 2020 09:53:41 +0800 Subject: [PATCH 014/133] Increase Docker Compose test timeout for retrieval --- tests/e2e/basic-ingest-redis-serving.py | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/tests/e2e/basic-ingest-redis-serving.py b/tests/e2e/basic-ingest-redis-serving.py index 619b8ca6ce2..4209ce1c220 100644 --- a/tests/e2e/basic-ingest-redis-serving.py +++ b/tests/e2e/basic-ingest-redis-serving.py @@ -129,7 +129,7 @@ def test_basic_ingest_success(client, cust_trans_df, driver_df): time.sleep(5) -@pytest.mark.timeout(60) +@pytest.mark.timeout(90) @pytest.mark.run(order=12) def test_basic_retrieve_online_success(client, cust_trans_df): # Poll serving for feature values until the correct values are returned @@ -171,7 +171,7 @@ def test_basic_retrieve_online_success(client, cust_trans_df): break -@pytest.mark.timeout(45) +@pytest.mark.timeout(90) @pytest.mark.run(order=13) def test_basic_retrieve_online_multiple_featureset(client, cust_trans_df, driver_df): # Poll serving for feature values until the correct values are returned @@ -359,7 +359,7 @@ def test_all_types_ingest_success(client, all_types_dataframe): client.ingest(all_types_fs, all_types_dataframe) -@pytest.mark.timeout(45) +@pytest.mark.timeout(90) @pytest.mark.run(order=22) def test_all_types_retrieve_online_success(client, all_types_dataframe): # Poll serving for feature values until the correct values are returned @@ -484,7 +484,7 @@ def test_large_volume_ingest_success(client, large_volume_dataframe): client.ingest(cust_trans_fs, large_volume_dataframe) -@pytest.mark.timeout(45) +@pytest.mark.timeout(90) @pytest.mark.run(order=32) def test_large_volume_retrieve_online_success(client, large_volume_dataframe): # Poll serving for feature values until the correct values are returned From 35f8f188c8294db4ed45d94df48d3ebf63b1cbe3 Mon Sep 17 00:00:00 2001 From: Willem Pienaar Date: Sun, 24 May 2020 14:45:36 +0800 Subject: [PATCH 015/133] Update Feast documentation based on 0.4 and new functionality --- README.md | 11 +- docs/.gitbook/assets/architecture.png | Bin 0 -> 24680 bytes .../assets/basic-architecture-diagram (2).svg | 1 + .../assets/basic-architecture-diagram (3).svg | 1 + .../feast-docs-overview-diagram-2 (2).svg | 1 + .../feast-docs-overview-diagram-2 (3).svg | 1 + docs/README.md | 4 +- docs/SUMMARY.md | 51 +- docs/{.gitbook => }/assets/feast_logo.png | Bin docs/contributing/adding-a-new-store-1.md | 87 +++ docs/contributing/adding-a-new-store.md | 87 +++ docs/contributing/contributing.md | 519 +----------------- docs/contributing/release-process.md | 45 +- docs/contributing/style-guide.md | 22 +- docs/getting-help.md | 36 ++ docs/getting-started/README.md | 14 + .../connecting-to-feast-1/README.md | 21 + .../connecting-to-feast.md | 38 ++ .../connecting-to-feast-1/python-sdk.md | 20 + .../getting-started/deploying-feast/README.md | 16 + .../deploying-feast/docker-compose.md | 112 ++++ .../deploying-feast/kubernetes.md | 211 +++++++ docs/installation/gke.md | 2 +- docs/reference/api.md | 10 + docs/reference/api/README.md | 10 + docs/reference/configuration-reference.md | 157 ++++++ docs/roadmap.md | 31 ++ docs/user-guide/architecture.md | 41 ++ docs/user-guide/data-ingestion.md | 126 +++++ docs/user-guide/entities.md | 43 ++ docs/user-guide/feature-retrieval.md | 116 ++++ docs/user-guide/feature-sets.md | 54 ++ docs/user-guide/features.md | 38 ++ docs/user-guide/overview.md | 58 ++ docs/user-guide/sources.md | 32 ++ docs/user-guide/stores.md | 37 ++ docs/why-feast.md | 32 ++ 37 files changed, 1512 insertions(+), 573 deletions(-) create mode 100644 docs/.gitbook/assets/architecture.png create mode 100644 docs/.gitbook/assets/basic-architecture-diagram (2).svg create mode 100644 docs/.gitbook/assets/basic-architecture-diagram (3).svg create mode 100644 docs/.gitbook/assets/feast-docs-overview-diagram-2 (2).svg create mode 100644 docs/.gitbook/assets/feast-docs-overview-diagram-2 (3).svg rename docs/{.gitbook => }/assets/feast_logo.png (100%) create mode 100644 docs/contributing/adding-a-new-store-1.md create mode 100644 docs/contributing/adding-a-new-store.md create mode 100644 docs/getting-help.md create mode 100644 docs/getting-started/README.md create mode 100644 docs/getting-started/connecting-to-feast-1/README.md create mode 100644 docs/getting-started/connecting-to-feast-1/connecting-to-feast.md create mode 100644 docs/getting-started/connecting-to-feast-1/python-sdk.md create mode 100644 docs/getting-started/deploying-feast/README.md create mode 100644 docs/getting-started/deploying-feast/docker-compose.md create mode 100644 docs/getting-started/deploying-feast/kubernetes.md create mode 100644 docs/reference/api.md create mode 100644 docs/reference/api/README.md create mode 100644 docs/reference/configuration-reference.md create mode 100644 docs/roadmap.md create mode 100644 docs/user-guide/architecture.md create mode 100644 docs/user-guide/data-ingestion.md create mode 100644 docs/user-guide/entities.md create mode 100644 docs/user-guide/feature-retrieval.md create mode 100644 docs/user-guide/feature-sets.md create mode 100644 docs/user-guide/features.md create mode 100644 docs/user-guide/overview.md create mode 100644 docs/user-guide/sources.md create mode 100644 docs/user-guide/stores.md create mode 100644 docs/why-feast.md diff --git a/README.md b/README.md index 533c123874d..2c2970c6d9c 100644 --- a/README.md +++ b/README.md @@ -1,17 +1,16 @@

- - - + + +


[![Unit Tests](https://github.com/feast-dev/feast/workflows/unit%20tests/badge.svg?branch=master)](https://github.com/feast-dev/feast/actions?query=workflow%3A%22unit+tests%22+branch%3Amaster) [![Docker Compose Tests](https://github.com/feast-dev/feast/workflows/docker%20compose%20tests/badge.svg?branch=master)](https://github.com/feast-dev/feast/actions?query=workflow%3A%22docker+compose+tests%22+branch%3Amaster) [![Code Standards](https://github.com/feast-dev/feast/workflows/code%20standards/badge.svg?branch=master)](https://github.com/feast-dev/feast/actions?query=workflow%3A%22code+standards%22+branch%3Amaster) -[![Docs latest](https://img.shields.io/badge/Docs-latest-blue.svg)](https://docs.feast.dev/) +[![Docs Latest](https://img.shields.io/badge/docs-latest-blue.svg)](https://docs.feast.dev/) [![GitHub Release](https://img.shields.io/github/release/feast-dev/feast.svg?style=flat)](https://github.com/feast-dev/feast/releases) - ## Overview Feast (Feature Store) is a tool for managing and serving machine learning features. Feast is the bridge between models and data. @@ -24,7 +23,7 @@ Feast aims to: ![](docs/.gitbook/assets/feast-docs-overview-diagram-2.svg) -TL;DR: Feast decouples feature engineering from feature usage. Features that are added to Feast become available immediately for training and serving. Models can retrieve the same features used in training from a low latency online store in production. +Feast decouples feature engineering from feature usage, allowing independent development of features and consumption of features. Features that are added to Feast become available immediately for training and serving. Models can retrieve the same features used in training from a low latency online store in production. This means that new ML projects start with a process of feature selection from a catalog instead of having to do feature engineering from scratch. ``` diff --git a/docs/.gitbook/assets/architecture.png b/docs/.gitbook/assets/architecture.png new file mode 100644 index 0000000000000000000000000000000000000000..bc655b60f327b7d7d508885666064e0c713522b5 GIT binary patch literal 24680 zcmc$`WmH^Uw>3xv2@o_`u;A|Q8r%ZG-66QU2Y08?5G(}u6a^ISg%$4Z?oL;p_qq4G zJx2HFAN`H){ijZ?QRi%5Ywo$`sYn$i>G#MVk>TLr-pk5Js=~p&v4DenMUR94v-pom zw8K8$x`@kuMS>MyB(n(EcLLYXTCQpi7Ooz~&gO8I_6~ODOfII*=H~V;Rt~NwZ@|KE zaG&60CB?paW*;sEeEn*fd-C8B^!fANS611%fqZ@@4W7shNrt@;SLx2(iq#y1;*xP| z^vrTGi)JzxUBP5M+a2b9{<0yT-Y~)wEFWs;ZK7(LpUE7-HXpNry7|0!$= zG2^cy{wa%lDH)OfT;VH30{&y``2XItO+egkhUybv7Z4ysTylRFIy{?{UB09uTIrHM zo!i+nvMPSM_H`5upUc_p0bp{kVLb9UEplbdI? zbZ2TNZhh|xlbt5vc9))8@PU!x=-bieJ;KLEwXe|k16gw*j_}V?h?LIZ8BY`zA$@Z| zKfKr+zqjkyvPv)4|^zeS%*$dd2KRfESTNo1w%^; z5pQd}#BC)v(Mt2cIP9fFuzTJQ)Am^_$z#eoVK%};+s#dF)~Z`!pWp5^E}A$jA|U+J z@rha4_FXu0C48&W{2Gz(d|n)>Q|gDMzb#JCbIF6q_-}Y7Y+M7$$<<8oN5?mA1A9i> z6s9V34u8*g-J}DIesyR8z7_r4yER_N2K~ zzPBXNAyW6zImNo*ZnC?V(3@uRTH~= zG)79F1_1KL<2p&}vG-g34e!_amK7u53;lhVOgX%8D~G9lv_<+PULm{=aXoqD_^m{f z)I2YBat39jFYj&bRX7>_UFv_tyVQnp7CEH#WQ#dk5uop>4W&06{b@ChwxU{mpB{Z5?*h~S3I#%5pk$G%Z%fnLq;E1yHiX&PLW%hYJbGJVK z760!W6~Fo{vJ3L;GORtjeKXoCc!Q<3C|-o!U&}+!8zlrtdgI4z5j#^tUrB`&U$kXp z^m_M$nFMzZO-JkObw*yhs$E$Up0=FQFKAkOFWAvxnBYg4l9eG3Y3-sIH@5cVS~B#O zz9Cx{)=^aSLu{2GTgPKEbrZuJ<->bLe7JZ@!M~wd%|9m{-luRp1nO@vEJGXA;~H}c zwl_LcQ^w?X3O>drM%~oo&tYp{c<=Lz|1jSH2@AL6{HT-MnIulHAHkvV<%4Hdn29bR=nL&Rfldv}M`IovY)33*z-gH^#Lz(I?Ls|QJBu>g&>MvMn5RJY#lRHT))j!TE zUX>dENY`8vm|K}ckCdfkLLKw_iF~t+7P}15d2@*d$4KckUKD%Vwb|@ZV`GWQz(Otc zC61t>u+EC(fo3fe$eAt3P5)BjnB#)@n6pYm=!adoVsFz{Oj|UZQ5v@eE?UH z1Sm77KS2p4xGbMG1r*gF@t!rm5=f`P#CyYN&oNy&5(%dtSPtiUM)Z9=*Gk{w^LMP} zEw!J5?6}Ndj<~K*yyEI&nJ&YJ85ukQLINIV&TGiSx4VMNL)9g@1%711L&&rY53fXM z7dpQYqPmgv6f9pc;*fuM&5G^Nw%lluW>>jMYOpYRt^P`9Ux1)fYCGEA|1^YsFjqf} zZ?bPZ%^*Ua8g%;5a@s|DK9Pn;$KYr}E!k8;c+q7;r$tCG9;-Z3{Hag2Wj{`@M{4_8 z=+O1#C!+u+uYAA_+?c$)JSmxN`EpyhzJN!KzOFBV<9B7yPL z9+-;hwmA4i!Jh{xLv?=5t3yau=AzZ?AYB`H>|+a6%@mpu6vD#nCaKl5s;|N%x>yej z0^qa~hJVX+N%{H;T0~58IPmz98STCWMJekw_62po4U}n^*jTS^#-q^M>hu;2Y1G~? z*m!+KnOi1XudbpuvD`;kKDI5asR_kiDzRR?qp)Q;q1+ifSa%FC)9476*1mHr6lQki zmtCh^YF$FDW~|NOOO!5vPE~3mPW^x=nnLlhvr7e06ZR?dYa)-+c3pdGnv)VFYl%Oh z?L<#aI=os(u;TTtdn%F4)oI8m2nUtUsfx; zt{&x0LFe*B&*zCWgqy>HrDqcS>SLJHOZb-7$*PTuO`}|!u~m80{!IH1upgkvOnTSW zP5$f$p1o&hWkr=WwX8@FOE>JnmLs>6zvdDKXAI-zg3OvJ5~?$dM-3L$X^;CEBm{yC~? zO7zV;a#8+7>bhZ!EkQ!5^DSx4*I?HQ8zZ9V+=YRm0cmSnzM!4ScORAWR(B&IEiD}3 z;Z#UjJX9&F^c_2o*%6RosyDUU*X2g#7D?e@?jbU2CYeEE72(J zn1mr3J57k+3;ChGiY()DA>2-|#W5jH^G0Qfx4*VCeO6l?f>MgIS{|O4)7Em9W#p*2T@SY; zjqc+4QbyFqD?P8!;uU;N_qLlV=|hD>?k2wux8Q#(Q!V!qbzac=fx>`3|L_IXM4@CQP#(%laOkz*@M(x2!=Yumm%P3zQOYqMu9 za^{qq$cs@#o^_#KKJX5`*0Am0GX8P$TX{}9CrfUlQc3Cg79Ke(NsD(@zKJDn3gt$% zdNvH;f4}c>-gWGGube~uHlv_a%ePZ}Jt%*}$BQfhl7{!y!>l$*tJRw=_Tjl)8y_r2!D^9!Cqhq-Dto=9WbREr@+1x#UK+vumtlP_p3<7u)vN%_i64 zay~r%N^VQrK;7I-0sw}`GwrXzm#=d_lfkBg5A-SZy{r@O1`JGXeqY|GuXi0=03lfV zvXGD_B*-K&1sz8&t3j!n0JBF6O)oSBA%BWP-tiAGE_Pn_9J*~+DrTM?^k-pG2`BXPhN}Rvj!eWFYNoAx zQI@48@Y)S&UyUndW9-Q%%miEe3bMthm(stq%)>kN!WJ|eu5|@Lo&I&aSo7#WvSql8 zPcA(7InG6R2@;$AI6FAeuwr&6&btB~Rt`9Gx_o-5VRkzoGh$it^YdK=XZ5AioQl0* zQ2OUAsvCB+Ia5W4w_e`7_o~x@P9Q=C*@kk8^)-Mc}`JYhy&w4@GAO|`d zFeo*4sG^&l^x;ic_Vwjy$C%ggLuRx(I)vwbhC+>2c<-QG$2Ul91@ZFw`b_Lo*h5>= zU>MU&qR8{0FMbJHxqfm-?e|yyD^Pid6%4^!UYi#a!WF+xoW5k&rLaoxIdf?OXS5@_cDJ!?#+X~MCN@jnXWxw5gAauK)iJK zxX22EduKSX3QQuOW{d)bK10RRiv8g)5QSRKqLq}c;Uf&2qk><58z_JpDxtyo_o18o zSEq3i5&cIp|4jo}+&SOD9tDXm8LsNw{(M89KGcA4;@XnN-{{xKlfZOhgaejF;R@e>8SI8}9kLu5M z=l)Pos30k6k(xM;KN4s4C}AIi#eoTcy0WOktOY0qQqrGXVqvC#N(dbrYy@G2E#v}Y|Sylk#~4^m$smw;Ct8FUW|NYhZ#F2Hnz)L9=Ws2NMq_*$jkkZ)3R5LVgM+) z{+0k;R#H;(STtE_SV~&@U8p!7HanGNFDpPHLxI}F)D-Br_S(tGsk^s#WN*{9vM)e@ zfPjFF|Lq2&R`)t0p22wtl@>i+!bMdH0nW@6&H26(^hb0QQL9$urx2&?BK@rYt%WaQd0 z=!BaeHS+x7DJCxNBXJnnw$Qnd!Q#DJL01Cbu=flt`mzlhKk6$45tQ?wm0enUeG_ri z>09d-3CiT_j#WN{(UI6^2R%I^6kJADepIp~iMyv{hZySAjL9O$qd6SiRE8UY%h~%Ho|;_;ZtXv0hc8!8v_A& zh}ro!yDKUymnbcy?@ilYFmrE{!J8t5ewcw{C(nQx|MU2kaSl7B>6RL2n>}B$ZGWM@?5D1vLlow`Os1f-P|56cPyfX9>x`G-4wnI!s zH0T?`A~C$e_XxPRQEFM=p7m)zC7f)1;8(`tGml*y1``ZGE1o=goC1C!`naAY? zP2Cu*(Gii6Uz{n};ide6Axr`Hci+PMz(`i_svZ3XQVXOb# zHfex2!U2UmE9nd2^ICg60WV?m?Xrv&zOC|kHwEIZ&q1E&-OD3jqTElfjpe9gg%>2WPbf zwhv3|^5y_HjR8A}0A5O%?*K^DCxu7!OOohy;z8O= z41|47*;Gjal0JkJ1Xy9pz9XHO;ecaYKO1&PI!~oq^w?9Gg{{3OkVkEec(sUuW-RkFhT4i{|k*;^^cFk(~r|-N7hQJ33apa|< z_jGBqO%iI}g0;7-6V!q(Q-9GK|}ouz6etXYsnmr)@t z#x%Z$l!FY>PogF!KmYD;pOaReowhz(TC45F2!t1NT;3ZpdGL9{_!p$p%9}T@T^@ag z;noQ@^Sgm(OWI8!o)s^=?Uh;0tnMu7>R!lWY?HF^-FGVav|&%6bAt-4fP(VKl_T8jKc&?-6^KiF;iCKQvCL037GR}YiYO}zi-Z# zP7@U0?`b~B3Y&Q3Og-rZ19P{dh{~ZV!4dehvt9W~i_tUfi%AlfZyoH?+va`s>_?fx zw{2KxR&ZK{Epb$rKU!}D>%X6DR8sfBD_lA%HZ`ZF+h{wXwA^Ep8xmSvuln4}HqO`P zC6*ZlrL)h>tlM&+HS_YOOP}7BS#jbkEiFwX&7N-7}P)a7^-|F?c&qr#QMZ_T3{rg!4nO>Z12nd`X*M;SP?oiy>f&#|M>|tS?3;U z^k*2EUgA{fYpoKP;~F`G)Fl8nkObJ7)G&U;7dMmGrn#fl8!QtxnSC9bo0GM)q=DeL zf0SRWr+3Noc3R~dYcG0Pcv5t$N8HGW;_wxMjtj5T8m+XwCQ`#*ug@@7BUqqw+xkLi zhyNIUF3Vy~C6kaV(X-jC_h&+X6O2Xh{(#;iQC&;iUxUAN2ufFe6qKuj9Bs3^_dEYj z_4RrA>`9-4Y&R~kclB1vml`huu+5 zdMnIRj^Z6r6*fB$xD+L8E^&0+Ay2scqErpn+cU$FF6mR4x#?Oek738mBJdngz|E4=vi@jAaM$AeD{XzTW&_lch=aLtD{^jHGyk(- zO@qBrqxuTa4}Q$ukG2!naAMLIV+M0`wTNp>&*rRrs-W8Yz(!JY3C+OY$*u2PmE8x#!z}dW9uBs_fHkiQ?{I@y*oc7HZ#u zT@I_;Z+C{I?F{I(y?})!>8H?n!3p_=0)dNF_bt_<_Y*4XKkHLvi2Ip}Ia#7FC%@|T zn9h)pRvfqVk2<5!4J#H^TW8<4C{Et7&((0jE9)P8rzqCg;h$g>a^36X-eQbHod|a0 z)Uze>nPZ1UMj$YevK&8aB=07KeqZVr+x%8;g7~P#-}$4 zTy5-N>w^Y{s(O4Fjy5mC_lJd;V3PTU!qrpe-w5q?nL_Kgyb$asO_oB=Wv!>z13e5D1J7$$Cmj=UKb) z<^>i;_(33?q;pZ`C-#}WVb&q;+$T206~N#W4Iac}=7Kh0GKFrw#Yy-3;iFxV=}cIF z4)G9fd2ZAM23^3@*;Y7g7zNoqUVKK+<6Ir1r zebM;*B`W7C_3E&{e!YU(xCtB)VrN%Rych#+`Mw{GS5UKIi2AlhP#=>o4C3#e?vc=1 z1!!!f&(3X4=d2}X*Iqh_KTj8VcYdFRhm;LIHt9`Hss+YyDzjEU z+y~_aJS_}4Aa>HUb5YTkJFaOvB5V6%J40^$3B!`o9F^Yos`H|Z}jrcjn zc7nZ5_H5E`8QdJaq9zSm5gYuV_$L!3OnOxyz-E06+yS_NYAUf>(fH(~HoqAkC|pL^ ze*~24@->F$g-xh>Pk*++BBrZTRcpy&{!!BW{i6+mH2(r`?FK>lc--W7Hya`zsIdR? z@n&3}vYvY&&z!Ilhr~=qe{oJ)In)jU;TRUDnhIIlD%7o3*p-y^NgGcG^>cT8 zvNnK2xlSGFU*Zmb<8ul*czk`d`IS3)94aTFPh6zEHQRJl@a-uzBop^go>-Dv)021l zxCfdXJ@g*q?SkCn^_8HhlUk-Bhc|2Re`x_EFhqpJP#F5WPp^v-K}xuY3`Fkb)@Y1@ zkmGi&%MHZilmTn-1Cswme&ov;?jjy~rMdC55#20JL)4ksD4vm_SQHMmprKjPnJdF_ zr?2~)HWCqc5hL^ck@OxhxlMP9#(Gp+d!9SQV2sQ43mPdb!RYrT9L)jYZ-&@J#B6!< zYQEHgzB@Z`kA6`9VqDj5-Y}>9a9}VI#ZFN(>pYvGF|zC~0)H(ZPud|o>-chW8L9?z-*{^uAhhy0p9Pti@8M5Mt@fJ!^-b zrIpz&&L~-B*IzIMrOIX`d~%~(;W|FymZc9>DWb90F{N-`sjS*|Z}b-r4{q@!_1ZZ! z@9xSP>(}Gi$!oJIJy?*LLxnRrKvs6Y3Mywm16>di8JNKaJ_`$ErqcC(PSyVU$e?&H zBZ~BTS^xQXuiZU5G6sHjEL8r}yQka7>1YmeM?uBRs_Y3>k-1T`TGPCZb(aR*viH#| zeOk$WAbcR4a6m@p3nn=c+SzTo>8h|m&v)@mbZ0?S`*NogmrUVI0A+`?& zABDuqk1Dk4%Bs7}M?==MWHaBlcc|(ntlW3ZOoeWlm46Y7?l$ZxE;AX8kF;$B5+6CPg=8Gnvi3hMHX ziChSNHwYfT9`7*sBirG<>o!k4DqiT@<9cX&>;ouzP6n4-Ck799WJH7^t6BSdn<=binaf>3W#K6;zwV zx*Z*US=AdINA#pST^qxlIh|{@BOV)PIczR7`WF$gp?kL<>)PTK7~5X9PtaCy@TfgU zx_9C6Z&9DmRuqV0p7WM7aJNgT*c2|AEY!T(PDy``Jy9Q^9aBIN6nlnBLHk{59&OEo z(R>M7Jsreoj6zijy1qw8YRLPgqnBgL^ZmyyOT6xrY8LI92#qa5@yes4k2h;Ye4UKi zoEtrZ#D#Y=c)UEm7kMwcb*yaENcs_!)(+S46FY=k;D+xRq?Y6&rX=wzD#X`N=)BSx zKvr(vJ6BnE^j_lbk2_CI8!b{C`Slc@OjY40=ztSAQ5c zxIOUh1UWBf`(@?H>OO@IE6)9?ll!$dXm&xB1s~GBG~+W*i6q8AAP}blaN27W{`Ss) zxqF_y)XIHgYF^GppG}*KuwZ6pSC}!4(7SR=0vs7|l)G&F>$!t962+md^)ZKKSl@z| zG??`?dijf>U)T4p(^YZomo-9*Hg0{x3%)fz;r^s|qccUeW)=&j*0!tu@u`RO0?BPg z1}Bp4Q(tB_s50`D-iVIHk1hl{JW<0OwORner9WppkMN%wejGf3s4afk>)7i$<`}7G zA_92qQ?NfG`22Oo9~P2lXj7N3u^jV;Sa72Pck$tts2amGH6bB7|EkLA`UC6uUcC`; zKQ?kc*l1JGaJzRwdLCV`E!yckasa6GzzeaM%i314VLr-u4j_;Jw2-vZuozRfYSr#IELNN^ub#on&q z&1)R@|3No#cFyJ0Zn^`gF9in>p@mysPPx(#kFEgDyD)&mB9EgO;9cT1$a9v@({TZc z=8;Rep8yZhb4x`QC(5!E4;83S0|RaS;toc5X%Q)JT#^_$T#E15RBWrv@E)ZKCEI?Z zb#SXb9<+|ZmRs9PVc!2a>Uh_zWRNoqFEp9*hi|iW_I4db2e|MWH<~Il@pBa<7V-4< zRSVJYgU-9*RFwExayv~({;umWGnzWjT36H(NjDx-*NlpSy*1u-%#|Rx07+W2Z^sTi z6W(LNQ(NVl%6dBOQ_%AC3;_#2x~81s2*gIZ|5_nM<-lrv$I3)4*oU1{Ri@1`W9&d* z(whX~nsxxjEbURw>fn`fbTBJfs!;#&Y6663wmhLEqX^eU-oCkDH1Vf(VtGyiK&GjZGSi4Xs}Q7o&Axwaw`X|3rauej$>~) zh=?*+7tU4iYU-R3MOPN@U#wqsb=!4#7XR=gqRIQyCR-m59UjB1lMgR=MU-_@Q`Igu z*sM^l=sz7Bi_hEi1ql2;vFN&pi*g-f_f@CL$#N{QgCors`Qo^EfAs<{eXU{ZtI8Tv zK4lp~07i9jbxrMSn$@9^A1#xgBe(j^J>GWo-bWExFGD3u2=pq^^j;;hIuJ{C+1QvZq*5* zLX4JPPFMIDAGjz$QCb6a=N%6Xj8$?w`N@Cm* z{HJtbjj|;_ywN;t)30l|>*aWxpV0x#Blo!r=IZd_v2Zgy`Ki)MNaD8IY8Ie7_Ga<8 z?1R5go8<=-Z!q67#u3rev)5?lm!DHo6){hQXq3mfrKrcvR8%Je>?}_PY^ej{dwcI| zD_RUP3)=67l=}iGzFG-sa;O)A7p{56kH{U0U`hM%0@nQxiz7SD_zQ z7|S9~sT!2K;7n)7bMb-;%?4qP&i+x^QpvwL-Gu>%AoK%BwPU-Rq+ZXT0IgAN#*FG= zmCBSXjdR*OWz~K+QttxwygOZ7Fwxhv32_}lp|w%THR!$pxH(1g1+kysc-63e45J}I zuA_#ZATna>qRER{KJ}+D8JLB|U7VNRVFsx!yYed94amChga=~BcHys|j1xP|zl`*a z*|fDaY`GbpcdP6k8;urK9a{W|ESV3~@4DR-fBI!x6nAuV985?3PeKb}xGKphhu@7p zGh->F$E1Uk*oDC1%k_(mi2~zHCDqv2j@3$g5V+}pG1i~3>G zM-R_Mw5&EKmpoGBOg{(ENfeGX?Y!eA1#aryrhYBEZa6FQ@?|{zkpWapwq^N(qguhNh zZN%YH)mx5ok?!SXyR~vAZEtMY4}+b#)*}n?+Y){8E{gr~|tOT20 zpwVxOs&Z##n69;77#{;1Tq&FALKdW{EZ8ejQX>10Rx1HHYXTMJ)}w0bFD>K7a7IIm zVxbi_!8HCMv)mfBU0wneJ-5J`y(o_l(IHnjh}sUWL@pTv-;-*s&0HUT!IEr}lBgyf z$1Tns32g4vzVecr+?*(ye|c0`0oKv}4_5IX6XySh^wR8mDhoEY!3_)Sj{qH=Ypl}` zBt|{S+yxRqb30-O$9mw@F6-NBp<9f2!TZ2*V}(JW%g!t~06cEISQ~V4Q@vZM9E-M* z2CMKC;^6Nxv7pS)#*qbK)T&;s&0}E8@-me@pS_ESj?RKcYQwS?e0+Scj0(B)P!g$E z%%(I62R|w>RuFKFC>8$6C1J_F&%TL}xR;3!`#ZRt>&)Zeq?{&FXK-r;`uf`SaMe;;Hv{RcalUFF{daNTh0nUpl^ z#eX&7f9S{S=?fwj6SsRZweU&hd3`M_ZaE3{@k2Y>iSI>&A|hl`U1FAL zfH({;4}OI0*E=HE0W%z$sv>zT6U1Q|gC_QqIVJ|5I2bl2YyTIu{4Y~APA@Tb4K zo$T-U9e?s4eRWN80n2;B8mlJ;d&HuyKQxB5mC?WBHA`-VsMy9lr6G`ReSJk@3 z`g;loFG9I~IYp|aRCy<3+`XbBxLcvA4;DbjqLO5FJ$X%xc6kYFX6E$h%S%1Ek1e+! z<@#3^LaR!@w`f`zc`Cu?o?7wWdUytvUuKYW{hf2|8g}k>4C7b@O5hlg&+Fa)pMPg<}hVez%S6T9IXQUAe=CkY*;7cApw2L^4o2X? zNHLk}%2$R;@DEX{)#e(Gr(PvBeYCJKOb7gvpNH;kaH!1ii9dm36FjRTr|2$jL(FTW z!rkoBKIFRVymFkiyY&;>-TTp(tEe3XU!71piRSPc#v)yLTaCKZ;C;gHHn|eth8Erp z(lK2nKPgZPT=P}G_HKrGS2Op&S^lv5Q#IaIE~g()7B{R|etyiT89xJfKEDtn+>5-c zd-042utl#izYz4L>I}$b;5B zR?Nw-7;0eD6#59-l36!`thQ{F2g`e*EYD@k!h=Gdq+fiwy?acS4iDb$h1zEg8qjYG zl2Xd=LWvJ%n@oj91(;m_#Es)DN@n)F?)Wh-?E>fnza?xl&YfmQ^9a*2n>zHvOax4Q z1~O#Lr?k3iSQt3FjP>O|x;EZ5o!#^D;L=Qd{gd762L8$J@_6Id;I;YWP6#n~acORY z?KQR(zUgjLL*$D-<;BIF@L;xVLZBd<&&|e{On)2#rs!C>a79cr-PEVYdJ&E=}p#Ndv4e4l zH;955JFJ}5+S325)9ula@Z)&0A-;H;^x@DiSOV}@cH`h3m8-Js=o8*X;(U_8PyPq| zzxh3kx|G#qak=CvG&yzWxv#k#Kqh%~w!Z~Ci_4;PM%x;%pw1s)D^XLPtZS|-2!WI@ z%NFjEa}rZwsNLzT!xBdG_Z-?^Vb;&~&*}`s5ecl8w6-K9NeF2jDjoVqY`Tkiqou0U zHlYMy91=48YI%>_kclFMiJ2Kdrn%i@{s^MPVL_nMZ z^WSb2dg(#n%Ao!xD{o#EOYhyrS$9GJPB;bWu-M&Xle}NQ!7~av}l{(VuNZWv)hrSoQHWOoReh3XY34 zQyUJdeBF4DKDMvbC$^9^3AbAE=VCewLQwYFI8zwH)ex<5qjL5QT z=kPDXY4u8MqNSH;;p^XB^tV7Mk$APhBXS1%4Dx;?s?({)8=Hl~b_A9}YeuXQLM7+F zKMVUds9{z)*_U_;|8omF%A7Ozx)puLk*uj{*j188`M-zhbN#F~YBbM7!DNe`V^=I& zho3Y^t38B%)R4q&%Gs|H$>{C>Fapm6!auBkVE#v612C9Y;O1YtoSjZ|@A;Hrss^vS z!brxm^nScYgnRDQgNAz{j%tP*Y8E7n$|{oad2LZ@Ya2_>1?r^BILp!a#hSQen6HZ^jT40k_2KT1kUe2unBm4Et8{=Y#3RZaM! zp#cL2?SQ(SJG+&WzqsahwC~iZU2?(?F9vNZ?N$WE2*@+A%L?=2s6{Y9c`U=02HNx7 z;E@V_&ViKxuUpsa8tw1zSxcHG_!NwG(Qc4t1?d*KeI@|R7`_mCO!ygZI zDCM5)kX;qn!!~jE2>zcXO^NSv_*{B-F)X@6QF<$#%X>s%x5C`6_|wx9k%+&q)X*cu z(r~@ZWAt!lzFeo_o74~}>0ezGVC;NrNW0DVE++Nn8RJpEArg;UNcNDJ6MTZk|Mf2_ z!UP<}Jr2GN-0?s_WbgVnuJ)IoRh5ttg_)Mb#+h0CtF}tiUULOUa)`0M5(((*K8+a1 zgb@83%Gntn4;5JfaM;}$tKAL?QsJDY{}rLvq?jd(*$nVw9|bsHznt5cnr6!WHyR=a zhmOI2BJz7x(#q;O-T%uX?%#H^WqXW$cN3mHL2*fkPr?laS_-=cI9e)LPTu_cj@%k| zVc1*(@`*1JRaHlPUP!crr3c)91E#Y%uj5}+_SD!T2 zf*pYLR+^8Hh>Jf(xA7Bm7(A{%sbvVqPFFTMB$nTt9Z;>s_;=Od2Cxl#Oh^F_Dz#wv zoC+pBLq$hd_I@Cg8-{@_t!kr=_lQ)`jk;0cE?e%uD4GqR07K0yyR4M^i$E8-q|f$A zUYvoD<}nzi|AQgerF{(BSGwFHuy-vEg#HSh<~T^-7_#!m$igIQFhR#+C67EwM9X$_ zVqzk!68qih<=x%IKWdI8e`Q_cWQ(;6YxwpwJ0m$9|MUpsc6b+qlHT+A+G=QIg}Tp4 zfY*oH$Q3Dh`{p4QP0T)NTkjuG81X-F>3=Pc|5v8|^7VqLoNxxAc+KXRIG9ifrWG7y zlpk)U^$_YjpLCsoLS>ale^$?jykzE`P3SC$YMy?~s!vbI4A{+lc=sP+2;LCHgJ-RE zS~$o^^G!oVn~ejdnpx()RCQ42R!_asA2ETVN*O@uQ2J^AxfSsMeVh~_%NgLdyA0@xRy&VBZU0UpmA2m02hYVq7Hy@0ZsOuH*zGbbty22C^fU}jA5W-6 z&ZoDf4>p}iw9k=B_x1`)K(s9yMWFKGgDLRAj4=hIwi~rwY`lYeetzCcK=rN9koBK! zmeHY_R=1gBU*#k!Q|ZFRMn7A$W9pTqkp_1d`D=8-cEZ(|P--r@Bx5S=QaJ2)kiJUG zkIxi9IHEguBikWc#^6PrBTSY6I;VBG*|4D{ z3vj?|E5tbOP5(S58u<-?tFaj&!Bw%6&`Anzljj z(w|g6CKuv~FTZ3Q6vEDaxyDj;iKc7)ee7jT3yGa9{pVOQrY4^cij1Eh8h}(4qn!E} zZrxv%CC*Rg^9SB;j*wx8V+6@P7FfU`7Z?G1G1vy?Cl3AZnWa&XEw6<8_OFlRFo|U zRS9WIcXMxT&t}@9V$bw2(r-O z6_6RQWk?33^|{N}KpVNf4JE}_N{?N(Sy3Lm#!3$hDO@R$d!6Z`^s;wx;*$GvA-Ka? zLunMtW)eEP7VW}|HtQ5e>1%rvgtH*+=XK5dRcA_$7ru#@qgob~(;{B8Lv??-h5Mns zmw!T0-z(v#X0RcVfa;g|lDM3Xb15{1EBBrS`O-Naw@(e>zgWJFj0nqowyvTpspkYe zijspKuV#bA1_{wO3ni`$uEznWCBmb;#JzaEnRN6OKZ`kKAZ^{KKu#$iFA51D(Y!UsG|<x@PDOazu8`sF!%N!6ET^5zAF`K2vi<09RxK5N9ydh2Q8iV} zGGD$VoHu)a3+5uy`ia?Su9ZPGtA;m>rkw6!a$--miIBt=Cj=FBpcFUz^UhS6$9QP; z;EDLVpnv8)=+|W-zLgz~dKU52Zk6KrGQmi$BO{NwP*+UcHAchAk!j&PdE&o2l5pTrJtHRi)8B*_s=72RqrZO0AE%}?U} zcTM_k^jpq6PDe?_EyQ+yd>(wi&C@di7bAhA_B9Doa+GUrvvW>TpcCsT> z{)}Ewpq~OaTisfe*FTPAm-}j7x^V)BV9P3iYe4AX3w;!$!u|znz++e09@SJuH8wTDl#wcDYSbRnn~y z&*u*W^4x2G?E22uB!I^BnVCHC3mP|&;4s>F!PK>pB)^utA~YCt&@NfX2#7Lkw39VEbOWL!`4nzt6j zNO_mGawBJz-MqZ-V_V_0yI4TbFaMAWYB(t#SnCebxCSlY3-b>;F1Q~%FDWSDFfOS< z(qtu0SgW!`=@ncQIUeND&NH>e0pm(GOa8WuSdAUm3uq>6Xnjv#Patv1-tL#IM6W%3 z-wq$W6`VVAxSq|<=!y3A-!G$42`*{PX%;3tq@=UK=61ZWfnF!l2%qoj0Ib(E(z-e%pR(bZ^i5 zj)1z@8jeH(8giFF$`WY3-Vn#K#rAo#?RxJHHQ*v1`|(Od>2$Hc>#>+6Pcv_%#BqGI z7~%PdxdE|3PV8;8)}!hgZ2-aRdW3Z0nI9 zuD1N<6c@OI%&z%b794opXZva**sw0XEG6sD5dWewt7lL9-j15+9sOo(Ug<(&>T3$; zIO(})d5#r(ua+X6PjN2Pk*2Ojs9lfwb{8u9yib#S`O?L!n>$?K>GFz2C|5dnC@8Q+ z_YC-I@wOOJq`O#QXYP6WporRTnu$+=%RPs=ysyGtBG^^|wFtr?HXGIy%OYG%#ld$A z>DKi%V*!u2PhP*mw;vWo(FZA1lV*b~6csx2O;2Y;+|D);ZjYEwPk1b~K+bCmjdmd_ z6?sB_#^qQdYaNtSO#Ztg&%%7%2vo#x7vZP{e*3w`cend||D(+|WzTkMciryvNtpfV zA)u1Oz=1ao^4m^0>w3wK&lXc4iM(1M1tJd(-y;E-{EuJrabBMN~PzMc^Zp?v7sJ%`mDOpDx?$1m8&7D>R>|fj${X#C+ z^8XCLc?637&Uy;RzWwbdn)D_%N9+Hn?Ye`S`noMh6KOUGMTk-&Dq;vd5l{pXg#dyA z(u?%aLJL(zKxvU8y{I(l2neBr5Ks^hg9?NWp$0+`LQP)q=Qs1dnQz|An>X*y+<#8) zJvV2ceac#U?R&4XEOKBzFfWO#`0V?*;5$u*Z{@DcJ=8n+T*_unb%Segi+t-`Yh$z- z5RZ6Nyda71X?wz9$I!%L(KFz&+83PkbskOK?tZ7zj}$P&%+=vlT2idk>4#?Ay^p!K z^(%jMCt6gY9<5%poUa+hLglM^@^+T9LXv%c&T>S>7F2cRb18>4sXS{^@mv?^SN)N6Ju6is*H~`IlF^B^ znq=i5@#T$~b}}&E?wvKh^_KL?4VyGGSo^+;GWD7LL&WCdw9#CWuT@f{r^0HR3#4M^ zl1qCGz0rqL)|c|*`&z+7m-tEhhVq14183j9iZH;W*2h8iisyG4ed65%PJsj zNetOqS|;$qRIQ6zI>FI}L`I3IM1&>p6K;b0Xm}*P>xZY~ykS|zrgK$+ruTLe z*K~*gIG3Qs>FXOO$7eyP>n~#f(_NLm*mls!Bkbh1x&LL0c9JrDF-xJ)47-z%N#w)d z)M(YdT#tTl4}kG3!Zq^kS|w;GJY!;GIP$==+HCh8Wo zyp56rVtPURoU-~t#Eu!ZN!hjA0uxa2TxWW&)07iia4y=Qpb*VBi) zP7oo+>%uac@)qQ8a@U>VX*dtFK(Y7;sSN;Lx3ZzPdvmO+A@2LOkT3$~P*hUzQRc^DE&`YjeYdW9oM zbWnOQA)M(~sU$q6FG~qlP-4^)d}xgoMtW_`Y6_W;cDuAS@Bfm_8ls!kS4*W# zW20Qsewp4jW(S5GNEMSlbQ3eoJ`-L(9_p%Q4Bl`}iwn`*%v=(~M65Hc=1B8V7H!j3 z`UzDYW9%IBJuTLoRXP#~OGqhKLIGys(fE8<$j;+5sxPG1t?^a=6`ffNH-iZW4tta} zt}~CzHO_clDt??!Sw5F}={|6hujkY-bdlg>0SY@aeuXeH++hgX*f$LEWd5rtW5dA7k!K_8Ta<9$xC!4+UdaMlIx^}!w}Rjk8vm%^tRUAmpA0drkDr=;%^s=&x?}%_m=`Ey zUaYFEW_#)V(UVcFhtm@v6gffjvuy|kKg1{rAP$w;J$v!_{(!-4nudVI6kE(ROQ0_! zws14)ZhO+B_$AJqGKTzdEg|=GdBTUHlaN&6n_)lx$e>+#gqZNg4LvJbWM|_EvhEFS z*L~&06LuHVl_D%r~ z9wC(zF!O+BtT=UoM}tw*Cv09ZUyVw!W3(^?w!6DqV%G3;5o{-x2}jbRB?av%arGUv zN*O=LrtyVMn&KeE9KWd>kF)1@WX7}}Jb;_&`^4XuXFtcK2UBJfbXGA3H0ce`S)Av)zjGGPn3jlz zd|FBh#&NfyR9aE7ZW5Ek_NVjiwF(u@U;$CbnM3udcO5xRkmGRSMzp(d?FaRAi>Xvg z*awsBWP>s*wr{uf6EsbrHGjBYS+F7&!v%e(ekFU zm-2;0{t4#_aKe!|<2_+nhC;khKpu^os(A74-T4Q-45BWr{O?v@-(F2|s?2>@ZXWdO zG0pP3%M%Tzl0v~dQ!vREpOq1&u8*dFhM2MNZA}(h5JR6T>=iZ_Ts+@lDIeiE9xrDU zcLE1!`fxf@S|P`Mz73HiAmW{jodbROoOP4iPH{the5(?_nqL;gF3=hnGd}(c zoXP24V;%Z@J)ZxZFba1~i+>Ldf%+%rXzUk$tans?m0=|^ox8RCYjem3-MbTz+@r3p zj@3p75w2LY5j=Y_2R+wXGH|?^xku)ml?>ip7Y*5~HB{USPFsK3a82c6p?Ib@Au);h zQmK=jI4cc9IOG>qCw~8ao)E(zTJf^Ra6s8dd0*D)jN7$5y9on^L7{ie?^DGuaSoo< zGuKxK4{4pPu-<=v?OL7;Z*JpyYc&(PrK4-e^7r27WTl@PL;9+%L8Y|x8yc%=b0XZ>L)Bz)w?#8RsR(f}9Lj{lpy|Ke3Z8y3nt?IxTU&Vk{X-|Aox0Ue&hd zd7|!Op+wNnL2T`p#YBVnoJ!xx@QNBeQeW0R&71SFI-Eg&zmxyH$j~`#mZaMgKl@8< zMf}~N1TPB7=k7h&OZ#88Z^~YDu9a#!Nz3tKT^8PqKs&xgXPz-v!^p3SY79fg=2?m!a2W5H48*`o4E5n zfmaair-b7|2lJsc^}FtOpI0@lw}#8be0jSo5!bnuqGNsZtkDC*{?9ANkH-yIjg0O) z0x?aMG)Fu0f~?K0Bxe(3lk$Vxk66g;*_b}pR_eu}@j}giP0)!z*D!>y62R4_XEil| z3t|Lbcz-`9w-|X&?EA!XlK%qkDrtU;!l^2xqJ3y);4g7{UFXs=ALumSV>jdIzQgr% zjoW=tq;k8(q3yFdDlMPJ?=`RsXNsw0R8RYdTuo|*=ft47_i)rR?~l??j@@EM)--8*|^pZ ztr}nXrkwU_VhPArtxBZrVPc#b>yO>BT0Q!Cc|9=MCs1nC(B)!!&nvk0FcY$>n1V zOXackJAF3+FzQ!qcK8$kSeJs$vzCUihmwLw0mRX|1n=daADn(%Huql?Yu+CtZTu(? z_8#%3jcY8BG6meA%~X}NpXCrE9knfoh~}+Qw_cT3kM97%SA{*%-Tg=_wEYgDi`*a9 z-}7MOtLQEY92f2dZKoa*L?h4zyWjZ%CfVIN2S1M4NnaX>tvStJ!(i#>i^I7=AW5Q0 z@84AmwleqoNT?>B)o?5?&k?vICeetjd(lIqL4^PW=@I1F5CRZ9uE5Ove)P|G27U5oVm2rv!DO4f>rcs`bbN&;|v@$!~2%=*eVfX z(xj!Wo*ulLUbp_*<#BFR7L@HyUd4fLu$1HoZDtsrZoXC)-QN9aw>)HNRP+kl)W(CI z^RizGk$ijOeU{2;T|Wyp#J9OC?C9|=33yV}e4E9bRfw;2=9d%Q{M+9NQg<5p+QpaC zF6SOr|0Y`t@JzL99^-iH!i>6H$l+CK|8@G~(&_jyz55sBMPAZZTc1YWL&b}kcW_``!T$wjdY*6~3ft>d2Pk&2(#`X6iZRGC#TadK0XPYt~n z0@S;swmmNmfHZDx$}DQWAAuaoo|K2&h4PLG>`l|d*moSELp`wqS76Rh_O@pX$mG)- zH!$nm%=3ax|BzBMaGT>b9F!5KlDP|=3aY~2)Lz284^fDJh^^j`%}Dt^-YRB?%nPoe zj<ymx+P0^6TRrkqLKDg=?Q;08YE+u+MF}(H|Br)H~Np!8q>+^c02TLp=)c zjXPJwHNDSpHfSz88>u~l{;Vv2LN#&pVACl zj6z*<_GAbyV$fWIUKmWcT4vXsj`w+68}F2p*k>db0COGoEd+i60sKwIx_;9*#xTuHCG+s6n&Gr@DtWfzB7Tx-!ZZo|*_ zn}M;8v*XG3 z^O=z=c3e&^e7cxd`BTH!Lh56)Nif0ft-Y^w>4Tdi91L!nY)Y=JW~!-r8bQ6ch$|Da zL8EU$&|_PJ$>_;-Kd_4k9-O^S`mOiSdcf!bdY~nF>?%F~awsE5!jp*(go)PcLUOeh zvs{_49|aMX>$Thi8XTbGS`R7&*}25Dm56C~l`OoE`wqQ>x+8G7`Q8;nMSt;2(!9>kl9j9fp5pg}OCoSS_FVWykfk^v3moZ3(xg zEgn+iCZ}P!`QyOA3@<2$t=p`gu5N22_Z6dBS3wKv-ml>rga8U9&!yN*rR*O}vZ&CV zS5e9S^huSr5sjH373AhJwqt71C(39-VoAjCRpIzse}L+Hv?kB5)J;BRrR=HQu8owb zvR?OHLzm#~mtryZ^2t^kX8~ru6rxB$kQb=k58AG8k%cP5^Y?9ZvKYap=)CQ?kDo3lM8iMDbguX{(-MMrP}?Yz9s_Eg$! z(c^ychClta{FtVnDY0mw?9HtEWsHxHD@gx*$G#skd#tU6b+V7{(Dr6gp> zUh5t0P35l7PKQ}E+|mOGrpPx7_RLsCdKu0ZFSpUKS3jGFMN@!Hhe0<9bo~_(V zLt04wc5D43a7a58k-Hm&47W`h_w}nKdqrDW=C2;I@{TEak*VcBkT-j4z&{Y{OKN2$ zq`SMDXG;+Zg8{tv;ooRN%|FNg3@HB{W6s_<;zSrzA*gI38)0zcZP0~#H~(PHFHgzT ztuxAZK^}D(v1%nzGcMzMvIXr%71j^ZjbK({- zyi}kjboOtnZOzL}>pnG*nxm|E;SrvOz`!+nlNuKfXf)rypd)G+;~}Df7V?Hh8?XCa zhex<;969}xoV2iKKhz%JiBb*H5*Ds(EIHy|tVb8YvqJCigs4Ae`j2q?->q{*q5K8q zcVg?l#q>XRrM2<165_#Acp$CbgOb5r&AB^3iij{-MkM5!#>2-IIFu~}%1TS6s4|rV zTw(iu76Z!UR9}^%wQ4rYFeN!3d~L;x_o(+b^Z!czg*pG_5YynlUi5z`+NWxOzXmY~ U{S>BsDu7N?9j;cUYWeKH07gwlNdN!< literal 0 HcmV?d00001 diff --git a/docs/.gitbook/assets/basic-architecture-diagram (2).svg b/docs/.gitbook/assets/basic-architecture-diagram (2).svg new file mode 100644 index 00000000000..b707f490461 --- /dev/null +++ b/docs/.gitbook/assets/basic-architecture-diagram (2).svg @@ -0,0 +1 @@ + \ No newline at end of file diff --git a/docs/.gitbook/assets/basic-architecture-diagram (3).svg b/docs/.gitbook/assets/basic-architecture-diagram (3).svg new file mode 100644 index 00000000000..b707f490461 --- /dev/null +++ b/docs/.gitbook/assets/basic-architecture-diagram (3).svg @@ -0,0 +1 @@ + \ No newline at end of file diff --git a/docs/.gitbook/assets/feast-docs-overview-diagram-2 (2).svg b/docs/.gitbook/assets/feast-docs-overview-diagram-2 (2).svg new file mode 100644 index 00000000000..7f30963ec78 --- /dev/null +++ b/docs/.gitbook/assets/feast-docs-overview-diagram-2 (2).svg @@ -0,0 +1 @@ + \ No newline at end of file diff --git a/docs/.gitbook/assets/feast-docs-overview-diagram-2 (3).svg b/docs/.gitbook/assets/feast-docs-overview-diagram-2 (3).svg new file mode 100644 index 00000000000..7f30963ec78 --- /dev/null +++ b/docs/.gitbook/assets/feast-docs-overview-diagram-2 (3).svg @@ -0,0 +1 @@ + \ No newline at end of file diff --git a/docs/README.md b/docs/README.md index 9593e9f26e9..aa0be6090d4 100644 --- a/docs/README.md +++ b/docs/README.md @@ -1,4 +1,4 @@ -# What is Feast? +# Introduction Feast \(**Fea**ture **St**ore\) is a tool for managing and serving machine learning features. @@ -13,7 +13,7 @@ Feast aims to: ![](.gitbook/assets/feast-docs-overview-diagram-2.svg) -**TL;DR:** Feast decouples feature engineering from feature usage. Features that are added to Feast become available immediately for training and serving. Models can retrieve the same features used in training from a low latency online store in production. +Feast decouples feature engineering from feature usage. Features that are added to Feast become available immediately for training and serving. Models can retrieve the same features used in training from a low latency online store in production. This means that new ML projects start with a process of feature selection from a catalog instead of having to do feature engineering from scratch. diff --git a/docs/SUMMARY.md b/docs/SUMMARY.md index 345b49fe0d1..f6139c90212 100644 --- a/docs/SUMMARY.md +++ b/docs/SUMMARY.md @@ -1,27 +1,33 @@ # Table of contents -* [What is Feast?](README.md) - -## Introduction - -* [Why Feast?](introduction/why-feast.md) -* [Getting Help](introduction/getting-help.md) -* [Roadmap](introduction/roadmap.md) +* [Introduction](README.md) +* [Why Feast?](why-feast.md) +* [Getting Started](getting-started/README.md) + * [Deploying Feast](getting-started/deploying-feast/README.md) + * [Docker Compose](getting-started/deploying-feast/docker-compose.md) + * [Kubernetes \(GKE\)](getting-started/deploying-feast/kubernetes.md) + * [Connecting to Feast](getting-started/connecting-to-feast-1/README.md) + * [Python SDK](getting-started/connecting-to-feast-1/python-sdk.md) + * [Feast CLI](getting-started/connecting-to-feast-1/connecting-to-feast.md) +* [Roadmap](roadmap.md) * [Changelog](https://github.com/feast-dev/feast/blob/master/CHANGELOG.md) +* [Community](getting-help.md) -## Concepts - -* [Concepts](concepts/concepts.md) - -## Installation +## User Guide -* [Overview](installation/overview.md) -* [Docker Compose](installation/docker-compose.md) -* [Google Kubernetes Engine \(GKE\)](installation/gke.md) +* [Concepts](user-guide/overview.md) +* [Architecture](user-guide/architecture.md) +* [Feature Sets](user-guide/feature-sets.md) +* [Entities](user-guide/entities.md) +* [Features](user-guide/features.md) +* [Sources](user-guide/sources.md) +* [Data ingestion](user-guide/data-ingestion.md) +* [Stores](user-guide/stores.md) +* [Feature retrieval](user-guide/feature-retrieval.md) ## Tutorials -* [Basic](https://github.com/feast-dev/feast/blob/master/examples/basic/basic.ipynb) +* [Basic Tutorial](https://github.com/feast-dev/feast/blob/master/examples/basic/basic.ipynb) * [Churn Prediction \(XGBoost\)](https://github.com/feast-dev/feast/blob/master/examples/feast-xgboost-churn-prediction-tutorial/Telecom%20Customer%20Churn%20Prediction%20%28with%20Feast%20and%20XGBoost%29.ipynb) ## Administration @@ -30,11 +36,13 @@ ## Reference -* [Python SDK](https://api.docs.feast.dev/python/) -* [Go SDK](https://godoc.org/github.com/feast-dev/feast/sdk/go) -* [gRPC Types](https://api.docs.feast.dev/grpc/feast.types.pb.html) -* [Core gRPC API](https://api.docs.feast.dev/grpc/feast.core.pb.html) -* [Serving gRPC API](https://api.docs.feast.dev/grpc/feast.serving.pb.html) +* [Configuration Reference](reference/configuration-reference.md) +* [API](reference/api/README.md) + * [Core gRPC API](https://api.docs.feast.dev/grpc/feast.core.pb.html) + * [Serving gRPC API](https://api.docs.feast.dev/grpc/feast.serving.pb.html) + * [gRPC Types](https://api.docs.feast.dev/grpc/feast.types.pb.html) + * [Go SDK](https://godoc.org/github.com/feast-dev/feast/sdk/go) + * [Python SDK](https://api.docs.feast.dev/python/) ## Contributing @@ -42,4 +50,5 @@ * [Development Guide](contributing/development-guide.md) * [Style Guide](contributing/style-guide.md) * [Release Process](contributing/release-process.md) +* [Adding a New Store](contributing/adding-a-new-store.md) diff --git a/docs/.gitbook/assets/feast_logo.png b/docs/assets/feast_logo.png similarity index 100% rename from docs/.gitbook/assets/feast_logo.png rename to docs/assets/feast_logo.png diff --git a/docs/contributing/adding-a-new-store-1.md b/docs/contributing/adding-a-new-store-1.md new file mode 100644 index 00000000000..56c06b9b4b6 --- /dev/null +++ b/docs/contributing/adding-a-new-store-1.md @@ -0,0 +1,87 @@ +# Adding a New Store + +The following guide will explain the process of adding a new store through the introduction of a storage connector. + +## 1. Storage API + +Feast has an external module where storage interfaces are defined: [Storage API](https://github.com/gojek/feast/tree/master/storage/api/src/main/java/feast/storage/api) + +Feast interacts with a store at three points . + +1. **During initialization:** Store configuration is loaded into memory by Feast Serving and synchronized with Feast Core +2. **During ingestion of feature data.** [writer interfaces](https://github.com/gojek/feast/tree/master/storage/api/src/main/java/feast/storage/api/writer) are used by the Apache Beam ingestion jobs in order to populate stores \(historical or online\). +3. **During retrieval of feature data:** [Retrieval interfaces](https://github.com/gojek/feast/tree/master/storage/api/src/main/java/feast/storage/api/retriever) are used by Feast Serving in order to read data from stores in order to create training datasets or to serve online data. + +All three of these components should be implemented in order to have a complete storage connector. + +## 2. Adding a Storage Connector + +### 2.1 Initialization and configuration + +Stores are configured in Feast Serving. Feast Serving publishes its store configuration to Feast Core, after which Feast Core can start ingestion/population jobs to populate it. + +Store configuration is always in the form of a map<String, String>. The keys and configuration for stores are defined in [protos](https://github.com/gojek/feast/blob/master/protos/feast/core/Store.proto). This must be added in order to define a new store + +Then the store must be configured to be loaded through Feast Serving. The above configuration is loaded through [FeastProperties.java](https://github.com/gojek/feast/blob/a1937c374a4e39b7a75d828e7b7c3b87a64d9d6e/serving/src/main/java/feast/serving/config/FeastProperties.java#L175). + +Once configuration is loaded, the store will then be instantiated. + +* Feast Core: The [StoreUtil.java](https://github.com/gojek/feast/blob/master/ingestion/src/main/java/feast/ingestion/utils/StoreUtil.java#L85) instantiates new stores for the purposes of feature ingestion. +* Feast Serving: The [ServingServiceConfig](https://github.com/gojek/feast/blob/a1937c374a4e39b7a75d828e7b7c3b87a64d9d6e/serving/src/main/java/feast/serving/config/ServingServiceConfig.java#L56) instantiates new stores for the purposes of retrieval + +{% hint style="info" %} +In the future we plan to provide a plugin interface for adding stores. +{% endhint %} + +### 2.2 Feature Ingestion \(Writer\) + +Feast creates and manages ingestion/population jobs that stream in data from upstream data sources. Currently Feast only supports Kafka as a data source, meaning these jobs are all long running. Batch ingestion \(from users\) results in data being pushed to Kafka topics after which they are picked up by these "population" jobs and written to stores. + +In order for ingestion to succeed, the destination store must be writable. This means that Feast must be able to create the appropriate tables/schemas in the store and also write data from the population job into the store. + +Currently Feast Core starts and manages these population jobs that ingest data into stores \(although we are planning to move this responsibility to the serving layer\). Feast Core starts an [Apache Beam](https://beam.apache.org/) job which synchronously runs migrations on the destination store and subsequently starts consuming [FeatureRows](https://github.com/gojek/feast/blob/master/protos/feast/types/FeatureRow.proto) from Kafka and writing it into stores using a [writer](https://github.com/gojek/feast/tree/master/storage/api/src/main/java/feast/storage/api/writer). + +Below is a "happy path" of a batch ingestion process which includes a blocking step at the Python SDK. + +![](https://user-images.githubusercontent.com/6728866/74807906-91e73c00-5324-11ea-8ba5-2b43c7c5282b.png) + + + +The complete ingestion flow is executed by a [FeatureSink](https://github.com/gojek/feast/blob/master/storage/api/src/main/java/feast/storage/api/writer/FeatureSink.java). Two methods should be implemented + +* [prepareWrite\(\)](https://github.com/gojek/feast/blob/a1937c374a4e39b7a75d828e7b7c3b87a64d9d6e/storage/api/src/main/java/feast/storage/api/writer/FeatureSink.java#L45): Sets up storage backend for writing/ingestion. This method will be called once during pipeline initialisation. Typically this is used to apply schemas. +* [writer\(\)](https://github.com/gojek/feast/blob/a1937c374a4e39b7a75d828e7b7c3b87a64d9d6e/storage/api/src/main/java/feast/storage/api/writer/FeatureSink.java#L53): Retrieves an Apache Beam PTransform that is used to write data to this store. + +### 2.2 Feature Serving \(Retriever\) + +Feast Serving can serve both historical/batch features and online features. Depending on the store that is being added, you should implement either a historical/batch store or an online storage. + +#### 2.2.1 Historical Serving + +The historical serving interface is defined through the [HistoricalRetriever](https://github.com/gojek/feast/blob/master/storage/api/src/main/java/feast/storage/api/retriever/HistoricalRetriever.java) interface. Historical retrieval is an asynchronous process. The client submits a request for a dataset to be produced, and polls until it is ready. + +![High-level flow for batch retrieval](https://user-images.githubusercontent.com/6728866/74797157-702a8c80-5305-11ea-8901-bf6f4eb075f9.png) + +The current implementation of batch retrieval starts and ends with a file \(dataset\) in a Google Cloud Storage bucket. The user ingests an entity dataset. This dataset is loaded into a store \(BigQuery0, joined to features in a point-in-time correct way, then exported again to the bucket. + +Additionally, we have also implemented a [batch retrieval method ](https://github.com/gojek/feast/blob/a1937c374a4e39b7a75d828e7b7c3b87a64d9d6e/sdk/python/feast/client.py#L509)in the Python SDK. Depending on the means through which this new store will export data, this client may have to change. At the very least it would change if Google Cloud Storage isn't used as the staging bucket. + +The means through which you implement the export/import of data into the store will depend on your store. + +#### 2.2.2 Online Serving + +In the case of online serving it is necessary to implement an [OnlineRetriever](https://github.com/gojek/feast/blob/master/storage/api/src/main/java/feast/storage/api/retriever/OnlineRetriever.java). This online retriever will read rows directly and synchronously from an online database. The exact encoding strategy you use to store your data in the store would be defined in the FeatureSink. The OnlineRetriever is expected to read and decode those rows. + +## 3. Storage Connectors Examples + +Feast currently provides support for the following storage types + +Historical storage + +* [BigQuery](https://github.com/gojek/feast/tree/master/storage/connectors/bigquery) + +Online storage + +* [Redis](https://github.com/gojek/feast/tree/master/storage/connectors/redis) +* [Redis Cluster](https://github.com/gojek/feast/tree/master/storage/connectors/rediscluster) + diff --git a/docs/contributing/adding-a-new-store.md b/docs/contributing/adding-a-new-store.md new file mode 100644 index 00000000000..56c06b9b4b6 --- /dev/null +++ b/docs/contributing/adding-a-new-store.md @@ -0,0 +1,87 @@ +# Adding a New Store + +The following guide will explain the process of adding a new store through the introduction of a storage connector. + +## 1. Storage API + +Feast has an external module where storage interfaces are defined: [Storage API](https://github.com/gojek/feast/tree/master/storage/api/src/main/java/feast/storage/api) + +Feast interacts with a store at three points . + +1. **During initialization:** Store configuration is loaded into memory by Feast Serving and synchronized with Feast Core +2. **During ingestion of feature data.** [writer interfaces](https://github.com/gojek/feast/tree/master/storage/api/src/main/java/feast/storage/api/writer) are used by the Apache Beam ingestion jobs in order to populate stores \(historical or online\). +3. **During retrieval of feature data:** [Retrieval interfaces](https://github.com/gojek/feast/tree/master/storage/api/src/main/java/feast/storage/api/retriever) are used by Feast Serving in order to read data from stores in order to create training datasets or to serve online data. + +All three of these components should be implemented in order to have a complete storage connector. + +## 2. Adding a Storage Connector + +### 2.1 Initialization and configuration + +Stores are configured in Feast Serving. Feast Serving publishes its store configuration to Feast Core, after which Feast Core can start ingestion/population jobs to populate it. + +Store configuration is always in the form of a map<String, String>. The keys and configuration for stores are defined in [protos](https://github.com/gojek/feast/blob/master/protos/feast/core/Store.proto). This must be added in order to define a new store + +Then the store must be configured to be loaded through Feast Serving. The above configuration is loaded through [FeastProperties.java](https://github.com/gojek/feast/blob/a1937c374a4e39b7a75d828e7b7c3b87a64d9d6e/serving/src/main/java/feast/serving/config/FeastProperties.java#L175). + +Once configuration is loaded, the store will then be instantiated. + +* Feast Core: The [StoreUtil.java](https://github.com/gojek/feast/blob/master/ingestion/src/main/java/feast/ingestion/utils/StoreUtil.java#L85) instantiates new stores for the purposes of feature ingestion. +* Feast Serving: The [ServingServiceConfig](https://github.com/gojek/feast/blob/a1937c374a4e39b7a75d828e7b7c3b87a64d9d6e/serving/src/main/java/feast/serving/config/ServingServiceConfig.java#L56) instantiates new stores for the purposes of retrieval + +{% hint style="info" %} +In the future we plan to provide a plugin interface for adding stores. +{% endhint %} + +### 2.2 Feature Ingestion \(Writer\) + +Feast creates and manages ingestion/population jobs that stream in data from upstream data sources. Currently Feast only supports Kafka as a data source, meaning these jobs are all long running. Batch ingestion \(from users\) results in data being pushed to Kafka topics after which they are picked up by these "population" jobs and written to stores. + +In order for ingestion to succeed, the destination store must be writable. This means that Feast must be able to create the appropriate tables/schemas in the store and also write data from the population job into the store. + +Currently Feast Core starts and manages these population jobs that ingest data into stores \(although we are planning to move this responsibility to the serving layer\). Feast Core starts an [Apache Beam](https://beam.apache.org/) job which synchronously runs migrations on the destination store and subsequently starts consuming [FeatureRows](https://github.com/gojek/feast/blob/master/protos/feast/types/FeatureRow.proto) from Kafka and writing it into stores using a [writer](https://github.com/gojek/feast/tree/master/storage/api/src/main/java/feast/storage/api/writer). + +Below is a "happy path" of a batch ingestion process which includes a blocking step at the Python SDK. + +![](https://user-images.githubusercontent.com/6728866/74807906-91e73c00-5324-11ea-8ba5-2b43c7c5282b.png) + + + +The complete ingestion flow is executed by a [FeatureSink](https://github.com/gojek/feast/blob/master/storage/api/src/main/java/feast/storage/api/writer/FeatureSink.java). Two methods should be implemented + +* [prepareWrite\(\)](https://github.com/gojek/feast/blob/a1937c374a4e39b7a75d828e7b7c3b87a64d9d6e/storage/api/src/main/java/feast/storage/api/writer/FeatureSink.java#L45): Sets up storage backend for writing/ingestion. This method will be called once during pipeline initialisation. Typically this is used to apply schemas. +* [writer\(\)](https://github.com/gojek/feast/blob/a1937c374a4e39b7a75d828e7b7c3b87a64d9d6e/storage/api/src/main/java/feast/storage/api/writer/FeatureSink.java#L53): Retrieves an Apache Beam PTransform that is used to write data to this store. + +### 2.2 Feature Serving \(Retriever\) + +Feast Serving can serve both historical/batch features and online features. Depending on the store that is being added, you should implement either a historical/batch store or an online storage. + +#### 2.2.1 Historical Serving + +The historical serving interface is defined through the [HistoricalRetriever](https://github.com/gojek/feast/blob/master/storage/api/src/main/java/feast/storage/api/retriever/HistoricalRetriever.java) interface. Historical retrieval is an asynchronous process. The client submits a request for a dataset to be produced, and polls until it is ready. + +![High-level flow for batch retrieval](https://user-images.githubusercontent.com/6728866/74797157-702a8c80-5305-11ea-8901-bf6f4eb075f9.png) + +The current implementation of batch retrieval starts and ends with a file \(dataset\) in a Google Cloud Storage bucket. The user ingests an entity dataset. This dataset is loaded into a store \(BigQuery0, joined to features in a point-in-time correct way, then exported again to the bucket. + +Additionally, we have also implemented a [batch retrieval method ](https://github.com/gojek/feast/blob/a1937c374a4e39b7a75d828e7b7c3b87a64d9d6e/sdk/python/feast/client.py#L509)in the Python SDK. Depending on the means through which this new store will export data, this client may have to change. At the very least it would change if Google Cloud Storage isn't used as the staging bucket. + +The means through which you implement the export/import of data into the store will depend on your store. + +#### 2.2.2 Online Serving + +In the case of online serving it is necessary to implement an [OnlineRetriever](https://github.com/gojek/feast/blob/master/storage/api/src/main/java/feast/storage/api/retriever/OnlineRetriever.java). This online retriever will read rows directly and synchronously from an online database. The exact encoding strategy you use to store your data in the store would be defined in the FeatureSink. The OnlineRetriever is expected to read and decode those rows. + +## 3. Storage Connectors Examples + +Feast currently provides support for the following storage types + +Historical storage + +* [BigQuery](https://github.com/gojek/feast/tree/master/storage/connectors/bigquery) + +Online storage + +* [Redis](https://github.com/gojek/feast/tree/master/storage/connectors/redis) +* [Redis Cluster](https://github.com/gojek/feast/tree/master/storage/connectors/rediscluster) + diff --git a/docs/contributing/contributing.md b/docs/contributing/contributing.md index 805721e3d22..2be097d6ffd 100644 --- a/docs/contributing/contributing.md +++ b/docs/contributing/contributing.md @@ -1,12 +1,10 @@ # Contribution Process -## 1. Contribution process +We use [RFCs](https://en.wikipedia.org/wiki/Request_for_Comments) and [GitHub issues](https://github.com/gojek/feast/issues) to communicate development ideas. The simplest way to contribute to Feast is to leave comments in our [RFCs](https://drive.google.com/drive/u/0/folders/1Lj1nIeRB868oZvKTPLYqAvKQ4O0BksjY) in the [Feast Google Drive](https://drive.google.com/drive/u/0/folders/0AAe8j7ZK3sxSUk9PVA) or our GitHub issues. You will need to join our [Google Group](../getting-help.md) in order to get access. -We use [RFCs](https://en.wikipedia.org/wiki/Request_for_Comments) and [GitHub issues](https://github.com/feast-dev/feast/issues) to communicate development ideas. The simplest way to contribute to Feast is to leave comments in our [RFCs](https://drive.google.com/drive/u/0/folders/1Lj1nIeRB868oZvKTPLYqAvKQ4O0BksjY) in the [Feast Google Drive](https://drive.google.com/drive/u/0/folders/0AAe8j7ZK3sxSUk9PVA) or our GitHub issues. +We follow a process of [lazy consensus](http://community.apache.org/committers/lazyConsensus.html). If you believe you know what the project needs then just start development. If you are unsure about which direction to take with development then please communicate your ideas through a GitHub issue or through our [Slack Channel](../getting-help.md) before starting development. -Please communicate your ideas through a GitHub issue or through our Slack Channel before starting development. - -Please [submit a PR ](https://github.com/feast-dev/feast/pulls)to the master branch of the Feast repository once you are ready to submit your contribution. Code submission to Feast \(including submission from project maintainers\) require review and approval from maintainers or code owners. +Please [submit a PR ](https://github.com/gojek/feast/pulls)to the master branch of the Feast repository once you are ready to submit your contribution. Code submission to Feast \(including submission from project maintainers\) require review and approval from maintainers or code owners. PRs that are submitted by the general public need to be identified as `ok-to-test`. Once enabled, [Prow](https://github.com/kubernetes/test-infra/tree/master/prow) will run a range of tests to verify the submission, after which community members will help to review the pull request. @@ -14,514 +12,3 @@ PRs that are submitted by the general public need to be identified as `ok-to-tes Please sign the [Google CLA](https://cla.developers.google.com/) in order to have your code merged into the Feast repository. {% endhint %} -## 2. Development guide - -### 2.1 Overview - -The following guide will help you quickly run Feast in your local machine. - -The main components of Feast are: - -* **Feast Core:** Handles feature registration, starts and manages ingestion jobs and ensures that Feast internal metadata is consistent. -* **Feast Ingestion Jobs:** Subscribes to streams of FeatureRows and writes these as feature - - values to registered databases \(online, historical\) that can be read by Feast Serving. - -* **Feast Serving:** Service that handles requests for features values, either online or batch. - -### 2.**2 Requirements** - -#### 2.**2.1 Development environment** - -The following software is required for Feast development - -* Java SE Development Kit 11 -* Python version 3.6 \(or above\) and pip -* [Maven ](https://maven.apache.org/install.html)version 3.6.x - -Additionally, [grpc\_cli](https://github.com/grpc/grpc/blob/master/doc/command_line_tool.md) is useful for debugging and quick testing of gRPC endpoints. - -#### 2.**2.2 Services** - -The following components/services are required to develop Feast: - -* **Feast Core:** Requires PostgreSQL \(version 11 and above\) to store state, and requires a Kafka \(tested on version 2.x\) setup to allow for ingestion of FeatureRows. -* **Feast Serving:** Requires Redis \(tested on version 5.x\). - -These services should be running before starting development. The following snippet will start the services using Docker. - -```bash -# Start Postgres -docker run --name postgres --rm -it -d --net host -e POSTGRES_DB=postgres -e POSTGRES_USER=postgres \ --e POSTGRES_PASSWORD=password postgres:12-alpine - -# Start Redis -docker run --name redis --rm -it --net host -d redis:5-alpine - -# Start Zookeeper (needed by Kafka) -docker run --rm \ - --net=host \ - --name=zookeeper \ - --env=ZOOKEEPER_CLIENT_PORT=2181 \ - --detach confluentinc/cp-zookeeper:5.2.1 - -# Start Kafka -docker run --rm \ - --net=host \ - --name=kafka \ - --env=KAFKA_ZOOKEEPER_CONNECT=localhost:2181 \ - --env=KAFKA_ADVERTISED_LISTENERS=PLAINTEXT://localhost:9092 \ - --env=KAFKA_OFFSETS_TOPIC_REPLICATION_FACTOR=1 \ - --detach confluentinc/cp-kafka:5.2.1 -``` - -### 2.3 Testing and development - -#### 2.3.1 Running unit tests - -```text -$ mvn test -``` - -#### 2.3.2 Running integration tests - -_Note: integration suite isn't yet separated from unit._ - -```text -$ mvn verify -``` - -#### 2.3.3 Running components locally - -The `core` and `serving` modules are Spring Boot applications. These may be run as usual for [the Spring Boot Maven plugin](https://docs.spring.io/spring-boot/docs/current/maven-plugin/index.html): - -```text -$ mvn --projects core spring-boot:run - -# Or for short: -$ mvn -pl core spring-boot:run -``` - -Note that you should execute `mvn` from the Feast repository root directory, as there are intermodule dependencies that Maven will not resolve if you `cd` to subdirectories to run. - -#### 2.3.4 Running from IntelliJ - -Compiling and running tests in IntelliJ should work as usual. - -Running the Spring Boot apps may work out of the box in IDEA Ultimate, which has built-in support for Spring Boot projects, but the Community Edition needs a bit of help: - -The Spring Boot Maven plugin automatically puts dependencies with `provided` scope on the runtime classpath when using `spring-boot:run`, such as its embedded Tomcat server. The "Play" buttons in the gutter or right-click menu of a `main()` method [do not do this](https://stackoverflow.com/questions/30237768/run-spring-boots-main-using-ide). - -A solution to this is: - -1. Open `View > Tool Windows > Maven` -2. Drill down to e.g. `Feast Core > Plugins > spring-boot:run`, right-click and `Create 'feast-core [spring-boot'…` -3. In the dialog that pops up, check the `Resolve Workspace artifacts` box -4. Recommended: add `-Dspring-boot.run.fork=false` to the `Command line` field to get Debug working too -5. Click `OK`. You should now be able to select this run configuration for the Play button in the main toolbar, keyboard shortcuts, etc. - -It is recommend to have IntelliJ delegate building to Maven, if this is not enabled out of the box when you import the project, for greater assurance that build behavior is consistent with CI / production builds. This is set in Preferences at `Build, Execution, Deployment > Build Tools > Maven > Runner > Delegate IDE build/run actions to Maven`. - -### 2.**4** Validating your setup - -The following section is a quick walk-through to test whether your local Feast deployment is functional for development purposes. - -**2.4.1 Assumptions** - -* PostgreSQL is running in `localhost:5432` and has a database called `postgres` which - - can be accessed with credentials user `postgres` and password `password`. Different database configurations can be supplied here \(`/core/src/main/resources/application.yml`\) - -* Redis is running locally and accessible from `localhost:6379` -* \(optional\) The local environment has been authentication with Google Cloud Platform and has full access to BigQuery. This is only necessary for BigQuery testing/development. - -#### 2.4.2 Clone Feast - -```bash -git clone https://github.com/feast-dev/feast.git && cd feast && \ -export FEAST_HOME_DIR=$(pwd) -``` - -#### 2.4.3 Starting Feast Core - -To run Feast Core locally using Maven: - -```bash -# Feast Core can be configured from the following .yml file -# $FEAST_HOME_DIR/core/src/main/resources/application.yml -mvn --projects core spring-boot:run -``` - -Test whether Feast Core is running - -```text -grpc_cli call localhost:6565 ListStores '' -``` - -The output should list **no** stores since no Feast Serving has registered its stores to Feast Core: - -```text -connecting to localhost:6565 - -Rpc succeeded with OK status -``` - -#### 2.4.4 Starting Feast Serving - -Feast Serving is configured through the `$FEAST_HOME_DIR/serving/src/main/resources/application.yml`. Each Serving deployment must be configured with a store. The default store is Redis \(used for online serving\). - -The configuration for this default store is located in a separate `.yml` file. The default location is `$FEAST_HOME_DIR/serving/sample_redis_config.yml`: - -```text -name: serving -type: REDIS -redis_config: - host: localhost - port: 6379 -subscriptions: - - name: "*" - project: "*" - version: "*" -``` - -Once Feast Serving is started, it will register its store with Feast Core \(by name\) and start to subscribe to a feature sets based on its subscription. - -Start Feast Serving GRPC server on localhost:6566 with store name `serving` - -```text -mvn --projects serving spring-boot:run -``` - -Test connectivity to Feast Serving - -```text -grpc_cli call localhost:6566 GetFeastServingInfo '' -``` - -```text -connecting to localhost:6566 -version: "0.4.2-SNAPSHOT" -type: FEAST_SERVING_TYPE_ONLINE - -Rpc succeeded with OK status -``` - -Test Feast Core to see whether it is aware of the Feast Serving deployment - -```text -grpc_cli call localhost:6565 ListStores '' -``` - -```text -connecting to localhost:6565 -store { - name: "serving" - type: REDIS - subscriptions { - name: "*" - version: "*" - project: "*" - } - redis_config { - host: "localhost" - port: 6379 - } -} - -Rpc succeeded with OK status -``` - -In order to use BigQuery as a historical store, it is necessary to start Feast Serving with a different store type. - -Copy `$FEAST_HOME_DIR/serving/sample_redis_config.yml` to the following location `$FEAST_HOME_DIR/serving/my_bigquery_config.yml` and update the configuration as below: - -```text -name: bigquery -type: BIGQUERY -bigquery_config: - project_id: YOUR_GCP_PROJECT_ID - dataset_id: YOUR_GCP_DATASET -subscriptions: - - name: "*" - version: "*" - project: "*" -``` - -Then inside `serving/src/main/resources/application.yml` modify the following key `feast.store.config-path` to point to the new store configuration. - -After making these changes, restart Feast Serving: - -```text -mvn --projects serving spring-boot:run -``` - -You should see two stores registered: - -```text -store { - name: "serving" - type: REDIS - subscriptions { - name: "*" - version: "*" - project: "*" - } - redis_config { - host: "localhost" - port: 6379 - } -} -store { - name: "bigquery" - type: BIGQUERY - subscriptions { - name: "*" - version: "*" - project: "*" - } - bigquery_config { - project_id: "my_project" - dataset_id: "my_bq_dataset" - } -} -``` - -#### 2.4.5 Registering a FeatureSet - -Before registering a new FeatureSet, a project is required. - -```text -grpc_cli call localhost:6565 CreateProject ' - name: "your_project_name" -' -``` - -When a feature set is successfully registered, Feast Core will start an **ingestion** job that listens for new features in the feature set. - -{% hint style="info" %} -Note that Feast currently only supports source of type `KAFKA`, so you must have access to a running Kafka broker to register a FeatureSet successfully. It is possible to omit the `source` from a Feature Set, but Feast Core will still use Kafka behind the scenes, it is simply abstracted away from the user. -{% endhint %} - -Create a new FeatureSet in Feast by sending a request to Feast Core: - -```text -# Example of registering a new driver feature set -# Note the source value, it assumes that you have access to a Kafka broker -# running on localhost:9092 - -grpc_cli call localhost:6565 ApplyFeatureSet ' -feature_set { - spec { - project: "your_project_name" - name: "driver" - version: 1 - - entities { - name: "driver_id" - value_type: INT64 - } - - features { - name: "city" - value_type: STRING - } - - source { - type: KAFKA - kafka_source_config { - bootstrap_servers: "localhost:9092" - topic: "your-kafka-topic" - } - } - } -} -' -``` - -Verify that the FeatureSet has been registered correctly. - -```text -# To check that the FeatureSet has been registered correctly. -# You should also see logs from Feast Core of the ingestion job being started -grpc_cli call localhost:6565 GetFeatureSet ' - project: "your_project_name" - name: "driver" -' -``` - -Or alternatively, list all feature sets - -```text -grpc_cli call localhost:6565 ListFeatureSets ' - filter { - project: "your_project_name" - feature_set_name: "driver" - feature_set_version: "1" - } -' -``` - -#### 2.4.6 Ingestion and Population of Feature Values - -```text -# Produce FeatureRow messages to Kafka so it will be ingested by Feast -# and written to the registered stores. -# Make sure the value here is the topic assigned to the feature set -# ... producer.send("feast-driver-features" ...) -# -# Install Python SDK to help writing FeatureRow messages to Kafka -cd $FEAST_HOMEDIR/sdk/python -pip3 install -e . -pip3 install pendulum - -# Produce FeatureRow messages to Kafka so it will be ingested by Feast -# and written to the corresponding store. -# Make sure the value here is the topic assigned to the feature set -# ... producer.send("feast-test_feature_set-features" ...) -python3 - <` property, and commit. -1. Push. For a new release branch, open a PR against master. -1. When CI passes, merge. (Remember _not_ to delete the new release branch). -1. Tag the merge commit with the release version, using a `v` prefix. Push the tag. -1. Bump to the next working version and append `-SNAPSHOT` in `pom.xml`. -1. Commit the POM and open a PR. -1. Create a [GitHub release](https://github.com/feast-dev/feast/releases) which includes a summary of important changes as well as any artifacts associated with the release. Make sure to include the same change log as added in [CHANGELOG.md]. Use `Feast vX.Y.Z` as the title. -1. Create one final PR to the master branch and also update its [CHANGELOG.md]. +2. Update the [CHANGELOG.md](https://github.com/feast-dev/feast/blob/master/CHANGELOG.md). See the [Creating a change log](release-process.md#creating-a-change-log) guide. +3. In the root `pom.xml`, remove `-SNAPSHOT` from the `` property, and commit. +4. Push. For a new release branch, open a PR against master. +5. When CI passes, merge. \(Remember _not_ to delete the new release branch\). +6. Tag the merge commit with the release version, using a `v` prefix. Push the tag. +7. Bump to the next working version and append `-SNAPSHOT` in `pom.xml`. +8. Commit the POM and open a PR. +9. Create a [GitHub release](https://github.com/feast-dev/feast/releases) which includes a summary of important changes as well as any artifacts associated with the release. Make sure to include the same change log as added in [CHANGELOG.md](https://github.com/feast-dev/feast/blob/master/CHANGELOG.md). Use `Feast vX.Y.Z` as the title. +10. Create one final PR to the master branch and also update its [CHANGELOG.md](https://github.com/feast-dev/feast/blob/master/CHANGELOG.md). When a tag that matches a Semantic Version string is pushed, CI will automatically build and push the relevant artifacts to their repositories or package managers \(docker images, Python wheels, etc\). JVM artifacts are promoted from Sonatype OSSRH to Maven Central, but it sometimes takes some time for them to be available. -[CHANGELOG.md]: https://github.com/feast-dev/feast/blob/master/CHANGELOG.md - ### Creating a change log -We use an [open source change log generator](https://hub.docker.com/r/ferrarimarco/github-changelog-generator/) to generate change logs. The process still requires a little bit of manual effort. -1. Create a GitHub token as [per these instructions ](https://github.com/github-changelog-generator/github-changelog-generator#github-token). The token is used as an input argument (`-t`) to the changelog generator. -2. The change log generator configuration below will look for unreleased changes on a specific branch. The branch will be `master` for a major/minor release, or a release branch (`v0.4-branch`) for a patch release. You will need to set the branch using the `--release-branch` argument. -3. You should also set the `--future-release` argument. This is the version you are releasing. The version can still be changed at a later date. -4. Update the arguments below and run the command to generate the change log to the console. -``` +We use an [open source change log generator](https://hub.docker.com/r/ferrarimarco/github-changelog-generator/) to generate change logs. The process still requires a little bit of manual effort. 1. Create a GitHub token as [per these instructions ](https://github.com/github-changelog-generator/github-changelog-generator#github-token). The token is used as an input argument \(`-t`\) to the changelog generator. 2. The change log generator configuration below will look for unreleased changes on a specific branch. The branch will be `master` for a major/minor release, or a release branch \(`v0.4-branch`\) for a patch release. You will need to set the branch using the `--release-branch` argument. 3. You should also set the `--future-release` argument. This is the version you are releasing. The version can still be changed at a later date. 4. Update the arguments below and run the command to generate the change log to the console. + +```text docker run -it --rm ferrarimarco/github-changelog-generator \ --user feast-dev \ --project feast \ @@ -55,8 +50,10 @@ docker run -it --rm ferrarimarco/github-changelog-generator \ --max-issues 1 \ -o ``` -5. Review each change log item. - - Make sure that sentences are grammatically correct and well formatted (although we will try to enforce this at the PR review stage). - - Make sure that each item is categorized correctly. You will see the following categories: `Breaking changes`, `Implemented enhancements`, `Fixed bugs`, and `Merged pull requests`. Any unlabeled PRs will be found in `Merged pull requests`. It's important to make sure that any `breaking changes`, `enhancements`, or `bug fixes` are pulled up out of `merged pull requests` into the correct category. Housekeeping, tech debt clearing, infra changes, or refactoring do not count as `enhancements`. Only enhancements a user benefits from should be listed in that category. - - Make sure that the "Full Changelog" link is actually comparing the correct tags (normally your released version against the previously version). - - Make sure that release notes and breaking changes are present. + +1. Review each change log item. + * Make sure that sentences are grammatically correct and well formatted \(although we will try to enforce this at the PR review stage\). + * Make sure that each item is categorized correctly. You will see the following categories: `Breaking changes`, `Implemented enhancements`, `Fixed bugs`, and `Merged pull requests`. Any unlabeled PRs will be found in `Merged pull requests`. It's important to make sure that any `breaking changes`, `enhancements`, or `bug fixes` are pulled up out of `merged pull requests` into the correct category. Housekeeping, tech debt clearing, infra changes, or refactoring do not count as `enhancements`. Only enhancements a user benefits from should be listed in that category. + * Make sure that the "Full Changelog" link is actually comparing the correct tags \(normally your released version against the previously version\). + * Make sure that release notes and breaking changes are present. + diff --git a/docs/contributing/style-guide.md b/docs/contributing/style-guide.md index 322f1e3d034..1cf1c69671f 100644 --- a/docs/contributing/style-guide.md +++ b/docs/contributing/style-guide.md @@ -1,6 +1,8 @@ # Style Guide -## 1. Java +## 1. Language Specific Style Guides + +### 1.1 Java We conform to the [Google Java Style Guide](https://google.github.io/styleguide/javaguide.html). Maven can helpfully take care of that for you before you commit: @@ -17,11 +19,25 @@ $ mvn verify -Dspotless.check.skip If you're using IntelliJ, you can import [these code style settings](https://github.com/google/styleguide/blob/gh-pages/intellij-java-google-style.xml) if you'd like to use the IDE's reformat function as you develop. -## 2. Go +### 1.2 Go Make sure you apply `go fmt`. -## 3. Python +### 1.3 Python We use [Python Black](https://github.com/psf/black) to format our Python code prior to submission. +## 2. Formatting and Linting + +Code can automatically be formatted by running the following command from the project root directory + +```text +make format +``` + +Once code that is submitted through a PR or direct push will be validated with the following command + +```text +make lint +``` + diff --git a/docs/getting-help.md b/docs/getting-help.md new file mode 100644 index 00000000000..32b21461b1d --- /dev/null +++ b/docs/getting-help.md @@ -0,0 +1,36 @@ +# Community + +## Chat + +* Come and say hello in [\#Feast](https://join.slack.com/t/kubeflow/shared_invite/zt-cpr020z4-PfcAue_2nw67~iIDy7maAQ) over in the Kubeflow Slack. + +## GitHub + +* Feast's GitHub repo can be [found here](https://github.com/feast-dev/feast/). +* Found a bug or need a feature? [Create an issue on GitHub](https://github.com/feast-dev/feast/issues/new) + +## Community Call + +We have a community call every 2 weeks. Alternating between two times. + +* 11 am \(UTC + 8\) +* 5 pm \(UTC + 8\) + +Please join the [feast-dev](getting-help.md#feast-development) mailing list to receive the the calendar invitation. + +## Mailing list + +### Feast discussion + +* Google Group: [https://groups.google.com/d/forum/feast-discuss](https://groups.google.com/d/forum/feast-discuss) +* Mailing List: [feast-discuss@googlegroups.com](mailto:feast-discuss@googlegroups.com) + +### Feast development + +* Google Group: [https://groups.google.com/d/forum/feast-dev](https://groups.google.com/d/forum/feast-dev) +* Mailing List: [feast-dev@googlegroups.com](mailto:feast-dev@googlegroups.com) + +## Google Drive + +The Feast community also maintains a [Google Drive](https://drive.google.com/drive/u/0/folders/0AAe8j7ZK3sxSUk9PVA) with documents like RFCs, meeting notes, or roadmaps. Please join one of the above mailing lists \(feast-dev or feast-discuss\) to gain access to the drive. + diff --git a/docs/getting-started/README.md b/docs/getting-started/README.md new file mode 100644 index 00000000000..9945ab197d9 --- /dev/null +++ b/docs/getting-started/README.md @@ -0,0 +1,14 @@ +# Getting Started + +If you want to learn more about Feast concepts and its architecture, see the user guide + +If you would like to connect to an existing Feast deployment then click on `Connecting to Feast` + +{% page-ref page="connecting-to-feast-1/" %} + +If you would like to deploy a new installation of Feast, then click on `Deploying Feast` + +{% page-ref page="deploying-feast/" %} + +If you are connected to a running Feast deployment, then have a look at our [example tutorials](https://github.com/gojek/feast/tree/master/examples). + diff --git a/docs/getting-started/connecting-to-feast-1/README.md b/docs/getting-started/connecting-to-feast-1/README.md new file mode 100644 index 00000000000..7cc7e5d8c5b --- /dev/null +++ b/docs/getting-started/connecting-to-feast-1/README.md @@ -0,0 +1,21 @@ +# Connecting to Feast + +## Python SDK + +* Define, register, and manage entities and features +* Ingest data into Feast +* Build and retrieve training datasets +* Retrieve online features + +{% page-ref page="python-sdk.md" %} + +## Feast CLI + +* Define, register, and manage entities and features from the terminal +* Ingest data into Feast +* Manage ingestion jobs + +{% page-ref page="connecting-to-feast.md" %} + + + diff --git a/docs/getting-started/connecting-to-feast-1/connecting-to-feast.md b/docs/getting-started/connecting-to-feast-1/connecting-to-feast.md new file mode 100644 index 00000000000..a2cafda0db2 --- /dev/null +++ b/docs/getting-started/connecting-to-feast-1/connecting-to-feast.md @@ -0,0 +1,38 @@ +# Feast CLI + +The Feast CLI is installed through pip: + +```bash +pip install feast +``` + +Configure the CLI to connect to your Feast Core deployment + +```text +feast config set core_url your.feast.deployment +``` + +{% hint style="info" %} +By default, all configuration is stored in `~/.feast/config` +{% endhint %} + +The CLI is a wrapper around the [Feast Python SDK](python-sdk.md) + +```aspnet +$ feast + +Usage: feast [OPTIONS] COMMAND [ARGS]... + +Options: + --help Show this message and exit. + +Commands: + config View and edit Feast properties + feature-sets Create and manage feature sets + ingest Ingest feature data into a feature set + projects Create and manage projects + version Displays version and connectivity information +``` + + + diff --git a/docs/getting-started/connecting-to-feast-1/python-sdk.md b/docs/getting-started/connecting-to-feast-1/python-sdk.md new file mode 100644 index 00000000000..2fea0c31af4 --- /dev/null +++ b/docs/getting-started/connecting-to-feast-1/python-sdk.md @@ -0,0 +1,20 @@ +# Python SDK + +The Feast SDK can be installed directly using pip: + +```bash +pip install feast +``` + +Users should then be able to connect to a Feast deployment as follows + +```python +from feast import Client + +# Connect to an existing Feast Core deployment +client = Client(core_url='feast.example.com:6565') + +# Ensure that your client is connected by printing out some feature sets +client.list_feature_sets() +``` + diff --git a/docs/getting-started/deploying-feast/README.md b/docs/getting-started/deploying-feast/README.md new file mode 100644 index 00000000000..4a9aac11bac --- /dev/null +++ b/docs/getting-started/deploying-feast/README.md @@ -0,0 +1,16 @@ +# Deploying Feast + +## Docker Compose + +* Fastest way to get Feast up and running. +* Provides a pre-installed Jupyter Notebook with sample code. + +{% page-ref page="docker-compose.md" %} + +## Kubernetes \(GKE\) + +* Recommended way to install Feast for production use. +* The guide has dependencies on BigQuery, and Google Cloud Storage. + +{% page-ref page="kubernetes.md" %} + diff --git a/docs/getting-started/deploying-feast/docker-compose.md b/docs/getting-started/deploying-feast/docker-compose.md new file mode 100644 index 00000000000..a8895a02529 --- /dev/null +++ b/docs/getting-started/deploying-feast/docker-compose.md @@ -0,0 +1,112 @@ +# Docker Compose + +### Overview + +This guide will bring Feast up using Docker Compose. This will allow you to: + +* Create, register, and manage feature sets +* Ingest feature data into Feast +* Retrieve features for online serving +* Retrieve features for batch serving \(only if using Google Cloud Platform\) + +This guide is split into three parts: + +1. Setting up your environment +2. Starting Feast with **online serving support only** \(does not require GCP\). +3. Starting Feast with support for **both online and batch** serving \(requires GCP\) + +{% hint style="info" %} +The docker compose setup uses Direct Runner for the Apache Beam jobs that populate data stores. Running Beam with the Direct Runner means it does not need a dedicated runner like Flink or Dataflow, but this comes at the cost of performance. We recommend the use of a dedicated runner when running Feast with very large workloads. +{% endhint %} + +### 0. Requirements + +* [Docker compose](https://docs.docker.com/compose/install/) must be installed. +* The following list of TCP ports must be free: + * 6565, 6566, 8888, and 9094. + * Alternatively it is possible to modify port mappings in `/docker-compose/docker-compose.yml`. +* \(for batch serving only\) For batch serving you will also need a [GCP service account key](https://cloud.google.com/iam/docs/creating-managing-service-account-keys) that has access to [Google Cloud Storage](https://cloud.google.com/storage) and [BigQuery](https://cloud.google.com/bigquery). +* \(for batch serving only\) [Google Cloud SDK ](https://cloud.google.com/sdk/install)installed, authenticated, and configured to the project you will use. + +## 1. Set up environment + +Clone the [Feast repository](https://github.com/feast-dev/feast/) and navigate to the `docker-compose` sub-directory: + +```bash +git clone https://github.com/feast-dev/feast.git && \ +cd feast && export FEAST_HOME_DIR=$(pwd) && \ +cd infra/docker-compose +``` + +Make a copy of the `.env.sample` file: + +```bash +cp .env.sample .env +``` + +## 2. Docker Compose for Online Serving Only + +### 2.1 Start Feast \(without batch retrieval support\) + +If you do not require batch serving, then its possible to simply bring up Feast: + +```javascript +docker-compose up -d +``` + +A Jupyter Notebook environment is now available to use Feast: + +[http://localhost:8888/tree/feast/examples](http://localhost:8888/tree/feast/examples) + +## 3. Docker Compose for Online and Batch Serving + +{% hint style="info" %} +Batch serving requires Google Cloud Storage to function, specifically Google Cloud Storage \(GCP\) and BigQuery. +{% endhint %} + +### 3.1 Set up Google Cloud Platform + +Create a [service account ](https://cloud.google.com/iam/docs/creating-managing-service-accounts)from the GCP console and copy it to the `infra/docker-compose/gcp-service-accounts` folder: + +```javascript +cp my-service-account.json ${FEAST_HOME_DIR}/infra/docker-compose/gcp-service-accounts +``` + +Create a Google Cloud Storage bucket. Make sure that your service account above has read/write permissions to this bucket: + +```bash +gsutil mb gs://my-feast-staging-bucket +``` + +### 3.2 Configure .env + +Configure the `.env` file based on your environment. At the very least you have to modify: + +| Parameter | Description | +| :--- | :--- | +| FEAST\_CORE\_GCP\_SERVICE\_ACCOUNT\_KEY | This should be your service account file name, for example `key.json`. | +| FEAST\_BATCH\_SERVING\_GCP\_SERVICE\_ACCOUNT\_KEY | This should be your service account file name, for example `key.json` | +| FEAST\_JUPYTER\_GCP\_SERVICE\_ACCOUNT\_KEY | This should be your service account file name, for example `key.json` | +| FEAST\_JOB\_STAGING\_LOCATION | Google Cloud Storage bucket that Feast will use to stage data exports and batch retrieval requests, for example `gs://your-gcs-bucket/staging` | + +### 3.3 Configure .bq-store.yml + +We will also need to configure the `bq-store.yml` file inside `infra/docker-compose/serving/` to configure the BigQuery storage configuration as well as the feature sets that the store subscribes to. At a minimum you will need to set: + +| Parameter | Description | +| :--- | :--- | +| bigquery\_config.project\_id | This is you [GCP project Id](https://cloud.google.com/resource-manager/docs/creating-managing-projects). | +| bigquery\_config.dataset\_id | This is the name of the BigQuery dataset that tables will be created in. Each feature set will have one table in BigQuery. | + +### 3.4 Start Feast \(with batch retrieval support\) + +Start Feast: + +```javascript +docker-compose up -d +``` + +A Jupyter Notebook environment is now available to use Feast: + +[http://localhost:8888/tree/feast/examples](http://localhost:8888/tree/feast/examples) + diff --git a/docs/getting-started/deploying-feast/kubernetes.md b/docs/getting-started/deploying-feast/kubernetes.md new file mode 100644 index 00000000000..a7432836f3a --- /dev/null +++ b/docs/getting-started/deploying-feast/kubernetes.md @@ -0,0 +1,211 @@ +# Kubernetes \(GKE\) + +### Overview + +This guide will install Feast into a Kubernetes cluster on GCP. It assumes that all of your services will run within a single Kubernetes cluster. Once Feast is installed you will be able to: + +* Define and register features. +* Load feature data from both batch and streaming sources. +* Retrieve features for model training. +* Retrieve features for online serving. + +{% hint style="info" %} +This guide requires [Google Cloud Platform](https://cloud.google.com/) for installation. + +* [BigQuery](https://cloud.google.com/bigquery/) is used for storing historical features. +* [Google Cloud Storage](https://cloud.google.com/storage/) is used for intermediate data storage. +{% endhint %} + +## 0. Requirements + +1. [Google Cloud SDK ](https://cloud.google.com/sdk/install)installed, authenticated, and configured to the project you will use. +2. [Kubectl](https://kubernetes.io/docs/tasks/tools/install-kubectl/) installed. +3. [Helm](https://helm.sh/3) \(2.16.0 or greater\) installed on your local machine with Tiller installed in your cluster. Helm 3 has not been tested yet. + +## 1. Set up GCP + +First define the environmental variables that we will use throughout this installation. Please customize these to reflect your environment. + +```bash +export FEAST_GCP_PROJECT_ID=my-gcp-project +export FEAST_GCP_REGION=us-central1 +export FEAST_GCP_ZONE=us-central1-a +export FEAST_BIGQUERY_DATASET_ID=feast +export FEAST_GCS_BUCKET=${FEAST_GCP_PROJECT_ID}_feast_bucket +export FEAST_GKE_CLUSTER_NAME=feast +export FEAST_SERVICE_ACCOUNT_NAME=feast-sa +``` + +Create a Google Cloud Storage bucket for Feast to stage batch data exports: + +```bash +gsutil mb gs://${FEAST_GCS_BUCKET} +``` + +Create the service account that Feast will run as: + +```bash +gcloud iam service-accounts create ${FEAST_SERVICE_ACCOUNT_NAME} + +gcloud projects add-iam-policy-binding ${FEAST_GCP_PROJECT_ID} \ + --member serviceAccount:${FEAST_SERVICE_ACCOUNT_NAME}@${FEAST_GCP_PROJECT_ID}.iam.gserviceaccount.com \ + --role roles/editor + +gcloud iam service-accounts keys create key.json --iam-account \ +${FEAST_SERVICE_ACCOUNT_NAME}@${FEAST_GCP_PROJECT_ID}.iam.gserviceaccount.com +``` + +## 2. Set up a Kubernetes \(GKE\) cluster + +{% hint style="warning" %} +Provisioning a GKE cluster can expose your services publicly. This guide does not cover securing access to the cluster. +{% endhint %} + +Create a GKE cluster: + +```bash +gcloud container clusters create ${FEAST_GKE_CLUSTER_NAME} \ + --machine-type n1-standard-4 +``` + +Create a secret in the GKE cluster based on your local key `key.json`: + +```bash +kubectl create secret generic feast-gcp-service-account --from-file=key.json +``` + +For this guide we will use `NodePort` for exposing Feast services. In order to do so, we must find an External IP of at least one GKE node. This should be a public IP. + +```bash +export FEAST_IP=$(kubectl describe nodes | grep ExternalIP | awk '{print $2}' | head -n 1) +export FEAST_CORE_URL=${FEAST_IP}:32090 +export FEAST_ONLINE_SERVING_URL=${FEAST_IP}:32091 +export FEAST_BATCH_SERVING_URL=${FEAST_IP}:32092 +``` + +Add firewall rules to open up ports on your Google Cloud Platform project: + +```bash +gcloud compute firewall-rules create feast-core-port --allow tcp:32090 +gcloud compute firewall-rules create feast-online-port --allow tcp:32091 +gcloud compute firewall-rules create feast-batch-port --allow tcp:32092 +gcloud compute firewall-rules create feast-redis-port --allow tcp:32101 +gcloud compute firewall-rules create feast-kafka-ports --allow tcp:31090-31095 +``` + +## 3. Set up Helm + +Run the following command to provide Tiller with authorization to install Feast: + +```bash +kubectl apply -f - < + + + Option + Description + Default + + + + + FEAST_CONFIG + + Location of Feast configuration file + /.feast/config + + + + CONFIG_FEAST_ENV_VAR_PREFIX + + +

Default prefix to Feast environmental variable options.

+

Does not apply to FEAST_CONFIG +

+ + FEAST_ + + + + PROJECT + + Default Feast project to use + default + + + + CORE_URL + + URL used to connect to Feast Core + localhost:6565 + + + + CORE_ENABLE_SSL + + Enables TLS/SSL on connections to Feast Core + False + + + + CORE_AUTH_ENABLED + + Enable user authentication when connecting to a Feast Core instance + False + + + + CORE_AUTH_TOKEN + + Provide a static JWT token to authenticate with Feast Core + Null + + + + CORE_SERVER_SSL_CERT + + Path to certificate(s) used by Feast Client to authenticate TLS connection + to Feast Core (not to authenticate you as a client). + Null + + + + SERVING_URL + + URL used to connect to Feast Serving + localhost:6566 + + + + SERVING_ENABLE_SSL + + Enables TLS/SSL on connections to Feast Serving + False + + + + SERVING_SERVER_SSL_CERT + + Path to certificate(s) used by Feast Client to authenticate TLS connection + to Feast Serving (not to authenticate you as a client). + None + + + + GRPC_CONNECTION_TIMEOUT_DEFAULT + + Default gRPC connection timeout to both Feast Serving and Feast Core (in + seconds) + 3 + + + + GRPC_CONNECTION_TIMEOUT_APPLY + + Default gRPC connection timeout when sending an ApplyFeatureSet command + to Feast Core (in seconds) + 600 + + + + BATCH_FEATURE_REQUEST_WAIT_S + + Time to wait for batch feature requests before timing out. + 600 + + + +### Usage + +#### Configuration File + +Feast Configuration File \(`~/.feast/config`\) + +```text +[general] +project = default +core_url = localhost:6565 +``` + +#### Environmental Variables + +```bash +FEAST_CORE_URL=my_feast:6565 FEAST_PROJECT=default feast projects list +``` + +#### Feast SDK + +```python +client = Client( + core_url="localhost:6565", + project="default" +) +``` + diff --git a/docs/roadmap.md b/docs/roadmap.md new file mode 100644 index 00000000000..a047cf41256 --- /dev/null +++ b/docs/roadmap.md @@ -0,0 +1,31 @@ +# Roadmap + +## Feast 0.5 \(Technical Release\) + +[Discussion](https://github.com/gojek/feast/issues/527) + +### New functionality + +1. Streaming statistics and validation \(M1 from [Feature Validation RFC](https://docs.google.com/document/d/1TPmd7r4mniL9Y-V_glZaWNo5LMXLshEAUpYsohojZ-8/edit)\) +2. Support for Redis Clusters \([\#478](https://github.com/gojek/feast/issues/478), [\#502](https://github.com/gojek/feast/issues/502)\) +3. Add feature and feature set labels, i.e. key/value registry metadata \([\#463](https://github.com/gojek/feast/issues/463)\) +4. Job management API \([\#302](https://github.com/gojek/feast/issues/302)\) + +### Technical debt, refactoring, or housekeeping + +1. Clean up and document all configuration options \([\#525](https://github.com/gojek/feast/issues/525)\) +2. Externalize storage interfaces \([\#402](https://github.com/gojek/feast/issues/402)\) +3. Reduce memory usage in Redis \([\#515](https://github.com/gojek/feast/issues/515)\) +4. Support for handling out of order ingestion \([\#273](https://github.com/gojek/feast/issues/273)\) +5. Remove feature versions and enable automatic data migration \([\#386](https://github.com/gojek/feast/issues/386)\) \([\#462](https://github.com/gojek/feast/issues/462)\) +6. Tracking of batch ingestion by with dataset\_id/job\_id \([\#461](https://github.com/gojek/feast/issues/461)\) +7. Write Beam metrics after ingestion to store \(not prior\) \([\#489](https://github.com/gojek/feast/issues/489)\) + +## Feast 0.6 \(Feature Release\) + +### New functionality + +1. User authentication & authorization \([\#504](https://github.com/gojek/feast/issues/504)\) +2. Batch statistics and validation \(M2 from [Feature Validation RFC](https://docs.google.com/document/d/1TPmd7r4mniL9Y-V_glZaWNo5LMXLshEAUpYsohojZ-8/edit)\) +3. Online feature/entity status metadata \([\#658](https://github.com/gojek/feast/pull/658)\) + diff --git a/docs/user-guide/architecture.md b/docs/user-guide/architecture.md new file mode 100644 index 00000000000..5de6fe66eba --- /dev/null +++ b/docs/user-guide/architecture.md @@ -0,0 +1,41 @@ +# Architecture + +![Feast high-level flow](../.gitbook/assets/blank-diagram-4.svg) + +### **Feast Core** + +Feast Core is the central management service of a Feast deployment. It's role is to: + +* Allows users to create [entities](entities.md) and [features](features.md) through the creation and management of [feature sets](feature-sets.md). +* Starts and manages [ingestion jobs](data-ingestion.md). These jobs populate [stores](stores.md) from [sources](sources.md) based on the feature sets that are defined and the subscription\(s\) that a [store](stores.md) has. + +{% hint style="info" %} +Job management may move out of Feast Core to Feast Serving in the future. +{% endhint %} + +### **Feast Ingestion** + +Before a user ingests data into Feast, they should register one or more feature sets. These [feature sets](feature-sets.md) tell Feast where to find their data, how to ingest it, and also describe the characteristics of the data for validation purposes. Once a feature set is registered, Feast will start an Apache Beam job in order to populate a store with data from a source. + +In order for stores to be populated with data, users must publish the data to a [source](sources.md). Currently Feast only supports Apache Kafka as a source. Feast users \(or pipelines\) ingest batch data through the [Feast SDK](../getting-started/connecting-to-feast-1/connecting-to-feast.md) using its `ingest()` method. The SDK publishes the data straight to Kafka. + +Streaming systems can also ingest data into Feast. This is done by publishing to the correct Kafka topic in the expected format. Feast expects data to be in [FeatureRow.proto](https://api.docs.feast.dev/grpc/feast.types.pb.html#FeatureRow) format. The topic and brokers can be found on the feature set schema using the [Python SDK](../getting-started/connecting-to-feast-1/python-sdk.md). + +### **Stores** + +Stores are nothing more than databases used to store feature data. Feast loads data into stores through an ingestion process, after which the data can be served through the Feast Serving API. Stores are documented in the following section. + +{% page-ref page="stores.md" %} + +### **Feast Serving** + + `Feast Serving` is the data access layer through which end users and production systems retrieve feature data. Each `Serving` instance is backed by a [store](stores.md). + +Since Feast supports multiple store types \(online, historical\) it is common to have two instances of Feast Serving deployed, one for online serving and one for historical serving. However, Feast allows for any number of `Feast Serving` deployments, meaning it is possible to deploy a `Feast Serving` deployment per production system, with its own stores and population jobs. + +`Serving` deployments can subscribe to a subset of feature data. Meaning they do not have to consume all features known to a `Feast Core` deployment. + +Feature retrieval \(and feature references\) are documented in more detail in subsequent sections + +{% page-ref page="feature-retrieval.md" %} + diff --git a/docs/user-guide/data-ingestion.md b/docs/user-guide/data-ingestion.md new file mode 100644 index 00000000000..464a51bbfd4 --- /dev/null +++ b/docs/user-guide/data-ingestion.md @@ -0,0 +1,126 @@ +# Data ingestion + +Users don't necessarily have to provide their external data sources in order to load data into Feast. Feast also allows users to exclude the source when registering a feature set. The following is a valid feature set specification. + +```python +feature_set = FeatureSet( + name="stream_feature", + entities=[Entity("entity_id", ValueType.INT64)], + features=[Feature("feature_value1", ValueType.STRING)], +) +``` + +If a user does not provide a source of data then they are required to publish data to Feast. This process is called ingestion. + +### Ingesting data + +The following example demonstrates how data ingestion works. For a full tutorial please see the [Telco Customer Churn Prediction Notebook](https://github.com/gojek/feast/blob/master/examples/feast-xgboost-churn-prediction-tutorial/Telecom%20Customer%20Churn%20Prediction%20%28with%20Feast%20and%20XGBoost%29.ipynb). + +1. Connect to Feast Core and load in a Pandas DataFrame. + +```python +from feast import FeatureSet, Client, Entity +import pandas as pd + +# Connect to Feast core +client = Client(core_url="feast-core.example.com") + +# Load in customer data +df = pd.read_csv("customer.csv") +``` + +1. Create, infer, and register a feature set from the DataFrame. This is a once off step that is required to initially register a feature set with Feast + +```python +# Create an empty feature set +customer_churn_fs = FeatureSet("customer_churn") + +# Infer the schema of the feature set from the Pandas DataFrame +customer_churn_fs.infer_fields_from_df( + df, + entities=[Entity(name='customer_id', + dtype=ValueType.STRING)] + ) + +# Register the feature set with Feast +client.apply(customer_churn_fs) +``` + +1. We can also test that the feature set was correctly registered with Feast by retrieving it again and printing it out + +```text +customer_churn_fs = client.get_feature_set('customer_churn') +print(client.get_feature_set('customer_churn')) +``` + +```yaml +{ + "spec": { + "name": "customer_churn", + "entities": [ + { + "name": "customer_id", + "valueType": "STRING" + } + ], + "features": [ + { + "name": "churn", + "valueType": "INT64" + }, + { + "name": "contract_month_to_month", + "valueType": "INT64" + }, + { + "name": "streamingmovies", + "valueType": "INT64" + }, + { + "name": "paperlessbilling", + "valueType": "INT64" + }, + { + "name": "contract_two_year", + "valueType": "INT64" + }, + { + "name": "partner", + "valueType": "INT64" + } + ], + "maxAge": "0s", + "source": { + "type": "KAFKA", + "kafkaSourceConfig": { + "bootstrapServers": "10.202.250.99:31190", + "topic": "feast" + } + }, + "project": "default" + }, + "meta": { + "createdTimestamp": "2020-03-15T07:47:52Z", + "status": "STATUS_READY" + } +} +``` + +Once we are happy that the schema is correct, we can start to ingest the DataFrame into Feast. + +```text +client.ingest(customer_churn_fs, telcom) +``` + +```text +100%|██████████| 7032/7032 [00:02<00:00, 2771.19rows/s] +Ingestion complete! + +Ingestion statistics: +Success: 7032/7032A rows ingested +``` + +{% hint style="warning" %} +Feast ingestion maintains the order of data that is ingested. This means that data that is written later will replace those that are written prior in stores. This is important to note when ingesting data that will end in a production system. +{% endhint %} + diff --git a/docs/user-guide/entities.md b/docs/user-guide/entities.md new file mode 100644 index 00000000000..f2383e440cb --- /dev/null +++ b/docs/user-guide/entities.md @@ -0,0 +1,43 @@ +# Entities + +An entity is any domain object that can be modeled and that information can be stored about. Entities are usually recognizable concepts, either concrete or abstract, such as persons, places, things, or events which have relevance to the modeled system. + +More formally, an entity is an instance of an entity type. An entity type is the class of entities where entities are the instances. + +* Examples of entity types in the context of ride-hailing and food delivery: `customer`, `order`, `driver`, `restaurant`, `dish`, `area`. +* A specific driver, for example a driver with ID `D011234` would be an entity of the entity type `driver` + +An entity is the object on which features are observed. For example we could have a feature `total_trips_24h` on the driver `D01123` with a feature value of `11`. + +In the context of Feast, entities are important because they are used as keys when looking up feature values. Entities are also used when joining feature values between different feature sets in order to build one large data set to train a model, or to serve a model. + +Example entity properties + +{% code title="customer\_id.yaml" %} +```yaml +# Entity name +name: customer_id + +# Entity value type +value_type: INT64 + +``` +{% endcode %} + +Entities can be created through the [Feast SDK](../getting-started/connecting-to-feast-1/connecting-to-feast.md) as follows + +```python +from feast import Entity, ValueType, FeatureSet + +# Create a customer entity +customer = Entity("customer_id", ValueType.INT64) + +# Create a feature set with only a single entity +customer_feature_set = FeatureSet("customer_fs", entities=[customer]) + +# Register the feature set with Feast +client.apply(customer_feature_set) +``` + +Please see the [EntitySpec](https://api.docs.feast.dev/grpc/feast.core.pb.html#EntitySpec) for the entity specification API. + diff --git a/docs/user-guide/feature-retrieval.md b/docs/user-guide/feature-retrieval.md new file mode 100644 index 00000000000..e49db24fe99 --- /dev/null +++ b/docs/user-guide/feature-retrieval.md @@ -0,0 +1,116 @@ +# Feature retrieval + +## 1. Overview + +Feature retrieval \(or serving\) is the process of retrieving either historical features or online features from Feast, for the purposes of training or serving a model. + +Feast attempts to unify the process of retrieving features in both the historical and online case. It does this through the creation of feature references. One of the major advantages of using Feast is that you have a single semantic reference to a feature. These feature references can then be stored alongside your model and loaded into a serving layer where it can be used for online feature retrieval. + +## 2. Feature references + +In Feast, each feature can be uniquely addressed through a feature reference. A feature reference is composed of the following components + +* Feature Set +* Feature + +These components can be used to create a string based feature reference as follows + +`:` + +Feast will attempt to infer both the `feature-set` name if it is not provided, but a feature reference must provide a `feature` name. + +```python +# Feature references +features = [ + 'partner', + 'daily_transactions', + 'customer_feature_set:dependents', + 'customer_feature_set:has_phone_service', + ] + +target = 'churn' +``` + +Feature references only apply to a single `project`. Features cannot be retrieved across projects in a single request. + +## 3. Historical feature retrieval + +Historical feature retrieval can be done through either the Feast SDK or directly through the Feast Serving gRPC API. Below is an example of historical retrieval from the [Churn Prediction Notebook](https://github.com/gojek/feast/blob/master/examples/feast-xgboost-churn-prediction-tutorial/Telecom%20Customer%20Churn%20Prediction%20%28with%20Feast%20and%20XGBoost%29.ipynb). + +```python +# Add the target variable to our feature list +features = self._features + [self._target] + +# Retrieve training dataset from Feast. The "entity_df" is a dataframe that contains +# timestamps and entity keys. In this case, it is a dataframe with two columns. +# One timestamp column, and one customer id column +dataset = client.get_batch_features( + feature_refs=features, + entity_rows=entity_df + ) + +# Materialize the dataset object to a Pandas DataFrame. +# Alternatively it is possible to use a file reference if the data is too large +df = dataset.to_dataframe() +``` + +In the above example, Feast does a point in time correct query from a single feature set. For each timestamp and entity key combination that is provided by `entity_df`, Feast determines the values of all the features in the `features` list at that respective point in time and then joins features values to that specific entity value and timestamp, and repeats this process for all timestamps. + +This is called a point in time correct join. + +Feast allows users to retrieve features from any feature sets and join them together in a single response dataset. The only requirement is that the user provides the correct entities in order to look up the features. + +### **Point-in-time-correct Join** + +Below is another example of how a point-in-time-correct join works. We have two dataframes. The first is the `entity dataframe` that contains timestamps, entities, and labels. The user would like to have driver features joined onto this `entity dataframe` from the `driver dataframe` to produce an `output dataframe` that contains both labels and features. They would then like to train their model on this output + +![Input 1: Entity DataFrame](https://lh3.googleusercontent.com/ecS5sqj3FHLFSm06XF11NmTQSru-bQ4Az3Kuko_vg5YlBxXjHadlsGwmo7d7wUx4fA1ssdZvxrESDKfkGWjj3HNJg_jIqXY0avz2JzCcEOXLBLmtXNEY8k2u3f4QusHdDWdqRARQHYE) + +![Input 2: Driver DataFrame](https://lh3.googleusercontent.com/LRtCOzmcfhLWzpyndbRKZSVPanLLzfULoHx2YxY6N3i1gQd2Eh6MS1igahOe8ydA7zQulIFJEaQ0IXFXOsdkKRobOC6ThSOnT4hACbCl1jeM4O2JDVC_kvw8lwTCezVUD3d6ZUYj31Q) + +Typically the `input 1` DataFrame would be provided by the user, and the `input 2` DataFrame would already be ingested into Feast. To join these two, the user would call Feast as follows: + +```python +# Feature references +features = [ + 'conv_rate', + 'acc_rate', + 'avg_daily_trips', + 'trip_completed' + ] + + +dataset = client.get_batch_features( + feature_refs=features, # this is a list of feature references + entity_rows=entity_df # This is the entity dataframe above + ) + +# This prints out the dataframe below +print(dataset.to_dataframe()) +``` + +![Output: Joined DataFrame](https://lh5.googleusercontent.com/Gm-4Ru68KyIQ2tQtaVTDFngqO7pMtlMP1YAQO-bqln6_Mo2XAPdbij6w5ACnHAmQ053XUPu6G-c2aYRVJxPqPTMN_BcH6PY0-E1kCwXQAdW1CcQo5tc0g5ilcuVAtqsHcJB1R5mBdLo) + +Feast is able to intelligently join feature data with different timestamps to a single basis table in a point-in-time-correct way. This allows users to join daily batch data with high-frequency event data transparently. They simply need to know the feature names. + +Point-in-time-correct joins also prevents the occurrence of feature leakage by trying to accurate the state of the world at a single point in time, instead of just joining features based on the nearest timestamps. + +## Online feature retrieval + +Online feature retrieval works in much the same way as batch retrieval, with one important distinction: Online stores only maintain the current state of features. No historical data is served. + +```python +features = [ + 'conv_rate', + 'acc_rate', + 'avg_daily_trips', + ] + +data = client.get_online_features( + feature_refs=features, # Contains only feature references + entity_rows=entity_rows, # Contains only entities (driver ids) + ) +``` + +Online serving with Feast is built to be very low latency. Feast Serving provides a [gRPC API](https://api.docs.feast.dev/grpc/feast.serving.pb.html) that is backed by [Redis](https://redis.io/). We also provide support for [Python](https://api.docs.feast.dev/python/), [Go](https://godoc.org/github.com/gojek/feast/sdk/go), and Java clients. + diff --git a/docs/user-guide/feature-sets.md b/docs/user-guide/feature-sets.md new file mode 100644 index 00000000000..48ad81a9e58 --- /dev/null +++ b/docs/user-guide/feature-sets.md @@ -0,0 +1,54 @@ +# Feature Sets + +Feature sets are both a schema and a means of identifying data sources for features. + +Data typically comes in the form of flat files, dataframes, tables in a database, or events on a stream. Thus the data occurs with multiple columns/fields in multiple rows/events. + +Feature sets are a way for defining the unique properties of these data sources, how Feast should interpret them, and how Feast should source them. Feature sets allow for groups of fields in these data sources to be [ingested](data-ingestion.md) and [stored](stores.md) together. Feature sets allow for efficient storage and logical namespacing of data within [stores](stores.md). + +{% hint style="info" %} +Feature sets are a grouping of feature sets based on how they are loaded into Feast. They ensure that data is efficiently stored during ingestion. Feature sets are not a grouping of features for retrieval of features. During retrieval it is possible to retrieve feature values from any number of feature sets. +{% endhint %} + +### Customer Transactions Example + +Below is an example of a basic `customer transactions` feature set that has been exported to YAML: + +{% tabs %} +{% tab title="customer\_transactions\_feature\_set.yaml" %} +```yaml +name: customer_transactions +entities: +- name: customer_id + valueType: INT64 +features: +- name: daily_transactions + valueType: FLOAT +- name: total_transactions + valueType: FLOAT +``` +{% endtab %} +{% endtabs %} + +The dataframe below \(`customer_data.csv`\) contains the features and entities of the above feature set + +| datetime | customer\_id | daily\_transactions | total\_tra**nsactions** | +| :--- | :--- | :--- | :--- | +| 2019-01-01 01:00:00 | 20001 | 5.0 | 14.0 | +| 2019-01-01 01:00:00 | 20002 | 2.6 | 43.0 | +| 2019-01-01 01:00:00 | 20003 | 4.1 | 154.0 | +| 2019-01-01 01:00:00 | 20004 | 3.4 | 74.0 | + +In order to ingest feature data into Feast for this specific feature set: + +```python +# Load dataframe +customer_df = pd.read_csv("customer_data.csv") + +# Create feature set from YAML (using YAML is optional) +cust_trans_fs = FeatureSet.from_yaml("customer_transactions_feature_set.yaml") + +# Load feature data into Feast for this specific feature set +client.ingest(cust_trans_fs, customer_data) +``` + diff --git a/docs/user-guide/features.md b/docs/user-guide/features.md new file mode 100644 index 00000000000..224ca798a54 --- /dev/null +++ b/docs/user-guide/features.md @@ -0,0 +1,38 @@ +# Features + +A feature is an individual measurable property or characteristic of a phenomenon being observed. Features are the most important concepts within a feature store. Feature data is used both as input to models during training and when models are served in production. + +In the context of Feast, features are values that are associated with either one or more entities over time. In Feast, these values are either primitives or lists of primitives. Each feature can also have additional information attached to it. + +The following is a YAML representation of a feature specification. This specification would form part of a larger specification within a [feature set](feature-sets.md). + +{% code title="total\_trips\_feature.yaml" %} +```yaml +# Entity name +name: total_trips_24h + +# Entity value type +value_type: INT64 +``` +{% endcode %} + + Features can be created through the[ Feast SDK](../getting-started/connecting-to-feast-1/connecting-to-feast.md) as follows + +```python +from feast import Entity, Feature, ValueType, FeatureSet + +# Create a driver entity +driver = Entity("driver_id", ValueType.INT64) + +# Create a total trips 24h feature +total_trips_24h = Feature("total_trips_24h", ValueType.INT64) + +# Create a feature set with a single entity and a single feature +driver_fs = FeatureSet("customer_fs", entities=[driver], features=[total_trips_24h]) + +# Register the feature set with Feast +client.apply(driver_fs) +``` + +Please see the [FeatureSpec](https://api.docs.feast.dev/grpc/feast.core.pb.html#FeatureSpec) for the complete feature specification API. + diff --git a/docs/user-guide/overview.md b/docs/user-guide/overview.md new file mode 100644 index 00000000000..0d45825368b --- /dev/null +++ b/docs/user-guide/overview.md @@ -0,0 +1,58 @@ +# Concepts + +## Using Feast + +Feast acts as the interface between ML models and data. Feast enables your team to + +1. Create feature specifications to manage features and load in data that should be managed +2. Retrieve historical features for training models +3. Retrieve online features for serving models + +{% hint style="info" %} +Feast currently does not apply feature transformations to data. +{% endhint %} + +### 1. Creating and managing features + +Feature creators model the data within their organization into Feast through the creation of [feature sets](feature-sets.md). + +Feature sets are specifications that contain both schema and data source information. They allow Feast to know how to interpret your data, and optionally where to find it. Feature sets allow users to define domain [entities](entities.md) along with the [features](features.md) that are available on these entities. Feature sets also allow users to define schemas that describe the properties of the data, which in turn can be used for validation purposes. + +Once a feature set has been registered, Feast will create the relevant schemas to store feature data within it's feature [stores](stores.md). These stores are then automatically populated by by jobs that ingest data from data [sources](sources.md), making it possible for Feast to provide access to features for training and serving. It is also possible for users to [ingest](data-ingestion.md) data into Feast instead of using an external source. + +Read more about [feature sets](feature-sets.md). + +### 2. Retrieving historical features during training + +Both online and historical retrieval are executed through an API call to `Feast Serving` using [feature references](feature-retrieval.md). In the case of historical serving it is necessary to provide Feast with the entities and timestamps that feature data will be joined to. Feast eagerly produces a point-in-time correct dataset based on the features that have been requested. These features can come from any number of feature sets. + +Stores supported: [BigQuery](https://cloud.google.com/bigquery) + +### 3. Retrieving online features during serving + +Feast also allows users to call `Feast Serving` for online feature data. Feast only stores the latest values during online serving for each feature, as opposed to historical serving where all historical values are stored. Online serving allows for very low latency requests to feature data at very high throughput. + +Stores supported: [Redis](https://redis.io/), [Redis Cluster](https://redis.io/topics/cluster-tutorial) + +## Concept Hierarchy + +![](../.gitbook/assets/image%20%283%29.png) + +Feast resources are arranged in the above hierarchy, with projects grouping one or more [feature sets](feature-sets.md), which in turn groups multiple [features](features.md) or [entities](entities.md). + +The logical grouping of these resources are important for namespacing as well as retrieval. During retrieval time it is necessary to reference individual features through feature references. These references uniquely identify a feature or entity within a Feast deployment. + +## Concepts + +[Entities](entities.md) are objects in an organization that model a specific construct. Examples of these include customers, transactions, and drivers. + +[Features](features.md) are measurable properties that are observed on entities. Features are used as inputs to models. + +[Feature Sets](feature-sets.md) are schemas that define logical groupings of entities, features, data sources, and other related metadata. + +[Stores](stores.md) are databases that maintain feature data that gets served to models during training or inference. + +[Sources](sources.md) are either internal or external data sources where feature data can be found. + +[Ingestion](data-ingestion.md) is the process of loading data into Feast. + diff --git a/docs/user-guide/sources.md b/docs/user-guide/sources.md new file mode 100644 index 00000000000..35fcb41a11b --- /dev/null +++ b/docs/user-guide/sources.md @@ -0,0 +1,32 @@ +# Sources + +A `source` is a data source that can be used to find feature data. Users define sources as part of [feature sets](feature-sets.md). Once a feature set is registered with a source, Feast will automatically start to populate its stores with data from this source. + +{% hint style="info" %} +Feast only supports [Kafka](https://kafka.apache.org/) as a source currently. +{% endhint %} + +An example of a user provided source can be seen in the following code snippet + +```python +feature_set = FeatureSet( + name="stream_feature", + entities=[ + Entity("entity_id", ValueType.INT64) + ], + features=[ + Feature("feature_value1", ValueType.STRING) + ], + source=KafkaSource( + brokers="mybroker:9092", + topic="my_feature_topic" + ) +) +``` + +Once this feature set is registered, Feast will start an ingestion job that retrieves data from this source and starts to populate all [stores](stores.md) that subscribe to it. + +In most cases a feature set \(and by extension its source\) will be used to populate both an online store and a historical store. This allows users to both train and serve their model with the same feature data. + +Feast will ensure that the source complies with the schema of the feature set. The event data has to be [Protobuf](https://developers.google.com/protocol-buffers) encoded and must contain the necessary [FeatureRow](https://api.docs.feast.dev/grpc/feast.types.pb.html#FeatureRow) structure. + diff --git a/docs/user-guide/stores.md b/docs/user-guide/stores.md new file mode 100644 index 00000000000..c15cbb3db02 --- /dev/null +++ b/docs/user-guide/stores.md @@ -0,0 +1,37 @@ +# Stores + +In Feast, a store describes a database that is populated with feature data in order to be served to models. + +Feast supports two classes of stores + +* Historical stores +* Online stores + +In order to populate these stores, Feast Core creates a long running ingestion job that streams in data from all feature sources to all stores that subscribe to those feature sets. + +![](../.gitbook/assets/image%20%282%29.png) + +## Historical Stores + +Historical stores maintain a complete history of feature data for the feature sets they are subscribed to. + +Feast currently only supports [Google BigQuery](https://cloud.google.com/bigquery) as a feature store, but we have [developed a storage API ](https://github.com/gojek/feast/issues/482)that makes adding a new store possible. + +Each historical store models its data differently, but in the case of a relational store \(like BigQuery\), each feature set maps directly to a table. Each feature and entity within a feature set maps directly to a column within a table. + +Data from historical stores can be used to train a model. In order to retrieve data from a historical store it is necessary to connect to a Feast Serving deployment and request historical features. Please see feature retrieval for more details. + +{% hint style="danger" %} +Data is persisted in historical stores like BigQuery in log format. Repeated ingestions will duplicate the data is persisted in the store. Feast will automatically deduplicate data during retrieval, but it doesn't currently remove data from the stores themselves. +{% endhint %} + +## Online Stores + +Online stores maintain only the latest values for a specific feature. Feast currently supports Redis as an online store. Online stores are meant for very high throughput writes from ingestion jobs and very low latency access to features during online serving. + +Please continue to the [feature retrieval](feature-retrieval.md) section for more details on retrieving data from online storage. + +## Subscriptions + +Stores are populated by ingestion jobs \(Apache Beam\) that retrieve feature data from sources based on subscriptions. These subscriptions are typically defined by the administrators of the Feast deployment. In most cases a store would simply subscribe to all features, but in some cases it may subscribe to a subset in order to improve performance or efficiency. + diff --git a/docs/why-feast.md b/docs/why-feast.md new file mode 100644 index 00000000000..75048ace2cc --- /dev/null +++ b/docs/why-feast.md @@ -0,0 +1,32 @@ +# Why Feast? + +## Lack of feature reuse + +**Problem:** The process of engineering features is one of the most time consuming activities in building an end-to-end ML system. Despite this, many teams continue to redevelop the same features from scratch for every new project. Often these features never leaving the notebooks or pipelines they are built in. + +**Solution:** A centralized feature store allows organizations to build up a foundation of features that can be reused across projects. Teams are then able to utilize features developed by other teams, and as more features are added to the store it becomes easier and cheaper to build models. + +## Serving features is hard + +**Problem:** Serving up to date features at scale is hard. Raw data can come from a variety of sources, from data lakes, to even streams, to data warehouse, to simply flat files. Data scientists need the ability to produce massive datasets of features from this data in order to train their models offline. These models then need access to real-time feature data at low latency and high throughput when they are served in production. + +**Solution:** Feast is built to be able to ingest data from a variety of sources, supporting both streaming and batch sources. Once data is loaded into Feast as features, they become available through both a batch serving API as well as an real-time \(online serving\) API. These APIs allows data scientists and ML engineers to easily retrieve feature data for their development, training, or in production. Feast also comes with a Java, Go, and Python SDK to make this experience easy. + +## **Models need point-in-time correctness** + +**Problem:** Most data sources are not built with ML use cases in mind and by extension don't provide point-in-time correct lookups of feature data. One of the reasons why features are often re-engineered is because ML practitioners need to ensure that their models are trained on a dataset that accurately models the state of the world when the model runs in production. + +**Solution:** Feast allows end users to create point-in-time correct datasets across multiple entities. Feast ensures that there is no data leakage, that cross feature set joins are valid, and that models are not fed expired data. + +## Definitions of features vary + +**Problem:** Teams define features differently and there is no easy access to the documentation of a feature. + +**Solution:** Feast becomes the single source of truth for all feature data for all models within an organization. Teams are able to capture documentation, metadata and metrics about features. This allows teams to communicate clearly about features, test features data, and determine if a feature is useful for a particular model. + +## **Inconsistency between training and serving** + +**Problem:** Training requires access to historical data, whereas models that serve predictions need the latest values. Inconsistencies arise when data is siloed into many independent systems requiring separate tooling. Often teams are using Python for creating batch features off line, but these features are redeveloped with different libraries and languages when moving to serving or streaming systems. + +**Solution:** Feast provides consistency by managing and unifying the ingestion of data from batch and streaming sources into both the feature warehouse and feature serving stores. Feast becomes the bridge between your model and your data, both for training and serving. This ensures that there is a consistency in the feature data that your model receives. + From 6874eb11adebda91ef3e8d03fd3275744bf5d665 Mon Sep 17 00:00:00 2001 From: Willem Pienaar Date: Sun, 24 May 2020 10:55:05 +0000 Subject: [PATCH 016/133] GitBook: [master] 29 pages and 4 assets modified --- docs/.gitbook/assets/blank-diagram-4 (1).svg | 1 + .../feast-docs-overview-diagram-2 (3) (1).svg | 1 + docs/.gitbook/assets/image (2) (1).png | Bin 0 -> 149255 bytes docs/.gitbook/assets/image (3) (1).png | Bin 0 -> 17434 bytes docs/SUMMARY.md | 1 + .../connecting-to-feast-1/README.md | 6 ++++++ 6 files changed, 9 insertions(+) create mode 100644 docs/.gitbook/assets/blank-diagram-4 (1).svg create mode 100644 docs/.gitbook/assets/feast-docs-overview-diagram-2 (3) (1).svg create mode 100644 docs/.gitbook/assets/image (2) (1).png create mode 100644 docs/.gitbook/assets/image (3) (1).png diff --git a/docs/.gitbook/assets/blank-diagram-4 (1).svg b/docs/.gitbook/assets/blank-diagram-4 (1).svg new file mode 100644 index 00000000000..fb5e0659e55 --- /dev/null +++ b/docs/.gitbook/assets/blank-diagram-4 (1).svg @@ -0,0 +1 @@ + \ No newline at end of file diff --git a/docs/.gitbook/assets/feast-docs-overview-diagram-2 (3) (1).svg b/docs/.gitbook/assets/feast-docs-overview-diagram-2 (3) (1).svg new file mode 100644 index 00000000000..7f30963ec78 --- /dev/null +++ b/docs/.gitbook/assets/feast-docs-overview-diagram-2 (3) (1).svg @@ -0,0 +1 @@ + \ No newline at end of file diff --git a/docs/.gitbook/assets/image (2) (1).png b/docs/.gitbook/assets/image (2) (1).png new file mode 100644 index 0000000000000000000000000000000000000000..d3b359a5988f3b0d75afe44b0ae2cd7d9ef0aab1 GIT binary patch literal 149255 zcmeFZbySsa*DeZ+1uUc%T}nzQDJdP&Ac81ef`qhyfOI#4K}m`zs30IET_Q?KgGh>$ zD4h~}E_r|7-urvsG0s2dpL52?U^s>kYdvw_Gp>2fYd+E1S}Me6=+9tbVG*mTDqhFJ z!j;6rLYAWtaD=&9<30R~rKTuNacGTGJclbcz)Qd8~&n>7nvL}Y8ENI9&3`;`koMja|BC^BOq{{0t0DanD23c~vD zzn>mrQP!R<3rYPCC&Y?5=l<`P!9vcdV#Sy~FqGH)_scM2wLK93PuF)lkJZN4UdYUW z#=`#Zzd@GmXaD^u7PdHfkY$RM9-896-!wdiT&Wewzdub-&{+bx(#}iVQ3U_CJXpw_ z(M$OM;W6bNz+Zl(Qhh-Fhjr6lNyYgOkIAfp&|Y!tI%A3Wx2M6v=JM?N4@c37Z15Lx z8{A95|F#Q3Sd^0U|KTX=|I6F+>4BV)krCoJI7o>g@8!#2GV+x5{=@$u|?_y}tDl6yag^)&~` zP2QT-*C^pf_7r8xz0oUD7YH=(^H}PC@vqZRaB9_AKAjcBoQ5j~PLtbjeXd!d)cC*| z|A>OiQxic&5oD=<*X@$90*bQs-Zx&$C(IT~U;#y3W9oDlbyu33n+d_ex>R!lPmZ?& zMu=i21_lP=*ska^uLUAAGBQvs^ExYR%5hj8XaUY(=ZTs=*=g++_3EiB8DJ@RV60;@{`ddgu$Xd79RO;2zkz#YEPx4J&$Zr!9-7iNI&Pm$L_dS#3<}7iY zQQ-3A&c>mYK@}vyeH#hAnZ7Gu%6<9%8PanxrZWVJd&S^g$7kf;7v2WrQ}1E4#K%T| zQ2}3MI%iUb0^^FcP{~JRm0FP(WWpJ~3ONpa54EgX-m+R_&asE@yw)lf#K&)Y+g109 z#a_xv{|fabya{;pdu7d3ZJ1^8J97yOyz?C9Sde_h4{$T%2lW-ZL~YK$gyepGkF?bH zjVSW*(_Ycb3?O1e-na@^A6mLOKbPkT{~1LKQrPn}0pdg%4QZ%N}SrhK)Cr%!bwGEWR^5DMVRY(Ti1) zz!YQpi>`8cUB1-#-oYhrw6+MW&&_=Ow9rsXeVfTavBMijvf!e# zx>49*H;b^&Z*hoT^TOIrxUCe`+j4n2WMkVi$^JhsDwn6oZ8EDbm}L=TFwv2Mi?-#X zaUi;Vk%wdWv^R_&z=eK{8vFKYuMErJ49Wy>c~;ntzHY~Fa~@i1pO%*!g4Ms7X=Y7A z5@VWnU-!zO_KN){){i;q@I`0Q_qpD%lQGv=@F2|Qgu*crfr84jaG}o9s-3K7qecFB z1^$Qdo!*k038XO-uLlyELxWz!j=Yhm?>KvwvQ}{OWkkWV(V;}8IC(Mn;)J8C(mGs} zDqDdF;=LR`96M*B?IaEtO15(OC4HsT_z#ixpFQ|a)~z>;Oq8`Z^O-{Nv2S8*=AKWI z_+>6n>-nq1I#){Hu*A}aI>Q%h#-?Lt;iB9!;j|Fhf=J+)L{rXJH@FaQ`{Y;O2+OJ+ zx}2SL_>Od|p0o^?=Nm0qJW|AK%#JK?U37QVUg_vcr_YVB{D0F>pS&0?h6v)fzk!W| zSME6WPOr#F#D36jut29bOFI8rcEHwR-{GO3HAl1h^9V~Cb{TGNhtI=Zo9^G0f3qrtu2Mvkjb zx0VKMcUH#owKDtdD_DK~o(RDTi`(LtLMC_;hDGVCw)m__4aKEYda}QeZCc||T3X6d zb8@rL;O_cto4B|*2MccgM_8Ahs0P1%H#pUg&Xf`9+UU$o(`*_4M~@#Ha#P;c!ivdc zUxaHNA0PX?6l2Nr*_7zB!U;LCiqPnenaxpy`fddJ7c z`iv#Kymp7)=*tEkXIwLJcK+=(nxUysL2{qLoKrGbq-fKE4yk;roF^7DO~TJmy)~_U zX?w`6|MvE--=#~Je0c7vk${oo%10$6*gp3B7)R%`;r4U5=&6Xq_3vN}i=`jf(;#lq zAfMg+{?Vg* zy*({Y-0OEnfR#zbW!U{EPbfvL;Qn9BvKbXs-rt-bJg(oJ9B_$yLk&jkyiUoC$pRD* zsKv+(-t)>}r*ZGJCoq*UG+4DisCUayQ&XD?`17l{@%Ui2jZp1-S63Hoj@RnsL}zF0 zG4=HF^4F4EtqC#*1#9D<23xbFeHBhNU5*a6AIad#U-n#~tuUs+ z%3Hu=@ym#rZ+CO^%x16b0`Hyu*|ylnoYx?`ZaWm$>w%SNWlHdxykDDXjSS~CysD+u z^D&;mcW1@$)pd4!e0)jC+88SC$~yxbf`WqV?BQvjOTdayEr~2w7KuoSi2SS{!16{A zGdwsa>ACyc)bt~GBfXG~W+)!<;xWmC2M^55%={0wq8~g+K2&06V++;l`|{-r=e4Xw zH%Xr@JNuo*KDJ4lZ}0DxXZ_xzJ$LTc&z~BS-CbRCytwk8Iyw%P-xT}6dEe;q^79wN zttHT2n$6D+AWw^y;o#uVN)>npal^w@ZtR`4!^m4{%MVd(Di5M~e(aFN9QDec+zxEw zH>=z0KxRtacLRT-;WPPC5p3t<11lVhgFu-1{;IhCLY+*ZBZ(VqzHDvH)zrm8NkwJg zLXA~Bi7EEVWPSQqh)H5~-QXN5KX+HB_)#tK3}Oe{KNo)g{yjIh)kb5!mR#<4e7H|Q z!6EE2`E6jJ45A;z8^db19|QT<2#w!8jf;z;;@0!{{cWtXQ)4qb5S9g2EG{Odf7AbV zH?ILCZ)QIaVg$3rb+9##Ho}EA7H0Rhtdx|6>1nyu{rT>6{R}cUrw5GE%fEhg|7KRg zg}o^$DQO{`w(Uxl=M4=F6}LUnHL3HyRIxOWPa1d&qG@@++^2_k?%Y`^vQhX15!SmC zA~6Z0RO(cAboj#T?7Lg7c$29~fyes`^dj~G8amBwZA!hxL|xG@&R{Iw?_eok*dZ$` zD}2H1;q^^evj+b^G4og_rDpX$;ep9M2U{1)Ahrgc_vPW@vJeFvw%{~cd8;>DHe1M+ zM2Icy_U+qi$&y~b3k^zHU79+AgM)LK5O%GA_gJn(EQ+(_LKHj*?489C2&9WSIzIZt z)0r;nbn(_1&y`X6hl1ID&6AS>5ShbkX)jqKMgA;4PdH=#LWC*s@KObxq$iJykpF>) zk*xcCSGt%>!_HTS!gj+loAqSar`54C&e{$dDQRhNp>Bq<4Y+p?{|hJb{0jCP%cypMD30z7h3)B@bG%lglr)jgyNI(8CUW+ zaP9L%k@jD==JJ$BS2|C02wH!tAvOK%v%kUK=%o)K(6%R|8eIB?MhXe7Kn}q6@{}`F z7Y~nir|KSOWSpfA;%m#w%R_D}MZ#;sk#m{4t4J|7E?muWw{TgchXZy|o!00S1Qj zmd2WZlcQb7V@YxG?cWo=;CDi{Ju6dNQ&gn*2rd;Dg697cI3;Bg1 zg|m(>81l;>2xp@c2ALA~9zA*#b7=+qq2c)8=ioK4x-;A-)}6_vW;?&e+r3I}eU)kq<=GlRT;2d{!^p^ZZ+TGMb0t;p4Ha?*K*OM*m{=85QeAO@*|M^- z+0s?zrKRE?i)i<(q@tGHf3J<*NCx@HvrZJiN+qu;r zdi2hUyX8R+R#g1(y*&YT|LI8B8N>u{3FeOW&%60;B!e;q-79X4=|K z=F)z90*^xge;u;#KY0dub#JAtcku&Tn6SgJ=2qy#;LkPg3nY@cwTMSnN0%U%+ z4G-@e?r+u^?;L(ZxAmMPUPQHYr3u|x155REb+sEV92+aMedWF*-TmoPfo?vD2!puW z`No~RoSfgrDUu1fn14GrDgAf}w0 zWCdHtR9ygDY@ljz#{0CGYloaG9xZ$K?y2w{g;i4&i>Rn5gxLMrSboLcgDq0Yix*>j zT;1J$*QSH>*72GAWKHh3%gs1K?pD-)io~(rx1INZ6^WqRjap3sSfhe|z-$JhpzZ zStj~=W5Cg)%a`sLC#ygGa`EPCzm3nF>+9<^$s5!p^RVK4e0=5muvCN$sF1av#f|Sq zIJCk_}v#1`N6>mYj z2V_?i2yX6hbMMuKihXeCrRS=?zJDNSUW(kfAXiEll9-&VS7CoMc^1+b#vQ;&S>s>J zIykPYR@e`nXBynhKMw|-vb!@A$<#l&&9wtgex2c=WwwX|HD)cqJg;86wjezqI6iB( zaig!UPHN}r=%7TO0k$e`D@*?i;r~R2yF;4>KO%8C}3! zVz@W=c*wB#xgkRR{#M<(gMR5HuoWmREe)anEFgVD*^4;XBLH%C#(ion<^lWcMVw(_k&&@Jc}4|*w+59*H2NexSG<2!jL?fZ ze&T=RGG58+Byy`Iti}0GH<&o8j#E76I3R+}SNN`>YLY52;^%B*z5saJme0^gPZ zBm`gzQwZcnb`=&eqsA+po?kUFF*(0YB=Y;qvnw;=kF&CR4u)QUE4X@i8sdo%P$r)FG{ zVOd0RiW*%^CdVw;-pU2xbjPqnc$9mqlK_X7B1z8mKejH?>!kMOH?Gvwcia6nu97J! zA+a$Pa;r6>&qq4Oe|hlLU9{o%^%5ff)sXT`p6EhWRkUJgG9t= zuWYf8L&RGDSu++-$vZkaDtxE^{Kbo}OB_FSb=`JT?Y-1!RFph+7!q1;S--?`!HWQ*A^I9H^KEN!u zQVP#8Sxn}&X2{un#pbdcabtxRxKMb&a~>W4S@q*KVia>eD?)K(asJ(HZXTmb$IpE0 zzM)GxD=UUIcI~V1;C*kEgtGT-g-TTUeM-dA0H1y~q2hy>pS^S3Be_c1V82e!<(V7; zh*>F?dTESfpr_|8JRO6W^W4&mxr#3NTAnnJEa-Fb_x#%KHD)boNay_TuE`??;VXPKUj z@@65BKowZ9V&EEru&RwShqA3l*?WKE2hdXqbV4ryhd5^2d+uIAy}JIQ15$cwJu?b| zyA@uQ%hWu7bjOO0pQ0kNRyg3$`>RQU<sd<~NKQPSr@MnPj9Kh6{gB)Rtw^7`!E5 zf&2~$ol?5cdF1W6ioVSj+{R8*fFp(@V$Ro!T8-l~+UC@Wu2hZ;l31+K%Z1a%|3x!Wyb zQxlWDh=b0%ckh-<#w)WO@Ws-i6S5NEuAl29@tGR!uJE<*)VtriH}t*xdP>gd_T%2p z&RXyFS=bC|?=_1`G8~pXh43=~H>{n;jx(K|oeQtOYkJxJt=c_J-ok=8Q^SW+@{gpH z$D;PIHX2#^qM*2#-@La2uvu;oRLxx%%?&Ns-j$a23DtKdakuxFkG{YAoqOWx7_e_$ zO?~G88I1vVm`*GrA^0RHmr7Fh6Oa>NrtgkQ_a?qK9PJqU=>C|VZW6m-60ker^@{xj zkAM83R)VFitw;1u?Sts(XC#2@%-&1FIwQQjyu8$KKC4$cr%8y-1wt8kaj7FDNqa>p zB=BJCPUTc>d^~ZZugMGbq{XuafM*!90eji#hM#-!vC`#B@#xscm0XI6F}5vf-yK!Y zZj$8jy8C5$!ME9c4LT6R<`9yfY*~$|b9uJz%lhub@&0BGTMKj?Ysdnc>C0;3QAwK% zeSfKyip2dbE>8{_rF>sLgOVWN9V&}%k2qrcJy(uDPkf%6hSw-BP===A^~oFFvDjDK zbmib8sWtq2!1`igyJKi(PNywvH1-ger-V_Z#@WUd*WKR}TX>5}y`>O%)Ohhp8N1d> ztrHKgp08CWbgcK0lfw#(uHTw`f7hxZE6LE&y5q^Mq+*_uDP|N4q+>#n=_&9@^QGY~ zJXj9gfQ6oOmEBY)2hNb77wb$*EyEeJfd{ZvXXfRNH`?Il*{Y-J9WITf?iXU`uf7_} zjN++;V73%})#LN{czVRwy8G@pHeQRsXDBhwt&zpl7Z_U9ez+q{@0m9SzHLMV==g?jsz-!x`IphD%B5g`-wOqOTukkW2=(>E&ykYcLB;JHnfpp0@b3MM6wG zRE86K+@R@|)EsOncJ8(}9yYq<9|HUURiIg#pPygWzOo`R0kQlc0oCWWIU@9UP-Y-^ z)`a3R#A}j5-F&SshDJAo6w3v#ETKbT{z`5^7R8Mw78$&;e*Ac>>PpIA3dUt2W=F;Eg{3X`RcaoC98S?b_8rmY@;d{HDU@57 zhy4uPQsb!OZG8NaDQWZlGLG>v=gGSJg+KXBYqYb$&pEUG6rA>|-XP^Ia&M$q+Di6f z@aFbjM@!AUpLSkaT3W@z$wkc4UNx0G=DlTr=uY3$HAglspzK)0q#+eG4;H*%pv`gGbB=U}*%gUX?A{`<%H$ByhMgzh$z-x{_{nfJQo zSH1FY(PI!yBI%!8ys3Wj*1Is883nM!lFeshjv!D?9w0a_JK?hsEEHulaH$Vnn@7(g z=L$zLIUT5wDoD#==!aTg!a4Z)69_Dh(-_D5e1Ord`8q#0S5j6%a`tjlmkm%8t8W2( zHCMrl(C8e`O% z_N*sQJ}mDe#$N`bAEl;#GPl%Nyw`2EL!-QPK(IO&$rNA!Y%k>3o-ApNpRKK}ODPH? zofrJv$C8UoO3mN9J_GU;xEiS7?n5oXa%Z-e{&wJ)S)F(D11Es98GN!AE?n^FK`ZQ4 zFtkRJ=5Mbyo;2Jx1~3=kO+L2hb3bPH$_e3%1=;J6HKOC_BnpufJ88DGScIIh^>%&H6fL$ zvfto&U!C`Q$S}mP^vTU73HcrzPpM=iz@J7p8fNIU9-EvBQI*kj!D820c>m@r&*3vY?}D^NF1?rO-`Dn zI66!=SECbFQ`&7VYNk`|jIY3*_$2O~@^ayLZiv+lI3U&Q5_U%0VH)c21T;6pTp<2zznfuJEs zv6QuMGZ&EZw7>&7y>;;Fg^kFf_xAQKIf)T*nat3O(F$7d1y(oy!Gqxnz3{4XRcMZ} zj`m%E+((DWeb|CET=dxD+@8m-=M;dJxsqy@bat-VePPu2A=E`bR=)#lukMlnH3U1; z$in=5rQ_J6uTeLojg9VDS-J0TSnG`7;^IpCY~=t(%Z<#?{N?NG3m_%|_B=N>?A+P2 ziYm4`=}`RZoWQ3nib$Abx+H!~p)Bx#FH!qe>16Dg|B1n%va4mh=H8nnR!li6JvZiI zN;$p>#x?f98AAs{R^EEw(bHoi6pVi7G%h+hIx>>dgWtD)qbuSl^+lhZ;_C*WKg*r$ z?6}Ho!}s>QcS0#y6t@x|OIv(v@RxL8y;xGB5>L-wa~{`hIm-!%6erwySs-TKY{xxtI9Y$JoMJ^T#R`0 z=FQyK__#Qt%H3eqAtrIRFs(c&b!f~2pmEZAyg{LGu)Es3+o|R6Fj*&V-kSn;pm7AW z1XaKZh9k**)f3n_N=}8reN=Jcwv9AfKSvU7cH{%WoNdBP4%zrSCL8lz^AMYrlY;#Sz7r{=#>I#5rL z##mWaMo2;;Uf4yzpH&;|hDK`Wt{qk0XPC5`to)_Wk>Eh;sF!hn@V50=?wF^7A==nU?AyuZJXq41%1py37me9Yn4J7~{lmOp1!o=9n(5az8&OrBNg^sy))7{mz0N6&*EZ7egH0yZ`yn&sD z+r1b!4l#U?FGZ$OU|D7bif5zAmQ1>k|(R_?F>Xsb6; zL#=OZt@J%_haiH$!QsA9&{P*sdnu%mhElwAT7}E=jI4x&k#Hmd3#1lkk6MHCrR7)? z_%CF7L2Km<=Yt}f{De$Q|L572$epEuNNGIIFsN=q19!tDJ_DV0{l%G0b6CQlpdi=` zoo2(gCe@uO{9TkQR|uPGj$0xT+SyZHaURrU_ab0ls`|D(RUl$&v?Of%5S7 zPI-y_&_xa-@Fif!gzUcL04e^?iH)nE#ib$Y&|sO3vX&VI{37Z2>(K6C@aI2!d*wjX zfj_KNOVlsV%ru|ae|Gcw(Lq&-3`O@tQvpbOsJco3Xgem=rl*gCiF7?JL7WV^y18L` zm;jIe^vZjC??GvE)tBbT@dKsrz{{&?RW0Fvc2~=smI+t@-C`Py%R_J40}7yK8GC>C zdBF@og(VHv@}B@{cbA78)~1`mU2Tdd(F$8kSS%ZNdZ0p_&x>z>9NMJ@RZT{DUy3}2 zCB&8gVqXCq*{{)3w%VD;JWnp@+CU@3>-V=lk&Q@FCSb{fr28qw?|+XM18SLfOz+Lm zJKo}bJt&xAd%JskZ9AR8sJeYE7Vg}?e}8>Dyb}_EW$*nxJ`@Ys7o=RiZ;I$J=<0wS zZnk8({QUj90otxq0}A3bn!#ZUL*n7R z%*&&6W<|BYzKxFeyF>2jIH@zZoD_xzTKbyI{&`;B ztNITBBaamsi+!#POC&ibu_$qw4KPp&^7)Jy8InarBq^|3ZtzQIXJ=j@1N#-hI`WtaSG^J>Hd>n++|>F< z=f&KcVV1UOP;q{Q3XDTD6-`SEoWBb665#&;{yFo2i$qYQW_i{C(jW;Zsoh*+It~Q@VC^ zkdJ8uXliQ4MMW(_9||6Qyux7@Xtb`UA{yleSKiG);EW_@7zPuA2j5qYM*30c&*_B7f6-K|HZ6$NGH2@4~C4`A9ktA0lh{m8mwXu1bC;g_*^Oq51 zmLO~#r~%{}%V7D&%WM;lpRX9WYfyn?MTzkUDvVT&Ht7{XYv4lyE{#%}``&iUR=}$X zYLO^F?&tDklM5rP#z+PnY|u0y8BoJW2A(i~Wxxos`X07C3x@U`F%nlO96qFPH;b-h z!5nyoWB?SwjyYyIe1j20OdS+F-)H1YJ1>i)x>1;x9~K$koI2p|w`uYwK#{ToMQV0E z#LmS7F0ZU|Ch{NM871aJ&!Vr4MqudfFmgu-I!`cqhiF6H`(4^A7q6AXMdG*deV=0% zX2X00q4++L8*aVY2JbFLkMabgMf)9mrklbQWfCFgt z5=eI1tZdK7vBeD7=MR|M8UW11^GD8`CdlXOG%Z z#%K&Y@?7nh;mbJk&6>~PLsU;MhdE(N_$yq^c%VUwhGR*oB1!E;ZErV_o!;L6v&jFm z$p2%>|9=S=LTvy-Vki%Bu!YDC%4~@Ddm91XKK<f6? zNl4H>L8^RddZ6|q8IlWolgsl99<~OMXn}1`)O?{J5{`$++%IT@yAC4ZWj_2PK?qYs zxM!9@h4!`~!%`8W;pBdQ^8;GGsRpBm)o-A>^!3As4`h4#CJ2y;R2**>0|aZ^4)$aV z*#AK@;BWX+GY#>fjYAuHuK;cL4-Tluap-~MZ34wJ^edT791+lO&>BBE^aX)H>@Os7 zu}QUM4IN1@JOi^u0Nla6?E^;nrZ6OjR)#8*2oaOax3GyO5tKnxP*D*tOCCrksF#^0 zLV^egx-unGjZsvXdYBuOcP|;MPhRh`k7nD{$$RPg0pvY z(y@xBHf4yrFEY74ZkV@ny(|Er2)>($3y*7huCQVJu6H3A z+NJ{xGV#j5y*}^Pu{6+Ap9bF2!-qnFQO3{i1*L%FHn|1ELz4zS!}{kRK0(5R-xy`A zn0mW93u1&Irz`OK9Cx5o&AMqv#t1ZF69$xS$$0%XdHA)37CCp#;&5->BLceaSC0Ew z6;7XG6O$cDZMP5z8&+3Pu5;W0`Iya1vp!-}YIqY=9AjIP@2q1D4-`3>Eo2bSRT5qb z9Rf_7|4Ai%D)Ik>k?huOhAztGL?HnXk#gJt<;>>Z`gc}|AaT^El&0zC(6Oy!&@;?2 zaIxY9TawX=I?`s&7;NeA1lawZM_7iziMKs{uA zpdTb}x}u9tWC{j_#`QKcP>mR+UcQ@q*rKS&tbH2+wNk!r4PeZ2ShamtUwIsK;-z4~ zi)f5^jq^J#8U5c|3#tZ$k)D3}00&!_f(wX3mOMbEAXN)X1c?AU2glPIU1)abQqV%z zm1^02q31<7G_g|u9(5;QTVH1hgF1I2BQux(bjQ-b?o#fDpka&Cf>KgwSmvM@hHerY zm8gu2Oj=r+l4lqmGcyAMRF6<6ZmnAx89h~g&hl4DB8tt5MhhUYU0hrgR3Ws)PZRQ; zn$jQu9#h`550~Yp8AX=L-7?eF<(xjjtCXWU@tBwK% zZ*{;?%?4g*lspKk4=S>a)#B69lZA^`QOJ{DSaEps##hiR&*y#m=;l7vPwkIrt> zB?&4Di{cNfQ;p{6Fko*a-oOqO&FWa$=_v*MQrG|lY%tVx$>8*%j{qrcZ#LhAyTI;q zZUx0+UNL(7>sKhyt2!7+Su1@IV`F1f7jNX)GR@7;p9xu88524Xeg}2a=*{tE&M0*( zDZjnMV-gYQ&?ONOwEm4CXaPJ31XkK%r07J7VEItIzVQY?_5JTbO`zf+Opg&6a^_o* z5u)mzHbE7!h_Ua4G!e<_1F-2ra$K}FUodDJzT7`v%Fn*&MHzMW??g`1BHV8L14)v9{7yY(9pC;Zom!n`+x`$qasMr z#={;Xw-9^@B~mW4aiybLzQ^_Jj~2;gLV{*dj-w^PJquuk7x=u9e=`cRg#^UQmsW(m z%ocQrDgWJb^}md3J&U+Ppq}X02utjh`9nk)q~y1zcIL70%uWkXL}n6h6Vm-M2HU7FHn#xM1gwC%q;62t4jIligRFu8rSPwNf6Y` z+S=N33GM@BZ2E99Aan`_@4C81fo{~kf;x)IG$hFRZ^G8~0Q$P72>IF_#R9sfa7G~$ zAWbPClCg(*-@pG^N-GTi?;Mte4J^D3-hgn(A$9NbO8A~3h z{eS^VVDyDkxRa}DX;oHLafZEr|2{7-Z*l1WLUb2-F~v{pa1A$(oFRJ|M&z(0V245_IVq<$Bo-O7ribmpCkuk0Gh+7(w&ckkUh+UVq8FEJ&(WFkl!3`8e5 zPG9z8Ou^Zq1;xEqJm}g(2oBXv1QJ<-VS{Kll%t8*R8&u|@DgPE&h{;9q_9M2YHDSd zV^HxG>w9LWrjCLV-u@a2LlwBWZEzbmV<9zFg3j4budcd$ipB%4g+eyp-5W$;FA0gV zCA`tg`h5rI%`aNCW#-Y) znubUL!iD@YF%FKGswB|PR%E~|THkXOC3`|EbO}!|Fg{^DlMp2gM9H^E{_Z-n80@-R zCu$ZOn+^T`-cRcKeFhmn?}y*0C@Bj~YlkN$GDSJV!0e~z1B6j)z2Fa72JaS6M6m#@j!j4q zvngD&-bT*%WNL640Q;)m;PMTWEIEC4f3=DXqK+j#HCW7E2ndlcHUmM^?3O^aeu`*P zF+9fD!{=Af0))XJF7G#Pg8syly4J9;FtAO*J}uOp{ya5JO`_y8DH*&{Jb>7>c;l7b z7r(GP#{xDv7Q_{%w1=QhV`$^d2nvcu@tD=62H*nqW1Gu2P5n1?$yI})C8bL$k{}NL zM#f2mR>ozWY*U@L=cIkgA*R+jV`er$B%hj4&;x;HlICrN2l5Q2te zz~LS^16>uHx2|+>I3Z1c^ldAv%^xY|^!|0wVYJPamXP?mcszqf#wp}L3oR@WRL!() z3>kkB7^)x+7OjIkW+teq5eZ!>6bm%=`OO;+tAMiLHcZ(3MblxF6zoaU4m zz*KK8SJOVNphy+Cg&4XBlIu%kv4ewy(3^Y#aO|$OH z-(a>tBA}6xD;5Hbr(lTW1RqF@8et*=n8A&jiH{#K}Izt=h3aBP1l${pCUp7WNesMy~2%a|{za5u>fZ3&Dk(zGA8{ zrocER*#@W=>pb}N-@>TIcOWHS4B2;6;Q2h{4ThB# zwfhpJMG9Oqfc2-mgJ4LdN@u}mf!SjtfKHk|C@Bv6;eZ0M+@@f+jdRNBEpxa}=#WTIz8zOX6Nfl=IA)QzyR)PCoSH zY8)=|0}@!-mzekicRlN1bCQviM8&``4zmX;Dk?8i7E4{hkfza!sj7`{6=0)CJI(0%o~%)B=1gW)Lw3 z%;ybP8toaBDp}6Y&rtV3Mapw;0&x$7&lBZ$bb*;py`77g{P+>0j09k>sjl84dXHK) zPN%l3rzZ>CYT{2K`Z_Eu?EO%I^pn9c*pG|)Z#5TSdI9w9$Q!z9B)h>E|Iq@Z<@)QO zz%IkJfDt&LdU4M2)Po9RAu!zg!SK970RbM7FBT5k;l%MZCfR_q6ciL_bd$$9;v~jp zMsjj;7$|u7kfo}Gte2Z0L)v$^y*<*}{X-NPObvNkb8j+80RiAL)xgsO27ChHb)FPQ z+K4k4I-UL1yF>cshMd@O^rsGlhK-KU4k1Qyr8WVgR(3gqPejW+ehV7#zF@%+BS9l? zu@OxT!!IyHSOL8esOdUI1CSq4P(dgwEB9B>hM_@k3|fu|(j{Iljg1ZH zobBuiQc_Y>YbX@_{QO#4TPeuN*Ln_5EFX8~1Fln!r!V~gNd6Zn)!YcQT+YJ&uJ7*J zKCH4IT80AI$9S3*^(3kZJbdw84gij%Ph|AK^T4(LH5DDV9vt*{VCY7SA4D7V868k* zf`%E=q!`49B`!T^VRk_J^xj*$wSR*5jB=ViZWFZfWMpLEL&7Q`;D)(M{b5pWXoy2! z8q*NJ^2JUVMidO5=(I!!&)1*|jjJB?ltcGr`JKG&QC8(Z!Dpx6X_P*S|_@JLG zfPtl}KdKfzZ2}zn>TtBeA?1wR+`>XO%-VoZA3`wYIWm<<9yAhf>IC4xW=b@c&Ac^>&j;G;pVDRQJyVb{m{ z<+O^$!v5doF~Q$@l;H!kA(M!f8e2ibsstA1?ft&7QFc5xgCzOYF0`{gefk9K9+X%o zm8~Z{SMw6ljeT;|B)@+yUu9&uilWec$IyxiXt> z6h_aCDNMduDfyjN0m+yuV2LOv5G_Cx3^Sl;1pwwrj~-dRmFb?irmT#2b2T6!0KTSa zR~iCYSa(%D@c#HWSKMcCx8Pmy>A3vJ$aa9aLr0RT5YQIc_hHTnrvAnc5Br`=pw4r0 z4!CGT7klUuYaHU#(ep4BU<^%xUXT{$^ z>rc~t0~banVLzR}y(eRcqtd!(X=w=x7dk#D$3SB+S?!)*@?zNmWCZ zwW|%juzy3OXK{q7gF%A_S}cXAH*dH}@`b5y?)}Ya_b@5cE8vm@dXEVK!wk=Ae!_J0 ziF@-?nQx7OvM`q;EL;xpL^FH?S6^R0Fz{q;Z4IhNXg(THFsvK&W=hgfQhq>QWXS{g z2VfIDZaB6838CC+Jh__=?hX_MFv@I9heuR(8M*U&Lc_KP3WwMMMyFHes1Qb`Yqx)# zWvjGAz*+OeVsY?@N-T*^NRD79&HcA;g4==6I2DV6>h;Yx!%&PtoPqE!#m4r1>f>v^ zOmTPRydef*yOO|@fSaT>cA)^^T?Pxt(s|+bVbsNbBDmPj&Y~z?4*Xy-Lrq=1-zl8? z)aaPk_<^hoG|WLFUUNeI3!PsLRG*zNNVRVsdB+Mjfxg5S;umChFyZgn7lqW6QGfv* zMoVU(CWSH6$Em3W9n!7WG9{q=`jAiqBdjpteEEFwn#V;@F2%q}F`5z9ZjfjGZGKU-i^K~~&yHYP*fmVF80H$h8Ohi*rQIV6+e*fOO zm0AEb2zaD|fWc1A|w7ViLzeA+(L+uh!dkB@Jw^gGC>9r^`j@j||eoLm!tsco?j zHyzl}^z`bSpFzCn;AGYAeVSckK(1Iny7&c+q*R-R8-%bOhQ4;K7X}TL9yG7WLpFgR zs08U67x%um_gYIFzoew(pFe*h!^1&&$mMX;9c~6xpZldSEb9jZ`j_R1-U{#$03(=8 z(LM^oL(mb^{YEBVXj^gRtg7-24V&1qcVM5kbsX6)j-qaVu67a2v%v?5a~Mc#R>DJIXD!nR61F zAR#RpshgXdODPVibObqohzdqwU{4`;0rrCQR(J5D^F|YlL2#Tuf7`}}etJBNeerBJG3OeL z!GHRs3RJZW8O*lEL&+#}JHqNl?hnvN*!52Zp7^tWOh`TT10D@%#=kr4lSCQBH39pw zboDmG3LfgPdtQD=2lpzmvI4O>xg``(jiIIw_2jRxcDM%2;8W6ZwB@k?5e5O0OaTA_ zBxQ^KhUrrpS1t)s%gr=BqBBS$)-UkBgo`A1v6AHLH1BKy1_Kfa6w~%l9RTePNsV3ItxV2#E) zpm7BMcR~@tT=KCx5fT}r$;`+1U{W1s@}GZ-?`wL31ZCPHVh^GTj4qo>j>6k0I@;Sg zFJDg0Vu5H^57WRfRu`=Q$Ky>IV6AjPvpqfz?9cI7zzJ6Zm;%UUDWN1JPG&bzBXT5@ z1)Pr7T^4GL`(k0Y0jr{~`5MiIqRj;K7Q~AH$tN@8B6XGtTEN&i^wyGm&{F!WfTf9Vr3P>H$6a!ghiYw|+s;2VBbM!=^3U-3=BD>!(BYq+MJsNdd`E7s?a`W3p z(IVrta)9RB>e+x+oU#{5q%uolX688xiUBD9%o@kW$Kj08K&RBPM_tcauOz(4WaEOc3bxy~sk4No*<{yGJ ze0|0pZk?K%IwBM+?$a38Ti`C?B`kMd!Z?8gWwz_7rCC(Kju(k;(_Tg$k_JNr?fcUS zN$RK<*RKQ}41NKr2oh9)X-!d)6TBP-0s&;3(%BlXRWqp702v=Yo;$-I+Kyvp^`gmD z)+U`QA`;ozf`mY%e3Ai}sCMn<;3*FlB=-<@fQEo3PF}782`PVx*9InDZ%C?!6UE6~ zErJF_H|ogzKzJL#(Mv{Ta&!9A5}iZyYBzQ%DHD&u?5Zj+P-jAY8rP4TL75;2kQr=b z@l_fI$gF4Nz)M>G7J35UJ*ZJ0f znm3fj$)cxS4MoNqT7^{lxhw=AKvc96!m1xD(bz;&jX;`LIlR_J_}ps!Zq|KYWtY>u z`d^J%W^Ez_^sa^K`?fYWTa&HYunbl>9;sA_OG?5U0Xlxsol9<}k?iDf;<_J#MG_`- z!Ch?u>IIkvV-6b~8#Ar**6GJfIz1_~Hs(b*N-elnXl?Oez%%JFJF5+Uw0>MkYF|4a zP91n7U8-e}1~<9+=X0w`he!A=Nxa4YV=3`l(m1IA;5`pixT2uYHNfU`O6zD#1Ns=s zr^P`Av!X8t9!pIu6v8|!%mOIc5e(#_xps7%l$JgSRw|j|F#*j0?=GUD*#%;;Q?and z>_>H-!o5^>JWNoBj!_@}3bcHaOXU7pXiA`ief!d$;lfM1ctTcSXm?AI8E1>Fa_!r@zn6x>_3Q2xyEikF}t^Up!1I^aP}^m9PNmOkAj)cPPgT^))X+8f%;ps3mo zVg6#h2Hpj4e}kNLIJ4( zR7)BvDC~ZKMOv>Z{fNVviBa!D|AD;W?c+m^At6`4Ra>c2jQ-1gp#N|x9z>V$A;1_J zsZ_x(hebBX6ZGU&Dw4blIlO|b*Ir)Xxvzyy62{HA2B>#HHkE=ylPe3q{GV49n!;J` zJp@+5KqMF(r}oCH>tTsdeZa6$|7$~9DQ{KgP#vlP0hpFD$6KA5Vewh(3G{Gx-`w3L zB_xcEj#kss`Y7nFw&)XJ(Gu+X=))O(8Q+~Q=**z6*LW;J*OE$9j`&}$_|NODcu@su za`1wJAz3g|-ObAyQlfP>CYE7sm^*H>ZLj#iNKJ{pmV4+DJp`1v19rk;XaGMe8|>nQ9_NOr85slM-8-K-3WP5NgA_h2EUciQ0Kzg) zC_kRPGz0sr+-0>a=%s?Sv4uV1HQs@+`bf0kRPurq{0K5UKY?^PE_Mb@6YcMX-ES*sbtnHc@MTwAOC;T1Gn33EV=5WC=N{Obkyva&vrncVE3m(bDyRxH%!Xn)Q_S>^2l32<+Mf6|A<|Kr>~Lc-~-If<~1YbE$Hf4 zv`ckE`P4BJwSkPb-A`Y1;%)yEPZYqH*pX55%sFldNDWW`aD?3`(8dgOTx4>3Hgk*f zmxAh`iO!1k8g!KG=QpNGXN1E}9_=-rGzOkDz)Mb8@&MGpOw4yoC;0TV%1~1{E2g>v zls~`Lr^3{?pbGq@{sQWSGVWbgrM8-7#m?!aYx-NY+IxZZb#+iPu|hJNogJ)KV+i<| zYaUusA|OG`%EngoGd8&Iw2_&b0*iS6{&O+3HghmEEP!nxSAc$N|G)s%1lBvTac@zvhH_xG&z-0OLsKc4rW_g?GXYiH-WzTeMyp2v9{ z#|ihBB)Mvz=dGmZKL^KQBzyy%A;TLcs>Mu({67`10JW<)JMQs`2nGc3 zasdBuKY!j}B<7Xkt2a!e1#w|^&5c8kzIuaJEdfOp`3hitwztw9kC_LET^m5SO+Nnc z$$Cs!pJ%oH-5L$n!_jEZ=`vr~UukNF;1|3(4ic0c26pypK0^L|C%RTX^5A3JoB_1R2*cuyqw(xLk`t362!{5GD14JcKVRqKq#+WrY{gH&hFouW6FqKRz zKhLx-+E3sYWa9GYdfE^c`Prdfl&s*A)b6naE}JJiO!j%$V^Tl+*AWg#G6k-@084I@}hGLx^n& zO-v+W8vGA2jjU|6WN<4oggLS5J{$=w0w>}QL?y^Y*rQ;6c8J;O)B z1$PJW5n*$U0Jf=_M$9g44rzfvC|M1@Pjwto_dRxz>g&7S z>}onSJ^dA81)NIA3~pD)US6<0Y0YO8>QZn@?mtDVBQ?$16C1CLPQ z8h0LEa#=&|VHW|*a2$p4A7~D*Vh-BajDkYAP&M-Ao|Gz>DI@twcO@K1Hk?Q;<^UKI zB2*<02Mdc7U?MuDB{#9YaKR!EB2sBB@?FXd@=*vZIDIt`SS(ME96I2=$B+nvqa6Ue zjBv19{lReyxiB0jZribNldfr7Ab*wlw7eK-ub1E|Ms6r2 zCKjb}DUI*N&5cKbk&R>#)S%Up+cucAwBI2}Sf%WLf=V^bMynnr`vXX+|H?iW z$SrmsXe6{nHbKIM`6o@S55U7bE!8d4rbpMRoegbvW&}5jNgIyu199X z&n@BUbvq`_`d9wxYDs&-#^_c(Ae@7y+FDxQV}LN+m4!cF_a(e62)knTh@xMcWoBd? zhzGEE#qb@Xg)I`O#J^Vd`Til480m1k&<5-sQ3;$ZP0CHJt@nr_)zQ@*kpNRh)cz-A zJ;aE>R_u`J`)9rhrqEM8fAL~*X{n=R!$JY0g!kpk2D5qzU_hF;52MVtkdK0du)X}5 zR~+vTbC9sw4m5&VBi#rvj9{GHY<~a#{lLILeZ4ibR87~`#>2ku3sGlzdiLD8yay&Y zLx>jun?=I@_44zE7cbg-4x$*^vs+qP|5G8d!q8^a2nfR`vkiOPG~4s=qo;Ag3>fk| zrTvdxHo9E?@SO2L2_kFa(Sfz^Z{IC1AMh#J z{rhk;w9&iITzN2p#P4z=f>N{gJm5~mheJzstMnI?UY8nyY#fTe?Cni_xX;Yn{kLOq zJ&j6i+~*@^KRP^oD>=E<3>m*kyBqF^r$F^uMTrOAEhBTe5$uP9rh0(N6uf^6StkEbIz(Y7>e!+!^F zp$}7v1C1!lMLx!IDS6NfBd;3#d}w@phkYD`{btUC&&=ENrdW+ZiV&3mgx;XsUwN_M zfeGBHumbDA8Q5>*MpdW_W^gY&Yqi{s`R{W{(+eQe))eN5rFd{hHX>=8>Uh6DpT4#J5hAoow70O za9o|T&##4|2mFw0Z@dQJ&bWOVUTGhMz^OVA&q>x?sa*h1V~Yf6L(R&a4rRA(LZT`! zew&T~Y3Wv}AcVSjZR%sWo0dRBm?q|KEq~ihB9Fn(gYqZR7uK7=pe1r_)8o)tS0J8R z@-yjU+H=qnN6UJn>Z!$ff>`XgPe2tI-3MnaU5szCbp^|_!tWq3)m*Fsu2?6(aYUMH;^G$Sn|BEqSVe|1{FTk;eT|fUHKNB%HYS^q?z&a|DUO|da_@T5 z_=@IlXzvue!b`$`$;_D3eRSlj1&8CYv2?J!3f9!QFd@ zaYl#}VY-aXnm2-DO0yYt@(A%}gA%t%kxADLul`sT2MMNz6T{f*Bl| ztR2MsYBlpP(XMmFj!ug?S^Wc8TN;KPBu04~gc;dey1#*t!6Uah#C1mo?i;<%+v68U zM@FtTnnfviHHP-atjHuhWBVU7>tNMnx-c6$LpWQh7NySG6lsR_ty^Ui`r zn{9%W%><_sq{>bfhJ-pZpXv3c`mlSW+RSL5&QG`TIuO-jXa zoEFm`T#e}CN<0WU=c#~*;LYg$5z7&ygKS)W6!K?HiI>;czl_hw%8De0W6xP!qg-Wz zVBoKpR#rCayv9+dasW)J+u0dk146R-MMU@PiEH+PJTWZ>^qI4mME&`yYp`f4Ce}D7 zYj$pUVq&6}1UMP5=uSSpRFoZ;yWzy+q*s`h-T>eLT*O~NK7^;aIa{N!Q<3lkF8m2Q zFFo>^>nIr?dFKW+kYb?oa9@^VTMCunS#DJF&(Z)!yQjir-}U(daH-JrbDMCivR(oO zb587m%HF-k7@>VDzuXnyY(0^G^0g_5HWB#i3JMCu8kKssKz)OYj3L}ZL*tZY)h%PJ zPXR+0)|gIRxxDvk9~`*j)396~-Uh}HPs#N0s$XEhZNm~f=`;Q5YIYn9n-pH`b9}ec z;E)~*Vw)@R(6qRze1!i*geepc_d@R^CQ3Mrj*mB9KdYssb=A~Owc>)0ap&zNTGRpV zX7?f(?crhyA`%IML(Z&!#m4d(l%Rg>Mg^LP>nJnXgS0b+_@_6-2aO9K>}{# zv;0g%G$ia02QL{@=fD5D9)BDTkQ0PipBqgzN+3L}uOx<_!M?zt1lL2+&-U;WX)%aO zI7!36LoRrcK{M29CX>nK0jQR9mVHXvPL#jmXCd-u=HuepA~68atuIPSoRhL<{{kMX*SQEJ?rO#eOG1|N;V`r|3XWiMq`hY0LS zJW`c7VqJ~3rKZJz;$tk)D>;Q|ANp>VBPq{<~aD%g!6#{<>XU^TJf30i>vaJsdD; zuG!_u?QCaA+1h`}_<}iu=+>?6@1K6<)-|uwDe?tLfUuvuesaf-hgdM~+`Aq;*V>3* zCV{0ggg_F9AVFpahe@qeBMpo&kPBdrCX;yDTHgj9L#XQ-i1El?1eXm#*ZD(Se5Gq- z6@3-$?CoE(vI~efHc>OKK23x$l_5o|(BOjDQ8Kx%^rCG<` z+!6R^11Al-WY<@^U4i|b=8)yCTLFw5uNoVbFHfI=yAJ#uev?Tjy+TkyOMuw};e6eH z?sAn81!)9cCO8i`Z25syuhv_v3lK*wzDmKZMDaVk`Ok>-a>au7f}^W^9ffs2&7ie{ zpxL!+q=-FK#{7|Hx0}_vz~-AJP#6tef_pNK0P#GKZaZsZ(HhQ6NMI4h)z#GjC?S2| z=R(y669duE(24dBK|1dd^q(%Ypt|-GyBdQNXO^Jm=GuIN&^dK|TI>Fn61xA^8ZI?z zoi&VOk!!>~eQ!Ni)TyDY5(z!e9PgHGSB}KK$0is0qz+W{yjSjf5o&pl|BIwMIK%eJ0<^uLjtK_dGge>mKU2Vm`l;Yl&MM~`4*;AnPm51v3zhC)E zBp||^O=U)ldQ`6lW^Va>t-iu)H&0?EA(3JlED%4IdF_1I%&ajjODHK~MpK~^b)uU;E*g_*Vm2S=54WmA^A!(j`^j*eWe4*@EKy zi;J$V;GM?(ishrib7*kuE7-mVO#wrIfMjIfS*b^R0Zt%EcnH>el~(i)BWKvfw3F?p zzW)7K>bgI?Nmy+zG5lHcO%ViV5pE1X>n!h^rsJBn{Z)~W6lhhRrYpQ{+e-jo97mCM zsKm&ss`}$dor*Yz^*7jkF?QB?J0_v&x*BvOV07XlKBOh}%dGg!Yq0eQYm@1XPRTGn zKu#rFoRM)BoVN|OuRw0XBi-OQ3JZOeFuBJaYsMo}<}_n_G6h% zsBj{t*tws*8aEa)p1?vRiye)8@P5U`lWIv2Ps_fJg%e+^M%@sFW!~XzXh_K%@H6%r zVBBe1dd1TL49}cOg>UCUR#@n-NKdyuc~a0N@Nevk*qy{u*NfX7`g@Y5${S`x_Mm;c zB`r#rSMb=pf+fr=M%oJmg~;?NND)GwR{tC@t6#K_UcV^8bIJ0H$2QlGFIf9>&^6-S zz>?|)EM=gFnQu`P36r|99M0w}9Qc8(mSdQtc-{(4GWvIWfE9?y1X`PmetyqFDnKEA z3KPs3x}#u7Vh0~!sH-4YVjOAuV&FNoOum%;vy)@@V1;G+Qm0rcOw84{kL%%6373%& z4iMM|s#ZR8a&yyYoPiixzRzmI$_}bbjEuA-QHh0jHRK*ITUu_2J1rIf_XDZ`Nvwy2 zo`Zwe2)zzuYe;%!lyrAZDc=bGu3i!gM*5F*>H1EGu_JUc5onn?92sy5bAvC|KgmeEXN=arUul8FywtH0 zT^bmvhH~L7l_0N6CS6dc)=9avbP%^ue?5D^)x4>F}jvWML_(R-|bhgzoU`hiy$T zshzv}C@(K9J>6qk%bNIBl|Wox&~Ve9iW3Rx>FWy%34x9Gp@fRoIGO#a`fXhwA)Uy) z&fEh<94A@)L>_~;bl{)2)Cu4M#|R#*JH*7K**L83pOaoke567k)X69s3JR8<{`3=- z<8-%y_V%OGm6;6*;_uin9qJr_B!r;qC_Zz*>!omrv8gFw@_9fmq0=Vit|Gh>APWFI z+|ijNfFNJL0D>NeFF`hc<}!f9qKSh!u*S+*)Z!HB+0vwPU);M)b_oWMgLY1En34spnXg^rh^^v-3D4mQV>2^?{#CL|MF7yY`PL&z zdDm6?gjcU!EO$ZdkudAXTekxEIiVm-6FTXn-3XctX`|bnzr>>XZ#Cp@q7tp)c=h_V z2{_Q<;q(vxUYZ@x`gzP%#-=G{4&6KeEm?fHx(&(Zp^3*Z_HhpM{`Y3%BQ8_xTQ7W~ z)QYyc!dk4aR!G75{%FMe3XjHyhERczZ>Hi+h7B;Pnv9@OPBQrp*0&?ar zTq;@6T-zb#4j;tX$(m%f{s(+s$Bs4TMEKKs$ z(;mgUWIRL5f`@_%O#{l}0XHrJmb4Rd!)ms_`4g40#@DakT3iiZ-9~*Jh+LcydV_{M zHV0H2NsM-v!j2u|gfqTPYHDgO-G!S+a~^CAap^0~QCi=B^Uum0J-nuR(x&gx&4_{0 zzu6B-1*<_6#2|u2g@tZaL78^DHL} z+OwSoX(W;7eSLr>)VtRFiEqzvl_NIu z8Bn~=;6(B2_{|R+{`saq=L+$K#Hp#iCaeuAZTM3?Mlf@-+D~Qs_Ae;FAUyy_H6w8d z=>Mx%B=gn97SAAd_f2eUEVxpE&g_hguiw0}@(p7XI(QR}YHVL7NVeKO-ecX1!5zW80BYA`i6fgFp#SZ@gOgSY`4930yG z_be(lh$rKhXF87j`0=Bxk@RSFR=F7A4fhli;mVu~=f)+QCBPOIjE4&?xq-W617w7Z z%+-}37G?or@L&ElHZ}%)>CjG&3X#=)skUoV0A_YQHy-El#+Xx%)ZqA8DVK+4Vm$0^ zZRd&FVpf)8BBKB{X8`E|H!^KooZ(*vUPHnL&s5oWh^oFgOohHLR}9A0c}R*C84NMZ zYUdGc4wRa4{nuv`hUkmMrp~5jW)OCWqj(Gm*7S2&*y`5UzUm^euHk_~HUPL2xrPTC z6Yl1kpu1BvIfebt-s6!XX0B2N@yFiLfx&^wi@KFtjlD!uZmH?%-GC{i!_5;}DkyHC zy4o4bzCn`+v|kbFR5i}FL}%T!7XZu`E_<3t1hx?0iV(m8(FFi$w9CmjFCKz~fX%jv zmXSw;8b9Qm=ys@%<(V1_{#jPi%y_!y zw78<%IyD?59L&0^x1Y~Cmi1ro^z>VpN)=%W+qJreqH73aMUU~!5#0m9abUh<_e6>R zd%S$a@#5?=8oQZGY#-@@x&Ms`4w#YTF9g)Y3*-KM&VVwNETmi5T~!l0dLQH6wel_( zDXtEp3^AYA1lB){AkFAmz~-;rtAJ_8D^TLrCWjHdz1|+UUU(| z5qqi3yOYPU@NN}wujRlmTM-wu6sV!chLoH>#Je|qGOTgHU? zPI0?Fjo5!~}lOTK(E>|9q_>b$oa7 zF)<#*%vqg1e**Ki^?^7a?ur?Ze{gGi@pRwsSbu>fggA!9PT_qMJiXKYM6->;DVD#V z09$&^a4mmzjrnpZA{U3qn2C2{!IIgR{L+~Cd7u>$OV}&eAk1E@tVX-NKM~uQj@rPY zJN6{JI+W-%+*ikxiV>Eesi^G6YbodNv8|!2Xy|34X))^3d#E9v%sKHWUg<4SvduVZ?{>G2`cKNK_8kLQwNSSG z9RuT21?t@KR@|&#tM1atXcI?*ohWp+=>*OnF?Zz(!}a`2SVgO+R=_O=gtuIFO>~U0 zG=Vrz#d(;pNP4-4vTpFe$yHSH~Wif@jUB9=Z%jE4?2 zLgPFX?>_3i!J>I}VViQ)6ja(%sI_SzT|Ue=g&D*-l_KbM+Kar#<>{dGZ}Y(v$*FS$Kn+pKHepNSDn5qk&(go z7Z(E12GiQYDAv14few(5G`|%|RQNjLBl~`1DMQLmRzO2$ z)co-~jkZ@YTpHH`NTM8gHY$P7K@o{GK?!~MQ$FjKIQi_Jv(!B2r7q2Hb6a!Kp12%5 z@3%2AjwNGYe;zi#2VCKd3LxP_x2SzQsbA!-=6Y;B^=MAd-GgV<)dP>_bpAUs%iE~^ z1}0P$MV(Tt?uD=>Mg?rZ(Q%KR^FNR8KwVuB2Nxe-HtVLJc#;vBUq?t-gS8^|G`y9f z=f*-vGitx4gbhuuDm8Qo1aJ(=Bec~gZ8z^FuiFJY?R#6Truhy7&PMa>mOJOIh=KC`$d8-ue0 zF?sUQXZrAIj8PZ&H?(lS^O}&;+_(SV-4KWeU~E%dXGjs` z=U9dFu<`{R!0sT zxC~AS&c;%8(Lua8wwZb>K6{*B49B=Bv5(}$%fhmIy3%TCbZVlAQOJrUMnUEpcHh7& z*6a93&92HiKrB-CIZcJB>Q^Uc$NzeAI0j~qV6NbcuPK`H=b_urghrTi3+)Xfu@La9 zPbu;Ivh@tvJ=fdD@Q*e8C2hocGH@wXj;n1WF+e8&TLT1uTGxO6-u?Sc8|$4Wh_!Wt z+f;<=H(pROdHa8O)|$P%OK8`II@2>iOHLfGA7h@9hy6FQ9dUw5qTZfon5}z{spm__hlUj(znjnraQp8wGF6V^ZtF^mNqNiIENe|89M4vlH9) zFBjndXsul|r0nG!cx+HK-om0172rqg+SdPcfMDsbKe*couSR-RnQry@>e#dE5#|?d z#sWz*tB353|GUQqGXKTJfk9-+|A!lcosEs~M?$py=--3yIfK_$yicpzEo{l=~Zl~GqG=9 zFh9B#u)I1c$YTGex8{GFlq-{qxb@6e4)5B;)pi4qH=$@>43y?7+XfA4I>e*q`%M3N zC%lc_1zhf7dm-ywy66wKV7I$n31fJ~z`1yV9j2d%?;I5UPxFZ(0UQQ?58&!||NTC? z`TPH`_tE>Tm#9|zU!K2+Sb5&$Mp`_(t6>sG)dGi0TB=T)vVJ>JSlp<7s)b&;SBbS+e zcw*;f!-prXKV33It;+tmGz6vxnv)SC7@5Q7X?rrd!+hP5IsAdx)PIeM5v@nOXbv4b zc>8pL6+c=?#+*t*4y*8l2^C4h89H>gbocE#3xlBOn;;WDvl5VG!x}%QODhf^rhFrBkeh zC}TqBM!4W4zvz%hbe|Z)ZU`;C@^ZUUgfFVsGyM;d337(dJ&Ukyj}Yp+SLut2+=0`3 zF(x{}h!WPdZ`$xul!qYsJaK>4rt9MJZIj;n2a#B;49(%##8EIIOIn!x22kUlcynEC zCcvyO?;iaPnyz7^JlMgzzAXc{%=Ri3`(}_uoZ&OZ6nLMvwU7|89+h~L3VfjK9VB>P zfFi+$1w@eNcPBs;n3g*$$ug?hl0<;)m+WRTm7T|XiYn~af5PLbuGPZ2n zIbI0~r2zNK9+RC7icY2x18Bi~)6*Zfq};%XO0+SpVmSh*D_KWN;*NI;{O@=Ct+x_{ z9sGx^o;?wO8?#g+up~#>UiXGPrgOb!&Zf;ht%au$a0E;+(=KTfk@&G6d>hkd5MJEE zMP2I5Xb`E&$V%+*I7$HxJ2TYC)P8t`$B@##W-U9?gobpX$m0^k*p7yLu537WE%75u z!~n{U%A&R%=&$LSjw7_zj{GdUC(_g!2uy1p&nlYGMS_ByM+hZq5s44y@d34Tm@b|w z76bh2yIrQY^o!QB%Go@SpgLwifpgkb?|J%%(b#QpPT3Bs3H)2oq8k#2Us&UEMoYlZ zQA1;8W*+Y?A+%x0-5DQ7!z<+W3#7XFc2Q7;pc;ny1R)8BZXG@|0&r8oiiJv5+<{&B z?!gi8eh7~+qURC*5-1sjQ~u*~z0+t}UI>HqaJGhq22kQ|EJG;!{{GtS@CVqjBeD;L z)y=>QA?;VZSn(4T5*(a200@AF2)}=8flx$aVssP({IJTtXM{ zU%!8&UpGi%>GR#A>O@)?aRMC}@W?1pUxz^z_>p*IWU=GGB!RPsg^luf`}r+{hXl?E zSZn<-cO>ezAh?6E_SH>x+VA9!Q~O-+=u;pFpW%YvAlIX5uwij9trrmJgKdw${|dH3 zfoYkzf`SnZF+Sk?z*hGOY7&_0;o)IuNjf1+jeIc(@0AJ$Y#s>h(YHYtt+XKPN7*(R zd@K|=(^68@5}5~9;Hd%#7@fk=+({X9D%dt6*;8^KW%+y;b{^38d^zyVg0L;}24ZpG zI;`sUyPO8QWvJis2X+_AhOkGH+d}xHp_z~m9u|W~yI%e~s2FW0!Livt4;2pV%|PcY z#dKg$P-7#EocjBXPw%nkvj!u%qCyc>3Yt;;M}UziK7f-2BFf$DMj$eZ4a;EHqYY~Y zKADP&3d?|rnOPaOI;px_1x|ibKL6;_Pn~AnA_!2>gXgLL$})!K7s}KCOjJ;AM8O?q zMeCS;pp;4!t_QdEnJ ze6X-)cB4wNb-*1JadM>XVVe8AE~Gvj4Ju1#qYr+zLkQXp*4(x^NrLYP zm_ZN6b@R<0x761MyQd}qF(7QnKzb6X1f%eZDHmDIr9RK)Z9yfREKn9dAbSOMPq4Rs z)lriG z5m|ow_N#C9v<#rr`S%YE=T3dwfuM5VWvokxE@ngpcUgr9GhlxQK2GVq)A91TSi94ETy-ju8u6LhqD%&JI%BOnCG!25N|*f z#4MxqlB1n%QZfqM>G;5wZ&VHY0F8AC!(W1MkW?DQ@0p>=W3JJK3$hY>gsfMvTv2B} zKHiQ^@#)i<;}2kg@sE7 zgQzFWd>c$&Zpp3i_VU7N;t>SpFU3J)9-m0p6hW@r%j zHC?SDY_&Es+k*$+?BD3CNf23bapMAnH!;uB7c)!_v$3`HhdrUIl&8BpSSIG!zJ&7_ z!}jC3Sj1TZ3eJaF4V6PdHnpnMg_$xsjO4(iWl5$D9d^3D7l8;*&V(leIC0=BJ%kVm z)<%^1>1m>yp3}>k8Z~T6RFJ$nQO^B2NU&)2*Jtni+bA=fEPd@XY4dpHWH*Xp8tXi9zKGhLpp)`qtAgh2Z zc<%1V62Na{8r!R)a=dbA%I6H4%FGPYK01W%8&y?Rz1uj`RBZD3!rl{5Md1OlIdB6z zet04f!F@B+@9E(IGlui1T%cFdaGrBl?p^UagyZAAV5oL6HVLx!R!&a33`AG-le!== z4EKXv^A^ML@!$N)Ybq<30QBzX`1D7MJzj-bckW*2*{_uvM zmsiUj<{`Zm9y=t=fP`ZJaXA{I2rqUsIXQX5{w(#Pn}U08?`X%$w~W2PFXKnrQ z^XG2#H(994RH?wIJlq6I<^?x5>eF~R_*~&Z5x&%#zinyJJ3dwI^YgavnbV-|fgwdl zODhL+D_>V>%Q{m07buXv{AQXKHz^h!;O60ZU|Kj|{&Fv`veP71%g3nw#PTIVUOCB=vINCHWagLRHWb2?Q@`6F@Lhdsoi_40;Ek8G1XlPuw&181_?(?olY9n>7B#2OjPh&{&kt}u?uWG zKJU29ElVd7;&f1*1ERP!P4J}b{ka0J8sx6hVWZ!^weqeJRnZLz2`S`ug|mg~&rjT@ zCg`6HmT$|zs|4b$6&M@#p%a+bJ2c+u$NGSb49lKsb^*LLEFJ?IvJRxtg9C(d!Xgjwpq)4X>0G0?Vm`Fkd;9uY;$*@TS>aeBaw*ybYC9wY@8-dt zO1**7NWXO(?9X5daAQMRQITb2bW)Pue2Gl3aGp{_zM(i_X)V+Umw9O1XavrJp;0y^BIa>doGY1w1XIW;|9gui# z9(Vi&koRX(vu08KMBSS=4hV}te4x7kCMddOretB4t1P)6af0Yb_omn7%jf4`po(9c zrv&+ubIcMtG38j}f*_djb1@+1+<_@?o8%pGYQQ=NyBkwiI0mi5ZN!-v!4^&BeQ*!y z*agoOulNWnspGjkRX@4NpkFRaEUzqwE3Cl^goc8K9{NmLD-RqTg&Vn0vL5@lep(O9b9^L}y>Q`j;1&>gBT&=0J2%*< zlY$kIb%R3xI%E4BCMP(oxU`g~SyD=h{gePfm>_9>vgAn)Ek>rXw0EE>}&wn$y;7Mavpc0{(4*ixBIyY!HfJTl>s7pDqmOZr_$DA)1rK76_mxbT!CI{4SLn-h$Q#*kqG;RJ ztwG&L5CVRppk|?ZFJR*Wn7~Eh{R0E3PkQOa@|p=2^z|{8Gw<5l?2&IXCWKgp^S6XV z!BX$)J1;`WUrOW|j%ue34dx#lEdqrgdmXh$! zs@=bl3gOYA7Aw>+Yv)lG>p{v;JYw5{G$h>v8d%HZ^$2qXa|BpIy6673TOxku#y~rb{TG)!^nU*nq-Ho0F?`Yd`=+bJX7F6ycb#D)hFyD4E z?Zs$HkLB&if`S4u5_-|^^F7!tE{;9$_vV8K55`Mdl3iV0@x@&#$poRK$^iz0Eib~K z1fC0zl0$QeFBlr+aW7fjYGy;NDLeD$le8O_-zTV@pu!Tbn{Vkg&Y_y82g$DxtkUjB6 z)}mFPNortI*#X$33&cv|jl=vTxe;3;aW)l0kvO42Y!CYZ*n(J?6{wb^9T0a=IdIw+EaBf;GUC zoZ9V)HggyWQo)I!>vrTIt8k;^Pu^~jT*kd(zX$fZJ#}h>-(jaZbkgEZ{@k_-Q?8z| zXbWEDN<%rr(?8ZMAni}bW?*531NG*7G=o=XQD_D+;%3D6LVr++O@ zNjM|i>LHlZibh?&m#}ki!ReP^fP=mv{i!t?5aX5$Ga-YMM0K&8>x)1}gq@}%b13TRqt&!i=tEG=uFvD4B837>;t#wK;x>~>z2yuCgJ={XJI#8P7@ z4mBk8LaxLc<$^TxgVEL?>8u_mYQCCy$+c1jm)&3 zVQZaS$0szTOPtYFAL|XV1#veg-k?o{Os?}rAkZR~EaoU6-j!DVWh zb)WRGr}t)+|Id+orINVjx;c_3MoN-x!}*tU$E7t&qoaA^X+=H(3n(01z%4r+N1SBG zC18x$dHgcsubEPUZP!y{=t^d&^xDGL_M9+ybpgwJxD68W;k|gKs>Suj3gFy0ylxSP zoole%c1D5~2$n@WEKGr0pgSz%6IVvYh~GFj_4T@>ObWkSO0KEwS>N==^7GchQ|NMv zY_zyd+CBV7`%}h% z%ByADcxfGp>YdTbr#nY#ZXYU)NHib4Bhz8e%|pPjHMh(?ZeN?>Ih~&-wHtK}APd+9@TEC$7 z)-g(uFr;B>H-kJ%3k%U5gh&bI69xKG5OXK@gk3>H5+arG47w}g78r}{SuOBt8o36S ziqF#t=zG5Q1X{7Pv(qPFiM1)0g{d2|?CC?h+3qXkC_uBqM(F$0P z9Q6A}>Ldo4=w!bGmJfKnZ(&V?WW`pAS>NA_atm(BbVF{tX0{>@4gow59$aja+t+7; zA`H}Go=CK7sD-?mJDyldcwvi7g=#4lcDva{T1Uif$Ocif9VIzw%?~~fVHSg;hE37ijKI{uHl+G z6JyT8&f^WM*_xwrRFNgF$#nWsee0$K87`B!EqvOd5y?U**!& znw^$8ijc&{y*2GPVXfX<(70{54pHXO>eMvaf#YK>%X#l>BQ`3LNb zF3pD!O+8-ukxhcEY;7AHV_l>e$k3oY;#{I)GyZt2K0xMLk(r6f)f2jPdioT=S)_}e zC!UVZ+aQcOG+x(oF8s&y+FB~B+^ux%8xH#-89jbez=<6TCp)srO7-HTV1KBr&ns=A z*NFR$UW5wHQ#Q0L+~9&ogwQnry}m#nksld8GUYvZ*4nzBmxWiEL*_GzChpF^H<@0% zbBsa>5P!+Tvjl6UA&&lLxu)W0pjr1n>%Hr3$+dMUbCYJwopQJz;ApnXeE04h#qJt{ zt8u;a9k0u|>S7bfsh4KI@(R*65cKIfJxMBS3S}KdaebnEsxQkLepF0jt(*Yw%0v;!zg)gv|Tvn>pCyAG4xN zJyWu-pVl?5=G`!{uG;>yn>LfwuVtbi8{{V zS%^sdcW%WO&ZNIbAG;)12w?V;8;H)$%NB(@7Tq`@(PdB-r*t>^L5Eb4 zoap%G=Rx$s{joJK=5)fLf4Km$#i&bN*^J%IbuE6|fx2lUxZDI&Na_KticVmJ542{g z7nS%J(@0hUnF4-;(=*rX*@nf;e$$hMro@z$$5OW5e@xf{9X7q>{o!H3GpmW-nOmFp z+m-b7^o%)1>|B3(3HkVhNlpnSfNe*F{lW`TG0!>YJMr;MkGHb^P={7bw;A~weF}7b zmYmmdF4p<^Zb2Jk`^=60zji&3##VKMgm5GIJY&XDMk9qv?NKnZ?RS+cGmf)GB#zVz zxPHMUiZUh99{9}b+Z6HjdHlabols^QU~de9fqLQ66WOQv4J~vL5fMS7@1~>)^PbLg z$U#pUTRVeRa-_0nhbNBfV)H_aK)vTJ$*S!4f-7uqKm!}@7d;n|xYincM%jJyTqxKj z#^;m^q+3blOE}ieQjfrYJ$~NoI!ZpZ199Utj6rKoPgr%keP&{bfHuiLTqA|W{N%}U zg%`DS&(`c`u=F?Vp^{`x7J38U_ok*0J6YI>5G|TLs#F|kN+z{bbDyisXGj*#eZgyr z>=(;I=fW^8{nYq={k^X6>J5!OiGId0pI@T@VCM+g&IulGV`Dx!i*4m-wFUs8rYA9s zN?~DYgMw6NR~N5eaBr&jUrIzYUq%44m4gdJhPWD6eG4pceB$mw-*5^tzc~)Edhks> zRUZJWKHO?c29uGQVw-iVuV6pk+X&SzOp zskfS}t#sX*)q@&xc0Q-t<;nL|>(7A^&&m|#yI+U}t+m+_w~-khzfe3}Zq=%YZS^V^NF^i2`_*v%cZD_+9Dp=6$Jzjo)wDue~MP?%fh*NQ*GfF|70R z*ipY_%a+>oF(TW(&eBhKB_JP8=}~a%dyaw8J9v;9N*V))R-#;3{7hOSV!fEPlUh-U zDt{p|4+a7x^y0D6LdFm$|c^NeCDTs*48KcQVw$8Z6rzn3Bm|>Omgi5R?w~ z&zxtp+Y1l7xsk1Jud!4>=i&3TExy5HWDmAtIxkZwdEiYgT-4kyT!0?~Gb3Z*ygE=S z4JmH&m9gS-aQ8#xf*D1C-3PDbNBMUmFxz{23j?)}@*Uq%F@_*$V4IEVm1CpcdLM}W zoVNh-HVj1~Jg>37Wx|p;%)6Clql%YdYijdW!3$qX1(oFl`I@aMur|t^*Ei_WJw*HF zj@cT)eGgAnT(F2Ya3Q z&m$wX)D)BubEQ(m@4f*D^Hu;bYw>B*u$%>SM$|VCXN=yX{G!G4GIV-|;ct!|Jvzkf zn(RBr|Mz|#QYyNLl#2FuF#YvNy~t5Dw*z>pz`MRaFP!3SOYj#H@lksZG<#0Nvuy;x zC?%TSfoQTAn?Oi%ScmF^eqn{8yo?%aSfT890A+1QdwY5crE1)5Ui1>hjS?)J1skH% zTi0=Z9_1{Z-Yx6B<|sc$o7Z2NLLyDTy&Tko7S^_%%t2Af$?OGYMrdtfW^sTc1ZtqV zkygDblQIY1s>27P#W;n8ng@1eZ|?t~eRBh=)=}}hGkIG>2=?$?M~G5GQbGcAt2VHp z%*Eq#sFrp74LE3MadL|Lg~}s~61s*Ky&+px!wyL30=iG4Kg7^S0l*l9rhK1D6( znxQy;Tlw!SHnmQr|l%&n6`a;B-4r1kn}7Tl-FQG~8kIg4y|`)yjHa^b_n1na)y`bUB6amC|j)jp*~NgrvrF=#Et^f_r}H&`?+lc$f2 zP4b*?V3VSE;G@S|JxYm*jb+d%4v9js9_Y3hx<9~%fGM=@?c49DMy1RchWiXRxUIHs z1Iwx!HwhLWirf4rZXOf5zXECr_UW)fI_hpn--z9M_;3?4%AqaVWQHXPCAb&ez4Op8 z2bkIr(%O2RVx; z|l48Se4 ze&oZbOyFEyQ8-cy9U~cjl8=E3{`J=+@*5~R&z++{`(Zhg#IYOwKAN- zEbyE*GCHeIK^XDjT=veaZGS~Z9)iYK`-WgI+At}lKR^Y5dF{@W{l~+75Qtmu(Z*b$ zh6D$3IEA$&C=|eVCuOfAd>n6xLRk)7W8LrPjg5}TU$AtRaruH3Pn2{oX%M}i@7FlP zip|4Nj0d%LGzlIfKxM}VdPN=lg&?fyzl3MA6e>{lPv+Z^qMmKw*m2s4natTr8<+ip^PMn~rrG{10P zv}sLZJzXFglAT{hHDQdteM-%>ct#wDpdFb2FSN50+iT(pqix@N>gMJwMMOfK3nY|Q zy&TqHd+m#hi@)|1eT2whMEp+jX2k)g3h!B9t!PxS2nkJ(cH!LgQ#gzb{H_XorsRHhDFbIYMB}Ft- z_0hu88voBtG>`AbZ@}UP78)Md=kOd}TwFvlyiQSHC+*1EG27N>IQ_83`t1KIEW~M! zs@LC`?c>TF1@u5LFUDF9r+}MHHlzPoN*eSLspx795r{xmtI<;YF2D4nDLl<0K?K(1>v1Jqs z=F12X$IrC%9=JptC}<{pX=WEu>H1>Nzjo~lp`d1j2?hy3mTUFo2@}kXehVUEVq<_7 z96{wG`cD)bs5hPYdLkBrdW6{HXeJ0o#FZ%wCfw^25^g<5@KT2d-+x^Id5T1=fCfRf zFoa(A@ZoC}FHfPC3GNXZF~r2gfN+7SPqRdrbdVEAvK#-~W&}n}$*R@(UDtYmrJATv z;C3nKOUROhlS*2Dr-Zi%Lmj9M%w?PsH-#A>h=x`ee!wa1Rhn11P#=axd{KoN$Me_C z&E|idGeOz-@GnTw3yx*Apk`}%Lxr#7IVQ(`&lhG|a5r)C*!LE15h{8NZ)n zlRht;k_Uh9r8A0)IO}GLyQbdXmlwkZMtlL-%^bGB<;mrh2q)hAzPurnTku}33GV-( za%pro!gv>$9;IdhBn6b>2f(2~#RVnCx!Gj=#xlrCKv2(SR0aoYATGy-%GV1g8qPwh zj=rkbLPNFS=!fNCJT}XgAxEa>t>hFYJWf`RGSE>at0mnwwb_;;M(u0z^_rbL(oKOG z3fP&K0gqt$j(^?JyiY~NJn`VYwfdPxCr-pvNcyGT0!Ayk#1w??!zq@{J!E%S?M`cdnAv>t8+kj(|2D!-sv#8$T2^tH2q-4G~%X;q;_OxXVy)(Z6AkL z4n!H)CBdb0%xk`5`{vDjtD+-6;1+1hxg`e zm(hhaZ+d%8Jf0N8$xW$da%d>wVVyu5Z0-y=#5z zyI1RftLyR`&S9T>AN$zHe)ar@$QsdT8u6(b2KLjB7Qur~W&_NWz$NRn z6Fi15yoGSOfd5IPa-e2LeX%HRBUTKG;{Hoj+5x6WTO?-xEUdRYyZl1)CUlU^JDC5f zM)}6lHLYnNRre(Q;Dc*`BNK-}OYfa>Uod?w4m1C_>A8NzZ~`ehP$ESKF6UW6fX!?( zp{%EsJ(RhgR?Y;YNSG9Yf;Al-96PcZHh{s0bFOYM*;m!I)!K*@r7U*^o?;q#>CzYm zkKm3IN6E2%32u-^@PduTE>y1hqK11hS)13XOX)LwiQjYkoRye`coW$A$dO(@1asE_s-d+%a(EH z)}rtwNe)A*NiIza@8q$Dvb`KDjVX8Kw_JoSGVf}&0< z?rS^H#hMga9Kj7q@yLG^TKvDLq9iE|QL>6iS=?U%t3nQOzpKk~E{C*{H@w7+=lCDC z7G9AVHQRqHMaYA2E52O?fMiMg&T1Djrg|g~jdCwkcQag!A>4Vq`SM8ekC^$JWaVC5 z|1B&sEj!e8;{h@ngIRcvf2&SGDFS@Rl_#eCkt|aW-f)G-=q?3-QV|n0&XKV9334Uc&JXP>rf)Y(|j$8i9Q-ci&SMZI3c7%+BEVm zzM?i1xt|jgLSqZ-r0r~|;(H6lOVMGt%M>Urh+c}uJ!BWSxe!_t(tWx{=a;Tp*C1W| zWk)Bp^#*}&9C>(eOL*L4h>1hl>Ku!zY6Z0XmspTFFRnGB5Dn8!n;;k}1(6P1be#1d zy#*=Iv{2^sg%@}BV{M-)1E)1*8-jR9KMS0Tm$Fq66Wis(3TEc#nK64DJqqhnKAIR) zkx&*>8GP*7e+8db&J|p} zgAvh~vfLxawS4^i$6Q@^F+iJB%=ghS+!X4)cA^JzPr{T{li>ZCJJ}1tqmm3@bj7^E zL4)ITYiGg-%`jj9jJj>`BB7yKiK1LvjaLpCoj|U>jEofZ-^dC_CY9p;e(TJ$EoTJa zHDC1P#&Pktz;~Gi71Yx##c!YmthFwD_Dm;CTZ<5nv+YnZAoaSbbhRc0dWj{&@u?P6 zxGeJnJ@)WDU2gzCG~ZWaRFaVJ1`bh_;SA&6IFM){B)~$tP}o(S%AY20r*{OfQx~aJ#mXJSye}FX#f62ljiuRSX=>pN+TDy~jO>9}ZSOr2z@J=BG(B;( zzFu86RL(qaZk95&*kXJTdgVI{c=-5kPfIS|;#H(Ncn%vko~ta!x2xvHn_6322`!6T zJ`bI%Y=(#;IkC4eT1t>H``5wpRBQ^|Ktz?SqtGHh)2?}a!D9T+T22lj3tNQks$jDp2N5{aiUo0kCo!bd=l) zhd}s%Xb8gw+#U3$5m4NI+CsWLvucwQO(a^jKOA0LVO-0k8|Hs<#ReFyGxN1dKLD$% zI300!4}wQL|CSi&^kt@0)M1n!2ekGBtO@O=6pkApXh+@jUt&y! z#^Xt5DUk!2k!j0_EEHOHxF;KF%;*@#Y#KnVhJJzCfod9H2hB4G{D&1wW_09iYc)`@ zaMF%dJI65X)`REZnUW6SBcRT#{=H(!OJOlSy_aSb=)GGMSt}?hbt+v%xoS)5k3fI` z+&d}TgFCQ{6cp~Yde^M?2M7YSaD>RYivB1vW2_?1>f_+=o{H+~#H~jSN3yEF2L{I5 zyW~8Br>L?r=@ zQV|q%fNfFv0d?W$P|aQ^Wy8tG2NG>63Qk}Mx6j@>l$?~btK|&5@ssk>(qpKr0+0pL zH_XYcF;6It^QxV|QB;eRb1!Jz_&@WfHGG_hI%eIt^Ezou2F``9odtR3%VrlmU)a^6 zjErhjFm>^(`0ivmhfGfUOK>+`Jdx>PR6qXi5(8{EbeE0;*2CDNXT+A&H{4)(4?qC7 z5Y`<56NU$Y4Z4DKoL*<(;W3K+g2;}CgTo(*D+CN}k6_jcEE!h=69+&hhYf}`3`?)u zl&FV{*<--YIKQBYh710_^LW8DQOxP)hRkf@{Ue=KYKtwaK);rLgRrnOCIak^R0X%U zBn%zavoO8+$*H8Drw@=7J^`1Z086=JaW$r(s+MIEC3fi9I8 zE1`*Iq2w@DzhvUSi7tcx7FtX!_0JcJ?ie-{(P_K?8AI-BwlkLKiL=vhAWT$^*WVcj z@G{A(Hf%(O1UYU=$rnh9`>oN2I2dru+=JQ#TMY^dvf|iDy0A_?Rxm$^nE`T1$MMkE z3U8~+o0*KVKNk9fpmA;ayo42f@9>njrRtFu1atmHTN^8N1L^?I;`A)=d0lKDNN`$^ z^Ic-V61eN~0-}KMZva?wuUxi*pZ_}evfE!rjlyR?g^a%1S{l7Yl(=7JIR{z&ue}kP zfZGb*+@NLQ=DoMO0kpd2ZbboT?)B5&etw6}CW@LkpfJy7U#A*Ms1(p)y>;xLwYp*L zEH-REeHFS+WjZer+Yca8C6CE|IrUX`)Us(1D%#AV{0{y>Ulafq*VN^{tQSS~*4Hx# zD813;baGy(nS!)8dZmU`6}p^5^CS=w@+2>!qo^#7<>=_u%}Z}VsG1Rc@NK8N;UWx| zo7v;o1x3q^B^G@3(0Z<;p`aB5a4cOrXY-_VFBJ>$W)c#|o>~Xw&y_or9OR=;Se0 z`hzfFgbY*ZM;1ewJ9;@dzv%;DnK1=1vKvC`o)TGs`#83D2Xy|bGavytG%x-YfG-jA z+ox)t@y*mwu8>_&?)%gOiN&@=8oc?hH# zhrr#CJH?%Eq}o@WB)yQGpLw8>=H+h>&Nz@xJ2-)?9Jkt{4N03qfk*G7U5Pag7avR_ zdnaIR&8T{BFBOh;X#D`J*ew}|c7E{>d06526j5rDw@?%5@sPF}n@v$I#Bt(=E9yod%U1jb`DtX5VH`;T*U2{Z>d@mb zVMz6UwHaPOhV+a4g9~*Xc9f@q>cIDg8h0pah24_Kh2+}s_Ehw}LywzcI~>a0SJpzI z{!JG|s*Ue}!3p~5j!9QZ+d0xQ6f+IhwS!M+DZq=bPJ-6?f9iW5guFBV8GPzx&^1;z z@6GY&&fR1PNYD`$kDnB09kkdLxAXFDF7I7yyg8nhUzt7LJ?8&OWAv=d?N1tsKZ8r= zf;WFF^f`3>?zD1q(nMPBP*6wjY{hA}qL!}1?>6$sKs0wpFLd34J|yL78iOC_OOdOw=>vlAg)0uiJ-;N5oE& zEA{v6u|-7@n7BIVVea`}ZxS@Ng_ht2A$stz($xaSQ5%Mu>T(=OmPOVkoj7(I!{QtX zQ4?J~QDOloGWpuI6L9}QbtL}W?eC~`mO4C;iayzEH(Q&>UxUYjmL#rt59}PU)A7(7 zIvP2BzUvrX5ZiiDbqXdkY9(|h-`zIx#}k85$5INsmF!wDg^^F(2bodzp@Y%yG_D)L z`71*ha1bTHHKbt@>d*#+e3N=#Z*oA21Zj3rw1IGqQ1ioN&Q_^l!V2t zcqMLD);8p&>h}HZr3;?^3=WaaHs#+@KEQDX#dEKF2PJKdIA^|wt}F2q;i*^`Wu}T* zsJ-xv^X`z^Aor$Cn<8iRf!yL0yt&s88fj!L?bSm$*zDvNA1KXw(ILVFAtzx}?Z{9D(qKmBs#~@cBd|e6rUIrB zu^S>`9l1G}Q^{@XHq|40{RO!$8E{Tf+lg=O-n9!IXGn_~Fy4>Gh9 zC0Yvrf0_mI0#^gW)2p?Lya&2*1{`-70k;{r3Hx8hZ2#b~DjvR71UO;tA$1P#n2H8S~`A}loBS|pSB8+ zIRX0p#^U?u2XQ-{#{(_b?ZtXtY~rclIw^QZl}B~6`0Z%Ut}aL#>`7gZP9HZU1%kgF zn16sI{Eh+k53zSE*ArytVb&zdo|}F^4bxy~1eY4vF1Y7$KHLOnk2ySX!v=`3YCjJg z?N$~fTd3XSh)H$Yw)c8=*tu8@(Jm%<4za9j+D5nU!HAT78iA2^t#X|NL zWg|mF57K*KKg`X5DyyKizW&O}EY!9wc^|9~e6c|dirAL7pTZ9qwx@uHJ`=mMy-_ln z0%WF%j{Ee8N%sD2(CWd_Mp(IbeVRW64Dq(zMv;`QvQCQpX_dfjiGrP*hV6MU`(cir zm!wtve)FQA+KU5nybZX6i?uQEuA>Ra7UWL5#QR^vHkN9hgD|%`vP!752pwsGCAmAg zKH#X^wDqr8~dJh?=hk_taB-CckZ%W0GD)(bUJFgta`N!hu0nj)>Boz<=Tllk8 zr^_-)3o!a<3RtNlamSDV{+OIDGxc2LdV>~fZ*6YqTtHK)z^J@7N4CKy4Y0c4T5gJb zTcwhY6HnS+V*{s>rvSUSs^6dorS)cR?$GlqdQR~@pMdFnSvgB|n?v+J0tQ+eK!{y~ zv1j)@baIN`1K*S~sR)@-B)DLcE=yTJWT5KzSBdwlMnSg|)QyaIg~-d}j?(W_1>=1b zQvRc#0D|3-wi{tvrPX^|u83R`%rnc45lxiIvHclh7=jgsFhT5{@(|D-c2;W8@i72C z=I5qw$fa4;+nl?#!qvQ@oyfD;{x3psV7*(h9}jGn)! z*c-d!PA#I%Ymj3{VzC#?B-v?y;XauWy6Z5RYZRg(8l8AIUK^Ux8)H#a-#EP=3x5th z?+|M$K@ZhW`znLBzg;}-DNO7Qo4S2C_Y&494}oR6;Yr|FgUi`gBj{~K5Da;u1UH4e#Up&7wwz$t*}_pn!AGY z@;yTXKWgLB8$4f;$K=q`2l6=9BCif1&C4bMU%c>T2Xfxs>u5}!0D%RVkP8mc65WmU zc2M#U%-;6%wV7afw-8R8wW?2_K6U5+VkVlrajH?`PHVFPTZtn_w#y$RR)3-S*jp)@ zr^>m5@6a5`{kpvgFBsiYXSU5!@0Pajbij@mbkx)gDJhh(t0gLrej2)i(fX+;%T)>Japx<&49xvkQ$xeMg zDuSywhf=q?CuiiA>sjBrg(cn`0)|@#N@M-iRceI zcE=aJMQoWj6;`FrKN2p=$dptlK`Qyy_job0JdOC+rCr*P4GHQu3*_o#)6Demd=CpJiP7G5a_YA` zPCI@9055Ncj@wVSR&;LXxQuk3q58ua%)IU#&a0*;(5@AkQVqP6#8YuLO8eCuGOgyK zd+zQ%R2QeY5iXsm_*KPYcEnJejY*SPhyJZ_0#JO{mCSaM9TMp2p2n8wQ*@jf4-XmFjL|KTH5g)2|DNykoOV<>k>tW`TF8W{^H3dZ}S&pEg}g&r%otbxpjg zmF8c5-QRReYeE&Le};LqPkL;uz^2>p_Xo!u`T=)OOxmrX&1Dbc@~qUt+(|gDB8FWs zQ|-<0f-a;*Q*g$=S|IE0{xmw&AvG|(JUuyCf7*`s!*>vl(^fr}C69g;tq@VM0#xID z8D)|w{?i-bz{%kaM}`kcHK+0=!r+r>&wh;Td}y{Q*fAl!Qm(Ev zil-tF{<@U^*tAhSyq897-I1E>qclwQqfECVTq8pUW7O5v(|T{eHq?kO*v%m`^=O^C z{M*~x{6P|aXgY%t8*K0<9k zdj_oa_xIfPB2E7lZc-Cz!N{b0R|@sI2OppL!^4Up%$aZgj^h;MPTo4#isJC_4AS^Il`IKQcsuvSa|psHA8-Mc+Xeupo- z`?}emTKaY`VELm_SEflPD64L`%gA*mGMgm7WgdG|z%g=@esj1lnrWM|wVEV^TD_1%_U-QGvA-av~ zKjXI_a0^c~bLpm9xW^m9kEd>`Ux3N#Ry{d?muQp&hEo9E;~~6!`ErS4DeaYpb)Asn z8O%688N?F!=MZ8c@vU3S0QF+su?MvQ7erR5O=3-?BkI2I3SevKv~vwi!K9coPv>5t_lha z3_CJ?#c{7)Ktk<6=V0F?r&a#7vwLbynsh~7-%;M*rjd?9Oti;682fxUGSz3{!=Hr? zE$7nk`1LCA%4Z+~@5;*UzK&w%MIFc(*Y1fUMWKww~Xyn10_++tj>Gl z<1?nyk{@HKovttLSI!J(Fs1rb@tOa@9iQ&LZ5()zB+AeeIZN9@Ar@xN(0R;l(k}PXAVcm!-Cvz ztb_DJKKEhTKyTomBG?)!t z!LcJ9Z^KnM7Zm@55=f)LQ?Xnii(!bN^-W3Dd+a_jeCGRelwWwH|JA!|V4>eYT2qyzd`(#nHcyZKvaDae;f> zF=@u-sm0xtKV!jv*oKNfamfQ0Iysyf9({eJ>;Ac6UH?Fd>J^fc*5PwrPsmDKV;oI^ z>nD?7kel z#YzO8f=y(TevQS^b24X7lM`oQ`gozVf`SDX4W9dVm)jlG5#9teN)Q9Y|0Fm+BN3Bk zXIiY-a)zg(M4pB>caeTUCY?pWhV-65*{_SX$Z4*p%v(d?qe)n4CZ z)y0FmdV2FE&dkAM`A9Fd_cl)$Pz}^gjznrpMXPny^5*ME@2aK*W(f6ls?%`kD(O{w z*PjMB_)xaBDnajOmIUXk!^y^@H90vsrt6mTUE_1uQsEg}k9IzWSMQahoIW8IT#?djP=5i?m^kCc`qAnoKFwd! z<4>`LHh)C{5Q07J+4FLhG}5Y55FWafd-;RvnpZYDCG#4F&)XR<|qO*46x1%YDIs$R*nk(wddvxzRu$M8c)Iz`1AM zXLx3DT8WS4Lat!M>&p6_{-&a5+*TY>IOw}MX4}xXZhaLIw9}-%{IaDp*Vp6yZ1bwx z%!5{^>kL0wMMOt$%ue;3ZV*Zwd@he2=kk=Yf3-3s%Ri<)hBs%$({-%IYP1t>pNo9Q z>8&;y*&MHHMM~`be(CWqx3u$!Zz`F;wdH5#9G_~??F$RE5fKrsc2w_Xm^QJl8%lo3 zO<$=a`(S8BrDGiv_inureiXWC;`f?vox2;8cC{()jsR$pLi-*Rulo_fDx}kkQ3J;2 z_FlLp*o;SYPguhwSA`n0rBoQtEIJPfjU~35N8V+V7!c6sj)nyK*J} zJaA~AGRtQV0M@-eBFs?qEK`ZlbRutHKJ%PdQwEOk0az>2^rKE$*5uC8H5(STUKRDL zp{Ry9evg$_nL+P(NEh@IP)_gL3YZyLD&ive$%q7&nel+7GWqVu!u5 z(|5WdYxCBdCc0?LivsJI%pnWuA)K)E~zcCp6&=r225b!|sIE!13=JFMc)>j|l?4BY5Z2gTk_x8|p_7 zC_6l?Lhm1G-3+sNN_e_2aImhv zzCG+SV4#ub&lZr<0ge`BW^OebYx!UmxbbP9wv)t>L`i#~sBirh89$V=nl@elgWgQo zxxQIB_t=p`zm=Jl1(f{;-4aXA^PnLm{Rk@9v-Y|+Y-~4~Sb1I6u^9p=4p|baMR_nH)(m zJ?_92$@T?$9=!nMZVKESTCSdU_-c(2oM`rw#bZ#>ES z3q@%sxl&3Z=z}~j%j76cNk5)5jW0O0K2`8$b~X-_441YWu+LDD%I!45oIJIgQJ(@n zZY)1@KJjd}pi`w8HY36IB5))(qSB zBP$vxs-BKmeXfC=bE4U86UmzConE7m8BKLfOwLP5!%vMTh19dw*Pza*SRbE$O+L zv3`_WQ8^J?xYvhi<;qtvhJf-eq3u;}H|9}rZ=D@bd6bewu?oZ8PxLZcXrT~kXPA^_Ug&oFThzO;K@xBmE>l{z}sheD%5F}{g%&bAM>vK`F6j# z%#{^NephIehbF&W{qsPP|Lqc1jajOkN;cBYKJS?iHXH}*LQ(yXsCk|-nqKi{b44+$ zeGjQ*UpIU|Nj=c-B*kRrP-Z3T(;^|}do@pSF=m?Ze}lpap=|q%+nvfo`=7!pCodnY zJ|)hD@6=ko-KL>qn+3WUloWNp~6Vf@U@Kqs%wFrt$~%^-{Hd$tRd zFWC6T&(w4cyZE+Wg7qH9(nMys0xzr)&MQluz=B~XxiV5W@jibS+1~0AC^Iq5kK<|F z>i&#k57Arf7~kouFpfJfCnt+H3)k%nTgiU(<@Oyr>VS1`PX68?bgA{R0*a~NH`L$m7QU>2Fky|t-rlBIg=Qd)X~KsJ zrfuut-5(G5(TBP^h(dKwX-y;J0uJ*|{xd6>nfKaV1$A%h#jVFGf3BM^jIcnh)E-YP z-4>T^LpJH}{GqTj!;;;jGHxu!X9(AtMhTukY=7s=CC9@ZbC;^|pXj4~pD6eIn`|BR z<%ZW~1xR6iOy0hLa^f^ zk?}$T6bW!WxUF1{z^rfN@<&sE1Mvgjuv3cESh7&Pgc(JG0RTM^rhv+wZ(C88C*o+3 zy!`rIuOY047`@CzsiJxI4=kg&g;2U|X{Wbqo#LKQ~<7J~O%O;x;HNen5F41{X@w6wm$| zXfrvrq^HSjg;d6PH6qtYl=ki1DPU9%-}Cprx>eGTp!0YRk>F+|h~b#X(<*>$IUb=G zqYlW>Zub{1csXG#()-JzEt}vMc9NVGRyqvB1$L(|OxobI=ImY~ntjC)T6o1H-@#M6 zhmxUJMLTF8A(c|rdjJsnN0H~868nk|0I{vI!h>;!I6;AwCS9*#TXkI7~Cw7^;+P72LQ|*ZVt|7ebSYm13o9=R7Qexo0u5McoRPFSi@QgO9SOUgFn~0AETIPA&r#Bz$`*S zPM&|D{fAE}sF`RJAi9XzalV;o73Vw{`)SWf&&_wXiS&wa_#`-2?CdZS{PAkHbX&G~ zg0@?_ghO4YnUT8i?OR{0ja|qCy#qs4o2h#%og)>r6G^m>rwAjHPz+T zRpK(V{@5d}c7^;~F$7yvVp@Zfn385G5Q!ZBCiNd>9R0x2N^J`cfrRXPO!xUUIG0|B zb8$k9Z?InYS~=azHjCtPp!@FU6bYj@I}I>;UB?j~apFHMiOqp5!%Qm9DAsEc)y^dBshRz6=ZUA z8;E9=Zj~K*fy6s9zzLU=R$5aW!l*K$9Kt2YzrHyAk?;~k--b*rp^pA^S9$Z1I~;lh@X;a9;*d-l1dGtt+r)n%HOT85TxYnGYyntt;@moaP*8MF zLP*FFxg7avQ}YaLQ$jqO(vR=~gUTiOhV&#TdxY{~)rd6CDRc-fHMGMna*lx@E;)C&a(ET3sGnAo!5Mf&e#Hl;b0kQefmJ$-Bmx8FlEk z!HTXC7UyV(nl@lqNHK#MXpN<>Kudcn}x@tPF4-B>^#&Fnzr%RlpD!P>1LXcYAUz~-PU+5mC) z^70fGG&Hi6&|2T#V5sCL6wl+<-QRg7F7BD!bx6q$W`85hYcrxHI@RnH$i4ulQrGTx zy7J4O6VOCd9s-Wy-kK3Q_Y@Z6B*fhZb@>PSfFB(N;jED3?4?r@2T|}Nuc)D>X7j+M z7kqQX2<#vDKo&vqXm)?<1aPCs-#8Y`Z=X+%iHc(54z$0w)4+hF28XXoF*0E^Jlb;@ z_oQBSJe2J5I`6Wmxz=SA&;ras8X2RQk7E^w!3bFYr8#@01eKKu*>ZA$>CF~+IEYCp zvhAU=b}+N3d}F!0UUWBn)jXx zp8*R|@8@MUa*ez=**1f>{{csL@K(4q1NVG9H@-W$+i++b!3ha`2OPg~vT-Ar9*_Ub zOTWCCB>-_(YR8@HBWxVa0qBRRGy&SCY>;t(%w@eVdoH=x;3QWdC;1-`c5nGF9Q_b< zul#1=`sO_`gdObA900P1Yrmks@y+Mvz9uQ@>0t5`ekW+B@cQc;EPetGM;SHm`t`7J z$*@OVZ;9MHFd45cBS!!u0c4E1c+s}h6 z-;d3OVBo`vPljZA}|28Rcv^dKCYji#(!#h4#b z;{CS_eq`o3^u++hz7MPtKYa^ROxU)7+NjST6*xFKpWxYnBGL(=xGDEJK%TtBhnZ}S>*`tsdIE|Q zcmxAuQxPVz z-29b>%(&?BvYtt!dW66~;S6*I-fz1a_Z&oqO|*g)Kk2)HIQwg%INBko^3OIwWxvCZ zMW_DIe*YuT7g3GTc$ShEO@WD5jZ+C;9n%wV=m?Xd@)QvH$iNu*ppBFHh1McZ-w$-A z&EL4zzeZrvpAH-}R{t;rn9iG5fykzI_nM<3Gd*iG%Xg!Ck8{ys+)93{4hqIA(031Y zf}|w*c?|-{XF-?VxoYHA-r?GRc7@4Y0*!5gb;ak@>J{}K3#Hyy9EVaL&wNg)`Kt75 zUDos8?NPQ@jXrF#l5uL`rU#-V_?E)Vz)*~iqqO05JDh)x6^1s;6@m7%dPvf8B|>7A zwJ|+QVolEZkeecwV*LqjlFHuAPmi2@+TAsC`g~LP76~>bU@<4$HPELHC&*Prc7)KT z)5=ru>2!5;w$$CYabtB=d;d|Nlh6S4SSe~h=e?C_QXoTE^LiGk(kP{qQ-!%4%TK?&jLqMcC;^Hvzl7>9ydmYmZzL%R~7tX=w1tC2x%Hw@t(S9 zl-M$?q2!l!llpD(IidKqHs&`D-n;foJ1)fz5kK~uNh}+^FFky;jlJ9w=6R4{8}eOa zHakqMmGetfuAx*!H(!^-uFaSRh2Sr=U4$iH1^>dU%^18T1C9yb+h6@(PwKL~Z)teI z!f2%#!a#am)D%^YH%fG|-`x4d8wnm~-}tto7+>_;rl^vaaYN2glo z3d97j*@>=X_)RWHP;kJ*_g_wr88`nQUq5OC#L7!vq-`ry`7CSYuj)xL$BYkpfJAc6?;yNnrBoHeWiX$D!K$MwVYlc?RS6jAfN=4nC=B4#g?V4Qzw10f5 z@C+ru)@rLs?clwf2l-hl5v;2v5Z~>cY0%!#uNp z63_oxPRsHuk%njXv82tO0KAE{!oZTyLIawZo>OE0x(1xc)-QxQa30$W%Uf-`4h?Mw z!@s>YH@(^p{D#^Q%*pEcEp%ib@c!d=lB+~82=sB&D*1JkKY!5I>CC=1e`z3NsE1%#zYuZxQZsZiVi)C=S7;oc{)$3EaOeB zYDZk)WftDErIlHTByHOJJ2#(41z!IH{{_gfAMkQ{ykdF%sG{w~^Cx-9^Vrh;>)xx# zy+d-GLLK*p9B5}C>j3Z{Qa7U@#6}Cq-U-E3Sy&}E;hAKN&n=;SieF&NDv>znct2{{ za?R>Aa_{!AzxV!?>p1PH_fN)_h9~fIbOsIWG)o9RCf0&X0SB1^nmy0y>TlCL$zi1< z{$A2+j4FKu^(|Y7uBg zV@%SlrX%mM;eY2nJgMTWL-&DB96rw@aDb6KKF@#i_;|;2al06KDt3h&@l*0kh?scq zNo|(f&csrDo;-2PfAhqcIPK>*m&NpK)KpOAsnCBm*(R0i;G?W z(*SzNDxL~D`jj1*;`NzMrb~!NpUfL$EBtY8D^JD5i`XUfg&XiJ=|W_6N6^qoVQCP0 zz)~cK&g>DwTDoW~n~yaI+rw0UdyUs=P`m;THPF(%U0g)0|I0;`cZeUMVi&;3uI+W|Yn zQF7m{qnSZ)0()*^_1J}wG0KOtPqP`7{1S^durcf-J3~$9zgaynBj{e`6b&V^`v1HH zP5o(Rro^qVS2gDi1P;)W_0IDDulGaYj(aaCJqFOPCO1Gs>qZW!A;i*)N6UX*<|ITk zIMUa-htB*ifosMW)5iJr#qQBC12Trjp&>tsL6;*2orP7idMVb7l%`opHGrRD*K;Ak zDpQcfCKN=t3p-enI>@GqU|2^CH>YS#XvjhQ!g-T->@}44x@VUN^cfO!_T^uO8|Qjp zc^2HR)jYM%hBPtiK0AyX@!XUA-&ilipWR$J;s528@K4UNi)6Xitq^A^PA7&k=f7X~ zRmhnj7)Akq&wno3kUJGU|998@KI|F8i2En%{I7s@cB?J4~(W zbo=v`D`wt)-R+zPO4NsK3mo*C)xyBw|`(duS8%`5sk(>`UJJ*BMY=gFj}^@ow~xM;9r`U$1tQ>u=^-|r8*IA_5f zJ)NxOx*Na#S;YArlagPPaik_DBNR0X)Th^5*8D<*5!pKfv!BwkwdRAw>5tD=Ol)&{ ze9?2Dfi-wr=&7=Kpk{1OI|#$=?nTYoY1_xAdydufUtz3Y%6WR*AqHe9>jt=nx4)B- z=nYrdZDyh5$GDa{E*ROo@RG7Z&L2pie*X@d6U-_g&g*h-JsU_IF1CNK+V_1Fqimv6W#tT@^9;94A*C zr>A9UCG)KLM2ZX=4effmJCDZo0hg&B|LWxPY_n=#=jOV|F(hf%_MalI?wy~UUsury zc*5gXHC+1lmxSoQtHYQ*+PkEFv@+;ya=1Zu*E_82+=c zePj;|mEuJfTI_~VW(?!(Q{L<|WPt8qpHJW8tGHj~iQZk+S3i)yS4iod)eD_8_FqQw>I9p>mp`OtV@VUz^oc%({%R$I~6IwbM-@Pf&f54!mtBhrWjD zS|M{$RKytNTiyYFJV623sKVi=Fk#b42&95WY1T3?F2D>dirZOdg7C?044r=h&p38* z_hBrp=?6=U)&8!h7(8Um$|-zOu8(=O@NQU+=e>@*e;8uY{6W?M4IOdGRm3~-NuM}g ze(&=QxOH_9-uLu%Rhrtf7i{rn64$x3H%71{~G3M`y!eJ)#NRg&$)K8lE=nS#)kH2}#bJCZr#Pk-v zy5h2&+@(Ab-}r1-H5vmNvPVtC*dPX=^3hpib<98@ejd5}#9dIDq#_C&7mDThSGZ(?Px`JQA|{LqZ;7FHI+=?@-keXu=L_@wSazhW886v_u4v&VG{B zorZRdSSaA~Jl$hX~} zWxl7l2QMP8O^=Z*--hpvG5g1_BzyT^LqYBRjbiM8*0jfv9S^*8U={t`8C==kSB@J? z<1&_pCDo#Iaf1|!Y=LSk=nXH^)!+YaeHeiu`#)(i{?FO5$O}QJg$*2?BOZYcr_)ke zdu*xE0`>5dXG1=a0h%?Vp;;V%mQeJIg3twzs^5#)u2KdJR!!iNzHc7?i!%e?7#;jJ zI_yau-4y-Hj<`Whstj+iWsCLjGanZfkQN3#7AvGl{*?5dm`c!_+S=M6tX=Txyt2<{ zoo@@!y%&{$;9j1Y?)0|~qpM!cLoZWuEdbS8TEY!-amM#cqJ64d+60SGc#%*tjx@Tp z6%;9<%2#Lu{M+)u(RZ>|SCf-(CFttueJJk!YW3}h^KU@INV|Evyr}LGjzG6)b43X3>8b`{e#2Ynp*6-kd zk{7HFp|u`_GL`cnu5+&heU%#Uf@qU_uMs(whDK!C-JDUXm4G24DjEe~2i+{}zI*tR z-e0}7W-l75s!FxR~rt7-QNr9&?`i^f|sx0jIrSigh#5V8?oRU8j%n_;tX;Txi4Zsmjq}}#wH-k)7FUY8j`p# zGBvH@r1(n$f(-XcP)yV)+x(z%RWTGo8XA$kab2=7-Xu+C1~qL(-f;O!#=!tt9LpU$ z&KHvLvgG365NU#u$zNQeXt{0)wv9~z8E47eyLY#fu}tt+gIagC1;9;XrUqr``CYOk zb7e(lt>`FH(zr=4X0vDx-V^w@wX>;dTwymt1xw@!i{X{P z%VW{Di-V*hAOSf4M#4oNWm8-Mim!PVSockh;I_CpW1h*%h7f&`H>go1;~?KzzXM~C zY#)#{IU2#XT%?n&4NcjKIK-iNptJVzZ|r~bl`hE1D&3sB;~jPqjVyHqwt|b55Q0H$ z7e}G0lmv3)`3W+0uiD$4oq^)tYa}eM5|D4*oTDj73qEfIb;LD+D2zzrzmSh*x%S;r>;c5ZJ()g@WL$O^ZKClt&fT5@wHnT|EZ$c$PKqU)aXAd=*z zB`A}{+6q95+OWVJV< zS`Mq=5lCm&&Zv)aB1-`yVYYe-9K=kE(@27QS0mlx;v6kAs9}rWxd)zER|Bn?nb|$r zx@+G1i!hef&PZeqgrrWOW}?XS0RuXZ-?w~wa~GTN0Wjz+^KIl9d6bML0>2gAj}Jk0 z_ZxBcZ|DHKrHJ}wue}AG28s@?2QhNagD9#hEkW4}HhBOzZ>5dMH#anr6;cQrj%92I z4HVpBn=E>w77?vY{c1>pxF!IGxtxI1@Y_bnXl=l#!Ow3MPpG(o4D#DXzJLK%;RE1W zVGcHZMgehY3Cy`T?T3@!HW!%&tOMtRuSgO(X8mfYDYztnI&^i>&RThT7Gsek^Jr5* zMR;3iU|{gNk<+n}Ee6Bjk^ph>_$79aPMAnIMvF0T?7DyKjs;YVESX62z@liGoX!rU znOzbHV`+(ZGa0gvTR{ZNx4^HPuei2(|5m(>>zh0F+lmSy#G_xGmnT}}4bEDLH_W!R z73S0#6NC)8^&kwb^B@d=X$c{rVL$N)Qf2V#cX`;@vJzn1ELQx2yugtJtn$PJVt*RN z&GQ5>7+}!87Ec)31$KXHqkf`P5q%*P2e3qgps3YOwDEK33oQz+;$I|D8g$muGDDMA z_yE?4lQ2(3P5t4m{hj?tczIIgQ=d>sn(fP}5JwR8u7t`gs|SZ?XbHF}JTYm_%awqr zt8j5TdQu@y@}n{YDe(GT!Hq9On`dxNWqPfXahB%&qnQ6< zumXqt7r2&i4Lv)u-yn8}TC3r2D~uKZ*>JEMF?y-#v8EGXI4c zNExpipRy#!#6JWvT7P?kZ>zm{y3j1Kp5_t zgBP5kZDv_^{F~*2F{L4L=8Grfb3uQkwGs01h4d(`tFX*hJ5uiZCC)O(;Ba5+SoZ_7 z0`nYt&@*)<%sg6rn!!vjvJf{LL!H{K1hG3WJe#3}fxB*P#Pi&f9elEB=oGS+cr{;g z@>wU1GcgU#0cNo#2h!kZMbK=S+$EtgWhsPzIvV2x zxl$*1(5P@{vYRFPd}a!lh2XB!oG1gGL(HLl&zJOiHQm%LPCJL3|6+OoiD zj~O*zPuiSE41HCrwme|>^73kQqFv5$0Pj*xBNSJLpURzPTdsIp?z;za0sF3Jz<_>* z(^#fI4InXP7Jcz}>Lov(TZ|+suu6f)|8~^Al!;!AL=NrGO!yE~=#m(W?!p!l0@2V} z0!FwWx5)wVMm6OV54_y(CJ5Oyy1eqXl5on&odj?d!EGum-{lcQ8nNjG;Mdk6KD^Jc zu?FA3pMASNCvNObjobx=<09)NhC$<|=dq#6T;T0ZYI zM=APTBr`XFv%Z(~gl?j0aG@Uun@w3bGzmsuzSQa=Aq`@9fHPBtU3XnL7|PSCLE9!w zZL z8$W;UTq^4EKm5w5#2DG_?t}BXEL^&7A0#0km2Joa1omGBdwR_^$Phf)-HX*)abG-w ziA4ozd=kag&W0gQ6;t{LTRrG2)0%(;4k0eGqAb55}k@?NCS~3hFa`fPwd_3`M z(ZN1)OKw9Z(1yjamB5nTzl)8cS-ow=YtXEp2p`kbtcKv&+?CbV*nfS4^f-AxANb`E zO>%BcDIo+a!bSgmN708Oj#4QfD1sJ_h%;=2gb5wGej>rpTerF3;S{WF&YUs2#; z1>loZjff8p=7{b1)VGe7>-`xccpp2%3`77^=fS4+#=+Cn^zG9B$oJ3{|q96~;<`2}XM}uUY=Co+h$-#%-gQoe2qNAmurY!YDd&RyW@o2^&d6%SI2^d(~RejA+8n zmSbIG_0Q7b#?pD~sk0U+J75zKGiT!N;}AuzAA_Kwm4U-({CfgU0=RC!bzYk6TAYv2 zLYP!_w`pnX<#c?)#*jOhiSq|@nr|~-*dlgQKz~)zN|4zedL^C;YY0rcxS;;SKJ^XJ z|DGN*N_+&_r|&-8E*cn7X4aihFb4hm;sHb?lbMZKxBgCq)io^Db)z=&j412mq)Q7W0c?QM1?f#xdhb0EkkARehe&UMK!899gtKt( z@7(iz+w=E6_nznc12ik^U2Cp6#~gF4`Q8RYC9H#UqismEA1Ucsw|^UyywVpy`~f*Xh|61*ao_6ydA9%O7_P9s0Uk2f1Vu$& z41N2XeA7A%fDjNHmStCd`u&^-UjfKhdr+Y|dPsryCZH!&KW?o%|J6YY;>9bF>46(f zoVl`p$H2>B+y|kj#eQ@7!5(-ikMAm&{j2@sHI$WVq}+$G!4aU+h9~ zw_K0kp8ym4kJ=@U>kKQafc8@3K3L8%ro+Ou8{sV`g$kBF#1>wHN`A^?U=DXK3kh{5 z`(G|$2Yze`m@mfDYjOR?)uyEUu7*aH=tdtfBFD8eXU_b%1d3&1K_{Cy&YgR<(sFhC zpfUkHdUBd#ndpPQ#8w@9_LK*B@R~4i*qLAfD4maN@bp7I=){^g`{2=KA6ym}1a#o^ z89_GqC;xSDulb)LzE&f_l~AYcyRd^p0ybeKac^?4^euENM(*C7%U=MuE2Xdvq;>5n zDVKj!S@E36Gl#&}L%<5bEm7O9xj>L}-dF7(S=2jWwrbn2VaE=If(MdPKwJUz>T9=y zf=#FQ+LAu#E6|<n{vMN@rFN>el1K z!rn##w_uI;P0#On*F)$J+YQWYHmaU16 z<>cW*2Q>)t1l?J%Z8Z_#XE#!y$>7NlBatUVNB>z=upezaC;uZv`@+9}Z0GAUoyjJtpmhij^N1^xzrW5$1rRKJDcleQnPBI%;UY zH-nGp&G|@J*4=w@5!*1A%U9p4_CfEr)E$l^92Q6 zx-gs{%bw+JSNNJ59p7BvB7BUi*|^KhRMe)|6T!od6EUL%$j3)h5iX;@3_hNp{`PPb zc)owzFgXFrDQcs*PQpd|AU($q`_|VNi=Y9GFYorFMz^;H#7b0kRtYldxjN z`wv3nCj^AVE$nuCwW#0BuMl)mQ3%XJ{Fx^#8Hn(jv zV#?_Bq&_-jPd}inFUOwLQf71Ui=hP@d@=T;5z6Wkqi5)oXn*-fgBkjsgevN@jD!kr zr`<^_qbSvV7HOs_4>?%HZlv(>ew|W4Wd3fwk1@QMsN$M*-QZ3P zXU_#e;(Uy`0UL#zDP+VMFZ8|&rH2+AJr~IJ~BV|Hy`#?uWpQ)4nV$gx*bcnsY_2Z>HoVcxSz7xsTRQwoNqWoQol7S#y zI-=jP5rzMK^xyyf22iDNmXd=na9%y)9%mP zdpZKzr!59v;oCxuQvKfh`Iawto9I2%c+66c7!7m~zntXY(>D@j43?*^aP);p1?c;? z`)jE@L~a&SHd%fGv@3Kb$&n7pDSZF5+%cFWR|?O_twSQr+J6{x&VJaP!wR5vku}nG zI`nw3&9_LPqZm;39E9orrC2=1{cA6z){B6f?^!H-s-ZTD7<$95@4c(v7NXE2Fs#TS zT={%on%2{j;ZwJkc9OGX=UzWUkkRdj5UOaOqxm>>-sdjcxpP@L52Y;KzTOVD#D_4C zK!7UE`!dO(iQ2!v7{CddOVEYX>VEP+TD$*@FA*k*9?0uW+N6W~`W>Tl#BuJcP!zu! zCkDI%x7cQzM1BJaeh|-%``I(v6^_<`rb4xIE@dAcI+i}mn1>$2sXO#%`Zw2KvLW@m zk#@std^+hbDDC;gS3%_6Xq4}IfB!N>*hY_5| zs9&}}3pMnsw+8v>su@Db9_=M{z0sYcaP8BPR-TJ#_q@;;Hd(Jsn9S|57FH3PDRX&} zSuEqU@*!Y(Pi_fFlQzC9Hc3X8QS~FQAaB>=ba&q=cvogqV=R zYoj-k2DrWL_Jf=wF3c@F7u#(AS-K8ykMCP(9;iEQHgfMH+|S;4XMJF%JNn!M8=5kR z#>x(~5uc~|pfp}I3tJScz_rpXyO(B|*Abvl?Qm=Fh{#g@=?fjkU&$2V123d4Zc;sv zFA(>Z{OyzssZ9x;6FUecy?00pwnDj}w_-_gPrHIOeHuF!$}X3!FKnB(UMck8Gu;>= z@Bd8R)bR(?Nf=eQD^UE3*P`*?9t`YQMENOj_1SLZ+&g<~@|uk~zc%BbS*6pQp3D5( zghFsgrJZJ5KZy+vnHx;0k8eX^hBMKN;9Z?P5^PZXn&=EQ0j1u_-9uudG=$10$%!hF zn%Te#e|DZh+$_JYa;Z64NyCd6`^f1IXw1C7?u6gja}tuSUHwI7WUmTnnR7zZIw{uf ztk9zAQEbJ-I{P(naP8MHr=kXD>^Tr}zBPr2(ZPHY({V4uUsZqjGUla4NJ8ajgL5JdAFW&>8r&vAKFRx*WQ-pS&?({2vRwW z%x8s?(O0hA!DwBl8W2K-ILTY*S5zz!oRI=o`E~9;|Ah^OOd~E=53O4j>pz+f4)n>Q zJfbnb=z`Xz`a5~8d(;N{EAe>(RVlh6U$Ml3(EqC?LZ^L2zv8+5lWf1u6(N*ViHS-# zBU}JuBrBU`Ip-G0R88qnSyK$<$3(`+PO8AA=k(*#_eqy4 z;G##y7K0iXPo;Z(x@8ZJxO^B!NjnP)YzvsOCzV(4=Q)fbGv}jK_o zG-6<-bn~mzxb4|bwGYrE7A`gE>p#w3_*ts6mwcTfuK4&IQSH65%@(7Ow&B_9@q|)) zO9O+&EyuRziS$cM)_eV*;=yPnt>`s6majab^8 zBRO2Uo{xMGf?eAhitgCGBkM6RPz+ooUlY1@6IAUnoNRi9HJ-g{Ez}hQE!*_`>e(Sp zjOuspNjuJX0oPu85Ww=Dg-ZJ1w?ms4_fr#t$Exl$N==G=s8w|H5U}6>&N+w{P4dv=qdI5*qXig!Qib0x z!`)0b93V~&HY*CKq{(S7@|`6u&DUK?F&)0ZO`fCQ z_Mo%WTQ2ot!Zlork5jCJIsMho?01I~=}5`nG7n7*Uz2}HSzh(9v9N`(RD17ZplMp( z3ntM$v;HqrKPYhsT)FDT$#IRND>74;-0v2r?NdtaQ}chnoxHuj|5A=i9KRe&9kkq4 zt(%MJlv}^(JuQ?R)^(?sQDCjs?ELSby%e&yC4b}RN0u{Zy1w5jB}YKmN2!$9+6F1X z#s1BBKZcnWz7Y8*HNTd7_qX$U*BV|Zm9dhS(qAY*ZC|;p4}Dq&e;6&8#|eYldRM^F z$q7=O8j5Fm-+8VT2~MI$@)v(|mt+-LsF*)g<$#<%=fZQj&)ui1nW2>JlG_!8SI&aVAOd>t9#I z#_~3ogvUJgQXapdWkqStymQMA-QCm-*tZ*%;-k_Sx=&l zk+nH%RP_W^A9Js9RFfS~{Z8l)gtt1bwEXJ~ha6ydIcmcF=xf9Xt#`2Oo$O6T5(w$h z#TSH)wDAll=-c)ldyksWDeSHAUO9g9bW-tBpPWZ>O}vhtN65Y z+BGi2($(i@z5#qtGqAl9-dM@mn$Px?8+X-7_NMKM^~`+zySbTTLJHn$XK#%o^3Vc5 z%RrieQjIB|{OQ`HXB#>|Av9h603aJct5-rEOgCqGh2yq+g?%p@+@CJFal&c$YADW2 zsoXX-3!Ivlyco@0;F)~dLn4Zs5^Dm*t7be_T^1;?%NvG=CQ5m=e>eVmnOnZtds>bq~)+;na^kc{a^m3jdZNfIVt z;fIpTbMBdX5T~u6BY``g;YH0yq>E}dcbs)XLnCNpJ&NR-ykE?P(SBPIf)M9kYkW3b zVj`V9Q3)5@7)*}q?gmaRth}#tk_sH!*GDu&MLy>#BN#qHX^b6l32K^hr50&pTYXx# zzsvZ=UcK?x7bE%Ab_QqFFzCi<7 zRuEo)j-YZ9=`QyfN<&b3;{7?&F#^?GVz?6K^3705v*0tW|v$1Le zs+{n_LNPRl&}YiXiT*suD?F!EEo32$q%t#IbbSvs>*jDlUMJ<&a>A|Mzz_I+1X^Le&;L%ysQ&OBSB)3AcfF0W)z-* zN^glc6CrLIYd`oqi#S6yRuYC|LAPAm*%dgt>5AP7yet%D6--C65)8VXXZ{2PZ+!%(m_2Rq$i4d7SaH38GGp z(BWko{Aw;xx*+$87(id-O4%-848l5e^D}mhUuh(@IU7p4A7NC7BwqtQ8{QIXPh>RrRm>M*7k|cVv z!updiE~&K%rZ(S`H_&`Y{>`DpjPQ%bW5yQ5)OlM0Iraj({xisH%vN^^0fSAxD33Fu z1#Z`>@EC@?+}yn9NI`akL0y^jre!f_vK-Nz4o#&#(H#-@%fG;+y41ck*`^bU`MyUi z@6}-M^;?X_EVa0q&bTo&JgsC!_DBrBmVkTP@KGdxcJwb0;5<bq8N7Gkv7aOkT!eY8oO+hSw6azdT|4yQ`hwa`fJOLLmpye2@D@xEP9Q$=<%PjuCrm z2^yJe+;n%>9IjD#%cCYh_a&)RvTi6+18FTql94!7zWZHm3T>iTE~w5Jw3kfbjMf@u zlj*vfaMGM=aO1^eXX`T4#(Z%Gysp^iMeh)^9wwWnilQfCxLBW-#t*0;W zf}N%3#b`IJ5)?Ens>G5{Ctq}tiPzVvB}*K>euxg;iDP;-5r*(k2>238mo9v5JDtNF zT=!YL!^z}5@%XP9tD0BroG3%ShQ(aoE%oW8$Y|3)odGz+q#>($LDWM1y=M2O<*)>4 zSMgDSb7Gm-u~<5lP>;4eBi@lA{|H_VzQ z<9e`$2%E)kD>eFsgsl`Y#=WjojfJez`9g#YUwfEz_FC~gV^FRBJAxiAYm>#`;qyDw zcSahsu0>~Z=bjz{OP@7OH)G@7CKHwYWRNZ4b=QrJ+~9~n0iR?V&P9PJFm+zschK{^ zQgm}=H&*rQqSIfDk;P*&o-b@U9~4CYbj2Qe0z#Nf3mH^v|@o9!mk(%ttp zZdV#uVt3i;d(q4TQpyn{k)qRdCAYui=Gr&Kr9iIv#Zq)~3MM+@ChmBVkGHnngw6E+ zE^mBL{tGQMTk%px;8SqzTt*AtwxxTR`>cMydGoCStel!Y5?=QrRaeeOE#Bb_`*{uY z_!o0pZ_T^G*v1>&qtU)Itt~@j?%ss~uK{e_zV~=ze?5Z{9%9deO{C%k=3jWQFa{+e z9k=r~F$g9->5cMrmzTTF)DoHJgIBzZgz(Um$<)kmku-EtnD{ZpbI~k?)$|CHQ>3yr3N|Ekw_3xQTy;-UhZxh>c&dyh%FjQQSOI z9J-zkn=I7TOcKld*7Epc-ups<W zrukCq3->)t3As!+s(lPUX7&E)MyS%ESB$D;+j14Ryuq@1G{iz&j_-OBehyC;Id_Y! z?dqJkQ0G)P%}OvQbFC@;_S(#ss^x`te{$?=%SX<)m{FyMMSN==%6E1rU+q-9qC+Fy z8XhiEH+Ejo$q%Hf$PI-197m|cm|A@-u^MaNl(53Dr(GIn`y_IY-Z zxtL2?J>>MbudF~+he>vp+w!7foCLp8zORNTH2oQaE%Uvpa9+y?-oa9IXb$mOI>^@z zE6u0ZBYbyomsZG&eQ7)rF(Opnfm+*^P?e2tZ7;*^3oCcrI1+t}Y^6Qr^(R~ZXsUAf zUox0@&-nE-P(sBVvORQ36Rq zqB^;QL-j_u-84C=>xvV7An)!>w0 zr|$ho#2XH~48!s(40U%swmMMT$FF|=?G~m@@v5=oXg`h-N&D6qNwvjjM`wu|i?@61 z{W(u#e3av9pbBL+ht=XmEa&vXM5@rn`Pz=w56Yb_+rp#x&1oXgNa?;;klH}!lX~zr zPDq(#&P`$Er{J73IBba-vtY$+WMGe#9q>C9tG{Bg#~0SH=jUGm+Z^&49AeiT(u7CQ zRG~Xj77kH9@Uo`G$Qo7BSIaYOnnUr`oW|Nd>wUHnDd&t%Ky~E3ewS)vjy}h~2nkFp zJ~uq{4LaFt-rlZmC@B2H!q!A(E8)9y&e`H~4gElPGLFagvayTiPt8(HXQs zdrr2PRifZIemBn2@++?<)6`+PrcTsOV33?jnmF1>H&_|;o^fYMwcab5E@WcrIzMGJ z(l1)Br2k|;`x0f{z)#BUVR(s)I>V!$w}phDOAo|Vkyk?Bb~)qUQ6$qIog-#lJo$v% zL|1Zf)k!|F6TMA$8aWd!&;Xpz?=tmeon~FY-41bSD68B`SjQBWud~;0sbv&arz^Mh#9S;43~c~L=#MeJyGgGPd(N(+ zm8l$=oLxZOu-i|y;T1H7n*aFZ9NjxSmsaeS8BGHvY-djl7&-^m7GI8t%D4Qqyv)8G zJ=D$Z;tcJpNSsLPeKC{$SUyYO$ANon=^u)Hx#uAf6Q!JQP<&|^eBOtZwbgi@TRFV4 zaa=Fq(agyL6!HpE?K+Xve@`z}$t_O0~afiToG{?&e1AUenQOQswo#wd;Ov z;@)|4L;J|`$yFFikGh;|awXKhEm=J4B6}bWG*lWAI;+|yE^4t&kk4T@;v*&~v+HdI zn6GPcoBSetFJMb1Yn>=L{1~bcIq*ts?sH`P_APK=y$m1U>fe{KIt_t3JE~X_eW&|GQ+ye*l)`2` zr|GirIgPzjylZ7J*lz6>h=Ty++6t=+L7mkRo9DnedEw+!dhaw!Mi$}f9P*K+j@Ef_6Lz0+BD+!ke~ zOh>Y2diM@jlTP#LCe5y#3Jv+lDK1fPCy)eX@>4Y4Mrn=lg*ck-WZ4Pka49QnvRaSb z)-GP#y5u=*&|MU{ooj|im!}b`)&$KL#ch>#YmXqdG9dY~>(iEY-W)draQCy1kT}GA z^V5k@3M=Wa#AW9N;CJ7Q+;95H5X!wl&m5w3-sC#_xWy z*_xO*s#;FU;jE)RAWpYGJd9?oUgOr*FBfz>%|KmnTg(F>bnsne)h&5c>1WAheVgrb z9sP};fPZyHtGDH@sSK}qS*0)ZrkSZ!_|k5gE2Z}iH){_<*^{xl;S$9Y>fqSiWg<-+ z;hFTp6FB^4+U>U$K3p|Uiej3BQWnPsR`b-9!}HkU@01fa6(ygM>UxHou>PcvPQ!!-`~v$c(s^cBvwO<@W$@Xlp9fEhad=<)Jw(^3)$ zM${S>+n#T6Qj#^W6sv}!zF%7HPOPk~q`6`^_EnapZ$;akk{a@=NnC5$;#57SiQZ?3 zKkd@^Hz+%@ZpHN&c-4u<&?t7paROs6m=Zn8Y_h$#ZvBcX>*uPxXWJAQ5(zRB0Auyrp^Qe1GP=L_sa*1jPPScHY&`ktR$e?gg?4aE~9x6vHz z=Q$lGWo{^+YF}=9lC<75Lry{h%8EGkCG(BgHCUd!C-=rL&f>vkW*M;x&xSM@ z%#T%=TiJRfg*Mq3_!HEc*!73i$am3rX%Vp(ye;b>*v)tKn77Mbe_l7czNPmbn3cHo zXzb^Euxt0g2PU4*=uWG@LpQ@7ab6St>1noU)?qj{R?Vp*@h;%q|(tzvf6lO#Zhp-^!jTZMDZ@$hww*?IQ%Yso)x>`$Flwh0+ z>Hc25m7l8&aklk7t&QRQVC|v6%gRsN(jregY*FUCKLEo8Ic5CaW7^jvLhW2#9cvI5 z1_5f2Jdodgxd?KT+@O+x5Y3ARMv2;UGJF@sZo7O6!`WjM&~lf0m3v#qx8+oD6m$72 zjax?0=oYiP!E%X8JW(mOwVv|aJ^Iu<_kqjPKPpv|q_?4}6Z2*cgdc;SGj!F;N;o*j z3_bQ1MlT{WF-?V!VJ=Q9@?$t%PpQP(HXoJxtJ^zj>4|v&3~Ey>4HAv8X77do1!b*4 zqtD(|A3?`yeA;a_w0+olEtYkiPdl$@GxXHs!^)bkL2AXXWp@xWj~JlCDw(Md5}ukp zoKxq&t{kpzDF!w~)Ra~FhyQ*@G#8&MKNGT}%Nw^&kH_1ZI|Qw_$F0<-{DlmZ7m~I9 zX4>%(Qhn3}YMD~ymeE1b(S|v@?%Z$^MD>E9yWY)&a}AFyKD~w5{rc)>zC8HG$Tzju zLlP5n)h!#6u01R%MEP7`yjgZbu0^A+tAa9qI$qZjlcXAsp_CA`r!Aw_y z&O_|{R@#Z2_%YPT#-wJM$_TYQR_e4r?3XKJ(VP&epp;VdEy6k0y0nSLm$jN(_&GPf zHq^)rqrPs}JXC#(xoMDhXlNe}!N$;}{7xlT%A0?x9s0!d!eEQ#w!_D7?;^SQA}mAX zp+|BIcQ#s zJbQBnE3x~re< z)lnNSUlQea=heRH#$PyXVZvoU!2)S>9v4|ec<)8%sxY>iN?!rs45b)yW753B z#Lv^5_^<=@$gcg%ta=D1D(BftydZ{m@NIgK)s#_lguQTjZSmVc=YTZ&wiSH=pOx2s zQ^UdBrP)jq@HKR&{Z(E0d|qtdkMj1gPOth^MxNwx6Y*~XHVayp1(w?kO{ZE9aR(r3 zECXN@)AOsEf!65qrTD;zt^;oLzde6dP~x<;$42a0T{f+HEY9_V2_SZr+Vv#i*rde4 zq{m0-eC?t+?BsoX*2xsck$9V8^6uVdx0sDoQA4826oM^DPaeCb2v%{AHHt+dKfCIc^Z&kO2IVK(pCc6^HS_Gv0 zsIc+40!SW}`ApT!V2d;0)M{QWU5W6kSs#f2gx9;YV=!jXJVJ#Pzg;TARLSJ+Mn$%f)|iqJ8-<;^^n-cWY-l{ z&yXeaUqlr{MHY5*W++OgN^T)0lI2aGMeNcjr3S?n$(0)!do8u+nbemi6~4~6pkVNU z(C1R_>l80u zia*f5eSNzKpoxEF@ox)#9vr%N{mjENAYCQ4FrkwK?at70%OCvqU#565NbXL6FZ%K- zGjJ^+hr9_}>p#uRqXOc=znH*F6w3=Y-yeQ+^z4D>MVR7SdX70ho;-i>+ka79bwlUB z>%KhopZU^3nM{_$O#k1R?4{6Q%73%~e^Z$MKbiRd>66XX$*Cj6Ml+7 zFD?(bT}U_g)!)>m_STmZ^|bORMu1m7$tZ9Um#`G=WCXV_(zXT>?HjT zOfMp}kf5jV)vVi)J8*WfD{BW37|Bn@%knw&)9xpUauc@^IxAv+ir49mp0;Dv$r|$S zwq5gT1TjVTWoDk*pKQB&GjFIyQ+NTF47h|R`C3XsZp@-9>E8DYU<3!E2?Te=3wdN@|rxnSqXy7ubki5vb@m^{dL!?{vb6D?+_#V3~Y zN=d)@ETR}G&_6v@eden2)7hX*!mfO@pgjm$)NA@hKKBsur|{U9fZ=c0@gV@W^!-UL zBWAIT&+J!lL~&+qOMUC@2CgV69i9k!c^&iiVGrOa@nmN#XZ*7aquPfcaPvs6Vx`bA*4EUT|0V;}% z63|VUwnK-HYgyTNY8yk0tixci<%KJJbbw>A@CZ45YGR9083ZU%?;heNi2EQtZ6}mY zEhg_8KL&TQe(zCiVH3{iNuNM9m$FP^$BJu%haOi{{q%_lGq1go~92xNg z6itrl2*%@_vY2}+0i2%9k5{=~IBn#* z0@73evS3Jw-Pl5gGTPwcfNU+Y!Z>$Q@DLSs`ZO$0CFqxaQx_Drue`Q~N+8s&p>c%i?Dz%_my2Pun+j z;SaKx&M-m-`pS1 zzO5Z+3ShDiPD+n)y3Gsm%JfbhLhmJEg)fof=1>5~uBHz%6(^!`Mr$DD%7l)%GZDXA zfTss_B$T1{UBx7sGG z=X5iM-yKW_splNz{=hHraIv^R_O|BAHq<;4v6oiN?h&22s_6E8@z*~*G3liFsq!B= z`qMUoPXOr@jmumEE+OJU=?fpva4y%ET8WZg-Sq&2$7!8%^HK@lfJU4Q6{3k9odSRJ z(Da7ADu8;(KvHkHJrAfvw`5Y2DOWt4EX`{Yn6Y(L@DT3`?oZb^9*K~E+sos;X9{gKlkO@GwF`! z{JFj*teHDtdbh{lb|!_piusSO;Av`xQ#a`v{-^+OV%2#Hsh<3R76eeARyC>V#iC0U z@ZnFdBhQp9znX#>6-_tWIS?)@hXd@s=&~TtZ*|KBeYK5*Bu&#NhscRpv~@=LsKoNS z7811{v>Ug=J_7=FbaQoBWgEO$cQDnu7Ok};kyL#pAm4JvBc4Qd!RuwXTWyJ3fjn?* zOmk+9pJ=x(33$L2BstmvlNm82e!m?7+`wR;xnDHk6vRa4;M;Qf6YlVwV9Q0o>H62rylRp+{l1e z^)hq^UKMb*HUk-knB@I@0$9K#Jq1~sYsyc7`A-tOU)pvkv}%Tb2kFSF(SXo0hBPC! zp#|ob=^5O4_DQYJD3MF=Ifzqz3 ztl;bLjWo;NIg7J96XbPp3~wY1_}Z~|EkQ;CTk1ctd*_(P&uY1k0?s$U#frY=z5|PS zQNBL^9|+%@u(qE0-a=kTzfE6RaZA+4n665c8Hjg zQM0(pP0D?fn~2@fEYIMex%xL<&HzOL+lIfzc6LgVCJ#q+TGSTyR{sPv$aOk%)7)(- zhPyK5xr#m`h`Q>qIp7RAhD80e7~8jeqE`lmmNkM0g*AS3j015R9!_ z3yQPB;svt;8k)C$2vlVnUW^P6vzkEDwN~T>Z%0CsHJXz4c-rw@KJJ#FB>EW zGyx=k?dqywiIyBGK&hT;(<-so)=LWnTAp$Puc&ui%BcY}apGD1L=U<25G&2hG5;BU zXY;M&dR_4n*H;qXF^sOyRprP>C@q5R*;Y#y+*%rcz3J-gI+?GCAv$0dZ(O3ZpYW6H z7w~9-xxn@>kksMI;UFZi_6doYx(6yBQcH(1*&#hSQlnS#H z0bx}~IL~f{FT2bKDY{4w1D7D0Y*MqN z8-k$Gdv~&Qv1P}5bgbZvW84NuQQ#`v0cAU$If&xTC*@GUhHM|Ky)L!U#ORH~%H>IN# zD99vs&UTeKie`#g;^$7Qb}!iKD&g!m%|mrHwrqHnpGK)#>ArJHB@#`C>86L@IwNW& zW||A?1k<>bc}d%vOjX|Ybwe$S+1l6dvjd*O0eVk`3DZ$;fr+B3ZtUNP*~jdKf%I{h zoj#mvIp`94%c#8lrnwfM$xR(t=$N$ZSILlRtL@O2Em!B-5H_pBfIdruIPwHro7=vg4N6?`7yhh}-9m>_D9u1qqd<9vxhMkU z;*wm-qLoThY};}E(Tq=r!f0x(&KoI|KgX8@7yvy-<~MmE1Yl>^2p*IcCleKrw{UpFTMv-jhuN8P8X!zQ(MWBbk;fht z@o=$;IQ#6$AF*46JTntZsfa8u*=Q*_xV(sEDcEa@AX3|;>ZrERsXm)vXgJ7L-RhqR zf}LH@f=iJaqS(HHr@*3SH57nvextE5wY)GU_VfEo)0H{FT*`pTnTE!RlLl%}@gi~J z0wqeat@iEofE6UF9YcFhP228>k=LyYqz$e?H9$}ksHWY=$fb%Y78Dj{ z8s@#Wy&@H0)j4FfP{FsKSMgvu^VR0I>WJKtj}dhFUa zP={=rV`gLqAN~tfJsaFs{5eia2s5}oQ(l-dT=d;M&=Z}Is1!d>tl&}x751;5$Y1M5 z(u^p#IR~<*wzr36YC_dEe3mM18iKR1g|> z4U%XB@vFal$)=*R|Gd_X$Y`2)qRv%(q!Ip-{!w>mND$dB+QcTg+Pdtdm@=ZjG&`|gqKlIQqrMsktl z#kSe>5TK^ouKu*ZQoVeSd#Xk}F_}V~BQ0dU z_j7#|hd(G{Mo+Zlt;X93Tgd$pu+=dDEn-{e@f$U@2~x0u?TG!3D1rPISG=`^WWx6W z>)m(0IRi;Gb+C!aS@n*WwN*0mxfxHjhdh=~%t*epzI?DaO2!{jA%|ULjA}VYJB-Vl zh}AROF*&NI%n1=<_L+6ROL$Vum*8C+LU@Ma3>_Gq{R4DTgP0d6au@l4B>bwTdH14^ z=3A6!`_*`H(mR8up7vLSnuWG{kTH>N&`Y7C`9Ap*gR0r?djUD2(&hjYkX2 z9+*`yQX*%?%5)gfDr+-aQnbRqbNo+gem>I5Yp!A&%| zFNEKN1QQffwem&_md>Dnnk5!;M*fheI~B-0$)2CVfBPGnKF8Y{Nhyb$4F;GKAE4zy zLTP*6s|IdQ9cy8i>tRj%V5`^D`W^eda6N8^*eM#oBM`(f6x4BKSBeeZ zDgTK*+O$lh;M;z<3~4vDE?ul`SM@fck6Uf?b78(usODqc&HHg-blpxj5j+PGlY z+gcCm$htr3Q%{9zm#vVqeT}}a->muK{$tt$Ar>(>r1uU4kVnsQ9`Mc}US#&BI$&-7 z^0N6s``rmh26Ur2uJJtOJ{j~k)HwyD$@8Ijhv|503zOxvP`YyTQ0!)-(dn^^-5%Xl=^Qd_KT}rCMp{G^SPf{HxsGM(X_oRWoLL<-1(DSmpg;ZJ79WNHv zt*>$SM$v^9Fj&-UO}Gf%9|>kk2nlm<{ar-@`E|a(9-Ah{}f%tBN zM3gmFG-GIpr#vjf8>Jl{RSdL8&Q2;d{p>xf=YSW~|G8!Zq)>fpR*G(wy=OgUJIg^9 z{WrPl#I*u|Hny%at^%R3)XdKhQAF0>JxPfdz);y3*U#&p^NLYJKIj8 zo+m2REup)TqKhhi;MmY=fm*ta)@B)x50Wl&!Ol9I3o9=cy;xnE-|<5KCTaBxGKbzL zn(FY0ldwwYOPd9RygWJ7)XAJDcuRN<(UVm-fgMMg%S%qe=%Cgb(%TR>Yj8AKy0a^`iRL6>6HRXJ1T_mvg{ zmkuNMkGb=fFkv{aJKJtP?k;Cw!tM2XtJDFHpATLGD%{o^-qk_1I z5ThF$B3tCt?94}kQn`7HsrytRmofksmSaKvUBO58_AFy9m2X)@{!92HIcJ*!`w>8p@;#vUm&OTl3OMU4g@n%JqDi{YAs*0vIVS4_#X)YYB z6QnG!Zm0FsFaXus4s}Cw8WJ^a-<}m3URZVM;~IDZbiPj_GR9jE=koKmHB~^c3K|0a z3NQjRzB@p{xsYwVTHK;YhhH--f>yORE>oK3L%+= z_)53Mj8HmIjlJjHrQ_>ZA71XJ4#sNz&qvW#Tt0#>RJR;=t+slG1mrU2n5d*$&Xtjm4Ec7+a)n2xif!!~+PoIL zsqM^~2KIbJ?LXmt>|3Fz9NP*ibRxuFAQ%vgNX*b*FnLQT^!%BsMt2m_yI1es=geL% z7iAJq78D>oPF@rVX}9LdANn!!Fr2G;GVwSR=#+3SBq3*Y<#6!#=QTLM%CIHU)7`-Y z4Oxxdo|5>@8j0S775xONQ~jGIN0}+qj&BR0a-O}{Pulbhd9-1fMS@t?eF5G%$l+tq zk_Lr26M@oOpF0^7k{Q5C5xPOuK1|*6T6J(jQmpV3uJjxxuN(lF+LQ+Tk|sDMzA#Ia z72UR}PgR>DwFrGXb}4F-Rc^H*gnutS6u4Wu?5)C>;@#2Wb~5j(LM5b|<)HrZFMgBp zd*Kt-2BYZU5t!s)yC)O}r*P*oM7M}F0r7>Ihz>jQd+)+f?ATB9B{PEsXjfUoH49-Z zc=&#hy=u_mT4exZBhi#cjL$Fxda9gY2{dvhlyaUie?k1!2H>tQ)K%P;ss#YlqDBHJ zdPbl>KQ5u^#f=Y4mCSPligF}QE!qG{n`vW&;XwZ8{TBK zA&lnAOU%;QFm8QAnM32T)uvoG$i`{+Ic!#fO=Qw(T^GhvBvF8rx`j=&-(CL8#$#!N z&A_0+makeW#mB9oR)J>Sys%&#zS?dN*i|^tEfV5_upRw9t!aH7sd0h2~Uv84lR@?Oh>0$GdBNeRN( z=q(FheLu0v1oFx5EGI-UD7;>{uKzTgD=~K&WQi|(!9AM!Ja~k#yFtm+aU+Iku1s(# zr%y_TXMCH5QlINTR0ua~u~07?OstBn?|!AZ1pc=DI*08{l`oB>v&Kc&xk$$&!GnKm zGk*->(0*wyV!Q0@Ba)p71$tI^ADn^6qz)&^jaNAw!GBjS935Kb(G_jD`_EJzoz zO$yP@tg33fwIdp@n0Q`_S`6dLd`!*R_4U##iFS*kU)q;bY>O8-zn^&62C+4Uka-V) zSL@Xp_@xKs^D5y#QFiL!?#0@wcsSW=YNTbSC=^Kk!E1%&iqis7)o(}edJ1os2kYuW z_-|Zb=@Y6O@xlFAZTi1WxeD$eRi8(mKK%hjJQ$&zq(a4d}f$>oFMWrPDb-+%)xFAXR}eFi0wcrkHN5)gL*l30LJ zjxff66)7cVe!h}8VU7>yGJiOCkV(mZk%pq&ypce99d)l0&kUH=`aE?M{8AaTqYF_f zCY@G5*y)wZxdXIg;uGQ1VjW3djnTVgIwL_2wcEc4?#mO8ylyiJMgxSu`19e z=I)2g*lS;f(zL}-)0JFj8L97#$b}GK2FikS;Vo$Y(0K?(Ecb`Gq;uK^kfN@r-*DXs(o7@^(CiVqonn z^&b&x*DEpxo!yJ?6vBtUeb+CxPDS>S)nOHX(tx?=!M!*Scr4FTT%Q-oQ1M9nFC3TG z+>egQy;&QAtnX=PX++n@bolnqzuRayyDVD&o<`Jz`EYJGn|%1gr-W5{RF4#%4;A=& z*IFZi1Mh}U#tZ{6Yc>=!)0f02vdHsR=To~Z55F*UUGLF z0d-brtRPKfraE~wkF}k#&(d`>J}&$7ga>z~zSBQQ{tg6V6^xS`RRc}py+jsHbu%Rs z@1?WaXf+V+~kwvkG>Mf0he`4vh*%~MORe|+iy0MPhblZW&_={25pmbGsT%zPh( zP-lshEhHhAR~sswg$44$!#|836ds745^lWx!~tGhq<8PMNa2jRxIl*~0ol(fGt{f7 zo)s1960a_Nn^Fl6H1??+Fxt8Kn>QQd*T3pxFjQrZt{)5pxg)ru?o~0`ekZAQx0ts) zAQ;1(UFqWOJ(#!Y!mM}j!>dZQ$T9VLM74zZW2dkab} z^y}W~-Ra)_kIBen(67@!jm(Td(@vHx_q~`6Jn16SYS6gxlTaQ&ndPCj^;Nz4ZMpo5 zzKb{C4xUV3|F;%kP-H`-aAA@6oz{AqR%C2R-1AA18-FulkY!+YPmxF>-|?1t$K4vK;5(D#^=@!8aI^3-*U zr2yH&$!84Gm9@{Bq|@>U69PGRO=k zB*M-&Mwd(+J(TD_v$>j(hS0Gw(hx1|^Lu}p_z5EAIZP*jh4 zO5J{EKN|V&PSc63aSi|=lJ7=xe1|q#_*mmp70&iX%&31D(Y8<~h z2wC&#Jniy}7pAwDDE=PT61;IOsm9j(N=VrB>JJTydu68AC8`IcgsrbJI^W3k3KkEsB0*?W(` z#d>;yKahEujm{`N?&UpKw`!mLY&XLj9Y}W{uM{=-x4iUERul2a0Q@ z7jv(=o->~ge2}E%uw0pFeUS&8;p~A8Rp&!lME}H}Y{;Ka>3oDJ;tCow$vSL1mKj;t zGpf%}Da2LW-SYU)s-eg<%@}|J+p7)B(*e7c+y$Ykd|OxhnZN{Cqi` zOBDDiFkreG*E$IyK71;?XfO%jWncIS(a*|>n0 z?&|fq%@JEfhk~ zKKbH(l}u=BysbvYrPG`2^NG_OrZoS!{g||hIiV4T+z;{wg}Bhfo$0w8Z%K=Pd~a$o zoLKKtdA^g;?Qi(S;T=@CP-^QMZ1MjpzmhgZNESrDk{x>Q3`A1+2G|eTv+Gf9o>Fy4+I zy&9AEx7I%w=$3Jc{6_gY6GnfMopd_y}1?H#V`6%XA>3l-nD zgX|ADKS{`FlY(O34%|!>z1{Fs(bPbDgO$V%TsOm5e`Bi2HbTUDQKi)AjZ>)9T@??^ zI&SRm7Ek>JF*NImhcAxxle{8lHVV{Q&2?4ohdl0*2d-efwfdFD@gL!y`$wQW9UuaJ zkMHlpk1poi(d%$&lWPKQYg=kzFEVR^u6#W@ul=n$x3JQmDP1Orpgq^;3{ZV~8`)1d zqe=PNck0zA5_!hmw@f z9Jed15{mp>U&1|=@YMd5HOPk;k+kL_y0k&c26j*5+Xe$yVjN5s2|AdY1J(^Y_C4GE zPfQ^93BKRkh*kFST8a?AN2DEPMVZBX}s& z`s@(J{XgoO|3hvUNquPLrZ456t#>R~@4NaEDY3lt|6l|QDIZTxZX+@NpFCK!+x{~* zB|m}4`3J$@|8oT57Qc6U5dCLr|1-7!vD*K)90CpD$bFW;5(DAw&f&qnEgd-If{ruh z?&KQIGw-_+5;RU(X_5c^t0Zbu5HtQj(^a20KmIg%Fof&4C&2JlCcpDxLDA+!oe$m~ z9sUSjt5NHHu=w5EyDQMgjF3IEo}^yEJFH3N@DhM8kIltdsYMfMgZ1_$FIIqFZYF^Y^r>wRMi=Ua5(@n|P(R zZc^a>!#AJ zT0r!-kUd^XNkI{V2Oj=-)|Tn<8O+Ok2n|18{=F2vC78a3@J?m?n;7o}Ars<|+exCO z1b_SOmGn5%V;g+gCQKrff$+L@5>`yjjjQ($l4eBU=Tf)cF2*xlom`^-^X5wmiu?HV z<=L|eOpgt5-GRLf8as$=Agi|WSiAoF&y#Urc#pT* zf98RSi3y+j-ntnfDOpDtB!gHAIdKo$7NgFErJ)~qVo6CgJl@$I+b#Tj6)>C&BKC50 zxOaW%;J+Fr5D^A>cz+5BF&D1Wp8dk?MDgJ?e)f3-qwJr4h&KtdEpl9rj6@5=$@}Cd zP3RhO+_h-1?||IdCT`Xezrt>L&*Klk`mV7gWp+QrpFq%Fp~W6 zCoHLN!H~(n_z@j@sJV70m(k0?8P9d7l+~ zV>>CD$~CNvH8nLUZk7A@&5mGF?g??d-HDq$Mgzf%iE|>NqBWk&8oUhwV(<;&;(i{B zQ)4$oxXqsJJ1WEC!6YO}*sWWL{&nUJ?Xkz1o*6ZebGN#9d3v^8mwQVIk8+ZP>q3_C z$nJ&J=^CrRUF7=-zD!b*2Ya}nerva{Z#gO2rLpOi-{^>+3k(d*n)q;^7Y%|6*^i@P zpjBEv;jbS1|9!J-*zp1Mv@F|kvk%o?tE!1}X>t#U$?CF=gdVVxn%{kiCVSS_)@Ouo z?O_nRq^M<|^LNYtJGO2o(UQH27I;ZwgEn2ylE*{Ic49}E`$>*{N|g}4MUNhYWE}Rv z|GJ1UE5dweb*m-)137^B3+q3g_M=ucv-`UMMrFzuLpIC_U9O)`Jk zLU6%bNY25t9&)>JpN=%+)=RYM;f+8O=(tZA2dwY8y=QNu4wRxDoMMEFh3-w9U;Ic7p#VDTXfhxrX5_{jj z4|xOu>7JO#+dX*5$2vu(Y^rWHHa2+E(FI3pIV_i?2MM2krMbgOI7=qFYW58I9sS#) zoZ=oFbPuUuX5pA(cnYB}-tf%zNB{i++s<7@ADKeU0d%Fy z8ryQ88~Y(}Cti*!d-@h$K_^`+X8+^wHBS(5?(tK;-AdA?f>-3SlarILaZcm++r8D^ zqV2fkQIhSusb8uZSbR!Z#@0RMFe#+;cjX_zM9(wO4{s;tIyysW2%TQpnQx%oO~j;! zFXU1RISi9Zc$L@<>Swdq_x-)yf3Z$wCJv4}DsI?=x#LD>=kWK~BQxk0I@w+HdG~X4 ziBqOOLw@k@s$asYU%S2iuAlD;>6^N`?(XgaO??>TJpC2|4Ek`~4_l;gtf{$~_skjn zT9$u59WlYRoT8T7MiNfs7b{?e89_TG_FJ{Dm=U7QZ@d2Jo1^)e9#<`pY9x!62*bg@Ri$~~Zr<0FZF zwddVI=J`LJj|8o|9uKTNST^ukpND5Yx4Hi0=zoDG=HjUrvM&~t_{)W7y3Qk-ihK7IDMRU;w-GbV7=3(l zzPt3mKcId71@x{oPuLd9ND?09`}FD2U_E9BQ&@;=kmSJCnJ_yWm3X(Je2 z39LEWB;`~Dd$6ALtq|!0f8;y9pg2yRBcYHZA)@v;@;I3cyMYUC>%s}89tFz!yxiIP zk3r|~ES{%8s?Z!ECS|#~C0KmlhY#DZbO-QUS=4MctUHZ5J6wm%&;L=I(}FVFy^n~5 z^6ZxSL{jq~XuR}z7gqDa?uXCA{R^zDDz6tGAuuG2d#EcM0&%weE?}}M$*CLY?4-60 z^T-t_e_8%@SBb!YknRJFCNgY$u%58=xh;O@@D%LCwB<0-3=Bl?Ky6Ac%eBcKTD!x! zav04!*{$_?@ythjC6C}bDxy7l@TOWANM6vjPZz@h_(KFkFS1}XBTtEpv|A|lNiyxj zb$5LO`LGaZ&t#-scie(o9&omvPL`VzqnZ1Nv6|f>Wx7vFe0<@Yr1T+7=LN!a8a^tH z#96J&Lm^KO+{I{?&Hpo<|C!GJA6w_ad2{sN_F5WDY6xSz7dtfCN`~#k`DgQ~$)PJ& zsImc?P^(ibobSIj$wkYPf`0Y)!A2~Qe3cqXA#CGa?+i>CXJ==@2cUQTuPh%Q=jj12 zpsiy?*~b_JaP_Ae2_OXnPqWvP1joWba4g=rr07Z(5*n&sWZIl{A-f~@vJ2iAi~gRT zxyBKbE(qDyEANsdYQlxa$HA<3x-wtJE}TG|%{2>%l0Wxlf_61g+~X7b+USR5Y58-q zZ~H5~*WJ+o9o_mA9vs;wsyw_IhgRB2-YcAtzP;^hX4_^YcyBu$59Jc!bRXZ|NWS7s1yZ<)7diAQ7)_wyEuVplO3)ppFh2@BRc=hu4I8m3W zON`4C&#locZwzxZ)wk?)QvO}l5N(A+k3nZwQzW-pGvm?0G~E)*cVR15=Uo@4^uE+c zAYN%7ad1Tk-^XWfO-@d_GO;yfYNW=Zi|i5S*$d_8WMm4*vH{@?SpD(~HLrjL>MGDg z5Mv=M4)NQr^RG$o7)|QrYzTWU*}o>&Na;|HI}L4v*XqJuxpB0%#T=zj_2GF{V^3M2 zQyu`9BE{2XvOL)#uwChEQU4`M4-yLnw)IWi^;o)wY!J>G_}z#ClnE_7dgV#HSG%?&SSw%9vXldFrky;+@*qn)2PKbCBw zfmDK>)n<|mSez4<`!M|Ug2@5DVdKfxB#FU76JRIDdZY(N^})}c9iLzvu6Cav$7Vp! z=;6^(LBLFOYrOKp{uS*_Y%ny-Cd0pW@7}$0C!&5UUbh@9TL0$Fl?vyOJhkPer7Nq` zqi8T%|1-o2Eq)tXTyRzoJ(A7v{x8iJ_Fuo%-P@e0-HSG%;Wkgg2+1l|dF$4#*!dDR zJ;j7uReQ!fr#;lAj-ULAzUdtk^V5ShVL?H$CyZq9;_LO*#ZA%2Rv1gY*D`I;ubHpX z7dO#<^pTGbFjrkEHlPTZ;_Wp;pUD0SK@JXcw7tiQ^*pzsQw*GD<*xDB+(1)y^iRiQ zT&Z#w_!<8t;KRSQ09yH-5}r%<>s0$3%L?gPe;*ANPks~J%Z?MWXg2yORS}JRMXtT} zKY#Dw&+Z}s@Qo8^jE7>EmX^?8yCp%ib8v95wbkzX`xqFm=f+yub;Y4r5T9sZF3!Qh zAtZzzz$HFwvkmynq#5Y2aYG47LA_}*2-%CZn*xpQjb}{XKYB2qAawnS^~2eQRK;j_ zclQn~37pmMQoFuMq;Kf*8jo(epr8P|AdcVU9efjm znCr_>bW8Q{JT}B%SLXbaHm=G5^OTm4LvYpW=t!^g9^H1bYOW8-oICd>f>U#CeI36E z*Q5~l{s4o7cO}wqpgxjq+vr7Z{J`|*i~#LkT0?uwn2FVq@oSR?p}a=`kv;X`lJlDw zJp|7(xlynF>?L%-w5fD1#|colM2yYu+uP0To=Ufq34w)F3M+PKAn!ccHESTfyor9} z1$q_EQ&UrIiD((Gb*cOYXM*468~*B}s3Yt&-nMo7?q@SCMKI{a$?ng?jD-N!U|_l# zLr;vB+6}fKoESHT*NZAga$N{x5O;l8{Op`LOg;EkT#kIw8S&oE6#0irr{&a4&}P2T z4bPXVnXZCw@`n%1g!>gRp79jIbFS?SnOv){uQ$b= z*v=|FI{vD@K5#mYO_}z)dd??&j>fa~UPq4}O-M+{d3p_bS--)rz%+a&zrk2K~M1h0bya`2feNy z9)lGw`G{*dXI))|<`oKwfiEme=vFZd^pI`ZjOyv_ZDLFHT%L=mobi|*5brKkS5Z+Z zYfm|MC|@m^_}-PAODAHfIa5gv1d`Lg zW4BwK-fQrXSvn9@Lz5vor3RFM!)d{PmRKhy+OrQ)Q5jiTHPL?XYdZ2Q3@w+hHJe`( zbF61TcTe4g$!;{IdH6eFTlVl~Z|-HmAuo^;@*S+0b{_cz6+!swj0}#0DUyD~_5C$B z_5`mhQpupNtzSOs78@mRjcQDHX7qPW_gBc8WiT)>IF7bdx&2Dh&-cMkBW7f2WxiH1 z2V2#VqjU7BJ$lfW_kBcYhYND{nXhn}Ms_NzourW>$6T>{V`f|7lbUHt7YB+}`!i?e zMifPwFlE0M7i(N+pI|5mYI%AUs8XF9eNb=U_#sGba>1LOJsl`)b$NLY-RVj6tiQ2h zZe_(?AsVre*Es>V)8af^yH`T3O4x|~9<{i8F^fEQ-@Q9`TxY)c>F3v=fqYGkM1pgv z#pl2$1ybhb9Q2Ct{n*P5&(>^-uc?RX zReIPuBBaOaf=$O4gh#ZgHBQ0&Pg*P8Y_3w#(<=`S zL;nr3XI2zASs9yEXA%9ax9!+bM7IG`uQF|pweBt)yHW1FzEZFqvplrQrc6_xcq+aC zsi@TxuHkF8Y|= zTE;Nq&Di!yOskqwm41!+$0HpddwY9R?CknpJkVnW7CGr-*z;6&Z^wy!+^z8b@Z-;> z@cxd93IK$}GVY{a_NE(dT!2!5`loFr&=xwXGA=Gosd8-XQ&ST+Cg#TLcQ-42HWS}? zyt=uq{s9~yD5f!w+pu0sY5LmNuQN3AJ3l8Z;N>KkXu&jb+diTCH{IPaHATM>vk<9_ z;s|RUMtl!Qz+OK5hv9du#?m_B@$=$~vO?Cud=Ix{!H=h1hmr8;6h1Q;g6o{ylXs;| zs-|i!iLxX{%#DBI$Cocm9%<x%VmNe2cF?fcsE(c};!)j-4Xd zJz$aW03)E5H6<8E-AzRBz?*R=7sfuP7uDk#pFe+IU;F#_m}Ys<8RyAMyD~Z_v~w=K zTIuQO0XDcln9@0co5lfLt!(4C{5&V;ETyM^0XvcLBzUqf*?_h_2PV5RS&8xPMcFWh zk3ahm5Cz=gvB>#anUtdQ5Bik`#(gFeTs*bTA6jG&@c-ap_vE*}l&r4g1GXdRue{A- zAw2uokJBSd2)Qaj_CE0dysof%CSo$6=pX07mW z@mbyRK0$Qk{i5-7%e2bGCxT-}lwC$sZl_;`F9;NRtZ%(X07bEgdR5(5BnL_fB2H-B zJpT64t)O*;3$b^~ymRyOxjIGPXTH{I=UvGZ<&FQD{1PSzr>yFhTcWGbIDPh1?IhY( zr&dbc!G61-`69w$Y36Ihmy%BXqDt=6VX8#eogA!1Xwp-m+y_o$)_AUTe zSaXHtWwWuevuAVPcA4x-9&l2j4(h+&5_oZPoH|qB?hBjV(<_R3mkOQ)ar5&R!gXL9 z!2*&-$T(vKYA<`3KTjH#myDwNo*A@t2YE29fI`uM)LTJ%3N-uny|UxTP%m>BaUV@^ zNmEU#cA3_1w%S%wa5z!KiOYmX?Xhpk$od)si)`rO@ylXP*FJLRS3Wrtley;aPoh6j zjIZZu?fEbmMQcQL#Uqcsud#y!JVmwUE#tQf1z^SLKmQ8NXH%e{$+m=QHX;gr<+TNTregD|~ z-l1cOvHEPIPHF{;oQ&e`Z8bhV%{SeqO0he#k?ElixmF2xCtqq`fzb|Da#GS~9Xin( zQ-d{0;THNAW)#|jX-32~nU?FRY^ zPX&D~nF7-rALw$su;8^Lz`fQB3aO(OFXC_Pkt`4m?fejwusP|oGR98F@h*~E)+b}! z*8cRTU%XM#WQX{M!_>G~zU=HQMc;5nU~{CQxi-r! zRGkv5lPBpq3j6wzJFpMAFiPFYui1zlYJVv@T`A3mrtESd=`(k!*nikcO8S^5i2RRo*8^0d+To6C;!UrZ?xfRQ6(+i}^KN)@S~axxD?oyiBk7ii|9PB0ytNeFxR0Erf|Uz58qAZ9vAt2k++&PV-(#k=r*v zg}__(v>SL4=|F~yjeqymRnt1h#%Mmh736~eeQ&Eop5NXI=q2p_o5L)5i)uvf#oXBM zQSQsbjZ|FhRvkIxGkQq3IjfO7S-K0nUu|qfC-;zO4HT6mP!CWL@?Lx49-Vs6^r9!4 z_i?NvNH?9!UroCyyDTN*ZI3%uU&X3B+i%IMV~&-79@_U zPo{_Ofp52{wTN;m&IEkDG1V8dp%#72pyfg4&Y6o3VlQxco|CE%rokF#dpFqZ&!)9* z$Z_QZY+GI&^zoUlT3ou^Y9LKM{Pw|6uU$=h<-t>M7X<-7L)cwid@=hDRpTOSuxR^A z+AsZMK?iv)?*>GF!-iCEhUskn{;b+|#K2$>?Z~opgx1wzfQ>XHs@_QckpoVla@JtRcrb&ujxSZq#k}d!iJ{AORRX9fjV>4(L>QXM z#ww*DMQMpSY4+u%A25R0u+$~uty=|Fe|`+LDjo@Uy80Dy+W6Q^cb&$&z&&&to|_x% z-P@JFG%)(4oI*J5==&BK1CAl~a-T~lhqL~0+0|FX$_J~5_sPx2o9W+K{V2}&cQ5eQPDqm61(q*iGP zoNm!ksPNkBr*C`Ql@LgKz;6qwPz+qyL%*lxqg3{cb&1EE7@bxOkW7^Ah)bl zik&I94LjQ$u!JBQGf}3s)>63@H5hYB2dD_^AtE+9H`dLR#O zADPuqunHn<>NK9yY^()_i%25rfGZl(EWL|_LL$OPm5`dLgt!nQn{d23p;Ys0Cwy0u zu%#zC(W3ffRIjf@!wpWItm~uS%ZM(>-M)Q0TUQ*`-W)41fxQ55v11SI{(}cQs=Yk8 zK4=s?v;mE`d(WPcm|n}D!j9jI;}0ed1SQ$Znm;gQ=r}XQbP4WhWb&3 zobCsqP3e-^7jrqfQTbcCC?!R_v(4An7s%E@Y+DYeGsAS@bt(Q$nqUU;*n1=EPXt7d zI}-1j5piplGEgqOQSNW(ayTcBW8qqDU}_03PrJ1bQF{dC8E@gV9s6o)AbcwQv2aygyP z=()l3SRSx^bfBW#G&1^gZ`v zDnLzXCA~JP0|!i&<%~%!efLU9I8RbH{Q5|VOz4oT#>-g4&5>w>3>hmkvy29_jOl%@ z!c`Qnd)PcR6gu7mTYY{s?5~*DUq7M6{>YDup!fI^03_q{RZji!Q&&93 z9q5psjKD&zAUWh~K&qxm+Z&|Uyo$AkH!mQ6|ByXQ#N}m7P{M6ne+sd*{vtC~OLc6p z3q_hVrgbe|=KaCQNYb_yW1@%U_lQ;2*!GJwUBTW@mI`=#)k>6C(IhLteSk8~(x`DA zK0Cv{-=a23rh0}N`~Cr~b^!r(Z>}X)Yzy4phB?ytFg|6eGUn!9W>c4U(0gs^`o}M(u)Jbfv)xMb&3cXH9CB=dM`Z)Su5*YN6JNu4uyh@h7&5?#g zAWA2Pa{-LJdLEb^-S1DRz16_WMt^>m`Mk4WFLH@@$1Xi%8gIm7%DQFvf;V0lw}bGg+gLn6YxP9>)OK$d0eC)>IW zOd0~;213R2^rxFXpHtx%kJ23T7jl_8+#kztvg`aoE-g9b_;>b*y&M%iVV%uQN7ukn zKGYZ}b7V&gd(vxVBoT#|H76@&I(r6MvJABw8a;mhIGZ+3KBk#EuA&8)`N2l6Tt@I*&uzilQ*oXr=8Lho4?*Mgp9}mq$bR z!RHMkao%7APpx}0j|H!Kupy3LZ* ztB~B6yf;)XXpK2}+X1<3C;y?kXNJdgi#gKEZj4cy?3c=z2p1yS=IWz-;iuK7ltI@0 z9xDas8&TNJW8G3WrMr=#ciT7lfGA@z$eqXC!%FFyn{*e-98AV3y_QF!H$6r-^;lV1 zXCtij>hiihSmtxol9e!_x-bJyp4^R(l6Z``9tZ|R0I_wGr|lHPl;H^6*q?)#dbpI_|dsOZZ3 z8(^_GIXUa@9XcfSW9rp4qtP-1+VHrni@BCM+c1rzrCT-L4G$aJZh)b^@oSDDN$L*% z*La$IryR$^jk+n73;_XzfG?Ovc=trwrCMqIys zy>cv~T=}|!!Y)R<hn|Om{Fqm;{iWFTOuAzc>jb^9gU2O-o6M;82$$eSm`1n>TOn z-=`Qa)8!*d4aUX?o6YewNL}*~xATQ|T=*=e4$Uq7#asGnNryVS8nD$!5Yc2q{kL1|Vx{+&~ZYD$1!m zCoC1N*5wH&hp2YA&bMm|==^Yad!Ko75W~|2* zz@TgQyViRkinzrH_6x){1B1@Z&SqZBr9Ch;J?%5vLSOEIIvDLSd6*afrd|>JW+_^k-tk&0&e6*guD3u znx9{X8j6REkOL*%%#)w#vC3C3Fz}oWFp*K-ZI^qPfkDN}mY^V$uZ`~5FWTtM>_1Y! z8XtXN6*Di)&&nE5+Rp`Qq?{-+Vh#`iGY)USo-}1@!*q{KDGZ=H6w_KWZq!92ju9P{bsZrDfcIm7L1_^kzBKL*@Hft35uFqJNB<4?a!=7 zEiQYk{W$l6h|_q+tXPN?DS=zfC{2j~;)b6Y;glQKB2|9(yTad4RpL1Z4FE|A0gyB? zmP}9_FE>(S zFg=Vqi{gn!ssf0w_%wKK(4$ojBS8@ksK5990@Uk4au#-2*a2laB)8|zpU0$!akmGj zZZ7oN#m{0v^^q;Oy1rOpINKn-8REY3JM%lq5+2lRq1y~W@)zRXX(;-zDEZYZTtY&R zvMHo~B-(W3>4~aP(b5ul&91I4ctb+Q_R^P##}F)L>Oy2{gthO_Jzb5%7^gZfph3EC z$oe>-RcL7e1#ANO-=^Zk=0**wF$bwZ7t)>aT*8xqRhtDm`Q&~ZwZT~>f8gdUw*;y{ zLznoX(+sM5jc}Q$*P$MG12q#M(1tf} z)_?&k4N-CYfq;+|h9ZshHdyoGZAR}qckalox6oqNM|gz;D0B*q$GQrJFqCK>9ZDLS zGK4_XDV!%dLKqK8-GP;$(w4bhjuQnuRfEIK=fD-SE63jW{o^C5o}hN#qoj2)SJ(I& zND$Ksg?W1+I=W|$PRYs1TKi)njv1V>5*b7!a-~}hd}g?pocw!MYJ_P~8BnpYFU2@a9~*wt+?XtO>lsSaAx_fny5A6I2x=7C{`0RK>QFST2PK4#+7f zY0=O8_;^%zZzh}K`Yy|djlbtR4SuOb>fSMVarM>B;)yuREdfblZl4fLn|LghfE!QM zD{{K+v*0@P6p30w6IwsNxIPq^dSZX=yDKMG6WGa8OoP2}kAV;%1d)(is13i`$#EahMVXc^8o*c^b*@vaKlE3Y@UH4E^^>c2 z6$g-Np^E2pQo|VDmN#T{eIS6MqgW6aUn^{oN-E>m{x;*&9R5wgJ=--so7t@G(Aq!@EeOZtV-3BvXxp4*!>d3kw3ra`@LUowqeIuA^6n2)Yvdq&9`SfEWOyB6Sr z?9x+T5kJ)xnyq_LeNyv0cg~-U%m98N?o-aCg$od!LqjTm?yOe9&7;Ehpq)@gkK#~2 za{Tyl4vq}$4loo$d$N~*zW|HlaFD=1(98bRts8)Bb)}aMRjH#kckkVsfcl_D;@h+l zaDhxP^);KKM$9U>X5Qn+?|&v^w(Q7R2BGup1OzAS=cMl7kzsM+w!nhT(6gdcDHd`E z3VZQY?AeS8F?|JL@b`&BK`7imK!0EPZb7kT|T)StYOW%T;pH$8-a zvwbha`_HetmoVHt)W^TT6bnv$#~vTfA>gYs@d7%DsIAO39dkTL*R%npj7+0tXI}2X zLO|xA(gG%P&?XUSC<0mP??Nrl7_1PXDw`k*P^b#Cr4_VYgG)N&Hg`^;akL5?SUta; zdxK;_y5O}49LV{?MXWZNcO=X`m(o7hu+%cDN)Q3x5rh|J0POu&2&Uk`l$7P+$*{e2 zfyG|k+QAzguE43xrs?UqhJ=|%@nnn=QXCz?aaY$RjQq51pX>big(l0|V4I-+uh~f#BRPB{T;GHwuFHf`UYRHY%sG#Z@ds z?SWLFq99bAvF%_I>zGGW{;Poqvz%p9*v42%NUqB`|@L7cTC@?e{4EcK$2DTd=@eNG)aP5}fv# zLvLu4wj?naRC~HLb#DL+xcCuo^QVBQqE+uyoXT+wcE|tz{iWZoz+mz;NFI{B7HF#M zt$Uy6Do)-ublK^w1@+$-5?GgtZ!p&3@i=^ zauobr2diBMy*e-0+(axEBcwX&H$5mxckX0Xo|>3goo!%5rCbnD!RGp0su+3#T~&zwNC^O9(Ph)9Lh9k`K8^e#&5535JMYg z)n6{K_qk)fAS&n5fxE98=X*DtKw92Hfn^NjcxAIw%zMohiDA_3uAR)l@y~Hi{*W&D zpNYni3?@A!@#)<9`)&84s^F2q9QKivg@xsutn9JN&w0eeJmFvg(NeaC==G9=h*&0& zI25cego%o}JyDSdifi^>&_Q;}rMjWm>3sIjkC9`>r7+`s{$W}3xWR1e)q2?@osTd)WE5BDMF zeo29=_IzcV<_A4pu@=%%PDS|9D$F_zZ40q8g|A`hR@y>@e(IaJin(uC4;+9;bj?Ry zmm@H*7lN=GvpbZMFUc`JXG!UU;O?bym7srIWs@9-^Yomc?t}{NLch5E&&m^%cAZ5& zNpAiY67j2~xGK>8p$pB=0t4O|Mu;P zwB)zQ5>R8x2mC}p;f9@~u7skEE(9j{rzi4D2z}%f6bc{ozq%}7*~WC#XJgr{Bm+fv z%{5qjo9pQ9>AtdG@3`|xnNc-@gEbN+AH{Yd-YbUP!FPN4Uae27B%MjqGU{jR0bt zg>WN>$B3{_d+m?J;?Tuo#VtX4hsciT$>YDHH&CNe^~ynQcZ}`6MxxO5Bd1Qab#_K; z=&qJtDli1P_rZy~5{JWgU3^dw3JwiL`IACwKi`!l!12)H&e}PIo+-6f%}JC8b2ZaL zDPA`;m;qvNJy{&^2unN()t%aqjsLHaCRv^8jTI{a9e0ielrrDc*CT>!<-Xl{O;CH1 z?O~>EfBEn>)mTfCx+(t}8!aMiT1u=1->Xms8`x@TiX)C@ynHFieD1-K^OTg7JXCU= zU;~hH=V$#!1-5!^t_ejYXfO$g-d&MY9dsR%uOA0TZoX70S!c^1^JCw&pZn4?SIVMse2cXRSa!4T z&9T#GH8dzoMiA_eJs&QPN)xT;OBM6k@KjT~PjTe(b8Dn-s=D`_T7Scg#tvP=fs`*o z1gAjr#Q%^IrhMxuR@PK3PuqO(ojV`;Qk#xdFWsahW%l&)0%f?5h9)y+-5Y=tWC3g}_Q+PgmydyWe?0ht8t&krkpTZV|MmxinKgbgq6ad?~y8#rC=yi~K z(79&CeoXs$1dZ?ujcElv=h2F&si3Q0za{5|og>U9teoj??6D z1A!9j<1X!g*EXlxA_Hz|kI)ND+f{Rdb8}8IqupDVw;})N*+>r8q5Oe!0VpX|`PapO1ky7ly;tx>SvAAl^?2Hkd3(C$N9>uiOhUtNV(Neio? z%F4mw$bcb55{_PF%3eTL(tn+Ri(_CrNa=FFm+VtfP{`6q-DSdqikqp8o4L7ph*{F3 zN7S4q(8{3_Qe@Ipe_(la6&f5UYdQ*dSp`+W%tn&DHkO9=3TWdAGWlr7B~T6lc9R#> zZ~_sBDc=AfZ@XgIWqlf?`(ycsY{-wUqQDPMe{(!L-+D2bpjUXOk*_aaqLjY}A$7`K zRb#Th-0A7Dw}4}rir=?!EzpZPe>nOLDQ-y)HyhjcU#${|={e<>WwD)mO6_Xg=Pl=V zI)8adoDo}rLX{7sVd0RT5VUAIux5o9SxjvQfO z@h`gHjE6+|y|3IUTldUu8}0>IgS`b1p5Q3IIl|fF&|BR=1+8GiH?EwM`(h2V^tgzS;6bzFAzl!9y}0s7*?w&I?Sh)aerjMsbhad z$>;tTHIE)W60#c*>v^4!{Y4U@2$aQ)yYqnR0lAVM>TCypU~Kf7phMRgg%+-((9`ao zAH38M!_zJ*DykymMyWwOvC3Gun_p0Wz5XohZg@I##N`U9I+EX|CD%fGqzUB~{uE#j zE?h2foi)5TU#`QYm2q|X&SlivSZzD=`oRyoO?X@==pEoDFx?=0I&wcYG^9i89c8x* zgK8^xvg+qKENuRPsl;jj^IVj1Jk2voqWPmt%5AVud&{>663W&Dg8QuR+l_C)Rwp$d zHoiI9i#`3_l^ok3y43;c5p6>{+YCfpi7FEIUnB(-K{R9@dI2FZDA?82Rm7|cqz?Tc zYZ1{1LRqhK%6l&<#9XXv5&sVA42nvVmiRIV`w%($@HJfL)+EmSMsU#cDJ^FzJYtqU zUdE68_8CVv4ZN34V8PJYAP^Z95Y`=2WIi`%0vTK6@oTS1LVx!VkW$_vbzQY}H~)&I zR)*Ru+Jo#I&ZnhG30hL%2Cy=#kV$c0%vt(!YcIc9i;~Rt9Xmerq=0k_g`BVS0+rNm zY=yO=R)|txFvCTDy*=7=tVZnmw>LOWEh8&y*Y)fgxVhr6a*8LAC*Hk#H(pVp%(9&; zs{l9zT5AfarPWo4m<4V79xZ+^0ND$tA!=r+rfN*>YLVkpLvV%Nd9DDpJkWycCl;=#!?($jvDacg@8Dwl_AV^Tb z_@=AJp@-%oU~J0$;lgnDqD2Iy%zSF0apRMUO9)FL5t5{{Aaoq+wYGy;l9&BuQOM z*#72w=lO?Vng}|n`7@A1bw9`ER7;_d`laRgU7U*_*y@ z%YjN^*(^WsqH40dC$F7<YYGffJZoNak=2v0wY+h3a{1EqZ(T)8_y%_Y}+c}jpWw*$Vf{9p?+Y=w+nSmRMk=| z2~pmw2T}hPC9Ko;UV@+aBS?qh;!IZlwfWI zGPg#Hy4W@yh}3d`wt=G=lzMylr>riwsdCf!q$bFyyuH1(qRXx57pHICya_qz%P|2z zDBvEUTyW(d1mpMO{HFyftkeSOhc*Ak~9V>u{WF_Ah`B1$b1{CUf%c9pH{yS$}WC;at=mFJN`ivLDBJas-qWbSj)} z6i%DIr&f6mX#ks18TIY(F)ruCmZN_PkwZ~eLBCGSjTE*EJ5a8=O#cD ze(A3O*0HIVpxs-1VaDMc6%C2n^H=@{2%3fgrT*JH@&t~Si#g}UoG89Arm?b%jQQrx zn`c&jJOxQM@0MznxTns0_S<>B8DyK~-JIcqAJtMPq5Bf5C|8l?mDMiT1mPI3zG&}J zLPGcTFM*eHrHuc`5kvTquaMm6YMv|-7YOn~regdkm@c84~Uya=*EF}lWtFQePr>N2i`Miiog5z#5*(f*s^Fc_S^vc%*rooLJ7 zMS**{7eKU{+v-#p+r%g!hg(}33KJNdg6R4b*vL0`_Nl9@myh=c8fur;xopjuYFsw}{d`GzG1mfE#0Ptz^1INaUYkRxH8;#w-q-(pt>n6eO_>Mibk0NM zuD1+FC5~5Y#p5muuYFQH)-Cz>H9)JybK|1qE|;G@b=`2m zMo*ofph}4tzeWXo^5iY{#pR~#>@FSZ(WdBDU73E@hmCDj9xl&gh=Cq?alQ)<%}Q)_ z=KVI&{i{gkGv9+<9izP(0hZ?f3=CfqkEC(@qMe2^X2L-NLK2rDtWauPy$nkjua%Zy zev(14`GbbvN0(BwRn zuy>mlTL{R-dvSMNOL{2IOZ080f94Xl!+iB4T69mDF!8tbSMJSC`e1FU;qlzt+bR#cZi}QrG%7# zigXAHNOQ*Hde`;tecpGUYo8DM!@16XUH=bD{nhh4_dV}9#~fqK{PagtfC*DHPBnDp zneI5>qwSxZ+2z{fm3lF%T{r&VaGQWH6(9|$QHaja@lFNa%5$3C30 zo14$m1S-?6<4T&787wG3i4D?{OE3N0xZ|F2_4RM0;sZWZ=psevrXB}7TSmScy`!o^ zXo~%~KoG)DD)U=Ixu9=1-kFQ@mlveqrjG`sj43~kkkU$XC_4{2Oj=vFq+2|5HOlf8*pG8ZK!yU&7(upc z2oBNXi@~%j^hUQ{JyDa_tWFmTj1&EW{beZv4jR$67Fh?z&&q7%J|+0LfLA5;;(m?~ zZ%f-2!mLa5aX^(fo}^hpuV_r+3CN-cFX}(A8(#w`TZr6i}*{MVz?)FU6l8hR3DEtx*o zn&m@2EcCfmuVMNM;N!=l?YKf@5~Kd?J|&)OWrAEhT9axYpf%<%Joj*~b=ofM-X9*E z9oc-syjvR0PX5eGXcqDd2uL2cA8t&quZ>u}Nr5J;EW0Ve#`=lQb%)DtC|zD6M5k;WE}V^j~gUU{gr5rbu29{jWwqbSYFw#5x_|ur0Nc32diaX&emu(-0a0~ z`EQH4HiCj@lnrPOY*1NQ*~^NYKIZ)uOXx3ghsd!b*+;|TLx`*|Rhe$Y$Q{eaMrsWv zbrM(K?i7^BMs}NBRd|MVUd*X!J7#A6-aX5XCLbQY#!0qAM~)cblH`-d-rIjCI6!wu zw@_S9?%axv=UhpowH(rYlG3kPD``caUi$>fr$Hr0$|`cRQO+!$AXhMdlqGuduLZuH z1Qc^IieL2wN@B}td3Go%0eLyf&rFy9%-BvO3A5;iSXLY`GkvlDYPI78-4yHi;9ydf zS8@fwnc+Su2%O}ejO*M)GwYHS>Iy(j_wJz|J;N6j{ZO^#(j@vg6$14=e2LNWeG7p4 zZ45TQ2Bv<9=nyeVnRv}2Ttr!5hdAzn{$mC}^yA{40Jl?7;f89He;XO;L`j)@^&K?I zIB2e+IEN+K0paVH2@#3_j@ICs-#b-m0H?zCU7Uksp zL?(n*_XiuX>t9KfV~EVe%texe!_TTfNWzYQ5?x&g={>4zn5Gql{sLcyAZ9p|A@v%* z`@PBRq$G(y4STKKk-gN^IIJL#cV&75_WdHQC6JHyBmN^C3qYCG9iZFB7w^0CzY0xS zH}Z$SI3-v%M6HcqT)0wJ+x^ps;)(HQU(ePHz z{>zoZf+bu?MQ;VwQT$k>t=u|!$JmuQN~L`%%9v-`sXXg0UyTvBq7vg}0G9gHEx>qW zYyhQo(Q(B#JzOP9$=9Nad4g|HdU{l~XZIMB+36 z*%mi!I$>S0SRu!B4jyN)^T=io4|Mo5Ki~1?y8^OM(7d$=S;1@sZ;-JgzC!G++f{* za{)l%cJ^bH%W&p?jpGP#mxgG`mq$rb_>G>6-?mx(R!FZwwWt0aJs2I$GkjTHf_kTN zNj?Wu{?;x_J&CJP%}~g%%hl!ObW${wr>3VTNYT^P8B-8;HlS$1kdFDU8=8RQ+RlZ10XC> z_iaT4ge46KYmeLe7#{58OIe`6Br0j+jspZs?$6r;I`b^fS%Q*f|KmMD5CeY@2QYD4 zI0C&y-50Fq@&iw+Yp6qUF>?*xSdXRPjBttqV+0Gc_=>@S3ngb!0Jyd z4yaL3D2M+Wg;NSuR0K6gsd=?5zjsvvT z#RvNb^c;+#Swad^B*2aphJOcYus)gJvi|$&26PB+Ky$`59iq7?8N?Vx!DLG;t&0Fi z^-SP%^VOuN8DL$2I~u+4@-uh0u7T||p6tQnw;2Hy8jvY1BaaUBX29ldjycuxi)9dd z2`2@iw#FI#ar*(lTD&?~Gq2CzU=$Jx_cX#WW*tjjGu9`Dm^!@sjw>KnC*Ui^2M_b7 z(jyDo)r|yKJ#8uNsb}k#+7xCoESN|*xEMENo;}R|ar56k)H%S;xRxZzhd2?ssd68m zJ-eNU=_B-fZX`VtkKeEqb*r-fcimfq!Ji;(w4RU!^wVw-Ay9ukN|O{%II-Ung?w9d zr+OO!YwO_i$CW0L+m;SqSGlu7^&l`X?Wa4uRman0O9I!z!=-Zd_|NP_3V97(+&?=d z-849#b#eC*!q1SRL!wSuLk4fU&JY%cHyMJft2dBIOtHOU&L_gPCZ+l$=pU)+Y=G84 zCv%LYl)p|*3h{ix)39%^@0UN6+Z~PGGL9Tr-(J^4ZEZVo;HThDzrNN#csTFT_o6xq z_;94{l=FlOoOg>P7+l&fy{ov8>Xj8L4z~G_KQvHc zgrX_cw({2p&T)(DJf|A>Y{o&HaK8eNVH$0#zPQT4l%g}a6Q?~9J#W@1BijK5{bm~@ ze;C=KT^5QP?{w+A##$C#^b2A{PIF2hz0q9C9rmS^w*lJGaB5y8e*)z+s;-j2E4y_` z|Lb)GbhP6)jaSml=}K6(vmF>4?=8;h_jr@ySA$~--HYQsY}zj`>+9=h{Blt|3P{6X z3Jp`_%fPPbqyaU8Xepw!ou$N2%HIALcqvMfEJ%Xv!}gD<@`A6Raz2W<9w}VU;R=$2 z>mZ=4YY#sNJrL6uuld9 zfNA{yjwbLA_u~Hm_u`+M`p-@M-(d;ql(c^Xw)3YZwI$8)Li&gbRkF&If#vOnH0zqr+6*wdr|(FbSvA@>pFp@hKpR*)HkJ zd+cO)VlCgR`IJcizFfzNu!WtUGt*WEkMj`$V{ z9Xm$mb@nZbRA3JN{b2F-m#b)f@y>}pswMPtHFb!a29qQjl%xz7o&lv7`1m9!=<5~5 zvkv%Yv6vbZUt_cLV`)#_oD@Et2H6V#Z3esKR8an94)T58Qplh<#8yDJ93UC4Jx9D? zk@q&bO09!ds4q4iz23mJ5dVEOMm*aExJUIh04k(s3Kh0K;4|N0;v+?Q$NZO`05g2ZbWhE)W%kyQ}X z$Uln*xc|-!_#b~D{(e&|88tAtIf9?0dgc&e!TO)KoPkU%0I#_(LesQq(@)~05wE(S zfR{_vNa5b>`oR@RLAsToaFMXSBCbyyU+Showe7!N6yAGf_jw1Y8h3n?dtlBdHR5)h zpu!QgS!74*9q&!}`-3%dC-%VBlu(f{(7quaKhC^J{K@~Y_xPEb*Tj1x&we^YHN${! z`sddCb8G%tHUDg=|9b%aC)WHEhW|OC{yDt=$!q@KrF?1Kn*jF3&P+OhyeV*PePAaj z$O(z?JzcGK{xU~+^#K4mfIosX_rkDpJXN!nD*~N9XLe5w=prr?Du|WtwNuxK?u9jI z48I42PpuK|hSbL|E+SEmHG6va@cYov(C5#eZ^z%oi!A{Fy&j$ZoE$l2HhD>$F;ug* zC1EW{$l-Byh?;s(_s-zEK>8Mm`@0X@e+-6utGX(1EvJ`ktS0dEM7yQC{xZ`W-s;uw zKJu!*--v|ZU;H3%{9grj<+#&=Kf6ajelOZM0r2+%JBTpv+0lMeTJwQ*)AeNctq}LI z`Md`azpXLm#1+z4BH)2qawsY(c}`>(mkeBbu9DFHL;3(R))wZkPJZG5ELW}b=zy&o znVXNIkE{Qka)pKfj(GG-p7C`_`H8c$r$apX;|Z`iuml`IUyUA49vDJ*0z*9rhd<9t z&wkLKC-GZ5|KywqvUj4S0iOam(P*(JD2+u+UZDSn+JjRS`^%SY0iBRKu-suK>8}{* zy~4QwvhAL4kdXoXU4kQp0_E<{9|$K$j1sFi$qOf$B|en-YpS34i4!k>Y;=WCmw#4_^Hs)>yvMR5yal79s;wrE~TaobVoN{Z95`7AeK=H zB6M@k0m0P&V5?!&m3Is-jV2Q@V{D1oY|Hn(cXf?(=H1x~KT2j4gNxnpN)6lvW7yF2Px@kaqN z|ApJJys{GF4Hu0h+YfX81!r@in$jV^lHS$U@}A!C7kFIT7r|mSa{1Q6JriSBwMSYi zSCT$d^ca5KHr|@8cxZ%jMf642RE=#^3P58}mBQy!oiFOr65d2URA3TXx*asrsj4^m z0MkX9a+dvXyuB63_g&0AnFRKyFBx>k+#;tv@uB?JmD zmGkbix#juJmk;93?85Oo{j7_y4L)2k$8fcHHX=x=7-TaznL@ERRoK~D&v3AgpjK6& z*gF_{eFs_Wg_`F?Bd0A>rjPsV#p8szZaIiW=pI1$OZY}Y_J*I4pL>86+C30v@K-NX zleU_Ub><8Jvga0ma=fkzi{jAdlz^>5GoQ)+;R7nsbaL$8Wqt;Uxrly{Dcgle*+hlcnm;A!#~Tjqo?KxZWeg5 z`Dl`$^#$?b6>z}`!@rm=enQm^V|lxZ*{$N`0Pf}KEEg2J;c<(u1$mX^&_&S@Zs!_6V@F$C8)QNsD(tV+j8ir} z#|VmUVguM;nL5H^@O^?=x1)yvf{rO{5rXSYAkk67(t@fKHi>f0EGcj{o+R1`Af4g? zYJw00p9zA%4@33n3%g*`2MWnIyQ(EKn4(BaOJg941ne`x+tyMX6u$!L)U2Q`1z-S_ z;5X_#zrWc&-bGl^k3W@>m;ZpfgW%Kyz4mhUD%ibd&9Ujz74Bf%;)kv88SAeUT{9Ud zb&%m6hY3CKepfUDi0z2whxp(Hkv5R4= zI0=U#s5O0y_z5~Uy8Oi;>EZwV>@EI@eg=1l>P|<-Ufl0|GsWJ+?=?6U@K)8=7Uctb zS>!*m{9#D%#})m7jg}Px3|fycA{_vo06_Yv{j<5_Ar=TWgnJvB^S;-PjQ4{yb>5~2 zjhEWmT7th@S%phgpMp3iOwIO7iE1aihBVyu`zp}~j}ZCmn*I`?2!d{`@SXz)@UT0u zP3%e8Q|e80^GqJ54luyXG0sqsZ{8Yh1@vuxp0aEo1&!YZKS*1<3%tcELNS8uH@1_; zpTUMO_{$vB?N6G0gN4GOnI%hkps5<`ph$m*iBJsaP7`Rp%Uz7X({8=B(U#mrkQW<_ zU5k|);zoePv#o*aSBk}o7LG+89Ndqwvs)1SAcx`hF9ge*)5|Om<^^tr336OM<{+}8 zb0eHmCVBHj5Yhh>zW`)u8!~uc*dIjbQT5Y4LDH(v@FMtRs3!ijlHl7 z4hL7zV2qyYzg{1g4bMtDbKV|uphA2ASq^JWmWj52&7O~qI<9=|K00NflSsBc{fI{* zVz_I@aIU_-{^_k>4>2RsEut1GQlMUKT=AgZS5hTH`+xxZg&u2u@1<)tBc(Wk(1C8t z^Mhea)9w=JR6v{id^!ldwDy~f>gin@VAD19_)>jw4<(vAQqGxX@F9RAUWcL&LFiJt z+WF!xF#f<+46d$2M$oEwLA;Sr9!=+|3nzvCg26ya`8eaY?6)ADb)(zTJ&f;acEPTP z8evou@;2H+whwt%UT>W@J`E zOM@r#Lbq@n0WDR=l2CA*@&YgFEB13NdO|p7z@I4vNdiZG+g)PxjwNlZgkW0vVKJ8z zVq&rJ!Dx%$wCzJ?kuIfxv-X%M&eX&E_Gu$jZP`HwvbYy$d63>6bw{QX0p>J1Ge}|_ z&;$fiBIi%deoDI*%L8gzB@X+OULjOxc18P*l3m+dCC*oj+3~a6! z>a8j`*uh^%Wf?ssl@75AWy#($zqkPpxLf#I4Zw?&b{+cCYo>4xgXLetrq8vFhDBOE(wZEK=8{ky#5+z%O>n7VH%C80_eQvc&Z6n3 z+U9;1E_Vkj0)*~(%Mh9&*1G1Ld2nTsmDS%$SNb;>;1;4B6x_;D{Jy8{r5_;5qp1Zd zCj&$CNYt()60x3ht`y(UXO*wcCY}`DP6R;oz!BNA!Rn{kKwg;P6{XP*j$k`f)J*5_ zi;x5eLMbAjl-aehJi#0NSqq(g+y`syN63<~-4oX@fN#(q@hPhYq6%S*H9_L>^t7Y= zl$7V6KIaVa*!Xh3af98g$B(PMmWv4I)OyTQ`lke^LLMt375JR~0Rdl-iC{QFN>#M_yxQ1dk;y-UAAsY;#k0KZtDcUej$T( zG=a9DgK!-qB%XmGAgt|GBFNx}?u&{QDYN6#NUCAR555C36&b0Is zv^c%bwNT3>_kaMAnR-Q}52}%-@+t7t%^JcL$rHM*yyN@8pkMthwFQ)?OE2_w$iKnA z2s2Y&3?XONC>*Ajl^+`!0gLd`i5qUL3&WFFgsjsQ-^66`##J=M(Z$Dz%PJI7tAM6Ah_SDKf|FLYKnCdtp{z+GuwE} zUXVR}s2~|qIjp$fRSLK9e01XV?W7l2FU!_dfo9Jj3-gD~D{wyLLqz?n%zqvzt26_l z<#r}ePtIiCNDc)hW1o<><4EsB^|_6E+fj{pYq0!(2hLv_fP69 zTTKG1<7~=ofP@lPiWqJqcr#p9s%sT}9x>kqeC@^?z{5gi-*uda&>ezK&g`_UY^~I|kG6wJ8hRd8k|H(l2 zl*}xcEy?e(IBv-A*5+{P@947kj73F&(HFbg)wd&ECHyj=&amBf46jp| zNJ!tZps1@vA%nV@_%Ow~sDF(=kqOv0+wcB!_7DdX;Ul9lCpO~Rnk;82b{{3^dF5Sw z+(W|qw`tenN&}deijT#?->Vj+KZi$$K=V#zGH;k47qde0FcInrWlpV>VZ=W8D)iof#E?gTy#5=zA;|D ziSlWDpXi!?>abC--x}D80!Olu#*CAlw({cs($LE0q&pr9o^T)2{`~HXD+L|t>h~2= zH(g^ohjA7%_BC)S{d%A3PPtmFw!ABN$~fMh^9j|8Nv=X&6FZ4w8p+HP_4dv~`o5ur zrd|5M(er-Kp9s|oWIscqXnDZO&jS^fM)|Zi*;CHs-QI+vE)T`Sg=}egqA)J2yUG+z zhO8r*Ozc&N(SB`P?&NxZ1*(~2q_!DdV=ESe0|?~M^%>@;yn`n-8-#qTdo%xBk;~y^ zB&rQ;WTfvoI{C-Hci%rntNy3|$3XB+wMkUr(&SCY*p_3G(hNZ#;@Tjyh=vv%w5Dk% z92@3`ByMR_Uf{|eqLZHCux&6FxuBH`|A5CC*u{~yv@nnfy$n8?KFv1NkRZ5MqHhV3 z?B46^zKh75gro3?2H)=D_!4c*3*U6AM4WW`G?L30w)OhSp=W$c$N==L{*I2yJ9G`g zSC)fFEeMrkgU2qBlt0KwE8lgRvy;2*1K4vZR|}-m(k9BlXI+|L(HYVhWNGK(oQ8dB z{ad5mfATcy)*;o5zWsbZg=Ry-_5}ok5!vaw^t`B_x*Sf?Ht~=NDTRFSHr*ENMbV*I zK~<-8m$2;nyv%`=cDtfo`(eu}1X`qbNLDVay?|pE*1~os!jnNLeC2BNA5sCGj-J!R zgsE|w^urAx)~T7m%2d}kG*~o|NKb8c+wE)gKcEyZR@7zT`AtXa>A)@dh@0LNG1k_22s-Jnv z9Vb%5v5LY^^}F0nl^!#bdt^Ma#}oQ##Bk{c*h?{)!#WP(X{BN=Gg6&<8{Q)^v%gIc z=H{njvt*wf#Ri5|=bs)`=jqhvy}wWJ{hdTwV}nk4rXOLvRU}^bn+l>$NW@*#1(r*> zR1?oZ!;1ehyLW2$QEGRTArVUsupE)U%FtAWwBXxA`&IYk?!h>IJeHpN3jCDl;B14` z)XnKd-?3v*L0*q|?c{y`y1B&~fiJ*@$Vvx#x209aK0c>uI@mqjNqAxmQfguBp;ycc zpR>NZIG*!>Sh2vBx^LIcomrUq0&eF~W@hfp%O?TLoTa|5UiU)O0#CPTZs-=I0XGwV zA7EX-r?(^EkVvpR++j%HPvK7;PQsW!JBpkSrA+BpDl$2iq!ci^LXCrN@?Z(o?}(K9UeiD$Ro1i zT?l_11kkyOPF^mqNq^od5ZyS36dxHAQX7_{)UGjFWt83}mWFg^((>&qLW5an^bX`a zB<#ITMmG&3t;KkX$3@+PZ?JHYg^qaMjjqdo{pIW(<3H9iNsf$PU9oN2_7K>z)kvixcARG}lt(zGcB5 z1eSJM(NFk6f)n;jRbrN2+ z442ZN-i2*QS*GnnnUh>N`Cp9+OHnUocjvsddfZTHKO_%?1$9rmKtW&_MJ%T@30Z(r zjVxilU|eHAAqWdqBF6B9J&uA()1wzZJ-sR*d9q?G#iov7|9+9T-;-?hf{)iJIplYo ztS{ll12|>d$am;+U9*4AXqqhfjo-r?7C)N?9W(kjHXOG;{uaq`mM!61pO~EavOs#j zg;K~xQ;rnX(8T9KI?rWk)MTQ@4CME4BpfyQ5ELaWd~Yz;;pf0$=kl!=f0n1+M+^RV z3|zBYab17z__OkJ;fwi-X?MMx%2b2krW;2SjUyt`_Y_M$)b4~vMsNm(L_WNnL5Hm< zlDY^=wBmX7Nw|d3uLKfz=0*02|ppuD)CXb zc6iFEBmiztSVH)BUar~`*Dx zCf!p*wXUYtIdI%$bqwXv?x#r^j&@A6<&)7gs<{p0?I@viD#{>Lo~Ns`Kfjlsa6h?O zx)`_@s~{Q%V!-$2tGwOZ$AAuPFf*In^Zv7a@nY`Jonjr98GdJ-pKrgEGbg@r&V*OT)8eK+;O;W#+98XUhFAGlZn&;2JwcIaHrd@FziKF*a;9Lj|rExvfkg4x5Fh6t0 zw_>kqE}n;h9SI?iqH9%xzpg;*gM)9^R|YmdJMT#9A}G_&M@omn^=hpm3}zbz?d-qz z5#i8(^wFAi+N=B5IDgRUa1xfKGj$p;Tc}K2)+vb^U?P3gY&F$rn)EHZe>IPc!#c1yh8h19DU>jD+r61HG zo`#0;hIem%f5>{_P!}OEg^~XlqFj9d`;djNszwib1^k9Rh3XLs2sU-Ho6T)?ac`C%wa%|*D=p44;4As>`M z>C=>!!F_j$58Kvj#*dzt;f7~BVTM#>BMQJ!L`sQf1@NlkO1P;9((?55^ynpgEw8g_ z&LS#&ZJzNrQP<+rLr>4QtG|Bjj2ShS>c#zTnW4+TTITB>j68UsxB6ft6DfQ$&N{ow z`S8Y~13(uL{{itIU1;**+7Bx7ZP&F(WjyR9sx~R|Lz#GV4^<{1!7Kqv2t0&#){42@ zw?Wpi!0rKiyX5{xrv zP-2Iv5u*Fz0%OdqxadN8uI!>bySi3`iae9i(1b+B)C{9YS`h^OLag)eSfAd?Lr8Ph89kkH?U(7FM*%d)z0c+@HrKvTOY=rNoKfdf|oWQ}j(~$Va&u{^X^)v2DGg zLafqn{SCEpz`i7K(j|X4o@})P@=O>KdJTnXT@baE`sscod%(&eGLUL!&!!D_rv^fG zoHCGjDNFlP(FRd5&bKX9txV_J2*6Jw7&rw-%8SzB0}q2nPxr<@NA1I_llQ}y-tc*L z)Y+;}>)d-{zHN_+FlA7&P&&d-TOR+1_C~ud!dY8Up>eU|Y0EDqxeFUm*uQiYgl(KS z?G)bovd?c(b$#gxGoND-BnYD%7nT#qn{8nWv(qx%so*Eui{W zL%N}zbMtBkf)D&n^I&@;B=dBh+Ft>a^OTJqt!yQ%b-qhc?PXhFJQpP-{WTJOEM*_# z&J8s<((-e1HY^l&SGpH}u0+-m`fwu8J*xCbkf{#(8=oeQ!nc_xX43L5PTwrU$}+VK zYz8x}WTcwQM?AXyn{P7e=jw&0+#iCj)0QELn!@+A^K_PCB@U9F2eAn&IBKHJB$r2- z^EVUznxzNn>li3}iL5qMUP0lcds1XU<{ z3dwWOu3Cu}2?oG#q&$2Gdc0Gtq5$sJ?(Ez9ZRW1kqw1z@XG46h(rKxS9O< zWlmE)ycILG*vkNCza-01uQ7vGU%?(zy*$pjHd_Dwy$cab&}D+kD7z3qDA)blaW`A# z0JpU>Pwl25j9Chc0k7LYJcs`_5Od7^2*_DJ{h)0L)9OSDtvj;n*LD{3<}o)5P}4?5 zTmpm(UD0C(@iO4&od(Wvw)4op4&UIHE?G18>&xRZC{)XUtXo-pzTN$2W_6*KeDf9f{d{kDdwV;)-U9aVB|@cv%-T_2ULFW2BvEPiyD^2h^IsUHK7FXi zPk6>YbNHOk~=p zQ_|7~99qPIp+k>D%&jV5PiapIl;HS`o6VH=Dc*g$6#hP_EG7}#;0VOp=cppzZFo_( z`>OD5W667W71|iDTT@HrvP$Q|iRMgT5=RP8>5pKv9s{x7kjvmgHGs{|OMV#UH(QD* z!>*;2SiFodTD|yv$6DPAd}rqmt^!n-`1PKdd41V^GWXq_{3I5sV!T-Qs^14ylRZXp zIHf>408AIHmI3P*2v|MgK7v|1tK3&U9kwh+7Z}Z2;<*pMhp=^Yck=X$F9^$fM1yXq zPP%=S|7ZXm6EF;T&)&o%`ccx2+D-R(h;DcXEc%*C5oLfLBm6Gn>M<&0sGy(#90YA4 z>*}%8n10uh!<7cPT-;o!3LvXQ3yXzXAtlBG0H8Bc48%lq<2)Bn|-tb$&NYY`D7>bCk{VDpWV_E`Rsu{otr`-~=>l+{yFVDAeRm5_0r4<o> z3Ol-sh=O%3UyrFwzIrIg?OVfl0cjz5ovqGifK%tgqBc~aiG}@opZ-xWpO9`u(`g9@=tK9VBH1^W6nc@UuS^jS!~NHiuZ(qA5Ri_v-hQF75{1e6vT>)X`LPMXspu^P5hz)YtEc;$A}MJ+qVE8eYB0Axgm$ zZn?=KB&l}rCRSJPE3gQ${(UT^Sz=`p=LLLQ42<~bL;CF-xaHFW0(Nds|3H@v1hYu89llMXl0L6bc z@&bmltGXQ7^%G_pLf;4Jpa()4?!`48mb@Xryk`+C2|IPf6(p(A3gu@LJ`xRYcopGj z8Gp5e=;raO^QlPbd$SX(gZLbPkMlU8#b7arE=7W8BwPv3#RS)14mZVa6v0U&#Hn|1 z6|>Xw^73$L`PWw@*XBY}uf&{s;jj+ld9)+Euz^u-5ZXXAiC+NC`XU>^3H!{C3#fw* z^YsGx2huut?Wso*5;!1U@E<|*F+Xvts0;#Wya|Zz0&=;@_VhMX^)@~Fpr|0T#bi}0 ziCM%zBKn~Th$M}0@itq>{EX_E+8ww9HfYu&aGS<05q2(NVWytqXfP(}G2IW1b~+aK z^r<9}R4qeSVrL*X8imy{%=fPPxZ}Yh^4}t=AR@o9t0H`wkaPPzG2%FVoSJRnDu0#$ z1?d?^_!g?E{JDtRx>s}*O{R|`Lr)DF@{lfP@s5}Jr266vp2J0p^QPA1}490zfpw{?tH_`3{tA>b$$V<;9KYMl= zy}}l;-;EF=i$XD%zk)th0?dMoaV}7b%}7P?7-0C~i+3mS%;@^H(8@E^)C{*7VV{-X zi%x3cd~cJ|Gn2N5U%q_7*H_{xAzv*2cJJVH=}0_Q<=fGbUccXaNUNB?iquVu-Vxrp zi98Zn?(to%RNmf!sHi(k+Lm(eG?rn&A1;gsDF0xFqO9yDN%Ei(1uWX z<0W6iLB-%c^WiKB>G4f8<#Klg@OR9lY)(hE?9gH&*+fouAFd+8xv42svRgONxbw5A zZzE$2BEbifN!1-AW5i2JSilid2>OL@mxG3(m2uZ5^2G=ET$xw3m&m0^NMA8|8&fJ} zS+J7@pFsfH`JsK&Ch}w6_~1<{eA}T|0C>3lSy$iQWHB`irXesI6Uw2JK|m2v9^>+G%iY$;3eLA_}Rr|?3Dl9j(={)|JPQB zf~bJ%zqtVaY@C1M$UkBApHt?4@6aXZx=mOgsCNy-IrqB`8F{rxtYl3MgFJRvTT|c= zDb>sY!Xm3DUR7Ca*2nW!AByNAJ)n{pkCELO?JfB?51=B$iT)QVVPqmhW)>SAok6kI ziA3CI@wO^)LT|)M0TriMn^ObZyRp80X(UXA6vyuWm_#tXCj;`MvdHrj)#;r^zapIC z@kdXf(NQZ|HC^#rNm(TDPjLkG!$E+_S)Qsd{y=<025jcjm2KpE^t7_M#Xr} z5XbM|*2d<&=F!`4pW72e9dLEd?WyZ<)VWK62E^y`kP844eVebq9u^;8W!1?;FQ`KE zA9vFasS~XP1@2wADe+8m%PYWDA)z6XEy}%-!Z(a3-)VtGAwDYfAK&$#$+?D02L0$P9{<*%zKQseX=H?f zqj_LX=8Liq>6bqIy&)uI-_PQ0M;>e6ma5r~s+Gk!f|!mC@M+Wiv%m2Sc&UY9L=5gA zv&1XRnwd0hncekYzYo~+(FJEzx-m0l>3yY;IPp}d$VrjoQYBtI$P~1bl-Nkw4RBmv zLMm?3u88CrlLM(R@XrpkNgL&t*j8mHXV+{IXZcke1QD+k3EqmuRWe~juEg3uJB9Wj+^XlMG9V0w%9Ap7y4s8 z3=kv#&&!{LG+GKYFOZ#YM*L}0mfW&~c+f}o5OfL8TX7D}A``WG?F#0Qr!zCM{yC`E*?UL7KmJQ06P@do6jCZjH|IgD3vJ{x7!w&eFt!p3Lu_l%b&^7_i`q83}Z# za}qD0C)>Jn-*ph}%TGxC`ys`m&KTTD(@t<|LKbpx>9=2kdmI4h7@yn?4dDlFfVtQG|@Ee6MMzB16wo*vEuHT?PUsy zU65D|onOUXlRFmldIYqP=8WSB<1yLGAx%EA_0YM~nZS3LWM_BSiJ@-c; zne+#%pSaYk)N}@B6o5ylXv*F1X?uHbMj5v4BO57-z&rSL^j9dtL)aEB*TsBxdx$@K z2td`b-*^-0Zd}G;sU&i&^Bs7Lkq^7JQexL#T07kd7X10oC-@ayd$5IE7$Ml1(-70;8>wn%KXG8=PZ|gRmEoRR((u2+T!FXvoy`TOaqB37% z*VU8OePxB~f`u4I;?!I_-~nj5pb5_Jv{j)I4%1z^XNzUo3o(#|-xvSidwN4|{yu@s?#||Fzfz z_shV{ujud84IeVh#Vns{VDkj_530}ZL^Y0+Cx1schFLga< zvG7~FchENIehXYHDU|Jy&a2I!(%Gjxjo4IA&L#K1eQVHnEfU4$w>*#$~><% zqTTiIMW*@G$+y$-UafFDb1PeYV~I}7Vrd)t3WYQp1a+sx&hZquhp`iT_mK3ZVpSZHYx;wT|g>)4O!T3&FO`b;X>WlCHVXJv8NMwYO1R zNgX0m9CB*)StgYqewBp4WwHtGfjyJsEJnHhWspl+-x@97chv|J*^tkX$%6RT1Ed>} zH)tG``}vc>@s2QXiV_+3tY?Wh6ms(9Ank*rxp$4711n`|%GE3}bquPz+VHd2cy=nkDkX7}zQTVUKhngrtxshcMgIjYSAj1n zAk#=<_{ZrOHIB!3t-m?msM`eOdb4@aQBZWCX;V^AP?x7wBSBk0af4*&RoD-{1=e^M zY0(+<3VJ_^tS>6lx`WE06e_u&$%C zv#WcNiF?^IH8(eh-bh_@W1k7o?eerZlR8fnVtvf>;5|bT^PVQ4gH=^0LKJS6;OGhuu1v@RlkaHN!MZC?h1yaOc)YUt*g)AkuHpmWtiHf3=NU% zU%h$uD&`&$zWxl#RLryc*!`n=Y-|h>1tp)?;iN?TaH~y>De|FgpP*yyv#XkGRhXa0 z_P&2m1TA$OIOtGaWY44~v~FKYW})?a?Jta(LwJAZTdVfxdp>1N8Ba=>+S*Q|jcCGp zYgQ7E6sibf?z#2nxp!DYnu`-Z!Lk;`Xez8S)9%$fTXsNO`uwL-6hDjc4UbtoUe>sq z!OiRwq?nEyw$wQQZ_*?AlQ-TSPIIt~62}bKhYuFNAcpU|f07{I?m8&co*(V_BwEl4 zqy&wF6+f$13f7=}jF~^q^T^n_MW`OrGSFCqwRrp?gwdp~b;+5U&C6Crcv|>L<2TRM z8LFt9z(nI6ycH^*=YE9|J=Xy+8=#9Zcg)t#O~jDsVap}pMpikMPOt842{$E#MKP8T z{+!zZRcOg~KhDM@zG@uY+*28uFZ`S;jDOL(gTJ?KqX|+~UKYq5`VLKkWd^wAMNUna zrVHQ%Y?R0o?GFKH6xt1Vf;U#Q8g7c>S3%MNz)Mry2;Q48kqxlHR3|~j9D;A07iAVA zf`WwvEx$Y$Oe0sM^+zEe0Ez!8 za3QlN5HsGwf-AEicxEq*!At-4=usyvjeCWx@iqeaLxku&dPh=o561%9neBO5UTYIZ zf7u_B7fwh(-_H#Gbf;{45=4s^*kJmRfd4QkVc%bPf*c?Zj z(?+}V)3i_g=1c+S=`qCH4^z=Ak9LXtNaoik-hYg90f%x!9^rP%p3>UZ1`QrV_V(`X zZZIud)xUE$1dp68F$Vvj6ucDc0lD6Hci$U9nTyj1>nfjX0($R` zK7W+`EV}#Pt1b~IqiowbHp91oMtc26H#0LcpeuCXgW=Dh|0u->HBC}e?mJPV*$IG( z)qc8m2X8DSDVU~$tDgm`gN=|5u25Q>Q#702{#~ChfR@G4Hw$VHICw7jN01E+;7^YD zVLEwQP;egaDeRcA)~qPqG!Gu&Xo+5UOblo57-&)$-)x4o0`FJ@wQ}q3LoPiQX%_7k z78a1o6uRr;$8!8DXp4fGd6rtdGxr+Z#rx!ZV%2fUSnCr2CdrXis>I1dQ2&QqX{ z=e0M80c!FfU}S?70oCgi(T=a9H+t`qsy2=ij80n6;yX zMKpgUmTlEbTIG*2KmlGNQ@9rInu;|5BjJHwFo4cm)gqtG{tbKYzPv0VczErk??-qm zuL=teCYA!e&J8a|i#`sbDUg|mtT4>0xOcp#+N;{yTnjU~aXMo5O&GxF5#&St7-ek7>|>;0&j)VoC`Ew(B15XN0ByMi^->&5?g?v&gw3bq@PH) z)yBU#w0M9@3;pE^z6z0f$ZLJXEBP(>q%qGD*`1bt#o{!@X%L_`9<0FQijsj2gG8~u zZvl@SZfEKhV|HluQ5(O#jRDrq{U9v>nj@wFic{ObW&OLFG(B5FSJU;7dn9C!0o*KH zWsiO*tLLYmyMa8lOko-E`hrPxT%19kiCl`tqXU8z6chx*q@4Mi-x^-hSqu+{m&SDz zF@-8@bM5NNV>_dwqHtcHJgM(KRk*B)fJ97c09V!u{@95lVQNKu*Tza{;9Nh<;^&LI zC9<~^oGGEfW*U%`hEFu0w&24VJ(F-^mPuFIFv7&dq%m5sA$$s0J*uD2+DalrG%+)D z9`1FtS|(;@o;Z8VW)WM6^fBU*kO?7MYizt2_b%4!pki)=KqnGVf1Wb7Tj|%Z1PYNc zXGAsA(9n>N5D@og|E7k9a?p$CBa?Xy-`_279%7Ol0j5V&)`H z!$!SuQyVaE5p+-ARm(sIWw!PA;UP%9P5N8MaIK@P4wjbV%93j)X0lbr$QFX`>EEcM zO^dbvfb?21;-+runLejpo1#5-4S5*(I$?e+iG2Z=D_^;P|9*mci9%89@#rE08@-w@R+%}~}N=uV>~=7suTX6d~V-SbjjD=MB!rH)r3LzUfj@ZkJ@jV>OKp#v z#2T^BBlvyD#@MJ3rPFfd?O>tT%I$_YU2P;{z#nUjaSBNa{iAyq&$HK13jUi5K!_C@#oXkc zsExyeUMNRF_w5qa|2#WC>)(xhteX2%p_SH}|I0@$tVX%_rJ0ttbEZ@wYmUhbH-+}T z1WQ#nq}R|!)~q{v9n9o#e?GIin2d6$M)pMX2H;#+N$u?N*U6uRDalaTl5hrK-zmlC z%|e)5VWVDUw_Nt%2f|o)zu5RrSQ_k6f#N<$p85&ZH8AUI{1wTio{^*E^rw=zBy(KF z=jx=r*dn;EIimYPHzjQyr-On!m%Z61+vk@)srFDP})h3`I`?Jpp2D zGjx5{Ny}<07~|pi1Y2Yc*|vROcu>kPv9Mhg-tI*GzdyHa2`8q-3gTl+QB z_DL|cP7e2(`x$JFz!7x%WOefIU%!4d`HKwaq~&v@%pEnUw>$o$%Q8iypf9ay^@dzh zSKn3CW|C6{Z3Qg^NzbSVNxnn{ZQejk#lsL)b@g0goZtnItJZ!?Pd8R&Hq?mpMsft% z?Wn(e`UAUNTn!kmY6LK#`7?OdvnNlT($cKH&)f0DnjoM{mJL1@ZT?~|zC_TUEvzRM zl|f)b2aZm)NjIFOWdj{tt!PY1&(STEA1{1(i+vq*UIhk+v|@9=HA2a;>Q?6y8T+lTxHiUS&Se8$oJgNR>2^%HNrYBV} zIt3Exl;3PA8289i{Nq_6f~G9bJysKbFC69?5LscoC1|Fkq=Z2!g`11GcjENQ zwhL2pEArJaIsO;}AZ*%Vx81-2Htt`PAf5jebjeiwlt@~;;8~Z`s2p&%&@#HYRDg|K zw1xdw6AU&AcE0%FNtbF&&u$3Ct@+Wmye!ncOZH$}2KH=n-L9RX!5*{udqQdP&}K?8 z#!hepxwbMtKB%DN%6F|1QHUlge1U7Ilx=XgXD>%V#6;AI|_An#Zb2FueRUll!jJy`99a-0od9QPiEyX#%AaA zZdFdEh&%D_)Q_ye^64^#h(hjpfd=4-`?KAU3jFwtc%IyVQ6+YobCP3*oGFbwdG2*Q z@qh=+CcAQxzgK{5hy*6JGI%2%r2nG2=SZPCoOAHhM<2IDgCpD=9vy9|WEk`$cIHv~ z8^YIYXS{sDov!V zDWQtyclezCxP)YNAx-3#m^e%orUs&FxIdIu-E_kTk9NQN1hJJ|)bC3+gG(w zqhp_ z;wlzwu*|qP3eSq76R)q(6>upqWMeaj#8@vK_Bxj%69sgBA={b zacrtaoxpfQyA97(;~`}2B7LA2LBY_lgT`@u>e z`fWRj$M89eT>`j*loxB0Oq8h(AgyQXJW=u}4||pq)4PmZhiE-!k zCwPe3aExZYp?YqN|BXcU1r(1KF{lkv51+K*SO7jUsy8f_vrpljX=#Ys2eEDa1xF$T z;F_?H4}Hb|?0TMm-qp0LUOM3;ds=KF<^JFx*_PL0a@v}9Jv8M3N%x*kvTWsU=)n}} zxf*1(&+QC=e=i%qdwjU5v8dqC;XlCW>bbj33>(*NYGaL}h3j63b0K${N(xmQ?swt; zGFQA?;U0R`a<7M^${+ML%L(oGJGGJrZ;DRiLX3JE{-A8Y5e=E@7E|Mw?MJt(P67{) z@JD{wsz@%`q5MlboMqw%t75c?p|bt~^eJsHiVnZkc=-#G=oAf9#ip*M?-!BceJDHF zkO!-^^lG0};|Mu6_~4vqx|e3ER%sozgyO~WZV_ujcGjV1mz5-f;|~8wTPxn~Alh&sS18rO^1QYg|1w3^o>kz4NsZMo|S6M?!a zllr|j`4GqoZ44!)cMaXR*1Q?RbDo$Ku{64xeDhDRj_SL0F1I}4|7!2b1F1~kwwg*Z z6Gf?}l$|I=*$Kl@gzQPEV=p@)qE$GQNMs!;N@M99hHNRN#X3@!V@qV~jIwWWzUwtJ z-z#Ipf)=$+x->iR4jl*J}Nz51;cX$P2?IllOt5$H}tFH2}%OJbqkYOL3Ws)1ATy8v^ zU^%HF?$hxYas@p_F$W{4N2S>bVcN(JA9dR>JM*}so?!*@z{)o~#&GB6#s8#lXE=<0 zn2lC4FkL|5BO`R4T~eH4d6|Dl_1d4gY!p=0)Xvw))uQRYFey|VQ`(rA(r@k^pCw|* z(5E|_Yh<8V3q*&E)o!mUv2}g*)-p*%23uRx?X=P6EEL}pmIfWv>MFr)M%1anMlD5i zsM=fhoi3U~(G_a8IIHRK`l#fR#{R!~hOPO&$w?R)37<&LLuFXpjoCeVLGJkhrwp=e zm+(zy(@UQfv3B#2Ezm>Y+I4dw(aDHfo#p364U9NVk~*#{_H*EzPowjhM$yUX(Px{$ zhHnA;{ps0K{RJW#`*=m&wKK=MY?#jm2X7H;a_CU1VrIdu?X9a`W)K4LtP-`p6*}F= zW%Qe(S_fz4T+?G}h4TD@f*yIZ)*9H!`wbF!OMLmY!Cuixz(!h659sJzzj1};&n#$j z6~9wxFB90F|7(__^hLc-Ab2AE3EQE%q(!anJ{03@lB0!`j)HAn`P%?&Tk$n6f!xj6 zUZ%0Hm)*eH{<6j$gWt1Tl#ChmG|zr?yS#3mkO{#VZGD%xQRkdwGa^+r+(7ZZ7jt^T zOjArq@}=!}rL{s9i#af}iI9^O_MYdoxoHMmmZCvrUlRYlofiWu^z9Bp;++3fGUasN z#^k%#D1pEP6v_>&3Uahg$gV4yk>SG|_p+-3*d0+%%d!1ONcL6*3;N3dgCB028KbyNjSHEuevKQ&3* zVUTud{Bf_Xq=jsqgt%g-SUNNS{DrnLj0owvYvIBAaz=~jGnC`qB5pA6GnjS9qo*Aj+xiA2cU))DFUGzQl=+ej>U46QruEwXNg0TfkM4d5y6v+eHJ3Y#9Ct+o z$RlTivb+dCJ<193H}B=-J=u`l+OqexVFL|Kml7$KlO(Ey&O)Z4cDxM(tuk*vH#Rm3 z$ULJ-;1jvpXcFc>?gQ}ZX!qesPRx<=pul|R;ap_P($CNi^G+AD+U>c}FIMZiHv~+Z zyTah1*)pEk>A9M*Wz{ZRhrS5nkkif-LWwu)kG;`mJaD2tAl(=`KB3Pb5gP5#)+Ws3 z!nPzmCORw=cbRz%9F%=?m8lY4i1)8~8z@WaG((yY zy7x%z1k8v|gM&kL@yW4cyTsDD0N>o$nHU=@OxI7jRPfE?8XCmBaq()t9jD^j2_)&w z$wzI`-%`1-)!6mC3JsA&;^s7n-Dh(HlMrKPH1QZx6D3bP^=<#<=bIGE=TB`!37YlQ zZDIO1?Zyl+0yp|z8p+bTKli1iYG*>Gy}$LL$f2)u{c|e~yw;DLbrab(fmt@Rpba}s z=RWvhnjz^;N3yJ(-ptztj;cV(^H4VU22_(coR99Aui}WEHbQQ&Xo~)@b?S=DIh)}u zS68)QVM!by|=Ly?3(Qgt1#D ze0`-WueXE(qF3i1(FldasGC3DWII65bE?d7+e)ZVU zPq|l3KjBzDc%$IN^=)mr8Q?^nl{WqwoVdV*9y28YXenx%V?d>Fkp;) z2-}O;tBBa!8MZv-i{yELzkNiT(YW#Iv(%+!HcH*5?MCLTMv{LJ7ES-Zr_D8r&iliH%nbT<7C;bx+`9i|!p>eG#A}s5!b-iL(gjxf(fgLg^Ovp(t)p=6kn-N+nuN+o$0&XP4;R zR&+5^X*6`F1&k=%m5a{Ic+O`W>@PwAeXs{a+YL3Rv2}NP7Gi=WGeg(QM2h*x~8!30Gx@B)6R74 znM`XjGBQdBG>tTpC+rBOODzG+wbIftQhn80V@i4!_ETFC4v7Xg&d-FTO15XMV=mPzPqNRc@JgZ10-IPE`H2Ed5t;s+u)_QgTB0MK_V2qz~rnS-};)}9l z(4e|NDF1-s0~uq2#3pE^0|$!UU10A`l^V+eqUY)5CHe5((vBTaANB&fEk)rO_@EI0 z3L=ytFt$LXgT)0-*Xp?EAR*J+H}CCYBxV9L$T@&#z(I)KUpON@M4k^i>dHpxK6nN+ zia$T4=T+!aI|e52$Y)V7D%;-zBt(nGkOjtVD`wKl4vic)#$ zcsAl<6YBFKUX>4rw@339Nld*h1Rk#Y#01+Z@Jb0Bpoa|}7i-dL;sDoI|LKr+Wy{+G z+bKFkO}GJ>+fT4s9rp6^DafLkYHKES?G(`k_+V!))ozI!hhVz8W{5EG9qzDr6pJJ- zGJ6QA`YVo7V8K8EI)N2I@>qz`hQJ&hBn9Y&AYQi6N3aJ4?;}-a?|l(P>RsU8tv>Hv z_3Bl;oJZQqcOXOCVujbe+EBuhq;#5=%{O~{v(BxU6Kw^*;wr#y0PbNw&f01kXXT&C z=*^9h5~C4E!?~F)d3kw4`be_j3Pj!L;_z@$%HC>YB6bSi7xjR;5F6kG z$$*V4eu900aCmi$VYju6M|~iU;UgLvS(dn=TOdGs=lRL|E7z5{w39+^rKYNCSb}?a zYLn;Kz~9lqW(HRh@MPMPwoR=d8i7dO3AqzPk?}3VZfY+hZ3)>W+?%RmN4rmW;f-ni zr=CQVFB}vA+jV^U*hA^coPadnc8`$Ki@U^E>BAji6y^@Q^h4G^K{#VVm%VeAz@Sy>KwPq~MaMR~y5(h_`2aFT!P(Uo}nr)*%c*t$o5Ej+VTn z&KE^$FeSBo@n){ULmiJ^7l5Y~n3brQMiF`SrrPDur<%rkdUYmdmUcmWIx(Vr;yDu& z*b;*4^9v=aV;nM;ryIi?8XPfN!n1Niwz5UV#fe_?78m<$A(J(5N)wo)?jf6FLwll$Ba z4)N`s;8#O8YOsZfBWwaUM~gM*$e;n5lVB>E3-7Qr^J@_4Ti&o5Yc!w?*gl%b3GVL^&zf!GXYgY&{m!nIc!Pyww}Q}!;1NKEYL znIXb{PhL%|fDG^NySu1cnni9TPy%tQo*l-0H0#u$YzY~=^*3W)vu(67(+=y8Ci+US zRT56aEn;vAAn-2V`_`p?)JIlSvZh#c=rW4(8e=LNL{7Qj1<;SDMg0E?7W(SsQ(Mf zjV}L>*?`e;;%cW-c0XuRIu+3?@!Mc^h`t-4n`ZrX!!H;?Sd$XRIqddt1K`^_(`-Hc zer7<0^e_*dbvV$0FtmpKI|;@MfWt=sXKQ^VlSG-#qT{GN!5rRJW`aHk;o7dbW3&Zi>`zF@$aB~lQTTm$w?EoxH;fIJWV^Qu|ECqaSBR&Z| zi+DcbYy~nODGhpOh1pCOUV0HQxun4OZlx4LV+i>Lg=?-T-QEtjj5ujSm32JeoJP>hC%XZeQ~&7)stC8_b3T^y{OG2aRG&895)Was z6i*KC56i%G1`!=fHZOdjccDj2AuS$s%~9E-BA=~AE$NRyOA9A^5T85fMl@c1yH({2 zG~VGdJ9L`1R>~O(?EF=e`eL~N388?42nD-D)qL41dB@#UchM#9PIaJ+gnXueMA%-f z4~DvAs-PrnE)-W?zZgQks1vp!o^Pzt9#zqi0++U+`x;74LQ3U{fJbD<7bOI^-Hz%8 zwtH@nN3BpfR?7<=JFhw#lkIDwJ&x$~;bd3ge*mN3w$25xlq$gEKU}$EI0Pbnrfq(v zU3TMFF>+NU@4#o)tPvEJFa*tL{rLt#@RT8)5gfx+S+6c$b%7r|<}D;fRZAvDIgPaK zY;C>a|2KJj^T-Jain5g;^&8wQdTKjPu6eTcdCSb|W#Q*KgI&?_hAQ;qcykxJGnfs# z{XBYu$-W$1zm4<$cla%kYn*@V2Yo-1<#BYDIoJ^$K;lj=YE+T$_O2tO8R#_CmvX;9 z7qB#f%2XEcRNb~4$^CHA;*c&~wi&TS@f>Lll71fi{?SUK2T2-KR`JgKCLQ8+x6dy= z1gAR`Si>xvG%+!(@s>IMgHJryR`O=~H6D?K>ZBxc1uCB(Bvc>&AMK0>xC0)%*F9OL zflH`tB7k2i%Mw;xeRZnAGD1Lvq*bzlTSCNu@q{9qJ+T)5pK}D@K-BU7-XI!6{~bXr z&wuCO|HmRZedT&23(NA!0~%_^|E}o&{GO;uC5(7}9A$-+FG>|D+zrF#qh7iXcojTc zvP~K2*E&dUzqe{B=QQCRGn8A7u>Nd9_R4NTUaE;C!|K_R)LHkXX)P1`%D(U{<=ig? zG3T^b=4Z2fvcBibd#jgn5<=yQeYr1@p8iEa=QHSY?VM^9cKedkeQ7P>y*fJS#7n;t z^SwqxofdI}+*{5=!YVNk$;b0649>|7!B-QIr$nJn^&Mf)SQK5>BU%LP6qH+r{v1L~ z3)Sgw%1_P*66}?A&42$L@#lA%_)JOC@*n?r;h*mj^?CHoUoY|7UnGglW%u^!uRlVS zI=1D{cmMHDzbFLXaDyiz|GXpU`;tHXd4qqv^Km$0msP|nmEUE?f4Q$ij5Dl%`S0&P zS`o5Gvhim^_xktCog?=v{{HxX`O9a1L_nU~7xe%6a+W3Bid|m~+_`f+Xoo4=un7Wk literal 0 HcmV?d00001 diff --git a/docs/.gitbook/assets/image (3) (1).png b/docs/.gitbook/assets/image (3) (1).png new file mode 100644 index 0000000000000000000000000000000000000000..2442410112fc9413bf4969dddf296cb06f201d3a GIT binary patch literal 17434 zcmeIa2{e@b|1fMzX%&SQTO`R|b_S)0P*L_JOJWSkzLUEm%Y>rrcZ#f&eK%8CLKzBE zGq&uE!C>se^Bu*wfB$EB&v~Ecyze>Bede5To$GsDpYLa1zb5Ruj{1SU$M@3E&>Xm` zp>~sohJK8OhBk5!Be?T$mkB=&4G+y#HC25tT0&3wi$S&T;+A$a$5a@f(b`b{ID*eJ z+C=cNLB-PsGX5pLkrkX!R;n&zVN;gDoqWErRMg*_KfY4p#{JK8m(c zPwUp63%dSBbH^(G*r!ZenmudhcvL+vQUx|+NXt|?89@_lN6WNg9a4a0+Loq9+i5$7 zJsNGv`>29-YiI5 zJVmMZ+Jb`^%pP0`3Sy`UvCL(n61)YXbq5pVnV+R;weA7mp3D(a!P~mG1)`^ds*ftW z=~&#Fj?vPw%pZJMAh1o{HV6fF>b)@)m4~8oeBirRRlJLV%Dz7!+(zlB_c#G>esf4+ zPeTIL1hzm-nNshi(F5LWa>!@*2SZjWL$^TuKV$NWWz$cYKEJtYv~K2iOC?N?RIseT z`YP)&s??ah01y2|p!RVaYjfyVb=M#P%fs=yeF3Y4?~8qb8}sYr)l4|Ivh-=ig|`gk zf^{W~sA`}J!|l^l!JXn!RX%TRtktF!GZ%vjTwfm5<%aml7G?O~-XN6qrMx~w#Ng*! z#E^=CWm2WKr&~+qVZhzeTdeck0ZYG5|NQz!h8wv$Rh5Z`tWKW3 z3LhG60kM?EN+P}ME8)xjgd?Ob_Xj;*YXCAka(ex{u5wKWCN2@d_cIbl z9ITovne^f!bo;QQ^cB)Krh99Ar|i1e;Xm^YQ`|GO6ifB|;vp*u_z~-3Ic%+aU-59% zn*4$IOOM~~r0MibQh!9!^CaA&7R&s*bbjSj{bOM^qT!=1kCZGLtrOGA%Dyzp6FhsY zd%G8VFah;SEPcis-E$SQ>8?cHOTwu*=@Io)=7<%Hhc+z!-VG1_NrJo{V>|nIk6^1x zgX$Zi+ia($+ee2EYhB>2mO^N^@kQtAX}1+kiq$R+$VtF$iSc@-6|MY2&GrZz^^dEa z1{O8BtOjp!Sppvkg+CxI@Wy+({B?4d{Y%ysK82Vf0zO}F2`wC;e|#-t(81Fex^i>% z_1cA}3zwA$3+)XvEdlyS$J9pq;0q?9>}-dr9^rjD%JtNFB~uws%`|~flUQVep*o#$ zreHe%6*SX*?Wg!kDPpccDqg+-W`@wVjA!YubR3^>)&7Q?#UEX&V5sfg#f<#RA`|;q zUOCmSr@gV_EcSVZ9{0^JFqxIL3pj~h!Gvu|(**CLsG8Bw3ML=^aFTB)Yyc-P<{IV+ zKs(?>JNV^de#w75jgrcWfUmlKKM1$+F_S`{|XrY{40&xzzv9*8F@h1Rd(YTgr1A85D5$6!}#rYa6GIv5mk!d|&D-6voW$n}*q+Dm?) z#TVm|XA8#fZ53GDcZKT-X2KELdk2Hr(JB&(?Lhdei-pvxL zSnn7#U7e^Tg#=-{$_YvVN)Q&0@5^-cvj_iHKe|Khi-FrdKld`>uZBeg-v_6Sky>@g ztz3WY>K)y);jYw?Xf-j*Y3A7E17) z7-DCzM#G@fqcwJ?_KkAi{-i@%4zKtiXzP)KF8{#z+9upOr|8dJLS+IaB(`}G?)^4} zN12?HM_Br?i1J5kF=ttZbIZu?3wgs|OPuiw?T=Ud$y&IAlG?Ij^Qo#Z**k1IY|#@$ zs?=9^I@LV1URU2<>td)0n64ya?!q`rzDTPk117)`W^>3-b8&MkgD$Yu0r&Yc0gr0P zrxz*~v|mWbAOfzxUzq+e(#+vf%RoMO*kA{L*+6WzyK4p4Y#S_HxRy2$-x8nUQ&L0b zS}>(Mc@@^n#Z~sW5npjOt{(Ypb%{8bbIYMet8Bo|jDSo@gj(nyDa%p6i0s-hlu;Jk zbN;(@uwpISz@y7M1UM-OoN8QEbuQ^0gVJ$kgHPwf5~>a;#r#<7!xI+K=@D;%relcE zF;_j9x-Yxxs_IAm*A@qGRkRMPSO#S8WoB;XFWPsW=@2%8Q*jMzne0WJgORUy#5==H z{=DiOD%93N3r~2Hd=z}1*Ic;#F~J~(k2aEDKHq9|LkRvxDt68g(pj58Q|5kI(hQ+E zy1{{-9qfxA-cW|mjygR;TyY8aUbi6qfHZ#@lACZ~U8;n29K|(Q2;fWnD-?0;aFhMm z0ADgfWh(xW*PX*VM5M6yw=U7iy`V`o>UL#G4r@1l$YtBu`Kw$q1$Ry93QJ!vHiclU z*=C)r@e(huH{4iVF{FskG8oKk4waW2Z-osMkeN@FY2)u%kPnxe_Cg+i=r&_4fuJ|e zT;K1xU+$>@HJPx~QF*wIT(kLJq;Z5VItrP^;Ht-K`&N*zrG>v(IK}^XlO2C{^w^b; zTyUbi(=lg+Cg!5Utl4N8I$b2%1~WUjEJu!#9Pn5&zEW+jmGp_fEF3oI4K>5XhuQKE z7py0!;tUWQM>QHNEbl;ekhu!T+}+Z8D)_6K|4#5_{%mb6YWR{0G6+=*U~);OC*zovCvV3fMn|E_ z9i^_ZNBaI&k-M@GDX)2E2lH-%DO+UY_Aq zLZ`O37e5QkL8Rw~c(65i>WuZ9S5#Y>5&&t-?SpSi7(77y=o6XB4&8R_s5bz;GB@=P z3{j_fR8u3O@3L$&y*v?QDTZ>-yI6i`Qy5pCbBdaz#sc(uzH#nimHn5%|Jx*}cxKGj za*8xw7xZDQ5oPmUJa${3XoCHBUC~vWw~(CT@bce3dBXj<&bH=LE{f=0y=l$2q*imk z_4_>%=wYC>6t!fcsA2FeU?4)#N2&HfSx2n{-6KbKZY%%qi`%>S$8yweZ>#U`i`9J^ zZHqFu>9@^j^TP1xvor*Y5w*hGBw$hu7*U_$0jlXxHnzN_@~~FZW*1fWDeM0tyd{+X zQyji;61 zuPN;D3%K%-3!kt>!5?`CCo`-&Uy4~D)p0);N7d*}g;~<=?|BS>N&FXJd;KbSRpIW} z{;4!Us8%|yvWiAY)_*D<&;8xOZ*ko{y#Y@+)As&TrI;yiRMqrIP$Sl+=}ZZVyC+uR zaMR`V`j`JnX)9(sUO4cFT{ii^<1ewc9t7^!> z3M!pDx#cC)r@XM{W`#mY@Q6~w56=uUZByu99iUrv4<7u4f$s%%jl-CFM1t*mev9~JYHc0uwe08AtuwcA)g63o zK$R~SX!`u_>;-Bp-8xI%S{L~1-(9@ahRcZm*MjJJivnFJ@{z!dRi-4J%u%nIXBuyI zh>>BP^I4zRuJZg0P@Uq(an8t^vFXgzfXe&_CeuPQl&%pOG=p?40+wwB!o=XIpgr}w zvGwcI6EmE+&o34?yB(c7_)*!8qw^XAnk+6|bI!HP8Mw7``A3j}8|&j`eF6B=NMET* z0K=r4|<&%gyWMyS^%>|>j^q2Nk9#8F*^wz_R>O{^9 zCnuz-3~t=3c;=MMmkJM&Y;#q(+Em^(;;#>(Om`+fUE7OXnaGKMRR_oeCx(=yv5^g?g}xk6EQoHGX-(@tx@-ZvX-1r>1L|v0Cg)4?pnuJpau6N{!IV^ z!C=FgPB_H0QhtU`X*($%yJWeg@7JoOF&$g}Z2xLGuBdzrH4)T!eI8<&%k$A85osAiv`) z3zJ`{{FqAWTak!d*`sZ&x~=P$iov4{<6ZRRn&$92;n+}DDs*2!6(1S+Ggc$M1^=oSl*Ma@&klxaoiz zM9aSTYv^o#%3aB$Oosd^#B|$=R{c}Q@ts48G5yuqCtFU9uT2)A&71Ej>xZhztrtd> zzfPJf6p8-aS?6-L02(_}>14*{A1R++>t&AP#Hm#kX}7OytQA=hIq;Z(%ZXyI91;hz z{RnfXhwG*L{1```hJEyuoP`kUjv3w(u1p1eBJtsgs`;MYa64#;OW$Q66_QKn^2hOguMZlF3~q zh!yhd>4r`#hdu-WZp!8UMVoXfDocK()OToY661B}I-Kf8=Xu3#9-7CFRJ@U8eVZ^} z5_X&aPV{7bpXsR+!!|M8Ol=*{!`e+vN(IJB^0b(II%MJ*)VYh4T)E|KxE)?A7?|fP zEE=+sb7i;&KSeAVtME@!I@FGZW-WOJjkt9TX~4Uev)%kM-1cWw1v$DU9x@dA;PnAl zZkA%g_%HW`od0Bxtm8v20~nt-ZpDDL6ttRdjh386%;ulAaUJep@D++i*WDB4&BdWR z56`8;jcya1g>#zbO_d7FYy7K3((@EnVn{F-$1*e~x?8SC`8rWKq|+qhaO( z%6g>X!ivAd;~NLR6%XmL&Sy#6RFvq-Au=0%&iRsWtzj4C8uQ2+)2_{7YKoAhd(*9< z(_hh?>?Sx@1gWquq+X?m8tB_1QO}rsOufI_41MEXmOsN=Ev?#@Z~?<#O{Pf~HPLK? zn)qr9yY|&v>qxmMpDAc1;H~2V?d76#G5)1)jTiG4+tEdRJ!)qT@NOp$;x8#M$wwEZ zqk9tgJ2zZwZeOX{Q+&BYeuTb1Ve1`b)%&$+A z8~PU5()G{AYkOC^vP#ABZ(_3IDm?7egg2Im);(*(-tsyG_?YWD6d8ZuZi?`W^F>XU z?CgB2!gP*P6K_lLU|!bw&kr3W%iQ)oFhiJ=^O>AYnNkw0lFJhk4Lta0vu%f@Uvbng zzR1i-7($=Mw6_}TR++<`t8(*-dQ+4-dkvK#%%KZcaOu6S&QIWnjyw4BU9~vJr}RCC z%v+6ZTLrhe+LWmcm3K4&v;odqZmZfQCmicQVr~a<89bmPs&uf+9jg&tnk*YNy`Qho1LRU-9_>Bhr z)q%axT=td$1UbsBn0G`y<_P!kYdt=kI2+)&6%jDgStH{(e|zq9XW|Ey(sL7E)-T5- z8ARd;qTZZ@@H+X4>{Z^?V_bKRz_60G-ks-6)7M0X)nZa$O$4k@W>li1^Smw81owcO zB#&M1v*2#u=}h+v+<(VFz7ZDg!xRB#<=nz6`0eqUiYKUc4oiyp^S=i}p_*l_z+1Q*m4a zy%}DYg1y0q!BWWOK&m{)K(iNeapWzWK^oLJRpIF4YFinf_)9>}r2(s>bbMfGclbKh z*xHi8(qCV+xS1R&rvTVfrM7#emg0b%$945W3|GL?AvQAAd!;FWr9H3WAuK;ACm2K_ zQ0A>P9Ayc}IS>|pYtO#|1KRWd8-an8zh?iJ8b~sSI0u~mwLYLcC7k_3;^qY6xz?p$*pT&D8|8e)rvOQzzATWNCyk0p)lNAoUqI)*4g9tgJ!!v6f9?fUo9(xcAJID*btFz2qi zQBIem+JC~x*bNlT`^K8r+F-Alb5l{<2;K)C|CdL_K~|po%(TR19tF+wTM@7f;86YI z%>xiD@0)TR2mseDX1vvM{GpAaxi4e93vjbv2%skfIkCmqs56!{Ue*2?^$dF=czGc3f>ct> zAA!xBvZPtM_=^butYl$Mt-^91zw`&c9~J=M@3Brl3joL>h1(LxL16%}KRI)pVNbO^ zSkj(0Pu}81HJdn%AI$0?fat@_oa*~yA9kx6`Ej+MJnptqvAjCyCSPx%plqH<$vnXL zcVNyxX?xf^cBOxCe10|%J=grD`_FJ7colGH(dG8OKO6#iH8)zL&Dw0UN8hck;#UhV zGTzE#dUj3;{tI|b%NE#kC+BI0c>DddbRht6$mG2q{y$j0LB-(&dwNrhxd6xSD+B5R zx*WEW7B}9R;m4F7d~bdJw>?1GtU%hhU)wpzp5P4=jc?YkHtm6nj|ZaMV3zkM@d|#z zRBZxK{vRz0_gp#Fve}TI;Q+fzOgU`nxS~E76=;@~DZ9nLO9FyJ(+w2SZ_U3zh1WqL zhkKHz4P=IuKrO1eMUHQ)h3@Znr$_T_K;GyG)U!;}2)-r$#|#scmjf>;+&0+5EL#oC zZprK&{T9k|zn7KKtypmDhn|XGub#ort<{6OHea5;zuTULmhRKuQ1quMAI?85#4Bu} z6h1;`52sBt?3owhfh<A)$+Chkq=!$n`k$oa^b?cHjf>+}uaj(DPIH}#>~h4zf!89$1H zQey1&`k`=b5lbE@S~fu!Ugb<69m9T_)&2bgMT_82z<_Hi(I^KjzE2FIO^a2g1=e2m z=ts_8w$vIKCSWv(z3I~6YzK-6-b!9$Vg}Cdyhk{?H<1A-tAB%C9?(aW6L?E!ona3L z%@mszLdei>p_g9;*nv<4meP}KWEH1zn?0if93A?H2*lT!g$Ib>SZWC+^{Hawppjo@ z1-`S~B(x^mlK9gX%z=r^+2Rh?KnN#w0m)6^%p*MtVqLaHu$^C*5LlcP*e2MCcOm=X z7Qs0m-$hbPMv+QzM9;qj|F0#y8?wAXY~)6-Zn()Sk-><;PRnoZtBe>fwWi=eC2Qzl zKN2~3xv=b~6|thqGmP=DS47a#nsi7tWuRrLiK3Zd9K~)AMwT`QBdX_G#V$S!sQqcu z`;q3pkQ&4-WvP1Rad^46eK#2#*JaZz4OIyQe)p1fK{G{a20+#)lAb_co9MhomztP2 zz8I^?;0OhW3fIS{ksATGF8jS_c5m$K2$CudgJs5Dz9XU1O0YQU)bNX;xxfd*^QO>< zmgxP<>_|c8$s#)?1}LpaLuF)guHQMvEzh&95f#69rdG+%FMLe0>RXx$ZF~| z@&4auWz3$FCn_Rk2=6~iGlio6RF|8Ek*E{da;}r1rtI8#@0&=Q_v{9rt$mqRX4eDH zY0Vk_6u}BEnNraG`XbEnWLbP&*`wFom19L{h;Q$-bC&L*UH*J8!qKCTb@7yVS`JK2 zM%KGeL6veaIb~pBz9g#@AM?TdV36~xG-vxoW2A7OI-gz1b+z6}EoOA2z{ed(PUJ2Od84`uS!ljrzK4KTRh3k=@t4 zu$jF4w8sAP0W@cw1L45~aOL$SNw1bz66LVAL)sBsUFqgTX@#g5c>V0|Wa7)Y!jv(u zu4H#V(DfF>cXAi!D1W=Kn=PdL-eNqMihYX{mciEAGAEFm2(uC^Lsbc`v!iHTOrrX+ zUFN_wi=AxnyBt{`D-t%d!>Wt6?K+;=0uLBvLEa@9w?b1l3QCTSlejcOCO8I zSbdF?d>tiSkze4IeV`<;!jE98=Z|e2E%b``@yd3@+AI)yGD^*ND%;18`I)VlQfu-) z2j46H=H0!1{ohrAi;EDQ@%wg?^p|H&0Oms{Hq9QI>@68ZYq5XtU}Nv88)(|=ifHbj zaCc};flhJOLAF3}nts$tH^%KvZo0G7Y_A_7l*Gp9|?Pq9D%72w$iKXH|2ttL>O}U8;1Y9VIFtGI-pCBJ4ZK}8|`YOcsD7s zQLsov1{3KsXZmcoewn8XK1>XDP`I3nmM#-5aPk#>w*WKZL+X5r8i>uHWOXPVI`o;= zXkDMOF<g(`69`^FY%T87EysCp<>Hb=JW@Xw#MeA7~?u*PNeDD^rn;#wbFXmZz zj5`Mrt3C@|s}NumJ{=Duk@k8fepa_S?&2mCcUP`JyB3`>Fk$>9>cNB6w?0-i6S1UG zYcuK2C{t&x2g&%6ig()Yp=NnTPAA|lQUUhfW}bDU6@7j8tG`Q8lAI+FU}qTZRuabI zV#iuk9ALfrtP|3DEw1LAYsDcQB_GUTiyf9?IB}Vv9Cth4fdcc+6rr;G6KLepK)78_ zX-X0|AyiU(b??wR8)wyFW?%-(#{&ndTH*r)ja!Vgn4w=B2MHJTl#QNG7fJWlYhoHE zjc*VPfPUZ$7oK{W$>1s%RH2S~IP;cwbN-Fh%}I-ji{SMw$dbiCp2^Fe!E1UTp7!9*dG<$y-I2v4{=4m_ERPQaBa=fnd zKpdkn=HNmX_BuWvmQA{Q%!C^%N38C>Y1|TuV5vhYJym@(E=^az=oLK)D>d{pmo0{_ zoo$@biYdQ?Kmwrny}I`Uf~b9U6q7^bX8(Y z5j9RYb7YHo+1J1VfjiD53#6!%^Lgy}<{6G^khs-C4y7n%9<<3|gxRqh*4oKRh2{)* z2H#c?tn_#vPyet&x}y@BII+l96p;rnN%G?&KUJ$SAI~1XTqt@&H?M^|d|WGNX6SDa z$3Olu9n?Ls$|mhw^6unfV$7;3GLl59nk>-i)jo>vM0N)V<(r=En0rS>G+^mZ#i%%_@5!IH?J1qR+kSH{~eR)+8uyy+VroIX@SNlxl^e zV~HI+`Ef?OjR|RE*blwNCmec8w4va?UC{nqVP5+Ou!m>|N}Eci;VCAVd-&1yUggPm zchZAE`XP`@%#8Hkmsn%0AGK=GQrh}%G-!#(WnAWbVQtorS~dl!5_B|S;`@TNy-~gi zKEVN+@7ZKsHzA&|((evt1(m(c(%#*U5uCRv#bz))NDO-WC(U+CLPIk0I)|IQV3zv3 zjoYK-%@lr^KriL_=JzTgLe&b*5e>yiIt$#;a;3`DG<{=HS(h8|O7HWw(V70@U2=ag z!bf5G=El}OH52Un0)sC2Tmb|5epKZ z&zp%pTb*=&DL$(DM|fx{CLfW^$iMe>1{U6f9+Q%8kR}wf6{Y(xLz1zOmK<(|x)p*x z4kwy}nEMo7PWGG5TE|4y;N`8BEBd&Pg#SMH<#_}&DL%Za6bRD`n>u}KsL9KonQd6~ zKyeqNFO*;o{bbrBA^RoRY;Di=>Xbw)qqUDN3gS_v^=nJKBMl>IZA%tnVhF&+XUrz~bk&${uWKy3J_V_yy(8Pp`2KICM&>(|dQ9b1ky$vh4n zjaduTy`gJeHxD64^SMqA*jjx{J9i;breFwERHHKiZR}GZ;mAbA=V^IGCC5K z{Ra1{AG`~#%4B@u&chLsv!;yt*($Hn>b=ZCpI_18IJVgnk!6ozs_!3XDPe; zYk|%h{&hE`IjXd-lhh6AGOM~63!9S_c^0>i`0@2BD6qdjnV9STlkk+kYiRc3R&xyF zU@Z{h8Wg&(?Sor!IF(Q!JyP#;*z^2ygscUQv&E|Bnkn>J>x0+p#unNGc=^NJ={*u+ zR?g(MG=d-OQdO?MnX+RZ>8*Cs2}zG@^}mb_WYtpBb=yY;`KB~9QU&cMOgl>L>oE3y zeavs9Da3pI1&ZARt@G~JZ|S@i3yBk%!>V3&Vhu@rn{_ZCqbuj?gSc1jogSmr##cD^ zjFb{zemCxspvl2^NKVX2%b{njXW|SJCsqh^97non9i66$iopXbp@wf-F%(D@RyJ@Mq>FnNZhCEiNACeh9TC#amU@F~c!tQ5I$uElK( z7U?=3KHTBC3`VyKGf2aN1PU5Ux@WNeX*xUu4;(<%zG*(Rp@Y3p<+#C_|4x#G8xcC zJvls?pUQBXLL`?3Lgc!ix!he50dded^@GWlC;_h?#RvM5%|FeQnQ}oK@?bBoAAMon zWkr_`8Voz@sPv>BQY`8zkhc?PZam!aRQ%%`I7%1S3z+ZrQ4|I9Hd9RuoS zQ*3r(A|R@6qh!xf!bw5t%+9Ag*ikz+`xhm@|4+Cyt^}0=KNQeJ`CWj2h4`m~zc6e|+&9A0Q zyRIl7Bv}Z1?5~C}avguJPd#G$XD0C<2j3~a{7>8d-wR4){-1F}{OrsI?Sl8U{sO4X@eE8o0r3@a9 literal 0 HcmV?d00001 diff --git a/docs/SUMMARY.md b/docs/SUMMARY.md index f6139c90212..aaea553e95a 100644 --- a/docs/SUMMARY.md +++ b/docs/SUMMARY.md @@ -42,6 +42,7 @@ * [Serving gRPC API](https://api.docs.feast.dev/grpc/feast.serving.pb.html) * [gRPC Types](https://api.docs.feast.dev/grpc/feast.types.pb.html) * [Go SDK](https://godoc.org/github.com/feast-dev/feast/sdk/go) + * [Java SDK](https://javadoc.io/doc/dev.feast/feast-sdk) * [Python SDK](https://api.docs.feast.dev/python/) ## Contributing diff --git a/docs/getting-started/connecting-to-feast-1/README.md b/docs/getting-started/connecting-to-feast-1/README.md index 7cc7e5d8c5b..fd8fe8e4982 100644 --- a/docs/getting-started/connecting-to-feast-1/README.md +++ b/docs/getting-started/connecting-to-feast-1/README.md @@ -17,5 +17,11 @@ {% page-ref page="connecting-to-feast.md" %} +## Online Serving Clients +Feast provides three ways to retrieve data in production for online serving + +* [Python SDK](https://api.docs.feast.dev/python/) \(as above\) +* [Go Client library](https://godoc.org/github.com/feast-dev/feast/sdk/go) +* [Java Client library](https://javadoc.io/doc/dev.feast/feast-sdk) From 85e8b1de9b676c775e8d00f895bf2188d89d50fc Mon Sep 17 00:00:00 2001 From: Willem Pienaar Date: Sun, 24 May 2020 10:56:52 +0000 Subject: [PATCH 017/133] GitBook: [master] 28 pages modified --- docs/reference/api/README.md | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/docs/reference/api/README.md b/docs/reference/api/README.md index c30736dcc2d..86a0576e15f 100644 --- a/docs/reference/api/README.md +++ b/docs/reference/api/README.md @@ -5,6 +5,7 @@ The following API docs are available * [Feast Core gRPC API](https://api.docs.feast.dev/grpc/feast.core.pb.html): This is the gRPC API used by Feast Core. Feast Core has a dual function of schema registry and job manager. This API contains RPCs for creating and managing feature sets, stores, projects, and jobs. * [Feast Serving gRPC API](https://api.docs.feast.dev/grpc/feast.serving.pb.html): This is the gRPC API used by Feast Serving. It contains RPCs used for the retrieval of online feature data or historical feature data. * [Feast gRPC Types](https://api.docs.feast.dev/grpc/feast.types.pb.html): These are the gRPC types used by both Feast Core, Feast Serving, and the Go, Java, and Python clients. -* [Go Client SDK](https://godoc.org/github.com/feast-dev/feast/sdk/go): This SDK is used for the retrieval of online features from Feast. +* [Go Client SDK](https://godoc.org/github.com/feast-dev/feast/sdk/go): The Go library used for the retrieval of online features from Feast. +* [Java Client SDK](https://javadoc.io/doc/dev.feast/feast-sdk): The Java library used for the retrieval of online features from Feast. * [Python SDK](https://api.docs.feast.dev/python/): This is the complete reference to the Feast Python SDK. The SDK is used to manage feature sets, features, jobs, projects, and entities. It can also be used to retrieve training datasets or online features from Feast Serving. From d5f6c7c447097026b103cfb854f47c1d7339eb18 Mon Sep 17 00:00:00 2001 From: Zhi Ling Date: Tue, 26 May 2020 02:59:16 +0000 Subject: [PATCH 018/133] GitBook: [master] 28 pages modified --- docs/user-guide/feature-sets.md | 30 ++++++++++++++++++++++++++++++ 1 file changed, 30 insertions(+) diff --git a/docs/user-guide/feature-sets.md b/docs/user-guide/feature-sets.md index 48ad81a9e58..3d72dc1b02b 100644 --- a/docs/user-guide/feature-sets.md +++ b/docs/user-guide/feature-sets.md @@ -52,3 +52,33 @@ cust_trans_fs = FeatureSet.from_yaml("customer_transactions_feature_set.yaml") client.ingest(cust_trans_fs, customer_data) ``` +### **Making changes to Feature Sets** + +In order to facilitate the need for feature set definitions to change over time, a limited set of changes can be made to existing feature sets. + +To apply changes to a feature set: + +```python +# With existing feature set +cust_trans_fs = FeatureSet.from_yaml("customer_transactions_feature_set.yaml") + +# Add new feature, avg_basket_size +cust_trans_fs.add(Feature(name="avg_basket_size", dtype=ValueType.INT32)) + +# Apply changed feature set +client.apply(cust_trans_fs) +``` + +Permitted changes include: + +* Adding new features +* Deleting existing features \(note that features are tombstoned and remain on record, rather than removed completely; as a result, new features will not be able to take the names of these deleted features\) +* Changing features' TFX schemas +* Changing the feature set's source and max age + +Note that the following are **not** allowed: + +* Changes to project or name of the feature set. +* Changes to entities. +* Changes to names and types of existing features. + From 3ac50dc57dc2b1d45d281ce87d46244540182fd2 Mon Sep 17 00:00:00 2001 From: Willem Pienaar Date: Tue, 26 May 2020 11:05:36 +0000 Subject: [PATCH 019/133] GitBook: [master] one page modified --- docs/user-guide/features.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/user-guide/features.md b/docs/user-guide/features.md index 224ca798a54..eb9adf01d0c 100644 --- a/docs/user-guide/features.md +++ b/docs/user-guide/features.md @@ -28,7 +28,7 @@ driver = Entity("driver_id", ValueType.INT64) total_trips_24h = Feature("total_trips_24h", ValueType.INT64) # Create a feature set with a single entity and a single feature -driver_fs = FeatureSet("customer_fs", entities=[driver], features=[total_trips_24h]) +driver_fs = FeatureSet("driver_fs", entities=[driver], features=[total_trips_24h]) # Register the feature set with Feast client.apply(driver_fs) From c869e37d28722147eef73fa08198903675b828db Mon Sep 17 00:00:00 2001 From: Willem Pienaar Date: Wed, 27 May 2020 01:19:48 +0000 Subject: [PATCH 020/133] GitBook: [master] 28 pages modified --- docs/administration/troubleshooting.md | 2 +- docs/contributing/development-guide.md | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/docs/administration/troubleshooting.md b/docs/administration/troubleshooting.md index 7924a33e733..e293945acce 100644 --- a/docs/administration/troubleshooting.md +++ b/docs/administration/troubleshooting.md @@ -1,6 +1,6 @@ # Troubleshooting -If at any point in time you cannot resolve a problem, please see the [Getting Help](https://github.com/feast-dev/feast/tree/75f3b783e5a7c5e0217a3020422548fb0d0ce0bf/docs/getting-help.md) section for reaching out to the Feast community. +If at any point in time you cannot resolve a problem, please see the [Community](../getting-help.md) section for reaching out to the Feast community. ## How can I verify that all services are operational? diff --git a/docs/contributing/development-guide.md b/docs/contributing/development-guide.md index 81de60202b5..677504e987d 100644 --- a/docs/contributing/development-guide.md +++ b/docs/contributing/development-guide.md @@ -77,7 +77,7 @@ $ mvn verify ### 3.3 Running components locally -The `core` and `serving` modules are Spring Boot applications. These may be run as usual for [the Spring Boot Maven plugin](https://docs.spring.io/spring-boot/docs/current/maven-plugin/index.html): +The `core` and `serving` modules are Spring Boot applications. These may be run as usual for the Spring Boot Maven plugin: ```text $ mvn --projects core spring-boot:run From 47758e16b9804f513bec5a8d8c8b7681c83f9083 Mon Sep 17 00:00:00 2001 From: Zhu Zhan Yan Date: Wed, 27 May 2020 21:08:06 +0800 Subject: [PATCH 021/133] Fix Docker Compose Batch Serving (#744) * Fix issue where external Feast client cannot access docker-compose Kafka This issue occurs when users attempt to interact with an internal docker-compose Kafka from a extenral. When Feast Core provides the Kakfa bootstrap servers, its relative to inside the docker-compose network, and not resolvable from the outside. This adds an external alias for the internal Kafka to bootstrap servers so that external clients work correctly. * Fix batch serving listening on wrong port and incorrect active-store typo * Fix extra slash in gcs store url. --- infra/docker-compose/.env.sample | 1 - infra/docker-compose/core/core.yml | 2 +- infra/docker-compose/serving/batch-serving.yml | 11 +++++++---- 3 files changed, 8 insertions(+), 6 deletions(-) diff --git a/infra/docker-compose/.env.sample b/infra/docker-compose/.env.sample index a267f5e8577..be071be994c 100644 --- a/infra/docker-compose/.env.sample +++ b/infra/docker-compose/.env.sample @@ -13,7 +13,6 @@ FEAST_SERVING_IMAGE=gcr.io/kf-feast/feast-serving # Feast Serving - Batch (BigQuery) FEAST_BATCH_SERVING_CONFIG=batch-serving.yml FEAST_BATCH_SERVING_GCP_SERVICE_ACCOUNT_KEY=placeholder.json -FEAST_BATCH_JOB_STAGING_LOCATION=gs://your-gcs-bucket/staging # Feast Serving - Online (Redis) FEAST_ONLINE_SERVING_CONFIG=online-serving.yml diff --git a/infra/docker-compose/core/core.yml b/infra/docker-compose/core/core.yml index 99d8d6c7ddb..f54d05a36b2 100644 --- a/infra/docker-compose/core/core.yml +++ b/infra/docker-compose/core/core.yml @@ -11,4 +11,4 @@ feast: type: kafka options: topic: feast-features - bootstrapServers: kafka:9092 + bootstrapServers: "kafka:9092,localhost:9094" diff --git a/infra/docker-compose/serving/batch-serving.yml b/infra/docker-compose/serving/batch-serving.yml index 375380bcee3..c34aba277c7 100644 --- a/infra/docker-compose/serving/batch-serving.yml +++ b/infra/docker-compose/serving/batch-serving.yml @@ -1,15 +1,15 @@ feast: core-host: core - active-store: batch + active-store: historical stores: - name: historical type: BIGQUERY # Changes required for batch serving to work # Please see https://api.docs.feast.dev/grpc/feast.core.pb.html#Store for configuration options config: - project_id: my_project - dataset_id: my_dataset - staging_location: gs://mybucket/myprefix + project_id: project + dataset_id: dataset + staging_location: gs://gcs_bucket/prefix initial_retry_delay_seconds: 1 total_timeout_seconds: 21600 subscriptions: @@ -18,3 +18,6 @@ feast: job_store: redis_host: redis redis_port: 6379 + +grpc: + port: 6567 From 75400fe598ff7ac7d64d10e1e677d9c4903c4f30 Mon Sep 17 00:00:00 2001 From: Zhu Zhan Yan Date: Thu, 28 May 2020 09:32:58 +0800 Subject: [PATCH 022/133] Update Example Notebooks to Default Project and Update Feature Refrences (#743) * Update basic example jupyter notebook to v0.5 * Update Telecom customer churn prediction example jupyter notebook to v0.5 --- examples/basic/basic.ipynb | 620 +++++++++++++++--- ... Prediction (with Feast and XGBoost).ipynb | 25 +- 2 files changed, 541 insertions(+), 104 deletions(-) diff --git a/examples/basic/basic.ipynb b/examples/basic/basic.ipynb index b9e0ba9e1a4..a56121328c5 100644 --- a/examples/basic/basic.ipynb +++ b/examples/basic/basic.ipynb @@ -28,7 +28,7 @@ }, { "cell_type": "code", - "execution_count": null, + "execution_count": 1, "metadata": {}, "outputs": [], "source": [ @@ -60,9 +60,55 @@ }, { "cell_type": "code", - "execution_count": null, - "metadata": {}, - "outputs": [], + "execution_count": 2, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Requirement already satisfied: feast in /home/zzy/.conda/envs/feast-ml/lib/python3.7/site-packages (0.5.0.post0)\n", + "Requirement already satisfied: google in /home/zzy/.conda/envs/feast-ml/lib/python3.7/site-packages (from feast) (2.0.3)\n", + "Requirement already satisfied: tabulate==0.8.* in /home/zzy/.conda/envs/feast-ml/lib/python3.7/site-packages (from feast) (0.8.7)\n", + "Requirement already satisfied: pandavro==1.5.* in /home/zzy/.conda/envs/feast-ml/lib/python3.7/site-packages (from feast) (1.5.1)\n", + "Requirement already satisfied: pandas==0.* in /home/zzy/.local/lib/python3.7/site-packages (from feast) (0.25.0)\n", + "Requirement already satisfied: google-cloud-core==1.0.* in /home/zzy/.conda/envs/feast-ml/lib/python3.7/site-packages (from feast) (1.0.3)\n", + "Requirement already satisfied: grpcio==1.* in /home/zzy/.conda/envs/feast-ml/lib/python3.7/site-packages (from feast) (1.29.0)\n", + "Requirement already satisfied: fastavro<0.23,>=0.22.11 in /home/zzy/.conda/envs/feast-ml/lib/python3.7/site-packages (from feast) (0.22.13)\n", + "Requirement already satisfied: PyYAML==5.1.* in /home/zzy/.conda/envs/feast-ml/lib/python3.7/site-packages (from feast) (5.1.2)\n", + "Requirement already satisfied: googleapis-common-protos==1.* in /home/zzy/.conda/envs/feast-ml/lib/python3.7/site-packages (from feast) (1.51.0)\n", + "Requirement already satisfied: tqdm==4.* in /home/zzy/.conda/envs/feast-ml/lib/python3.7/site-packages (from feast) (4.46.0)\n", + "Requirement already satisfied: numpy in /home/zzy/.local/lib/python3.7/site-packages (from feast) (1.17.4)\n", + "Requirement already satisfied: confluent-kafka in /home/zzy/.conda/envs/feast-ml/lib/python3.7/site-packages (from feast) (1.4.2)\n", + "Requirement already satisfied: google-cloud-bigquery-storage==0.7.* in /home/zzy/.conda/envs/feast-ml/lib/python3.7/site-packages (from feast) (0.7.0)\n", + "Requirement already satisfied: toml==0.10.* in /home/zzy/.conda/envs/feast-ml/lib/python3.7/site-packages (from feast) (0.10.1)\n", + "Requirement already satisfied: protobuf>=3.10 in /home/zzy/.conda/envs/feast-ml/lib/python3.7/site-packages (from feast) (3.12.1)\n", + "Requirement already satisfied: google-cloud-bigquery==1.18.* in /home/zzy/.conda/envs/feast-ml/lib/python3.7/site-packages (from feast) (1.18.1)\n", + "Requirement already satisfied: pyarrow>=0.15.1 in /home/zzy/.conda/envs/feast-ml/lib/python3.7/site-packages (from feast) (0.17.1)\n", + "Requirement already satisfied: google-api-core==1.14.* in /home/zzy/.conda/envs/feast-ml/lib/python3.7/site-packages (from feast) (1.14.3)\n", + "Requirement already satisfied: google-cloud-storage==1.20.* in /home/zzy/.conda/envs/feast-ml/lib/python3.7/site-packages (from feast) (1.20.0)\n", + "Requirement already satisfied: google-auth==1.6.* in /home/zzy/.conda/envs/feast-ml/lib/python3.7/site-packages (from feast) (1.6.3)\n", + "Requirement already satisfied: kafka-python==1.* in /home/zzy/.conda/envs/feast-ml/lib/python3.7/site-packages (from feast) (1.4.7)\n", + "Requirement already satisfied: Click==7.* in /home/zzy/.conda/envs/feast-ml/lib/python3.7/site-packages (from feast) (7.1.2)\n", + "Requirement already satisfied: beautifulsoup4 in /home/zzy/.conda/envs/feast-ml/lib/python3.7/site-packages (from google->feast) (4.9.1)\n", + "Requirement already satisfied: six>=1.9 in /home/zzy/.conda/envs/feast-ml/lib/python3.7/site-packages (from pandavro==1.5.*->feast) (1.14.0)\n", + "Requirement already satisfied: python-dateutil>=2.6.1 in /home/zzy/.local/lib/python3.7/site-packages (from pandas==0.*->feast) (2.8.0)\n", + "Requirement already satisfied: pytz>=2017.2 in /home/zzy/.local/lib/python3.7/site-packages (from pandas==0.*->feast) (2019.1)\n", + "Requirement already satisfied: setuptools in /home/zzy/.local/lib/python3.7/site-packages (from protobuf>=3.10->feast) (41.6.0)\n", + "Requirement already satisfied: google-resumable-media<0.5.0dev,>=0.3.1 in /home/zzy/.conda/envs/feast-ml/lib/python3.7/site-packages (from google-cloud-bigquery==1.18.*->feast) (0.4.1)\n", + "Requirement already satisfied: requests<3.0.0dev,>=2.18.0 in /home/zzy/.conda/envs/feast-ml/lib/python3.7/site-packages (from google-api-core==1.14.*->feast) (2.23.0)\n", + "Requirement already satisfied: rsa>=3.1.4 in /home/zzy/.conda/envs/feast-ml/lib/python3.7/site-packages (from google-auth==1.6.*->feast) (4.0)\n", + "Requirement already satisfied: cachetools>=2.0.0 in /home/zzy/.conda/envs/feast-ml/lib/python3.7/site-packages (from google-auth==1.6.*->feast) (4.1.0)\n", + "Requirement already satisfied: pyasn1-modules>=0.2.1 in /home/zzy/.conda/envs/feast-ml/lib/python3.7/site-packages (from google-auth==1.6.*->feast) (0.2.8)\n", + "Requirement already satisfied: soupsieve>1.2 in /home/zzy/.conda/envs/feast-ml/lib/python3.7/site-packages (from beautifulsoup4->google->feast) (2.0.1)\n", + "Requirement already satisfied: chardet<4,>=3.0.2 in /home/zzy/.conda/envs/feast-ml/lib/python3.7/site-packages (from requests<3.0.0dev,>=2.18.0->google-api-core==1.14.*->feast) (3.0.4)\n", + "Requirement already satisfied: idna<3,>=2.5 in /home/zzy/.conda/envs/feast-ml/lib/python3.7/site-packages (from requests<3.0.0dev,>=2.18.0->google-api-core==1.14.*->feast) (2.9)\n", + "Requirement already satisfied: urllib3!=1.25.0,!=1.25.1,<1.26,>=1.21.1 in /home/zzy/.conda/envs/feast-ml/lib/python3.7/site-packages (from requests<3.0.0dev,>=2.18.0->google-api-core==1.14.*->feast) (1.25.8)\n", + "Requirement already satisfied: certifi>=2017.4.17 in /home/zzy/.local/lib/python3.7/site-packages (from requests<3.0.0dev,>=2.18.0->google-api-core==1.14.*->feast) (2019.6.16)\n", + "Requirement already satisfied: pyasn1>=0.1.3 in /home/zzy/.conda/envs/feast-ml/lib/python3.7/site-packages (from rsa>=3.1.4->google-auth==1.6.*->feast) (0.4.8)\n" + ] + } + ], "source": [ "!pip install feast" ] @@ -76,7 +122,7 @@ }, { "cell_type": "code", - "execution_count": null, + "execution_count": 3, "metadata": {}, "outputs": [], "source": [ @@ -103,45 +149,13 @@ }, { "cell_type": "code", - "execution_count": null, + "execution_count": 4, "metadata": {}, "outputs": [], "source": [ "client = Client(core_url=FEAST_CORE_URL, serving_url=FEAST_ONLINE_SERVING_URL)" ] }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "Create a project workspace" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": {}, - "outputs": [], - "source": [ - "client.create_project('customer_project')" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "Set the active project" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": {}, - "outputs": [], - "source": [ - "client.set_project('customer_project')" - ] - }, { "cell_type": "markdown", "metadata": {}, @@ -158,7 +172,7 @@ }, { "cell_type": "code", - "execution_count": null, + "execution_count": 5, "metadata": {}, "outputs": [], "source": [ @@ -170,9 +184,142 @@ }, { "cell_type": "code", - "execution_count": null, - "metadata": {}, - "outputs": [], + "execution_count": 6, + "metadata": {}, + "outputs": [ + { + "data": { + "text/html": [ + "
\n", + "\n", + "\n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + "
datetimecustomer_iddaily_transactionstotal_transactions
02020-05-25 00:00:00+00:0010018.70480277
12020-05-25 00:00:00+00:0010027.16388731
22020-05-25 00:00:00+00:0010039.93597668
32020-05-25 00:00:00+00:0010041.10798078
42020-05-25 00:00:00+00:0010058.30738136
52020-05-26 00:00:00+00:0010012.41681114
62020-05-26 00:00:00+00:0010024.8177359
72020-05-26 00:00:00+00:0010034.40971495
82020-05-26 00:00:00+00:0010046.6173176
92020-05-26 00:00:00+00:0010051.03252586
\n", + "
" + ], + "text/plain": [ + " datetime customer_id daily_transactions \\\n", + "0 2020-05-25 00:00:00+00:00 1001 8.704802 \n", + "1 2020-05-25 00:00:00+00:00 1002 7.163887 \n", + "2 2020-05-25 00:00:00+00:00 1003 9.935976 \n", + "3 2020-05-25 00:00:00+00:00 1004 1.107980 \n", + "4 2020-05-25 00:00:00+00:00 1005 8.307381 \n", + "5 2020-05-26 00:00:00+00:00 1001 2.416811 \n", + "6 2020-05-26 00:00:00+00:00 1002 4.817735 \n", + "7 2020-05-26 00:00:00+00:00 1003 4.409714 \n", + "8 2020-05-26 00:00:00+00:00 1004 6.617317 \n", + "9 2020-05-26 00:00:00+00:00 1005 1.032525 \n", + "\n", + " total_transactions \n", + "0 77 \n", + "1 31 \n", + "2 68 \n", + "3 78 \n", + "4 36 \n", + "5 14 \n", + "6 9 \n", + "7 95 \n", + "8 6 \n", + "9 86 " + ] + }, + "execution_count": 6, + "metadata": {}, + "output_type": "execute_result" + } + ], "source": [ "customer_features = pd.DataFrame(\n", " {\n", @@ -183,7 +330,7 @@ " }\n", ")\n", "\n", - "print(customer_features.head(500))" + "customer_features.head(10)" ] }, { @@ -203,7 +350,7 @@ }, { "cell_type": "code", - "execution_count": null, + "execution_count": 7, "metadata": {}, "outputs": [], "source": [ @@ -223,9 +370,19 @@ }, { "cell_type": "code", - "execution_count": null, - "metadata": {}, - "outputs": [], + "execution_count": 8, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Feature daily_transactions (ValueType.DOUBLE) added from dataframe.\n", + "Feature total_transactions (ValueType.INT64) added from dataframe.\n", + "\n" + ] + } + ], "source": [ "customer_fs.infer_fields_from_df(customer_features, replace_existing_features=True)" ] @@ -246,19 +403,53 @@ }, { "cell_type": "code", - "execution_count": null, - "metadata": {}, - "outputs": [], - "source": [ - "client.apply(customer_fs)\n" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": {}, - "outputs": [], - "source": [ + "execution_count": 9, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Feature set created: \"customer_transactions\"\n", + "{\n", + " \"spec\": {\n", + " \"name\": \"customer_transactions\",\n", + " \"entities\": [\n", + " {\n", + " \"name\": \"customer_id\",\n", + " \"valueType\": \"INT64\"\n", + " }\n", + " ],\n", + " \"features\": [\n", + " {\n", + " \"name\": \"daily_transactions\",\n", + " \"valueType\": \"DOUBLE\"\n", + " },\n", + " {\n", + " \"name\": \"total_transactions\",\n", + " \"valueType\": \"INT64\"\n", + " }\n", + " ],\n", + " \"maxAge\": \"432000s\",\n", + " \"source\": {\n", + " \"type\": \"KAFKA\",\n", + " \"kafkaSourceConfig\": {\n", + " \"bootstrapServers\": \"kafka:9092,localhost:9094\",\n", + " \"topic\": \"feast-features\"\n", + " }\n", + " },\n", + " \"project\": \"default\"\n", + " },\n", + " \"meta\": {\n", + " \"createdTimestamp\": \"2020-05-27T03:58:07Z\",\n", + " \"status\": \"STATUS_PENDING\"\n", + " }\n", + "}\n" + ] + } + ], + "source": [ + "client.apply(customer_fs)\n", "customer_fs = client.get_feature_set(\"customer_transactions\")\n", "print(customer_fs)" ] @@ -272,9 +463,52 @@ }, { "cell_type": "code", - "execution_count": null, - "metadata": {}, - "outputs": [], + "execution_count": 10, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Waiting for feature set to be ready for ingestion...\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "100%|██████████| 15/15 [00:01<00:00, 13.99rows/s]" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Ingestion complete!\n", + "\n", + "Ingestion statistics:\n", + "Success: 15/15\n", + "Removing temporary file(s)...\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "\n" + ] + }, + { + "data": { + "text/plain": [ + "'3b988d56-6885-36c6-804e-73ea76b7eae6'" + ] + }, + "execution_count": 10, + "metadata": {}, + "output_type": "execute_result" + } + ], "source": [ "client.ingest(\"customer_transactions\", customer_features)" ] @@ -302,9 +536,37 @@ }, { "cell_type": "code", - "execution_count": null, - "metadata": {}, - "outputs": [], + "execution_count": 11, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "field_values {\n", + " fields {\n", + " key: \"customer_id\"\n", + " value {\n", + " int64_val: 1001\n", + " }\n", + " }\n", + " fields {\n", + " key: \"daily_transactions\"\n", + " value {\n", + " double_val: 2.460333315469021\n", + " }\n", + " }\n", + " fields {\n", + " key: \"total_transactions\"\n", + " value {\n", + " int64_val: 11\n", + " }\n", + " }\n", + "}\n", + "\n" + ] + } + ], "source": [ "online_features = client.get_online_features(\n", " feature_refs=[\n", @@ -355,20 +617,117 @@ }, { "cell_type": "code", - "execution_count": null, - "metadata": {}, - "outputs": [], + "execution_count": 19, + "metadata": {}, + "outputs": [ + { + "data": { + "text/html": [ + "
\n", + "\n", + "\n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + "
datetimecustomer_id
02020-05-25 00:00:00+00:001001
12020-05-25 00:00:00+00:001002
22020-05-25 00:00:00+00:001003
32020-05-25 00:00:00+00:001004
42020-05-25 00:00:00+00:001005
52020-05-26 00:00:00+00:001001
62020-05-26 00:00:00+00:001002
72020-05-26 00:00:00+00:001003
82020-05-26 00:00:00+00:001004
92020-05-26 00:00:00+00:001005
\n", + "
" + ], + "text/plain": [ + " datetime customer_id\n", + "0 2020-05-25 00:00:00+00:00 1001\n", + "1 2020-05-25 00:00:00+00:00 1002\n", + "2 2020-05-25 00:00:00+00:00 1003\n", + "3 2020-05-25 00:00:00+00:00 1004\n", + "4 2020-05-25 00:00:00+00:00 1005\n", + "5 2020-05-26 00:00:00+00:00 1001\n", + "6 2020-05-26 00:00:00+00:00 1002\n", + "7 2020-05-26 00:00:00+00:00 1003\n", + "8 2020-05-26 00:00:00+00:00 1004\n", + "9 2020-05-26 00:00:00+00:00 1005" + ] + }, + "execution_count": 19, + "metadata": {}, + "output_type": "execute_result" + } + ], "source": [ - "event_timestamps = [datetime.utcnow().replace(tzinfo=utc) - timedelta(days=randrange(15), hours=randrange(24), minutes=randrange(60)) for day in range(30)]\n", - "\n", "entity_rows = pd.DataFrame(\n", " {\n", - " \"datetime\": event_timestamps,\n", - " \"customer_id\": [customers[idx % len(customers)] for idx in range(len(event_timestamps))],\n", + " \"datetime\": [day for day in days for customer in customers],\n", + " \"customer_id\": [customer for day in days for customer in customers],\n", " }\n", ")\n", "\n", - "print(entity_rows.head(10))" + "entity_rows.head(10)" ] }, { @@ -387,12 +746,11 @@ }, { "cell_type": "code", - "execution_count": null, + "execution_count": 20, "metadata": {}, "outputs": [], "source": [ - "batch_client = Client(core_url=FEAST_CORE_URL, serving_url=FEAST_BATCH_SERVING_URL)\n", - "batch_client.set_project(\"customer_project\")" + "batch_client = Client(core_url=FEAST_CORE_URL, serving_url=FEAST_BATCH_SERVING_URL)" ] }, { @@ -404,7 +762,7 @@ }, { "cell_type": "code", - "execution_count": null, + "execution_count": 21, "metadata": { "scrolled": true }, @@ -412,8 +770,8 @@ "source": [ "job = batch_client.get_batch_features(\n", " feature_refs=[\n", - " f\"customer_project/daily_transactions\", \n", - " f\"customer_project/total_transactions\", \n", + " f\"daily_transactions\", \n", + " f\"total_transactions\", \n", " ],\n", " entity_rows=entity_rows\n", " )" @@ -428,20 +786,108 @@ }, { "cell_type": "code", - "execution_count": null, - "metadata": {}, - "outputs": [], + "execution_count": 22, + "metadata": {}, + "outputs": [ + { + "data": { + "text/html": [ + "
\n", + "\n", + "\n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + "
event_timestampcustomer_iddaily_transactionstotal_transactions
02020-05-26 00:00:00+00:0010012.41681114
12020-05-26 00:00:00+00:0010046.6173176
22020-05-26 00:00:00+00:0010034.40971495
32020-05-26 00:00:00+00:0010051.03252586
42020-05-26 00:00:00+00:0010024.8177359
\n", + "
" + ], + "text/plain": [ + " event_timestamp customer_id daily_transactions \\\n", + "0 2020-05-26 00:00:00+00:00 1001 2.416811 \n", + "1 2020-05-26 00:00:00+00:00 1004 6.617317 \n", + "2 2020-05-26 00:00:00+00:00 1003 4.409714 \n", + "3 2020-05-26 00:00:00+00:00 1005 1.032525 \n", + "4 2020-05-26 00:00:00+00:00 1002 4.817735 \n", + "\n", + " total_transactions \n", + "0 14 \n", + "1 6 \n", + "2 95 \n", + "3 86 \n", + "4 9 " + ] + }, + "execution_count": 22, + "metadata": {}, + "output_type": "execute_result" + } + ], "source": [ "df = job.to_dataframe()\n", - "print(df.head(10))" + "df.head()" ] } ], "metadata": { "kernelspec": { - "display_name": "Python 3", + "display_name": "feast-ml-py374", "language": "python", - "name": "python3" + "name": "feast-ml-py374" }, "language_info": { "codemirror_mode": { @@ -457,5 +903,5 @@ } }, "nbformat": 4, - "nbformat_minor": 2 + "nbformat_minor": 4 } diff --git a/examples/feast-xgboost-churn-prediction-tutorial/Telecom Customer Churn Prediction (with Feast and XGBoost).ipynb b/examples/feast-xgboost-churn-prediction-tutorial/Telecom Customer Churn Prediction (with Feast and XGBoost).ipynb index e88fe970d54..c29c01efffb 100644 --- a/examples/feast-xgboost-churn-prediction-tutorial/Telecom Customer Churn Prediction (with Feast and XGBoost).ipynb +++ b/examples/feast-xgboost-churn-prediction-tutorial/Telecom Customer Churn Prediction (with Feast and XGBoost).ipynb @@ -6176,8 +6176,7 @@ "source": [ "os.environ['FEAST_CORE_URL'] = 'localhost:6565'\n", "os.environ['FEAST_ONLINE_URL'] = 'localhost:6566'\n", - "os.environ['FEAST_BATCH_URL'] = 'localhost:6567'\n", - "os.environ['FEAST_PROJECT'] = 'default'" + "os.environ['FEAST_BATCH_URL'] = 'localhost:6567'" ] }, { @@ -6195,8 +6194,7 @@ "metadata": {}, "outputs": [], "source": [ - "client = Client(core_url=os.environ['FEAST_CORE_URL'])\n", - "client.set_project(os.environ['FEAST_PROJECT'])" + "client = Client(core_url=os.environ['FEAST_CORE_URL'])" ] }, { @@ -6490,7 +6488,6 @@ "name": "stderr", "output_type": "stream", "text": [ - "\r", " 0%| | 0/7032 [00:00 Date: Thu, 28 May 2020 19:32:36 +0800 Subject: [PATCH 023/133] Fix broken links in README.md --- README.md | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/README.md b/README.md index 2c2970c6d9c..d8f60e2614b 100644 --- a/README.md +++ b/README.md @@ -56,9 +56,9 @@ Please see the links below to set up Feast for batch/historical serving with Big Please refer to the official documentation at - * [Why Feast?](https://docs.feast.dev/introduction/why-feast) - * [Concepts](https://docs.feast.dev/concepts/concepts) - * [Installation](https://docs.feast.dev/installation/overview) + * [Why Feast?](https://docs.feast.dev/why-feast) + * [Concepts](https://docs.feast.dev/user-guide/overview) + * [Installation](https://docs.feast.dev/getting-started) * [Examples](https://github.com/feast-dev/feast/blob/master/examples/) * [Roadmap](https://docs.feast.dev/roadmap) * [Change Log](https://github.com/feast-dev/feast/blob/master/CHANGELOG.md) From f72cc033a8c47f4ec17af5b6a7cac4c338ab1268 Mon Sep 17 00:00:00 2001 From: Oleksii Moskalenko Date: Thu, 28 May 2020 15:46:58 +0300 Subject: [PATCH 024/133] redis-cluster is being merged into redis (#752) styling handle null case styling --- docs/coverage/java/pom.xml | 6 - ingestion/pom.xml | 6 - .../java/feast/ingestion/utils/StoreUtil.java | 3 +- serving/pom.xml | 6 - .../serving/config/ServingServiceConfig.java | 2 +- storage/connectors/pom.xml | 1 - storage/connectors/redis/pom.xml | 7 + .../RedisClusterOnlineRetriever.java | 2 +- .../writer/RedisClusterIngestionClient.java | 2 +- .../redis/writer/RedisCustomIO.java | 28 +- .../redis/writer/RedisFeatureSink.java | 49 ++- .../RedisClusterOnlineRetrieverTest.java | 2 +- .../writer/RedisClusterFeatureSinkTest.java | 10 +- storage/connectors/rediscluster/pom.xml | 81 ----- .../retriever/FeatureRowDecoder.java | 82 ----- .../writer/RedisClusterCustomIO.java | 296 ------------------ .../writer/RedisClusterFeatureSink.java | 75 ----- .../writer/RedisIngestionClient.java | 49 --- 18 files changed, 70 insertions(+), 637 deletions(-) rename storage/connectors/{rediscluster/src/main/java/feast/storage/connectors/rediscluster => redis/src/main/java/feast/storage/connectors/redis}/retriever/RedisClusterOnlineRetriever.java (99%) rename storage/connectors/{rediscluster/src/main/java/feast/storage/connectors/rediscluster => redis/src/main/java/feast/storage/connectors/redis}/writer/RedisClusterIngestionClient.java (98%) rename storage/connectors/{rediscluster/src/test/java/feast/storage/connectors/rediscluster => redis/src/test/java/feast/storage/connectors/redis}/retriever/RedisClusterOnlineRetrieverTest.java (99%) rename storage/connectors/{rediscluster/src/test/java/feast/storage/connectors/rediscluster => redis/src/test/java/feast/storage/connectors/redis}/writer/RedisClusterFeatureSinkTest.java (98%) delete mode 100644 storage/connectors/rediscluster/pom.xml delete mode 100644 storage/connectors/rediscluster/src/main/java/feast/storage/connectors/rediscluster/retriever/FeatureRowDecoder.java delete mode 100644 storage/connectors/rediscluster/src/main/java/feast/storage/connectors/rediscluster/writer/RedisClusterCustomIO.java delete mode 100644 storage/connectors/rediscluster/src/main/java/feast/storage/connectors/rediscluster/writer/RedisClusterFeatureSink.java delete mode 100644 storage/connectors/rediscluster/src/main/java/feast/storage/connectors/rediscluster/writer/RedisIngestionClient.java diff --git a/docs/coverage/java/pom.xml b/docs/coverage/java/pom.xml index 05ffaf374e0..9666b7092c7 100644 --- a/docs/coverage/java/pom.xml +++ b/docs/coverage/java/pom.xml @@ -59,12 +59,6 @@ ${project.version} - - dev.feast - feast-storage-connector-redis-cluster - ${project.version} - - dev.feast feast-ingestion diff --git a/ingestion/pom.xml b/ingestion/pom.xml index a004c6d9df6..36d5cd6269b 100644 --- a/ingestion/pom.xml +++ b/ingestion/pom.xml @@ -126,12 +126,6 @@ ${project.version} - - dev.feast - feast-storage-connector-redis-cluster - ${project.version} - - dev.feast feast-storage-connector-bigquery diff --git a/ingestion/src/main/java/feast/ingestion/utils/StoreUtil.java b/ingestion/src/main/java/feast/ingestion/utils/StoreUtil.java index 54323869cf8..4efec994f7e 100644 --- a/ingestion/src/main/java/feast/ingestion/utils/StoreUtil.java +++ b/ingestion/src/main/java/feast/ingestion/utils/StoreUtil.java @@ -26,7 +26,6 @@ import feast.storage.api.writer.FeatureSink; import feast.storage.connectors.bigquery.writer.BigQueryFeatureSink; import feast.storage.connectors.redis.writer.RedisFeatureSink; -import feast.storage.connectors.rediscluster.writer.RedisClusterFeatureSink; import java.util.HashMap; import java.util.Map; import org.slf4j.Logger; @@ -84,7 +83,7 @@ public static FeatureSink getFeatureSink( StoreType storeType = store.getType(); switch (storeType) { case REDIS_CLUSTER: - return RedisClusterFeatureSink.fromConfig(store.getRedisClusterConfig(), featureSetSpecs); + return RedisFeatureSink.fromConfig(store.getRedisClusterConfig(), featureSetSpecs); case REDIS: return RedisFeatureSink.fromConfig(store.getRedisConfig(), featureSetSpecs); case BIGQUERY: diff --git a/serving/pom.xml b/serving/pom.xml index c7a30166f6b..8304899b515 100644 --- a/serving/pom.xml +++ b/serving/pom.xml @@ -110,12 +110,6 @@ ${project.version} - - dev.feast - feast-storage-connector-redis-cluster - ${project.version} - - dev.feast feast-storage-connector-bigquery diff --git a/serving/src/main/java/feast/serving/config/ServingServiceConfig.java b/serving/src/main/java/feast/serving/config/ServingServiceConfig.java index 46ffdb2fcc8..41a92e4bc64 100644 --- a/serving/src/main/java/feast/serving/config/ServingServiceConfig.java +++ b/serving/src/main/java/feast/serving/config/ServingServiceConfig.java @@ -28,8 +28,8 @@ import feast.storage.api.retriever.HistoricalRetriever; import feast.storage.api.retriever.OnlineRetriever; import feast.storage.connectors.bigquery.retriever.BigQueryHistoricalRetriever; +import feast.storage.connectors.redis.retriever.RedisClusterOnlineRetriever; import feast.storage.connectors.redis.retriever.RedisOnlineRetriever; -import feast.storage.connectors.rediscluster.retriever.RedisClusterOnlineRetriever; import io.opentracing.Tracer; import java.util.Map; import org.slf4j.Logger; diff --git a/storage/connectors/pom.xml b/storage/connectors/pom.xml index 280b0d40bf1..6cd949acfdf 100644 --- a/storage/connectors/pom.xml +++ b/storage/connectors/pom.xml @@ -16,7 +16,6 @@ redis - rediscluster bigquery diff --git a/storage/connectors/redis/pom.xml b/storage/connectors/redis/pom.xml index 6c50895bd20..2a19c0e8287 100644 --- a/storage/connectors/redis/pom.xml +++ b/storage/connectors/redis/pom.xml @@ -70,6 +70,13 @@ test + + net.ishiis.redis + redis-unit + 1.0.3 + test + + junit diff --git a/storage/connectors/rediscluster/src/main/java/feast/storage/connectors/rediscluster/retriever/RedisClusterOnlineRetriever.java b/storage/connectors/redis/src/main/java/feast/storage/connectors/redis/retriever/RedisClusterOnlineRetriever.java similarity index 99% rename from storage/connectors/rediscluster/src/main/java/feast/storage/connectors/rediscluster/retriever/RedisClusterOnlineRetriever.java rename to storage/connectors/redis/src/main/java/feast/storage/connectors/redis/retriever/RedisClusterOnlineRetriever.java index f80de0fc8af..18619252c27 100644 --- a/storage/connectors/rediscluster/src/main/java/feast/storage/connectors/rediscluster/retriever/RedisClusterOnlineRetriever.java +++ b/storage/connectors/redis/src/main/java/feast/storage/connectors/redis/retriever/RedisClusterOnlineRetriever.java @@ -14,7 +14,7 @@ * See the License for the specific language governing permissions and * limitations under the License. */ -package feast.storage.connectors.rediscluster.retriever; +package feast.storage.connectors.redis.retriever; import com.google.protobuf.AbstractMessageLite; import com.google.protobuf.InvalidProtocolBufferException; diff --git a/storage/connectors/rediscluster/src/main/java/feast/storage/connectors/rediscluster/writer/RedisClusterIngestionClient.java b/storage/connectors/redis/src/main/java/feast/storage/connectors/redis/writer/RedisClusterIngestionClient.java similarity index 98% rename from storage/connectors/rediscluster/src/main/java/feast/storage/connectors/rediscluster/writer/RedisClusterIngestionClient.java rename to storage/connectors/redis/src/main/java/feast/storage/connectors/redis/writer/RedisClusterIngestionClient.java index 5bd70d08b43..389db4be3ad 100644 --- a/storage/connectors/rediscluster/src/main/java/feast/storage/connectors/rediscluster/writer/RedisClusterIngestionClient.java +++ b/storage/connectors/redis/src/main/java/feast/storage/connectors/redis/writer/RedisClusterIngestionClient.java @@ -14,7 +14,7 @@ * See the License for the specific language governing permissions and * limitations under the License. */ -package feast.storage.connectors.rediscluster.writer; +package feast.storage.connectors.redis.writer; import com.google.common.collect.Lists; import feast.proto.core.StoreProto; diff --git a/storage/connectors/redis/src/main/java/feast/storage/connectors/redis/writer/RedisCustomIO.java b/storage/connectors/redis/src/main/java/feast/storage/connectors/redis/writer/RedisCustomIO.java index eeac5b3d964..b7fd68755c1 100644 --- a/storage/connectors/redis/src/main/java/feast/storage/connectors/redis/writer/RedisCustomIO.java +++ b/storage/connectors/redis/src/main/java/feast/storage/connectors/redis/writer/RedisCustomIO.java @@ -19,7 +19,7 @@ import feast.proto.core.FeatureSetProto.EntitySpec; import feast.proto.core.FeatureSetProto.FeatureSetSpec; import feast.proto.core.FeatureSetProto.FeatureSpec; -import feast.proto.core.StoreProto.Store.RedisConfig; +import feast.proto.core.StoreProto.Store.*; import feast.proto.storage.RedisProto.RedisKey; import feast.proto.storage.RedisProto.RedisKey.Builder; import feast.proto.types.FeatureRowProto.FeatureRow; @@ -28,7 +28,7 @@ import feast.storage.api.writer.FailedElement; import feast.storage.api.writer.WriteResult; import feast.storage.common.retry.Retriable; -import io.lettuce.core.RedisConnectionException; +import io.lettuce.core.RedisException; import java.io.IOException; import java.util.ArrayList; import java.util.HashMap; @@ -63,21 +63,22 @@ public class RedisCustomIO { private RedisCustomIO() {} - public static Write write(RedisConfig redisConfig, Map featureSetSpecs) { - return new Write(redisConfig, featureSetSpecs); + public static Write write( + RedisIngestionClient redisIngestionClient, Map featureSetSpecs) { + return new Write(redisIngestionClient, featureSetSpecs); } /** ServingStoreWrite data to a Redis server. */ public static class Write extends PTransform, WriteResult> { private Map featureSetSpecs; - private RedisConfig redisConfig; + private RedisIngestionClient redisIngestionClient; private int batchSize; private int timeout; - public Write(RedisConfig redisConfig, Map featureSetSpecs) { - - this.redisConfig = redisConfig; + public Write( + RedisIngestionClient redisIngestionClient, Map featureSetSpecs) { + this.redisIngestionClient = redisIngestionClient; this.featureSetSpecs = featureSetSpecs; } @@ -95,7 +96,7 @@ public Write withTimeout(int timeout) { public WriteResult expand(PCollection input) { PCollectionTuple redisWrite = input.apply( - ParDo.of(new WriteDoFn(redisConfig, featureSetSpecs)) + ParDo.of(new WriteDoFn(redisIngestionClient, featureSetSpecs)) .withOutputTags(successfulInsertsTag, TupleTagList.of(failedInsertsTupleTag))); return WriteResult.in( input.getPipeline(), @@ -111,9 +112,10 @@ public static class WriteDoFn extends DoFn { private int timeout = DEFAULT_TIMEOUT; private RedisIngestionClient redisIngestionClient; - WriteDoFn(RedisConfig config, Map featureSetSpecs) { + WriteDoFn( + RedisIngestionClient redisIngestionClient, Map featureSetSpecs) { - this.redisIngestionClient = new RedisStandaloneIngestionClient(config); + this.redisIngestionClient = redisIngestionClient; this.featureSetSpecs = featureSetSpecs; } @@ -140,7 +142,7 @@ public void setup() { public void startBundle() { try { redisIngestionClient.connect(); - } catch (RedisConnectionException e) { + } catch (RedisException e) { log.error("Connection to redis cannot be established ", e); } featureRows.clear(); @@ -165,7 +167,7 @@ public void execute() throws ExecutionException, InterruptedException { @Override public Boolean isExceptionRetriable(Exception e) { - return e instanceof RedisConnectionException; + return e instanceof RedisException; } @Override diff --git a/storage/connectors/redis/src/main/java/feast/storage/connectors/redis/writer/RedisFeatureSink.java b/storage/connectors/redis/src/main/java/feast/storage/connectors/redis/writer/RedisFeatureSink.java index 171f17daac2..3ddaef7779e 100644 --- a/storage/connectors/redis/src/main/java/feast/storage/connectors/redis/writer/RedisFeatureSink.java +++ b/storage/connectors/redis/src/main/java/feast/storage/connectors/redis/writer/RedisFeatureSink.java @@ -20,6 +20,7 @@ import feast.proto.core.FeatureSetProto.FeatureSet; import feast.proto.core.FeatureSetProto.FeatureSetSpec; import feast.proto.core.StoreProto; +import feast.proto.core.StoreProto.Store.RedisClusterConfig; import feast.proto.core.StoreProto.Store.RedisConfig; import feast.proto.types.FeatureRowProto.FeatureRow; import feast.storage.api.writer.FeatureSink; @@ -28,6 +29,7 @@ import io.lettuce.core.RedisConnectionException; import io.lettuce.core.RedisURI; import java.util.Map; +import javax.annotation.Nullable; import org.apache.beam.sdk.transforms.PTransform; import org.apache.beam.sdk.values.PCollection; @@ -46,8 +48,17 @@ public static FeatureSink fromConfig( return builder().setFeatureSetSpecs(featureSetSpecs).setRedisConfig(redisConfig).build(); } + public static FeatureSink fromConfig( + RedisClusterConfig redisConfig, Map featureSetSpecs) { + return builder().setFeatureSetSpecs(featureSetSpecs).setRedisClusterConfig(redisConfig).build(); + } + + @Nullable public abstract RedisConfig getRedisConfig(); + @Nullable + public abstract RedisClusterConfig getRedisClusterConfig(); + public abstract Map getFeatureSetSpecs(); public abstract Builder toBuilder(); @@ -60,6 +71,8 @@ public static Builder builder() { public abstract static class Builder { public abstract Builder setRedisConfig(RedisConfig redisConfig); + public abstract Builder setRedisClusterConfig(RedisClusterConfig redisConfig); + public abstract Builder setFeatureSetSpecs(Map featureSetSpecs); public abstract RedisFeatureSink build(); @@ -67,22 +80,36 @@ public abstract static class Builder { @Override public void prepareWrite(FeatureSet featureSet) { - - RedisClient redisClient = - RedisClient.create(RedisURI.create(getRedisConfig().getHost(), getRedisConfig().getPort())); - try { - redisClient.connect(); - } catch (RedisConnectionException e) { + if (getRedisConfig() != null) { + RedisClient redisClient = + RedisClient.create( + RedisURI.create(getRedisConfig().getHost(), getRedisConfig().getPort())); + try { + redisClient.connect(); + } catch (RedisConnectionException e) { + throw new RuntimeException( + String.format( + "Failed to connect to Redis at host: '%s' port: '%d'. Please check that your Redis is running and accessible from Feast.", + getRedisConfig().getHost(), getRedisConfig().getPort())); + } + redisClient.shutdown(); + } else if (getRedisClusterConfig() == null) { throw new RuntimeException( - String.format( - "Failed to connect to Redis at host: '%s' port: '%d'. Please check that your Redis is running and accessible from Feast.", - getRedisConfig().getHost(), getRedisConfig().getPort())); + "At least one RedisConfig or RedisClusterConfig must be provided to Redis Sink"); } - redisClient.shutdown(); } @Override public PTransform, WriteResult> writer() { - return new RedisCustomIO.Write(getRedisConfig(), getFeatureSetSpecs()); + if (getRedisClusterConfig() != null) { + return new RedisCustomIO.Write( + new RedisClusterIngestionClient(getRedisClusterConfig()), getFeatureSetSpecs()); + } else if (getRedisConfig() != null) { + return new RedisCustomIO.Write( + new RedisStandaloneIngestionClient(getRedisConfig()), getFeatureSetSpecs()); + } else { + throw new RuntimeException( + "At least one RedisConfig or RedisClusterConfig must be provided to Redis Sink"); + } } } diff --git a/storage/connectors/rediscluster/src/test/java/feast/storage/connectors/rediscluster/retriever/RedisClusterOnlineRetrieverTest.java b/storage/connectors/redis/src/test/java/feast/storage/connectors/redis/retriever/RedisClusterOnlineRetrieverTest.java similarity index 99% rename from storage/connectors/rediscluster/src/test/java/feast/storage/connectors/rediscluster/retriever/RedisClusterOnlineRetrieverTest.java rename to storage/connectors/redis/src/test/java/feast/storage/connectors/redis/retriever/RedisClusterOnlineRetrieverTest.java index deed53e1ae8..45842cef6ee 100644 --- a/storage/connectors/rediscluster/src/test/java/feast/storage/connectors/rediscluster/retriever/RedisClusterOnlineRetrieverTest.java +++ b/storage/connectors/redis/src/test/java/feast/storage/connectors/redis/retriever/RedisClusterOnlineRetrieverTest.java @@ -14,7 +14,7 @@ * See the License for the specific language governing permissions and * limitations under the License. */ -package feast.storage.connectors.rediscluster.retriever; +package feast.storage.connectors.redis.retriever; import static org.hamcrest.MatcherAssert.assertThat; import static org.hamcrest.Matchers.equalTo; diff --git a/storage/connectors/rediscluster/src/test/java/feast/storage/connectors/rediscluster/writer/RedisClusterFeatureSinkTest.java b/storage/connectors/redis/src/test/java/feast/storage/connectors/redis/writer/RedisClusterFeatureSinkTest.java similarity index 98% rename from storage/connectors/rediscluster/src/test/java/feast/storage/connectors/rediscluster/writer/RedisClusterFeatureSinkTest.java rename to storage/connectors/redis/src/test/java/feast/storage/connectors/redis/writer/RedisClusterFeatureSinkTest.java index 8c5afd3d679..7bdc9bba2bb 100644 --- a/storage/connectors/rediscluster/src/test/java/feast/storage/connectors/rediscluster/writer/RedisClusterFeatureSinkTest.java +++ b/storage/connectors/redis/src/test/java/feast/storage/connectors/redis/writer/RedisClusterFeatureSinkTest.java @@ -14,7 +14,7 @@ * See the License for the specific language governing permissions and * limitations under the License. */ -package feast.storage.connectors.rediscluster.writer; +package feast.storage.connectors.redis.writer; import static feast.storage.common.testing.TestUtil.field; import static org.hamcrest.CoreMatchers.equalTo; @@ -67,7 +67,7 @@ public class RedisClusterFeatureSinkTest { private RedisClusterClient redisClusterClient; private RedisClusterCommands redisClusterCommands; - private RedisClusterFeatureSink redisClusterFeatureSink; + private RedisFeatureSink redisClusterFeatureSink; @Before public void setUp() throws IOException { @@ -123,7 +123,7 @@ public void setUp() throws IOException { .build(); redisClusterFeatureSink = - RedisClusterFeatureSink.builder() + RedisFeatureSink.builder() .setFeatureSetSpecs(specMap) .setRedisClusterConfig(redisClusterConfig) .build(); @@ -141,8 +141,8 @@ static boolean deleteDirectory(File directoryToBeDeleted) { @After public void teardown() { - redisClusterClient.shutdown(); redisCluster.stop(); + redisClusterClient.shutdown(); deleteDirectory(new File(String.valueOf(Paths.get(System.getProperty("user.dir"), ".redis")))); } @@ -259,7 +259,7 @@ public void shouldProduceFailedElementIfRetryExceeded() { .build(); Map specMap = ImmutableMap.of("myproject/fs", spec1); redisClusterFeatureSink = - RedisClusterFeatureSink.builder() + RedisFeatureSink.builder() .setFeatureSetSpecs(specMap) .setRedisClusterConfig(redisClusterConfig) .build(); diff --git a/storage/connectors/rediscluster/pom.xml b/storage/connectors/rediscluster/pom.xml deleted file mode 100644 index 5c3cb6e42d3..00000000000 --- a/storage/connectors/rediscluster/pom.xml +++ /dev/null @@ -1,81 +0,0 @@ - - - - dev.feast - feast-storage-connectors - ${revision} - - - 4.0.0 - feast-storage-connector-redis-cluster - - Feast Storage Connector for Redis Cluster - - - - io.lettuce - lettuce-core - - - - org.apache.commons - commons-lang3 - 3.9 - - - - com.google.auto.value - auto-value-annotations - 1.6.6 - - - - com.google.auto.value - auto-value - 1.6.6 - provided - - - - org.mockito - mockito-core - 2.23.0 - test - - - - org.apache.beam - beam-runners-direct-java - ${org.apache.beam.version} - test - - - - org.hamcrest - hamcrest-core - test - - - - org.hamcrest - hamcrest-library - test - - - - net.ishiis.redis - redis-unit - 1.0.3 - test - - - - junit - junit - 4.12 - test - - - - diff --git a/storage/connectors/rediscluster/src/main/java/feast/storage/connectors/rediscluster/retriever/FeatureRowDecoder.java b/storage/connectors/rediscluster/src/main/java/feast/storage/connectors/rediscluster/retriever/FeatureRowDecoder.java deleted file mode 100644 index d398752aa47..00000000000 --- a/storage/connectors/rediscluster/src/main/java/feast/storage/connectors/rediscluster/retriever/FeatureRowDecoder.java +++ /dev/null @@ -1,82 +0,0 @@ -/* - * SPDX-License-Identifier: Apache-2.0 - * Copyright 2018-2020 The Feast Authors - * - * Licensed under the Apache License, Version 2.0 (the "License"); - * you may not use this file except in compliance with the License. - * You may obtain a copy of the License at - * - * https://www.apache.org/licenses/LICENSE-2.0 - * - * Unless required by applicable law or agreed to in writing, software - * distributed under the License is distributed on an "AS IS" BASIS, - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - * See the License for the specific language governing permissions and - * limitations under the License. - */ -package feast.storage.connectors.rediscluster.retriever; - -import feast.proto.core.FeatureSetProto.FeatureSetSpec; -import feast.proto.core.FeatureSetProto.FeatureSpec; -import feast.proto.types.FeatureRowProto.FeatureRow; -import feast.proto.types.FieldProto.Field; -import java.util.Comparator; -import java.util.List; -import java.util.stream.Collectors; -import java.util.stream.IntStream; - -public class FeatureRowDecoder { - - private final String featureSetRef; - private final FeatureSetSpec spec; - - public FeatureRowDecoder(String featureSetRef, FeatureSetSpec spec) { - this.featureSetRef = featureSetRef; - this.spec = spec; - } - - /** - * Validates if an encoded feature row can be decoded without exception. - * - * @param featureRow Feature row - * @return boolean - */ - public Boolean isEncodingValid(FeatureRow featureRow) { - return featureRow.getFieldsList().size() == spec.getFeaturesList().size(); - } - - /** - * Decoding feature row by repopulating the field names based on the corresponding feature set - * spec. - * - * @param encodedFeatureRow Feature row - * @return boolean - */ - public FeatureRow decode(FeatureRow encodedFeatureRow) { - final List fieldsWithoutName = encodedFeatureRow.getFieldsList(); - - List featureNames = - spec.getFeaturesList().stream() - .sorted(Comparator.comparing(FeatureSpec::getName)) - .map(FeatureSpec::getName) - .collect(Collectors.toList()); - List fields = - IntStream.range(0, featureNames.size()) - .mapToObj( - featureNameIndex -> { - String featureName = featureNames.get(featureNameIndex); - return fieldsWithoutName - .get(featureNameIndex) - .toBuilder() - .setName(featureName) - .build(); - }) - .collect(Collectors.toList()); - return encodedFeatureRow - .toBuilder() - .clearFields() - .setFeatureSet(featureSetRef) - .addAllFields(fields) - .build(); - } -} diff --git a/storage/connectors/rediscluster/src/main/java/feast/storage/connectors/rediscluster/writer/RedisClusterCustomIO.java b/storage/connectors/rediscluster/src/main/java/feast/storage/connectors/rediscluster/writer/RedisClusterCustomIO.java deleted file mode 100644 index 712012a2fb7..00000000000 --- a/storage/connectors/rediscluster/src/main/java/feast/storage/connectors/rediscluster/writer/RedisClusterCustomIO.java +++ /dev/null @@ -1,296 +0,0 @@ -/* - * SPDX-License-Identifier: Apache-2.0 - * Copyright 2018-2019 The Feast Authors - * - * Licensed under the Apache License, Version 2.0 (the "License"); - * you may not use this file except in compliance with the License. - * You may obtain a copy of the License at - * - * https://www.apache.org/licenses/LICENSE-2.0 - * - * Unless required by applicable law or agreed to in writing, software - * distributed under the License is distributed on an "AS IS" BASIS, - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - * See the License for the specific language governing permissions and - * limitations under the License. - */ -package feast.storage.connectors.rediscluster.writer; - -import feast.proto.core.FeatureSetProto.EntitySpec; -import feast.proto.core.FeatureSetProto.FeatureSetSpec; -import feast.proto.core.FeatureSetProto.FeatureSpec; -import feast.proto.core.StoreProto.Store.RedisClusterConfig; -import feast.proto.storage.RedisProto.RedisKey; -import feast.proto.storage.RedisProto.RedisKey.Builder; -import feast.proto.types.FeatureRowProto.FeatureRow; -import feast.proto.types.FieldProto.Field; -import feast.proto.types.ValueProto; -import feast.storage.api.writer.FailedElement; -import feast.storage.api.writer.WriteResult; -import feast.storage.common.retry.Retriable; -import io.lettuce.core.RedisException; -import java.io.IOException; -import java.util.ArrayList; -import java.util.HashMap; -import java.util.List; -import java.util.Map; -import java.util.concurrent.ExecutionException; -import java.util.stream.Collectors; -import org.apache.beam.sdk.transforms.DoFn; -import org.apache.beam.sdk.transforms.PTransform; -import org.apache.beam.sdk.transforms.ParDo; -import org.apache.beam.sdk.transforms.windowing.GlobalWindow; -import org.apache.beam.sdk.values.PCollection; -import org.apache.beam.sdk.values.PCollectionTuple; -import org.apache.beam.sdk.values.TupleTag; -import org.apache.beam.sdk.values.TupleTagList; -import org.apache.commons.lang3.exception.ExceptionUtils; -import org.joda.time.Instant; -import org.slf4j.Logger; -import org.slf4j.LoggerFactory; - -public class RedisClusterCustomIO { - - private static final int DEFAULT_BATCH_SIZE = 1000; - private static final int DEFAULT_TIMEOUT = 2000; - - private static TupleTag successfulInsertsTag = - new TupleTag("successfulInserts") {}; - private static TupleTag failedInsertsTupleTag = - new TupleTag("failedInserts") {}; - - private static final Logger log = LoggerFactory.getLogger(RedisClusterCustomIO.class); - - private RedisClusterCustomIO() {} - - public static Write write( - RedisClusterConfig redisClusterConfig, Map featureSetSpecs) { - return new Write(redisClusterConfig, featureSetSpecs); - } - - /** ServingStoreWrite data to a Redis server. */ - public static class Write extends PTransform, WriteResult> { - - private Map featureSetSpecs; - private RedisClusterConfig redisClusterConfig; - private int batchSize; - private int timeout; - - public Write( - RedisClusterConfig redisClusterConfig, Map featureSetSpecs) { - - this.redisClusterConfig = redisClusterConfig; - this.featureSetSpecs = featureSetSpecs; - } - - public Write withBatchSize(int batchSize) { - this.batchSize = batchSize; - return this; - } - - public Write withTimeout(int timeout) { - this.timeout = timeout; - return this; - } - - @Override - public WriteResult expand(PCollection input) { - PCollectionTuple redisWrite = - input.apply( - ParDo.of(new WriteDoFn(redisClusterConfig, featureSetSpecs)) - .withOutputTags(successfulInsertsTag, TupleTagList.of(failedInsertsTupleTag))); - return WriteResult.in( - input.getPipeline(), - redisWrite.get(successfulInsertsTag), - redisWrite.get(failedInsertsTupleTag)); - } - - public static class WriteDoFn extends DoFn { - - private final List featureRows = new ArrayList<>(); - private Map featureSetSpecs; - private int batchSize = DEFAULT_BATCH_SIZE; - private int timeout = DEFAULT_TIMEOUT; - private RedisIngestionClient redisIngestionClient; - - WriteDoFn(RedisClusterConfig config, Map featureSetSpecs) { - - this.redisIngestionClient = new RedisClusterIngestionClient(config); - this.featureSetSpecs = featureSetSpecs; - } - - public WriteDoFn withBatchSize(int batchSize) { - if (batchSize > 0) { - this.batchSize = batchSize; - } - return this; - } - - public WriteDoFn withTimeout(int timeout) { - if (timeout > 0) { - this.timeout = timeout; - } - return this; - } - - @Setup - public void setup() { - this.redisIngestionClient.setup(); - } - - @StartBundle - public void startBundle() { - try { - redisIngestionClient.connect(); - } catch (RedisException e) { - log.error("Connection to redis cannot be established ", e); - } - featureRows.clear(); - } - - private void executeBatch() throws Exception { - this.redisIngestionClient - .getBackOffExecutor() - .execute( - new Retriable() { - @Override - public void execute() throws ExecutionException, InterruptedException { - if (!redisIngestionClient.isConnected()) { - redisIngestionClient.connect(); - } - featureRows.forEach( - row -> { - redisIngestionClient.set(getKey(row), getValue(row)); - }); - redisIngestionClient.sync(); - } - - @Override - public Boolean isExceptionRetriable(Exception e) { - return e instanceof RedisException; - } - - @Override - public void cleanUpAfterFailure() {} - }); - } - - private FailedElement toFailedElement( - FeatureRow featureRow, Exception exception, String jobName) { - return FailedElement.newBuilder() - .setJobName(jobName) - .setTransformName("RedisClusterCustomIO") - .setPayload(featureRow.toString()) - .setErrorMessage(exception.getMessage()) - .setStackTrace(ExceptionUtils.getStackTrace(exception)) - .build(); - } - - private byte[] getKey(FeatureRow featureRow) { - FeatureSetSpec featureSetSpec = featureSetSpecs.get(featureRow.getFeatureSet()); - List entityNames = - featureSetSpec.getEntitiesList().stream() - .map(EntitySpec::getName) - .sorted() - .collect(Collectors.toList()); - - Map entityFields = new HashMap<>(); - Builder redisKeyBuilder = RedisKey.newBuilder().setFeatureSet(featureRow.getFeatureSet()); - for (Field field : featureRow.getFieldsList()) { - if (entityNames.contains(field.getName())) { - entityFields.putIfAbsent( - field.getName(), - Field.newBuilder().setName(field.getName()).setValue(field.getValue()).build()); - } - } - for (String entityName : entityNames) { - redisKeyBuilder.addEntities(entityFields.get(entityName)); - } - return redisKeyBuilder.build().toByteArray(); - } - - private byte[] getValue(FeatureRow featureRow) { - FeatureSetSpec spec = featureSetSpecs.get(featureRow.getFeatureSet()); - - List featureNames = - spec.getFeaturesList().stream().map(FeatureSpec::getName).collect(Collectors.toList()); - Map fieldValueOnlyMap = - featureRow.getFieldsList().stream() - .filter(field -> featureNames.contains(field.getName())) - .distinct() - .collect( - Collectors.toMap( - Field::getName, - field -> Field.newBuilder().setValue(field.getValue()).build())); - - List values = - featureNames.stream() - .sorted() - .map( - featureName -> - fieldValueOnlyMap.getOrDefault( - featureName, - Field.newBuilder() - .setValue(ValueProto.Value.getDefaultInstance()) - .build())) - .collect(Collectors.toList()); - - return FeatureRow.newBuilder() - .setEventTimestamp(featureRow.getEventTimestamp()) - .addAllFields(values) - .build() - .toByteArray(); - } - - @ProcessElement - public void processElement(ProcessContext context) { - FeatureRow featureRow = context.element(); - featureRows.add(featureRow); - if (featureRows.size() >= batchSize) { - try { - executeBatch(); - featureRows.forEach(row -> context.output(successfulInsertsTag, row)); - featureRows.clear(); - } catch (Exception e) { - featureRows.forEach( - failedMutation -> { - FailedElement failedElement = - toFailedElement(failedMutation, e, context.getPipelineOptions().getJobName()); - context.output(failedInsertsTupleTag, failedElement); - }); - featureRows.clear(); - } - } - } - - @FinishBundle - public void finishBundle(FinishBundleContext context) - throws IOException, InterruptedException { - if (featureRows.size() > 0) { - try { - executeBatch(); - featureRows.forEach( - row -> - context.output( - successfulInsertsTag, row, Instant.now(), GlobalWindow.INSTANCE)); - featureRows.clear(); - } catch (Exception e) { - featureRows.forEach( - failedMutation -> { - FailedElement failedElement = - toFailedElement(failedMutation, e, context.getPipelineOptions().getJobName()); - context.output( - failedInsertsTupleTag, failedElement, Instant.now(), GlobalWindow.INSTANCE); - }); - featureRows.clear(); - } - } - } - - @Teardown - public void teardown() { - redisIngestionClient.shutdown(); - } - } - } -} diff --git a/storage/connectors/rediscluster/src/main/java/feast/storage/connectors/rediscluster/writer/RedisClusterFeatureSink.java b/storage/connectors/rediscluster/src/main/java/feast/storage/connectors/rediscluster/writer/RedisClusterFeatureSink.java deleted file mode 100644 index 160df6553cc..00000000000 --- a/storage/connectors/rediscluster/src/main/java/feast/storage/connectors/rediscluster/writer/RedisClusterFeatureSink.java +++ /dev/null @@ -1,75 +0,0 @@ -/* - * SPDX-License-Identifier: Apache-2.0 - * Copyright 2018-2020 The Feast Authors - * - * Licensed under the Apache License, Version 2.0 (the "License"); - * you may not use this file except in compliance with the License. - * You may obtain a copy of the License at - * - * https://www.apache.org/licenses/LICENSE-2.0 - * - * Unless required by applicable law or agreed to in writing, software - * distributed under the License is distributed on an "AS IS" BASIS, - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - * See the License for the specific language governing permissions and - * limitations under the License. - */ -package feast.storage.connectors.rediscluster.writer; - -import com.google.auto.value.AutoValue; -import feast.proto.core.FeatureSetProto; -import feast.proto.core.StoreProto.Store.RedisClusterConfig; -import feast.proto.types.FeatureRowProto; -import feast.storage.api.writer.FeatureSink; -import feast.storage.api.writer.WriteResult; -import java.util.Map; -import org.apache.beam.sdk.transforms.PTransform; -import org.apache.beam.sdk.values.PCollection; - -@AutoValue -public abstract class RedisClusterFeatureSink implements FeatureSink { - - /** - * Initialize a {@link RedisClusterFeatureSink.Builder} from a {@link RedisClusterConfig}. - * - * @param redisClusterConfig {@link RedisClusterConfig} - * @param featureSetSpecs - * @return {@link RedisClusterFeatureSink.Builder} - */ - public static FeatureSink fromConfig( - RedisClusterConfig redisClusterConfig, - Map featureSetSpecs) { - return builder() - .setFeatureSetSpecs(featureSetSpecs) - .setRedisClusterConfig(redisClusterConfig) - .build(); - } - - public abstract RedisClusterConfig getRedisClusterConfig(); - - public abstract Map getFeatureSetSpecs(); - - public abstract Builder toBuilder(); - - public static Builder builder() { - return new AutoValue_RedisClusterFeatureSink.Builder(); - } - - @AutoValue.Builder - public abstract static class Builder { - public abstract Builder setRedisClusterConfig(RedisClusterConfig redisClusterConfig); - - public abstract Builder setFeatureSetSpecs( - Map featureSetSpecs); - - public abstract RedisClusterFeatureSink build(); - } - - @Override - public void prepareWrite(FeatureSetProto.FeatureSet featureSet) {} - - @Override - public PTransform, WriteResult> writer() { - return new RedisClusterCustomIO.Write(getRedisClusterConfig(), getFeatureSetSpecs()); - } -} diff --git a/storage/connectors/rediscluster/src/main/java/feast/storage/connectors/rediscluster/writer/RedisIngestionClient.java b/storage/connectors/rediscluster/src/main/java/feast/storage/connectors/rediscluster/writer/RedisIngestionClient.java deleted file mode 100644 index 5a0b54e6970..00000000000 --- a/storage/connectors/rediscluster/src/main/java/feast/storage/connectors/rediscluster/writer/RedisIngestionClient.java +++ /dev/null @@ -1,49 +0,0 @@ -/* - * SPDX-License-Identifier: Apache-2.0 - * Copyright 2018-2020 The Feast Authors - * - * Licensed under the Apache License, Version 2.0 (the "License"); - * you may not use this file except in compliance with the License. - * You may obtain a copy of the License at - * - * https://www.apache.org/licenses/LICENSE-2.0 - * - * Unless required by applicable law or agreed to in writing, software - * distributed under the License is distributed on an "AS IS" BASIS, - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - * See the License for the specific language governing permissions and - * limitations under the License. - */ -package feast.storage.connectors.rediscluster.writer; - -import feast.storage.common.retry.BackOffExecutor; -import java.io.Serializable; - -public interface RedisIngestionClient extends Serializable { - - void setup(); - - BackOffExecutor getBackOffExecutor(); - - void shutdown(); - - void connect(); - - boolean isConnected(); - - void sync(); - - void pexpire(byte[] key, Long expiryMillis); - - void append(byte[] key, byte[] value); - - void set(byte[] key, byte[] value); - - void lpush(byte[] key, byte[] value); - - void rpush(byte[] key, byte[] value); - - void sadd(byte[] key, byte[] value); - - void zadd(byte[] key, Long score, byte[] value); -} From 17411d6fa316a5be3b773cc3475a97eaa7327549 Mon Sep 17 00:00:00 2001 From: Khor Shu Heng <32997938+khorshuheng@users.noreply.github.com> Date: Thu, 28 May 2020 22:58:58 +0800 Subject: [PATCH 025/133] Remove usage of parallel stream for feature value map generation (#751) Co-authored-by: Khor Shu Heng --- .../main/java/feast/serving/service/OnlineServingService.java | 1 - 1 file changed, 1 deletion(-) diff --git a/serving/src/main/java/feast/serving/service/OnlineServingService.java b/serving/src/main/java/feast/serving/service/OnlineServingService.java index d5c615059d1..7339a19c1a4 100644 --- a/serving/src/main/java/feast/serving/service/OnlineServingService.java +++ b/serving/src/main/java/feast/serving/service/OnlineServingService.java @@ -98,7 +98,6 @@ public GetOnlineFeaturesResponse getOnlineFeatures(GetOnlineFeaturesRequest requ if (isStale(featureSetRequest, entityRow, featureRow)) { featureSetRequest .getFeatureReferences() - .parallelStream() .forEach( ref -> { populateStaleKeyCountMetrics(project, ref); From 30ff1c3dcc6e12425d06094c6c2dbc5324ee7672 Mon Sep 17 00:00:00 2001 From: Zhu Zhanyan Date: Fri, 29 May 2020 04:32:54 +0000 Subject: [PATCH 026/133] GitBook: [master] 28 pages modified --- docs/contributing/development-guide.md | 25 ++++++++--------------- docs/reference/configuration-reference.md | 4 +++- docs/user-guide/feature-retrieval.md | 14 ++++++++++++- docs/user-guide/feature-sets.md | 11 ++++++++-- docs/user-guide/features.md | 4 ++-- 5 files changed, 35 insertions(+), 23 deletions(-) diff --git a/docs/contributing/development-guide.md b/docs/contributing/development-guide.md index 677504e987d..964e667009a 100644 --- a/docs/contributing/development-guide.md +++ b/docs/contributing/development-guide.md @@ -32,7 +32,7 @@ The following components/services are required to develop Feast: * **Feast Core:** Requires PostgreSQL \(version 11 and above\) to store state, and requires a Kafka \(tested on version 2.x\) setup to allow for ingestion of FeatureRows. * **Feast Serving:** Requires Redis \(tested on version 5.x\). -These services should be running before starting development. The following snippet will start the services using Docker. For the +These services should be running before starting development. The following snippet will start the services using Docker: ```bash # Start Postgres @@ -269,20 +269,6 @@ store { ### 4.5 Registering a FeatureSet -Before registering a new FeatureSet, a project is required. - -```text -grpc_cli call localhost:6565 CreateProject ' - name: "your_project_name" -' -``` - -When a feature set is successfully registered, Feast Core will start an **ingestion** job that listens for new features in the feature set. - -{% hint style="info" %} -Note that Feast currently only supports source of type `KAFKA`, so you must have access to a running Kafka broker to register a FeatureSet successfully. It is possible to omit the `source` from a Feature Set, but Feast Core will still use Kafka behind the scenes, it is simply abstracted away from the user. -{% endhint %} - Create a new FeatureSet in Feast by sending a request to Feast Core: ```text @@ -342,6 +328,12 @@ grpc_cli call localhost:6565 ListFeatureSets ' ' ``` +When a feature set is successfully registered, Feast Core will start an **ingestion** job that listens for new features in the feature set. + +{% hint style="info" %} +Note that Feast currently only supports source of type `KAFKA`, so you must have access to a running Kafka broker to register a FeatureSet successfully. It is possible to omit the `source` from a Feature Set, but Feast Core will still use Kafka behind the scenes, it is simply abstracted away from the user. +{% endhint %} + ### 4.6 Ingestion and Population of Feature Values ```text @@ -403,7 +395,6 @@ Ensure that Feast Serving returns results for the feature value for the specific ```text grpc_cli call localhost:6566 GetOnlineFeatures ' features { - project: "your_project_name" name: "city" version: 1 max_age { @@ -430,7 +421,7 @@ field_values { } } fields { - key: "your_project_name/city:1" + key: "city" value { string_val: "JAKARTA" } diff --git a/docs/reference/configuration-reference.md b/docs/reference/configuration-reference.md index cefd478ef9d..76baf8a6edd 100644 --- a/docs/reference/configuration-reference.md +++ b/docs/reference/configuration-reference.md @@ -128,7 +128,9 @@ The following list is a subset of all configuration properties. Please see [this -### Usage + + +### Usage #### Configuration File diff --git a/docs/user-guide/feature-retrieval.md b/docs/user-guide/feature-retrieval.md index e49db24fe99..0ebf79ca88e 100644 --- a/docs/user-guide/feature-retrieval.md +++ b/docs/user-guide/feature-retrieval.md @@ -31,6 +31,10 @@ features = [ target = 'churn' ``` +{% hint style="info" %} +Where the Features from different Feature Sets use the same name, the Feature Set name `feature-set` is required to disambiguate which feature is specified. +{% endhint %} + Feature references only apply to a single `project`. Features cannot be retrieved across projects in a single request. ## 3. Historical feature retrieval @@ -47,13 +51,17 @@ features = self._features + [self._target] dataset = client.get_batch_features( feature_refs=features, entity_rows=entity_df - ) +) # Materialize the dataset object to a Pandas DataFrame. # Alternatively it is possible to use a file reference if the data is too large df = dataset.to_dataframe() ``` +{% hint style="info" %} +When no project is specified when retrieving features with `get_batch_features()`, Feast infers that the features specified belong to the `default` project. To retrieve from another project, specify the `default` parameter when retrieving features. +{% endhint %} + In the above example, Feast does a point in time correct query from a single feature set. For each timestamp and entity key combination that is provided by `entity_df`, Feast determines the values of all the features in the `features` list at that respective point in time and then joins features values to that specific entity value and timestamp, and repeats this process for all timestamps. This is called a point in time correct join. @@ -112,5 +120,9 @@ data = client.get_online_features( ) ``` +{% hint style="info" %} +When no project is specified when retrieving features with `get_online_feature()`, Feast infers that the features specified belong to the `default` project. To retrieve from another project, specify the `project` parameter when retrieving features. +{% endhint %} + Online serving with Feast is built to be very low latency. Feast Serving provides a [gRPC API](https://api.docs.feast.dev/grpc/feast.serving.pb.html) that is backed by [Redis](https://redis.io/). We also provide support for [Python](https://api.docs.feast.dev/python/), [Go](https://godoc.org/github.com/gojek/feast/sdk/go), and Java clients. diff --git a/docs/user-guide/feature-sets.md b/docs/user-guide/feature-sets.md index 3d72dc1b02b..fb03adad345 100644 --- a/docs/user-guide/feature-sets.md +++ b/docs/user-guide/feature-sets.md @@ -12,7 +12,7 @@ Feature sets are a grouping of feature sets based on how they are loaded into Fe ### Customer Transactions Example -Below is an example of a basic `customer transactions` feature set that has been exported to YAML: +Below is an example specification of a basic `customer transactions` feature set that has been exported to YAML: {% tabs %} {% tab title="customer\_transactions\_feature\_set.yaml" %} @@ -30,7 +30,7 @@ features: {% endtab %} {% endtabs %} -The dataframe below \(`customer_data.csv`\) contains the features and entities of the above feature set +The dataframe below \(`customer_data.csv`\) contains the features and entities of the above feature set. | datetime | customer\_id | daily\_transactions | total\_tra**nsactions** | | :--- | :--- | :--- | :--- | @@ -48,10 +48,17 @@ customer_df = pd.read_csv("customer_data.csv") # Create feature set from YAML (using YAML is optional) cust_trans_fs = FeatureSet.from_yaml("customer_transactions_feature_set.yaml") +# Apply new feature set +client.apply(cust_trans_fs) + # Load feature data into Feast for this specific feature set client.ingest(cust_trans_fs, customer_data) ``` +{% hint style="info" %} +When applying a Feature Set without specifying a project in its specification, Feast creates/updates the Feature Set in the `default` project. To create a Feature Set in another project, specify the project of choice in the Feature Set specification's project field. +{% endhint %} + ### **Making changes to Feature Sets** In order to facilitate the need for feature set definitions to change over time, a limited set of changes can be made to existing feature sets. diff --git a/docs/user-guide/features.md b/docs/user-guide/features.md index eb9adf01d0c..32a7b57b098 100644 --- a/docs/user-guide/features.md +++ b/docs/user-guide/features.md @@ -8,10 +8,10 @@ The following is a YAML representation of a feature specification. This specific {% code title="total\_trips\_feature.yaml" %} ```yaml -# Entity name +# Feature name name: total_trips_24h -# Entity value type +# Feature value type value_type: INT64 ``` {% endcode %} From b91f2dd4965f655fe20212cb0e221e668d61628d Mon Sep 17 00:00:00 2001 From: Chen Zhiling Date: Fri, 29 May 2020 13:37:58 +0800 Subject: [PATCH 027/133] Recompile golang protos (#755) --- go.mod | 4 +-- go.sum | 6 ++++ sdk/go/protos/feast/core/CoreService.pb.go | 2 +- sdk/go/protos/feast/core/FeatureSet.pb.go | 36 ++++++++++--------- .../feast/core/FeatureSetReference.pb.go | 2 +- sdk/go/protos/feast/core/IngestionJob.pb.go | 2 +- sdk/go/protos/feast/core/Runner.pb.go | 2 +- sdk/go/protos/feast/core/Source.pb.go | 2 +- sdk/go/protos/feast/core/Store.pb.go | 2 +- .../protos/feast/serving/ServingService.pb.go | 2 +- sdk/go/protos/feast/storage/Redis.pb.go | 2 +- sdk/go/protos/feast/types/FeatureRow.pb.go | 2 +- .../feast/types/FeatureRowExtended.pb.go | 2 +- sdk/go/protos/feast/types/Field.pb.go | 2 +- sdk/go/protos/feast/types/Value.pb.go | 2 +- .../tensorflow_metadata/proto/v0/path.pb.go | 2 +- .../tensorflow_metadata/proto/v0/schema.pb.go | 2 +- 17 files changed, 42 insertions(+), 32 deletions(-) diff --git a/go.mod b/go.mod index 9a4456616b7..1be2396f6e6 100644 --- a/go.mod +++ b/go.mod @@ -25,9 +25,9 @@ require ( golang.org/x/lint v0.0.0-20200302205851-738671d3881b // indirect golang.org/x/net v0.0.0-20200513185701-a91f0712d120 golang.org/x/sys v0.0.0-20200515095857-1151b9dac4a9 // indirect - golang.org/x/tools v0.0.0-20200515220128-d3bf790afa53 // indirect - google.golang.org/genproto v0.0.0-20200515170657-fc4c6c6a6587 // indirect + golang.org/x/tools v0.0.0-20200528185414-6be401e3f76e // indirect google.golang.org/grpc v1.29.1 + google.golang.org/protobuf v1.24.0 // indirect gopkg.in/russross/blackfriday.v2 v2.0.0 // indirect gopkg.in/yaml.v2 v2.2.4 istio.io/gogo-genproto v0.0.0-20191212213402-78a529a42cd8 // indirect diff --git a/go.sum b/go.sum index 7608c511c25..1b2aa8bf63b 100644 --- a/go.sum +++ b/go.sum @@ -462,6 +462,8 @@ golang.org/x/tools v0.0.0-20200504022951-6b6965ac5dd1 h1:C8rdnd6KieI73Z2Av0sS0t4 golang.org/x/tools v0.0.0-20200504022951-6b6965ac5dd1/go.mod h1:EkVYQZoAsY45+roYkvgYkIh4xh/qjgUK9TdY2XT94GE= golang.org/x/tools v0.0.0-20200515220128-d3bf790afa53 h1:vmsb6v0zUdmUlXfwKaYrHPPRCV0lHq/IwNIf0ASGjyQ= golang.org/x/tools v0.0.0-20200515220128-d3bf790afa53/go.mod h1:EkVYQZoAsY45+roYkvgYkIh4xh/qjgUK9TdY2XT94GE= +golang.org/x/tools v0.0.0-20200528185414-6be401e3f76e h1:jTL1CJ2kmavapMVdBKy6oVrhBHByRCMfykS45+lEFQk= +golang.org/x/tools v0.0.0-20200528185414-6be401e3f76e/go.mod h1:EkVYQZoAsY45+roYkvgYkIh4xh/qjgUK9TdY2XT94GE= golang.org/x/xerrors v0.0.0-20190717185122-a985d3407aa7/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0= golang.org/x/xerrors v0.0.0-20191011141410-1b5146add898/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0= golang.org/x/xerrors v0.0.0-20191204190536-9bdfabe68543/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0= @@ -487,6 +489,7 @@ google.golang.org/genproto v0.0.0-20200319113533-08878b785e9c h1:5aI3/f/3eCZps9x google.golang.org/genproto v0.0.0-20200319113533-08878b785e9c/go.mod h1:55QSHmfGQM9UVYDPBsyGGes0y52j32PQ3BqQfXhyH3c= google.golang.org/genproto v0.0.0-20200515170657-fc4c6c6a6587 h1:1Ym+vvUpq1ZHvxzn34gENJX8U4aKO+vhy2P/2+Xl6qQ= google.golang.org/genproto v0.0.0-20200515170657-fc4c6c6a6587/go.mod h1:YsZOwe1myG/8QRHRsmBRE1LrgQY60beZKjly0O1fX9U= +google.golang.org/genproto v0.0.0-20200526211855-cb27e3aa2013/go.mod h1:NbSheEEYHJ7i3ixzK3sjbqSGDJWnxyFXZblF3eUsNvo= google.golang.org/grpc v1.13.0/go.mod h1:yo6s7OP7yaDglbqo1J04qKzAhqBH6lvTonzMVmEdcZw= google.golang.org/grpc v1.19.0/go.mod h1:mqu4LbDTu4XGKhr4mRzUsmM4RtVoemTSY81AxZiDr8c= google.golang.org/grpc v1.19.1 h1:TrBcJ1yqAl1G++wO39nD/qtgpsW9/1+QGrluyMGEYgM= @@ -512,6 +515,9 @@ google.golang.org/protobuf v1.21.0/go.mod h1:47Nbq4nVaFHyn7ilMalzfO3qCViNmqZ2kzi google.golang.org/protobuf v1.22.0/go.mod h1:EGpADcykh3NcUnDUJcl1+ZksZNG86OlYog2l/sGQquU= google.golang.org/protobuf v1.23.0 h1:4MY060fB1DLGMB/7MBTLnwQUY6+F09GEiz6SsrNqyzM= google.golang.org/protobuf v1.23.0/go.mod h1:EGpADcykh3NcUnDUJcl1+ZksZNG86OlYog2l/sGQquU= +google.golang.org/protobuf v1.23.1-0.20200526195155-81db48ad09cc/go.mod h1:EGpADcykh3NcUnDUJcl1+ZksZNG86OlYog2l/sGQquU= +google.golang.org/protobuf v1.24.0 h1:UhZDfRO8JRQru4/+LlLE0BRKGF8L+PICnvYZmx/fEGA= +google.golang.org/protobuf v1.24.0/go.mod h1:r/3tXBNzIEhYS9I1OUVjXDlt8tc493IdKGjtUeSXeh4= gopkg.in/alecthomas/kingpin.v2 v2.2.6/go.mod h1:FMv+mEhP44yOT+4EoQTLFTRgOQ1FBLkstjWtayDeSgw= gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405 h1:yhCVgyC4o1eVCa2tZl7eS0r+SDo693bJlVdllGtEeKM= gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0= diff --git a/sdk/go/protos/feast/core/CoreService.pb.go b/sdk/go/protos/feast/core/CoreService.pb.go index 75f988ed528..4f22ae4c373 100644 --- a/sdk/go/protos/feast/core/CoreService.pb.go +++ b/sdk/go/protos/feast/core/CoreService.pb.go @@ -16,7 +16,7 @@ // Code generated by protoc-gen-go. DO NOT EDIT. // versions: -// protoc-gen-go v1.23.0 +// protoc-gen-go v1.24.0 // protoc v3.10.0 // source: feast/core/CoreService.proto diff --git a/sdk/go/protos/feast/core/FeatureSet.pb.go b/sdk/go/protos/feast/core/FeatureSet.pb.go index 3c6441f2ac9..acd111614d2 100644 --- a/sdk/go/protos/feast/core/FeatureSet.pb.go +++ b/sdk/go/protos/feast/core/FeatureSet.pb.go @@ -16,7 +16,7 @@ // Code generated by protoc-gen-go. DO NOT EDIT. // versions: -// protoc-gen-go v1.23.0 +// protoc-gen-go v1.24.0 // protoc v3.10.0 // source: feast/core/FeatureSet.proto @@ -48,9 +48,10 @@ const _ = proto.ProtoPackageIsVersion4 type FeatureSetStatus int32 const ( - FeatureSetStatus_STATUS_INVALID FeatureSetStatus = 0 - FeatureSetStatus_STATUS_PENDING FeatureSetStatus = 1 - FeatureSetStatus_STATUS_READY FeatureSetStatus = 2 + FeatureSetStatus_STATUS_INVALID FeatureSetStatus = 0 + FeatureSetStatus_STATUS_PENDING FeatureSetStatus = 1 + FeatureSetStatus_STATUS_JOB_STARTING FeatureSetStatus = 3 + FeatureSetStatus_STATUS_READY FeatureSetStatus = 2 ) // Enum value maps for FeatureSetStatus. @@ -58,12 +59,14 @@ var ( FeatureSetStatus_name = map[int32]string{ 0: "STATUS_INVALID", 1: "STATUS_PENDING", + 3: "STATUS_JOB_STARTING", 2: "STATUS_READY", } FeatureSetStatus_value = map[string]int32{ - "STATUS_INVALID": 0, - "STATUS_PENDING": 1, - "STATUS_READY": 2, + "STATUS_INVALID": 0, + "STATUS_PENDING": 1, + "STATUS_JOB_STARTING": 3, + "STATUS_READY": 2, } ) @@ -873,18 +876,19 @@ var file_feast_core_FeatureSet_proto_rawDesc = []byte{ 0x06, 0x73, 0x74, 0x61, 0x74, 0x75, 0x73, 0x18, 0x02, 0x20, 0x01, 0x28, 0x0e, 0x32, 0x1c, 0x2e, 0x66, 0x65, 0x61, 0x73, 0x74, 0x2e, 0x63, 0x6f, 0x72, 0x65, 0x2e, 0x46, 0x65, 0x61, 0x74, 0x75, 0x72, 0x65, 0x53, 0x65, 0x74, 0x53, 0x74, 0x61, 0x74, 0x75, 0x73, 0x52, 0x06, 0x73, 0x74, 0x61, - 0x74, 0x75, 0x73, 0x2a, 0x4c, 0x0a, 0x10, 0x46, 0x65, 0x61, 0x74, 0x75, 0x72, 0x65, 0x53, 0x65, + 0x74, 0x75, 0x73, 0x2a, 0x65, 0x0a, 0x10, 0x46, 0x65, 0x61, 0x74, 0x75, 0x72, 0x65, 0x53, 0x65, 0x74, 0x53, 0x74, 0x61, 0x74, 0x75, 0x73, 0x12, 0x12, 0x0a, 0x0e, 0x53, 0x54, 0x41, 0x54, 0x55, 0x53, 0x5f, 0x49, 0x4e, 0x56, 0x41, 0x4c, 0x49, 0x44, 0x10, 0x00, 0x12, 0x12, 0x0a, 0x0e, 0x53, 0x54, 0x41, 0x54, 0x55, 0x53, 0x5f, 0x50, 0x45, 0x4e, 0x44, 0x49, 0x4e, 0x47, 0x10, 0x01, 0x12, - 0x10, 0x0a, 0x0c, 0x53, 0x54, 0x41, 0x54, 0x55, 0x53, 0x5f, 0x52, 0x45, 0x41, 0x44, 0x59, 0x10, - 0x02, 0x42, 0x58, 0x0a, 0x10, 0x66, 0x65, 0x61, 0x73, 0x74, 0x2e, 0x70, 0x72, 0x6f, 0x74, 0x6f, - 0x2e, 0x63, 0x6f, 0x72, 0x65, 0x42, 0x0f, 0x46, 0x65, 0x61, 0x74, 0x75, 0x72, 0x65, 0x53, 0x65, - 0x74, 0x50, 0x72, 0x6f, 0x74, 0x6f, 0x5a, 0x33, 0x67, 0x69, 0x74, 0x68, 0x75, 0x62, 0x2e, 0x63, - 0x6f, 0x6d, 0x2f, 0x66, 0x65, 0x61, 0x73, 0x74, 0x2d, 0x64, 0x65, 0x76, 0x2f, 0x66, 0x65, 0x61, - 0x73, 0x74, 0x2f, 0x73, 0x64, 0x6b, 0x2f, 0x67, 0x6f, 0x2f, 0x70, 0x72, 0x6f, 0x74, 0x6f, 0x73, - 0x2f, 0x66, 0x65, 0x61, 0x73, 0x74, 0x2f, 0x63, 0x6f, 0x72, 0x65, 0x62, 0x06, 0x70, 0x72, 0x6f, - 0x74, 0x6f, 0x33, + 0x17, 0x0a, 0x13, 0x53, 0x54, 0x41, 0x54, 0x55, 0x53, 0x5f, 0x4a, 0x4f, 0x42, 0x5f, 0x53, 0x54, + 0x41, 0x52, 0x54, 0x49, 0x4e, 0x47, 0x10, 0x03, 0x12, 0x10, 0x0a, 0x0c, 0x53, 0x54, 0x41, 0x54, + 0x55, 0x53, 0x5f, 0x52, 0x45, 0x41, 0x44, 0x59, 0x10, 0x02, 0x42, 0x58, 0x0a, 0x10, 0x66, 0x65, + 0x61, 0x73, 0x74, 0x2e, 0x70, 0x72, 0x6f, 0x74, 0x6f, 0x2e, 0x63, 0x6f, 0x72, 0x65, 0x42, 0x0f, + 0x46, 0x65, 0x61, 0x74, 0x75, 0x72, 0x65, 0x53, 0x65, 0x74, 0x50, 0x72, 0x6f, 0x74, 0x6f, 0x5a, + 0x33, 0x67, 0x69, 0x74, 0x68, 0x75, 0x62, 0x2e, 0x63, 0x6f, 0x6d, 0x2f, 0x66, 0x65, 0x61, 0x73, + 0x74, 0x2d, 0x64, 0x65, 0x76, 0x2f, 0x66, 0x65, 0x61, 0x73, 0x74, 0x2f, 0x73, 0x64, 0x6b, 0x2f, + 0x67, 0x6f, 0x2f, 0x70, 0x72, 0x6f, 0x74, 0x6f, 0x73, 0x2f, 0x66, 0x65, 0x61, 0x73, 0x74, 0x2f, + 0x63, 0x6f, 0x72, 0x65, 0x62, 0x06, 0x70, 0x72, 0x6f, 0x74, 0x6f, 0x33, } var ( diff --git a/sdk/go/protos/feast/core/FeatureSetReference.pb.go b/sdk/go/protos/feast/core/FeatureSetReference.pb.go index 3da39ad922a..dfa47a25a7c 100644 --- a/sdk/go/protos/feast/core/FeatureSetReference.pb.go +++ b/sdk/go/protos/feast/core/FeatureSetReference.pb.go @@ -16,7 +16,7 @@ // Code generated by protoc-gen-go. DO NOT EDIT. // versions: -// protoc-gen-go v1.23.0 +// protoc-gen-go v1.24.0 // protoc v3.10.0 // source: feast/core/FeatureSetReference.proto diff --git a/sdk/go/protos/feast/core/IngestionJob.pb.go b/sdk/go/protos/feast/core/IngestionJob.pb.go index bf30f25168f..a54b23c4498 100644 --- a/sdk/go/protos/feast/core/IngestionJob.pb.go +++ b/sdk/go/protos/feast/core/IngestionJob.pb.go @@ -16,7 +16,7 @@ // Code generated by protoc-gen-go. DO NOT EDIT. // versions: -// protoc-gen-go v1.23.0 +// protoc-gen-go v1.24.0 // protoc v3.10.0 // source: feast/core/IngestionJob.proto diff --git a/sdk/go/protos/feast/core/Runner.pb.go b/sdk/go/protos/feast/core/Runner.pb.go index 54e03714d2a..b1255e52ba6 100644 --- a/sdk/go/protos/feast/core/Runner.pb.go +++ b/sdk/go/protos/feast/core/Runner.pb.go @@ -16,7 +16,7 @@ // Code generated by protoc-gen-go. DO NOT EDIT. // versions: -// protoc-gen-go v1.23.0 +// protoc-gen-go v1.24.0 // protoc v3.10.0 // source: feast/core/Runner.proto diff --git a/sdk/go/protos/feast/core/Source.pb.go b/sdk/go/protos/feast/core/Source.pb.go index b29dc5af96c..2aa0c3f12e6 100644 --- a/sdk/go/protos/feast/core/Source.pb.go +++ b/sdk/go/protos/feast/core/Source.pb.go @@ -16,7 +16,7 @@ // Code generated by protoc-gen-go. DO NOT EDIT. // versions: -// protoc-gen-go v1.23.0 +// protoc-gen-go v1.24.0 // protoc v3.10.0 // source: feast/core/Source.proto diff --git a/sdk/go/protos/feast/core/Store.pb.go b/sdk/go/protos/feast/core/Store.pb.go index 866dc71fe94..a36c6afc6ab 100644 --- a/sdk/go/protos/feast/core/Store.pb.go +++ b/sdk/go/protos/feast/core/Store.pb.go @@ -16,7 +16,7 @@ // Code generated by protoc-gen-go. DO NOT EDIT. // versions: -// protoc-gen-go v1.23.0 +// protoc-gen-go v1.24.0 // protoc v3.10.0 // source: feast/core/Store.proto diff --git a/sdk/go/protos/feast/serving/ServingService.pb.go b/sdk/go/protos/feast/serving/ServingService.pb.go index 9e5a151b7ee..fc2748d5d84 100644 --- a/sdk/go/protos/feast/serving/ServingService.pb.go +++ b/sdk/go/protos/feast/serving/ServingService.pb.go @@ -15,7 +15,7 @@ // Code generated by protoc-gen-go. DO NOT EDIT. // versions: -// protoc-gen-go v1.23.0 +// protoc-gen-go v1.24.0 // protoc v3.10.0 // source: feast/serving/ServingService.proto diff --git a/sdk/go/protos/feast/storage/Redis.pb.go b/sdk/go/protos/feast/storage/Redis.pb.go index cfc22a458e5..92c1040ca5a 100644 --- a/sdk/go/protos/feast/storage/Redis.pb.go +++ b/sdk/go/protos/feast/storage/Redis.pb.go @@ -15,7 +15,7 @@ // Code generated by protoc-gen-go. DO NOT EDIT. // versions: -// protoc-gen-go v1.23.0 +// protoc-gen-go v1.24.0 // protoc v3.10.0 // source: feast/storage/Redis.proto diff --git a/sdk/go/protos/feast/types/FeatureRow.pb.go b/sdk/go/protos/feast/types/FeatureRow.pb.go index 3b42b44889e..4c094379eb1 100644 --- a/sdk/go/protos/feast/types/FeatureRow.pb.go +++ b/sdk/go/protos/feast/types/FeatureRow.pb.go @@ -15,7 +15,7 @@ // Code generated by protoc-gen-go. DO NOT EDIT. // versions: -// protoc-gen-go v1.23.0 +// protoc-gen-go v1.24.0 // protoc v3.10.0 // source: feast/types/FeatureRow.proto diff --git a/sdk/go/protos/feast/types/FeatureRowExtended.pb.go b/sdk/go/protos/feast/types/FeatureRowExtended.pb.go index 1692a55dd4f..3cb15a21ce0 100644 --- a/sdk/go/protos/feast/types/FeatureRowExtended.pb.go +++ b/sdk/go/protos/feast/types/FeatureRowExtended.pb.go @@ -15,7 +15,7 @@ // Code generated by protoc-gen-go. DO NOT EDIT. // versions: -// protoc-gen-go v1.23.0 +// protoc-gen-go v1.24.0 // protoc v3.10.0 // source: feast/types/FeatureRowExtended.proto diff --git a/sdk/go/protos/feast/types/Field.pb.go b/sdk/go/protos/feast/types/Field.pb.go index 901075e2620..08b35f66272 100644 --- a/sdk/go/protos/feast/types/Field.pb.go +++ b/sdk/go/protos/feast/types/Field.pb.go @@ -15,7 +15,7 @@ // Code generated by protoc-gen-go. DO NOT EDIT. // versions: -// protoc-gen-go v1.23.0 +// protoc-gen-go v1.24.0 // protoc v3.10.0 // source: feast/types/Field.proto diff --git a/sdk/go/protos/feast/types/Value.pb.go b/sdk/go/protos/feast/types/Value.pb.go index 317768fd549..bb777a07661 100644 --- a/sdk/go/protos/feast/types/Value.pb.go +++ b/sdk/go/protos/feast/types/Value.pb.go @@ -15,7 +15,7 @@ // Code generated by protoc-gen-go. DO NOT EDIT. // versions: -// protoc-gen-go v1.23.0 +// protoc-gen-go v1.24.0 // protoc v3.10.0 // source: feast/types/Value.proto diff --git a/sdk/go/protos/tensorflow_metadata/proto/v0/path.pb.go b/sdk/go/protos/tensorflow_metadata/proto/v0/path.pb.go index fe4cc3ee47a..f58247299d9 100644 --- a/sdk/go/protos/tensorflow_metadata/proto/v0/path.pb.go +++ b/sdk/go/protos/tensorflow_metadata/proto/v0/path.pb.go @@ -15,7 +15,7 @@ // Code generated by protoc-gen-go. DO NOT EDIT. // versions: -// protoc-gen-go v1.23.0 +// protoc-gen-go v1.24.0 // protoc v3.10.0 // source: tensorflow_metadata/proto/v0/path.proto diff --git a/sdk/go/protos/tensorflow_metadata/proto/v0/schema.pb.go b/sdk/go/protos/tensorflow_metadata/proto/v0/schema.pb.go index 97117947b78..a04f5bba8ca 100644 --- a/sdk/go/protos/tensorflow_metadata/proto/v0/schema.pb.go +++ b/sdk/go/protos/tensorflow_metadata/proto/v0/schema.pb.go @@ -15,7 +15,7 @@ // Code generated by protoc-gen-go. DO NOT EDIT. // versions: -// protoc-gen-go v1.23.0 +// protoc-gen-go v1.24.0 // protoc v3.10.0 // source: tensorflow_metadata/proto/v0/schema.proto From 714b0c25f50d2514187fface32fdec5b90a13e6b Mon Sep 17 00:00:00 2001 From: Zhu Zhan Yan Date: Fri, 29 May 2020 18:01:58 +0800 Subject: [PATCH 028/133] Fix Go and Java SDK Regressions (#729) * Fix nil error cause by adding items to a nil map in go SDK's client.GetOnlineFeatures() * Fix issue where go sdk client.GetOnlineFeatures() does not return entity values * Fixed issue where Row is unable to parse Value type due to proto package rename. * Fix wrong path of test_feature python SDK test. * Revert stub and channel private fields to final in FeastClient java SDK Co-authored-by: Zhu Zhanyan --- sdk/go/client.go | 8 ++++---- sdk/go/types.go | 4 ++-- sdk/java/src/main/java/com/gojek/feast/FeastClient.java | 7 ++++--- sdk/java/src/main/java/com/gojek/feast/Row.java | 2 +- sdk/python/{feast => tests}/test_feature.py | 2 +- 5 files changed, 12 insertions(+), 11 deletions(-) rename sdk/python/{feast => tests}/test_feature.py (92%) diff --git a/sdk/go/client.go b/sdk/go/client.go index 38c9e2fb7e5..ef8ec8b8cf9 100644 --- a/sdk/go/client.go +++ b/sdk/go/client.go @@ -52,7 +52,7 @@ func (fc *GrpcClient) GetOnlineFeatures(ctx context.Context, req *OnlineFeatures resp, err := fc.cli.GetOnlineFeatures(ctx, featuresRequest) // collect unqiue entity refs from entity rows - var entityRefs map[string]struct{} + entityRefs := make(map[string]struct{}) for _, entityRows := range req.Entities { for ref, _ := range entityRows { entityRefs[ref] = struct{}{} @@ -61,7 +61,7 @@ func (fc *GrpcClient) GetOnlineFeatures(ctx context.Context, req *OnlineFeatures // strip projects from to projects for _, fieldValue := range resp.GetFieldValues() { - var stripFields map[string]*types.Value + stripFields := make(map[string]*types.Value) for refStr, value := range fieldValue.Fields { _, isEntity := entityRefs[refStr] if !isEntity { // is feature ref @@ -69,9 +69,9 @@ func (fc *GrpcClient) GetOnlineFeatures(ctx context.Context, req *OnlineFeatures if err != nil { return nil, err } - stripRefStr := toFeatureRefStr(featureRef) - stripFields[stripRefStr] = value + refStr = toFeatureRefStr(featureRef) } + stripFields[refStr] = value } fieldValue.Fields = stripFields } diff --git a/sdk/go/types.go b/sdk/go/types.go index 7fe400e45bb..dd6553163bd 100644 --- a/sdk/go/types.go +++ b/sdk/go/types.go @@ -21,12 +21,12 @@ func (r Row) equalTo(other Row) bool { return true } -// StrVal is a int64 type feast value +// StrVal is a string type feast value func StrVal(val string) *types.Value { return &types.Value{Val: &types.Value_StringVal{StringVal: val}} } -// Int32Val is a int64 type feast value +// Int32Val is a int32 type feast value func Int32Val(val int32) *types.Value { return &types.Value{Val: &types.Value_Int32Val{Int32Val: val}} } diff --git a/sdk/java/src/main/java/com/gojek/feast/FeastClient.java b/sdk/java/src/main/java/com/gojek/feast/FeastClient.java index c6e7edd0764..a81fdab21a2 100644 --- a/sdk/java/src/main/java/com/gojek/feast/FeastClient.java +++ b/sdk/java/src/main/java/com/gojek/feast/FeastClient.java @@ -23,6 +23,7 @@ import feast.proto.serving.ServingAPIProto.GetOnlineFeaturesRequest.EntityRow; import feast.proto.serving.ServingAPIProto.GetOnlineFeaturesResponse; import feast.proto.serving.ServingServiceGrpc; +import feast.proto.serving.ServingServiceGrpc.ServingServiceBlockingStub; import feast.proto.types.ValueProto.Value; import io.grpc.ManagedChannel; import io.grpc.ManagedChannelBuilder; @@ -40,7 +41,7 @@ public class FeastClient implements AutoCloseable { private static final int CHANNEL_SHUTDOWN_TIMEOUT_SEC = 5; private final ManagedChannel channel; - private final ServingServiceGrpc.ServingServiceBlockingStub stub; + private final ServingServiceBlockingStub stub; /** * Create a client to access Feast @@ -161,9 +162,9 @@ public List getOnlineFeatures( .collect(Collectors.toList()); } - private FeastClient(ManagedChannel channel) { + protected FeastClient(ManagedChannel channel) { this.channel = channel; - stub = ServingServiceGrpc.newBlockingStub(channel); + this.stub = ServingServiceGrpc.newBlockingStub(channel); } public void close() throws Exception { diff --git a/sdk/java/src/main/java/com/gojek/feast/Row.java b/sdk/java/src/main/java/com/gojek/feast/Row.java index 2ac2581cac9..4a3035f8f19 100644 --- a/sdk/java/src/main/java/com/gojek/feast/Row.java +++ b/sdk/java/src/main/java/com/gojek/feast/Row.java @@ -76,7 +76,7 @@ public Row set(String fieldName, Object value) { fields.put( fieldName, Value.newBuilder().setBytesVal(ByteString.copyFrom((byte[]) value)).build()); break; - case "feast.types.ValueProto.Value": + case "feast.proto.types.ValueProto.Value": fields.put(fieldName, (Value) value); break; default: diff --git a/sdk/python/feast/test_feature.py b/sdk/python/tests/test_feature.py similarity index 92% rename from sdk/python/feast/test_feature.py rename to sdk/python/tests/test_feature.py index a4f0ff58ff2..bc83683e0fe 100644 --- a/sdk/python/feast/test_feature.py +++ b/sdk/python/tests/test_feature.py @@ -17,7 +17,7 @@ class TestFeatureRef: def test_str_ref(self): - original_ref = FeatureRef(project="test", name="test") + original_ref = FeatureRef(feature_set="test", name="test") ref_str = repr(original_ref) parsed_ref = FeatureRef.from_str(ref_str) assert original_ref == parsed_ref From 0ec49cd96568470abf3acc07818f64ca255c2eaf Mon Sep 17 00:00:00 2001 From: Terence Lim Date: Sat, 30 May 2020 09:45:58 +0800 Subject: [PATCH 029/133] Fix feature_set_describe cli (#731) Co-authored-by: Terence --- sdk/python/feast/cli.py | 12 ++++++++++-- 1 file changed, 10 insertions(+), 2 deletions(-) diff --git a/sdk/python/feast/cli.py b/sdk/python/feast/cli.py index 36d1d489e41..6d1df292764 100644 --- a/sdk/python/feast/cli.py +++ b/sdk/python/feast/cli.py @@ -156,12 +156,20 @@ def feature_set_create(filename): @feature_set.command("describe") @click.argument("name", type=click.STRING) -def feature_set_describe(name: str): +@click.option( + "--project", + "-p", + help="Project that feature set belongs to", + type=click.STRING, + default="default", +) +def feature_set_describe(name: str, project: str): """ Describe a feature set """ feast_client = Client() # type: Client - fs = feast_client.get_feature_set(name=name) + fs = feast_client.get_feature_set(name=name, project=project) + if not fs: print(f'Feature set with name "{name}" could not be found') return From d22fb88c09f1a1dae29d6a40d7393820ee489d7c Mon Sep 17 00:00:00 2001 From: Willem Pienaar <6728866+woop@users.noreply.github.com> Date: Sat, 30 May 2020 13:48:25 +0800 Subject: [PATCH 030/133] Refactor end-to-end tests to reduce duplication (#758) * Externalize common scripts in e2e tests * Fix script directory * Fix script path for e2e-batch test * Externalize redis-cluster e2e test scripts * Add wait-for-it to Kafka and Zookeper initialization --- infra/scripts/setup-common-functions.sh | 132 ++++++++++++ infra/scripts/test-end-to-end-batch.sh | 189 ++--------------- .../scripts/test-end-to-end-redis-cluster.sh | 190 ++--------------- infra/scripts/test-end-to-end.sh | 196 ++---------------- 4 files changed, 180 insertions(+), 527 deletions(-) create mode 100755 infra/scripts/setup-common-functions.sh diff --git a/infra/scripts/setup-common-functions.sh b/infra/scripts/setup-common-functions.sh new file mode 100755 index 00000000000..e158e0f5672 --- /dev/null +++ b/infra/scripts/setup-common-functions.sh @@ -0,0 +1,132 @@ +#!/usr/bin/env bash + +# Get Feast project repository root and scripts directory +export PROJECT_ROOT_DIR=$(git rev-parse --show-toplevel) +export SCRIPTS_DIR=${PROJECT_ROOT_DIR}/infra/scripts + +install_test_tools() { + apt-get -qq update + apt-get -y install wget netcat kafkacat build-essential +} + +install_gcloud_sdk() { + print_banner "Installing Google Cloud SDK" + if [[ ! $(command -v gsutil) ]]; then + CURRENT_DIR=$(dirname "$BASH_SOURCE") + . "${CURRENT_DIR}"/install-google-cloud-sdk.sh + fi + + export GOOGLE_APPLICATION_CREDENTIALS + gcloud auth activate-service-account --key-file ${GOOGLE_APPLICATION_CREDENTIALS} +} + +install_and_start_local_redis() { + print_banner "Installing and tarting Redis at localhost:6379" + # Allow starting serving in this Maven Docker image. Default set to not allowed. + echo "exit 0" >/usr/sbin/policy-rc.d + apt-get -y install redis-server >/var/log/redis.install.log + redis-server --daemonize yes + redis-cli ping +} + +install_and_start_local_redis_cluster() { + print_banner "Installing Redis at localhost:6379" + echo "exit 0" >/usr/sbin/policy-rc.d + ${SCRIPTS_DIR}/setup-redis-cluster.sh + redis-cli -c -p 7000 ping +} + +install_and_start_local_postgres() { + print_banner "Installing and starting Postgres at localhost:5432" + apt-get -y install postgresql >/var/log/postgresql.install.log + service postgresql start + # Initialize with database: 'postgres', user: 'postgres', password: 'password' + cat </tmp/update-postgres-role.sh +psql -c "ALTER USER postgres PASSWORD 'password';" +EOF + chmod +x /tmp/update-postgres-role.sh + su -s /bin/bash -c /tmp/update-postgres-role.sh postgres + export PGPASSWORD=password + pg_isready +} + +install_and_start_local_zookeeper_and_kafka() { + print_banner "Installing and starting Zookeeper at localhost:2181 and Kafka at localhost:9092" + wget -qO- https://www-eu.apache.org/dist/kafka/2.3.0/kafka_2.12-2.3.0.tgz | tar xz + mv kafka_2.12-2.3.0/ /tmp/kafka + + nohup /tmp/kafka/bin/zookeeper-server-start.sh /tmp/kafka/config/zookeeper.properties &>/var/log/zookeeper.log 2>&1 & + ${SCRIPTS_DIR}/wait-for-it.sh localhost:2181 --timeout=20 + tail -n10 /var/log/zookeeper.log + + nohup /tmp/kafka/bin/kafka-server-start.sh /tmp/kafka/config/server.properties &>/var/log/kafka.log 2>&1 & + ${SCRIPTS_DIR}/wait-for-it.sh localhost:9092 --timeout=40 + tail -n10 /var/log/kafka.log + kafkacat -b localhost:9092 -L +} + +build_feast_core_and_serving() { + print_banner "Building Feast Core and Feast Serving" + infra/scripts/download-maven-cache.sh \ + --archive-uri gs://feast-templocation-kf-feast/.m2.2019-10-24.tar \ + --output-dir /root/ + + # Build jars for Feast + mvn --quiet --batch-mode --define skipTests=true clean package + + ls -lh core/target/*jar + ls -lh serving/target/*jar +} + +start_feast_core() { + print_banner "Starting Feast Core" + + if [ -n "$1" ]; then + echo "Custom Spring application.yml location provided: $1" + export CONFIG_ARG="--spring.config.location=file://$1" + fi + + nohup java -jar core/target/feast-core-$FEAST_BUILD_VERSION.jar $CONFIG_ARG &>/var/log/feast-core.log & + ${SCRIPTS_DIR}/wait-for-it.sh localhost:6565 --timeout=90 + + tail -n10 /var/log/feast-core.log + nc -w2 localhost 6565 /var/log/feast-serving-online.log & + ${SCRIPTS_DIR}/wait-for-it.sh localhost:6566 --timeout=60 + + tail -n100 /var/log/feast-serving-online.log + nc -w2 localhost 6566 /usr/sbin/policy-rc.d -apt-get -y install redis-server > /var/log/redis.install.log -redis-server --daemonize yes -redis-cli ping - -echo " -============================================================ -Installing Postgres at localhost:5432 -============================================================ -" -apt-get -y install postgresql > /var/log/postgresql.install.log -service postgresql start -# Initialize with database: 'postgres', user: 'postgres', password: 'password' -cat < /tmp/update-postgres-role.sh -psql -c "ALTER USER postgres PASSWORD 'password';" -EOF -chmod +x /tmp/update-postgres-role.sh -su -s /bin/bash -c /tmp/update-postgres-role.sh postgres -export PGPASSWORD=password -pg_isready +source ${SCRIPTS_DIR}/setup-common-functions.sh -echo " -============================================================ -Installing Zookeeper at localhost:2181 -Installing Kafka at localhost:9092 -============================================================ -" -wget -qO- https://www-eu.apache.org/dist/kafka/2.3.0/kafka_2.12-2.3.0.tgz | tar xz -mv kafka_2.12-2.3.0/ /tmp/kafka -nohup /tmp/kafka/bin/zookeeper-server-start.sh /tmp/kafka/config/zookeeper.properties &> /var/log/zookeeper.log 2>&1 & -sleep 5 -tail -n10 /var/log/zookeeper.log -nohup /tmp/kafka/bin/kafka-server-start.sh /tmp/kafka/config/server.properties &> /var/log/kafka.log 2>&1 & -sleep 20 -tail -n10 /var/log/kafka.log -kafkacat -b localhost:9092 -L +install_test_tools +install_gcloud_sdk +install_and_start_local_redis +install_and_start_local_postgres +install_and_start_local_zookeeper_and_kafka if [[ ${SKIP_BUILD_JARS} != "true" ]]; then - echo " - ============================================================ - Building jars for Feast - ============================================================ - " - - infra/scripts/download-maven-cache.sh \ - --archive-uri gs://feast-templocation-kf-feast/.m2.2019-10-24.tar \ - --output-dir /root/ - - # Build jars for Feast - mvn --quiet --batch-mode --define skipTests=true clean package - - ls -lh core/target/*jar - ls -lh serving/target/*jar + build_feast_core_and_serving else echo "[DEBUG] Skipping building jars" fi -echo " -============================================================ -Starting Feast Core -============================================================ -" -# Start Feast Core in background -cat < /tmp/core.application.yml -grpc: - port: 6565 - enable-reflection: true - -feast: - jobs: - polling_interval_milliseconds: 10000 - job_update_timeout_seconds: 240 - - active_runner: direct - - runners: - - name: direct - type: DirectRunner - options: {} - - metrics: - enabled: false - - stream: - type: kafka - options: - topic: feast-features - bootstrapServers: localhost:9092 - replicationFactor: 1 - partitions: 1 - -spring: - jpa: - properties.hibernate: - format_sql: true - event: - merge: - entity_copy_observer: allow - hibernate.naming.physical-strategy=org.hibernate.boot.model.naming: PhysicalNamingStrategyStandardImpl - hibernate.ddl-auto: update - datasource: - url: jdbc:postgresql://localhost:5432/postgres - username: postgres - password: password -EOF - -nohup java -jar core/target/feast-core-${FEAST_BUILD_VERSION}.jar \ - --spring.config.location=file:///tmp/core.application.yml \ - &> /var/log/feast-core.log & -sleep 35 -tail -n10 /var/log/feast-core.log -nc -w2 localhost 6565 < /dev/null - -echo " -============================================================ -Starting Feast Warehouse Serving -============================================================ -" +export FEAST_JOBS_POLLING_INTERVAL_MILLISECONDS=10000 +start_feast_core DATASET_NAME=feast_$(date +%s) - bq --location=US --project_id=${GOOGLE_CLOUD_PROJECT} mk \ --dataset \ --default_table_expiration 86400 \ @@ -232,35 +108,11 @@ server: EOF -nohup java -jar serving/target/feast-serving-${FEAST_BUILD_VERSION}.jar \ - --spring.config.location=file:///tmp/serving.warehouse.application.yml \ - &> /var/log/feast-serving-warehouse.log & -sleep 15 -tail -n100 /var/log/feast-serving-warehouse.log -nc -w2 localhost 6566 < /dev/null +start_feast_serving /tmp/serving.warehouse.application.yml -echo " -============================================================ -Installing Python 3.7 with Miniconda and Feast SDK -============================================================ -" -# Install Python 3.7 with Miniconda -wget -q https://repo.continuum.io/miniconda/Miniconda3-4.7.12-Linux-x86_64.sh \ - -O /tmp/miniconda.sh -bash /tmp/miniconda.sh -b -p /root/miniconda -f -/root/miniconda/bin/conda init -source ~/.bashrc - -# Install Feast Python SDK and test requirements -make compile-protos-python -pip install -qe sdk/python -pip install -qr tests/e2e/requirements.txt +install_python_with_miniconda_and_feast_sdk -echo " -============================================================ -Running end-to-end tests with pytest at 'tests/e2e' -============================================================ -" +print_banner "Running end-to-end tests with pytest at 'tests/e2e'" # Default artifact location setting in Prow jobs LOGS_ARTIFACT_PATH=/logs/artifacts @@ -282,12 +134,7 @@ fi cd ${ORIGINAL_DIR} -echo " -============================================================ -Cleaning up -============================================================ -" +print_banner "Cleaning up" bq rm -r -f ${GOOGLE_CLOUD_PROJECT}:${DATASET_NAME} - exit ${TEST_EXIT_CODE} \ No newline at end of file diff --git a/infra/scripts/test-end-to-end-redis-cluster.sh b/infra/scripts/test-end-to-end-redis-cluster.sh index de41edf4741..a11aebf0197 100755 --- a/infra/scripts/test-end-to-end-redis-cluster.sh +++ b/infra/scripts/test-end-to-end-redis-cluster.sh @@ -10,9 +10,13 @@ test -z ${TEMP_BUCKET} && TEMP_BUCKET="feast-templocation-kf-feast" test -z ${JOBS_STAGING_LOCATION} && JOBS_STAGING_LOCATION="gs://${TEMP_BUCKET}/staging-location" # Get the current build version using maven (and pom.xml) -FEAST_BUILD_VERSION=$(mvn help:evaluate -Dexpression=project.version -q -DforceStdout) +export FEAST_BUILD_VERSION=$(mvn help:evaluate -Dexpression=project.version -q -DforceStdout) echo Building version: $FEAST_BUILD_VERSION +# Get Feast project repository root and scripts directory +export PROJECT_ROOT_DIR=$(git rev-parse --show-toplevel) +export SCRIPTS_DIR=${PROJECT_ROOT_DIR}/infra/scripts + echo " This script will run end-to-end tests for Feast Core and Online Serving. @@ -23,161 +27,20 @@ This script will run end-to-end tests for Feast Core and Online Serving. tests/e2e via pytest. " -apt-get -qq update -apt-get -y install wget netcat kafkacat build-essential - -echo " -============================================================ -Installing Redis at localhost:6379 -============================================================ -" -# Allow starting serving in this Maven Docker image. Default set to not allowed. -echo "exit 0" > /usr/sbin/policy-rc.d -infra/scripts/setup-redis-cluster.sh -redis-cli -c -p 7000 ping - -echo " -============================================================ -Installing Postgres at localhost:5432 -============================================================ -" -apt-get -y install postgresql > /var/log/postgresql.install.log -service postgresql start -# Initialize with database: 'postgres', user: 'postgres', password: 'password' -cat < /tmp/update-postgres-role.sh -psql -c "ALTER USER postgres PASSWORD 'password';" -EOF -chmod +x /tmp/update-postgres-role.sh -su -s /bin/bash -c /tmp/update-postgres-role.sh postgres -export PGPASSWORD=password -pg_isready +source ${SCRIPTS_DIR}/setup-common-functions.sh -echo " -============================================================ -Installing Zookeeper at localhost:2181 -Installing Kafka at localhost:9092 -============================================================ -" -wget -qO- https://www-eu.apache.org/dist/kafka/2.3.0/kafka_2.12-2.3.0.tgz | tar xz -mv kafka_2.12-2.3.0/ /tmp/kafka -nohup /tmp/kafka/bin/zookeeper-server-start.sh /tmp/kafka/config/zookeeper.properties &> /var/log/zookeeper.log 2>&1 & -sleep 5 -tail -n10 /var/log/zookeeper.log -nohup /tmp/kafka/bin/kafka-server-start.sh /tmp/kafka/config/server.properties &> /var/log/kafka.log 2>&1 & -sleep 20 -tail -n10 /var/log/kafka.log -kafkacat -b localhost:9092 -L +install_test_tools +install_and_start_local_redis_cluster +install_and_start_local_postgres +install_and_start_local_zookeeper_and_kafka if [[ ${SKIP_BUILD_JARS} != "true" ]]; then -echo " -============================================================ -Building jars for Feast -============================================================ -" - -infra/scripts/download-maven-cache.sh \ - --archive-uri gs://feast-templocation-kf-feast/.m2.2019-10-24.tar \ - --output-dir /root/ - -# Build jars for Feast -mvn --quiet --batch-mode --define skipTests=true clean package - -ls -lh core/target/*jar -ls -lh serving/target/*jar + build_feast_core_and_serving else echo "[DEBUG] Skipping building jars" fi -echo " -============================================================ -Starting Feast Core -============================================================ -" -# Start Feast Core in background -cat < /tmp/core.application.yml -grpc: - port: 6565 - enable-reflection: true - -feast: - jobs: - polling_interval_milliseconds: 30000 - job_update_timeout_seconds: 240 - - active_runner: direct - - runners: - - name: direct - type: DirectRunner - options: {} - - metrics: - enabled: false - - stream: - type: kafka - options: - topic: feast-features - bootstrapServers: localhost:9092 - replicationFactor: 1 - partitions: 1 - -spring: - jpa: - properties.hibernate: - format_sql: true - event.merge.entity_copy_observer: allow - hibernate.naming.physical-strategy=org.hibernate.boot.model.naming: PhysicalNamingStrategyStandardImpl - hibernate.ddl-auto: update - datasource: - url: jdbc:postgresql://localhost:5432/postgres - username: postgres - password: password - -management: - metrics: - export: - simple: - enabled: false - statsd: - enabled: false -EOF - -nohup java -jar core/target/feast-core-${FEAST_BUILD_VERSION}.jar \ - --spring.config.location=file:///tmp/core.application.yml \ - &> /var/log/feast-core.log & -sleep 35 -tail -n10 /var/log/feast-core.log -nc -w2 localhost 6565 < /dev/null - -echo " -============================================================ -Starting Feast Online Serving -============================================================ -" -# Start Feast Online Serving in background -cat < /tmp/serving.store.redis.cluster.yml -name: serving -type: REDIS_CLUSTER -redis_cluster_config: - nodes: - - host: localhost - port: 7000 - - host: localhost - port: 7001 - - host: localhost - port: 7002 - - host: localhost - port: 7003 - - host: localhost - port: 7004 - - host: localhost - port: 7005 -subscriptions: - - name: "*" - version: "*" - project: "*" -EOF +start_feast_core cat < /tmp/serving.online.application.yml feast: @@ -213,35 +76,12 @@ spring: EOF -nohup java -jar serving/target/feast-serving-${FEAST_BUILD_VERSION}.jar \ - --spring.config.location=file:///tmp/serving.online.application.yml \ - &> /var/log/feast-serving-online.log & -sleep 15 -tail -n100 /var/log/feast-serving-online.log -nc -w2 localhost 6566 < /dev/null +start_feast_serving /tmp/serving.online.application.yml -echo " -============================================================ -Installing Python 3.7 with Miniconda and Feast SDK -============================================================ -" -# Install Python 3.7 with Miniconda -wget -q https://repo.continuum.io/miniconda/Miniconda3-4.7.12-Linux-x86_64.sh \ - -O /tmp/miniconda.sh -bash /tmp/miniconda.sh -b -p /root/miniconda -f -/root/miniconda/bin/conda init -source ~/.bashrc +install_python_with_miniconda_and_feast_sdk -# Install Feast Python SDK and test requirements -make compile-protos-python -pip install -qe sdk/python -pip install -qr tests/e2e/requirements.txt +print_banner "Running end-to-end tests with pytest at 'tests/e2e'" -echo " -============================================================ -Running end-to-end tests with pytest at 'tests/e2e' -============================================================ -" # Default artifact location setting in Prow jobs LOGS_ARTIFACT_PATH=/logs/artifacts diff --git a/infra/scripts/test-end-to-end.sh b/infra/scripts/test-end-to-end.sh index 452d646f2f2..c90059b7a05 100755 --- a/infra/scripts/test-end-to-end.sh +++ b/infra/scripts/test-end-to-end.sh @@ -10,9 +10,13 @@ test -z ${TEMP_BUCKET} && TEMP_BUCKET="feast-templocation-kf-feast" test -z ${JOBS_STAGING_LOCATION} && JOBS_STAGING_LOCATION="gs://${TEMP_BUCKET}/staging-location" # Get the current build version using maven (and pom.xml) -FEAST_BUILD_VERSION=$(mvn help:evaluate -Dexpression=project.version -q -DforceStdout) +export FEAST_BUILD_VERSION=$(mvn help:evaluate -Dexpression=project.version -q -DforceStdout) echo Building version: $FEAST_BUILD_VERSION +# Get Feast project repository root and scripts directory +export PROJECT_ROOT_DIR=$(git rev-parse --show-toplevel) +export SCRIPTS_DIR=${PROJECT_ROOT_DIR}/infra/scripts + echo " This script will run end-to-end tests for Feast Core and Online Serving. @@ -23,195 +27,25 @@ This script will run end-to-end tests for Feast Core and Online Serving. tests/e2e via pytest. " -apt-get -qq update -apt-get -y install wget netcat kafkacat build-essential - -echo " -============================================================ -Installing Redis at localhost:6379 -============================================================ -" -# Allow starting serving in this Maven Docker image. Default set to not allowed. -echo "exit 0" > /usr/sbin/policy-rc.d -apt-get -y install redis-server > /var/log/redis.install.log -redis-server --daemonize yes -redis-cli ping - -echo " -============================================================ -Installing Postgres at localhost:5432 -============================================================ -" -apt-get -y install postgresql > /var/log/postgresql.install.log -service postgresql start -# Initialize with database: 'postgres', user: 'postgres', password: 'password' -cat < /tmp/update-postgres-role.sh -psql -c "ALTER USER postgres PASSWORD 'password';" -EOF -chmod +x /tmp/update-postgres-role.sh -su -s /bin/bash -c /tmp/update-postgres-role.sh postgres -export PGPASSWORD=password -pg_isready +source ${SCRIPTS_DIR}/setup-common-functions.sh -echo " -============================================================ -Installing Zookeeper at localhost:2181 -Installing Kafka at localhost:9092 -============================================================ -" -wget -qO- https://www-eu.apache.org/dist/kafka/2.3.0/kafka_2.12-2.3.0.tgz | tar xz -mv kafka_2.12-2.3.0/ /tmp/kafka -nohup /tmp/kafka/bin/zookeeper-server-start.sh /tmp/kafka/config/zookeeper.properties &> /var/log/zookeeper.log 2>&1 & -sleep 5 -tail -n10 /var/log/zookeeper.log -nohup /tmp/kafka/bin/kafka-server-start.sh /tmp/kafka/config/server.properties &> /var/log/kafka.log 2>&1 & -sleep 20 -tail -n10 /var/log/kafka.log -kafkacat -b localhost:9092 -L +install_test_tools +install_and_start_local_redis +install_and_start_local_postgres +install_and_start_local_zookeeper_and_kafka if [[ ${SKIP_BUILD_JARS} != "true" ]]; then -echo " -============================================================ -Building jars for Feast -============================================================ -" - -infra/scripts/download-maven-cache.sh \ - --archive-uri gs://feast-templocation-kf-feast/.m2.2019-10-24.tar \ - --output-dir /root/ - -# Build jars for Feast -mvn --quiet --batch-mode --define skipTests=true clean package - -ls -lh core/target/*jar -ls -lh serving/target/*jar + build_feast_core_and_serving else echo "[DEBUG] Skipping building jars" fi -echo " -============================================================ -Starting Feast Core -============================================================ -" -# Start Feast Core in background -cat < /tmp/core.application.yml -grpc: - port: 6565 - enable-reflection: true - -feast: - jobs: - polling_interval_milliseconds: 30000 - job_update_timeout_seconds: 240 - - active_runner: direct - - runners: - - name: direct - type: DirectRunner - options: {} - - metrics: - enabled: false - - stream: - type: kafka - options: - topic: feast-features - bootstrapServers: localhost:9092 - replicationFactor: 1 - partitions: 1 - -spring: - jpa: - properties.hibernate: - format_sql: true - event: - merge: - entity_copy_observer: allow - hibernate.naming.physical-strategy=org.hibernate.boot.model.naming: PhysicalNamingStrategyStandardImpl - hibernate.ddl-auto: update - datasource: - url: jdbc:postgresql://localhost:5432/postgres - username: postgres - password: password - -EOF - -nohup java -jar core/target/feast-core-$FEAST_BUILD_VERSION.jar \ - --spring.config.location=file:///tmp/core.application.yml \ - &> /var/log/feast-core.log & -sleep 35 -tail -n10 /var/log/feast-core.log -nc -w2 localhost 6565 < /dev/null +start_feast_core +start_feast_serving +install_python_with_miniconda_and_feast_sdk -echo " -============================================================ -Starting Feast Online Serving -============================================================ -" - -cat < /tmp/serving.online.application.yml -feast: - core-host: localhost - core-grpc-port: 6565 - - active_store: serving - - # List of store configurations - stores: - - name: serving - type: REDIS # Type of the store. REDIS, BIGQUERY are available options - config: - host: localhost - port: 6379 - subscriptions: - - name: "*" - project: "*" - version: "*" - - tracing: - enabled: false - -grpc: - port: 6566 - enable-reflection: true - -server: - port: 8081 +print_banner "Running end-to-end tests with pytest at 'tests/e2e'" -EOF - -nohup java -jar serving/target/feast-serving-${FEAST_BUILD_VERSION}.jar \ - --spring.config.location=file:///tmp/serving.online.application.yml \ - &> /var/log/feast-serving-online.log & -sleep 15 -tail -n100 /var/log/feast-serving-online.log -nc -w2 localhost 6566 < /dev/null - -echo " -============================================================ -Installing Python 3.7 with Miniconda and Feast SDK -============================================================ -" -# Install Python 3.7 with Miniconda -wget -q https://repo.continuum.io/miniconda/Miniconda3-4.7.12-Linux-x86_64.sh \ - -O /tmp/miniconda.sh -bash /tmp/miniconda.sh -b -p /root/miniconda -f -/root/miniconda/bin/conda init -source ~/.bashrc - -# Install Feast Python SDK and test requirements -make compile-protos-python -pip install -qe sdk/python -pip install -qr tests/e2e/requirements.txt - -echo " -============================================================ -Running end-to-end tests with pytest at 'tests/e2e' -============================================================ -" # Default artifact location setting in Prow jobs LOGS_ARTIFACT_PATH=/logs/artifacts From a5c6dce3cbe75c20bb2e033bee2a4d9100f8295b Mon Sep 17 00:00:00 2001 From: Willem Pienaar <6728866+woop@users.noreply.github.com> Date: Sat, 30 May 2020 13:50:14 +0800 Subject: [PATCH 031/133] Add support for version method in Feast SDK and Core (#759) * Add support for version method * Fix typo in e2e tests for version method --- .../java/feast/core/grpc/CoreServiceImpl.java | 17 +++++++++-- sdk/python/feast/client.py | 8 +++++- tests/e2e/basic-ingest-redis-serving.py | 28 ++++++++++++++----- 3 files changed, 43 insertions(+), 10 deletions(-) diff --git a/core/src/main/java/feast/core/grpc/CoreServiceImpl.java b/core/src/main/java/feast/core/grpc/CoreServiceImpl.java index 412d3d31198..db65328b821 100644 --- a/core/src/main/java/feast/core/grpc/CoreServiceImpl.java +++ b/core/src/main/java/feast/core/grpc/CoreServiceImpl.java @@ -18,6 +18,7 @@ import com.google.api.gax.rpc.InvalidArgumentException; import com.google.protobuf.InvalidProtocolBufferException; +import feast.core.config.FeastProperties; import feast.core.exception.RetrievalException; import feast.core.grpc.interceptors.MonitoringInterceptor; import feast.core.model.Project; @@ -64,6 +65,7 @@ @GRpcService(interceptors = {MonitoringInterceptor.class}) public class CoreServiceImpl extends CoreServiceImplBase { + private final FeastProperties feastProperties; private SpecService specService; private AccessManagementService accessManagementService; private JobService jobService; @@ -72,17 +74,28 @@ public class CoreServiceImpl extends CoreServiceImplBase { public CoreServiceImpl( SpecService specService, AccessManagementService accessManagementService, - JobService jobService) { + JobService jobService, + FeastProperties feastProperties) { this.specService = specService; this.accessManagementService = accessManagementService; this.jobService = jobService; + this.feastProperties = feastProperties; } @Override public void getFeastCoreVersion( GetFeastCoreVersionRequest request, StreamObserver responseObserver) { - super.getFeastCoreVersion(request, responseObserver); + try { + GetFeastCoreVersionResponse response = + GetFeastCoreVersionResponse.newBuilder().setVersion(feastProperties.getVersion()).build(); + responseObserver.onNext(response); + responseObserver.onCompleted(); + } catch (RetrievalException | StatusRuntimeException e) { + log.error("Could not determine Feast Core version: ", e); + responseObserver.onError( + Status.INTERNAL.withDescription(e.getMessage()).withCause(e).asRuntimeException()); + } } @Override diff --git a/sdk/python/feast/client.py b/sdk/python/feast/client.py index 5e4baf9d598..1c34213ad88 100644 --- a/sdk/python/feast/client.py +++ b/sdk/python/feast/client.py @@ -195,7 +195,13 @@ def version(self): """ Returns version information from Feast Core and Feast Serving """ - result = {} + import pkg_resources + + result = { + "sdk": {"version": pkg_resources.get_distribution("feast").version}, + "serving": "not configured", + "core": "not configured", + } if self.serving_url: self._connect_serving() diff --git a/tests/e2e/basic-ingest-redis-serving.py b/tests/e2e/basic-ingest-redis-serving.py index 4209ce1c220..da0967fd764 100644 --- a/tests/e2e/basic-ingest-redis-serving.py +++ b/tests/e2e/basic-ingest-redis-serving.py @@ -30,6 +30,7 @@ FLOAT_TOLERANCE = 0.00001 PROJECT_NAME = 'basic_' + uuid.uuid4().hex.upper()[0:6] + @pytest.fixture(scope='module') def core_url(pytestconfig): return pytestconfig.getoption("core_url") @@ -62,6 +63,7 @@ def client(core_url, serving_url, allow_dirty): return client + def basic_dataframe(entities, features, ingest_time, n_size): offset = random.randint(1000, 100000) # ensure a unique key space is used df_dict = { @@ -79,6 +81,7 @@ def basic_dataframe(entities, features, ingest_time, n_size): def ingest_time(): return datetime.utcnow() + @pytest.fixture(scope="module") def cust_trans_df(ingest_time): return basic_dataframe(entities=["customer_id"], @@ -86,6 +89,7 @@ def cust_trans_df(ingest_time): ingest_time=ingest_time, n_size=5) + @pytest.fixture(scope="module") def driver_df(ingest_time): return basic_dataframe(entities=["driver_id"], @@ -93,6 +97,13 @@ def driver_df(ingest_time): ingest_time=ingest_time, n_size=5) + +def test_version_returns_results(client): + version_info = client.version() + assert not version_info['core'] is 'not configured' + assert not version_info['serving'] is 'not configured' + + @pytest.mark.timeout(45) @pytest.mark.run(order=10) def test_basic_register_feature_set_success(client): @@ -117,6 +128,7 @@ def test_basic_register_feature_set_success(client): # reset client's project for other tests client.set_project() + @pytest.mark.timeout(300) @pytest.mark.run(order=11) def test_basic_ingest_success(client, cust_trans_df, driver_df): @@ -157,16 +169,16 @@ def test_basic_retrieve_online_success(client, cust_trans_df): returned_daily_transactions = float( response.field_values[0] - .fields["daily_transactions"] - .float_val + .fields["daily_transactions"] + .float_val ) sent_daily_transactions = float( cust_trans_df.iloc[0]["daily_transactions"]) if math.isclose( - sent_daily_transactions, - returned_daily_transactions, - abs_tol=FLOAT_TOLERANCE, + sent_daily_transactions, + returned_daily_transactions, + abs_tol=FLOAT_TOLERANCE, ): break @@ -206,8 +218,8 @@ def test_basic_retrieve_online_multiple_featureset(client, cust_trans_df, driver def check_response(ingest_df, response, feature_ref): returned_value = float( response.field_values[0] - .fields[feature_ref] - .float_val + .fields[feature_ref] + .float_val ) feature_ref_splits = feature_ref.split(":") if len(feature_ref_splits) == 1: @@ -223,9 +235,11 @@ def check_response(ingest_df, response, feature_ref): returned_value, abs_tol=FLOAT_TOLERANCE, ) + if all([check_response(df, response, ref) for ref, df in feature_ref_df_mapping]): break + @pytest.mark.timeout(300) @pytest.mark.run(order=19) def test_basic_ingest_jobs(client): From f9ce8ee7291a1b9c0ead061069715db26bd604ab Mon Sep 17 00:00:00 2001 From: Chen Zhiling Date: Mon, 1 Jun 2020 10:32:59 +0800 Subject: [PATCH 032/133] Refactor runner configuration, add labels to dataflow options (#718) * Refactor runner configuration, add labels to dataflow options * Remove util method * Add javadocs for RunnerConfig class * Apply spotless --- .../feast/core/config/FeastProperties.java | 2 +- .../java/feast/core/config/JobConfig.java | 24 +++++- .../core/job/dataflow/DataflowJobManager.java | 23 +++--- .../job/dataflow/DataflowRunnerConfig.java | 78 ++++++++----------- .../core/job/direct/DirectRunnerConfig.java | 36 +++++++++ .../job/direct/DirectRunnerJobManager.java | 15 ++-- .../feast/core/job/option/RunnerConfig.java | 75 ++++++++++++++++++ .../java/feast/core/util/TypeConversion.java | 16 ---- .../job/dataflow/DataflowJobManagerTest.java | 23 +++--- .../direct/DirectRunnerJobManagerTest.java | 9 +-- .../feast/core/util/TypeConversionTest.java | 12 --- protos/feast/core/Runner.proto | 3 + 12 files changed, 202 insertions(+), 114 deletions(-) create mode 100644 core/src/main/java/feast/core/job/direct/DirectRunnerConfig.java create mode 100644 core/src/main/java/feast/core/job/option/RunnerConfig.java diff --git a/core/src/main/java/feast/core/config/FeastProperties.java b/core/src/main/java/feast/core/config/FeastProperties.java index eb50728baf5..6dad2782429 100644 --- a/core/src/main/java/feast/core/config/FeastProperties.java +++ b/core/src/main/java/feast/core/config/FeastProperties.java @@ -103,7 +103,7 @@ public static class Runner { * Job runner configuration options. See the following for options * https://api.docs.feast.dev/grpc/feast.core.pb.html#Runner */ - Map options = new HashMap<>(); + Map options = new HashMap<>(); /** * Gets the job runner type as an enum. diff --git a/core/src/main/java/feast/core/config/JobConfig.java b/core/src/main/java/feast/core/config/JobConfig.java index 69636963bea..30023de0641 100644 --- a/core/src/main/java/feast/core/config/JobConfig.java +++ b/core/src/main/java/feast/core/config/JobConfig.java @@ -16,11 +16,16 @@ */ package feast.core.config; +import com.google.gson.Gson; +import com.google.protobuf.InvalidProtocolBufferException; +import com.google.protobuf.util.JsonFormat; import feast.core.config.FeastProperties.JobProperties; import feast.core.job.JobManager; import feast.core.job.dataflow.DataflowJobManager; import feast.core.job.direct.DirectJobRegistry; import feast.core.job.direct.DirectRunnerJobManager; +import feast.proto.core.RunnerProto.DataflowRunnerConfigOptions; +import feast.proto.core.RunnerProto.DirectRunnerConfigOptions; import java.util.Map; import lombok.extern.slf4j.Slf4j; import org.springframework.beans.factory.annotation.Autowired; @@ -31,6 +36,7 @@ @Slf4j @Configuration public class JobConfig { + private final Gson gson = new Gson(); /** * Get a JobManager according to the runner type and Dataflow configuration. @@ -39,18 +45,28 @@ public class JobConfig { */ @Bean @Autowired - public JobManager getJobManager(FeastProperties feastProperties) { + public JobManager getJobManager(FeastProperties feastProperties) + throws InvalidProtocolBufferException { JobProperties jobProperties = feastProperties.getJobs(); FeastProperties.JobProperties.Runner runner = jobProperties.getActiveRunner(); - Map runnerConfigOptions = runner.getOptions(); + Map runnerConfigOptions = runner.getOptions(); + String configJson = gson.toJson(runnerConfigOptions); + FeastProperties.MetricsProperties metrics = jobProperties.getMetrics(); switch (runner.getType()) { case DATAFLOW: - return new DataflowJobManager(runnerConfigOptions, metrics); + DataflowRunnerConfigOptions.Builder dataflowRunnerConfigOptions = + DataflowRunnerConfigOptions.newBuilder(); + JsonFormat.parser().merge(configJson, dataflowRunnerConfigOptions); + return new DataflowJobManager(dataflowRunnerConfigOptions.build(), metrics); case DIRECT: - return new DirectRunnerJobManager(runnerConfigOptions, new DirectJobRegistry(), metrics); + DirectRunnerConfigOptions.Builder directRunnerConfigOptions = + DirectRunnerConfigOptions.newBuilder(); + JsonFormat.parser().merge(configJson, directRunnerConfigOptions); + return new DirectRunnerJobManager( + directRunnerConfigOptions.build(), new DirectJobRegistry(), metrics); default: throw new IllegalArgumentException("Unsupported runner: " + runner); } diff --git a/core/src/main/java/feast/core/job/dataflow/DataflowJobManager.java b/core/src/main/java/feast/core/job/dataflow/DataflowJobManager.java index 7b9df0abd52..2c3da255f5b 100644 --- a/core/src/main/java/feast/core/job/dataflow/DataflowJobManager.java +++ b/core/src/main/java/feast/core/job/dataflow/DataflowJobManager.java @@ -33,12 +33,12 @@ import feast.core.job.Runner; import feast.core.job.option.FeatureSetJsonByteConverter; import feast.core.model.*; -import feast.core.util.TypeConversion; import feast.ingestion.ImportJob; import feast.ingestion.options.BZip2Compressor; import feast.ingestion.options.ImportOptions; import feast.ingestion.options.OptionCompressor; import feast.proto.core.FeatureSetProto; +import feast.proto.core.RunnerProto.DataflowRunnerConfigOptions; import feast.proto.core.SourceProto; import feast.proto.core.StoreProto; import java.io.IOException; @@ -46,7 +46,6 @@ import java.util.ArrayList; import java.util.Collections; import java.util.List; -import java.util.Map; import lombok.extern.slf4j.Slf4j; import org.apache.beam.runners.dataflow.DataflowPipelineJob; import org.apache.beam.runners.dataflow.DataflowRunner; @@ -61,21 +60,20 @@ public class DataflowJobManager implements JobManager { private final String projectId; private final String location; private final Dataflow dataflow; - private final Map defaultOptions; + private final DataflowRunnerConfig defaultOptions; private final MetricsProperties metrics; public DataflowJobManager( - Map runnerConfigOptions, MetricsProperties metricsProperties) { + DataflowRunnerConfigOptions runnerConfigOptions, MetricsProperties metricsProperties) { this(runnerConfigOptions, metricsProperties, getGoogleCredential()); } public DataflowJobManager( - Map runnerConfigOptions, + DataflowRunnerConfigOptions runnerConfigOptions, MetricsProperties metricsProperties, Credential credential) { - DataflowRunnerConfig config = new DataflowRunnerConfig(runnerConfigOptions); - + defaultOptions = new DataflowRunnerConfig(runnerConfigOptions); Dataflow dataflow = null; try { dataflow = @@ -89,11 +87,10 @@ public DataflowJobManager( throw new IllegalStateException("Unable to initialize DataflowJobManager", e); } - this.defaultOptions = runnerConfigOptions; this.dataflow = dataflow; this.metrics = metricsProperties; - this.projectId = config.getProject(); - this.location = config.getRegion(); + this.projectId = defaultOptions.getProject(); + this.location = defaultOptions.getRegion(); } private static Credential getGoogleCredential() { @@ -270,9 +267,9 @@ private ImportOptions getPipelineOptions( List featureSets, StoreProto.Store sink, boolean update) - throws IOException { - String[] args = TypeConversion.convertMapToArgs(defaultOptions); - ImportOptions pipelineOptions = PipelineOptionsFactory.fromArgs(args).as(ImportOptions.class); + throws IOException, IllegalAccessException { + ImportOptions pipelineOptions = + PipelineOptionsFactory.fromArgs(defaultOptions.toArgs()).as(ImportOptions.class); OptionCompressor> featureSetJsonCompressor = new BZip2Compressor<>(new FeatureSetJsonByteConverter()); diff --git a/core/src/main/java/feast/core/job/dataflow/DataflowRunnerConfig.java b/core/src/main/java/feast/core/job/dataflow/DataflowRunnerConfig.java index 6fe93ca80cd..85628d2cd0e 100644 --- a/core/src/main/java/feast/core/job/dataflow/DataflowRunnerConfig.java +++ b/core/src/main/java/feast/core/job/dataflow/DataflowRunnerConfig.java @@ -16,9 +16,9 @@ */ package feast.core.job.dataflow; -import java.lang.reflect.Field; -import java.util.Map; -import java.util.Set; +import feast.core.job.option.RunnerConfig; +import feast.proto.core.RunnerProto.DataflowRunnerConfigOptions; +import java.util.*; import javax.validation.*; import javax.validation.constraints.NotBlank; import lombok.Getter; @@ -27,77 +27,65 @@ /** DataflowRunnerConfig contains configuration fields for the Dataflow job runner. */ @Getter @Setter -public class DataflowRunnerConfig { - - public DataflowRunnerConfig(Map runnerConfigOptions) { - - // Try to find all fields in DataflowRunnerConfig inside the runnerConfigOptions and map it into - // this object - for (Field field : DataflowRunnerConfig.class.getFields()) { - String fieldName = field.getName(); - try { - if (!runnerConfigOptions.containsKey(fieldName)) { - continue; - } - String value = runnerConfigOptions.get(fieldName); - - if (Boolean.class.equals(field.getType())) { - field.set(this, Boolean.valueOf(value)); - continue; - } - if (field.getType() == Integer.class) { - field.set(this, Integer.valueOf(value)); - continue; - } - field.set(this, value); - } catch (IllegalAccessException e) { - throw new RuntimeException( - String.format( - "Could not successfully convert DataflowRunnerConfig for key: %s", fieldName), - e); - } - } +public class DataflowRunnerConfig extends RunnerConfig { + + public DataflowRunnerConfig(DataflowRunnerConfigOptions runnerConfigOptions) { + this.project = runnerConfigOptions.getProject(); + this.region = runnerConfigOptions.getRegion(); + this.zone = runnerConfigOptions.getZone(); + this.serviceAccount = runnerConfigOptions.getServiceAccount(); + this.network = runnerConfigOptions.getNetwork(); + this.subnetwork = runnerConfigOptions.getSubnetwork(); + this.workerMachineType = runnerConfigOptions.getWorkerMachineType(); + this.autoscalingAlgorithm = runnerConfigOptions.getAutoscalingAlgorithm(); + this.usePublicIps = runnerConfigOptions.getUsePublicIps(); + this.tempLocation = runnerConfigOptions.getTempLocation(); + this.maxNumWorkers = runnerConfigOptions.getMaxNumWorkers(); + this.deadLetterTableSpec = runnerConfigOptions.getDeadLetterTableSpec(); + this.labels = runnerConfigOptions.getLabelsMap(); validate(); } /* Project id to use when launching jobs. */ - @NotBlank public String project; + @NotBlank String project; /* The Google Compute Engine region for creating Dataflow jobs. */ - @NotBlank public String region; + @NotBlank String region; /* GCP availability zone for operations. */ - @NotBlank public String zone; + @NotBlank String zone; /* Run the job as a specific service account, instead of the default GCE robot. */ - public String serviceAccount; + String serviceAccount; /* GCE network for launching workers. */ - @NotBlank public String network; + @NotBlank String network; /* GCE subnetwork for launching workers. */ - @NotBlank public String subnetwork; + @NotBlank String subnetwork; /* Machine type to create Dataflow worker VMs as. */ - public String workerMachineType; + String workerMachineType; /* The autoscaling algorithm to use for the workerpool. */ - public String autoscalingAlgorithm; + String autoscalingAlgorithm; /* Specifies whether worker pools should be started with public IP addresses. */ - public Boolean usePublicIps; + Boolean usePublicIps; /** * A pipeline level default location for storing temporary files. Support Google Cloud Storage * locations, e.g. gs://bucket/object */ - @NotBlank public String tempLocation; + @NotBlank String tempLocation; /* The maximum number of workers to use for the workerpool. */ - public Integer maxNumWorkers; + Integer maxNumWorkers; /* BigQuery table specification, e.g. PROJECT_ID:DATASET_ID.PROJECT_ID */ - public String deadLetterTableSpec; + String deadLetterTableSpec; + + Map labels; /** Validates Dataflow runner configuration options */ public void validate() { diff --git a/core/src/main/java/feast/core/job/direct/DirectRunnerConfig.java b/core/src/main/java/feast/core/job/direct/DirectRunnerConfig.java new file mode 100644 index 00000000000..ebd327f2f75 --- /dev/null +++ b/core/src/main/java/feast/core/job/direct/DirectRunnerConfig.java @@ -0,0 +1,36 @@ +/* + * SPDX-License-Identifier: Apache-2.0 + * Copyright 2018-2020 The Feast Authors + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package feast.core.job.direct; + +import feast.core.job.option.RunnerConfig; +import feast.proto.core.RunnerProto.DirectRunnerConfigOptions; + +public class DirectRunnerConfig extends RunnerConfig { + /** + * Controls the amount of target parallelism the DirectRunner will use. Defaults to the greater of + * the number of available processors and 3. Must be a value greater than zero. + */ + Integer targetParallelism; + + /* BigQuery table specification, e.g. PROJECT_ID:DATASET_ID.PROJECT_ID */ + String deadletterTableSpec; + + public DirectRunnerConfig(DirectRunnerConfigOptions runnerConfigOptions) { + this.deadletterTableSpec = runnerConfigOptions.getDeadLetterTableSpec(); + this.targetParallelism = runnerConfigOptions.getTargetParallelism(); + } +} diff --git a/core/src/main/java/feast/core/job/direct/DirectRunnerJobManager.java b/core/src/main/java/feast/core/job/direct/DirectRunnerJobManager.java index 2e2b43047ea..715adbdd430 100644 --- a/core/src/main/java/feast/core/job/direct/DirectRunnerJobManager.java +++ b/core/src/main/java/feast/core/job/direct/DirectRunnerJobManager.java @@ -27,18 +27,17 @@ import feast.core.model.Job; import feast.core.model.JobStatus; import feast.core.model.Project; -import feast.core.util.TypeConversion; import feast.ingestion.ImportJob; import feast.ingestion.options.BZip2Compressor; import feast.ingestion.options.ImportOptions; import feast.ingestion.options.OptionCompressor; import feast.proto.core.FeatureSetProto; +import feast.proto.core.RunnerProto.DirectRunnerConfigOptions; import feast.proto.core.StoreProto; import java.io.IOException; import java.util.ArrayList; import java.util.Collections; import java.util.List; -import java.util.Map; import lombok.extern.slf4j.Slf4j; import org.apache.beam.runners.direct.DirectRunner; import org.apache.beam.sdk.PipelineResult; @@ -49,15 +48,15 @@ public class DirectRunnerJobManager implements JobManager { private final Runner RUNNER_TYPE = Runner.DIRECT; - protected Map defaultOptions; + private DirectRunnerConfig defaultOptions; private final DirectJobRegistry jobs; private MetricsProperties metrics; public DirectRunnerJobManager( - Map defaultOptions, + DirectRunnerConfigOptions directRunnerConfigOptions, DirectJobRegistry jobs, MetricsProperties metricsProperties) { - this.defaultOptions = defaultOptions; + this.defaultOptions = new DirectRunnerConfig(directRunnerConfigOptions); this.jobs = jobs; this.metrics = metricsProperties; } @@ -95,9 +94,9 @@ public Job startJob(Job job) { private ImportOptions getPipelineOptions( String jobName, List featureSets, StoreProto.Store sink) - throws IOException { - String[] args = TypeConversion.convertMapToArgs(defaultOptions); - ImportOptions pipelineOptions = PipelineOptionsFactory.fromArgs(args).as(ImportOptions.class); + throws IOException, IllegalAccessException { + ImportOptions pipelineOptions = + PipelineOptionsFactory.fromArgs(defaultOptions.toArgs()).as(ImportOptions.class); OptionCompressor> featureSetJsonCompressor = new BZip2Compressor<>(new FeatureSetJsonByteConverter()); diff --git a/core/src/main/java/feast/core/job/option/RunnerConfig.java b/core/src/main/java/feast/core/job/option/RunnerConfig.java new file mode 100644 index 00000000000..4b937074a31 --- /dev/null +++ b/core/src/main/java/feast/core/job/option/RunnerConfig.java @@ -0,0 +1,75 @@ +/* + * SPDX-License-Identifier: Apache-2.0 + * Copyright 2018-2020 The Feast Authors + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package feast.core.job.option; + +import feast.core.util.TypeConversion; +import java.lang.reflect.Field; +import java.util.ArrayList; +import java.util.List; +import java.util.Map; + +/** + * Value class containing the application-default configuration for a runner. When a job is started + * by core, all fields in the object will be converted into --key=value args to seed the beam + * pipeline options. + */ +public abstract class RunnerConfig { + + /** + * Converts the fields in this class to a list of --key=value args to be passed to a {@link + * org.apache.beam.sdk.options.PipelineOptionsFactory}. + * + *

Ignores values that are proto-default (e.g. empty string, 0). + * + * @return Array of string args in the format --key=value. + * @throws IllegalAccessException + */ + public String[] toArgs() throws IllegalAccessException { + List args = new ArrayList<>(); + for (Field field : this.getClass().getFields()) { + if (field.get(this) == null) { + continue; + } + Class type = field.getType(); + if (Map.class.equals(type)) { + String jsonString = + TypeConversion.convertMapToJsonString((Map) field.get(this)); + args.add(String.format("--%s=%s", field.getName(), jsonString)); + continue; + } + + if (String.class.equals(type)) { + String val = (String) field.get(this); + if (!val.equals("")) { + args.add(String.format("--%s=%s", field.getName(), val)); + } + continue; + } + + if (Integer.class.equals(type)) { + Integer val = (Integer) field.get(this); + if (val != 0) { + args.add(String.format("--%s=%d", field.getName(), val)); + } + continue; + } + + args.add(String.format("--%s=%s", field.getName(), field.get(this))); + } + return args.toArray(String[]::new); + } +} diff --git a/core/src/main/java/feast/core/util/TypeConversion.java b/core/src/main/java/feast/core/util/TypeConversion.java index 6ee990fc1c5..e6b7ef33cbc 100644 --- a/core/src/main/java/feast/core/util/TypeConversion.java +++ b/core/src/main/java/feast/core/util/TypeConversion.java @@ -16,12 +16,10 @@ */ package feast.core.util; -import com.google.common.base.Strings; import com.google.gson.Gson; import com.google.gson.reflect.TypeToken; import java.lang.reflect.Type; import java.util.*; -import java.util.Map.Entry; public class TypeConversion { private static Gson gson = new Gson(); @@ -72,18 +70,4 @@ public static Map convertJsonStringToMap(String jsonString) { public static String convertMapToJsonString(Map map) { return gson.toJson(map); } - - /** - * Convert a map of key value pairs to a array of java arguments in format --key=value - * - * @param map - * @return array of string arguments - */ - public static String[] convertMapToArgs(Map map) { - List args = new ArrayList<>(); - for (Entry arg : map.entrySet()) { - args.add(Strings.lenientFormat("--%s=%s", arg.getKey(), arg.getValue())); - } - return args.toArray(new String[] {}); - } } diff --git a/core/src/test/java/feast/core/job/dataflow/DataflowJobManagerTest.java b/core/src/test/java/feast/core/job/dataflow/DataflowJobManagerTest.java index 632c9d22a25..ea9caa91ff6 100644 --- a/core/src/test/java/feast/core/job/dataflow/DataflowJobManagerTest.java +++ b/core/src/test/java/feast/core/job/dataflow/DataflowJobManagerTest.java @@ -40,6 +40,8 @@ import feast.proto.core.FeatureSetProto; import feast.proto.core.FeatureSetProto.FeatureSetMeta; import feast.proto.core.FeatureSetProto.FeatureSetSpec; +import feast.proto.core.RunnerProto.DataflowRunnerConfigOptions; +import feast.proto.core.RunnerProto.DataflowRunnerConfigOptions.Builder; import feast.proto.core.SourceProto; import feast.proto.core.SourceProto.KafkaSourceConfig; import feast.proto.core.SourceProto.SourceType; @@ -49,9 +51,7 @@ import feast.proto.core.StoreProto.Store.Subscription; import java.io.IOException; import java.util.Collections; -import java.util.HashMap; import java.util.List; -import java.util.Map; import org.apache.beam.runners.dataflow.DataflowPipelineJob; import org.apache.beam.runners.dataflow.DataflowRunner; import org.apache.beam.sdk.PipelineResult.State; @@ -70,19 +70,21 @@ public class DataflowJobManagerTest { @Mock private Dataflow dataflow; - private Map defaults; + private DataflowRunnerConfigOptions defaults; private DataflowJobManager dfJobManager; @Before public void setUp() { initMocks(this); - defaults = new HashMap<>(); - defaults.put("project", "project"); - defaults.put("region", "region"); - defaults.put("zone", "zone"); - defaults.put("tempLocation", "tempLocation"); - defaults.put("network", "network"); - defaults.put("subnetwork", "subnetwork"); + Builder optionsBuilder = DataflowRunnerConfigOptions.newBuilder(); + optionsBuilder.setProject("project"); + optionsBuilder.setRegion("region"); + optionsBuilder.setZone("zone"); + optionsBuilder.setTempLocation("tempLocation"); + optionsBuilder.setNetwork("network"); + optionsBuilder.setSubnetwork("subnetwork"); + optionsBuilder.putLabels("orchestrator", "feast"); + defaults = optionsBuilder.build(); MetricsProperties metricsProperties = new MetricsProperties(); metricsProperties.setEnabled(false); Credential credential = null; @@ -137,6 +139,7 @@ public void shouldStartJobWithCorrectPipelineOptions() throws IOException { expectedPipelineOptions.setRegion("region"); expectedPipelineOptions.setUpdate(false); expectedPipelineOptions.setAppName("DataflowJobManager"); + expectedPipelineOptions.setLabels(defaults.getLabelsMap()); expectedPipelineOptions.setJobName(jobName); expectedPipelineOptions.setStoreJson(Lists.newArrayList(printer.print(store))); diff --git a/core/src/test/java/feast/core/job/direct/DirectRunnerJobManagerTest.java b/core/src/test/java/feast/core/job/direct/DirectRunnerJobManagerTest.java index 42d3189a730..0128f5aa0b3 100644 --- a/core/src/test/java/feast/core/job/direct/DirectRunnerJobManagerTest.java +++ b/core/src/test/java/feast/core/job/direct/DirectRunnerJobManagerTest.java @@ -42,6 +42,7 @@ import feast.ingestion.options.OptionCompressor; import feast.proto.core.FeatureSetProto; import feast.proto.core.FeatureSetProto.FeatureSetSpec; +import feast.proto.core.RunnerProto.DirectRunnerConfigOptions; import feast.proto.core.SourceProto; import feast.proto.core.SourceProto.KafkaSourceConfig; import feast.proto.core.SourceProto.SourceType; @@ -51,9 +52,7 @@ import feast.proto.core.StoreProto.Store.Subscription; import java.io.IOException; import java.util.Collections; -import java.util.HashMap; import java.util.List; -import java.util.Map; import org.apache.beam.runners.direct.DirectRunner; import org.apache.beam.sdk.PipelineResult; import org.apache.beam.sdk.options.PipelineOptionsFactory; @@ -71,12 +70,12 @@ public class DirectRunnerJobManagerTest { @Mock private DirectJobRegistry directJobRegistry; private DirectRunnerJobManager drJobManager; - private Map defaults; + private DirectRunnerConfigOptions defaults; @Before public void setUp() { initMocks(this); - defaults = new HashMap<>(); + defaults = DirectRunnerConfigOptions.newBuilder().setTargetParallelism(1).build(); MetricsProperties metricsProperties = new MetricsProperties(); metricsProperties.setEnabled(false); @@ -123,7 +122,7 @@ public void shouldStartDirectJobAndRegisterPipelineResult() throws IOException { expectedPipelineOptions.setAppName("DirectRunnerJobManager"); expectedPipelineOptions.setRunner(DirectRunner.class); expectedPipelineOptions.setBlockOnRun(false); - expectedPipelineOptions.setProject(""); + expectedPipelineOptions.setTargetParallelism(1); expectedPipelineOptions.setStoreJson(Lists.newArrayList(printer.print(store))); expectedPipelineOptions.setProject(""); diff --git a/core/src/test/java/feast/core/util/TypeConversionTest.java b/core/src/test/java/feast/core/util/TypeConversionTest.java index 02f0a7cee45..c44bf50129e 100644 --- a/core/src/test/java/feast/core/util/TypeConversionTest.java +++ b/core/src/test/java/feast/core/util/TypeConversionTest.java @@ -74,16 +74,4 @@ public void convertMapToJsonStringShouldReturnEmptyJsonForAnEmptyMap() { Map input = new HashMap<>(); assertThat(TypeConversion.convertMapToJsonString(input), equalTo("{}")); } - - @Test - public void convertJsonStringToArgsShouldReturnCorrectListOfArgs() { - Map input = new HashMap<>(); - input.put("key", "value"); - input.put("key2", "value2"); - - String[] expected = new String[] {"--key=value", "--key2=value2"}; - String[] actual = TypeConversion.convertMapToArgs(input); - assertThat(actual.length, equalTo(expected.length)); - assertTrue(Arrays.asList(actual).containsAll(Arrays.asList(expected))); - } } diff --git a/protos/feast/core/Runner.proto b/protos/feast/core/Runner.proto index 544972286df..91c1e99485f 100644 --- a/protos/feast/core/Runner.proto +++ b/protos/feast/core/Runner.proto @@ -70,4 +70,7 @@ message DataflowRunnerConfigOptions { /* BigQuery table specification, e.g. PROJECT_ID:DATASET_ID.PROJECT_ID */ string deadLetterTableSpec = 12; + + /* Labels to apply to the dataflow job */ + map labels = 13; } \ No newline at end of file From a33883fbbf0c8d4b27711d85bca313ab11da81b6 Mon Sep 17 00:00:00 2001 From: Terence Lim Date: Mon, 1 Jun 2020 16:11:59 +0800 Subject: [PATCH 033/133] Increase Jaeger Tracing coverage (#719) * Add jaegertracing logs to getOnlineFeatures * Apply spotless --- .../main/java/feast/serving/service/OnlineServingService.java | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/serving/src/main/java/feast/serving/service/OnlineServingService.java b/serving/src/main/java/feast/serving/service/OnlineServingService.java index 7339a19c1a4..bb73e34f51c 100644 --- a/serving/src/main/java/feast/serving/service/OnlineServingService.java +++ b/serving/src/main/java/feast/serving/service/OnlineServingService.java @@ -16,6 +16,7 @@ */ package feast.serving.service; +import com.google.common.collect.ImmutableMap; import com.google.common.collect.Maps; import com.google.protobuf.Duration; import feast.proto.serving.ServingAPIProto.*; @@ -75,6 +76,9 @@ public GetOnlineFeaturesResponse getOnlineFeatures(GetOnlineFeaturesRequest requ // feature set request. List> featureRows = retriever.getOnlineFeatures(entityRows, featureSetRequests); + if (scope != null) { + scope.span().log(ImmutableMap.of("event", "featureRows", "value", featureRows)); + } // For each feature set request, read the feature rows returned by the retriever, and // populate the featureValuesMap with the feature values corresponding to that entity row. From a941f62520374b1e0f6362f3ce410313e5861940 Mon Sep 17 00:00:00 2001 From: Chen Zhiling Date: Wed, 3 Jun 2020 22:22:00 +0800 Subject: [PATCH 034/133] Set pipeline config fields to public, add tests (#765) --- .../job/dataflow/DataflowRunnerConfig.java | 26 +++---- .../core/job/direct/DirectRunnerConfig.java | 4 +- .../feast/core/job/option/RunnerConfig.java | 4 +- .../dataflow/DataflowRunnerConfigTest.java | 68 +++++++++++++++++++ .../job/direct/DirectRunnerConfigTest.java | 44 ++++++++++++ 5 files changed, 129 insertions(+), 17 deletions(-) create mode 100644 core/src/test/java/feast/core/job/dataflow/DataflowRunnerConfigTest.java create mode 100644 core/src/test/java/feast/core/job/direct/DirectRunnerConfigTest.java diff --git a/core/src/main/java/feast/core/job/dataflow/DataflowRunnerConfig.java b/core/src/main/java/feast/core/job/dataflow/DataflowRunnerConfig.java index 85628d2cd0e..df0cb325388 100644 --- a/core/src/main/java/feast/core/job/dataflow/DataflowRunnerConfig.java +++ b/core/src/main/java/feast/core/job/dataflow/DataflowRunnerConfig.java @@ -47,45 +47,45 @@ public DataflowRunnerConfig(DataflowRunnerConfigOptions runnerConfigOptions) { } /* Project id to use when launching jobs. */ - @NotBlank String project; + @NotBlank public String project; /* The Google Compute Engine region for creating Dataflow jobs. */ - @NotBlank String region; + @NotBlank public String region; /* GCP availability zone for operations. */ - @NotBlank String zone; + @NotBlank public String zone; /* Run the job as a specific service account, instead of the default GCE robot. */ - String serviceAccount; + public String serviceAccount; /* GCE network for launching workers. */ - @NotBlank String network; + @NotBlank public String network; /* GCE subnetwork for launching workers. */ - @NotBlank String subnetwork; + @NotBlank public String subnetwork; /* Machine type to create Dataflow worker VMs as. */ - String workerMachineType; + public String workerMachineType; /* The autoscaling algorithm to use for the workerpool. */ - String autoscalingAlgorithm; + public String autoscalingAlgorithm; /* Specifies whether worker pools should be started with public IP addresses. */ - Boolean usePublicIps; + public Boolean usePublicIps; /** * A pipeline level default location for storing temporary files. Support Google Cloud Storage * locations, e.g. gs://bucket/object */ - @NotBlank String tempLocation; + @NotBlank public String tempLocation; /* The maximum number of workers to use for the workerpool. */ - Integer maxNumWorkers; + public Integer maxNumWorkers; /* BigQuery table specification, e.g. PROJECT_ID:DATASET_ID.PROJECT_ID */ - String deadLetterTableSpec; + public String deadLetterTableSpec; - Map labels; + public Map labels; /** Validates Dataflow runner configuration options */ public void validate() { diff --git a/core/src/main/java/feast/core/job/direct/DirectRunnerConfig.java b/core/src/main/java/feast/core/job/direct/DirectRunnerConfig.java index ebd327f2f75..c7459cb9d16 100644 --- a/core/src/main/java/feast/core/job/direct/DirectRunnerConfig.java +++ b/core/src/main/java/feast/core/job/direct/DirectRunnerConfig.java @@ -24,10 +24,10 @@ public class DirectRunnerConfig extends RunnerConfig { * Controls the amount of target parallelism the DirectRunner will use. Defaults to the greater of * the number of available processors and 3. Must be a value greater than zero. */ - Integer targetParallelism; + public Integer targetParallelism; /* BigQuery table specification, e.g. PROJECT_ID:DATASET_ID.PROJECT_ID */ - String deadletterTableSpec; + public String deadletterTableSpec; public DirectRunnerConfig(DirectRunnerConfigOptions runnerConfigOptions) { this.deadletterTableSpec = runnerConfigOptions.getDeadLetterTableSpec(); diff --git a/core/src/main/java/feast/core/job/option/RunnerConfig.java b/core/src/main/java/feast/core/job/option/RunnerConfig.java index 4b937074a31..44550f9608e 100644 --- a/core/src/main/java/feast/core/job/option/RunnerConfig.java +++ b/core/src/main/java/feast/core/job/option/RunnerConfig.java @@ -30,8 +30,8 @@ public abstract class RunnerConfig { /** - * Converts the fields in this class to a list of --key=value args to be passed to a {@link - * org.apache.beam.sdk.options.PipelineOptionsFactory}. + * Converts the public-access fields in this class to a list of --key=value args to be passed to a + * {@link org.apache.beam.sdk.options.PipelineOptionsFactory}. * *

Ignores values that are proto-default (e.g. empty string, 0). * diff --git a/core/src/test/java/feast/core/job/dataflow/DataflowRunnerConfigTest.java b/core/src/test/java/feast/core/job/dataflow/DataflowRunnerConfigTest.java new file mode 100644 index 00000000000..d5c69af6a72 --- /dev/null +++ b/core/src/test/java/feast/core/job/dataflow/DataflowRunnerConfigTest.java @@ -0,0 +1,68 @@ +/* + * SPDX-License-Identifier: Apache-2.0 + * Copyright 2018-2020 The Feast Authors + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package feast.core.job.dataflow; + +import static org.hamcrest.CoreMatchers.equalTo; +import static org.hamcrest.Matchers.containsInAnyOrder; +import static org.junit.Assert.*; + +import com.google.common.collect.Lists; +import feast.proto.core.RunnerProto.DataflowRunnerConfigOptions; +import java.util.Arrays; +import java.util.List; +import org.junit.Test; + +public class DataflowRunnerConfigTest { + @Test + public void shouldConvertToPipelineArgs() throws IllegalAccessException { + DataflowRunnerConfigOptions opts = + DataflowRunnerConfigOptions.newBuilder() + .setProject("my-project") + .setRegion("asia-east1") + .setZone("asia-east1-a") + .setTempLocation("gs://bucket/tempLocation") + .setNetwork("default") + .setSubnetwork("regions/asia-east1/subnetworks/mysubnetwork") + .setMaxNumWorkers(1) + .setAutoscalingAlgorithm("THROUGHPUT_BASED") + .setUsePublicIps(false) + .setWorkerMachineType("n1-standard-1") + .setDeadLetterTableSpec("project_id:dataset_id.table_id") + .putLabels("key", "value") + .build(); + + DataflowRunnerConfig dataflowRunnerConfig = new DataflowRunnerConfig(opts); + List args = Lists.newArrayList(dataflowRunnerConfig.toArgs()); + String[] expectedArgs = + Arrays.asList( + "--project=my-project", + "--region=asia-east1", + "--zone=asia-east1-a", + "--tempLocation=gs://bucket/tempLocation", + "--network=default", + "--subnetwork=regions/asia-east1/subnetworks/mysubnetwork", + "--maxNumWorkers=1", + "--autoscalingAlgorithm=THROUGHPUT_BASED", + "--usePublicIps=false", + "--workerMachineType=n1-standard-1", + "--deadLetterTableSpec=project_id:dataset_id.table_id", + "--labels={\"key\":\"value\"}") + .toArray(String[]::new); + assertThat(args.size(), equalTo(expectedArgs.length)); + assertThat(args, containsInAnyOrder(expectedArgs)); + } +} diff --git a/core/src/test/java/feast/core/job/direct/DirectRunnerConfigTest.java b/core/src/test/java/feast/core/job/direct/DirectRunnerConfigTest.java new file mode 100644 index 00000000000..c4114dd2104 --- /dev/null +++ b/core/src/test/java/feast/core/job/direct/DirectRunnerConfigTest.java @@ -0,0 +1,44 @@ +/* + * SPDX-License-Identifier: Apache-2.0 + * Copyright 2018-2020 The Feast Authors + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package feast.core.job.direct; + +import static org.hamcrest.Matchers.containsInAnyOrder; +import static org.hamcrest.Matchers.equalTo; +import static org.junit.Assert.*; + +import com.google.common.collect.Lists; +import feast.proto.core.RunnerProto.DirectRunnerConfigOptions; +import java.util.List; +import org.junit.Test; + +public class DirectRunnerConfigTest { + @Test + public void shouldConvertToPipelineArgs() throws IllegalAccessException { + DirectRunnerConfigOptions opts = + DirectRunnerConfigOptions.newBuilder() + .setTargetParallelism(1) + .setDeadLetterTableSpec("project_id:dataset_id.table_id") + .build(); + DirectRunnerConfig directRunnerConfig = new DirectRunnerConfig(opts); + List args = Lists.newArrayList(directRunnerConfig.toArgs()); + assertThat(args.size(), equalTo(2)); + assertThat( + args, + containsInAnyOrder( + "--targetParallelism=1", "--deadletterTableSpec=project_id:dataset_id.table_id")); + } +} From 83d9cfa4fdc3188d7cfd6e959b916a6e1d6d8984 Mon Sep 17 00:00:00 2001 From: Willem Pienaar Date: Thu, 4 Jun 2020 01:42:59 +0000 Subject: [PATCH 035/133] GitBook: [master] one page modified --- docs/roadmap.md | 27 +++++++++++++++++++-------- 1 file changed, 19 insertions(+), 8 deletions(-) diff --git a/docs/roadmap.md b/docs/roadmap.md index a047cf41256..09d564d45bd 100644 --- a/docs/roadmap.md +++ b/docs/roadmap.md @@ -1,5 +1,24 @@ # Roadmap +## Feast 0.6 \(Feature Release\) + +[Discussion](https://github.com/feast-dev/feast/issues/767) + +[GitHub Milestone](https://github.com/feast-dev/feast/milestone/3) + +### New functionality + +1. Batch statistics and validation [\#612](https://github.com/feast-dev/feast/pull/612) +2. Authentication and authorization [\#554](https://github.com/feast-dev/feast/pull/554) +3. Online feature and entity status metadata [\#658](https://github.com/feast-dev/feast/pull/658) +4. Improved searching and filtering of features and entities +5. Python support for labels [\#663](https://github.com/feast-dev/feast/issues/663) + +### Technical debt, refactoring, or housekeeping + +1. Improved job life cycle management [\#761](https://github.com/feast-dev/feast/issues/761) +2. Compute and write metrics for rows prior to store writes [\#763](https://github.com/feast-dev/feast/pull/763) + ## Feast 0.5 \(Technical Release\) [Discussion](https://github.com/gojek/feast/issues/527) @@ -21,11 +40,3 @@ 6. Tracking of batch ingestion by with dataset\_id/job\_id \([\#461](https://github.com/gojek/feast/issues/461)\) 7. Write Beam metrics after ingestion to store \(not prior\) \([\#489](https://github.com/gojek/feast/issues/489)\) -## Feast 0.6 \(Feature Release\) - -### New functionality - -1. User authentication & authorization \([\#504](https://github.com/gojek/feast/issues/504)\) -2. Batch statistics and validation \(M2 from [Feature Validation RFC](https://docs.google.com/document/d/1TPmd7r4mniL9Y-V_glZaWNo5LMXLshEAUpYsohojZ-8/edit)\) -3. Online feature/entity status metadata \([\#658](https://github.com/gojek/feast/pull/658)\) - From 4be0eff52e1f74b89971a609921bbbd635c612fe Mon Sep 17 00:00:00 2001 From: Zhu Zhan Yan Date: Fri, 5 Jun 2020 12:11:00 +0800 Subject: [PATCH 036/133] Expand Go and Java SDK Test Coverage (#720) * Added TestGetOnlineFeatures() client_test to test client.GetOnlineFeatures() * Added unit test for Java SDK's FeastClient * Make Python SDK's client.get_online_features() unit test more comprehensive * Remove code added to debug the Python SDK tests * Change repo referenced in go.mod to mrzzy fork to test modules retrieval * Fix python sdk lint * Revert "Change repo referenced in go.mod to mrzzy fork to test modules retrieval" This reverts commit 13008ccef6b6cbee6299029589b5e9e864eee7ae. * Optmisation: Init stripFields map to expected size in client.GetOnlineFeatures() * Make the name of the client test case more descriptive * Add missing mock files * Fix rebase markers in project root go mod * Fix compilation issue cause by downgrade to java 8 * Revert row count in python SDK client test to 300 Co-authored-by: Zhu Zhanyan --- go.mod | 3 +- go.sum | 6 +- sdk/go/client.go | 2 +- sdk/go/client_test.go | 95 ++++++++++++ sdk/go/go.mod | 1 + sdk/go/go.sum | 8 + sdk/go/mocks/serving_mock.go | 116 ++++++++++++++ sdk/java/pom.xml | 26 +++- .../java/com/gojek/feast/FeastClientTest.java | 146 ++++++++++++++++++ sdk/python/feast/client.py | 9 +- sdk/python/tests/test_client.py | 61 ++++---- 11 files changed, 435 insertions(+), 38 deletions(-) create mode 100644 sdk/go/client_test.go create mode 100644 sdk/go/mocks/serving_mock.go create mode 100644 sdk/java/src/test/java/com/gojek/feast/FeastClientTest.java diff --git a/go.mod b/go.mod index 1be2396f6e6..d5917cb33e3 100644 --- a/go.mod +++ b/go.mod @@ -25,7 +25,8 @@ require ( golang.org/x/lint v0.0.0-20200302205851-738671d3881b // indirect golang.org/x/net v0.0.0-20200513185701-a91f0712d120 golang.org/x/sys v0.0.0-20200515095857-1151b9dac4a9 // indirect - golang.org/x/tools v0.0.0-20200528185414-6be401e3f76e // indirect + golang.org/x/tools v0.0.0-20200521211927-2b542361a4fc // indirect + google.golang.org/genproto v0.0.0-20200515170657-fc4c6c6a6587 // indirect google.golang.org/grpc v1.29.1 google.golang.org/protobuf v1.24.0 // indirect gopkg.in/russross/blackfriday.v2 v2.0.0 // indirect diff --git a/go.sum b/go.sum index 1b2aa8bf63b..90a517da7e4 100644 --- a/go.sum +++ b/go.sum @@ -462,8 +462,10 @@ golang.org/x/tools v0.0.0-20200504022951-6b6965ac5dd1 h1:C8rdnd6KieI73Z2Av0sS0t4 golang.org/x/tools v0.0.0-20200504022951-6b6965ac5dd1/go.mod h1:EkVYQZoAsY45+roYkvgYkIh4xh/qjgUK9TdY2XT94GE= golang.org/x/tools v0.0.0-20200515220128-d3bf790afa53 h1:vmsb6v0zUdmUlXfwKaYrHPPRCV0lHq/IwNIf0ASGjyQ= golang.org/x/tools v0.0.0-20200515220128-d3bf790afa53/go.mod h1:EkVYQZoAsY45+roYkvgYkIh4xh/qjgUK9TdY2XT94GE= -golang.org/x/tools v0.0.0-20200528185414-6be401e3f76e h1:jTL1CJ2kmavapMVdBKy6oVrhBHByRCMfykS45+lEFQk= -golang.org/x/tools v0.0.0-20200528185414-6be401e3f76e/go.mod h1:EkVYQZoAsY45+roYkvgYkIh4xh/qjgUK9TdY2XT94GE= +golang.org/x/tools v0.0.0-20200519205726-57a9e4404bf7 h1:nm4zDh9WvH4jiuUpMY5RUsvOwrtTVVAsUaCdLW71hfY= +golang.org/x/tools v0.0.0-20200519205726-57a9e4404bf7/go.mod h1:EkVYQZoAsY45+roYkvgYkIh4xh/qjgUK9TdY2XT94GE= +golang.org/x/tools v0.0.0-20200521211927-2b542361a4fc h1:6m2YO+AmBApbUOmhsghW+IfRyZOY4My4UYvQQrEpHfY= +golang.org/x/tools v0.0.0-20200521211927-2b542361a4fc/go.mod h1:EkVYQZoAsY45+roYkvgYkIh4xh/qjgUK9TdY2XT94GE= golang.org/x/xerrors v0.0.0-20190717185122-a985d3407aa7/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0= golang.org/x/xerrors v0.0.0-20191011141410-1b5146add898/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0= golang.org/x/xerrors v0.0.0-20191204190536-9bdfabe68543/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0= diff --git a/sdk/go/client.go b/sdk/go/client.go index ef8ec8b8cf9..1f0cbaa3b96 100644 --- a/sdk/go/client.go +++ b/sdk/go/client.go @@ -61,7 +61,7 @@ func (fc *GrpcClient) GetOnlineFeatures(ctx context.Context, req *OnlineFeatures // strip projects from to projects for _, fieldValue := range resp.GetFieldValues() { - stripFields := make(map[string]*types.Value) + stripFields := make(map[string]*types.Value, len(fieldValue.Fields)) for refStr, value := range fieldValue.Fields { _, isEntity := entityRefs[refStr] if !isEntity { // is feature ref diff --git a/sdk/go/client_test.go b/sdk/go/client_test.go new file mode 100644 index 00000000000..1ad6629e9af --- /dev/null +++ b/sdk/go/client_test.go @@ -0,0 +1,95 @@ +package feast + +import ( + "context" + "testing" + + "github.com/feast-dev/feast/sdk/go/mocks" + "github.com/feast-dev/feast/sdk/go/protos/feast/serving" + "github.com/feast-dev/feast/sdk/go/protos/feast/types" + "github.com/golang/mock/gomock" + "github.com/google/go-cmp/cmp" + "github.com/opentracing/opentracing-go" +) + +func TestGetOnlineFeatures(t *testing.T) { + tt := []struct { + name string + req OnlineFeaturesRequest + recieve OnlineFeaturesResponse + want OnlineFeaturesResponse + wantErr bool + err error + }{ + { + name: "Valid client Get Online Features call", + req: OnlineFeaturesRequest{ + Features: []string{ + "driver:rating", + "rating", + }, + Entities: []Row{ + {"driver_id": Int64Val(1)}, + }, + Project: "driver_project", + }, + // check GetOnlineFeatures() should strip projects returned from serving + recieve: OnlineFeaturesResponse{ + RawResponse: &serving.GetOnlineFeaturesResponse{ + FieldValues: []*serving.GetOnlineFeaturesResponse_FieldValues{ + { + Fields: map[string]*types.Value{ + "driver_project/driver:rating": Int64Val(1), + "driver_project/rating": Int64Val(1), + }, + }, + }, + }, + }, + want: OnlineFeaturesResponse{ + RawResponse: &serving.GetOnlineFeaturesResponse{ + FieldValues: []*serving.GetOnlineFeaturesResponse_FieldValues{ + { + Fields: map[string]*types.Value{ + "driver:rating": Int64Val(1), + "rating": Int64Val(1), + }, + }, + }, + }, + }, + }, + } + + for _, tc := range tt { + t.Run(tc.name, func(t *testing.T) { + // mock feast grpc client get online feature requestss + ctrl := gomock.NewController(t) + defer ctrl.Finish() + cli := mock_serving.NewMockServingServiceClient(ctrl) + ctx := context.Background() + _, traceCtx := opentracing.StartSpanFromContext(ctx, "get_online_features") + rawRequest, _ := tc.req.buildRequest() + resp := tc.recieve.RawResponse + cli.EXPECT().GetOnlineFeatures(traceCtx, rawRequest).Return(resp, nil).Times(1) + + client := &GrpcClient{ + cli: cli, + } + got, err := client.GetOnlineFeatures(ctx, &tc.req) + + if err != nil && !tc.wantErr { + t.Errorf("error = %v, wantErr %v", err, tc.wantErr) + return + } + if tc.wantErr && err.Error() != tc.err.Error() { + t.Errorf("error = %v, expected err = %v", err, tc.err) + return + } + // TODO: compare directly once OnlineFeaturesResponse no longer embeds a rawResponse. + if !cmp.Equal(got.RawResponse.String(), tc.want.RawResponse.String()) { + t.Errorf("got: \n%v\nwant:\n%v", got.RawResponse.String(), tc.want.RawResponse.String()) + } + }) + } +} diff --git a/sdk/go/go.mod b/sdk/go/go.mod index 06bd6936162..656810354ba 100644 --- a/sdk/go/go.mod +++ b/sdk/go/go.mod @@ -3,6 +3,7 @@ module github.com/feast-dev/feast/sdk/go go 1.13 require ( + github.com/golang/mock v1.4.3 github.com/golang/protobuf v1.4.0-rc.4.0.20200313231945-b860323f09d0 github.com/google/go-cmp v0.4.0 github.com/opentracing/opentracing-go v1.1.0 diff --git a/sdk/go/go.sum b/sdk/go/go.sum index f08f3026170..39e6ef923b4 100644 --- a/sdk/go/go.sum +++ b/sdk/go/go.sum @@ -12,7 +12,10 @@ github.com/golang/glog v0.0.0-20160126235308-23def4e6c14b h1:VKtxabqXZkF25pY9ekf github.com/golang/glog v0.0.0-20160126235308-23def4e6c14b/go.mod h1:SBH7ygxi8pfUlaOkMMuAQtPIUF8ecWP5IEl/CR7VP2Q= github.com/golang/groupcache v0.0.0-20190702054246-869f871628b6 h1:ZgQEtGgCBiWRM39fZuwSd1LwSqqSW0hOdXCYYDX0R3I= github.com/golang/groupcache v0.0.0-20190702054246-869f871628b6/go.mod h1:cIg4eruTrX1D+g88fzRXU5OdNfaM+9IcxsU14FzY7Hc= +github.com/golang/mock v1.1.1 h1:G5FRp8JnTd7RQH5kemVNlMeyXQAztQ3mOWV95KxsXH8= github.com/golang/mock v1.1.1/go.mod h1:oTYuIxOrZwtPieC+H1uAHpcLFnEyAGVDL/k47Jfbm0A= +github.com/golang/mock v1.4.3 h1:GV+pQPG/EUUbkh47niozDcADz6go/dUwhVzdUQHIVRw= +github.com/golang/mock v1.4.3/go.mod h1:UOMv5ysSaYNkG+OFQykRIcU/QvvxJf3p21QfJ2Bt3cw= github.com/golang/protobuf v1.2.0/go.mod h1:6lQm79b+lXiMfvg/cZm0SGofjICqVBUtrP5yJMmIC1U= github.com/golang/protobuf v1.3.1/go.mod h1:6lQm79b+lXiMfvg/cZm0SGofjICqVBUtrP5yJMmIC1U= github.com/golang/protobuf v1.3.2 h1:6nsPYzhq5kReh6QImI3k5qWzO4PEbvbIW2cwSfR/6xs= @@ -63,6 +66,7 @@ golang.org/x/sys v0.0.0-20180830151530-49385e6e1522/go.mod h1:STP8DvDyc/dI5b8T5h golang.org/x/sys v0.0.0-20190215142949-d0b11bdaac8a/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY= golang.org/x/sys v0.0.0-20190502145724-3ef323f4f1fd h1:r7DufRZuZbWB7j439YfAzP8RPDa9unLkpwQKUYbIMPI= golang.org/x/sys v0.0.0-20190502145724-3ef323f4f1fd/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= +golang.org/x/text v0.0.0-20170915032832-14c0d48ead0c/go.mod h1:NqM8EUOU14njkJ3fqMW+pc6Ldnwhi/IjpwHt7yyuwOQ= golang.org/x/text v0.3.0 h1:g61tztE5qeGQ89tm6NTjjM9VPIm088od1l6aSorWRWg= golang.org/x/text v0.3.0/go.mod h1:NqM8EUOU14njkJ3fqMW+pc6Ldnwhi/IjpwHt7yyuwOQ= golang.org/x/text v0.3.2 h1:tW2bmiBqwgJj/UpqtC8EpXEZVYOwU0yG4iWbprSVAcs= @@ -71,6 +75,8 @@ golang.org/x/tools v0.0.0-20180917221912-90fa682c2a6e/go.mod h1:n7NCudcB/nEzxVGm golang.org/x/tools v0.0.0-20190114222345-bf090417da8b/go.mod h1:n7NCudcB/nEzxVGmLbDWY5pfWTLqBcC2KZ6jyYvM4mQ= golang.org/x/tools v0.0.0-20190226205152-f727befe758c/go.mod h1:9Yl7xja0Znq3iFh3HoIrodX9oNMXvdceNzlUR8zjMvY= golang.org/x/tools v0.0.0-20190311212946-11955173bddd/go.mod h1:LCzVGOaR6xXOjkQ3onu1FJEFr0SW1gC7cKk1uF8kGRs= +golang.org/x/tools v0.0.0-20190425150028-36563e24a262/go.mod h1:RgjU9mgBXZiqYHBnxXauZ1Gv1EHHAz9KjViQ78xBX0Q= +golang.org/x/tools v0.0.0-20190524140312-2c0ae7006135 h1:5Beo0mZN8dRzgrMMkDp0jc8YXQKx9DiJ2k1dkvGsn5A= golang.org/x/tools v0.0.0-20190524140312-2c0ae7006135/go.mod h1:RgjU9mgBXZiqYHBnxXauZ1Gv1EHHAz9KjViQ78xBX0Q= golang.org/x/xerrors v0.0.0-20191204190536-9bdfabe68543 h1:E7g+9GITq07hpfrRu66IVDexMakfv52eLZ2CXBWiKr4= golang.org/x/xerrors v0.0.0-20191204190536-9bdfabe68543/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0= @@ -100,3 +106,5 @@ gopkg.in/yaml.v2 v2.2.2 h1:ZCJp+EgiOT7lHqUV2J862kp8Qj64Jo6az82+3Td9dZw= gopkg.in/yaml.v2 v2.2.2/go.mod h1:hI93XBmqTisBFMUTm0b8Fm+jr3Dg1NNxqwp+5A1VGuI= honnef.co/go/tools v0.0.0-20190102054323-c2f93a96b099/go.mod h1:rf3lG4BRIbNafJWhAfAdb/ePZxsR/4RtNHQocxwk9r4= honnef.co/go/tools v0.0.0-20190523083050-ea95bdfd59fc/go.mod h1:rf3lG4BRIbNafJWhAfAdb/ePZxsR/4RtNHQocxwk9r4= +rsc.io/quote/v3 v3.1.0/go.mod h1:yEA65RcK8LyAZtP9Kv3t0HmxON59tX3rD+tICJqUlj0= +rsc.io/sampler v1.3.0/go.mod h1:T1hPZKmBbMNahiBKFy5HrXp6adAjACjK9JXDnKaTXpA= diff --git a/sdk/go/mocks/serving_mock.go b/sdk/go/mocks/serving_mock.go new file mode 100644 index 00000000000..bea754c1bf2 --- /dev/null +++ b/sdk/go/mocks/serving_mock.go @@ -0,0 +1,116 @@ +// Code generated by MockGen. DO NOT EDIT. +// Source: github.com/feast-dev/feast/sdk/go/protos/feast/serving (interfaces: ServingServiceClient) + +// Package mock_serving is a generated GoMock package. +package mock_serving + +import ( + context "context" + serving "github.com/feast-dev/feast/sdk/go/protos/feast/serving" + gomock "github.com/golang/mock/gomock" + grpc "google.golang.org/grpc" + reflect "reflect" +) + +// MockServingServiceClient is a mock of ServingServiceClient interface +type MockServingServiceClient struct { + ctrl *gomock.Controller + recorder *MockServingServiceClientMockRecorder +} + +// MockServingServiceClientMockRecorder is the mock recorder for MockServingServiceClient +type MockServingServiceClientMockRecorder struct { + mock *MockServingServiceClient +} + +// NewMockServingServiceClient creates a new mock instance +func NewMockServingServiceClient(ctrl *gomock.Controller) *MockServingServiceClient { + mock := &MockServingServiceClient{ctrl: ctrl} + mock.recorder = &MockServingServiceClientMockRecorder{mock} + return mock +} + +// EXPECT returns an object that allows the caller to indicate expected use +func (m *MockServingServiceClient) EXPECT() *MockServingServiceClientMockRecorder { + return m.recorder +} + +// GetBatchFeatures mocks base method +func (m *MockServingServiceClient) GetBatchFeatures(arg0 context.Context, arg1 *serving.GetBatchFeaturesRequest, arg2 ...grpc.CallOption) (*serving.GetBatchFeaturesResponse, error) { + m.ctrl.T.Helper() + varargs := []interface{}{arg0, arg1} + for _, a := range arg2 { + varargs = append(varargs, a) + } + ret := m.ctrl.Call(m, "GetBatchFeatures", varargs...) + ret0, _ := ret[0].(*serving.GetBatchFeaturesResponse) + ret1, _ := ret[1].(error) + return ret0, ret1 +} + +// GetBatchFeatures indicates an expected call of GetBatchFeatures +func (mr *MockServingServiceClientMockRecorder) GetBatchFeatures(arg0, arg1 interface{}, arg2 ...interface{}) *gomock.Call { + mr.mock.ctrl.T.Helper() + varargs := append([]interface{}{arg0, arg1}, arg2...) + return mr.mock.ctrl.RecordCallWithMethodType(mr.mock, "GetBatchFeatures", reflect.TypeOf((*MockServingServiceClient)(nil).GetBatchFeatures), varargs...) +} + +// GetFeastServingInfo mocks base method +func (m *MockServingServiceClient) GetFeastServingInfo(arg0 context.Context, arg1 *serving.GetFeastServingInfoRequest, arg2 ...grpc.CallOption) (*serving.GetFeastServingInfoResponse, error) { + m.ctrl.T.Helper() + varargs := []interface{}{arg0, arg1} + for _, a := range arg2 { + varargs = append(varargs, a) + } + ret := m.ctrl.Call(m, "GetFeastServingInfo", varargs...) + ret0, _ := ret[0].(*serving.GetFeastServingInfoResponse) + ret1, _ := ret[1].(error) + return ret0, ret1 +} + +// GetFeastServingInfo indicates an expected call of GetFeastServingInfo +func (mr *MockServingServiceClientMockRecorder) GetFeastServingInfo(arg0, arg1 interface{}, arg2 ...interface{}) *gomock.Call { + mr.mock.ctrl.T.Helper() + varargs := append([]interface{}{arg0, arg1}, arg2...) + return mr.mock.ctrl.RecordCallWithMethodType(mr.mock, "GetFeastServingInfo", reflect.TypeOf((*MockServingServiceClient)(nil).GetFeastServingInfo), varargs...) +} + +// GetJob mocks base method +func (m *MockServingServiceClient) GetJob(arg0 context.Context, arg1 *serving.GetJobRequest, arg2 ...grpc.CallOption) (*serving.GetJobResponse, error) { + m.ctrl.T.Helper() + varargs := []interface{}{arg0, arg1} + for _, a := range arg2 { + varargs = append(varargs, a) + } + ret := m.ctrl.Call(m, "GetJob", varargs...) + ret0, _ := ret[0].(*serving.GetJobResponse) + ret1, _ := ret[1].(error) + return ret0, ret1 +} + +// GetJob indicates an expected call of GetJob +func (mr *MockServingServiceClientMockRecorder) GetJob(arg0, arg1 interface{}, arg2 ...interface{}) *gomock.Call { + mr.mock.ctrl.T.Helper() + varargs := append([]interface{}{arg0, arg1}, arg2...) + return mr.mock.ctrl.RecordCallWithMethodType(mr.mock, "GetJob", reflect.TypeOf((*MockServingServiceClient)(nil).GetJob), varargs...) +} + +// GetOnlineFeatures mocks base method +func (m *MockServingServiceClient) GetOnlineFeatures(arg0 context.Context, arg1 *serving.GetOnlineFeaturesRequest, arg2 ...grpc.CallOption) (*serving.GetOnlineFeaturesResponse, error) { + m.ctrl.T.Helper() + varargs := []interface{}{arg0, arg1} + for _, a := range arg2 { + varargs = append(varargs, a) + } + ret := m.ctrl.Call(m, "GetOnlineFeatures", varargs...) + ret0, _ := ret[0].(*serving.GetOnlineFeaturesResponse) + ret1, _ := ret[1].(error) + return ret0, ret1 +} + +// GetOnlineFeatures indicates an expected call of GetOnlineFeatures +func (mr *MockServingServiceClientMockRecorder) GetOnlineFeatures(arg0, arg1 interface{}, arg2 ...interface{}) *gomock.Call { + mr.mock.ctrl.T.Helper() + varargs := append([]interface{}{arg0, arg1}, arg2...) + return mr.mock.ctrl.RecordCallWithMethodType(mr.mock, "GetOnlineFeatures", reflect.TypeOf((*MockServingServiceClient)(nil).GetOnlineFeatures), varargs...) +} diff --git a/sdk/java/pom.xml b/sdk/java/pom.xml index 75d82edcc01..7856afd47e7 100644 --- a/sdk/java/pom.xml +++ b/sdk/java/pom.xml @@ -18,6 +18,7 @@ 5.5.2 + 2.28.2 @@ -40,6 +41,10 @@ io.grpc grpc-stub + + io.grpc + grpc-testing + com.google.protobuf protobuf-java-util @@ -55,7 +60,7 @@ slf4j-api - + org.junit.jupiter junit-jupiter-engine @@ -80,7 +85,24 @@ 3.6 compile - + + org.mockito + mockito-core + ${mockito.version} + test + + + org.mockito + mockito-inline + ${mockito.version} + test + + + org.junit.vintage + junit-vintage-engine + ${junit.version} + test + diff --git a/sdk/java/src/test/java/com/gojek/feast/FeastClientTest.java b/sdk/java/src/test/java/com/gojek/feast/FeastClientTest.java new file mode 100644 index 00000000000..717792244ee --- /dev/null +++ b/sdk/java/src/test/java/com/gojek/feast/FeastClientTest.java @@ -0,0 +1,146 @@ +/* + * SPDX-License-Identifier: Apache-2.0 + * Copyright 2018-2019 The Feast Authors + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package com.gojek.feast; + +import static org.junit.jupiter.api.Assertions.assertEquals; +import static org.mockito.AdditionalAnswers.delegatesTo; +import static org.mockito.Mockito.mock; + +import com.google.protobuf.Timestamp; +import feast.proto.serving.ServingAPIProto.FeatureReference; +import feast.proto.serving.ServingAPIProto.GetOnlineFeaturesRequest; +import feast.proto.serving.ServingAPIProto.GetOnlineFeaturesRequest.EntityRow; +import feast.proto.serving.ServingAPIProto.GetOnlineFeaturesResponse; +import feast.proto.serving.ServingAPIProto.GetOnlineFeaturesResponse.FieldValues; +import feast.proto.serving.ServingServiceGrpc.ServingServiceImplBase; +import feast.proto.types.ValueProto.Value; +import io.grpc.ManagedChannel; +import io.grpc.Status; +import io.grpc.inprocess.InProcessChannelBuilder; +import io.grpc.inprocess.InProcessServerBuilder; +import io.grpc.stub.StreamObserver; +import io.grpc.testing.GrpcCleanupRule; +import java.time.Instant; +import java.util.Arrays; +import java.util.HashMap; +import java.util.List; +import org.junit.Before; +import org.junit.Rule; +import org.junit.Test; + +public class FeastClientTest { + @Rule public GrpcCleanupRule grpcRule; + private ServingServiceImplBase servingMock = + mock( + ServingServiceImplBase.class, + delegatesTo( + new ServingServiceImplBase() { + @Override + public void getOnlineFeatures( + GetOnlineFeaturesRequest request, + StreamObserver responseObserver) { + if (!request.equals(FeastClientTest.getFakeRequest())) { + responseObserver.onError(Status.UNKNOWN.asRuntimeException()); + } + + responseObserver.onNext(FeastClientTest.getFakeResponse()); + responseObserver.onCompleted(); + } + })); + private FeastClient client; + + @Before + public void setup() throws Exception { + this.grpcRule = new GrpcCleanupRule(); + // setup fake serving service + String serverName = InProcessServerBuilder.generateName(); + this.grpcRule.register( + InProcessServerBuilder.forName(serverName) + .directExecutor() + .addService(this.servingMock) + .build() + .start()); + + // setup test feast client target + ManagedChannel channel = + this.grpcRule.register( + InProcessChannelBuilder.forName(serverName).directExecutor().build()); + this.client = new FeastClient(channel); + } + + @Test + public void shouldGetOnlineFeatures() { + List rows = + this.client.getOnlineFeatures( + Arrays.asList("driver:name", "rating"), + Arrays.asList( + Row.create().set("driver_id", 1).setEntityTimestamp(Instant.ofEpochSecond(100))), + "driver_project"); + + assertEquals( + rows.get(0).getFields(), + new HashMap() { + { + put("driver_id", intValue(1)); + put("driver:name", strValue("david")); + put("rating", intValue(3)); + } + }); + } + + private static GetOnlineFeaturesRequest getFakeRequest() { + // setup mock serving service stub + return GetOnlineFeaturesRequest.newBuilder() + .addFeatures( + FeatureReference.newBuilder() + .setProject("driver_project") + .setFeatureSet("driver") + .setName("name") + .build()) + .addFeatures( + FeatureReference.newBuilder().setProject("driver_project").setName("rating").build()) + .addEntityRows( + EntityRow.newBuilder() + .setEntityTimestamp(Timestamp.newBuilder().setSeconds(100)) + .putFields("driver_id", intValue(1))) + .build(); + } + + private static GetOnlineFeaturesResponse getFakeResponse() { + return GetOnlineFeaturesResponse.newBuilder() + .addFieldValues( + FieldValues.newBuilder() + .putAllFields( + new HashMap() { + { + put("driver_id", intValue(1)); + put("driver:name", strValue("david")); + put("rating", intValue(3)); + } + }) + .build()) + .build(); + } + + private static Value strValue(String val) { + return Value.newBuilder().setStringVal(val).build(); + } + + private static Value intValue(int val) { + return Value.newBuilder().setInt32Val(val).build(); + } +} diff --git a/sdk/python/feast/client.py b/sdk/python/feast/client.py index 1c34213ad88..f10204c59d9 100644 --- a/sdk/python/feast/client.py +++ b/sdk/python/feast/client.py @@ -658,14 +658,11 @@ def get_online_features( # strip the project part the string feature references returned from serving strip_fields = {} for ref_str, value in field_value.fields.items(): - # find and ignore entities - if ref_str in entity_refs: - strip_fields[ref_str] = value - else: - strip_ref_str = repr( + if ref_str not in entity_refs: + ref_str = repr( FeatureRef.from_str(ref_str, ignore_project=True) ) - strip_fields[strip_ref_str] = value + strip_fields[ref_str] = value strip_field_values.append( GetOnlineFeaturesResponse.FieldValues(fields=strip_fields) ) diff --git a/sdk/python/tests/test_client.py b/sdk/python/tests/test_client.py index 380557ce928..75d45a583fc 100644 --- a/sdk/python/tests/test_client.py +++ b/sdk/python/tests/test_client.py @@ -47,11 +47,12 @@ from feast.core.Source_pb2 import KafkaSourceConfig, Source, SourceType from feast.core.Store_pb2 import Store from feast.entity import Entity -from feast.feature_set import Feature, FeatureSet, FeatureSetRef +from feast.feature import Feature +from feast.feature_set import FeatureSet, FeatureSetRef from feast.job import IngestJob +from feast.serving.ServingService_pb2 import DataFormat, FeastServingType +from feast.serving.ServingService_pb2 import FeatureReference as FeatureRefProto from feast.serving.ServingService_pb2 import ( - DataFormat, - FeastServingType, GetBatchFeaturesResponse, GetFeastServingInfoResponse, GetJobResponse, @@ -210,42 +211,50 @@ def test_get_online_features(self, mocked_client, mocker): def int_val(x): return ValueProto.Value(int64_val=x) - # serving can return feature references with projects, - # get_online_features() should strip the project part. - field_values = GetOnlineFeaturesResponse.FieldValues( - fields={ - "driver_project/driver:driver_id": int_val(1), - "driver_project/driver_id": int_val(9), - } + request = GetOnlineFeaturesRequest() + request.features.extend( + [ + FeatureRefProto( + project="driver_project", feature_set="driver", name="age" + ), + FeatureRefProto(project="driver_project", name="rating"), + ] ) - - response = GetOnlineFeaturesResponse() - entity_rows = [] + recieve_response = GetOnlineFeaturesResponse() for row_number in range(1, ROW_COUNT + 1): - response.field_values.append(field_values) - entity_rows.append( + request.entity_rows.append( GetOnlineFeaturesRequest.EntityRow( - fields={"customer_id": int_val(row_number)} + fields={"driver_id": int_val(row_number)} ) + ), + field_values = GetOnlineFeaturesResponse.FieldValues( + fields={ + "driver_id": int_val(row_number), + "driver_project/driver:age": int_val(1), + "driver_project/rating": int_val(9), + } ) + recieve_response.field_values.append(field_values) mocker.patch.object( mocked_client._serving_service_stub, "GetOnlineFeatures", - return_value=response, + return_value=recieve_response, ) - - # NOTE: Feast Serving does not allow for feature references - # that specify the same feature in the same request - response = mocked_client.get_online_features( - entity_rows=entity_rows, - feature_refs=["driver:driver_id", "driver_id"], + got_response = mocked_client.get_online_features( + entity_rows=request.entity_rows, + feature_refs=["driver:age", "rating"], project="driver_project", ) # type: GetOnlineFeaturesResponse + mocked_client._serving_service_stub.GetOnlineFeatures.assert_called_with( + request + ) + got_fields = got_response.field_values[0].fields assert ( - response.field_values[0].fields["driver:driver_id"].int64_val == 1 - and response.field_values[0].fields["driver_id"].int64_val == 9 + got_fields["driver_id"] == int_val(1) + and got_fields["driver:age"] == int_val(1) + and got_fields["rating"] == int_val(9) ) @pytest.mark.parametrize( @@ -635,7 +644,7 @@ def test_feature_set_ingest_throws_exception_if_kafka_down( ) with pytest.raises(exception): - test_client.ingest("driver-feature-set", dataframe) + test_client.ingest("driver-feature-set", dataframe, timeout=1) @pytest.mark.parametrize( "dataframe,exception,test_client", From 7b537f02861457c7721904e46308d5bdae26052b Mon Sep 17 00:00:00 2001 From: Khor Shu Heng <32997938+khorshuheng@users.noreply.github.com> Date: Sun, 7 Jun 2020 12:31:01 +0800 Subject: [PATCH 037/133] Publish helm chart script should not modify the chart content (#779) Co-authored-by: Khor Shu Heng --- .prow/config.yaml | 9 --------- 1 file changed, 9 deletions(-) diff --git a/.prow/config.yaml b/.prow/config.yaml index f970ded0814..09f791ef4a4 100644 --- a/.prow/config.yaml +++ b/.prow/config.yaml @@ -493,15 +493,6 @@ postsubmits: mv /tmp/linux-amd64/helm /usr/bin/helm helm init --client-only - sed -i "/version: /c\version: ${PULL_BASE_REF:1}" infra/charts/feast/Chart.yaml - sed -i "/ version: /c\ version: ${PULL_BASE_REF:1}" infra/charts/feast/requirements.yaml - - sed -i "/version: /c\version: ${PULL_BASE_REF:1}" infra/charts/feast/charts/feast-core/Chart.yaml - sed -i "/ tag: /c\ tag: ${PULL_BASE_REF:1}" infra/charts/feast/charts/feast-core/values.yaml - - sed -i "/version: /c\version: ${PULL_BASE_REF:1}" infra/charts/feast/charts/feast-serving/Chart.yaml - sed -i "/ tag: /c\ tag: ${PULL_BASE_REF:1}" infra/charts/feast/charts/feast-serving/values.yaml - infra/scripts/sync-helm-charts.sh volumeMounts: - name: service-account From fefcc9b5b2c0e1f47f4f5edb1875778ea04233f2 Mon Sep 17 00:00:00 2001 From: suwik Date: Sun, 7 Jun 2020 18:22:01 +0700 Subject: [PATCH 038/133] assert for the actual applied labels (#740) Co-authored-by: Krzysztof Suwinski --- .../java/feast/core/service/SpecServiceTest.java | 16 +++------------- 1 file changed, 3 insertions(+), 13 deletions(-) diff --git a/core/src/test/java/feast/core/service/SpecServiceTest.java b/core/src/test/java/feast/core/service/SpecServiceTest.java index e584ee71e00..b5fd03fc7fc 100644 --- a/core/src/test/java/feast/core/service/SpecServiceTest.java +++ b/core/src/test/java/feast/core/service/SpecServiceTest.java @@ -543,8 +543,6 @@ public void applyFeatureSetShouldFailWhenProjectIsArchived() @Test public void applyFeatureSetShouldAcceptFeatureLabels() throws InvalidProtocolBufferException { - List entitySpecs = new ArrayList<>(); - entitySpecs.add(EntitySpec.newBuilder().setName("entity1").setValueType(Enum.INT64).build()); Map featureLabels0 = new HashMap<>() { @@ -583,7 +581,6 @@ public void applyFeatureSetShouldAcceptFeatureLabels() throws InvalidProtocolBuf FeatureSetSpec.newBuilder() .setProject("project1") .setName("featureSetWithConstraints") - .addAllEntities(entitySpecs) .addAllFeatures(featureSpecs) .build(); FeatureSetProto.FeatureSet featureSet = @@ -592,22 +589,15 @@ public void applyFeatureSetShouldAcceptFeatureLabels() throws InvalidProtocolBuf ApplyFeatureSetResponse applyFeatureSetResponse = specService.applyFeatureSet(featureSet); FeatureSetSpec appliedFeatureSetSpec = applyFeatureSetResponse.getFeatureSet().getSpec(); - // appliedEntitySpecs needs to be sorted because the list returned by specService may not - // follow the order in the request - List appliedEntitySpecs = new ArrayList<>(appliedFeatureSetSpec.getEntitiesList()); - appliedEntitySpecs.sort(Comparator.comparing(EntitySpec::getName)); - // appliedFeatureSpecs needs to be sorted because the list returned by specService may not // follow the order in the request List appliedFeatureSpecs = new ArrayList<>(appliedFeatureSetSpec.getFeaturesList()); appliedFeatureSpecs.sort(Comparator.comparing(FeatureSpec::getName)); - var featureSpecsLabels = - featureSpecs.stream().map(e -> e.getLabelsMap()).collect(Collectors.toList()); - assertEquals(appliedEntitySpecs, entitySpecs); - assertEquals(appliedFeatureSpecs, featureSpecs); - assertEquals(featureSpecsLabels, featureLabels); + var appliedFeatureSpecsLabels = + appliedFeatureSpecs.stream().map(e -> e.getLabelsMap()).collect(Collectors.toList()); + assertEquals(appliedFeatureSpecsLabels, featureLabels); } @Test From 5d3d81f3be75b042e8f1a276921d21601218c547 Mon Sep 17 00:00:00 2001 From: Khor Shu Heng <32997938+khorshuheng@users.noreply.github.com> Date: Mon, 8 Jun 2020 11:24:01 +0800 Subject: [PATCH 039/133] Update change log due to release 0.5.1 (#783) Co-authored-by: Khor Shu Heng --- CHANGELOG.md | 22 ++++++++++++++++++++++ 1 file changed, 22 insertions(+) diff --git a/CHANGELOG.md b/CHANGELOG.md index 5339b8c5f57..77f666cf395 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -1,5 +1,27 @@ # Changelog +## [0.5.1](https://github.com/feast-dev/feast/tree/0.5.1) (2020-06-06) + +[Full Changelog](https://github.com/feast-dev/feast/compare/v0.5.0...v0.5.1) + +**Implemented enhancements:** +- Add support for version method in Feast SDK and Core [\#759](https://github.com/feast-dev/feast/pull/759) ([woop](https://github.com/woop)) +- Refactor runner configuration, add labels to dataflow options [\#718](https://github.com/feast-dev/feast/pull/718) ([zhilingc](https://github.com/zhilingc)) + +**Fixed bugs:** +- Fix pipeline options toArgs\(\) returning empty list [\#765](https://github.com/feast-dev/feast/pull/765) ([zhilingc](https://github.com/zhilingc)) +- Fix project argument for feature set describe in CLI [\#731](https://github.com/feast-dev/feast/pull/731) ([terryyylim](https://github.com/terryyylim)) +- Fix Go and Java SDK Regressions [\#729](https://github.com/feast-dev/feast/pull/729) ([mrzzy](https://github.com/mrzzy)) +- Remove usage of parallel stream for feature value map generation [\#751](https://github.com/feast-dev/feast/pull/751) ([khorshuheng](https://github.com/khorshuheng)) +- Restore Feast Java SDK and Ingestion compatibility with Java 8 runtimes [\#722](https://github.com/feast-dev/feast/pull/722) ([ches](https://github.com/ches)) +- Python sdk bug fixes [\#723](https://github.com/feast-dev/feast/pull/723) ([zhilingc](https://github.com/zhilingc)) + +**Merged pull requests:** +- Increase Jaeger Tracing coverage [\#719](https://github.com/feast-dev/feast/pull/719) ([terryyylim](https://github.com/terryyylim)) +- Recompile golang protos to include new FeatureSetStatus [\#755](https://github.com/feast-dev/feast/pull/755) ([zhilingc](https://github.com/zhilingc)) +- Merge Redis cluster connector with Redis connector [\#752](https://github.com/feast-dev/feast/pull/752) ([pyalex](https://github.com/pyalex)) +- Remove unused Hibernate dep from Serving [\#721](https://github.com/feast-dev/feast/pull/721) ([ches](https://github.com/ches)) + ## [v0.5.0](https://github.com/feast-dev/feast/tree/v0.5.0) (2020-05-19) [Full Changelog](https://github.com/feast-dev/feast/compare/v0.4.7...v0.5.0) From 561d6f415c3842d74e33ba97fed4cfdf041767e7 Mon Sep 17 00:00:00 2001 From: Willem Pienaar <6728866+woop@users.noreply.github.com> Date: Mon, 8 Jun 2020 15:22:02 +0800 Subject: [PATCH 040/133] Add YAML export to Python SDK (#782) * Add feature set export method * Fix equality in feature set and remove YAML path export * Remove spec from yaml export * Conditionally delete meta from feature set when empty --- sdk/go/client.go | 2 +- sdk/python/feast/feature_set.py | 41 ++++++++++++++++++++++++++-- sdk/python/tests/test_feature_set.py | 22 +++++++++++++++ 3 files changed, 61 insertions(+), 4 deletions(-) diff --git a/sdk/go/client.go b/sdk/go/client.go index 1f0cbaa3b96..30b3dc4bf09 100644 --- a/sdk/go/client.go +++ b/sdk/go/client.go @@ -54,7 +54,7 @@ func (fc *GrpcClient) GetOnlineFeatures(ctx context.Context, req *OnlineFeatures // collect unqiue entity refs from entity rows entityRefs := make(map[string]struct{}) for _, entityRows := range req.Entities { - for ref, _ := range entityRows { + for ref := range entityRows { entityRefs[ref] = struct{}{} } } diff --git a/sdk/python/feast/feature_set.py b/sdk/python/feast/feature_set.py index aebee52ca42..a181c9378bd 100644 --- a/sdk/python/feast/feature_set.py +++ b/sdk/python/feast/feature_set.py @@ -17,9 +17,10 @@ import pandas as pd import pyarrow as pa +import yaml from google.protobuf import json_format from google.protobuf.duration_pb2 import Duration -from google.protobuf.json_format import MessageToJson +from google.protobuf.json_format import MessageToDict, MessageToJson from google.protobuf.message import Message from pandas.api.types import is_datetime64_ns_dtype from pyarrow.lib import TimestampType @@ -79,12 +80,18 @@ def __eq__(self, other): if key not in other.fields.keys() or self.fields[key] != other.fields[key]: return False + if self.fields[key] != other.fields[key]: + return False + if ( self.name != other.name or self.project != other.project or self.max_age != other.max_age ): return False + + if self.source != other.source: + return False return True def __str__(self): @@ -783,13 +790,18 @@ def from_proto(cls, feature_set_proto: FeatureSetProto): entities=[ Entity.from_proto(entity) for entity in feature_set_proto.spec.entities ], - max_age=feature_set_proto.spec.max_age, + max_age=( + None + if feature_set_proto.spec.max_age.seconds == 0 + and feature_set_proto.spec.max_age.nanos == 0 + else feature_set_proto.spec.max_age + ), source=( None if feature_set_proto.spec.source.type == 0 else Source.from_proto(feature_set_proto.spec.source) ), - project=feature_set_proto.spec.project + project=None if len(feature_set_proto.spec.project) == 0 else feature_set_proto.spec.project, ) @@ -828,6 +840,29 @@ def to_proto(self) -> FeatureSetProto: return FeatureSetProto(spec=spec, meta=meta) + def to_dict(self) -> Dict: + """ + Converts feature set to dict + + :return: Dictionary object representation of feature set + """ + feature_set_dict = MessageToDict(self.to_proto()) + + # Remove meta when empty for more readable exports + if feature_set_dict["meta"] == {}: + del feature_set_dict["meta"] + + return feature_set_dict + + def to_yaml(self): + """ + Converts a feature set to a YAML string. + + :return: Feature set string returned in YAML format + """ + feature_set_dict = self.to_dict() + return yaml.dump(feature_set_dict, allow_unicode=True, sort_keys=False) + class FeatureSetRef: """ diff --git a/sdk/python/tests/test_feature_set.py b/sdk/python/tests/test_feature_set.py index 04e75c9e76d..4052623e699 100644 --- a/sdk/python/tests/test_feature_set.py +++ b/sdk/python/tests/test_feature_set.py @@ -11,6 +11,7 @@ # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. # See the License for the specific language governing permissions and # limitations under the License. + import pathlib from concurrent import futures from datetime import datetime @@ -244,6 +245,27 @@ def test_export_tfx_schema(self): for actual, expected in zip(actual_schema.feature, expected_schema.feature): assert actual.SerializeToString() == expected.SerializeToString() + def test_feature_set_import_export_yaml(self): + + test_feature_set = FeatureSet( + name="bikeshare", + entities=[Entity(name="station_id", dtype=ValueType.INT64)], + features=[ + Feature(name="name", dtype=ValueType.STRING), + Feature(name="longitude", dtype=ValueType.FLOAT), + Feature(name="location", dtype=ValueType.STRING), + ], + ) + + # Create a string YAML representation of the feature set + string_yaml = test_feature_set.to_yaml() + + # Create a new feature set object from the YAML string + actual_feature_set_from_string = FeatureSet.from_yaml(string_yaml) + + # Ensure equality is upheld to original feature set + assert test_feature_set == actual_feature_set_from_string + def make_tfx_schema_domain_info_inline(schema): # Copy top-level domain info defined in the schema to inline definition. From ef47f3c6eba58b79e2d722e4485c5505f61b4699 Mon Sep 17 00:00:00 2001 From: Willem Pienaar Date: Tue, 9 Jun 2020 00:16:53 +0000 Subject: [PATCH 041/133] GitBook: [master] 28 pages modified --- docs/user-guide/feature-retrieval.md | 6 +++++- 1 file changed, 5 insertions(+), 1 deletion(-) diff --git a/docs/user-guide/feature-retrieval.md b/docs/user-guide/feature-retrieval.md index 0ebf79ca88e..12ad64c4f3e 100644 --- a/docs/user-guide/feature-retrieval.md +++ b/docs/user-guide/feature-retrieval.md @@ -101,6 +101,10 @@ print(dataset.to_dataframe()) Feast is able to intelligently join feature data with different timestamps to a single basis table in a point-in-time-correct way. This allows users to join daily batch data with high-frequency event data transparently. They simply need to know the feature names. +{% hint style="info" %} +Feast can retrieve features from any amount of feature sets, as long as they occur on the same entities. +{% endhint %} + Point-in-time-correct joins also prevents the occurrence of feature leakage by trying to accurate the state of the world at a single point in time, instead of just joining features based on the nearest timestamps. ## Online feature retrieval @@ -121,7 +125,7 @@ data = client.get_online_features( ``` {% hint style="info" %} -When no project is specified when retrieving features with `get_online_feature()`, Feast infers that the features specified belong to the `default` project. To retrieve from another project, specify the `project` parameter when retrieving features. +When no project is specified when retrieving features with get\_online\_feature\(\), Feast infers that the features specified belong to the default project. To retrieve from another project, specify the project parameter when retrieving features. {% endhint %} Online serving with Feast is built to be very low latency. Feast Serving provides a [gRPC API](https://api.docs.feast.dev/grpc/feast.serving.pb.html) that is backed by [Redis](https://redis.io/). We also provide support for [Python](https://api.docs.feast.dev/python/), [Go](https://godoc.org/github.com/gojek/feast/sdk/go), and Java clients. From 1b2e3a6fd2b6cb982693301a6cafd0ed36f4d06f Mon Sep 17 00:00:00 2001 From: zhilingc Date: Tue, 19 May 2020 17:27:54 +0800 Subject: [PATCH 042/133] Add method to retrieve statistics over historical data --- .../core/dao/FeatureStatisticsRepository.java | 31 + .../java/feast/core/grpc/CoreServiceImpl.java | 54 +- .../feast/core/model/FeatureStatistics.java | 243 +++++++ .../java/feast/core/service/StatsService.java | 635 ++++++++++++++++++ .../feast/core/service/StatsServiceTest.java | 378 +++++++++++ protos/feast/core/CoreService.proto | 46 ++ .../proto/v0/statistics.proto | 427 ++++++++++++ sdk/python/feast/cli.py | 2 +- sdk/python/feast/client.py | 80 ++- sdk/python/feast/feature_set.py | 3 +- sdk/python/feast/loaders/ingest.py | 2 +- sdk/python/feast/type_map.py | 4 +- sdk/python/setup.py | 2 +- .../api/statistics/FeatureSetStatistics.java | 48 ++ .../api/statistics/StatisticsRetriever.java | 46 ++ .../BigQueryStatisticsRetriever.java | 154 +++++ .../FeatureSetStatisticsQueryInfo.java | 102 +++ .../FeatureStatisticsQueryInfo.java | 99 +++ .../bigquery/statistics/StatsQueryResult.java | 332 +++++++++ .../statistics/StatsQueryTemplater.java | 78 +++ .../main/resources/templates/basic_stats.sql | 85 +++ .../main/resources/templates/hist_stats.sql | 41 ++ tests/e2e/requirements.txt | 3 + 23 files changed, 2864 insertions(+), 31 deletions(-) create mode 100644 core/src/main/java/feast/core/dao/FeatureStatisticsRepository.java create mode 100644 core/src/main/java/feast/core/model/FeatureStatistics.java create mode 100644 core/src/main/java/feast/core/service/StatsService.java create mode 100644 core/src/test/java/feast/core/service/StatsServiceTest.java create mode 100644 protos/tensorflow_metadata/proto/v0/statistics.proto create mode 100644 storage/api/src/main/java/feast/storage/api/statistics/FeatureSetStatistics.java create mode 100644 storage/api/src/main/java/feast/storage/api/statistics/StatisticsRetriever.java create mode 100644 storage/connectors/bigquery/src/main/java/feast/storage/connectors/bigquery/statistics/BigQueryStatisticsRetriever.java create mode 100644 storage/connectors/bigquery/src/main/java/feast/storage/connectors/bigquery/statistics/FeatureSetStatisticsQueryInfo.java create mode 100644 storage/connectors/bigquery/src/main/java/feast/storage/connectors/bigquery/statistics/FeatureStatisticsQueryInfo.java create mode 100644 storage/connectors/bigquery/src/main/java/feast/storage/connectors/bigquery/statistics/StatsQueryResult.java create mode 100644 storage/connectors/bigquery/src/main/java/feast/storage/connectors/bigquery/statistics/StatsQueryTemplater.java create mode 100644 storage/connectors/bigquery/src/main/resources/templates/basic_stats.sql create mode 100644 storage/connectors/bigquery/src/main/resources/templates/hist_stats.sql diff --git a/core/src/main/java/feast/core/dao/FeatureStatisticsRepository.java b/core/src/main/java/feast/core/dao/FeatureStatisticsRepository.java new file mode 100644 index 00000000000..4046295b75e --- /dev/null +++ b/core/src/main/java/feast/core/dao/FeatureStatisticsRepository.java @@ -0,0 +1,31 @@ +/* + * SPDX-License-Identifier: Apache-2.0 + * Copyright 2018-2019 The Feast Authors + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package feast.core.dao; + +import feast.core.model.Feature; +import feast.core.model.FeatureStatistics; +import java.util.Date; +import java.util.Optional; +import org.springframework.data.jpa.repository.JpaRepository; + +/** JPA repository supplying Statistics objects for features keyed by id. */ +public interface FeatureStatisticsRepository extends JpaRepository { + Optional findFeatureStatisticsByFeatureAndDatasetId( + Feature feature, String datasetId); + + Optional findFeatureStatisticsByFeatureAndDate(Feature feature, Date date); +} diff --git a/core/src/main/java/feast/core/grpc/CoreServiceImpl.java b/core/src/main/java/feast/core/grpc/CoreServiceImpl.java index db65328b821..18b8b870fba 100644 --- a/core/src/main/java/feast/core/grpc/CoreServiceImpl.java +++ b/core/src/main/java/feast/core/grpc/CoreServiceImpl.java @@ -25,31 +25,9 @@ import feast.core.service.AccessManagementService; import feast.core.service.JobService; import feast.core.service.SpecService; +import feast.core.service.StatsService; import feast.proto.core.CoreServiceGrpc.CoreServiceImplBase; -import feast.proto.core.CoreServiceProto.ApplyFeatureSetRequest; -import feast.proto.core.CoreServiceProto.ApplyFeatureSetResponse; -import feast.proto.core.CoreServiceProto.ArchiveProjectRequest; -import feast.proto.core.CoreServiceProto.ArchiveProjectResponse; -import feast.proto.core.CoreServiceProto.CreateProjectRequest; -import feast.proto.core.CoreServiceProto.CreateProjectResponse; -import feast.proto.core.CoreServiceProto.GetFeastCoreVersionRequest; -import feast.proto.core.CoreServiceProto.GetFeastCoreVersionResponse; -import feast.proto.core.CoreServiceProto.GetFeatureSetRequest; -import feast.proto.core.CoreServiceProto.GetFeatureSetResponse; -import feast.proto.core.CoreServiceProto.ListFeatureSetsRequest; -import feast.proto.core.CoreServiceProto.ListFeatureSetsResponse; -import feast.proto.core.CoreServiceProto.ListIngestionJobsRequest; -import feast.proto.core.CoreServiceProto.ListIngestionJobsResponse; -import feast.proto.core.CoreServiceProto.ListProjectsRequest; -import feast.proto.core.CoreServiceProto.ListProjectsResponse; -import feast.proto.core.CoreServiceProto.ListStoresRequest; -import feast.proto.core.CoreServiceProto.ListStoresResponse; -import feast.proto.core.CoreServiceProto.RestartIngestionJobRequest; -import feast.proto.core.CoreServiceProto.RestartIngestionJobResponse; -import feast.proto.core.CoreServiceProto.StopIngestionJobRequest; -import feast.proto.core.CoreServiceProto.StopIngestionJobResponse; -import feast.proto.core.CoreServiceProto.UpdateStoreRequest; -import feast.proto.core.CoreServiceProto.UpdateStoreResponse; +import feast.proto.core.CoreServiceProto.*; import io.grpc.Status; import io.grpc.StatusRuntimeException; import io.grpc.stub.StreamObserver; @@ -69,11 +47,13 @@ public class CoreServiceImpl extends CoreServiceImplBase { private SpecService specService; private AccessManagementService accessManagementService; private JobService jobService; + private StatsService statsService; @Autowired public CoreServiceImpl( SpecService specService, AccessManagementService accessManagementService, + StatsService statsService, JobService jobService, FeastProperties feastProperties) { this.specService = specService; @@ -126,6 +106,32 @@ public void listFeatureSets( } } + @Override + public void getFeatureStatistics( + GetFeatureStatisticsRequest request, + StreamObserver responseObserver) { + try { + GetFeatureStatisticsResponse response = statsService.getFeatureStatistics(request); + responseObserver.onNext(response); + responseObserver.onCompleted(); + } catch (IllegalArgumentException e) { + log.error("Illegal arguments provided to GetFeatureStatistics method: ", e); + responseObserver.onError( + Status.INVALID_ARGUMENT + .withDescription(e.getMessage()) + .withCause(e) + .asRuntimeException()); + } catch (RetrievalException e) { + log.error("Unable to fetch feature set requested in GetFeatureStatistics method: ", e); + responseObserver.onError( + Status.NOT_FOUND.withDescription(e.getMessage()).withCause(e).asRuntimeException()); + } catch (Exception e) { + log.error("Exception has occurred in GetFeatureStatistics method: ", e); + responseObserver.onError( + Status.INTERNAL.withDescription(e.getMessage()).withCause(e).asRuntimeException()); + } + } + @Override public void listStores( ListStoresRequest request, StreamObserver responseObserver) { diff --git a/core/src/main/java/feast/core/model/FeatureStatistics.java b/core/src/main/java/feast/core/model/FeatureStatistics.java new file mode 100644 index 00000000000..e06e7d9a6f3 --- /dev/null +++ b/core/src/main/java/feast/core/model/FeatureStatistics.java @@ -0,0 +1,243 @@ +/* + * SPDX-License-Identifier: Apache-2.0 + * Copyright 2018-2020 The Feast Authors + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package feast.core.model; + +import com.google.protobuf.InvalidProtocolBufferException; +import java.io.*; +import java.util.Date; +import java.util.List; +import javax.persistence.*; +import javax.persistence.Entity; +import lombok.Getter; +import lombok.NoArgsConstructor; +import lombok.Setter; +import org.tensorflow.metadata.v0.*; + +@NoArgsConstructor +@Getter +@Setter +@Entity +@Table( + name = "feature_statistics", + indexes = { + @Index(name = "idx_feature_statistics_feature", columnList = "feature_id"), + @Index(name = "idx_feature_statistics_dataset_id", columnList = "datasetId"), + @Index(name = "idx_feature_statistics_date", columnList = "date"), + }) +public class FeatureStatistics { + @Id + @GeneratedValue(strategy = GenerationType.AUTO) + private int id; + + @ManyToOne private Feature feature; + + // Only one of these fields should be populated. + private String datasetId; + private Date date; + + // General statistics + private String featureType; + private long count; + private long numMissing; + private long minNumValues; + private long maxNumValues; + private float avgNumValues; + private long totalNumValues; + private byte[] numValuesHistogram; + + // Numeric statistics + private double mean; + private double stdev; + private long zeroes; + private double min; + private double max; + private double median; + private byte[] numericValueHistogram; + private byte[] numericValueQuantiles; + + // String statistics + @Column(name = "n_unique") + private long unique; + + private float averageLength; + private byte[] rankHistogram; + private byte[] topValues; + + // Byte statistics + private float minBytes; + private float maxBytes; + private float avgBytes; + + // Instantiates a Statistics object from a tensorflow metadata FeatureNameStatistics object and a + // dataset ID. + public static FeatureStatistics createForDataset( + Feature feature, FeatureNameStatistics featureNameStatistics, String datasetId) + throws IOException { + FeatureStatistics featureStatistics = FeatureStatistics.fromProto(featureNameStatistics); + featureStatistics.setFeature(feature); + featureStatistics.setDatasetId(datasetId); + return featureStatistics; + } + + // Instantiates a Statistics object from a tensorflow metadata FeatureNameStatistics object and a + // date. + public static FeatureStatistics createForDate( + Feature feature, FeatureNameStatistics featureNameStatistics, Date date) throws IOException { + FeatureStatistics featureStatistics = FeatureStatistics.fromProto(featureNameStatistics); + featureStatistics.setDate(date); + featureStatistics.setFeature(feature); + return featureStatistics; + } + + public FeatureNameStatistics toProto() throws InvalidProtocolBufferException { + FeatureNameStatistics.Builder featureNameStatisticsBuilder = + FeatureNameStatistics.newBuilder() + .setType(FeatureNameStatistics.Type.valueOf(featureType)) + .setPath(Path.newBuilder().addStep(feature.getName())); + CommonStatistics commonStatistics = + CommonStatistics.newBuilder() + .setNumNonMissing(count - numMissing) + .setNumMissing(numMissing) + .setMaxNumValues(maxNumValues) + .setMinNumValues(minNumValues) + .setTotNumValues(totalNumValues) + .setNumValuesHistogram(Histogram.parseFrom(numValuesHistogram)) + .build(); + + switch (featureNameStatisticsBuilder.getType()) { + case INT: + case FLOAT: + NumericStatistics numStats = + NumericStatistics.newBuilder() + .setCommonStats(commonStatistics) + .setMean(mean) + .setStdDev(stdev) + .setNumZeros(zeroes) + .setMin(min) + .setMax(max) + .setMedian(median) + .addHistograms(Histogram.parseFrom(numericValueHistogram)) + .addHistograms(Histogram.parseFrom(numericValueQuantiles)) + .build(); + featureNameStatisticsBuilder.setNumStats(numStats); + break; + case STRING: + StringStatistics.Builder stringStats = + StringStatistics.newBuilder() + .setCommonStats(commonStatistics) + .setUnique(unique) + .setAvgLength(averageLength); + if (rankHistogram == null) { + stringStats.setRankHistogram(RankHistogram.getDefaultInstance()); + } else { + stringStats.setRankHistogram(RankHistogram.parseFrom(rankHistogram)); + } + try (ByteArrayInputStream bis = new ByteArrayInputStream(topValues)) { + ObjectInputStream ois = new ObjectInputStream(bis); + List freqAndValueList = + (List) ois.readObject(); + stringStats.addAllTopValues(freqAndValueList); + } catch (IOException | ClassNotFoundException e) { + throw new InvalidProtocolBufferException( + "Failed to parse field: StringStatistics.TopValues. Check if the value is malformed."); + } + featureNameStatisticsBuilder.setStringStats(stringStats); + break; + case BYTES: + BytesStatistics bytesStats = + BytesStatistics.newBuilder() + .setCommonStats(commonStatistics) + .setAvgNumBytes(avgBytes) + .setMinNumBytes(minBytes) + .setMaxNumBytes(maxBytes) + .build(); + featureNameStatisticsBuilder.setBytesStats(bytesStats); + break; + case STRUCT: + StructStatistics structStats = + StructStatistics.newBuilder().setCommonStats(commonStatistics).build(); + featureNameStatisticsBuilder.setStructStats(structStats); + break; + } + return featureNameStatisticsBuilder.build(); + } + + private static FeatureStatistics fromProto(FeatureNameStatistics featureNameStatistics) + throws IOException, IllegalArgumentException { + FeatureStatistics featureStatistics = new FeatureStatistics(); + featureStatistics.setFeatureType(featureNameStatistics.getType().toString()); + CommonStatistics commonStats; + switch (featureNameStatistics.getType()) { + case FLOAT: + case INT: + NumericStatistics numStats = featureNameStatistics.getNumStats(); + commonStats = numStats.getCommonStats(); + featureStatistics.setMean(numStats.getMean()); + featureStatistics.setStdev(numStats.getStdDev()); + featureStatistics.setZeroes(numStats.getNumZeros()); + featureStatistics.setMin(numStats.getMin()); + featureStatistics.setMax(numStats.getMax()); + featureStatistics.setMedian(numStats.getMedian()); + for (Histogram histogram : numStats.getHistogramsList()) { + switch (histogram.getType()) { + case STANDARD: + featureStatistics.setNumericValueHistogram(histogram.toByteArray()); + case QUANTILES: + featureStatistics.setNumericValueQuantiles(histogram.toByteArray()); + default: + // invalid type, dropping the values + } + } + break; + case STRING: + StringStatistics stringStats = featureNameStatistics.getStringStats(); + commonStats = stringStats.getCommonStats(); + featureStatistics.setUnique(stringStats.getUnique()); + featureStatistics.setAverageLength(stringStats.getAvgLength()); + featureStatistics.setRankHistogram(stringStats.getRankHistogram().toByteArray()); + try (ByteArrayOutputStream bos = new ByteArrayOutputStream()) { + ObjectOutputStream oos = new ObjectOutputStream(bos); + oos.writeObject(stringStats.getTopValuesList()); + featureStatistics.setTopValues(bos.toByteArray()); + } + break; + case BYTES: + BytesStatistics bytesStats = featureNameStatistics.getBytesStats(); + commonStats = bytesStats.getCommonStats(); + featureStatistics.setUnique(bytesStats.getUnique()); + featureStatistics.setMaxBytes(bytesStats.getMaxNumBytes()); + featureStatistics.setMinBytes(bytesStats.getMinNumBytes()); + featureStatistics.setAvgBytes(bytesStats.getAvgNumBytes()); + break; + case STRUCT: + StructStatistics structStats = featureNameStatistics.getStructStats(); + commonStats = structStats.getCommonStats(); + break; + default: + throw new IllegalArgumentException("Feature statistics provided were of unknown type."); + } + featureStatistics.setCount(commonStats.getNumMissing() + commonStats.getNumNonMissing()); + featureStatistics.setNumMissing(commonStats.getNumMissing()); + featureStatistics.setMinNumValues(commonStats.getMinNumValues()); + featureStatistics.setMaxNumValues(commonStats.getMaxNumValues()); + featureStatistics.setAvgNumValues(commonStats.getAvgNumValues()); + featureStatistics.setTotalNumValues(commonStats.getTotNumValues()); + featureStatistics.setNumValuesHistogram(commonStats.getNumValuesHistogram().toByteArray()); + + return featureStatistics; + } +} diff --git a/core/src/main/java/feast/core/service/StatsService.java b/core/src/main/java/feast/core/service/StatsService.java new file mode 100644 index 00000000000..1fddfaea05c --- /dev/null +++ b/core/src/main/java/feast/core/service/StatsService.java @@ -0,0 +1,635 @@ +/* + * SPDX-License-Identifier: Apache-2.0 + * Copyright 2018-2020 The Feast Authors + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package feast.core.service; + +import static java.lang.Math.*; + +import com.google.common.annotations.VisibleForTesting; +import com.google.protobuf.InvalidProtocolBufferException; +import com.google.protobuf.Timestamp; +import feast.core.dao.FeatureSetRepository; +import feast.core.dao.FeatureStatisticsRepository; +import feast.core.dao.StoreRepository; +import feast.core.exception.RetrievalException; +import feast.core.model.*; +import feast.core.model.Feature; +import feast.proto.core.CoreServiceProto.GetFeatureStatisticsRequest; +import feast.proto.core.CoreServiceProto.GetFeatureStatisticsResponse; +import feast.proto.core.StoreProto; +import feast.proto.core.StoreProto.Store.StoreType; +import feast.storage.api.statistics.FeatureSetStatistics; +import feast.storage.api.statistics.StatisticsRetriever; +import feast.storage.connectors.bigquery.statistics.BigQueryStatisticsRetriever; +import java.io.IOException; +import java.time.Instant; +import java.util.*; +import java.util.stream.Collectors; +import lombok.extern.slf4j.Slf4j; +import org.joda.time.DateTime; +import org.joda.time.DateTimeZone; +import org.joda.time.format.DateTimeFormat; +import org.joda.time.format.DateTimeFormatter; +import org.springframework.beans.factory.annotation.Autowired; +import org.springframework.stereotype.Service; +import org.springframework.transaction.annotation.Transactional; +import org.tensorflow.metadata.v0.*; +import org.tensorflow.metadata.v0.FeatureNameStatistics.Builder; + +/** Facilitates the retrieval of feature set statistics from historical stores. */ +@Slf4j +@Service +public class StatsService { + + private StoreRepository storeRepository; + private FeatureStatisticsRepository featureStatisticsRepository; + private FeatureSetRepository featureSetRepository; + + @Autowired + public StatsService( + StoreRepository storeRepository, + FeatureStatisticsRepository featureStatisticsRepository, + FeatureSetRepository featureSetRepository) { + this.storeRepository = storeRepository; + this.featureStatisticsRepository = featureStatisticsRepository; + this.featureSetRepository = featureSetRepository; + } + + /** + * Get {@link DatasetFeatureStatistics} for the requested feature set in the provided datasets or + * date range for the store provided. The {@link DatasetFeatureStatistics} will contain a list of + * {@link FeatureNameStatistics} for each feature requested. Results retrieved will be cached + * indefinitely. To force Feast to recompute the statistics, set forceRefresh to true. + * + *

Only one of ingestionIds or startDate/endDate should be provided. If both are provided, the + * former will be used over the latter. + * + *

If multiple datasetIds or if the date ranges over a few days, statistics will be retrieved + * for each single unit (dataset id or day) and results aggregated across that set. As a result of + * this, in such a scenario, statistics that cannot be aggregated will be dropped. This includes + * all histograms and quantiles, unique values, and top value counts. + * + * @param request {@link GetFeatureStatisticsRequest} containing feature set name, subset of + * features, dataset ids or date range, and store to retrieve the data from. + * @return {@link GetFeatureStatisticsResponse} containing {@link DatasetFeatureStatistics} with + * the feature statistics requested. + * @throws IOException + */ + @Transactional + public GetFeatureStatisticsResponse getFeatureStatistics(GetFeatureStatisticsRequest request) + throws IOException { + + // Validate the request + validateRequest(request); + + // Get the stats retriever for the store requested + StatisticsRetriever statisticsRetriever = getStatisticsRetriever(request.getStore()); + + // 1. Retrieve the feature set spec from the db + FeatureSet featureSet = getFeatureSet(request.getFeatureSetId()); + if (featureSet == null) { + throw new IllegalArgumentException( + String.format( + "Illegal request. Unable to find feature set %s", request.getFeatureSetId())); + } + + // 2. Filter out the features requested by the user. If none are provided, + // use all features in the feature set. + List features = request.getFeaturesList(); + if (features.size() == 0) { + features = + featureSet.getFeatures().stream() + .filter(feature -> !feature.isArchived()) + .map(Feature::getName) + .collect(Collectors.toList()); + } + + // 3. Retrieve the statistics from the StatsRetriever. + List> featureNameStatisticsList = new ArrayList<>(); + if (request.getIngestionIdsCount() == 0) { + Timestamp endDate = request.getEndDate(); + Timestamp startDate = request.getStartDate(); + // If no dataset provided, retrieve by date + + long timestamp = startDate.getSeconds(); + while (timestamp < endDate.getSeconds()) { + List featureNameStatistics = + getFeatureNameStatisticsByDate( + statisticsRetriever, featureSet, features, timestamp, request.getForceRefresh()); + featureNameStatisticsList.add(featureNameStatistics); + timestamp += 86400; // advance by a day + } + if (featureNameStatisticsList.size() == 0) { + DateTimeFormatter fmt = DateTimeFormat.forPattern("yyyy-MM-dd"); + DateTime startDateTime = new DateTime(startDate.getSeconds() * 1000, DateTimeZone.UTC); + DateTime endDateTime = new DateTime(endDate.getSeconds() * 1000, DateTimeZone.UTC); + throw new RetrievalException( + String.format( + "Unable to find any data over provided dates [%s, %s)", + fmt.print(startDateTime), fmt.print(endDateTime))); + } + } else { + // else, retrieve by dataset + for (String datasetId : request.getIngestionIdsList()) { + List featureNameStatistics = + getFeatureNameStatisticsByDataset( + statisticsRetriever, featureSet, features, datasetId, request.getForceRefresh()); + featureNameStatisticsList.add(featureNameStatistics); + if (featureNameStatisticsList.size() == 0) { + throw new RetrievalException( + String.format( + "Unable to find any data over provided data sets %s", + request.getIngestionIdsList())); + } + } + } + + // Merge statistics values across days/datasets + List featureNameStatistics = mergeStatistics(featureNameStatisticsList); + long totalCount = getTotalCount(featureNameStatistics.get(0)); + return GetFeatureStatisticsResponse.newBuilder() + .setDatasetFeatureStatisticsList( + DatasetFeatureStatisticsList.newBuilder() + .addDatasets( + DatasetFeatureStatistics.newBuilder() + .setNumExamples(totalCount) + .addAllFeatures(featureNameStatistics))) + .build(); + } + + /** + * Get {@link FeatureNameStatistics} by dataset id. + * + * @param statisticsRetriever {@link StatisticsRetriever} corresponding to the store to get the + * data from. + * @param featureSet {@link FeatureSet} requested by the user + * @param features features to retrieve + * @param datasetId dataset id to subset the data by + * @param forceRefresh whether to override the values in the cache + * @return {@link FeatureNameStatistics} for the data within the dataset id provided + * @throws IOException + */ + private List getFeatureNameStatisticsByDataset( + StatisticsRetriever statisticsRetriever, + FeatureSet featureSet, + List features, + String datasetId, + boolean forceRefresh) + throws IOException { + List featureNameStatistics = new ArrayList<>(); + List featuresMissingStats = new ArrayList<>(); + Map featureNameToFeature = + featureSet.getFeatures().stream().collect(Collectors.toMap(Feature::getName, f -> f)); + + // For each feature requested, check if statistics already exist in the cache + // If not refreshing data in the cache, retrieve the cached data and add it to the + // list of FeatureNameStatistics for this dataset. + // Else, add to the list of features we still need to retrieve statistics for. + for (String featureName : features) { + Feature feature = featureNameToFeature.get(featureName); + Optional cachedFeatureStatistics = Optional.empty(); + if (!forceRefresh) { + cachedFeatureStatistics = + featureStatisticsRepository.findFeatureStatisticsByFeatureAndDatasetId( + feature, datasetId); + } + if (cachedFeatureStatistics.isPresent()) { + featureNameStatistics.add(cachedFeatureStatistics.get().toProto()); + } else { + featuresMissingStats.add(featureName); + } + } + + // Retrieve the balance of statistics after checking the cache, and add it to the + // list of FeatureNameStatistics. + if (featuresMissingStats.size() > 0) { + FeatureSetStatistics featureSetStatistics = + statisticsRetriever.getFeatureStatistics( + featureSet.toProto().getSpec(), featuresMissingStats, datasetId); + + // Persist the newly retrieved statistics in the cache. + for (FeatureNameStatistics stat : featureSetStatistics.getFeatureNameStatistics()) { + if (isEmpty(stat)) { + continue; + } + Feature feature = featureNameToFeature.get(stat.getName()); + FeatureStatistics featureStatistics = + FeatureStatistics.createForDataset(feature, stat, datasetId); + Optional existingRecord = + featureStatisticsRepository.findFeatureStatisticsByFeatureAndDatasetId( + featureStatistics.getFeature(), datasetId); + existingRecord.ifPresent(statistics -> featureStatistics.setId(statistics.getId())); + featureStatisticsRepository.save(featureStatistics); + featureNameStatistics.add(stat); + } + } + return featureNameStatistics; + } + + /** + * Get {@link FeatureNameStatistics} by date. + * + * @param statisticsRetriever {@link StatisticsRetriever} corresponding to the store to get the + * data from. + * @param featureSet {@link FeatureSet} requested by the user + * @param features features to retrieve + * @param timestamp timestamp of the date to subset the data + * @param forceRefresh whether to override the values in the cache + * @return {@link FeatureNameStatistics} for the data within the dataset id provided + * @throws IOException + */ + private List getFeatureNameStatisticsByDate( + StatisticsRetriever statisticsRetriever, + FeatureSet featureSet, + List features, + long timestamp, + boolean forceRefresh) + throws IOException { + Date date = Date.from(Instant.ofEpochSecond(timestamp)); + List featureNameStatistics = new ArrayList<>(); + List featuresMissingStats = new ArrayList<>(); + Map featureNameToFeature = + featureSet.getFeatures().stream().collect(Collectors.toMap(Feature::getName, f -> f)); + + // For each feature requested, check if statistics already exist in the cache + // If not refreshing data in the cache, retrieve the cached data and add it to the + // list of FeatureNameStatistics for this date. + // Else, add to the list of features we still need to retrieve statistics for. + for (String featureName : features) { + Feature feature = featureNameToFeature.get(featureName); + Optional cachedFeatureStatistics = Optional.empty(); + if (!forceRefresh) { + cachedFeatureStatistics = + featureStatisticsRepository.findFeatureStatisticsByFeatureAndDate(feature, date); + } + if (cachedFeatureStatistics.isPresent()) { + featureNameStatistics.add(cachedFeatureStatistics.get().toProto()); + } else { + featuresMissingStats.add(featureName); + } + } + + // Retrieve the balance of statistics after checking the cache, and add it to the + // list of FeatureNameStatistics. + if (featuresMissingStats.size() > 0) { + FeatureSetStatistics featureSetStatistics = + statisticsRetriever.getFeatureStatistics( + featureSet.toProto().getSpec(), + featuresMissingStats, + Timestamp.newBuilder().setSeconds(timestamp).build()); + + // Persist the newly retrieved statistics in the cache. + for (FeatureNameStatistics stat : featureSetStatistics.getFeatureNameStatistics()) { + if (isEmpty(stat)) { + continue; + } + Feature feature = featureNameToFeature.get(stat.getName()); + FeatureStatistics featureStatistics = FeatureStatistics.createForDate(feature, stat, date); + Optional existingRecord = + featureStatisticsRepository.findFeatureStatisticsByFeatureAndDate( + featureStatistics.getFeature(), date); + existingRecord.ifPresent(statistics -> featureStatistics.setId(statistics.getId())); + featureStatisticsRepository.save(featureStatistics); + featureNameStatistics.add(stat); + } + } + return featureNameStatistics; + } + + /** + * Get the {@link StatisticsRetriever} corresponding to the store name provided. + * + * @param storeName name of the store to retrieve statistics from + * @return {@link StatisticsRetriever} + */ + private StatisticsRetriever getStatisticsRetriever(String storeName) + throws InvalidProtocolBufferException { + Store store = + storeRepository + .findById(storeName) + .orElseThrow( + () -> + new RetrievalException( + String.format("Could not find store with name %s", storeName))); + StoreProto.Store storeProto = store.toProto(); + if (storeProto.getType() != StoreType.BIGQUERY) { + throw new IllegalArgumentException( + String.format( + "Invalid store %s with type %s specified. Batch statistics are only supported for BigQuery stores", + store.getName(), store.getType())); + } + return BigQueryStatisticsRetriever.create(storeProto.getBigqueryConfig()); + } + + private FeatureSet getFeatureSet(String featureSetId) { + String[] split = featureSetId.split("/"); + String project; + String featureSetName; + if (split.length == 1) { + project = Project.DEFAULT_NAME; + featureSetName = split[0]; + } else { + project = split[0]; + featureSetName = split[1]; + } + FeatureSet featureSet = + featureSetRepository.findFeatureSetByNameAndProject_Name(featureSetName, project); + return featureSet; + } + + /** + * Merge feature statistics by name. This method is used to merge statistics retrieved over + * multiple days or datasets. + * + * @param featureNameStatistics {@link FeatureNameStatistics} retrieved from the store + * @return Merged list of {@link FeatureNameStatistics} by name + */ + @VisibleForTesting + public List mergeStatistics( + List> featureNameStatistics) { + List unnestedList = new ArrayList<>(); + + featureNameStatistics.forEach(unnestedList::addAll); + Map> groupByPath = + unnestedList.stream() + .collect(Collectors.groupingBy(FeatureNameStatistics::getPath, Collectors.toList())); + + List merged = new ArrayList<>(); + for (Path key : groupByPath.keySet()) { + List featureNameStatisticsForKey = groupByPath.get(key); + if (featureNameStatisticsForKey.size() == 1) { + merged.add(featureNameStatisticsForKey.get(0)); + } else { + switch (featureNameStatisticsForKey.get(0).getType()) { + case INT: + case FLOAT: + merged.add(mergeNumStatistics(featureNameStatisticsForKey)); + break; + case STRING: + merged.add(mergeCategoricalStatistics(groupByPath.get(key))); + break; + case BYTES: + merged.add(mergeByteStatistics(groupByPath.get(key))); + break; + case STRUCT: + merged.add(mergeStructStats(groupByPath.get(key))); + break; + default: + throw new IllegalArgumentException( + "Statistics are only supported for string, boolean, bytes and numeric features"); + } + } + } + return merged; + } + + private FeatureNameStatistics mergeStructStats( + List featureNameStatisticsList) { + Builder mergedFeatureNameStatistics = + FeatureNameStatistics.newBuilder() + .setPath(featureNameStatisticsList.get(0).getPath()) + .setType(featureNameStatisticsList.get(0).getType()); + + long totalCount = 0; + long missingCount = 0; + long totalNumValues = 0; + long maxNumValues = + featureNameStatisticsList.get(0).getStructStats().getCommonStats().getMaxNumValues(); + long minNumValues = + featureNameStatisticsList.get(0).getStructStats().getCommonStats().getMinNumValues(); + + for (FeatureNameStatistics featureNameStatistics : featureNameStatisticsList) { + StructStatistics structStats = featureNameStatistics.getStructStats(); + totalCount += structStats.getCommonStats().getNumNonMissing(); + missingCount += structStats.getCommonStats().getNumMissing(); + totalNumValues += + structStats.getCommonStats().getAvgNumValues() + * structStats.getCommonStats().getNumNonMissing(); + maxNumValues = max(maxNumValues, structStats.getCommonStats().getMaxNumValues()); + minNumValues = min(minNumValues, structStats.getCommonStats().getMinNumValues()); + } + + StructStatistics mergedStructStatistics = + StructStatistics.newBuilder() + .setCommonStats( + CommonStatistics.newBuilder() + .setTotNumValues(totalNumValues) + .setNumNonMissing(totalCount) + .setAvgNumValues((float) totalNumValues / totalCount) + .setMaxNumValues(maxNumValues) + .setMinNumValues(minNumValues) + .setNumMissing(missingCount)) + .build(); + + return mergedFeatureNameStatistics.setStructStats(mergedStructStatistics).build(); + } + + private FeatureNameStatistics mergeNumStatistics( + List featureNameStatisticsList) { + Builder mergedFeatureNameStatistics = + FeatureNameStatistics.newBuilder() + .setPath(featureNameStatisticsList.get(0).getPath()) + .setType(featureNameStatisticsList.get(0).getType()); + + FeatureNameStatistics first = featureNameStatisticsList.remove(0); + double max = first.getNumStats().getMax(); + double min = first.getNumStats().getMin(); + double var = pow(first.getNumStats().getStdDev(), 2); + long totalCount = first.getNumStats().getCommonStats().getNumNonMissing(); + double totalVal = totalCount * first.getNumStats().getMean(); + long missingCount = first.getNumStats().getCommonStats().getNumMissing(); + long zeroes = first.getNumStats().getNumZeros(); + + for (FeatureNameStatistics featureNameStatistics : featureNameStatisticsList) { + NumericStatistics numStats = featureNameStatistics.getNumStats(); + max = max(numStats.getMax(), max); + min = min(numStats.getMin(), min); + long count = numStats.getCommonStats().getNumNonMissing(); + double sampleVar = pow(numStats.getStdDev(), 2); + float aggMean = (float) totalVal / totalCount; + var = getVar(var, totalCount, aggMean, sampleVar, count, numStats.getMean()); + totalVal += numStats.getMean() * count; + totalCount += count; + missingCount += numStats.getCommonStats().getNumMissing(); + zeroes += numStats.getNumZeros(); + } + NumericStatistics mergedNumericStatistics = + NumericStatistics.newBuilder() + .setMax(max) + .setMin(min) + .setMean(totalVal / totalCount) + .setNumZeros(zeroes) + .setStdDev(sqrt(var)) + .setCommonStats( + CommonStatistics.newBuilder() + .setTotNumValues(totalCount) + .setNumNonMissing(totalCount) + .setAvgNumValues(1) + .setMaxNumValues(1) + .setMinNumValues(1) + .setNumMissing(missingCount)) + .build(); + return mergedFeatureNameStatistics.setNumStats(mergedNumericStatistics).build(); + } + + // Aggregation of sample variance follows the formula described here: + // https://www.tandfonline.com/doi/abs/10.1080/00031305.2014.966589 + private double getVar( + double s1Var, long s1Count, double s1Mean, double s2Var, long s2Count, double s2Mean) { + long totalCount = s1Count + s2Count; + return ((s1Count - 1) * s1Var + + (s2Count - 1) * s2Var + + ((float) s1Count * s2Count / totalCount) * pow(s1Mean - s2Mean, 2)) + / (s1Count + s2Count - 1); + } + + private FeatureNameStatistics mergeCategoricalStatistics( + List featureNameStatisticsList) { + Builder mergedFeatureNameStatistics = + FeatureNameStatistics.newBuilder() + .setPath(featureNameStatisticsList.get(0).getPath()) + .setType(featureNameStatisticsList.get(0).getType()); + long totalCount = 0; + long missingCount = 0; + long totalLen = 0; + for (FeatureNameStatistics featureNameStatistics : featureNameStatisticsList) { + StringStatistics stringStats = featureNameStatistics.getStringStats(); + totalCount += stringStats.getCommonStats().getNumNonMissing(); + missingCount += stringStats.getCommonStats().getNumMissing(); + totalLen += stringStats.getAvgLength() * stringStats.getCommonStats().getNumNonMissing(); + } + StringStatistics mergedStringStatistics = + StringStatistics.newBuilder() + .setAvgLength((float) totalLen / totalCount) + .setCommonStats( + CommonStatistics.newBuilder() + .setTotNumValues(totalCount) + .setNumNonMissing(totalCount) + .setAvgNumValues(1) + .setMaxNumValues(1) + .setMinNumValues(1) + .setNumMissing(missingCount)) + .build(); + return mergedFeatureNameStatistics.setStringStats(mergedStringStatistics).build(); + } + + private FeatureNameStatistics mergeByteStatistics( + List featureNameStatisticsList) { + Builder mergedFeatureNameStatistics = + FeatureNameStatistics.newBuilder() + .setPath(featureNameStatisticsList.get(0).getPath()) + .setType(featureNameStatisticsList.get(0).getType()); + + long totalCount = 0; + long missingCount = 0; + float totalNumBytes = 0; + float maxNumBytes = featureNameStatisticsList.get(0).getBytesStats().getMaxNumBytes(); + float minNumBytes = featureNameStatisticsList.get(0).getBytesStats().getMinNumBytes(); + + for (FeatureNameStatistics featureNameStatistics : featureNameStatisticsList) { + BytesStatistics bytesStats = featureNameStatistics.getBytesStats(); + totalCount += bytesStats.getCommonStats().getNumNonMissing(); + missingCount += bytesStats.getCommonStats().getNumMissing(); + totalNumBytes += bytesStats.getAvgNumBytes() * bytesStats.getCommonStats().getNumNonMissing(); + maxNumBytes = max(maxNumBytes, bytesStats.getMaxNumBytes()); + minNumBytes = min(minNumBytes, bytesStats.getMinNumBytes()); + } + + BytesStatistics mergedBytesStatistics = + BytesStatistics.newBuilder() + .setAvgNumBytes(totalNumBytes / totalCount) + .setMinNumBytes(minNumBytes) + .setMaxNumBytes(maxNumBytes) + .setCommonStats( + CommonStatistics.newBuilder() + .setTotNumValues(totalCount) + .setNumNonMissing(totalCount) + .setAvgNumValues(1) + .setMaxNumValues(1) + .setMinNumValues(1) + .setNumMissing(missingCount)) + .build(); + + return mergedFeatureNameStatistics.setBytesStats(mergedBytesStatistics).build(); + } + + private long getTotalCount(FeatureNameStatistics featureNameStatistics) { + CommonStatistics commonStats; + switch (featureNameStatistics.getType()) { + case STRUCT: + commonStats = featureNameStatistics.getStructStats().getCommonStats(); + break; + case STRING: + commonStats = featureNameStatistics.getStringStats().getCommonStats(); + break; + case BYTES: + commonStats = featureNameStatistics.getBytesStats().getCommonStats(); + break; + case FLOAT: + case INT: + commonStats = featureNameStatistics.getNumStats().getCommonStats(); + break; + default: + throw new RuntimeException("Unable to extract dataset size; Invalid type provided"); + } + return commonStats.getNumNonMissing() + commonStats.getNumMissing(); + } + + private void validateRequest(GetFeatureStatisticsRequest request) { + if (request.getIngestionIdsCount() == 0) { + Timestamp startDate = request.getStartDate(); + Timestamp endDate = request.getEndDate(); + if (!request.hasStartDate() || !request.hasEndDate()) { + throw new IllegalArgumentException( + "Invalid request. Either provide dataset ids to retrieve statistics over, or a start date and end date."); + } + if (endDate.getSeconds() < startDate.getSeconds()) { + throw new IllegalArgumentException( + String.format( + "Invalid request. Start timestamp %d is greater than the end timestamp %d", + startDate.getSeconds(), endDate.getSeconds())); + } + } + } + + private boolean isEmpty(FeatureNameStatistics featureNameStatistics) { + switch (featureNameStatistics.getType()) { + case STRUCT: + return featureNameStatistics + .getStructStats() + .getCommonStats() + .equals(CommonStatistics.getDefaultInstance()); + case STRING: + return featureNameStatistics + .getStringStats() + .getCommonStats() + .equals(CommonStatistics.getDefaultInstance()); + case BYTES: + return featureNameStatistics + .getBytesStats() + .getCommonStats() + .equals(CommonStatistics.getDefaultInstance()); + case FLOAT: + case INT: + return featureNameStatistics + .getNumStats() + .getCommonStats() + .equals(CommonStatistics.getDefaultInstance()); + default: + return true; + } + } +} diff --git a/core/src/test/java/feast/core/service/StatsServiceTest.java b/core/src/test/java/feast/core/service/StatsServiceTest.java new file mode 100644 index 00000000000..1146e75a3c3 --- /dev/null +++ b/core/src/test/java/feast/core/service/StatsServiceTest.java @@ -0,0 +1,378 @@ +/* + * SPDX-License-Identifier: Apache-2.0 + * Copyright 2018-2020 The Feast Authors + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package feast.core.service; + +import static org.hamcrest.Matchers.equalTo; +import static org.junit.Assert.assertThat; +import static org.mockito.Mockito.when; +import static org.mockito.MockitoAnnotations.initMocks; + +import com.google.protobuf.Timestamp; +import feast.core.dao.FeatureSetRepository; +import feast.core.dao.FeatureStatisticsRepository; +import feast.core.dao.StoreRepository; +import feast.core.model.Project; +import feast.core.model.Store; +import feast.proto.core.CoreServiceProto.GetFeatureStatisticsRequest; +import feast.proto.core.StoreProto; +import feast.proto.core.StoreProto.Store.BigQueryConfig; +import feast.proto.core.StoreProto.Store.StoreType; +import java.io.IOException; +import java.util.Arrays; +import java.util.Optional; +import org.junit.Before; +import org.junit.Rule; +import org.junit.Test; +import org.junit.rules.ExpectedException; +import org.mockito.Mock; +import org.tensorflow.metadata.v0.*; +import org.tensorflow.metadata.v0.FeatureNameStatistics.Type; + +public class StatsServiceTest { + + private StatsService statsService; + @Mock private FeatureStatisticsRepository featureStatisticsRepository; + @Mock private StoreRepository storeRepository; + @Mock private FeatureSetRepository featureSetRepository; + + @Rule public final ExpectedException expectedException = ExpectedException.none(); + + @Before + public void setUp() { + initMocks(this); + statsService = + new StatsService(storeRepository, featureStatisticsRepository, featureSetRepository); + } + + @Test + public void shouldThrowExceptionIfNeitherDatesNorDatasetsProvided() throws IOException { + GetFeatureStatisticsRequest request = GetFeatureStatisticsRequest.newBuilder().build(); + + expectedException.expect(IllegalArgumentException.class); + expectedException.expectMessage( + "Invalid request. Either provide dataset ids to retrieve statistics over, or a start date and end date."); + statsService.getFeatureStatistics(request); + } + + @Test + public void shouldThrowExceptionIfInvalidDatesProvided() throws IOException { + GetFeatureStatisticsRequest request = + GetFeatureStatisticsRequest.newBuilder() + .setStartDate(Timestamp.newBuilder().setSeconds(1)) + .setEndDate(Timestamp.newBuilder().setSeconds(0)) + .build(); + + expectedException.expect(IllegalArgumentException.class); + expectedException.expectMessage( + "Invalid request. Start timestamp 1 is greater than the end timestamp 0"); + statsService.getFeatureStatistics(request); + } + + @Test + public void shouldThrowExceptionIfInvalidStoreProvided() throws IOException { + GetFeatureStatisticsRequest request = + GetFeatureStatisticsRequest.newBuilder() + .setStartDate(Timestamp.newBuilder().setSeconds(0)) + .setEndDate(Timestamp.newBuilder().setSeconds(1)) + .setStore("redis") + .build(); + + when(storeRepository.findById("redis")) + .thenReturn( + Optional.of( + Store.fromProto( + StoreProto.Store.newBuilder() + .setName("redis") + .setType(StoreType.REDIS) + .build()))); + + expectedException.expect(IllegalArgumentException.class); + expectedException.expectMessage( + "Invalid store redis with type REDIS specified. Batch statistics are only supported for BigQuery stores"); + statsService.getFeatureStatistics(request); + } + + @Test + public void shouldThrowExceptionIfFeatureSetNotFound() throws IOException { + GetFeatureStatisticsRequest request = + GetFeatureStatisticsRequest.newBuilder() + .setStartDate(Timestamp.newBuilder().setSeconds(0)) + .setEndDate(Timestamp.newBuilder().setSeconds(1)) + .setStore("bigquery") + .setFeatureSetId("my_feature_set") + .build(); + + StoreProto.Store storeProto = + StoreProto.Store.newBuilder() + .setName("bigquery") + .setType(StoreType.BIGQUERY) + .setBigqueryConfig( + BigQueryConfig.newBuilder().setProjectId("project").setDatasetId("dataset")) + .build(); + when(storeRepository.findById("bigquery")).thenReturn(Optional.of(Store.fromProto(storeProto))); + when(featureSetRepository.findFeatureSetByNameAndProject_Name( + "my_feature_set", Project.DEFAULT_NAME)) + .thenReturn(null); + expectedException.expect(IllegalArgumentException.class); + expectedException.expectMessage("Illegal request. Unable to find feature set my_feature_set"); + statsService.getFeatureStatistics(request); + } + + @Test + public void shouldAggregateNumericStatistics() { + FeatureNameStatistics stat1 = + FeatureNameStatistics.newBuilder() + .setNumStats( + NumericStatistics.newBuilder() + .setMax(20) + .setMin(1) + .setMean(6) + .setNumZeros(0) + .setStdDev(7.90569415) + .setCommonStats( + CommonStatistics.newBuilder() + .setTotNumValues(5) + .setNumNonMissing(5) + .setAvgNumValues(1) + .setMaxNumValues(1) + .setMinNumValues(1) + .setNumMissing(0))) + .setType(Type.INT) + .setPath(Path.newBuilder().addStep("feature").build()) + .build(); + FeatureNameStatistics stat2 = + FeatureNameStatistics.newBuilder() + .setNumStats( + NumericStatistics.newBuilder() + .setMax(10) + .setMin(0) + .setMean(4) + .setNumZeros(1) + .setStdDev(3.807886553) + .setCommonStats( + CommonStatistics.newBuilder() + .setTotNumValues(5) + .setNumNonMissing(5) + .setAvgNumValues(1) + .setMaxNumValues(1) + .setMinNumValues(1) + .setNumMissing(1))) + .setPath(Path.newBuilder().addStep("feature").build()) + .setType(Type.INT) + .build(); + + FeatureNameStatistics expected = + FeatureNameStatistics.newBuilder() + .setNumStats( + NumericStatistics.newBuilder() + .setMax(20) + .setMin(0) + .setMean(5) + .setNumZeros(1) + .setStdDev(5.944184833146219) + .setCommonStats( + CommonStatistics.newBuilder() + .setTotNumValues(10) + .setNumNonMissing(10) + .setAvgNumValues(1) + .setMaxNumValues(1) + .setMinNumValues(1) + .setNumMissing(1))) + .setPath(Path.newBuilder().addStep("feature").build()) + .setType(Type.INT) + .build(); + + assertThat( + statsService.mergeStatistics(Arrays.asList(Arrays.asList(stat1, stat2))), + equalTo(Arrays.asList(expected))); + } + + @Test + public void shouldAggregateCategoricalStatistics() { + FeatureNameStatistics stat1 = + FeatureNameStatistics.newBuilder() + .setStringStats( + StringStatistics.newBuilder() + .setCommonStats( + CommonStatistics.newBuilder() + .setTotNumValues(5) + .setNumNonMissing(5) + .setAvgNumValues(1) + .setMaxNumValues(1) + .setMinNumValues(1) + .setNumMissing(0)) + .setUnique(4) + .setAvgLength(6)) + .setType(Type.STRING) + .setPath(Path.newBuilder().addStep("feature").build()) + .build(); + FeatureNameStatistics stat2 = + FeatureNameStatistics.newBuilder() + .setStringStats( + StringStatistics.newBuilder() + .setCommonStats( + CommonStatistics.newBuilder() + .setTotNumValues(5) + .setNumNonMissing(5) + .setAvgNumValues(1) + .setMaxNumValues(1) + .setMinNumValues(1) + .setNumMissing(1)) + .setUnique(4) + .setAvgLength(4)) + .setType(Type.STRING) + .setPath(Path.newBuilder().addStep("feature").build()) + .build(); + + FeatureNameStatistics expected = + FeatureNameStatistics.newBuilder() + .setStringStats( + StringStatistics.newBuilder() + .setCommonStats( + CommonStatistics.newBuilder() + .setTotNumValues(10) + .setNumNonMissing(10) + .setAvgNumValues(1) + .setMaxNumValues(1) + .setMinNumValues(1) + .setNumMissing(1)) + .setAvgLength(5)) + .setType(Type.STRING) + .setPath(Path.newBuilder().addStep("feature").build()) + .build(); + assertThat( + statsService.mergeStatistics(Arrays.asList(Arrays.asList(stat1, stat2))), + equalTo(Arrays.asList(expected))); + } + + @Test + public void shouldAggregateBytesStatistics() { + FeatureNameStatistics stat1 = + FeatureNameStatistics.newBuilder() + .setBytesStats( + BytesStatistics.newBuilder() + .setCommonStats( + CommonStatistics.newBuilder() + .setTotNumValues(5) + .setNumNonMissing(5) + .setAvgNumValues(1) + .setMaxNumValues(1) + .setMinNumValues(1) + .setNumMissing(0)) + .setUnique(4) + .setAvgNumBytes(6) + .setMaxNumBytes(10) + .setMinNumBytes(0)) + .setType(Type.BYTES) + .setPath(Path.newBuilder().addStep("feature").build()) + .build(); + FeatureNameStatistics stat2 = + FeatureNameStatistics.newBuilder() + .setBytesStats( + BytesStatistics.newBuilder() + .setCommonStats( + CommonStatistics.newBuilder() + .setTotNumValues(5) + .setNumNonMissing(5) + .setAvgNumValues(1) + .setMaxNumValues(1) + .setMinNumValues(1) + .setNumMissing(1)) + .setUnique(4) + .setAvgNumBytes(4) + .setMaxNumBytes(20) + .setMinNumBytes(1)) + .setType(Type.BYTES) + .setPath(Path.newBuilder().addStep("feature").build()) + .build(); + + FeatureNameStatistics expected = + FeatureNameStatistics.newBuilder() + .setBytesStats( + BytesStatistics.newBuilder() + .setCommonStats( + CommonStatistics.newBuilder() + .setTotNumValues(10) + .setNumNonMissing(10) + .setAvgNumValues(1) + .setMaxNumValues(1) + .setMinNumValues(1) + .setNumMissing(1)) + .setAvgNumBytes(5) + .setMaxNumBytes(20) + .setMinNumBytes(0)) + .setType(Type.BYTES) + .setPath(Path.newBuilder().addStep("feature").build()) + .build(); + assertThat( + statsService.mergeStatistics(Arrays.asList(Arrays.asList(stat1, stat2))), + equalTo(Arrays.asList(expected))); + } + + @Test + public void shouldAggregateStructStatistics() { + FeatureNameStatistics stat1 = + FeatureNameStatistics.newBuilder() + .setStructStats( + StructStatistics.newBuilder() + .setCommonStats( + CommonStatistics.newBuilder() + .setTotNumValues(5) + .setNumNonMissing(5) + .setAvgNumValues(1) + .setMaxNumValues(1) + .setMinNumValues(1) + .setNumMissing(0))) + .setType(Type.STRUCT) + .setPath(Path.newBuilder().addStep("feature").build()) + .build(); + FeatureNameStatistics stat2 = + FeatureNameStatistics.newBuilder() + .setStructStats( + StructStatistics.newBuilder() + .setCommonStats( + CommonStatistics.newBuilder() + .setTotNumValues(5) + .setNumNonMissing(5) + .setAvgNumValues(1) + .setMaxNumValues(1) + .setMinNumValues(1) + .setNumMissing(1))) + .setType(Type.STRUCT) + .setPath(Path.newBuilder().addStep("feature").build()) + .build(); + + FeatureNameStatistics expected = + FeatureNameStatistics.newBuilder() + .setStructStats( + StructStatistics.newBuilder() + .setCommonStats( + CommonStatistics.newBuilder() + .setTotNumValues(10) + .setNumNonMissing(10) + .setAvgNumValues(1) + .setMaxNumValues(1) + .setMinNumValues(1) + .setNumMissing(1))) + .setType(Type.STRUCT) + .setPath(Path.newBuilder().addStep("feature").build()) + .build(); + assertThat( + statsService.mergeStatistics(Arrays.asList(Arrays.asList(stat1, stat2))), + equalTo(Arrays.asList(expected))); + } +} diff --git a/protos/feast/core/CoreService.proto b/protos/feast/core/CoreService.proto index 3cd3c756830..05755b701d8 100644 --- a/protos/feast/core/CoreService.proto +++ b/protos/feast/core/CoreService.proto @@ -21,6 +21,8 @@ option go_package = "github.com/feast-dev/feast/sdk/go/protos/feast/core"; option java_outer_classname = "CoreServiceProto"; option java_package = "feast.proto.core"; +import "google/protobuf/timestamp.proto"; +import "tensorflow_metadata/proto/v0/statistics.proto"; import "feast/core/FeatureSet.proto"; import "feast/core/Store.proto"; import "feast/core/FeatureSetReference.proto"; @@ -41,6 +43,11 @@ service CoreService { // sets currently stored in the registry. rpc ListFeatureSets (ListFeatureSetsRequest) returns (ListFeatureSetsResponse); + // Get feature statistics computed over the data in the batch stores. + // + // Returns a dataset containing TFDV statistics mapped to each valid historical store. + rpc GetFeatureStatistics (GetFeatureStatisticsRequest) returns (GetFeatureStatisticsResponse); + // Retrieve store details given a filter. // // Returns all stores matching that filter. If none are found, an empty list will be returned. @@ -270,3 +277,42 @@ message StopIngestionJobRequest { // Request from stopping an ingestion job message StopIngestionJobResponse {} + +message GetFeatureStatisticsRequest { + // Feature set to retrieve the statistics for. A fully qualified feature set + // id in the format of project/feature_set must be provided. + string feature_set_id = 1; + + // Optional filter which filters returned statistics by selected features. These + // features must be present in the data that is being processed. + repeated string features = 2; + + // Optional filter to select store over which the statistics will retrieved. + // Only historical stores are allowed. + string store = 3; + + // Optional start and end dates over which to filter statistical data + // Start date is inclusive, but end date is not. + // Only dates are supported, not times. + // Cannot be used with dataset_ids. + // If this period spans multiple days, unaggregatable statistics will be dropped. + google.protobuf.Timestamp start_date = 4; + google.protobuf.Timestamp end_date = 5; + + // Optional list of ingestion Ids by which to filter data before + // retrieving statistics. + // Cannot be used with the date ranges + // If multiple dataset ids are provided, unaggregatable statistics will be dropped. + repeated string ingestion_ids = 6; + + // Setting this flag to true will force a recalculation of statistics and overwrite results currently in the + // cache, if any. + bool force_refresh = 7; +} + +message GetFeatureStatisticsResponse { + // Contains statistics for the requested data. + // Due to the limitations of TFDV and Facets, only a single dataset can be returned in, + // despite the message being of list type. + tensorflow.metadata.v0.DatasetFeatureStatisticsList dataset_feature_statistics_list = 1; +} diff --git a/protos/tensorflow_metadata/proto/v0/statistics.proto b/protos/tensorflow_metadata/proto/v0/statistics.proto new file mode 100644 index 00000000000..b07f5e5fc29 --- /dev/null +++ b/protos/tensorflow_metadata/proto/v0/statistics.proto @@ -0,0 +1,427 @@ +// Copyright 2017 The TensorFlow Authors. All Rights Reserved. +// +// Licensed under the Apache License, Version 2.0 (the "License"); +// you may not use this file except in compliance with the License. +// You may obtain a copy of the License at +// +// http://www.apache.org/licenses/LICENSE-2.0 +// +// Unless required by applicable law or agreed to in writing, software +// distributed under the License is distributed on an "AS IS" BASIS, +// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +// See the License for the specific language governing permissions and +// limitations under the License. +// ============================================================================= + +// Definitions for aggregated feature statistics for datasets. +// TODO(b/80075690): make a Javascript build rule for this. +// TODO(b/80075691): migrate Facets to use this. +syntax = "proto3"; +option cc_enable_arenas = true; + +package tensorflow.metadata.v0; + +option java_package = "org.tensorflow.metadata.v0"; +option java_multiple_files = true; +option go_package = "github.com/gojek/feast/sdk/go/protos/tensorflow_metadata/proto/v0"; + +import "tensorflow_metadata/proto/v0/path.proto"; + +// Copied from Facets feature_statistics.proto +// Must be kept binary-compatible with the original, until all usages +// are updated to use this version, or we write a proto-to-proto converter. + +// A list of features statistics for different datasets. If you wish to compare +// different datasets using this list, then the DatasetFeatureStatistics +// entries should all contain the same list of features. +message DatasetFeatureStatisticsList { + repeated DatasetFeatureStatistics datasets = 1; +} + +// The feature statistics for a single dataset. +message DatasetFeatureStatistics { + // The name of the dataset. + string name = 1; + // The number of examples in the dataset. + uint64 num_examples = 2; + + // Only valid if the weight feature was specified. + // Treats a missing weighted feature as zero. + double weighted_num_examples = 4; + // The feature statistics for the dataset. + repeated FeatureNameStatistics features = 3; + + // Cross feature statistics for the dataset. + repeated CrossFeatureStatistics cross_features = 5; +} + +message CrossFeatureStatistics { + // The path of feature x. + Path path_x = 1; + // The path of feature y. + Path path_y = 2; + + // Number of occurrences of this feature cross in the data. If any of + // the features in the cross is missing, the example is ignored. + uint64 count = 3; + + oneof cross_stats { + NumericCrossStatistics num_cross_stats = 4; + CategoricalCrossStatistics categorical_cross_stats = 5; + } +} + +message NumericCrossStatistics { + // Pearson product-moment correlation coefficient. + float correlation = 1; + // Standard covariance. E[(X-E[X])*(Y-E[Y])] + float covariance = 2; +} + +message CategoricalCrossStatistics { + LiftStatistics lift = 1; +} + +message LiftStatistics { + // Lift information for each value of path_y. Lift is defined for each pair of + // values (x,y) as P(path_y=y|path_x=x)/P(path_y=y). + repeated LiftSeries lift_series = 1; + // Weighted lift information for each value of path_y. Weighted lift is + // defined for each pair of values (x,y) as P(path_y=y|path_x=x)/P(path_y=y) + // where probabilities are computed over weighted example space. + repeated LiftSeries weighted_lift_series = 2; +} + +// Container for lift information for a specific y-value. +message LiftSeries { + // A bucket for referring to binned numeric features. + message Bucket { + // The low value of the bucket, inclusive. + double low_value = 1; + // The high value of the bucket, exclusive (unless the high_value is + // positive infinity). + double high_value = 2; + } + + // The particular value of path_y corresponding to this LiftSeries. Each + // element in lift_values corresponds to the lift a different x_value and + // this specific y_value. + oneof y_value { + int32 y_int = 1; + string y_string = 2; + Bucket y_bucket = 3; + } + + // The number of examples in which y_value appears. + oneof y_count_value { + uint64 y_count = 4; + double weighted_y_count = 5; + } + + // A container for lift information about a specific value of path_x. + message LiftValue { + oneof x_value { + int32 x_int = 1; + string x_string = 2; + } + // P(path_y=y|path_x=x) / P(path_y=y) for x_value and the enclosing y_value. + // In terms of concrete fields, this number represents: + // (x_and_y_count / x_count) / (y_count / num_examples) + double lift = 3; + // The number of examples in which x_value appears. + oneof x_count_value { + uint64 x_count = 4; + double weighted_x_count = 5; + } + // The number of examples in which x_value appears and y_value appears. + oneof x_and_y_count_value { + uint64 x_and_y_count = 6; + double weighted_x_and_y_count = 7; + } + } + + // The lifts for a each path_x value and this y_value. + repeated LiftValue lift_values = 6; +} + +// The complete set of statistics for a given feature name for a dataset. +message FeatureNameStatistics { + // The types supported by the feature statistics. When aggregating + // tf.Examples, if the bytelist contains a string, it is recommended to encode + // it here as STRING instead of BYTES in order to calculate string-specific + // statistical measures. + enum Type { + INT = 0; + FLOAT = 1; + STRING = 2; + BYTES = 3; + STRUCT = 4; + } + + // One can identify a field either by the name (for simple fields), or by + // a path (for structured fields). Note that: + // name: "foo" + // is equivalent to: + // path: {step:"foo"} + // Note: this oneof must be consistently either name or path across all + // FeatureNameStatistics in one DatasetFeatureStatistics. + oneof field_id { + // The feature name + string name = 1; + + // The path of the feature. + Path path = 8; + } + + // The data type of the feature + Type type = 2; + + // The statistics of the values of the feature. + oneof stats { + NumericStatistics num_stats = 3; + StringStatistics string_stats = 4; + BytesStatistics bytes_stats = 5; + StructStatistics struct_stats = 7; + } + + // Any custom statistics can be stored in this list. + repeated CustomStatistic custom_stats = 6; +} + +// Common weighted statistics for all feature types. Statistics counting number +// of values (i.e., avg_num_values and tot_num_values) include NaNs. +// If the weighted column is missing, then this counts as a weight of 1 +// for that example. +message WeightedCommonStatistics { + // Weighted number of examples not missing. + double num_non_missing = 1; + // Weighted number of examples missing. + // Note that if the weighted column is zero, this does not count + // as missing. + double num_missing = 2; + // average number of values, weighted by the number of examples. + double avg_num_values = 3; + // tot_num_values = avg_num_values * num_non_missing. + // This is calculated directly, so should have less numerical error. + double tot_num_values = 4; +} + +// Stores the name and value of any custom statistic. The value can be a string, +// double, or histogram. +message CustomStatistic { + string name = 1; + oneof val { + double num = 2; + string str = 3; + Histogram histogram = 4; + RankHistogram rank_histogram = 5; + } +} + +// Statistics for a numeric feature in a dataset. +message NumericStatistics { + CommonStatistics common_stats = 1; + // The mean of the values + double mean = 2; + // The standard deviation of the values + double std_dev = 3; + // The number of values that equal 0 + uint64 num_zeros = 4; + // The minimum value + double min = 5; + // The median value + double median = 6; + // The maximum value + double max = 7; + // The histogram(s) of the feature values. + repeated Histogram histograms = 8; + + // Weighted statistics for the feature, if the values have weights. + WeightedNumericStatistics weighted_numeric_stats = 9; +} + +// Statistics for a string feature in a dataset. +message StringStatistics { + CommonStatistics common_stats = 1; + // The number of unique values + uint64 unique = 2; + + message FreqAndValue { + string value = 2; + + // The number of times the value occurs. Stored as a double to be able to + // handle weighted features. + double frequency = 3; + + // Deleted fields. + reserved 1; + } + // A sorted list of the most-frequent values and their frequencies, with + // the most-frequent being first. + repeated FreqAndValue top_values = 3; + + // The average length of the values + float avg_length = 4; + + // The rank histogram for the values of the feature. + // The rank is used to measure of how commonly the value is found in the + // dataset. The most common value would have a rank of 1, with the second-most + // common value having a rank of 2, and so on. + RankHistogram rank_histogram = 5; + + // Weighted statistics for the feature, if the values have weights. + WeightedStringStatistics weighted_string_stats = 6; + + // A vocabulary file, used for vocabularies too large to store in the proto + // itself. Note that the file may be relative to some context-dependent + // directory. E.g. in TFX the feature statistics will live in a PPP and + // vocabulary file names will be relative to this PPP. + string vocabulary_file = 7; +} + +// Statistics for a weighted numeric feature in a dataset. +message WeightedNumericStatistics { + // The weighted mean of the values + double mean = 1; + // The weighted standard deviation of the values + double std_dev = 2; + // The weighted median of the values + double median = 3; + + // The histogram(s) of the weighted feature values. + repeated Histogram histograms = 4; +} + +// Statistics for a weighted string feature in a dataset. +message WeightedStringStatistics { + // A sorted list of the most-frequent values and their weighted frequencies, + // with the most-frequent being first. + repeated StringStatistics.FreqAndValue top_values = 1; + + // The rank histogram for the weighted values of the feature. + RankHistogram rank_histogram = 2; +} + +// Statistics for a bytes feature in a dataset. +message BytesStatistics { + CommonStatistics common_stats = 1; + // The number of unique values + uint64 unique = 2; + + // The average number of bytes in a value + float avg_num_bytes = 3; + // The minimum number of bytes in a value + float min_num_bytes = 4; + // The maximum number of bytes in a value + float max_num_bytes = 5; +} + +message StructStatistics { + CommonStatistics common_stats = 1; +} + +// Common statistics for all feature types. Statistics counting number of values +// (i.e., min_num_values, max_num_values, avg_num_values, and tot_num_values) +// include NaNs. +message CommonStatistics { + // The number of examples with at least one value for this feature. + uint64 num_non_missing = 1; + // The number of examples with no values for this feature. + uint64 num_missing = 2; + // The minimum number of values in a single example for this feature. + uint64 min_num_values = 3; + // The maximum number of values in a single example for this feature. + uint64 max_num_values = 4; + // The average number of values in a single example for this feature. + float avg_num_values = 5; + // tot_num_values = avg_num_values * num_non_missing. + // This is calculated directly, so should have less numerical error. + uint64 tot_num_values = 8; + // The quantiles histogram for the number of values in this feature. + Histogram num_values_histogram = 6; + WeightedCommonStatistics weighted_common_stats = 7; + // The histogram for the number of features in the feature list (only set if + // this feature is a non-context feature from a tf.SequenceExample). + // This is different from num_values_histogram, as num_values_histogram tracks + // the count of all values for a feature in an example, whereas this tracks + // the length of the feature list for this feature in an example (where each + // feature list can contain multiple values). + Histogram feature_list_length_histogram = 9; +} + +// The data used to create a histogram of a numeric feature for a dataset. +message Histogram { + // Each bucket defines its low and high values along with its count. The + // low and high values must be a real number or positive or negative + // infinity. They cannot be NaN or undefined. Counts of those special values + // can be found in the numNaN and numUndefined fields. + message Bucket { + // The low value of the bucket, inclusive. + double low_value = 1; + // The high value of the bucket, exclusive (unless the highValue is + // positive infinity). + double high_value = 2; + + // The number of items in the bucket. Stored as a double to be able to + // handle weighted histograms. + double sample_count = 4; + + // Deleted fields. + reserved 3; + } + + // The number of NaN values in the dataset. + uint64 num_nan = 1; + // The number of undefined values in the dataset. + uint64 num_undefined = 2; + + // A list of buckets in the histogram, sorted from lowest bucket to highest + // bucket. + repeated Bucket buckets = 3; + + // The type of the histogram. A standard histogram has equal-width buckets. + // The quantiles type is used for when the histogram message is used to store + // quantile information (by using equal-count buckets with variable widths). + enum HistogramType { + STANDARD = 0; + QUANTILES = 1; + } + + // The type of the histogram. + HistogramType type = 4; + + // An optional descriptive name of the histogram, to be used for labeling. + string name = 5; +} + +// The data used to create a rank histogram of a non-numeric feature of a +// dataset. The rank of a value in a feature can be used as a measure of how +// commonly the value is found in the entire dataset. With bucket sizes of one, +// this becomes a distribution function of all feature values. +message RankHistogram { + // Each bucket defines its start and end ranks along with its count. + message Bucket { + // The low rank of the bucket, inclusive. + uint64 low_rank = 1; + // The high rank of the bucket, exclusive. + uint64 high_rank = 2; + + // The label for the bucket. Can be used to list or summarize the values in + // this rank bucket. + string label = 4; + + // The number of items in the bucket. Stored as a double to be able to + // handle weighted histograms. + double sample_count = 5; + + // Deleted fields. + reserved 3; + } + + // A list of buckets in the histogram, sorted from lowest-ranked bucket to + // highest-ranked bucket. + repeated Bucket buckets = 1; + + // An optional descriptive name of the histogram, to be used for labeling. + string name = 2; +} \ No newline at end of file diff --git a/sdk/python/feast/cli.py b/sdk/python/feast/cli.py index 6d1df292764..4db41f92f18 100644 --- a/sdk/python/feast/cli.py +++ b/sdk/python/feast/cli.py @@ -18,8 +18,8 @@ import click import pkg_resources -import yaml +import yaml from feast.client import Client from feast.config import Config from feast.core.IngestionJob_pb2 import IngestionJobStatus diff --git a/sdk/python/feast/client.py b/sdk/python/feast/client.py index f10204c59d9..316573ce93d 100644 --- a/sdk/python/feast/client.py +++ b/sdk/python/feast/client.py @@ -13,6 +13,7 @@ # limitations under the License. +import datetime import logging import os import shutil @@ -25,9 +26,10 @@ import grpc import pandas as pd +from google.protobuf.timestamp_pb2 import Timestamp + import pyarrow as pa import pyarrow.parquet as pq - from feast.config import Config from feast.constants import ( CONFIG_CORE_SECURE_KEY, @@ -48,6 +50,7 @@ GetFeastCoreVersionRequest, GetFeatureSetRequest, GetFeatureSetResponse, + GetFeatureStatisticsRequest, ListFeatureSetsRequest, ListFeatureSetsResponse, ListIngestionJobsRequest, @@ -76,6 +79,7 @@ GetOnlineFeaturesResponse, ) from feast.serving.ServingService_pb2_grpc import ServingServiceStub +from tensorflow_metadata.proto.v0 import statistics_pb2 _logger = logging.getLogger(__name__) @@ -859,6 +863,80 @@ def ingest( return ingestion_id + def get_statistics( + self, + feature_set_id: str, + store: str, + features: List[str] = [], + ingestion_ids: Optional[List[str]] = None, + start_date: Optional[datetime.datetime] = None, + end_date: Optional[datetime.datetime] = None, + force_refresh: bool = False, + default_project: Optional[str] = None, + ) -> statistics_pb2.DatasetFeatureStatisticsList: + """ + Retrieves the feature featureStatistics computed over the data in the batch + stores. + + Args: + feature_set_id: Fully qualified feature set id in the format + project/feature_set to retrieve batch featureStatistics for. If project + is not provided, the default ("default") will be used. + store: Name of the store to retrieve feature featureStatistics over. This + store must be a historical store. + features: Optional list of feature names to filter from the results. + ingestion_ids: Optional list of dataset Ids by which to filter data + before retrieving featureStatistics. Cannot be used with start_date + and end_date. + If multiple dataset ids are provided, unaggregatable featureStatistics + will be dropped. + start_date: Optional start date over which to filter statistical data. + Data from this date will be included. + Cannot be used with dataset_ids. If the provided period spans + multiple days, unaggregatable featureStatistics will be dropped. + end_date: Optional end date over which to filter statistical data. + Data from this data will not be included. + Cannot be used with dataset_ids. If the provided period spans + multiple days, unaggregatable featureStatistics will be dropped. + force_refresh: Setting this flag to true will force a recalculation + of featureStatistics and overwrite results currently in the cache, if any. + default_project: Manual override for default project. + + Returns: + Returns a tensorflow DatasetFeatureStatisticsList containing TFDV featureStatistics. + """ + + self._connect_core() + if ingestion_ids is not None and ( + start_date is not None or end_date is not None + ): + raise ValueError( + "Only one of dataset_id or [start_date, end_date] can be provided." + ) + + if default_project != "" and "/" not in feature_set_id: + feature_set_id = f"{default_project}/{feature_set_id}" + + request = GetFeatureStatisticsRequest( + feature_set_id=feature_set_id, + features=features, + store=store, + force_refresh=force_refresh, + ) + if ingestion_ids is not None: + request.ingestion_ids.extend(ingestion_ids) + else: + if start_date is not None: + request.start_date.CopyFrom( + Timestamp(seconds=int(start_date.timestamp())) + ) + if end_date is not None: + request.end_date.CopyFrom(Timestamp(seconds=int(end_date.timestamp()))) + + return self._core_service_stub.GetFeatureStatistics( + request + ).dataset_feature_statistics_list + def _build_feature_references( feature_ref_strs: List[str], project: Optional[str] = None diff --git a/sdk/python/feast/feature_set.py b/sdk/python/feast/feature_set.py index a181c9378bd..c94f32130e4 100644 --- a/sdk/python/feast/feature_set.py +++ b/sdk/python/feast/feature_set.py @@ -23,8 +23,8 @@ from google.protobuf.json_format import MessageToDict, MessageToJson from google.protobuf.message import Message from pandas.api.types import is_datetime64_ns_dtype -from pyarrow.lib import TimestampType +import pyarrow as pa from feast.core.FeatureSet_pb2 import FeatureSet as FeatureSetProto from feast.core.FeatureSet_pb2 import FeatureSetMeta as FeatureSetMetaProto from feast.core.FeatureSet_pb2 import FeatureSetSpec as FeatureSetSpecProto @@ -40,6 +40,7 @@ pa_to_feast_value_type, python_type_to_feast_value_type, ) +from pyarrow.lib import TimestampType from tensorflow_metadata.proto.v0 import schema_pb2 diff --git a/sdk/python/feast/loaders/ingest.py b/sdk/python/feast/loaders/ingest.py index b439dbd3027..5e8e5da4d41 100644 --- a/sdk/python/feast/loaders/ingest.py +++ b/sdk/python/feast/loaders/ingest.py @@ -4,8 +4,8 @@ from typing import Iterable, List import pandas as pd -import pyarrow.parquet as pq +import pyarrow.parquet as pq from feast.constants import DATETIME_COLUMN from feast.feature_set import FeatureSet from feast.type_map import ( diff --git a/sdk/python/feast/type_map.py b/sdk/python/feast/type_map.py index 85def25fcb9..7a483a6c2a0 100644 --- a/sdk/python/feast/type_map.py +++ b/sdk/python/feast/type_map.py @@ -17,10 +17,9 @@ import numpy as np import pandas as pd -import pyarrow as pa from google.protobuf.timestamp_pb2 import Timestamp -from pyarrow.lib import TimestampType +import pyarrow as pa from feast.constants import DATETIME_COLUMN from feast.types import FeatureRow_pb2 as FeatureRowProto from feast.types import Field_pb2 as FieldProto @@ -36,6 +35,7 @@ from feast.types.Value_pb2 import Value as ProtoValue from feast.types.Value_pb2 import ValueType as ProtoValueType from feast.value_type import ValueType +from pyarrow.lib import TimestampType def python_type_to_feast_value_type( diff --git a/sdk/python/setup.py b/sdk/python/setup.py index 69ea44a1871..b0d49ccabcf 100644 --- a/sdk/python/setup.py +++ b/sdk/python/setup.py @@ -42,7 +42,7 @@ "tabulate==0.8.*", "toml==0.10.*", "tqdm==4.*", - "pyarrow>=0.15.1", + "pyarrow<0.16.0,>=0.15.1", "numpy", "google", "confluent_kafka", diff --git a/storage/api/src/main/java/feast/storage/api/statistics/FeatureSetStatistics.java b/storage/api/src/main/java/feast/storage/api/statistics/FeatureSetStatistics.java new file mode 100644 index 00000000000..a328df46b97 --- /dev/null +++ b/storage/api/src/main/java/feast/storage/api/statistics/FeatureSetStatistics.java @@ -0,0 +1,48 @@ +/* + * SPDX-License-Identifier: Apache-2.0 + * Copyright 2018-2020 The Feast Authors + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package feast.storage.api.statistics; + +import com.google.auto.value.AutoValue; +import com.google.common.collect.ImmutableList; +import org.tensorflow.metadata.v0.FeatureNameStatistics; + +/** Feature statistics for a feature set over a bounded set of data. */ +@AutoValue +public abstract class FeatureSetStatistics { + + public abstract long getNumExamples(); + + public abstract ImmutableList getFeatureNameStatistics(); + + public static Builder newBuilder() { + return new AutoValue_FeatureSetStatistics.Builder(); + } + + @AutoValue.Builder + public abstract static class Builder { + public abstract Builder setNumExamples(long numExamples); + + protected abstract ImmutableList.Builder featureNameStatisticsBuilder(); + + public Builder addFeatureNameStatistics(FeatureNameStatistics featureNameStatistics) { + featureNameStatisticsBuilder().add(featureNameStatistics); + return this; + } + + public abstract FeatureSetStatistics build(); + } +} diff --git a/storage/api/src/main/java/feast/storage/api/statistics/StatisticsRetriever.java b/storage/api/src/main/java/feast/storage/api/statistics/StatisticsRetriever.java new file mode 100644 index 00000000000..bdd51de2cba --- /dev/null +++ b/storage/api/src/main/java/feast/storage/api/statistics/StatisticsRetriever.java @@ -0,0 +1,46 @@ +/* + * SPDX-License-Identifier: Apache-2.0 + * Copyright 2018-2020 The Feast Authors + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package feast.storage.api.statistics; + +import com.google.protobuf.Timestamp; +import feast.proto.core.FeatureSetProto.FeatureSetSpec; +import java.util.List; + +public interface StatisticsRetriever { + + /** + * Get feature set statistics for a single feature set, for a single dataset id. + * + * @param featureSetSpec feature set spec of the requested feature set + * @param features subset of features to retrieve. + * @param dataset dataset id to filter the data by + * @return {@link FeatureSetStatistics} containing statistics for the requested features. + */ + FeatureSetStatistics getFeatureStatistics( + FeatureSetSpec featureSetSpec, List features, String dataset); + + /** + * Get feature set statistics for a single feature set, for a single day. + * + * @param featureSetSpec feature set spec of the requested feature set + * @param features subset of features to retrieve. + * @param date date to filter the data by + * @return {@link FeatureSetStatistics} containing statistics for the requested features. + */ + FeatureSetStatistics getFeatureStatistics( + FeatureSetSpec featureSetSpec, List features, Timestamp date); +} diff --git a/storage/connectors/bigquery/src/main/java/feast/storage/connectors/bigquery/statistics/BigQueryStatisticsRetriever.java b/storage/connectors/bigquery/src/main/java/feast/storage/connectors/bigquery/statistics/BigQueryStatisticsRetriever.java new file mode 100644 index 00000000000..ef572a03bcf --- /dev/null +++ b/storage/connectors/bigquery/src/main/java/feast/storage/connectors/bigquery/statistics/BigQueryStatisticsRetriever.java @@ -0,0 +1,154 @@ +/* + * SPDX-License-Identifier: Apache-2.0 + * Copyright 2018-2020 The Feast Authors + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package feast.storage.connectors.bigquery.statistics; + +import com.google.auto.value.AutoValue; +import com.google.cloud.bigquery.*; +import com.google.common.collect.Streams; +import com.google.protobuf.Timestamp; +import feast.proto.core.FeatureSetProto.FeatureSetSpec; +import feast.proto.core.FeatureSetProto.FeatureSpec; +import feast.proto.core.StoreProto.Store.BigQueryConfig; +import feast.storage.api.statistics.FeatureSetStatistics; +import feast.storage.api.statistics.StatisticsRetriever; +import java.io.IOException; +import java.util.List; +import java.util.Map; +import java.util.stream.Collectors; +import org.tensorflow.metadata.v0.FeatureNameStatistics; + +@AutoValue +public abstract class BigQueryStatisticsRetriever implements StatisticsRetriever { + + public abstract String projectId(); + + public abstract String datasetId(); + + public abstract BigQuery bigquery(); + + public static BigQueryStatisticsRetriever create(BigQueryConfig config) { + BigQuery bigquery = BigQueryOptions.getDefaultInstance().getService(); + return newBuilder() + .setBigquery(bigquery) + .setDatasetId(config.getDatasetId()) + .setProjectId(config.getProjectId()) + .build(); + } + + static Builder newBuilder() { + return new AutoValue_BigQueryStatisticsRetriever.Builder(); + } + + @AutoValue.Builder + abstract static class Builder { + + public abstract Builder setProjectId(String projectId); + + public abstract Builder setDatasetId(String datasetId); + + public abstract Builder setBigquery(BigQuery bigquery); + + public abstract BigQueryStatisticsRetriever build(); + } + + @Override + public FeatureSetStatistics getFeatureStatistics( + FeatureSetSpec featureSetSpec, List features, String dataset) { + FeatureSetStatisticsQueryInfo featureSetStatisticsQueryInfo = + new FeatureSetStatisticsQueryInfo( + featureSetSpec.getProject(), featureSetSpec.getName(), dataset); + return getFeatureSetStatistics(featureSetSpec, features, featureSetStatisticsQueryInfo); + } + + @Override + public FeatureSetStatistics getFeatureStatistics( + FeatureSetSpec featureSetSpec, List features, Timestamp date) { + FeatureSetStatisticsQueryInfo featureSetStatisticsQueryInfo = + new FeatureSetStatisticsQueryInfo( + featureSetSpec.getProject(), featureSetSpec.getName(), date); + return getFeatureSetStatistics(featureSetSpec, features, featureSetStatisticsQueryInfo); + } + + private FeatureSetStatistics getFeatureSetStatistics( + FeatureSetSpec featureSetSpec, + List features, + FeatureSetStatisticsQueryInfo featureSetStatisticsQueryInfo) { + List featuresList = featureSetSpec.getFeaturesList(); + + FeatureSetSpec.Builder featureSetSpecBuilder = featureSetSpec.toBuilder().clearFeatures(); + for (FeatureSpec featureSpec : featuresList) { + if (features.contains(featureSpec.getName())) { + featureSetStatisticsQueryInfo.addFeature(featureSpec); + featureSetSpecBuilder.addFeatures(featureSpec); + } + } + featureSetSpec = featureSetSpecBuilder.build(); + + try { + // Generate SQL for and retrieve non-histogram statistics + String getFeatureSetStatsQuery = + StatsQueryTemplater.createGetFeatureSetStatsQuery( + featureSetStatisticsQueryInfo, projectId(), datasetId()); + QueryJobConfiguration queryJobConfiguration = + QueryJobConfiguration.newBuilder(getFeatureSetStatsQuery).build(); + TableResult basicStats = bigquery().query(queryJobConfiguration); + + // Generate SQL for and retrieve histogram statistics + String getFeatureSetHistQuery = + StatsQueryTemplater.createGetFeatureSetHistQuery( + featureSetStatisticsQueryInfo, projectId(), datasetId()); + queryJobConfiguration = QueryJobConfiguration.newBuilder(getFeatureSetHistQuery).build(); + TableResult hist = bigquery().query(queryJobConfiguration); + + // Convert to map of feature_name:row containing the statistics + Map basicStatsValues = getTableResultByFeatureName(basicStats); + Map histValues = getTableResultByFeatureName(hist); + + int totalCountIndex = basicStats.getSchema().getFields().getIndex("total_count"); + String ref = features.get(0); + FeatureSetStatistics.Builder featureSetStatisticsBuilder = + FeatureSetStatistics.newBuilder() + .setNumExamples(basicStatsValues.get(ref).get(totalCountIndex).getLongValue()); + + // Convert BQ rows to FeatureNameStatistics + for (FeatureSpec featureSpec : featureSetSpec.getFeaturesList()) { + FeatureNameStatistics featureNameStatistics = + StatsQueryResult.create() + .withBasicStatsResults( + basicStats.getSchema(), basicStatsValues.get(featureSpec.getName())) + .withHistResults(hist.getSchema(), histValues.get(featureSpec.getName())) + .toFeatureNameStatistics(featureSpec.getValueType()); + featureSetStatisticsBuilder.addFeatureNameStatistics(featureNameStatistics); + } + return featureSetStatisticsBuilder.build(); + } catch (IOException | InterruptedException e) { + throw new RuntimeException( + String.format( + "Unable to retrieve statistics from BigQuery for Feature set %s, features %s", + featureSetSpec.getName(), features), + e); + } + } + + private Map getTableResultByFeatureName(TableResult basicStats) { + return Streams.stream(basicStats.getValues()) + .collect( + Collectors.toMap( + fieldValueList -> fieldValueList.get(0).getStringValue(), + fieldValueList -> fieldValueList)); + } +} diff --git a/storage/connectors/bigquery/src/main/java/feast/storage/connectors/bigquery/statistics/FeatureSetStatisticsQueryInfo.java b/storage/connectors/bigquery/src/main/java/feast/storage/connectors/bigquery/statistics/FeatureSetStatisticsQueryInfo.java new file mode 100644 index 00000000000..b0119f3e587 --- /dev/null +++ b/storage/connectors/bigquery/src/main/java/feast/storage/connectors/bigquery/statistics/FeatureSetStatisticsQueryInfo.java @@ -0,0 +1,102 @@ +/* + * SPDX-License-Identifier: Apache-2.0 + * Copyright 2018-2020 The Feast Authors + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package feast.storage.connectors.bigquery.statistics; + +import com.google.protobuf.Timestamp; +import feast.proto.core.FeatureSetProto.EntitySpec; +import feast.proto.core.FeatureSetProto.FeatureSpec; +import java.util.ArrayList; +import java.util.List; +import org.joda.time.DateTime; +import org.joda.time.DateTimeZone; +import org.joda.time.format.DateTimeFormat; +import org.joda.time.format.DateTimeFormatter; + +/** + * Value class for Feature Sets containing information necessary to template stats-retrieving + * queries. + */ +public class FeatureSetStatisticsQueryInfo { + // Feast project name + private final String project; + + // Feature set name + private final String name; + + // Dataset ID to retrieve statistics over + private String datasetId = ""; + + // Date to retrieve statistics over + private String date = ""; + + // List of entity names in this feature set + private final List entityNames; + + // List of fields to get stats for + private final List features; + + public FeatureSetStatisticsQueryInfo(String project, String name, String datasetId) { + this.project = project; + this.name = name; + this.entityNames = new ArrayList<>(); + this.features = new ArrayList<>(); + this.datasetId = datasetId; + } + + public FeatureSetStatisticsQueryInfo(String project, String name, Timestamp date) { + this.project = project; + this.name = name; + this.entityNames = new ArrayList<>(); + this.features = new ArrayList<>(); + DateTime dateTime = new DateTime(date.getSeconds() * 1000, DateTimeZone.UTC); + DateTimeFormatter fmt = DateTimeFormat.forPattern("yyyy-MM-dd"); + this.date = fmt.print(dateTime); + } + + public void addFeature(FeatureSpec featureSpec) { + this.features.add(FeatureStatisticsQueryInfo.fromProto(featureSpec)); + } + + public void addEntity(EntitySpec entitySpec) { + this.entityNames.add(entitySpec.getName()); + this.features.add(FeatureStatisticsQueryInfo.fromProto(entitySpec)); + } + + public String getProject() { + return project; + } + + public String getName() { + return name; + } + + public String getDatasetId() { + return datasetId; + } + + public String getDate() { + return date; + } + + public List getEntityNames() { + return entityNames; + } + + public List getFeatures() { + return features; + } +} diff --git a/storage/connectors/bigquery/src/main/java/feast/storage/connectors/bigquery/statistics/FeatureStatisticsQueryInfo.java b/storage/connectors/bigquery/src/main/java/feast/storage/connectors/bigquery/statistics/FeatureStatisticsQueryInfo.java new file mode 100644 index 00000000000..8d73fad5dde --- /dev/null +++ b/storage/connectors/bigquery/src/main/java/feast/storage/connectors/bigquery/statistics/FeatureStatisticsQueryInfo.java @@ -0,0 +1,99 @@ +/* + * SPDX-License-Identifier: Apache-2.0 + * Copyright 2018-2020 The Feast Authors + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package feast.storage.connectors.bigquery.statistics; + +import feast.proto.core.FeatureSetProto.EntitySpec; +import feast.proto.core.FeatureSetProto.FeatureSpec; +import feast.proto.types.ValueProto.ValueType.Enum; + +/** + * Value class for Features containing information necessary to template stats-retrieving queries. + */ +public class FeatureStatisticsQueryInfo { + // Name of the field + private final String name; + + // Type of the field + private final String type; + + private FeatureStatisticsQueryInfo(String name, String type) { + this.name = name; + this.type = type; + } + + public static FeatureStatisticsQueryInfo fromProto(FeatureSpec featureSpec) { + Enum valueType = featureSpec.getValueType(); + switch (valueType) { + case FLOAT: + case DOUBLE: + case INT32: + case INT64: + case BOOL: + return new FeatureStatisticsQueryInfo(featureSpec.getName(), "NUMERIC"); + case STRING: + return new FeatureStatisticsQueryInfo(featureSpec.getName(), "CATEGORICAL"); + case BYTES: + return new FeatureStatisticsQueryInfo(featureSpec.getName(), "BYTES"); + case BYTES_LIST: + case BOOL_LIST: + case FLOAT_LIST: + case INT32_LIST: + case INT64_LIST: + case DOUBLE_LIST: + case STRING_LIST: + return new FeatureStatisticsQueryInfo(featureSpec.getName(), "LIST"); + default: + throw new IllegalArgumentException( + String.format("Invalid feature type provided: %s", valueType)); + } + } + + public static FeatureStatisticsQueryInfo fromProto(EntitySpec entitySpec) { + Enum valueType = entitySpec.getValueType(); + switch (valueType) { + case FLOAT: + case DOUBLE: + case INT32: + case INT64: + case BOOL: + return new FeatureStatisticsQueryInfo(entitySpec.getName(), "NUMERIC"); + case STRING: + return new FeatureStatisticsQueryInfo(entitySpec.getName(), "CATEGORICAL"); + case BYTES: + return new FeatureStatisticsQueryInfo(entitySpec.getName(), "BYTES"); + case BYTES_LIST: + case BOOL_LIST: + case FLOAT_LIST: + case INT32_LIST: + case INT64_LIST: + case DOUBLE_LIST: + case STRING_LIST: + return new FeatureStatisticsQueryInfo(entitySpec.getName(), "LIST"); + default: + throw new IllegalArgumentException( + String.format("Invalid entity type provided: %s", valueType)); + } + } + + public String getName() { + return name; + } + + public String getType() { + return type; + } +} diff --git a/storage/connectors/bigquery/src/main/java/feast/storage/connectors/bigquery/statistics/StatsQueryResult.java b/storage/connectors/bigquery/src/main/java/feast/storage/connectors/bigquery/statistics/StatsQueryResult.java new file mode 100644 index 00000000000..f725f2924f0 --- /dev/null +++ b/storage/connectors/bigquery/src/main/java/feast/storage/connectors/bigquery/statistics/StatsQueryResult.java @@ -0,0 +1,332 @@ +/* + * SPDX-License-Identifier: Apache-2.0 + * Copyright 2018-2020 The Feast Authors + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package feast.storage.connectors.bigquery.statistics; + +import com.google.auto.value.AutoValue; +import com.google.cloud.bigquery.*; +import com.google.cloud.bigquery.Schema; +import com.google.common.collect.ComparisonChain; +import com.google.common.collect.Ordering; +import feast.proto.types.ValueProto.ValueType; +import java.util.HashMap; +import java.util.List; +import java.util.Map; +import java.util.stream.Collectors; +import javax.annotation.Nullable; +import org.tensorflow.metadata.v0.*; +import org.tensorflow.metadata.v0.Histogram.Bucket; +import org.tensorflow.metadata.v0.Histogram.HistogramType; +import org.tensorflow.metadata.v0.StringStatistics.FreqAndValue; + +@AutoValue +public abstract class StatsQueryResult { + // Map converting Feast type to TFDV type + private static final Map TFDV_TYPE_MAP = + new HashMap<>(); + + static { + TFDV_TYPE_MAP.put(ValueType.Enum.INT64, FeatureNameStatistics.Type.INT); + TFDV_TYPE_MAP.put(ValueType.Enum.INT32, FeatureNameStatistics.Type.INT); + TFDV_TYPE_MAP.put(ValueType.Enum.BOOL, FeatureNameStatistics.Type.INT); + TFDV_TYPE_MAP.put(ValueType.Enum.FLOAT, FeatureNameStatistics.Type.FLOAT); + TFDV_TYPE_MAP.put(ValueType.Enum.DOUBLE, FeatureNameStatistics.Type.FLOAT); + TFDV_TYPE_MAP.put(ValueType.Enum.STRING, FeatureNameStatistics.Type.STRING); + TFDV_TYPE_MAP.put(ValueType.Enum.BYTES, FeatureNameStatistics.Type.BYTES); + TFDV_TYPE_MAP.put(ValueType.Enum.BYTES_LIST, FeatureNameStatistics.Type.STRUCT); + TFDV_TYPE_MAP.put(ValueType.Enum.STRING_LIST, FeatureNameStatistics.Type.STRUCT); + TFDV_TYPE_MAP.put(ValueType.Enum.INT32_LIST, FeatureNameStatistics.Type.STRUCT); + TFDV_TYPE_MAP.put(ValueType.Enum.INT64_LIST, FeatureNameStatistics.Type.STRUCT); + TFDV_TYPE_MAP.put(ValueType.Enum.BOOL_LIST, FeatureNameStatistics.Type.STRUCT); + TFDV_TYPE_MAP.put(ValueType.Enum.FLOAT_LIST, FeatureNameStatistics.Type.STRUCT); + TFDV_TYPE_MAP.put(ValueType.Enum.DOUBLE_LIST, FeatureNameStatistics.Type.STRUCT); + } + + // Schema of the table returned by the basic stats retrieval query + @Nullable + abstract Schema basicStatsSchema(); + + // Table values returned by the basic stats retrieval query + @Nullable + abstract FieldValueList basicStatsFieldValues(); + + // Schema of the table returned by the histogram retrieval query + @Nullable + abstract Schema histSchema(); + + // Table values returned by the histogram retrieval query + @Nullable + abstract FieldValueList histFieldValues(); + + public static StatsQueryResult create() { + return StatsQueryResult.newBuilder().build(); + } + + private static StatsQueryResult.Builder newBuilder() { + return new AutoValue_StatsQueryResult.Builder(); + } + + abstract Builder toBuilder(); + + /** + * Add basic stats query results to the StatsQueryResult. + * + * @param basicStatsSchema BigQuery {@link Schema} of the retrieved statistics row for the + * non-histogram statistics. Used to retrieve the column names corresponding to each value in + * the row. + * @param basicStatsFieldValues BigQuery {@link FieldValueList} containing a single row of + * non-histogram statistics retrieved from BigQuery + * @return {@link StatsQueryResult} + */ + public StatsQueryResult withBasicStatsResults( + Schema basicStatsSchema, FieldValueList basicStatsFieldValues) { + return toBuilder() + .setBasicStatsSchema(basicStatsSchema) + .setBasicStatsFieldValues(basicStatsFieldValues) + .build(); + } + + /** + * Add histogram stats query results to the StatsQueryResult. + * + * @param histSchema BigQuery {@link Schema} of the retrieved statistics row for the histogram + * statistics. Used to retrieve the column names corresponding to each value in the row. + * @param histFieldValues BigQuery {@link FieldValueList} containing a single row of histogram + * statistics retrieved from BigQuery + * @return {@link StatsQueryResult} + */ + public StatsQueryResult withHistResults(Schema histSchema, FieldValueList histFieldValues) { + return toBuilder().setHistSchema(histSchema).setHistFieldValues(histFieldValues).build(); + } + + @AutoValue.Builder + abstract static class Builder { + abstract Builder setBasicStatsSchema(Schema basicStatsSchema); + + abstract Builder setBasicStatsFieldValues(FieldValueList basicStatsFieldValues); + + abstract Builder setHistSchema(Schema histSchema); + + abstract Builder setHistFieldValues(FieldValueList histFieldValues); + + public abstract StatsQueryResult build(); + } + + /** + * Convert BQ-retrieved statistics to the corresponding TFDV {@link FeatureNameStatistics} + * specific to the feature type. + * + * @param valueType {@link ValueType.Enum} denoting the value type of the feature + * @return {@link FeatureNameStatistics} + */ + public FeatureNameStatistics toFeatureNameStatistics(ValueType.Enum valueType) { + Map valuesMap = new HashMap<>(); + + // Convert the table values to a map of field name : table value for easy retrieval + FieldList basicStatsfields = basicStatsSchema().getFields(); + for (int i = 0; i < basicStatsSchema().getFields().size(); i++) { + valuesMap.put(basicStatsfields.get(i).getName(), basicStatsFieldValues().get(i)); + } + + FieldList histFields = histSchema().getFields(); + for (int i = 0; i < histSchema().getFields().size(); i++) { + valuesMap.put(histFields.get(i).getName(), histFieldValues().get(i)); + } + + FeatureNameStatistics.Builder featureNameStatisticsBuilder = + FeatureNameStatistics.newBuilder() + .setPath(Path.newBuilder().addStep(valuesMap.get("feature_name").getStringValue())) + .setType(TFDV_TYPE_MAP.get(valueType)); + + switch (valueType) { + case FLOAT: + case BOOL: + case DOUBLE: + case INT32: + case INT64: + NumericStatistics numStats = getNumericStatistics(valuesMap); + featureNameStatisticsBuilder.setNumStats(numStats); + break; + case STRING: + StringStatistics stringStats = getStringStatistics(valuesMap); + featureNameStatisticsBuilder.setStringStats(stringStats); + break; + case BYTES: + BytesStatistics bytesStats = getBytesStatistics(valuesMap); + featureNameStatisticsBuilder.setBytesStats(bytesStats); + break; + case BYTES_LIST: + case BOOL_LIST: + case FLOAT_LIST: + case INT32_LIST: + case INT64_LIST: + case DOUBLE_LIST: + case STRING_LIST: + StructStatistics structStats = getStructStatistics(valuesMap); + featureNameStatisticsBuilder.setStructStats(structStats); + break; + default: + throw new IllegalArgumentException( + "Invalid feature type provided. Only statistics for numeric, bytes, string, boolean and list features are supported."); + } + return featureNameStatisticsBuilder.build(); + } + + private BytesStatistics getBytesStatistics(Map valuesMap) { + if (valuesMap.get("total_count").getLongValue() == 0) { + return BytesStatistics.getDefaultInstance(); + } + + return BytesStatistics.newBuilder() + .setCommonStats( + CommonStatistics.newBuilder() + .setNumMissing(valuesMap.get("missing_count").getLongValue()) + .setNumNonMissing(valuesMap.get("feature_count").getLongValue()) + .setMinNumValues(1) + .setMaxNumValues(1) + .setAvgNumValues(1) + .setTotNumValues(valuesMap.get("feature_count").getLongValue())) + .setUnique(valuesMap.get("unique").getLongValue()) + .setMaxNumBytes((float) valuesMap.get("max").getDoubleValue()) + .setMinNumBytes((float) valuesMap.get("min").getDoubleValue()) + .setAvgNumBytes((float) valuesMap.get("mean").getDoubleValue()) + .build(); + } + + private StringStatistics getStringStatistics(Map valuesMap) { + if (valuesMap.get("total_count").getLongValue() == 0) { + return StringStatistics.getDefaultInstance(); + } + + RankHistogram.Builder rankHistogram = RankHistogram.newBuilder(); + valuesMap + .get("cat_hist") + .getRepeatedValue() + .forEach( + v -> { + FieldValueList recordValue = v.getRecordValue(); + rankHistogram.addBuckets( + RankHistogram.Bucket.newBuilder() + .setLabel(recordValue.get(0).getStringValue()) + .setSampleCount(recordValue.get(1).getLongValue())); + }); + + List topCount = + rankHistogram.getBucketsList().stream() + .sorted( + (a, b) -> + ComparisonChain.start() + .compare( + a.getSampleCount(), b.getSampleCount(), Ordering.natural().reverse()) + .result()) + .limit(5) + .map( + bucket -> + FreqAndValue.newBuilder() + .setValue(bucket.getLabel()) + .setFrequency(bucket.getSampleCount()) + .build()) + .collect(Collectors.toList()); + + return StringStatistics.newBuilder() + .setUnique(valuesMap.get("unique").getLongValue()) + .setAvgLength((long) valuesMap.get("mean").getDoubleValue()) + .setCommonStats( + CommonStatistics.newBuilder() + .setNumMissing(valuesMap.get("missing_count").getLongValue()) + .setNumNonMissing(valuesMap.get("feature_count").getLongValue()) + .setMinNumValues(1) + .setMaxNumValues(1) + .setAvgNumValues(1) + .setTotNumValues(valuesMap.get("feature_count").getLongValue())) + .setRankHistogram(rankHistogram) + .addAllTopValues(topCount) + .build(); + } + + private NumericStatistics getNumericStatistics(Map valuesMap) { + if (valuesMap.get("total_count").getLongValue() == 0) { + return NumericStatistics.getDefaultInstance(); + } + + // Build quantiles + long quantileCount = valuesMap.get("feature_count").getLongValue() / 10; + Histogram.Builder quantilesBuilder = Histogram.newBuilder().setType(HistogramType.QUANTILES); + + List quantilesRaw = valuesMap.get("quantiles").getRepeatedValue(); + for (int i = 0; i < quantilesRaw.size() - 1; i++) { + quantilesBuilder.addBuckets( + Bucket.newBuilder() + .setLowValue(quantilesRaw.get(i).getDoubleValue()) + .setHighValue(quantilesRaw.get(i + 1).getDoubleValue()) + .setSampleCount(quantileCount)); + } + // Build histogram + Histogram.Builder histBuilder = Histogram.newBuilder().setType(HistogramType.STANDARD); + + // Order of histogram records is defined in the query hist_stats.sql:L35 + valuesMap + .get("num_hist") + .getRepeatedValue() + .forEach( + v -> { + FieldValueList recordValue = v.getRecordValue(); + histBuilder.addBuckets( + Bucket.newBuilder() + .setHighValue(recordValue.get(2).getDoubleValue()) + .setLowValue(recordValue.get(1).getDoubleValue()) + .setSampleCount(recordValue.get(0).getLongValue())); + }); + + return NumericStatistics.newBuilder() + .setMax(valuesMap.get("max").getDoubleValue()) + .setMin(valuesMap.get("min").getDoubleValue()) + .setMedian(quantilesRaw.get(5).getDoubleValue()) + .setNumZeros(valuesMap.get("zeroes").getLongValue()) + .setStdDev(valuesMap.get("stdev").getDoubleValue()) + .setMean(valuesMap.get("mean").getDoubleValue()) + .setCommonStats( + CommonStatistics.newBuilder() + .setNumMissing(valuesMap.get("missing_count").getLongValue()) + .setNumNonMissing(valuesMap.get("feature_count").getLongValue()) + .setMinNumValues(1) + .setMaxNumValues(1) + .setAvgNumValues(1) + .setTotNumValues(valuesMap.get("feature_count").getLongValue())) + .addHistograms(histBuilder) + .addHistograms(quantilesBuilder) + .build(); + } + + private StructStatistics getStructStatistics(Map valuesMap) { + if (valuesMap.get("total_count").getLongValue() == 0) { + return StructStatistics.getDefaultInstance(); + } + + return StructStatistics.newBuilder() + .setCommonStats( + CommonStatistics.newBuilder() + .setNumMissing(valuesMap.get("missing_count").getLongValue()) + .setNumNonMissing(valuesMap.get("feature_count").getLongValue()) + .setMinNumValues(valuesMap.get("min").getLongValue()) + .setMaxNumValues(valuesMap.get("max").getLongValue()) + .setAvgNumValues(valuesMap.get("mean").getLongValue()) + .setTotNumValues( + valuesMap.get("feature_count").getLongValue() + * valuesMap.get("mean").getLongValue())) + .build(); + } +} diff --git a/storage/connectors/bigquery/src/main/java/feast/storage/connectors/bigquery/statistics/StatsQueryTemplater.java b/storage/connectors/bigquery/src/main/java/feast/storage/connectors/bigquery/statistics/StatsQueryTemplater.java new file mode 100644 index 00000000000..8103b2cb48a --- /dev/null +++ b/storage/connectors/bigquery/src/main/java/feast/storage/connectors/bigquery/statistics/StatsQueryTemplater.java @@ -0,0 +1,78 @@ +/* + * SPDX-License-Identifier: Apache-2.0 + * Copyright 2018-2019 The Feast Authors + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package feast.storage.connectors.bigquery.statistics; + +import com.mitchellbosecke.pebble.PebbleEngine; +import com.mitchellbosecke.pebble.template.PebbleTemplate; +import java.io.IOException; +import java.io.StringWriter; +import java.io.Writer; +import java.util.HashMap; +import java.util.Map; + +public class StatsQueryTemplater { + + private static final PebbleEngine engine = new PebbleEngine.Builder().build(); + private static final String BASIC_STATS_TEMPLATE_NAME = "templates/basic_stats.sql"; + private static final String HIST_STATS_TEMPLATE_NAME = "templates/hist_stats.sql"; + + /** + * Generate the query for getting basic statistics about a given feature set + * + * @param featureSetInfo Information about the feature set necessary for the query templating + * @param projectId google project ID + * @param datasetId feast bigquery dataset ID + * @return point in time correctness join BQ SQL query + */ + public static String createGetFeatureSetStatsQuery( + FeatureSetStatisticsQueryInfo featureSetInfo, String projectId, String datasetId) + throws IOException { + + PebbleTemplate template = engine.getTemplate(BASIC_STATS_TEMPLATE_NAME); + Map context = new HashMap<>(); + context.put("featureSet", featureSetInfo); + context.put("projectId", projectId); + context.put("datasetId", datasetId); + + Writer writer = new StringWriter(); + template.evaluate(writer, context); + return writer.toString(); + } + + /** + * Generate the query for getting histograms for features in a given feature set + * + * @param featureSetInfo Information about the feature set necessary for the query templating + * @param projectId google project ID + * @param datasetId feast bigquery dataset ID + * @return point in time correctness join BQ SQL query + */ + public static String createGetFeatureSetHistQuery( + FeatureSetStatisticsQueryInfo featureSetInfo, String projectId, String datasetId) + throws IOException { + + PebbleTemplate template = engine.getTemplate(HIST_STATS_TEMPLATE_NAME); + Map context = new HashMap<>(); + context.put("featureSet", featureSetInfo); + context.put("projectId", projectId); + context.put("datasetId", datasetId); + + Writer writer = new StringWriter(); + template.evaluate(writer, context); + return writer.toString(); + } +} diff --git a/storage/connectors/bigquery/src/main/resources/templates/basic_stats.sql b/storage/connectors/bigquery/src/main/resources/templates/basic_stats.sql new file mode 100644 index 00000000000..d987a013772 --- /dev/null +++ b/storage/connectors/bigquery/src/main/resources/templates/basic_stats.sql @@ -0,0 +1,85 @@ +WITH subset AS ( +SELECT * FROM `{{ projectId }}.{{ datasetId }}.{{ featureSet.project }}_{{ featureSet.name }}` +{% if featureSet.datasetId == "" %} +WHERE DATE(event_timestamp) = '{{ featureSet.date }}' +{% else %} +WHERE dataset_id='{{ featureSet.datasetId }}' +{% endif %} +) +{% for feature in featureSet.features %} +SELECT + "{{ feature.name }}" as feature_name, + -- total count + COUNT(*) AS total_count, + -- count + COUNT({{ feature.name }}) as feature_count, + -- missing + COUNT(*) - COUNT({{ feature.name }}) as missing_count, + {% if feature.type equals "NUMERIC" %} + -- mean + AVG({{ feature.name }}) as mean, + -- stdev + STDDEV({{ feature.name }}) as stdev, + -- zeroes + COUNTIF({{ feature.name }} = 0) as zeroes, + -- min + MIN({{ feature.name }}) as min, + -- max + MAX({{ feature.name }}) as max, + -- hist will have to be called separately + -- quantiles + APPROX_QUANTILES(CAST({{ feature.name }} AS FLOAT64), 10) AS quantiles, + -- unique + null as unique + {% elseif feature.type equals "CATEGORICAL" %} + -- mean + AVG(LENGTH({{ feature.name }})) as mean, + -- stdev + null as stdev, + -- zeroes + null as zeroes, + -- min + null as min, + -- max + null as max, + -- quantiles + ARRAY[] AS quantiles, + -- unique + COUNT(DISTINCT({{ feature.name }})) as unique + {% elseif feature.type equals "BYTES" %} + -- mean + AVG(BIT_COUNT({{ feature.name }})) as mean, + -- stdev + null as stdev, + -- zeroes + null as zeroes, + -- min + MIN(BIT_COUNT({{ feature.name }})) as min, + -- max + MAX(BIT_COUNT({{ feature.name }})) as max, + -- hist will have to be called separately + -- quantiles + ARRAY[] AS quantiles, + -- unique + COUNT(DISTINCT({{ feature.name }})) as unique + {% elseif feature.type equals "LIST" %} + -- mean + AVG(ARRAY_LENGTH({{ feature.name }})) as mean, + -- stdev + null as stdev, + -- zeroes + null as zeroes, + -- min + MIN(ARRAY_LENGTH({{ feature.name }})) as min, + -- max + MAX(ARRAY_LENGTH({{ feature.name }})) as max, + -- hist will have to be called separately + -- quantiles + ARRAY[] AS quantiles, + -- unique + null as unique + {% endif %} +FROM subset +{% if loop.last %}{% else %}UNION ALL {% endif %} +{% endfor %} + diff --git a/storage/connectors/bigquery/src/main/resources/templates/hist_stats.sql b/storage/connectors/bigquery/src/main/resources/templates/hist_stats.sql new file mode 100644 index 00000000000..bb835af382d --- /dev/null +++ b/storage/connectors/bigquery/src/main/resources/templates/hist_stats.sql @@ -0,0 +1,41 @@ +WITH subset AS ( +SELECT * FROM `{{ projectId }}.{{ datasetId }}.{{ featureSet.project }}_{{ featureSet.name }}` +{% if featureSet.datasetId == "" %} +WHERE DATE(event_timestamp) = '{{ featureSet.date }}' +{% else %} +WHERE dataset_id='{{ featureSet.datasetId }}' +{% endif %} +) +{% for feature in featureSet.features %} +, {{ feature.name }}_stats AS ( +{% if feature.type == 'NUMERIC' %} + WITH stats AS ( + SELECT min+step*i as min, min+step*(i+1) as max + FROM ( + SELECT MIN({{ feature.name }}) as min, MAX({{ feature.name }}) as max, (MAX({{ feature.name }})-MIN({{ feature.name }}))/10 step, GENERATE_ARRAY(0, 10, 1) i + FROM subset + ), UNNEST(i) i + ), counts as ( + SELECT COUNT(*) as count, min, max, + FROM subset + JOIN stats + ON subset.{{ feature.name }} >= stats.min AND subset.{{ feature.name }}>[] as cat_hist FROM counts +{% elseif feature.type == 'CATEGORICAL' %} + WITH counts AS ( + SELECT {{ feature.name }}, COUNT({{ feature.name }}) AS count FROM subset GROUP BY {{ feature.name }} + ) + SELECT '{{ feature.name }}' as feature, ARRAY>[] as num_hist, ARRAY_AGG(STRUCT({{ feature.name }} as value, count as count)) as cat_hist FROM counts +{% elseif feature.type == 'BYTES' %} + SELECT '{{ feature.name }}' as feature, ARRAY>[] as num_hist, ARRAY>[] as cat_hist +{% elseif feature.type == 'LIST' %} + SELECT '{{ feature.name }}' as feature, ARRAY>[] as num_hist, ARRAY>[] as cat_hist +{% endif %} +) +{% endfor %} +{% for feature in featureSet.features %} +SELECT * FROM {{ feature.name }}_stats +{% if loop.last %}{% else %}UNION ALL {% endif %} +{% endfor %} \ No newline at end of file diff --git a/tests/e2e/requirements.txt b/tests/e2e/requirements.txt index 0ba345a000f..6a032bb7eb4 100644 --- a/tests/e2e/requirements.txt +++ b/tests/e2e/requirements.txt @@ -7,3 +7,6 @@ pytest-benchmark==3.2.2 pytest-mock==1.10.4 pytest-timeout==1.3.3 pytest-ordering==0.6.* +tensorflow-data-validation==0.21.2 +deepdiff==4.3.2 +tensorflow==2.1.0 From 7d207d9861ede3b03527449e1653e179383156eb Mon Sep 17 00:00:00 2001 From: zhilingc Date: Tue, 26 May 2020 15:48:49 +0800 Subject: [PATCH 043/133] Add stats tests, group existing tests --- .../scripts/test-end-to-end-batch-dataflow.sh | 2 +- infra/scripts/test-end-to-end-batch.sh | 4 +- .../scripts/test-end-to-end-redis-cluster.sh | 2 +- infra/scripts/test-end-to-end.sh | 2 +- .../FeatureSetStatisticsQueryInfo.java | 12 +- .../main/resources/templates/basic_stats.sql | 4 +- .../main/resources/templates/hist_stats.sql | 4 +- tests/e2e/{ => bq}/bq-batch-retrieval.py | 0 tests/e2e/bq/feature-stats.py | 305 ++++++++++++++++++ .../all_types_parquet/all_types_parquet.yaml | 0 .../{ => redis}/basic-ingest-redis-serving.py | 0 .../e2e/{ => redis}/basic/cust_trans_fs.yaml | 0 tests/e2e/{ => redis}/basic/data.csv | 0 tests/e2e/{ => redis}/basic/driver_fs.yaml | 0 .../large_volume/cust_trans_large_fs.yaml | 0 15 files changed, 320 insertions(+), 15 deletions(-) rename tests/e2e/{ => bq}/bq-batch-retrieval.py (100%) create mode 100644 tests/e2e/bq/feature-stats.py rename tests/e2e/{ => redis}/all_types_parquet/all_types_parquet.yaml (100%) rename tests/e2e/{ => redis}/basic-ingest-redis-serving.py (100%) rename tests/e2e/{ => redis}/basic/cust_trans_fs.yaml (100%) rename tests/e2e/{ => redis}/basic/data.csv (100%) rename tests/e2e/{ => redis}/basic/driver_fs.yaml (100%) rename tests/e2e/{ => redis}/large_volume/cust_trans_large_fs.yaml (100%) diff --git a/infra/scripts/test-end-to-end-batch-dataflow.sh b/infra/scripts/test-end-to-end-batch-dataflow.sh index e8fb3b95653..4cb7f1998b4 100755 --- a/infra/scripts/test-end-to-end-batch-dataflow.sh +++ b/infra/scripts/test-end-to-end-batch-dataflow.sh @@ -274,7 +274,7 @@ core_ip=$(kubectl get -o jsonpath="{.spec.clusterIP}" service ${HELM_RELEASE_NAM serving_ip=$(kubectl get -o jsonpath="{.spec.clusterIP}" service ${HELM_RELEASE_NAME}-feast-batch-serving) set +e -pytest -v bq-batch-retrieval.py -m dataflow_runner --core_url "$core_ip:6565" --serving_url "$serving_ip:6566" --gcs_path "gs://${TEMP_BUCKET}/" --junitxml=${LOGS_ARTIFACT_PATH}/python-sdk-test-report.xml +pytest -v bq/bq-batch-retrieval.py -m dataflow_runner --core_url "$core_ip:6565" --serving_url "$serving_ip:6566" --gcs_path "gs://${TEMP_BUCKET}/" --junitxml=${LOGS_ARTIFACT_PATH}/python-sdk-test-report.xml TEST_EXIT_CODE=$? if [[ ${TEST_EXIT_CODE} != 0 ]]; then diff --git a/infra/scripts/test-end-to-end-batch.sh b/infra/scripts/test-end-to-end-batch.sh index 988c346bde7..c225a8e2991 100755 --- a/infra/scripts/test-end-to-end-batch.sh +++ b/infra/scripts/test-end-to-end-batch.sh @@ -120,7 +120,7 @@ ORIGINAL_DIR=$(pwd) cd tests/e2e set +e -pytest bq-batch-retrieval.py -m ${PYTEST_MARK} --gcs_path "gs://${TEMP_BUCKET}/" --junitxml=${LOGS_ARTIFACT_PATH}/python-sdk-test-report.xml +pytest bq/* -m ${PYTEST_MARK} --gcs_path "gs://${TEMP_BUCKET}/" --junitxml=${LOGS_ARTIFACT_PATH}/python-sdk-test-report.xml TEST_EXIT_CODE=$? if [[ ${TEST_EXIT_CODE} != 0 ]]; then @@ -137,4 +137,4 @@ cd ${ORIGINAL_DIR} print_banner "Cleaning up" bq rm -r -f ${GOOGLE_CLOUD_PROJECT}:${DATASET_NAME} -exit ${TEST_EXIT_CODE} \ No newline at end of file +exit ${TEST_EXIT_CODE} diff --git a/infra/scripts/test-end-to-end-redis-cluster.sh b/infra/scripts/test-end-to-end-redis-cluster.sh index a11aebf0197..ed6cd42bc05 100755 --- a/infra/scripts/test-end-to-end-redis-cluster.sh +++ b/infra/scripts/test-end-to-end-redis-cluster.sh @@ -89,7 +89,7 @@ ORIGINAL_DIR=$(pwd) cd tests/e2e set +e -pytest basic-ingest-redis-serving.py --junitxml=${LOGS_ARTIFACT_PATH}/python-sdk-test-report.xml +pytest redis/* --junitxml=${LOGS_ARTIFACT_PATH}/python-sdk-test-report.xml TEST_EXIT_CODE=$? if [[ ${TEST_EXIT_CODE} != 0 ]]; then diff --git a/infra/scripts/test-end-to-end.sh b/infra/scripts/test-end-to-end.sh index c90059b7a05..c37ad20eb48 100755 --- a/infra/scripts/test-end-to-end.sh +++ b/infra/scripts/test-end-to-end.sh @@ -53,7 +53,7 @@ ORIGINAL_DIR=$(pwd) cd tests/e2e set +e -pytest basic-ingest-redis-serving.py --junitxml=${LOGS_ARTIFACT_PATH}/python-sdk-test-report.xml +pytest redis/* --junitxml=${LOGS_ARTIFACT_PATH}/python-sdk-test-report.xml TEST_EXIT_CODE=$? if [[ ${TEST_EXIT_CODE} != 0 ]]; then diff --git a/storage/connectors/bigquery/src/main/java/feast/storage/connectors/bigquery/statistics/FeatureSetStatisticsQueryInfo.java b/storage/connectors/bigquery/src/main/java/feast/storage/connectors/bigquery/statistics/FeatureSetStatisticsQueryInfo.java index b0119f3e587..7c9665d92bb 100644 --- a/storage/connectors/bigquery/src/main/java/feast/storage/connectors/bigquery/statistics/FeatureSetStatisticsQueryInfo.java +++ b/storage/connectors/bigquery/src/main/java/feast/storage/connectors/bigquery/statistics/FeatureSetStatisticsQueryInfo.java @@ -37,8 +37,8 @@ public class FeatureSetStatisticsQueryInfo { // Feature set name private final String name; - // Dataset ID to retrieve statistics over - private String datasetId = ""; + // Ingestion ID to retrieve statistics over + private String ingestionId = ""; // Date to retrieve statistics over private String date = ""; @@ -49,12 +49,12 @@ public class FeatureSetStatisticsQueryInfo { // List of fields to get stats for private final List features; - public FeatureSetStatisticsQueryInfo(String project, String name, String datasetId) { + public FeatureSetStatisticsQueryInfo(String project, String name, String ingestionId) { this.project = project; this.name = name; this.entityNames = new ArrayList<>(); this.features = new ArrayList<>(); - this.datasetId = datasetId; + this.ingestionId = ingestionId; } public FeatureSetStatisticsQueryInfo(String project, String name, Timestamp date) { @@ -84,8 +84,8 @@ public String getName() { return name; } - public String getDatasetId() { - return datasetId; + public String getIngestionId() { + return ingestionId; } public String getDate() { diff --git a/storage/connectors/bigquery/src/main/resources/templates/basic_stats.sql b/storage/connectors/bigquery/src/main/resources/templates/basic_stats.sql index d987a013772..dfc931720bb 100644 --- a/storage/connectors/bigquery/src/main/resources/templates/basic_stats.sql +++ b/storage/connectors/bigquery/src/main/resources/templates/basic_stats.sql @@ -1,9 +1,9 @@ WITH subset AS ( SELECT * FROM `{{ projectId }}.{{ datasetId }}.{{ featureSet.project }}_{{ featureSet.name }}` -{% if featureSet.datasetId == "" %} +{% if featureSet.ingestionId == "" %} WHERE DATE(event_timestamp) = '{{ featureSet.date }}' {% else %} -WHERE dataset_id='{{ featureSet.datasetId }}' +WHERE ingestion_id='{{ featureSet.ingestionId }}' {% endif %} ) {% for feature in featureSet.features %} diff --git a/storage/connectors/bigquery/src/main/resources/templates/hist_stats.sql b/storage/connectors/bigquery/src/main/resources/templates/hist_stats.sql index bb835af382d..1f0a7605faa 100644 --- a/storage/connectors/bigquery/src/main/resources/templates/hist_stats.sql +++ b/storage/connectors/bigquery/src/main/resources/templates/hist_stats.sql @@ -1,9 +1,9 @@ WITH subset AS ( SELECT * FROM `{{ projectId }}.{{ datasetId }}.{{ featureSet.project }}_{{ featureSet.name }}` -{% if featureSet.datasetId == "" %} +{% if featureSet.ingestionId == "" %} WHERE DATE(event_timestamp) = '{{ featureSet.date }}' {% else %} -WHERE dataset_id='{{ featureSet.datasetId }}' +WHERE ingestion_id='{{ featureSet.ingestionId }}' {% endif %} ) {% for feature in featureSet.features %} diff --git a/tests/e2e/bq-batch-retrieval.py b/tests/e2e/bq/bq-batch-retrieval.py similarity index 100% rename from tests/e2e/bq-batch-retrieval.py rename to tests/e2e/bq/bq-batch-retrieval.py diff --git a/tests/e2e/bq/feature-stats.py b/tests/e2e/bq/feature-stats.py new file mode 100644 index 00000000000..d2300701550 --- /dev/null +++ b/tests/e2e/bq/feature-stats.py @@ -0,0 +1,305 @@ +import pandas as pd +import pytest +import pytz +import uuid +import time +import os +from datetime import datetime, timedelta + +from feast.client import Client +from feast.entity import Entity +from feast.feature import Feature +from feast.feature_set import FeatureSet +from feast.type_map import ValueType +from google.protobuf.duration_pb2 import Duration +import tensorflow_data_validation as tfdv +from deepdiff import DeepDiff +from google.protobuf.json_format import MessageToDict + + +pd.set_option("display.max_columns", None) + +PROJECT_NAME = "batch_" + uuid.uuid4().hex.upper()[0:6] +STORE_NAME = "historical" +os.environ['CUDA_VISIBLE_DEVICES'] = "0" + + +@pytest.fixture(scope="module") +def core_url(pytestconfig): + return pytestconfig.getoption("core_url") + + +@pytest.fixture(scope="module") +def serving_url(pytestconfig): + return pytestconfig.getoption("serving_url") + + +@pytest.fixture(scope="module") +def allow_dirty(pytestconfig): + return True if pytestconfig.getoption("allow_dirty").lower() == "true" else False + + +@pytest.fixture(scope="module") +def gcs_path(pytestconfig): + return pytestconfig.getoption("gcs_path") + + +@pytest.fixture(scope="module") +def client(core_url, allow_dirty): + # Get client for core and serving + client = Client(core_url=core_url) + client.create_project(PROJECT_NAME) + client.set_project(PROJECT_NAME) + + # Ensure Feast core is active, but empty + if not allow_dirty: + feature_sets = client.list_feature_sets() + if len(feature_sets) > 0: + raise Exception( + "Feast cannot have existing feature sets registered. Exiting tests." + ) + + return client + + +@pytest.fixture(scope="module") +def feature_stats_feature_set(client): + fv_fs = FeatureSet( + "feature_stats", + features=[ + Feature("strings", ValueType.STRING), + Feature("ints", ValueType.INT64), + Feature("floats", ValueType.FLOAT), + ], + entities=[Entity("entity_id", ValueType.INT64)], + max_age=Duration(seconds=100), + ) + client.apply(fv_fs) + return fv_fs + + +@pytest.fixture(scope="module") +def feature_stats_dataset_basic(client, feature_stats_feature_set): + + N_ROWS = 20 + + time_offset = datetime.utcnow().replace(tzinfo=pytz.utc) + df = pd.DataFrame( + { + "datetime": [time_offset] * N_ROWS, + "entity_id": [i for i in range(N_ROWS)], + "strings": ["a", "b"] * int(N_ROWS / 2), + "ints": [int(i) for i in range(N_ROWS)], + "floats": [10.5 - i for i in range(N_ROWS)], + } + ) + + expected_stats = tfdv.generate_statistics_from_dataframe( + df[["strings", "ints", "floats"]] + ) + clear_unsupported_fields(expected_stats) + + # Since TFDV computes population std dev + for feature in expected_stats.datasets[0].features: + if feature.HasField("num_stats"): + name = feature.path.step[0] + std = combined_df[name].std() + feature.num_stats.std_dev = std + + ingestion_id = client.ingest(feature_stats_feature_set, df) + time.sleep(10) + return { + "df": df, + "id": ingestion_id, + "date": datetime(time_offset.year, time_offset.month, time_offset.day).replace( + tzinfo=pytz.utc + ), + "stats": expected_stats, + } + + +@pytest.fixture(scope="module") +def feature_stats_dataset_agg(client, feature_stats_feature_set): + time_offset = datetime.utcnow().replace(tzinfo=pytz.utc) + start_date = time_offset - timedelta(days=10) + end_date = time_offset - timedelta(days=7) + df1 = pd.DataFrame( + { + "datetime": [start_date] * 5, + "entity_id": [i for i in range(5)], + "strings": ["a", "b", "b", "b", "a"], + "ints": [4, 3, 2, 6, 3], + "floats": [2.1, 5.2, 4.3, 0.6, 0.1], + } + ) + ingestion_id_1 = client.ingest(feature_stats_feature_set, df1) + df2 = pd.DataFrame( + { + "datetime": [start_date + timedelta(days=1)] * 3, + "entity_id": [i for i in range(3)], + "strings": ["a", "b", "c"], + "ints": [2, 6, 7], + "floats": [1.6, 2.4, 2], + } + ) + ingestion_id_2 = client.ingest(feature_stats_feature_set, df2) + + combined_df = pd.concat([df1, df2])[["strings", "ints", "floats"]] + expected_stats = tfdv.generate_statistics_from_dataframe(combined_df) + clear_unsupported_agg_fields(expected_stats) + + # Since TFDV computes population std dev + for feature in expected_stats.datasets[0].features: + if feature.HasField("num_stats"): + name = feature.path.step[0] + std = combined_df[name].std() + feature.num_stats.std_dev = std + + time.sleep(10) + + return { + "ids": [ingestion_id_1, ingestion_id_2], + "start_date": datetime( + start_date.year, start_date.month, start_date.day + ).replace(tzinfo=pytz.utc), + "end_date": datetime(end_date.year, end_date.month, end_date.day).replace( + tzinfo=pytz.utc + ), + "stats": expected_stats, + } + + +def test_feature_stats_retrieval_by_single_dataset(client, feature_stats_dataset_basic): + stats = client.get_statistics( + f"{PROJECT_NAME}/feature_stats", + features=["strings", "ints", "floats"], + store=STORE_NAME, + ingestion_ids=[feature_stats_dataset_basic["id"]], + ) + + assert_stats_equal(feature_stats_dataset_basic["stats"], stats) + + +def test_feature_stats_by_date(client, feature_stats_dataset_basic): + stats = client.get_statistics( + f"{PROJECT_NAME}/feature_stats", + features=["strings", "ints", "floats"], + store=STORE_NAME, + start_date=feature_stats_dataset_basic["date"], + end_date=feature_stats_dataset_basic["date"] + timedelta(days=1), + ) + assert_stats_equal(feature_stats_dataset_basic["stats"], stats) + + +def test_feature_stats_agg_over_datasets(client, feature_stats_dataset_agg): + stats = client.get_statistics( + f"{PROJECT_NAME}/feature_stats", + features=["strings", "ints", "floats"], + store=STORE_NAME, + ingestion_ids=feature_stats_dataset_agg["ids"], + ) + assert_stats_equal(feature_stats_dataset_agg["stats"], stats) + + +def test_feature_stats_agg_over_dates(client, feature_stats_dataset_agg): + stats = client.get_statistics( + f"{PROJECT_NAME}/feature_stats", + features=["strings", "ints", "floats"], + store=STORE_NAME, + start_date=feature_stats_dataset_agg["start_date"], + end_date=feature_stats_dataset_agg["end_date"], + ) + assert_stats_equal(feature_stats_dataset_agg["stats"], stats) + + +def test_feature_stats_force_refresh( + client, feature_stats_dataset_basic, feature_stats_feature_set +): + df = feature_stats_dataset_basic["df"] + + df2 = pd.DataFrame( + { + "datetime": [df.iloc[0].datetime], + "entity_id": [10], + "strings": ["c"], + "ints": [2], + "floats": [1.3], + } + ) + client.ingest(feature_stats_feature_set, df2) + time.sleep(10) + + actual_stats = client.get_statistics( + f"{PROJECT_NAME}/feature_stats", + features=["strings", "ints", "floats"], + store="historical", + start_date=feature_stats_dataset_basic["date"], + end_date=feature_stats_dataset_basic["date"] + timedelta(days=1), + force_refresh=True, + ) + + combined_df = pd.concat([df, df2]) + expected_stats = tfdv.generate_statistics_from_dataframe(combined_df) + + clear_unsupported_fields(expected_stats) + + # Since TFDV computes population std dev + for feature in expected_stats.datasets[0].features: + if feature.HasField("num_stats"): + name = feature.path.step[0] + std = combined_df[name].std() + feature.num_stats.std_dev = std + + assert_stats_equal(expected_stats, actual_stats) + + +def clear_unsupported_fields(datasets): + dataset = datasets.datasets[0] + for feature in dataset.features: + if feature.HasField("num_stats"): + feature.num_stats.common_stats.ClearField("num_values_histogram") + for hist in feature.num_stats.histograms: + sorted_buckets = sorted(hist.buckets, key=lambda k: k["highValue"]) + del hist.buckets[:] + hist.buckets.extend(sorted_buckets) + elif feature.HasField("string_stats"): + feature.string_stats.common_stats.ClearField("num_values_histogram") + for bucket in feature.string_stats.rank_histogram.buckets: + bucket.ClearField("low_rank") + bucket.ClearField("high_rank") + elif feature.HasField("struct_stats"): + feature.string_stats.struct_stats.ClearField("num_values_histogram") + elif feature.HasField("bytes_stats"): + feature.string_stats.bytes_stats.ClearField("num_values_histogram") + + +def clear_unsupported_agg_fields(datasets): + dataset = datasets.datasets[0] + for feature in dataset.features: + if feature.HasField("num_stats"): + feature.num_stats.common_stats.ClearField("num_values_histogram") + feature.num_stats.ClearField("histograms") + feature.num_stats.ClearField("median") + elif feature.HasField("string_stats"): + feature.string_stats.common_stats.ClearField("num_values_histogram") + feature.string_stats.ClearField("rank_histogram") + feature.string_stats.ClearField("top_values") + feature.string_stats.ClearField("unique") + elif feature.HasField("struct_stats"): + feature.struct_stats.ClearField("num_values_histogram") + elif feature.HasField("bytes_stats"): + feature.bytes_stats.ClearField("num_values_histogram") + feature.bytes_stats.ClearField("unique") + + +def assert_stats_equal(left, right): + left_stats = MessageToDict(left)["datasets"][0] + right_stats = MessageToDict(right)["datasets"][0] + assert ( + left_stats["numExamples"] == right_stats["numExamples"] + ), f"Number of examples do not match. Expected {left_stats['numExamples']}, got {right_stats['numExamples']}" + + left_features = sorted(left_stats["features"], key=lambda k: k["path"]["step"][0]) + right_features = sorted(right_stats["features"], key=lambda k: k["path"]["step"][0]) + diff = DeepDiff(left_features, right_features, significant_digits=4) + assert len(diff) == 0, f"Feature statistics do not match: \nwanted: {left_features}\n got: {right_features}" diff --git a/tests/e2e/all_types_parquet/all_types_parquet.yaml b/tests/e2e/redis/all_types_parquet/all_types_parquet.yaml similarity index 100% rename from tests/e2e/all_types_parquet/all_types_parquet.yaml rename to tests/e2e/redis/all_types_parquet/all_types_parquet.yaml diff --git a/tests/e2e/basic-ingest-redis-serving.py b/tests/e2e/redis/basic-ingest-redis-serving.py similarity index 100% rename from tests/e2e/basic-ingest-redis-serving.py rename to tests/e2e/redis/basic-ingest-redis-serving.py diff --git a/tests/e2e/basic/cust_trans_fs.yaml b/tests/e2e/redis/basic/cust_trans_fs.yaml similarity index 100% rename from tests/e2e/basic/cust_trans_fs.yaml rename to tests/e2e/redis/basic/cust_trans_fs.yaml diff --git a/tests/e2e/basic/data.csv b/tests/e2e/redis/basic/data.csv similarity index 100% rename from tests/e2e/basic/data.csv rename to tests/e2e/redis/basic/data.csv diff --git a/tests/e2e/basic/driver_fs.yaml b/tests/e2e/redis/basic/driver_fs.yaml similarity index 100% rename from tests/e2e/basic/driver_fs.yaml rename to tests/e2e/redis/basic/driver_fs.yaml diff --git a/tests/e2e/large_volume/cust_trans_large_fs.yaml b/tests/e2e/redis/large_volume/cust_trans_large_fs.yaml similarity index 100% rename from tests/e2e/large_volume/cust_trans_large_fs.yaml rename to tests/e2e/redis/large_volume/cust_trans_large_fs.yaml From 3f66f0eacbfa8e0995c1d7a61e92df1aee168baf Mon Sep 17 00:00:00 2001 From: zhilingc Date: Tue, 26 May 2020 15:48:56 +0800 Subject: [PATCH 044/133] Add notebook example --- ...atistics with Feast, TFDV and Facets.ipynb | 694 ++++++++++++++++++ 1 file changed, 694 insertions(+) create mode 100644 examples/statistics/Historical Feature Statistics with Feast, TFDV and Facets.ipynb diff --git a/examples/statistics/Historical Feature Statistics with Feast, TFDV and Facets.ipynb b/examples/statistics/Historical Feature Statistics with Feast, TFDV and Facets.ipynb new file mode 100644 index 00000000000..1db737824ff --- /dev/null +++ b/examples/statistics/Historical Feature Statistics with Feast, TFDV and Facets.ipynb @@ -0,0 +1,694 @@ +{ + "cells": [ + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "# Historical Feature Statistics with Feast, TFDV and Facets\n", + "\n", + "This tutorial covers how Feast can be used in conjunction with TFDV and Facets to retrieve statistics about feature datasets. \n", + "\n", + "The notebook showcases how Feast's integration with TFDV allows users to:\n", + "\n", + "1. Define TFX feature schemas and persist these properties in the Feature Store\n", + "2. Validate new data against the defined schema\n", + "3. Validate data already in Feast against the defined schema\n", + "\n", + "**Prerequisites**:\n", + "\n", + "- Feast running with at least 1 BigQuery warehouse store. This example uses a bigquery store with the name `historical`." + ] + }, + { + "cell_type": "code", + "execution_count": 1, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "setting project to statistics...\n" + ] + } + ], + "source": [ + "import pandas as pd\n", + "import pytest\n", + "import pytz\n", + "import uuid\n", + "import time\n", + "from datetime import datetime, timedelta\n", + "\n", + "from feast.client import Client\n", + "from feast.entity import Entity\n", + "from feast.feature import Feature\n", + "from feast.feature_set import FeatureSet\n", + "from feast.type_map import ValueType\n", + "from google.protobuf import json_format\n", + "from google.protobuf.duration_pb2 import Duration\n", + "from tensorflow_metadata.proto.v0 import statistics_pb2\n", + "from tensorflow_metadata.proto.v0 import schema_pb2\n", + "import tensorflow_data_validation as tfdv\n", + "\n", + "PROJECT_NAME = \"statistics\"\n", + "IRIS_DATASET = \"http://archive.ics.uci.edu/ml/machine-learning-databases/iris/iris.data\"\n", + "BIGQUERY_STORE_NAME = \"historical\"\n", + "client = Client(core_url=\"localhost:6565\")\n", + "print(f\"setting project to {PROJECT_NAME}...\")\n", + "try:\n", + " client.create_project(PROJECT_NAME)\n", + "except:\n", + " print(\"project already exists, skipping.\")\n", + "client.set_project(PROJECT_NAME)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "In this example, we are using the iris dataset. More information about this dataset can be found [here](http://archive.ics.uci.edu/ml/datasets/iris)." + ] + }, + { + "cell_type": "code", + "execution_count": 2, + "metadata": {}, + "outputs": [ + { + "data": { + "text/html": [ + "

\n", + "\n", + "\n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + "
sepal_lengthsepal_widthpetal_lengthpetal_widthclassdatetime
05.13.51.40.2Iris-setosa2020-05-25 07:31:28.230582+00:00
14.93.01.40.2Iris-setosa2020-05-25 07:31:28.230582+00:00
24.73.21.30.2Iris-setosa2020-05-25 07:31:28.230582+00:00
34.63.11.50.2Iris-setosa2020-05-25 07:31:28.230582+00:00
45.03.61.40.2Iris-setosa2020-05-25 07:31:28.230582+00:00
\n", + "
" + ], + "text/plain": [ + " sepal_length sepal_width petal_length petal_width class \\\n", + "0 5.1 3.5 1.4 0.2 Iris-setosa \n", + "1 4.9 3.0 1.4 0.2 Iris-setosa \n", + "2 4.7 3.2 1.3 0.2 Iris-setosa \n", + "3 4.6 3.1 1.5 0.2 Iris-setosa \n", + "4 5.0 3.6 1.4 0.2 Iris-setosa \n", + "\n", + " datetime \n", + "0 2020-05-25 07:31:28.230582+00:00 \n", + "1 2020-05-25 07:31:28.230582+00:00 \n", + "2 2020-05-25 07:31:28.230582+00:00 \n", + "3 2020-05-25 07:31:28.230582+00:00 \n", + "4 2020-05-25 07:31:28.230582+00:00 " + ] + }, + "execution_count": 2, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "iris_feature_names = [\"sepal_length\",\"sepal_width\",\"petal_length\",\"petal_width\"]\n", + "df = pd.read_csv(IRIS_DATASET, names=iris_feature_names + [\"class\"])\n", + "\n", + "# Add datetime to satisfy Feast\n", + "current_datetime = datetime.utcnow().replace(tzinfo=pytz.utc)\n", + "df['datetime'] = current_datetime - timedelta(days=1)\n", + "\n", + "df.head()" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## TFDV schema as part of the feature set definition\n", + "\n", + "An integral part of TFDV is the feature [schemas](https://github.com/tensorflow/metadata/blob/master/tensorflow_metadata/proto/v0/schema.proto) that describe the expected properties of the data in a dataset, such as:\n", + "- expected feature presence\n", + "- type\n", + "- expected domains of features\n", + "\n", + "These schemas, which can be [manually defined or generated by TFDV](https://www.tensorflow.org/tfx/data_validation/get_started#inferring_a_schema_over_the_data), can be then used to extend the definition of features within the feature set. As part of the spec, the schema is persisted within Feast, and is used for both in-flight data validation, as well as offline integration with TFDV.\n" + ] + }, + { + "cell_type": "code", + "execution_count": 3, + "metadata": { + "scrolled": true + }, + "outputs": [ + { + "name": "stderr", + "output_type": "stream", + "text": [ + "WARNING:root:Ignoring feature datetime of type datetime64[ns, UTC]\n", + "/Users/zhiling/.pyenv/versions/3.7.2/envs/test-feast/lib/python3.7/site-packages/tensorflow_data_validation/arrow/arrow_util.py:236: FutureWarning: Calling .data on ChunkedArray is provided for compatibility after Column was removed, simply drop this attribute\n", + " types.FeaturePath([column_name]), column.data.chunk(0), weights):\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Entity class(ValueType.STRING) manually updated (replacing an existing field).\n", + "Feature sepal_length (ValueType.DOUBLE) added from dataframe.\n", + "Feature sepal_width (ValueType.DOUBLE) added from dataframe.\n", + "Feature petal_length (ValueType.DOUBLE) added from dataframe.\n", + "Feature petal_width (ValueType.DOUBLE) added from dataframe.\n", + "\n", + "{\n", + " \"spec\": {\n", + " \"name\": \"iris\",\n", + " \"entities\": [\n", + " {\n", + " \"name\": \"class\",\n", + " \"valueType\": \"STRING\"\n", + " }\n", + " ],\n", + " \"features\": [\n", + " {\n", + " \"name\": \"sepal_length\",\n", + " \"valueType\": \"DOUBLE\",\n", + " \"presence\": {\n", + " \"minFraction\": 1.0,\n", + " \"minCount\": \"1\"\n", + " },\n", + " \"shape\": {\n", + " \"dim\": [\n", + " {\n", + " \"size\": \"1\"\n", + " }\n", + " ]\n", + " }\n", + " },\n", + " {\n", + " \"name\": \"sepal_width\",\n", + " \"valueType\": \"DOUBLE\",\n", + " \"presence\": {\n", + " \"minFraction\": 1.0,\n", + " \"minCount\": \"1\"\n", + " },\n", + " \"shape\": {\n", + " \"dim\": [\n", + " {\n", + " \"size\": \"1\"\n", + " }\n", + " ]\n", + " }\n", + " },\n", + " {\n", + " \"name\": \"petal_length\",\n", + " \"valueType\": \"DOUBLE\",\n", + " \"presence\": {\n", + " \"minFraction\": 1.0,\n", + " \"minCount\": \"1\"\n", + " },\n", + " \"shape\": {\n", + " \"dim\": [\n", + " {\n", + " \"size\": \"1\"\n", + " }\n", + " ]\n", + " }\n", + " },\n", + " {\n", + " \"name\": \"petal_width\",\n", + " \"valueType\": \"DOUBLE\",\n", + " \"presence\": {\n", + " \"minFraction\": 1.0,\n", + " \"minCount\": \"1\"\n", + " },\n", + " \"shape\": {\n", + " \"dim\": [\n", + " {\n", + " \"size\": \"1\"\n", + " }\n", + " ]\n", + " },\n", + " \"floatDomain\": {\n", + " \"min\": 0.0\n", + " }\n", + " }\n", + " ]\n", + " },\n", + " \"meta\": {}\n", + "}\n" + ] + } + ], + "source": [ + "# Infer a schema over the iris dataset. These values can be tweaked as necessary.\n", + "stats = tfdv.generate_statistics_from_dataframe(df)\n", + "schema = tfdv.infer_schema(statistics=stats)\n", + "width_domain = schema_pb2.FloatDomain(min=0)\n", + "tfdv.set_domain(schema, 'petal_width', width_domain)\n", + "\n", + "# Create a new FeatureSet or retrieve an existing FeatureSet in Feast\n", + "feature_set = FeatureSet(name=\"iris\")\n", + "feature_set.infer_fields_from_df(df[['datetime'] + iris_feature_names], \n", + " entities=[Entity(name=\"class\", dtype=ValueType.STRING)])\n", + "\n", + "# Update the entities and features with constraints defined in the schema\n", + "feature_set.import_tfx_schema(schema)\n", + "print(feature_set)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Computing statistics over an ingested dataset\n", + "\n", + "Feast is able to compute statistics for any data that has been ingested into the system. Statistics can be computed over either discrete datasets using *dataset_ids* or periods of time using a specified time range.\n", + "\n", + "These statistics are computed at a historical store (caveat: only BQ is supported at the moment). The feature statistics returned in the form of TFX's `DatasetFeatureStatisticsList`, which can then be directly fed back into TFDV methods to either visualise the data statistics, or validate the dataset." + ] + }, + { + "cell_type": "code", + "execution_count": 4, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Feature set created: \"iris\"\n", + "Waiting for feature set to be ready for ingestion...\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "100%|██████████| 150/150 [00:01<00:00, 122.33rows/s]" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Ingestion complete!\n", + "\n", + "Ingestion statistics:\n", + "Success: 150/150\n", + "Removing temporary file(s)...\n", + "\n", + "ingestion id: 73ed84b1-1218-3702-b4c6-673503233264\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "\n" + ] + } + ], + "source": [ + "# Apply the featureset\n", + "client.apply(feature_set)\n", + "\n", + "# When a dataset is ingested into Feast, a unique ingestion id referencing the ingested dataset is returned. \n", + "ingestion_id = client.ingest(feature_set, df)\n", + "print(\"\\ningestion id: \" + ingestion_id)" + ] + }, + { + "cell_type": "code", + "execution_count": 6, + "metadata": {}, + "outputs": [], + "source": [ + "# Get statistics from Feast for the ingested dataset.\n", + "# The statistics are calculated over the data in the store specified.\n", + "stats = client.get_statistics(\n", + " feature_set_id=f'{PROJECT_NAME}/iris', \n", + " store=BIGQUERY_STORE_NAME, \n", + " features=iris_feature_names, \n", + " ingestion_ids=[ingestion_id])" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "# Visualising statistics with facets\n", + "\n", + "Since Feast outputs statistics in a format compatible with the TFDV API, the stats object can be directly passed to `tfdv.visualize_statistics()` to visualise, in-line, the output statistics on [Facets](https://pair-code.github.io/facets/), allowing for easy and interactive exploration of the shape and distribution of the data inside Feast." + ] + }, + { + "cell_type": "code", + "execution_count": 7, + "metadata": { + "scrolled": false + }, + "outputs": [ + { + "data": { + "text/html": [ + "\n", + " " + ], + "text/plain": [ + "" + ] + }, + "metadata": {}, + "output_type": "display_data" + } + ], + "source": [ + "tfdv.visualize_statistics(stats)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "# Validating correctness of subsequent datasets \n", + "\n", + "While it is useful to explore dataset statistics using facets, since we have already defined a schema that specifies a dataset's bounds of correctness, we can leverage TFDV's `validate_statistics` to validate if subsequent datasets are problematic or not. " + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "It is possible to validate correctness of a new dataset prior to ingestion by retrieving the schema from the feature set, and comparing computed statistics against that schema. \n", + "\n", + "This can be useful if we want to avoid ingesting problematic data into Feast." + ] + }, + { + "cell_type": "code", + "execution_count": 8, + "metadata": {}, + "outputs": [ + { + "name": "stderr", + "output_type": "stream", + "text": [ + "WARNING:root:Ignoring feature datetime of type datetime64[ns, UTC]\n" + ] + }, + { + "data": { + "text/html": [ + "
\n", + "\n", + "\n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + "
Anomaly short descriptionAnomaly long description
Feature name
'petal_width'Out-of-range valuesUnexpectedly low values: -1<-1(upto six significant digits)
'class'New columnNew column (column in data but not in schema)
\n", + "
" + ], + "text/plain": [ + " Anomaly short description \\\n", + "Feature name \n", + "'petal_width' Out-of-range values \n", + "'class' New column \n", + "\n", + " Anomaly long description \n", + "Feature name \n", + "'petal_width' Unexpectedly low values: -1<-1(upto six significant digits) \n", + "'class' New column (column in data but not in schema) " + ] + }, + "metadata": {}, + "output_type": "display_data" + } + ], + "source": [ + "# Ingest a new dataset with obviously incorrect data\n", + "df_2 = pd.DataFrame(\n", + " {\n", + " \"datetime\": current_datetime,\n", + " \"class\": [\"Iris-setosa\", \"Iris-virginica\", \"Iris-nonsensica\"],\n", + " \"sepal_length\": [4.3, 6.9, 12],\n", + " \"sepal_width\": [3.0, 2.8, 1.1],\n", + " \"petal_length\": [1.2, 4.9, 2.2],\n", + " \"petal_width\": [0.1, 1.8, -1.0]\n", + " }\n", + ")\n", + "\n", + "# Validate correctness\n", + "stats_2 = tfdv.generate_statistics_from_dataframe(df_2)\n", + "anomalies = tfdv.validate_statistics(statistics=stats_2, schema=feature_set.export_tfx_schema())\n", + "tfdv.display_anomalies(anomalies)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Alternatively, the data can be ingested into Feast, and the statistics computed at the store. This has the benefit of offloading statistics computation for large datasets to Feast." + ] + }, + { + "cell_type": "code", + "execution_count": 9, + "metadata": { + "scrolled": false + }, + "outputs": [ + { + "name": "stderr", + "output_type": "stream", + "text": [ + "\r", + " 0%| | 0/3 [00:00\n", + "\n", + "\n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + "
Anomaly short descriptionAnomaly long description
Feature name
'petal_width'Out-of-range valuesUnexpectedly low values: -1<-1(upto six significant digits)
\n", + "" + ], + "text/plain": [ + " Anomaly short description \\\n", + "Feature name \n", + "'petal_width' Out-of-range values \n", + "\n", + " Anomaly long description \n", + "Feature name \n", + "'petal_width' Unexpectedly low values: -1<-1(upto six significant digits) " + ] + }, + "metadata": {}, + "output_type": "display_data" + } + ], + "source": [ + "# Ingest the data into Feast\n", + "ingestion_id_2 = client.ingest(feature_set, df_2)\n", + "time.sleep(10) # Sleep is not necessary if not using DirectRunner\n", + "\n", + "# Compute statistics over the new dataset\n", + "stats_2 = client.get_statistics(\n", + " feature_set_id=f'{PROJECT_NAME}/iris', \n", + " store=BIGQUERY_STORE_NAME, \n", + " features=iris_feature_names, \n", + " ingestion_ids=[ingestion_id_2])\n", + "\n", + "# Detect anomalies in the dataset\n", + "anomalies = tfdv.validate_statistics(statistics=stats_2, schema=feature_set.export_tfx_schema())\n", + "tfdv.display_anomalies(anomalies)" + ] + } + ], + "metadata": { + "kernelspec": { + "display_name": "Python 3", + "language": "python", + "name": "python3" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.7.2" + } + }, + "nbformat": 4, + "nbformat_minor": 2 +} From 729a06a702b45d87dcfc79353c6c90ebeda91fa7 Mon Sep 17 00:00:00 2001 From: zhilingc Date: Tue, 26 May 2020 16:44:01 +0800 Subject: [PATCH 045/133] Add documentation --- docs/assets/statistics-sources.png | Bin 0 -> 165507 bytes docs/user-guide/statistics.md | 160 +++++++++++++++++++++++++++++ 2 files changed, 160 insertions(+) create mode 100644 docs/assets/statistics-sources.png create mode 100644 docs/user-guide/statistics.md diff --git a/docs/assets/statistics-sources.png b/docs/assets/statistics-sources.png new file mode 100644 index 0000000000000000000000000000000000000000..02be233968d3c97e932f8073b66d41b49247782c GIT binary patch literal 165507 zcmeFZbzGC{|33~0GC~1SK#74UptPXW7^Q+rnIMgbfHWJz=mrHDD&44rQX->MYDx@b z3eq`7$3~6(-P3c#^ZCBdpTGZ(f9x@~d)Iwm^?E(yb@^0RTm1Oak~{37IDkWx1qk3x4FL4tb;cu-pFW_gL(AOod$|GH$RoxGDh%jeH8mFVw%tI z5Q*i_;!x0Zg%ETH*>5Y-k6ynfs(rZI*H2clB0VR_-(QmE^F1-~a}-4ies(WTr@!%= zm*agRA5=rxq#f`9>GwT|TJBJvjGvwVWW?q4lM&*6b}47htiaD)`ewpBy5_70TBDnS4x*40z%Hg7VNIp?1FaN+T4Z zXWwwps^%$2_NpbYDXp0chVB`%0xxF8B#Aqqn{z|Gx79bo*w8u`1O%a$+;7aJ$IjiUoExn1)I zj;?UU)2GQd`u*?Mb6UFF9NftP_E)!n4vLe%5tkH`5dYma@KXr+Q+Zt*cS}3{%Qp6w z4lv*zP)TW7X~@qX{6AkE-13i~-aq)MjLfA!fBDBZfBh07PVT`UJ^D3VKR*S=3rY_W z|2_6ldZcCUdkP9=ifflu?zmIV50UDQe^1$8;TQVI@OeDI>Hcx*X78uM))4}arNd9M zzDfv{eARB@lequTCn-TwfcRPS^@_q(!KVz1%x=$bO46SeD))cQ+xq$=DZlq?XUfL~ z&zZJ`+OpiGTrbp+v2oiwtCaUQYUWmF0->TIlj(j`^g{phrJk9c<5E-ENQa&>CCzc& z|MkMlz!7iA^1r_$UoS+>*q(p*U)l$5e4Og=)V)Lh(m2onJ>?--uK&M1pdVGRpMS;o z|Mg(MKJ!e;4x{>01gMU**gc<evqYl6v6s)=&2N3VV4f76z`%*&>tcYoZ7 zy?Jn7|5N@oo*P-Nb3K{9+hx8tYgb=lb-75k5D{;K6kJ_yQZj6<3mLj>IQW_-X7ZvW!Sv?#oceOy*j2+8Aug) zR++aJhR3XK%@yLeDiFk}!G5m~trH%9ci6v{o+EzaZu`vN9{>+(%}UvD&0tYG6(VQs zySp~Ciq+}#ugG^Fcu0Hk9mjm<)%9~j>5S|3&$${1Y5P0)nddUYpPOyo&IcMh^cMaN zx`Kd5)!5AY`%j^C z@Gy6f@vk}Ncnq*qHG5=L%>kI@=2x~Ic4xlRVw=i+oI`|iZp7?MmC0W}unqz4Ds|jA z0jT&{ebIf=P%2s{4qhG^Aq=3Xb#_~mzUKi z6zKi+O(LH}-bn`MlLt@of8X6PuCK42_3R=DA570b-$`Eqt~eJVcKP6L|9qD|3v~GI zNMNk#KR-W6*6HVfE40kveE;|E2aAqfLZWbs2etFBS7@_6Y-R06~D zeIp(Bulj@aGvJEb9L8aP4D>(lr~I9Uu|2nzR)MbDebz~^7D$O6;|nc%O%f3~e$MensWlf7Ud-In?LnYtcP(U#puh(!KlP`Vub zL0=G&y|mnL-Dm0%;a2xPTditjh)%TzC6Tl+U(fvgT1874LtMgyzq_pzgq?t z+7bH8UEGw2cD9TsB;sZSp0G)nBUs^jiy)~M9#FS~KILQqlXr0=Df8YR)9!CVy*}Uf zYS?Fc0p&1Mi2`DUw8!eSwD;DkG!UbtAGOCr@R8gK-QnW3@udz!iRjK$MTH|^C`588 zR@AUNODn0n+RA$)GC=D!*-oq5^k?eFK- zr2)Efe*SD*&hL+au56!350+`%UaCt=^&)&({b7O3v>zzNp=rThQGp9-d#l;+a&v`u zNu9ph!g>Ym=JMvU*v@1*sT9|q`*ILLP~BWXRky-Y^(!_J{Vwj;X)uhOO-sbdnVIY~ z3~_0&LW*`8&Fy9+KM^72BBV3Dc(6@%wJ3d8ixamyZvDB7m7i1ilhnc9(W#z`6{|@j zI|5}qf?TH6?#_u9EUz1sJihDTs?D)CXSy#xjqJ>C z0V5_~405;S7S<*zDb2in)qOV3=w+2{Mq8|&3F2(Ne$naB+3M`@69F)T$TW}HcNNT@ z8;c$Oa-T@S>?ZNra=7SfuLgouDnh{6Yo*y{{zuWa_=npE%N>;otSG}pHimN?Pmb{=nPf>&pcxEK^-Z+VZ@de20u9sMGTK|sgD_`4T}s;Ug* zH%jKU4|aGL57m<|-8|c$ZXG-!kbjhDtj)D^YP`4CFPmxq(3 z+^%;#=JLzBOYH|-;%l8JS_Y3W5U~Nt$|<$&OOmCA2ka-B-rB?8xC^ca&C1W2YQX z-UmH-_g@*l@Wm^PT(e4czUrW*Z3{cx0GrCrW#2j{7^Q?yh7qhp_E*s0&NlrIW-5si z=BlYOzSH{$Pja68BwvOeW&Gnw?)lXxi`@iO6qb)a+j*0mt5=B0kQ>TkPQ=p6iPqa( zt92S{K>6%$4c=u)Bl0q5ce#I6AWz$*LM8;Wzmg^=_oiuMkxZ@!`gVJ7O;;j&2~pZ}EAcz&-e2fPULU|qfo=s! zS-+Em(9p%C$#WJv&l(yQw3bP)@LAoUK77@s^~mNqc7-th-e$ zBfYyme>}BPP?qdL{GJr)T!QVXU9y3q{|fKQcp$sG?LnU~98BVtkTN?Le4@y|<02#%X>=YrmQuFqdf^CDbxxN<{6qnEoKds{=p)-cP8B&p>%XV#n-ctKK}sC%`? zzDv)o%*%XoFT!K1I(r2dEOS8%v6Zo&D1TiMd2i~QapYMZ#l9G%V0C<`Bi-dV*^&^E zB-gHzqjMG>m7_EE+|ChFOl_H-snZ47Pe`?Y?Kc06vu0*c)Owj=?(fm~0HzE;X(e6C z^jxZuA&2pe+Z{}oP2gXzATTZ(TDt0C9dNfnXZ8LHnAOqQM<@la!0Nco(UvX*J|zykD%Gib6hrqRVSR6VNrzL$p*?MdzhGpuMTUSZ zjm@?+=^=ii1-}LED6R-3M>$!+9VkEoh)h-8oe(u3+ENp8@8!47{rF(6wwSy3u}$m- zIYn>45d!xR(-vf#^MthgWvEtU67J$0Z6$rL$FRoo4${hHiXG9a+Bfp)lZ;Q z#Jat;!k>wCOfLV9j_g<>;N$&&&9zCo4p;9T-T$@_|TaOo3*h}a273(JFlp5B}zd8FKdFb zzF9lg48i%=doA>rAh;o{(O}Q{BGM;#(GCGp`&UZw&)^5<9ll;-c!2i0(kb``$#|s7 zJ^n6=O$kV(RfaxsjnXFA2eI`Ft-^)#N81yu#+#nH_O?L0t_lxGTC_|qC&iXBu!(%e z#>I7lhOsqG^L@qG6M9B34CjxvVjGf4pd;MJt@S9W2vVI8DK}(;Wb^iIZMn0(yZx1z zOUuvcBGU?e)^cZn%+k4*R_W6&iJt7mA>6e#ga_?jUy(!8h<>||CZ~3EFU*`<>QVb= zzMZYLIilp1umgz23pjs@2Vhh;{fzSTCCk?6)wM!j#jhV@VsSv|)wX%I?|IEkeS;|o zW^8O+Q57@om~d}`5$A{82yzwKPqZz#20!%4b1Ff?oC@q~W9GB*Q}M* zKSPRehF3LoyluV`cXe>Se?(rk#iqz-cda+OV7%PYBhoeYt|NCI`fsWj66xnp>gwxt zcy|yV4^aj6Quphbdb=K-2hCnZ@(fu-wXCkHXxpxEtg~0M__5{?!~DcjEl%RBdOi_s z{sJGs0-}UUxO+pB+19VLtypen%vX?yGFGXY*NO^mCzRtF>u%N34lIyaxP50Z7 z9+d=r=m=K=pf#B2==gi(HCp{mm*vTH*~+vk^8la#q#(Uxa+_i9!HeeriD2z#0j*os z5+LopaR`eVsIWvA6|S;BPJW=*W?NoSv^QHjrJsA82^Bu~7ryjgKMt_P&fN#^4iewt z`+m#$U)c(dmx!AiYbr4l885ua)f;8-hre;H{W=q{p`48ui2j{QblD^%#BDbZHm$OM zCk<$H>r}&2U}t|nw#-r3v-c78>!BHg{K~Mjs(r41Vz=J`j||jNHGZP)13v8c|CWMu zmHj_4|0uT7F%dPs^uxZw2KV@7ZGQ4#p04=95HCQW`)J_iSEfSuLeH6WxMuQ3x^Hr zq3yH=gKGzB4yvSnxW*IP@|+Xv=(|0z+PS~eIrCI7)q0L#APYQI5`|3jlcnL5dj&=b zd%Gk1io0`ld$wfyQG2e%>Tg&ofdYUc6tz)jcmRoY`IXzSDGF|6-3ix|opi~*^)~a{ zKD6_ZSjQ7NITUDOPrf0xzr+^ijZV@Iem>+v0c*3$NxIn)Ar_D3Rv_P);>*>P{7PQ0+F7=76o zHv&?72Q@$@8Wz6cSuu+7b-bjP?j85YY`Av2<>kQwVtlP^RDNenHU2_(OuKHC6S!%1 z?7b&5ukQ28&u12RFwHC6or-q%GS%ZYC<`aN7cRw(#F@s)q$V*+`XaoFO*T-&{kY!Y zsHYbseHSh~9_bac&8bFbeBXA=(QkQjw7Jq!@mbACH9kWhnV8*MdG=&{8K&p#0aL!C zpH9y%5&a@6dD(idC*KB}rBy0X=#-M{@s%Gn_|fu4Kd?&+lAI^r8JFeLqe*-7PE4Lz z&wK3*7uLUX>Y4b6ggd^UAFfeIGaQGt07+3e&EkQPV`ov`Zhmn!<7`_P3qL_Cj-zvD zFldZZsYoQXO>JSDV8v1pXS@lCy_;Hk+q8ILCu|Mi6ofr~S-0sliZy!l$^;q@re7NA zelICAP)*iDfm}QV3tY?D-?kU(d9#;$r2QCUJ8er}$?v(PzoY3F&Y>Fe#G!;cqo-=A zHpz$7W}20GBW^1FG)NrpkELFn=ImT+HR0q@@9NNE zq-D-5j*adhVHU%=gbbs=;DwbpQK#F!JxYNk8Rx*_*4bmD;%gb zr#xLG4?z1?(Mo%g_9L~5ac5_BAxs98!le&pnVI&U%h~s%-~h0}dmN9v*?Ye9KKEUt zL&V7b6o-g|EnQ4K{vazw+yE2;WW1EdJbbFB7yw2aDBj*bQIA^%mhuU)O*%a?scG!s z7{SvVt!KPh>dxbL_fj=I*MthG)l#IcMcyIxD^)UYvTjmrI`s(T5}+9!a#D2;aDrO& zTVKK0PMHsbTH-@`86i?fC>7=&H!}}tY-$PdjA)M{zUAB`sc3u&?40pU?K~}Ebt+)( zDXZX=Q;Ni^-k>E5bYus%cplW7Al`GcN^uOcDWta`eHQ`VUIgR~4KhHkl{_YM?1y$} zEuAh{qPYMNmpT%pvK1t(j7UG`2VM@i&E^Pl?TgoLJ~-M=*O{+QTif}X=VlxrioJ*g zg8QNeeDsDZ<`E#K%iP9V9&AW>5QRT8dm!~?)Po4{VY}?-{PX0*y|se$O>UGBs~vw` zn$l`>KO~D{&AB0nZU%+bZlknR2+;^?4r*W=bRR$b&5Vhw(V5GFjj8p@hlW5GR->Cw z42X5R>jNY%WuaR0#t=wkr(CGykxtZ6yest#t}Dsa3Cee8ip5_#pU0W+PL}3o*Y0J0 zd}IGE_rSMTj+&ivarUow7~9no-mEDMbUyi@sL6?4=K3UWDF6Wj;m@rLBdhwP6uBV3 zHT{N2ys6F|qvi^oP>JW^GmAdg!!l66NwD%V3HJAB479HWSOcLW_GKLDuz!b&i>H(>k zX%HiKz`*2n{la4bxk%90)qAMYb;z`AO0_6O^;km&f28bbZ-BIaVgImjr}StZo5L+I zjjTz|8|vtwq6O*8(?mj>cs8KY35Irgar*!OJkTZLHV=RORaD!~62k#jx-aF$L^CwSsdNvR?MttiO^3E6`IjP+x6tNOap9F~JPlX3sE;&hs z+T#*7zTH+`EqL85(WyUnkMlM9iMmGX1ql-pC(wPmr+%2svdGN)ZZ?9mEF$F|3EBH#OJh7%=ZibD*5-Qgux;I3b*(_V zv4H+LQ!&v&UG`sz`&ahnr=sQqFo-r2bkl5aR?;*k)Atrt{u73)pbsFgR&zuCfoW-a z#vlqb1D|pgw~UGQ)|ZQkCb%R4CeIRp9|(@O;1-K zWsmEF)Tz7kLVM@maG;+M>(6K56lLf^?TD`H+=5$1_)q25Hl0>MGaT4n=tQ4G0E|oW zjRIflOb?FCI+#@9I}KqUuOK9CG8yI`_vjAaYc-8=k+I$0uHu?~;KdqfX)YTb^222A z=Ki|;S{UqTum`djtbE^}K4lj5ta#-6G3#cUc^|Zr#^ zLW>lZ5Vm2*%7hu4$AY>i6!Saix@7G8qYuFJVYC7P`#O`^+sD(}zDG~t3c2kv?9z{l6uOSSaPly}E2x_-%UrW_m} z{JLqREf)o#-b3#eDi7ewd43==Etrg+r;$R2^VC7@G#8RN#IlLxk0&sv#h_Oz zpH%j5lwf7Hy>;do6rhEA>+#4WO2~)$v#K4S-~<^}IF)Oq<5-qZXE`h|z*||5lX?!N zDtHGkaG2QroMGFKR&=WqLTrbRKNXzfWWO2WfwL{W?Hj8uhZwzlzcEXxjq5xyw&sB^ zQ;dD#5GF@`%uMn9iCL0&GJit>{@m_E2_T-(-X3v1z$K^@3O=eKb`e0u4ZREhUOyVF zHL!4=;cY!7Ls!2CpoNlvHF&q`S!^R?zC4Cbg9Btc;=M7n3h0#Cbvrom86ab}4Ha~G zgZRWVWR;??>+!B$>>!0Y#iqwn$nr#|WtbpNqPI>*e_>s}VDB}I>BKFF)Gg+>+bQvx z$ToRJgtN`Hcj6gQs-^(x-@AEJ6<%4Bp^$tx_a1K1d0H)X*$dYiGm~JI`en#`cSgsz zb$jmHZ5^%bWS8rU+sWRnbL|Z4zVz^NWiCVR+61iuNKN9^zHS--e?|8OAN2E%mS4RW zpfS$?{P@c!I%6&gcmLZZiZ+N7IpXz2nmN}P#=!fVXt}{jf%~~gX9EdV6k!|mQ~SuH3OU~UCj5tc6&8HJ3*4G>>8Uw{}lF; zz!!X9^|znqPsx_P#>dCz|I}29$IMGFyPg8X{IUgUN|0B@-pfnuX${H^dha&ms^@SM z<1y`-430iyb~ZXQ>4d?{+hQ1nn*ynCPQY{9`kyU0mRW|_*0{tNh7>cszM>>^XPO5O z*hgAqO<_;vrccxwS<&6aU{f&mWka!zt8HRcBEB$xPVnHo(@#|*hitO{Q_|<4@E=x< zq13O*1y}+lb6D&^zW#|Z;UO=;pWqt;L~N2u84ooMpXTf1!~EU_<}g_9T;U4S!AS@$ zUCXMN@RVPYN_7N(9Z%yt<;C!{z-7A8t0VE^3;+eS$fCIe;-_`AA2mI)J2a9>!@vd~ z*|c`JW}HX{S<1q|fEm7<`*@Ha&kHkux(3mEq9#&KMZYRkzcTdRBDp@f2e*pQnseD| zgaU#Pc4Vhx?Cg+;uzma> zN5*%O8k6F;p?wHn)lc%STmo3Mxg~WX2mVt%1;b_K3-!6KuJl3@%2fHXE%ybK1hxS4 z>WmM^OK|w4`qjdk^?ovD%qBxHwl+=nxxyn>wAMspEiiTKw$S=O&doP2ZlcuFtNz-) zc(Oh&iqV0rGz|`EBWj^}m}@`#dH9;y58nLV@8=IFUsn$mJhMJfw+WD7iBoshNJ02U0LkMG*)Y1nvt_%~OV?j+m5lLcX!5Hkj zh1#w>TU8%S?v*InhNsJAxcl44%Xv>h`5B-t-S6+_c9SWGAtPbJAwp(3XV#=-klyR2 zpC1gMR5O`zCavi8hF}IF($UBH{a)Qf^`}ZuM;c-(0cF?saVPKvpbY}y8Y4ZNK3BJ< zEbhPC!XF}PSPI|Y6jUKapSw+v;n>Xb|F=n24yOUhkO8v#!?WFS_Bss|{T|wP2hHVl!BxsJqWPn> z`5urvGYuUo&K7RJydidttm^KKY4n^FjZT2_M7UXl{Xw7k&mfbyyu!@@uvbyIHQ-Ps z>ekp${^gZmna-H#)1SK_4OolYO}M55Z7W4g+38C8+EkL`Pb5_Mt+*rT42#c2J7#m! z*C!qjsp>*{mxAc;+~YM^ES)MJbKe8F6f8{39aDDQ;G)wox#Z-T)6R$SM}q@1dUacO zJ=XiIuswm*B6^EkQsh<3ggGzSX*kxb-5Xa|Bi06SIPX3%!x`JM}~*A)FyI;1M4Ek(Q>9t)6pZvs|`4ukg){ZX~ONhEGPny|1ABj|(&C;*r3IpHHfPzXB zz}8*`+~_Gsuw*KD{frPk1Pyp0L}*r>hd~Nwd}u)@ITEjt<6&8+NYO#J8EB^2#y;~S zHG54w?L7ce-%TfYJl`-p1Pd<7Q+ zL%GUZ|2*^cKsZZdcXrh?aO2=zE~&YmYn7R~*BKTkkV?=MP8voczH{kl zGBHeGAL$x-Qc7M_-^nXc!A)Bpi1x6}>%Afna`ndGc(Tv;rSgeLu2^z~Mw)NsvwPFj zc%TqKhgDF!U1cB8P-atKodyZ$T7@yon#83chHTWR$L;j=qst^A<8n|7gUTtsc>&N! zrK_z^^-2pTLdp4dAowW`rb-xV?%?$K_KK62Y?>CCvQHMcwnTC#8KxF{0kaCE$60^_ z>$#2GcR9%JY#?TM*Mq3oR4~e`Dkg+71PQ%<$N^fi1`zHja|R zBAGOv4nkrdOHVszXEmrDPjiTR#w@3}yMssHWQkoGGPcuaiI=enbGY@|QjQXyORIW6 z;GU{u!%_tdpH=9E6Q235XWHTYj*MSJ@$d~5+3oHd;+a68mSAZh@TTKJRq3o|YJ)Ai zTvCYNflSo>`2|1mIYnvU+NyN4VYYe>7p04RS5@7HV*VSYiF3CRIUBX5gPPz0W@1mR z6@XbyB_SxIN|G>EdmXfZgLNkoEyvJ#lpVsUbk2oR#%b}`}rgWBg<%^T;W zhkt7DN!Kpz#9tG9uiencc*z?<8f4@}?E#y_#qB)&o$9Dou2Tdl zLr;Vbjc|em^h|1^1DBJdlI@$BZaC;Vy?L;;Iz#$|L>)_>xE{tMBN&gaudbHyBD2#w zdO%35+{V28Bh~8iswTQa)9+qUyB0t^U4H#lc)@*Z#lC>$CBIg@COztvG8bp#h_KIc zF$j(TtIb$OJZ^yjgk>4RWz_{H{w-uwYS(VNjRR$Wtipl zmlgZWO)KPSx8*#H9=d6wR7h8WV8$Hl#sGjkv@)nF=u^-%oV_8?EFM~Nl08FyOagn1 zipaYhd2b4(Avldopq+c_m)r|Kp%(slDkGf5AQyW=6Lr;!C(9i1{3y%mxU%9h#5o1g zTDrQE@o8-1JAMKKH>|aVol0MICE1@@o%&RbIwq)rzetokl2)|0Y{^+!S?R)o(W0)z z(K-6m1vm%c?F8A(j}I`nJzEe$BAuZ>C-IIUFs1$5o!x&-sfHh~fC2MWvu}lWPxxyc zZAC!+7MZt79IWo8(^C!J8GnI{>0ytbsN$Nwy4<&Q(bxHiVt$-cQYWaG?gO1Okbe8P znOUi^j}v(l91LI2#ZG%BnI$NvbkfC7go==o5ffbMHnM0IMr=T$>(wqyB`Lz@9Me#g zM8Vxd<}xZWoza22YWL|H`tA)A+W~K0f}5#Tr(WlVcf(#IlM-ZvVpNfd%8AS7BF2gu zAB3}$A`;978d5A<2*mKt{a)-QLHd^mn^Ouff|8B2%!9fY>leyF5~2px;dhn&J*d(` zqt~F1*ULfB`wk@6;$CbV?tAQ;^+(+PVxJrk>35|?3lYp>te1$qcLclyWCb+^A=$V~ zonNioP*=0hUQ&@?N|0$-2ldcpCoGlpnzYRc71ovY;y91hXrZk%Z|LgSItDlf*d8uT zh)P;vu_Go@aDl0otS^Fh8;I>k4#%?cDG^(qQ}#3x5SLHh*UML$;P5!L-_Y z^1bn|SLpGUS(a53pWu*Ib9b+MN1M;lwmr|t?p>)Mr&2#EH66grpI(5K>HoIW(%C5+ z2xQrB2q-DA+1*;3A(sJ=X_ebDwk?lbHh6-W7jjnFq{hS5pb~C#G_bIm(26qq6d+Cw zXdCBnelo3}FR>hr3kBu!YfQD6QlnSyi-d2in4R2o5{sB=rBs+Oy!hb=^)N)pFi)w} z-x;S~(Ox9FVkE}jX4y~Bb*v=de@!|^CD}}M8>zw_jG8MoC~%Xq2JreaV}$jxPD(xE zi0^fFiKe>CHY%I#`GQ>BQ1+Va>mP0BPtWf!y?AUdwk-dN$8a2)MOUf`!g2`>6ZU@1 z1wn5;FhbNwilw%t0eXqo!gl|R_b<0j83+}jLBEFR*PXq@lx?qXt!6S%Rh|bZD%o!f z%RF%rEjtvsT!{^>J>0_441vUrl=Z)_Ht9LBWZOG{-Rz(7`l~%tGH!m2`^#SaIr6H0 z%mTFyU{TS~rjV)^^o%V2AOn8t$PKlJM9%(pgIE0A>QraZ9{%P~m2s zLP1Fa+)Mxsnj}U7 zX?3Mm3Oqo0wwN=_|{6Iz@bO3Cg z9G8%GMAtk>IBuQlhju557{C10O3G<72BLP~D~HfOTK%Ij07}+T1HP&_)aqbZr|*zY z226nuab2?0AOC982-LFm+ayh{qjb|*4IjW5osy-NE$ULrpNr-l6Z_?+`$wLTkXZU* zJQs4HZmz1Qv#Jr6d4&ec98GKa%$#`6Lwo6&dDX+(SC|WJvvX%XOGDF>Gq9!k@3)&= zeg*8Vqrh@s_`ojm6W{+5j|V`^ypDh=lz+ciz#ibQ+<;SL17vkMqXL&4HcGjHu6OxU z16jvh%~Wz^ZidS9L97gi0bk22xwvWyD4YQdpFw|5jMlAR1owZOHIO&(6Fc>jXaX=3 z1=JkD$>kPH-b+k;yz_D-u@Vf(nLOW>rUV-uNWt!eWUCuiK;5?6o zlHP9N!s?fX8Mm51DFb=sHEf~wPhGH&eE3hPqUh$iggp}?^P<1qgM z4Nx8b>XeiN+m)239#TMQE1Bkygw}TF-PSlqZTDn z91Y3fA5bH=Zl8uItu(P>du!KAR$~gi&T<0<%m{vF(0p%a{Z&7VDHtNKZ*Z-s&nyT1B19Gzk1}nn0h;VTf_#=5^aUEluD6_ zXuA&N6@MU1G12{^0Ka0P5H*MnDL+u0vRP7X6at0F`l^hEcr4yC*iAoi1ua|uHw`X35i?xnP&An4q zd>w&$ot>1B2Ao5ZgHNEnb9L7-IWHWp+@%>eI)=UuDOU7i!~1b`j+{0@2X>y+Q{X0| zA=Qt8R)px1@&XGCWS=0KAP}@x9l^xhptS>^m*ntn^Y9VlYNta9Yd;T< z0slD$&#_;)`d2-|as>tZw9yaZsO#OssuY#x6&u5zSl|5(Up!m=s+>0vjIz$&h@I5+ zGvf=OOXZd~^z+taaCS)FSud;8B1<|)YwEU#mZoBHUzrd(d@>V=WdP9I^O;SlE78U8 zjqGpx&h%NOb$nvuJ1+AO)ceM#q)J4^)Xoi3=9wUxMit0cFR5kSK?-z?2WtQMM;=ai z)tB7{716z^kSu$$(7)pr#(8Tnf`3xrGF}}u!zN-71~WN*94|Y$^T|=j$zcrANFa^b zQ`z^ZNOZ1hqKjHReRASPCV6EAS0C+=?bxbBd7ws$Afpj4*5F)wp9oCmBdAZ(7*M&6 z$?vHnbvQoRPbqAzu$7qgo4aF{1B4N#62-;!19@2b46_R2Zh&+EwZWbuP&F|F=z{*nWgd<=p~`$jR=SI&5+|Gc6=08z{$HgQ{s|u5HSz&vz^;31widSF+r+0hX^M|p`BUc*<{_k%VsQW(#-N~-dya*A^{Eb1B zLqW{xch6ns_u@55Z#&|X{vy^p2xq$8-|NDwJDhsnUN7`cwn`skr9%41ilJjo$@aiu zrJXS*EFgtd`Oe1#Tp60qH))clz7b@x<1~M<*XUL4M$HoL!?UPom1w4bV-N3IhAq6m zju6eJIwYzbr8(zJWffp$5T54JdX6L`xyemIWv#lJo$7Huv@2);zIM6I#UX3OA|*Lo z)~%}{DR!265fJhnYW>mh9S%us_v_@vJMpnS(7C2*_QIyqC9scAa8Qz z-8N}$9Z@q#WBsPI{==doSl*W%@d09x$=c1jxbb9>)7r5Y$m0uq%gs~N3pWLBWc&m+ zKVRz88M+#tXAB*>*g)CZs@J|$O~m|E0NH%WMawSw7^ckTCzI@Zg%dP6tU*1fQX?|q zK<3cu8985B3<-`Mr8OwDfU3d_0!)AkPB~>@gZadft7!%m59ME_d;OsQ$>G9Q=V44p zY<@A#xa?HCU`|%siH3z&I#4;cjwZZ#^|I>TJ=SBWA5(?`$yT6RQ=lec(3_qpl^Q!r zJ?n*8QUB2KT+v~*`#qqmQ21=GFI3RT523ow9v^yI#26yPIueypB5y7y`ZXc8z|?vO z;{s}@>mzM{COvb-(zLJ6U||S3J;G04>oK6%+FtpK6#T+>eDySPY7o4P{ic%GIvtz! z*fx+DCu9ABa69Q1-x~-Z@y42=g`~UOtpi&1-$wl9RW=~Ld&D$Y+x1u-y>r01CKHFd zeG#r796^PZ&%onVv+r|V+{C{rRr+R(co{ghAh*}tJPqW0zZJPV;3|zusqP*h{V}R9 zU^n%{1ki7{fb(0arlvr(`qP3sYd-Bb9TYvjy0JrknDr6sOW4r|au877Nw**7p|}r; ziv}3L6JA`rW9_Z|SxZ(Y?mS!&IO(?FY+xdgA&=Egx+GfWVa0=SooIPJ=Rl*A2EmwoW9t;(=)8DxdNEEG}oge>@C@%Yjd)5YEdTU6hWJL%1UDNZ7HnDCZd^Dz^ zb%a)7ETHMZUf3QAc`|gg^{$vLY0W>jB0CDF9M}8Vt4KS`9?p3xrQqutD-{cf#l*26 zb#8xih@7S?pE*^XG)F`Sy1?vod&^bSIIeJe0i|D9ve%Z|Su5M*LOYuI45IWl18M$#PmSfHn$1oJ7mZA(?xK8}SHtr|K4%h;VA-3mf;#6*KZ7wHRNZ zhHa`o*})PtMYr9i6vR#Zd8`nqW28Nhrj85IIgM$h<-jdJ(Lj~-pJ3$hli^E`4ku;a zLVXCO|YbsHy8pllQmO7s>wm@y+Df3EOse*Rn z7YEv&$jJNZh<&wh0E{gcMSY{y``T_a9r{le>->Wc;eUcZd z&l*d748m{On$Asl(#Wg$O$5{OZIWw}kTLsPh5N0>Y0OcudDdsxBlIfL zLnRW-`Y`@yb-55&uMd1sra@Zd&L?lNy!A>&NV;>z6@41|IknSh$L%imG)+=!9ERy}DW! zh-e)MKcXT{NuM7|ciTXb*@yVXHK-}*GmZe%5fY9~GWQPyhZ!m|1%|l2_P0lrMw7Tj zq#QPxAzu}#)_`InoetMGWrT0Bgrk1J^a4r#GC^v zVlAujDd&S9#b6;}f8%fbsVDpM2iHoQol}Inj;=+)&KX0#F0lJrO(zE58$$`;t6NEt z6p6ekj`CkGvigQkp9+SZPAg_Kf#t1?UYwAvb?;h|^Q{_pq`>E4b(9p-)F-N2NXhE$ zot15Elkt&j5~=Dt(}GHntsX;1+?tcalp|H!7}vGwVa&*B+p7x>p&s8N7MxDfGXk~L zB<06dVh2RJlzp_}Ww?WZ0&(;m7rk*XfR~IgWK^>#13Gp(V*1^@Go|r1LWj~E zvDI(WnidnMms$YPia77ua>Iq+5hmX44=2*2=n)fB0%b<(7n<@~hj9Rf=e2LczYwV4=4&cx% zuHZz@t03E*c(ADYKv9gGp>p_rgQlzH>&^LFc|6e@rnx(hd*?i9RCzIqE2((@g3xGx zi(sGT3b{6>$H$Cdk(tO9VD@E3;wJ{Q1~(^ncE$Gz;v_EZ{mH?I2R@i<$w`|@*Cn*~ z@yTEgS#_t>*Ou~lI|CnBF*3;+p4L!i$pd(I)podAx;Ooo-pR`cm zv@oe_aLM7o>#O&nE>9mHuSy98Nm`$4>+PI*798dOQGNeIO-$O%tg%Kc`g_%f2;i*M z_gbW@RI0jtSZBFD8|C3KhB4X!wXnVcW=pGtAQ{wr3$M>~v^N@=;ezE~x5&$VQXyGu zLb5Qbv1x4^gRRqw?$P-^@W?t>C)U~c{D{G$?w2nm!dyPqIz1)5X2c;pJvuIchUtny zMKhf71;-5JbfKleYaf`%XIKa0cn#wj z_rfgMIz>(?;gWPPw_QmMHiFgzBD!$N?njj2 zB+KcpaWSziPvILz99MIAmMl2i*k4u`q=?0G6IW^>S_?ap-kOcugzCIF=0N7) z7ZpS~&Z#5JhV`uwyKvh2OT>0*oNKgcT-;0}ZMz{Xn9Na4~<$f~PS z41TSIYK=O_bH$!BQjqE)#wJ2JeVI&@I)vU_C>u$w>r?9+-?=lFC|Iz(|3e4wl8MsuTRB+0^z8}6zqqz>m*$lPrmd>XJ1Y_pZtqu+&?J zm)Wi>glb**5h@}wbV zrH%BAH_5CD(p2d_RepL@d_*Fmldg6`-`pqyX<2kMGCw{Gl3I6dCqJz7!5*O$Ef-=r z`l-GmwP$%}!7Oz%0;X7C+AFad%iQnvknw^M zeN#_X(lvMNaU)c~dD4#U=&gE0Lfwmb&ppxl5$L+iT2x)|pz;)?^Ern@*RqfI%8^U4 zfX{h~TUULv>Wbp%h~Q1+`|?gv7z_aLNpmyZ2RKb$8Ne6}LSo&Ch?UC`Vu4hQ70ixpyzbNyD%m)*(Q!`XSVCysO4?XEFk<6ui;ICrcjbv;0_qHp1 zfoff`YPyr40y-adag${6a?)26`J}^gQhId#3kJQc3{(BB5}axrIOAao*^jB*n$0B! zpPv9_&nzXmUT7-4rCl2S@uN@H*?F@*>T5w=)I&yG#P1%ljawBOE&h0*9!WY*K~=l; z`bfpxx9O<+Arg}%n%H5(-i()F1 zi(G#sCeA*Q$I4{)Vd^SnTl4F|j|l`xZWXCVOp2?zVvPV5#p(*%x0l~kmIGlyfuMkM zw)TiOgB@OvxT$rhkH@<-1`s|Baj8c;)~QJzL0`McS}SdnTJ^%x%LcQ>4_XHCG=>sZ zcr`5QijHvoe{8*FR8?#D|E++qIkYIbT_HLrQi{Cv?ScrI1^SpDKgzsHdY zmbnBlj%`sX3N|&xAJ*OFSNdcLb}6^s!f!&`POvU2607Jcn+Tx}oqMNSP=_-Y86_f} zyM}x-*QAz9Yqs$&PAlG1dge!~V~%iTsCzfFoYzNz-38Nk?gvNc^MoT0TH36e5w4<7 z2fUp53K7@%Vh=RaiZ@4l3o=C?|36J|bU&C4^o0@^Wj#HF( zk6`}OUhA3pPR-quNmAYeC!Mua>i z1h^yyOey%ia36kk{HWf?EP&=C3}eE;Uhma(y%u7hQPy-FHJEJGKoiKQqUG?VxwZLX zO2dA8FhZdMRd^eiapMWxc`jt7`@;J;fVacYqmr9EpX@bci?SN3% zy_nv3*%~KqP3fBw>?2@?F#6I%TS8xL$T@EiIR>f4nJFz9YhPlQV^zQ*!XPkNrfMSR z@smW>r14EME)>d^ox+fv1Tz6U%x4QWH>53a8^m;gaQAYBa`&Ee4)N~d_26!4?vidH z5MiYCob<9J@tr#&SC*vlADOvlLkl@P<=6Fjx5(A8&p5o}`UnQ9_&4zd?>7-R205L2 z&cotzc?r80n2dCnFHF>+(95e2Mwv?|GPplPpst+B?I}XDByK zxO}-*`oSAVyq-FBiG9n4`!|$tJAk*Xrq2Z_;MMn%@4sVP z5h`#6`V-J8bvTwPmi$;O`VvH5M4UJ}weQ4m(?!W7NiPz0Q_z#`UQ^hWdjQRaJT>Q!F=plCcgFD&5sH$`>s#UvizDkpR&jL$@0C>IeF18rbw6L6^Gg zJObn@Z9}~NW1uOq>0&AtK{G!BUe`~55%?+2+tP>+?lC4AALabl00(G$*dNJxYIgn+ z$_fK3jM0zDfqMEE{E8>%C&wviHM88IOEInWGw}dlRKln+(yrY1<7d>nQ^)%C7glT^ zDw-Z-G4psUJwXQgPb>_YfmovF;)?sCY^AM{;vQe21{^H?7?D$CG zyVmA>hLaT>({CBgwsM9jP66K7D)|llz0BVhLcM?}+9CVjZX8R3MQu zzAc{?x8FM(D+YQ~yiVx8viYX;1VQWaH zh85z-WFE-z$1OF(9rn$L^A7ne{E4@&ZUZI>OHHQZm03}`(I>~eRiEhAluvv<-8P0? zylXK0-SGp2>8vO^)p-uw90(v@XUZ2_Yw?yVZGzOTzNAxa6)uE2%f?`zi*38W$fSeJ zMx6G1&=&b`6JpyFS#!;pSo-sso z9Vyn3W7i|9#xdy;-sCA%0k#9mYrU-YdR$oO4?-M?*7l*d1P0471G18s4oUb--;&Fl zOuKq*KemdR%fe9YBConyJLEY&G?zBNaDcdWa0MoZ5Y!UfLAgI$(+*rjo#Ar~f_ewm z6C#XW(FmeDcCcWW*Mxd4Er6Yi>BBH_1nPe{hQ1gaN6uM9diV&`+M5khT04p-=tKQ zp z4{HP!PHD}v!1P952T9>l8*nkvl}1dL5Axa$Jza;|qyXj1HGd>^7}qHsB30rDRqc8i znI5)_d>l64eCm5<O{Pbh?i$%Bq|lMu3HcQfM%~f96u#7%}m$beg6`Lo`9>FE3Lo)RYbzeo>Yi^ivbKv zx+|RxGr>BHHc?n$Uwu{~LsS%k&zaJ~*`t6~uh2K7Y^A9UG+XQMqE21v9MwErh__sZ z4TgyHHSTt+TkJYd^=|;`*%gQ9uMw`6aqeWR=9!TL&a%`A@?9|t`}&m74ljv%B88)Z zmW3kdG6`J&eE`DZOlOM@(6GBO@9z$iX@F#lagp+S64z%AEsvdo4U^Z}oWo&>Vv_A` zE$n4FSUpV;hi^9F$2qCJygaLm5&mZ+V*B8-&J>@c3OgIw&yp9X z&Xl&4mEi@V+)e{|yFVia7G1`iZ6AdfY=Jpxt{3ab}l5P+t?R z+~cR_R~DKbicFVmnfaD@GbT_1c_9=sb>pWUlYUVV`hXH9Q`us~wr*d800ej!x-HQ5 z>Y3(pF;6I_z|42lxn>>|Mk{vNv`_oMUiLUu$73meig7HXkHs$Aoga&V{9^Xw)cmxC z#6IIDDJ00!K;WeKRdZ~sqtcc^I!E?kYICUTQu&Y`!O_47-_Z0tFZ9M-hr_jKav1wa zjj_iwg(@3p2j98nY1^W{+uyiX;3;$8)wR%>=qQLMa9YrXcB9Z2(n!l_G!}_32Qu$L zro)Ek=fBa>#!27~KtoRkb%>VdPhIbd9C6F<=T3gnXW!$NoS>ASUOPGu)hIAt-w6|D zF-)RbaG+h%h)j@g8fo5ZcvCL`n7g^_AYmb&^Ni*Z6F~H8Sj7t`EW^POuk}89yy! zwAnV+D~*QeOn|{2F`gwBa_Trbi8SARdDGgk@3auUsAfs9u#{gMPB2XV7zcEu=Bg|P(Cn}AgZKsH~8D=cj>Vu5rMtSjC>=u z3fZDSgt&>^?bm_Hui=D&S&FCYh^O8jGXl>d)^KScm#V^uw0E}~vgA2#U~4l!Ko4p zy)SNj9um})cTH;MO}k+Ibab5uBhXI3p?VW)WRD71w7N{)s)XRMOKiiS3kMi3rUCbi z*~1flSP3qqsHKq8vzzNuFuut&fGZ$aU{$zy8geM+=>BUEp`p1F(d zUJmgIRaawPPXM*5eDAKJwRQTOF}{uYf4o^$*b8i(Dsi#`l7(NZ0r;n{)OG&4-pLn} zjPA%AVNkz1TuNrQ&h2s-<&!WIvQ^N)nl!BP9~zBE9Lt)=BX-@3s}HZN6R= z^3jBOA&kHfj>xo@Ro=8PH%*tWdkL(`^y<)1!L8Z?U8($MT*=K^!g1WE_zOCK^cL?= z+`W;NL_E9u9tN}daiqeTlE5%g)7`GTA5H5umV!8g9byh*t0&Oj*wW=`$Y1S-+uLl9 zb-;LyxpD@XYWU_Lv_076cHYxJy}Fa-&oXuz8n>T{0WRj1ygLoCgSVY6a~R`5XVRzU{E*$_dKaZYx~3!CMHauoWksGynn2^Fij<0P*rYpWG8 zklZZSBN;@`&ABYBYRM>Ffg`)cv?hAGif)T_mv=y4r!wU|s>1SGeeAs8;#S+IkwVRY z8MIK^7UQMUZx>6fCHbUNBP?0G7yV=7_s|nMUv8jOM!X%c_q2sXy0V_qgCe?1GO#|A zrOB(ru#3xKL`Jb?Ou8j8?47d5DPhZ?t3I5F&_Mqt=)x|1t0(H_#9&x@fz9&#NOm^w z^22;b{pIs4f~P%V`g$!hr}df_Ifv!R_DHMaF-P#hiuEvHx0CL^As&;{-X6j(#%~v6 zZ4OO)yjwi)ht^WMAM+Vj3DLT{hOn&aY%2Y4@h%`Od{ChBrQ3+$s{8aQpN96ryX9iS z<8fGGabrPxcFmf9vG$W^LIBpz^Uy&WuRB7CkZ`{Mp`mm{Mg#>n-Pzm8|8iR`(ub# zLYxO(#MGYO$mshN!HqzHdp*?p`fbY2cU5)IrLI>W+DQ2a2gKfIvKW!&WE#CI9xl2V6G8M*OcPcMD6 zmIk|Ju%~%lIDP6=6UTm3Ch~do3tA#K(^92O4yoVxE`2JC*x$y^6psJKPjgWE#o#L* z?>FACdt}Ir(+D-b2!~tEP;cEyJ2<=KcdYz374%igGerSh zXRTcCQ^hSVB1HB2SuxegW$sQpm$H=83Y;ES&dY+z(Ncj6c+cH8cm8L@TNpkpn?cHX z1gt3n332P;9 z$XtL{zNgA>IE89P;G{Ox()h`)G74WKWxX-DB`}3iP5W1-E6Zc%>&rEr9uQJuzF4)eNd)l0qdE3 zQ;Yu8omp5>lEq--uIN<28WRiDfD|M+J#7i>iB^v2F9JQK%jn?qR#}G$2cM~0txgmp|tGv)RYuGWjU->z#2&AyEy zt#!*Lk5@PmON*5bntiriy|T5{Q_;4%$%;V|X-9*pmuCD`)wqYwusJY(!p(4x_;w1L zz7p{TKFVNy*+y2vAvp+Hl^?rbQ&fT$XoCM`r}r^ZbdSXco}r??H+01(!}rECQT+Fy zTaC9@)F*!=`F;mr^k}m+^+uJ$XholJug;4q(qwi}gtMB7Q?mOs#*aT%{CpYVDZ zr}41fNnX|4&(kcAU*4p7Tnd#*VLs*3^s-4;SXvn$4-u~hS)*^XpEVxxV(YAe42nJ^ zV4JvXXu&#Mq5CX}`81Ar9+5OW54(Qf8hf|2MmoF{b@+pGwSyoL3lkk8IsMB~sGo!f z;|YQyeY=W&!Lgs8@JQN0DaQa%KI12ODU-Pf+y5HBz}zHt0FR@+qwt*KS#B3>rSrlQ}=s5r_uXGN4xc5SkoAex?y$#!hXb1M#;++;a2V!>H0c27`McG5BzNx8^3#JJF&C6g!^ zN!y9vCm^0WR zZ0M|YDTBC@-Q_j8T5b;?rawn$yDK={o-uxDENPJ=Wpzy8P(G;WNt+xnPx}#9s;U55 zCIPmNsnN0Q-YSG^iU;HLzS4Vdj^bcEEV4ua5f?d|ub<#;|LUSNsUtGH*8=l;X*w+O?0 zeiEkME{7V1Ax>*|B-gz5BW}&mPG4dO&Q8$X(g){FY;5oiy*2L^4D5(m%UBZ!#02aM zUoQnZavMH6R#9I>vIsUQ77z+X5Yj*+UrOv(Q)OZ!Z6)G6Dcy5thfFGOf-=;yiHZKanYpwAOyNaAr5pG3?C42 z=%5IxA~}dbftG8-!%Sa;#W8k|qbk}bAHvEO(8%c0+;p@;SyVi07m24eo0J6ZiWx}J zS6>?mEa4vUtk$+*PNyZ}hp8I@Ol$kvyk~9=sO)8i4W_c@ijHkQuly~hzyU2b75Y6n z0d-%kJEP8unm1dNS8m+6^`oq|DTu`23 zKInzur>txdj`$-f(aKcCR&DW>UXiXbf5CPRk-tydt^MU*svW-KFG)F1Ezyr{5u05Q zf)RhoHgQI|v~$IETKuax4i(z9kyer6zGE&MY{L}UEa~egS%02dqt$rmGLZ|;r|mFj zfRRez;IL*f4iY4b1W~~J^TXcDO9*#uGn93hTD<|nTH+yytHAnc9l0HOh5;aU?Y5si z=HLEiHUEkEq!Y4scO3YWo@24@R(W8foA{_+zFL1)c5*jPoD)TNm;_}Zs0E zLDeg;Dy)RKIw0v$@Dunz;A`w~)Q`%Xbe8O!71+^7cHMG1M7>@2xSlX3nb`ofN^N8MPQveH%#2-z0MkOv}x6tN8oXsoC7O6srcnbYrZEkcfIb;%&qnmX(}`s z{&vxYb19I#nUCk9*p0cs^m5{kH7CClL);v{zMAB&NNa+S> zkgEOR6(%s9OIRRB=iobFpir8>BT~Ga*JLRKWlt;wE4E>YC|;;}3hf8l91~O$KU1zc zKAW0nO#NxGswza|*L`Cg7F6u!^HHocl++O8$W4Ae8)ZDtRl-f8Pp}?YAUWkS586fD zNriZ(&hpRi0erd31A^j+aF3|1h$pw}gj&C3@;hWA(;k2uZ&K`%tLkrmf} z1>JwWYJ9Z1ySmuBCfvF%d2nsIySUnInO{!dR%@a~)311`5Y8#uY?lhZntG4ivOBWd zvVQYAs;d2AhcMj3x zb3`ge;w~}#P$o)vnIvkFi>qJx3#Q{$h(Z;#CKp#pDUYEVPw0~EO`fVIlKf zH^Q%mFsN&T7}V`Uuo)SWY$e~r*10-kqO9=Cl);TxA0r~In( zTcV-~t`Z^%hu;ovgg;xx_v%RPHi-2x60<`|Uq<OIU0x8 zt6^|X@`=aA4W&cE?a6OMo0BI#Sx0S4iW~$XvgNRmY4N{bKE>JD78GnS`S6lug8#4H zpDy(D+RG%0QTdcFF{U6MhHta92)MbNr_r1x+_sxpLTzH@o;^(ob9tNfj_?vUxqNkC zuXaw+DFnxIWE>J$@xsL9h^>L_j1`Xj%Kqn_1bmVUI5BQ05$TsraJuIMLMAxw}-< z&l@Q*u&b33sS8so6ywdC@)vs7O*tl9pt=di+kOg5$+$4l(p{`)94#OJPh2Po2ao4X z@^ubDQ4$WJSSranE|h#A-P|%b;CE)VhoGn7Mz|W}~%a)EzhucpfpOd`y$YkaHUfK=nGQkHQJl0$PhLD`esHc$s!t|eh z9Mr|cT}vp|IRK;ee^@|g0Oi;mB=kIJr~A`-Vn|UMM8@>Pk(m~W@%nFC)~;tFlTJn| zK;9>mF@_O@+07g!ii99A@q06pQ%}VO8ux zf%`(rWFb{zhotaPVRLT|-W(_%^u^s=&rR7MX;f~OXkJku3AUpxtk9^49MfY`gXV9hC z&1wvxVV#}vMbx)zeoEtWr~UvIIy7jk-Ao^1VVd=cMj|wV>1IISM5vk8?mxp7XmQqSPvEPMtz# zFZ~HRf!9jf7!JUjgh~A1Ki_%Phgn$O_?xZ3Y_fqJW6dL{zDWX^ zn{xyG;9DVbQ;&5g6^cC^tE}#Fmw53UYEF%#Rnz&MHctSxSPEG-R3I7`Vs0z)`L76e zCeKdanhNI=H_C&6Iv)2B3b6$ES!hvHGIiD@9UUBh#b4yF(hIE-433Zrfb?t$($78E*i;m8Scy(C7 z_AMp7okI^?^zvFE5u}iNE!n17DNcrR@~G#G7Y!cfR3R~;mxghvu(9aQf$h|Z(0jS| z^+n?i&%cS#Ly%`C4aEsJP|5A zB7j_-Qn3_1P4`}oS-@LB5a2K@9n>8x>dC?q_&{O5o!TpN8i$t5LgXSO-OBEczJf;hH$CHWx|{F;r30 zKi0|yqaZ%|M3Fhxsr!!oC;|&eJtCCBCSNiCCv%pB%Y%3ceL*flt|F|WjEZ==(0Td- zB89bs8;Wl}&lP@Me8-P*CP813Cx$44r+6)}6N1^KTP8nmuP{J|WY^E)42RL!0lFvQ zY`~a;8|{@q5fbF5#Z)Gs#sn#-*X(PNi_aEcYSt&nCLZ%Cvao}jfE}S=M7SgPfyNs((vRbV zYim^j!`~m$#Ab#O&EEeu%-&diqqYyfP{uTWfLuZsccuY**!sv7%VmYAqVX~9>KC^9 z(b7ejQ;NTtzw8XH(}*bIRxO(pzNc(E=RyS2K*$Xha16wW0M{ka45E1Pf1{lP5DsK~ zsVWq|-p5b)FOp(Y*RdQiP^#mcnOI{=@#b&UF9*Q5b zSCP0?BT+9kIXVuM@5j>*$w|^;!Fvd^xeh)+dP& zx(I9X1xb@@zF-rNkV_b|aOC{z2x=9|7}*=$FVV4d{gf38@MWiqaG{QZYH@+59iph3 zviyyV=`r>#vhXwlV7Ej3BR2`Hi!_@>C;9Y!zKgD-x+6m9P%z%X7}r*c(&cBtZMVGr zLZ16i?0%VY38okva=4Q5KsufsCTRW6XSDm}T|4Ms767|a`hST!@Y0yPKsci0IVFI+ zkG{AUJeOrRzyta6bd&z12EW3F|GcWI{LHvxMnT`u#Wj?Vdb3B#27dzz6Qd8)$IC9q zpiw-a)w%gddKLX*n{sXFlJ*oY(-~NDdpo4=SeBTd;q7+XM}=KMyOeD8)jdD0Eb+;g z_&x*uq@ScB1;w+|Fq+T@r9NOL+l7(-S#+`*Ks<8zP>bM^K@!2ja7ov$0DCcnT>xf$ z2|?!jZ}@lw+Hm zm9l6eL@x*{-kt1 z?0UX0RHimwQ8pk#qHtj*A- zQXRB*_k=UH@i^cmDv6LFp8Z&jv3~b&;pOi`7z%=cpO?gk<2q1-L$Z`gg2p-H+mL5Q zvBMw2FNN1Tf(Ua0oP!uE4gfkTHC!tI5YE6Ijnu1ydNRyNDSd5S zDW78)dGVZ*=>-RJn%<$WQ?6bIO=yo-D9HKp%n@;pSxC%~OGTya=a8pGMMD$|iDD%5 zdVUhPO{du{1MOCGqUa;6X8Un`(w1-P8v%;nq?Kc0=EYv-+?8` z)2b0n7iGa(cJa5N(NKwk?lg4hCg|Qfx}rdnBwNryCo>XyoI# zt)=MY!#f6xgS&$?-ZDJ+%cZ+iwN!!ko2fvvI9=F|_AG6dyp3=9FL?rtgZmhXel!7W z22IsN(wnG*exEwh>DVuDEvH%grY$)^=EC?)FkQ6K2_Uv---(|thTHaE>GwIJ0F&$f zCK_lrlg6{rmydm$# z2m+bAyfV3aO)_lO$CB;yqUoS!{-YMhXds(P55@#U3BiaBAUUzxOG6 zI7zqn`MC^?GQyDgf~X%%{whKUG?29eVBajewrwakF zvEu`4&qkRE;I27petI}kVVjPca|rB=krQ?4tbOEdcP!x%r7zw9#0=}ecb*TIvThAe z(@4cgq5_K8#_RsX;86XWfCwp66ZWITdS+5sw=3Os!B4e~&QstPooS3_XS>sBih;3Q z>4CrCjpSNpCis8pa2jy}PUtugM4|T>nDDJBTW^v}9ASr>dz{nhp2?aL1~y#q*8-Ze zBFpXH&I(9+fEt|On^A6xEnJ)N^Lz0SIsstjAE_LNWi^Ixe?4R^pw3Xg&DxOWd$cn1 zSid!%1DiC4LDeU-DbngO6e^g0W_$RmZ&B)Oj){$Bw_a0>EHbzf>SM3PMBk%jx(s}4 zd$?NGkum0EMH9}BCNwj|v`1m%O4ekYvseBl{{0!6VPrR$m#7&%M#2z1IS{mDMwKwn zY=aW$*fToPNzFq*y}B})-kNqde;<$AvF=<;~J)Z|vHQ2=vw z=JBvVs{?*DR8x5*@uw{+QRgg=?}iNfP`W@%f*rHMYa{2#u}E84z~Gk1A{9?4uEp0& zVVN+k`V!UbrJYO(>M37{nlJ}B-Ed;yP{RpxvP9N|M&$*~I>B@sw%-Z9V*I3f3sh($!{?YUHz$I^3d^fz>t55_Ve>`!jGMOacH3;4i6g3c-YN zmfWFWHe$B1?BzOCSLc2XGS+U!er_cZ5ECYCKU_3^uY*y6Tmmi6pt7?uOF$koE1iLX zfxisCpDV^qeu^}4``^qG+<`=j-cw@G&_1%jx*tGCE{q{$?~G+Mq&TCrP$M5Go|5vh zGsYR2RN3YDuDOJZ-M{ynnmEUmykxgw^1~bl^urjl%8YRx@cw_xz@($gU+w|0>E~zFi`ip3 z_Bs(VWW_o0LPJM^>OmAzCST54Au^T)WALB>HB!Hy2$ygS>BJ-g{$3&^lcq?{Pj=<7 zr=phsW3p5hq-F10rO1l_%4HdrBzHYDO)g*7yXiS{xZKkit7G)YmfaB75tOf~QSdn~ z4Szd#N@2jmApRzm#FJS7s_>Jt9E+ceHR~((?~6KWfA||gxF66fD39M0zk>-A_Odak z=*gf~{s_silZCt!ME3(#E`5NI9nbCed_>x)e`iq@jJ{P4P<8*34eL({mHB@q?ba01 zj0m^gANfC}8R~NYWN%=w@g+@uisJq{ybzN~RI*+xM84vE{gs|mtr_gc=G~VnI>ZOBs zfC!6n4XF?G4klm{Ach{A_|C%{YO|wytA6lC7L3tgwjGxp2o55p;PQ?;_$aM3TP+$q zF21hqc2F%GS1DBYb@HeF-YI}2q8pe*%Nplk(=?78(?AJC2zKXC!A)4Dgi#V}W!d{cKY#b!=WzgH^?2yCMG}$y zAhXdpC8iMku+aBfq^hbOOCsYh{)A&!K)_UCnEE3NfMOS0B(2Q@SpBl=@yCbH>xoBy zZruk_uSGlBA4Tu0T>Gs(W;vQq!t~GFaAXlgiTOg!_Dp7RtIM($kTn{@wg|>Jip&UV zcn8k^eEeU}*EwFhP+U5|Ux-DCOwi&2M1q6s5%CGt+rR26VCU?%m}^5vL^BjXBm;>` znI8g~YBO=ePRHeVicRB7TOM!ysnaboR=9@jH`QktP2x=FEoB_<&(a;iD0{$ z-_!@PSxMQmOlqM-$ltyw0P`hY?!FU%6mB@zql~jOy#`Ba_fZl=Kd3*??Ef~j zHO)k&B1`-!s#}H51n&-D><0dlaY)wkETkF#^OgYXV>7|M`ctep%qE`} z8x4UhG5Ei@yMP?{CiQL4Vm}HN+{?hdIvId(rJ4#X-iuWNZ2*p^)--#n+qq?SlSxxv--nqluov;bQ9?>Eq8L%&`hl{DV^{5pWHvijv3KpRu> z%l1X(F0k-QFmy2)JUv_?5j^$;ZiSyfhF$6k&|5-&*-{3~7#aoSYQR2{3>6P@0)bZ{ zAbzYe*gC^2`%AIszc^S{$m_o64HAl7bQAl{0zh;q1h7B#bAVXHlgiWfB4^U;r%(yt z9{Psx(qY~(v4bKJ5PkVG(IN>)!ZhZ`HAKoD|2(s*-{B6-W|uw-aaYWm%+QVKe+MDs z!T5O!KLLYYpV^t+u2%zTlU<*^NSEVHKL5p!3KvU`@wYkkD1s3JkJn%0wH$wdFDui( zJXahC7AC8|{SyuZwQ7>1dr*Xp6_zyoG>xp31p~4x$^b$_f|)!|-kyo3W>g$Mb}|r9 zYF^Tm;OmeFjD&$)SDxr2MY2VOl$cs@XbQu>#}dfKUV=h%q~d;4mJP;X?E~a;ZFU!- z^8>(po7r>ib+?iW+7F8x0kW%%>fa;)1!%ReR)St|+y3i}$)#222_`oH6G(f}^Xb8# zO(JJrvQ|a!ii~f^M(qHbt3nPAJZHu%^u!S!D|d9=&V59{xjYV7-hC8duk__am6QpyZ)h6_bdxpMX! zF%0XO&c6W3sa5r&urF=2xekh8v&xF;j)$HzKy(4T>2;c+c~Q!E4evHh9D zq?FG^%H4QCeMhtPyB_8LE7*i`!myuJrUwj_6@#C(!-My0ZC2YWMi894$p;pQk7gkq zNqzEg0WjW0>mpHM|M+j<9b^=OLf_~Cqke?4({E{&KeM}MX@Of`BE+DiDyr^&o&7j+ z8|qad+qj^3_M9RhA6gnV0;m3JlA*=Tq2r$~I6vs)EasAiI{yd~AQYJ+|F5&xU!XYj- z0MRut@-p-lmbM+QvD4`59q<Sr1)$z!WKU&5BTlJ5)8`MpV$ z??7{j`g8WS5nyf!@D-Mw*TW{Cof&1+vt+D(V@m4jpR@iyLq`)BI5h1GUONEWf+4A7 zgzrl8!k^JSa}Ll_9x(P3I-P9mTMlZ?Ajz;uMs@{ULa-%xgX_S~SYb-dB$Z3;quf9W z{4%f+t0mL^Ja?FTSWcY$e?E)IW-BlpwphLTC->9+nKZHC z#U!B0s7X51GrwIKjxLa3-^X7V{_#-`E&y=H8iXUR{JbtnG|#1uzmIQ&cWYqds?_)c zL`cpwe*Vo#W%`2!>EJW~Y@1;C>I!dsPbuaVX+7BqI3ra`-x?rR(~p0v#}|LtLHuYl z%2KQH;(#Ah8G=;ElVG91;j5WG;A}ke&$sRf^lCndRnew#95~Pt*?iWDeUIrEFUSYZ+HJ4VMJ>_as~_biA2L$i^`$?G3TUXfxFp}V6`~y!1PC| zf1C&rj3IyCRZ&9&KqwgSs{ot!eSbFzK&!OaJ<^M;MH5bX#3wB>$fy3kJB~>+tXcq9 zkej9?m{??q!w<(I_Qzbwej|EEkq)Ay(ESjjwyIkV@c(o_k6r`|p_6V*I|kzuyul zjbqf&R>n#y1Efuq3IScH7^#JwUXLZQ4APP3u(H*LGy^O_7_ryA2+y5=%CcFh`@ChOa zW{&`~6AQ&&`jG#se&nq%MtuUzXi>jBc;N+HBxovf{*86iMoBnwYEXg0yzpE{rv7Z}SoGxW3Js#VzcD^FV~kMxar#xY zJrF>YUbTv*Sj{(CHguV9AsCS7A(4GD)dl98K%{LSdNlL847W9PNE zkOawKIWG{E=gL^uL*t0)@U1`2>Aw(mGXe4g#%SfLwJx>AjfwF;cHb(4@tZi{;FRa6 zsnWT!hsfdMMknZkQQWOM{PU-Fj>5^^~M3D{& zi9xzkq(xe~Q{sP(d++b>d0wl>;mm!{b**)-&v~w|AgCnLwcdxDQ0RB==o(LnIE6tf z_XhY0a42IMwX)8+~Y&zQd~fVz^~))`iqrUmvWk{ z1On(6`J_3V>!wLIlReaX`KCwzT!T!QZ?ZYc++H}6q}74gl0>DQB!llmcc0yFeM2jH z)ovTt7U3-qy##B#7|{g?Z1d}*Z8G>C&^Irda6iR1Z~4{jDBS&!b1573qiC6h=TaLs zT!QttpHPW&5w3z7EL9b+s0t&lc?KQChfrUX2;}G~iEQdL(maTOkCDg=6qEX#PZ;|J z-eC8PsDxkU_nYJtfK>{~&s!2h7O6xJIovTW+zLom+Yn|E44D%lm;OtBFVTBW+|Apr-d~eSkIXk9jCR$1s4|c1feH6a zxxSc-{>5nlm-%om)#wFWru0=q4QdaSqiVplsq~w~zYj}|HkX}#Z4Ue+&{xS&+P9#M zBQmVHn6*QZC)2=pdltL6TibkGvH=iu(nnidoZ&5ua08Kc-8EE21M>LinT_3aqME!Irm2)S_4`vm?9w=SQGRz7tjc6RVCQl{2|ou7r|nF-G2`grX4&wv<%D_S z)UC7v_OHO()=S^-I%A`y6f`BxdRUcqo?7u2*)yQSe3Q&E;UH*HF%5+YhJKP}ysJ=W z>!RB#*^fLLee#e+h7TFC)O=_zc9@3yPSNTe`E)gwBT$|^wRc>=R#uXOOTbGO#t5#a zR&J?gc0SR+YCeUQqLdHXS1!K|xs2ZjCJccR(zNBSb3Qj68P42MBe(Vkw&L2Ko)336 z3NEBJMPB`PgK~g^F=)9F#Ctb-{tCd3m<!i&QpVuqK-8mEg~N_S z%_pwmxniJ=nlZh6BXXLt=_+c@2cpj0ib1B#Q_a#zyM=!4Ar>z-3UCvJDCO|m>BYy- z%&<8uY%J9}!f%f51)O-tQEQ$()#Hk32HD<4Abcx1Nu_s0^QF!6161aa%_Z)yX7sdx zzgiN91f<`Vhw~QBcs%VgzBh}HbgJ9SHeKupQa?~~_eLF{COAAeeDmIiqv50I6NNf4 zA;UjSQ(isp(!THgo(RkGs(JIG%6*4&8aJ5|{H=k%B__mqopMN4q$7!v7hQW&yJ0bc zIU>##=pI_AMkZX^s}AORrtkNYYVE|W(~D}i!{f8B>%ma%_GugP;_Bp7=vAcZY?&YV zDZF`msNg`PIcr<{(!v~$xBT&-WlQa&zyM$IlrOadZmaIAe3(M+GpzrU#>QkN#4VEM z4p#c`ELAuS!QN)s2Jf%F(WheP>H;yS8-ha@+=3ed+TR!4YAGM2UOuth_>>p(s3ikJ`6jFdk5DWA^9>NyXKKCiXOn3gT5w^@xpLtT%dDu zUT%z8vzU==rQdyueH(1WD@1KFc@3N-vWD!v9#U=`?|+*7&6s(%q?NMV29keDi9i_Z0`ofjp{mgAIE%ex^TI23#K$Y z=#%4m$4fmlGjth@Hsirl(H@JyK#QRU3l8+tlya}g7dqQ5K5PIyA7x7XZ} zSH1vecN3b+>1{f3TND~RF25?B-%TvNI6Sdy@L;u0pjL1?k7e1FJ>N0QJP(=WzeQvJ zIfJuj45!aFH0%0b-4P!fJ&cv86O~Xk!p5+qMkgT@Z(}y|@ps>Gp$SV-uH(b+_K6uv zvUF~m9>N8?!$^x zn=H6t&(PwG*x>MwFAXPd(>ZI$yHJZRHv7n`+9Tf1mx-UzZ{~MY*g{C zr$!*dj0a*()!`he@(la4G$NO?1XeYQWqwKIm;XLyGI#h3@Yf+67`_#1T>2E{eH<{T zdSD3r>{@3K?<`Mf+8!MRCK|K1Ab0s^8%h zF8*p3aG&tGexr5|3^MaDk1ts ziO)wP=tc|!1}=`>wbLDuH_{aPWc0(*PM8?^JmVo!u!USzcnfijMM_tcgUX5nX5BaWKlCTl;z^40n+}*k!N=~a1s_3!WMGQV2{rqUN-{>0;bcEAaRJABq^Ef$qVeQ zx`NX&$7dY)Zf3!GOx&Xk zUtqbx`zhc59d{bCG#BDNBHZpKB3GASk?Rvbk>`F0PgOtJK_fX;zvo)GslOY%Q4_16 zApeS6Ptz$%UaiZEPSilK_XkQnqBI!Y)yt*&$*f3KNxevsQwpO3PcuwZ+1gn&a*SCW zH7}#lqIjc-C@*7snobDWu5r3Lr4w>JbP;rEaT~32@)^s5jguRw#4bl$ytB|f?|Dd7yQXz zjVy3(P7>PUo5&=f~}J@adIR93qToQs#@du)E-MCsYa8D!%#YfGY9%2JZ%m0ycg#WD4E|a z7^~fgJyM-wcOjZOVuo1|qB}hwg6kTS_9}LIcA9nab_R4(^Cj@CzEd;FH0fV%e$Ly; z-D$b}da191#KZP>Rnowv<4`JI{En#c19L9-sBqz)rGcI_@u;3>(+f0hSxf>C5;_FK z9E^Et|9WI-CDpd47ez=}a2R0+5r_+N;p2dO|M$-oKwh#e(aBD1F z$K;x?FE)$7G^V`K&LhRm1}dF*E4)~`Rx)akY)W}D!XEB}^;Y-Qr{Q>zq_hPwq8GTq zQROLZ%Y;a_iniX94C)MYr#%XBnKDhe_mr$6T0kDNYJiOr6Wm=UPZF$JJ5cNp#16BO zV>4K=*g|7C|JMZYN9#eX+WC?B(W`agm^v#c`f{Y3hQ77Ot(xVGX|CDnq@+56dv-Y} z96j|cdXFTT!{-pJIv_SopzY+45R#P?cU(H#jz*p}qA8yq_!rJ@=&& z?H4BTDn$-FNe9(cvXixweMfD7Soh46+NtrAB&B=f^A0vX5J{U|K&H}}_& zc6T3*+Dlbb|8B4YOD*!Gsw~-$Xd}fBh>44?cwkClEN4#7PGAvQfkR-?D?Tl@R7~o& z*A+nH7E2e;`Oq)gG|CvdQ9T&&8LGK>7SdMph_H4|37#K6L*)xb@u_o?`38c+QcY@VGtJH-N4%3}*ct@bU|j_;qhh+#9~z&kMluQyOgN zMInvMjT%Bi5?)z`DrJizL5}{?rh);W}p)rjv2p+*d zr8^LUuvgO~Sc#YPjO$0|+rM(Wnjcz!$|c-~+&_h`G(R+z(MS*&q38(?iAj~6?hwNzrX^Nm=#pfFYH9oiVSBklzC+~6+xbrQP9{QR zv@%6&*nBH5AI}L-r{nw!`~d5%ww%4l&Myv6%Q7nhjubH%C_7s_y~FHZw0v`kVvUAR z$I`S%r%`fZ@v?mg5&!X2BB6P>o1ZO_57Q&c%@7^WIDzzu9LSE>5;*GBcW=ci=rrgw z9!<7=4QXNDfN#l7G0FF;CFV(%5IH4f8nT2Bve8RqsNr@z@wHN~p+mIiH#uA?H1-g!sp z2Gj-BELt%FxD3r@^G?V)4>n~9PKf;S zteq=URwp*dDdQnn@`u08IQmUgK7 zx>FfK=A9MsH(Tp!VL>Ta3}-G9kYJa^Psj}QO8|w?{9Xxh}z8Ul~Mc3P<3k* zv7_R(N#4!)oJ*=vTH%xyBqR;B=F4yMt1Qrw$I7_OmNY6?JCa~+|zSq0X z^RB&e;FsgJmGc(9<@gs^BkP*FV_|Y(H?)LIMg}m z`w%|gUkJ&$MReWnHp)H_`~bti%re!37Pv91Keg*i-*s^QQ6UVma-l7iS-m=I-whvJ zWcNOo4NGLZj?D9RM!0_)_6mMPUs|n@v1?2x**y9*_n5@Y0^;&9%T;G$`?E@7fuslG zM#?_cahXMY9PdS_y_G?WSV=gs%Lk3@JCDC|Xa6fwb4yDgB{%ImBJNSADvq%abM1a8 z>SJxfXShPqR`t@kg~yGedXm}X80v_Hs0L_Tr^>YAngDbt%+#D-NER`n;AGq(;JsGM znc^002;H96UFHFR<4lRkP7-{L9GjFJi_cH7y&i`Ke< zg$I`uofHly4KauANU+V&_kwv~)ck3P;`J{wDwUG1#n6WSJ7skJ{GEp3}axNz*_ zoYE~dO924g{6@@{k41AZv_P@m#~k{4ls(Nd2mf;NQDYCYx%l7kT zNsqz-c{P1?rimTJA(u#0uwi+S?f!jfn<_9}v!+e~uB61l!Q56OD7;j?ZTWU{630EB z1f6=@xo-bHADKeDOXr z)xeM{R1V@-*Y1jmr-2XzMW-AVnh)1hh_yclgZYc84@0GWDsz}J$ozRX2>c?$_D#Gm zY7jC@2X)b9iJFUIO5d0^eKj#H=inN{_o$O2K8BP?P*`C7M6cS(7yd1}%qa6y7O$^N zb{BuAzz*%i$Bkgv!HvnhyQ&v?1a~~b+?aETZwREJ(bpYyEi^%xcj`}O1k3k@XWdVi z&N(taP>1@ba3m+k?%bwoi|ed^jL{+NRTx~9Qc}l{zowAlQ16(3`OB=5VnEEzFmfWy zk@&^X&$T1N8VEkGA=R&=yq0xqs269DYD-uQB)~dPAxI>jTA7hMKFlC^0hN_u?Pg`& z_{5(U3e`CaVl9FA7_NWa!4J&Q@{5ir6$y=O!wC$^NrVfnTX~!@*t@!5r^u`&7DKexb0_#t(mJ+%poXSmZv)?am!gC_Up2Y)fZJBw?$4T@Cl-5T!D%>VZ5 zwjd@|5j$TtdabNKzFq!?){lz-%%o{^*%GdLQUTJU{Q&-Ht&=tASr@$2A^DE)#LVXl z?o;9Vx*=k^HJt|Xc2DoIbv+R)@k99Ky--h)>3JI@pNf6(VTsk!z}`pS(ol^Xgs*Vd zpKYY@s|J4S!PN23$D}tC`=mA1CvZ{q)x;;4os0seSQC-~(Qj$+!Bb^hldqr{Ags9d z6+H^}KtAHQd+`mMnq`@fE-9JC@!t7o4X&3a@NTfDp%R4*F;A~no4~5VtWO%I9^Gl3 zqJd8J67qfX7gsPIvdsU(Dk6oyZT4jUM>QYJR6YVBRGF4={V5nLMJbPlGX8D1={01= z%WMNS@e{AaZB5}mGX6}`5q=;Vse)b~ex&!EE03Qi`|Sj`owviDPA;?S#?Tz1B+p za-zni!l5?<6sRZpSg%~mx{zbLo(mjVQbd9Fd-$yGB-88wmBqnB3328r^l?J5B;V-7 zaooD4WNPfkT0_ zRzG@|KT>p(AO0zX$(inV0m0MiDX^CBCm?j~nWH|ltbuj_=7KuI9sEAk;jM3AqPU8a=td^}T zcxbXjr77sLVU?G74W2yq=L_|J7IK{TRZ}nR+;$$Aj2_9+JMGYh5a2GQQ)ix@#j25z zpwDWDe(dt2lJ>&=^GH!j3DaPdWmG}$Tq z+UzWe*A%?iTQ`$)Hr@p&Cf0vN##5adkg}D(sZhk#SFUL!HHq%2PRUifRSc)42? zgWtq$ZoEx$nC?2TJbEw7hoMyWihOq>TpF~M4L{57EN;wu{@4+bUliIXor`rRaVXLM zGeBK>)lkPC!x3wbh)8jx+zXKY=(f2llSx;6H~vSCYw_%w`PG$>?e;?Z>EOZJA|#x! zbjjl?Z9e{|r+Y^N)fKD&2kIc3JeKc95?dhT!oKw<@}pwXi4c)PO7mB|#=Ok4qk<>uVHHwd$VQ2?jgHpDQXV-s-#f9k~4~sQjYIoiG0!smS*y#Mps;SDXO+aBa@;6>X3qF;tr#wm00T3* z%<`b9ym88Wy6qn{JPlc#S;0joUZpd;;a%YF*A^|^o^MDlPyumZk@MU0T~`9N;&#Xr zy;&!!JE}tnPpX9u!Jo04D(1WP4z_+D$F43!^G{`7cua8q6h2;a@q<%2un#&Q@0(jn zl8rT+7j)Vl^9*F{e3z6@)d}wjFKidKk5+JFf4zc$LN);wlL=5n81lasnqO&pou`bZ^ok z?y78l-j7^*|9grzu(PK4U)P~?hfo|*h~SggI8sWQ9M~g0u8u?S1T=c|S8}%&qF4$H z2b+rsMxY~2nb360yL2kuu>j?2qQGJ_WV0-|z91yr;GenX`)~!;=f2{*;R&|~dermz zKR-_+%;?@{N7f}qUM8R_7w}doU{;S$JCF8T$3~ppX5&jXT*wx35dZ%OK)oZ!ke(MZAD6^X$d1CLcDe#;sBV?|BBUMkn(> zN^)+9zwb7RKSd4A``6R(zw|hzPCSnL-xu`X%Si+K{g)4d4?`ouoPyZ-`qM%{1%WSw zu3*;AKB_Bdj?{gyc2VNP|C}X{kZv%Z3S#51Yvv7GvY!IP+mpHbohYA&f|H35d0q0+ zn8GfJy7>6f+JCTX0$f7H*lY`|cewzBFBlah4P^b9v)#;jAG_f;nE+3+Lp^DJXt(E2 z0^+gQ|NdJBbtol#_7~PWQnC+N2K}1LY`@!{$I0{ZR#(AVyXOH&S<9kt?wcRZ%m}67 zL%5Kp(v*$&=|Q0iPNXziF@VZ8nz?%WgpH}Df82-2LU_AcV~WS+S)WS5KY1Alx?DfV z%!<etZ66s0moiguSe9}A8xQg(5gL(%O;Bm*^j!_5E|NDP^hmzv7QZUs5 zZCf)|^yI}cATWu??(v~36O5@d@3bZ81&WUPyYeMSYpT=SOSH>)$rsH5a;*icIQ!{n zMZbU3V=IwQ_gjWj7uNjr+}r@%7_-b3a4i<0OtebxX$RK^GPH1I-_o{e@N+!`KLU

mz@o-R0PF)}F&-z&I_3>RkSu5;TUk7QJm&@=bT7_FiribV1&4OFADCyV zvjQEWn4=!dGCu?O;w$v530L?EHNgj8Un&*)1?y@D%Hg;HxUxrPN3LLDna{o>_J3Ib zuL1k}sB-qmJ7aDptQySVN-t_b1n`FN{JBc_(3W?98PJ4t6(JUOskn_`vsRJcd5T}* zHOL02zqC!ioTcyQV*Br~YGI@WbLIMenbZ+_*-Y|FngN3DI^qbRE}Bcgat-wu`X`FD zHFy@%lPQj)3Q5WK!1^%O1g)Guvuls%wp?_TSzee{0&I+!Rd%_az<&rpQ@xHifMzI~ zP*=G5SDP;I_itW2p3pBtp}4P)H^xnmzoYbW#EU;4w5dj9Qa5Xhqejge0rt6~-WwgD zmQ94hG<|BgG$?DCM60rNe;oe*D0z;Tm~W=(pWv6K`jC>7Gt*~1bW;#M$xf>Xb2Gy; z@y&q%B8=x#8lR*XH)s27_=gYuIW%J#i$)T65ymVfwbifYw&W>^-{5e&+Q(DbdtUB$ z75%*r)~LY-Cd>DQOd$`KP?QWj+}}5lt4-DKV(c|=bpaRv?090%;ITrA?#la(5H{OW zLXQ5$0={qZPYG_G?;hM|OQ_1<#4AO_dE!> zj$omY?RR=gC8dF?du{{ z3QCzE!}K2w2e;y)ZK!>YMx>OmY3G97N6DhbDaozNB~iYFB(%rKB@&Y>m^_7gbw2iTg5mb@otmFXON}v(T}X9R?`~p4&QlqMa}=r zd2lg^5zbo)f1LJ>SOPIYASH;zP{!9XL(umD#`skxqfK)fuGmj7^InDNr(b)an$Wa{PjuA{~8#arCM4dw;g?MNAKjt z2z{PplYb+lKf2zaH#f!~PkF436{^hT^rML9BQm_@6K$>!Cm;U6fn?Fzw(+D2*OwE# z`sgz`XAPr$;lchyI%FWG@$|@-HQyPpZeKzQzv^)puU!o$?vXcc5&9o@OVU4dzh@5< z;jCZC#z?ch4FV-JA?{uLguC&_O-pXNUJRET#06es(QEG4Jjc!ZlPf9udRd@MkabY- z?LKt|0_vXyj**iuWj-ZBfFd?u9l7Q6p>gtNbK0t`1hw|8a#txiZ^|^6E8pgQ%jt2M zUsE@7+vd-fRCk$qP|GmUYK!_W=X8Zintsa$u0A6Yd0J&~Z_}X*F@`=|>g8XEHvl0* z@51J^hSqvvcf@fosl zA=*&rXS|GmR6$N|f4Nadgi(VxfZo`qSDY*xM7Y{$5G(Bx@S*S)z;FA87E`Phd~g(@ z<6n$y$CaA3GvR-m5iVCy8s$MbB!1%^-!o_Y2)6&>>bQi%JCgWRc-m|T?rpw%Fw>yn zvhUOGR=K;3^M2+`wAn!}xPjV(WZOyPP#0;aYa+|s>Vtw=%~$VJW|IeDzS|+k7N;xr zyeuF;y{J{bU_~x};A)}&Kv1)J#>Z3|1iin4UNBhUVBbE|q+-gHeP=^(Bam+q=zhhF z>zvq;ex#j8CvdQ1=aZ_OTwR+^44qGqBNa}ouGw!jm(2chj+cgQ;^a@(W#Y|IfUT?q zw(KwUWSp=t%f9LO@BP*NAI64nz`BV?E`;GX;i}&6$*f?15T92ojqmYA6KD94ag>|; z0)XK;DjDdTLnLcW>e$(UX}0M zwEY0UP`D7DrWpCl>d>%z!2$;ft^%#SW+%&jl_w|@UWsMNZ<#R9u7KrIzcniH*`>2x z32)jorql<*@Dxkugf!#D;ncR&=_)-zV{A<1k)qQP(TS9lN(6tU{#R*Xu?yUNd@hQ5 zHs#T~?R8$7#AT)bG7k-45yxZ%PGaIgCdMFd=??a_~zIZYighe)d-H(%u?-O3ZVx z9~B8Xz#Ii9<95)8vN?r6N>?!ap zc6x;7IZEfe{0S=4zQ@!=#ZKdX%2bEm@V`az4qa~3(#Ddy4L#RjfQTFv-bP7`%4pS*MzHIwDmRr-crP4vqW@qJ-y?!urxk-wV}tk3 zmcq37mzZ_F_C=dZZtN$nz98aOS9&9t3Fle}RN!~G zCGR?swSXlp^R)-NW)il}b2t}zTUUv(Wnd~mm?O=~+ThFAI@)2cXlbvD1|Qa(Bs3MB zQNRx$bjmjE3-{*E=Pjd0GRl`$Q7D&Twy3;(TUrl^!Zf7X%TPwh{N9?><*Mq($H4gG zB>PBi_8=3|kA;>+gcd6oEPdX*lKe8Fv;IKrzRE|%)Y|YFls$KY%5d!IP=Ry7B8YPkVKZ8& zO{SBD0^9*rCyXL=2$Z>}CBNZKJ+b9A``~XhUhTkokbICbZ}_pnRwtNZQ+Q2)26lQy zf^92waPIt*HCy#iwY~6KJF7ePPk`~&bd!L=CcmR70uJ3!aL!1l8-lGVwa^SKG_Cru z`pSIWMU{k=xQ`ICfjuWfnoQ({@K(*KEt?!2E|Oo|Sl{-kWh7OvvjQ z>ndgK-a(RL9NH5D%dDXgIexT+A8! z_pd?lq-@eVmW>sNjx^KsFr}#dkVxjHT^tr075R>sc4Fuc8m2OLBlqb0AwXbgd@I!T zG1;_YkwwFzu4;x?)(*&(j-1zs*7$h3aViwQg&;1r2=t5ljj(Uq!b6|Kok}e?*m?T% z7h$8Zt_#oB_jOJ&jG3b@@MIsfM%!wZF3gRqVFo zRmiv+Hif-}adn^Z=xGFVDWh2jswtMz;VryLC!0Rk1RbHXl*B8u3rv{j_;am$i_b*O zlgm=L9t9@X$Du+4P!^3RWG({v4~FvIm2x;8C~x`cgW zbu#X)6Y6eE1cY$kH}TYM2qogsQ|F`L`J-$=O3uN_G|xDxrXrO;CT1QqnHum^Z}K*O z&8N_T-T#~^VD*3kY*mOAI|16y9*kA0!dBg%KuTVWo>_4<51EX56K$_U7#7C3ga}Wi zS`URg(~()2(MVaO5z$z7y?8XubALM!hns8<^Vi{!IFcpw!?8uKk^kF&DbQSD$ZrM~ z9~3|SL_8q(4EQfzDBwE2$ZGtp5;k^vyU?`0Ib7d|3H^yU!d5X*iF)Xg4rW1zzWh9Y z4P06;m1MR7Ru{st`&{%sRmlp7qf6pQLzC&G`&M$-^lKcYqlw^Dmi6Nx5x$Hl_mC}I z4djw9LsV7Mh3KOI*~}$HU2K3?GJSi1!Qlb-lfuYi;w*A!nn$@M8JTFmA6xRzdpo&+ zfV!llqA)k#&>NT@=o0_F^H>-tr!V|T|Jmr*FKC})2n}N<#~xH-q-5!USTji}iF#Qc z8mh!?V7W~^u$eHW|0*egL%ttlOrw;=jsG|{Z`!$b^$#1j&n zDC=8veU^Q&VeS zHexj&o)=keTepAIlF`vPWLipsM4_W^xP7-Vm;Zil`yt0q!9%A9w4Hh`>mO1yQRZt} z-JXojyhLtj1ulp@Z4f$T#N%{IhDrc5I-z zllsK{#FFLNjqkzm;eWCcf`;R5u;91boMwDEajhf|p293`55+0u~byc zOl%Ml5u!p@D_p2h65~IZsVC$&+(LV_)p!HbQl{ttMXIEGDZg!k(rH>x&9pvZWhSbp z7cEFxJ9ni60w-9%hywW(Y|Qw-@7qo|6+KIJMX{O@S&ojYs4Z4f`A^M8e#S|{X9f5l zG^odBT#lM+*T-maEPPuPAwRpXYcJ_Cz&klh}qt(Svn) zP$LQ)vtM%l^{5;R*4o+1$q^po+5CW)UY#p0bQ~~rOn#k_ND|gZedx*%ZS>bThGke7 z8+{pe&DuPDIN{bv9it&O8zP&O*wCDBL;(}Q7~ArE1D0uUB`FNYM!Q4be-yBmGYCyT zN=>G7qK*CQgB0vfvo4^vjF2{1=T-%yJ)_#>bxgMK;(}$hPB<);5`Cl6Q-!U7qvOGi zOGC^UTWuLs`vBmqtic<=Os}bQi|%C*(01Eh!=0z&2;8QL9sE$>MNzO<4wMRD}M)uzO^d6M4-J zs0~;6U#^0r#8Ob|HKEJK(8_&~F%bg4VFOYqsU>h?$r*=T3A9s{1fq;CN%R_(`T<^o0imY<$CwXbD&ckBqkkFMr zUAl3HVlfqU8Gxjj;|8s5s0X=rFH0Y=T|aLEiIVryM3(mi?l4s5C4BCdH$N{%Rxy3@ zb^~rdzq#)4g^`|jN|t}$oiTZXXV;D7Q^7ZX6y>cD$u86C_7Q=I|q)13Lq6Q zSFzBRF-p=vvTUJI#>ndkl*j4f+Z|su>!>lF9Gm(RQOUTHBajx+U{O9cD0qFg#C#l{ z?4_X?7w-75Jde_@?WvBubtU_&uhr&c7epNlIsaIax`6=}hQ+62K9p2bBWiL27V}BT z4b`x3@bU#s#vYR;fI%fG0`vR6-h|UO#aPG*$=Az+zQ`VNvbE0l*}P4Z6a|+%vO0;> zk9iR?TPx1MNlsG55?_1q?f-iTPjlD}(LJ$+8E^HB+$wM2^;+G@ap-)o^58B9W||(} zv+e!9O&J3Ogz;mr{mBuS2u2R0i-U}*2>)x~#4mSO3=-n2cQF<+ed(j7zlF93tj=}T zERPydEF<)2%cn>>Xv2cI6DzkuGD~>_W!{s@T|2mx9GHqi%^X30>~s5WK+PNqL$*Gz z+<*Pl1;~c!n#i#sck>s2rCrd_V4ef~1q!dQ7hMIN#1Dl&!) zhlP1Zh>Uij)Id|E_REJ<3xyxEA0gbqlJuN^l_H)80pCokzJ`4sBs@2jj`@9N@wT$# zvQPiI5*3;_)e=PcWGF9zIK&+pIigs0H=vX@*PG6l)k8j}LC*Fip&wBCNETug--A>H zAA?%XUbiOe+WEYP--;s#WJFV$_ zl2y!KPrEFK$1?W+rkLc;KxF*B6(@-Wv6!w9{wbA&^43#EU*BOo+M)jZ_D)mVi zh>+-e%e=y?O8?6(BGm=)PJZIM&enYTlIi5L>kyl_?f~S!$4z3L$*x;h{+@TU>1&bO z`{O?#T-*W|y_bJXnomok+3zTi`rg70&{J4yx{~acJAlh^RGnD_Q`&T{z{dKL9c3I) zqnSrJKebT#Gqim+W;`s@6Qh*m{#-pqIIK?S(>Ru{QI(N3Lk~2;Ij1g7(|6S_SZb52 zehy50i9K>Ut0-4;S_{F$;BcSRKOfV&oN}-q1j*!1K#uaV6FNZWn^4)M`uqx6`R)>U zMkADbP}S=wD#C|dW;6Mvt+vU5>j4EJt=BmispFY{*EOIx>zpFRRcvy!pk3aCtW*W z0z=pT8$UiUd60A5piPaO%#&YeG~Ml-8)kuz-n%9*b`VeJ4ev^tK}Ft8t|0PJqJ?Z^ zI)ay$5@t=LOZFrAfsUmi2D$6dc&JrreL-fL#}qIK;=zVggtk)+hgr(ZDF_tE{aaVT z!OGmnYS@Xm1WAHiZ(aNv17s*`LE=*sF8YRa?=5ip*6)F!jAES%BiSja4zd!+dxP*R>rGUk7{)J!7Kv=#a?B|7LaD&wLpC{3H3Y!3O@i2^!l4D9zRVttAsiwgEq zIUj;~9$G{*&nq~YX4uCNf4rw4y$7O3SeCzmyF~+w7@N{rElsAA>37FSy9vAr8Ap6B^v!s33n^2wWu^|kjue8~{hGg=$GFt|;o@4h|=b?CX9kFMCHA&x96{IEb}}cj_sdnVbsyil+_{u3S9wfLVeyIUUk&%)5jkE(^MVa_0e>6WDQ}oiYa?|6bBd7O&^c9>M z@F?i!98UBu38#M0jB0~HY{Ew(<3zX~!xdZU`PeNF1q{3r_uZlUymQSG~GPum(X!aRybAW$*_e3X!O zCro8oa3OA6l#GR^D9LVD=zTGK2DZ%isIoe{Gwb3#6@F0yA38^sQn#bd!)eSSb&Ffy ztWvWs@0g>9AQF3s3OuSq`wvJ-hdY&kKFy3iEkk5-sN*H}LfM!dgDA@6HYU~-?Z#um zw?(5}lYbU~u*94%m}z}v=`u+3D+n6?;dGjxo`fM5oA$a*=ME}qlOLv*lSWnzW2m2` zr=FPc9K<>ajX1z9G5Lc}+pa3G5Ny=kMmm;#$Ara!dp%5zAv7cZVeD&ppFDA??9HHJ zZ$#cp+U;R;%OZqT+57p3sw35t0TZykfj}!(gW+alXw!0=7+LqxMm|d8fP;{(;SD+@ zFcV7*gT{@%pP`$9K5ftr zy#CCTv~Pz;b@R_rfn7YxX`du$#)Y?V8vEtLCE!psJscOz*HW%^t_y>_hpd>AI%_D= zWF$LR5?1h1&a^eG2BJ)ZUU{Jk57#D7KgMF=u<2sAr#z~ve((a6jCdVy{-F9)1OWk$ z2zT7C#6pl$N<){+GUm@`uUP;inwDP<{<_Q2!dIx!pma5HG6L0Yba&u<^1&zS9z#%= zB>xgVGm#Y!JxC%s8I9$zG%plV!tSx0-`Jj|=@4B?2lGNAIMn5(sC;Uu2=~6$oEI5= zoojpz*TJ}+1meCW-^;mrkcBDr>+QEF!Kop>Ym<$~-skUjX;WUFf6}MRpc_38mx;xg zDzY2%Tb~Ggmhv9aTb|HSIOcS&U?!^rZ#dp{_`M`835ntGxjtX(drg=@$>&#@_IJ^Ia&F4r0Li*6TcKj0YG*b@+MQ zd%4;Vh%iyT_=9#P32u4SOiK5NGVxj5oF)e+wmZn5 zOW_+gDH&(e-1+U6IZRe%8Mj@v2&%{&OOc9KnIVBikY^F z-0^eO?vlTFW_OL!f?9}Lfe~LhH(ZIQUg#eNjAyP7DQ7tr zjLMBTTv5F=N=Qg!RHI6LFX>+~WdS82dbFnqjtX@23tJtt?%U+0wEbYJk`j`uhP0F# zBh{*bF%r_FZ;Q2KO1-$MeV6j#Fxieg@o@Kv{>4*Ht&m_+UTiHgpXAD$p_@z9iqpp< z{y6OJKmNoO7{iguxYxka=#$0>^RtZ`9d?f5_5~q*bWmgAg5zs(uTv`Pm+Etoe&KRK ztL93H?WD&$#&?ORA)jd27jB`;FFy>eZRBZ}|Nh8$e83MW{#LyBYtplarba!_TQIt% zf6}nWcS*{?B-JIKc~eAp-e*#G;?*;5JRIW~v^4wi(`&nF!npK{+-|y8si67L&PblT zTkQV!i0I9#-E~16I!#t0mDU=0F3+!N0htLSE3%*{bxuOG5JBF_$HBGLI0h~C`-~J{ z{d;RDD#jJ8x8Ii~=fDp}YdeBDv0*;Bje;S!BJy@INHLB@$)Eqq#M+8sbxfqBDYfp@ zuc@snac*~@JgKYeWe{jk&ADBJ*K6gC6)ddec-Vg@)|*evU-z8C{r%RrvbWuNHyc8c zf39{qszeyD-(vIxEVnp2(4ZZnZ5B~oUd9AWZW2e}_XKFKNU(0Y2&ax8usvs6!OjGl zfUT*jVkb7c6ZPxGLg}gOq?Wp0ie4ax^^~pYXw=AplaYmJb=$;?4C^-Bh*R1~(>;DA zm6h=Km_I>d+jdz~{rO}?jm^xLXmb8!C-=k#{XTv;d0Yz(cJUS@7Uh3bAIuyPpFVQ9%x z#G0xeE3|F^93UbKuJOcT$a{DLj-}I!`Zg9KPOxU+%GfNRoM)%Zulfkk3VTYsxty>yj2{^`*TVXVb{v`?bfDh>9 z$*k`0#ZPs^bl`zw0p_x@5cWC*_z^VEdQGh`ayfsOR*MngQP0jhWV=)+q&sYPEy3p^ zM-OU1EYQ4DXG^O-5brc@mg=H7L(Ycp<4&smh!I(EpUlEZ5t}>WUdX?6_6TF?eREWI z!!UG&{ilzCy>`^*gWJE6TU?%n-F0MQvTiv@pxZXM21(Ng#B&fN%u(6NFzp=i{WmkC zaH<1tSFLrz$1;F~SICAB%>A5KN ze1qcxb{6z0}MNnbqKF6kw{pxcu=glK1u?ua;h-NOUS7;H~d2)y) z?6fv~#T90^sK2L;I(Op(P6hVS)}tYJ6Nbzi&GVyy%SOZ=d7lLLFntJ%y*yj;BaK@D z5@(C)jFyGY);GNv+FwjP1w<4GJq&1BWb)(N(;DH2_Pw0B6r@fm(qsRnMDcTyC%-0% zv`KN4rd(%flpeKg-2n=6UPRnwBwr+g=Wrko8p_AbeH;$R+~TlJFbl95jzSn?WD4vhl6@UVW7O?;HMHdQc=d z%9Devlt}njs`zmC{Rz3)>*CU((U~DYOtuk=+&Tk@)W^smq>Sm?LEyf%Ttik}VEt+e zlWMSIvWbUPG&l*6nn{8UYrKat8_L{;?Kp4U{fY zc6cfP;_reQ&R|--V44Bi9Ezx4YmLC~S-MAJLbu3vcVvUdB?n6CSTUwGAh|2N>`vEl z!S#O@=^LbNLl$SfZNoAr0a&&EyHJqLklUcsSN80PThASx#U~5X`WL;LIRq4t?P=AP zDnUS2%lpD2#xr@h(VC9Z$Kf0B-fZYWakF-@9~Yg+R7}-wutr!qEr(M9rdn&Gs#%r@ zN~ZcbKw(oU<#635oYKJzPgpY?XkgczVR5-Dh za9a2gz><~Uboezw8QNq};)M1N5di#@R46SqslF2V|E8qe{NmmQF0 zu`=qDUP1oV*yUurZUuBRmJdgiF1&ZG^mlX}IvzC2Ttv$xCKhS$PXaIhrA8Ig^`@d} zN7zjp<9fpg-t6TweP+ZXu zkIf@iKDSp}qAyxz*DPq)oer1Vx}+D}`*^C4fpzAKUCnJ=2YOCz%P=4T{b`u`f@&;dzU*}o)6wa=nkMtywTc2?h{%K zP?e|GH|Sfh3likbL)iR?@Gdyv3fsSQ1+Yf(MYhMR@2CG_!M4>7-%kafM7PCp!?v6X z0xU~LvSh@EQz0`|Pm7DJjxtHcq!YzH-I!qh?-}1D04{6vYn8_gz*o)Vjv)S1Nz{>K zN1q24+N$K?<6LEK_{BwJpZ69siPD2{pa3HjyZH!)w1oftX51k@W%MSZ< zx1-(zz`fak&Ao4R>td z>WgDNwPk|ZDX|%2s6ZcZ){TweX3_N2EYJ!&n|kR8fy@3U|MyV$_>edOsU3-?>bE%H zSSv%JAbpVdCHdvjJk!k|ra0Cnrpu`=i5=5HFX!ZxiJSJ~85o+gkE?lSscg z1P0xSuwANHJ3Zxv#$mgEOycT#Z?o*q=NGT%f0@_u{AlHxU6=y5PaM?W&<#k3nj^kp zU9A2R-a@_ip^exU|Jkqj#=^gZGTXsH4hP*;5~oHo`^}KPrn6HKO0#T_m1X1gX!=b- zXdzkF89l&C&rYV}`eMDITU?pN^M9XZ<&Uh_u@lT=@FRBD&}@KKwtLjP9PI#p^AjC@ zP&6Rjtrkn_ixcB@S?yi_7C&M4Nz+*)`ay&y^Fr$NbZRWmgW)GxWnwEhl5HK3xZA(x zGF(ibLOpY_LlB%z966t+m2Y->YiyawqMgYSKBn$x(SD*;<||p|Jio(L3*LBuoW^og zO!e)KnE*AiY;-B#RYi;`nRH|J0o7$M#m~UtImSu?(i@Uy>dfIthiP^)3Xddk0yJ{m{|(AgrMmi*{XB32xGcG}bjA zWf{Wgnd{5kmw|g0Z~|%ua@r3~e4rZ2PM*a>NH?f&_auyWQS^MGy6Ig*f0xS8r%%4- z7k~ovz=A)O-c&sxbQh9)0Yvj*#PyE8ey@M^n0ol{7^zh=5%lM424lH^)n;b;;U)z= zv$7(tG4434$hraS8vQSRza%-{0G{S-Lumm>W2W5`TaA3Xcyihh7AfauBD2 z(4^VRYC@ z!izXH-G~x@weUI0TsQYs@;C95IG4$jnQeT}%7$@cZP7fXC;ul{V5^UpBt)r@EkkLK z6tl0`<)C*0or7e5vGxHB+mF(i3#gAL4FAIawr#!#g_pknz5|>1KGA?GQSGiSiUjuf5QV{HAcy{O234%*ryU7; z(US4<-|P+jySV^Px;oyD-g>Gu{j4Ha%4TkrD2d4nBeoCRce>4vI`g0&H6SZb@nT`0 z#y2x%fAj+NoKYpo>#Tl}hbx^CI8O^$0XWs+glD1g5R*rkPIuT(Zyi%ZJG+9_l#@d9 z2cB}?Y+sba;rqMVNY_{X9j&?&TT@yK_VxG(xNm`5gMJ+ z`Bk;dgrK0s^RUIB!o!BwBzY_i>y3cqsCPYbV+gTu1t!=?EKxfTQQ|BHoHk)z%tGz6#DM2u;iN_!pEy4k{mW*L_n!p&qJZt-{1t+&1rQ)drVa!kM2ASdInG4@TrE!l zF;dum^FmfL8)K%d1n4dcH3+{HZyhghl=DS+0`*m?4T+i(6b-EBqE`WXGLIB|3P%*{ zZd$4CXTBZRHSgG4$78oYL*R#1mF_4YUoy}k+0qWSZ@tS-L8iOhmn~}aS=r@q`!HAL z+7b*`!M8Nt-r7?0c4zO+wsnh!?E9#JUVHjQ_+Uy+;%AsXAlFRBq0(5hhGmsv^r2?} zsApYH&pbVJ`)EVv)@}OdCV-mWod;6N#uD%Ao#*Svu9#ZvjXQnooRLc{bH`)@ro7(;~+q)hR2Wp6JfEg><8kQro;)6Y4zOBUb?cfz4#Xnu2Oa7hRH;{5wLJXuk~ zCc80NF^lQo=qkvSxp^*S31+V13v!0beQ&tWHX!Pvn&QES49u}H{%hO>jEN}RwqHwq z6AkaNcO?=v zU~^@t4j{0r7I)KiN{bO%oGqpxo-@6P7uhjQr0h~cx#_EX+t!VCwx11uLEur87uTeaY;+EL%*jD+=83D1^+)ZuYQ z_UkGE3+nuVQIi!}EkbzqxWbrrc4-ZyQ*+g>^IR8Sa~03K{eOflg!-(bx%hK>8J}={ z;M{$c^TIleqCYmcVf9QsK)lp{T`Y2Eo8oLQ^{#IkZb-Z!Vs2xYDfXc*UjInZG@{b4 z*84Hr1j%q}(6^>-fM~?Uq~;04iUmX)-GVwBbHLib?`-!iMM7y1F4p|C7H){3-1oGH)HAO>Y_uNIzD>5c=$$tF#FE7=PXHS2$V^*dhtP=;YC$;?Jk+bt0j7ABbNZJqO5Q_=YB0InMUmVn|mQp30l0vb*W8y!Z_0Oe3zJzX1Jb;yancbFW*K3Tbg@CO>eh(bQ=0 zF4S|$EfUkcQDIlVKlNP;^JGTU?N>*{jch_((;MBJKGc;G8cQ`y0TSTqj7@B(LC4>c z2~%fzbVsjl;&97|o?kEW?&9MbtS0_@C0L(Izy)s$5DzC2fNRV(bz+_m2ddKBn5@yC zO2Ts;>p5`U4md%)>y%5Yb#C2p5^%3`UOfL9rW0n!O_h_K>>3>Ls_)8PRwo^0A=!D) z6Ed$=KdA|~?LSW1Q|hAJwA>3`NjXsb5%PKC!)#{TNYv&FXXBN!3aPQ%;a#!nmtFOv z$0W4vYr%W;I!ez6YS_JClkBu~vHnsfcxleRuAyCr|N3zTDT4p2rC;=ed$Woj!jBKw zDq|v}?!JV!hFi%v5fom5oh^Pff>uP^o|344!y^#kvtMwcfS=%MHH#9|!6|GR^Wq$Z zsyOEe2PmP~iSqOw{&45fp!6_>W8X9bm7c%!e{VPlZvKu=`S_8Qt5r!Ci8;x;<6bktzH#w zgv9Cdu{`A!y(b(2wi{PECLU{sF#9zvWOZR_s_(nCwwz}yb7PByjDCdGk$w+wWDP3- z3VK%Zrd-3{K?y_p7sGla4=^j8J=0l#OAoj!^`M(<7iNR~={N-Ul{amz-h5n9gsIM2GTY8EW`pr;( zn_)Bk#;;^qkf4yZG;3aH@-#0uYt{>Ih+&&)Q#F_r{p0(_GH4vzlKpjCB{d4(8HL!Z z|83+752j=k02<0u1Phav13Pvn=?lsR34hO5WNucB#Qu1WTT#LN7+q*|1i>QJ!i@9M zAc_1Qt}Dai8LHzXr$Nrk-ORCe(xpCJVwR~!H-g^2HWghkMA<@FVK@~;2EWw<5I<|x z5*}ke7XE8KOgR2gjY$fqqr(fE5l!UZ^FD85T9BZrYMRztTa8xZHMlQ@gb_Um0q(FTeNVd>QNd6TJu|R6Ra7^h?efpO&y>9p=fq!alLQbaL~D+p~&#g+47*M0?*#cZiZR$KJqC)L+M@`bZp~T zSJtLrZVn39xwcG#IQK=D<6h5JCjwObT%#R()jEJa;BM!AIg8l&aYHg3PHCzAFw;B3 z`_(uEJuLAGg)p;Oz8Bky&8O6r|;r}&6LLf1G$UY!n)s0co^NtZ;qEYXJ zmcvtcA58pg2lsQv%1U%TY)+D=C0KsrHq<8M(agdWCY=ALpKuZVnQZd$9a5^VqIUL| zbu)yq4-H<`1xNaKt|Ya{rf;m_z%@5yKc8@d+-X3~_|Z=&TrGtjdC-5;tabG$jF_9^ zRhU0W>e;^o-5-7Czl#rl@<)aSZKPfjD#4}v7u;{a0%`p)d;Yptr;n-h>TmfF@1mCM zkGo8>rIB_9wBI*)ESz9-e=V)$6?7=;`UzbK+JV&+?vC@?JRVa+?*guBrlBLQrU^_W z?KNiBf<8FHhh;DWanM(iQrg70;=2Y*HH||)x@hm)OC=_2B3zRZt58qfSaIj{p&Fou zzpQ@F0dmyMcJ<17sxotdXF@hZA4n2OOQ|jBQeFHZ(gmSDrN( zTgOsLO?-|OD5EUk7N>9^8$ksrR+F~_nxR92gChgr^TSEAj%>|~7tjr63^O8-O1yDc zTPAN{tUSAo4QI|t%fRmyyK^S*gxqKh5YR$Xf!=jQ>PDh5_9O<i$;I7i|$eNGVvz}5?&pE`~2}iILGcyGZj>sE+5cB7|scp%6{j! zw~S6mr@o1uweQ>gB5lalFw}Nz21(ws8%vlNC@RL0K-%l=o=wEciX{=Qi;=v)+>-d( zyyUhL#=`jVen6R~>z6{5-XB&*fnP`&kdv$`6(mu>t+Vv)(2NOaeI+Y?E>b2%U;0i@ zH!A{wCwd%kE2c_Y>G{=-bgIx=6pWnK_mZCEM}P=GFFe)X-x@ZIl#Fwk)O`rZvZ|rK zlel!vP;hP&=L{Cva_Ny+%%tl`vegPxq6hRnR$hPQGbOeGz(8 z5{}1q*XPVC;ZLmTPh)D1otj~^Vg&o2&yw~OP?W!R&sNM9ux+dVH^VYs2XzemdX|!e zeD`|eou=-)m|iRbm?N3!!HnPJ|E~q0{5r)?5Gqz_8c4dY@qYI8#+&D9xHwwi(gvxI znJo{ySsrvKiUpOx{)2b?5sFB_dI)md?4?e42uGX|G)Fx&hy_JBUWP-6_P?2j9=&OC zGT`L*;oiS0FL?x7ka7l%xm81aJ)Jce1Ym-&I0)QQ{;t*?4>N9o?WFjEz^)wOCa#l^ z7v}hBtX&r7t6_pmfzrq_ICj{24eN=IZ~^y%^;YS0r`fFwf-RT)r56sJF1sX-=E~=E zJ%76H8h&UM-bYY0(0{HCd)vt(*iufudHhqe@n@(&TffqWPNCUEYQiGh(xI|oI-Rf6 zRiRhJLPLf}sJ;r~`ocs#M}tcGG`eAp2tbzy#{$x!(mC=jOz4<4@Gd;M^4J2Cc#0P(?g7Gp65}$`~?YH79)D>OKblAylt&{h8OBRC{qpEgls62H%c|7&d^dF~Gj~U%FczzDQq$1aE zfXj>yI$>kUA~p$V+L2Mhu9R{RghH@0n{=5!l+(UyVQP-sc>SQqegS}O5$w0n`5kue zXaCifw@{*L7Poh)l?CJW>IA$DHWizlFrGoito7E;Y@9=~4DNueuJHBpSjY)Mhf`Mc zc>~m-XMZ^HlEBTG;b%n6udiRRlURypq3JJr6d7Jdu!p;^=Q{xkJttTPhr_1h-ON4y z=@xRDsXy~X5ryH+YA`5%L5M$o(n!h|URr4jql73bd>~I2d^V3W63DQscv@%x2h@Eu zK~8R45RbVdY;lR7=_NPEduTlcOUypbkQq?~bMP2KB{nm;4bMDiw<(e_%DYpFEN;|w zPtz@GZ$Fv7rNw-*0d+%SXk3EI2mS^+cle)Ao=gi6O7|}Ug>_=TL<1<6@N_X+ve0@oV zud$^Oh)8cct{V{u4KLEoGPVLnu&73Af>Hb|ofO(Mh}vh_(pRL6ey|}$aOr7udhsGe zOkkB-HM9!UU;kRPnLZN#<0$FW(c4-27sCs9Tg%XC7KLUpu6M@|8>D-L`AnkaN2uWJ zZ3$xg93`1et92i43e088Zy;I)dVqeFfT%^VZ`|6Z)~klKQG%Ivg%auUUOI9Z8<~py zDWQc$?yjB+T@&c@F}iaoT2&$+DZcBvObb6JHE9Mp0EeSw<&IpBb(_b1$4Jxs=3NjvC$YQB+$R2TmH^rPUX(&?UBGOw;z(oL1|$g`@M@m@AJUX>Jf*N zy9;aLdx>Y8R?8(;RUb8;dVUecvKAZ%q(?=!+1IjV%HI)6B{P_GaL*?&+PovY?7rEM zbzBK-3Y6wSYk6`EuA9k66yZfqqC2-WDr!V`Y@}9SN>z4goaJy5z0uTI$nKSRnPj=F zG%5kZR&X#qv)BLID9`DGe116eUlLjjvJ9Ux#mLBt`-l6!(lEIqEYoxX_xrDCoYj>(NLkab2Voz%WK02Mperx=U3mM_#zRObzA!&oAuha2azUfn zVTl_r*|%f?w6{&nfs=*vQDfMLbphhOgy1-hImKd2sPXo|_dyMDzua@AK`%n_LD1$d zr!4^qP}^QKone_oY9QwAi>p?B*#~-{xttcIprpixN5&SK0fxA+VNK8yO?DN+h<^L+ zYKzVp;2Sjkvax$UH?G;NhAAD1gmkvkjK z6+Tr>eR^ZRJ#xAw=KLN_UEsmx&&ySOdCV+_!N@a9)CZ&xb%%Q?+^UG4*^nS5*fae* z5xOY+&C_`j&tl<&KGD<((OLB#P5N-|W3%v=AZNEN7MtL1!vID;HRBHMfa30vQa8z$ zKd9d)8IGlFsX(#8Io$pA6KoYovxbzLlY5wTuCY)31IRV)rSlZ>Gh^O-#x?I`e;yI< ze;whLeekrJk7=#4h#oW#ezK6R{^pLRRnX9zcS}45NnJb;rp5h9Puak71cf^i!e>83 z1B!9&^?*Bm?W@5r{Zk}H;qtn{b?zt}zE{A7T45x0!4F*0g=YF0{?T<<<0C~~f(6R| z6ZMP3vjdw-J9i_#UkQDkv#a)#8>4rDIp@n%eClhKYo`3JNxSX%jhOritk4$d**lt5 z%*+I~Y_62Ct%H6ZDk^HaRlJcVl5F~HJv#c6N$=C2DE%V@V0g5^HV-rzXH6fi#NS>N z6s3W*hX;2s)WUqxEDdfaN1vifB5vO1)G!IxbotFqXq}G!PK|cm+FVq*f45Wmp~Wd8^6=SJx2VOL7s`NW zvBtp|9FIJaPsVp&Nf88kKC6Gk>x}w94v+ML8~SA|H=*)u`>pq)3;Pyk#@D-Fn5k+RD1mMLj17IHY>}=f(r>(|Fu#NMv9bp-&l|K6B`JJM2ROK0o7g6eIfUPKx=x zImj2tl6d&A3xh9gLL~gjVSLHwP0yiuMYYwEFbC_qm`zzWgDsu+24(T>E~!!`SHm)R zVRA#e2{PA&x4E<*>DcqU za^gj26oU-z**d7I>lJ2yrF*aGjWW<3l3Q#GFwvX zl=WsGe|!(or9gREZ6jF-CFjdiIwDr*g$EFiw4lIIMuK-qJy5*fqwnS^^(1(+A|3rM z4LcuYTp4;3U@Us8_l6#&Ua=#jR~ zFx4gtwo(=jy^2_CIt2>8v>YJ^5~C4kf$0X?z4QTX*e zdou3F9Yni288P$xaOC)x0a=5F2_#*@r2&Q$Z|S30vrel_qrG3qbY+LU%brtEPRG_GfN zy&ZEpVUUAGdox`vK?&y<9`K!JMZlGkvw4W?z_bS+E@qE;V>>^j&`^PodrNrtfhHKQ zFgIJgRgaSh`h_I3-}~jryC-s#`;pLHpslF6-M?CM7%A@Xx5rlAKxA#}^@V0zxrez2 z{hf>PYJK|_yyuERUT?i+NlsKiW`ZKTwWCcvjZLp3?dGd%D7$JW0$IW@d)NO4UAAaX zh6lY}6b*kVnNm3xWQ7i558{%#E|*T=d8R+!S?THdRC+X`!qO6M9_Q=O_f!H<81Nc- zAil^gY`pt;%|hhr9iOuZABET}+4q0)hmg7;1_rXU_xbB=m3|1dNW6E*-H3<24 zCrNR6He)DRH25F3k~`#+ZKeDD1re_8NHym?P|sK zU@SUcRj}z~Iu2psdBK?|oXDkbiZzG{(fD-hariO-{0DA5T0_#i!Jq!v=7o$zBZO=M ze?qixkEXd(W`*s;s((o@hxbs?1XvX!ztWb`K6vK3 zm3YC1H6A<8Sx7g#Z&l$OQj>h4fjd?a)9eO&79tUg#!6ek?Igj?{$!>Pm^LH$*leuj z_5mD5WdoOLvfV0^xhekRNFL8W@{mxF@0aX_kh4kmIH1vH5>Z8 zRaaaO+(PhgdbW^?w>GRZ`O=hdRkog4v2(iPHMdi7S;n61Qvl#j@%^K)Mt;d;)d@o< zd~)T3)a$|cQPw#3rg4Nz81F<}0}}vUJXR_X&-MHc9~XtFoivWxH)D@!n)F9jxVjiOG?q4R+E7Rj>z7V|zWce)X-p;%Zj1Hd@Jsg(===qyF zkY@={w-HN{JfRn?*(6IKB5h^u;3ofvuY=Q+g;YJS@R6-3S-AgR@+o`93+hpx9S1a3S$s4 z2;2!)fIACajiLnv5*Jxs9 z+UH_A9H?PE!YV|q=k*w-I_>5v#~Ks?+D zO^f9LIl1KB6bVPOH6;NOzj)k2Tg^y)-ZUzH-_%0^0Uy$pI~J7VfgV1B#|e2l#}zxD z1?s9;mLsa-4_78t=#CXn8LWs#gHNfP3!KhiA$}hvD}>zRf{Dn%+rfom6+`I^B#utDHe|csBtbbfcqC2U=*yF+FyaiKJZGCfQRuwun3wuBRs!rkm_h#np&K4? zPff-o)R+_;;ugD-Y-JqjHG58Z5U=D~w&8L;aZ;^yghH_Mg)n*pen}1yq6#?-nFF zbl()D2$*Cp(3BmTcH_{vi9&qbws92TNk_u?gZcW7er$*G?~zf>68Pzte#a-vL8QW^ zh11kHn|K5|mAL2lzL`;zzp)?!q%Bv%aM#W~ev-T>%1rX2P@7rgR3|Qf|7;(wEvUb@ zxKIQC_;jUYWBJwOw@+7&&o00-n+h^3G5b&OYM0#okW6?eEbs_rfjYPL1#L1d zT)HIApbqZ6f>_xg;y!wzaNMl)@>PXI1|LQp~cwjA!FYgpXeR zY&E5@NI3F2dZ(}3pgX7hG%I=RfpU@#7y`nFfB*8>8zV9ab}tsb)edHIX zXrUh2>(czNmzSBxgsz|TClkyHwfMRAQs0c5KwPj%d*qAaahR9mYUW`KOJFlMYJGCrUYBS8x> zQpka+&fW%+o_wBan$|hGBt5YO z%yAbQQC@Qs55O9V+kzyrFOd&dn;&y{elvv5t#CpUASu!t;zKr}9f~Gk8(HK_C$bM~ zoN#DbTQ{yWL<>~=UNBPW7(_Q`Zj$Mr{wY`pLZKT)B92Q2{*=vXBS@2GA5-42zlloA zY@$L*v7j5;kwohN^!X5*edvIx4wfGjyAY1@mAIQgV8L*)`h+2grGifx>8)xK4K2qUX=-WVWt< zy_!4%Pr7k`SGkt;%)ly5&i8717Wc3Svf<7CPzwRTIUsG3&Vt58qh*S5sa`;2V&XX$oM30AR zs~ih-=58a(GEZO3`jR!yXeyIHRhIL|@~kkub!PJ9yhcRXn)#v*^qs8HeNE+Z)q?rO zJU|zA2^)DM{Wg)pFY!xRnQ6^{VoOXs?MO6FV{V|T_&E(V`}|RoCei*Nk&@yuJJ(vs z8OH{ZG=Um-;9f>X!+$gza<-XQN9q%?y$8$(2gIcH9J75R5w--<5eG`N%&`I+dNJ%N zLf;wWulwyVc%{TOTh~inj&2)AYvN=!g7B}4Tz4bvOpY$X;{Q5%UFbJ5cpV)yLQxmV z3-ycM?@668NBDDOGnSn$-Q`xD2Ji|TXE$$%g2&P{R9A}Wz4`=)=rnG7_pF^Tp64?E ziyl*ATz5u?;m0Xw4QQ_{Cbwrv9h+yxe%0}$Gv6yp*Fb!FFZgxqRk=ok8vxhZ(Kn3d z#|Y8oPcqKA67Y30-&4#5GivR@uIOmIYsKV})9Tw?&Y>WxXrq&(4zEdPK1y@?vk_9{ zf!uq<94RemSXOy;p4ShyPNlyE!8r_H2_);uwCdwYh zzOL}zPp&Zk2-sn_;02}U$R~?Lf7kt?j*FYD5t{ryJ7>`+E+~24rowi`L*BR{hRS%e=!c_I9;oCv+}0zuK+c;Nse}YP@w6z7xD7!XzIu7Jh4w z&x#!;cfdaBsTyC5%P@y-5bDYyT-APvTIMh<$w8<|Gi!OvT^r3&1}`A~so(Ugi(<-p zUUq&SsHxxExXrC@o1_i8swX{5IY6LhXIH!nccijwZw-w9UKbu1y%E(&l{3}VPZ8K3 z=i;+)%+S}jAkw_lJNlMmMZ)zmnZJFMyDqi z)}eKO@$leVgIZI>abZD_bK|S!My?XuZP<$k|sgi(N41B%z@m~_hsq!x!auWnN19e#=nsk zycFY|p0Ug(XIoWeqFQ9%R#fXYfDz89aCE|3e4en;T%Lp-?4_BzJ+s(gI@5H2|K8^1 z%nX5q;M9Rv$MKZAKe)KfBdwE=IWXPv`QzeJW@| z@2EECfPb72iW_qjCz{i+E>gl-2a-~dY1HB$bbR8#p>(-Y(<@KaOlLDNH5Xp4HRI-e ziUkx0?P#q$75g518G`5ov!bjvifcYS+%l-#IxyHSF;-O)vFUt5iB6reVvGv~WRrVSEiI@{9qs|RS) z=3J2MtN?4bX#)v4K?Mgz2c2;z=Es}6Z zkGx}+8>g80<3Z*I1G%oaSa`U2E%p}2IQ>fpILzJj`i(I_Y zr=2NyHGn0SDj#oozrTOnC@tTeQ&4Ss7bB0Koga~Q8!hLUYd29X_9I3}Z}SIqx1KWd ztqy-;7JAg2RAEt_q~aAh_0QC5s3fV;+9+d(oMcL|$jcC3RvAL+o})%Q?)IDv;_^Q?+`JUt#)5iQ_oyftd(8EN))@mE~ZP+P`WH1LL%ex zRf;#gKYU>F^(A>bs!R%;`tb3oGXa4mnl){S^pDRzL`!oq`6uKyW9e8FniUcp!YLXb zI95Q9no)~wcSIHgq846{st3XzkJmPOb6Pf25Iiw%u`?j{GympXoTW_TdO{y0Vb*;( zV@OXhgRCk^7Tf8g0sMnrdmhseff)0Us+Z<7#W--c_IrvH%`d!}Z60>|I7Z!ew)%8k z|GWs=AVp1DBd6fkqT;mil?Ucmbr;m8#@0V?#l|BPWT=+tR$}6-Z`8a8k$BSEW7%uE zzgp}~+qXqe_&O;)dzpPgqqvwa0KKG-olMQ40;^cF2loQlDZpzP)z@(h$2WLX(gPud zl@32A>3O34II#06v#2&c^TLAK}4TAStqgAe#110`jlzn6~PpM+T&uKQ~ zjtt+xY2tnW)4bhmXK|JweL}93E!ABfsgy>W$UM~0rxO0myW*q7>zaA;9|EW-mp={z z6ED=fdZQ7j>HVciHJSxn_g4)kYLbunfNpOLJDhR%zspR(s;r73l3r}fICUmpP*L(X zV~=IlNN@2`d@pyXpVy($XhCvnLr;08{(iXYtZ=8~UUlk;lA?CygSy1UB%1O0PYSC_ zds~~UM#IeBo@C;0!@u9;ksszSe(j<4e2!e*qoL|H+$T=mcE-o%XtGtT()0-g}plMrj(Ek`{8N}AdCj_w_ick4hZf2?I#OJ{XI~*Xd@r-}{jQy7;lTkwOM3FF zr11&qh1`!CAkx+G)@^1ii|~>pN*oU>gj@xo(`1I9Hz-nWepnuUmqiYvb#)U$?b#cg zu*kah@{7b+IJ+o5_+qX=q`K->M9-`g1BYrmebL~t5lk=?q|cibXZe=izi&RJvuzvq z@mk#ij_&Ahx`Ztv^L?E&7?U~qW9)_2-~TGKgZ<~$2TtLXH9^WV+g>l2)RgjQR&a1- z$3L=e1zE@*m?f$Ag4trIOjqZZy!J;fPtY&L_c|(_+KHPrBxJ)kBUX}^)IKWG9(aA6 z+&GoIC%QN3kX^v268aJT#~%-AI0d=~5+ouEpoN6B{#@X&AKVsf5YHOpyRMmxxJk>r z?!U$~c8rFf&2GS}J!MLuE1;W>bT5W&q;)_Pqst2tq!1zTof<&M@_{yv@F(^^41~w^ z0Kech0;dU$y1nU0`z=%9(^y#FvVG?^lVrTFIo++~Y)7#%k_#1-(^)-HTQROc+>~$p z;^z(7c?$?y1+wi3aqW#zTs)gY5TUgEK@=~}m`{kFfvG`vvQ1m@nyt7;&%|_C?+Q=o zU*^pLqv@YG{N0*NJl$$c_*tJp*3@~ju?e{2U8q0Ab{xziZtOpLtzgVYm{y*!u3B?8 zf;mMhy@o9nyC>JV0ugd0=<(D~biD?@G-zl195-b~tYPDf`_OaVp)oa?#(e_*OkyxJ zR0yMIxmggFElU=7mbG!FhuD30Zlt``Wej8I>a5+M^LK`Wx;h09cCuo zX4F?;*RCSU-W3j8cThoCet#F(V-U`=)YMOD>yXl*;-U?vP5CVf5$N-lc1~n z=5snk+`qtw&`UIR6?PSzq)4?}e|C%U!OF^+nUNE;2d) z%0-{9nx>BgPl_;0AZUs(+~&Kov!eL4)hcGAkalQ}qI-I|g(k+)VUte4?Y@Ujc$0^8 zJ$B^K??M-*YQDl>dPD(9bE6#UQhskH3LvU$jX;NAGPL;nInw2M^!GIKIGSS^Q9_|{ z5DFCnsqCRqv9IEp-*B4m7`8)NiN=eGkCE>fr+hJl&++nJr>D0=9>5_14kf&qGQsB$ z=|&$?g}=f-PLJLoT8qi}HOwIps`i>5@BgFft)rrf+Q#jHp}QLfkWxCNV`z~UC8Qff zx@O3sJCv3#5h;=GZlo1Oxi=pILc98=@oadwI%@kk3f6{!3*|1A(e6@YqShvmC7neu`;x z#lySns?8!EgfxOMuUcA`9>x3J5kBw4L@N2kP?)sb!%9=Yn7*YS6$=dFVnzpltjwI6 z!c-N+v$oyIPaEs}eL5dhNu>#Ty$u7Z*^Loy<_Yz95tA{Jx&`(n8#e2Cig}O^lss4|@CvQPA2)7gh;Cy+HU1Pq z?y7unnty(fM2AQ2%{lMTPXDa%ozko77qljy_}9KgaLV_4gWl|e=q268Q(m4E-l{yOH zTqn4@EO8nn8`on@64#lvdUdtB@WLT|%WKY2y8mS3K)Ll({dG1>BmO6Zex``Tpg2>B zh<7o@TQbJiv-OgGe3kAZyI59FP}Zh}xgIaaVLS(AGRR`pj-BoNlyw``wMbYy! z;pAdbAL*Ps#8he{S?8sx;TEr><81H8H^ReZIdu}8T9PtAG3f|qYFq=;!1Mi{`#9+< zoU!$ogJ{Uu@Q)|r2tWT1(l7rMzDKqT?VcJ{v4t}H@S3c;S{8yw-lnSRwDzqQkejAV z6Q4Gh>|b?0zd`I5{@_j>vk+47$r)(5EKW#00IcS7Ei2))IL;_#cKVKAPTo4w^!V`F zMG4T3oEj3i6+r~UKR)q{?qE`Wj*krRcP8l=67dfU(T}|w4h>inh4b$)Y}BG2JTi%k zcG)es;u_y;<=p_3Kcj2-VN?jF@zWEQ%PhEbD!A+rpCMDjq#_ZFficXtcD`^J;H|Bo zo35awNpLGdps?k{xE#gQx+Qh^6s#=qy^*Z^(cO3PxW|A8T~(mjzl6655X?kB3kj(< z&k(oH`A$8uM)e>m|A<#bh3Vko?HwO94F_(k;RA%!3KuD=utxVstL#BwVH{7A&n-f8IIyYzELHd820%+o_;vG zyfIBI8v}nVWSCgqB_HJ_T9PW6B90Ysj#bJ9%ejs~af)R|;7sOMJ;dCblCU zV~S~Q;c^tG7R@ewbd?J$Da->^F~%AAz3JK+y{y5RM&F^5tfiD0l7Ey{-g@WzKVc&5$l+*jnw z$gN2eW7R79`2os9oXVcYI0xIEL79Uc(>-Y9>6!(+%Q38zR(jv|y1fiu zuRo_ z@l(Hd;E)ulbTDKKbHrDitv>F1N9ThhHqE~63z~;od*Whk$f{uqt&lgXw#*;T4*ZoK zzVXm7d}8c>_OtB~AG#g{*_STP9{W~_RHVoo1b=0EsEBKq3MM_jE?nb6Ky2WTJVbd`sYNtjIcAOe)4~%%;3T*F!cqb8U5?>G zo^vthq$kpJL0_E}e|Ghn+t^-3$7gX?q|m_cVuCeZSnWcf$-?arm^g6V6<5E8EH9WJ zyFB-U{E}e$%NBQP4zfKfNifui4_EB@HlqzGQe{ zweS^JM%cL4y0(+W=zVv+S^kl>513`ERAC77wbnY`%%Yw%nal-8@1iRoh%3ytGjs}q zvuDAYnCb!QCSyILKjTRlbgsS_aD~JTM`;8x7Y5=bV4e!(`GJ0T317b0!RG}7=R8cj z{#Yy8mq2-ih8YQ2a$bNA0T5RUC<4!NI9c3m=EQ2X)GQO^7;LWKx(T-sh$?8apZ@%@ zvKV2PABlXcKZ~%F4zP8gUuNG(IH~AohWalLdf~2Wz6uYn!mHs<&`3|Dzi6Qw6m2gvbu{qL6T423wNG&S_}o+W zpw`p#bLC)0vn^5kq5mQ0cVD0)JJ~1hoVV(Q?S!ALRF3fHs-q8Fqy)|Y(8m`j2A!IV zUWi;@`{eW1?)q~>+rDGE$#sB$)&;o$b**&obk14C)I`ZnE(Z2>7gbW0V}2Kr@jJB- zrrd?Dj;$w#kk62mRB5~F+TIHJBAum}CC~Y5?~X0ig&5m2c&XnWO!cmx(7#&PhCE5y zpaa{g{O|8q_INkc{$0K4w?P~0!D_s^ke*?P%pW1Bpc_Y^8C%n~o_Ei%_w$`EBDFNL z01M^5G`8)rGBpMaZmxTZD2H9^uGQVtjm8$Lp(yo-)`|COf(Wy*{Shw^jAAnn4!(BN zJU^w{mKxOugqnDBzGE$7(!iZ8jM}2AOh2YoM@#P5&QmJWQqK3tUngAVm-(<6TUOsn z>Jj)8wEY(Jp>+A4sv@MK+%!K6$;W+&`!M`Z7Rr%$>#jYMG>A6@Aez`kvwXS^KV_y2Wqi7+Y^NNiV9j^48Yn zVBBFeoVNa7M(YVCl^Q!Ud}P=0X}~Hg->c2vbIR%cl0qMy7`j3q5qIQxr7UH0wN(VJ;k7Zr8O0Vu0239&3OF z`m(>pmO;}$VSYzrh?2Q zn@~&-z%9t!U@j03%D-e695ECAR#@*PEcT+W%lDN37nKEoX#OIMK67CLvr(qi;eiX_ z%*a?jv>p08|H&Dkz(C0o#Q4c?DNoZ%_K2Y<7|rlNc_%1<;9q?~X-buVhXHN(o;Y41 z3~Z$Oa@h@AkLo*O)&w0d6aKL3`tC6d>Lw%ATtcH&Bg~R=DFor&7Sh=TrR!%Q*h_gtS_(1e;K)pfxOUx^99` z^>7z*_O~+!4ztJ-}F@+{_LPsb*>h@JK*)w z^c5sPv7}nqMTLZ(OYH9jc;J(iEct$bWK3+)w#=0AlYOBP;yyokdyvE_E8VXp5Q(v- ze=6vp5=eGf@Ta#}SR?$7;>c90zj3>{GU@t)V>?=CG9nkDM4Ed_AVhN`hBo01+JFY; z<+)9GmcJE(zf3C-1WD|muXkNm?9&SV{D746b4(+!Va@S4#UCbAm|;&hEAa*?!(n?4 z+oYG_(zgW?bb+_X4UuIWW00yij#T$E5q~hDH34D&K>(B0kQnR`4{9su3mIgUGR*ww zFur#gu$Os+SPRM1EQqKJ@m|2>xTg28pFQ9VoOkGu7tm>@XS4oD*3u<|WEOe_&Lp7v6A8caMF zd@4qRIX+le|BlPV?$p$cWKCgTlbO=Im~TIxEhDKl87V zKwRF<>@^+V3sF2aGOEt#Jv@gkkTlH~YIHnIXr*8e;?)4(aw%$gaqr55#jrO=?=|{M zNKfPk@NMERi$m@=IG*YT9|1hNNc-}0izIbok-^#; z{3Y*C+HIGgawd=+^8W)Zlil)(Qyr19!gUy^;rY($_X&t zg&{r4ODY#Uh4G_wW>ZYh0XvYH&TjxXkU*A@u8t2QKx{6oXEc9g7w1537hmSjnxi7# zgF2D%U07s&r2R#g!$Xbz;MQ->JXMT4$K4;|yP?Q zo8v&bI0BqO*u^o;ilC@>KqPEdnV)N2RvgSQ5us9<34G8iC^XIfYI*l*s6#*eU(Wq_ zH3>DN!!_DnytLNDxC*B62~n{3ZYR|S4WVxLfcc5a9gh$KB@Y{6eAyefiD-os^!RsPYds~lK&{*7IJ^r65Py&?sn%EYf zbh}*zaRF>#Rw}R=Nla^lq9q-J_B=4IW2;{;OyU+PR~0)<(r#mTpf?jZTb@O;i3dxn zk52R<&h5bM)8yM4FjZ52hMi1AGUrCweK?Qu)|7V!mrI_ZR)pP?!*!zcmZqzmh#G9L zWPvL7erxf*)ecbLuK917l9NTdu|YtAqxd&-2Dm&JR9l zyAFn)t}EZmFQuhQXZ}kUve3!%OqHWtw@bc#EqzDa^5J&Jd6fYkl^3_S2k3y_9c39= z0Z65_PCvWvcP|$weQD0iy#ez3olzpWQc!oeoG2wa#GDB^Nt`1n2}f)s3k_3jOxBZk zs=qAog@($`iEYHm5@8&?!*_ve3!&8Ep)-h)y%5rT%Y-Mb^_uGa9-w1Hk}m(F?&~Kp zK>%uGsea~-Z~;6s+u!_ytpknP!nY-nU=QjnZOyZ$1p-N2pSg~{K6yp~MYVufd5oY_ zl+#%3m*U`V_i*zLXg*llFnCz?rTBT;X12T+1&w^_fKObRVI zbjVR>FW8b7kA4<5q)!B}%|vp1O?@tLd#zbs0VgKu`upVMAwAh2iprDlwmlOq`$^jQ zf}8QumHk)SSkJApO5RW*dsf-ov94)7u^V5dNtPj~Kve!H%UKO@1Vi0F!kAQhHOz+s zdJyJZlBoly&Xa}&)J+G~E#e8~%%8j&>FBTD-CVQv=LM9RGJ1FB&b`3P*r5z8^__~w z9*S(3t6jabm-My{1|1HeEqCf$NC@Pmv_lYI7bA>iCaZRQQA)2VzFU)EZi|0e6g==r z@G|2EIe-M0iS^YkWI0{5d}n1QMdCsh!JlZ?;<$$q5A`u8 z`N%}V!zs#bLV}GF%@6cuO@0;yi-O{y02N%SjXYdX50bZ2DUXGSRR5hEgYj`{RRFfJ z2lm?8OSze_0E2+3yn!k63Ff?{dl*pYM0P)6yp49v4&{-48X>?#UbH23`{8r@PpYTR zOm+s(;>@LKy#wy-ltaLWgCMad5o{_mOe4vhKCNByRo;`gF|6!rkZ$Pm7fh+Ne z=a<%fffhEw$q^KhiJz7zQpUJ zH4W;9joS6)+du>vjLEFZgTXNuzLA z{+PN3-*%`t#5GygOrK9Fh$dOWOc*^k$flITC)mq=W~cWqFC#x+8cCpju*L)q?kc|@ zV*5s>A8dQ6v!xyR?(QVVkD7>}V*$6PgxsGw&8}PKxI!}aNzPxnqteXczeyc&BV@-_g#E0XQ}N)*p^7JzpNznTC5!X!&T!)r~dNvUr7(JS*ruD)2qxB5Ga`X7CZ%o%H)LKAT9$!s4Rcnj5q8&s<73vcvdM_AT0B~y}bV4Hs4O3`T&JP|J7NXB}(3$==)&Q zsvXi(#@S)u)p3Y~h?8b3T%96!hf#zsT-Ec~7KYHC8RKdq7Cms8`lp(?uH{Zql-l;V z7>s3ejw??toaU9wZqI7Uuutg@lU#jX7p>t@HTXpT8;PK7bB6yG4s|j!y*}dF*P9(i zVAfZn969h-(=)%lX{-4ppf)YB%MCM`uBZ%}pnDVh!3s5Sl~LwBfg9tD8;i#F$Aj7N zT}g_Yg?QdWaVfY1AydzK*uNx4_;d?uURvtK*&CE=-c;wsLut$zXEaYk(*ggBIRUb9 zfV=WV>|3+;R}H4X|Dw0W))BI)$Ae|T4Qvxpn3Nj}3{&TdHS;k*f7ubyCmgP?Yo zp-uKj7A*fV=_9r!v9)&3OIRJgbo2Us-{W5^&`;$Ys{lFe((BLLTHMmMka6n&;4k&` zKr%1rAfMR_bPcGBAu<&-N0~@;s>E=IMO>IM<`t-qx+K6?8UA#u zKCap*-Ik%J+x<#hA{7Uq2oCTvHvYsL=6w-@(p5Ne&L6Aj+i6I^eIy#&Ivl5XuTny= zzjytIC14H3(%4Y2NsgeIjw;vAYA`=rvaCgR*P?ENIqewK4x%b&XauB1BW`v#cY;2q z7tOON&RmK6i7k~s&)$u6OkE^rx;h@e^tV?AD%htr=RT`IdZvvs^~O=G&dNptE_4mY z%s?zG!-xp9{t5tKf>p<0}dxYB`5I5GD}mgZSSYx>s0dac)m@E>|Spb zKs%Zv_t_!t_uw2XSNJ>ge(AIEUrnPn8s+U52wBo^o$iMdN#%2eF$ygnUL2OEl8V1Phu|Zx^%DQJ}v^;>MxV?(=Wbp-439(%XxEM0q(%Rfy)>pc|T^!+@vS`COC4tXMQPLGT}9F_!g= zlfKK{LrhgCv0WSkL;u+iMk~5ko#>LE6KA__Kz@1XG!r*%P$(T6 z!_jf<6TZlyGf|)h>$6*5h$mq96)Ss73<;#_O8tEN>D4A5Niyef=&$B-SL3Ca+YpH7 z726qY74q3GspGIpzxs=rjZCsQ&-QJm6FZiKIhMJUo*M0XW&0#m(1!|=LB^2tM@i9=dl7-G(Bq|I4f!Y zOM$MC^nvnvr!ZpZn+~yZGrA;ct=Y?WnFRi^L!T-mwkcbLb(ali@(G_#X^PriDxqMa znC0EiiI(Iz`l4FRK-`57T>c#y<6Z}m+vyG{*$dmH`*s{kwxM9%(v#=Fk=U7g^H!F{JEpCpR$Z)rw8PjBz%hN!_oFDJv5Mw zVJ16dk<^jWLx~I0}t`K=TFONyTz%rC-QxHDSC6vTDyk;!0))1nh zq<0=LA0zZ1i2MH^@Wr_dLXQzp+*w_*#HGMs?u%vr={i8KcaHs;O2DGI$Rj_@LKR+o zDS$)8hZ}cH)p>-YU%mNdJD;pD40FcR;x^7PBs&++3#;zB(aC_Pw$!KK^-x51HA2~a z)VuT0+n|K-2w8;2p+I!d#&(_Gkiqh5Z4`Oob2m^)U0XEx^c%-GKzV2c>n1%+zLds}CLaxk}% zdNTyvuUVG%SfA1kwul8`&=kf}PqynlA+=KBvq>*n>2i=1@){badO^AC6Zw6hiSu-| zs+vNyG-n&*lxR(~b5@}dxp2AiR*GsXnrYwqcbI8aM~w1%?&ct84Ex zj0SF87sh&?xBd3EjqO{<`=)LilM>9MEYd_0gZnIDn(+=uQCLo-RSxfg zb@(j8&d~1Eq&Wljp##=$Z8(cETHMq=Nqm-l{08%KBhq_U()Oz1c#0CSdis8eu%If5 zcK@#xK5qc$Pf=o5X5{?2^zI^s*c4;SB_J@J>4 zojJd#KL1erh;7W?0&T<|ANIsT4f;XffJ1=X`hG##-xcjiS0#6X^Fxkb$i{9&K{XjD z&+$sIe5Eyfp0<%0MhjZzXa}dY>TrCsEBICr71Tp|0{~n&Tv6Iq=zF(I=21VP48(ed z=eQmB7t{)J1WlhGO96DpX!bX-xsln$AIbh_EfZ9)Ko~gf4j84kliSqN{LeO6A*zy- z_*vc%tY;99C>_il(4Qe(grSikdtNU85EV+^h_W#`?)iGA54Avr>oX!nZpl6ya0>qEWSE2va&5r!rD5@PazpXKKt z(Tx%!-Mb#cGc}DtgbIe|tYT-D5&!69a!e@Yjrb~FObxk~g-ki&)`CT2e)4dt8 zfN-j4Unm)$au{N7i7fwW$*C4AcFHF%kr9{T*lB-UWU>IZC=(aeTb-m8<88rHs0rgtFSUh_CkZD=j_qm7jO$eN^S1zlXv8f~m<-;r31Yj@At zZFC~MdgQzR?gwTa4M^s||MK4z#h@9VvF6I=Kia6`1A#T?H9~wjX+jYlSADzC?3IL$(PN1{)(Cy1Z2#r! zyxrDf2rsYkYjY@o8nwPs3dbU1*stuyfV)pJ=ZVdh) zzF`)gJPLg$T*kYIfL;nHo5h*)G)aLJ*PChLNfc3R(8V_XTwOIG z-+M)Dvrig@&LRZIGw8wKX?3S`G(t^}wPufPOb@DSqP8wk>#2ozLd{hUIX(S){&@6q zk{&2pDDkTk_}jLf4g#25ij>Vgmgxc3nX%Iv5*4Djp#&@bd6%1;h1FBV%J~^%>a!o% zr7*En)AFktd8HV7J>H6E2ml*{+n%Rv&7do9w!^yjUvIty?o37nl^D_>-W;hlZE&~2 zao34Dj#A6hYx$W}>t}eyz-~#HZpzApS6Y-^djQa-aFF^+d+VCc{-bh7A1inrW4w)> z9zcBORG;NEC$e8@K>}1Y{oD?6yyUBnr6VXw8T;kkcBJ37$qOfkl~06#&aNwY&cG*Z z&IXI?&{@%M>Jqy&)+(An=~yD&HVBi$F;;H4t%3`dsWk?tIB_zs)w<+B$mykUd1$|Q zTGvU|j|Rpx6OTeLyyW#c`dwFpNV~gyL48iZsz*KN`m`T7R)FVo(YlobhS8OQRvG#} zb<@vIrC^2#*l}h&e=t%VXA7C zXn=~P1;PDcg$H)}D7(}-f#zjzD2?&8YV8N6D2VwVuoY-{)1)#b9@W~yA|H}$Dq~hK z@|A=%Lh`j^6F5}70;LqTQF>y}Im>80F8YuS-px=0wZKxq#lj+aEE&sBaa`)8g4*4i z7`YF@P)BuJaL%bjP}-k#(e#Z@3I_1N9Lpp;eYPS~&MR*agQK<3;BvH#+>%0`Zt9P^qjFd~&kSE4)C}pv)@x4+oqH6g85-kKbplO~!B>cPY(^{BhT5mi+;)Ov+Cl zt6(Q^P+(ZjZDCRliDqeg<}jnl%7&)<-ZfGqA?_R@a2=zJ3H0N%JX0HO%}bv0;!RQ;7a4 z^i(fd)}#@kxcIV_6L!KEn+zwhHn7j(j9TJ7V7!=fott&ketU5BcC}&J$+ThGtL=5x zdCfJTlG-8g;a%0-9c4tNNpv=OZ}@3rre11?#Ec->yvV?7>=KO+RDf)B+*S`xh+P0_ z`iYWE42PV4&UAWacKvtpyuJdeBGZ(P_M>oE$N103elO7aklX#)-mGSMu93ai&jkKq z=`Lr#HCoj+e}Fm4L(0gO9P3yw6v>S<$aFMJ^}$?NEH44Y14+V4di_$s{4f6UXRk5b z1$WTb`x&d8db8kcWj<#&UnvJt$&c!dHvt$6cE)Sjc&%Bf?I<$I>hOPi1ROsPSAr%| z+_ux`w*iQldZvSA9t=6!6u>)7Hm=G9x!YQRF*h~?>hGDF5u?$B!b zmFsl8jQAxI#z23lNL{2Ckta5GCfE^I;z+YAiZsdEvqeh4w&r0GU-H zU7HYSa~~5c@k1lr?&qj||8q(|I(0&~^R0^KE=fi5LYNIM)}rXSpGUbfl+ok`aF%0w z80Gsu=m>=J2#`e${#};K#1iT%Qz|f=djT=SIqKz;r_KAWieeU&W={5ZmLlECFd^*o z;kV}g^_8M4v3EJtnD6&3oox_=O;dUrI!!}VpsXw;o{(3mDyVed3x;_tj!E@JXYFmQ zW8tRrXBjbqM003${SSW77i`d$JvLE8GYgy6Ddt=-Ilu0{=zGpS(gg~P`p)G#C))5i z3AkIap~2X-6=~V2)W$LYFlbaSb!yvkKy;RueLPfj)-@IATxY&j0lyC6{#DtS-?9cO zRYd=Gf%Med6G_VqPfPmK13=ai6?v9TC-0WZrs+u%mycqe z)F$1X-M$A+C4RSRS8jCa{cla^e^}9^V@p!{<@VLx4WVQ);f~FPq$SX@5M7A3&d1<)AR7&|LABB-um(?mFs=HfERFt70@{;P&ISjFyk1q`QN>2ePSkZfYD_=%8 z`;Lc4`5Zc*`h*LO1Kd+6!f)WO6)WP2t$0EJ<7qg6ay<)DyhhENXK5umR1lwbW7~r* zm&Y|+@pEr%(_}MAu!Rp1v^VQ%maCPGdc@5p71&bjI#Ueum0pkgC4o=6A4k37&ox<( zqh0}>8HX4MD>KvIIB28(S+&vWW8sLI7~wyABd7VQiC9y}rU1%vp;W5b30Vz`=?nDl zklACCSuvg|9%5bkCBMi zs`~zM}955H& z(Kl{fxRfs))0BDAG9}@m@Uu<%=V?Ss4mwr6^ds zUssdu3#_$paxp`4rDbswVO@XnO^-jp-`B(x07#=leNQr%j9S-iiSe|Ze0~4cELbV~ z;nnP&)n&yN$_(eVi3+e_Vyr+1$I1s|kL|YTutZ;A^+(x3LXKG0@cM(+cg*~T_5b%y zxCurvP-nPv*q%Q=w`3R4;MyfJA#fhB{{HoO^ogU4P;0PfxyS)dkK1Rly-%P1`fepa zq>jo*;$J(UjJ0om16Vye=1qlzVu5^_57VM!Gt6cW^!NMGw%0$vnNZA~Ks%oT8Snee zIal|So^7MH&vm+H>g)8w%2Qtdq{ws8c0hAK60nJQcr5cgr8VXqcsat|9yN8zc+7<= zJzNyKtC8L(kQ%(v@!Zvn;;$DU%XNcG^q??N*3ctl@_$unJGvkLgL)t3#-s`&l(}^| z1V+T&Y=EF)O-sS}D;Ka3T*LzX;$iN??H1|NwDaDcg?9voQdgjz|Jd}}CAj@@bFR()YDnzQNNhY* zd*@8*zP7(#oSTb8LrLLqlV#arWDHi2B_^xGUw9X>Q*Y}N=kR$}Z9KxD#-<|U`I7Bf z#C^lUSZuzbAsHUW3nDZX)MtzaLmvwnse3wdD3|{BA?N*GWuN@@{#hYT?gjh$Zg#Qb z?w4VL3xR+KP3DsKeMaqbe3SfvtXep; zN>Asv0;LMylAsV1{pa(A00n#VXMT~t0u(Je9u)AlnrpzZ$9-H%@1TaxVF++}aGDv! zJC9iGbG+h_zKwFAFp=#i@xD6VSQ-5AMA2}wpe6E6UMKT@z>HnLLlnL#iAj0f28WV`SR zI?f6C3Sx5lp#RUs{r3`qZw?GplF^tIuCI8^jHu6<5&H3Oo;#u6A4SN7CP-g%Br6`! zx`-gG#Vo{DwfHfIBfdml_6__afJoDAj^{c{M-wBXCtua3kCzO4Jw;JC0Tecwxi0hJ z<;E@b^TU#9v%>aFzpIRzrleD&d=plBA|kx@=UBhT%gztTFGB!8Y+jym@eY*YZN0?-233Y zb5gl}b$6&crH6k`1vBwOnrVQ2`v}~3ZG9+d1rRY4JW-CXO$hid_4L<dpB+sl72nM87ih z=>Py{wA8&#J^`d(e6u30T{>z(TS1zAuX$1w(G-@MQ93&Od3b;;{+|yB=&8*FKtFvM z4Pxf!p}DkM_@Z5?_7BJ{;tYrmmO5n0O*^CkA5#|>Yc&D)IpRws`oTBzp3!$6fW&Nb z)_hlZX)fe@s1^f`|yUVqNAJ?B`#W9MN zVyVni59js%sL2>^L=VK+X)N&DPqC1WmkQea_+-5#pZ8v+Nygwyp(}`BtVqQFe4xfXg`oJR9Nre;d+T{tVWFcA4=Gau$8|xJk|5*4h!XX=K^)7({Q}U?qbC zH$yP+zl(V6&Q>{xIO4tJ(@#7_B-UFF#2@`#x);6$1Ryc^ek^Tj!yQK@y`-jF>!tbA?o?gG&|!L zbM3K+E1%PC^HrMEkAR2@8|XAO9x-iep@bHGoCyq4b4L1p$3yHZz&F`s%g+0LT&ej= zRQbqpu@z1wVD*9{Mmj6Dc=O+%nx1CQz4BO23t9(-v5OfA{49_e6yZFuhVivBZ3n}5m%*8%gUiOUGJ7T%CY2w-#oM6U!}iK zaRx?3mxsHHOj=wn*QJ0PNB63upL(W4b6?jFKCV9HV;0lhd>rSgh?^9yfbEqZX=3-I z2AkL8$yhJ5WwTUMxv>3Sr*8npnlNPscrVv{sc*C**A`Abm-Mo*d3UJGe%sfC8R=YGqdSc0yz=QF|zHIy9uVFZFcyV}8`uzXilVYXi=vSQIhB;B0 z6CtP`FNyMr3yHrG#q$_7nq@TYek~itNrJrhM#)_I7ZJurPkpYOXHI|bZjei3^v@=6 ziBNVx@tL@M18C+BERHs@{G^NV91!*X_CeRBtQqrK~-n5q_WkTfD4sEcWG-|LL4p>_^6R zPaoi~9z#cTB1leH+MoFAQCu9NZy-hgV5U@Od$Fy>--Dzp%AbHv)a=(H@DdSLysZ;Z z{Q*R?jLwDc)*}UZmQ?04&(QN2B6NCtB(G*doJt%O&CM9O_6eQS9fN%HM<(SWYz?1t6OcvP zKrT6TC#!ys6%V^VMcKth(=;m7B9sJtfnKIh-jf1*WVYQLBS>+?(i`-pToln>w@_Gn z1mcZ^1)vwV{4pznB+CSB-&6`k8a7bWhohMu{iB+AadagWPKh&fgLRwD$9!jBtW$vL zezcnWnjPpFFHmPxW{myN?mO=HI?C~8{;HIfkzku&vF`HAwt z!JZdDE4+7`gkKyiuzkjZNK4*sE6WH>zB}3cVYA`~i1tDO?0}@l_C!~&=dULoYvO2K zQj4Ws&*=HH2z8mePl7tsGlkL1XNiAcH@@if<`2fa4|VNYMRj>*iNy z3YaNvulP%ED0`e?-MC8!TmY?dl8(aZY0S^!U@Vq6(1o~X7eG`rH`og40GySCnlg8_ z_?51KYp+;Ls$Q4G&eP)E^LHv_K>}~90lB$gMV`T4-^ zZ@uFsVAHfX^O;g`uLS21<9&3Ck8w?`O{`?vP((YH?ZnjKpUaD%%JjGr#eJ!R26P|b zgOh|nJL#6>9Zta_qel^E#j4N`m}UY%LW%y7gxa*Gf68<#1RBsS`(vy1EL}5#ixWygM)z41D#Z*N3?`57;Vs;C$)PjSEM(@ML^XAG!!&vt*9e{o>UuDvUTB(nP6{J(5QFr1B zq^cMgBH!vYJ=MX2=)@dWs76bXoBIAwf~6mBPtGTo9wT^t^BOR{t!Xg+BKUh&{&wzM zJb10NglfeDDBD+|CwGlNcjmY8m`ec_80nXqo)g(f(r{6NpY<|SPP$p zyQ|WGD?={p5^HZcJAmkuLb`l^em%660?}@$KyIllSWHd+Ur_qP!q_F5=|SEAjDN&O zV<_~*3gnb+aD0k1PJLU8xJIW1anfdZ97N<+2cAQyDE?Mx5e8T287Mg0{>0~ip_Is z&4zSM?$K}M|CTyBqE>Hhw7@HL7eYWQnLzbJ3vpwz2xZv&MwP5pg;6TCD{VuK{n7-f zB77+@=1lyMecyb}-qEVH0280lVw@X+zADFuN)*rGa)X-gy{ZnU3-pj6<7N+RnR{e{ zI@bWZQ|SU_YkqJsn^X2B1!|>%34DXR%-J;}sh3FRLS?iAn7}Y=Jx5vCdLuZ+@INi1 z^$v|CJ#CI!53a~Kjz(1E-z`V2jqR{8U80%U82ns>VqHEQ5BrkK^t)A5+plr2^5?`w zMAnipcFY*}_%3EY3~xUuo^X`uZno~IL=(|wE@OkLc~ zZoj2-Rgn%#U1@b(b^gAL5$(*nTp8t%IfS`$DUAw#r^Y3SHIPTwJ`8^na>(OCczLJe z`n9hmtoc)e>vDJJnxS)>b+DN8#qw9SdnHZeE3Kjb28jRr{;0Ox@>S=fYGG>(0*fUW z#M~6~5=%4kO(aL7irrLE6!xlq@Y-=mg2z~n)ZgAY^PcbwXjo%B;|2;7oop7&7f6|O z*CPcR-$K82af6s8bayWm2cqd^BkojY*(ncQsi&(v%9UW`nIj#7UN@{L^y> zn@XAAsT$=Lwd@ zP?vrVKHlK5+5}vq+dzjwG0z6AvBUmh(kp+l;O@)+rwfyedg23jzn%=DC!H?`mY)RA znTB}VFT2H`K2eJ0gLFn#N0MR9UG^&(nFp4WSL0N1FupT8y9HFanJ+Y28+D;aO7n^{ z-HM&D)PmirylrHu+=e#vg;Kuf`pVdxFK*q|(k?rLz-&j$1wod7>l`RI$FgZZ1G$hK zb?QN{UUXLlQcMQVjW4oO%zCv|g^RbZtUzdZ)Qo<%W8C1pOCmQ7BWgE=MC0IZws@8g z<GS=+wrA4LY)rR=5b60{(d@%rWAL0R8cqgipBzurPGVI&+Be6U zcjAC$Tbg&hrld9Mzt0?cj{SmfT-vf2Xow?j6mfPnELMHrmY<@K`1LhktabgPrtN5$ z1|X=zDuvv{$`|W|{1oAlkEHF&M38eh854O%rNt9ThhMp{>TY1QM~cw(NcVOP?0!=k zIK%0vX50`p>TL`w^)47J;T<^9!5K;Z%O1*)=J8;Rv!O?=8UthiU^%2imUUP(c4tI$ zN^>${!7(@1?qacO8iw?sn!o8Fp4d{WSdTaDptr$k{|B^_=<$a7NGJ(5<3~Zp@Ea*S z-b;#j+)Wj(gvXHl-{+Yq2Fzdt2VU_5cllZtgbbZECeD4duwii35q~E(gfC?8BM~EH zx0#kJahWNN`s7OqXIMx6*QeOVEnd$#nd(VJcKB@}y>9ayvA~>00Um0?R$omxqMtk+ z#LT-MBW5u^kuUEkvFD;Lx7_#&ANjo`r{R@#fg2?+O6A3+Cqu`5Ov8*PFSX}0=QS-} zou!isu=J{=?O4_kAW5IuU67-Pody|hN|KTR5hgkfbRimIh4X&{Eb}rDNhPpTP zUV?&X_8ie1DcNApQ}r&cB3VWx%j2e3tNB%wbSc*nxx|3E zr;~8v`02l4=d~K5IZ5{ABFU;cy~4?JhhO?q4=1?~^G3#R{5e&K|onL+xbSf>BK_1srI6P5jawA{)2?igk0qZu*z`l^g%7XV&owFJqwLmXg*kw%~QF@cveF)^1 z(WucS|5kq7hk>m7k2Zrs-T_*x)2ccpo)A_e=IAdejaKQOQWw$r6WA`5Yf7SAwtI8W zJSc{9A7d{|3Xq2uX<{aOxlp`cHW|(L|U0I{zP4R~;2~*L8^zK|mNl zx{*+jmK-{jHV8pdO6iaqN@@Tp=`IC>MkS@YhLY|Yq`SfIjy~`EKHpli<_}%WFYdkP zp0m&1`|bnd9{zBrKqu9$C5&CjKLDeLE~$`G>Oqm=8&*?Ib*zS;Gk9Y2~2i-PWejT6~rdk=R9 zquA=Rg3C9tzyrssm{6j1cRj1P%6O+j{tp38iWu|$M9!w>=sa#B?dFYj(M$-}nbngr zhDlv*C1%TfWju5sk{-d;9NoWnqoC(HAvKwLy4Wq?^8KU4eb=;nF0bg=Qf7o-D;&|8 zm?8HKBVSIQkqL4^f9#Q0t~QylEwVH3F_U2cQY!D_kb{r=tL+@OGyRU$exdgKU(Mu9 zfGn~Sz|fnUdO$vOV^=g0esnx%}yu`&dFD}NTz;{ zd~GO!OtDPx6J}eCP>?5U0+X*fn2@Fk_+D8g2our>3C=E`#J<(fwiHqdQ~Og+FzP_8 zuo{d+H~)c55TBz?O(RM!eN)aaT?rFo-(6 zI00uuHVuAI>Agl^Z#V$l(I8kqRmu#TkUiE-PRh8HBvan+2B~oO2 z`^4vik}4~@!1G^%g6gi;S$mOIV6YcE_4+rdg3HD}xti}pb_h_3~wArPeZa$8^cw%XUdYTSWzZxyNpZOh>hviW|rlA7x69-c5N}pSQ)1!B4erF8xh{pap@ADMIPGbE9D;%8J zZQ&To$^E@OlcyOQ8K0ezKZuR zZy`P|7}oCol6NYALUnytz(L+ORdG=jQ%>RDiJt&A_w*9_MBi^=hF%U%hb1ftD}Zvw)US98TT)SjLX{9@((4bB@?Tgs5|daYaU z^W(J4ZXictXLRUuSoglH0UVB>4IeKv_g7?RSUde$>~jlbH?JQomgT6snOl2ga89=G zzY{B79Shrw@r3jJ4))ha_G9|-j4lIYs%OyR>f#jB_dBG(wk?v5W(!1%{tqe?dWjVK zLD^+sqQ}u<%I|`#AUIntWa7GXX{EeDgBt$kn+ce2`L!6!L>jtidpzB7=0bM`Mqh|k zrc>tQV}~fqGf8&CGsA;#AcJF4!RK_Acb@r5b^m4DPhSbogOcr&xya}_SEBawrMQ#A zXm6~U3KrJB>zGGA=?5K(LxayF;B$Ji$>^ZQxJah?qd5r z2HN)56c(p+$Z~sePzi2^xQZ8tloL(D7ew}&J&EcfMAyB;?Af1xZ|`mr>p}!cOwP?Q zlos4oj}q^6=3%K7vERuuIo)k%NZo#RF}jzBF)5l~n}-aM@j6^HXTUq$k9~pDRr6##BN^=Z?EAG(#YbV4ryb-G+zG!<2*D;J_6fEFy^{&L-8%qP3y!1{pOB*u zx3#BzFVs@Y*IDY~;26=gZUJzv3Ro)*#iw-DJ^!avgxBRgAWx2V=N!+sPtt7b=my3W zoF(Z)8GG%q%vA|}Xd@Ud%vi?W5nX)rJgu$uv4Ke>;Qd;#_#td4LA2{iTVsjLC}E+F zn{!pcxy#}8)R9DK@J-^KX-3=tG6T+3VcQ!j_1*%;0C#EgkHk(0u8l;#I!k-3dsg*; z`ueiYpGV4a!h&g(vbE>)*y2r<8JkCPa?iZx3~UgHAd1}!rf`Ko%TWEH9uE1#~*DQ_~nB}C=3r1zPwgwmwjN>B1p-tyLTP4mI8p=OX{t7_&K z2M6&laBjX>Q-K^0`;gk&b9A56h?3pW>Gus zlPva23rXExejH=U$bo!|er#cJ0Kb$AXI|Q3N--#4Y6hVJWPcLo9i4Awfj6GNR9MbN z2GFWFaHSX5Y>B;sGd(OLpe%@Ei_&L4S3fh@K$iCS6%D#>g0=N08ZLb*LoQ zIouo~d16T)_3|dvXHz&A+MwJ;~MwUc!Y)y^_-@m^6L8eDZ*9y@T><;6>Jzt{07a>}rD5Vo- zF3FmISSuK4-0@o8k$RdyomY$8XFuMB`sw@o7Cn^ZEB3=5wM4wnzP5%2!K|U|cIJb8 zJ8-gzH0MREDq()_tib&nkHwgD<9Nair4%T~8PXY!KA3bGcdriQ7Vb-jV0_2F9 zcNJay4I0Cd7)Z4>FL^6@mJDd6MdQP$?>Lhe<3=@PYd&O1p$RcGj`zd#cev37Ga6a! z@~T@it||)MDX?Yap$V*2QxQ+Dqr4>M>SR|-d4aX@j#Z_g zSG67^d|!ALK6Iu@$-4WRH(2c;Q6xbUyZgXo@Q{Xs=8}yK%etN2$dV>}hhh16jw?wV zJ3~ell3Cs(S$j>*H>!@w`)R^2XQ%Z3?GHp_{ zYo{{qN~;+HmXW8Sxup*>I@p;7=37EZS4L^;$8>{Ce)hC0bP8yjxWIRLLRe`c825Su zqtU5E@{R@zE)@P`Nh4YvCM(n=Wc`>MVk|bWlqiEGZ@5eA+SSyJslw+MFvq^q z1yuE$$@3VDGUv%72ajP4TiVi1GD5b4h_T3Uc--;ZzGK&zWeF6M;mV0qHOxBwPmGl= z!_!2ffdL$K3eX(J{!G9fg)kuR&+BlT^H7>Z9AO0E3r1<;I z8Cf&y8TWy$RldN}Czhodl7##2j8PdHnfZ26zx!64Ud||YkMbRxk9gHBIvX)iwCX8U zeVP@WRi*T8ObNYWo}Dz!k&^R$SI5caA!NcEG(hC7&(+D-XPT8Lj8SVZPaWuT!EHC- z`=^`ge;W;)2bRZW!}j{~n%l3FL+zn&p#D_%EalVG?_%+u4@65>T1XILv^>8E8q%DL zx2Coc?6CSeuZZoRq}cRk)r_5|{-TU-vfkI1v>U2s?Z7BhGxu;YpbcM}M3-*=2DA34 zpco<+hxpW39h-dM^c0m@A>uwAae#;=S}6esH{%dSXvn+BdrJ~@i4_RfYANKt3%-Mm zff#kjGKdQZCLy;YAdhB{x1s*5J7gx)u)ZkxUO5Wh=nN1G0-T3Am7pJeWB1x{EupWf3+iaKCqn>oF=L-#vh|NDz4E7&nz=Vv-l#S}Dkd z3GSrIb9v1YpK{oHnXSTLo-`&KtNr|CmGyT?`s+h=NnmOkD$al3)>8KBN&BUnMXE zBn%VsHG!74*@zX``h}=ot)(0*ssG?@>Dh+KhN*Qf6m$Q|q4DODy~6fU%ze?zSkNIH zPpvx;{YW!S$mWfI=Ac^J_)y|uJyv#_B$@Xuwlinknu23V;M8V&65&XE`Mq)g|1=cU zy`N~eby?*y$zR%c4%zilQrPwU&7I-a_o05^?La*K;(?Tja%NnkCP78g+*>R4dR9ZB zv8I9R_gB~B6)q{xK0C&w)q0#|9Ea`cbF5!h;c!xVUg5L)24xVf-Js-;2gV>zD$2K! zbj9s!1Htn^uJ{lP9WCxTTD$Uej^1`lx*Zf&^(j&D_#E~5wjQc;1ogomR->H zeP$N4xj{iB=tQ}kZz5*^yy+459}zEr{rLHnpHke)q2eAgkWxc5xdemJa8C_81czvZ z3NbE7UutKmep-6v15Q`8;l+GK80a>L(uFv5b^8eBSiN9+{rG0VVCfNK{mJbm5#51~ zoKgi(hd07t_gTi>Te>!%7EE{A`E(~N*_nW+i?q%hzR^DHrAjnJlAw*5(5p{y2(6hhYr;p;8 zYrYxfC+mm8-`p3&Etxmp!Cx$kY7W^gEDQ-QS5|nwaMzbz-IWq#2uB z`r7)-KALEvAGeZ#ae*4bLbq}tzWR!<3X|+od}impy?inv@rrzRpf5i|L0(BU-$*ty z)U^lNKOVK1G(=(2Fu3U8pFX=3dzGUxUQ1oeAWQwe+y0E1`=xW;s$0KOc5#SB!lt$Qovg<>i%+7l9 zZ9ahQ0oc^2oF$xk8~aa}#WhYJGT<17G$@Q}^&%monxBl6?&l}eu-r~UMYovjF_?HNADS3Z%R!JlyMq|F?c{YSvOYbzFj0`yWizDa z6AbIv?n@UhBxsSDutA9|SS*OJjSM+735!K=M&_5c@=sa`GUXvt?sqRVx8g6ea8nm9 zMFropj@DRUmU1b1ka`KbmtdmQOq(dEn?H0y1Lb3<6GIuk#*S(AeIG!n?jibnhqWHk zhmD#*ibtq+wSCFqiUo0`3zVsgRdtjQW8;9vYXz5+@`MLCmdKFi$y}l$yJ_;s3LARr zn=~XvF9%Od^>X3=6d-Z*m>Cf;QV+W+pYu(&qaAGu@_3lWDq@*)r?$;>A0_p>N`j~~ zkUiQG5Q+SkNVDiv=>q1vz?nj?Ueye}?p0#sQ9WS5<$65YUu?1vp*yjm`nB;_e!qQg zVIRM(bD#;=hh(eba(b=gD*}7}K_wP-jc{{{i`rsYefq|XP;-Jel(q64N7YecsTcji zUD{c15z}#)Y%|4s!U^+pZ$!MIXFLZ=Nvk2H=?#v35wmfzp;zR~PG8_SioX&c8J^{- z#zkEkFQJXyN|sLth^>-1$k)^U97gw(VUA5w~X(a6rI-4Fpn zerlEi%e}et0mkL%J-Laui-nW-uMoeLEjg_J}UfO+0=LBfQO( z|LFkQZOy@S62%}TS$j0`YUOj#XzQsvd%9PJE~5_)10q(W1V@GpSlAyPiB6e9zT9{ZCL8> zHd9MHCn{5eIY8@M>mu!uuusFsLV;{{xrWkfcYX`sQC+Ia2brd(0cd&=hkOdaok4c! z-1+2&E17?_FKzfMl`+fgqnuaj=CH=sAwHL;j!wdK;;Ig zn33$yY^aYo6>Q>a><=-VK3Ar`Jw_Tpf_XCwrnGcID;}R)vm}AtmxVdp76*%iC<03( zz(ph&Mk*Dd*kVdeh&xBXHfIs2c)Oi9Mu=PAA^*)&5u?IPG1t$b^Kle0NumBYV&{;B ze3>>si1oV!=QsGw4Y^U0FE^mQj+g=6G&tLiLjvcZSB^!HiranD0L_^E$&o>EQB? zu-Ka-7ODOFndh$Z={Oo|+(fNDbRMGk-rS9~{YcH=JteaMns*Il!jd1Ch~I*w`vtSu z8y@Wel|=6Q_}K6_i#XSG?n3T%oa_o_Sv?u>lT~dMrr&zHqtNv#LeK_x#vzw~&OLLe z!)!cAdx713xzS#%2e;>{)d`-;JTch7@;owsH^jK_i@9z9lU@7b;Q<1v6p&mlW&UA6 z$)_}7<*DLb4r=X<=8ijr_ZH+~E4c}9%D510>wzS*63hew$$niHBM5bB%QbfD^cJOO zy5EM3v$;zAsfri-IS%(JaO5EmislU#X|xa|&6yl{7qXz+fkou1%9mK{=riu7Euj%JV<1I%oe~I5$H94XlZ5*L>p~nJA`>CDh zIlGd0j-nJA>{pZB?P-T~Q4EQ3ud&56z3vp(B6P)oOc+WykZ6U#tF)!@t4 zme6HJYw5@~&6W(o1*Lm7{oIUyfb1r&i1=3~gFxuQ;p0uypIXVxAoM@JjW0-VCrvG> zi*@qK6A3&9DI^+ZCEmfy%%@Ke6*5tNLw)H_mp{~ZdiAJ zI{$^{n1GV;7j&+wT_6QDfPk_z0ngD_NY%P~P33WxpQXYdMnyjS)B1}iMT|zxu%U1# zR(I#lJtn7p1&;Bw;u|C-TNe?>*bD0dKQ=CzkSx39J)T5Mt=+56vq;qLA;7-WqrQ-> zz*Ia^zZ4YoRL0*8QO$TdRel))iXKu$`Uw5k8y-nv-W(5LA#Bc-X_8$k-vZ#+!W zqu!!$w8*t+p_KEKyEIU$7;=AVe~JOOxd#9F1bsWca7(d5I3<}snMvxy=nzg>zBhsk zeDzd%w{ITF)UM2{C?zhIbj<(`8s6F_OAK}9FTt1Ym*H9u3^I1Kc;sIlv8e3@V?<_c z73W3^k@qb{xsYbA^B|LV&tQ zS)6=!XE#RT$Q2)7-mYM^)Pemu{msR;2muOi8#xe`aN*8P*Kcwp+bTeeE8fcJo- zm@G~Rgjs~NHOX96Qx+dM0P35EA~K1OWjoy~BWi}}0A*pF!4&@t5*F(>yBHVgRhzW6 z@2E0ooSYD zY;>6JdFulClNGunjy!8(T^r+1yc6^?lEc*Ac;qRQTfwH>aT`H8aCYe;~je0*E2X;Hl%ksBRg-@Hh z=wveC(F{14B_v8oN`a`Pc6zTK$^J(By@z$miM3VrPsBD5i@G7@47-e;sy+jx6akXO!rYnOU%I zPZUVVq)ohMcB(D^w>7KB+EAWhDLjjvO}N*D&oib}yIr|>(~CU+J7HN$vhETLzkCn& z_CvM;$eu5t{SrQR&WF#q4~4yk>^p+yQn)u&oWB^|XKkqJ{(1VvdgfiV?;XSZuj5Hh zT12-Gt3O)TsqBhu zR3M+;Xb(sbJ!jND7Yhqkev1(?Nz!5yc^t{TULqKkzg#?28tEM>e!qlpp1Okd{R3ko zq72EANdgSwu}F*v_ukh?Y%C$!N`~e--5gSw?S)bd4-fl-NV}-;h}RjHz?k1Tl|j3E ztq<3)M|OLyy9Mlw?-CVY>80EDH2XVIa8h_i2p7kLzj-1FTZ3~viZx&)ocvx{MZH67 z+p(mT&WV7C4#Mg>??*L#9KOs(J9JgCz}#>dfqC-~|G27RDEv!YGEc-AelkeWC|$3d z9@MxG4Z1$5MQO9vD1%jxXMpD-oot1hhrb)fUANJGBIZ-RLF(u+#npbiKg*5PUg04L zr#o*MJ@s7JX%IVNlGrXe^C_?#_iabrS{^?>_8#Yj2gJss$x3}6yvDftt zDckzxZ!MmC)c(^MM@Zj+&WUzpiQA{ocLBV)PW;W6`#4uW)MD`25LT7yi>X|dZ&C%V z-|5wPrLs_XXi|e2G0xKPjXGYusAvagK>O})?wZeTS+`Z3I>Dw-uKp?diTxk3G?YWNXx*#<<*ihB z%poM$>T=vNb9{g5dQo{V&bm{b?;9ys_^18pTe3xi$k(T2fS)d1Qla$SewN+>nPGb@ z8esM>M#NveK-t5Fpm|lccM&22@%6HO`gvU+-T(F=taUVj)umNu`hdLQb|%Jg<<|aErc(Uh*C7qze*i|6&H66)pJ@Y-e>C_n z25sm{+b*6<2-*&zg}L+g0M;2ih19Vq@LoX=4`5%D7Sa6SZiHjF;%y~fNTM?SAMLy& zRMq`%qlaz+d4Cl@Ol2JJ5V79D97Mxl*V}=z z;#Fs~@SnNK*x(i;_IiVIs|rQGZ_>vvbL`=mW)v^`XE3E9YqH?(pRfMX{rBC!C2hEm zU)KXV64d0}238DnX#$q&t5a3ZR>SgT)t+6J+ZMxm{FGVwI%IuJ zP2Syc>yXcQ21lER=A*hUY6?>SG_&{Qz-4?YIiLRfG7oW$e@28uE^ZLf!$cU9%*+8+ zQE|PX$Jyjijk-!XpM+rIHt1lPW8JSq@}v=&{8VD%a%T_3zRrv(L4t&ee6X`PRgG6Z zAZXxa9IDdCHQU0?etu4Jr!wpgx-b4uHbJzk_|KrCpLk9mt5U%lt$_eEc)i5!=^Ei$ zkT_*4MbT#$6mYR?uT;KdR2KPt^xldMOF=Pf;#kDBanvYiXS|3w<9Gc|Z>AysY?=sH za&1RZh%Cp=W}d|3UsMwAIY6P%-gs3Jz#kMXzWk^8_%Hx;KL$mRcR&2ew*EGTBQf51 zpjpX@+|QeAE2haor_Mrq1$VWIxTpSXjgxe_GX%2V@?5g31oz1UjMB98G(Vb;cenS8 znb&8VLuALwg-MVsS^}PvtOlnN`UycE@s;1?TyY&FGS2w}1uVI5_NBpmw0JEWR`|(W zz)VLk5(6~ej^FiA?qBaAOB+J`MIXps6r@Km|oO`rPe#+kWJ&8FWJyttScu zkbkhz<(kesXv`-XXE+XqX+rjt=44;=)_5MS?fkx){f*D{dr|7jj8ect^-v%%Bh-QZ zr$!@}f3A7On<5{s`;!6@?hna{Pw^y)8+cy-*x2iCI_3^nov)`@=6m+~%cjeAADr%| z7e!w>zLC$V`HP^7ry^b}HX^QPPfpIdJMQ0dRKnUCceZS%04}wt(;e&kZh~ z_+G(#XA3@>(TD9i6md?q@ns=X9nhOtE4o~8OxKA2*8&Fu%{QXyg8-@xkqN-j%V((I z0y!!X;F@;0d;Eg1aL|f6wXhZRqT^B+AWEKodYs?X7;s&US1JIxnt~Ij196_J&T1pk zP5dRDrgecBnT3;vHgK`z8InJ!i=m7>Nfr8ORgWK|>B)T;)oCT>edZ$Kbz~#rvG-z$ z`CL_m0&nR;QPrHrb>^2y*Yk>9~tnf_=k`nIERKSZ{545$nZdjWPQx% z@TZPaH(`!&9PxND!ev!3FvpdftvODDJG?J{SG7VaxznP7P?vo0!`rmkDrZzMpjVB6 zQ(xJS{Xl)(&plr)&-MG4hA;J2&9-DX?#NGS+Rx9+#2$jl8UpANDB|z${;k~n2s9k| zK_@dLg9yMc4>eX8v&=G!NMC0S^A5Dp=(s6+}d(^vQZ+80}3 zwntPNd2_yGUNWTD3kB-556|8AqD~@k`wl=)E)jo`J*QDJ(B1&KV{tP8+eX*~Mxt5D zmcQ|S`H{=mDMostXq`Q+$Ry%vrcr{rpC1L75=BJ-7H&2ixLK*&=lJ(}xX zc1K{|)R1QdYIIP!{c@xrSq-snJ2^v)w|S2lv+k1w`&eXjp8&SW4=^i^#M2qhi*sBa zv|Gua?tHyBZ*%ztCwGFa_jLKf%%o2m5BsK$;>g8~`|J&e9nz%Sla)A8iUG=hrkv8; zV7~2cyawyP-w+2D7UC9yPI%fSHgxHP-}iXhO|IVO^5u$RHZ_+rzSeRujUBHeo{0?D z7qTjG8D0L!wlw@y$mO449s3r_(M7t-h;bXlQcyUL_oedawq_!@en0^VB4uYf4dya7 z=^^vhd*Z%3q~~66LvDHFgz3!tId`~95v75In8;CXt+?UJt%X6P2S;Lej$6?awp#2c*i!Z ztgLL0-%&!-@W>zNzTmzzFghO+Db;smFl5{puj8$GOQ~}`tD(H!RvG`mc-G9ROEEzA z@^;^CrP%ahr*`i^QE5#uM@yhtt?k-;e*vYxRDc)J7&+Iq1JF_pPVE9*G}*R0;T}G^ zu8d*femBgj2e}&wxs53EiIHeAymJ#{+%l z{L}|hS=ss(e+ki086b&D}v-dvgUQv2m8zr)-)TfVnt{@wKqOo*KHd9_tmp-dQ!mL9&UKz3hOc zM{#lI5zfDJK1M%Yf4?C>+xA2Bepx=@XArW{r35Tg*JsbiKH>8%NQ*Sxh>CRh-jZFt zq}2`pgW(7aX7(*{N(YvuJ;{7S$C!w-;IF39E2c!SRAjv*;VpHD z$<#GRWp}Di$cqXT6St3-54^8#Bazx*Sp9is=WIs(u*jmvzU{`khpBj)vGn*eP;D6E z@NmkqtCq^Tb$dgceVlJfo-UD=&mFsZ9c|zNjh!Q!r~B4dPf(<>J>NsTvG9nhgVnt* z{Tr&Zf5m}0N>cgB>gm1PK51q}O!X@_?$~#s)$tZ+2?3WzE8Bm_;vl?6D`2@iDov9w zXuyK^rHRgiLvrzPS8|o2`mDN0p!{KvV?w1Hq0%BQ1H6;Ta#s7&)w+@_ZQMq6Ff_$_ z#q`EJ?L4{6%>jxEM)(+|Kke>u^?DI!rWvLh-EwSr*89bUlKf5~qezs0S$ScpFEaQ5LZ}HKL%!w~kz&cI zNDU@pu)FFPwR}JSODDk+=_QF;Lhh#^;kWN?%1RYKSp0MmCY_BI)PJ;8JpFehRaU`R zf53IF7)eVdWH&{5axwx+kBxYb78h5GA=9CUU}RxX={Ofhk}r;R^}K*RRO0gM5veKE za{RhXT7@NXXE(Eog`{d{pt-25y#cMIZzf`LU1DW8<^x&n#x8%pe*0@F3LDGh2L7u~ zN4d?v3d5x)Adv=0jO!|afs+`7_znoazpg4UN!veU;E1{ zPms2jctGlv66g4@xrxc{SLx4*1t((R`T>a179UOrlo*n~Wp3+?ikjXS3BhH!70TnP zCf5#v;!>0!e&3Rx7fkc=WGWY1!uG@FBC`)uoyVbqpIzLs6?bAG?Z~&|$0?`!VS^EOyv~4$FqFNy{;yHCD zPM2)+0Sh@8kCZL2=y^?wSJFnj{69hY1zMtMGx&7sFWmL_TaU>v$v^xJIG^<%<1^#Z zD)-%mhW2rObMwd~S+PI6-ve;d+*ua6aRk-#0_Ol!!{zyL@F+>jx9*g^{rV~3@Y`OM zOy|%Fa}J9xg)S!57-ww4Epj&=gbMTt^-Tr2FbKAGF$iT&2x%ihzDrj-ETA;3-_UJGRu>>Zra&0dd%rX#78uI17g*kv-RTCepjHld`9gGoqLr|MF$XrXh*E6?vwmB63RsNseK@l z_am%}Ny7i&s+Q#9P}8Kul1!}rNyBF^HZzd>d39XP2BU<=5Xx0ezuhuH`l7bwngh-g z$86`z!_idnbnU`tL3bYKjeR==-Q-CRJ5c^w2;`2T%M)oxnflybL4MZ1YD09@f)8Hv zj(MbXBxeGUo{W9j1<&ADI$enM>K&||UexJ!EwJjasH%It`Aoc4$nodx=bLCXy*Sns zsuGzb?pCDdV?SFbksDgWQE%Q&UM?p2dP8@0w%#`h=23>Xf5MR;DGN8>daNjr-BBjn zfQ(0qhSak9)sSB#648kbbmJtQS^HfZI%2WH|wXbyxL?LR@b{6z(k`OOTdZVw()3XyU)H}?}iuhc(kA1a%JodxO&BN2C*!J>?fh9}ThtN!^1b~WIZRrAs78@i{ zi_>vzv46}`hUKyW$RQ7KJ3>5ur8~#}>s5q>VdPZlC-0w~1Lb^g7aYiY{fWwd3cAB% z+0<^J`QySjT9`_6^KO5u$0g&&G_Z03Qy2!$2W2yMTN}Rf>R3c}4J-t#t(+bE=}N~% zfCbnCXzY9C>Sxb9hhi$E-UYlg(z$K!?v|S&s1&`f7?Wn*7bAQSD}U!Z9t-HTnXs#l z-}*0l8Famnm7_jk)KLz5D9@$#powAIj}mKl zmYyB1TV6^YpM6qGrD0atU$pSA1sYXeknGS5n;8?mI>sTSrX-^bIYZ>iWn@R#v zdLAdzpNw3(J9Z7;!!-7_&>8RJ>HaS|JYQ-Y@PVqeqM=>pR4Cv&%Q3d7fV?)tP4!WR07)ko{oiGl{B z4$$p+o_wuaj4Yi$nhZ&eKSk@S*)!oJK;=PB&SOZMB4}+|*ZZlwi&=+HlIi_bjJ7&O ze*i8{@3))w!Vy#gY*X<4UXV^swrW&6{fCFU0?ba3v?{Ftiu?=Upksg%7A~Ex`O)_5 zto*)?p$98S9+3sY?5at|lHpL9hgu?Ci`H6$;18gf#a`5cR+PZq1v*h-&w!S%0MYzf z$%{@BQa#7TQjosTtS$z$o(Q)uTmT0y{=~<)D3DI!3r{%&R9zWI^FD82@cgXdvL;b{ z4$s&cdtdB~2{;frJ)d&iYYR@8QOyPDL=$&0EgZ106ZGR2eU?FOzG7booU^h)5PoeK ze3R~F!g3mEhE(75K2$>v?QsM?olMbJc&TX@ocaHIGVmXo>KK?{LJP%QLv0>?h86co z0h7<_VD4^JEi}c)mrc)pO(Cq?rTAKiTqe?6F zdLzlP?ZXWQ(`B>t%H5`V*gEk*D2>$dPOE2HV>non;|(^T`03e(jr@MElHqRW?gQGO z_J6mPB{V#m=KLWI-EaNR2MAmT@EZ}Mi|B*tcZMHp1Obgy5V%?}+8N7!{BBN3FM=T+ zPyJ`8+&TCwG9iD1S=9TGq^>1QM;ahe7%%G}#1HUBROpHc%!L&1BPGwdXr+8$z%+q^ zEzc}Iaz_OY=ZLHBWtTTtXv!CwBpyHQO{<1FQeakoKh!t)c*Ot0Nj(|YQSoO4Ln*b? zqAI&ahG1mn6MwpTXqQR1Gz1QOMqZI7EE6y{K!hrH76>tL80RJcOFgw@D`XLEBy;XT zZ?bTDzqkiGPPnBQB{qEY9}j4h5BiB z2eyhN^q^gosu5?aK-RmWlzOpnOk2TJphT;Pu_DYYMvBQd+UhRvVi3F&alOT`E0prB z$i~P(kDk({a$QmFUYC>qw1V-P_K{geVS`7bTV*1)(1qfLMc)y~NRon10J60qysmeh zsDeC%0-F%AkY*|3$hE5_lsOl{(Km->&+ZBY?~>JBAvzwByd4cKKD-g(OD!nW0F`-o z_`=^!s^cCAu~o~3!++n;-|sS2jCm}T#V@ETCIP0;#N;=c?uS>s|$D9y=JRYi23%qm*t+nG6KCkiGR7`VN1 zS<~PehK?oID8EgL9EczJ`*+o!qIYfIjrg+vO@4%Q z0xqKkqaY!z2rulQU+(Daa^l-7YzC~>Fyff^%;C~46kHUib2JI52#vo$Y(tQ)F?37__paO-CFmvRIr%kGeiv$WU^9?f#aa+9u)ZRLE&U%F`fK}d=v`bH#x@WO%@ zgNatfkpW>mzPRJW&)#zC>EUfBzWS#+Lr80pg1T}zUnhw4zgdX?ASiUsq-~+1l`F@^qZYUG^+-Y88jTisvI`zlgo)r`L+~@ z1D-FL({vlwRc7_gH2^Y@nt|o`bc%6n#Lw?Fk|xQRa)GXc{zO=?1@R`h>l+mNH?DX( z6}^&^ysbPNh(R|a+-y3sqqjv;H?YPba;Mj4%OZFdcdx;2?A_JrX+I>^9pj-ti8u+o zCAI)%jsPxd(vDDRg4VT4&{eR=?&W9tCfV`g(>jUU#n=fymOZSbSHXy42pZ#ff8ji@ zPjsf$wSr5DOi33d>{v$UPbbXV^MzEoeV+vBY~If7z~aIZ<{T-K;W<^fu4f_Pyl~*| ztv*$k;Za*q*CUz1lMt%%w_=lqEP#@&-P%g;UnzRlN8r`R7`D$;fv!YB=@kSvmmsub zX*NLhR&s^EH6T=hjT9l;aiz*^e$fk}N0aw%&>pJHhPieva2#mW0%tE)gxNBf)5;~S znKDo({L2phNmUxVhr5^04USzM{il;#7WZAplVYzUL)tT|BWo*@w$1m=V3|H#&Oc`z zssF>94+ELsXI$&QI>x`jUOESlF~_)6#JIta*KVrvPhkiUx8UXO#1K$Frz|sHb17{- z25G~WP}MR?)q`I;D+-MFFd+iJ` zl;`%CxnjfqowqPzB@#<7&9gY@Mxc4ff_76Pz~E5`-AKNOV-sO?M0UARN_XG{I_ zY`_};cdj>8m0O) z6V{_%TD{Hj-yigU$0nT`PUN=-MP8?OZ@5U=hxY3jdEegvK5G*%966m6%;z)*6W`u2 zS^_$DKyYT|w+sdd@nFtcAaOmQr4k*UHw8zAd6aQ*o-@uYnU6ni4TtDT*=&|5gp((~N&pSnVBiEdIYZErg{kA&Q(U|E@Z8FZZBQGX zUkAzg)lnadHdWPQ8i-UqR{!r#wukPIk#`R5`vA5M@jgEa0XY(_nn@_ef;P7$Oynzt0!`6rr{-M zy`ph|)6Nz;{NghY2S+$4tuVi^MeZ)b8PEnD|9yrsIltFBAdP4OCSZj>j?QO)OL7@h z_Rrv4oY(*M67ZZOYy?j2fRZ5wvh+S{%>w3VVJzJQvwlHf^yZ;T%_lr>iH9WQfPsthD@fGZB);TXV|ZvCt9sSLzG z%FSTZ0x+>T%J;(TT;v?|0?7@%51*bN^#D>vLE-E9rXZC_mAxo;eSx~PCH_0xRqUDs z3;d%}l08-R$-3I;?*BiZKi77ZHX|8;-DgZ{m*SGMjJO=!O!X}A373A3>rlA6%U>jg zhjL~5WcK&{hxub5fosDouuhmSZKuc2S5LnItVeF5T^Z@bUlV{qxq%k==CD1#EzkI# z@S!M&+hF%&f>EQvu4qp})_cEqF9zv>4We@;@Cx6kTOIxsyA>+Jc<-OKkTf3Hs56#h zjkJ^k4|i>AUT@w2F#?XpjtY9p;p$}Gxa{g_xM;69m6Q`#4x#^r9^$C=pI;~dGGm>v z0+2a;&{5*`T>%DLxS+puhax4mt-Y`#`LTK)NE8EsgnLry7s!=qB56f)jx4d%ZSp2; z4DfxPyO%CH=5qrVSuQ{j=);LW;n)K1l(r!rw7ZuQb+VbUmT#4+U4EG^gX2Wf%*AiV zJpG=`;RF4M=YM-t{QjN%BalLNNP$W10lWU>sP-y^XCKtjIELr(K_2uP%6P01u2fro z=)T-&Rt@f5Cdl2-|11RqSq!G+Z6srgnPXTquzP0+0hL9HF4`E;DTdEFhr(8UO3gT{ zQ5s`;qQpe*8tfn(?V-_Guq~3~wYE!6`9{IkKdRSiXu_~zwi8JINBch zFgDJMdD^I^TY&G*=CWr#L6M3xAK@`#0MxqvvJ9gLP}8jEsU zlq78f0z>8LN-B1f2T9kH)t=aBsg_-^AEweM&*aQ`@^Eqr?BdDMvrQEX0#j!}tNsQY z?aHIkLVXJDA_HrC_8o1mfCr%6ZsI=Yf4wQ)ZHDSug`I7yM#`$ZPiz9}wt;1rnIoyR zRb+T?oRAMgYejyM8-JbT{S0@5x`}xs!;Z$q_}~n`NhGSU=fZhpDa9Njb9# zCb^uqMh$?BAJB`zB5dDRZ|(VL#eK27T$=Ro+5e(1iJadNRl38JhVfHi+ z%*iTXXrSs>yW($XO<^^#pL^t?cQ}Lgp{eRy<7|f@E$7?hDVjHy89m`%7U` z!2J0HctlvWpYyqPSw#}%!K=F5m_{f>+X|ORR2#N$g+*^wHNIvma{gZawsQ0Q{BV2y zbaUM%KhC!<0!3!8c)UF;WdMrWY_NYFr5!CkUNrkrybs??opU^s)Gc}}yzs8~SgkYI z2UP!Y9O&sGx%*_w&;upP`LIqN&s{E8KD^lUpP1F)hTCu5Mt98~Q*yV3bUx)(5#N9Z zusUIrH>;9xTIzW7t&7d~gxweTxvFj`!5G9}+qf^qC^Z3#Wh^^OgPs%h`ENNKc)%`v z`wVy{a`&W(4#{)L?*Z??e#w{NH01K1TrNo_!U$+fk-(FNx{Epd0KNAn8hUn0GJ|P{ zObt&2#*>0B8)E|tj#{7lO_*1h5@L+HGUMPBVmiykB6tqIJMj5Q3OH>IU_BnemmYkHf>>TABX+b%NWLX(AV(+VGqpzde`LLPG~50E|6ilTPEezW zL?|k1)s7vb!)RSCYEz>{ji9kt5W7l^TBTars!?iH%@A9a+BIUesJ-X!<+|RV_xt*u z-_bvu(?8@SuRNcRalha0Lm08U!(iS%Q7T{g6qkt)^qY70o?YuJJ%_bj*>&I{RS}7p z_!An75(w|Xl_$udy_Hd0;_PI&nn%#J_fVnUWurIDAurFjdWJTflR2HnmpmeJuPXfa z3W5|O$P{{wGiPe&fM)uQ*Ri`s)ABTB(Kkqr`9ma5h9~cWO&cI~Xo(T4cU10zP_3Si zUe3v0%8t?Nje*x-tm^g4*!P}T0ZO?(Gm7eknU{^9GMz3sxl=w-CUs1M;eopA<3~dE z**g{vf@$FS07FPL6^U~1F+z$2XMXro6Y@m*$L=h0QhLq3IoQS@Rtbi_okrTQ3u`Y z4xDfg^md2=nt~AQPlq#JzQ^!41Y1P3x)9Uv1TMstbO&Bk@6{mZz zP4a*%&SB&vIo@S|;LiC|t5wMNm$%Zs%mP&Kvzz> z82#V_f3*W-W&O9FMNMBBUUb`k_kUYZ$vf}cc>g+^_={73n&S~4l{%S^cCcHS*-*s~ zt(U)eYM^Q0LU7tSIGl?T7(_oe@4~mxZ=ikN|^lhvn{$IF&y3IJ~9Ex7yx01mUDS$Cd=5`DSIG7>1Zbm=Ev z8xXCsdZs0(bYK%eeCn0kwgJ7WW5B#!BS$%)NN+lhO?kwpPiZgt3w6s9w}ij;wwg6n z83%4wzPnL6Pwtu_nDFxwU7z_Tny1B^#IG{Z0QzP8X6rtKx_rv=99I{1Gt8>jot~eW zI?PYmEzPL?=J{S}8YmORstxn#|DWAf(uFKX!u6uh>80wgDH2|c1EhOharE`Sfve2% zpRZ*FAs4`;dM#Y+TUw3$!g5|X%aAi6{Ig=-g%pBuzqNFvT<7w7ww5lgpLVO5AqpNm zP52B16a*6*JAjq3`4f}XVNDm>IU=bNG>o1=R6D5Mwr!-U63y3M4{%D-lS!DVZ@POa zmx+w=Da-kHWeRX-P~5pZEc7I+XAkV=6Z%GW9Ed};D!wxzE5|mAQmN^yCa&}7p$0jD zf=otMwEE7BPmGsB-+Msn+qVQp4!@qVErM?lj(tKNo-BI^R*$u#h{89x| zx#c{ktnDwTgnJsVjnckN{782sbY91+UIlK5cj)uk=YN}&S93Jqq7w8y>wdqgdiSY6r!X2vE3T>ac4vb(b*?PNE@xdV9D){UxdrDed>S%DS-i_#bi2SG98`Y$?q zzBfRW8r(C-#`BESb~n+>t$z_B*5htrjssN3hs6p z4B8p>DwZc+MDnjE8u zvVS8V6-*Cy0h?|941cKv#x6`93JSD?oIu2}E>qMZk24Ndl_{2yTH0N`qB)b91(v?8}4Y= z8xR2^P7k>TZgFYooSs1S?cPsGBMefPtL%jzt(k#e={1FBRNXhWhtW%9)W#${^5~m& z;J&p1j3?5b^|-DI<`hmrddpVr0TCnDhWl4A?;c`@>iz3AlmmU` zFXc?U?_^hx5QhzAH*}u^b^1$#-bgi0evz1>$&J?Daalsi--Duez`I+(-2cyV=%4QS zeD<`M-s=q{F%(JVT&?w8e5t26G*Pz{w=)Ki-MVYN{rJsFqL3~@7;XNS8877J5apXX zs%HRq1Tj@~)a`Zg^bLe^Jtkg+UEu^zZQPm|QDE5LVP%QFfBco?8)h^DIo#m+<9^T6 zIrn1$l3ov1Kg|Vozc{sSELHpJZW0PZPtN4LuH7eZBM#39xxVqDtMTIT4VJ`$yvEZx zhK4C1*D1Eg{nk3~*#_^GmA^dA?3SaFoNR#Wo{YEI4P9;HG4t-ztW$a1GHYUV6N5js ztxxAT8-C$<73dSRM1MIY5j~06Y)%8%x*~MK=EcsvpD`P=H+@D+9vy39cx*blk~?ID zZbrL$sz|k8J}I|ZTvnK~Xq>0@lG?e{C$c0%i7Be$biNux>3AH6z;4AcEn%{?2gB(p znZX0;*m^!2;7^D7#mb^MRP7s|1r+L^Z_amQhDiGwi;6sf++neUkMR4q{il;qcMCjF z4L)1ZvslLUnLAV|H8SJcPi(x=*YeYh?Ss>OVV3?3@u$EA#mWtw#TTF6Oosoj!~}<^ z!wgIo(_KKn@um1g5IO#0iEakl!q=lGSiw(nu0AU|`-`?jjq(r0vC%A1?)|5hS|!nb zMBd>`o`(<)a4!)^637dq`+{eE$7gYseOHZDHIskZm*Ru`rjg0%XsYcjSy_zj)tgBt zB4Y&#?gz@*vLu_$;2l^PEBQn!2(N2eAJfu1CU5Ho$B>KQj}o#`)50W z++Fux5IKDhxtEtwuF8TglxYPyfnyT~$8&-zWi zz|rdNTq2|tBh;hPM?BxbNBaWmq5Henzu0?T^3hyr4rOVAhIESRI~m5u+82kmThT3g zHkGhcj(_Ir5jjb71!A|Hc{0Iw%D@%?o6&sS|5h%N*-+r+@U{)UDEpmj-q8|Q@qlCL z&a!L$)e^j=gidl=Ch5sHg4>8a%hs3xhb*0szcq*FcxRxSWG%Of?T@;Qz4z zfY1DY6z%_=e<9@X=};59QEOf_pIOC(e4xKj3}uUbB3Ak2^Kf&Wz^#^(of5{L%Vv*w z4(SpvR(q`=;Ay=ma`G&H$}5E6^HN>^UeVBR0*kDAXbjD}11qH=PoN%_aZ<*4Q#x3< zIGEncI*sns_K_P(kS!Cx35X1qYWFL#%CqioR$27JY_W0BL`pKsnU;^`0V;h7#5H%r z`vfi=q=?mhUfrq&;C!}m=0>V=zw>cWau@f32K35=7W61zx)=}a63OU#$?}+*1|G({ ztu$>V=-}E{$e9-8IUOfPtrkLJ93 zHSGUGFo!kKFNfz!ulB9pyC_V!^a?kDG`NJ`|JZPdc&ro;VWf{=;WHzLJ>}k8r zM7@J0_T={&O6DAWAzW@+~RahUrYq$vXBQ;RR??cQq3tYa2D!mJ8?Dfqn@Cn+wc zemPTcbKY+EQ&X5MIFn_g+L2YiB+rg?g<;YRrUKH+zn3I;HTsLfvV7{bht@NFrsQVB z$zh+~+8SkHF`tS2J61l{p))9nJU|q4_L487%18f9RfSw|1EZ zi}>izb6KX=_DFGA_MN^Xu5(?XoJ#eA8ZQ#d;+LU`nNA6qzRyG@uBpCR>$7N*P0oJc z%5v_f>Z@Z~oTy%GpvcXZfllH?@q3{LCz$}|h2xEcTs3~mDdrl%8o0+6uKo52m0@UV zculGpm(KzN*<^m6vbDjtzKk5x@A;E?Pues&!(LEs5`1?Z#0LNhOXWb-(H+50OksHL z*~N%WzBB~hZ~fDz3spjrEy&K!t}=dyeqyE})Axt@D|b;hm&2`yVTMJH(T{XE^Et3M9@YA^|6;5TRr?!qjlxoW`?;rZehNr0883*2S z)elXoQOuIR>sU|>@ft!4FWd=w7F@glRn{aEfvPshxOxaXBsJr9sw32j_^ZEJso0$9 zFeP441IKv#f8o0e_#uOXy~KwCj4Am%;kBEzN+qSUyB8s>BI?LPuz*hFwtG|Xub9GS zWqgbArgb8|G4zc@o{Y`4c{PI>IpooFTYXVvSPb-G#xY-`)HBt~^k9Sglw1U+K1H^k zeHFnhVDLKX-5K!$!Pl36Bk6V{CYRd?UL620$zPs_pBW>b8xcGwE*i7}BW05VHXvDj*# z&(=aZFSf|xkK)Dt03~q`(f++b)3t>vPxn_#qU~03}KV&^%4Fs(vxGpr)!vqqw(b;4U4w)izdojRWAylr3spkPpRA!=3Jr3 z<;RM{S^y0mcB|st?~DEWslR+9n71$PgiQBz_tA1r-B)V)tKO7lK}^=5?jpwG;eG6#rT2{3d=KxK9i7ZA{MNWCZIW!mo%Nnj|> z&hB%AOI@s)63!{~v|EB$T^i?h)65k21pOHG1=8q~$erD>oq%>a*fw_qqdJ(?G8R z3(I|XoLg@TeN&wF65wQ6G7g5FJB>h_BdyJimyFD20??G2rS=lRsH2tg1Rl?CYLm+J zShE~Wy5@;>=39rc3@XPnOj(-VPhdmSCSA=`b>M0f<5#-Qz%21>RjkYQPMH5lpBDZ? z-SlrkuIv?YwQWD3u{n7}LD&JTM?Z65F#Sq%>K6x*@=biS-p!tlo1HjK4F5_fg{`WJ z3Afcoa2ZV*>k-{s=|vj9yPqHHT6q6HdXGJ@4)9`io-P+G%V6*>ON{Z+9idvF-@-n4 zSpaCp%m0ko@*HXTsx&cz<_RzKww`!h9j_1Q(RVu3QCh(qnL4g0JDHqlPj}kT$DokL z5(~&@;#KJmBWf9m3CrpJW`5mwlrbMSw?X={VCAsX3Xg0}ielc*&28EMYt#m;4Uy{W z@V3CTI>F7-rnn5uW}`G@D7YtBthp;8Ym9qEADdo+bZ+Hi5=)*7L@Q2AIfiW_N!HW{ zW$DY=%j~AcaxE4XI#F>=A%3)b=tFQ!Ohv17lNSTUijZBjpI80Cv+*bQY|JhVWcU4U zn1AHH5XZmTdov)A?`P_&uKss7-ux+~n}_SHug3bDDLA*bUkygQs;AB0p?fqh&fiWR zkgL?jJ1j@Zd?N0f8GSui>W0qxawz#$lmpSBuF%CA<+Nqc0^nV6$&y(d%UuI*ewRqq zM`sEZJn>8K{=H%s;S2u8nk&CP-F14Qq$K0GbWy-%h%up^|C%v1S9R0jZX6=MY_+e! zu}ZJHX;LRoERW*PLqmV;V(HLpBw?|?G|V(Bw37M6@*ts{Q;;vRDk=q24vdkO2}F zHC`12;I895P!}O{Wz=PtOP)~5wlUApZX}%=2#Wb|W!FVOyn~L$r)odq9YIKtKFE1c zEscFl6{RU>2dA*_rZH||Jq2L~i4tu%3G_3xqXT!@PO_oihWi;&<_JvB z*J1FhSgUT7-4kM${6TjT4U*ekbn_v_fr$|520im1iFlNJ(@wjgxEY<@Q9JEKMi4~-;JRR+WbbPAM=e#ovTcc$$JCc255#)6yBPz zkS3YMoaSm5Zm_N8<$=g_VS771NRtWlx)`H@9-Quk? z@vVXK3(`jfsY~9yWjdWxl&hkEIO0Qs*q(*Odd8)`*B{eFEPS|*BK_oJ+inseR~V|z+4AfN_zjZg8EytG{>xJJg>b2v|?splBON@E<%fK zv#4umR{bKMcF*{DkyC1KWov1Va|Y|#{-kh{k`tZ-{uu33udRr`j~_ZWs`|MhR;OK* z6@0qkn@H0X3))n)N#HudSCe~aLgDr5O7+ z*k8B>+C<-X7bjFd0JX@#M)2twTtQ0{u60s3{NkE|{P=XzGsaH*ElAFL@LOayj0f+E z4;|6-!{1$jXz1Ym=H5VM4kMo`{OnZ&kL*Hh9%yH$hq0 zW9_M->j&vPF5mUp#HO_Zka`ABG5lj{uyFPOtvc=LSqwg6XFL(DLD^POGkQFreB!KY z+MRW{T&P05MG^9%;3`KFz|&W>jkgsE2oaouEGMbdl!SQw%Czx#N-}-%Ro*Y&+-p{M zcOh)R+3f*ZtqARG|77MO_fB%O$_|3k<4>75K{`aI+npqXnF8HOcgiY*l^KUywIACT zjUe)qaz_5(hr9iXS}biSWfBKV@m6JTA#>XunP^oEpxv09ApFSw z_rYqyqUzivvOBr`OEL6vt~A&kle9c0`IS~gv0Ww19*aCcww{= zKl~CHU|N{hVIN(+|N8y2svgs>PTjcsC75>_!@b+X8jUQY%jitkrJTD6^UD{XDN{MDZ*-A=3J{P0%+mcEl(*Uas^20mlP zaZ$VNwr8K-@H~I^^-ABEcZPf8EUW&b1!=ykB*|U(6U;}xo6lYC#=yuZk*~g*v{l~| zokU%JPdA72j?!TkN4p+zM{V_?1iC5Ikuf*A&(>`$O*6{p!w?u@v?bdxR`TM}hMeF8 z1oKi8PU^(;NJfJ1)x?9J-Plf)CvrRJNiM$s@F4)u(NbB=BZb_uzEQ1+LllP?mtXD2~-dvt`tdn#v=(TqUe;xYI#GU%DsH0K2g@#on{S zv0urZE6SLNKUK~(A9V(n9Ny9BG= zlYG`FdEE3ZIrv!1Q3e1EmDr+&zqnXXEd$nrl>Fu*N$TH+0ek_q41fHL`=90rKXR0u z_L(tO9k-winb8M3y-Tg9G8&s!L~VFRTpX4nLOwi7ZI$axj6`Qu@QTI7)sj#?a&1p# zs`)_W>jYA_i9^0pc3CBHG_akM#dMPkpV^DX?H z-^;x&#ix~D?-#IND;V7!YQ9)$)lZbxluAmdz2Po~_oMY|i;{3q<6{fqFwHieF)K5S z6P4b}FD)2xxt?DiQPFeUYEfx5^sM%UHn6310W=1aGDP#Q|0w1D83YzaULcX1f@F3c zieY{2eC4M@=Y-{Ceey_Nx7B~~#nNUI@zbE2*Z^iN3No4*yb_TnSJk4fvLm4#7b!Ve z=9B@Zfn=?JL;t0ITU{ql_fW5B3g4k?L3srw2$^4Rk4wG;YY%gf&dwaU)le8S3%cje zxbbom#L17NSJpNMDdD(O*IT5vMJx}Z7h7T^DKSenF&nfqIHg^!fDzR+4oWk0eJPps zv(ssT80yFOZ|5@BC*@@qPBF3VNBUPp{luSuq6b5=KpmwaQgGyYew=q%dLPIF_q=!e*!{H5ViRFeHRo9RinrL!PCIUO@YnTV*Eah2UI=Rvx&O2)pg&Oa5sOIxa@ zMyVJ_2`O)!uBbV_H_5q5eG6^Z7Eu%Y>w&rXA6tTSaO$;CkB^SfW!6XUZ4km8mA?b> zMO=I?8@8p$XjfrT63J-H+PvWQWG@_kW+#zwL~XIUowH#duPuLg+v4sZ5ems(R%8op z*XI+@O6qM;yLO;6-ry!^&}=pqzD6;ftzkalx}LHK?evP&R*a!u$Jg5O z?v(Uo*G`8xCPb>(-nCrHw0LrKU}RaTr=Nc@e*1dME9lpv;UivOOOGylmvD{JH#ysh z8&*&ceZ;9^z`U6R%O4wJ^d)30(u>P6tiCOcz5J`cSHJ55!uEEai*x&CN6Lp~tW1LP zd69L25?BbZ8vbjk0APpQ$U!L}@wz(ZT!ec3^*@o`%&gA~wM+;}6pS2BrUk|eO3$a` zg7Lu`rkNVkv=ZE8bc*0Sp_u(fs%Sz~b!ePnyX6cc}NBr@6^Wyg6KViw&~k@FQ%&gb) z#K-cuTf?M~$e_{tZ_2JE=QV_H66Bo;U7U&OeRar`gC2*vXEb31ubC~_rRJErfU&WA zKkyB|d+HylcKhzE*eSZ!ykNLRKF@Xoxey2uOL&6}y>1(?YPNz7n$bN0?Ut(O3Sr_Qr-b_n;_IA7yuE%r+@$}MGSvPdevX`E;$)pQz){u&Z4<5xMWwB)PSh_OP-C2_TJPrpQNX? z$95JdZoE*ZawyP#lMG`?4(WhG-w_IB+7K~z5_EZl$P7;nT)2VhZf(lystYZz~ zT$g?0y-hruVge@X4gz`V4#z~4*{#j8R!^Lg3RMnGjFjfASJ!1DefsziF@H^9YD%C(cmTDce~h%Cw$!5u5@Zh8ROWK$yVx zm-#54QA^Oy(D&0?zw`vuZMC2Xd_rOCt?l*ZdB8T%h2vDks)N)~vl~GKB&9H=3)(Fn z?hPMjer%gFmxsv;E!oS` z98y_^0lZ^S%dD*m4Zkwg5C+g8LHtUa)QQ?dIF3RFu?z2=<%zetOcExSoy8eRPJmZ! zk0++3&j=%>1+Lhqe3ubObHGV#A(s7_(5fx&5D4eyTba4!_xnxikgq@sqdWB%9g_`r zT};l{Mg7ChG%E+($iZK2%>KCz`Nnc;`Oz2Q1QE8Ym!BU;-^d)`*Bc#`dyby*{95E} z#S=ajwM#MV~#0OiQdTfj_VnJ89j@6jZQcCZQ@@ zv3|LHr%74*+YMHv$-ZN~-F$8K@25#eD)a-R+gD>>7Cb79bjmy;r%S-kt!wZvDw>7* zjTn^v(aN6lonE^*o1m@YUo-5!wvgRYEy+84<%jH*{*rcLKzf5HlBy}xM=#p{_^Va! ziw2hDnFm&o8#m4+G)sI2edb?^Z4|Ek$ENYmv-&YrwO=!tHKX#@@U46^tJ%o^7oXLYnc|-24>uJiVI;v+1{6(fN=;9G8E`*Jm6^IiG zp%x_P4CT%<tH`Q2`-}y;q2HF(_nI$cdwpBp_-n|a@Oq>C$%Di-OgHp@{Er-6c zqS~zyT%bo#!?78|$jk0%ukL?q;Lsk^{N^AL*0+6xW6@i`c!5pOF)UT|yLKDX(Qkwc zd>^s&O@Gr$G1O9{)gE{`vr&Z{lS?B5W>Kvd*lu&&!rf{^5We`j=*<3!dUf_;KziD8 zKqXLV5I0p=6;EX!e2He90fvmB?S@XKs?{7H{I zC5Ej8{4-q}Kj^Pz<-i>k*7G%_&TI;4WA4^;(Sm6!`JnT2-{)2mTb_(h+Blvx7+xTi zlYVCQ6jUEd%j0ed3!yFtwkGiSZ^extzqxk5 z>ofa+Jplp3Z3i;JY${M$73-{emf6~b*kwuf_Y?S+K~5_-wj@uq+gUn^C(tYSxrwQ1 z%A9rn1Rrz5)1)9_%UHCfai-mq#*9i6=Q-%nT++aaa#uA$N$HU155{F~QbnHK+c3os zG+&c|S&LB+(mq%L>J@$ax`PZz-8;cepbQYjgw*%|##}!T?`g60_L`l<$U8>Bh1Utt z!|Oh^-5UCTtij@B?BDnMb12CJyOV5kPBN7i(l?L88XFS=&^MM^!$Q0JrdA#U8bpv)=b!#c+yirJG^!%i%mm}bNi@3%4nt*1#?8M3 zbE&Nn@rg{pPgos($DNT&PD`HPjPu+q58o#`tk@-_#euZ*80$n4D&F5^WqKjCRw4XJJZ;&+Bwb44i#v(St~?zJ$YMNxh;nt7;ZCTrySvsShdN{yx0^d}$m{J{r0 zP3e7ek96|%+LGl%);ls)C)5o(JzR9mbvrbs9GOb|ciqs))?$vB(4Dts8_8^GcSH0) zyp*IqdN#!oBRPzl`Gc)OU7XO^eVAMEQgHU+s9i)a)> z|8!KAN-ep3vW`RIzmCWQgHX#->!wJb@cnK>KG5X0z=H5F^;S-JDVSvx0mH;;1&CWW zg_p(o^rEiB(%HCxX0aIUpA)Iz*Bf#UcVe9Q*+RgQ@>&+o!4h7j#)O>?n|lOyL8jq; zh&K1k0rq)BpurV3lCjXV;vR@Cs+yW_qI?={z8!@b)Q}T8>>`U3Z#UQO$uAz`tydQAKWWyIX^U{TdEy~9XTcfuX|(>XzL^6z5xQC4B#==Wix>mbj4A%a=enC$gQmQ-e4>o;1CV9Q(J-G)$ zIHecT!9gLZ#$O2zt>x@wFf^|A%V7GYSF_kvWpqlRqDVg4!g&t$u3lkziHSIR`S~wiJ}9&tGze-1DdS&& zd@}`Gq5W6xrof4%icTyAzY(ppJJJ$vL}`Pb)q4~>rjn#i<+Wn< zfn!`c!<1NODz%qayf8+b2$7|;??x=B5KYV!R!Up^>6irz(gEQ-P{s2<-1 zK4VGhS${K(Svj&9_E!@12IrGZhXfu#-JhB|t9W37CUr25;hnNE-9MI=9`D%@EtVQn zIM|)FE@%sB70t`dm&Vloq8kUewm-c$*RxKwl_ruL9UPf6Az0xUhW-?;2YP4XFTUsC zWCNlodI4Yq=Q zqgqU(28?^3fYAFN>}sWnPShbePSL32IGd1%_P!6Iwz`ejj5xTOYruJnfG}+LqHH!4 zFX&?Imb;3T1gT}UGA}tGHuhqDX}ZEiM_Q+}d!?xN{W@6KQT;6ewAR_)v-O!>#qToz z(gRvxFRaLv-9(jm&5XG)vdKZ*S5+hs)>uI&Ia)V*oE&n?+_^Lgoj!rO z<6!RB2^&N_loTwDVc_C2H8OW$Rfr$CiS%25&nBcr=v+Dx`>T?>*IIC5UF!-fWXk+6 zUE=rrQXs|9^hGlmBH6~$x!|DLnX*LJJxaRV$cUb;Ro`^GhMrH`+E6+K6EQ?_<5qH$ zxda(ZSGaj*-%zVVZ!0T7SCXKneBA$db<4IUt(XGoLJWGcLoq026$KG0x5n^(})rwizsA`mT#OG1N%DGR0F=3ob-G`aN z*YTUn(QfkRTM@CjGDi(g_2>;&N=(!dwTs&npV%Il(p7nj^l<5|?rIWlbmkF$4Zs3E zVT!4&=s4oT+5mc%!UI6q`bVG33K8@HWY?mIEKCwasEhB#|KdGwHH`s6a?M-7x&4nR z`JXXAJUH@l6z_45fXt2oHeQg+D9a%F49|hD0`)%fjsgb&)svIA$J08LdRw*no zP?7Irqsa|pC<`G@$Vcwk;>$G+Krg(_st!{1PD8pmq+5n`sdu8go4rpbLf2f{_#kBWp7 zh2B`{1#$Y>aBcCa;;t^hF%*gK-=q&pLr3lno5d{`c;8O4N>jLNbk7ch0*oF+ak==# z)z?z)sQSg#S^t$;)D$Mcz4%+1|03;=)C`q9zvQ@p%!B}Lv1KJbp9irzXNQAM+LHx%87!#+N#dAv*f2(;Oc4t~EJIMnR4UeyI%Oy)$UBmUJJd^@CBf1UpAGJ#T%07}T( z3ajI0Q8|QM!nzB(oBC8_VaM>74%7v{QB%_5E`A@QrG&rKHiz7A5^>cy!+}zFp2XSq z#Dz#$t5G%`16!>xT_!KRhcfW{7;=G*>9<|(?**c$BW zW_8L8msZ}|V}oSJEi=1wxo3k#j&oN02cmdz^*uKTh0e%{UlFGxw}#G?j&ZjweZ6`nF`Muu$6knBO1Tb5QdW5btoGB zV4+qUS~1c?WrFdg0p()OC5Kgb;EeFi%xUn4=+XNUOQB>W?it%&z?`eNn29;Y2#>I~ zSp4YMqDhX_I~Cp^^+WBwbdJW9B^)vAp;2ucw2O8Gb_;B714r@}a-Z}Lx~IGAj2~UO z;m!y_W#jS~#_CA0)5i|x^DSvNqc5Tw==$3J)MaO-k!YNM+_;stPin$nj+eH6A9>dH6LY9!`3L6P}5=yb6;ueK%PKk@TxR4NY+O08{?h? zr3>5HND9(~G%P}V6c#YTdZ6K0psbuExYWxWZQE@r@V9eG8(4fOUsd>jj+wVKI1p>I zu6&ECkZL}E`~eV$ypTsOrlFArgVXD<3N1pA^)B5s%`xEj4&yFWUj0jz<4-VRkI~oe zf(&(Clsaec#5>6EAROT-h=>{K2dl!xYt_o?XpyI1GP;!a335^e(=Ba_{#=uF`1~d* zC&3-J>^Rn)#8lf+$$qMXyZ~j9&)j5fbP}1>Gx!mll}#?PEosA+ESmk4nLV?57B#AF zEQE7vekYy+_y_Y+Po~3|oB@OE=|fsEhyXp^EO4qgs8@q6V#AB#WX9TC@{~@t2X+DB zLH}7L+#=zE-x>r{J+|LVK$S;SGRB>kr=sA{GWCWMHnghWHG;NuBXF0J+%XXu(;_uQyVBtgWrR zZ|36sv*$&QQ;6|0wi)qlBYlt++GB^gRT1_L--vJQ2uKuXoK2 zs^?1j73=vdVr2Hnr;|PobQ-S+oefv(oQ z8+ufeT_KIVhDdgE&qTUC{N7+!K)iQhm=h-%)KtEWdzK@6wQhx_iMv00Z9D9l_)Ak4 z7nu(YeBZWSRN@X7+3MMkC9J?_IVt}fs zMs?$lYtZ=Y4Fnd(_Kk^#@?(&sR2wKB4~Ent;tHD*bn z)wiBpZcL&8O(5%GPxuU)A!^V4=&~Cp#C%2(B>ugZY1DH$KoE3HHWr3>*^*I1?x+3) z7B9uSaTx&0DIOmIr~Xdmj$OU}R(93x>rE;xCzzjHdAbuXq9Jz8P1?C zCI?HRVgHn_fqx5H!d?K_r9S~fqC+Iqx6tp8A^TcLz|*cs7e>-Tr?5d7ZtF~U zV6_>7s|mK^*<)B)Si34#Xm7Dz6{V!lHEkDf)mv@I6eBj+M9z>W@8$OKZeW7a?mEB+ ztLuie(5PuP>z%TF(xzXW)XZbde(*PY0yy!@fc@=}mS4U{gJn(P;rSNPzDzN0D`mu5M}qn-d}4RV$S)=JyJi%i*VqKUTB#}^u3VmdZsW2N z36C&CBZA)G_049x&q`|sy;EyXZMui5sl|t|n|e9QUu7zo_^yy6V#Kk(`65VfVt|p= zXt`4THUNcLQU48Ckbj2pO0divMT-=#%WGrf0hHsg<{ip@hisoVcFVVVSxXizf|+ii zF$;ABjs#COd!24eLjO%G<5+j0VTFCJu<0HS=J=>1nmuYZJ@9+^z7zgc_)1ML3EI3L z{)#8a7E-QJ9Uq8jbV$$)NVcnoKQm}wEh1t4)GL~di~P6B=>}7RaF^{GHUjF=4wt#!DnnFc$|WyT zd*{WXO5md>&o$=QI`5yMK64Ou`KYIT>a-a0K8xN+EaN4@6?FudJOoe+zP#R{;R&;s z>|6iqY%{fC#m1-7OaRF{{VE5s&iP@YFUTvAG|&xRda5---nO@78#Rkj{XyPKR8+s=SX;3 zCrOhUGVS%#YnXFJ{2HJp`~on)1Fiu_0^>E~=LzBLIAOE_?lxM#SqvQX@*v#G8`$0W zI$Ntd3w?%?BcH+U0@6AWaLMds@IHP|o+~FEkvXe~31b)CBxx;h5aru$K_ORZzBIFl z=Y!<{E8`qo884V1*V_e!j6$y0M9wh83U$R??q~_P`Oc0;yf}6dAm2UKC>4*Fc$?`U zlI+p97+Bz3Z_C~HG^Z;y)z%s4*M#aiX%?Gf)VF@9{k(a8hl2q@GBwS4fAzn&1*QSR zwe9z3$=}=ME#oAkDA7mjqQ#(Df^}p-y=zVI&>Z9|B6qies0r{TpL~-UZ3LHd*fz6F zU%WkGrOYz02Sj8^H`{=B$;C+Emtg&kUj8TY&}{65Ee|D&wsr?L({|G+;bOpMMOdz5 z?Y34gav&v2?P60LG;%2gn~%V=nFWmNP$D9WV?J41hvCk1AFY7F5K!-g4P#rTn%g2U zvgcoVSnR$=YR&!eTbK&<6EctA4NcW@?4%lH;=@wyl__ zY|ly)eICSDvi*wC1FD(pQh=MShu*B`-`4BDs6zH7c#b+t4_QQ7o>1zdma%un(1(!``HbK!@-v{(`voyG&TY>GtUoHV>fj`y2$r1Pi80ugjLM5E>L4S zw(doBUoX)8-(0y>=B%Y-R^gZ^mJh@8Nw#t4GW3?0jWn~+o_E0K*0ZnekhYq>Y3p8c zv%&E7-e&Hdxro|O#jV&`g5qKzfsc{+jzZsF8PQn|h{k;0ncChjy5sslY2ATa=Y=?% zwX49$SOw&cFfc>6IVVEge);~^zdq4dKftOWyc@-fC7(F)V2)O3?xTj2-EyV9iGiuDZ@`SpXV%) z7vYfdNMP?x+ObBXea_L`4RQ6`f}VYVVR|Wh`aH8lrKZ_t;)RjL@6basYVPU1oHU~* z#ijPo>HdSazKpuVC%OQv#={B-My9H$zKWi#4XJov)K;_Qf!*W}>AlPM!#se`r9uFx zj>f+`ekl2Gxe5HT!wkD7mR*TsZw%W6zHD1y3OxCXLt8~O!wRvhTp?WEiY_-Sz`vON znPi19d3RRY-pZ^3-u4A3OxZ1KEBT_BgeFc2pe!oIE1tZYtG{H6nocq|=r9u8gEYIZ z0F3|B?|OADkggy=e|~h!nr5w~3l+uFlScvrf-HO@8 zhv)^EG@%wa;l|*Bs7Kj-*_9~!rGMajs*lcL1Pn$$uKusM=QC)c1V3JXj4phnORftt zD09GrKYp4L!HzuiMSyU|>7viGs6RfMbU5P3`7Ofd>iqF*L;u?0uG{aSj~TA`8JxPc z8k`iNeUPT&Hp}hfW4#ElMt<4IH}Abt3qxfx#l!=Mx2pe-wfBx{YU{d(1tSJgA%KE{ zK8=~BieReubma*ylK&Hg^T-@ zpM$_H)Eh){4;59Sk57pUawp=5NFSjYH`Dlyq<(Y{C~PcrK>BvSGL?+AH*d7Or0(yg zu=lpCI>+lUdU=JWZHZKP`-7^e^vGT6SYGRxxR{ngT!(PlLzy;Z?e1?jfZ}5?7alaJ(X=Z3L?Y-AmqrqB0TFOh z4RVHjgqXb2CxBjLNGM__)x2a%xXxeu^m*DDLQw9=s)MxyPPm~fh?KROYbl~`aAcGp z7Wq#@e{$j_E^6}!oizJ_cp1`(KdKN*=Q^K7_x_ZZJ4gq0rhzV?HD33zj`gUPw@e?n z7ku?J9#PM&F1}I`;{x(EW$p|%f-~~w(Z3ZF*~!5+DMk4C=|9JVDCF^J0>~P{Ng^CL zE0-$ChVPMgJg(aYrSWfhveYp20FavEfBm&pstzf8PX7_+UMtCKKPh2g0k%auJ%nN7 zqaoIhM86IkY1x5(CLeRkoMQQaN*uQYw!CDAAfW9&vw~R!x(ge-_*|W*&F_{1Ds0fH zzXj+xPfo96|Ek%BzQ@l53e|&u%>?NaV&@0~Gf$8=UbEVMk_o+y5!IT0((QOid=p}Y zY<@QAsOHHr3zX3Z*J#>okxe9nRBN^YhS7r0HeFU!r&&@!`X1f$tmRG*4#=J#7{$3R z7A9pW8Lv(>(4C*c?>BCjlN4%YVt)_F{Rg00V7Bwc>0k7vzbYtDLb%m9%b^dqTg5vr z?WaK289N9(_qu`5d(|3p^*dac>4WsEwqE2;E9}cDr9&VDn1|0nKsI3jpbfhWJ8=Wp zt4Sc)lo`w(W1xL|E>5S9T{%dM(`PQS(SNW~BxHv@~*rK5PyKX+3N!jLfP%rAH? z)<6G}WeG6|W*sdm?u?Y?nEI#SYy);z7S1co#BIiY*|DD>6cEdNgQSrbz9&>%rYWG; zAXY$iK40axFfC|a#-qt(SEB#LV9kM`R-)g4V#XZHbG?+glEsyl3%NUVb+cSu+qaZV z=Iu|d*Y+fn|23yFognMi^cJ>1=N#E;t@=YaZ-LPV7(%e9-n*;e=9D_yB(E1j~82m4BMn6>MMT63-ub zkanH?GlE`ua0b$~ytt(7;|nA$=!8KZFb5dia{ys-zFvj5YLB#|3`Fvq#mZO>0*K^O zwe6%HyBMJElkR-1-~&F4AF|9U&1Z7$m5KA0I+UO3>QlM5iM)QQlex+GliI`bV^ftX zC-1eDlUHml?cM8YvmU-Y8Rzk&1sFk{<+7@%SKy9oxic;fc zg7)6J!l^)#MAg@Y4SkgjJ}Y--`9_sucGX%ssIPP=ba$CWz1lucfSv;~l6~g|=k-K( z8WaZ7`glBkS^NagxwZ~aB13K7C7DFkFfPU&T-Pv2 z5tcXDsk&ydQKfMEqw^$;044+Z;&$glWZv92@w<;vv){9lwZ5j~9WsEDO?prN%v?!w zck<1@=Z77BeMH@r$UlyOg`OiAJ15;dde_gaTHoODyFXe09CvRo4fb24q@@m)wmnvf z4Z3$Yr-pD3fty3g9>iFxT)u_)vuiQn-hZ4MWmU*&3Ga>Bg>1;6&kE?q9WSvPBfJy)D_ z8k8+J#=_FRSf6R);UbQlYPmy4nk1lpc? zwT6+6_Cbim9foG|HZ23)oq!DAU4~k>zI~}TX0|V=vbY|b)}m~6##QLXYt%EhJB2M- zmUr7;J{i4iz3Qk#m;1!$&*h2+;{!Ev8I^zGG(}KK7A*PrXzf}D#VM`nk3nWx zk}J)c8#XESe|^?>Juu=l0rxK*pVfVY1d_SHF`*Zz99rKpZqDR(|M>ucM-AGE+v36g zqu2|9)_5x(&gZoC;pwtHn}BVi4;l{Bo~5eCX;i)tnoZFD)UCi!%U@f%v867A4_Lgq zrA3eGhIeZp2_3h&QYRsVP5~jb9N^0_jF$_e>gtXcxB%b|b_qhQX&_sgi`Vad=O1!ZW}y{d*_5GH40JJ?UvHFZAwDgOTWLS)x3Z>I5O=<=kzF0D=mL7Y+~pmC zOYu>VX>%@|-(j@Fx?+L)ey-Qu;4j>T+?u5JL6{mf*T?k3yCA&))Fev6zI zi?tTsty2+G1^2K2-icGsffBzijmuSFF7)>#{yqyGG3zV$^rQjg5dSNJi(eDAlvYeN z2NhE!EW^Px!9o z${qrF+h}-0eF7t7SUU|QvbtD-d?nsiRj8~9(>N)4JT2_KA)2J-*? zoBh_-v3mp4^T75a8;?5-IZs~^+#Y#h2`ZYIaEX^+uWO8!Q07zS=vo%057>_Ffhyoa z#cBFnpuSqDUS`_P*bk&BEGO!N^8vh_Z>04*b$WzfKNkcYZXT~KeF%lzhjn}n-TYz& zo>VPN^!tgq~u3uK4Cc+WTVnJRsSv=t8Jrn@gDn zPiI7fmf6<)(U&y8PAbe4G_6O+HO-S1NfH;E54^Pco%$`ck#)C(9;TZ%*%D`pC874d zPWy_|jq$m*!hGq1_Z!$^*H2-SVq6*%6=JP`TvrxYUy?86 z2+ESC^*yJl%8+!H`vnO8(@TMLEGYkF?7S2`T&^PR2WCWWvOqOydQ|Nfqj2Q;Nw@X{ zki8EvZxpC37B+7H@Lh%PrPbX3J{-XkGrj2rw)}0wZRk}L-PON# zwU5vyXr$KkCrZB0k=G@)D{~*Qk=jYF@EF#j)Jf4A$QD@JibV|Yr7LH4Xn4c2*b<=V zj|;*C6$o&LYFU?VoaHuwIONj_xuA5=J4Fwnv-4GSi?)RaZ3pu1?bxDOo&Ir+$ZSWS z=-xG3P7BihN~_x25ZhTs5uI`1;pSkw4}!91i#B)!ZXkpfpxcu+b(*w^oaz>NU8;*a zSYCd(69T7q5+4H!=h;+5oe|95ocpcxE@}iz-pepG+K~7ctrR#{5I;G^Ciz-CoV6IA(G5?(Q2D^T&V|H!P=*%FjTKwpIF<8Jtt?zq+3qX zl46i6cHn6H%H!Lo!g1L;_3zS%DRtnrnrhM(qZ$oysJ2H&6Q2|hn8z1I#*OXgK`(eV zz7pjTfa+T(Hw-->a?;y0qrZcXb#X?}~s^CRA2 zC}4qL5h90J;i-u91hCr{^;!TaC#-=BA^Ntj(A!R; z61}j}B$YP@*}(4m!zkEoc%$Ch9S`ku2?+NY?uIkw-w-F^9<3^VJq~x8G!R|C%A7tm zHSKEH)X8?@VnSeT-=p*c)7vRd_p}y&6e^8z*vDlVm@981?&7KwYbQ+hFGrShSl-Sb z{$%ySJOZt_3j}(LZR$yQS4OxTq5+@&K!U#4wC&W)oiz?*%?iUXsNh)a!=!PQ4kGWv zkgd8v%-t1WWDB3dbZ$a)7nS`(s2nigNVznPn>4ELIo?cfR+ya&ThxWl(wxP8lNxdC z(d;?3&ShRby7_tGvchakZPVTUJZqEahrNz(s1@kmJ!j|^CSKQmliZAYceUbV+oI8D z`rP8H?z-R?kr(;g|D9*bOG**2*lAU%{*hSD0;+Yg%r6L`@I_j+W=I_aAW)&`%- ze@MU~RtkD-U(V=yvp#lwt5v}H1*4lnXY42sk@YTQ1x9?oSZ%-me*zs5diiXc{%Bj; z709Dm41Be9sw8IH9^cwmY?Yh%RRt^3eut((UzPk4C*fPf9vn#@Yy`xXxlk)O5yrEX z7YQ%L#P5&~S)fTlN&Ka&!Ve<4&lmxeOMIH6fZLz`SO zeh_&pS-nDFw)TJ;ElR}nBvv-xEp_I!KN-d&SIh(X#IIdTO+@b%8@oF(oL1GMkYm?N zkLVg-Rag-)Kd`sHRv`Ok!lG;S!sadscW<+r@#Tkl#gB;ObAvPJGvm*pAD{*LKG6H9 zx0C8nV!q!KNe)ARmRu1-@&i4I6KL+{dIb@(b?QAgzZ-NDNy8I$DX99_@Vp)Sy-MoW zo*>B-K7@+Wu(ZEc@7=k0a%Ri*p4-Q>bS3RX?#WGw)FM;Ro->B>H0YdqHonPxg4458 zE@MSwu4NuKK+_1j0?oSD9C6$&UZvCa2O!VSPsu3jTXO@8wEzS_g1qLVOP|vwo;LQw zwLi+gywDmu%_;R};<2)D@7oIo#kos*+7EGWJ>2*DzTXb&x8|*UcU|MDKUEi|iiC<4 z`_o}5g5N%I*poUe;+FYTQ}z=I@wGa$#gaF*^EDJr0dJ~ec%1c9~Ze<26jC0SKi*EJ!$B4c@)Qp;1jCwYdN2*+66Lcd6 zGim+F+}CyF)#SB~LQ+eeS&B-5X|+U<&iTIm+~$xZ;T!bX`bwGvd2K)2O1ZhOe7AY@ z;vN4JbL*KEKZ|IJRusz-h#{2WTr6bGvHNF4^_pX<=9X1=q4MfCg&sn2K7Z2BcA9U$ zrrqzk@%ifY%&dCFTSI1TcH%9U17fXLMScp0szTpum6Ung4(l(j#cN!o7zuL~>QYnC z6lrr0$P^j~E8MD50i}`;8+ek^{}9+?%%DM-T(pR2t|Y_m&{o-Wp7AL`2v8*E1*#o_ zKte+sa@Ukr*zIeyW4Fd)bPz-EfmjQ60R%NIU`is3K~SPSFxQcUb)USZ``MDoT-Z_% z2>lKi0&q96K$-^Nqj}Mw2>8;(q6|T1?75Mm+k55Nz^LVS--IB6MvwIb`a;n+3#VgY zpMcq%{4dJZE%6ij#HJ?|dfY4-9{#Xy)TgN21$!u-Z?j?o<||a|S2@{Z_a~Q!5)r{R zprGX4YU&}1Z#_k&rH5zD_*3Na0I$lu-ZqbSHID=lxhvr4>&_PwAxj)B1?hbEdO|+4 zzJn^x)$A;H$pmBM@^(l!d~&HTE#->cGJF&0`V90MY3MVFuQ6efV+WGe5+9_(UrK4G zsy`~+9`j!SPQ+^-3?3kc7!_7{+S1_mArP1>^#kb65|1{lNuB_#7*Dfm#a5%5EnqCr z5BAJi`!N1G#tKr;ZHazEtdKCvt9JU;obc7i;r&`X-%|otTf7Y-(9Khf5R-=(v(Sn} z*A9}a+(k2tu~oSh$dBiAys4czq&>(FSdV~u)vP}W${1kJ!A?K!v2}5gNfnyDC_H~H zouW^Vj?r`p#5A!*@8W@n9iH8`k}Ye2UeNcwVho%K?G`HU=bzhcM3&5FHzDVVy+y`l z+=JT!xz|aR*@&~v#xx%ul1y537N#yCFNQ^d5}y`Y8Vtm*!LZ z;XQ`DuuA!oJN>G{drQ?PKbKr>3A=5gn)q63pzU&3f?GuC56VMKSyyw&do9!e+e+~j z0}pAcMDtd11oM@-@al$jdLx04nYV=AWVY;^#$LQA;4~XvGF+@4>*$(>L?tiY=s9B-vpTdzFBn*vQ!V&4M;oo2}w;sfiO%7o*X3Zlh#_@(LNJ%n?* z)L8pIXoXsvMzGWDNSru3L~BA4X`oOj?srq^8BRs*5~Pqh#KnWERh3oD#&MKyK39&W zeR#JIv#${u&mGkimhJvQE8n7QpSEere}A6jQ$5vYCU9}PT&Xbir@gVOqVe^Wsr@ZOA zHu%H4-i0Gowr%42kg#)O4z%MA@rd>heo6bvhD_?7LVJVPi~6cHQSz3o3bvZNS-o}i ziEa1J`Kuyb>Uq@`08rX5R3aTf?tjf2Bk5&ja_r1D|Z@ zq{W8zl9XwYifY9e;}#ZlXn3wzIb@F*zq8%nmBK@nYpP)wM;5{NxWidX2 z>i7Xk8dTpHivoK98I0IR>}#kSN8h)Uue>|Xfu5{5?i2lv zc#|XzTF8rAfk*Fj!0eel@k=FzE4xXJgAn|Iy94MTJxnt z`}RB9yj&lPlUt`9ki?T*w1&)#h%wvM<;_sY>f39|+wazORUYR0v6L|qw>}7A6CiZ0 zdN|PpbMJ_!Xu0*UJ zI?K^NJQ(7ps_K|_pxzsN|9&>=oaOGuLd%{A?bC|TfIU60G!NG@m5uxQnL%{cPK}?9 zInD+p-HCc{z|ce2bJ{*;j%9dbFw~{r?$V;~{=wu>#U3dhk$Ox8Dj;d*$S#`WdkQ1D zC>ir4g`9n)U&*-^g|garW)>cK1YYkS3BC;AFwnkx!@A&z%Q)m$c+;-gV=bS8~>>U~dWtLo}CG{KDWO}ilwexDzy|ev`axsnf z6m`T?7bvSrn#rG~zb{q`)Hkc$+nTp!9((QA%f{bV-a|W0vn2fiB2AXm@vbsn-9$LU z=+(=dyFa>Lwd3M&SW-^Y)4gw(5acS7ca$vEqYmg|OBH($bL7dJI{Z~n zZ`=_aMdeT^WKn9B*7oI*H767o_E^#Ol>LOvmOi^(`t&U*L7IvY#sbj;F~iQ|kP*PG zDZFa;$*g3DW(*CRZX-k%Zs*GyuBmy@+Kd(+> zbY-gDdHU^Wayr^_zm(G7j@&R_Ej$1Doz6SMrvdI#??N@g?<<7hn6QzhdtSYmmS*9C z5EivV<^VQWiC2$?VogzIrhA$soylCEv|mwcSjWm8!>&6*5xHHM>Zd*HC@Gf;Zu`{W z)Y(}TD`06AZ8Qsn^S)NFqD%G}~?_UtTxcPd+hq4>YQAwE$HRN7vG>R3{5hffLU0+o^ z>hNRqo4g<^nTW^4MRV|Vi23@iziNHQ@)pq=av7fdG_^DR36FR|!B5A)Z>Gs##&%sk zYclP@t~>@-Mq#S`2T5yu4T>>0X(=QAV@tyq!D=Cm!jun4a!7=cg)jx{P7nFG$3Off zMflq*^WT}Tt#Wer1P5z_`?q`!J_K>2em4Htb^K1GL}A1-%~~Yk4LJis%{$V6|5@zw z7_Y14Ji$`<;rrkJT^9v4WD-%ekpJU7;75~q0{Tv%ALsE3_P=VMplLxkVEKK|ct+W}Fyj)x)RKcL6o=*RIF zkMVOH?+12#7{3$Zaj_E)cR2n%yoyASFOA=bB96Zo z=&u4Oxyz=W`q);4FEkpnDY8tK^hTtJ{qqBMW&Gec=skJ&@4FTyCQf~9nQ!NlmXB>w zP0d{w`}3@SUw7&Y>f^aJ{Tsx!ThqAvlYj57yE$2Xe@Tn%DaS|u z;Pk6t3}djbg8pN9Z;f19;DEy;=$ZMlz{g#qI6SokR*fum~t27zg1<_=af`RD^1~-;9v7w ziMYP1B$-bo;?H;f{t^BTuM=EM?8{==5nF;L*Tny-ErZ#LCqV231#%v*A^tTPy&73E zpZfIvV9DPdJZ0SUE016C_1nRsE+oEyj&OXS|A*HQ`<7pe z{(oEL;QKOQ-kV?k|M(zL0QqWl{FxnpkMw`Mz;k}y6aC+V!uT4XkUh8mj|;;8$V(F6 zW%=())PG+hGgyUp&i%(l{O6YpVAZ6X{dp$;+r{}khuF*gO@a7d#_a#kq4GK({~y=% z9RW>!oY#m07S7X}hOTnGZ?CC61}J>wBM;!kF~q!}+%wyiVm|luizSX?Y zcNq6PGd<1bMZr7OK?Vh%caPuxESuu~(OOEm+~j`@x2Z89{QFDjKC4dfXGfVbgnnIu170^pdzR{3aw-g|7(PXL8Yx^#0Q?M9o}ygG0-xUO3^ z;Ig$C&E|Xl_$CS#mveXlvwPlg+?0Wkh4ne_E$@9%_Ox}XY+Cw6*UC;wlkl}eS=L8B% zoqewA?j0K7k}Z0MF$p58y(^ z^)S|D_A}RG3agI*%aWT_Ln~ECAj#`wa%-k>V@N{o%1uzkd`e^WHBKA!`CHWi7dxQ` z1y+M#>*ZQB@&Jsdeqy#Lka7EDE9vIGP>hnJZEK<8sgn>*JOGs5&KS_Mf{l>($hR01 z+*z)yYsCYtctV?DZNQ3!{%PI#7A<{&GU*|l?-l5lg2+Lx5Fns&0o-Gr!MZF|sC7D1 zXg26(=Y=|)$y^Y!@1N7@IkZy&Y{auBA8G%!`tdm1JCW^?lUrrAIVHejG9UCE?edg- zj(jh^FZESB-y)(EkT$u*w}7>HGa{J?{El}NZ65hRSURUh*C9deR4Q(-7R8_eFGLp?oguu*c%Wrm|?5uMdNj8%p0v19{ftQ+)(^*=+ z@4K4|a|m+sm#d+0dZ*XEh5^Kg4v|^!>-psQ#&~dk4b`556HBHJKTCb%Th;gPp7YWH zCfJrJf1luy@hcr7$4BB6IL^2+(ro{Vxjx(^W+)odJ{?*rUJli}U&0t>)@TpYbZ zhcoZEC8;Z(yFW%%Zo@-G@E$h|ZNd!e?*0gymloL#{)LH_F1fc!WPV=+p0wt8<%fmB z=@>3XLpCGmnz%L|)jZg)6G1>s<_|!#=OX+`k<-G{NXz^P|5@(x{pALEKg#LPM2u1b*FRo5~s1G49G$we_^jLp$YjU)g}VUdq7 z`ok*v!|(m^bUN=`02Wk)3*y5D9__>){d8L00Ykd!Vsi1(an0`0SN*MDtp7c{IYGkR!0eRt zqQ+AW=*{E~k>A1#e;r|7-uUx}3@d+p+%Te(gqssE-t7z4B z>a8g))HpjL^+}r67TqpPHvnuL9Jv>dwy{SxZB7;+g^@p;vpB{Cy?KXLI~_aZczeO` zj@A)ph&MaU_X7UG0vLmpEPc25qZ_04i{UG!!2H^2)Q74MK&=a$WV1`4SWW|aWJzX1 z6OKd&1fKZ`Psn%hRwpiiI4 z?To$#rb>>Hbd1;FR$aZW8Y4F&FZZzIOs!_LVP`QgPwWL?$GkXSodFYsWT(E02Vy4a zAYMQw?z0w5rC`0P{{UedfR3JEkv;1S6?~Dzi_mu8=vo{MvJke&py}i;Na7jV)(+f-=uFq+Wl~3{4#22r$8LD zn#ma)E0oXB9OC=>D(d~XoV>?a7?TB{puIQO%`e*(HWc(fhng|u-t-B_!{e@|QOPe> zMtl--rG?9s6Vnf+`4{{pPP8_A;_9|<5y@RT-Kb1zcyDKbC%`Ea*(zP!ERv3W`7x8f z;P7Q=OBmaHkjHdWN2p=%;g52I-hN8WM|Xcc`wxzI%~gRqmC}C|0leMsP)sPGwa zy)*T>5-s3p2aDP6MM_Hvtx3|3vP0)rs9V=fWRH}?&Q0zc3GFJ+_ zLP{{h@=>e!NFomYZw2FshI{&LbR)99_=KaBmr=CtJJTXp}ZZYYAhD{Vs-Vs4!4arr22V z9`xKFrx)G=%^7;0z1GTe{&t;BLeCW+EbARoT=YA9W0#S|p7f^^eIl6!6jb4JDWn65f%1gOuO~LebC}t)fYa8!9fo(@1X8bTxV15Gqn6X#tE7t zE)+G+2RB2-J=G@G#1sqPu2E)Nd834E)0gJEq$BIntzps&xMFJk#@dcQQ7h0J9_TqS zb`#hO=WO84zU!|Z!BisijwOv+ z^wPNoIYHgy5(@ODkMNC#*p4F*Ja@(<(C*YLXl!;?Wy!b&?KJNsM>?tVF=#0wf_Zy_ z=65Vy_04k|_-8+6poQYGzqd6%QAxad^|| zpC?Ns`*A-wIr6%$IFz-y0pG;UowMJL^?35D*Q8TT+wm>FiR#dxwWzU2h0R^u6fYsa zKN(ZyOWGEew@K|pw&w;qr*G>-@#|MY=K2dVM#X{v$(r%*8o4ssycKrqpz&J9dYJW_ z)7&_+AF$123K*RjEMpX?Uu!tJ?67e^DCI3xU-u^Avcc-j?tmj}Waay3UR?u=<%g_^ zdw6nFF`%z3fSwHdr4c_0w!Z5H17rCir?avi8Ai0_=ewM3wH1QDx`jO&KHBpX7KWDN z?Fe|L{9dHx_ciLiZ2C%hnWmM;I(tv6Oi6meaOe;OD=RlS>z`!2ozJ7H>s`x*cq7+B zcV|v-1ZVGTH}A;ww)Gm~oVCf?Fr5L83v2&32BPNCY0N1{bALz?m0Yd!6lSrc}HI z!n*UqWPExi48E;azWWvrLvIcr9Sn2b_gu7(UCU&`e1{n`uSyH$)=l%tm!etVXKMj{ z*%+UHEZ`Fq;HcX>9>Y`*1e*ATm7PEkx`{V_RQ*L*um;u@i&?IK z5%N`SyUkB*BZ^uRJm<(9h;Y<_hxmi&EcHm`?H(;Ox!;ajtWRgvyN&!-5A15xbOTXI zt46>m2jrsvGOfwb%7ldD`B@curiSbuEI5u;Y9_`43D@ylV#`OlJXkuTpCe zH6G|5H115N@MihZX>sBFMetiOyy{9}e+o9CuS8XLB@)VSe~M3W`8y`WKk*N;PzDRQ zA}pIA!w6zRE^?Y|o2;G8ke@34QMbb?Tn8A_w~f0nMUCEhKn;6X?s4$6$<__Dl5pk2 zImeX(74JGjBp_OqGT&KwzQS^M^65gK_ za7D7vd%c@vE2h|0pngrU>LWUR3Wz*imoC;=(xQe>z^egXp9-F3#b9lnMRD#zlYbdYs$w?GnFZN}upLQ{jI-lFSXJ>Q}DK9-ILP zmU-Mc%-r4=#2u~1_{1HVOUz-dImAj)*U4i))3=lAhoc(tl4f-fX_;|8?;E3>WNpj) zLBjAM6Xmi^vW|M`v#Ui9%k1Eoobczbukcjy2Zc+I-P-7oM)v0TAptYAna-|)yh12) zMo`~iSv&~wlP#9XD55s2o`j^tLY0?uHx32mu~69UW1d-PAm9`Ho!1=?p6!+D3l? zde42^VsI?F!vn*&ksIIQVc^F2*493T$gc`BE_SHR!qG4T-cjt>WqmPd9IfCl4XVjj zgG(DsojK52Ldv0q%-tGpkI^x{O$sZJiE*ulC9gW%?TlDR`KAAjFD>)nQ&(2GE|z~C zN=C8_)x~a%+Hv;1T$58x2XxMBSEkkT7u=l1A#D3+bSA1rhd^mbE1)WeGWb0A;}Szy zE7=$Oxt2b%g)BDRG6a+7y!~7Pr;|}r2=83`dri+Mpy111&M zknOhw&4n}H73@Xs4wOKsQ*cwQ(Q`aYK+D?#l%9OA*nA@>4$>9hLt0%9uCP6CHS(RuLBNYjv=bey? z=(`3y{sQ5i_#Q+5QBF?yYw7-|X|p$g-%`=UR68Cae;H&2+(k-sg6DzM%k+yL{sRLh zZhs|Q{oTgRHqyT{D0W70x=5ivfBE<65=A_5CQU<=B>GxtM!Fr4o1<&miZ-ksxvP#> zs#$n%p1yK}4LXq&W%tT;nz0#PX>_VAHE@Mr_W($g2^VNhOro@>>y>+U25znhZ3cc8 zS)J-Xjdh!LNZld6Aw3K%*hwWSLtqKY&C7Vby~B@$-4j8ca1t=HM&F06swTbW43)ZF zn-l5t5ua>IEvCMfrHcs=bz>#s>*)Zz@Oi@9OVE*1`KTB6r9NbV=Q_opjFl(hPngt0 ze9`I(_4Jh_uwq(xloOvkJ5|gWw7*_K+eRJmnF}hssWyCO+MYqS*{h!ExVRo{^tj48 zB3Je6<4-o6;1E(W8V*Umg;!E^N@gh@pT7N{<6p%$VUd$g7QBshA>^SqQ;+vz1_ zoAHv6t8q(nu}N(BaH@6780!bUvaiJEx`gBacVuQDAeIp&Bv)rAgqEY(#Tdqu6DgZ1 zeN!HlSlxq=VPM&yolOUbdhM?QMR2KR>+03+J6VyTYt&*9FZP=K3FY=fCLda~7Ew+X zeMZY|LPgn4;`~tK>(e0j#ok~?OY5&aMtd9GcTM%({_ANhYe3z8 z<~+(z|NFt<@9z6HWQxV9#oVDYrCVy?OIxJcftzYxuyd3$?KHo^B}SyPzkoiy7#S&U zYBxK%zjNq~G{f|H?`3fAYl6OdpC36UZ9j^iE#YR~@{OYf^kY`iY{;IXzq3N?G|FRS zOQA!ZYm_0*s-=l((sa(um6u?9^cl}8Q*zBZSPjCN3_7r>67#D8B4wb{psuzARJWe! z6Fj#=Tj^JQ$9Gwz=Clfo62uIf9TfP1%U-n{XYe1L{AAfD*$543bqhfwbU^Z_LzM{B({BVY9O2i0;9x8vNn7pF2nZ zsQC9Wnr><3xC z3)L(5fV4dW6v&18jt-VQ=_6gNzK*{9q+RR;yKm%kK^0?7&FxS&7wy#d885j4Kcu&B z<~gO{0P#TVNpgySc8&XUk1=2x^~GEFQuVc&$7ID#1AU6i{!mo@xO3ZZBw*Mh!(nw% z>T_s+v5{%(N%J0|*f}rYHrW=$*Mw(_LQOE*p>3gKR^5}AV!oj2 z#?HBPsi^tr$$uph7gnUpP^nJ)68%Le1|AN_<@{ubr(xt5XIy!g^@9B_a(v!8&*RmT zs2p8nTXvUQ-xtoBKy!IO$}_38Fy1~*ug~q^n2m&VNdDnr}_QzzqI`j&cC8(QjOf$5{~C_K6JyjvY1Kz8kdraV)v&u!tnhHW1OVhVvZRnzmN1@js0Z4EdOi;G zS7W35h{{aVhM1tQk6jw)c(rv+=M&I2chjE+@&iz>E73UTwXp zpkyK{;u^=Ss`OiL>lluz;H|FguDRTy-*1h)ENXJ&m6LPsUvWM_|BY4V7*H z0=DGTYZPV&oKU6JNycJ2How}AkDdCiTiG0C*T?}A_m=IGc94Kdqu4^5neLD}uTqPQ zGa*UznJ|U2vYTo;&Maemq?c|oS4u_!Vm!?AQUy#Y!?Hhrl1*giUa#9)iv*;6@}1@x zBfo?FZzoC7d_jCN^?o3t&DC=G`N=2iTWF0Jz!wtc!QohORzWiaa*@r8GIq`fz&{+a zYHMGuxXQ~0yob1-x{~yzPyV%It}!i7_IYA9V|C%N|6aU-5)4)E8BB*k)eJ756HL$J z%O)R1O|xKGIQb5~$;ilbCg&%eI8WsUNNea;s{Ql>aDzD)4XMR05m002fW8H{q&+fg z<-Bw^|T!39_NB#JCPk-sn11^qg zSX~D#ycjI!QM1ENWrYg{H<&VEL@&v>Fhl5&ij#D4k%16c3+UpbWE9XDtyVWF2{=hW z8yvY}H)4tX5Of+zH@ZK}2(hfS0a!qdLDlYDhAzI#Z{%eizk#sMOn@Ixj?K51anKR4 zEm~UK-CpldkakBl=|5-3%2l=?LF)C%(K4hO)K8}?Vq8nQo`D2mdFYep8u?8c6QaJ3 zclQ~Da;Lk>r>6Amch<7+WSIE(4VCn*M*J1Z=|idC80nj8sQzpDN1cejS$S<#73w2S z{rY-7jirwq5@M{?Avx8={=307uN^fM7Aa0+&5X_*qouS_+%`TXR0Xoe|TtFo~9{hbu# zTJx<*fJI{E_+S|v3hFS)h14d|RXVZD1eaYcuD&YZjTu_#FTmcohJF+$>}TKSA5gyk z;(BEf*~?R6SCnYP-haOdT}DfbA(PG;2o6ny2d>=2ckl22B9v&qOkD?HauD&JdP_lC`v??-~0M77oS zwR(?Nq03fldg(mRsEjHRPN2g{_eauC`vdimITQW&h?%yepUMd~1+l*l&&*{AWiwSw z#GSvR8hoe>7z2G{EwX$tbT4QJdk274LGiRqC6-Y&=ml@nXal7)cQJ(>iE{p#aIDB7 z3R8nNs{qR=JakN@TaJ3dM7*AHqCtvEo3akbkiz2h$Rc2?etUAw4iEbpgg>5x*h{(_ z_}vP6>Y1hB+=`p;N>M|HT;jKTQmu-Y;gcA^V~+QS%iPB`HU$?<`n&c@YUg6m?#THS zqzR0Hm$9X+^8F@pF`;0iy`o6Q+F6nGOKO|Zh%W|w z$TF8wkiYw>=S^$D7bAuHnIk_F(*QADWD7BixYi=DfRV;fRF8id31xgF#M&pVX9b4klCT>} z>_(Ct@>|xk!<}jA?YF663)4a-79&@Jd=bFlZF)WUg$LPDq|J3ycn;Y&<3zwXF$MJE zHClM2B1iQlwn_DIW8tHZ-|k)6eQv@RvqA5+EQYZ%^GEJh#*4ee7z)g(MK%a@Av3~c z3HP5qm$L-i8ExE&FkO=yz7f;@S9!KpdoKAmPK{+(eHF3v{7j-SYu_?PxkoD-gX9-i z2(IHPhg_StnbLehO0+4YM|yfy_OBp|`Gc?Xn`O||j!$TRFCUiYCn{l;KbjwxEPyi# zY4AMOw)PUtq{Q*M2cQ03c_4&E=0|OHrkF#fCE^y_*qiYy9smaNoe5$H=;1eTagkBx z^|7Dr_OHP7qw<(q0VP?0CQc1PBGQ)az7a#ufVfb6O zac&`s+!HwjcyYtZlt^S{4%v&S?Qu}hdz>pJ$gi4C#X8#CN)(GvQ!q=uw01H9VT#wG zan%lN>mq-c=*}`e$x)~TZBG|zPUIAFZGj+IXaCBG zHKi-F;=5BRyP+kx4zE%Q9QOJ0qiboo4=uU{pVYsuB&y#Z^YJO%+4wwEw)K0tJt0@T zp?h;UCE?9KiG=zqVutsyO7uEt^LqKZWY}ZR#}`4Dfl;&w^7|$^Lk2NLHo|t6@3Tdb z3}pc%TdpFKV#x+#kN6?!Ccd4(N!NqQR8D%IDF-D-P=oF@`-E0d_tx@QiyRKUbjuiR z>hoO0akg`L#Sme1(!J?TTy< zevq0HXp|?2&@UxzBH$9;<*=>4Rf$9Um)2@;46{m1>#wMfEFG8z& z4Fzq(FRccPY6@D0As(MAhk44jQk%SKdT4QS{h@~RXh36vA9Af0V3*!Y-9g;a0Xf*% z;FHoX(YuEq2Q88gcLuzi;lHN&IXTgak~rPiXy7vN_cr)#m|V18>}+N6x5VIB`oQEJqM(U0Vv?1#ZXDr|=SPs#oJZVDeE~ z_PaDSv?g3@XII{R=ZuOt{r}qg@^`4$|9?ptgqcQ`FpMcnSwgmqeJq_UNgb33*=oub zMs@}xTO#`som41mq9*I0Y@GkD zV|$Kyj`^R|;!a2l^JwSHmbZD-$jGBD=fn}XOWSh)xPm0_J904}|(`T1_M5*93ZnWzvozEZnqeuD8o$I>nqNiQZaNbEiJ#$h~5o)icf4MQ;{tk&*^_`BAuIes2xiB`r`B>@$p z1nGfS&qXE0F3cg3oOMD7*;HQIGcuQZz7 z4vva=Yr>>Y?1kl&Hd^4~QdCPLSrW7f(G@y;qzX`&9FIv!9mO@iCsQy*6({#C0)3Gz zzEl7e@cJ0fB5XsTB=j)SffqIK82@>QFKPm~Ep59W%pD>rHS5@XZ50?K3obwJqS_Uya zsFh}O=`+xqloE8SgeWln@e$jO5pG*+i{6`*G^Y`Bgrt>inC{V$n?AGs3ffF>#S{x2 zc3XAMIMp#8$o}fD(;l?UEj$N?t=wu73f$!ZIv6~&>cTUSy=P0OCgWAk??}DfJE#z# z4hCwWZonDkqpyMpouEDttzt)JOfB)*#paEdIRW&X_xIIasb9#nsiFofDB*29Pp8af z)L41EISgAS)UpHNm^uF58tC-|gOn^*?}Or4jmJUg@$84XK{WRl@isew!D-Oe2p}zZ z;o2%}uK4h3Kr?C|2Nq_(O(R2-MbJix)$}P`MgFns#le4gVFl7m%mCh81w|Cal>zCF zQ^Ln-YZg0oSC#i?(>n``n=Jm-K~hq$8hyroTw*`&GYbzE;m^V`sMnR-r#3xEUkoK? zh6JNZT0-t&w4L?P4-tsZE^MU&S}_>JGBj3P=K>*E06nUy*}#pMBbX4*jv+rjsKThx zO2d>5f=bNio<$Lil(|VBBUQ6D=YL0rhh*i6BFW5O#vK7;93`Mp_QW$F!z zTbSGT*}s)@CJkq=_xViO!*feov(@Dht+dbGt4z2Z$bjU*Qlu1{@+R$`z0om!N&U;Z zRpSA0R=molfzi<63QjWYDGUq7x3xna;7g+KbrQ7>{sxuNk-5DfBXrQ*(%~+&qP(^jnaR4PBk4rm#l6QlUUyf&XvKauL)=7T zVqh7hsY<-Xr!kOGk|rxU)#chxHy~<#e)uXTt+QCu<87QPJzA1#pR;G%8aV^!&$78) z@<>ZCFio4k*Y<7nd_1qL2*-9Z?8q$)_6GMpXuIvn)>NZ?wTp*8?&}sp=PL(Ii7=Ovx@Yqxxxy(syjC-GA0VP+OIz4Jf~{q)T-{D z6L1*ZC7+f6bba!|(!^A1!SZ|WWb($^vxKcST1OwYz^Zb*x)d_@vYBm-Sz}d;M48wc z0U|quUlJWD#9z|p8;**ty4C9sZhmgXY)qA|vtum-4LG^GJgD`|k#(^Q@sWz|8*2~t zW}LN4yo$L@EON=I?$kE&>P(E+%o+SG`^|l8P|@1IjZkBX=+Qn}>fEfC4tc%V&j{NF ze5V7>?)}6^g?e>(r9`7T6Ye8Dl-dE5CUIzt<9eGXcL`mSU7be*p`lz`x{G?3M2W1~ zLz`2K!)tVeAco!bI+Gh7%PXARm>e6p@JZ`*B%cF-jt4CX!kpxr`0LycoO1$2D&YmU z!j-rchU&p+9P8{pU2c!aLu$Ze!kF6uw^s%bIsg zM7H+qrEFLMB!S!28H+zS3YE0o&yuWS?u1Wfz`P#*k3jE z&+=N64SD*2k=bieEF^FCocYBANVT#;iT?c=nY;rKB4HZaeaZlixj}AiE;8q`bYzwtR`!ffNH zdOJY7!^OWa;+3=2q5jY=TIGb6| zXMs!U+a;X4a?eAH=1+@N{C3?M%-9<~$4%n)e($I8=^i5bz*yt1c#n}TpsG+*hCJ6X z;O5>(hj)smEWfu$2UCMXeRtQQUrPV{O22=|2MG)!+{|`aCmE#@G|yS+#W;EzE2fbA z(kwnfx*+d>;NO_=6woMvNm~B8_YNbJ2i_jizB?4P1!z)CmVoOs_L@EY$j=Z&zrRqg zST{@d@Y<`h(_;z^DjBqq!^Swc;y8xsA;&cha5W>{Jjtd;xF!vt#VJKf@@PfQfB>BjprF`AQ$FD}#T6|SBNXqgxZ42@ z#B*11kL}BAvan>JR|O@RbpBSYxidjrHD@jpuc~m1KDeis#We(FW#J5Q zuBw6uh~alxG||>DHamI~xSRQTzACtph;F<1j*sasV== z?4v`DgJ-&$xHBRvpEijFfs-4>#A2s5YZbJ$G`yz<)LAWh4}j2LI^ez!9R}ISsCHM?Ep# zTvX_>lCK*28fR2vig60Ag??`}V>V)7sF>b-ty9^C4?Nl&E^b_}B2H0lywHkiE(zPKGBEb`)+A70`shV^Y zxfSRkxCb6Az+b{1KT-v3EVP#(X!tr3`4Xr%RoFd_vYbSbARhbtzWFEBq*T}j6{8zNPqc-2jLF0WD)w+LXlO9DAK=(Om+FR6^IMh#QWVLp z^`)c0Q1g4c1Lh0B%+!fxuxBhsLofM*MB1d`#z_4Dk}cke(gwuQFcEpg^w<}Wem2}^ zs0?j*C2S3k6tLxi@!0YfdtX=rBlcXl6UlZYcyvAxMDyJRiF_D7^dlA>?ZK#1 z)<%nLCccWbq>rG+HU&i93&JyF13vAlA+Na9_+As6Ng2c-7*X!#U+Di5yZuJKbtv4# z2KtWgsJ(@KtU}(7zY^(0OSC{jjw=l22YAUCqm!@|mZ;Dwqk!i2!DJvilfz%bdv1Rl z+GCP|`(U4gS3~d2`#&^U%;24i*^SKJ!-Sq2UT(A5E)xLg#XN_tW-W_@js`8uT*X)F zN1vh(PQ3YnG_~Ykn@C2OCQ<9PcKditZ=)AtN$(5>rP20>n3!*GohVs6qbCU|11B3S z*&Wjs^uon4PlDt*ZK|k@*>btWCv*m=?MSS8M4e;gN*9fd^ojNP?v=<1OO1_>pwe~y z;Q5T4W;3`_p{u?;zc&!n?2QX|(M%d=p3S+mdD`#4q(1!Q^Rufx)FUld(^jH|PgQYH zq-1MY)Iu$yd1DRYQcqnZCc{!;-k?htv>rt}jh<|X9;!FYbZyN6=`_d@kr$}f_WZe7 zyaQmalwFb*#_8KIixD636|5H4P{@DvorYaL$`~FRUinBrH8Jj4>h$7c^|v-tvU{;_ z)saT4iq3@qSAXEuVV3EE3!rVFO z{cu6vGG6^n9QW}_jUWwNvDHFmtqoxccu_2gSGkWF#Ps~UmyAF+xs$rJRBO!dC%J6> zD`vj`5mXbZcUF7y%o%|Q~1^EdXLIul1rs&6YlAE#I?rU74i zuQD*XBsQgP%<6mLX}gqOH}Moe_sri}mxP1&H&5#iukM2cr#XrL8?PAFNrC%F)~& z#8S@Zw~wk9FZgD^nHW-UJFPyaGNAx)rN<|d-rTfu;-eKvEG)E_V++sMo);@FH_px| zNFhIJ^h!R-|H1F+$+VR5a%^_FRez`Fl5aAycpZH$F~X4EpL?nfUJIMTUK`-?aP!Flzd6C|)ZT_pkRsOLxe%$b zY1^%Psi82@Z6B`LyU4dFzc7fHU^&of%8lHkUSO8|*o@^(kyk*Qt# zfL`JnE@=NoD*FoLgQhYweiEo63P1Y{28@Fnbf0Mk@4J{tDpy5REa+G@&=?j$S&y;T z)a~ZB12Sib7tCUc8S``S*ECo4Dq3D9qqQ|l)`l+nd^vD!^zlpJ>q{~MF-%V%s?7o# zyS0Sw!U+?aealR`sr&rMZ1XFM+Cq(|Rzw2zy1a&dvI8w!PQ^qy@pap#WKLy?eycYw zdr8?W4pW^;ZZ|)(^`fjHHg;pT-i!QydEH$I~~bs@}B8 z^(iXKeg2mXYmH`JCk?;;bcXA&XEtV=vX3E9`KhQ;6v!EhstGGs-O8Z8OOf5zA`6N z>Kk><(BF>9x%EMbONLckC_<_4O8b zn=yg2vCo0B68e3>sbK9=Qz)@w+!C*YSbypSo8{2V~Xl=dNW`JfeL z?J|M$(bR9E$6G;|C+he<5%m>_{eAuTxUdf1zEzn$^6PIDLivZK2@5ePzMm?u-OnMc zfRxYc_@H~7q}+%>kN^y`MLgnU=H>!kt%|A$p=e%uOkA!Fmgyv4lajN7_iwu@1G(5} z*E5cqxh8rnIZV$@f0|GaI`Fo;ya4yPd%<`7!%A_DWaady>rZQ!)|M)#=UfVFIv-vO z3iPI|5Nj^}(9qnQhioczT^e2bg7`a4X#~|wZ6%4O9l~IVEq%P4b6=ySUxuLYy1<7b5 zJ5hU18FwJNnbD6ucYnosmT%kPRu3F3@^Qo?du<9obptE;y`3?RMN<17Mp-9~Jj!Fh5S^p5D@Y%lcI_1SHaP>oGN3t#al#U=!6;G0C|Hg6b*EOv$!xo5Z;#r<8(@W zg-2Af2O)C{zg0xBGx;!i^wWoohM|^vW2nG4u@8=SMb_W#lEY;^e;P$OgdX!1!VSUI zBa=p>_;*|^7>WAa8mVH^p_08<4IKk_vo*J!`QGrB;OH#tyxWI@{~*M{Of_zdgepQ4 zVeHVizDxVxCT{|xu7VvFK5`l4%}LPjrzD+T-Q)R)JR1FCC+P}{?FHJGVuT$!_ky_o zMPfN@5LU%=b|z1wPYes}9c5rDgeJTk7%wGTxI|OAy2Iq78F92koZ}TNqsmGuv%pfW!9l+p(Zwrnqf~wi|Cnb#z?KEj zwWAD4z80= zZ#Ewa=)|ecG@s7my{o#7!0gCt8$*c7iPkvPX~6Tv zU^UJ#4i>#Iua60NvsQ+y?CBCQvjRMWLGvwQ0@dJ$fK^TFNUTSttt z3zyR+sVRo7!?KZq^rh+yi$M9n}A zY@8pkd2%i95$^5+4;+st-gGT+XZcG*SXk}RnJ0CF!QhOPrd?ioUPJ`R;h@1LN7fB) z_e@Q^&wujkbF0aivwIhlz2Y5Akxr7HvR+aRV9ih%mJFY#RU@}xk(qA7gpZIm0WeuY zhJGqf&}%2~6H+grkPMUR2TsYn;<7#a(aN&}(s5_l3gRaRI3*fLHI!uiMcH$$Rv4mVg8QX=Lac(Rw zgfJrrPzlmS6Wg2@OHsspTH!b@j-6>8T*AlKJ;1XbF3!r+Y~r_=u=@4T`s;>W{DC5w zs3LNz?ETLG)-M$gVU}|Vyrz%baYu*;X{u7mFYv_fU1W7}!RY_(MYZ^YKH3`U*`g~}t|iDl#L`aIgCblmj( zo9)-j+N6)t9TS8mFzdZXabSqFM_)m6shtBa;lS%e5I1iKb;}TPsZ5S*M6cqL(JsuFdLE|-R>DS8z2h^!*5cX)75Nz>P zk{jyi2uLc`^0fUNgX+Oo2`N*-%36#WF#1tWNM+POZqL z>&JLj!}l)0ky#kNz<~OZGZ7qi*QuFvAuD8g4hkY_F2bodS9+ygqm{aqpdWzsBo|AT zm}$6{YF#+g-;B6+JF)|tGSPcTNVWHnJ2^yfKbu8G6Q64DJ;bbsXuNmHsX<)!=pV+* zwKX~2;~B%50V9Ta*vfFJ*4qW66yG=nUq}8VvirFWT?AQNUwtcIn#FH1XIZ?(%WEd- z3ZS0ahd@`+^-;rj zW3?Izg+qP5zH@l09HB;hPM?ZL1L2qv;xUjQ(5~nb-7$yz0dGot0c2LKfcrU3&CEB# z`!&>3_r1Ld9z7P|hI7M;Xn%2QWFH6quXA{#HK04>9q4O}U}u~2B&qe!^%UFiow(tg zrsn5ONE}@2^l50z=n}&^z5r&t2&YZJ7qaHcKGGMmIHb9$&t-4QKz7qEs0(J|L^n&6 ze>{Ix*P23p_4wkkPQzZ&Lv6vkcH1p$k@(+DB8QQF<|iCK;x0KuDFh2x^TA*Fp6fU+$$znL*y@ z(9J9@YQ%xs{9ztNIzhHLPv^}8<*yDWta`8M(QBa9m(7Dmr%GRY1Ls0Zo=CI^$#Oz) zUg?gH-j40HKQZDG!`b+Co1Lr0^DU!)EALCX;hH$nO_uC-f)AZq*TFW(%e@jUY%pe3 z>(=RNl!xzPl}*K)i}Ukl15wqmOPoRYz076t$XH3T0<(8X)KF5M-~_SLyIjUHa`{o> zh`ZVNYt*^v-#<<8%>w>O?i*IA?XA+8ffl1#A(})+rM!?T#0eio*Gb?8>62|e+ZNET z&VLQMb+?-g_zAD&SdR*u!QU&tR?9-D)+b31#hCWz9k-2x7JpZ-H8`5yGSZ~;N2>~Z zYe|!zt34&Ng+*M6^UZV-_je6r_)(^d-04uqwR0_C#_A{!t0WYeiF$#mDL+lp{*eBx z!~jwa`B>2wY96tc>$;d=LjX!l71IdbX~^5n{*XOFm?=8@Uo3BddjV>yQCCPF6^uKB zEg}4=_1Xl{8PvB0sbfA8o?tIFg`_b7M9!SN6f_5w>?@QKlfFMlW z5|GL($2~kW^L*bPx)dp-SrmwF{)O)O?$OxZ{>BybZ=-Me=3vW;C$Od z;&@_C54sVtKiNbeqGMD;Y_{_htn+C%Cptn-z%C9RvG*k*u@UVG8Y`xL9l+ozxQGt*ccLfo>qIPACI-)8ba3^&C<;-Kwz z>~=@&6I~Pk-pP4oBL{VElx_Z9n4q4nk=V)$W$IgC^HU zw&9LZu|99?VR7?7Q|>N}G+RB)TDx2YGA$Ylv*tow5whulBFpl|frsUMX=AHPEVYZm zR_wWiNsR#-F2F1@khX6ap_StrWn*`mV>q)3|CyPA?A||=A|iPiX3QEQFt`2j$p!M( z@+zgYEg|aaI?Gk8NEhn$s8px6aGn}sx!2+I%tqY^Q|tMbsh7>kf z3b?0s=GE5v2BjC{u;YSbY)^~v>9_=Sh841eH7GzJdQMUxl`sM>9_!zt&$7bz)J7?4yk-^{H!5uQSnjMD}~_D(8DyomfKN z`^wzPU6-(mzk!rwkKVn=oSUNvqYc-o-j_k!hlDl3vfp#e9r(1Sg#5JdqX&uA1*mH2 zK+3CMeHV59EW#LMUMUQW2(`{8s7iQukTuWE3?GCMpiKD4Iutx4${*V)bM9J4F;?BC z>qPVmtIEOdv{I2}SbW-a+_PBqzDr#0McIuLeZ~3fR|a@a-CN&ItNMu~UGKUH~TQos{cYNE0&ddz43wsC!? z>p`TT369tSn~Z|T;bN3jSmJLiAIAN`y*>sI@@&<)`G#hljAi9XR4DoyQe^aGX9#pB zF=z7-X?y0vt@cbyD`8m#7U*T%DZ=iY^6Qbk#?^*ajk2#AEKk3_X(>u};cq64N^(Gh zvY?V$V(uRo!wcqJ&@b-dn`c?T)%Rd%SDhCh;Z9)3Wr+fzM=3RTCgDLc1zVl6aJ{{-gY^hg)usTv z7t`iGQ;fR4=V%z;s7_J7VcgxTG=sDjm07or5M^?`g1x-}-9{3_QXJUO2*%A@qxJf$X`2k=tE3Z%yZwsmJiDl zJ$lcuOX=P$5+u^D3;oLatXRFpZj*VkF_?A(@(wuNDCO=@R zC*Y;?N=Y7p-i+E_2wZ)}!5>E3 zvQK3H-RWaP?J>Qd^3IaWP`dnZ;Qsx;HT6Kg|9}u?aZZMy&^Nh1vs&C&x;`~|Jy+HQ z8m%(nW2Uwn5`V?+?&x>`(9wtUo)U-J-5Lb-9E=rT^%>IOQoOxUNO`z*CBx_y1aAlU zfxC3DpHSGTW_uP|0c$u(Si;$$^z7mFys(_z^#a8UV0aA1cf=2M#VJTR(Slc)^ajjO zK#a(ZYWs^KP7KUoDN#fI;1+Euwk->c+mm6HNY;8e+y5+@DvM2S&3}YVy~WVkd>^E? z905TnxAyF^tA!tlXkx}+Lr*_$+?$QJv^8a@&(sa$>HwBK#K0(FDkh_e3Ek=|Q}Tk6 ze${SQn9aAL*iK53n--}BuC~D+wj%WXKzFtPpcnhb)J)Z{NY8g#^DLvpbyVIRXaS3u z_6T3-8Zeg&X7_+%G3vH3qyj4*#Sr^^0Zg9f)0R~%U{-|QFKjqYep4u;-*+XC1MwD@ zHagr9k&8=43B$dk+@uI~gz?1$D#&rbC2|U>6Jri49HF^H-|QBK=x`3uZc#@>xfQK@2!%=dlZPc)a!*sG#hBQ#Cm{WJp85gck8FTv z+!h9suj}0a;_%U->JhI26*W%BKDuh8BDTeI`aE{_CM-pf#VFKig{}^U29%prS0DDG zjx~FxV|*JD(PutU7eKWZWgzXi{D^@i!w3OU_=%(dhs}#%U zcq*92-#;HWq0)RMkm9g~El&6g%Czi#vJHL*^9qJH0+GNC{&H|#ISE*yVs z|FfMC9eTR;RV7$bh;HTBR9_R_oPxb>ulqvIJz&0UKJ9B);RDV?jV>F~YV z7Wn%$$E}&5B|cx=J*UCK(axKhW@n%F8>oJ8dXszpd9`{%j^exP&#XU4k1OBt`Pc^b z?PkT@C6$NI{gVr261~yifpfrCbTty^2Xj%s9yh=6qu&85T}@5S1ziSH_`kN*Uu#A; z?}DmpNAT|g(7)qb`z@I0UWQ#5qksGyyf}BVW`MwK_!C#(KdUxwHSh)LlFvr}>s3R1 z_ZyU=$qSM*f9V(0mB68p!+(YU7uEL9kCYT@unjn;r00WyFX<`*^3H05o=(TJmUOK^eeExEOFCUygyzrr#|zLuaXN#xVuRPM9-D1i%0$1gE1=KD z)10y;=sMIjdXS8Gv2 z1ry<{0qpQZTnCtj1D+E%X>H>xwL1Z8U>ddQM{B>?veqMoNkOZTK@hFYC(j3t1g$m- zwE_%`-(eeM1L~#$g_Hu8KDKI{{;1!U>AwHv!0#5h!u{sDE8+ZiAN;iUoZM@&kk-&;g>)t#a-a)ysPlh0`(0TkNDlYKr-RWcOdn$ ztj{syu7p`W<|Xa{C+{&~ES*{jFT*`}FG@DJMX~8RRBHMCZ97BgOfl(dS>iw6bk`l0 zp|`Ql##LN+PwmIh&^@{QobH%w=;jogz%?iUM3;&qc~-%ibFa~@FCbRd=eKx-+Hb(2 z3Nq@D=e9va(L*axQSF&?Cw4^n8soT0G%&xaW&Fb6`tNJNV{NV%C%}goV-~z|x(`Sr zcb{AN_N5JIOw8#s&uPFuiJ44EZttr3>FPYs2=+uWm3t=a&%OQUTRwh#m*Pjv>w!yQ z?5%*CvHk`kUVD4t@iedlUtqyC60SU&v<9ph+LHR0CRV{DHJ!y~RA%aE5)xTbOgHU6 zF~)kJn_%s49s50Z4oe|KflmUkYD%|+DVVbRUkIQ zYEAfvXT|8m2^-^hMiPBMtWPFTv<{EQv1bPW zcM$S5D%HO6hbUwJ?K&chzso>;z>$9)mAC6QA;Q5ZEGiCO?hoin4t>=mWL_OiMp>q5@kf>3H}=+xq&mKEy*A&?aFm$ zZtW82Cv>>$Q$Y8(KuPoB8bH7(QpA>PT?wLG-+@B2Byqgf;!NdF4J7CllOI1hw&NRB z?XdEG;8$Puj~#X}lp*(b$U_>a`6E)+fLBRiU>Gt1nLcEf1<%O@^v8265>mg*ad#Iu zQ^ltC`v{oK7bqKNdTQ4~$!Pm@`}m!!CNI(VFKX8o0rKj>xx(LJD8Wyxl-qJYUyK4Q zJ^lm!{{P-kAfx4LN`z*ISGMe(JTHnlI|{czJg=VP^or;@wP`u6L7 Date: Tue, 26 May 2020 16:53:09 +0800 Subject: [PATCH 046/133] Fix end to end tests --- tests/e2e/bq/feature-stats.py | 11 +++++------ tests/e2e/redis/basic-ingest-redis-serving.py | 11 ++++++----- tests/e2e/requirements.txt | 1 + 3 files changed, 12 insertions(+), 11 deletions(-) diff --git a/tests/e2e/bq/feature-stats.py b/tests/e2e/bq/feature-stats.py index d2300701550..89f871e63b5 100644 --- a/tests/e2e/bq/feature-stats.py +++ b/tests/e2e/bq/feature-stats.py @@ -49,7 +49,6 @@ def client(core_url, allow_dirty): # Get client for core and serving client = Client(core_url=core_url) client.create_project(PROJECT_NAME) - client.set_project(PROJECT_NAME) # Ensure Feast core is active, but empty if not allow_dirty: @@ -171,7 +170,7 @@ def feature_stats_dataset_agg(client, feature_stats_feature_set): def test_feature_stats_retrieval_by_single_dataset(client, feature_stats_dataset_basic): stats = client.get_statistics( - f"{PROJECT_NAME}/feature_stats", + f"feature_stats", features=["strings", "ints", "floats"], store=STORE_NAME, ingestion_ids=[feature_stats_dataset_basic["id"]], @@ -182,7 +181,7 @@ def test_feature_stats_retrieval_by_single_dataset(client, feature_stats_dataset def test_feature_stats_by_date(client, feature_stats_dataset_basic): stats = client.get_statistics( - f"{PROJECT_NAME}/feature_stats", + "feature_stats", features=["strings", "ints", "floats"], store=STORE_NAME, start_date=feature_stats_dataset_basic["date"], @@ -193,7 +192,7 @@ def test_feature_stats_by_date(client, feature_stats_dataset_basic): def test_feature_stats_agg_over_datasets(client, feature_stats_dataset_agg): stats = client.get_statistics( - f"{PROJECT_NAME}/feature_stats", + "feature_stats", features=["strings", "ints", "floats"], store=STORE_NAME, ingestion_ids=feature_stats_dataset_agg["ids"], @@ -203,7 +202,7 @@ def test_feature_stats_agg_over_datasets(client, feature_stats_dataset_agg): def test_feature_stats_agg_over_dates(client, feature_stats_dataset_agg): stats = client.get_statistics( - f"{PROJECT_NAME}/feature_stats", + "feature_stats", features=["strings", "ints", "floats"], store=STORE_NAME, start_date=feature_stats_dataset_agg["start_date"], @@ -230,7 +229,7 @@ def test_feature_stats_force_refresh( time.sleep(10) actual_stats = client.get_statistics( - f"{PROJECT_NAME}/feature_stats", + "feature_stats", features=["strings", "ints", "floats"], store="historical", start_date=feature_stats_dataset_basic["date"], diff --git a/tests/e2e/redis/basic-ingest-redis-serving.py b/tests/e2e/redis/basic-ingest-redis-serving.py index da0967fd764..3b57aef9fb0 100644 --- a/tests/e2e/redis/basic-ingest-redis-serving.py +++ b/tests/e2e/redis/basic-ingest-redis-serving.py @@ -29,6 +29,7 @@ FLOAT_TOLERANCE = 0.00001 PROJECT_NAME = 'basic_' + uuid.uuid4().hex.upper()[0:6] +DIR_PATH = os.path.dirname(os.path.realpath(__file__)) @pytest.fixture(scope='module') @@ -108,8 +109,8 @@ def test_version_returns_results(client): @pytest.mark.run(order=10) def test_basic_register_feature_set_success(client): # Register feature set without project - cust_trans_fs_expected = FeatureSet.from_yaml("basic/cust_trans_fs.yaml") - driver_fs_expected = FeatureSet.from_yaml("basic/driver_fs.yaml") + cust_trans_fs_expected = FeatureSet.from_yaml(f"{DIR_PATH}/basic/cust_trans_fs.yaml") + driver_fs_expected = FeatureSet.from_yaml(f"{DIR_PATH}basic/driver_fs.yaml") client.apply(cust_trans_fs_expected) client.apply(driver_fs_expected) cust_trans_fs_actual = client.get_feature_set("customer_transactions") @@ -118,7 +119,7 @@ def test_basic_register_feature_set_success(client): assert driver_fs_actual == driver_fs_expected # Register feature set with project - cust_trans_fs_expected = FeatureSet.from_yaml("basic/cust_trans_fs.yaml") + cust_trans_fs_expected = FeatureSet.from_yaml(f"{DIR_PATH}/basic/cust_trans_fs.yaml") client.set_project(PROJECT_NAME) client.apply(cust_trans_fs_expected) cust_trans_fs_actual = client.get_feature_set("customer_transactions", @@ -466,7 +467,7 @@ def large_volume_dataframe(): @pytest.mark.run(order=30) def test_large_volume_register_feature_set_success(client): cust_trans_fs_expected = FeatureSet.from_yaml( - "large_volume/cust_trans_large_fs.yaml") + f"{DIR_PATH}/large_volume/cust_trans_large_fs.yaml") # Register feature set client.apply(cust_trans_fs_expected) @@ -604,7 +605,7 @@ def all_types_parquet_file(): def test_all_types_parquet_register_feature_set_success(client): # Load feature set from file all_types_parquet_expected = FeatureSet.from_yaml( - "all_types_parquet/all_types_parquet.yaml") + f"{DIR_PATH}/all_types_parquet/all_types_parquet.yaml") # Register feature set client.apply(all_types_parquet_expected) diff --git a/tests/e2e/requirements.txt b/tests/e2e/requirements.txt index 6a032bb7eb4..040febc4983 100644 --- a/tests/e2e/requirements.txt +++ b/tests/e2e/requirements.txt @@ -10,3 +10,4 @@ pytest-ordering==0.6.* tensorflow-data-validation==0.21.2 deepdiff==4.3.2 tensorflow==2.1.0 +tfx-bsl==0.21.* # lock to 0.21 \ No newline at end of file From b11e54cae273fddda807df0c3390db053a99ee89 Mon Sep 17 00:00:00 2001 From: zhilingc Date: Tue, 26 May 2020 16:55:59 +0800 Subject: [PATCH 047/133] Fix documentation formatting --- docs/user-guide/statistics.md | 8 ++++++-- 1 file changed, 6 insertions(+), 2 deletions(-) diff --git a/docs/user-guide/statistics.md b/docs/user-guide/statistics.md index 2cd2b0030e3..cfb36dea1a8 100644 --- a/docs/user-guide/statistics.md +++ b/docs/user-guide/statistics.md @@ -19,7 +19,7 @@ Statistics can be retrieved from Feast using the python SDK's `get_statistics` m Feature statistics can be retrieved for a single feature set, from a single valid warehouse store. Users can opt to either retrieve feature statistics for a discrete subset of data by providing an `ingestion_id` , a unique id generated for a dataset when it is ingested into feast: ```{python} -#A unique ingestion id is returned for each batch ingestion +# A unique ingestion id is returned for each batch ingestion ingestion_id=client.ingest(feature_set,df) stats = client.get_statistics( @@ -142,7 +142,11 @@ The following metrics are written at the end of each window as [statsd gauges](h feast_ingestion_feature_value_min feast_ingestion_feature_value_max feast_ingestion_feature_value_mean -feast_ingestion_feature_value_percentile_25 feast_ingestion_feature_value_percentile_50 feast_ingestion_feature_value_percentile_90 feast_ingestion_feature_value_percentile_95 feast_ingestion_feature_value_percentile_99 +feast_ingestion_feature_value_percentile_25 +feast_ingestion_feature_value_percentile_50 +feast_ingestion_feature_value_percentile_90 +feast_ingestion_feature_value_percentile_95 +feast_ingestion_feature_value_percentile_99 ``` {% hint style="info" %} From ec01df1409a1e110ff5aaa6b601a6e0740e2e631 Mon Sep 17 00:00:00 2001 From: zhilingc Date: Tue, 26 May 2020 17:40:11 +0800 Subject: [PATCH 048/133] Add missing slash to filepath --- tests/e2e/redis/basic-ingest-redis-serving.py | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/tests/e2e/redis/basic-ingest-redis-serving.py b/tests/e2e/redis/basic-ingest-redis-serving.py index 3b57aef9fb0..31a7c8d73d7 100644 --- a/tests/e2e/redis/basic-ingest-redis-serving.py +++ b/tests/e2e/redis/basic-ingest-redis-serving.py @@ -110,7 +110,7 @@ def test_version_returns_results(client): def test_basic_register_feature_set_success(client): # Register feature set without project cust_trans_fs_expected = FeatureSet.from_yaml(f"{DIR_PATH}/basic/cust_trans_fs.yaml") - driver_fs_expected = FeatureSet.from_yaml(f"{DIR_PATH}basic/driver_fs.yaml") + driver_fs_expected = FeatureSet.from_yaml(f"{DIR_PATH}/basic/driver_fs.yaml") client.apply(cust_trans_fs_expected) client.apply(driver_fs_expected) cust_trans_fs_actual = client.get_feature_set("customer_transactions") From 78871ffa63f9a00c6c2e60b080aeab117244aafe Mon Sep 17 00:00:00 2001 From: zhilingc Date: Tue, 26 May 2020 18:36:10 +0800 Subject: [PATCH 049/133] Mock retriever for statsService test --- core/src/main/java/feast/core/service/StatsService.java | 2 +- .../test/java/feast/core/service/StatsServiceTest.java | 8 +++++++- .../bigquery/statistics/BigQueryStatisticsRetriever.java | 7 ++++++- 3 files changed, 14 insertions(+), 3 deletions(-) diff --git a/core/src/main/java/feast/core/service/StatsService.java b/core/src/main/java/feast/core/service/StatsService.java index 1fddfaea05c..12111866e60 100644 --- a/core/src/main/java/feast/core/service/StatsService.java +++ b/core/src/main/java/feast/core/service/StatsService.java @@ -315,7 +315,7 @@ private List getFeatureNameStatisticsByDate( * @param storeName name of the store to retrieve statistics from * @return {@link StatisticsRetriever} */ - private StatisticsRetriever getStatisticsRetriever(String storeName) + StatisticsRetriever getStatisticsRetriever(String storeName) throws InvalidProtocolBufferException { Store store = storeRepository diff --git a/core/src/test/java/feast/core/service/StatsServiceTest.java b/core/src/test/java/feast/core/service/StatsServiceTest.java index 1146e75a3c3..276d8df4b7c 100644 --- a/core/src/test/java/feast/core/service/StatsServiceTest.java +++ b/core/src/test/java/feast/core/service/StatsServiceTest.java @@ -18,7 +18,7 @@ import static org.hamcrest.Matchers.equalTo; import static org.junit.Assert.assertThat; -import static org.mockito.Mockito.when; +import static org.mockito.Mockito.*; import static org.mockito.MockitoAnnotations.initMocks; import com.google.protobuf.Timestamp; @@ -31,6 +31,7 @@ import feast.proto.core.StoreProto; import feast.proto.core.StoreProto.Store.BigQueryConfig; import feast.proto.core.StoreProto.Store.StoreType; +import feast.storage.connectors.bigquery.statistics.BigQueryStatisticsRetriever; import java.io.IOException; import java.util.Arrays; import java.util.Optional; @@ -127,6 +128,11 @@ public void shouldThrowExceptionIfFeatureSetNotFound() throws IOException { when(featureSetRepository.findFeatureSetByNameAndProject_Name( "my_feature_set", Project.DEFAULT_NAME)) .thenReturn(null); + + statsService = spy(statsService); + BigQueryStatisticsRetriever retriever = mock(BigQueryStatisticsRetriever.class); + doReturn(retriever).when(statsService).getStatisticsRetriever(storeProto.getName()); + expectedException.expect(IllegalArgumentException.class); expectedException.expectMessage("Illegal request. Unable to find feature set my_feature_set"); statsService.getFeatureStatistics(request); diff --git a/storage/connectors/bigquery/src/main/java/feast/storage/connectors/bigquery/statistics/BigQueryStatisticsRetriever.java b/storage/connectors/bigquery/src/main/java/feast/storage/connectors/bigquery/statistics/BigQueryStatisticsRetriever.java index ef572a03bcf..3ea4c64c394 100644 --- a/storage/connectors/bigquery/src/main/java/feast/storage/connectors/bigquery/statistics/BigQueryStatisticsRetriever.java +++ b/storage/connectors/bigquery/src/main/java/feast/storage/connectors/bigquery/statistics/BigQueryStatisticsRetriever.java @@ -41,7 +41,12 @@ public abstract class BigQueryStatisticsRetriever implements StatisticsRetriever public abstract BigQuery bigquery(); public static BigQueryStatisticsRetriever create(BigQueryConfig config) { - BigQuery bigquery = BigQueryOptions.getDefaultInstance().getService(); + BigQuery bigquery = + BigQueryOptions.getDefaultInstance() + .toBuilder() + .setProjectId(config.getProjectId()) + .build() + .getService(); return newBuilder() .setBigquery(bigquery) .setDatasetId(config.getDatasetId()) From d623039918cc8785aca35f3f39be72cfd0b2c2bb Mon Sep 17 00:00:00 2001 From: zhilingc Date: Fri, 29 May 2020 13:56:20 +0800 Subject: [PATCH 050/133] Update golang package path to feast-dev --- .../proto/v0/statistics.proto | 2 +- .../proto/v0/statistics.pb.go | 3120 +++++++++++++++++ 2 files changed, 3121 insertions(+), 1 deletion(-) create mode 100644 sdk/go/protos/tensorflow_metadata/proto/v0/statistics.pb.go diff --git a/protos/tensorflow_metadata/proto/v0/statistics.proto b/protos/tensorflow_metadata/proto/v0/statistics.proto index b07f5e5fc29..3123dad8746 100644 --- a/protos/tensorflow_metadata/proto/v0/statistics.proto +++ b/protos/tensorflow_metadata/proto/v0/statistics.proto @@ -23,7 +23,7 @@ package tensorflow.metadata.v0; option java_package = "org.tensorflow.metadata.v0"; option java_multiple_files = true; -option go_package = "github.com/gojek/feast/sdk/go/protos/tensorflow_metadata/proto/v0"; +option go_package = "github.com/feast-dev/feast/sdk/go/protos/tensorflow_metadata/proto/v0"; import "tensorflow_metadata/proto/v0/path.proto"; diff --git a/sdk/go/protos/tensorflow_metadata/proto/v0/statistics.pb.go b/sdk/go/protos/tensorflow_metadata/proto/v0/statistics.pb.go new file mode 100644 index 00000000000..3d9e7da362d --- /dev/null +++ b/sdk/go/protos/tensorflow_metadata/proto/v0/statistics.pb.go @@ -0,0 +1,3120 @@ +// Copyright 2017 The TensorFlow Authors. All Rights Reserved. +// +// Licensed under the Apache License, Version 2.0 (the "License"); +// you may not use this file except in compliance with the License. +// You may obtain a copy of the License at +// +// http://www.apache.org/licenses/LICENSE-2.0 +// +// Unless required by applicable law or agreed to in writing, software +// distributed under the License is distributed on an "AS IS" BASIS, +// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +// See the License for the specific language governing permissions and +// limitations under the License. +// ============================================================================= + +// Definitions for aggregated feature statistics for datasets. +// TODO(b/80075690): make a Javascript build rule for this. +// TODO(b/80075691): migrate Facets to use this. + +// Code generated by protoc-gen-go. DO NOT EDIT. +// versions: +// protoc-gen-go v1.24.0 +// protoc v3.10.0 +// source: tensorflow_metadata/proto/v0/statistics.proto + +package v0 + +import ( + proto "github.com/golang/protobuf/proto" + protoreflect "google.golang.org/protobuf/reflect/protoreflect" + protoimpl "google.golang.org/protobuf/runtime/protoimpl" + reflect "reflect" + sync "sync" +) + +const ( + // Verify that this generated code is sufficiently up-to-date. + _ = protoimpl.EnforceVersion(20 - protoimpl.MinVersion) + // Verify that runtime/protoimpl is sufficiently up-to-date. + _ = protoimpl.EnforceVersion(protoimpl.MaxVersion - 20) +) + +// This is a compile-time assertion that a sufficiently up-to-date version +// of the legacy proto package is being used. +const _ = proto.ProtoPackageIsVersion4 + +// The types supported by the feature statistics. When aggregating +// tf.Examples, if the bytelist contains a string, it is recommended to encode +// it here as STRING instead of BYTES in order to calculate string-specific +// statistical measures. +type FeatureNameStatistics_Type int32 + +const ( + FeatureNameStatistics_INT FeatureNameStatistics_Type = 0 + FeatureNameStatistics_FLOAT FeatureNameStatistics_Type = 1 + FeatureNameStatistics_STRING FeatureNameStatistics_Type = 2 + FeatureNameStatistics_BYTES FeatureNameStatistics_Type = 3 + FeatureNameStatistics_STRUCT FeatureNameStatistics_Type = 4 +) + +// Enum value maps for FeatureNameStatistics_Type. +var ( + FeatureNameStatistics_Type_name = map[int32]string{ + 0: "INT", + 1: "FLOAT", + 2: "STRING", + 3: "BYTES", + 4: "STRUCT", + } + FeatureNameStatistics_Type_value = map[string]int32{ + "INT": 0, + "FLOAT": 1, + "STRING": 2, + "BYTES": 3, + "STRUCT": 4, + } +) + +func (x FeatureNameStatistics_Type) Enum() *FeatureNameStatistics_Type { + p := new(FeatureNameStatistics_Type) + *p = x + return p +} + +func (x FeatureNameStatistics_Type) String() string { + return protoimpl.X.EnumStringOf(x.Descriptor(), protoreflect.EnumNumber(x)) +} + +func (FeatureNameStatistics_Type) Descriptor() protoreflect.EnumDescriptor { + return file_tensorflow_metadata_proto_v0_statistics_proto_enumTypes[0].Descriptor() +} + +func (FeatureNameStatistics_Type) Type() protoreflect.EnumType { + return &file_tensorflow_metadata_proto_v0_statistics_proto_enumTypes[0] +} + +func (x FeatureNameStatistics_Type) Number() protoreflect.EnumNumber { + return protoreflect.EnumNumber(x) +} + +// Deprecated: Use FeatureNameStatistics_Type.Descriptor instead. +func (FeatureNameStatistics_Type) EnumDescriptor() ([]byte, []int) { + return file_tensorflow_metadata_proto_v0_statistics_proto_rawDescGZIP(), []int{7, 0} +} + +// The type of the histogram. A standard histogram has equal-width buckets. +// The quantiles type is used for when the histogram message is used to store +// quantile information (by using equal-count buckets with variable widths). +type Histogram_HistogramType int32 + +const ( + Histogram_STANDARD Histogram_HistogramType = 0 + Histogram_QUANTILES Histogram_HistogramType = 1 +) + +// Enum value maps for Histogram_HistogramType. +var ( + Histogram_HistogramType_name = map[int32]string{ + 0: "STANDARD", + 1: "QUANTILES", + } + Histogram_HistogramType_value = map[string]int32{ + "STANDARD": 0, + "QUANTILES": 1, + } +) + +func (x Histogram_HistogramType) Enum() *Histogram_HistogramType { + p := new(Histogram_HistogramType) + *p = x + return p +} + +func (x Histogram_HistogramType) String() string { + return protoimpl.X.EnumStringOf(x.Descriptor(), protoreflect.EnumNumber(x)) +} + +func (Histogram_HistogramType) Descriptor() protoreflect.EnumDescriptor { + return file_tensorflow_metadata_proto_v0_statistics_proto_enumTypes[1].Descriptor() +} + +func (Histogram_HistogramType) Type() protoreflect.EnumType { + return &file_tensorflow_metadata_proto_v0_statistics_proto_enumTypes[1] +} + +func (x Histogram_HistogramType) Number() protoreflect.EnumNumber { + return protoreflect.EnumNumber(x) +} + +// Deprecated: Use Histogram_HistogramType.Descriptor instead. +func (Histogram_HistogramType) EnumDescriptor() ([]byte, []int) { + return file_tensorflow_metadata_proto_v0_statistics_proto_rawDescGZIP(), []int{17, 0} +} + +// A list of features statistics for different datasets. If you wish to compare +// different datasets using this list, then the DatasetFeatureStatistics +// entries should all contain the same list of features. +type DatasetFeatureStatisticsList struct { + state protoimpl.MessageState + sizeCache protoimpl.SizeCache + unknownFields protoimpl.UnknownFields + + Datasets []*DatasetFeatureStatistics `protobuf:"bytes,1,rep,name=datasets,proto3" json:"datasets,omitempty"` +} + +func (x *DatasetFeatureStatisticsList) Reset() { + *x = DatasetFeatureStatisticsList{} + if protoimpl.UnsafeEnabled { + mi := &file_tensorflow_metadata_proto_v0_statistics_proto_msgTypes[0] + ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x)) + ms.StoreMessageInfo(mi) + } +} + +func (x *DatasetFeatureStatisticsList) String() string { + return protoimpl.X.MessageStringOf(x) +} + +func (*DatasetFeatureStatisticsList) ProtoMessage() {} + +func (x *DatasetFeatureStatisticsList) ProtoReflect() protoreflect.Message { + mi := &file_tensorflow_metadata_proto_v0_statistics_proto_msgTypes[0] + if protoimpl.UnsafeEnabled && x != nil { + ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x)) + if ms.LoadMessageInfo() == nil { + ms.StoreMessageInfo(mi) + } + return ms + } + return mi.MessageOf(x) +} + +// Deprecated: Use DatasetFeatureStatisticsList.ProtoReflect.Descriptor instead. +func (*DatasetFeatureStatisticsList) Descriptor() ([]byte, []int) { + return file_tensorflow_metadata_proto_v0_statistics_proto_rawDescGZIP(), []int{0} +} + +func (x *DatasetFeatureStatisticsList) GetDatasets() []*DatasetFeatureStatistics { + if x != nil { + return x.Datasets + } + return nil +} + +// The feature statistics for a single dataset. +type DatasetFeatureStatistics struct { + state protoimpl.MessageState + sizeCache protoimpl.SizeCache + unknownFields protoimpl.UnknownFields + + // The name of the dataset. + Name string `protobuf:"bytes,1,opt,name=name,proto3" json:"name,omitempty"` + // The number of examples in the dataset. + NumExamples uint64 `protobuf:"varint,2,opt,name=num_examples,json=numExamples,proto3" json:"num_examples,omitempty"` + // Only valid if the weight feature was specified. + // Treats a missing weighted feature as zero. + WeightedNumExamples float64 `protobuf:"fixed64,4,opt,name=weighted_num_examples,json=weightedNumExamples,proto3" json:"weighted_num_examples,omitempty"` + // The feature statistics for the dataset. + Features []*FeatureNameStatistics `protobuf:"bytes,3,rep,name=features,proto3" json:"features,omitempty"` + // Cross feature statistics for the dataset. + CrossFeatures []*CrossFeatureStatistics `protobuf:"bytes,5,rep,name=cross_features,json=crossFeatures,proto3" json:"cross_features,omitempty"` +} + +func (x *DatasetFeatureStatistics) Reset() { + *x = DatasetFeatureStatistics{} + if protoimpl.UnsafeEnabled { + mi := &file_tensorflow_metadata_proto_v0_statistics_proto_msgTypes[1] + ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x)) + ms.StoreMessageInfo(mi) + } +} + +func (x *DatasetFeatureStatistics) String() string { + return protoimpl.X.MessageStringOf(x) +} + +func (*DatasetFeatureStatistics) ProtoMessage() {} + +func (x *DatasetFeatureStatistics) ProtoReflect() protoreflect.Message { + mi := &file_tensorflow_metadata_proto_v0_statistics_proto_msgTypes[1] + if protoimpl.UnsafeEnabled && x != nil { + ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x)) + if ms.LoadMessageInfo() == nil { + ms.StoreMessageInfo(mi) + } + return ms + } + return mi.MessageOf(x) +} + +// Deprecated: Use DatasetFeatureStatistics.ProtoReflect.Descriptor instead. +func (*DatasetFeatureStatistics) Descriptor() ([]byte, []int) { + return file_tensorflow_metadata_proto_v0_statistics_proto_rawDescGZIP(), []int{1} +} + +func (x *DatasetFeatureStatistics) GetName() string { + if x != nil { + return x.Name + } + return "" +} + +func (x *DatasetFeatureStatistics) GetNumExamples() uint64 { + if x != nil { + return x.NumExamples + } + return 0 +} + +func (x *DatasetFeatureStatistics) GetWeightedNumExamples() float64 { + if x != nil { + return x.WeightedNumExamples + } + return 0 +} + +func (x *DatasetFeatureStatistics) GetFeatures() []*FeatureNameStatistics { + if x != nil { + return x.Features + } + return nil +} + +func (x *DatasetFeatureStatistics) GetCrossFeatures() []*CrossFeatureStatistics { + if x != nil { + return x.CrossFeatures + } + return nil +} + +type CrossFeatureStatistics struct { + state protoimpl.MessageState + sizeCache protoimpl.SizeCache + unknownFields protoimpl.UnknownFields + + // The path of feature x. + PathX *Path `protobuf:"bytes,1,opt,name=path_x,json=pathX,proto3" json:"path_x,omitempty"` + // The path of feature y. + PathY *Path `protobuf:"bytes,2,opt,name=path_y,json=pathY,proto3" json:"path_y,omitempty"` + // Number of occurrences of this feature cross in the data. If any of + // the features in the cross is missing, the example is ignored. + Count uint64 `protobuf:"varint,3,opt,name=count,proto3" json:"count,omitempty"` + // Types that are assignable to CrossStats: + // *CrossFeatureStatistics_NumCrossStats + // *CrossFeatureStatistics_CategoricalCrossStats + CrossStats isCrossFeatureStatistics_CrossStats `protobuf_oneof:"cross_stats"` +} + +func (x *CrossFeatureStatistics) Reset() { + *x = CrossFeatureStatistics{} + if protoimpl.UnsafeEnabled { + mi := &file_tensorflow_metadata_proto_v0_statistics_proto_msgTypes[2] + ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x)) + ms.StoreMessageInfo(mi) + } +} + +func (x *CrossFeatureStatistics) String() string { + return protoimpl.X.MessageStringOf(x) +} + +func (*CrossFeatureStatistics) ProtoMessage() {} + +func (x *CrossFeatureStatistics) ProtoReflect() protoreflect.Message { + mi := &file_tensorflow_metadata_proto_v0_statistics_proto_msgTypes[2] + if protoimpl.UnsafeEnabled && x != nil { + ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x)) + if ms.LoadMessageInfo() == nil { + ms.StoreMessageInfo(mi) + } + return ms + } + return mi.MessageOf(x) +} + +// Deprecated: Use CrossFeatureStatistics.ProtoReflect.Descriptor instead. +func (*CrossFeatureStatistics) Descriptor() ([]byte, []int) { + return file_tensorflow_metadata_proto_v0_statistics_proto_rawDescGZIP(), []int{2} +} + +func (x *CrossFeatureStatistics) GetPathX() *Path { + if x != nil { + return x.PathX + } + return nil +} + +func (x *CrossFeatureStatistics) GetPathY() *Path { + if x != nil { + return x.PathY + } + return nil +} + +func (x *CrossFeatureStatistics) GetCount() uint64 { + if x != nil { + return x.Count + } + return 0 +} + +func (m *CrossFeatureStatistics) GetCrossStats() isCrossFeatureStatistics_CrossStats { + if m != nil { + return m.CrossStats + } + return nil +} + +func (x *CrossFeatureStatistics) GetNumCrossStats() *NumericCrossStatistics { + if x, ok := x.GetCrossStats().(*CrossFeatureStatistics_NumCrossStats); ok { + return x.NumCrossStats + } + return nil +} + +func (x *CrossFeatureStatistics) GetCategoricalCrossStats() *CategoricalCrossStatistics { + if x, ok := x.GetCrossStats().(*CrossFeatureStatistics_CategoricalCrossStats); ok { + return x.CategoricalCrossStats + } + return nil +} + +type isCrossFeatureStatistics_CrossStats interface { + isCrossFeatureStatistics_CrossStats() +} + +type CrossFeatureStatistics_NumCrossStats struct { + NumCrossStats *NumericCrossStatistics `protobuf:"bytes,4,opt,name=num_cross_stats,json=numCrossStats,proto3,oneof"` +} + +type CrossFeatureStatistics_CategoricalCrossStats struct { + CategoricalCrossStats *CategoricalCrossStatistics `protobuf:"bytes,5,opt,name=categorical_cross_stats,json=categoricalCrossStats,proto3,oneof"` +} + +func (*CrossFeatureStatistics_NumCrossStats) isCrossFeatureStatistics_CrossStats() {} + +func (*CrossFeatureStatistics_CategoricalCrossStats) isCrossFeatureStatistics_CrossStats() {} + +type NumericCrossStatistics struct { + state protoimpl.MessageState + sizeCache protoimpl.SizeCache + unknownFields protoimpl.UnknownFields + + // Pearson product-moment correlation coefficient. + Correlation float32 `protobuf:"fixed32,1,opt,name=correlation,proto3" json:"correlation,omitempty"` + // Standard covariance. E[(X-E[X])*(Y-E[Y])] + Covariance float32 `protobuf:"fixed32,2,opt,name=covariance,proto3" json:"covariance,omitempty"` +} + +func (x *NumericCrossStatistics) Reset() { + *x = NumericCrossStatistics{} + if protoimpl.UnsafeEnabled { + mi := &file_tensorflow_metadata_proto_v0_statistics_proto_msgTypes[3] + ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x)) + ms.StoreMessageInfo(mi) + } +} + +func (x *NumericCrossStatistics) String() string { + return protoimpl.X.MessageStringOf(x) +} + +func (*NumericCrossStatistics) ProtoMessage() {} + +func (x *NumericCrossStatistics) ProtoReflect() protoreflect.Message { + mi := &file_tensorflow_metadata_proto_v0_statistics_proto_msgTypes[3] + if protoimpl.UnsafeEnabled && x != nil { + ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x)) + if ms.LoadMessageInfo() == nil { + ms.StoreMessageInfo(mi) + } + return ms + } + return mi.MessageOf(x) +} + +// Deprecated: Use NumericCrossStatistics.ProtoReflect.Descriptor instead. +func (*NumericCrossStatistics) Descriptor() ([]byte, []int) { + return file_tensorflow_metadata_proto_v0_statistics_proto_rawDescGZIP(), []int{3} +} + +func (x *NumericCrossStatistics) GetCorrelation() float32 { + if x != nil { + return x.Correlation + } + return 0 +} + +func (x *NumericCrossStatistics) GetCovariance() float32 { + if x != nil { + return x.Covariance + } + return 0 +} + +type CategoricalCrossStatistics struct { + state protoimpl.MessageState + sizeCache protoimpl.SizeCache + unknownFields protoimpl.UnknownFields + + Lift *LiftStatistics `protobuf:"bytes,1,opt,name=lift,proto3" json:"lift,omitempty"` +} + +func (x *CategoricalCrossStatistics) Reset() { + *x = CategoricalCrossStatistics{} + if protoimpl.UnsafeEnabled { + mi := &file_tensorflow_metadata_proto_v0_statistics_proto_msgTypes[4] + ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x)) + ms.StoreMessageInfo(mi) + } +} + +func (x *CategoricalCrossStatistics) String() string { + return protoimpl.X.MessageStringOf(x) +} + +func (*CategoricalCrossStatistics) ProtoMessage() {} + +func (x *CategoricalCrossStatistics) ProtoReflect() protoreflect.Message { + mi := &file_tensorflow_metadata_proto_v0_statistics_proto_msgTypes[4] + if protoimpl.UnsafeEnabled && x != nil { + ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x)) + if ms.LoadMessageInfo() == nil { + ms.StoreMessageInfo(mi) + } + return ms + } + return mi.MessageOf(x) +} + +// Deprecated: Use CategoricalCrossStatistics.ProtoReflect.Descriptor instead. +func (*CategoricalCrossStatistics) Descriptor() ([]byte, []int) { + return file_tensorflow_metadata_proto_v0_statistics_proto_rawDescGZIP(), []int{4} +} + +func (x *CategoricalCrossStatistics) GetLift() *LiftStatistics { + if x != nil { + return x.Lift + } + return nil +} + +type LiftStatistics struct { + state protoimpl.MessageState + sizeCache protoimpl.SizeCache + unknownFields protoimpl.UnknownFields + + // Lift information for each value of path_y. Lift is defined for each pair of + // values (x,y) as P(path_y=y|path_x=x)/P(path_y=y). + LiftSeries []*LiftSeries `protobuf:"bytes,1,rep,name=lift_series,json=liftSeries,proto3" json:"lift_series,omitempty"` + // Weighted lift information for each value of path_y. Weighted lift is + // defined for each pair of values (x,y) as P(path_y=y|path_x=x)/P(path_y=y) + // where probabilities are computed over weighted example space. + WeightedLiftSeries []*LiftSeries `protobuf:"bytes,2,rep,name=weighted_lift_series,json=weightedLiftSeries,proto3" json:"weighted_lift_series,omitempty"` +} + +func (x *LiftStatistics) Reset() { + *x = LiftStatistics{} + if protoimpl.UnsafeEnabled { + mi := &file_tensorflow_metadata_proto_v0_statistics_proto_msgTypes[5] + ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x)) + ms.StoreMessageInfo(mi) + } +} + +func (x *LiftStatistics) String() string { + return protoimpl.X.MessageStringOf(x) +} + +func (*LiftStatistics) ProtoMessage() {} + +func (x *LiftStatistics) ProtoReflect() protoreflect.Message { + mi := &file_tensorflow_metadata_proto_v0_statistics_proto_msgTypes[5] + if protoimpl.UnsafeEnabled && x != nil { + ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x)) + if ms.LoadMessageInfo() == nil { + ms.StoreMessageInfo(mi) + } + return ms + } + return mi.MessageOf(x) +} + +// Deprecated: Use LiftStatistics.ProtoReflect.Descriptor instead. +func (*LiftStatistics) Descriptor() ([]byte, []int) { + return file_tensorflow_metadata_proto_v0_statistics_proto_rawDescGZIP(), []int{5} +} + +func (x *LiftStatistics) GetLiftSeries() []*LiftSeries { + if x != nil { + return x.LiftSeries + } + return nil +} + +func (x *LiftStatistics) GetWeightedLiftSeries() []*LiftSeries { + if x != nil { + return x.WeightedLiftSeries + } + return nil +} + +// Container for lift information for a specific y-value. +type LiftSeries struct { + state protoimpl.MessageState + sizeCache protoimpl.SizeCache + unknownFields protoimpl.UnknownFields + + // The particular value of path_y corresponding to this LiftSeries. Each + // element in lift_values corresponds to the lift a different x_value and + // this specific y_value. + // + // Types that are assignable to YValue: + // *LiftSeries_YInt + // *LiftSeries_YString + // *LiftSeries_YBucket + YValue isLiftSeries_YValue `protobuf_oneof:"y_value"` + // The number of examples in which y_value appears. + // + // Types that are assignable to YCountValue: + // *LiftSeries_YCount + // *LiftSeries_WeightedYCount + YCountValue isLiftSeries_YCountValue `protobuf_oneof:"y_count_value"` + // The lifts for a each path_x value and this y_value. + LiftValues []*LiftSeries_LiftValue `protobuf:"bytes,6,rep,name=lift_values,json=liftValues,proto3" json:"lift_values,omitempty"` +} + +func (x *LiftSeries) Reset() { + *x = LiftSeries{} + if protoimpl.UnsafeEnabled { + mi := &file_tensorflow_metadata_proto_v0_statistics_proto_msgTypes[6] + ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x)) + ms.StoreMessageInfo(mi) + } +} + +func (x *LiftSeries) String() string { + return protoimpl.X.MessageStringOf(x) +} + +func (*LiftSeries) ProtoMessage() {} + +func (x *LiftSeries) ProtoReflect() protoreflect.Message { + mi := &file_tensorflow_metadata_proto_v0_statistics_proto_msgTypes[6] + if protoimpl.UnsafeEnabled && x != nil { + ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x)) + if ms.LoadMessageInfo() == nil { + ms.StoreMessageInfo(mi) + } + return ms + } + return mi.MessageOf(x) +} + +// Deprecated: Use LiftSeries.ProtoReflect.Descriptor instead. +func (*LiftSeries) Descriptor() ([]byte, []int) { + return file_tensorflow_metadata_proto_v0_statistics_proto_rawDescGZIP(), []int{6} +} + +func (m *LiftSeries) GetYValue() isLiftSeries_YValue { + if m != nil { + return m.YValue + } + return nil +} + +func (x *LiftSeries) GetYInt() int32 { + if x, ok := x.GetYValue().(*LiftSeries_YInt); ok { + return x.YInt + } + return 0 +} + +func (x *LiftSeries) GetYString() string { + if x, ok := x.GetYValue().(*LiftSeries_YString); ok { + return x.YString + } + return "" +} + +func (x *LiftSeries) GetYBucket() *LiftSeries_Bucket { + if x, ok := x.GetYValue().(*LiftSeries_YBucket); ok { + return x.YBucket + } + return nil +} + +func (m *LiftSeries) GetYCountValue() isLiftSeries_YCountValue { + if m != nil { + return m.YCountValue + } + return nil +} + +func (x *LiftSeries) GetYCount() uint64 { + if x, ok := x.GetYCountValue().(*LiftSeries_YCount); ok { + return x.YCount + } + return 0 +} + +func (x *LiftSeries) GetWeightedYCount() float64 { + if x, ok := x.GetYCountValue().(*LiftSeries_WeightedYCount); ok { + return x.WeightedYCount + } + return 0 +} + +func (x *LiftSeries) GetLiftValues() []*LiftSeries_LiftValue { + if x != nil { + return x.LiftValues + } + return nil +} + +type isLiftSeries_YValue interface { + isLiftSeries_YValue() +} + +type LiftSeries_YInt struct { + YInt int32 `protobuf:"varint,1,opt,name=y_int,json=yInt,proto3,oneof"` +} + +type LiftSeries_YString struct { + YString string `protobuf:"bytes,2,opt,name=y_string,json=yString,proto3,oneof"` +} + +type LiftSeries_YBucket struct { + YBucket *LiftSeries_Bucket `protobuf:"bytes,3,opt,name=y_bucket,json=yBucket,proto3,oneof"` +} + +func (*LiftSeries_YInt) isLiftSeries_YValue() {} + +func (*LiftSeries_YString) isLiftSeries_YValue() {} + +func (*LiftSeries_YBucket) isLiftSeries_YValue() {} + +type isLiftSeries_YCountValue interface { + isLiftSeries_YCountValue() +} + +type LiftSeries_YCount struct { + YCount uint64 `protobuf:"varint,4,opt,name=y_count,json=yCount,proto3,oneof"` +} + +type LiftSeries_WeightedYCount struct { + WeightedYCount float64 `protobuf:"fixed64,5,opt,name=weighted_y_count,json=weightedYCount,proto3,oneof"` +} + +func (*LiftSeries_YCount) isLiftSeries_YCountValue() {} + +func (*LiftSeries_WeightedYCount) isLiftSeries_YCountValue() {} + +// The complete set of statistics for a given feature name for a dataset. +type FeatureNameStatistics struct { + state protoimpl.MessageState + sizeCache protoimpl.SizeCache + unknownFields protoimpl.UnknownFields + + // One can identify a field either by the name (for simple fields), or by + // a path (for structured fields). Note that: + // name: "foo" + // is equivalent to: + // path: {step:"foo"} + // Note: this oneof must be consistently either name or path across all + // FeatureNameStatistics in one DatasetFeatureStatistics. + // + // Types that are assignable to FieldId: + // *FeatureNameStatistics_Name + // *FeatureNameStatistics_Path + FieldId isFeatureNameStatistics_FieldId `protobuf_oneof:"field_id"` + // The data type of the feature + Type FeatureNameStatistics_Type `protobuf:"varint,2,opt,name=type,proto3,enum=tensorflow.metadata.v0.FeatureNameStatistics_Type" json:"type,omitempty"` + // The statistics of the values of the feature. + // + // Types that are assignable to Stats: + // *FeatureNameStatistics_NumStats + // *FeatureNameStatistics_StringStats + // *FeatureNameStatistics_BytesStats + // *FeatureNameStatistics_StructStats + Stats isFeatureNameStatistics_Stats `protobuf_oneof:"stats"` + // Any custom statistics can be stored in this list. + CustomStats []*CustomStatistic `protobuf:"bytes,6,rep,name=custom_stats,json=customStats,proto3" json:"custom_stats,omitempty"` +} + +func (x *FeatureNameStatistics) Reset() { + *x = FeatureNameStatistics{} + if protoimpl.UnsafeEnabled { + mi := &file_tensorflow_metadata_proto_v0_statistics_proto_msgTypes[7] + ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x)) + ms.StoreMessageInfo(mi) + } +} + +func (x *FeatureNameStatistics) String() string { + return protoimpl.X.MessageStringOf(x) +} + +func (*FeatureNameStatistics) ProtoMessage() {} + +func (x *FeatureNameStatistics) ProtoReflect() protoreflect.Message { + mi := &file_tensorflow_metadata_proto_v0_statistics_proto_msgTypes[7] + if protoimpl.UnsafeEnabled && x != nil { + ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x)) + if ms.LoadMessageInfo() == nil { + ms.StoreMessageInfo(mi) + } + return ms + } + return mi.MessageOf(x) +} + +// Deprecated: Use FeatureNameStatistics.ProtoReflect.Descriptor instead. +func (*FeatureNameStatistics) Descriptor() ([]byte, []int) { + return file_tensorflow_metadata_proto_v0_statistics_proto_rawDescGZIP(), []int{7} +} + +func (m *FeatureNameStatistics) GetFieldId() isFeatureNameStatistics_FieldId { + if m != nil { + return m.FieldId + } + return nil +} + +func (x *FeatureNameStatistics) GetName() string { + if x, ok := x.GetFieldId().(*FeatureNameStatistics_Name); ok { + return x.Name + } + return "" +} + +func (x *FeatureNameStatistics) GetPath() *Path { + if x, ok := x.GetFieldId().(*FeatureNameStatistics_Path); ok { + return x.Path + } + return nil +} + +func (x *FeatureNameStatistics) GetType() FeatureNameStatistics_Type { + if x != nil { + return x.Type + } + return FeatureNameStatistics_INT +} + +func (m *FeatureNameStatistics) GetStats() isFeatureNameStatistics_Stats { + if m != nil { + return m.Stats + } + return nil +} + +func (x *FeatureNameStatistics) GetNumStats() *NumericStatistics { + if x, ok := x.GetStats().(*FeatureNameStatistics_NumStats); ok { + return x.NumStats + } + return nil +} + +func (x *FeatureNameStatistics) GetStringStats() *StringStatistics { + if x, ok := x.GetStats().(*FeatureNameStatistics_StringStats); ok { + return x.StringStats + } + return nil +} + +func (x *FeatureNameStatistics) GetBytesStats() *BytesStatistics { + if x, ok := x.GetStats().(*FeatureNameStatistics_BytesStats); ok { + return x.BytesStats + } + return nil +} + +func (x *FeatureNameStatistics) GetStructStats() *StructStatistics { + if x, ok := x.GetStats().(*FeatureNameStatistics_StructStats); ok { + return x.StructStats + } + return nil +} + +func (x *FeatureNameStatistics) GetCustomStats() []*CustomStatistic { + if x != nil { + return x.CustomStats + } + return nil +} + +type isFeatureNameStatistics_FieldId interface { + isFeatureNameStatistics_FieldId() +} + +type FeatureNameStatistics_Name struct { + // The feature name + Name string `protobuf:"bytes,1,opt,name=name,proto3,oneof"` +} + +type FeatureNameStatistics_Path struct { + // The path of the feature. + Path *Path `protobuf:"bytes,8,opt,name=path,proto3,oneof"` +} + +func (*FeatureNameStatistics_Name) isFeatureNameStatistics_FieldId() {} + +func (*FeatureNameStatistics_Path) isFeatureNameStatistics_FieldId() {} + +type isFeatureNameStatistics_Stats interface { + isFeatureNameStatistics_Stats() +} + +type FeatureNameStatistics_NumStats struct { + NumStats *NumericStatistics `protobuf:"bytes,3,opt,name=num_stats,json=numStats,proto3,oneof"` +} + +type FeatureNameStatistics_StringStats struct { + StringStats *StringStatistics `protobuf:"bytes,4,opt,name=string_stats,json=stringStats,proto3,oneof"` +} + +type FeatureNameStatistics_BytesStats struct { + BytesStats *BytesStatistics `protobuf:"bytes,5,opt,name=bytes_stats,json=bytesStats,proto3,oneof"` +} + +type FeatureNameStatistics_StructStats struct { + StructStats *StructStatistics `protobuf:"bytes,7,opt,name=struct_stats,json=structStats,proto3,oneof"` +} + +func (*FeatureNameStatistics_NumStats) isFeatureNameStatistics_Stats() {} + +func (*FeatureNameStatistics_StringStats) isFeatureNameStatistics_Stats() {} + +func (*FeatureNameStatistics_BytesStats) isFeatureNameStatistics_Stats() {} + +func (*FeatureNameStatistics_StructStats) isFeatureNameStatistics_Stats() {} + +// Common weighted statistics for all feature types. Statistics counting number +// of values (i.e., avg_num_values and tot_num_values) include NaNs. +// If the weighted column is missing, then this counts as a weight of 1 +// for that example. +type WeightedCommonStatistics struct { + state protoimpl.MessageState + sizeCache protoimpl.SizeCache + unknownFields protoimpl.UnknownFields + + // Weighted number of examples not missing. + NumNonMissing float64 `protobuf:"fixed64,1,opt,name=num_non_missing,json=numNonMissing,proto3" json:"num_non_missing,omitempty"` + // Weighted number of examples missing. + // Note that if the weighted column is zero, this does not count + // as missing. + NumMissing float64 `protobuf:"fixed64,2,opt,name=num_missing,json=numMissing,proto3" json:"num_missing,omitempty"` + // average number of values, weighted by the number of examples. + AvgNumValues float64 `protobuf:"fixed64,3,opt,name=avg_num_values,json=avgNumValues,proto3" json:"avg_num_values,omitempty"` + // tot_num_values = avg_num_values * num_non_missing. + // This is calculated directly, so should have less numerical error. + TotNumValues float64 `protobuf:"fixed64,4,opt,name=tot_num_values,json=totNumValues,proto3" json:"tot_num_values,omitempty"` +} + +func (x *WeightedCommonStatistics) Reset() { + *x = WeightedCommonStatistics{} + if protoimpl.UnsafeEnabled { + mi := &file_tensorflow_metadata_proto_v0_statistics_proto_msgTypes[8] + ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x)) + ms.StoreMessageInfo(mi) + } +} + +func (x *WeightedCommonStatistics) String() string { + return protoimpl.X.MessageStringOf(x) +} + +func (*WeightedCommonStatistics) ProtoMessage() {} + +func (x *WeightedCommonStatistics) ProtoReflect() protoreflect.Message { + mi := &file_tensorflow_metadata_proto_v0_statistics_proto_msgTypes[8] + if protoimpl.UnsafeEnabled && x != nil { + ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x)) + if ms.LoadMessageInfo() == nil { + ms.StoreMessageInfo(mi) + } + return ms + } + return mi.MessageOf(x) +} + +// Deprecated: Use WeightedCommonStatistics.ProtoReflect.Descriptor instead. +func (*WeightedCommonStatistics) Descriptor() ([]byte, []int) { + return file_tensorflow_metadata_proto_v0_statistics_proto_rawDescGZIP(), []int{8} +} + +func (x *WeightedCommonStatistics) GetNumNonMissing() float64 { + if x != nil { + return x.NumNonMissing + } + return 0 +} + +func (x *WeightedCommonStatistics) GetNumMissing() float64 { + if x != nil { + return x.NumMissing + } + return 0 +} + +func (x *WeightedCommonStatistics) GetAvgNumValues() float64 { + if x != nil { + return x.AvgNumValues + } + return 0 +} + +func (x *WeightedCommonStatistics) GetTotNumValues() float64 { + if x != nil { + return x.TotNumValues + } + return 0 +} + +// Stores the name and value of any custom statistic. The value can be a string, +// double, or histogram. +type CustomStatistic struct { + state protoimpl.MessageState + sizeCache protoimpl.SizeCache + unknownFields protoimpl.UnknownFields + + Name string `protobuf:"bytes,1,opt,name=name,proto3" json:"name,omitempty"` + // Types that are assignable to Val: + // *CustomStatistic_Num + // *CustomStatistic_Str + // *CustomStatistic_Histogram + // *CustomStatistic_RankHistogram + Val isCustomStatistic_Val `protobuf_oneof:"val"` +} + +func (x *CustomStatistic) Reset() { + *x = CustomStatistic{} + if protoimpl.UnsafeEnabled { + mi := &file_tensorflow_metadata_proto_v0_statistics_proto_msgTypes[9] + ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x)) + ms.StoreMessageInfo(mi) + } +} + +func (x *CustomStatistic) String() string { + return protoimpl.X.MessageStringOf(x) +} + +func (*CustomStatistic) ProtoMessage() {} + +func (x *CustomStatistic) ProtoReflect() protoreflect.Message { + mi := &file_tensorflow_metadata_proto_v0_statistics_proto_msgTypes[9] + if protoimpl.UnsafeEnabled && x != nil { + ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x)) + if ms.LoadMessageInfo() == nil { + ms.StoreMessageInfo(mi) + } + return ms + } + return mi.MessageOf(x) +} + +// Deprecated: Use CustomStatistic.ProtoReflect.Descriptor instead. +func (*CustomStatistic) Descriptor() ([]byte, []int) { + return file_tensorflow_metadata_proto_v0_statistics_proto_rawDescGZIP(), []int{9} +} + +func (x *CustomStatistic) GetName() string { + if x != nil { + return x.Name + } + return "" +} + +func (m *CustomStatistic) GetVal() isCustomStatistic_Val { + if m != nil { + return m.Val + } + return nil +} + +func (x *CustomStatistic) GetNum() float64 { + if x, ok := x.GetVal().(*CustomStatistic_Num); ok { + return x.Num + } + return 0 +} + +func (x *CustomStatistic) GetStr() string { + if x, ok := x.GetVal().(*CustomStatistic_Str); ok { + return x.Str + } + return "" +} + +func (x *CustomStatistic) GetHistogram() *Histogram { + if x, ok := x.GetVal().(*CustomStatistic_Histogram); ok { + return x.Histogram + } + return nil +} + +func (x *CustomStatistic) GetRankHistogram() *RankHistogram { + if x, ok := x.GetVal().(*CustomStatistic_RankHistogram); ok { + return x.RankHistogram + } + return nil +} + +type isCustomStatistic_Val interface { + isCustomStatistic_Val() +} + +type CustomStatistic_Num struct { + Num float64 `protobuf:"fixed64,2,opt,name=num,proto3,oneof"` +} + +type CustomStatistic_Str struct { + Str string `protobuf:"bytes,3,opt,name=str,proto3,oneof"` +} + +type CustomStatistic_Histogram struct { + Histogram *Histogram `protobuf:"bytes,4,opt,name=histogram,proto3,oneof"` +} + +type CustomStatistic_RankHistogram struct { + RankHistogram *RankHistogram `protobuf:"bytes,5,opt,name=rank_histogram,json=rankHistogram,proto3,oneof"` +} + +func (*CustomStatistic_Num) isCustomStatistic_Val() {} + +func (*CustomStatistic_Str) isCustomStatistic_Val() {} + +func (*CustomStatistic_Histogram) isCustomStatistic_Val() {} + +func (*CustomStatistic_RankHistogram) isCustomStatistic_Val() {} + +// Statistics for a numeric feature in a dataset. +type NumericStatistics struct { + state protoimpl.MessageState + sizeCache protoimpl.SizeCache + unknownFields protoimpl.UnknownFields + + CommonStats *CommonStatistics `protobuf:"bytes,1,opt,name=common_stats,json=commonStats,proto3" json:"common_stats,omitempty"` + // The mean of the values + Mean float64 `protobuf:"fixed64,2,opt,name=mean,proto3" json:"mean,omitempty"` + // The standard deviation of the values + StdDev float64 `protobuf:"fixed64,3,opt,name=std_dev,json=stdDev,proto3" json:"std_dev,omitempty"` + // The number of values that equal 0 + NumZeros uint64 `protobuf:"varint,4,opt,name=num_zeros,json=numZeros,proto3" json:"num_zeros,omitempty"` + // The minimum value + Min float64 `protobuf:"fixed64,5,opt,name=min,proto3" json:"min,omitempty"` + // The median value + Median float64 `protobuf:"fixed64,6,opt,name=median,proto3" json:"median,omitempty"` + // The maximum value + Max float64 `protobuf:"fixed64,7,opt,name=max,proto3" json:"max,omitempty"` + // The histogram(s) of the feature values. + Histograms []*Histogram `protobuf:"bytes,8,rep,name=histograms,proto3" json:"histograms,omitempty"` + // Weighted statistics for the feature, if the values have weights. + WeightedNumericStats *WeightedNumericStatistics `protobuf:"bytes,9,opt,name=weighted_numeric_stats,json=weightedNumericStats,proto3" json:"weighted_numeric_stats,omitempty"` +} + +func (x *NumericStatistics) Reset() { + *x = NumericStatistics{} + if protoimpl.UnsafeEnabled { + mi := &file_tensorflow_metadata_proto_v0_statistics_proto_msgTypes[10] + ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x)) + ms.StoreMessageInfo(mi) + } +} + +func (x *NumericStatistics) String() string { + return protoimpl.X.MessageStringOf(x) +} + +func (*NumericStatistics) ProtoMessage() {} + +func (x *NumericStatistics) ProtoReflect() protoreflect.Message { + mi := &file_tensorflow_metadata_proto_v0_statistics_proto_msgTypes[10] + if protoimpl.UnsafeEnabled && x != nil { + ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x)) + if ms.LoadMessageInfo() == nil { + ms.StoreMessageInfo(mi) + } + return ms + } + return mi.MessageOf(x) +} + +// Deprecated: Use NumericStatistics.ProtoReflect.Descriptor instead. +func (*NumericStatistics) Descriptor() ([]byte, []int) { + return file_tensorflow_metadata_proto_v0_statistics_proto_rawDescGZIP(), []int{10} +} + +func (x *NumericStatistics) GetCommonStats() *CommonStatistics { + if x != nil { + return x.CommonStats + } + return nil +} + +func (x *NumericStatistics) GetMean() float64 { + if x != nil { + return x.Mean + } + return 0 +} + +func (x *NumericStatistics) GetStdDev() float64 { + if x != nil { + return x.StdDev + } + return 0 +} + +func (x *NumericStatistics) GetNumZeros() uint64 { + if x != nil { + return x.NumZeros + } + return 0 +} + +func (x *NumericStatistics) GetMin() float64 { + if x != nil { + return x.Min + } + return 0 +} + +func (x *NumericStatistics) GetMedian() float64 { + if x != nil { + return x.Median + } + return 0 +} + +func (x *NumericStatistics) GetMax() float64 { + if x != nil { + return x.Max + } + return 0 +} + +func (x *NumericStatistics) GetHistograms() []*Histogram { + if x != nil { + return x.Histograms + } + return nil +} + +func (x *NumericStatistics) GetWeightedNumericStats() *WeightedNumericStatistics { + if x != nil { + return x.WeightedNumericStats + } + return nil +} + +// Statistics for a string feature in a dataset. +type StringStatistics struct { + state protoimpl.MessageState + sizeCache protoimpl.SizeCache + unknownFields protoimpl.UnknownFields + + CommonStats *CommonStatistics `protobuf:"bytes,1,opt,name=common_stats,json=commonStats,proto3" json:"common_stats,omitempty"` + // The number of unique values + Unique uint64 `protobuf:"varint,2,opt,name=unique,proto3" json:"unique,omitempty"` + // A sorted list of the most-frequent values and their frequencies, with + // the most-frequent being first. + TopValues []*StringStatistics_FreqAndValue `protobuf:"bytes,3,rep,name=top_values,json=topValues,proto3" json:"top_values,omitempty"` + // The average length of the values + AvgLength float32 `protobuf:"fixed32,4,opt,name=avg_length,json=avgLength,proto3" json:"avg_length,omitempty"` + // The rank histogram for the values of the feature. + // The rank is used to measure of how commonly the value is found in the + // dataset. The most common value would have a rank of 1, with the second-most + // common value having a rank of 2, and so on. + RankHistogram *RankHistogram `protobuf:"bytes,5,opt,name=rank_histogram,json=rankHistogram,proto3" json:"rank_histogram,omitempty"` + // Weighted statistics for the feature, if the values have weights. + WeightedStringStats *WeightedStringStatistics `protobuf:"bytes,6,opt,name=weighted_string_stats,json=weightedStringStats,proto3" json:"weighted_string_stats,omitempty"` + // A vocabulary file, used for vocabularies too large to store in the proto + // itself. Note that the file may be relative to some context-dependent + // directory. E.g. in TFX the feature statistics will live in a PPP and + // vocabulary file names will be relative to this PPP. + VocabularyFile string `protobuf:"bytes,7,opt,name=vocabulary_file,json=vocabularyFile,proto3" json:"vocabulary_file,omitempty"` +} + +func (x *StringStatistics) Reset() { + *x = StringStatistics{} + if protoimpl.UnsafeEnabled { + mi := &file_tensorflow_metadata_proto_v0_statistics_proto_msgTypes[11] + ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x)) + ms.StoreMessageInfo(mi) + } +} + +func (x *StringStatistics) String() string { + return protoimpl.X.MessageStringOf(x) +} + +func (*StringStatistics) ProtoMessage() {} + +func (x *StringStatistics) ProtoReflect() protoreflect.Message { + mi := &file_tensorflow_metadata_proto_v0_statistics_proto_msgTypes[11] + if protoimpl.UnsafeEnabled && x != nil { + ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x)) + if ms.LoadMessageInfo() == nil { + ms.StoreMessageInfo(mi) + } + return ms + } + return mi.MessageOf(x) +} + +// Deprecated: Use StringStatistics.ProtoReflect.Descriptor instead. +func (*StringStatistics) Descriptor() ([]byte, []int) { + return file_tensorflow_metadata_proto_v0_statistics_proto_rawDescGZIP(), []int{11} +} + +func (x *StringStatistics) GetCommonStats() *CommonStatistics { + if x != nil { + return x.CommonStats + } + return nil +} + +func (x *StringStatistics) GetUnique() uint64 { + if x != nil { + return x.Unique + } + return 0 +} + +func (x *StringStatistics) GetTopValues() []*StringStatistics_FreqAndValue { + if x != nil { + return x.TopValues + } + return nil +} + +func (x *StringStatistics) GetAvgLength() float32 { + if x != nil { + return x.AvgLength + } + return 0 +} + +func (x *StringStatistics) GetRankHistogram() *RankHistogram { + if x != nil { + return x.RankHistogram + } + return nil +} + +func (x *StringStatistics) GetWeightedStringStats() *WeightedStringStatistics { + if x != nil { + return x.WeightedStringStats + } + return nil +} + +func (x *StringStatistics) GetVocabularyFile() string { + if x != nil { + return x.VocabularyFile + } + return "" +} + +// Statistics for a weighted numeric feature in a dataset. +type WeightedNumericStatistics struct { + state protoimpl.MessageState + sizeCache protoimpl.SizeCache + unknownFields protoimpl.UnknownFields + + // The weighted mean of the values + Mean float64 `protobuf:"fixed64,1,opt,name=mean,proto3" json:"mean,omitempty"` + // The weighted standard deviation of the values + StdDev float64 `protobuf:"fixed64,2,opt,name=std_dev,json=stdDev,proto3" json:"std_dev,omitempty"` + // The weighted median of the values + Median float64 `protobuf:"fixed64,3,opt,name=median,proto3" json:"median,omitempty"` + // The histogram(s) of the weighted feature values. + Histograms []*Histogram `protobuf:"bytes,4,rep,name=histograms,proto3" json:"histograms,omitempty"` +} + +func (x *WeightedNumericStatistics) Reset() { + *x = WeightedNumericStatistics{} + if protoimpl.UnsafeEnabled { + mi := &file_tensorflow_metadata_proto_v0_statistics_proto_msgTypes[12] + ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x)) + ms.StoreMessageInfo(mi) + } +} + +func (x *WeightedNumericStatistics) String() string { + return protoimpl.X.MessageStringOf(x) +} + +func (*WeightedNumericStatistics) ProtoMessage() {} + +func (x *WeightedNumericStatistics) ProtoReflect() protoreflect.Message { + mi := &file_tensorflow_metadata_proto_v0_statistics_proto_msgTypes[12] + if protoimpl.UnsafeEnabled && x != nil { + ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x)) + if ms.LoadMessageInfo() == nil { + ms.StoreMessageInfo(mi) + } + return ms + } + return mi.MessageOf(x) +} + +// Deprecated: Use WeightedNumericStatistics.ProtoReflect.Descriptor instead. +func (*WeightedNumericStatistics) Descriptor() ([]byte, []int) { + return file_tensorflow_metadata_proto_v0_statistics_proto_rawDescGZIP(), []int{12} +} + +func (x *WeightedNumericStatistics) GetMean() float64 { + if x != nil { + return x.Mean + } + return 0 +} + +func (x *WeightedNumericStatistics) GetStdDev() float64 { + if x != nil { + return x.StdDev + } + return 0 +} + +func (x *WeightedNumericStatistics) GetMedian() float64 { + if x != nil { + return x.Median + } + return 0 +} + +func (x *WeightedNumericStatistics) GetHistograms() []*Histogram { + if x != nil { + return x.Histograms + } + return nil +} + +// Statistics for a weighted string feature in a dataset. +type WeightedStringStatistics struct { + state protoimpl.MessageState + sizeCache protoimpl.SizeCache + unknownFields protoimpl.UnknownFields + + // A sorted list of the most-frequent values and their weighted frequencies, + // with the most-frequent being first. + TopValues []*StringStatistics_FreqAndValue `protobuf:"bytes,1,rep,name=top_values,json=topValues,proto3" json:"top_values,omitempty"` + // The rank histogram for the weighted values of the feature. + RankHistogram *RankHistogram `protobuf:"bytes,2,opt,name=rank_histogram,json=rankHistogram,proto3" json:"rank_histogram,omitempty"` +} + +func (x *WeightedStringStatistics) Reset() { + *x = WeightedStringStatistics{} + if protoimpl.UnsafeEnabled { + mi := &file_tensorflow_metadata_proto_v0_statistics_proto_msgTypes[13] + ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x)) + ms.StoreMessageInfo(mi) + } +} + +func (x *WeightedStringStatistics) String() string { + return protoimpl.X.MessageStringOf(x) +} + +func (*WeightedStringStatistics) ProtoMessage() {} + +func (x *WeightedStringStatistics) ProtoReflect() protoreflect.Message { + mi := &file_tensorflow_metadata_proto_v0_statistics_proto_msgTypes[13] + if protoimpl.UnsafeEnabled && x != nil { + ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x)) + if ms.LoadMessageInfo() == nil { + ms.StoreMessageInfo(mi) + } + return ms + } + return mi.MessageOf(x) +} + +// Deprecated: Use WeightedStringStatistics.ProtoReflect.Descriptor instead. +func (*WeightedStringStatistics) Descriptor() ([]byte, []int) { + return file_tensorflow_metadata_proto_v0_statistics_proto_rawDescGZIP(), []int{13} +} + +func (x *WeightedStringStatistics) GetTopValues() []*StringStatistics_FreqAndValue { + if x != nil { + return x.TopValues + } + return nil +} + +func (x *WeightedStringStatistics) GetRankHistogram() *RankHistogram { + if x != nil { + return x.RankHistogram + } + return nil +} + +// Statistics for a bytes feature in a dataset. +type BytesStatistics struct { + state protoimpl.MessageState + sizeCache protoimpl.SizeCache + unknownFields protoimpl.UnknownFields + + CommonStats *CommonStatistics `protobuf:"bytes,1,opt,name=common_stats,json=commonStats,proto3" json:"common_stats,omitempty"` + // The number of unique values + Unique uint64 `protobuf:"varint,2,opt,name=unique,proto3" json:"unique,omitempty"` + // The average number of bytes in a value + AvgNumBytes float32 `protobuf:"fixed32,3,opt,name=avg_num_bytes,json=avgNumBytes,proto3" json:"avg_num_bytes,omitempty"` + // The minimum number of bytes in a value + MinNumBytes float32 `protobuf:"fixed32,4,opt,name=min_num_bytes,json=minNumBytes,proto3" json:"min_num_bytes,omitempty"` + // The maximum number of bytes in a value + MaxNumBytes float32 `protobuf:"fixed32,5,opt,name=max_num_bytes,json=maxNumBytes,proto3" json:"max_num_bytes,omitempty"` +} + +func (x *BytesStatistics) Reset() { + *x = BytesStatistics{} + if protoimpl.UnsafeEnabled { + mi := &file_tensorflow_metadata_proto_v0_statistics_proto_msgTypes[14] + ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x)) + ms.StoreMessageInfo(mi) + } +} + +func (x *BytesStatistics) String() string { + return protoimpl.X.MessageStringOf(x) +} + +func (*BytesStatistics) ProtoMessage() {} + +func (x *BytesStatistics) ProtoReflect() protoreflect.Message { + mi := &file_tensorflow_metadata_proto_v0_statistics_proto_msgTypes[14] + if protoimpl.UnsafeEnabled && x != nil { + ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x)) + if ms.LoadMessageInfo() == nil { + ms.StoreMessageInfo(mi) + } + return ms + } + return mi.MessageOf(x) +} + +// Deprecated: Use BytesStatistics.ProtoReflect.Descriptor instead. +func (*BytesStatistics) Descriptor() ([]byte, []int) { + return file_tensorflow_metadata_proto_v0_statistics_proto_rawDescGZIP(), []int{14} +} + +func (x *BytesStatistics) GetCommonStats() *CommonStatistics { + if x != nil { + return x.CommonStats + } + return nil +} + +func (x *BytesStatistics) GetUnique() uint64 { + if x != nil { + return x.Unique + } + return 0 +} + +func (x *BytesStatistics) GetAvgNumBytes() float32 { + if x != nil { + return x.AvgNumBytes + } + return 0 +} + +func (x *BytesStatistics) GetMinNumBytes() float32 { + if x != nil { + return x.MinNumBytes + } + return 0 +} + +func (x *BytesStatistics) GetMaxNumBytes() float32 { + if x != nil { + return x.MaxNumBytes + } + return 0 +} + +type StructStatistics struct { + state protoimpl.MessageState + sizeCache protoimpl.SizeCache + unknownFields protoimpl.UnknownFields + + CommonStats *CommonStatistics `protobuf:"bytes,1,opt,name=common_stats,json=commonStats,proto3" json:"common_stats,omitempty"` +} + +func (x *StructStatistics) Reset() { + *x = StructStatistics{} + if protoimpl.UnsafeEnabled { + mi := &file_tensorflow_metadata_proto_v0_statistics_proto_msgTypes[15] + ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x)) + ms.StoreMessageInfo(mi) + } +} + +func (x *StructStatistics) String() string { + return protoimpl.X.MessageStringOf(x) +} + +func (*StructStatistics) ProtoMessage() {} + +func (x *StructStatistics) ProtoReflect() protoreflect.Message { + mi := &file_tensorflow_metadata_proto_v0_statistics_proto_msgTypes[15] + if protoimpl.UnsafeEnabled && x != nil { + ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x)) + if ms.LoadMessageInfo() == nil { + ms.StoreMessageInfo(mi) + } + return ms + } + return mi.MessageOf(x) +} + +// Deprecated: Use StructStatistics.ProtoReflect.Descriptor instead. +func (*StructStatistics) Descriptor() ([]byte, []int) { + return file_tensorflow_metadata_proto_v0_statistics_proto_rawDescGZIP(), []int{15} +} + +func (x *StructStatistics) GetCommonStats() *CommonStatistics { + if x != nil { + return x.CommonStats + } + return nil +} + +// Common statistics for all feature types. Statistics counting number of values +// (i.e., min_num_values, max_num_values, avg_num_values, and tot_num_values) +// include NaNs. +type CommonStatistics struct { + state protoimpl.MessageState + sizeCache protoimpl.SizeCache + unknownFields protoimpl.UnknownFields + + // The number of examples with at least one value for this feature. + NumNonMissing uint64 `protobuf:"varint,1,opt,name=num_non_missing,json=numNonMissing,proto3" json:"num_non_missing,omitempty"` + // The number of examples with no values for this feature. + NumMissing uint64 `protobuf:"varint,2,opt,name=num_missing,json=numMissing,proto3" json:"num_missing,omitempty"` + // The minimum number of values in a single example for this feature. + MinNumValues uint64 `protobuf:"varint,3,opt,name=min_num_values,json=minNumValues,proto3" json:"min_num_values,omitempty"` + // The maximum number of values in a single example for this feature. + MaxNumValues uint64 `protobuf:"varint,4,opt,name=max_num_values,json=maxNumValues,proto3" json:"max_num_values,omitempty"` + // The average number of values in a single example for this feature. + AvgNumValues float32 `protobuf:"fixed32,5,opt,name=avg_num_values,json=avgNumValues,proto3" json:"avg_num_values,omitempty"` + // tot_num_values = avg_num_values * num_non_missing. + // This is calculated directly, so should have less numerical error. + TotNumValues uint64 `protobuf:"varint,8,opt,name=tot_num_values,json=totNumValues,proto3" json:"tot_num_values,omitempty"` + // The quantiles histogram for the number of values in this feature. + NumValuesHistogram *Histogram `protobuf:"bytes,6,opt,name=num_values_histogram,json=numValuesHistogram,proto3" json:"num_values_histogram,omitempty"` + WeightedCommonStats *WeightedCommonStatistics `protobuf:"bytes,7,opt,name=weighted_common_stats,json=weightedCommonStats,proto3" json:"weighted_common_stats,omitempty"` + // The histogram for the number of features in the feature list (only set if + // this feature is a non-context feature from a tf.SequenceExample). + // This is different from num_values_histogram, as num_values_histogram tracks + // the count of all values for a feature in an example, whereas this tracks + // the length of the feature list for this feature in an example (where each + // feature list can contain multiple values). + FeatureListLengthHistogram *Histogram `protobuf:"bytes,9,opt,name=feature_list_length_histogram,json=featureListLengthHistogram,proto3" json:"feature_list_length_histogram,omitempty"` +} + +func (x *CommonStatistics) Reset() { + *x = CommonStatistics{} + if protoimpl.UnsafeEnabled { + mi := &file_tensorflow_metadata_proto_v0_statistics_proto_msgTypes[16] + ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x)) + ms.StoreMessageInfo(mi) + } +} + +func (x *CommonStatistics) String() string { + return protoimpl.X.MessageStringOf(x) +} + +func (*CommonStatistics) ProtoMessage() {} + +func (x *CommonStatistics) ProtoReflect() protoreflect.Message { + mi := &file_tensorflow_metadata_proto_v0_statistics_proto_msgTypes[16] + if protoimpl.UnsafeEnabled && x != nil { + ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x)) + if ms.LoadMessageInfo() == nil { + ms.StoreMessageInfo(mi) + } + return ms + } + return mi.MessageOf(x) +} + +// Deprecated: Use CommonStatistics.ProtoReflect.Descriptor instead. +func (*CommonStatistics) Descriptor() ([]byte, []int) { + return file_tensorflow_metadata_proto_v0_statistics_proto_rawDescGZIP(), []int{16} +} + +func (x *CommonStatistics) GetNumNonMissing() uint64 { + if x != nil { + return x.NumNonMissing + } + return 0 +} + +func (x *CommonStatistics) GetNumMissing() uint64 { + if x != nil { + return x.NumMissing + } + return 0 +} + +func (x *CommonStatistics) GetMinNumValues() uint64 { + if x != nil { + return x.MinNumValues + } + return 0 +} + +func (x *CommonStatistics) GetMaxNumValues() uint64 { + if x != nil { + return x.MaxNumValues + } + return 0 +} + +func (x *CommonStatistics) GetAvgNumValues() float32 { + if x != nil { + return x.AvgNumValues + } + return 0 +} + +func (x *CommonStatistics) GetTotNumValues() uint64 { + if x != nil { + return x.TotNumValues + } + return 0 +} + +func (x *CommonStatistics) GetNumValuesHistogram() *Histogram { + if x != nil { + return x.NumValuesHistogram + } + return nil +} + +func (x *CommonStatistics) GetWeightedCommonStats() *WeightedCommonStatistics { + if x != nil { + return x.WeightedCommonStats + } + return nil +} + +func (x *CommonStatistics) GetFeatureListLengthHistogram() *Histogram { + if x != nil { + return x.FeatureListLengthHistogram + } + return nil +} + +// The data used to create a histogram of a numeric feature for a dataset. +type Histogram struct { + state protoimpl.MessageState + sizeCache protoimpl.SizeCache + unknownFields protoimpl.UnknownFields + + // The number of NaN values in the dataset. + NumNan uint64 `protobuf:"varint,1,opt,name=num_nan,json=numNan,proto3" json:"num_nan,omitempty"` + // The number of undefined values in the dataset. + NumUndefined uint64 `protobuf:"varint,2,opt,name=num_undefined,json=numUndefined,proto3" json:"num_undefined,omitempty"` + // A list of buckets in the histogram, sorted from lowest bucket to highest + // bucket. + Buckets []*Histogram_Bucket `protobuf:"bytes,3,rep,name=buckets,proto3" json:"buckets,omitempty"` + // The type of the histogram. + Type Histogram_HistogramType `protobuf:"varint,4,opt,name=type,proto3,enum=tensorflow.metadata.v0.Histogram_HistogramType" json:"type,omitempty"` + // An optional descriptive name of the histogram, to be used for labeling. + Name string `protobuf:"bytes,5,opt,name=name,proto3" json:"name,omitempty"` +} + +func (x *Histogram) Reset() { + *x = Histogram{} + if protoimpl.UnsafeEnabled { + mi := &file_tensorflow_metadata_proto_v0_statistics_proto_msgTypes[17] + ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x)) + ms.StoreMessageInfo(mi) + } +} + +func (x *Histogram) String() string { + return protoimpl.X.MessageStringOf(x) +} + +func (*Histogram) ProtoMessage() {} + +func (x *Histogram) ProtoReflect() protoreflect.Message { + mi := &file_tensorflow_metadata_proto_v0_statistics_proto_msgTypes[17] + if protoimpl.UnsafeEnabled && x != nil { + ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x)) + if ms.LoadMessageInfo() == nil { + ms.StoreMessageInfo(mi) + } + return ms + } + return mi.MessageOf(x) +} + +// Deprecated: Use Histogram.ProtoReflect.Descriptor instead. +func (*Histogram) Descriptor() ([]byte, []int) { + return file_tensorflow_metadata_proto_v0_statistics_proto_rawDescGZIP(), []int{17} +} + +func (x *Histogram) GetNumNan() uint64 { + if x != nil { + return x.NumNan + } + return 0 +} + +func (x *Histogram) GetNumUndefined() uint64 { + if x != nil { + return x.NumUndefined + } + return 0 +} + +func (x *Histogram) GetBuckets() []*Histogram_Bucket { + if x != nil { + return x.Buckets + } + return nil +} + +func (x *Histogram) GetType() Histogram_HistogramType { + if x != nil { + return x.Type + } + return Histogram_STANDARD +} + +func (x *Histogram) GetName() string { + if x != nil { + return x.Name + } + return "" +} + +// The data used to create a rank histogram of a non-numeric feature of a +// dataset. The rank of a value in a feature can be used as a measure of how +// commonly the value is found in the entire dataset. With bucket sizes of one, +// this becomes a distribution function of all feature values. +type RankHistogram struct { + state protoimpl.MessageState + sizeCache protoimpl.SizeCache + unknownFields protoimpl.UnknownFields + + // A list of buckets in the histogram, sorted from lowest-ranked bucket to + // highest-ranked bucket. + Buckets []*RankHistogram_Bucket `protobuf:"bytes,1,rep,name=buckets,proto3" json:"buckets,omitempty"` + // An optional descriptive name of the histogram, to be used for labeling. + Name string `protobuf:"bytes,2,opt,name=name,proto3" json:"name,omitempty"` +} + +func (x *RankHistogram) Reset() { + *x = RankHistogram{} + if protoimpl.UnsafeEnabled { + mi := &file_tensorflow_metadata_proto_v0_statistics_proto_msgTypes[18] + ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x)) + ms.StoreMessageInfo(mi) + } +} + +func (x *RankHistogram) String() string { + return protoimpl.X.MessageStringOf(x) +} + +func (*RankHistogram) ProtoMessage() {} + +func (x *RankHistogram) ProtoReflect() protoreflect.Message { + mi := &file_tensorflow_metadata_proto_v0_statistics_proto_msgTypes[18] + if protoimpl.UnsafeEnabled && x != nil { + ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x)) + if ms.LoadMessageInfo() == nil { + ms.StoreMessageInfo(mi) + } + return ms + } + return mi.MessageOf(x) +} + +// Deprecated: Use RankHistogram.ProtoReflect.Descriptor instead. +func (*RankHistogram) Descriptor() ([]byte, []int) { + return file_tensorflow_metadata_proto_v0_statistics_proto_rawDescGZIP(), []int{18} +} + +func (x *RankHistogram) GetBuckets() []*RankHistogram_Bucket { + if x != nil { + return x.Buckets + } + return nil +} + +func (x *RankHistogram) GetName() string { + if x != nil { + return x.Name + } + return "" +} + +// A bucket for referring to binned numeric features. +type LiftSeries_Bucket struct { + state protoimpl.MessageState + sizeCache protoimpl.SizeCache + unknownFields protoimpl.UnknownFields + + // The low value of the bucket, inclusive. + LowValue float64 `protobuf:"fixed64,1,opt,name=low_value,json=lowValue,proto3" json:"low_value,omitempty"` + // The high value of the bucket, exclusive (unless the high_value is + // positive infinity). + HighValue float64 `protobuf:"fixed64,2,opt,name=high_value,json=highValue,proto3" json:"high_value,omitempty"` +} + +func (x *LiftSeries_Bucket) Reset() { + *x = LiftSeries_Bucket{} + if protoimpl.UnsafeEnabled { + mi := &file_tensorflow_metadata_proto_v0_statistics_proto_msgTypes[19] + ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x)) + ms.StoreMessageInfo(mi) + } +} + +func (x *LiftSeries_Bucket) String() string { + return protoimpl.X.MessageStringOf(x) +} + +func (*LiftSeries_Bucket) ProtoMessage() {} + +func (x *LiftSeries_Bucket) ProtoReflect() protoreflect.Message { + mi := &file_tensorflow_metadata_proto_v0_statistics_proto_msgTypes[19] + if protoimpl.UnsafeEnabled && x != nil { + ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x)) + if ms.LoadMessageInfo() == nil { + ms.StoreMessageInfo(mi) + } + return ms + } + return mi.MessageOf(x) +} + +// Deprecated: Use LiftSeries_Bucket.ProtoReflect.Descriptor instead. +func (*LiftSeries_Bucket) Descriptor() ([]byte, []int) { + return file_tensorflow_metadata_proto_v0_statistics_proto_rawDescGZIP(), []int{6, 0} +} + +func (x *LiftSeries_Bucket) GetLowValue() float64 { + if x != nil { + return x.LowValue + } + return 0 +} + +func (x *LiftSeries_Bucket) GetHighValue() float64 { + if x != nil { + return x.HighValue + } + return 0 +} + +// A container for lift information about a specific value of path_x. +type LiftSeries_LiftValue struct { + state protoimpl.MessageState + sizeCache protoimpl.SizeCache + unknownFields protoimpl.UnknownFields + + // Types that are assignable to XValue: + // *LiftSeries_LiftValue_XInt + // *LiftSeries_LiftValue_XString + XValue isLiftSeries_LiftValue_XValue `protobuf_oneof:"x_value"` + // P(path_y=y|path_x=x) / P(path_y=y) for x_value and the enclosing y_value. + // In terms of concrete fields, this number represents: + // (x_and_y_count / x_count) / (y_count / num_examples) + Lift float64 `protobuf:"fixed64,3,opt,name=lift,proto3" json:"lift,omitempty"` + // The number of examples in which x_value appears. + // + // Types that are assignable to XCountValue: + // *LiftSeries_LiftValue_XCount + // *LiftSeries_LiftValue_WeightedXCount + XCountValue isLiftSeries_LiftValue_XCountValue `protobuf_oneof:"x_count_value"` + // The number of examples in which x_value appears and y_value appears. + // + // Types that are assignable to XAndYCountValue: + // *LiftSeries_LiftValue_XAndYCount + // *LiftSeries_LiftValue_WeightedXAndYCount + XAndYCountValue isLiftSeries_LiftValue_XAndYCountValue `protobuf_oneof:"x_and_y_count_value"` +} + +func (x *LiftSeries_LiftValue) Reset() { + *x = LiftSeries_LiftValue{} + if protoimpl.UnsafeEnabled { + mi := &file_tensorflow_metadata_proto_v0_statistics_proto_msgTypes[20] + ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x)) + ms.StoreMessageInfo(mi) + } +} + +func (x *LiftSeries_LiftValue) String() string { + return protoimpl.X.MessageStringOf(x) +} + +func (*LiftSeries_LiftValue) ProtoMessage() {} + +func (x *LiftSeries_LiftValue) ProtoReflect() protoreflect.Message { + mi := &file_tensorflow_metadata_proto_v0_statistics_proto_msgTypes[20] + if protoimpl.UnsafeEnabled && x != nil { + ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x)) + if ms.LoadMessageInfo() == nil { + ms.StoreMessageInfo(mi) + } + return ms + } + return mi.MessageOf(x) +} + +// Deprecated: Use LiftSeries_LiftValue.ProtoReflect.Descriptor instead. +func (*LiftSeries_LiftValue) Descriptor() ([]byte, []int) { + return file_tensorflow_metadata_proto_v0_statistics_proto_rawDescGZIP(), []int{6, 1} +} + +func (m *LiftSeries_LiftValue) GetXValue() isLiftSeries_LiftValue_XValue { + if m != nil { + return m.XValue + } + return nil +} + +func (x *LiftSeries_LiftValue) GetXInt() int32 { + if x, ok := x.GetXValue().(*LiftSeries_LiftValue_XInt); ok { + return x.XInt + } + return 0 +} + +func (x *LiftSeries_LiftValue) GetXString() string { + if x, ok := x.GetXValue().(*LiftSeries_LiftValue_XString); ok { + return x.XString + } + return "" +} + +func (x *LiftSeries_LiftValue) GetLift() float64 { + if x != nil { + return x.Lift + } + return 0 +} + +func (m *LiftSeries_LiftValue) GetXCountValue() isLiftSeries_LiftValue_XCountValue { + if m != nil { + return m.XCountValue + } + return nil +} + +func (x *LiftSeries_LiftValue) GetXCount() uint64 { + if x, ok := x.GetXCountValue().(*LiftSeries_LiftValue_XCount); ok { + return x.XCount + } + return 0 +} + +func (x *LiftSeries_LiftValue) GetWeightedXCount() float64 { + if x, ok := x.GetXCountValue().(*LiftSeries_LiftValue_WeightedXCount); ok { + return x.WeightedXCount + } + return 0 +} + +func (m *LiftSeries_LiftValue) GetXAndYCountValue() isLiftSeries_LiftValue_XAndYCountValue { + if m != nil { + return m.XAndYCountValue + } + return nil +} + +func (x *LiftSeries_LiftValue) GetXAndYCount() uint64 { + if x, ok := x.GetXAndYCountValue().(*LiftSeries_LiftValue_XAndYCount); ok { + return x.XAndYCount + } + return 0 +} + +func (x *LiftSeries_LiftValue) GetWeightedXAndYCount() float64 { + if x, ok := x.GetXAndYCountValue().(*LiftSeries_LiftValue_WeightedXAndYCount); ok { + return x.WeightedXAndYCount + } + return 0 +} + +type isLiftSeries_LiftValue_XValue interface { + isLiftSeries_LiftValue_XValue() +} + +type LiftSeries_LiftValue_XInt struct { + XInt int32 `protobuf:"varint,1,opt,name=x_int,json=xInt,proto3,oneof"` +} + +type LiftSeries_LiftValue_XString struct { + XString string `protobuf:"bytes,2,opt,name=x_string,json=xString,proto3,oneof"` +} + +func (*LiftSeries_LiftValue_XInt) isLiftSeries_LiftValue_XValue() {} + +func (*LiftSeries_LiftValue_XString) isLiftSeries_LiftValue_XValue() {} + +type isLiftSeries_LiftValue_XCountValue interface { + isLiftSeries_LiftValue_XCountValue() +} + +type LiftSeries_LiftValue_XCount struct { + XCount uint64 `protobuf:"varint,4,opt,name=x_count,json=xCount,proto3,oneof"` +} + +type LiftSeries_LiftValue_WeightedXCount struct { + WeightedXCount float64 `protobuf:"fixed64,5,opt,name=weighted_x_count,json=weightedXCount,proto3,oneof"` +} + +func (*LiftSeries_LiftValue_XCount) isLiftSeries_LiftValue_XCountValue() {} + +func (*LiftSeries_LiftValue_WeightedXCount) isLiftSeries_LiftValue_XCountValue() {} + +type isLiftSeries_LiftValue_XAndYCountValue interface { + isLiftSeries_LiftValue_XAndYCountValue() +} + +type LiftSeries_LiftValue_XAndYCount struct { + XAndYCount uint64 `protobuf:"varint,6,opt,name=x_and_y_count,json=xAndYCount,proto3,oneof"` +} + +type LiftSeries_LiftValue_WeightedXAndYCount struct { + WeightedXAndYCount float64 `protobuf:"fixed64,7,opt,name=weighted_x_and_y_count,json=weightedXAndYCount,proto3,oneof"` +} + +func (*LiftSeries_LiftValue_XAndYCount) isLiftSeries_LiftValue_XAndYCountValue() {} + +func (*LiftSeries_LiftValue_WeightedXAndYCount) isLiftSeries_LiftValue_XAndYCountValue() {} + +type StringStatistics_FreqAndValue struct { + state protoimpl.MessageState + sizeCache protoimpl.SizeCache + unknownFields protoimpl.UnknownFields + + Value string `protobuf:"bytes,2,opt,name=value,proto3" json:"value,omitempty"` + // The number of times the value occurs. Stored as a double to be able to + // handle weighted features. + Frequency float64 `protobuf:"fixed64,3,opt,name=frequency,proto3" json:"frequency,omitempty"` +} + +func (x *StringStatistics_FreqAndValue) Reset() { + *x = StringStatistics_FreqAndValue{} + if protoimpl.UnsafeEnabled { + mi := &file_tensorflow_metadata_proto_v0_statistics_proto_msgTypes[21] + ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x)) + ms.StoreMessageInfo(mi) + } +} + +func (x *StringStatistics_FreqAndValue) String() string { + return protoimpl.X.MessageStringOf(x) +} + +func (*StringStatistics_FreqAndValue) ProtoMessage() {} + +func (x *StringStatistics_FreqAndValue) ProtoReflect() protoreflect.Message { + mi := &file_tensorflow_metadata_proto_v0_statistics_proto_msgTypes[21] + if protoimpl.UnsafeEnabled && x != nil { + ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x)) + if ms.LoadMessageInfo() == nil { + ms.StoreMessageInfo(mi) + } + return ms + } + return mi.MessageOf(x) +} + +// Deprecated: Use StringStatistics_FreqAndValue.ProtoReflect.Descriptor instead. +func (*StringStatistics_FreqAndValue) Descriptor() ([]byte, []int) { + return file_tensorflow_metadata_proto_v0_statistics_proto_rawDescGZIP(), []int{11, 0} +} + +func (x *StringStatistics_FreqAndValue) GetValue() string { + if x != nil { + return x.Value + } + return "" +} + +func (x *StringStatistics_FreqAndValue) GetFrequency() float64 { + if x != nil { + return x.Frequency + } + return 0 +} + +// Each bucket defines its low and high values along with its count. The +// low and high values must be a real number or positive or negative +// infinity. They cannot be NaN or undefined. Counts of those special values +// can be found in the numNaN and numUndefined fields. +type Histogram_Bucket struct { + state protoimpl.MessageState + sizeCache protoimpl.SizeCache + unknownFields protoimpl.UnknownFields + + // The low value of the bucket, inclusive. + LowValue float64 `protobuf:"fixed64,1,opt,name=low_value,json=lowValue,proto3" json:"low_value,omitempty"` + // The high value of the bucket, exclusive (unless the highValue is + // positive infinity). + HighValue float64 `protobuf:"fixed64,2,opt,name=high_value,json=highValue,proto3" json:"high_value,omitempty"` + // The number of items in the bucket. Stored as a double to be able to + // handle weighted histograms. + SampleCount float64 `protobuf:"fixed64,4,opt,name=sample_count,json=sampleCount,proto3" json:"sample_count,omitempty"` +} + +func (x *Histogram_Bucket) Reset() { + *x = Histogram_Bucket{} + if protoimpl.UnsafeEnabled { + mi := &file_tensorflow_metadata_proto_v0_statistics_proto_msgTypes[22] + ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x)) + ms.StoreMessageInfo(mi) + } +} + +func (x *Histogram_Bucket) String() string { + return protoimpl.X.MessageStringOf(x) +} + +func (*Histogram_Bucket) ProtoMessage() {} + +func (x *Histogram_Bucket) ProtoReflect() protoreflect.Message { + mi := &file_tensorflow_metadata_proto_v0_statistics_proto_msgTypes[22] + if protoimpl.UnsafeEnabled && x != nil { + ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x)) + if ms.LoadMessageInfo() == nil { + ms.StoreMessageInfo(mi) + } + return ms + } + return mi.MessageOf(x) +} + +// Deprecated: Use Histogram_Bucket.ProtoReflect.Descriptor instead. +func (*Histogram_Bucket) Descriptor() ([]byte, []int) { + return file_tensorflow_metadata_proto_v0_statistics_proto_rawDescGZIP(), []int{17, 0} +} + +func (x *Histogram_Bucket) GetLowValue() float64 { + if x != nil { + return x.LowValue + } + return 0 +} + +func (x *Histogram_Bucket) GetHighValue() float64 { + if x != nil { + return x.HighValue + } + return 0 +} + +func (x *Histogram_Bucket) GetSampleCount() float64 { + if x != nil { + return x.SampleCount + } + return 0 +} + +// Each bucket defines its start and end ranks along with its count. +type RankHistogram_Bucket struct { + state protoimpl.MessageState + sizeCache protoimpl.SizeCache + unknownFields protoimpl.UnknownFields + + // The low rank of the bucket, inclusive. + LowRank uint64 `protobuf:"varint,1,opt,name=low_rank,json=lowRank,proto3" json:"low_rank,omitempty"` + // The high rank of the bucket, exclusive. + HighRank uint64 `protobuf:"varint,2,opt,name=high_rank,json=highRank,proto3" json:"high_rank,omitempty"` + // The label for the bucket. Can be used to list or summarize the values in + // this rank bucket. + Label string `protobuf:"bytes,4,opt,name=label,proto3" json:"label,omitempty"` + // The number of items in the bucket. Stored as a double to be able to + // handle weighted histograms. + SampleCount float64 `protobuf:"fixed64,5,opt,name=sample_count,json=sampleCount,proto3" json:"sample_count,omitempty"` +} + +func (x *RankHistogram_Bucket) Reset() { + *x = RankHistogram_Bucket{} + if protoimpl.UnsafeEnabled { + mi := &file_tensorflow_metadata_proto_v0_statistics_proto_msgTypes[23] + ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x)) + ms.StoreMessageInfo(mi) + } +} + +func (x *RankHistogram_Bucket) String() string { + return protoimpl.X.MessageStringOf(x) +} + +func (*RankHistogram_Bucket) ProtoMessage() {} + +func (x *RankHistogram_Bucket) ProtoReflect() protoreflect.Message { + mi := &file_tensorflow_metadata_proto_v0_statistics_proto_msgTypes[23] + if protoimpl.UnsafeEnabled && x != nil { + ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x)) + if ms.LoadMessageInfo() == nil { + ms.StoreMessageInfo(mi) + } + return ms + } + return mi.MessageOf(x) +} + +// Deprecated: Use RankHistogram_Bucket.ProtoReflect.Descriptor instead. +func (*RankHistogram_Bucket) Descriptor() ([]byte, []int) { + return file_tensorflow_metadata_proto_v0_statistics_proto_rawDescGZIP(), []int{18, 0} +} + +func (x *RankHistogram_Bucket) GetLowRank() uint64 { + if x != nil { + return x.LowRank + } + return 0 +} + +func (x *RankHistogram_Bucket) GetHighRank() uint64 { + if x != nil { + return x.HighRank + } + return 0 +} + +func (x *RankHistogram_Bucket) GetLabel() string { + if x != nil { + return x.Label + } + return "" +} + +func (x *RankHistogram_Bucket) GetSampleCount() float64 { + if x != nil { + return x.SampleCount + } + return 0 +} + +var File_tensorflow_metadata_proto_v0_statistics_proto protoreflect.FileDescriptor + +var file_tensorflow_metadata_proto_v0_statistics_proto_rawDesc = []byte{ + 0x0a, 0x2d, 0x74, 0x65, 0x6e, 0x73, 0x6f, 0x72, 0x66, 0x6c, 0x6f, 0x77, 0x5f, 0x6d, 0x65, 0x74, + 0x61, 0x64, 0x61, 0x74, 0x61, 0x2f, 0x70, 0x72, 0x6f, 0x74, 0x6f, 0x2f, 0x76, 0x30, 0x2f, 0x73, + 0x74, 0x61, 0x74, 0x69, 0x73, 0x74, 0x69, 0x63, 0x73, 0x2e, 0x70, 0x72, 0x6f, 0x74, 0x6f, 0x12, + 0x16, 0x74, 0x65, 0x6e, 0x73, 0x6f, 0x72, 0x66, 0x6c, 0x6f, 0x77, 0x2e, 0x6d, 0x65, 0x74, 0x61, + 0x64, 0x61, 0x74, 0x61, 0x2e, 0x76, 0x30, 0x1a, 0x27, 0x74, 0x65, 0x6e, 0x73, 0x6f, 0x72, 0x66, + 0x6c, 0x6f, 0x77, 0x5f, 0x6d, 0x65, 0x74, 0x61, 0x64, 0x61, 0x74, 0x61, 0x2f, 0x70, 0x72, 0x6f, + 0x74, 0x6f, 0x2f, 0x76, 0x30, 0x2f, 0x70, 0x61, 0x74, 0x68, 0x2e, 0x70, 0x72, 0x6f, 0x74, 0x6f, + 0x22, 0x6c, 0x0a, 0x1c, 0x44, 0x61, 0x74, 0x61, 0x73, 0x65, 0x74, 0x46, 0x65, 0x61, 0x74, 0x75, + 0x72, 0x65, 0x53, 0x74, 0x61, 0x74, 0x69, 0x73, 0x74, 0x69, 0x63, 0x73, 0x4c, 0x69, 0x73, 0x74, + 0x12, 0x4c, 0x0a, 0x08, 0x64, 0x61, 0x74, 0x61, 0x73, 0x65, 0x74, 0x73, 0x18, 0x01, 0x20, 0x03, + 0x28, 0x0b, 0x32, 0x30, 0x2e, 0x74, 0x65, 0x6e, 0x73, 0x6f, 0x72, 0x66, 0x6c, 0x6f, 0x77, 0x2e, + 0x6d, 0x65, 0x74, 0x61, 0x64, 0x61, 0x74, 0x61, 0x2e, 0x76, 0x30, 0x2e, 0x44, 0x61, 0x74, 0x61, + 0x73, 0x65, 0x74, 0x46, 0x65, 0x61, 0x74, 0x75, 0x72, 0x65, 0x53, 0x74, 0x61, 0x74, 0x69, 0x73, + 0x74, 0x69, 0x63, 0x73, 0x52, 0x08, 0x64, 0x61, 0x74, 0x61, 0x73, 0x65, 0x74, 0x73, 0x22, 0xa7, + 0x02, 0x0a, 0x18, 0x44, 0x61, 0x74, 0x61, 0x73, 0x65, 0x74, 0x46, 0x65, 0x61, 0x74, 0x75, 0x72, + 0x65, 0x53, 0x74, 0x61, 0x74, 0x69, 0x73, 0x74, 0x69, 0x63, 0x73, 0x12, 0x12, 0x0a, 0x04, 0x6e, + 0x61, 0x6d, 0x65, 0x18, 0x01, 0x20, 0x01, 0x28, 0x09, 0x52, 0x04, 0x6e, 0x61, 0x6d, 0x65, 0x12, + 0x21, 0x0a, 0x0c, 0x6e, 0x75, 0x6d, 0x5f, 0x65, 0x78, 0x61, 0x6d, 0x70, 0x6c, 0x65, 0x73, 0x18, + 0x02, 0x20, 0x01, 0x28, 0x04, 0x52, 0x0b, 0x6e, 0x75, 0x6d, 0x45, 0x78, 0x61, 0x6d, 0x70, 0x6c, + 0x65, 0x73, 0x12, 0x32, 0x0a, 0x15, 0x77, 0x65, 0x69, 0x67, 0x68, 0x74, 0x65, 0x64, 0x5f, 0x6e, + 0x75, 0x6d, 0x5f, 0x65, 0x78, 0x61, 0x6d, 0x70, 0x6c, 0x65, 0x73, 0x18, 0x04, 0x20, 0x01, 0x28, + 0x01, 0x52, 0x13, 0x77, 0x65, 0x69, 0x67, 0x68, 0x74, 0x65, 0x64, 0x4e, 0x75, 0x6d, 0x45, 0x78, + 0x61, 0x6d, 0x70, 0x6c, 0x65, 0x73, 0x12, 0x49, 0x0a, 0x08, 0x66, 0x65, 0x61, 0x74, 0x75, 0x72, + 0x65, 0x73, 0x18, 0x03, 0x20, 0x03, 0x28, 0x0b, 0x32, 0x2d, 0x2e, 0x74, 0x65, 0x6e, 0x73, 0x6f, + 0x72, 0x66, 0x6c, 0x6f, 0x77, 0x2e, 0x6d, 0x65, 0x74, 0x61, 0x64, 0x61, 0x74, 0x61, 0x2e, 0x76, + 0x30, 0x2e, 0x46, 0x65, 0x61, 0x74, 0x75, 0x72, 0x65, 0x4e, 0x61, 0x6d, 0x65, 0x53, 0x74, 0x61, + 0x74, 0x69, 0x73, 0x74, 0x69, 0x63, 0x73, 0x52, 0x08, 0x66, 0x65, 0x61, 0x74, 0x75, 0x72, 0x65, + 0x73, 0x12, 0x55, 0x0a, 0x0e, 0x63, 0x72, 0x6f, 0x73, 0x73, 0x5f, 0x66, 0x65, 0x61, 0x74, 0x75, + 0x72, 0x65, 0x73, 0x18, 0x05, 0x20, 0x03, 0x28, 0x0b, 0x32, 0x2e, 0x2e, 0x74, 0x65, 0x6e, 0x73, + 0x6f, 0x72, 0x66, 0x6c, 0x6f, 0x77, 0x2e, 0x6d, 0x65, 0x74, 0x61, 0x64, 0x61, 0x74, 0x61, 0x2e, + 0x76, 0x30, 0x2e, 0x43, 0x72, 0x6f, 0x73, 0x73, 0x46, 0x65, 0x61, 0x74, 0x75, 0x72, 0x65, 0x53, + 0x74, 0x61, 0x74, 0x69, 0x73, 0x74, 0x69, 0x63, 0x73, 0x52, 0x0d, 0x63, 0x72, 0x6f, 0x73, 0x73, + 0x46, 0x65, 0x61, 0x74, 0x75, 0x72, 0x65, 0x73, 0x22, 0xef, 0x02, 0x0a, 0x16, 0x43, 0x72, 0x6f, + 0x73, 0x73, 0x46, 0x65, 0x61, 0x74, 0x75, 0x72, 0x65, 0x53, 0x74, 0x61, 0x74, 0x69, 0x73, 0x74, + 0x69, 0x63, 0x73, 0x12, 0x33, 0x0a, 0x06, 0x70, 0x61, 0x74, 0x68, 0x5f, 0x78, 0x18, 0x01, 0x20, + 0x01, 0x28, 0x0b, 0x32, 0x1c, 0x2e, 0x74, 0x65, 0x6e, 0x73, 0x6f, 0x72, 0x66, 0x6c, 0x6f, 0x77, + 0x2e, 0x6d, 0x65, 0x74, 0x61, 0x64, 0x61, 0x74, 0x61, 0x2e, 0x76, 0x30, 0x2e, 0x50, 0x61, 0x74, + 0x68, 0x52, 0x05, 0x70, 0x61, 0x74, 0x68, 0x58, 0x12, 0x33, 0x0a, 0x06, 0x70, 0x61, 0x74, 0x68, + 0x5f, 0x79, 0x18, 0x02, 0x20, 0x01, 0x28, 0x0b, 0x32, 0x1c, 0x2e, 0x74, 0x65, 0x6e, 0x73, 0x6f, + 0x72, 0x66, 0x6c, 0x6f, 0x77, 0x2e, 0x6d, 0x65, 0x74, 0x61, 0x64, 0x61, 0x74, 0x61, 0x2e, 0x76, + 0x30, 0x2e, 0x50, 0x61, 0x74, 0x68, 0x52, 0x05, 0x70, 0x61, 0x74, 0x68, 0x59, 0x12, 0x14, 0x0a, + 0x05, 0x63, 0x6f, 0x75, 0x6e, 0x74, 0x18, 0x03, 0x20, 0x01, 0x28, 0x04, 0x52, 0x05, 0x63, 0x6f, + 0x75, 0x6e, 0x74, 0x12, 0x58, 0x0a, 0x0f, 0x6e, 0x75, 0x6d, 0x5f, 0x63, 0x72, 0x6f, 0x73, 0x73, + 0x5f, 0x73, 0x74, 0x61, 0x74, 0x73, 0x18, 0x04, 0x20, 0x01, 0x28, 0x0b, 0x32, 0x2e, 0x2e, 0x74, + 0x65, 0x6e, 0x73, 0x6f, 0x72, 0x66, 0x6c, 0x6f, 0x77, 0x2e, 0x6d, 0x65, 0x74, 0x61, 0x64, 0x61, + 0x74, 0x61, 0x2e, 0x76, 0x30, 0x2e, 0x4e, 0x75, 0x6d, 0x65, 0x72, 0x69, 0x63, 0x43, 0x72, 0x6f, + 0x73, 0x73, 0x53, 0x74, 0x61, 0x74, 0x69, 0x73, 0x74, 0x69, 0x63, 0x73, 0x48, 0x00, 0x52, 0x0d, + 0x6e, 0x75, 0x6d, 0x43, 0x72, 0x6f, 0x73, 0x73, 0x53, 0x74, 0x61, 0x74, 0x73, 0x12, 0x6c, 0x0a, + 0x17, 0x63, 0x61, 0x74, 0x65, 0x67, 0x6f, 0x72, 0x69, 0x63, 0x61, 0x6c, 0x5f, 0x63, 0x72, 0x6f, + 0x73, 0x73, 0x5f, 0x73, 0x74, 0x61, 0x74, 0x73, 0x18, 0x05, 0x20, 0x01, 0x28, 0x0b, 0x32, 0x32, + 0x2e, 0x74, 0x65, 0x6e, 0x73, 0x6f, 0x72, 0x66, 0x6c, 0x6f, 0x77, 0x2e, 0x6d, 0x65, 0x74, 0x61, + 0x64, 0x61, 0x74, 0x61, 0x2e, 0x76, 0x30, 0x2e, 0x43, 0x61, 0x74, 0x65, 0x67, 0x6f, 0x72, 0x69, + 0x63, 0x61, 0x6c, 0x43, 0x72, 0x6f, 0x73, 0x73, 0x53, 0x74, 0x61, 0x74, 0x69, 0x73, 0x74, 0x69, + 0x63, 0x73, 0x48, 0x00, 0x52, 0x15, 0x63, 0x61, 0x74, 0x65, 0x67, 0x6f, 0x72, 0x69, 0x63, 0x61, + 0x6c, 0x43, 0x72, 0x6f, 0x73, 0x73, 0x53, 0x74, 0x61, 0x74, 0x73, 0x42, 0x0d, 0x0a, 0x0b, 0x63, + 0x72, 0x6f, 0x73, 0x73, 0x5f, 0x73, 0x74, 0x61, 0x74, 0x73, 0x22, 0x5a, 0x0a, 0x16, 0x4e, 0x75, + 0x6d, 0x65, 0x72, 0x69, 0x63, 0x43, 0x72, 0x6f, 0x73, 0x73, 0x53, 0x74, 0x61, 0x74, 0x69, 0x73, + 0x74, 0x69, 0x63, 0x73, 0x12, 0x20, 0x0a, 0x0b, 0x63, 0x6f, 0x72, 0x72, 0x65, 0x6c, 0x61, 0x74, + 0x69, 0x6f, 0x6e, 0x18, 0x01, 0x20, 0x01, 0x28, 0x02, 0x52, 0x0b, 0x63, 0x6f, 0x72, 0x72, 0x65, + 0x6c, 0x61, 0x74, 0x69, 0x6f, 0x6e, 0x12, 0x1e, 0x0a, 0x0a, 0x63, 0x6f, 0x76, 0x61, 0x72, 0x69, + 0x61, 0x6e, 0x63, 0x65, 0x18, 0x02, 0x20, 0x01, 0x28, 0x02, 0x52, 0x0a, 0x63, 0x6f, 0x76, 0x61, + 0x72, 0x69, 0x61, 0x6e, 0x63, 0x65, 0x22, 0x58, 0x0a, 0x1a, 0x43, 0x61, 0x74, 0x65, 0x67, 0x6f, + 0x72, 0x69, 0x63, 0x61, 0x6c, 0x43, 0x72, 0x6f, 0x73, 0x73, 0x53, 0x74, 0x61, 0x74, 0x69, 0x73, + 0x74, 0x69, 0x63, 0x73, 0x12, 0x3a, 0x0a, 0x04, 0x6c, 0x69, 0x66, 0x74, 0x18, 0x01, 0x20, 0x01, + 0x28, 0x0b, 0x32, 0x26, 0x2e, 0x74, 0x65, 0x6e, 0x73, 0x6f, 0x72, 0x66, 0x6c, 0x6f, 0x77, 0x2e, + 0x6d, 0x65, 0x74, 0x61, 0x64, 0x61, 0x74, 0x61, 0x2e, 0x76, 0x30, 0x2e, 0x4c, 0x69, 0x66, 0x74, + 0x53, 0x74, 0x61, 0x74, 0x69, 0x73, 0x74, 0x69, 0x63, 0x73, 0x52, 0x04, 0x6c, 0x69, 0x66, 0x74, + 0x22, 0xab, 0x01, 0x0a, 0x0e, 0x4c, 0x69, 0x66, 0x74, 0x53, 0x74, 0x61, 0x74, 0x69, 0x73, 0x74, + 0x69, 0x63, 0x73, 0x12, 0x43, 0x0a, 0x0b, 0x6c, 0x69, 0x66, 0x74, 0x5f, 0x73, 0x65, 0x72, 0x69, + 0x65, 0x73, 0x18, 0x01, 0x20, 0x03, 0x28, 0x0b, 0x32, 0x22, 0x2e, 0x74, 0x65, 0x6e, 0x73, 0x6f, + 0x72, 0x66, 0x6c, 0x6f, 0x77, 0x2e, 0x6d, 0x65, 0x74, 0x61, 0x64, 0x61, 0x74, 0x61, 0x2e, 0x76, + 0x30, 0x2e, 0x4c, 0x69, 0x66, 0x74, 0x53, 0x65, 0x72, 0x69, 0x65, 0x73, 0x52, 0x0a, 0x6c, 0x69, + 0x66, 0x74, 0x53, 0x65, 0x72, 0x69, 0x65, 0x73, 0x12, 0x54, 0x0a, 0x14, 0x77, 0x65, 0x69, 0x67, + 0x68, 0x74, 0x65, 0x64, 0x5f, 0x6c, 0x69, 0x66, 0x74, 0x5f, 0x73, 0x65, 0x72, 0x69, 0x65, 0x73, + 0x18, 0x02, 0x20, 0x03, 0x28, 0x0b, 0x32, 0x22, 0x2e, 0x74, 0x65, 0x6e, 0x73, 0x6f, 0x72, 0x66, + 0x6c, 0x6f, 0x77, 0x2e, 0x6d, 0x65, 0x74, 0x61, 0x64, 0x61, 0x74, 0x61, 0x2e, 0x76, 0x30, 0x2e, + 0x4c, 0x69, 0x66, 0x74, 0x53, 0x65, 0x72, 0x69, 0x65, 0x73, 0x52, 0x12, 0x77, 0x65, 0x69, 0x67, + 0x68, 0x74, 0x65, 0x64, 0x4c, 0x69, 0x66, 0x74, 0x53, 0x65, 0x72, 0x69, 0x65, 0x73, 0x22, 0xab, + 0x05, 0x0a, 0x0a, 0x4c, 0x69, 0x66, 0x74, 0x53, 0x65, 0x72, 0x69, 0x65, 0x73, 0x12, 0x15, 0x0a, + 0x05, 0x79, 0x5f, 0x69, 0x6e, 0x74, 0x18, 0x01, 0x20, 0x01, 0x28, 0x05, 0x48, 0x00, 0x52, 0x04, + 0x79, 0x49, 0x6e, 0x74, 0x12, 0x1b, 0x0a, 0x08, 0x79, 0x5f, 0x73, 0x74, 0x72, 0x69, 0x6e, 0x67, + 0x18, 0x02, 0x20, 0x01, 0x28, 0x09, 0x48, 0x00, 0x52, 0x07, 0x79, 0x53, 0x74, 0x72, 0x69, 0x6e, + 0x67, 0x12, 0x46, 0x0a, 0x08, 0x79, 0x5f, 0x62, 0x75, 0x63, 0x6b, 0x65, 0x74, 0x18, 0x03, 0x20, + 0x01, 0x28, 0x0b, 0x32, 0x29, 0x2e, 0x74, 0x65, 0x6e, 0x73, 0x6f, 0x72, 0x66, 0x6c, 0x6f, 0x77, + 0x2e, 0x6d, 0x65, 0x74, 0x61, 0x64, 0x61, 0x74, 0x61, 0x2e, 0x76, 0x30, 0x2e, 0x4c, 0x69, 0x66, + 0x74, 0x53, 0x65, 0x72, 0x69, 0x65, 0x73, 0x2e, 0x42, 0x75, 0x63, 0x6b, 0x65, 0x74, 0x48, 0x00, + 0x52, 0x07, 0x79, 0x42, 0x75, 0x63, 0x6b, 0x65, 0x74, 0x12, 0x19, 0x0a, 0x07, 0x79, 0x5f, 0x63, + 0x6f, 0x75, 0x6e, 0x74, 0x18, 0x04, 0x20, 0x01, 0x28, 0x04, 0x48, 0x01, 0x52, 0x06, 0x79, 0x43, + 0x6f, 0x75, 0x6e, 0x74, 0x12, 0x2a, 0x0a, 0x10, 0x77, 0x65, 0x69, 0x67, 0x68, 0x74, 0x65, 0x64, + 0x5f, 0x79, 0x5f, 0x63, 0x6f, 0x75, 0x6e, 0x74, 0x18, 0x05, 0x20, 0x01, 0x28, 0x01, 0x48, 0x01, + 0x52, 0x0e, 0x77, 0x65, 0x69, 0x67, 0x68, 0x74, 0x65, 0x64, 0x59, 0x43, 0x6f, 0x75, 0x6e, 0x74, + 0x12, 0x4d, 0x0a, 0x0b, 0x6c, 0x69, 0x66, 0x74, 0x5f, 0x76, 0x61, 0x6c, 0x75, 0x65, 0x73, 0x18, + 0x06, 0x20, 0x03, 0x28, 0x0b, 0x32, 0x2c, 0x2e, 0x74, 0x65, 0x6e, 0x73, 0x6f, 0x72, 0x66, 0x6c, + 0x6f, 0x77, 0x2e, 0x6d, 0x65, 0x74, 0x61, 0x64, 0x61, 0x74, 0x61, 0x2e, 0x76, 0x30, 0x2e, 0x4c, + 0x69, 0x66, 0x74, 0x53, 0x65, 0x72, 0x69, 0x65, 0x73, 0x2e, 0x4c, 0x69, 0x66, 0x74, 0x56, 0x61, + 0x6c, 0x75, 0x65, 0x52, 0x0a, 0x6c, 0x69, 0x66, 0x74, 0x56, 0x61, 0x6c, 0x75, 0x65, 0x73, 0x1a, + 0x44, 0x0a, 0x06, 0x42, 0x75, 0x63, 0x6b, 0x65, 0x74, 0x12, 0x1b, 0x0a, 0x09, 0x6c, 0x6f, 0x77, + 0x5f, 0x76, 0x61, 0x6c, 0x75, 0x65, 0x18, 0x01, 0x20, 0x01, 0x28, 0x01, 0x52, 0x08, 0x6c, 0x6f, + 0x77, 0x56, 0x61, 0x6c, 0x75, 0x65, 0x12, 0x1d, 0x0a, 0x0a, 0x68, 0x69, 0x67, 0x68, 0x5f, 0x76, + 0x61, 0x6c, 0x75, 0x65, 0x18, 0x02, 0x20, 0x01, 0x28, 0x01, 0x52, 0x09, 0x68, 0x69, 0x67, 0x68, + 0x56, 0x61, 0x6c, 0x75, 0x65, 0x1a, 0xa8, 0x02, 0x0a, 0x09, 0x4c, 0x69, 0x66, 0x74, 0x56, 0x61, + 0x6c, 0x75, 0x65, 0x12, 0x15, 0x0a, 0x05, 0x78, 0x5f, 0x69, 0x6e, 0x74, 0x18, 0x01, 0x20, 0x01, + 0x28, 0x05, 0x48, 0x00, 0x52, 0x04, 0x78, 0x49, 0x6e, 0x74, 0x12, 0x1b, 0x0a, 0x08, 0x78, 0x5f, + 0x73, 0x74, 0x72, 0x69, 0x6e, 0x67, 0x18, 0x02, 0x20, 0x01, 0x28, 0x09, 0x48, 0x00, 0x52, 0x07, + 0x78, 0x53, 0x74, 0x72, 0x69, 0x6e, 0x67, 0x12, 0x12, 0x0a, 0x04, 0x6c, 0x69, 0x66, 0x74, 0x18, + 0x03, 0x20, 0x01, 0x28, 0x01, 0x52, 0x04, 0x6c, 0x69, 0x66, 0x74, 0x12, 0x19, 0x0a, 0x07, 0x78, + 0x5f, 0x63, 0x6f, 0x75, 0x6e, 0x74, 0x18, 0x04, 0x20, 0x01, 0x28, 0x04, 0x48, 0x01, 0x52, 0x06, + 0x78, 0x43, 0x6f, 0x75, 0x6e, 0x74, 0x12, 0x2a, 0x0a, 0x10, 0x77, 0x65, 0x69, 0x67, 0x68, 0x74, + 0x65, 0x64, 0x5f, 0x78, 0x5f, 0x63, 0x6f, 0x75, 0x6e, 0x74, 0x18, 0x05, 0x20, 0x01, 0x28, 0x01, + 0x48, 0x01, 0x52, 0x0e, 0x77, 0x65, 0x69, 0x67, 0x68, 0x74, 0x65, 0x64, 0x58, 0x43, 0x6f, 0x75, + 0x6e, 0x74, 0x12, 0x23, 0x0a, 0x0d, 0x78, 0x5f, 0x61, 0x6e, 0x64, 0x5f, 0x79, 0x5f, 0x63, 0x6f, + 0x75, 0x6e, 0x74, 0x18, 0x06, 0x20, 0x01, 0x28, 0x04, 0x48, 0x02, 0x52, 0x0a, 0x78, 0x41, 0x6e, + 0x64, 0x59, 0x43, 0x6f, 0x75, 0x6e, 0x74, 0x12, 0x34, 0x0a, 0x16, 0x77, 0x65, 0x69, 0x67, 0x68, + 0x74, 0x65, 0x64, 0x5f, 0x78, 0x5f, 0x61, 0x6e, 0x64, 0x5f, 0x79, 0x5f, 0x63, 0x6f, 0x75, 0x6e, + 0x74, 0x18, 0x07, 0x20, 0x01, 0x28, 0x01, 0x48, 0x02, 0x52, 0x12, 0x77, 0x65, 0x69, 0x67, 0x68, + 0x74, 0x65, 0x64, 0x58, 0x41, 0x6e, 0x64, 0x59, 0x43, 0x6f, 0x75, 0x6e, 0x74, 0x42, 0x09, 0x0a, + 0x07, 0x78, 0x5f, 0x76, 0x61, 0x6c, 0x75, 0x65, 0x42, 0x0f, 0x0a, 0x0d, 0x78, 0x5f, 0x63, 0x6f, + 0x75, 0x6e, 0x74, 0x5f, 0x76, 0x61, 0x6c, 0x75, 0x65, 0x42, 0x15, 0x0a, 0x13, 0x78, 0x5f, 0x61, + 0x6e, 0x64, 0x5f, 0x79, 0x5f, 0x63, 0x6f, 0x75, 0x6e, 0x74, 0x5f, 0x76, 0x61, 0x6c, 0x75, 0x65, + 0x42, 0x09, 0x0a, 0x07, 0x79, 0x5f, 0x76, 0x61, 0x6c, 0x75, 0x65, 0x42, 0x0f, 0x0a, 0x0d, 0x79, + 0x5f, 0x63, 0x6f, 0x75, 0x6e, 0x74, 0x5f, 0x76, 0x61, 0x6c, 0x75, 0x65, 0x22, 0xfd, 0x04, 0x0a, + 0x15, 0x46, 0x65, 0x61, 0x74, 0x75, 0x72, 0x65, 0x4e, 0x61, 0x6d, 0x65, 0x53, 0x74, 0x61, 0x74, + 0x69, 0x73, 0x74, 0x69, 0x63, 0x73, 0x12, 0x14, 0x0a, 0x04, 0x6e, 0x61, 0x6d, 0x65, 0x18, 0x01, + 0x20, 0x01, 0x28, 0x09, 0x48, 0x00, 0x52, 0x04, 0x6e, 0x61, 0x6d, 0x65, 0x12, 0x32, 0x0a, 0x04, + 0x70, 0x61, 0x74, 0x68, 0x18, 0x08, 0x20, 0x01, 0x28, 0x0b, 0x32, 0x1c, 0x2e, 0x74, 0x65, 0x6e, + 0x73, 0x6f, 0x72, 0x66, 0x6c, 0x6f, 0x77, 0x2e, 0x6d, 0x65, 0x74, 0x61, 0x64, 0x61, 0x74, 0x61, + 0x2e, 0x76, 0x30, 0x2e, 0x50, 0x61, 0x74, 0x68, 0x48, 0x00, 0x52, 0x04, 0x70, 0x61, 0x74, 0x68, + 0x12, 0x46, 0x0a, 0x04, 0x74, 0x79, 0x70, 0x65, 0x18, 0x02, 0x20, 0x01, 0x28, 0x0e, 0x32, 0x32, + 0x2e, 0x74, 0x65, 0x6e, 0x73, 0x6f, 0x72, 0x66, 0x6c, 0x6f, 0x77, 0x2e, 0x6d, 0x65, 0x74, 0x61, + 0x64, 0x61, 0x74, 0x61, 0x2e, 0x76, 0x30, 0x2e, 0x46, 0x65, 0x61, 0x74, 0x75, 0x72, 0x65, 0x4e, + 0x61, 0x6d, 0x65, 0x53, 0x74, 0x61, 0x74, 0x69, 0x73, 0x74, 0x69, 0x63, 0x73, 0x2e, 0x54, 0x79, + 0x70, 0x65, 0x52, 0x04, 0x74, 0x79, 0x70, 0x65, 0x12, 0x48, 0x0a, 0x09, 0x6e, 0x75, 0x6d, 0x5f, + 0x73, 0x74, 0x61, 0x74, 0x73, 0x18, 0x03, 0x20, 0x01, 0x28, 0x0b, 0x32, 0x29, 0x2e, 0x74, 0x65, + 0x6e, 0x73, 0x6f, 0x72, 0x66, 0x6c, 0x6f, 0x77, 0x2e, 0x6d, 0x65, 0x74, 0x61, 0x64, 0x61, 0x74, + 0x61, 0x2e, 0x76, 0x30, 0x2e, 0x4e, 0x75, 0x6d, 0x65, 0x72, 0x69, 0x63, 0x53, 0x74, 0x61, 0x74, + 0x69, 0x73, 0x74, 0x69, 0x63, 0x73, 0x48, 0x01, 0x52, 0x08, 0x6e, 0x75, 0x6d, 0x53, 0x74, 0x61, + 0x74, 0x73, 0x12, 0x4d, 0x0a, 0x0c, 0x73, 0x74, 0x72, 0x69, 0x6e, 0x67, 0x5f, 0x73, 0x74, 0x61, + 0x74, 0x73, 0x18, 0x04, 0x20, 0x01, 0x28, 0x0b, 0x32, 0x28, 0x2e, 0x74, 0x65, 0x6e, 0x73, 0x6f, + 0x72, 0x66, 0x6c, 0x6f, 0x77, 0x2e, 0x6d, 0x65, 0x74, 0x61, 0x64, 0x61, 0x74, 0x61, 0x2e, 0x76, + 0x30, 0x2e, 0x53, 0x74, 0x72, 0x69, 0x6e, 0x67, 0x53, 0x74, 0x61, 0x74, 0x69, 0x73, 0x74, 0x69, + 0x63, 0x73, 0x48, 0x01, 0x52, 0x0b, 0x73, 0x74, 0x72, 0x69, 0x6e, 0x67, 0x53, 0x74, 0x61, 0x74, + 0x73, 0x12, 0x4a, 0x0a, 0x0b, 0x62, 0x79, 0x74, 0x65, 0x73, 0x5f, 0x73, 0x74, 0x61, 0x74, 0x73, + 0x18, 0x05, 0x20, 0x01, 0x28, 0x0b, 0x32, 0x27, 0x2e, 0x74, 0x65, 0x6e, 0x73, 0x6f, 0x72, 0x66, + 0x6c, 0x6f, 0x77, 0x2e, 0x6d, 0x65, 0x74, 0x61, 0x64, 0x61, 0x74, 0x61, 0x2e, 0x76, 0x30, 0x2e, + 0x42, 0x79, 0x74, 0x65, 0x73, 0x53, 0x74, 0x61, 0x74, 0x69, 0x73, 0x74, 0x69, 0x63, 0x73, 0x48, + 0x01, 0x52, 0x0a, 0x62, 0x79, 0x74, 0x65, 0x73, 0x53, 0x74, 0x61, 0x74, 0x73, 0x12, 0x4d, 0x0a, + 0x0c, 0x73, 0x74, 0x72, 0x75, 0x63, 0x74, 0x5f, 0x73, 0x74, 0x61, 0x74, 0x73, 0x18, 0x07, 0x20, + 0x01, 0x28, 0x0b, 0x32, 0x28, 0x2e, 0x74, 0x65, 0x6e, 0x73, 0x6f, 0x72, 0x66, 0x6c, 0x6f, 0x77, + 0x2e, 0x6d, 0x65, 0x74, 0x61, 0x64, 0x61, 0x74, 0x61, 0x2e, 0x76, 0x30, 0x2e, 0x53, 0x74, 0x72, + 0x75, 0x63, 0x74, 0x53, 0x74, 0x61, 0x74, 0x69, 0x73, 0x74, 0x69, 0x63, 0x73, 0x48, 0x01, 0x52, + 0x0b, 0x73, 0x74, 0x72, 0x75, 0x63, 0x74, 0x53, 0x74, 0x61, 0x74, 0x73, 0x12, 0x4a, 0x0a, 0x0c, + 0x63, 0x75, 0x73, 0x74, 0x6f, 0x6d, 0x5f, 0x73, 0x74, 0x61, 0x74, 0x73, 0x18, 0x06, 0x20, 0x03, + 0x28, 0x0b, 0x32, 0x27, 0x2e, 0x74, 0x65, 0x6e, 0x73, 0x6f, 0x72, 0x66, 0x6c, 0x6f, 0x77, 0x2e, + 0x6d, 0x65, 0x74, 0x61, 0x64, 0x61, 0x74, 0x61, 0x2e, 0x76, 0x30, 0x2e, 0x43, 0x75, 0x73, 0x74, + 0x6f, 0x6d, 0x53, 0x74, 0x61, 0x74, 0x69, 0x73, 0x74, 0x69, 0x63, 0x52, 0x0b, 0x63, 0x75, 0x73, + 0x74, 0x6f, 0x6d, 0x53, 0x74, 0x61, 0x74, 0x73, 0x22, 0x3d, 0x0a, 0x04, 0x54, 0x79, 0x70, 0x65, + 0x12, 0x07, 0x0a, 0x03, 0x49, 0x4e, 0x54, 0x10, 0x00, 0x12, 0x09, 0x0a, 0x05, 0x46, 0x4c, 0x4f, + 0x41, 0x54, 0x10, 0x01, 0x12, 0x0a, 0x0a, 0x06, 0x53, 0x54, 0x52, 0x49, 0x4e, 0x47, 0x10, 0x02, + 0x12, 0x09, 0x0a, 0x05, 0x42, 0x59, 0x54, 0x45, 0x53, 0x10, 0x03, 0x12, 0x0a, 0x0a, 0x06, 0x53, + 0x54, 0x52, 0x55, 0x43, 0x54, 0x10, 0x04, 0x42, 0x0a, 0x0a, 0x08, 0x66, 0x69, 0x65, 0x6c, 0x64, + 0x5f, 0x69, 0x64, 0x42, 0x07, 0x0a, 0x05, 0x73, 0x74, 0x61, 0x74, 0x73, 0x22, 0xaf, 0x01, 0x0a, + 0x18, 0x57, 0x65, 0x69, 0x67, 0x68, 0x74, 0x65, 0x64, 0x43, 0x6f, 0x6d, 0x6d, 0x6f, 0x6e, 0x53, + 0x74, 0x61, 0x74, 0x69, 0x73, 0x74, 0x69, 0x63, 0x73, 0x12, 0x26, 0x0a, 0x0f, 0x6e, 0x75, 0x6d, + 0x5f, 0x6e, 0x6f, 0x6e, 0x5f, 0x6d, 0x69, 0x73, 0x73, 0x69, 0x6e, 0x67, 0x18, 0x01, 0x20, 0x01, + 0x28, 0x01, 0x52, 0x0d, 0x6e, 0x75, 0x6d, 0x4e, 0x6f, 0x6e, 0x4d, 0x69, 0x73, 0x73, 0x69, 0x6e, + 0x67, 0x12, 0x1f, 0x0a, 0x0b, 0x6e, 0x75, 0x6d, 0x5f, 0x6d, 0x69, 0x73, 0x73, 0x69, 0x6e, 0x67, + 0x18, 0x02, 0x20, 0x01, 0x28, 0x01, 0x52, 0x0a, 0x6e, 0x75, 0x6d, 0x4d, 0x69, 0x73, 0x73, 0x69, + 0x6e, 0x67, 0x12, 0x24, 0x0a, 0x0e, 0x61, 0x76, 0x67, 0x5f, 0x6e, 0x75, 0x6d, 0x5f, 0x76, 0x61, + 0x6c, 0x75, 0x65, 0x73, 0x18, 0x03, 0x20, 0x01, 0x28, 0x01, 0x52, 0x0c, 0x61, 0x76, 0x67, 0x4e, + 0x75, 0x6d, 0x56, 0x61, 0x6c, 0x75, 0x65, 0x73, 0x12, 0x24, 0x0a, 0x0e, 0x74, 0x6f, 0x74, 0x5f, + 0x6e, 0x75, 0x6d, 0x5f, 0x76, 0x61, 0x6c, 0x75, 0x65, 0x73, 0x18, 0x04, 0x20, 0x01, 0x28, 0x01, + 0x52, 0x0c, 0x74, 0x6f, 0x74, 0x4e, 0x75, 0x6d, 0x56, 0x61, 0x6c, 0x75, 0x65, 0x73, 0x22, 0xe7, + 0x01, 0x0a, 0x0f, 0x43, 0x75, 0x73, 0x74, 0x6f, 0x6d, 0x53, 0x74, 0x61, 0x74, 0x69, 0x73, 0x74, + 0x69, 0x63, 0x12, 0x12, 0x0a, 0x04, 0x6e, 0x61, 0x6d, 0x65, 0x18, 0x01, 0x20, 0x01, 0x28, 0x09, + 0x52, 0x04, 0x6e, 0x61, 0x6d, 0x65, 0x12, 0x12, 0x0a, 0x03, 0x6e, 0x75, 0x6d, 0x18, 0x02, 0x20, + 0x01, 0x28, 0x01, 0x48, 0x00, 0x52, 0x03, 0x6e, 0x75, 0x6d, 0x12, 0x12, 0x0a, 0x03, 0x73, 0x74, + 0x72, 0x18, 0x03, 0x20, 0x01, 0x28, 0x09, 0x48, 0x00, 0x52, 0x03, 0x73, 0x74, 0x72, 0x12, 0x41, + 0x0a, 0x09, 0x68, 0x69, 0x73, 0x74, 0x6f, 0x67, 0x72, 0x61, 0x6d, 0x18, 0x04, 0x20, 0x01, 0x28, + 0x0b, 0x32, 0x21, 0x2e, 0x74, 0x65, 0x6e, 0x73, 0x6f, 0x72, 0x66, 0x6c, 0x6f, 0x77, 0x2e, 0x6d, + 0x65, 0x74, 0x61, 0x64, 0x61, 0x74, 0x61, 0x2e, 0x76, 0x30, 0x2e, 0x48, 0x69, 0x73, 0x74, 0x6f, + 0x67, 0x72, 0x61, 0x6d, 0x48, 0x00, 0x52, 0x09, 0x68, 0x69, 0x73, 0x74, 0x6f, 0x67, 0x72, 0x61, + 0x6d, 0x12, 0x4e, 0x0a, 0x0e, 0x72, 0x61, 0x6e, 0x6b, 0x5f, 0x68, 0x69, 0x73, 0x74, 0x6f, 0x67, + 0x72, 0x61, 0x6d, 0x18, 0x05, 0x20, 0x01, 0x28, 0x0b, 0x32, 0x25, 0x2e, 0x74, 0x65, 0x6e, 0x73, + 0x6f, 0x72, 0x66, 0x6c, 0x6f, 0x77, 0x2e, 0x6d, 0x65, 0x74, 0x61, 0x64, 0x61, 0x74, 0x61, 0x2e, + 0x76, 0x30, 0x2e, 0x52, 0x61, 0x6e, 0x6b, 0x48, 0x69, 0x73, 0x74, 0x6f, 0x67, 0x72, 0x61, 0x6d, + 0x48, 0x00, 0x52, 0x0d, 0x72, 0x61, 0x6e, 0x6b, 0x48, 0x69, 0x73, 0x74, 0x6f, 0x67, 0x72, 0x61, + 0x6d, 0x42, 0x05, 0x0a, 0x03, 0x76, 0x61, 0x6c, 0x22, 0x92, 0x03, 0x0a, 0x11, 0x4e, 0x75, 0x6d, + 0x65, 0x72, 0x69, 0x63, 0x53, 0x74, 0x61, 0x74, 0x69, 0x73, 0x74, 0x69, 0x63, 0x73, 0x12, 0x4b, + 0x0a, 0x0c, 0x63, 0x6f, 0x6d, 0x6d, 0x6f, 0x6e, 0x5f, 0x73, 0x74, 0x61, 0x74, 0x73, 0x18, 0x01, + 0x20, 0x01, 0x28, 0x0b, 0x32, 0x28, 0x2e, 0x74, 0x65, 0x6e, 0x73, 0x6f, 0x72, 0x66, 0x6c, 0x6f, + 0x77, 0x2e, 0x6d, 0x65, 0x74, 0x61, 0x64, 0x61, 0x74, 0x61, 0x2e, 0x76, 0x30, 0x2e, 0x43, 0x6f, + 0x6d, 0x6d, 0x6f, 0x6e, 0x53, 0x74, 0x61, 0x74, 0x69, 0x73, 0x74, 0x69, 0x63, 0x73, 0x52, 0x0b, + 0x63, 0x6f, 0x6d, 0x6d, 0x6f, 0x6e, 0x53, 0x74, 0x61, 0x74, 0x73, 0x12, 0x12, 0x0a, 0x04, 0x6d, + 0x65, 0x61, 0x6e, 0x18, 0x02, 0x20, 0x01, 0x28, 0x01, 0x52, 0x04, 0x6d, 0x65, 0x61, 0x6e, 0x12, + 0x17, 0x0a, 0x07, 0x73, 0x74, 0x64, 0x5f, 0x64, 0x65, 0x76, 0x18, 0x03, 0x20, 0x01, 0x28, 0x01, + 0x52, 0x06, 0x73, 0x74, 0x64, 0x44, 0x65, 0x76, 0x12, 0x1b, 0x0a, 0x09, 0x6e, 0x75, 0x6d, 0x5f, + 0x7a, 0x65, 0x72, 0x6f, 0x73, 0x18, 0x04, 0x20, 0x01, 0x28, 0x04, 0x52, 0x08, 0x6e, 0x75, 0x6d, + 0x5a, 0x65, 0x72, 0x6f, 0x73, 0x12, 0x10, 0x0a, 0x03, 0x6d, 0x69, 0x6e, 0x18, 0x05, 0x20, 0x01, + 0x28, 0x01, 0x52, 0x03, 0x6d, 0x69, 0x6e, 0x12, 0x16, 0x0a, 0x06, 0x6d, 0x65, 0x64, 0x69, 0x61, + 0x6e, 0x18, 0x06, 0x20, 0x01, 0x28, 0x01, 0x52, 0x06, 0x6d, 0x65, 0x64, 0x69, 0x61, 0x6e, 0x12, + 0x10, 0x0a, 0x03, 0x6d, 0x61, 0x78, 0x18, 0x07, 0x20, 0x01, 0x28, 0x01, 0x52, 0x03, 0x6d, 0x61, + 0x78, 0x12, 0x41, 0x0a, 0x0a, 0x68, 0x69, 0x73, 0x74, 0x6f, 0x67, 0x72, 0x61, 0x6d, 0x73, 0x18, + 0x08, 0x20, 0x03, 0x28, 0x0b, 0x32, 0x21, 0x2e, 0x74, 0x65, 0x6e, 0x73, 0x6f, 0x72, 0x66, 0x6c, + 0x6f, 0x77, 0x2e, 0x6d, 0x65, 0x74, 0x61, 0x64, 0x61, 0x74, 0x61, 0x2e, 0x76, 0x30, 0x2e, 0x48, + 0x69, 0x73, 0x74, 0x6f, 0x67, 0x72, 0x61, 0x6d, 0x52, 0x0a, 0x68, 0x69, 0x73, 0x74, 0x6f, 0x67, + 0x72, 0x61, 0x6d, 0x73, 0x12, 0x67, 0x0a, 0x16, 0x77, 0x65, 0x69, 0x67, 0x68, 0x74, 0x65, 0x64, + 0x5f, 0x6e, 0x75, 0x6d, 0x65, 0x72, 0x69, 0x63, 0x5f, 0x73, 0x74, 0x61, 0x74, 0x73, 0x18, 0x09, + 0x20, 0x01, 0x28, 0x0b, 0x32, 0x31, 0x2e, 0x74, 0x65, 0x6e, 0x73, 0x6f, 0x72, 0x66, 0x6c, 0x6f, + 0x77, 0x2e, 0x6d, 0x65, 0x74, 0x61, 0x64, 0x61, 0x74, 0x61, 0x2e, 0x76, 0x30, 0x2e, 0x57, 0x65, + 0x69, 0x67, 0x68, 0x74, 0x65, 0x64, 0x4e, 0x75, 0x6d, 0x65, 0x72, 0x69, 0x63, 0x53, 0x74, 0x61, + 0x74, 0x69, 0x73, 0x74, 0x69, 0x63, 0x73, 0x52, 0x14, 0x77, 0x65, 0x69, 0x67, 0x68, 0x74, 0x65, + 0x64, 0x4e, 0x75, 0x6d, 0x65, 0x72, 0x69, 0x63, 0x53, 0x74, 0x61, 0x74, 0x73, 0x22, 0x93, 0x04, + 0x0a, 0x10, 0x53, 0x74, 0x72, 0x69, 0x6e, 0x67, 0x53, 0x74, 0x61, 0x74, 0x69, 0x73, 0x74, 0x69, + 0x63, 0x73, 0x12, 0x4b, 0x0a, 0x0c, 0x63, 0x6f, 0x6d, 0x6d, 0x6f, 0x6e, 0x5f, 0x73, 0x74, 0x61, + 0x74, 0x73, 0x18, 0x01, 0x20, 0x01, 0x28, 0x0b, 0x32, 0x28, 0x2e, 0x74, 0x65, 0x6e, 0x73, 0x6f, + 0x72, 0x66, 0x6c, 0x6f, 0x77, 0x2e, 0x6d, 0x65, 0x74, 0x61, 0x64, 0x61, 0x74, 0x61, 0x2e, 0x76, + 0x30, 0x2e, 0x43, 0x6f, 0x6d, 0x6d, 0x6f, 0x6e, 0x53, 0x74, 0x61, 0x74, 0x69, 0x73, 0x74, 0x69, + 0x63, 0x73, 0x52, 0x0b, 0x63, 0x6f, 0x6d, 0x6d, 0x6f, 0x6e, 0x53, 0x74, 0x61, 0x74, 0x73, 0x12, + 0x16, 0x0a, 0x06, 0x75, 0x6e, 0x69, 0x71, 0x75, 0x65, 0x18, 0x02, 0x20, 0x01, 0x28, 0x04, 0x52, + 0x06, 0x75, 0x6e, 0x69, 0x71, 0x75, 0x65, 0x12, 0x54, 0x0a, 0x0a, 0x74, 0x6f, 0x70, 0x5f, 0x76, + 0x61, 0x6c, 0x75, 0x65, 0x73, 0x18, 0x03, 0x20, 0x03, 0x28, 0x0b, 0x32, 0x35, 0x2e, 0x74, 0x65, + 0x6e, 0x73, 0x6f, 0x72, 0x66, 0x6c, 0x6f, 0x77, 0x2e, 0x6d, 0x65, 0x74, 0x61, 0x64, 0x61, 0x74, + 0x61, 0x2e, 0x76, 0x30, 0x2e, 0x53, 0x74, 0x72, 0x69, 0x6e, 0x67, 0x53, 0x74, 0x61, 0x74, 0x69, + 0x73, 0x74, 0x69, 0x63, 0x73, 0x2e, 0x46, 0x72, 0x65, 0x71, 0x41, 0x6e, 0x64, 0x56, 0x61, 0x6c, + 0x75, 0x65, 0x52, 0x09, 0x74, 0x6f, 0x70, 0x56, 0x61, 0x6c, 0x75, 0x65, 0x73, 0x12, 0x1d, 0x0a, + 0x0a, 0x61, 0x76, 0x67, 0x5f, 0x6c, 0x65, 0x6e, 0x67, 0x74, 0x68, 0x18, 0x04, 0x20, 0x01, 0x28, + 0x02, 0x52, 0x09, 0x61, 0x76, 0x67, 0x4c, 0x65, 0x6e, 0x67, 0x74, 0x68, 0x12, 0x4c, 0x0a, 0x0e, + 0x72, 0x61, 0x6e, 0x6b, 0x5f, 0x68, 0x69, 0x73, 0x74, 0x6f, 0x67, 0x72, 0x61, 0x6d, 0x18, 0x05, + 0x20, 0x01, 0x28, 0x0b, 0x32, 0x25, 0x2e, 0x74, 0x65, 0x6e, 0x73, 0x6f, 0x72, 0x66, 0x6c, 0x6f, + 0x77, 0x2e, 0x6d, 0x65, 0x74, 0x61, 0x64, 0x61, 0x74, 0x61, 0x2e, 0x76, 0x30, 0x2e, 0x52, 0x61, + 0x6e, 0x6b, 0x48, 0x69, 0x73, 0x74, 0x6f, 0x67, 0x72, 0x61, 0x6d, 0x52, 0x0d, 0x72, 0x61, 0x6e, + 0x6b, 0x48, 0x69, 0x73, 0x74, 0x6f, 0x67, 0x72, 0x61, 0x6d, 0x12, 0x64, 0x0a, 0x15, 0x77, 0x65, + 0x69, 0x67, 0x68, 0x74, 0x65, 0x64, 0x5f, 0x73, 0x74, 0x72, 0x69, 0x6e, 0x67, 0x5f, 0x73, 0x74, + 0x61, 0x74, 0x73, 0x18, 0x06, 0x20, 0x01, 0x28, 0x0b, 0x32, 0x30, 0x2e, 0x74, 0x65, 0x6e, 0x73, + 0x6f, 0x72, 0x66, 0x6c, 0x6f, 0x77, 0x2e, 0x6d, 0x65, 0x74, 0x61, 0x64, 0x61, 0x74, 0x61, 0x2e, + 0x76, 0x30, 0x2e, 0x57, 0x65, 0x69, 0x67, 0x68, 0x74, 0x65, 0x64, 0x53, 0x74, 0x72, 0x69, 0x6e, + 0x67, 0x53, 0x74, 0x61, 0x74, 0x69, 0x73, 0x74, 0x69, 0x63, 0x73, 0x52, 0x13, 0x77, 0x65, 0x69, + 0x67, 0x68, 0x74, 0x65, 0x64, 0x53, 0x74, 0x72, 0x69, 0x6e, 0x67, 0x53, 0x74, 0x61, 0x74, 0x73, + 0x12, 0x27, 0x0a, 0x0f, 0x76, 0x6f, 0x63, 0x61, 0x62, 0x75, 0x6c, 0x61, 0x72, 0x79, 0x5f, 0x66, + 0x69, 0x6c, 0x65, 0x18, 0x07, 0x20, 0x01, 0x28, 0x09, 0x52, 0x0e, 0x76, 0x6f, 0x63, 0x61, 0x62, + 0x75, 0x6c, 0x61, 0x72, 0x79, 0x46, 0x69, 0x6c, 0x65, 0x1a, 0x48, 0x0a, 0x0c, 0x46, 0x72, 0x65, + 0x71, 0x41, 0x6e, 0x64, 0x56, 0x61, 0x6c, 0x75, 0x65, 0x12, 0x14, 0x0a, 0x05, 0x76, 0x61, 0x6c, + 0x75, 0x65, 0x18, 0x02, 0x20, 0x01, 0x28, 0x09, 0x52, 0x05, 0x76, 0x61, 0x6c, 0x75, 0x65, 0x12, + 0x1c, 0x0a, 0x09, 0x66, 0x72, 0x65, 0x71, 0x75, 0x65, 0x6e, 0x63, 0x79, 0x18, 0x03, 0x20, 0x01, + 0x28, 0x01, 0x52, 0x09, 0x66, 0x72, 0x65, 0x71, 0x75, 0x65, 0x6e, 0x63, 0x79, 0x4a, 0x04, 0x08, + 0x01, 0x10, 0x02, 0x22, 0xa3, 0x01, 0x0a, 0x19, 0x57, 0x65, 0x69, 0x67, 0x68, 0x74, 0x65, 0x64, + 0x4e, 0x75, 0x6d, 0x65, 0x72, 0x69, 0x63, 0x53, 0x74, 0x61, 0x74, 0x69, 0x73, 0x74, 0x69, 0x63, + 0x73, 0x12, 0x12, 0x0a, 0x04, 0x6d, 0x65, 0x61, 0x6e, 0x18, 0x01, 0x20, 0x01, 0x28, 0x01, 0x52, + 0x04, 0x6d, 0x65, 0x61, 0x6e, 0x12, 0x17, 0x0a, 0x07, 0x73, 0x74, 0x64, 0x5f, 0x64, 0x65, 0x76, + 0x18, 0x02, 0x20, 0x01, 0x28, 0x01, 0x52, 0x06, 0x73, 0x74, 0x64, 0x44, 0x65, 0x76, 0x12, 0x16, + 0x0a, 0x06, 0x6d, 0x65, 0x64, 0x69, 0x61, 0x6e, 0x18, 0x03, 0x20, 0x01, 0x28, 0x01, 0x52, 0x06, + 0x6d, 0x65, 0x64, 0x69, 0x61, 0x6e, 0x12, 0x41, 0x0a, 0x0a, 0x68, 0x69, 0x73, 0x74, 0x6f, 0x67, + 0x72, 0x61, 0x6d, 0x73, 0x18, 0x04, 0x20, 0x03, 0x28, 0x0b, 0x32, 0x21, 0x2e, 0x74, 0x65, 0x6e, + 0x73, 0x6f, 0x72, 0x66, 0x6c, 0x6f, 0x77, 0x2e, 0x6d, 0x65, 0x74, 0x61, 0x64, 0x61, 0x74, 0x61, + 0x2e, 0x76, 0x30, 0x2e, 0x48, 0x69, 0x73, 0x74, 0x6f, 0x67, 0x72, 0x61, 0x6d, 0x52, 0x0a, 0x68, + 0x69, 0x73, 0x74, 0x6f, 0x67, 0x72, 0x61, 0x6d, 0x73, 0x22, 0xbe, 0x01, 0x0a, 0x18, 0x57, 0x65, + 0x69, 0x67, 0x68, 0x74, 0x65, 0x64, 0x53, 0x74, 0x72, 0x69, 0x6e, 0x67, 0x53, 0x74, 0x61, 0x74, + 0x69, 0x73, 0x74, 0x69, 0x63, 0x73, 0x12, 0x54, 0x0a, 0x0a, 0x74, 0x6f, 0x70, 0x5f, 0x76, 0x61, + 0x6c, 0x75, 0x65, 0x73, 0x18, 0x01, 0x20, 0x03, 0x28, 0x0b, 0x32, 0x35, 0x2e, 0x74, 0x65, 0x6e, + 0x73, 0x6f, 0x72, 0x66, 0x6c, 0x6f, 0x77, 0x2e, 0x6d, 0x65, 0x74, 0x61, 0x64, 0x61, 0x74, 0x61, + 0x2e, 0x76, 0x30, 0x2e, 0x53, 0x74, 0x72, 0x69, 0x6e, 0x67, 0x53, 0x74, 0x61, 0x74, 0x69, 0x73, + 0x74, 0x69, 0x63, 0x73, 0x2e, 0x46, 0x72, 0x65, 0x71, 0x41, 0x6e, 0x64, 0x56, 0x61, 0x6c, 0x75, + 0x65, 0x52, 0x09, 0x74, 0x6f, 0x70, 0x56, 0x61, 0x6c, 0x75, 0x65, 0x73, 0x12, 0x4c, 0x0a, 0x0e, + 0x72, 0x61, 0x6e, 0x6b, 0x5f, 0x68, 0x69, 0x73, 0x74, 0x6f, 0x67, 0x72, 0x61, 0x6d, 0x18, 0x02, + 0x20, 0x01, 0x28, 0x0b, 0x32, 0x25, 0x2e, 0x74, 0x65, 0x6e, 0x73, 0x6f, 0x72, 0x66, 0x6c, 0x6f, + 0x77, 0x2e, 0x6d, 0x65, 0x74, 0x61, 0x64, 0x61, 0x74, 0x61, 0x2e, 0x76, 0x30, 0x2e, 0x52, 0x61, + 0x6e, 0x6b, 0x48, 0x69, 0x73, 0x74, 0x6f, 0x67, 0x72, 0x61, 0x6d, 0x52, 0x0d, 0x72, 0x61, 0x6e, + 0x6b, 0x48, 0x69, 0x73, 0x74, 0x6f, 0x67, 0x72, 0x61, 0x6d, 0x22, 0xe2, 0x01, 0x0a, 0x0f, 0x42, + 0x79, 0x74, 0x65, 0x73, 0x53, 0x74, 0x61, 0x74, 0x69, 0x73, 0x74, 0x69, 0x63, 0x73, 0x12, 0x4b, + 0x0a, 0x0c, 0x63, 0x6f, 0x6d, 0x6d, 0x6f, 0x6e, 0x5f, 0x73, 0x74, 0x61, 0x74, 0x73, 0x18, 0x01, + 0x20, 0x01, 0x28, 0x0b, 0x32, 0x28, 0x2e, 0x74, 0x65, 0x6e, 0x73, 0x6f, 0x72, 0x66, 0x6c, 0x6f, + 0x77, 0x2e, 0x6d, 0x65, 0x74, 0x61, 0x64, 0x61, 0x74, 0x61, 0x2e, 0x76, 0x30, 0x2e, 0x43, 0x6f, + 0x6d, 0x6d, 0x6f, 0x6e, 0x53, 0x74, 0x61, 0x74, 0x69, 0x73, 0x74, 0x69, 0x63, 0x73, 0x52, 0x0b, + 0x63, 0x6f, 0x6d, 0x6d, 0x6f, 0x6e, 0x53, 0x74, 0x61, 0x74, 0x73, 0x12, 0x16, 0x0a, 0x06, 0x75, + 0x6e, 0x69, 0x71, 0x75, 0x65, 0x18, 0x02, 0x20, 0x01, 0x28, 0x04, 0x52, 0x06, 0x75, 0x6e, 0x69, + 0x71, 0x75, 0x65, 0x12, 0x22, 0x0a, 0x0d, 0x61, 0x76, 0x67, 0x5f, 0x6e, 0x75, 0x6d, 0x5f, 0x62, + 0x79, 0x74, 0x65, 0x73, 0x18, 0x03, 0x20, 0x01, 0x28, 0x02, 0x52, 0x0b, 0x61, 0x76, 0x67, 0x4e, + 0x75, 0x6d, 0x42, 0x79, 0x74, 0x65, 0x73, 0x12, 0x22, 0x0a, 0x0d, 0x6d, 0x69, 0x6e, 0x5f, 0x6e, + 0x75, 0x6d, 0x5f, 0x62, 0x79, 0x74, 0x65, 0x73, 0x18, 0x04, 0x20, 0x01, 0x28, 0x02, 0x52, 0x0b, + 0x6d, 0x69, 0x6e, 0x4e, 0x75, 0x6d, 0x42, 0x79, 0x74, 0x65, 0x73, 0x12, 0x22, 0x0a, 0x0d, 0x6d, + 0x61, 0x78, 0x5f, 0x6e, 0x75, 0x6d, 0x5f, 0x62, 0x79, 0x74, 0x65, 0x73, 0x18, 0x05, 0x20, 0x01, + 0x28, 0x02, 0x52, 0x0b, 0x6d, 0x61, 0x78, 0x4e, 0x75, 0x6d, 0x42, 0x79, 0x74, 0x65, 0x73, 0x22, + 0x5f, 0x0a, 0x10, 0x53, 0x74, 0x72, 0x75, 0x63, 0x74, 0x53, 0x74, 0x61, 0x74, 0x69, 0x73, 0x74, + 0x69, 0x63, 0x73, 0x12, 0x4b, 0x0a, 0x0c, 0x63, 0x6f, 0x6d, 0x6d, 0x6f, 0x6e, 0x5f, 0x73, 0x74, + 0x61, 0x74, 0x73, 0x18, 0x01, 0x20, 0x01, 0x28, 0x0b, 0x32, 0x28, 0x2e, 0x74, 0x65, 0x6e, 0x73, + 0x6f, 0x72, 0x66, 0x6c, 0x6f, 0x77, 0x2e, 0x6d, 0x65, 0x74, 0x61, 0x64, 0x61, 0x74, 0x61, 0x2e, + 0x76, 0x30, 0x2e, 0x43, 0x6f, 0x6d, 0x6d, 0x6f, 0x6e, 0x53, 0x74, 0x61, 0x74, 0x69, 0x73, 0x74, + 0x69, 0x63, 0x73, 0x52, 0x0b, 0x63, 0x6f, 0x6d, 0x6d, 0x6f, 0x6e, 0x53, 0x74, 0x61, 0x74, 0x73, + 0x22, 0x94, 0x04, 0x0a, 0x10, 0x43, 0x6f, 0x6d, 0x6d, 0x6f, 0x6e, 0x53, 0x74, 0x61, 0x74, 0x69, + 0x73, 0x74, 0x69, 0x63, 0x73, 0x12, 0x26, 0x0a, 0x0f, 0x6e, 0x75, 0x6d, 0x5f, 0x6e, 0x6f, 0x6e, + 0x5f, 0x6d, 0x69, 0x73, 0x73, 0x69, 0x6e, 0x67, 0x18, 0x01, 0x20, 0x01, 0x28, 0x04, 0x52, 0x0d, + 0x6e, 0x75, 0x6d, 0x4e, 0x6f, 0x6e, 0x4d, 0x69, 0x73, 0x73, 0x69, 0x6e, 0x67, 0x12, 0x1f, 0x0a, + 0x0b, 0x6e, 0x75, 0x6d, 0x5f, 0x6d, 0x69, 0x73, 0x73, 0x69, 0x6e, 0x67, 0x18, 0x02, 0x20, 0x01, + 0x28, 0x04, 0x52, 0x0a, 0x6e, 0x75, 0x6d, 0x4d, 0x69, 0x73, 0x73, 0x69, 0x6e, 0x67, 0x12, 0x24, + 0x0a, 0x0e, 0x6d, 0x69, 0x6e, 0x5f, 0x6e, 0x75, 0x6d, 0x5f, 0x76, 0x61, 0x6c, 0x75, 0x65, 0x73, + 0x18, 0x03, 0x20, 0x01, 0x28, 0x04, 0x52, 0x0c, 0x6d, 0x69, 0x6e, 0x4e, 0x75, 0x6d, 0x56, 0x61, + 0x6c, 0x75, 0x65, 0x73, 0x12, 0x24, 0x0a, 0x0e, 0x6d, 0x61, 0x78, 0x5f, 0x6e, 0x75, 0x6d, 0x5f, + 0x76, 0x61, 0x6c, 0x75, 0x65, 0x73, 0x18, 0x04, 0x20, 0x01, 0x28, 0x04, 0x52, 0x0c, 0x6d, 0x61, + 0x78, 0x4e, 0x75, 0x6d, 0x56, 0x61, 0x6c, 0x75, 0x65, 0x73, 0x12, 0x24, 0x0a, 0x0e, 0x61, 0x76, + 0x67, 0x5f, 0x6e, 0x75, 0x6d, 0x5f, 0x76, 0x61, 0x6c, 0x75, 0x65, 0x73, 0x18, 0x05, 0x20, 0x01, + 0x28, 0x02, 0x52, 0x0c, 0x61, 0x76, 0x67, 0x4e, 0x75, 0x6d, 0x56, 0x61, 0x6c, 0x75, 0x65, 0x73, + 0x12, 0x24, 0x0a, 0x0e, 0x74, 0x6f, 0x74, 0x5f, 0x6e, 0x75, 0x6d, 0x5f, 0x76, 0x61, 0x6c, 0x75, + 0x65, 0x73, 0x18, 0x08, 0x20, 0x01, 0x28, 0x04, 0x52, 0x0c, 0x74, 0x6f, 0x74, 0x4e, 0x75, 0x6d, + 0x56, 0x61, 0x6c, 0x75, 0x65, 0x73, 0x12, 0x53, 0x0a, 0x14, 0x6e, 0x75, 0x6d, 0x5f, 0x76, 0x61, + 0x6c, 0x75, 0x65, 0x73, 0x5f, 0x68, 0x69, 0x73, 0x74, 0x6f, 0x67, 0x72, 0x61, 0x6d, 0x18, 0x06, + 0x20, 0x01, 0x28, 0x0b, 0x32, 0x21, 0x2e, 0x74, 0x65, 0x6e, 0x73, 0x6f, 0x72, 0x66, 0x6c, 0x6f, + 0x77, 0x2e, 0x6d, 0x65, 0x74, 0x61, 0x64, 0x61, 0x74, 0x61, 0x2e, 0x76, 0x30, 0x2e, 0x48, 0x69, + 0x73, 0x74, 0x6f, 0x67, 0x72, 0x61, 0x6d, 0x52, 0x12, 0x6e, 0x75, 0x6d, 0x56, 0x61, 0x6c, 0x75, + 0x65, 0x73, 0x48, 0x69, 0x73, 0x74, 0x6f, 0x67, 0x72, 0x61, 0x6d, 0x12, 0x64, 0x0a, 0x15, 0x77, + 0x65, 0x69, 0x67, 0x68, 0x74, 0x65, 0x64, 0x5f, 0x63, 0x6f, 0x6d, 0x6d, 0x6f, 0x6e, 0x5f, 0x73, + 0x74, 0x61, 0x74, 0x73, 0x18, 0x07, 0x20, 0x01, 0x28, 0x0b, 0x32, 0x30, 0x2e, 0x74, 0x65, 0x6e, + 0x73, 0x6f, 0x72, 0x66, 0x6c, 0x6f, 0x77, 0x2e, 0x6d, 0x65, 0x74, 0x61, 0x64, 0x61, 0x74, 0x61, + 0x2e, 0x76, 0x30, 0x2e, 0x57, 0x65, 0x69, 0x67, 0x68, 0x74, 0x65, 0x64, 0x43, 0x6f, 0x6d, 0x6d, + 0x6f, 0x6e, 0x53, 0x74, 0x61, 0x74, 0x69, 0x73, 0x74, 0x69, 0x63, 0x73, 0x52, 0x13, 0x77, 0x65, + 0x69, 0x67, 0x68, 0x74, 0x65, 0x64, 0x43, 0x6f, 0x6d, 0x6d, 0x6f, 0x6e, 0x53, 0x74, 0x61, 0x74, + 0x73, 0x12, 0x64, 0x0a, 0x1d, 0x66, 0x65, 0x61, 0x74, 0x75, 0x72, 0x65, 0x5f, 0x6c, 0x69, 0x73, + 0x74, 0x5f, 0x6c, 0x65, 0x6e, 0x67, 0x74, 0x68, 0x5f, 0x68, 0x69, 0x73, 0x74, 0x6f, 0x67, 0x72, + 0x61, 0x6d, 0x18, 0x09, 0x20, 0x01, 0x28, 0x0b, 0x32, 0x21, 0x2e, 0x74, 0x65, 0x6e, 0x73, 0x6f, + 0x72, 0x66, 0x6c, 0x6f, 0x77, 0x2e, 0x6d, 0x65, 0x74, 0x61, 0x64, 0x61, 0x74, 0x61, 0x2e, 0x76, + 0x30, 0x2e, 0x48, 0x69, 0x73, 0x74, 0x6f, 0x67, 0x72, 0x61, 0x6d, 0x52, 0x1a, 0x66, 0x65, 0x61, + 0x74, 0x75, 0x72, 0x65, 0x4c, 0x69, 0x73, 0x74, 0x4c, 0x65, 0x6e, 0x67, 0x74, 0x68, 0x48, 0x69, + 0x73, 0x74, 0x6f, 0x67, 0x72, 0x61, 0x6d, 0x22, 0x83, 0x03, 0x0a, 0x09, 0x48, 0x69, 0x73, 0x74, + 0x6f, 0x67, 0x72, 0x61, 0x6d, 0x12, 0x17, 0x0a, 0x07, 0x6e, 0x75, 0x6d, 0x5f, 0x6e, 0x61, 0x6e, + 0x18, 0x01, 0x20, 0x01, 0x28, 0x04, 0x52, 0x06, 0x6e, 0x75, 0x6d, 0x4e, 0x61, 0x6e, 0x12, 0x23, + 0x0a, 0x0d, 0x6e, 0x75, 0x6d, 0x5f, 0x75, 0x6e, 0x64, 0x65, 0x66, 0x69, 0x6e, 0x65, 0x64, 0x18, + 0x02, 0x20, 0x01, 0x28, 0x04, 0x52, 0x0c, 0x6e, 0x75, 0x6d, 0x55, 0x6e, 0x64, 0x65, 0x66, 0x69, + 0x6e, 0x65, 0x64, 0x12, 0x42, 0x0a, 0x07, 0x62, 0x75, 0x63, 0x6b, 0x65, 0x74, 0x73, 0x18, 0x03, + 0x20, 0x03, 0x28, 0x0b, 0x32, 0x28, 0x2e, 0x74, 0x65, 0x6e, 0x73, 0x6f, 0x72, 0x66, 0x6c, 0x6f, + 0x77, 0x2e, 0x6d, 0x65, 0x74, 0x61, 0x64, 0x61, 0x74, 0x61, 0x2e, 0x76, 0x30, 0x2e, 0x48, 0x69, + 0x73, 0x74, 0x6f, 0x67, 0x72, 0x61, 0x6d, 0x2e, 0x42, 0x75, 0x63, 0x6b, 0x65, 0x74, 0x52, 0x07, + 0x62, 0x75, 0x63, 0x6b, 0x65, 0x74, 0x73, 0x12, 0x43, 0x0a, 0x04, 0x74, 0x79, 0x70, 0x65, 0x18, + 0x04, 0x20, 0x01, 0x28, 0x0e, 0x32, 0x2f, 0x2e, 0x74, 0x65, 0x6e, 0x73, 0x6f, 0x72, 0x66, 0x6c, + 0x6f, 0x77, 0x2e, 0x6d, 0x65, 0x74, 0x61, 0x64, 0x61, 0x74, 0x61, 0x2e, 0x76, 0x30, 0x2e, 0x48, + 0x69, 0x73, 0x74, 0x6f, 0x67, 0x72, 0x61, 0x6d, 0x2e, 0x48, 0x69, 0x73, 0x74, 0x6f, 0x67, 0x72, + 0x61, 0x6d, 0x54, 0x79, 0x70, 0x65, 0x52, 0x04, 0x74, 0x79, 0x70, 0x65, 0x12, 0x12, 0x0a, 0x04, + 0x6e, 0x61, 0x6d, 0x65, 0x18, 0x05, 0x20, 0x01, 0x28, 0x09, 0x52, 0x04, 0x6e, 0x61, 0x6d, 0x65, + 0x1a, 0x6d, 0x0a, 0x06, 0x42, 0x75, 0x63, 0x6b, 0x65, 0x74, 0x12, 0x1b, 0x0a, 0x09, 0x6c, 0x6f, + 0x77, 0x5f, 0x76, 0x61, 0x6c, 0x75, 0x65, 0x18, 0x01, 0x20, 0x01, 0x28, 0x01, 0x52, 0x08, 0x6c, + 0x6f, 0x77, 0x56, 0x61, 0x6c, 0x75, 0x65, 0x12, 0x1d, 0x0a, 0x0a, 0x68, 0x69, 0x67, 0x68, 0x5f, + 0x76, 0x61, 0x6c, 0x75, 0x65, 0x18, 0x02, 0x20, 0x01, 0x28, 0x01, 0x52, 0x09, 0x68, 0x69, 0x67, + 0x68, 0x56, 0x61, 0x6c, 0x75, 0x65, 0x12, 0x21, 0x0a, 0x0c, 0x73, 0x61, 0x6d, 0x70, 0x6c, 0x65, + 0x5f, 0x63, 0x6f, 0x75, 0x6e, 0x74, 0x18, 0x04, 0x20, 0x01, 0x28, 0x01, 0x52, 0x0b, 0x73, 0x61, + 0x6d, 0x70, 0x6c, 0x65, 0x43, 0x6f, 0x75, 0x6e, 0x74, 0x4a, 0x04, 0x08, 0x03, 0x10, 0x04, 0x22, + 0x2c, 0x0a, 0x0d, 0x48, 0x69, 0x73, 0x74, 0x6f, 0x67, 0x72, 0x61, 0x6d, 0x54, 0x79, 0x70, 0x65, + 0x12, 0x0c, 0x0a, 0x08, 0x53, 0x54, 0x41, 0x4e, 0x44, 0x41, 0x52, 0x44, 0x10, 0x00, 0x12, 0x0d, + 0x0a, 0x09, 0x51, 0x55, 0x41, 0x4e, 0x54, 0x49, 0x4c, 0x45, 0x53, 0x10, 0x01, 0x22, 0xec, 0x01, + 0x0a, 0x0d, 0x52, 0x61, 0x6e, 0x6b, 0x48, 0x69, 0x73, 0x74, 0x6f, 0x67, 0x72, 0x61, 0x6d, 0x12, + 0x46, 0x0a, 0x07, 0x62, 0x75, 0x63, 0x6b, 0x65, 0x74, 0x73, 0x18, 0x01, 0x20, 0x03, 0x28, 0x0b, + 0x32, 0x2c, 0x2e, 0x74, 0x65, 0x6e, 0x73, 0x6f, 0x72, 0x66, 0x6c, 0x6f, 0x77, 0x2e, 0x6d, 0x65, + 0x74, 0x61, 0x64, 0x61, 0x74, 0x61, 0x2e, 0x76, 0x30, 0x2e, 0x52, 0x61, 0x6e, 0x6b, 0x48, 0x69, + 0x73, 0x74, 0x6f, 0x67, 0x72, 0x61, 0x6d, 0x2e, 0x42, 0x75, 0x63, 0x6b, 0x65, 0x74, 0x52, 0x07, + 0x62, 0x75, 0x63, 0x6b, 0x65, 0x74, 0x73, 0x12, 0x12, 0x0a, 0x04, 0x6e, 0x61, 0x6d, 0x65, 0x18, + 0x02, 0x20, 0x01, 0x28, 0x09, 0x52, 0x04, 0x6e, 0x61, 0x6d, 0x65, 0x1a, 0x7f, 0x0a, 0x06, 0x42, + 0x75, 0x63, 0x6b, 0x65, 0x74, 0x12, 0x19, 0x0a, 0x08, 0x6c, 0x6f, 0x77, 0x5f, 0x72, 0x61, 0x6e, + 0x6b, 0x18, 0x01, 0x20, 0x01, 0x28, 0x04, 0x52, 0x07, 0x6c, 0x6f, 0x77, 0x52, 0x61, 0x6e, 0x6b, + 0x12, 0x1b, 0x0a, 0x09, 0x68, 0x69, 0x67, 0x68, 0x5f, 0x72, 0x61, 0x6e, 0x6b, 0x18, 0x02, 0x20, + 0x01, 0x28, 0x04, 0x52, 0x08, 0x68, 0x69, 0x67, 0x68, 0x52, 0x61, 0x6e, 0x6b, 0x12, 0x14, 0x0a, + 0x05, 0x6c, 0x61, 0x62, 0x65, 0x6c, 0x18, 0x04, 0x20, 0x01, 0x28, 0x09, 0x52, 0x05, 0x6c, 0x61, + 0x62, 0x65, 0x6c, 0x12, 0x21, 0x0a, 0x0c, 0x73, 0x61, 0x6d, 0x70, 0x6c, 0x65, 0x5f, 0x63, 0x6f, + 0x75, 0x6e, 0x74, 0x18, 0x05, 0x20, 0x01, 0x28, 0x01, 0x52, 0x0b, 0x73, 0x61, 0x6d, 0x70, 0x6c, + 0x65, 0x43, 0x6f, 0x75, 0x6e, 0x74, 0x4a, 0x04, 0x08, 0x03, 0x10, 0x04, 0x42, 0x68, 0x0a, 0x1a, + 0x6f, 0x72, 0x67, 0x2e, 0x74, 0x65, 0x6e, 0x73, 0x6f, 0x72, 0x66, 0x6c, 0x6f, 0x77, 0x2e, 0x6d, + 0x65, 0x74, 0x61, 0x64, 0x61, 0x74, 0x61, 0x2e, 0x76, 0x30, 0x50, 0x01, 0x5a, 0x45, 0x67, 0x69, + 0x74, 0x68, 0x75, 0x62, 0x2e, 0x63, 0x6f, 0x6d, 0x2f, 0x66, 0x65, 0x61, 0x73, 0x74, 0x2d, 0x64, + 0x65, 0x76, 0x2f, 0x66, 0x65, 0x61, 0x73, 0x74, 0x2f, 0x73, 0x64, 0x6b, 0x2f, 0x67, 0x6f, 0x2f, + 0x70, 0x72, 0x6f, 0x74, 0x6f, 0x73, 0x2f, 0x74, 0x65, 0x6e, 0x73, 0x6f, 0x72, 0x66, 0x6c, 0x6f, + 0x77, 0x5f, 0x6d, 0x65, 0x74, 0x61, 0x64, 0x61, 0x74, 0x61, 0x2f, 0x70, 0x72, 0x6f, 0x74, 0x6f, + 0x2f, 0x76, 0x30, 0xf8, 0x01, 0x01, 0x62, 0x06, 0x70, 0x72, 0x6f, 0x74, 0x6f, 0x33, +} + +var ( + file_tensorflow_metadata_proto_v0_statistics_proto_rawDescOnce sync.Once + file_tensorflow_metadata_proto_v0_statistics_proto_rawDescData = file_tensorflow_metadata_proto_v0_statistics_proto_rawDesc +) + +func file_tensorflow_metadata_proto_v0_statistics_proto_rawDescGZIP() []byte { + file_tensorflow_metadata_proto_v0_statistics_proto_rawDescOnce.Do(func() { + file_tensorflow_metadata_proto_v0_statistics_proto_rawDescData = protoimpl.X.CompressGZIP(file_tensorflow_metadata_proto_v0_statistics_proto_rawDescData) + }) + return file_tensorflow_metadata_proto_v0_statistics_proto_rawDescData +} + +var file_tensorflow_metadata_proto_v0_statistics_proto_enumTypes = make([]protoimpl.EnumInfo, 2) +var file_tensorflow_metadata_proto_v0_statistics_proto_msgTypes = make([]protoimpl.MessageInfo, 24) +var file_tensorflow_metadata_proto_v0_statistics_proto_goTypes = []interface{}{ + (FeatureNameStatistics_Type)(0), // 0: tensorflow.metadata.v0.FeatureNameStatistics.Type + (Histogram_HistogramType)(0), // 1: tensorflow.metadata.v0.Histogram.HistogramType + (*DatasetFeatureStatisticsList)(nil), // 2: tensorflow.metadata.v0.DatasetFeatureStatisticsList + (*DatasetFeatureStatistics)(nil), // 3: tensorflow.metadata.v0.DatasetFeatureStatistics + (*CrossFeatureStatistics)(nil), // 4: tensorflow.metadata.v0.CrossFeatureStatistics + (*NumericCrossStatistics)(nil), // 5: tensorflow.metadata.v0.NumericCrossStatistics + (*CategoricalCrossStatistics)(nil), // 6: tensorflow.metadata.v0.CategoricalCrossStatistics + (*LiftStatistics)(nil), // 7: tensorflow.metadata.v0.LiftStatistics + (*LiftSeries)(nil), // 8: tensorflow.metadata.v0.LiftSeries + (*FeatureNameStatistics)(nil), // 9: tensorflow.metadata.v0.FeatureNameStatistics + (*WeightedCommonStatistics)(nil), // 10: tensorflow.metadata.v0.WeightedCommonStatistics + (*CustomStatistic)(nil), // 11: tensorflow.metadata.v0.CustomStatistic + (*NumericStatistics)(nil), // 12: tensorflow.metadata.v0.NumericStatistics + (*StringStatistics)(nil), // 13: tensorflow.metadata.v0.StringStatistics + (*WeightedNumericStatistics)(nil), // 14: tensorflow.metadata.v0.WeightedNumericStatistics + (*WeightedStringStatistics)(nil), // 15: tensorflow.metadata.v0.WeightedStringStatistics + (*BytesStatistics)(nil), // 16: tensorflow.metadata.v0.BytesStatistics + (*StructStatistics)(nil), // 17: tensorflow.metadata.v0.StructStatistics + (*CommonStatistics)(nil), // 18: tensorflow.metadata.v0.CommonStatistics + (*Histogram)(nil), // 19: tensorflow.metadata.v0.Histogram + (*RankHistogram)(nil), // 20: tensorflow.metadata.v0.RankHistogram + (*LiftSeries_Bucket)(nil), // 21: tensorflow.metadata.v0.LiftSeries.Bucket + (*LiftSeries_LiftValue)(nil), // 22: tensorflow.metadata.v0.LiftSeries.LiftValue + (*StringStatistics_FreqAndValue)(nil), // 23: tensorflow.metadata.v0.StringStatistics.FreqAndValue + (*Histogram_Bucket)(nil), // 24: tensorflow.metadata.v0.Histogram.Bucket + (*RankHistogram_Bucket)(nil), // 25: tensorflow.metadata.v0.RankHistogram.Bucket + (*Path)(nil), // 26: tensorflow.metadata.v0.Path +} +var file_tensorflow_metadata_proto_v0_statistics_proto_depIdxs = []int32{ + 3, // 0: tensorflow.metadata.v0.DatasetFeatureStatisticsList.datasets:type_name -> tensorflow.metadata.v0.DatasetFeatureStatistics + 9, // 1: tensorflow.metadata.v0.DatasetFeatureStatistics.features:type_name -> tensorflow.metadata.v0.FeatureNameStatistics + 4, // 2: tensorflow.metadata.v0.DatasetFeatureStatistics.cross_features:type_name -> tensorflow.metadata.v0.CrossFeatureStatistics + 26, // 3: tensorflow.metadata.v0.CrossFeatureStatistics.path_x:type_name -> tensorflow.metadata.v0.Path + 26, // 4: tensorflow.metadata.v0.CrossFeatureStatistics.path_y:type_name -> tensorflow.metadata.v0.Path + 5, // 5: tensorflow.metadata.v0.CrossFeatureStatistics.num_cross_stats:type_name -> tensorflow.metadata.v0.NumericCrossStatistics + 6, // 6: tensorflow.metadata.v0.CrossFeatureStatistics.categorical_cross_stats:type_name -> tensorflow.metadata.v0.CategoricalCrossStatistics + 7, // 7: tensorflow.metadata.v0.CategoricalCrossStatistics.lift:type_name -> tensorflow.metadata.v0.LiftStatistics + 8, // 8: tensorflow.metadata.v0.LiftStatistics.lift_series:type_name -> tensorflow.metadata.v0.LiftSeries + 8, // 9: tensorflow.metadata.v0.LiftStatistics.weighted_lift_series:type_name -> tensorflow.metadata.v0.LiftSeries + 21, // 10: tensorflow.metadata.v0.LiftSeries.y_bucket:type_name -> tensorflow.metadata.v0.LiftSeries.Bucket + 22, // 11: tensorflow.metadata.v0.LiftSeries.lift_values:type_name -> tensorflow.metadata.v0.LiftSeries.LiftValue + 26, // 12: tensorflow.metadata.v0.FeatureNameStatistics.path:type_name -> tensorflow.metadata.v0.Path + 0, // 13: tensorflow.metadata.v0.FeatureNameStatistics.type:type_name -> tensorflow.metadata.v0.FeatureNameStatistics.Type + 12, // 14: tensorflow.metadata.v0.FeatureNameStatistics.num_stats:type_name -> tensorflow.metadata.v0.NumericStatistics + 13, // 15: tensorflow.metadata.v0.FeatureNameStatistics.string_stats:type_name -> tensorflow.metadata.v0.StringStatistics + 16, // 16: tensorflow.metadata.v0.FeatureNameStatistics.bytes_stats:type_name -> tensorflow.metadata.v0.BytesStatistics + 17, // 17: tensorflow.metadata.v0.FeatureNameStatistics.struct_stats:type_name -> tensorflow.metadata.v0.StructStatistics + 11, // 18: tensorflow.metadata.v0.FeatureNameStatistics.custom_stats:type_name -> tensorflow.metadata.v0.CustomStatistic + 19, // 19: tensorflow.metadata.v0.CustomStatistic.histogram:type_name -> tensorflow.metadata.v0.Histogram + 20, // 20: tensorflow.metadata.v0.CustomStatistic.rank_histogram:type_name -> tensorflow.metadata.v0.RankHistogram + 18, // 21: tensorflow.metadata.v0.NumericStatistics.common_stats:type_name -> tensorflow.metadata.v0.CommonStatistics + 19, // 22: tensorflow.metadata.v0.NumericStatistics.histograms:type_name -> tensorflow.metadata.v0.Histogram + 14, // 23: tensorflow.metadata.v0.NumericStatistics.weighted_numeric_stats:type_name -> tensorflow.metadata.v0.WeightedNumericStatistics + 18, // 24: tensorflow.metadata.v0.StringStatistics.common_stats:type_name -> tensorflow.metadata.v0.CommonStatistics + 23, // 25: tensorflow.metadata.v0.StringStatistics.top_values:type_name -> tensorflow.metadata.v0.StringStatistics.FreqAndValue + 20, // 26: tensorflow.metadata.v0.StringStatistics.rank_histogram:type_name -> tensorflow.metadata.v0.RankHistogram + 15, // 27: tensorflow.metadata.v0.StringStatistics.weighted_string_stats:type_name -> tensorflow.metadata.v0.WeightedStringStatistics + 19, // 28: tensorflow.metadata.v0.WeightedNumericStatistics.histograms:type_name -> tensorflow.metadata.v0.Histogram + 23, // 29: tensorflow.metadata.v0.WeightedStringStatistics.top_values:type_name -> tensorflow.metadata.v0.StringStatistics.FreqAndValue + 20, // 30: tensorflow.metadata.v0.WeightedStringStatistics.rank_histogram:type_name -> tensorflow.metadata.v0.RankHistogram + 18, // 31: tensorflow.metadata.v0.BytesStatistics.common_stats:type_name -> tensorflow.metadata.v0.CommonStatistics + 18, // 32: tensorflow.metadata.v0.StructStatistics.common_stats:type_name -> tensorflow.metadata.v0.CommonStatistics + 19, // 33: tensorflow.metadata.v0.CommonStatistics.num_values_histogram:type_name -> tensorflow.metadata.v0.Histogram + 10, // 34: tensorflow.metadata.v0.CommonStatistics.weighted_common_stats:type_name -> tensorflow.metadata.v0.WeightedCommonStatistics + 19, // 35: tensorflow.metadata.v0.CommonStatistics.feature_list_length_histogram:type_name -> tensorflow.metadata.v0.Histogram + 24, // 36: tensorflow.metadata.v0.Histogram.buckets:type_name -> tensorflow.metadata.v0.Histogram.Bucket + 1, // 37: tensorflow.metadata.v0.Histogram.type:type_name -> tensorflow.metadata.v0.Histogram.HistogramType + 25, // 38: tensorflow.metadata.v0.RankHistogram.buckets:type_name -> tensorflow.metadata.v0.RankHistogram.Bucket + 39, // [39:39] is the sub-list for method output_type + 39, // [39:39] is the sub-list for method input_type + 39, // [39:39] is the sub-list for extension type_name + 39, // [39:39] is the sub-list for extension extendee + 0, // [0:39] is the sub-list for field type_name +} + +func init() { file_tensorflow_metadata_proto_v0_statistics_proto_init() } +func file_tensorflow_metadata_proto_v0_statistics_proto_init() { + if File_tensorflow_metadata_proto_v0_statistics_proto != nil { + return + } + file_tensorflow_metadata_proto_v0_path_proto_init() + if !protoimpl.UnsafeEnabled { + file_tensorflow_metadata_proto_v0_statistics_proto_msgTypes[0].Exporter = func(v interface{}, i int) interface{} { + switch v := v.(*DatasetFeatureStatisticsList); i { + case 0: + return &v.state + case 1: + return &v.sizeCache + case 2: + return &v.unknownFields + default: + return nil + } + } + file_tensorflow_metadata_proto_v0_statistics_proto_msgTypes[1].Exporter = func(v interface{}, i int) interface{} { + switch v := v.(*DatasetFeatureStatistics); i { + case 0: + return &v.state + case 1: + return &v.sizeCache + case 2: + return &v.unknownFields + default: + return nil + } + } + file_tensorflow_metadata_proto_v0_statistics_proto_msgTypes[2].Exporter = func(v interface{}, i int) interface{} { + switch v := v.(*CrossFeatureStatistics); i { + case 0: + return &v.state + case 1: + return &v.sizeCache + case 2: + return &v.unknownFields + default: + return nil + } + } + file_tensorflow_metadata_proto_v0_statistics_proto_msgTypes[3].Exporter = func(v interface{}, i int) interface{} { + switch v := v.(*NumericCrossStatistics); i { + case 0: + return &v.state + case 1: + return &v.sizeCache + case 2: + return &v.unknownFields + default: + return nil + } + } + file_tensorflow_metadata_proto_v0_statistics_proto_msgTypes[4].Exporter = func(v interface{}, i int) interface{} { + switch v := v.(*CategoricalCrossStatistics); i { + case 0: + return &v.state + case 1: + return &v.sizeCache + case 2: + return &v.unknownFields + default: + return nil + } + } + file_tensorflow_metadata_proto_v0_statistics_proto_msgTypes[5].Exporter = func(v interface{}, i int) interface{} { + switch v := v.(*LiftStatistics); i { + case 0: + return &v.state + case 1: + return &v.sizeCache + case 2: + return &v.unknownFields + default: + return nil + } + } + file_tensorflow_metadata_proto_v0_statistics_proto_msgTypes[6].Exporter = func(v interface{}, i int) interface{} { + switch v := v.(*LiftSeries); i { + case 0: + return &v.state + case 1: + return &v.sizeCache + case 2: + return &v.unknownFields + default: + return nil + } + } + file_tensorflow_metadata_proto_v0_statistics_proto_msgTypes[7].Exporter = func(v interface{}, i int) interface{} { + switch v := v.(*FeatureNameStatistics); i { + case 0: + return &v.state + case 1: + return &v.sizeCache + case 2: + return &v.unknownFields + default: + return nil + } + } + file_tensorflow_metadata_proto_v0_statistics_proto_msgTypes[8].Exporter = func(v interface{}, i int) interface{} { + switch v := v.(*WeightedCommonStatistics); i { + case 0: + return &v.state + case 1: + return &v.sizeCache + case 2: + return &v.unknownFields + default: + return nil + } + } + file_tensorflow_metadata_proto_v0_statistics_proto_msgTypes[9].Exporter = func(v interface{}, i int) interface{} { + switch v := v.(*CustomStatistic); i { + case 0: + return &v.state + case 1: + return &v.sizeCache + case 2: + return &v.unknownFields + default: + return nil + } + } + file_tensorflow_metadata_proto_v0_statistics_proto_msgTypes[10].Exporter = func(v interface{}, i int) interface{} { + switch v := v.(*NumericStatistics); i { + case 0: + return &v.state + case 1: + return &v.sizeCache + case 2: + return &v.unknownFields + default: + return nil + } + } + file_tensorflow_metadata_proto_v0_statistics_proto_msgTypes[11].Exporter = func(v interface{}, i int) interface{} { + switch v := v.(*StringStatistics); i { + case 0: + return &v.state + case 1: + return &v.sizeCache + case 2: + return &v.unknownFields + default: + return nil + } + } + file_tensorflow_metadata_proto_v0_statistics_proto_msgTypes[12].Exporter = func(v interface{}, i int) interface{} { + switch v := v.(*WeightedNumericStatistics); i { + case 0: + return &v.state + case 1: + return &v.sizeCache + case 2: + return &v.unknownFields + default: + return nil + } + } + file_tensorflow_metadata_proto_v0_statistics_proto_msgTypes[13].Exporter = func(v interface{}, i int) interface{} { + switch v := v.(*WeightedStringStatistics); i { + case 0: + return &v.state + case 1: + return &v.sizeCache + case 2: + return &v.unknownFields + default: + return nil + } + } + file_tensorflow_metadata_proto_v0_statistics_proto_msgTypes[14].Exporter = func(v interface{}, i int) interface{} { + switch v := v.(*BytesStatistics); i { + case 0: + return &v.state + case 1: + return &v.sizeCache + case 2: + return &v.unknownFields + default: + return nil + } + } + file_tensorflow_metadata_proto_v0_statistics_proto_msgTypes[15].Exporter = func(v interface{}, i int) interface{} { + switch v := v.(*StructStatistics); i { + case 0: + return &v.state + case 1: + return &v.sizeCache + case 2: + return &v.unknownFields + default: + return nil + } + } + file_tensorflow_metadata_proto_v0_statistics_proto_msgTypes[16].Exporter = func(v interface{}, i int) interface{} { + switch v := v.(*CommonStatistics); i { + case 0: + return &v.state + case 1: + return &v.sizeCache + case 2: + return &v.unknownFields + default: + return nil + } + } + file_tensorflow_metadata_proto_v0_statistics_proto_msgTypes[17].Exporter = func(v interface{}, i int) interface{} { + switch v := v.(*Histogram); i { + case 0: + return &v.state + case 1: + return &v.sizeCache + case 2: + return &v.unknownFields + default: + return nil + } + } + file_tensorflow_metadata_proto_v0_statistics_proto_msgTypes[18].Exporter = func(v interface{}, i int) interface{} { + switch v := v.(*RankHistogram); i { + case 0: + return &v.state + case 1: + return &v.sizeCache + case 2: + return &v.unknownFields + default: + return nil + } + } + file_tensorflow_metadata_proto_v0_statistics_proto_msgTypes[19].Exporter = func(v interface{}, i int) interface{} { + switch v := v.(*LiftSeries_Bucket); i { + case 0: + return &v.state + case 1: + return &v.sizeCache + case 2: + return &v.unknownFields + default: + return nil + } + } + file_tensorflow_metadata_proto_v0_statistics_proto_msgTypes[20].Exporter = func(v interface{}, i int) interface{} { + switch v := v.(*LiftSeries_LiftValue); i { + case 0: + return &v.state + case 1: + return &v.sizeCache + case 2: + return &v.unknownFields + default: + return nil + } + } + file_tensorflow_metadata_proto_v0_statistics_proto_msgTypes[21].Exporter = func(v interface{}, i int) interface{} { + switch v := v.(*StringStatistics_FreqAndValue); i { + case 0: + return &v.state + case 1: + return &v.sizeCache + case 2: + return &v.unknownFields + default: + return nil + } + } + file_tensorflow_metadata_proto_v0_statistics_proto_msgTypes[22].Exporter = func(v interface{}, i int) interface{} { + switch v := v.(*Histogram_Bucket); i { + case 0: + return &v.state + case 1: + return &v.sizeCache + case 2: + return &v.unknownFields + default: + return nil + } + } + file_tensorflow_metadata_proto_v0_statistics_proto_msgTypes[23].Exporter = func(v interface{}, i int) interface{} { + switch v := v.(*RankHistogram_Bucket); i { + case 0: + return &v.state + case 1: + return &v.sizeCache + case 2: + return &v.unknownFields + default: + return nil + } + } + } + file_tensorflow_metadata_proto_v0_statistics_proto_msgTypes[2].OneofWrappers = []interface{}{ + (*CrossFeatureStatistics_NumCrossStats)(nil), + (*CrossFeatureStatistics_CategoricalCrossStats)(nil), + } + file_tensorflow_metadata_proto_v0_statistics_proto_msgTypes[6].OneofWrappers = []interface{}{ + (*LiftSeries_YInt)(nil), + (*LiftSeries_YString)(nil), + (*LiftSeries_YBucket)(nil), + (*LiftSeries_YCount)(nil), + (*LiftSeries_WeightedYCount)(nil), + } + file_tensorflow_metadata_proto_v0_statistics_proto_msgTypes[7].OneofWrappers = []interface{}{ + (*FeatureNameStatistics_Name)(nil), + (*FeatureNameStatistics_Path)(nil), + (*FeatureNameStatistics_NumStats)(nil), + (*FeatureNameStatistics_StringStats)(nil), + (*FeatureNameStatistics_BytesStats)(nil), + (*FeatureNameStatistics_StructStats)(nil), + } + file_tensorflow_metadata_proto_v0_statistics_proto_msgTypes[9].OneofWrappers = []interface{}{ + (*CustomStatistic_Num)(nil), + (*CustomStatistic_Str)(nil), + (*CustomStatistic_Histogram)(nil), + (*CustomStatistic_RankHistogram)(nil), + } + file_tensorflow_metadata_proto_v0_statistics_proto_msgTypes[20].OneofWrappers = []interface{}{ + (*LiftSeries_LiftValue_XInt)(nil), + (*LiftSeries_LiftValue_XString)(nil), + (*LiftSeries_LiftValue_XCount)(nil), + (*LiftSeries_LiftValue_WeightedXCount)(nil), + (*LiftSeries_LiftValue_XAndYCount)(nil), + (*LiftSeries_LiftValue_WeightedXAndYCount)(nil), + } + type x struct{} + out := protoimpl.TypeBuilder{ + File: protoimpl.DescBuilder{ + GoPackagePath: reflect.TypeOf(x{}).PkgPath(), + RawDescriptor: file_tensorflow_metadata_proto_v0_statistics_proto_rawDesc, + NumEnums: 2, + NumMessages: 24, + NumExtensions: 0, + NumServices: 0, + }, + GoTypes: file_tensorflow_metadata_proto_v0_statistics_proto_goTypes, + DependencyIndexes: file_tensorflow_metadata_proto_v0_statistics_proto_depIdxs, + EnumInfos: file_tensorflow_metadata_proto_v0_statistics_proto_enumTypes, + MessageInfos: file_tensorflow_metadata_proto_v0_statistics_proto_msgTypes, + }.Build() + File_tensorflow_metadata_proto_v0_statistics_proto = out.File + file_tensorflow_metadata_proto_v0_statistics_proto_rawDesc = nil + file_tensorflow_metadata_proto_v0_statistics_proto_goTypes = nil + file_tensorflow_metadata_proto_v0_statistics_proto_depIdxs = nil +} From 960a763f61734e79d094347c04f5214f42b243b9 Mon Sep 17 00:00:00 2001 From: zhilingc Date: Tue, 2 Jun 2020 11:51:21 +0800 Subject: [PATCH 051/133] Fix missing field after rebase --- core/src/main/java/feast/core/grpc/CoreServiceImpl.java | 1 + 1 file changed, 1 insertion(+) diff --git a/core/src/main/java/feast/core/grpc/CoreServiceImpl.java b/core/src/main/java/feast/core/grpc/CoreServiceImpl.java index 18b8b870fba..7a81690a582 100644 --- a/core/src/main/java/feast/core/grpc/CoreServiceImpl.java +++ b/core/src/main/java/feast/core/grpc/CoreServiceImpl.java @@ -60,6 +60,7 @@ public CoreServiceImpl( this.accessManagementService = accessManagementService; this.jobService = jobService; this.feastProperties = feastProperties; + this.statsService = statsService; } @Override From 1e12d3f8c67ed6f657b8472e16693af2162b4eea Mon Sep 17 00:00:00 2001 From: zhilingc Date: Tue, 9 Jun 2020 10:49:48 +0800 Subject: [PATCH 052/133] Fix formatting issues after rebase --- sdk/python/feast/feature_set.py | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-) diff --git a/sdk/python/feast/feature_set.py b/sdk/python/feast/feature_set.py index c94f32130e4..1e7ec132763 100644 --- a/sdk/python/feast/feature_set.py +++ b/sdk/python/feast/feature_set.py @@ -16,8 +16,6 @@ from typing import Dict, List, Optional import pandas as pd -import pyarrow as pa -import yaml from google.protobuf import json_format from google.protobuf.duration_pb2 import Duration from google.protobuf.json_format import MessageToDict, MessageToJson @@ -25,6 +23,7 @@ from pandas.api.types import is_datetime64_ns_dtype import pyarrow as pa +import yaml from feast.core.FeatureSet_pb2 import FeatureSet as FeatureSetProto from feast.core.FeatureSet_pb2 import FeatureSetMeta as FeatureSetMetaProto from feast.core.FeatureSet_pb2 import FeatureSetSpec as FeatureSetSpecProto From 6fd0874c609c938acce04ebb0d55eae7f922fa54 Mon Sep 17 00:00:00 2001 From: Joost Rothweiler Date: Tue, 9 Jun 2020 11:25:02 +0200 Subject: [PATCH 053/133] Add support for labels on feature sets (#707) * Update Python SDK to support labels * Format python code * Fix equals comparison FeatureSet * Fix bug labels returns presence and initialize labels by default as empty dict * Propagate KeyErrors labels and fields and update e2e tests with labels in constructor Co-authored-by: Joost Rothweiler <=> --- sdk/python/feast/feature.py | 5 +- sdk/python/feast/feature_set.py | 45 +++++++-- sdk/python/feast/field.py | 27 +++++- sdk/python/tests/test_client.py | 5 + sdk/python/tests/test_feature_set.py | 95 ++++++++++++++++++- tests/e2e/redis/basic-ingest-redis-serving.py | 29 ++++-- 6 files changed, 187 insertions(+), 19 deletions(-) diff --git a/sdk/python/feast/feature.py b/sdk/python/feast/feature.py index 35e0ea32aeb..054bf5ecc58 100644 --- a/sdk/python/feast/feature.py +++ b/sdk/python/feast/feature.py @@ -28,6 +28,7 @@ def to_proto(self) -> FeatureProto: return FeatureProto( name=self.name, value_type=value_type, + labels=self.labels, presence=self.presence, group_presence=self.group_presence, shape=self.shape, @@ -57,7 +58,9 @@ def from_proto(cls, feature_proto: FeatureProto): Feature object """ feature = cls( - name=feature_proto.name, dtype=ValueType(feature_proto.value_type), + name=feature_proto.name, + dtype=ValueType(feature_proto.value_type), + labels=feature_proto.labels, ) feature.update_presence_constraints(feature_proto) feature.update_shape_type(feature_proto) diff --git a/sdk/python/feast/feature_set.py b/sdk/python/feast/feature_set.py index 1e7ec132763..f4397f6cbd5 100644 --- a/sdk/python/feast/feature_set.py +++ b/sdk/python/feast/feature_set.py @@ -13,7 +13,7 @@ # limitations under the License. import warnings from collections import OrderedDict -from typing import Dict, List, Optional +from typing import Dict, List, MutableMapping, Optional import pandas as pd from google.protobuf import json_format @@ -56,6 +56,7 @@ def __init__( entities: List[Entity] = None, source: Source = None, max_age: Optional[Duration] = None, + labels: Optional[MutableMapping[str, str]] = None, ): self._name = name self._project = project @@ -68,6 +69,10 @@ def __init__( self._source = None else: self._source = source + if labels is None: + self._labels = OrderedDict() + else: + self._labels = labels self._max_age = max_age self._status = None self._created_timestamp = None @@ -84,7 +89,8 @@ def __eq__(self, other): return False if ( - self.name != other.name + self.labels != other.labels + or self.name != other.name or self.project != other.project or self.max_age != other.max_age ): @@ -217,6 +223,21 @@ def max_age(self, max_age): """ self._max_age = max_age + @property + def labels(self): + """ + Returns the labels of this feature set. This is the user defined metadata + defined as a dictionary. + """ + return self._labels + + @labels.setter + def labels(self, labels: MutableMapping[str, str]): + """ + Set the labels for this feature set + """ + self._labels = labels + @property def status(self): """ @@ -245,6 +266,18 @@ def created_timestamp(self, created_timestamp): """ self._created_timestamp = created_timestamp + def set_label(self, key: str, value: str): + """ + Sets the label value for a given key + """ + self.labels[key] = value + + def remove_label(self, key: str): + """ + Removes a label based on key + """ + del self.labels[key] + def add(self, resource): """ Adds a resource (Feature, Entity) to this Feature Set. @@ -279,11 +312,7 @@ def drop(self, name: str): Args: name: Name of Feature or Entity to be removed """ - if name not in self._fields: - raise ValueError("Could not find field " + name + ", no action taken") - if name in self._fields: - del self._fields[name] - return + del self._fields[name] def _add_fields(self, fields: List[Field]): """ @@ -796,6 +825,7 @@ def from_proto(cls, feature_set_proto: FeatureSetProto): and feature_set_proto.spec.max_age.nanos == 0 else feature_set_proto.spec.max_age ), + labels=feature_set_proto.spec.labels, source=( None if feature_set_proto.spec.source.type == 0 @@ -825,6 +855,7 @@ def to_proto(self) -> FeatureSetProto: name=self.name, project=self.project, max_age=self.max_age, + labels=self.labels, source=self.source.to_proto() if self.source is not None else None, features=[ field.to_proto() diff --git a/sdk/python/feast/field.py b/sdk/python/feast/field.py index be56823489b..6c015588b6b 100644 --- a/sdk/python/feast/field.py +++ b/sdk/python/feast/field.py @@ -11,7 +11,8 @@ # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. # See the License for the specific language governing permissions and # limitations under the License. -from typing import Union +from collections import OrderedDict +from typing import MutableMapping, Optional, Union from feast.core.FeatureSet_pb2 import EntitySpec, FeatureSpec from feast.value_type import ValueType @@ -24,11 +25,20 @@ class Field: features. """ - def __init__(self, name: str, dtype: ValueType): + def __init__( + self, + name: str, + dtype: ValueType, + labels: Optional[MutableMapping[str, str]] = None, + ): self._name = name if not isinstance(dtype, ValueType): raise ValueError("dtype is not a valid ValueType") self._dtype = dtype + if labels is None: + self._labels = OrderedDict() + else: + self._labels = labels self._presence = None self._group_presence = None self._shape = None @@ -47,7 +57,11 @@ def __init__(self, name: str, dtype: ValueType): self._time_of_day_domain = None def __eq__(self, other): - if self.name != other.name or self.dtype != other.dtype: + if ( + self.name != other.name + or self.dtype != other.dtype + or self.labels != other.labels + ): return False return True @@ -65,6 +79,13 @@ def dtype(self) -> ValueType: """ return self._dtype + @property + def labels(self) -> MutableMapping[str, str]: + """ + Getter for labels of this field + """ + return self._labels + @property def presence(self) -> schema_pb2.FeaturePresence: """ diff --git a/sdk/python/tests/test_client.py b/sdk/python/tests/test_client.py index 75d45a583fc..ee89b83737a 100644 --- a/sdk/python/tests/test_client.py +++ b/sdk/python/tests/test_client.py @@ -276,6 +276,7 @@ def test_get_feature_set(self, mocked_client, mocker): spec=FeatureSetSpecProto( name="my_feature_set", max_age=Duration(seconds=3600), + labels={"key1": "val1", "key2": "val2"}, features=[ FeatureSpecProto( name="my_feature_1", @@ -308,6 +309,10 @@ def test_get_feature_set(self, mocked_client, mocker): assert ( feature_set.name == "my_feature_set" + and "key1" in feature_set.labels + and feature_set.labels["key1"] == "val1" + and "key2" in feature_set.labels + and feature_set.labels["key2"] == "val2" and feature_set.fields["my_feature_1"].name == "my_feature_1" and feature_set.fields["my_feature_1"].dtype == ValueType.FLOAT and feature_set.fields["my_entity_1"].name == "my_entity_1" diff --git a/sdk/python/tests/test_feature_set.py b/sdk/python/tests/test_feature_set.py index 4052623e699..5f36d8fe691 100644 --- a/sdk/python/tests/test_feature_set.py +++ b/sdk/python/tests/test_feature_set.py @@ -13,6 +13,7 @@ # limitations under the License. import pathlib +from collections import OrderedDict from concurrent import futures from datetime import datetime @@ -62,7 +63,7 @@ def test_add_remove_features_success(self): assert len(fs.features) == 1 and fs.features[0].name == "my-feature-2" def test_remove_feature_failure(self): - with pytest.raises(ValueError): + with pytest.raises(KeyError): fs = FeatureSet("my-feature-set") fs.drop(name="my-feature-1") @@ -287,6 +288,98 @@ def make_tfx_schema_domain_info_inline(schema): feature.int_domain.MergeFrom(domain_ref_to_int_domain[domain_ref]) +def test_feature_set_class_contains_labels(): + fs = FeatureSet("my-feature-set", labels={"key1": "val1", "key2": "val2"}) + assert "key1" in fs.labels.keys() and fs.labels["key1"] == "val1" + assert "key2" in fs.labels.keys() and fs.labels["key2"] == "val2" + + +def test_feature_class_contains_labels(): + fs = FeatureSet("my-feature-set", labels={"key1": "val1", "key2": "val2"}) + fs.add( + Feature( + name="my-feature-1", + dtype=ValueType.INT64, + labels={"feature_key1": "feature_val1"}, + ) + ) + assert "feature_key1" in fs.features[0].labels.keys() + assert fs.features[0].labels["feature_key1"] == "feature_val1" + + +def test_feature_set_without_labels_empty_dict(): + fs = FeatureSet("my-feature-set") + assert fs.labels == OrderedDict() + assert len(fs.labels) == 0 + + +def test_feature_without_labels_empty_dict(): + f = Feature("my feature", dtype=ValueType.INT64) + assert f.labels == OrderedDict() + assert len(f.labels) == 0 + + +def test_set_label_feature_set(): + fs = FeatureSet("my-feature-set") + fs.set_label("k1", "v1") + assert fs.labels["k1"] == "v1" + + +def test_set_labels_overwrites_existing(): + fs = FeatureSet("my-feature-set") + fs.set_label("k1", "v1") + fs.set_label("k1", "v2") + assert fs.labels["k1"] == "v2" + + +def test_remove_labels_empty_failure(): + fs = FeatureSet("my-feature-set") + with pytest.raises(KeyError): + fs.remove_label("key1") + + +def test_remove_labels_invalid_key_failure(): + fs = FeatureSet("my-feature-set") + fs.set_label("k1", "v1") + with pytest.raises(KeyError): + fs.remove_label("key1") + + +def test_unequal_feature_based_on_labels(): + f1 = Feature(name="feature-1", dtype=ValueType.INT64, labels={"k1": "v1"}) + f2 = Feature(name="feature-1", dtype=ValueType.INT64, labels={"k1": "v1"}) + assert f1 == f2 + f3 = Feature(name="feature-1", dtype=ValueType.INT64) + assert f1 != f3 + f4 = Feature(name="feature-1", dtype=ValueType.INT64, labels={"k1": "notv1"}) + assert f1 != f4 + + +def test_unequal_feature_set_based_on_labels(): + fs1 = FeatureSet("my-feature-set") + fs2 = FeatureSet("my-feature-set") + assert fs1 == fs2 + fs1.set_label("k1", "v1") + fs2.set_label("k1", "v1") + assert fs1 == fs2 + fs2.set_label("k1", "unequal") + assert not fs1 == fs2 + + +def test_unequal_feature_set_other_has_no_labels(): + fs1 = FeatureSet("my-feature-set") + fs2 = FeatureSet("my-feature-set") + assert fs1 == fs2 + fs1.set_label("k1", "v1") + assert not fs1 == fs2 + + +def test_unequal_feature_other_has_no_labels(): + f1 = Feature(name="feature-1", dtype=ValueType.INT64, labels={"k1": "v1"}) + f2 = Feature(name="feature-1", dtype=ValueType.INT64) + assert f1 != f2 + + class TestFeatureSetRef: def test_from_feature_set(self): feature_set = FeatureSet("test", "test") diff --git a/tests/e2e/redis/basic-ingest-redis-serving.py b/tests/e2e/redis/basic-ingest-redis-serving.py index 31a7c8d73d7..63a3c794a5a 100644 --- a/tests/e2e/redis/basic-ingest-redis-serving.py +++ b/tests/e2e/redis/basic-ingest-redis-serving.py @@ -688,11 +688,16 @@ def get_feature_set(self, core_service_stub, name, project): @pytest.mark.run(order=51) def test_register_feature_set_with_labels(self, core_service_stub): feature_set_name = "test_feature_set_labels" - feature_set_proto = FeatureSet(feature_set_name, PROJECT_NAME).to_proto() - feature_set_proto.spec.labels[self.LABEL_KEY] = self.LABEL_VALUE + feature_set_proto = FeatureSet( + name=feature_set_name, + project=PROJECT_NAME, + labels={self.LABEL_KEY: self.LABEL_VALUE}, + ).to_proto() self.apply_feature_set(core_service_stub, feature_set_proto) - retrieved_feature_set = self.get_feature_set(core_service_stub, feature_set_name, PROJECT_NAME) + retrieved_feature_set = self.get_feature_set( + core_service_stub, feature_set_name, PROJECT_NAME + ) assert self.LABEL_KEY in retrieved_feature_set.spec.labels assert retrieved_feature_set.spec.labels[self.LABEL_KEY] == self.LABEL_VALUE @@ -701,12 +706,22 @@ def test_register_feature_set_with_labels(self, core_service_stub): @pytest.mark.run(order=52) def test_register_feature_with_labels(self, core_service_stub): feature_set_name = "test_feature_labels" - feature_set_proto = FeatureSet(feature_set_name, PROJECT_NAME, features=[Feature("rating", ValueType.INT64)]) \ - .to_proto() - feature_set_proto.spec.features[0].labels[self.LABEL_KEY] = self.LABEL_VALUE + feature_set_proto = FeatureSet( + name=feature_set_name, + project=PROJECT_NAME, + features=[ + Feature( + name="rating", + dtype=ValueType.INT64, + labels={self.LABEL_KEY: self.LABEL_VALUE}, + ) + ], + ).to_proto() self.apply_feature_set(core_service_stub, feature_set_proto) - retrieved_feature_set = self.get_feature_set(core_service_stub, feature_set_name, PROJECT_NAME) + retrieved_feature_set = self.get_feature_set( + core_service_stub, feature_set_name, PROJECT_NAME + ) retrieved_feature = retrieved_feature_set.spec.features[0] assert self.LABEL_KEY in retrieved_feature.labels From d23f0276366e51e59c3d24fa1581a1e39854d853 Mon Sep 17 00:00:00 2001 From: Willem Pienaar <6728866+woop@users.noreply.github.com> Date: Tue, 9 Jun 2020 21:00:37 +0800 Subject: [PATCH 054/133] Add statistics to gitbook --- docs/SUMMARY.md | 2 ++ 1 file changed, 2 insertions(+) diff --git a/docs/SUMMARY.md b/docs/SUMMARY.md index aaea553e95a..ae4a61f3e2e 100644 --- a/docs/SUMMARY.md +++ b/docs/SUMMARY.md @@ -24,11 +24,13 @@ * [Data ingestion](user-guide/data-ingestion.md) * [Stores](user-guide/stores.md) * [Feature retrieval](user-guide/feature-retrieval.md) +* [Statistics](user-guide/statistics.md) ## Tutorials * [Basic Tutorial](https://github.com/feast-dev/feast/blob/master/examples/basic/basic.ipynb) * [Churn Prediction \(XGBoost\)](https://github.com/feast-dev/feast/blob/master/examples/feast-xgboost-churn-prediction-tutorial/Telecom%20Customer%20Churn%20Prediction%20%28with%20Feast%20and%20XGBoost%29.ipynb) +* [Feast Statistics with TFDV and Facets](https://github.com/feast-dev/feast/blob/master/examples/statistics/Historical%20Feature%20Statistics%20with%20Feast%2C%20TFDV%20and%20Facets.ipynb) ## Administration From 577edca8c8f8c082a19f2b75a525b094771f9536 Mon Sep 17 00:00:00 2001 From: Oleksii Moskalenko Date: Tue, 9 Jun 2020 17:15:02 +0300 Subject: [PATCH 055/133] Upgrade ingestion to allow for in-flight updates to feature sets for sinks (#757) * In-Flight update of FeatureSetSpecs in Sinks (BQ & redis). BQ table schema update as part of data load * constant Co-authored-by: Willem Pienaar --- .prow/config.yaml | 4 +- .../java/feast/core/job/JobUpdateTask.java | 8 +- .../core/job/dataflow/DataflowJobManager.java | 32 +- .../core/job/direct/DirectRunnerConfig.java | 3 + .../main/resources/application-override.yaml | 0 core/src/main/resources/application.yml | 3 +- infra/scripts/setup-common-functions.sh | 2 +- .../scripts/test-end-to-end-batch-dataflow.sh | 12 +- infra/scripts/test-end-to-end-batch.sh | 17 +- .../values-end-to-end-batch-dataflow.yaml | 11 +- .../main/java/feast/ingestion/ImportJob.java | 19 +- .../ingestion/transform/ReadFromSource.java | 6 +- .../metrics/WriteSuccessMetricsTransform.java | 9 +- .../java/feast/ingestion/utils/StoreUtil.java | 4 +- pom.xml | 5 +- protos/feast/core/Runner.proto | 4 + protos/feast/core/Store.proto | 1 + serving/src/main/resources/application.yml | 2 + storage/connectors/bigquery/pom.xml | 8 +- .../bigquery/writer/BigQueryFeatureSink.java | 204 +++------ .../bigquery/writer/BigQuerySinkHelpers.java | 62 +++ .../bigquery/writer/BigQueryWrite.java | 268 +++++++++--- .../writer/FeatureSetSpecToTableSchema.java | 231 ++++++++++ .../bigquery/writer/GetTableDestination.java | 50 --- .../io/gcp/bigquery/BatchLoadsWithResult.java | 291 ++++++++++++ .../com/google/cloud/bigquery/FakeTable.java | 37 ++ .../bigquery/writer/BigQuerySinkTest.java | 413 ++++++++++++++++++ storage/connectors/redis/pom.xml | 5 + .../redis/writer/RedisCustomIO.java | 126 +++--- .../redis/writer/RedisFeatureSink.java | 22 +- .../writer/RedisClusterFeatureSinkTest.java | 8 +- .../redis/writer/RedisFeatureSinkTest.java | 11 +- tests/e2e/bq/bq-batch-retrieval.py | 286 +++++++----- 33 files changed, 1655 insertions(+), 509 deletions(-) create mode 100644 core/src/main/resources/application-override.yaml create mode 100644 storage/connectors/bigquery/src/main/java/feast/storage/connectors/bigquery/writer/BigQuerySinkHelpers.java create mode 100644 storage/connectors/bigquery/src/main/java/feast/storage/connectors/bigquery/writer/FeatureSetSpecToTableSchema.java delete mode 100644 storage/connectors/bigquery/src/main/java/feast/storage/connectors/bigquery/writer/GetTableDestination.java create mode 100644 storage/connectors/bigquery/src/main/java/org/apache/beam/sdk/io/gcp/bigquery/BatchLoadsWithResult.java create mode 100644 storage/connectors/bigquery/src/test/java/com/google/cloud/bigquery/FakeTable.java create mode 100644 storage/connectors/bigquery/src/test/java/feast/storage/connectors/bigquery/writer/BigQuerySinkTest.java diff --git a/.prow/config.yaml b/.prow/config.yaml index 09f791ef4a4..54f03226343 100644 --- a/.prow/config.yaml +++ b/.prow/config.yaml @@ -255,8 +255,8 @@ presubmits: secret: secretName: feast-e2e-service-account - name: docker-socket - hostPath: - path: /var/run/docker.sock + hostPath: + path: /var/run/docker.sock containers: - image: google/cloud-sdk:273.0.0 command: ["infra/scripts/test-end-to-end-batch-dataflow.sh"] diff --git a/core/src/main/java/feast/core/job/JobUpdateTask.java b/core/src/main/java/feast/core/job/JobUpdateTask.java index 056da34b3aa..0c79bb620db 100644 --- a/core/src/main/java/feast/core/job/JobUpdateTask.java +++ b/core/src/main/java/feast/core/job/JobUpdateTask.java @@ -38,6 +38,7 @@ import java.util.concurrent.TimeoutException; import lombok.Getter; import lombok.extern.slf4j.Slf4j; +import org.apache.beam.vendor.guava.v26_0_jre.com.google.common.hash.Hashing; /** * JobUpdateTask is a callable that starts or updates a job given a set of featureSetSpecs, as well @@ -172,8 +173,11 @@ private Job updateStatus(Job job) { String createJobId(String sourceId, String storeName) { String dateSuffix = String.valueOf(Instant.now().toEpochMilli()); - String sourceIdTrunc = sourceId.split("/")[0].toLowerCase(); - String jobId = String.format("%s-to-%s", sourceIdTrunc, storeName) + dateSuffix; + String[] sourceParts = sourceId.split("/", 2); + String sourceType = sourceParts[0].toLowerCase(); + String sourceHash = + Hashing.murmur3_128().hashUnencodedChars(sourceParts[1]).toString().substring(0, 10); + String jobId = String.format("%s-%s-to-%s-%s", sourceType, sourceHash, storeName, dateSuffix); return jobId.replaceAll("_", "-"); } diff --git a/core/src/main/java/feast/core/job/dataflow/DataflowJobManager.java b/core/src/main/java/feast/core/job/dataflow/DataflowJobManager.java index 2c3da255f5b..50a23224078 100644 --- a/core/src/main/java/feast/core/job/dataflow/DataflowJobManager.java +++ b/core/src/main/java/feast/core/job/dataflow/DataflowJobManager.java @@ -137,38 +137,15 @@ public Job startJob(Job job) { } /** - * Update an existing Dataflow job. + * Drain existing job. Replacement will be created on next run (when job gracefully stop) * * @param job job of target job to change - * @return Dataflow-specific job id + * @return same job as input */ @Override public Job updateJob(Job job) { - try { - List featureSetProtos = new ArrayList<>(); - for (FeatureSet featureSet : job.getFeatureSets()) { - featureSetProtos.add(featureSet.toProto()); - } - - String extId = - submitDataflowJob( - job.getId(), - featureSetProtos, - job.getSource().toProto(), - job.getStore().toProto(), - true); - - job.setExtId(extId); - job.setStatus(JobStatus.PENDING); - return job; - } catch (InvalidProtocolBufferException e) { - log.error(e.getMessage()); - throw new IllegalArgumentException( - String.format( - "DataflowJobManager failed to UPDATE job with id '%s' because the job" - + "has an invalid spec. Please check the FeatureSet, Source and Store specs. Actual error message: %s", - job.getId(), e.getMessage())); - } + abortJob(job.getExtId()); + return job; } /** @@ -283,7 +260,6 @@ private ImportOptions getPipelineOptions( pipelineOptions.setJobName(jobName); pipelineOptions.setFilesToStage( detectClassPathResourcesToStage(DataflowRunner.class.getClassLoader())); - if (metrics.isEnabled()) { pipelineOptions.setMetricsExporterType(metrics.getType()); if (metrics.getType().equals("statsd")) { diff --git a/core/src/main/java/feast/core/job/direct/DirectRunnerConfig.java b/core/src/main/java/feast/core/job/direct/DirectRunnerConfig.java index c7459cb9d16..e48ef15bbed 100644 --- a/core/src/main/java/feast/core/job/direct/DirectRunnerConfig.java +++ b/core/src/main/java/feast/core/job/direct/DirectRunnerConfig.java @@ -29,8 +29,11 @@ public class DirectRunnerConfig extends RunnerConfig { /* BigQuery table specification, e.g. PROJECT_ID:DATASET_ID.PROJECT_ID */ public String deadletterTableSpec; + public String tempLocation; + public DirectRunnerConfig(DirectRunnerConfigOptions runnerConfigOptions) { this.deadletterTableSpec = runnerConfigOptions.getDeadLetterTableSpec(); this.targetParallelism = runnerConfigOptions.getTargetParallelism(); + this.tempLocation = runnerConfigOptions.getTempLocation(); } } diff --git a/core/src/main/resources/application-override.yaml b/core/src/main/resources/application-override.yaml new file mode 100644 index 00000000000..e69de29bb2d diff --git a/core/src/main/resources/application.yml b/core/src/main/resources/application.yml index 51395cf6449..5dc5698e889 100644 --- a/core/src/main/resources/application.yml +++ b/core/src/main/resources/application.yml @@ -38,7 +38,8 @@ feast: runners: - name: direct type: DirectRunner - options: {} + options: + tempLocation: gs://bucket/tempLocation - name: dataflow type: DataflowRunner diff --git a/infra/scripts/setup-common-functions.sh b/infra/scripts/setup-common-functions.sh index e158e0f5672..44d900b1a9d 100755 --- a/infra/scripts/setup-common-functions.sh +++ b/infra/scripts/setup-common-functions.sh @@ -83,7 +83,7 @@ start_feast_core() { if [ -n "$1" ]; then echo "Custom Spring application.yml location provided: $1" - export CONFIG_ARG="--spring.config.location=file://$1" + export CONFIG_ARG="--spring.config.location=classpath:/application.yml,file://$1" fi nohup java -jar core/target/feast-core-$FEAST_BUILD_VERSION.jar $CONFIG_ARG &>/var/log/feast-core.log & diff --git a/infra/scripts/test-end-to-end-batch-dataflow.sh b/infra/scripts/test-end-to-end-batch-dataflow.sh index 4cb7f1998b4..9ad29185570 100755 --- a/infra/scripts/test-end-to-end-batch-dataflow.sh +++ b/infra/scripts/test-end-to-end-batch-dataflow.sh @@ -9,7 +9,7 @@ test -z ${GCLOUD_PROJECT} && GCLOUD_PROJECT="kf-feast" test -z ${GCLOUD_REGION} && GCLOUD_REGION="us-central1" test -z ${GCLOUD_NETWORK} && GCLOUD_NETWORK="default" test -z ${GCLOUD_SUBNET} && GCLOUD_SUBNET="default" -test -z ${TEMP_BUCKET} && TEMP_BUCKET="feast-templocation-kf-feast" +test -z ${TEMP_BUCKET} && TEMP_BUCKET="kf-feast-dataflow-temp" test -z ${K8_CLUSTER_NAME} && K8_CLUSTER_NAME="feast-e2e-dataflow" test -z ${HELM_RELEASE_NAME} && HELM_RELEASE_NAME="pr-$PULL_NUMBER" test -z ${HELM_COMMON_NAME} && HELM_COMMON_NAME="deps" @@ -124,6 +124,7 @@ Helm install common parts (kafka, redis, etc) --set "feast-core.enabled=false" \ --set "feast-online-serving.enabled=false" \ --set "feast-batch-serving.enabled=false" \ + --set "postgresql.enabled=false" "$HELM_COMMON_NAME" . } @@ -149,7 +150,6 @@ Helm install feast cd $ORIGINAL_DIR/infra/charts/feast helm install --wait --timeout 300s --debug --values="values-end-to-end-batch-dataflow-updated.yaml" \ - --set "postgresql.enabled=false" \ --set "kafka.enabled=false" \ --set "redis.enabled=false" \ --set "prometheus-statsd-exporter.enabled=false" \ @@ -172,6 +172,8 @@ function clean() { # Uninstall helm release before clearing PVCs helm uninstall ${HELM_RELEASE_NAME} + kubectl delete pvc data-${HELM_RELEASE_NAME}-postgresql-0 + # Stop Dataflow jobs from retrieved Dataflow job ids in ingesting_jobs.txt if [ -f ingesting_jobs.txt ]; then while read line @@ -270,11 +272,11 @@ LOGS_ARTIFACT_PATH=/logs/artifacts cd $ORIGINAL_DIR/tests/e2e -core_ip=$(kubectl get -o jsonpath="{.spec.clusterIP}" service ${HELM_RELEASE_NAME}-feast-core) -serving_ip=$(kubectl get -o jsonpath="{.spec.clusterIP}" service ${HELM_RELEASE_NAME}-feast-batch-serving) +core_ip=$(kubectl get -o jsonpath="{.status.loadBalancer.ingress[0].ip}" service ${HELM_RELEASE_NAME}-feast-core) +serving_ip=$(kubectl get -o jsonpath="{.status.loadBalancer.ingress[0].ip}" service ${HELM_RELEASE_NAME}-feast-batch-serving) set +e -pytest -v bq/bq-batch-retrieval.py -m dataflow_runner --core_url "$core_ip:6565" --serving_url "$serving_ip:6566" --gcs_path "gs://${TEMP_BUCKET}/" --junitxml=${LOGS_ARTIFACT_PATH}/python-sdk-test-report.xml +pytest -s -v bq/bq-batch-retrieval.py -m dataflow_runner --core_url "$core_ip:6565" --serving_url "$serving_ip:6566" --gcs_path "gs://${TEMP_BUCKET}/" --junitxml=${LOGS_ARTIFACT_PATH}/python-sdk-test-report.xml TEST_EXIT_CODE=$? if [[ ${TEST_EXIT_CODE} != 0 ]]; then diff --git a/infra/scripts/test-end-to-end-batch.sh b/infra/scripts/test-end-to-end-batch.sh index c225a8e2991..6763e417095 100755 --- a/infra/scripts/test-end-to-end-batch.sh +++ b/infra/scripts/test-end-to-end-batch.sh @@ -56,8 +56,20 @@ else echo "[DEBUG] Skipping building jars" fi -export FEAST_JOBS_POLLING_INTERVAL_MILLISECONDS=10000 -start_feast_core +cat < /tmp/core.warehouse.application.yml +feast: + jobs: + polling_interval_milliseconds: 10000 + active_runner: direct + runners: + - name: direct + type: DirectRunner + options: + tempLocation: gs://${TEMP_BUCKET}/tempLocation + +EOF + +start_feast_core /tmp/core.warehouse.application.yml DATASET_NAME=feast_$(date +%s) bq --location=US --project_id=${GOOGLE_CLOUD_PROJECT} mk \ @@ -87,6 +99,7 @@ feast: staging_location: ${JOBS_STAGING_LOCATION} initial_retry_delay_seconds: 1 total_timeout_seconds: 21600 + write_triggering_frequency_seconds: 1 subscriptions: - name: "*" project: "*" diff --git a/infra/scripts/test-templates/values-end-to-end-batch-dataflow.yaml b/infra/scripts/test-templates/values-end-to-end-batch-dataflow.yaml index 44a377db5fa..345d5234184 100644 --- a/infra/scripts/test-templates/values-end-to-end-batch-dataflow.yaml +++ b/infra/scripts/test-templates/values-end-to-end-batch-dataflow.yaml @@ -5,12 +5,12 @@ feast-core: enabled: true postgresql: existingSecret: feast-postgresql + service: + type: LoadBalancer image: tag: $IMAGE_TAG + logLevel: INFO application-override.yaml: - spring: - datasource: - url: jdbc:postgresql://$HELM_COMMON_NAME-postgresql:5432/postgres feast: stream: options: @@ -43,6 +43,8 @@ feast-online-serving: enabled: true image: tag: $IMAGE_TAG + service: + type: LoadBalancer application-override.yaml: feast: active_store: online @@ -66,6 +68,8 @@ feast-batch-serving: tag: $IMAGE_TAG gcpServiceAccount: enabled: true + service: + type: LoadBalancer application-override.yaml: feast: active_store: historical @@ -80,6 +84,7 @@ feast-batch-serving: staging_location: gs://$TEMP_BUCKET/stagingLocation initial_retry_delay_seconds: 3 total_timeout_seconds: 21600 + write_triggering_frequency_seconds: 1 subscriptions: - name: "*" project: "*" diff --git a/ingestion/src/main/java/feast/ingestion/ImportJob.java b/ingestion/src/main/java/feast/ingestion/ImportJob.java index 16efe11f55f..5cef8e57449 100644 --- a/ingestion/src/main/java/feast/ingestion/ImportJob.java +++ b/ingestion/src/main/java/feast/ingestion/ImportJob.java @@ -46,8 +46,8 @@ import org.apache.beam.sdk.PipelineResult; import org.apache.beam.sdk.options.PipelineOptionsFactory; import org.apache.beam.sdk.options.PipelineOptionsValidator; -import org.apache.beam.sdk.values.PCollectionTuple; -import org.apache.beam.sdk.values.TupleTag; +import org.apache.beam.sdk.transforms.*; +import org.apache.beam.sdk.values.*; import org.slf4j.Logger; public class ImportJob { @@ -104,7 +104,20 @@ public static PipelineResult runPipeline(ImportOptions options) throws IOExcepti featureSetSpecsByKey.put(ref, fs.getSpec()); }); - FeatureSink featureSink = getFeatureSink(store, featureSetSpecsByKey); + PCollection> staticSpecs = + pipeline + .apply(Create.of("")) + .apply( + "TemporarySpecSource", + ParDo.of( + new DoFn>() { + @ProcessElement + public void process(ProcessContext c) { + featureSetSpecsByKey.forEach((key, value) -> c.output(KV.of(key, value))); + } + })); + + FeatureSink featureSink = getFeatureSink(store, staticSpecs); // TODO: make the source part of the job initialisation options Source source = subscribedFeatureSets.get(0).getSpec().getSource(); diff --git a/ingestion/src/main/java/feast/ingestion/transform/ReadFromSource.java b/ingestion/src/main/java/feast/ingestion/transform/ReadFromSource.java index 45253735141..3fd01cf4f1e 100644 --- a/ingestion/src/main/java/feast/ingestion/transform/ReadFromSource.java +++ b/ingestion/src/main/java/feast/ingestion/transform/ReadFromSource.java @@ -23,6 +23,8 @@ import feast.proto.core.SourceProto.SourceType; import feast.proto.types.FeatureRowProto.FeatureRow; import feast.storage.api.writer.FailedElement; +import java.util.Arrays; +import java.util.stream.Collectors; import org.apache.beam.sdk.io.kafka.KafkaIO; import org.apache.beam.sdk.transforms.PTransform; import org.apache.beam.sdk.transforms.ParDo; @@ -97,6 +99,8 @@ public PCollectionTuple expand(PBegin input) { } private String generateConsumerGroupId(String jobName) { - return "feast_import_job_" + jobName; + String jobNameWithoutTimestamp = + Arrays.stream(jobName.split("-")).limit(4).collect(Collectors.joining("-")); + return "feast_import_job_" + jobNameWithoutTimestamp; } } diff --git a/ingestion/src/main/java/feast/ingestion/transform/metrics/WriteSuccessMetricsTransform.java b/ingestion/src/main/java/feast/ingestion/transform/metrics/WriteSuccessMetricsTransform.java index e75debf4a93..31915ea6701 100644 --- a/ingestion/src/main/java/feast/ingestion/transform/metrics/WriteSuccessMetricsTransform.java +++ b/ingestion/src/main/java/feast/ingestion/transform/metrics/WriteSuccessMetricsTransform.java @@ -68,10 +68,11 @@ public PDone expand(PCollection input) { input .apply( "FixedWindow", - Window.into( - FixedWindows.of( - Duration.standardSeconds( - options.getWindowSizeInSecForFeatureValueMetric())))) + Window.into( + FixedWindows.of( + Duration.standardSeconds( + options.getWindowSizeInSecForFeatureValueMetric()))) + .withAllowedLateness(Duration.millis(0))) .apply( "ConvertToKV_FeatureSetRefToFeatureRow", ParDo.of( diff --git a/ingestion/src/main/java/feast/ingestion/utils/StoreUtil.java b/ingestion/src/main/java/feast/ingestion/utils/StoreUtil.java index 4efec994f7e..429159a8a6a 100644 --- a/ingestion/src/main/java/feast/ingestion/utils/StoreUtil.java +++ b/ingestion/src/main/java/feast/ingestion/utils/StoreUtil.java @@ -28,6 +28,8 @@ import feast.storage.connectors.redis.writer.RedisFeatureSink; import java.util.HashMap; import java.util.Map; +import org.apache.beam.sdk.values.KV; +import org.apache.beam.sdk.values.PCollection; import org.slf4j.Logger; // TODO: Create partitioned table by default @@ -79,7 +81,7 @@ public class StoreUtil { } public static FeatureSink getFeatureSink( - Store store, Map featureSetSpecs) { + Store store, PCollection> featureSetSpecs) { StoreType storeType = store.getType(); switch (storeType) { case REDIS_CLUSTER: diff --git a/pom.xml b/pom.xml index 77db4d21373..4d1be1f2a6c 100644 --- a/pom.xml +++ b/pom.xml @@ -49,7 +49,7 @@ 3.10.0 3.10.0 2.0.9.RELEASE - 2.16.0 + 2.18.0 1.91.0 0.8.0 1.9.10 @@ -399,6 +399,9 @@ 1.7 + + src/main/java/**/BatchLoadsWithResult.java + diff --git a/protos/feast/core/Runner.proto b/protos/feast/core/Runner.proto index 91c1e99485f..1591ecb2bf3 100644 --- a/protos/feast/core/Runner.proto +++ b/protos/feast/core/Runner.proto @@ -31,6 +31,10 @@ message DirectRunnerConfigOptions { /* BigQuery table specification, e.g. PROJECT_ID:DATASET_ID.PROJECT_ID */ string deadLetterTableSpec = 2; + + // A pipeline level default location for storing temporary files. + // Support Google Cloud Storage locations or local path + string tempLocation = 3; } message DataflowRunnerConfigOptions { diff --git a/protos/feast/core/Store.proto b/protos/feast/core/Store.proto index 7279c484eea..be9750cfde4 100644 --- a/protos/feast/core/Store.proto +++ b/protos/feast/core/Store.proto @@ -116,6 +116,7 @@ message Store { string staging_location = 3; int32 initial_retry_delay_seconds = 4; int32 total_timeout_seconds = 5; + int32 write_triggering_frequency_seconds = 6; } message CassandraConfig { diff --git a/serving/src/main/resources/application.yml b/serving/src/main/resources/application.yml index 9158ee7fa58..b5eff65391b 100644 --- a/serving/src/main/resources/application.yml +++ b/serving/src/main/resources/application.yml @@ -40,6 +40,8 @@ feast: initial_retry_delay_seconds: 1 # BigQuery timeout for retrieval jobs total_timeout_seconds: 21600 + # BigQuery sink write frequency + write_triggering_frequency_seconds: 600 subscriptions: - name: "*" project: "*" diff --git a/storage/connectors/bigquery/pom.xml b/storage/connectors/bigquery/pom.xml index fab3739c43c..32c6dda4810 100644 --- a/storage/connectors/bigquery/pom.xml +++ b/storage/connectors/bigquery/pom.xml @@ -72,6 +72,13 @@ test + + org.slf4j + slf4j-simple + 1.7.30 + test + + org.apache.beam beam-runners-direct-java @@ -90,5 +97,4 @@ test - diff --git a/storage/connectors/bigquery/src/main/java/feast/storage/connectors/bigquery/writer/BigQueryFeatureSink.java b/storage/connectors/bigquery/src/main/java/feast/storage/connectors/bigquery/writer/BigQueryFeatureSink.java index 8dffe629769..d088e859794 100644 --- a/storage/connectors/bigquery/src/main/java/feast/storage/connectors/bigquery/writer/BigQueryFeatureSink.java +++ b/storage/connectors/bigquery/src/main/java/feast/storage/connectors/bigquery/writer/BigQueryFeatureSink.java @@ -16,42 +16,43 @@ */ package feast.storage.connectors.bigquery.writer; +import com.google.api.services.bigquery.model.TableSchema; import com.google.auto.value.AutoValue; import com.google.cloud.bigquery.*; -import com.google.common.collect.ImmutableMap; import feast.proto.core.FeatureSetProto; import feast.proto.core.StoreProto.Store.BigQueryConfig; import feast.proto.types.FeatureRowProto; import feast.storage.api.writer.FeatureSink; import feast.storage.api.writer.WriteResult; -import feast.storage.connectors.bigquery.common.TypeUtil; -import java.util.ArrayList; -import java.util.List; import java.util.Map; -import org.apache.beam.repackaged.core.org.apache.commons.lang3.tuple.Pair; +import javax.annotation.Nullable; +import org.apache.beam.sdk.coders.KvCoder; +import org.apache.beam.sdk.coders.StringUtf8Coder; +import org.apache.beam.sdk.io.gcp.bigquery.BigQueryServices; +import org.apache.beam.sdk.options.ValueProvider; import org.apache.beam.sdk.transforms.PTransform; +import org.apache.beam.sdk.transforms.ParDo; +import org.apache.beam.sdk.transforms.View; +import org.apache.beam.sdk.values.KV; import org.apache.beam.sdk.values.PCollection; -import org.slf4j.Logger; +import org.apache.beam.sdk.values.PCollectionView; +import org.joda.time.Duration; @AutoValue public abstract class BigQueryFeatureSink implements FeatureSink { - private static final Logger log = org.slf4j.LoggerFactory.getLogger(BigQueryFeatureSink.class); - - // Column description for reserved fields - public static final String BIGQUERY_EVENT_TIMESTAMP_FIELD_DESCRIPTION = - "Event time for the FeatureRow"; - public static final String BIGQUERY_CREATED_TIMESTAMP_FIELD_DESCRIPTION = - "Processing time of the FeatureRow ingestion in Feast\""; - public static final String BIGQUERY_INGESTION_ID_FIELD_DESCRIPTION = - "Unique id identifying groups of rows that have been ingested together"; - public static final String BIGQUERY_JOB_ID_FIELD_DESCRIPTION = - "Feast import job ID for the FeatureRow"; - public abstract String getProjectId(); public abstract String getDatasetId(); - public abstract BigQuery getBigQuery(); + public abstract Duration getTriggeringFrequency(); + + @Nullable + public abstract BigQueryServices getBQTestServices(); + + @Nullable + public abstract ValueProvider getBQClient(); + + public abstract PCollection> getFeatureSetSpecs(); /** * Initialize a {@link BigQueryFeatureSink.Builder} from a {@link BigQueryConfig}. This method @@ -59,15 +60,31 @@ public abstract class BigQueryFeatureSink implements FeatureSink { * your own client. * * @param config {@link BigQueryConfig} - * @param featureSetSpecs + * @param featureSetSpecs input stream of featureSet specs * @return {@link BigQueryFeatureSink.Builder} */ public static FeatureSink fromConfig( - BigQueryConfig config, Map featureSetSpecs) { - return builder() + BigQueryConfig config, + PCollection> featureSetSpecs) { + return BigQueryFeatureSink.builder() .setDatasetId(config.getDatasetId()) .setProjectId(config.getProjectId()) - .setBigQuery(BigQueryOptions.getDefaultInstance().getService()) + .setFeatureSetSpecs(featureSetSpecs) + .setBQTestServices(null) + .setBQClient( + new ValueProvider() { + @Override + public BigQuery get() { + return BigQueryOptions.getDefaultInstance().getService(); + } + + @Override + public boolean isAccessible() { + return true; + } + }) + .setTriggeringFrequency( + Duration.standardSeconds(config.getWriteTriggeringFrequencySeconds())) .build(); } @@ -82,133 +99,36 @@ public abstract static class Builder { public abstract Builder setDatasetId(String datasetId); - public abstract Builder setBigQuery(BigQuery bigQuery); + public abstract Builder setTriggeringFrequency(Duration triggeringFrequency); + + public abstract Builder setBQTestServices(BigQueryServices bigQueryServices); + + public abstract Builder setBQClient(ValueProvider bigQueryOptions); + + public abstract Builder setFeatureSetSpecs( + PCollection> featureSetSpecs); public abstract BigQueryFeatureSink build(); } /** @param featureSet Feature set to be written */ @Override - public void prepareWrite(FeatureSetProto.FeatureSet featureSet) { - BigQuery bigquery = getBigQuery(); - FeatureSetProto.FeatureSetSpec featureSetSpec = featureSet.getSpec(); - - DatasetId datasetId = DatasetId.of(getProjectId(), getDatasetId()); - if (bigquery.getDataset(datasetId) == null) { - log.info( - "Creating dataset '{}' in project '{}'", datasetId.getDataset(), datasetId.getProject()); - bigquery.create(DatasetInfo.of(datasetId)); - } - String tableName = - String.format("%s_%s", featureSetSpec.getProject(), featureSetSpec.getName()) - .replaceAll("-", "_"); - TableId tableId = TableId.of(datasetId.getProject(), datasetId.getDataset(), tableName); - - Table table = bigquery.getTable(tableId); - TableDefinition tableDefinition = createBigQueryTableDefinition(table, featureSet.getSpec()); - TableInfo tableInfo = TableInfo.of(tableId, tableDefinition); - if (table != null) { - log.info( - "Updating and writing to existing BigQuery table '{}:{}.{}'", - datasetId.getProject(), - datasetId.getDataset(), - tableName); - bigquery.update(tableInfo); - return; - } - - log.info( - "Creating table '{}' in dataset '{}' in project '{}'", - tableId.getTable(), - datasetId.getDataset(), - datasetId.getProject()); - bigquery.create(tableInfo); - } + public void prepareWrite(FeatureSetProto.FeatureSet featureSet) {} @Override public PTransform, WriteResult> writer() { - return new BigQueryWrite(DatasetId.of(getProjectId(), getDatasetId())); - } - /** - * Creates a BigQuery {@link TableDefinition} based on the provided FeatureSetSpec and the - * existing table, if any. If a table already exists, existing fields will be retained, and new - * fields present in the feature set will be appended to the existing FieldsList. - * - * @param existingTable existing {@link Table} retrieved using bigquery.GetTable(). If the table - * does not exist, will be null. - * @param spec FeatureSet spec that this table is for - * @return {@link TableDefinition} containing all tombstoned and active fields. - */ - private TableDefinition createBigQueryTableDefinition( - Table existingTable, FeatureSetProto.FeatureSetSpec spec) { - List fields = new ArrayList<>(); - log.info("Table will have the following fields:"); - - for (FeatureSetProto.EntitySpec entitySpec : spec.getEntitiesList()) { - Field.Builder builder = - Field.newBuilder( - entitySpec.getName(), TypeUtil.toStandardSqlType(entitySpec.getValueType())); - if (entitySpec.getValueType().name().toLowerCase().endsWith("_list")) { - builder.setMode(Field.Mode.REPEATED); - } - Field field = builder.build(); - log.info("- {}", field.toString()); - fields.add(field); - } - for (FeatureSetProto.FeatureSpec featureSpec : spec.getFeaturesList()) { - Field.Builder builder = - Field.newBuilder( - featureSpec.getName(), TypeUtil.toStandardSqlType(featureSpec.getValueType())); - if (featureSpec.getValueType().name().toLowerCase().endsWith("_list")) { - builder.setMode(Field.Mode.REPEATED); - } - Field field = builder.build(); - log.info("- {}", field.toString()); - fields.add(field); - } - - // Refer to protos/feast/core/Store.proto for reserved fields in BigQuery. - Map> - reservedFieldNameToPairOfStandardSQLTypeAndDescription = - ImmutableMap.of( - "event_timestamp", - Pair.of(StandardSQLTypeName.TIMESTAMP, BIGQUERY_EVENT_TIMESTAMP_FIELD_DESCRIPTION), - "created_timestamp", - Pair.of( - StandardSQLTypeName.TIMESTAMP, BIGQUERY_CREATED_TIMESTAMP_FIELD_DESCRIPTION), - "ingestion_id", - Pair.of(StandardSQLTypeName.STRING, BIGQUERY_INGESTION_ID_FIELD_DESCRIPTION), - "job_id", - Pair.of(StandardSQLTypeName.STRING, BIGQUERY_JOB_ID_FIELD_DESCRIPTION)); - for (Map.Entry> entry : - reservedFieldNameToPairOfStandardSQLTypeAndDescription.entrySet()) { - Field field = - Field.newBuilder(entry.getKey(), entry.getValue().getLeft()) - .setDescription(entry.getValue().getRight()) - .build(); - log.info("- {}", field.toString()); - fields.add(field); - } - - TimePartitioning timePartitioning = - TimePartitioning.newBuilder(TimePartitioning.Type.DAY).setField("event_timestamp").build(); - log.info("Table partitioning: " + timePartitioning.toString()); - - List fieldsList = new ArrayList<>(); - if (existingTable != null) { - Schema existingSchema = existingTable.getDefinition().getSchema(); - fieldsList.addAll(existingSchema.getFields()); - } - - for (Field field : fields) { - if (!fieldsList.contains(field)) { - fieldsList.add(field); - } - } - - return StandardTableDefinition.newBuilder() - .setTimePartitioning(timePartitioning) - .setSchema(Schema.of(FieldList.of(fieldsList))) - .build(); + PCollection> fs = getFeatureSetSpecs(); + PCollectionView>> schemasView = + fs.apply( + ParDo.of( + new FeatureSetSpecToTableSchema( + DatasetId.of(getProjectId(), getDatasetId()), getBQClient()))) + .setCoder( + KvCoder.of(StringUtf8Coder.of(), FeatureSetSpecToTableSchema.TableSchemaCoder.of())) + .apply(View.asMultimap()); + + return new BigQueryWrite(DatasetId.of(getProjectId(), getDatasetId()), schemasView) + .withTriggeringFrequency(getTriggeringFrequency()) + .withTestServices(getBQTestServices()); } } diff --git a/storage/connectors/bigquery/src/main/java/feast/storage/connectors/bigquery/writer/BigQuerySinkHelpers.java b/storage/connectors/bigquery/src/main/java/feast/storage/connectors/bigquery/writer/BigQuerySinkHelpers.java new file mode 100644 index 00000000000..e07cd6c637b --- /dev/null +++ b/storage/connectors/bigquery/src/main/java/feast/storage/connectors/bigquery/writer/BigQuerySinkHelpers.java @@ -0,0 +1,62 @@ +/* + * SPDX-License-Identifier: Apache-2.0 + * Copyright 2018-2020 The Feast Authors + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package feast.storage.connectors.bigquery.writer; + +import com.google.api.services.bigquery.model.TimePartitioning; +import com.google.cloud.bigquery.DatasetId; +import org.apache.beam.sdk.io.gcp.bigquery.TableDestination; + +public class BigQuerySinkHelpers { + + public static final String DEFAULT_PROJECT_NAME = "default"; + + /** + * Generating BQ table destination from dataset reference and featuresSet's project and name. If + * project is undefined "default" would be selected + * + * @param dataset {@link DatasetId} reference to bq project and dataset + * @param featureSetKey Feature Set reference with format <project>/<feature-set-name> + * @return {@link TableDestination} + */ + public static TableDestination getTableDestination(DatasetId dataset, String featureSetKey) { + String[] splitName = featureSetKey.split("/"); + String projectName, setName; + + if (splitName.length == 2) { + projectName = splitName[0]; + setName = splitName[1]; + } else { + projectName = DEFAULT_PROJECT_NAME; + setName = splitName[0]; + } + + TimePartitioning timePartitioning = + new TimePartitioning() + .setType("DAY") + .setField(FeatureRowToTableRow.getEventTimestampColumn()); + + return new TableDestination( + String.format( + "%s:%s.%s_%s", + dataset.getProject(), + dataset.getDataset(), + projectName.replaceAll("-", "_"), + setName.replaceAll("-", "_")), + String.format("Feast table for %s", featureSetKey), + timePartitioning); + } +} diff --git a/storage/connectors/bigquery/src/main/java/feast/storage/connectors/bigquery/writer/BigQueryWrite.java b/storage/connectors/bigquery/src/main/java/feast/storage/connectors/bigquery/writer/BigQueryWrite.java index b1309e59b5e..c8966bfa341 100644 --- a/storage/connectors/bigquery/src/main/java/feast/storage/connectors/bigquery/writer/BigQueryWrite.java +++ b/storage/connectors/bigquery/src/main/java/feast/storage/connectors/bigquery/writer/BigQueryWrite.java @@ -16,92 +16,238 @@ */ package feast.storage.connectors.bigquery.writer; -import com.google.api.services.bigquery.model.TableDataInsertAllResponse; -import com.google.api.services.bigquery.model.TableRow; +import com.google.api.services.bigquery.model.*; import com.google.cloud.bigquery.DatasetId; -import feast.proto.types.FeatureRowProto; +import com.google.common.collect.ImmutableList; +import com.google.common.collect.ImmutableSet; +import com.google.common.collect.Iterators; +import feast.proto.types.FeatureRowProto.FeatureRow; import feast.storage.api.writer.FailedElement; import feast.storage.api.writer.WriteResult; -import java.io.IOException; -import org.apache.beam.sdk.io.gcp.bigquery.BigQueryIO; -import org.apache.beam.sdk.io.gcp.bigquery.BigQueryInsertError; -import org.apache.beam.sdk.io.gcp.bigquery.InsertRetryPolicy; -import org.apache.beam.sdk.transforms.DoFn; -import org.apache.beam.sdk.transforms.PTransform; -import org.apache.beam.sdk.transforms.ParDo; -import org.apache.beam.sdk.values.PCollection; +import java.util.List; +import java.util.Map; +import org.apache.beam.sdk.coders.KvCoder; +import org.apache.beam.sdk.coders.StringUtf8Coder; +import org.apache.beam.sdk.io.gcp.bigquery.*; +import org.apache.beam.sdk.transforms.*; +import org.apache.beam.sdk.transforms.join.CoGbkResult; +import org.apache.beam.sdk.transforms.join.CoGroupByKey; +import org.apache.beam.sdk.transforms.join.KeyedPCollectionTuple; +import org.apache.beam.sdk.transforms.windowing.FixedWindows; +import org.apache.beam.sdk.transforms.windowing.Window; +import org.apache.beam.sdk.values.*; +import org.joda.time.Duration; import org.slf4j.Logger; /** - * A {@link PTransform} that writes {@link FeatureRowProto FeatureRows} to the specified BigQuery + * A {@link PTransform} that writes {@link FeatureRow FeatureRows} to the specified BigQuery * dataset, and returns a {@link WriteResult} containing the unsuccessful writes. Since Bigquery - * does not output successful writes, we cannot emit those, and so no success metrics will be - * captured if this sink is used. + * does not output failed writes, we cannot emit those. */ -public class BigQueryWrite - extends PTransform, WriteResult> { +public class BigQueryWrite extends PTransform, WriteResult> { private static final Logger log = org.slf4j.LoggerFactory.getLogger(BigQueryWrite.class); - // Destination dataset + private static final Duration BIGQUERY_DEFAULT_WRITE_TRIGGERING_FREQUENCY = + Duration.standardMinutes(1); + + private static final Duration BIGQUERY_JOB_MAX_EXPECTING_RESULT_TIME = Duration.standardHours(1); + private static final int BIGQUERY_MAX_JOB_RETRIES = 20; + private DatasetId destination; + private PCollectionView>> schemas; - public BigQueryWrite(DatasetId destination) { + private Duration triggeringFrequency = BIGQUERY_DEFAULT_WRITE_TRIGGERING_FREQUENCY; + private Duration expectingResultTime = BIGQUERY_JOB_MAX_EXPECTING_RESULT_TIME; + private BigQueryServices testServices; + + public BigQueryWrite( + DatasetId destination, PCollectionView>> schemas) { this.destination = destination; + this.schemas = schemas; + } + + public BigQueryWrite withTriggeringFrequency(Duration triggeringFrequency) { + this.triggeringFrequency = triggeringFrequency; + return this; + } + + public BigQueryWrite withTestServices(BigQueryServices testServices) { + this.testServices = testServices; + return this; } + public BigQueryWrite withExpectingResultTime(Duration expectingResultTime) { + this.expectingResultTime = expectingResultTime; + return this; + } + + /** + * BigQuery writer 1. choose destination based on featureSetName {@link + * FeatureDynamicDestinations} 2. dynamically pull destination's schema from schemas' view 3. + * convert {@link FeatureRow} into {@link TableRow} 4. group input into fixed windows (configured + * with setTriggeringFrequency) 5. write to bq (via BATCH FILE LOADING) {@link + * BatchLoadsWithResult} 6. join bq job output with input to produce successful inserts + * + * @param input stream of FeatureRows to write + * @return stream of successfully inserted FeatureRows + */ @Override - public WriteResult expand(PCollection input) { + public WriteResult expand(PCollection input) { String jobName = input.getPipeline().getOptions().getJobName(); - org.apache.beam.sdk.io.gcp.bigquery.WriteResult bigqueryWriteResult = + + FeatureDynamicDestinations dynamicDestinations = new FeatureDynamicDestinations(); + + BatchLoadsWithResult.Builder writerBuilder = + BatchLoadsWithResult.create() + .setWriteDisposition(BigQueryIO.Write.WriteDisposition.WRITE_APPEND) + .setCreateDisposition(BigQueryIO.Write.CreateDisposition.CREATE_IF_NEEDED) + .setDynamicDestinations(dynamicDestinations) + .setDestinationCoder(StringUtf8Coder.of()) + .setElementCoder(TableRowJsonCoder.of()) + .setTriggeringFrequency(triggeringFrequency) + .setMaxRetryJobs(BIGQUERY_MAX_JOB_RETRIES) + .setSchemaUpdateOptions( + ImmutableSet.of(BigQueryIO.Write.SchemaUpdateOption.ALLOW_FIELD_ADDITION)); + + if (testServices != null) { + writerBuilder.setBigQueryServices(testServices); + } + + PCollection inputInFixedWindow = input.apply( - "WriteTableRowToBigQuery", - BigQueryIO.write() - .to(new GetTableDestination(destination.getProject(), destination.getDataset())) - .withFormatFunction(new FeatureRowToTableRow(jobName)) - .withCreateDisposition(BigQueryIO.Write.CreateDisposition.CREATE_NEVER) - .withWriteDisposition(BigQueryIO.Write.WriteDisposition.WRITE_APPEND) - .withExtendedErrorInfo() - .withMethod(BigQueryIO.Write.Method.STREAMING_INSERTS) - .withFailedInsertRetryPolicy(InsertRetryPolicy.retryTransientErrors())); + "BatchingInput", + Window.into(FixedWindows.of(triggeringFrequency)) + .withAllowedLateness(expectingResultTime) + .discardingFiredPanes()); + PCollection> insertionResult = + inputInFixedWindow + .apply( + "PrepareWrite", + new PrepareWrite<>(dynamicDestinations, new FeatureRowToTableRow(jobName))) + .setCoder(KvCoder.of(StringUtf8Coder.of(), TableRowJsonCoder.of())) + .apply("WriteTableRowToBigQuery", writerBuilder.build()); + + PCollection successfulInserts = + mergeInputWithResult(inputInFixedWindow, insertionResult); + + // Since BigQueryIO does not support emitting failure writes, we set failedElements to + // an empty stream PCollection failedElements = - bigqueryWriteResult - .getFailedInsertsWithErr() + input + .getPipeline() + .apply(Create.of("")) .apply( - "WrapBigQueryInsertionError", + "dummy", ParDo.of( - new DoFn() { + new DoFn() { @ProcessElement - public void processElement(ProcessContext context) { - TableDataInsertAllResponse.InsertErrors error = - context.element().getError(); - TableRow row = context.element().getRow(); - try { - context.output( - FailedElement.newBuilder() - .setErrorMessage(error.toPrettyString()) - .setPayload(row.toPrettyString()) - .setJobName(context.getPipelineOptions().getJobName()) - .setTransformName("WriteTableRowToBigQuery") - .build()); - } catch (IOException e) { - log.error(e.getMessage()); - } - } + public void processElement(ProcessContext context) {} })); - // Since BigQueryIO does not support emitting successful writes, we set successfulInserts to - // an empty stream, - // and no metrics will be collected. - PCollection successfulInserts = - input.apply( - "dummy", - ParDo.of( - new DoFn() { - @ProcessElement - public void processElement(ProcessContext context) {} - })); - return WriteResult.in(input.getPipeline(), successfulInserts, failedElements); } + + /** + * Join input stream of FeatureRows with output stream from write jobs to produce rows that was + * successfully inserted. + * + *

In order to join we expect both streams to have identical windowing strategy. Join key - is + * write destination (table reference), hence as soon as bq load job responsible for specific + * table succeeded - we consider all rows routed to this destination in this window(!) as + * successfully written + * + * @param inputInFixedWindow input stream in fixed window + * @param successful output stream from BatchLoader {@link BatchLoadsWithResult} + * @return collection of successfully inserted rows + */ + private PCollection mergeInputWithResult( + PCollection inputInFixedWindow, + PCollection> successful) { + final TupleTag inputTag = new TupleTag<>(); + final TupleTag successTag = new TupleTag<>(); + + PCollection> inputWithResult = + KeyedPCollectionTuple.of( + inputTag, + inputInFixedWindow.apply( + "MakeElementKey", + ParDo.of( + new DoFn>() { + @ProcessElement + public void process(ProcessContext c) { + FeatureRow element = c.element(); + c.output( + KV.of( + BigQuerySinkHelpers.getTableDestination( + destination, element.getFeatureSet()) + .getTableSpec(), + element)); + } + }))) + .and( + successTag, + successful.apply( + "MakeResultKey", + ParDo.of( + new DoFn, KV>() { + @ProcessElement + public void process(ProcessContext c) { + c.output( + KV.of(c.element().getKey().getTableSpec(), c.element().getValue())); + } + }))) + .apply(CoGroupByKey.create()); + + return inputWithResult.apply( + "ProduceSuccessfulInserts", + ParDo.of( + new DoFn, FeatureRow>() { + @ProcessElement + public void process(ProcessContext c) { + CoGbkResult result = c.element().getValue(); + result + .getAll(successTag) + .forEach(success -> result.getAll(inputTag).forEach(c::output)); + } + })); + } + + /** + * DynamicDestination router allocates incoming {@link FeatureRow} to tables in BigQuery based on + * FeatureSet reference extracted from this row. + * + *

getDestination is called for every item and simply returns grouping key {@link String}. + * + *

On window triggering getTable and getSchema are called once per each group. + * + *

getSchema provides latest table schema calculated from PCollection<FeatureSetSpec>. + * This schema is attached to each BQ Load job to update tables in-flight. + */ + private class FeatureDynamicDestinations extends DynamicDestinations { + @Override + public String getDestination(ValueInSingleWindow element) { + return element.getValue().getFeatureSet(); + } + + @Override + public List> getSideInputs() { + return ImmutableList.of(schemas); + } + + @Override + public TableDestination getTable(String featureSetKey) { + return BigQuerySinkHelpers.getTableDestination(destination, featureSetKey); + } + + @Override + public TableSchema getSchema(String featureSet) { + Map> schemasValue = sideInput(schemas); + Iterable schemasIt = schemasValue.get(featureSet); + if (schemasIt == null) { + return null; + } + return Iterators.getLast(schemasIt.iterator()); + } + } } diff --git a/storage/connectors/bigquery/src/main/java/feast/storage/connectors/bigquery/writer/FeatureSetSpecToTableSchema.java b/storage/connectors/bigquery/src/main/java/feast/storage/connectors/bigquery/writer/FeatureSetSpecToTableSchema.java new file mode 100644 index 00000000000..f57a9a89029 --- /dev/null +++ b/storage/connectors/bigquery/src/main/java/feast/storage/connectors/bigquery/writer/FeatureSetSpecToTableSchema.java @@ -0,0 +1,231 @@ +/* + * SPDX-License-Identifier: Apache-2.0 + * Copyright 2018-2020 The Feast Authors + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package feast.storage.connectors.bigquery.writer; + +import com.google.api.services.bigquery.model.TableFieldSchema; +import com.google.api.services.bigquery.model.TableReference; +import com.google.api.services.bigquery.model.TableSchema; +import com.google.cloud.bigquery.*; +import com.google.common.collect.ImmutableList; +import com.google.common.collect.ImmutableMap; +import feast.proto.core.FeatureSetProto; +import feast.storage.connectors.bigquery.common.TypeUtil; +import java.io.IOException; +import java.io.InputStream; +import java.io.OutputStream; +import java.util.ArrayList; +import java.util.List; +import java.util.Map; +import java.util.stream.Collectors; +import org.apache.beam.repackaged.core.org.apache.commons.lang3.tuple.Pair; +import org.apache.beam.sdk.coders.Coder; +import org.apache.beam.sdk.coders.CoderException; +import org.apache.beam.sdk.coders.StringUtf8Coder; +import org.apache.beam.sdk.io.gcp.bigquery.BigQueryHelpers; +import org.apache.beam.sdk.io.gcp.bigquery.TableDestination; +import org.apache.beam.sdk.options.ValueProvider; +import org.apache.beam.sdk.transforms.DoFn; +import org.apache.beam.sdk.values.KV; +import org.slf4j.Logger; + +public class FeatureSetSpecToTableSchema + extends DoFn, KV> { + private BigQuery bqService; + private DatasetId dataset; + private ValueProvider bqProvider; + + private static final Logger log = + org.slf4j.LoggerFactory.getLogger(FeatureSetSpecToTableSchema.class); + + // Column description for reserved fields + public static final String BIGQUERY_EVENT_TIMESTAMP_FIELD_DESCRIPTION = + "Event time for the FeatureRow"; + public static final String BIGQUERY_CREATED_TIMESTAMP_FIELD_DESCRIPTION = + "Processing time of the FeatureRow ingestion in Feast\""; + public static final String BIGQUERY_INGESTION_ID_FIELD_DESCRIPTION = + "Unique id identifying groups of rows that have been ingested together"; + public static final String BIGQUERY_JOB_ID_FIELD_DESCRIPTION = + "Feast import job ID for the FeatureRow"; + + public FeatureSetSpecToTableSchema(DatasetId dataset, ValueProvider bqProvider) { + this.dataset = dataset; + this.bqProvider = bqProvider; + } + + @Setup + public void setup() { + this.bqService = bqProvider.get(); + } + + @ProcessElement + public void processElement( + @Element KV element, + OutputReceiver> output, + ProcessContext context) { + Schema schema = createSchemaFromSpec(element.getValue(), element.getKey()); + output.output(KV.of(element.getKey(), serializeSchema(schema))); + } + + private Table getExistingTable(String specKey) { + TableDestination tableDestination = BigQuerySinkHelpers.getTableDestination(dataset, specKey); + TableReference tableReference = BigQueryHelpers.parseTableSpec(tableDestination.getTableSpec()); + TableId tableId = + TableId.of( + tableReference.getProjectId(), + tableReference.getDatasetId(), + tableReference.getTableId()); + return bqService.getTable(tableId); + } + + /** + * Creates a BigQuery {@link Schema} based on the provided FeatureSetSpec and the existing table, + * if any. If a table already exists, existing fields will be retained, and new fields present in + * the feature set will be appended to the existing FieldsList. + * + * @param spec FeatureSet spec that this table is for + * @param specKey String for retrieving existing table + * @return {@link Schema} containing all tombstoned and active fields. + */ + private Schema createSchemaFromSpec(FeatureSetProto.FeatureSetSpec spec, String specKey) { + List fields = new ArrayList<>(); + log.info("Table {} will have the following fields:", specKey); + Table existingTable = getExistingTable(specKey); + + for (FeatureSetProto.EntitySpec entitySpec : spec.getEntitiesList()) { + Field.Builder builder = + Field.newBuilder( + entitySpec.getName(), TypeUtil.toStandardSqlType(entitySpec.getValueType())); + if (entitySpec.getValueType().name().toLowerCase().endsWith("_list")) { + builder.setMode(Field.Mode.REPEATED); + } + Field field = builder.build(); + log.info("- {}", field.toString()); + fields.add(field); + } + for (FeatureSetProto.FeatureSpec featureSpec : spec.getFeaturesList()) { + Field.Builder builder = + Field.newBuilder( + featureSpec.getName(), TypeUtil.toStandardSqlType(featureSpec.getValueType())); + if (featureSpec.getValueType().name().toLowerCase().endsWith("_list")) { + builder.setMode(Field.Mode.REPEATED); + } + + Field field = builder.build(); + log.info("- {}", field.toString()); + fields.add(field); + } + + // Refer to protos/feast/core/Store.proto for reserved fields in BigQuery. + Map> + reservedFieldNameToPairOfStandardSQLTypeAndDescription = + ImmutableMap.of( + "event_timestamp", + Pair.of(StandardSQLTypeName.TIMESTAMP, BIGQUERY_EVENT_TIMESTAMP_FIELD_DESCRIPTION), + "created_timestamp", + Pair.of( + StandardSQLTypeName.TIMESTAMP, BIGQUERY_CREATED_TIMESTAMP_FIELD_DESCRIPTION), + "ingestion_id", + Pair.of(StandardSQLTypeName.STRING, BIGQUERY_INGESTION_ID_FIELD_DESCRIPTION), + "job_id", + Pair.of(StandardSQLTypeName.STRING, BIGQUERY_JOB_ID_FIELD_DESCRIPTION)); + for (Map.Entry> entry : + reservedFieldNameToPairOfStandardSQLTypeAndDescription.entrySet()) { + Field field = + Field.newBuilder(entry.getKey(), entry.getValue().getLeft()) + .setDescription(entry.getValue().getRight()) + .build(); + log.info("- {}", field.toString()); + fields.add(field); + } + + List fieldsList = new ArrayList<>(); + if (existingTable != null) { + Schema existingSchema = existingTable.getDefinition().getSchema(); + fieldsList.addAll(existingSchema.getFields()); + } + + for (Field field : fields) { + if (!fieldsList.contains(field)) { + fieldsList.add(field); + log.info("- {}", field.toString()); + } + } + + return Schema.of(FieldList.of(fieldsList)); + } + + /** + * Convert Table schema into json-like object (prepared for serialization) + * + * @param schema bq table schema + * @return json-like schema + */ + private TableSchema serializeSchema(Schema schema) { + TableSchema tableSchema = new TableSchema(); + FieldList fields = schema.getFields(); + List tableFieldSchemas = + fields.stream() + .map( + field -> { + TableFieldSchema f = + new TableFieldSchema() + .setName(field.getName()) + .setType(field.getType().name()); + + if (field.getMode() != null) { + f.setMode(field.getMode().name()); + } + + if (field.getDescription() != null) { + f.setDescription(field.getDescription()); + } + return f; + }) + .collect(Collectors.toList()); + + tableSchema.setFields(tableFieldSchemas); + return tableSchema; + } + + public static class TableSchemaCoder extends Coder { + private static final StringUtf8Coder stringCoder = StringUtf8Coder.of(); + private static final TableSchemaCoder INSTANCE = new TableSchemaCoder(); + + public static TableSchemaCoder of() { + return INSTANCE; + } + + @Override + public void encode(TableSchema value, OutputStream outStream) + throws CoderException, IOException { + stringCoder.encode(BigQueryHelpers.toJsonString(value), outStream); + } + + @Override + public TableSchema decode(InputStream inStream) throws CoderException, IOException { + return BigQueryHelpers.fromJsonString(stringCoder.decode(inStream), TableSchema.class); + } + + @Override + public List> getCoderArguments() { + return ImmutableList.of(); + } + + @Override + public void verifyDeterministic() throws NonDeterministicException {} + } +} diff --git a/storage/connectors/bigquery/src/main/java/feast/storage/connectors/bigquery/writer/GetTableDestination.java b/storage/connectors/bigquery/src/main/java/feast/storage/connectors/bigquery/writer/GetTableDestination.java deleted file mode 100644 index b7774ae7ec4..00000000000 --- a/storage/connectors/bigquery/src/main/java/feast/storage/connectors/bigquery/writer/GetTableDestination.java +++ /dev/null @@ -1,50 +0,0 @@ -/* - * SPDX-License-Identifier: Apache-2.0 - * Copyright 2018-2019 The Feast Authors - * - * Licensed under the Apache License, Version 2.0 (the "License"); - * you may not use this file except in compliance with the License. - * You may obtain a copy of the License at - * - * https://www.apache.org/licenses/LICENSE-2.0 - * - * Unless required by applicable law or agreed to in writing, software - * distributed under the License is distributed on an "AS IS" BASIS, - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - * See the License for the specific language governing permissions and - * limitations under the License. - */ -package feast.storage.connectors.bigquery.writer; - -import com.google.api.services.bigquery.model.TimePartitioning; -import feast.proto.types.FeatureRowProto.FeatureRow; -import org.apache.beam.sdk.io.gcp.bigquery.TableDestination; -import org.apache.beam.sdk.transforms.SerializableFunction; -import org.apache.beam.sdk.values.ValueInSingleWindow; - -public class GetTableDestination - implements SerializableFunction, TableDestination> { - - private String projectId; - private String datasetId; - - public GetTableDestination(String projectId, String datasetId) { - this.projectId = projectId; - this.datasetId = datasetId; - } - - @Override - public TableDestination apply(ValueInSingleWindow input) { - String[] splitName = input.getValue().getFeatureSet().split("/"); - - TimePartitioning timePartitioning = - new TimePartitioning() - .setType("DAY") - .setField(FeatureRowToTableRow.getEventTimestampColumn()); - - return new TableDestination( - String.format("%s:%s.%s_%s", projectId, datasetId, splitName[0], splitName[1]), - String.format("Feast table for %s", input.getValue().getFeatureSet()), - timePartitioning); - } -} diff --git a/storage/connectors/bigquery/src/main/java/org/apache/beam/sdk/io/gcp/bigquery/BatchLoadsWithResult.java b/storage/connectors/bigquery/src/main/java/org/apache/beam/sdk/io/gcp/bigquery/BatchLoadsWithResult.java new file mode 100644 index 00000000000..89ce54b8219 --- /dev/null +++ b/storage/connectors/bigquery/src/main/java/org/apache/beam/sdk/io/gcp/bigquery/BatchLoadsWithResult.java @@ -0,0 +1,291 @@ +package org.apache.beam.sdk.io.gcp.bigquery; + +import static org.apache.beam.sdk.io.gcp.bigquery.BigQueryHelpers.resolveTempLocation; +import static org.apache.beam.vendor.grpc.v1p21p0.com.google.common.base.Preconditions.checkArgument; + +import com.google.api.services.bigquery.model.TableRow; +import com.google.auto.value.AutoValue; +import java.util.Collections; +import java.util.List; +import java.util.Set; +import javax.annotation.Nullable; +import org.apache.beam.sdk.coders.*; +import org.apache.beam.sdk.options.ValueProvider; +import org.apache.beam.sdk.transforms.*; +import org.apache.beam.sdk.transforms.windowing.*; +import org.apache.beam.sdk.values.*; +import org.apache.beam.vendor.guava.v26_0_jre.com.google.common.annotations.VisibleForTesting; +import org.apache.beam.vendor.guava.v26_0_jre.com.google.common.collect.Lists; +import org.joda.time.Duration; +import org.slf4j.Logger; +import org.slf4j.LoggerFactory; + +@AutoValue +public abstract class BatchLoadsWithResult + extends PTransform< + PCollection>, PCollection>> { + static final Logger LOG = LoggerFactory.getLogger(BatchLoadsWithResult.class); + + @VisibleForTesting + // Maximum number of files in a single partition. + static final int DEFAULT_MAX_FILES_PER_PARTITION = 10000; + + @VisibleForTesting + // Maximum number of bytes in a single partition -- 11 TiB just under BQ's 12 TiB limit. + static final long DEFAULT_MAX_BYTES_PER_PARTITION = 11 * (1L << 40); + + // The maximum size of a single file - 4TiB, just under the 5 TiB limit. + static final long DEFAULT_MAX_FILE_SIZE = 4 * (1L << 40); + + static final int DEFAULT_MAX_RETRY_JOBS = 3; + + @Nullable + abstract BigQueryServices getBigQueryServices(); + + abstract boolean getIgnoreUnknownValues(); + + abstract BigQueryIO.Write.WriteDisposition getWriteDisposition(); + + abstract BigQueryIO.Write.CreateDisposition getCreateDisposition(); + + abstract Set getSchemaUpdateOptions(); + + abstract DynamicDestinations getDynamicDestinations(); + + abstract Coder getDestinationCoder(); + + abstract Duration getTriggeringFrequency(); + + @Nullable + abstract ValueProvider getCustomGcsTempLocation(); + + @Nullable + abstract ValueProvider getLoadJobProjectId(); + + abstract Coder getElementCoder(); + + abstract RowWriterFactory getRowWriterFactory(); + + @Nullable + abstract String getKmsKey(); + + abstract int getMaxRetryJobs(); + + @AutoValue.Builder + public abstract static class Builder { + public abstract Builder setBigQueryServices(BigQueryServices bigQueryServices); + + public abstract Builder setIgnoreUnknownValues(boolean ignoreUnknownValues); + + public abstract Builder setWriteDisposition( + BigQueryIO.Write.WriteDisposition writeDisposition); + + public abstract Builder setCreateDisposition( + BigQueryIO.Write.CreateDisposition createDisposition); + + public abstract Builder setSchemaUpdateOptions( + Set schemaUpdateOptions); + + public abstract Builder setDynamicDestinations( + DynamicDestinations dynamicDestinations); + + public abstract Builder setDestinationCoder(Coder destinationCoder); + + public abstract Builder setTriggeringFrequency(Duration triggeringFrequency); + + public abstract Builder setCustomGcsTempLocation( + @Nullable ValueProvider customGcsTempLocation); + + public abstract Builder setLoadJobProjectId( + @Nullable ValueProvider loadJobProjectId); + + public abstract Builder setElementCoder(Coder elementCoder); + + public abstract Builder setRowWriterFactory( + RowWriterFactory rowWriterFactory); + + public abstract Builder setKmsKey(@Nullable String kmsKey); + + public abstract Builder setMaxRetryJobs(int maxRetryJobs); + + public abstract BatchLoadsWithResult build(); + } + + public static Builder create() { + return new AutoValue_BatchLoadsWithResult.Builder() + .setIgnoreUnknownValues(false) + .setBigQueryServices(new BigQueryServicesImpl()) + .setRowWriterFactory(RowWriterFactory.tableRows(SerializableFunctions.identity())) + .setSchemaUpdateOptions(Collections.emptySet()) + .setMaxRetryJobs(DEFAULT_MAX_RETRY_JOBS); + } + + public PCollection> expand( + PCollection> input) { + + // we assume that input must be already windowed and we allow only fixed window + // so our internal generator (JobIdPrefix) would be compatible (hence, joinable) + checkArgument( + input.getWindowingStrategy().getWindowFn() instanceof FixedWindows, + "Input to BQ writer must be windowed in advance"); + + final PCollection loadJobIdPrefixView = createLoadJobIdPrefixView(input); + final PCollectionView tempFilePrefixView = + createTempFilePrefixView(loadJobIdPrefixView); + + PCollection> results = + input + .apply( + "PutAllRowsInSingleShard", + ParDo.of( + new DoFn, KV, TableRow>>() { + @ProcessElement + public void process(ProcessContext c) { + c.output( + KV.of(ShardedKey.of(c.element().getKey(), 0), c.element().getValue())); + } + })) + .setCoder(KvCoder.of(ShardedKeyCoder.of(getDestinationCoder()), getElementCoder())) + .apply("GroupByDestination", GroupByKey.create()) + .apply( + "WriteGroupedRecords", + ParDo.of( + new WriteGroupedRecordsToFiles<>( + tempFilePrefixView, DEFAULT_MAX_FILE_SIZE, getRowWriterFactory())) + .withSideInputs(tempFilePrefixView)) + .setCoder(WriteBundlesToFiles.ResultCoder.of(getDestinationCoder())); + + TupleTag, List>> multiPartitionsTag = + new TupleTag<>("multiPartitionsTag"); + TupleTag, List>> singlePartitionTag = + new TupleTag<>("singlePartitionTag"); + + // Copied from original BatchLoads + PCollectionTuple partitions = + results + .apply("AttachSingletonKey", WithKeys.of((Void) null)) + .setCoder( + KvCoder.of( + VoidCoder.of(), WriteBundlesToFiles.ResultCoder.of(getDestinationCoder()))) + .apply("GroupOntoSingleton", GroupByKey.create()) + .apply("ExtractResultValues", Values.create()) + .apply( + "WritePartitionTriggered", + ParDo.of( + new WritePartition<>( + false, + getDynamicDestinations(), + tempFilePrefixView, + DEFAULT_MAX_FILES_PER_PARTITION, + DEFAULT_MAX_BYTES_PER_PARTITION, + multiPartitionsTag, + singlePartitionTag, + getRowWriterFactory())) + .withSideInputs(tempFilePrefixView) + .withOutputTags(multiPartitionsTag, TupleTagList.of(singlePartitionTag))); + + partitions + .get(multiPartitionsTag) + .setCoder( + KvCoder.of( + ShardedKeyCoder.of(NullableCoder.of(getDestinationCoder())), + ListCoder.of(StringUtf8Coder.of()))); + + return writeSinglePartitionWithResult( + partitions.get(singlePartitionTag), loadJobIdPrefixView.apply(View.asSingleton())); + } + + private PCollection createLoadJobIdPrefixView( + PCollection> input) { + // We generate new JobId per each (input) window + // To keep BQ job's name unique + // Windowing of this generator is expected to be synchronized with input window + // So generated ids can be applied as side input + return input + .apply( + "JobIdWindow", + Window.>configure() + .triggering(Repeatedly.forever(AfterProcessingTime.pastFirstElementInPane())) + .discardingFiredPanes()) + .apply( + "JustDummyEvent", + ParDo.of( + new DoFn, KV>() { + @ProcessElement + public void process(ProcessContext c) { + c.output(KV.of((Void) null, (Void) null)); + } + })) + .apply("OneEventPerWindow", GroupByKey.create()) + .apply( + "CreateJobId", + ParDo.of( + new DoFn>, String>() { + @ProcessElement + public void process(ProcessContext c, BoundedWindow w) { + c.output( + String.format( + "beam_load_%s_%s", + c.getPipelineOptions().getJobName().replaceAll("-", ""), + BigQueryHelpers.randomUUIDString())); + LOG.info("[BQ] New window {}, {}", c.timestamp(), w.maxTimestamp()); + } + })); + } + + private PCollectionView createTempFilePrefixView(final PCollection jobId) { + return jobId + .apply( + "GetTempFilePrefix", + ParDo.of( + new DoFn() { + @ProcessElement + public void getTempFilePrefix(ProcessContext c, BoundedWindow w) { + String tempLocationRoot; + if (getCustomGcsTempLocation() != null) { + tempLocationRoot = getCustomGcsTempLocation().get(); + } else { + tempLocationRoot = c.getPipelineOptions().getTempLocation(); + } + String tempLocation = + resolveTempLocation(tempLocationRoot, "BigQueryWriteTemp", c.element()); + LOG.info("[BQ] temp location generated {}, {}", tempLocation, w.maxTimestamp()); + c.output(tempLocation); + } + })) + .apply("TempFilePrefixView", View.asSingleton()); + } + + PCollection> writeSinglePartitionWithResult( + PCollection, List>> input, + PCollectionView loadJobIdPrefixView) { + List> sideInputs = Lists.newArrayList(loadJobIdPrefixView); + sideInputs.addAll(getDynamicDestinations().getSideInputs()); + Coder, List>> partitionsCoder = + KvCoder.of( + ShardedKeyCoder.of(NullableCoder.of(getDestinationCoder())), + ListCoder.of(StringUtf8Coder.of())); + // Write single partition to final table + return input + .setCoder(partitionsCoder) + // Reshuffle will distribute this among multiple workers, and also guard against + // reexecution of the WritePartitions step once WriteTables has begun. + .apply("SinglePartitionsReshuffle", Reshuffle.of()) + .apply( + "SinglePartitionWriteTables", + new WriteTables<>( + false, + getBigQueryServices(), + loadJobIdPrefixView, + getWriteDisposition(), + getCreateDisposition(), + sideInputs, + getDynamicDestinations(), + getLoadJobProjectId(), + getMaxRetryJobs(), + getIgnoreUnknownValues(), + getKmsKey(), + getRowWriterFactory().getSourceFormat(), + getSchemaUpdateOptions())); + } +} diff --git a/storage/connectors/bigquery/src/test/java/com/google/cloud/bigquery/FakeTable.java b/storage/connectors/bigquery/src/test/java/com/google/cloud/bigquery/FakeTable.java new file mode 100644 index 00000000000..7a3eb445602 --- /dev/null +++ b/storage/connectors/bigquery/src/test/java/com/google/cloud/bigquery/FakeTable.java @@ -0,0 +1,37 @@ +/* + * SPDX-License-Identifier: Apache-2.0 + * Copyright 2018-2020 The Feast Authors + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package com.google.cloud.bigquery; + +import java.io.IOException; +import java.io.ObjectInputStream; + +public class FakeTable extends Table { + FakeTable(BigQuery bigquery, BuilderImpl infoBuilder) { + super(bigquery, infoBuilder); + } + + public static FakeTable create(BigQuery bigQuery, TableId tableId, TableDefinition definition) { + BuilderImpl builder = new BuilderImpl(); + builder.setDefinition(definition); + builder.setTableId(tableId); + return new FakeTable(bigQuery, builder); + } + + private void readObject(ObjectInputStream in) throws IOException, ClassNotFoundException { + in.defaultReadObject(); + } +} diff --git a/storage/connectors/bigquery/src/test/java/feast/storage/connectors/bigquery/writer/BigQuerySinkTest.java b/storage/connectors/bigquery/src/test/java/feast/storage/connectors/bigquery/writer/BigQuerySinkTest.java new file mode 100644 index 00000000000..e36e2313456 --- /dev/null +++ b/storage/connectors/bigquery/src/test/java/feast/storage/connectors/bigquery/writer/BigQuerySinkTest.java @@ -0,0 +1,413 @@ +/* + * SPDX-License-Identifier: Apache-2.0 + * Copyright 2018-2020 The Feast Authors + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package feast.storage.connectors.bigquery.writer; + +import static feast.storage.common.testing.TestUtil.field; +import static feast.storage.connectors.bigquery.writer.FeatureSetSpecToTableSchema.*; +import static org.hamcrest.CoreMatchers.*; +import static org.junit.Assert.*; +import static org.mockito.Mockito.*; +import static org.mockito.MockitoAnnotations.initMocks; + +import com.google.api.services.bigquery.model.Job; +import com.google.api.services.bigquery.model.JobConfigurationLoad; +import com.google.api.services.bigquery.model.TableFieldSchema; +import com.google.api.services.bigquery.model.TableReference; +import com.google.cloud.bigquery.*; +import com.google.common.collect.ImmutableMap; +import com.google.common.collect.Iterators; +import feast.proto.core.FeatureSetProto.EntitySpec; +import feast.proto.core.FeatureSetProto.FeatureSetSpec; +import feast.proto.core.FeatureSetProto.FeatureSpec; +import feast.proto.types.FeatureRowProto.FeatureRow; +import feast.proto.types.ValueProto; +import feast.storage.api.writer.FeatureSink; +import feast.storage.api.writer.WriteResult; +import java.io.IOException; +import java.io.Serializable; +import java.util.ArrayList; +import java.util.Arrays; +import java.util.List; +import java.util.Random; +import java.util.logging.ConsoleHandler; +import java.util.logging.Logger; +import org.apache.beam.sdk.coders.KvCoder; +import org.apache.beam.sdk.coders.StringUtf8Coder; +import org.apache.beam.sdk.extensions.protobuf.ProtoCoder; +import org.apache.beam.sdk.io.gcp.bigquery.BatchLoadsWithResult; +import org.apache.beam.sdk.io.gcp.testing.FakeBigQueryServices; +import org.apache.beam.sdk.io.gcp.testing.FakeDatasetService; +import org.apache.beam.sdk.io.gcp.testing.FakeJobService; +import org.apache.beam.sdk.options.PipelineOptions; +import org.apache.beam.sdk.options.ValueProvider; +import org.apache.beam.sdk.testing.PAssert; +import org.apache.beam.sdk.testing.TestPipeline; +import org.apache.beam.sdk.testing.TestStream; +import org.apache.beam.sdk.transforms.Create; +import org.apache.beam.sdk.transforms.PTransform; +import org.apache.beam.sdk.transforms.windowing.AfterPane; +import org.apache.beam.sdk.transforms.windowing.GlobalWindows; +import org.apache.beam.sdk.transforms.windowing.Repeatedly; +import org.apache.beam.sdk.transforms.windowing.Window; +import org.apache.beam.sdk.values.KV; +import org.apache.beam.sdk.values.PCollection; +import org.joda.time.Duration; +import org.joda.time.Instant; +import org.junit.Before; +import org.junit.Rule; +import org.junit.Test; +import org.junit.rules.ExpectedException; +import org.mockito.Mock; +import org.mockito.invocation.InvocationOnMock; +import org.mockito.stubbing.Answer; + +public class BigQuerySinkTest { + + @Rule public transient TestPipeline p = TestPipeline.fromOptions(makePipelineOptions()); + @Rule public final ExpectedException exception = ExpectedException.none(); + + @Mock(serializable = true) + private BigQuery bigQuery; + + private FakeJobService jobService = new FakeJobService(); + private FakeDatasetService datasetService = new FakeDatasetService(); + private Random rd = new Random(); + + List commonFields = + Arrays.asList( + new TableFieldSchema() + .setName("event_timestamp") + .setType("TIMESTAMP") + .setDescription(BIGQUERY_EVENT_TIMESTAMP_FIELD_DESCRIPTION), + new TableFieldSchema() + .setName("created_timestamp") + .setType("TIMESTAMP") + .setDescription(BIGQUERY_CREATED_TIMESTAMP_FIELD_DESCRIPTION), + new TableFieldSchema() + .setName("ingestion_id") + .setType("STRING") + .setDescription(BIGQUERY_INGESTION_ID_FIELD_DESCRIPTION), + new TableFieldSchema() + .setName("job_id") + .setType("STRING") + .setDescription(BIGQUERY_JOB_ID_FIELD_DESCRIPTION)); + FeatureSetSpec spec; + FeatureSink defaultSink; + + public static PipelineOptions makePipelineOptions() { + PipelineOptions options = TestPipeline.testingPipelineOptions(); + options.setTempLocation("/tmp/feast"); + return options; + } + + public FeatureRow generateRow(String featureSet) { + return FeatureRow.newBuilder() + .setFeatureSet(featureSet) + .addFields(field("entity", rd.nextInt(), ValueProto.ValueType.Enum.INT64)) + .addFields( + field("feature", String.format("%f", rd.nextFloat()), ValueProto.ValueType.Enum.STRING)) + .build(); + } + + @Before + public void setUp() throws IOException, InterruptedException { + initMocks(this); + + Logger.getLogger(BatchLoadsWithResult.class.getName()).addHandler(new ConsoleHandler()); + + when(bigQuery.getTable(TableId.of("test-project", "test_dataset", "myproject_fs"))) + .thenReturn(null); + + FakeDatasetService.setUp(); + datasetService.createDataset("test-project", "test_dataset", "us-central1", "description", 0L); + + spec = + FeatureSetSpec.newBuilder() + .setName("fs") + .setProject("myproject") + .addEntities( + EntitySpec.newBuilder() + .setName("entity") + .setValueType(ValueProto.ValueType.Enum.INT64) + .build()) + .addFeatures( + FeatureSpec.newBuilder() + .setName("feature") + .setValueType(ValueProto.ValueType.Enum.STRING) + .build()) + .build(); + + defaultSink = + makeSink( + ValueProvider.StaticValueProvider.of(bigQuery), + p.apply( + "StaticSpecs", + Create.of( + ImmutableMap.of( + String.format("%s/%s", spec.getProject(), spec.getName()), spec)))); + } + + private FeatureSink makeSink( + ValueProvider bq, PCollection> specs) { + return BigQueryFeatureSink.builder() + .setDatasetId("test_dataset") + .setProjectId("test-project") + .setFeatureSetSpecs(specs) + .setBQTestServices( + new FakeBigQueryServices() + .withJobService(jobService) + .withDatasetService(datasetService)) + .setBQClient(bq) + .setTriggeringFrequency(Duration.standardSeconds(5)) + .build(); + } + + @Test + public void simpleInsert() { + FeatureRow row1 = generateRow("myproject/fs"); + FeatureRow row2 = generateRow("myproject/fs"); + + TestStream featureRowTestStream = + TestStream.create(ProtoCoder.of(FeatureRow.class)) + .advanceWatermarkTo(Instant.now()) + .addElements(row1, row2) + .advanceWatermarkToInfinity(); + + FeatureSink sink = + makeSink( + ValueProvider.StaticValueProvider.of(bigQuery), + p.apply( + Create.of( + ImmutableMap.of( + String.format("%s/%s", spec.getProject(), spec.getName()), spec)))); + PCollection successfulInserts = + p.apply(featureRowTestStream).apply(sink.writer()).getSuccessfulInserts(); + PAssert.that(successfulInserts).containsInAnyOrder(row1, row2); + p.run(); + + assert jobService.getAllJobs().size() == 1; + Job load = Iterators.getLast(jobService.getAllJobs().iterator()); + JobConfigurationLoad loadConfiguration = load.getConfiguration().getLoad(); + + ArrayList expectedFields = + new ArrayList<>( + Arrays.asList( + new TableFieldSchema().setName("entity").setType("INTEGER"), + new TableFieldSchema().setName("feature").setType("STRING"))); + + expectedFields.addAll(commonFields); + + assertThat(loadConfiguration.getSchema().getFields(), is(expectedFields)); + + assertThat( + loadConfiguration.getDestinationTable(), + is( + new TableReference() + .setDatasetId("test_dataset") + .setProjectId("test-project") + .setTableId("myproject_fs"))); + } + + @Test + public void uniqueJobIdPerWindow() { + TestStream featureRowTestStream = + TestStream.create(ProtoCoder.of(FeatureRow.class)) + .advanceWatermarkTo(Instant.now()) + .addElements(generateRow("myproject/fs")) + .addElements(generateRow("myproject/fs")) + .advanceWatermarkTo(Instant.now().plus(Duration.standardSeconds(10))) + .addElements(generateRow("myproject/fs")) + .addElements(generateRow("myproject/fs")) + .advanceWatermarkToInfinity(); + + p.apply(featureRowTestStream).apply(defaultSink.writer()); + p.run(); + + assertThat(jobService.getAllJobs().size(), is(2)); + assertThat( + jobService.getAllJobs().stream() + .map(j -> j.getJobReference().getJobId()) + .distinct() + .count(), + is(2L)); + } + + @Test + public void expectingJobResult() { + FeatureRow featureRow = generateRow("myproject/fs"); + TestStream featureRowTestStream = + TestStream.create(ProtoCoder.of(FeatureRow.class)) + .advanceWatermarkTo(Instant.now()) + .addElements(featureRow) + .advanceWatermarkToInfinity(); + + jobService.setNumFailuresExpected(3); + + PTransform, WriteResult> writer = + ((BigQueryWrite) defaultSink.writer()).withExpectingResultTime(Duration.standardSeconds(5)); + PCollection inserts = + p.apply(featureRowTestStream).apply(writer).getSuccessfulInserts(); + + PAssert.that(inserts).containsInAnyOrder(featureRow); + + p.run(); + } + + @Test + public void updateSchemaWithExistingTable() { + TableId tableId = TableId.of("test-project", "test_dataset", "myproject_fs_2"); + + when(bigQuery.getTable(tableId)) + .thenAnswer( + new TableAnswer( + TableId.of("test-project", "test_dataset", "myproject_fs_2"), + StandardTableDefinition.of( + Schema.of( + Field.of("old_feature_1", LegacySQLTypeName.FLOAT), + Field.of("old_feature_2", LegacySQLTypeName.INTEGER))))); + + FeatureSetSpec spec_fs_2 = + FeatureSetSpec.newBuilder() + .setName("fs_2") + .setProject("myproject") + .addEntities( + EntitySpec.newBuilder() + .setName("entity") + .setValueType(ValueProto.ValueType.Enum.INT64) + .build()) + .addFeatures( + FeatureSpec.newBuilder() + .setName("feature") + .setValueType(ValueProto.ValueType.Enum.STRING) + .build()) + .build(); + + FeatureSink sink = + makeSink( + ValueProvider.StaticValueProvider.of(bigQuery), + p.apply( + Create.of( + ImmutableMap.of( + String.format("%s/%s", spec_fs_2.getProject(), spec_fs_2.getName()), + spec_fs_2)))); + + TestStream featureRowTestStream = + TestStream.create(ProtoCoder.of(FeatureRow.class)) + .advanceWatermarkTo(Instant.now()) + .addElements(generateRow("myproject/fs_2")) + .advanceWatermarkToInfinity(); + + p.apply(featureRowTestStream).apply(sink.writer()); + p.run(); + + assert jobService.getAllJobs().size() == 1; + Job load = Iterators.getLast(jobService.getAllJobs().iterator()); + JobConfigurationLoad loadConfiguration = load.getConfiguration().getLoad(); + + ArrayList expectedFields = + new ArrayList<>( + Arrays.asList( + new TableFieldSchema().setName("old_feature_1").setType("FLOAT"), + new TableFieldSchema().setName("old_feature_2").setType("INTEGER"), + new TableFieldSchema().setName("entity").setType("INTEGER"), + new TableFieldSchema().setName("feature").setType("STRING"))); + expectedFields.addAll(commonFields); + assertThat(loadConfiguration.getSchema().getFields(), is(expectedFields)); + } + + @Test + public void updateSpecInFlight() { + FeatureSetSpec spec_fs_2 = + FeatureSetSpec.newBuilder() + .setName("fs_2") + .setProject("myproject") + .addEntities( + EntitySpec.newBuilder() + .setName("entity") + .setValueType(ValueProto.ValueType.Enum.INT64) + .build()) + .addFeatures( + FeatureSpec.newBuilder() + .setName("feature") + .setValueType(ValueProto.ValueType.Enum.STRING) + .build()) + .addFeatures( + FeatureSpec.newBuilder() + .setName("new_feature") + .setValueType(ValueProto.ValueType.Enum.FLOAT) + .build()) + .build(); + + TestStream> specsStream = + TestStream.create(KvCoder.of(StringUtf8Coder.of(), ProtoCoder.of(FeatureSetSpec.class))) + .advanceWatermarkTo(Instant.now()) + .addElements(KV.of("myproject/fs", spec)) + .advanceProcessingTime(Duration.standardSeconds(5)) + // .advanceWatermarkTo(Instant.now().plus(Duration.standardSeconds(5))) + .addElements(KV.of("myproject/fs", spec_fs_2)) + .advanceWatermarkToInfinity(); + + FeatureSink sink = + makeSink( + ValueProvider.StaticValueProvider.of(bigQuery), + p.apply("SpecsInput", specsStream) + .apply( + Window.>into(new GlobalWindows()) + .triggering(Repeatedly.forever(AfterPane.elementCountAtLeast(1))) + .withAllowedLateness(Duration.millis(0)) + .accumulatingFiredPanes())); + + TestStream featureRowTestStream = + TestStream.create(ProtoCoder.of(FeatureRow.class)) + .advanceWatermarkTo(Instant.now().plus(Duration.standardSeconds(10))) + .advanceProcessingTime(Duration.standardSeconds(10)) + .addElements(generateRow("myproject/fs")) + .advanceWatermarkToInfinity(); + + p.apply("FeaturesInput", featureRowTestStream).apply(sink.writer()); + p.run(); + + assert jobService.getAllJobs().size() == 1; + Job load = Iterators.getLast(jobService.getAllJobs().iterator()); + JobConfigurationLoad loadConfiguration = load.getConfiguration().getLoad(); + + ArrayList expectedFields = + new ArrayList<>( + Arrays.asList( + new TableFieldSchema().setName("entity").setType("INTEGER"), + new TableFieldSchema().setName("feature").setType("STRING"), + new TableFieldSchema().setName("new_feature").setType("FLOAT"))); + expectedFields.addAll(commonFields); + + assertThat(loadConfiguration.getSchema().getFields(), is(expectedFields)); + } + + public static class TableAnswer implements Answer, Serializable { + TableId tableId; + TableDefinition tableDefinition; + + public TableAnswer(TableId tableId, TableDefinition tableDefinition) { + this.tableId = tableId; + this.tableDefinition = tableDefinition; + } + + @Override + public Table answer(InvocationOnMock invocationOnMock) throws Throwable { + return FakeTable.create(mock(BigQuery.class), tableId, tableDefinition); + } + } +} diff --git a/storage/connectors/redis/pom.xml b/storage/connectors/redis/pom.xml index 2a19c0e8287..3aa863f6811 100644 --- a/storage/connectors/redis/pom.xml +++ b/storage/connectors/redis/pom.xml @@ -37,6 +37,11 @@ provided + + com.google.guava + guava + + org.mockito mockito-core diff --git a/storage/connectors/redis/src/main/java/feast/storage/connectors/redis/writer/RedisCustomIO.java b/storage/connectors/redis/src/main/java/feast/storage/connectors/redis/writer/RedisCustomIO.java index b7fd68755c1..ae16a464a28 100644 --- a/storage/connectors/redis/src/main/java/feast/storage/connectors/redis/writer/RedisCustomIO.java +++ b/storage/connectors/redis/src/main/java/feast/storage/connectors/redis/writer/RedisCustomIO.java @@ -16,10 +16,11 @@ */ package feast.storage.connectors.redis.writer; +import com.google.common.collect.Iterators; +import com.google.common.collect.Lists; import feast.proto.core.FeatureSetProto.EntitySpec; import feast.proto.core.FeatureSetProto.FeatureSetSpec; import feast.proto.core.FeatureSetProto.FeatureSpec; -import feast.proto.core.StoreProto.Store.*; import feast.proto.storage.RedisProto.RedisKey; import feast.proto.storage.RedisProto.RedisKey.Builder; import feast.proto.types.FeatureRowProto.FeatureRow; @@ -29,23 +30,16 @@ import feast.storage.api.writer.WriteResult; import feast.storage.common.retry.Retriable; import io.lettuce.core.RedisException; -import java.io.IOException; -import java.util.ArrayList; import java.util.HashMap; import java.util.List; import java.util.Map; import java.util.concurrent.ExecutionException; import java.util.stream.Collectors; -import org.apache.beam.sdk.transforms.DoFn; -import org.apache.beam.sdk.transforms.PTransform; -import org.apache.beam.sdk.transforms.ParDo; -import org.apache.beam.sdk.transforms.windowing.GlobalWindow; -import org.apache.beam.sdk.values.PCollection; -import org.apache.beam.sdk.values.PCollectionTuple; -import org.apache.beam.sdk.values.TupleTag; -import org.apache.beam.sdk.values.TupleTagList; +import org.apache.beam.sdk.transforms.*; +import org.apache.beam.sdk.transforms.windowing.*; +import org.apache.beam.sdk.values.*; import org.apache.commons.lang3.exception.ExceptionUtils; -import org.joda.time.Instant; +import org.apache.commons.lang3.tuple.ImmutablePair; import org.slf4j.Logger; import org.slf4j.LoggerFactory; @@ -64,20 +58,22 @@ public class RedisCustomIO { private RedisCustomIO() {} public static Write write( - RedisIngestionClient redisIngestionClient, Map featureSetSpecs) { + RedisIngestionClient redisIngestionClient, + PCollectionView>> featureSetSpecs) { return new Write(redisIngestionClient, featureSetSpecs); } /** ServingStoreWrite data to a Redis server. */ public static class Write extends PTransform, WriteResult> { - private Map featureSetSpecs; + private PCollectionView>> featureSetSpecs; private RedisIngestionClient redisIngestionClient; private int batchSize; private int timeout; public Write( - RedisIngestionClient redisIngestionClient, Map featureSetSpecs) { + RedisIngestionClient redisIngestionClient, + PCollectionView>> featureSetSpecs) { this.redisIngestionClient = redisIngestionClient; this.featureSetSpecs = featureSetSpecs; } @@ -95,28 +91,37 @@ public Write withTimeout(int timeout) { @Override public WriteResult expand(PCollection input) { PCollectionTuple redisWrite = - input.apply( - ParDo.of(new WriteDoFn(redisIngestionClient, featureSetSpecs)) - .withOutputTags(successfulInsertsTag, TupleTagList.of(failedInsertsTupleTag))); + input + .apply( + "CollectBatchBeforeWrite", + Window.into(new GlobalWindows()) + .triggering(Repeatedly.forever(AfterPane.elementCountAtLeast(batchSize))) + .discardingFiredPanes()) + .apply("AttachSingletonKey", WithKeys.of((Void) null)) + .apply("GroupOntoSingleton", GroupByKey.create()) + .apply("ExtractResultValues", Values.create()) + .apply( + ParDo.of(new WriteDoFn(redisIngestionClient, featureSetSpecs)) + .withOutputTags(successfulInsertsTag, TupleTagList.of(failedInsertsTupleTag)) + .withSideInputs(featureSetSpecs)); return WriteResult.in( input.getPipeline(), redisWrite.get(successfulInsertsTag), redisWrite.get(failedInsertsTupleTag)); } - public static class WriteDoFn extends DoFn { - - private final List featureRows = new ArrayList<>(); - private Map featureSetSpecs; + public static class WriteDoFn extends DoFn, FeatureRow> { + private PCollectionView>> featureSetSpecsView; private int batchSize = DEFAULT_BATCH_SIZE; private int timeout = DEFAULT_TIMEOUT; private RedisIngestionClient redisIngestionClient; WriteDoFn( - RedisIngestionClient redisIngestionClient, Map featureSetSpecs) { + RedisIngestionClient redisIngestionClient, + PCollectionView>> featureSetSpecsView) { this.redisIngestionClient = redisIngestionClient; - this.featureSetSpecs = featureSetSpecs; + this.featureSetSpecsView = featureSetSpecsView; } public WriteDoFn withBatchSize(int batchSize) { @@ -145,10 +150,11 @@ public void startBundle() { } catch (RedisException e) { log.error("Connection to redis cannot be established ", e); } - featureRows.clear(); } - private void executeBatch() throws Exception { + private void executeBatch( + Iterable featureRows, Map featureSetSpecs) + throws Exception { this.redisIngestionClient .getBackOffExecutor() .execute( @@ -160,7 +166,9 @@ public void execute() throws ExecutionException, InterruptedException { } featureRows.forEach( row -> { - redisIngestionClient.set(getKey(row), getValue(row)); + redisIngestionClient.set( + getKey(row, featureSetSpecs.get(row.getFeatureSet())), + getValue(row, featureSetSpecs.get(row.getFeatureSet()))); }); redisIngestionClient.sync(); } @@ -186,10 +194,9 @@ private FailedElement toFailedElement( .build(); } - private byte[] getKey(FeatureRow featureRow) { - FeatureSetSpec featureSetSpec = featureSetSpecs.get(featureRow.getFeatureSet()); + private byte[] getKey(FeatureRow featureRow, FeatureSetSpec spec) { List entityNames = - featureSetSpec.getEntitiesList().stream() + spec.getEntitiesList().stream() .map(EntitySpec::getName) .sorted() .collect(Collectors.toList()); @@ -209,9 +216,7 @@ private byte[] getKey(FeatureRow featureRow) { return redisKeyBuilder.build().toByteArray(); } - private byte[] getValue(FeatureRow featureRow) { - FeatureSetSpec spec = featureSetSpecs.get(featureRow.getFeatureSet()); - + private byte[] getValue(FeatureRow featureRow, FeatureSetSpec spec) { List featureNames = spec.getFeaturesList().stream().map(FeatureSpec::getName).collect(Collectors.toList()); Map fieldValueOnlyMap = @@ -244,46 +249,23 @@ private byte[] getValue(FeatureRow featureRow) { @ProcessElement public void processElement(ProcessContext context) { - FeatureRow featureRow = context.element(); - featureRows.add(featureRow); - if (featureRows.size() >= batchSize) { - try { - executeBatch(); - featureRows.forEach(row -> context.output(successfulInsertsTag, row)); - featureRows.clear(); - } catch (Exception e) { - featureRows.forEach( - failedMutation -> { - FailedElement failedElement = - toFailedElement(failedMutation, e, context.getPipelineOptions().getJobName()); - context.output(failedInsertsTupleTag, failedElement); - }); - featureRows.clear(); - } - } - } + List featureRows = Lists.newArrayList(context.element().iterator()); - @FinishBundle - public void finishBundle(FinishBundleContext context) - throws IOException, InterruptedException { - if (featureRows.size() > 0) { - try { - executeBatch(); - featureRows.forEach( - row -> - context.output( - successfulInsertsTag, row, Instant.now(), GlobalWindow.INSTANCE)); - featureRows.clear(); - } catch (Exception e) { - featureRows.forEach( - failedMutation -> { - FailedElement failedElement = - toFailedElement(failedMutation, e, context.getPipelineOptions().getJobName()); - context.output( - failedInsertsTupleTag, failedElement, Instant.now(), GlobalWindow.INSTANCE); - }); - featureRows.clear(); - } + Map latestSpecs = + context.sideInput(featureSetSpecsView).entrySet().stream() + .map(e -> ImmutablePair.of(e.getKey(), Iterators.getLast(e.getValue().iterator()))) + .collect(Collectors.toMap(ImmutablePair::getLeft, ImmutablePair::getRight)); + + try { + executeBatch(featureRows, latestSpecs); + featureRows.forEach(row -> context.output(successfulInsertsTag, row)); + } catch (Exception e) { + featureRows.forEach( + failedMutation -> { + FailedElement failedElement = + toFailedElement(failedMutation, e, context.getPipelineOptions().getJobName()); + context.output(failedInsertsTupleTag, failedElement); + }); } } diff --git a/storage/connectors/redis/src/main/java/feast/storage/connectors/redis/writer/RedisFeatureSink.java b/storage/connectors/redis/src/main/java/feast/storage/connectors/redis/writer/RedisFeatureSink.java index 3ddaef7779e..ebc1052dd02 100644 --- a/storage/connectors/redis/src/main/java/feast/storage/connectors/redis/writer/RedisFeatureSink.java +++ b/storage/connectors/redis/src/main/java/feast/storage/connectors/redis/writer/RedisFeatureSink.java @@ -31,7 +31,10 @@ import java.util.Map; import javax.annotation.Nullable; import org.apache.beam.sdk.transforms.PTransform; +import org.apache.beam.sdk.transforms.View; +import org.apache.beam.sdk.values.KV; import org.apache.beam.sdk.values.PCollection; +import org.apache.beam.sdk.values.PCollectionView; @AutoValue public abstract class RedisFeatureSink implements FeatureSink { @@ -40,16 +43,16 @@ public abstract class RedisFeatureSink implements FeatureSink { * Initialize a {@link RedisFeatureSink.Builder} from a {@link StoreProto.Store.RedisConfig}. * * @param redisConfig {@link RedisConfig} - * @param featureSetSpecs + * @param featureSetSpecs input stream of featureSet specs with featureSet name as key * @return {@link RedisFeatureSink.Builder} */ public static FeatureSink fromConfig( - RedisConfig redisConfig, Map featureSetSpecs) { + RedisConfig redisConfig, PCollection> featureSetSpecs) { return builder().setFeatureSetSpecs(featureSetSpecs).setRedisConfig(redisConfig).build(); } public static FeatureSink fromConfig( - RedisClusterConfig redisConfig, Map featureSetSpecs) { + RedisClusterConfig redisConfig, PCollection> featureSetSpecs) { return builder().setFeatureSetSpecs(featureSetSpecs).setRedisClusterConfig(redisConfig).build(); } @@ -59,7 +62,7 @@ public static FeatureSink fromConfig( @Nullable public abstract RedisClusterConfig getRedisClusterConfig(); - public abstract Map getFeatureSetSpecs(); + public abstract PCollection> getFeatureSetSpecs(); public abstract Builder toBuilder(); @@ -73,7 +76,8 @@ public abstract static class Builder { public abstract Builder setRedisClusterConfig(RedisClusterConfig redisConfig); - public abstract Builder setFeatureSetSpecs(Map featureSetSpecs); + public abstract Builder setFeatureSetSpecs( + PCollection> featureSetSpecs); public abstract RedisFeatureSink build(); } @@ -101,12 +105,16 @@ public void prepareWrite(FeatureSet featureSet) { @Override public PTransform, WriteResult> writer() { + + PCollectionView>> specsView = + getFeatureSetSpecs().apply(View.asMultimap()); + if (getRedisClusterConfig() != null) { return new RedisCustomIO.Write( - new RedisClusterIngestionClient(getRedisClusterConfig()), getFeatureSetSpecs()); + new RedisClusterIngestionClient(getRedisClusterConfig()), specsView); } else if (getRedisConfig() != null) { return new RedisCustomIO.Write( - new RedisStandaloneIngestionClient(getRedisConfig()), getFeatureSetSpecs()); + new RedisStandaloneIngestionClient(getRedisConfig()), specsView); } else { throw new RuntimeException( "At least one RedisConfig or RedisClusterConfig must be provided to Redis Sink"); diff --git a/storage/connectors/redis/src/test/java/feast/storage/connectors/redis/writer/RedisClusterFeatureSinkTest.java b/storage/connectors/redis/src/test/java/feast/storage/connectors/redis/writer/RedisClusterFeatureSinkTest.java index 7bdc9bba2bb..6c6c8785fa5 100644 --- a/storage/connectors/redis/src/test/java/feast/storage/connectors/redis/writer/RedisClusterFeatureSinkTest.java +++ b/storage/connectors/redis/src/test/java/feast/storage/connectors/redis/writer/RedisClusterFeatureSinkTest.java @@ -32,6 +32,7 @@ import feast.proto.types.FieldProto.Field; import feast.proto.types.ValueProto.Value; import feast.proto.types.ValueProto.ValueType.Enum; +import feast.storage.api.writer.FailedElement; import io.lettuce.core.RedisURI; import io.lettuce.core.cluster.RedisClusterClient; import io.lettuce.core.cluster.api.StatefulRedisClusterConnection; @@ -49,6 +50,7 @@ import org.apache.beam.sdk.testing.TestPipeline; import org.apache.beam.sdk.transforms.Count; import org.apache.beam.sdk.transforms.Create; +import org.apache.beam.sdk.transforms.windowing.*; import org.apache.beam.sdk.values.PCollection; import org.junit.After; import org.junit.Before; @@ -124,7 +126,7 @@ public void setUp() throws IOException { redisClusterFeatureSink = RedisFeatureSink.builder() - .setFeatureSetSpecs(specMap) + .setFeatureSetSpecs(p.apply("Specs", Create.of(specMap))) .setRedisClusterConfig(redisClusterConfig) .build(); } @@ -217,6 +219,7 @@ public void shouldRetryFailConnection() throws InterruptedException { p.apply(Create.of(featureRows)) .apply(redisClusterFeatureSink.writer()) .getFailedInserts() + .apply(Window.into(new GlobalWindows()).triggering(Never.ever())) .apply(Count.globally()); redisCluster.stop(); @@ -260,7 +263,7 @@ public void shouldProduceFailedElementIfRetryExceeded() { Map specMap = ImmutableMap.of("myproject/fs", spec1); redisClusterFeatureSink = RedisFeatureSink.builder() - .setFeatureSetSpecs(specMap) + .setFeatureSetSpecs(p.apply("Specs-2", Create.of(specMap))) .setRedisClusterConfig(redisClusterConfig) .build(); redisCluster.stop(); @@ -277,6 +280,7 @@ public void shouldProduceFailedElementIfRetryExceeded() { p.apply(Create.of(featureRows)) .apply(redisClusterFeatureSink.writer()) .getFailedInserts() + .apply(Window.into(new GlobalWindows()).triggering(Never.ever())) .apply(Count.globally()); PAssert.that(failedElementCount).containsInAnyOrder(1L); diff --git a/storage/connectors/redis/src/test/java/feast/storage/connectors/redis/writer/RedisFeatureSinkTest.java b/storage/connectors/redis/src/test/java/feast/storage/connectors/redis/writer/RedisFeatureSinkTest.java index 0da4c102cd4..0a0259f8448 100644 --- a/storage/connectors/redis/src/test/java/feast/storage/connectors/redis/writer/RedisFeatureSinkTest.java +++ b/storage/connectors/redis/src/test/java/feast/storage/connectors/redis/writer/RedisFeatureSinkTest.java @@ -33,6 +33,7 @@ import feast.proto.types.FieldProto.Field; import feast.proto.types.ValueProto.Value; import feast.proto.types.ValueProto.ValueType.Enum; +import feast.storage.api.writer.FailedElement; import io.lettuce.core.RedisClient; import io.lettuce.core.RedisURI; import io.lettuce.core.api.StatefulRedisConnection; @@ -47,6 +48,9 @@ import org.apache.beam.sdk.testing.TestPipeline; import org.apache.beam.sdk.transforms.Count; import org.apache.beam.sdk.transforms.Create; +import org.apache.beam.sdk.transforms.windowing.GlobalWindows; +import org.apache.beam.sdk.transforms.windowing.Never; +import org.apache.beam.sdk.transforms.windowing.Window; import org.apache.beam.sdk.values.PCollection; import org.junit.After; import org.junit.Before; @@ -110,7 +114,10 @@ public void setUp() throws IOException { StoreProto.Store.RedisConfig.newBuilder().setHost(REDIS_HOST).setPort(REDIS_PORT).build(); redisFeatureSink = - RedisFeatureSink.builder().setFeatureSetSpecs(specMap).setRedisConfig(redisConfig).build(); + RedisFeatureSink.builder() + .setFeatureSetSpecs(p.apply("Specs-1", Create.of(specMap))) + .setRedisConfig(redisConfig) + .build(); } @After @@ -199,6 +206,7 @@ public void shouldRetryFailConnection() throws InterruptedException { p.apply(Create.of(featureRows)) .apply(redisFeatureSink.writer()) .getFailedInserts() + .apply(Window.into(new GlobalWindows()).triggering(Never.ever())) .apply(Count.globally()); redis.stop(); @@ -252,6 +260,7 @@ public void shouldProduceFailedElementIfRetryExceeded() { p.apply(Create.of(featureRows)) .apply(redisFeatureSink.writer()) .getFailedInserts() + .apply(Window.into(new GlobalWindows()).triggering(Never.ever())) .apply(Count.globally()); redis.stop(); diff --git a/tests/e2e/bq/bq-batch-retrieval.py b/tests/e2e/bq/bq-batch-retrieval.py index 99b88a8dff7..2291bd1122e 100644 --- a/tests/e2e/bq/bq-batch-retrieval.py +++ b/tests/e2e/bq/bq-batch-retrieval.py @@ -64,6 +64,22 @@ def client(core_url, serving_url, allow_dirty): return client +def wait_for(fn, timeout: timedelta, sleep=5): + until = datetime.now() + timeout + last_exc = None + + while datetime.now() <= until: + try: + fn() + except Exception as exc: + last_exc = exc + else: + return + time.sleep(sleep) + + raise last_exc + + @pytest.mark.first @pytest.mark.direct_runner @pytest.mark.dataflow_runner @@ -151,6 +167,12 @@ def test_batch_get_batch_features_with_file(client): "feature_value1": [f"{i}" for i in range(N_ROWS)], } ) + + # feature set may be ready (direct runner set ready right after job submitted), + # but kafka consumer is not configured + # give some time to warm up ingestion job + time.sleep(20) + client.ingest(file_fs1, features_1_df, timeout=480) # Rename column (datetime -> event_timestamp) @@ -162,20 +184,24 @@ def test_batch_get_batch_features_with_file(client): file_path_or_buffer="file_feature_set.avro", ) - time.sleep(15) - feature_retrieval_job = client.get_batch_features( - entity_rows="file://file_feature_set.avro", - feature_refs=["feature_value1"], - project=PROJECT_NAME, - ) + time.sleep(10) - output = feature_retrieval_job.to_dataframe() - clean_up_remote_files(feature_retrieval_job.get_avro_files()) - print(output.head()) + def check(): + feature_retrieval_job = client.get_batch_features( + entity_rows="file://file_feature_set.avro", + feature_refs=["feature_value1"], + project=PROJECT_NAME, + ) - assert output["entity_id"].to_list() == [ - int(i) for i in output["feature_value1"].to_list() - ] + output = feature_retrieval_job.to_dataframe(timeout_sec=180) + print(output.head()) + + assert output["entity_id"].to_list() == [ + int(i) for i in output["feature_value1"].to_list() + ] + clean_up_remote_files(feature_retrieval_job.get_avro_files()) + + wait_for(check, timedelta(minutes=10)) @pytest.mark.direct_runner @@ -217,21 +243,23 @@ def test_batch_get_batch_features_with_gs_path(client, gcs_path): blob = bucket.blob(remote_path) blob.upload_from_filename(file_name) - time.sleep(15) - feature_retrieval_job = client.get_batch_features( - entity_rows=f"{gcs_path}{ts}/*", - feature_refs=["feature_value2"], - project=PROJECT_NAME, - ) + def check(): + feature_retrieval_job = client.get_batch_features( + entity_rows=f"{gcs_path}{ts}/*", + feature_refs=["feature_value2"], + project=PROJECT_NAME, + ) - output = feature_retrieval_job.to_dataframe() - clean_up_remote_files(feature_retrieval_job.get_avro_files()) - blob.delete() - print(output.head()) + output = feature_retrieval_job.to_dataframe(timeout_sec=180) + print(output.head()) + assert output["entity_id"].to_list() == [ + int(i) for i in output["feature_value2"].to_list() + ] - assert output["entity_id"].to_list() == [ - int(i) for i in output["feature_value2"].to_list() - ] + clean_up_remote_files(feature_retrieval_job.get_avro_files()) + blob.delete() + + wait_for(check, timedelta(minutes=5)) @pytest.mark.direct_runner @@ -258,16 +286,21 @@ def test_batch_order_by_creation_time(client): client.ingest(proc_time_fs, incorrect_df) time.sleep(15) client.ingest(proc_time_fs, correct_df) - feature_retrieval_job = client.get_batch_features( - entity_rows=incorrect_df[["datetime", "entity_id"]], - feature_refs=["feature_value3"], - project=PROJECT_NAME, - ) - output = feature_retrieval_job.to_dataframe() - clean_up_remote_files(feature_retrieval_job.get_avro_files()) - print(output.head()) - assert output["feature_value3"].to_list() == ["CORRECT"] * N_ROWS + def check(): + feature_retrieval_job = client.get_batch_features( + entity_rows=incorrect_df[["datetime", "entity_id"]], + feature_refs=["feature_value3"], + project=PROJECT_NAME, + ) + output = feature_retrieval_job.to_dataframe(timeout_sec=180) + print(output.head()) + + assert output["feature_value3"].to_list() == ["CORRECT"] * N_ROWS + + clean_up_remote_files(feature_retrieval_job.get_avro_files()) + + wait_for(check, timedelta(minutes=5)) @pytest.mark.direct_runner @@ -295,24 +328,26 @@ def test_batch_additional_columns_in_entity_table(client): } ) - time.sleep(15) - feature_retrieval_job = client.get_batch_features( - entity_rows=entity_df, - feature_refs=["feature_value4"], - project=PROJECT_NAME, - ) - output = feature_retrieval_job.to_dataframe().sort_values(by=["entity_id"]) - clean_up_remote_files(feature_retrieval_job.get_avro_files()) - print(output.head(10)) + def check(): + feature_retrieval_job = client.get_batch_features( + entity_rows=entity_df, + feature_refs=["feature_value4"], + project=PROJECT_NAME, + ) + output = feature_retrieval_job.to_dataframe(timeout_sec=180).sort_values(by=["entity_id"]) + print(output.head(10)) - assert np.allclose( - output["additional_float_col"], entity_df["additional_float_col"] - ) - assert ( - output["additional_string_col"].to_list() - == entity_df["additional_string_col"].to_list() - ) - assert output["feature_value4"].to_list() == features_df["feature_value4"].to_list() + assert np.allclose( + output["additional_float_col"], entity_df["additional_float_col"] + ) + assert ( + output["additional_string_col"].to_list() + == entity_df["additional_string_col"].to_list() + ) + assert output["feature_value4"].to_list() == features_df["feature_value4"].to_list() + clean_up_remote_files(feature_retrieval_job.get_avro_files()) + + wait_for(check, timedelta(minutes=5)) @pytest.mark.direct_runner @@ -343,17 +378,19 @@ def test_batch_point_in_time_correctness_join(client): client.ingest(historical_fs, historical_df) - time.sleep(15) - feature_retrieval_job = client.get_batch_features( - entity_rows=entity_df, - feature_refs=["feature_value5"], - project=PROJECT_NAME, - ) - output = feature_retrieval_job.to_dataframe() - clean_up_remote_files(feature_retrieval_job.get_avro_files()) - print(output.head()) + def check(): + feature_retrieval_job = client.get_batch_features( + entity_rows=entity_df, + feature_refs=["feature_value5"], + project=PROJECT_NAME, + ) + output = feature_retrieval_job.to_dataframe(timeout_sec=180) + print(output.head()) - assert output["feature_value5"].to_list() == ["CORRECT"] * N_EXAMPLES + assert output["feature_value5"].to_list() == ["CORRECT"] * N_EXAMPLES + clean_up_remote_files(feature_retrieval_job.get_avro_files()) + + wait_for(check, timedelta(minutes=5)) @pytest.mark.direct_runner @@ -390,27 +427,29 @@ def test_batch_multiple_featureset_joins(client): } ) - time.sleep(15) # Test retrieve with different variations of the string feature refs # ie feature set inference for feature refs without specified feature set - feature_retrieval_job = client.get_batch_features( - entity_rows=entity_df, - feature_refs=[ - "feature_value6", - "feature_set_2:other_feature_value7", - ], - project=PROJECT_NAME, - ) - output = feature_retrieval_job.to_dataframe() - clean_up_remote_files(feature_retrieval_job.get_avro_files()) - print(output.head()) + def check(): + feature_retrieval_job = client.get_batch_features( + entity_rows=entity_df, + feature_refs=[ + "feature_value6", + "feature_set_2:other_feature_value7", + ], + project=PROJECT_NAME, + ) + output = feature_retrieval_job.to_dataframe(timeout_sec=180) + print(output.head()) - assert output["entity_id"].to_list() == [ - int(i) for i in output["feature_value6"].to_list() - ] - assert ( - output["other_entity_id"].to_list() == output["feature_set_2__other_feature_value7"].to_list() - ) + assert output["entity_id"].to_list() == [ + int(i) for i in output["feature_value6"].to_list() + ] + assert ( + output["other_entity_id"].to_list() == output["feature_set_2__other_feature_value7"].to_list() + ) + clean_up_remote_files(feature_retrieval_job.get_avro_files()) + + wait_for(check, timedelta(minutes=5)) @pytest.mark.direct_runner @@ -429,18 +468,21 @@ def test_batch_no_max_age(client): ) client.ingest(no_max_age_fs, features_8_df) - time.sleep(15) - feature_retrieval_job = client.get_batch_features( - entity_rows=features_8_df[["datetime", "entity_id"]], - feature_refs=["feature_value8"], - project=PROJECT_NAME, - ) + def check(): + feature_retrieval_job = client.get_batch_features( + entity_rows=features_8_df[["datetime", "entity_id"]], + feature_refs=["feature_value8"], + project=PROJECT_NAME, + ) - output = feature_retrieval_job.to_dataframe() - clean_up_remote_files(feature_retrieval_job.get_avro_files()) - print(output.head()) + output = feature_retrieval_job.to_dataframe(timeout_sec=180) + print(output.head()) + + assert output["entity_id"].to_list() == output["feature_value8"].to_list() - assert output["entity_id"].to_list() == output["feature_value8"].to_list() + clean_up_remote_files(feature_retrieval_job.get_avro_files()) + + wait_for(check, timedelta(minutes=5)) @pytest.fixture(scope="module", autouse=True) @@ -510,22 +552,25 @@ def test_update_featureset_apply_featureset_and_ingest_first_subset( client.ingest(feature_set=update_fs, source=subset_df) - time.sleep(15) - feature_retrieval_job = client.get_batch_features( - entity_rows=update_featureset_dataframe[["datetime", "entity_id"]].iloc[:5], - feature_refs=[ - "update_feature1", - "update_feature2", - ], - project=PROJECT_NAME - ) + def check(): + feature_retrieval_job = client.get_batch_features( + entity_rows=update_featureset_dataframe[["datetime", "entity_id"]].iloc[:5], + feature_refs=[ + "update_feature1", + "update_feature2", + ], + project=PROJECT_NAME + ) - output = feature_retrieval_job.to_dataframe().sort_values(by=["entity_id"]) - clean_up_remote_files(feature_retrieval_job.get_avro_files()) - print(output.head()) + output = feature_retrieval_job.to_dataframe(timeout_sec=180).sort_values(by=["entity_id"]) + print(output.head()) + + assert output["update_feature1"].to_list() == subset_df["update_feature1"].to_list() + assert output["update_feature2"].to_list() == subset_df["update_feature2"].to_list() - assert output["update_feature1"].to_list() == subset_df["update_feature1"].to_list() - assert output["update_feature2"].to_list() == subset_df["update_feature2"].to_list() + clean_up_remote_files(feature_retrieval_job.get_avro_files()) + + wait_for(check, timedelta(minutes=5)) @pytest.mark.fs_update @@ -565,23 +610,26 @@ def test_update_featureset_update_featureset_and_ingest_second_subset( ) time.sleep(30) - feature_retrieval_job = client.get_batch_features( - entity_rows=update_featureset_dataframe[["datetime", "entity_id"]].iloc[5:], - feature_refs=[ - "update_feature1", - "update_feature3", - "update_feature4", - ], - project=PROJECT_NAME, - ) + def check(): + feature_retrieval_job = client.get_batch_features( + entity_rows=update_featureset_dataframe[["datetime", "entity_id"]].iloc[5:], + feature_refs=[ + "update_feature1", + "update_feature3", + "update_feature4", + ], + project=PROJECT_NAME, + ) - output = feature_retrieval_job.to_dataframe().sort_values(by=["entity_id"]) - clean_up_remote_files(feature_retrieval_job.get_avro_files()) - print(output.head()) + output = feature_retrieval_job.to_dataframe(timeout_sec=180).sort_values(by=["entity_id"]) + print(output.head()) + + assert output["update_feature1"].to_list() == subset_df["update_feature1"].to_list() + assert output["update_feature3"].to_list() == subset_df["update_feature3"].to_list() + assert output["update_feature4"].to_list() == subset_df["update_feature4"].to_list() + clean_up_remote_files(feature_retrieval_job.get_avro_files()) - assert output["update_feature1"].to_list() == subset_df["update_feature1"].to_list() - assert output["update_feature3"].to_list() == subset_df["update_feature3"].to_list() - assert output["update_feature4"].to_list() == subset_df["update_feature4"].to_list() + wait_for(check, timedelta(minutes=5)) @pytest.mark.fs_update @@ -613,7 +661,7 @@ def test_update_featureset_retrieve_valid_fields(client, update_featureset_dataf ], project=PROJECT_NAME, ) - output = feature_retrieval_job.to_dataframe().sort_values(by=["entity_id"]) + output = feature_retrieval_job.to_dataframe(timeout_sec=180).sort_values(by=["entity_id"]) clean_up_remote_files(feature_retrieval_job.get_avro_files()) print(output.head(10)) assert ( From 20e4bf785e48425a078cd18f7e8db8c5068867cb Mon Sep 17 00:00:00 2001 From: Ches Martin Date: Thu, 11 Jun 2020 07:03:02 +0700 Subject: [PATCH 056/133] Add v0.3.8 changelog (#788) --- CHANGELOG.md | 12 ++++++++++++ 1 file changed, 12 insertions(+) diff --git a/CHANGELOG.md b/CHANGELOG.md index 77f666cf395..e1c56cb75d4 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -216,6 +216,18 @@ See [Feast 0.5 Release Issue](https://github.com/feast-dev/feast/issues/527) for - Add readiness checks for Feast services in end to end test [\#337](https://github.com/feast-dev/feast/pull/337) ([davidheryanto](https://github.com/davidheryanto)) - Create CHANGELOG.md [\#321](https://github.com/feast-dev/feast/pull/321) ([woop](https://github.com/woop)) +## [v0.3.8](https://github.com/feast-dev/feast/tree/v0.3.8) (2020-06-10) + +[Full Changelog](https://github.com/feast-dev/feast/compare/v0.3.7...v0.3.8) + +**Implemented enhancements:** + +- v0.3 backport: Add feature and feature set labels [\#737](https://github.com/feast-dev/feast/pull/737) ([ches](https://github.com/ches)) + +**Merged pull requests:** + +- v0.3 backport: Add Java coverage reporting [\#734](https://github.com/feast-dev/feast/pull/734) ([ches](https://github.com/ches)) + ## [v0.3.7](https://github.com/feast-dev/feast/tree/v0.3.7) (2020-05-01) [Full Changelog](https://github.com/feast-dev/feast/compare/v0.3.6...v0.3.7) From 1aec92dd128349f790adafe01708a43c2ad2ce23 Mon Sep 17 00:00:00 2001 From: Willem Pienaar <6728866+woop@users.noreply.github.com> Date: Thu, 11 Jun 2020 08:06:57 +0800 Subject: [PATCH 057/133] Fix release badge to use latest by semver --- README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/README.md b/README.md index d8f60e2614b..8a3b2198d27 100644 --- a/README.md +++ b/README.md @@ -9,7 +9,7 @@ [![Docker Compose Tests](https://github.com/feast-dev/feast/workflows/docker%20compose%20tests/badge.svg?branch=master)](https://github.com/feast-dev/feast/actions?query=workflow%3A%22docker+compose+tests%22+branch%3Amaster) [![Code Standards](https://github.com/feast-dev/feast/workflows/code%20standards/badge.svg?branch=master)](https://github.com/feast-dev/feast/actions?query=workflow%3A%22code+standards%22+branch%3Amaster) [![Docs Latest](https://img.shields.io/badge/docs-latest-blue.svg)](https://docs.feast.dev/) -[![GitHub Release](https://img.shields.io/github/release/feast-dev/feast.svg?style=flat)](https://github.com/feast-dev/feast/releases) +[![GitHub Release](https://img.shields.io/github/v/release/feast-dev/feast.svg?style=flat&sort=semver&color=blue)](https://github.com/feast-dev/feast/releases) ## Overview From 53cc46adf9d46861271bbf532be493ce5ad40423 Mon Sep 17 00:00:00 2001 From: Willem Pienaar Date: Fri, 12 Jun 2020 10:30:50 +0800 Subject: [PATCH 058/133] Prevent latest image from being overwritten by old releases --- .prow/config.yaml | 15 +++++++++++---- 1 file changed, 11 insertions(+), 4 deletions(-) diff --git a/.prow/config.yaml b/.prow/config.yaml index 54f03226343..7035d5cc9be 100644 --- a/.prow/config.yaml +++ b/.prow/config.yaml @@ -451,11 +451,18 @@ postsubmits: --file infra/docker/serving/Dockerfile \ --google-service-account-file /etc/gcloud/service-account.json - docker tag gcr.io/kf-feast/feast-core:${PULL_BASE_REF:1} gcr.io/kf-feast/feast-core:latest - docker push gcr.io/kf-feast/feast-core:latest - docker tag gcr.io/kf-feast/feast-serving:${PULL_BASE_REF:1} gcr.io/kf-feast/feast-serving:latest - docker push gcr.io/kf-feast/feast-serving:latest + HIGHEST_SEMVER_TAG=$(git tag -l --sort -version:refname | head -n 1) + echo "Only push to latest tag if tag is the highest semver version $HIGHEST_SEMVER_TAG" + + if [ "${PULL_BASE_REF:1}" == "${HIGHEST_SEMVER_TAG:1}" ] + then + docker tag gcr.io/kf-feast/feast-core:${PULL_BASE_REF:1} gcr.io/kf-feast/feast-core:latest + docker push gcr.io/kf-feast/feast-core:latest + + docker tag gcr.io/kf-feast/feast-serving:${PULL_BASE_REF:1} gcr.io/kf-feast/feast-serving:latest + docker push gcr.io/kf-feast/feast-serving:latest + fi fi volumeMounts: From 1395f3f038a13ded29e1ff084e5bee8a0f2acb60 Mon Sep 17 00:00:00 2001 From: Ashwin Date: Fri, 12 Jun 2020 15:28:06 +0800 Subject: [PATCH 059/133] Add Kubernetes Pod labels to Core and Serving. (#795) * Add Kubernetes Pod labels to Core and Serving. * Missed a dot before Values. --- infra/charts/feast/charts/feast-core/README.md | 5 +++-- .../feast/charts/feast-core/templates/deployment.yaml | 3 +++ infra/charts/feast/charts/feast-core/values.yaml | 5 ++++- infra/charts/feast/charts/feast-serving/README.md | 7 ++++--- .../feast/charts/feast-serving/templates/deployment.yaml | 3 +++ infra/charts/feast/charts/feast-serving/values.yaml | 5 ++++- 6 files changed, 21 insertions(+), 7 deletions(-) diff --git a/infra/charts/feast/charts/feast-core/README.md b/infra/charts/feast/charts/feast-core/README.md index 40d6bd91129..f85b4223c84 100644 --- a/infra/charts/feast/charts/feast-core/README.md +++ b/infra/charts/feast/charts/feast-core/README.md @@ -12,7 +12,7 @@ Current chart version is `0.5.0-alpha.1` | Key | Type | Default | Description | |-----|------|---------|-------------| -| "application-generated.yaml".enabled | bool | `true` | Flag to include Helm generated configuration for Feast database URL, Kafka bootstrap servers and jobs metrics host. This is useful for deployment that uses default configuration for Kafka, Postgres and StatsD exporter. Please set `application-override.yaml` to override this configuration. | +| "application-generated.yaml".enabled | bool | `true` | Flag to include Helm generated configuration for http port, Feast database URL, Kafka bootstrap servers and jobs metrics host. This is useful for deployment that uses default configuration for Kafka, Postgres and StatsD exporter. Please set `application-override.yaml` to override this configuration. | | "application-override.yaml" | object | `{"enabled":true}` | Configuration to override the default [application.yaml](https://github.com/feast-dev/feast/blob/master/core/src/main/resources/application.yml). Will be created as a ConfigMap. `application-override.yaml` has a higher precedence than `application-secret.yaml` | | "application-secret.yaml" | object | `{"enabled":true}` | Configuration to override the default [application.yaml](https://github.com/feast-dev/feast/blob/master/core/src/main/resources/application.yml). Will be created as a Secret. `application-override.yaml` has a higher precedence than `application-secret.yaml`. It is recommended to either set `application-override.yaml` or `application-secret.yaml` only to simplify config management. | | "application.yaml".enabled | bool | `true` | Flag to include the default [configuration](https://github.com/feast-dev/feast/blob/master/core/src/main/resources/application.yml). Please set `application-override.yaml` to override this configuration. | @@ -42,7 +42,7 @@ Current chart version is `0.5.0-alpha.1` | ingress.http.https.secretNames | object | `{}` | Map of hostname to TLS secret name | | ingress.http.whitelist | string | `""` | Allowed client IP source ranges | | javaOpts | string | `nil` | [JVM options](https://docs.oracle.com/cd/E22289_01/html/821-1274/configuring-the-default-jvm-and-java-arguments.html). For better performance, it is advised to set the min and max heap:
`-Xms2048m -Xmx2048m` | -| livenessProbe.enabled | bool | `true` | Flag to enabled the probe | +| livenessProbe.enabled | bool | `false` | Flag to enabled the probe | | livenessProbe.failureThreshold | int | `5` | Min consecutive failures for the probe to be considered failed | | livenessProbe.initialDelaySeconds | int | `60` | Delay before the probe is initiated | | livenessProbe.periodSeconds | int | `10` | How often to perform the probe | @@ -51,6 +51,7 @@ Current chart version is `0.5.0-alpha.1` | logLevel | string | `"WARN"` | Default log level, use either one of `DEBUG`, `INFO`, `WARN` or `ERROR` | | logType | string | `"Console"` | Log format, either `JSON` or `Console` | | nodeSelector | object | `{}` | Node labels for pod assignment | +| podLabels | object | `{}` | Labels to be added to Feast Core pods | | postgresql.existingSecret | string | `""` | Existing secret to use for authenticating to Postgres | | prometheus.enabled | bool | `true` | Flag to enable scraping of Feast Core metrics | | readinessProbe.enabled | bool | `true` | Flag to enabled the probe | diff --git a/infra/charts/feast/charts/feast-core/templates/deployment.yaml b/infra/charts/feast/charts/feast-core/templates/deployment.yaml index 179e3a6a094..8b2343ee9c2 100644 --- a/infra/charts/feast/charts/feast-core/templates/deployment.yaml +++ b/infra/charts/feast/charts/feast-core/templates/deployment.yaml @@ -30,6 +30,9 @@ spec: app: {{ template "feast-core.name" . }} component: core release: {{ .Release.Name }} + {{- if .Values.podLabels }} + {{ toYaml .Values.podLabels | nindent 8 }} + {{- end }} spec: {{- with .Values.nodeSelector }} nodeSelector: diff --git a/infra/charts/feast/charts/feast-core/values.yaml b/infra/charts/feast/charts/feast-core/values.yaml index f56897d2852..d5ad50ea156 100644 --- a/infra/charts/feast/charts/feast-core/values.yaml +++ b/infra/charts/feast/charts/feast-core/values.yaml @@ -150,4 +150,7 @@ resources: {} nodeSelector: {} # envOverrides -- Extra environment variables to set -envOverrides: {} \ No newline at end of file +envOverrides: {} + +# podLabels -- Labels to be added to Feast Core pods +podLabels: {} diff --git a/infra/charts/feast/charts/feast-serving/README.md b/infra/charts/feast/charts/feast-serving/README.md index d30da2eb3a8..eb638359e7d 100644 --- a/infra/charts/feast/charts/feast-serving/README.md +++ b/infra/charts/feast/charts/feast-serving/README.md @@ -12,13 +12,13 @@ Current chart version is `0.5.0-alpha.1` | Key | Type | Default | Description | |-----|------|---------|-------------| -| "application-generated.yaml".enabled | bool | `true` | Flag to include Helm generated configuration for Feast Core host, Redis store and job store. This is useful for deployment that uses default configuration for Redis. Please set `application-override.yaml` to override this configuration. | +| "application-generated.yaml".enabled | bool | `true` | Flag to include Helm generated configuration for http port, Feast Core host, Redis store and job store. This is useful for deployment that uses default configuration for Redis. Please set `application-override.yaml` to override this configuration. | | "application-override.yaml" | object | `{"enabled":true}` | Configuration to override the default [application.yaml](https://github.com/feast-dev/feast/blob/master/serving/src/main/resources/application.yml). Will be created as a ConfigMap. `application-override.yaml` has a higher precedence than `application-secret.yaml` | | "application-secret.yaml" | object | `{"enabled":true}` | Configuration to override the default [application.yaml](https://github.com/feast-dev/feast/blob/master/serving/src/main/resources/application.yml). Will be created as a Secret. `application-override.yaml` has a higher precedence than `application-secret.yaml`. It is recommended to either set `application-override.yaml` or `application-secret.yaml` only to simplify config management. | | "application.yaml".enabled | bool | `true` | Flag to include the default [configuration](https://github.com/feast-dev/feast/blob/master/serving/src/main/resources/application.yml). Please set `application-override.yaml` to override this configuration. | | envOverrides | object | `{}` | Extra environment variables to set | | gcpProjectId | string | `""` | Project ID to use when using Google Cloud services such as BigQuery, Cloud Storage and Dataflow | -| gcpServiceAccount.enabled | bool | `false` | Flag to use [service account](https://cloud.google.com/iam/docs/creating-managing-service-account-keys) JSON key | +| gcpServiceAccount.enabled | bool | `false` | Flag to use [service account](https://cloud.google.com/iam/docs/creating-managing-service-account-keys) JSON key Cloud service account JSON key file. | | gcpServiceAccount.existingSecret.key | string | `"credentials.json"` | Key in the secret data (file name of the service account) | | gcpServiceAccount.existingSecret.name | string | `"feast-gcp-service-account"` | Name of the existing secret containing the service account | | image.pullPolicy | string | `"IfNotPresent"` | Image pull policy | @@ -51,6 +51,7 @@ Current chart version is `0.5.0-alpha.1` | logLevel | string | `"WARN"` | Default log level, use either one of `DEBUG`, `INFO`, `WARN` or `ERROR` | | logType | string | `"Console"` | Log format, either `JSON` or `Console` | | nodeSelector | object | `{}` | Node labels for pod assignment | +| podLabels | object | `{}` | Labels to be added to Feast Serving pods | | prometheus.enabled | bool | `true` | Flag to enable scraping of Feast Core metrics | | readinessProbe.enabled | bool | `true` | Flag to enabled the probe | | readinessProbe.failureThreshold | int | `5` | Min consecutive failures for the probe to be considered failed | @@ -65,5 +66,5 @@ Current chart version is `0.5.0-alpha.1` | service.grpc.targetPort | int | `6566` | Container port serving GRPC requests | | service.http.nodePort | string | `nil` | Port number that each cluster node will listen to | | service.http.port | int | `80` | Service port for HTTP requests | -| service.http.targetPort | int | `8080` | Container port serving HTTP requests | +| service.http.targetPort | int | `8080` | Container port serving HTTP requests and Prometheus metrics | | service.type | string | `"ClusterIP"` | Kubernetes service type | diff --git a/infra/charts/feast/charts/feast-serving/templates/deployment.yaml b/infra/charts/feast/charts/feast-serving/templates/deployment.yaml index bb8fdc55ae5..610dbe5feea 100644 --- a/infra/charts/feast/charts/feast-serving/templates/deployment.yaml +++ b/infra/charts/feast/charts/feast-serving/templates/deployment.yaml @@ -30,6 +30,9 @@ spec: app: {{ template "feast-serving.name" . }} component: serving release: {{ .Release.Name }} + {{- if .Values.podLabels }} + {{ toYaml .Values.podLabels | nindent 8 }} + {{- end }} spec: {{- with .Values.nodeSelector }} nodeSelector: diff --git a/infra/charts/feast/charts/feast-serving/values.yaml b/infra/charts/feast/charts/feast-serving/values.yaml index c94c7205264..18069c28634 100644 --- a/infra/charts/feast/charts/feast-serving/values.yaml +++ b/infra/charts/feast/charts/feast-serving/values.yaml @@ -145,4 +145,7 @@ resources: {} nodeSelector: {} # envOverrides -- Extra environment variables to set -envOverrides: {} \ No newline at end of file +envOverrides: {} + +# podLabels -- Labels to be added to Feast Serving pods +podLabels: {} From 02df03aaaaf7f16e82613bf2bb04591ead2ddc22 Mon Sep 17 00:00:00 2001 From: Willem Pienaar Date: Sat, 13 Jun 2020 05:14:09 +0000 Subject: [PATCH 060/133] GitBook: [master] 2 pages and one asset modified --- docs/.gitbook/assets/statistics-sources.png | Bin 0 -> 165507 bytes .../deploying-feast/docker-compose.md | 59 ++++++++---- docs/user-guide/statistics.md | 87 +++++++++--------- 3 files changed, 82 insertions(+), 64 deletions(-) create mode 100644 docs/.gitbook/assets/statistics-sources.png diff --git a/docs/.gitbook/assets/statistics-sources.png b/docs/.gitbook/assets/statistics-sources.png new file mode 100644 index 0000000000000000000000000000000000000000..02be233968d3c97e932f8073b66d41b49247782c GIT binary patch literal 165507 zcmeFZbzGC{|33~0GC~1SK#74UptPXW7^Q+rnIMgbfHWJz=mrHDD&44rQX->MYDx@b z3eq`7$3~6(-P3c#^ZCBdpTGZ(f9x@~d)Iwm^?E(yb@^0RTm1Oak~{37IDkWx1qk3x4FL4tb;cu-pFW_gL(AOod$|GH$RoxGDh%jeH8mFVw%tI z5Q*i_;!x0Zg%ETH*>5Y-k6ynfs(rZI*H2clB0VR_-(QmE^F1-~a}-4ies(WTr@!%= zm*agRA5=rxq#f`9>GwT|TJBJvjGvwVWW?q4lM&*6b}47htiaD)`ewpBy5_70TBDnS4x*40z%Hg7VNIp?1FaN+T4Z zXWwwps^%$2_NpbYDXp0chVB`%0xxF8B#Aqqn{z|Gx79bo*w8u`1O%a$+;7aJ$IjiUoExn1)I zj;?UU)2GQd`u*?Mb6UFF9NftP_E)!n4vLe%5tkH`5dYma@KXr+Q+Zt*cS}3{%Qp6w z4lv*zP)TW7X~@qX{6AkE-13i~-aq)MjLfA!fBDBZfBh07PVT`UJ^D3VKR*S=3rY_W z|2_6ldZcCUdkP9=ifflu?zmIV50UDQe^1$8;TQVI@OeDI>Hcx*X78uM))4}arNd9M zzDfv{eARB@lequTCn-TwfcRPS^@_q(!KVz1%x=$bO46SeD))cQ+xq$=DZlq?XUfL~ z&zZJ`+OpiGTrbp+v2oiwtCaUQYUWmF0->TIlj(j`^g{phrJk9c<5E-ENQa&>CCzc& z|MkMlz!7iA^1r_$UoS+>*q(p*U)l$5e4Og=)V)Lh(m2onJ>?--uK&M1pdVGRpMS;o z|Mg(MKJ!e;4x{>01gMU**gc<evqYl6v6s)=&2N3VV4f76z`%*&>tcYoZ7 zy?Jn7|5N@oo*P-Nb3K{9+hx8tYgb=lb-75k5D{;K6kJ_yQZj6<3mLj>IQW_-X7ZvW!Sv?#oceOy*j2+8Aug) zR++aJhR3XK%@yLeDiFk}!G5m~trH%9ci6v{o+EzaZu`vN9{>+(%}UvD&0tYG6(VQs zySp~Ciq+}#ugG^Fcu0Hk9mjm<)%9~j>5S|3&$${1Y5P0)nddUYpPOyo&IcMh^cMaN zx`Kd5)!5AY`%j^C z@Gy6f@vk}Ncnq*qHG5=L%>kI@=2x~Ic4xlRVw=i+oI`|iZp7?MmC0W}unqz4Ds|jA z0jT&{ebIf=P%2s{4qhG^Aq=3Xb#_~mzUKi z6zKi+O(LH}-bn`MlLt@of8X6PuCK42_3R=DA570b-$`Eqt~eJVcKP6L|9qD|3v~GI zNMNk#KR-W6*6HVfE40kveE;|E2aAqfLZWbs2etFBS7@_6Y-R06~D zeIp(Bulj@aGvJEb9L8aP4D>(lr~I9Uu|2nzR)MbDebz~^7D$O6;|nc%O%f3~e$MensWlf7Ud-In?LnYtcP(U#puh(!KlP`Vub zL0=G&y|mnL-Dm0%;a2xPTditjh)%TzC6Tl+U(fvgT1874LtMgyzq_pzgq?t z+7bH8UEGw2cD9TsB;sZSp0G)nBUs^jiy)~M9#FS~KILQqlXr0=Df8YR)9!CVy*}Uf zYS?Fc0p&1Mi2`DUw8!eSwD;DkG!UbtAGOCr@R8gK-QnW3@udz!iRjK$MTH|^C`588 zR@AUNODn0n+RA$)GC=D!*-oq5^k?eFK- zr2)Efe*SD*&hL+au56!350+`%UaCt=^&)&({b7O3v>zzNp=rThQGp9-d#l;+a&v`u zNu9ph!g>Ym=JMvU*v@1*sT9|q`*ILLP~BWXRky-Y^(!_J{Vwj;X)uhOO-sbdnVIY~ z3~_0&LW*`8&Fy9+KM^72BBV3Dc(6@%wJ3d8ixamyZvDB7m7i1ilhnc9(W#z`6{|@j zI|5}qf?TH6?#_u9EUz1sJihDTs?D)CXSy#xjqJ>C z0V5_~405;S7S<*zDb2in)qOV3=w+2{Mq8|&3F2(Ne$naB+3M`@69F)T$TW}HcNNT@ z8;c$Oa-T@S>?ZNra=7SfuLgouDnh{6Yo*y{{zuWa_=npE%N>;otSG}pHimN?Pmb{=nPf>&pcxEK^-Z+VZ@de20u9sMGTK|sgD_`4T}s;Ug* zH%jKU4|aGL57m<|-8|c$ZXG-!kbjhDtj)D^YP`4CFPmxq(3 z+^%;#=JLzBOYH|-;%l8JS_Y3W5U~Nt$|<$&OOmCA2ka-B-rB?8xC^ca&C1W2YQX z-UmH-_g@*l@Wm^PT(e4czUrW*Z3{cx0GrCrW#2j{7^Q?yh7qhp_E*s0&NlrIW-5si z=BlYOzSH{$Pja68BwvOeW&Gnw?)lXxi`@iO6qb)a+j*0mt5=B0kQ>TkPQ=p6iPqa( zt92S{K>6%$4c=u)Bl0q5ce#I6AWz$*LM8;Wzmg^=_oiuMkxZ@!`gVJ7O;;j&2~pZ}EAcz&-e2fPULU|qfo=s! zS-+Em(9p%C$#WJv&l(yQw3bP)@LAoUK77@s^~mNqc7-th-e$ zBfYyme>}BPP?qdL{GJr)T!QVXU9y3q{|fKQcp$sG?LnU~98BVtkTN?Le4@y|<02#%X>=YrmQuFqdf^CDbxxN<{6qnEoKds{=p)-cP8B&p>%XV#n-ctKK}sC%`? zzDv)o%*%XoFT!K1I(r2dEOS8%v6Zo&D1TiMd2i~QapYMZ#l9G%V0C<`Bi-dV*^&^E zB-gHzqjMG>m7_EE+|ChFOl_H-snZ47Pe`?Y?Kc06vu0*c)Owj=?(fm~0HzE;X(e6C z^jxZuA&2pe+Z{}oP2gXzATTZ(TDt0C9dNfnXZ8LHnAOqQM<@la!0Nco(UvX*J|zykD%Gib6hrqRVSR6VNrzL$p*?MdzhGpuMTUSZ zjm@?+=^=ii1-}LED6R-3M>$!+9VkEoh)h-8oe(u3+ENp8@8!47{rF(6wwSy3u}$m- zIYn>45d!xR(-vf#^MthgWvEtU67J$0Z6$rL$FRoo4${hHiXG9a+Bfp)lZ;Q z#Jat;!k>wCOfLV9j_g<>;N$&&&9zCo4p;9T-T$@_|TaOo3*h}a273(JFlp5B}zd8FKdFb zzF9lg48i%=doA>rAh;o{(O}Q{BGM;#(GCGp`&UZw&)^5<9ll;-c!2i0(kb``$#|s7 zJ^n6=O$kV(RfaxsjnXFA2eI`Ft-^)#N81yu#+#nH_O?L0t_lxGTC_|qC&iXBu!(%e z#>I7lhOsqG^L@qG6M9B34CjxvVjGf4pd;MJt@S9W2vVI8DK}(;Wb^iIZMn0(yZx1z zOUuvcBGU?e)^cZn%+k4*R_W6&iJt7mA>6e#ga_?jUy(!8h<>||CZ~3EFU*`<>QVb= zzMZYLIilp1umgz23pjs@2Vhh;{fzSTCCk?6)wM!j#jhV@VsSv|)wX%I?|IEkeS;|o zW^8O+Q57@om~d}`5$A{82yzwKPqZz#20!%4b1Ff?oC@q~W9GB*Q}M* zKSPRehF3LoyluV`cXe>Se?(rk#iqz-cda+OV7%PYBhoeYt|NCI`fsWj66xnp>gwxt zcy|yV4^aj6Quphbdb=K-2hCnZ@(fu-wXCkHXxpxEtg~0M__5{?!~DcjEl%RBdOi_s z{sJGs0-}UUxO+pB+19VLtypen%vX?yGFGXY*NO^mCzRtF>u%N34lIyaxP50Z7 z9+d=r=m=K=pf#B2==gi(HCp{mm*vTH*~+vk^8la#q#(Uxa+_i9!HeeriD2z#0j*os z5+LopaR`eVsIWvA6|S;BPJW=*W?NoSv^QHjrJsA82^Bu~7ryjgKMt_P&fN#^4iewt z`+m#$U)c(dmx!AiYbr4l885ua)f;8-hre;H{W=q{p`48ui2j{QblD^%#BDbZHm$OM zCk<$H>r}&2U}t|nw#-r3v-c78>!BHg{K~Mjs(r41Vz=J`j||jNHGZP)13v8c|CWMu zmHj_4|0uT7F%dPs^uxZw2KV@7ZGQ4#p04=95HCQW`)J_iSEfSuLeH6WxMuQ3x^Hr zq3yH=gKGzB4yvSnxW*IP@|+Xv=(|0z+PS~eIrCI7)q0L#APYQI5`|3jlcnL5dj&=b zd%Gk1io0`ld$wfyQG2e%>Tg&ofdYUc6tz)jcmRoY`IXzSDGF|6-3ix|opi~*^)~a{ zKD6_ZSjQ7NITUDOPrf0xzr+^ijZV@Iem>+v0c*3$NxIn)Ar_D3Rv_P);>*>P{7PQ0+F7=76o zHv&?72Q@$@8Wz6cSuu+7b-bjP?j85YY`Av2<>kQwVtlP^RDNenHU2_(OuKHC6S!%1 z?7b&5ukQ28&u12RFwHC6or-q%GS%ZYC<`aN7cRw(#F@s)q$V*+`XaoFO*T-&{kY!Y zsHYbseHSh~9_bac&8bFbeBXA=(QkQjw7Jq!@mbACH9kWhnV8*MdG=&{8K&p#0aL!C zpH9y%5&a@6dD(idC*KB}rBy0X=#-M{@s%Gn_|fu4Kd?&+lAI^r8JFeLqe*-7PE4Lz z&wK3*7uLUX>Y4b6ggd^UAFfeIGaQGt07+3e&EkQPV`ov`Zhmn!<7`_P3qL_Cj-zvD zFldZZsYoQXO>JSDV8v1pXS@lCy_;Hk+q8ILCu|Mi6ofr~S-0sliZy!l$^;q@re7NA zelICAP)*iDfm}QV3tY?D-?kU(d9#;$r2QCUJ8er}$?v(PzoY3F&Y>Fe#G!;cqo-=A zHpz$7W}20GBW^1FG)NrpkELFn=ImT+HR0q@@9NNE zq-D-5j*adhVHU%=gbbs=;DwbpQK#F!JxYNk8Rx*_*4bmD;%gb zr#xLG4?z1?(Mo%g_9L~5ac5_BAxs98!le&pnVI&U%h~s%-~h0}dmN9v*?Ye9KKEUt zL&V7b6o-g|EnQ4K{vazw+yE2;WW1EdJbbFB7yw2aDBj*bQIA^%mhuU)O*%a?scG!s z7{SvVt!KPh>dxbL_fj=I*MthG)l#IcMcyIxD^)UYvTjmrI`s(T5}+9!a#D2;aDrO& zTVKK0PMHsbTH-@`86i?fC>7=&H!}}tY-$PdjA)M{zUAB`sc3u&?40pU?K~}Ebt+)( zDXZX=Q;Ni^-k>E5bYus%cplW7Al`GcN^uOcDWta`eHQ`VUIgR~4KhHkl{_YM?1y$} zEuAh{qPYMNmpT%pvK1t(j7UG`2VM@i&E^Pl?TgoLJ~-M=*O{+QTif}X=VlxrioJ*g zg8QNeeDsDZ<`E#K%iP9V9&AW>5QRT8dm!~?)Po4{VY}?-{PX0*y|se$O>UGBs~vw` zn$l`>KO~D{&AB0nZU%+bZlknR2+;^?4r*W=bRR$b&5Vhw(V5GFjj8p@hlW5GR->Cw z42X5R>jNY%WuaR0#t=wkr(CGykxtZ6yest#t}Dsa3Cee8ip5_#pU0W+PL}3o*Y0J0 zd}IGE_rSMTj+&ivarUow7~9no-mEDMbUyi@sL6?4=K3UWDF6Wj;m@rLBdhwP6uBV3 zHT{N2ys6F|qvi^oP>JW^GmAdg!!l66NwD%V3HJAB479HWSOcLW_GKLDuz!b&i>H(>k zX%HiKz`*2n{la4bxk%90)qAMYb;z`AO0_6O^;km&f28bbZ-BIaVgImjr}StZo5L+I zjjTz|8|vtwq6O*8(?mj>cs8KY35Irgar*!OJkTZLHV=RORaD!~62k#jx-aF$L^CwSsdNvR?MttiO^3E6`IjP+x6tNOap9F~JPlX3sE;&hs z+T#*7zTH+`EqL85(WyUnkMlM9iMmGX1ql-pC(wPmr+%2svdGN)ZZ?9mEF$F|3EBH#OJh7%=ZibD*5-Qgux;I3b*(_V zv4H+LQ!&v&UG`sz`&ahnr=sQqFo-r2bkl5aR?;*k)Atrt{u73)pbsFgR&zuCfoW-a z#vlqb1D|pgw~UGQ)|ZQkCb%R4CeIRp9|(@O;1-K zWsmEF)Tz7kLVM@maG;+M>(6K56lLf^?TD`H+=5$1_)q25Hl0>MGaT4n=tQ4G0E|oW zjRIflOb?FCI+#@9I}KqUuOK9CG8yI`_vjAaYc-8=k+I$0uHu?~;KdqfX)YTb^222A z=Ki|;S{UqTum`djtbE^}K4lj5ta#-6G3#cUc^|Zr#^ zLW>lZ5Vm2*%7hu4$AY>i6!Saix@7G8qYuFJVYC7P`#O`^+sD(}zDG~t3c2kv?9z{l6uOSSaPly}E2x_-%UrW_m} z{JLqREf)o#-b3#eDi7ewd43==Etrg+r;$R2^VC7@G#8RN#IlLxk0&sv#h_Oz zpH%j5lwf7Hy>;do6rhEA>+#4WO2~)$v#K4S-~<^}IF)Oq<5-qZXE`h|z*||5lX?!N zDtHGkaG2QroMGFKR&=WqLTrbRKNXzfWWO2WfwL{W?Hj8uhZwzlzcEXxjq5xyw&sB^ zQ;dD#5GF@`%uMn9iCL0&GJit>{@m_E2_T-(-X3v1z$K^@3O=eKb`e0u4ZREhUOyVF zHL!4=;cY!7Ls!2CpoNlvHF&q`S!^R?zC4Cbg9Btc;=M7n3h0#Cbvrom86ab}4Ha~G zgZRWVWR;??>+!B$>>!0Y#iqwn$nr#|WtbpNqPI>*e_>s}VDB}I>BKFF)Gg+>+bQvx z$ToRJgtN`Hcj6gQs-^(x-@AEJ6<%4Bp^$tx_a1K1d0H)X*$dYiGm~JI`en#`cSgsz zb$jmHZ5^%bWS8rU+sWRnbL|Z4zVz^NWiCVR+61iuNKN9^zHS--e?|8OAN2E%mS4RW zpfS$?{P@c!I%6&gcmLZZiZ+N7IpXz2nmN}P#=!fVXt}{jf%~~gX9EdV6k!|mQ~SuH3OU~UCj5tc6&8HJ3*4G>>8Uw{}lF; zz!!X9^|znqPsx_P#>dCz|I}29$IMGFyPg8X{IUgUN|0B@-pfnuX${H^dha&ms^@SM z<1y`-430iyb~ZXQ>4d?{+hQ1nn*ynCPQY{9`kyU0mRW|_*0{tNh7>cszM>>^XPO5O z*hgAqO<_;vrccxwS<&6aU{f&mWka!zt8HRcBEB$xPVnHo(@#|*hitO{Q_|<4@E=x< zq13O*1y}+lb6D&^zW#|Z;UO=;pWqt;L~N2u84ooMpXTf1!~EU_<}g_9T;U4S!AS@$ zUCXMN@RVPYN_7N(9Z%yt<;C!{z-7A8t0VE^3;+eS$fCIe;-_`AA2mI)J2a9>!@vd~ z*|c`JW}HX{S<1q|fEm7<`*@Ha&kHkux(3mEq9#&KMZYRkzcTdRBDp@f2e*pQnseD| zgaU#Pc4Vhx?Cg+;uzma> zN5*%O8k6F;p?wHn)lc%STmo3Mxg~WX2mVt%1;b_K3-!6KuJl3@%2fHXE%ybK1hxS4 z>WmM^OK|w4`qjdk^?ovD%qBxHwl+=nxxyn>wAMspEiiTKw$S=O&doP2ZlcuFtNz-) zc(Oh&iqV0rGz|`EBWj^}m}@`#dH9;y58nLV@8=IFUsn$mJhMJfw+WD7iBoshNJ02U0LkMG*)Y1nvt_%~OV?j+m5lLcX!5Hkj zh1#w>TU8%S?v*InhNsJAxcl44%Xv>h`5B-t-S6+_c9SWGAtPbJAwp(3XV#=-klyR2 zpC1gMR5O`zCavi8hF}IF($UBH{a)Qf^`}ZuM;c-(0cF?saVPKvpbY}y8Y4ZNK3BJ< zEbhPC!XF}PSPI|Y6jUKapSw+v;n>Xb|F=n24yOUhkO8v#!?WFS_Bss|{T|wP2hHVl!BxsJqWPn> z`5urvGYuUo&K7RJydidttm^KKY4n^FjZT2_M7UXl{Xw7k&mfbyyu!@@uvbyIHQ-Ps z>ekp${^gZmna-H#)1SK_4OolYO}M55Z7W4g+38C8+EkL`Pb5_Mt+*rT42#c2J7#m! z*C!qjsp>*{mxAc;+~YM^ES)MJbKe8F6f8{39aDDQ;G)wox#Z-T)6R$SM}q@1dUacO zJ=XiIuswm*B6^EkQsh<3ggGzSX*kxb-5Xa|Bi06SIPX3%!x`JM}~*A)FyI;1M4Ek(Q>9t)6pZvs|`4ukg){ZX~ONhEGPny|1ABj|(&C;*r3IpHHfPzXB zz}8*`+~_Gsuw*KD{frPk1Pyp0L}*r>hd~Nwd}u)@ITEjt<6&8+NYO#J8EB^2#y;~S zHG54w?L7ce-%TfYJl`-p1Pd<7Q+ zL%GUZ|2*^cKsZZdcXrh?aO2=zE~&YmYn7R~*BKTkkV?=MP8voczH{kl zGBHeGAL$x-Qc7M_-^nXc!A)Bpi1x6}>%Afna`ndGc(Tv;rSgeLu2^z~Mw)NsvwPFj zc%TqKhgDF!U1cB8P-atKodyZ$T7@yon#83chHTWR$L;j=qst^A<8n|7gUTtsc>&N! zrK_z^^-2pTLdp4dAowW`rb-xV?%?$K_KK62Y?>CCvQHMcwnTC#8KxF{0kaCE$60^_ z>$#2GcR9%JY#?TM*Mq3oR4~e`Dkg+71PQ%<$N^fi1`zHja|R zBAGOv4nkrdOHVszXEmrDPjiTR#w@3}yMssHWQkoGGPcuaiI=enbGY@|QjQXyORIW6 z;GU{u!%_tdpH=9E6Q235XWHTYj*MSJ@$d~5+3oHd;+a68mSAZh@TTKJRq3o|YJ)Ai zTvCYNflSo>`2|1mIYnvU+NyN4VYYe>7p04RS5@7HV*VSYiF3CRIUBX5gPPz0W@1mR z6@XbyB_SxIN|G>EdmXfZgLNkoEyvJ#lpVsUbk2oR#%b}`}rgWBg<%^T;W zhkt7DN!Kpz#9tG9uiencc*z?<8f4@}?E#y_#qB)&o$9Dou2Tdl zLr;Vbjc|em^h|1^1DBJdlI@$BZaC;Vy?L;;Iz#$|L>)_>xE{tMBN&gaudbHyBD2#w zdO%35+{V28Bh~8iswTQa)9+qUyB0t^U4H#lc)@*Z#lC>$CBIg@COztvG8bp#h_KIc zF$j(TtIb$OJZ^yjgk>4RWz_{H{w-uwYS(VNjRR$Wtipl zmlgZWO)KPSx8*#H9=d6wR7h8WV8$Hl#sGjkv@)nF=u^-%oV_8?EFM~Nl08FyOagn1 zipaYhd2b4(Avldopq+c_m)r|Kp%(slDkGf5AQyW=6Lr;!C(9i1{3y%mxU%9h#5o1g zTDrQE@o8-1JAMKKH>|aVol0MICE1@@o%&RbIwq)rzetokl2)|0Y{^+!S?R)o(W0)z z(K-6m1vm%c?F8A(j}I`nJzEe$BAuZ>C-IIUFs1$5o!x&-sfHh~fC2MWvu}lWPxxyc zZAC!+7MZt79IWo8(^C!J8GnI{>0ytbsN$Nwy4<&Q(bxHiVt$-cQYWaG?gO1Okbe8P znOUi^j}v(l91LI2#ZG%BnI$NvbkfC7go==o5ffbMHnM0IMr=T$>(wqyB`Lz@9Me#g zM8Vxd<}xZWoza22YWL|H`tA)A+W~K0f}5#Tr(WlVcf(#IlM-ZvVpNfd%8AS7BF2gu zAB3}$A`;978d5A<2*mKt{a)-QLHd^mn^Ouff|8B2%!9fY>leyF5~2px;dhn&J*d(` zqt~F1*ULfB`wk@6;$CbV?tAQ;^+(+PVxJrk>35|?3lYp>te1$qcLclyWCb+^A=$V~ zonNioP*=0hUQ&@?N|0$-2ldcpCoGlpnzYRc71ovY;y91hXrZk%Z|LgSItDlf*d8uT zh)P;vu_Go@aDl0otS^Fh8;I>k4#%?cDG^(qQ}#3x5SLHh*UML$;P5!L-_Y z^1bn|SLpGUS(a53pWu*Ib9b+MN1M;lwmr|t?p>)Mr&2#EH66grpI(5K>HoIW(%C5+ z2xQrB2q-DA+1*;3A(sJ=X_ebDwk?lbHh6-W7jjnFq{hS5pb~C#G_bIm(26qq6d+Cw zXdCBnelo3}FR>hr3kBu!YfQD6QlnSyi-d2in4R2o5{sB=rBs+Oy!hb=^)N)pFi)w} z-x;S~(Ox9FVkE}jX4y~Bb*v=de@!|^CD}}M8>zw_jG8MoC~%Xq2JreaV}$jxPD(xE zi0^fFiKe>CHY%I#`GQ>BQ1+Va>mP0BPtWf!y?AUdwk-dN$8a2)MOUf`!g2`>6ZU@1 z1wn5;FhbNwilw%t0eXqo!gl|R_b<0j83+}jLBEFR*PXq@lx?qXt!6S%Rh|bZD%o!f z%RF%rEjtvsT!{^>J>0_441vUrl=Z)_Ht9LBWZOG{-Rz(7`l~%tGH!m2`^#SaIr6H0 z%mTFyU{TS~rjV)^^o%V2AOn8t$PKlJM9%(pgIE0A>QraZ9{%P~m2s zLP1Fa+)Mxsnj}U7 zX?3Mm3Oqo0wwN=_|{6Iz@bO3Cg z9G8%GMAtk>IBuQlhju557{C10O3G<72BLP~D~HfOTK%Ij07}+T1HP&_)aqbZr|*zY z226nuab2?0AOC982-LFm+ayh{qjb|*4IjW5osy-NE$ULrpNr-l6Z_?+`$wLTkXZU* zJQs4HZmz1Qv#Jr6d4&ec98GKa%$#`6Lwo6&dDX+(SC|WJvvX%XOGDF>Gq9!k@3)&= zeg*8Vqrh@s_`ojm6W{+5j|V`^ypDh=lz+ciz#ibQ+<;SL17vkMqXL&4HcGjHu6OxU z16jvh%~Wz^ZidS9L97gi0bk22xwvWyD4YQdpFw|5jMlAR1owZOHIO&(6Fc>jXaX=3 z1=JkD$>kPH-b+k;yz_D-u@Vf(nLOW>rUV-uNWt!eWUCuiK;5?6o zlHP9N!s?fX8Mm51DFb=sHEf~wPhGH&eE3hPqUh$iggp}?^P<1qgM z4Nx8b>XeiN+m)239#TMQE1Bkygw}TF-PSlqZTDn z91Y3fA5bH=Zl8uItu(P>du!KAR$~gi&T<0<%m{vF(0p%a{Z&7VDHtNKZ*Z-s&nyT1B19Gzk1}nn0h;VTf_#=5^aUEluD6_ zXuA&N6@MU1G12{^0Ka0P5H*MnDL+u0vRP7X6at0F`l^hEcr4yC*iAoi1ua|uHw`X35i?xnP&An4q zd>w&$ot>1B2Ao5ZgHNEnb9L7-IWHWp+@%>eI)=UuDOU7i!~1b`j+{0@2X>y+Q{X0| zA=Qt8R)px1@&XGCWS=0KAP}@x9l^xhptS>^m*ntn^Y9VlYNta9Yd;T< z0slD$&#_;)`d2-|as>tZw9yaZsO#OssuY#x6&u5zSl|5(Up!m=s+>0vjIz$&h@I5+ zGvf=OOXZd~^z+taaCS)FSud;8B1<|)YwEU#mZoBHUzrd(d@>V=WdP9I^O;SlE78U8 zjqGpx&h%NOb$nvuJ1+AO)ceM#q)J4^)Xoi3=9wUxMit0cFR5kSK?-z?2WtQMM;=ai z)tB7{716z^kSu$$(7)pr#(8Tnf`3xrGF}}u!zN-71~WN*94|Y$^T|=j$zcrANFa^b zQ`z^ZNOZ1hqKjHReRASPCV6EAS0C+=?bxbBd7ws$Afpj4*5F)wp9oCmBdAZ(7*M&6 z$?vHnbvQoRPbqAzu$7qgo4aF{1B4N#62-;!19@2b46_R2Zh&+EwZWbuP&F|F=z{*nWgd<=p~`$jR=SI&5+|Gc6=08z{$HgQ{s|u5HSz&vz^;31widSF+r+0hX^M|p`BUc*<{_k%VsQW(#-N~-dya*A^{Eb1B zLqW{xch6ns_u@55Z#&|X{vy^p2xq$8-|NDwJDhsnUN7`cwn`skr9%41ilJjo$@aiu zrJXS*EFgtd`Oe1#Tp60qH))clz7b@x<1~M<*XUL4M$HoL!?UPom1w4bV-N3IhAq6m zju6eJIwYzbr8(zJWffp$5T54JdX6L`xyemIWv#lJo$7Huv@2);zIM6I#UX3OA|*Lo z)~%}{DR!265fJhnYW>mh9S%us_v_@vJMpnS(7C2*_QIyqC9scAa8Qz z-8N}$9Z@q#WBsPI{==doSl*W%@d09x$=c1jxbb9>)7r5Y$m0uq%gs~N3pWLBWc&m+ zKVRz88M+#tXAB*>*g)CZs@J|$O~m|E0NH%WMawSw7^ckTCzI@Zg%dP6tU*1fQX?|q zK<3cu8985B3<-`Mr8OwDfU3d_0!)AkPB~>@gZadft7!%m59ME_d;OsQ$>G9Q=V44p zY<@A#xa?HCU`|%siH3z&I#4;cjwZZ#^|I>TJ=SBWA5(?`$yT6RQ=lec(3_qpl^Q!r zJ?n*8QUB2KT+v~*`#qqmQ21=GFI3RT523ow9v^yI#26yPIueypB5y7y`ZXc8z|?vO z;{s}@>mzM{COvb-(zLJ6U||S3J;G04>oK6%+FtpK6#T+>eDySPY7o4P{ic%GIvtz! z*fx+DCu9ABa69Q1-x~-Z@y42=g`~UOtpi&1-$wl9RW=~Ld&D$Y+x1u-y>r01CKHFd zeG#r796^PZ&%onVv+r|V+{C{rRr+R(co{ghAh*}tJPqW0zZJPV;3|zusqP*h{V}R9 zU^n%{1ki7{fb(0arlvr(`qP3sYd-Bb9TYvjy0JrknDr6sOW4r|au877Nw**7p|}r; ziv}3L6JA`rW9_Z|SxZ(Y?mS!&IO(?FY+xdgA&=Egx+GfWVa0=SooIPJ=Rl*A2EmwoW9t;(=)8DxdNEEG}oge>@C@%Yjd)5YEdTU6hWJL%1UDNZ7HnDCZd^Dz^ zb%a)7ETHMZUf3QAc`|gg^{$vLY0W>jB0CDF9M}8Vt4KS`9?p3xrQqutD-{cf#l*26 zb#8xih@7S?pE*^XG)F`Sy1?vod&^bSIIeJe0i|D9ve%Z|Su5M*LOYuI45IWl18M$#PmSfHn$1oJ7mZA(?xK8}SHtr|K4%h;VA-3mf;#6*KZ7wHRNZ zhHa`o*})PtMYr9i6vR#Zd8`nqW28Nhrj85IIgM$h<-jdJ(Lj~-pJ3$hli^E`4ku;a zLVXCO|YbsHy8pllQmO7s>wm@y+Df3EOse*Rn z7YEv&$jJNZh<&wh0E{gcMSY{y``T_a9r{le>->Wc;eUcZd z&l*d748m{On$Asl(#Wg$O$5{OZIWw}kTLsPh5N0>Y0OcudDdsxBlIfL zLnRW-`Y`@yb-55&uMd1sra@Zd&L?lNy!A>&NV;>z6@41|IknSh$L%imG)+=!9ERy}DW! zh-e)MKcXT{NuM7|ciTXb*@yVXHK-}*GmZe%5fY9~GWQPyhZ!m|1%|l2_P0lrMw7Tj zq#QPxAzu}#)_`InoetMGWrT0Bgrk1J^a4r#GC^v zVlAujDd&S9#b6;}f8%fbsVDpM2iHoQol}Inj;=+)&KX0#F0lJrO(zE58$$`;t6NEt z6p6ekj`CkGvigQkp9+SZPAg_Kf#t1?UYwAvb?;h|^Q{_pq`>E4b(9p-)F-N2NXhE$ zot15Elkt&j5~=Dt(}GHntsX;1+?tcalp|H!7}vGwVa&*B+p7x>p&s8N7MxDfGXk~L zB<06dVh2RJlzp_}Ww?WZ0&(;m7rk*XfR~IgWK^>#13Gp(V*1^@Go|r1LWj~E zvDI(WnidnMms$YPia77ua>Iq+5hmX44=2*2=n)fB0%b<(7n<@~hj9Rf=e2LczYwV4=4&cx% zuHZz@t03E*c(ADYKv9gGp>p_rgQlzH>&^LFc|6e@rnx(hd*?i9RCzIqE2((@g3xGx zi(sGT3b{6>$H$Cdk(tO9VD@E3;wJ{Q1~(^ncE$Gz;v_EZ{mH?I2R@i<$w`|@*Cn*~ z@yTEgS#_t>*Ou~lI|CnBF*3;+p4L!i$pd(I)podAx;Ooo-pR`cm zv@oe_aLM7o>#O&nE>9mHuSy98Nm`$4>+PI*798dOQGNeIO-$O%tg%Kc`g_%f2;i*M z_gbW@RI0jtSZBFD8|C3KhB4X!wXnVcW=pGtAQ{wr3$M>~v^N@=;ezE~x5&$VQXyGu zLb5Qbv1x4^gRRqw?$P-^@W?t>C)U~c{D{G$?w2nm!dyPqIz1)5X2c;pJvuIchUtny zMKhf71;-5JbfKleYaf`%XIKa0cn#wj z_rfgMIz>(?;gWPPw_QmMHiFgzBD!$N?njj2 zB+KcpaWSziPvILz99MIAmMl2i*k4u`q=?0G6IW^>S_?ap-kOcugzCIF=0N7) z7ZpS~&Z#5JhV`uwyKvh2OT>0*oNKgcT-;0}ZMz{Xn9Na4~<$f~PS z41TSIYK=O_bH$!BQjqE)#wJ2JeVI&@I)vU_C>u$w>r?9+-?=lFC|Iz(|3e4wl8MsuTRB+0^z8}6zqqz>m*$lPrmd>XJ1Y_pZtqu+&?J zm)Wi>glb**5h@}wbV zrH%BAH_5CD(p2d_RepL@d_*Fmldg6`-`pqyX<2kMGCw{Gl3I6dCqJz7!5*O$Ef-=r z`l-GmwP$%}!7Oz%0;X7C+AFad%iQnvknw^M zeN#_X(lvMNaU)c~dD4#U=&gE0Lfwmb&ppxl5$L+iT2x)|pz;)?^Ern@*RqfI%8^U4 zfX{h~TUULv>Wbp%h~Q1+`|?gv7z_aLNpmyZ2RKb$8Ne6}LSo&Ch?UC`Vu4hQ70ixpyzbNyD%m)*(Q!`XSVCysO4?XEFk<6ui;ICrcjbv;0_qHp1 zfoff`YPyr40y-adag${6a?)26`J}^gQhId#3kJQc3{(BB5}axrIOAao*^jB*n$0B! zpPv9_&nzXmUT7-4rCl2S@uN@H*?F@*>T5w=)I&yG#P1%ljawBOE&h0*9!WY*K~=l; z`bfpxx9O<+Arg}%n%H5(-i()F1 zi(G#sCeA*Q$I4{)Vd^SnTl4F|j|l`xZWXCVOp2?zVvPV5#p(*%x0l~kmIGlyfuMkM zw)TiOgB@OvxT$rhkH@<-1`s|Baj8c;)~QJzL0`McS}SdnTJ^%x%LcQ>4_XHCG=>sZ zcr`5QijHvoe{8*FR8?#D|E++qIkYIbT_HLrQi{Cv?ScrI1^SpDKgzsHdY zmbnBlj%`sX3N|&xAJ*OFSNdcLb}6^s!f!&`POvU2607Jcn+Tx}oqMNSP=_-Y86_f} zyM}x-*QAz9Yqs$&PAlG1dge!~V~%iTsCzfFoYzNz-38Nk?gvNc^MoT0TH36e5w4<7 z2fUp53K7@%Vh=RaiZ@4l3o=C?|36J|bU&C4^o0@^Wj#HF( zk6`}OUhA3pPR-quNmAYeC!Mua>i z1h^yyOey%ia36kk{HWf?EP&=C3}eE;Uhma(y%u7hQPy-FHJEJGKoiKQqUG?VxwZLX zO2dA8FhZdMRd^eiapMWxc`jt7`@;J;fVacYqmr9EpX@bci?SN3% zy_nv3*%~KqP3fBw>?2@?F#6I%TS8xL$T@EiIR>f4nJFz9YhPlQV^zQ*!XPkNrfMSR z@smW>r14EME)>d^ox+fv1Tz6U%x4QWH>53a8^m;gaQAYBa`&Ee4)N~d_26!4?vidH z5MiYCob<9J@tr#&SC*vlADOvlLkl@P<=6Fjx5(A8&p5o}`UnQ9_&4zd?>7-R205L2 z&cotzc?r80n2dCnFHF>+(95e2Mwv?|GPplPpst+B?I}XDByK zxO}-*`oSAVyq-FBiG9n4`!|$tJAk*Xrq2Z_;MMn%@4sVP z5h`#6`V-J8bvTwPmi$;O`VvH5M4UJ}weQ4m(?!W7NiPz0Q_z#`UQ^hWdjQRaJT>Q!F=plCcgFD&5sH$`>s#UvizDkpR&jL$@0C>IeF18rbw6L6^Gg zJObn@Z9}~NW1uOq>0&AtK{G!BUe`~55%?+2+tP>+?lC4AALabl00(G$*dNJxYIgn+ z$_fK3jM0zDfqMEE{E8>%C&wviHM88IOEInWGw}dlRKln+(yrY1<7d>nQ^)%C7glT^ zDw-Z-G4psUJwXQgPb>_YfmovF;)?sCY^AM{;vQe21{^H?7?D$CG zyVmA>hLaT>({CBgwsM9jP66K7D)|llz0BVhLcM?}+9CVjZX8R3MQu zzAc{?x8FM(D+YQ~yiVx8viYX;1VQWaH zh85z-WFE-z$1OF(9rn$L^A7ne{E4@&ZUZI>OHHQZm03}`(I>~eRiEhAluvv<-8P0? zylXK0-SGp2>8vO^)p-uw90(v@XUZ2_Yw?yVZGzOTzNAxa6)uE2%f?`zi*38W$fSeJ zMx6G1&=&b`6JpyFS#!;pSo-sso z9Vyn3W7i|9#xdy;-sCA%0k#9mYrU-YdR$oO4?-M?*7l*d1P0471G18s4oUb--;&Fl zOuKq*KemdR%fe9YBConyJLEY&G?zBNaDcdWa0MoZ5Y!UfLAgI$(+*rjo#Ar~f_ewm z6C#XW(FmeDcCcWW*Mxd4Er6Yi>BBH_1nPe{hQ1gaN6uM9diV&`+M5khT04p-=tKQ zp z4{HP!PHD}v!1P952T9>l8*nkvl}1dL5Axa$Jza;|qyXj1HGd>^7}qHsB30rDRqc8i znI5)_d>l64eCm5<O{Pbh?i$%Bq|lMu3HcQfM%~f96u#7%}m$beg6`Lo`9>FE3Lo)RYbzeo>Yi^ivbKv zx+|RxGr>BHHc?n$Uwu{~LsS%k&zaJ~*`t6~uh2K7Y^A9UG+XQMqE21v9MwErh__sZ z4TgyHHSTt+TkJYd^=|;`*%gQ9uMw`6aqeWR=9!TL&a%`A@?9|t`}&m74ljv%B88)Z zmW3kdG6`J&eE`DZOlOM@(6GBO@9z$iX@F#lagp+S64z%AEsvdo4U^Z}oWo&>Vv_A` zE$n4FSUpV;hi^9F$2qCJygaLm5&mZ+V*B8-&J>@c3OgIw&yp9X z&Xl&4mEi@V+)e{|yFVia7G1`iZ6AdfY=Jpxt{3ab}l5P+t?R z+~cR_R~DKbicFVmnfaD@GbT_1c_9=sb>pWUlYUVV`hXH9Q`us~wr*d800ej!x-HQ5 z>Y3(pF;6I_z|42lxn>>|Mk{vNv`_oMUiLUu$73meig7HXkHs$Aoga&V{9^Xw)cmxC z#6IIDDJ00!K;WeKRdZ~sqtcc^I!E?kYICUTQu&Y`!O_47-_Z0tFZ9M-hr_jKav1wa zjj_iwg(@3p2j98nY1^W{+uyiX;3;$8)wR%>=qQLMa9YrXcB9Z2(n!l_G!}_32Qu$L zro)Ek=fBa>#!27~KtoRkb%>VdPhIbd9C6F<=T3gnXW!$NoS>ASUOPGu)hIAt-w6|D zF-)RbaG+h%h)j@g8fo5ZcvCL`n7g^_AYmb&^Ni*Z6F~H8Sj7t`EW^POuk}89yy! zwAnV+D~*QeOn|{2F`gwBa_Trbi8SARdDGgk@3auUsAfs9u#{gMPB2XV7zcEu=Bg|P(Cn}AgZKsH~8D=cj>Vu5rMtSjC>=u z3fZDSgt&>^?bm_Hui=D&S&FCYh^O8jGXl>d)^KScm#V^uw0E}~vgA2#U~4l!Ko4p zy)SNj9um})cTH;MO}k+Ibab5uBhXI3p?VW)WRD71w7N{)s)XRMOKiiS3kMi3rUCbi z*~1flSP3qqsHKq8vzzNuFuut&fGZ$aU{$zy8geM+=>BUEp`p1F(d zUJmgIRaawPPXM*5eDAKJwRQTOF}{uYf4o^$*b8i(Dsi#`l7(NZ0r;n{)OG&4-pLn} zjPA%AVNkz1TuNrQ&h2s-<&!WIvQ^N)nl!BP9~zBE9Lt)=BX-@3s}HZN6R= z^3jBOA&kHfj>xo@Ro=8PH%*tWdkL(`^y<)1!L8Z?U8($MT*=K^!g1WE_zOCK^cL?= z+`W;NL_E9u9tN}daiqeTlE5%g)7`GTA5H5umV!8g9byh*t0&Oj*wW=`$Y1S-+uLl9 zb-;LyxpD@XYWU_Lv_076cHYxJy}Fa-&oXuz8n>T{0WRj1ygLoCgSVY6a~R`5XVRzU{E*$_dKaZYx~3!CMHauoWksGynn2^Fij<0P*rYpWG8 zklZZSBN;@`&ABYBYRM>Ffg`)cv?hAGif)T_mv=y4r!wU|s>1SGeeAs8;#S+IkwVRY z8MIK^7UQMUZx>6fCHbUNBP?0G7yV=7_s|nMUv8jOM!X%c_q2sXy0V_qgCe?1GO#|A zrOB(ru#3xKL`Jb?Ou8j8?47d5DPhZ?t3I5F&_Mqt=)x|1t0(H_#9&x@fz9&#NOm^w z^22;b{pIs4f~P%V`g$!hr}df_Ifv!R_DHMaF-P#hiuEvHx0CL^As&;{-X6j(#%~v6 zZ4OO)yjwi)ht^WMAM+Vj3DLT{hOn&aY%2Y4@h%`Od{ChBrQ3+$s{8aQpN96ryX9iS z<8fGGabrPxcFmf9vG$W^LIBpz^Uy&WuRB7CkZ`{Mp`mm{Mg#>n-Pzm8|8iR`(ub# zLYxO(#MGYO$mshN!HqzHdp*?p`fbY2cU5)IrLI>W+DQ2a2gKfIvKW!&WE#CI9xl2V6G8M*OcPcMD6 zmIk|Ju%~%lIDP6=6UTm3Ch~do3tA#K(^92O4yoVxE`2JC*x$y^6psJKPjgWE#o#L* z?>FACdt}Ir(+D-b2!~tEP;cEyJ2<=KcdYz374%igGerSh zXRTcCQ^hSVB1HB2SuxegW$sQpm$H=83Y;ES&dY+z(Ncj6c+cH8cm8L@TNpkpn?cHX z1gt3n332P;9 z$XtL{zNgA>IE89P;G{Ox()h`)G74WKWxX-DB`}3iP5W1-E6Zc%>&rEr9uQJuzF4)eNd)l0qdE3 zQ;Yu8omp5>lEq--uIN<28WRiDfD|M+J#7i>iB^v2F9JQK%jn?qR#}G$2cM~0txgmp|tGv)RYuGWjU->z#2&AyEy zt#!*Lk5@PmON*5bntiriy|T5{Q_;4%$%;V|X-9*pmuCD`)wqYwusJY(!p(4x_;w1L zz7p{TKFVNy*+y2vAvp+Hl^?rbQ&fT$XoCM`r}r^ZbdSXco}r??H+01(!}rECQT+Fy zTaC9@)F*!=`F;mr^k}m+^+uJ$XholJug;4q(qwi}gtMB7Q?mOs#*aT%{CpYVDZ zr}41fNnX|4&(kcAU*4p7Tnd#*VLs*3^s-4;SXvn$4-u~hS)*^XpEVxxV(YAe42nJ^ zV4JvXXu&#Mq5CX}`81Ar9+5OW54(Qf8hf|2MmoF{b@+pGwSyoL3lkk8IsMB~sGo!f z;|YQyeY=W&!Lgs8@JQN0DaQa%KI12ODU-Pf+y5HBz}zHt0FR@+qwt*KS#B3>rSrlQ}=s5r_uXGN4xc5SkoAex?y$#!hXb1M#;++;a2V!>H0c27`McG5BzNx8^3#JJF&C6g!^ zN!y9vCm^0WR zZ0M|YDTBC@-Q_j8T5b;?rawn$yDK={o-uxDENPJ=Wpzy8P(G;WNt+xnPx}#9s;U55 zCIPmNsnN0Q-YSG^iU;HLzS4Vdj^bcEEV4ua5f?d|ub<#;|LUSNsUtGH*8=l;X*w+O?0 zeiEkME{7V1Ax>*|B-gz5BW}&mPG4dO&Q8$X(g){FY;5oiy*2L^4D5(m%UBZ!#02aM zUoQnZavMH6R#9I>vIsUQ77z+X5Yj*+UrOv(Q)OZ!Z6)G6Dcy5thfFGOf-=;yiHZKanYpwAOyNaAr5pG3?C42 z=%5IxA~}dbftG8-!%Sa;#W8k|qbk}bAHvEO(8%c0+;p@;SyVi07m24eo0J6ZiWx}J zS6>?mEa4vUtk$+*PNyZ}hp8I@Ol$kvyk~9=sO)8i4W_c@ijHkQuly~hzyU2b75Y6n z0d-%kJEP8unm1dNS8m+6^`oq|DTu`23 zKInzur>txdj`$-f(aKcCR&DW>UXiXbf5CPRk-tydt^MU*svW-KFG)F1Ezyr{5u05Q zf)RhoHgQI|v~$IETKuax4i(z9kyer6zGE&MY{L}UEa~egS%02dqt$rmGLZ|;r|mFj zfRRez;IL*f4iY4b1W~~J^TXcDO9*#uGn93hTD<|nTH+yytHAnc9l0HOh5;aU?Y5si z=HLEiHUEkEq!Y4scO3YWo@24@R(W8foA{_+zFL1)c5*jPoD)TNm;_}Zs0E zLDeg;Dy)RKIw0v$@Dunz;A`w~)Q`%Xbe8O!71+^7cHMG1M7>@2xSlX3nb`ofN^N8MPQveH%#2-z0MkOv}x6tN8oXsoC7O6srcnbYrZEkcfIb;%&qnmX(}`s z{&vxYb19I#nUCk9*p0cs^m5{kH7CClL);v{zMAB&NNa+S> zkgEOR6(%s9OIRRB=iobFpir8>BT~Ga*JLRKWlt;wE4E>YC|;;}3hf8l91~O$KU1zc zKAW0nO#NxGswza|*L`Cg7F6u!^HHocl++O8$W4Ae8)ZDtRl-f8Pp}?YAUWkS586fD zNriZ(&hpRi0erd31A^j+aF3|1h$pw}gj&C3@;hWA(;k2uZ&K`%tLkrmf} z1>JwWYJ9Z1ySmuBCfvF%d2nsIySUnInO{!dR%@a~)311`5Y8#uY?lhZntG4ivOBWd zvVQYAs;d2AhcMj3x zb3`ge;w~}#P$o)vnIvkFi>qJx3#Q{$h(Z;#CKp#pDUYEVPw0~EO`fVIlKf zH^Q%mFsN&T7}V`Uuo)SWY$e~r*10-kqO9=Cl);TxA0r~In( zTcV-~t`Z^%hu;ovgg;xx_v%RPHi-2x60<`|Uq<OIU0x8 zt6^|X@`=aA4W&cE?a6OMo0BI#Sx0S4iW~$XvgNRmY4N{bKE>JD78GnS`S6lug8#4H zpDy(D+RG%0QTdcFF{U6MhHta92)MbNr_r1x+_sxpLTzH@o;^(ob9tNfj_?vUxqNkC zuXaw+DFnxIWE>J$@xsL9h^>L_j1`Xj%Kqn_1bmVUI5BQ05$TsraJuIMLMAxw}-< z&l@Q*u&b33sS8so6ywdC@)vs7O*tl9pt=di+kOg5$+$4l(p{`)94#OJPh2Po2ao4X z@^ubDQ4$WJSSranE|h#A-P|%b;CE)VhoGn7Mz|W}~%a)EzhucpfpOd`y$YkaHUfK=nGQkHQJl0$PhLD`esHc$s!t|eh z9Mr|cT}vp|IRK;ee^@|g0Oi;mB=kIJr~A`-Vn|UMM8@>Pk(m~W@%nFC)~;tFlTJn| zK;9>mF@_O@+07g!ii99A@q06pQ%}VO8ux zf%`(rWFb{zhotaPVRLT|-W(_%^u^s=&rR7MX;f~OXkJku3AUpxtk9^49MfY`gXV9hC z&1wvxVV#}vMbx)zeoEtWr~UvIIy7jk-Ao^1VVd=cMj|wV>1IISM5vk8?mxp7XmQqSPvEPMtz# zFZ~HRf!9jf7!JUjgh~A1Ki_%Phgn$O_?xZ3Y_fqJW6dL{zDWX^ zn{xyG;9DVbQ;&5g6^cC^tE}#Fmw53UYEF%#Rnz&MHctSxSPEG-R3I7`Vs0z)`L76e zCeKdanhNI=H_C&6Iv)2B3b6$ES!hvHGIiD@9UUBh#b4yF(hIE-433Zrfb?t$($78E*i;m8Scy(C7 z_AMp7okI^?^zvFE5u}iNE!n17DNcrR@~G#G7Y!cfR3R~;mxghvu(9aQf$h|Z(0jS| z^+n?i&%cS#Ly%`C4aEsJP|5A zB7j_-Qn3_1P4`}oS-@LB5a2K@9n>8x>dC?q_&{O5o!TpN8i$t5LgXSO-OBEczJf;hH$CHWx|{F;r30 zKi0|yqaZ%|M3Fhxsr!!oC;|&eJtCCBCSNiCCv%pB%Y%3ceL*flt|F|WjEZ==(0Td- zB89bs8;Wl}&lP@Me8-P*CP813Cx$44r+6)}6N1^KTP8nmuP{J|WY^E)42RL!0lFvQ zY`~a;8|{@q5fbF5#Z)Gs#sn#-*X(PNi_aEcYSt&nCLZ%Cvao}jfE}S=M7SgPfyNs((vRbV zYim^j!`~m$#Ab#O&EEeu%-&diqqYyfP{uTWfLuZsccuY**!sv7%VmYAqVX~9>KC^9 z(b7ejQ;NTtzw8XH(}*bIRxO(pzNc(E=RyS2K*$Xha16wW0M{ka45E1Pf1{lP5DsK~ zsVWq|-p5b)FOp(Y*RdQiP^#mcnOI{=@#b&UF9*Q5b zSCP0?BT+9kIXVuM@5j>*$w|^;!Fvd^xeh)+dP& zx(I9X1xb@@zF-rNkV_b|aOC{z2x=9|7}*=$FVV4d{gf38@MWiqaG{QZYH@+59iph3 zviyyV=`r>#vhXwlV7Ej3BR2`Hi!_@>C;9Y!zKgD-x+6m9P%z%X7}r*c(&cBtZMVGr zLZ16i?0%VY38okva=4Q5KsufsCTRW6XSDm}T|4Ms767|a`hST!@Y0yPKsci0IVFI+ zkG{AUJeOrRzyta6bd&z12EW3F|GcWI{LHvxMnT`u#Wj?Vdb3B#27dzz6Qd8)$IC9q zpiw-a)w%gddKLX*n{sXFlJ*oY(-~NDdpo4=SeBTd;q7+XM}=KMyOeD8)jdD0Eb+;g z_&x*uq@ScB1;w+|Fq+T@r9NOL+l7(-S#+`*Ks<8zP>bM^K@!2ja7ov$0DCcnT>xf$ z2|?!jZ}@lw+Hm zm9l6eL@x*{-kt1 z?0UX0RHimwQ8pk#qHtj*A- zQXRB*_k=UH@i^cmDv6LFp8Z&jv3~b&;pOi`7z%=cpO?gk<2q1-L$Z`gg2p-H+mL5Q zvBMw2FNN1Tf(Ua0oP!uE4gfkTHC!tI5YE6Ijnu1ydNRyNDSd5S zDW78)dGVZ*=>-RJn%<$WQ?6bIO=yo-D9HKp%n@;pSxC%~OGTya=a8pGMMD$|iDD%5 zdVUhPO{du{1MOCGqUa;6X8Un`(w1-P8v%;nq?Kc0=EYv-+?8` z)2b0n7iGa(cJa5N(NKwk?lg4hCg|Qfx}rdnBwNryCo>XyoI# zt)=MY!#f6xgS&$?-ZDJ+%cZ+iwN!!ko2fvvI9=F|_AG6dyp3=9FL?rtgZmhXel!7W z22IsN(wnG*exEwh>DVuDEvH%grY$)^=EC?)FkQ6K2_Uv---(|thTHaE>GwIJ0F&$f zCK_lrlg6{rmydm$# z2m+bAyfV3aO)_lO$CB;yqUoS!{-YMhXds(P55@#U3BiaBAUUzxOG6 zI7zqn`MC^?GQyDgf~X%%{whKUG?29eVBajewrwakF zvEu`4&qkRE;I27petI}kVVjPca|rB=krQ?4tbOEdcP!x%r7zw9#0=}ecb*TIvThAe z(@4cgq5_K8#_RsX;86XWfCwp66ZWITdS+5sw=3Os!B4e~&QstPooS3_XS>sBih;3Q z>4CrCjpSNpCis8pa2jy}PUtugM4|T>nDDJBTW^v}9ASr>dz{nhp2?aL1~y#q*8-Ze zBFpXH&I(9+fEt|On^A6xEnJ)N^Lz0SIsstjAE_LNWi^Ixe?4R^pw3Xg&DxOWd$cn1 zSid!%1DiC4LDeU-DbngO6e^g0W_$RmZ&B)Oj){$Bw_a0>EHbzf>SM3PMBk%jx(s}4 zd$?NGkum0EMH9}BCNwj|v`1m%O4ekYvseBl{{0!6VPrR$m#7&%M#2z1IS{mDMwKwn zY=aW$*fToPNzFq*y}B})-kNqde;<$AvF=<;~J)Z|vHQ2=vw z=JBvVs{?*DR8x5*@uw{+QRgg=?}iNfP`W@%f*rHMYa{2#u}E84z~Gk1A{9?4uEp0& zVVN+k`V!UbrJYO(>M37{nlJ}B-Ed;yP{RpxvP9N|M&$*~I>B@sw%-Z9V*I3f3sh($!{?YUHz$I^3d^fz>t55_Ve>`!jGMOacH3;4i6g3c-YN zmfWFWHe$B1?BzOCSLc2XGS+U!er_cZ5ECYCKU_3^uY*y6Tmmi6pt7?uOF$koE1iLX zfxisCpDV^qeu^}4``^qG+<`=j-cw@G&_1%jx*tGCE{q{$?~G+Mq&TCrP$M5Go|5vh zGsYR2RN3YDuDOJZ-M{ynnmEUmykxgw^1~bl^urjl%8YRx@cw_xz@($gU+w|0>E~zFi`ip3 z_Bs(VWW_o0LPJM^>OmAzCST54Au^T)WALB>HB!Hy2$ygS>BJ-g{$3&^lcq?{Pj=<7 zr=phsW3p5hq-F10rO1l_%4HdrBzHYDO)g*7yXiS{xZKkit7G)YmfaB75tOf~QSdn~ z4Szd#N@2jmApRzm#FJS7s_>Jt9E+ceHR~((?~6KWfA||gxF66fD39M0zk>-A_Odak z=*gf~{s_silZCt!ME3(#E`5NI9nbCed_>x)e`iq@jJ{P4P<8*34eL({mHB@q?ba01 zj0m^gANfC}8R~NYWN%=w@g+@uisJq{ybzN~RI*+xM84vE{gs|mtr_gc=G~VnI>ZOBs zfC!6n4XF?G4klm{Ach{A_|C%{YO|wytA6lC7L3tgwjGxp2o55p;PQ?;_$aM3TP+$q zF21hqc2F%GS1DBYb@HeF-YI}2q8pe*%Nplk(=?78(?AJC2zKXC!A)4Dgi#V}W!d{cKY#b!=WzgH^?2yCMG}$y zAhXdpC8iMku+aBfq^hbOOCsYh{)A&!K)_UCnEE3NfMOS0B(2Q@SpBl=@yCbH>xoBy zZruk_uSGlBA4Tu0T>Gs(W;vQq!t~GFaAXlgiTOg!_Dp7RtIM($kTn{@wg|>Jip&UV zcn8k^eEeU}*EwFhP+U5|Ux-DCOwi&2M1q6s5%CGt+rR26VCU?%m}^5vL^BjXBm;>` znI8g~YBO=ePRHeVicRB7TOM!ysnaboR=9@jH`QktP2x=FEoB_<&(a;iD0{$ z-_!@PSxMQmOlqM-$ltyw0P`hY?!FU%6mB@zql~jOy#`Ba_fZl=Kd3*??Ef~j zHO)k&B1`-!s#}H51n&-D><0dlaY)wkETkF#^OgYXV>7|M`ctep%qE`} z8x4UhG5Ei@yMP?{CiQL4Vm}HN+{?hdIvId(rJ4#X-iuWNZ2*p^)--#n+qq?SlSxxv--nqluov;bQ9?>Eq8L%&`hl{DV^{5pWHvijv3KpRu> z%l1X(F0k-QFmy2)JUv_?5j^$;ZiSyfhF$6k&|5-&*-{3~7#aoSYQR2{3>6P@0)bZ{ zAbzYe*gC^2`%AIszc^S{$m_o64HAl7bQAl{0zh;q1h7B#bAVXHlgiWfB4^U;r%(yt z9{Psx(qY~(v4bKJ5PkVG(IN>)!ZhZ`HAKoD|2(s*-{B6-W|uw-aaYWm%+QVKe+MDs z!T5O!KLLYYpV^t+u2%zTlU<*^NSEVHKL5p!3KvU`@wYkkD1s3JkJn%0wH$wdFDui( zJXahC7AC8|{SyuZwQ7>1dr*Xp6_zyoG>xp31p~4x$^b$_f|)!|-kyo3W>g$Mb}|r9 zYF^Tm;OmeFjD&$)SDxr2MY2VOl$cs@XbQu>#}dfKUV=h%q~d;4mJP;X?E~a;ZFU!- z^8>(po7r>ib+?iW+7F8x0kW%%>fa;)1!%ReR)St|+y3i}$)#222_`oH6G(f}^Xb8# zO(JJrvQ|a!ii~f^M(qHbt3nPAJZHu%^u!S!D|d9=&V59{xjYV7-hC8duk__am6QpyZ)h6_bdxpMX! zF%0XO&c6W3sa5r&urF=2xekh8v&xF;j)$HzKy(4T>2;c+c~Q!E4evHh9D zq?FG^%H4QCeMhtPyB_8LE7*i`!myuJrUwj_6@#C(!-My0ZC2YWMi894$p;pQk7gkq zNqzEg0WjW0>mpHM|M+j<9b^=OLf_~Cqke?4({E{&KeM}MX@Of`BE+DiDyr^&o&7j+ z8|qad+qj^3_M9RhA6gnV0;m3JlA*=Tq2r$~I6vs)EasAiI{yd~AQYJ+|F5&xU!XYj- z0MRut@-p-lmbM+QvD4`59q<Sr1)$z!WKU&5BTlJ5)8`MpV$ z??7{j`g8WS5nyf!@D-Mw*TW{Cof&1+vt+D(V@m4jpR@iyLq`)BI5h1GUONEWf+4A7 zgzrl8!k^JSa}Ll_9x(P3I-P9mTMlZ?Ajz;uMs@{ULa-%xgX_S~SYb-dB$Z3;quf9W z{4%f+t0mL^Ja?FTSWcY$e?E)IW-BlpwphLTC->9+nKZHC z#U!B0s7X51GrwIKjxLa3-^X7V{_#-`E&y=H8iXUR{JbtnG|#1uzmIQ&cWYqds?_)c zL`cpwe*Vo#W%`2!>EJW~Y@1;C>I!dsPbuaVX+7BqI3ra`-x?rR(~p0v#}|LtLHuYl z%2KQH;(#Ah8G=;ElVG91;j5WG;A}ke&$sRf^lCndRnew#95~Pt*?iWDeUIrEFUSYZ+HJ4VMJ>_as~_biA2L$i^`$?G3TUXfxFp}V6`~y!1PC| zf1C&rj3IyCRZ&9&KqwgSs{ot!eSbFzK&!OaJ<^M;MH5bX#3wB>$fy3kJB~>+tXcq9 zkej9?m{??q!w<(I_Qzbwej|EEkq)Ay(ESjjwyIkV@c(o_k6r`|p_6V*I|kzuyul zjbqf&R>n#y1Efuq3IScH7^#JwUXLZQ4APP3u(H*LGy^O_7_ryA2+y5=%CcFh`@ChOa zW{&`~6AQ&&`jG#se&nq%MtuUzXi>jBc;N+HBxovf{*86iMoBnwYEXg0yzpE{rv7Z}SoGxW3Js#VzcD^FV~kMxar#xY zJrF>YUbTv*Sj{(CHguV9AsCS7A(4GD)dl98K%{LSdNlL847W9PNE zkOawKIWG{E=gL^uL*t0)@U1`2>Aw(mGXe4g#%SfLwJx>AjfwF;cHb(4@tZi{;FRa6 zsnWT!hsfdMMknZkQQWOM{PU-Fj>5^^~M3D{& zi9xzkq(xe~Q{sP(d++b>d0wl>;mm!{b**)-&v~w|AgCnLwcdxDQ0RB==o(LnIE6tf z_XhY0a42IMwX)8+~Y&zQd~fVz^~))`iqrUmvWk{ z1On(6`J_3V>!wLIlReaX`KCwzT!T!QZ?ZYc++H}6q}74gl0>DQB!llmcc0yFeM2jH z)ovTt7U3-qy##B#7|{g?Z1d}*Z8G>C&^Irda6iR1Z~4{jDBS&!b1573qiC6h=TaLs zT!QttpHPW&5w3z7EL9b+s0t&lc?KQChfrUX2;}G~iEQdL(maTOkCDg=6qEX#PZ;|J z-eC8PsDxkU_nYJtfK>{~&s!2h7O6xJIovTW+zLom+Yn|E44D%lm;OtBFVTBW+|Apr-d~eSkIXk9jCR$1s4|c1feH6a zxxSc-{>5nlm-%om)#wFWru0=q4QdaSqiVplsq~w~zYj}|HkX}#Z4Ue+&{xS&+P9#M zBQmVHn6*QZC)2=pdltL6TibkGvH=iu(nnidoZ&5ua08Kc-8EE21M>LinT_3aqME!Irm2)S_4`vm?9w=SQGRz7tjc6RVCQl{2|ou7r|nF-G2`grX4&wv<%D_S z)UC7v_OHO()=S^-I%A`y6f`BxdRUcqo?7u2*)yQSe3Q&E;UH*HF%5+YhJKP}ysJ=W z>!RB#*^fLLee#e+h7TFC)O=_zc9@3yPSNTe`E)gwBT$|^wRc>=R#uXOOTbGO#t5#a zR&J?gc0SR+YCeUQqLdHXS1!K|xs2ZjCJccR(zNBSb3Qj68P42MBe(Vkw&L2Ko)336 z3NEBJMPB`PgK~g^F=)9F#Ctb-{tCd3m<!i&QpVuqK-8mEg~N_S z%_pwmxniJ=nlZh6BXXLt=_+c@2cpj0ib1B#Q_a#zyM=!4Ar>z-3UCvJDCO|m>BYy- z%&<8uY%J9}!f%f51)O-tQEQ$()#Hk32HD<4Abcx1Nu_s0^QF!6161aa%_Z)yX7sdx zzgiN91f<`Vhw~QBcs%VgzBh}HbgJ9SHeKupQa?~~_eLF{COAAeeDmIiqv50I6NNf4 zA;UjSQ(isp(!THgo(RkGs(JIG%6*4&8aJ5|{H=k%B__mqopMN4q$7!v7hQW&yJ0bc zIU>##=pI_AMkZX^s}AORrtkNYYVE|W(~D}i!{f8B>%ma%_GugP;_Bp7=vAcZY?&YV zDZF`msNg`PIcr<{(!v~$xBT&-WlQa&zyM$IlrOadZmaIAe3(M+GpzrU#>QkN#4VEM z4p#c`ELAuS!QN)s2Jf%F(WheP>H;yS8-ha@+=3ed+TR!4YAGM2UOuth_>>p(s3ikJ`6jFdk5DWA^9>NyXKKCiXOn3gT5w^@xpLtT%dDu zUT%z8vzU==rQdyueH(1WD@1KFc@3N-vWD!v9#U=`?|+*7&6s(%q?NMV29keDi9i_Z0`ofjp{mgAIE%ex^TI23#K$Y z=#%4m$4fmlGjth@Hsirl(H@JyK#QRU3l8+tlya}g7dqQ5K5PIyA7x7XZ} zSH1vecN3b+>1{f3TND~RF25?B-%TvNI6Sdy@L;u0pjL1?k7e1FJ>N0QJP(=WzeQvJ zIfJuj45!aFH0%0b-4P!fJ&cv86O~Xk!p5+qMkgT@Z(}y|@ps>Gp$SV-uH(b+_K6uv zvUF~m9>N8?!$^x zn=H6t&(PwG*x>MwFAXPd(>ZI$yHJZRHv7n`+9Tf1mx-UzZ{~MY*g{C zr$!*dj0a*()!`he@(la4G$NO?1XeYQWqwKIm;XLyGI#h3@Yf+67`_#1T>2E{eH<{T zdSD3r>{@3K?<`Mf+8!MRCK|K1Ab0s^8%h zF8*p3aG&tGexr5|3^MaDk1ts ziO)wP=tc|!1}=`>wbLDuH_{aPWc0(*PM8?^JmVo!u!USzcnfijMM_tcgUX5nX5BaWKlCTl;z^40n+}*k!N=~a1s_3!WMGQV2{rqUN-{>0;bcEAaRJABq^Ef$qVeQ zx`NX&$7dY)Zf3!GOx&Xk zUtqbx`zhc59d{bCG#BDNBHZpKB3GASk?Rvbk>`F0PgOtJK_fX;zvo)GslOY%Q4_16 zApeS6Ptz$%UaiZEPSilK_XkQnqBI!Y)yt*&$*f3KNxevsQwpO3PcuwZ+1gn&a*SCW zH7}#lqIjc-C@*7snobDWu5r3Lr4w>JbP;rEaT~32@)^s5jguRw#4bl$ytB|f?|Dd7yQXz zjVy3(P7>PUo5&=f~}J@adIR93qToQs#@du)E-MCsYa8D!%#YfGY9%2JZ%m0ycg#WD4E|a z7^~fgJyM-wcOjZOVuo1|qB}hwg6kTS_9}LIcA9nab_R4(^Cj@CzEd;FH0fV%e$Ly; z-D$b}da191#KZP>Rnowv<4`JI{En#c19L9-sBqz)rGcI_@u;3>(+f0hSxf>C5;_FK z9E^Et|9WI-CDpd47ez=}a2R0+5r_+N;p2dO|M$-oKwh#e(aBD1F z$K;x?FE)$7G^V`K&LhRm1}dF*E4)~`Rx)akY)W}D!XEB}^;Y-Qr{Q>zq_hPwq8GTq zQROLZ%Y;a_iniX94C)MYr#%XBnKDhe_mr$6T0kDNYJiOr6Wm=UPZF$JJ5cNp#16BO zV>4K=*g|7C|JMZYN9#eX+WC?B(W`agm^v#c`f{Y3hQ77Ot(xVGX|CDnq@+56dv-Y} z96j|cdXFTT!{-pJIv_SopzY+45R#P?cU(H#jz*p}qA8yq_!rJ@=&& z?H4BTDn$-FNe9(cvXixweMfD7Soh46+NtrAB&B=f^A0vX5J{U|K&H}}_& zc6T3*+Dlbb|8B4YOD*!Gsw~-$Xd}fBh>44?cwkClEN4#7PGAvQfkR-?D?Tl@R7~o& z*A+nH7E2e;`Oq)gG|CvdQ9T&&8LGK>7SdMph_H4|37#K6L*)xb@u_o?`38c+QcY@VGtJH-N4%3}*ct@bU|j_;qhh+#9~z&kMluQyOgN zMInvMjT%Bi5?)z`DrJizL5}{?rh);W}p)rjv2p+*d zr8^LUuvgO~Sc#YPjO$0|+rM(Wnjcz!$|c-~+&_h`G(R+z(MS*&q38(?iAj~6?hwNzrX^Nm=#pfFYH9oiVSBklzC+~6+xbrQP9{QR zv@%6&*nBH5AI}L-r{nw!`~d5%ww%4l&Myv6%Q7nhjubH%C_7s_y~FHZw0v`kVvUAR z$I`S%r%`fZ@v?mg5&!X2BB6P>o1ZO_57Q&c%@7^WIDzzu9LSE>5;*GBcW=ci=rrgw z9!<7=4QXNDfN#l7G0FF;CFV(%5IH4f8nT2Bve8RqsNr@z@wHN~p+mIiH#uA?H1-g!sp z2Gj-BELt%FxD3r@^G?V)4>n~9PKf;S zteq=URwp*dDdQnn@`u08IQmUgK7 zx>FfK=A9MsH(Tp!VL>Ta3}-G9kYJa^Psj}QO8|w?{9Xxh}z8Ul~Mc3P<3k* zv7_R(N#4!)oJ*=vTH%xyBqR;B=F4yMt1Qrw$I7_OmNY6?JCa~+|zSq0X z^RB&e;FsgJmGc(9<@gs^BkP*FV_|Y(H?)LIMg}m z`w%|gUkJ&$MReWnHp)H_`~bti%re!37Pv91Keg*i-*s^QQ6UVma-l7iS-m=I-whvJ zWcNOo4NGLZj?D9RM!0_)_6mMPUs|n@v1?2x**y9*_n5@Y0^;&9%T;G$`?E@7fuslG zM#?_cahXMY9PdS_y_G?WSV=gs%Lk3@JCDC|Xa6fwb4yDgB{%ImBJNSADvq%abM1a8 z>SJxfXShPqR`t@kg~yGedXm}X80v_Hs0L_Tr^>YAngDbt%+#D-NER`n;AGq(;JsGM znc^002;H96UFHFR<4lRkP7-{L9GjFJi_cH7y&i`Ke< zg$I`uofHly4KauANU+V&_kwv~)ck3P;`J{wDwUG1#n6WSJ7skJ{GEp3}axNz*_ zoYE~dO924g{6@@{k41AZv_P@m#~k{4ls(Nd2mf;NQDYCYx%l7kT zNsqz-c{P1?rimTJA(u#0uwi+S?f!jfn<_9}v!+e~uB61l!Q56OD7;j?ZTWU{630EB z1f6=@xo-bHADKeDOXr z)xeM{R1V@-*Y1jmr-2XzMW-AVnh)1hh_yclgZYc84@0GWDsz}J$ozRX2>c?$_D#Gm zY7jC@2X)b9iJFUIO5d0^eKj#H=inN{_o$O2K8BP?P*`C7M6cS(7yd1}%qa6y7O$^N zb{BuAzz*%i$Bkgv!HvnhyQ&v?1a~~b+?aETZwREJ(bpYyEi^%xcj`}O1k3k@XWdVi z&N(taP>1@ba3m+k?%bwoi|ed^jL{+NRTx~9Qc}l{zowAlQ16(3`OB=5VnEEzFmfWy zk@&^X&$T1N8VEkGA=R&=yq0xqs269DYD-uQB)~dPAxI>jTA7hMKFlC^0hN_u?Pg`& z_{5(U3e`CaVl9FA7_NWa!4J&Q@{5ir6$y=O!wC$^NrVfnTX~!@*t@!5r^u`&7DKexb0_#t(mJ+%poXSmZv)?am!gC_Up2Y)fZJBw?$4T@Cl-5T!D%>VZ5 zwjd@|5j$TtdabNKzFq!?){lz-%%o{^*%GdLQUTJU{Q&-Ht&=tASr@$2A^DE)#LVXl z?o;9Vx*=k^HJt|Xc2DoIbv+R)@k99Ky--h)>3JI@pNf6(VTsk!z}`pS(ol^Xgs*Vd zpKYY@s|J4S!PN23$D}tC`=mA1CvZ{q)x;;4os0seSQC-~(Qj$+!Bb^hldqr{Ags9d z6+H^}KtAHQd+`mMnq`@fE-9JC@!t7o4X&3a@NTfDp%R4*F;A~no4~5VtWO%I9^Gl3 zqJd8J67qfX7gsPIvdsU(Dk6oyZT4jUM>QYJR6YVBRGF4={V5nLMJbPlGX8D1={01= z%WMNS@e{AaZB5}mGX6}`5q=;Vse)b~ex&!EE03Qi`|Sj`owviDPA;?S#?Tz1B+p za-zni!l5?<6sRZpSg%~mx{zbLo(mjVQbd9Fd-$yGB-88wmBqnB3328r^l?J5B;V-7 zaooD4WNPfkT0_ zRzG@|KT>p(AO0zX$(inV0m0MiDX^CBCm?j~nWH|ltbuj_=7KuI9sEAk;jM3AqPU8a=td^}T zcxbXjr77sLVU?G74W2yq=L_|J7IK{TRZ}nR+;$$Aj2_9+JMGYh5a2GQQ)ix@#j25z zpwDWDe(dt2lJ>&=^GH!j3DaPdWmG}$Tq z+UzWe*A%?iTQ`$)Hr@p&Cf0vN##5adkg}D(sZhk#SFUL!HHq%2PRUifRSc)42? zgWtq$ZoEx$nC?2TJbEw7hoMyWihOq>TpF~M4L{57EN;wu{@4+bUliIXor`rRaVXLM zGeBK>)lkPC!x3wbh)8jx+zXKY=(f2llSx;6H~vSCYw_%w`PG$>?e;?Z>EOZJA|#x! zbjjl?Z9e{|r+Y^N)fKD&2kIc3JeKc95?dhT!oKw<@}pwXi4c)PO7mB|#=Ok4qk<>uVHHwd$VQ2?jgHpDQXV-s-#f9k~4~sQjYIoiG0!smS*y#Mps;SDXO+aBa@;6>X3qF;tr#wm00T3* z%<`b9ym88Wy6qn{JPlc#S;0joUZpd;;a%YF*A^|^o^MDlPyumZk@MU0T~`9N;&#Xr zy;&!!JE}tnPpX9u!Jo04D(1WP4z_+D$F43!^G{`7cua8q6h2;a@q<%2un#&Q@0(jn zl8rT+7j)Vl^9*F{e3z6@)d}wjFKidKk5+JFf4zc$LN);wlL=5n81lasnqO&pou`bZ^ok z?y78l-j7^*|9grzu(PK4U)P~?hfo|*h~SggI8sWQ9M~g0u8u?S1T=c|S8}%&qF4$H z2b+rsMxY~2nb360yL2kuu>j?2qQGJ_WV0-|z91yr;GenX`)~!;=f2{*;R&|~dermz zKR-_+%;?@{N7f}qUM8R_7w}doU{;S$JCF8T$3~ppX5&jXT*wx35dZ%OK)oZ!ke(MZAD6^X$d1CLcDe#;sBV?|BBUMkn(> zN^)+9zwb7RKSd4A``6R(zw|hzPCSnL-xu`X%Si+K{g)4d4?`ouoPyZ-`qM%{1%WSw zu3*;AKB_Bdj?{gyc2VNP|C}X{kZv%Z3S#51Yvv7GvY!IP+mpHbohYA&f|H35d0q0+ zn8GfJy7>6f+JCTX0$f7H*lY`|cewzBFBlah4P^b9v)#;jAG_f;nE+3+Lp^DJXt(E2 z0^+gQ|NdJBbtol#_7~PWQnC+N2K}1LY`@!{$I0{ZR#(AVyXOH&S<9kt?wcRZ%m}67 zL%5Kp(v*$&=|Q0iPNXziF@VZ8nz?%WgpH}Df82-2LU_AcV~WS+S)WS5KY1Alx?DfV z%!<etZ66s0moiguSe9}A8xQg(5gL(%O;Bm*^j!_5E|NDP^hmzv7QZUs5 zZCf)|^yI}cATWu??(v~36O5@d@3bZ81&WUPyYeMSYpT=SOSH>)$rsH5a;*icIQ!{n zMZbU3V=IwQ_gjWj7uNjr+}r@%7_-b3a4i<0OtebxX$RK^GPH1I-_o{e@N+!`KLU

mz@o-R0PF)}F&-z&I_3>RkSu5;TUk7QJm&@=bT7_FiribV1&4OFADCyV zvjQEWn4=!dGCu?O;w$v530L?EHNgj8Un&*)1?y@D%Hg;HxUxrPN3LLDna{o>_J3Ib zuL1k}sB-qmJ7aDptQySVN-t_b1n`FN{JBc_(3W?98PJ4t6(JUOskn_`vsRJcd5T}* zHOL02zqC!ioTcyQV*Br~YGI@WbLIMenbZ+_*-Y|FngN3DI^qbRE}Bcgat-wu`X`FD zHFy@%lPQj)3Q5WK!1^%O1g)Guvuls%wp?_TSzee{0&I+!Rd%_az<&rpQ@xHifMzI~ zP*=G5SDP;I_itW2p3pBtp}4P)H^xnmzoYbW#EU;4w5dj9Qa5Xhqejge0rt6~-WwgD zmQ94hG<|BgG$?DCM60rNe;oe*D0z;Tm~W=(pWv6K`jC>7Gt*~1bW;#M$xf>Xb2Gy; z@y&q%B8=x#8lR*XH)s27_=gYuIW%J#i$)T65ymVfwbifYw&W>^-{5e&+Q(DbdtUB$ z75%*r)~LY-Cd>DQOd$`KP?QWj+}}5lt4-DKV(c|=bpaRv?090%;ITrA?#la(5H{OW zLXQ5$0={qZPYG_G?;hM|OQ_1<#4AO_dE!> zj$omY?RR=gC8dF?du{{ z3QCzE!}K2w2e;y)ZK!>YMx>OmY3G97N6DhbDaozNB~iYFB(%rKB@&Y>m^_7gbw2iTg5mb@otmFXON}v(T}X9R?`~p4&QlqMa}=r zd2lg^5zbo)f1LJ>SOPIYASH;zP{!9XL(umD#`skxqfK)fuGmj7^InDNr(b)an$Wa{PjuA{~8#arCM4dw;g?MNAKjt z2z{PplYb+lKf2zaH#f!~PkF436{^hT^rML9BQm_@6K$>!Cm;U6fn?Fzw(+D2*OwE# z`sgz`XAPr$;lchyI%FWG@$|@-HQyPpZeKzQzv^)puU!o$?vXcc5&9o@OVU4dzh@5< z;jCZC#z?ch4FV-JA?{uLguC&_O-pXNUJRET#06es(QEG4Jjc!ZlPf9udRd@MkabY- z?LKt|0_vXyj**iuWj-ZBfFd?u9l7Q6p>gtNbK0t`1hw|8a#txiZ^|^6E8pgQ%jt2M zUsE@7+vd-fRCk$qP|GmUYK!_W=X8Zintsa$u0A6Yd0J&~Z_}X*F@`=|>g8XEHvl0* z@51J^hSqvvcf@fosl zA=*&rXS|GmR6$N|f4Nadgi(VxfZo`qSDY*xM7Y{$5G(Bx@S*S)z;FA87E`Phd~g(@ z<6n$y$CaA3GvR-m5iVCy8s$MbB!1%^-!o_Y2)6&>>bQi%JCgWRc-m|T?rpw%Fw>yn zvhUOGR=K;3^M2+`wAn!}xPjV(WZOyPP#0;aYa+|s>Vtw=%~$VJW|IeDzS|+k7N;xr zyeuF;y{J{bU_~x};A)}&Kv1)J#>Z3|1iin4UNBhUVBbE|q+-gHeP=^(Bam+q=zhhF z>zvq;ex#j8CvdQ1=aZ_OTwR+^44qGqBNa}ouGw!jm(2chj+cgQ;^a@(W#Y|IfUT?q zw(KwUWSp=t%f9LO@BP*NAI64nz`BV?E`;GX;i}&6$*f?15T92ojqmYA6KD94ag>|; z0)XK;DjDdTLnLcW>e$(UX}0M zwEY0UP`D7DrWpCl>d>%z!2$;ft^%#SW+%&jl_w|@UWsMNZ<#R9u7KrIzcniH*`>2x z32)jorql<*@Dxkugf!#D;ncR&=_)-zV{A<1k)qQP(TS9lN(6tU{#R*Xu?yUNd@hQ5 zHs#T~?R8$7#AT)bG7k-45yxZ%PGaIgCdMFd=??a_~zIZYighe)d-H(%u?-O3ZVx z9~B8Xz#Ii9<95)8vN?r6N>?!ap zc6x;7IZEfe{0S=4zQ@!=#ZKdX%2bEm@V`az4qa~3(#Ddy4L#RjfQTFv-bP7`%4pS*MzHIwDmRr-crP4vqW@qJ-y?!urxk-wV}tk3 zmcq37mzZ_F_C=dZZtN$nz98aOS9&9t3Fle}RN!~G zCGR?swSXlp^R)-NW)il}b2t}zTUUv(Wnd~mm?O=~+ThFAI@)2cXlbvD1|Qa(Bs3MB zQNRx$bjmjE3-{*E=Pjd0GRl`$Q7D&Twy3;(TUrl^!Zf7X%TPwh{N9?><*Mq($H4gG zB>PBi_8=3|kA;>+gcd6oEPdX*lKe8Fv;IKrzRE|%)Y|YFls$KY%5d!IP=Ry7B8YPkVKZ8& zO{SBD0^9*rCyXL=2$Z>}CBNZKJ+b9A``~XhUhTkokbICbZ}_pnRwtNZQ+Q2)26lQy zf^92waPIt*HCy#iwY~6KJF7ePPk`~&bd!L=CcmR70uJ3!aL!1l8-lGVwa^SKG_Cru z`pSIWMU{k=xQ`ICfjuWfnoQ({@K(*KEt?!2E|Oo|Sl{-kWh7OvvjQ z>ndgK-a(RL9NH5D%dDXgIexT+A8! z_pd?lq-@eVmW>sNjx^KsFr}#dkVxjHT^tr075R>sc4Fuc8m2OLBlqb0AwXbgd@I!T zG1;_YkwwFzu4;x?)(*&(j-1zs*7$h3aViwQg&;1r2=t5ljj(Uq!b6|Kok}e?*m?T% z7h$8Zt_#oB_jOJ&jG3b@@MIsfM%!wZF3gRqVFo zRmiv+Hif-}adn^Z=xGFVDWh2jswtMz;VryLC!0Rk1RbHXl*B8u3rv{j_;am$i_b*O zlgm=L9t9@X$Du+4P!^3RWG({v4~FvIm2x;8C~x`cgW zbu#X)6Y6eE1cY$kH}TYM2qogsQ|F`L`J-$=O3uN_G|xDxrXrO;CT1QqnHum^Z}K*O z&8N_T-T#~^VD*3kY*mOAI|16y9*kA0!dBg%KuTVWo>_4<51EX56K$_U7#7C3ga}Wi zS`URg(~()2(MVaO5z$z7y?8XubALM!hns8<^Vi{!IFcpw!?8uKk^kF&DbQSD$ZrM~ z9~3|SL_8q(4EQfzDBwE2$ZGtp5;k^vyU?`0Ib7d|3H^yU!d5X*iF)Xg4rW1zzWh9Y z4P06;m1MR7Ru{st`&{%sRmlp7qf6pQLzC&G`&M$-^lKcYqlw^Dmi6Nx5x$Hl_mC}I z4djw9LsV7Mh3KOI*~}$HU2K3?GJSi1!Qlb-lfuYi;w*A!nn$@M8JTFmA6xRzdpo&+ zfV!llqA)k#&>NT@=o0_F^H>-tr!V|T|Jmr*FKC})2n}N<#~xH-q-5!USTji}iF#Qc z8mh!?V7W~^u$eHW|0*egL%ttlOrw;=jsG|{Z`!$b^$#1j&n zDC=8veU^Q&VeS zHexj&o)=keTepAIlF`vPWLipsM4_W^xP7-Vm;Zil`yt0q!9%A9w4Hh`>mO1yQRZt} z-JXojyhLtj1ulp@Z4f$T#N%{IhDrc5I-z zllsK{#FFLNjqkzm;eWCcf`;R5u;91boMwDEajhf|p293`55+0u~byc zOl%Ml5u!p@D_p2h65~IZsVC$&+(LV_)p!HbQl{ttMXIEGDZg!k(rH>x&9pvZWhSbp z7cEFxJ9ni60w-9%hywW(Y|Qw-@7qo|6+KIJMX{O@S&ojYs4Z4f`A^M8e#S|{X9f5l zG^odBT#lM+*T-maEPPuPAwRpXYcJ_Cz&klh}qt(Svn) zP$LQ)vtM%l^{5;R*4o+1$q^po+5CW)UY#p0bQ~~rOn#k_ND|gZedx*%ZS>bThGke7 z8+{pe&DuPDIN{bv9it&O8zP&O*wCDBL;(}Q7~ArE1D0uUB`FNYM!Q4be-yBmGYCyT zN=>G7qK*CQgB0vfvo4^vjF2{1=T-%yJ)_#>bxgMK;(}$hPB<);5`Cl6Q-!U7qvOGi zOGC^UTWuLs`vBmqtic<=Os}bQi|%C*(01Eh!=0z&2;8QL9sE$>MNzO<4wMRD}M)uzO^d6M4-J zs0~;6U#^0r#8Ob|HKEJK(8_&~F%bg4VFOYqsU>h?$r*=T3A9s{1fq;CN%R_(`T<^o0imY<$CwXbD&ckBqkkFMr zUAl3HVlfqU8Gxjj;|8s5s0X=rFH0Y=T|aLEiIVryM3(mi?l4s5C4BCdH$N{%Rxy3@ zb^~rdzq#)4g^`|jN|t}$oiTZXXV;D7Q^7ZX6y>cD$u86C_7Q=I|q)13Lq6Q zSFzBRF-p=vvTUJI#>ndkl*j4f+Z|su>!>lF9Gm(RQOUTHBajx+U{O9cD0qFg#C#l{ z?4_X?7w-75Jde_@?WvBubtU_&uhr&c7epNlIsaIax`6=}hQ+62K9p2bBWiL27V}BT z4b`x3@bU#s#vYR;fI%fG0`vR6-h|UO#aPG*$=Az+zQ`VNvbE0l*}P4Z6a|+%vO0;> zk9iR?TPx1MNlsG55?_1q?f-iTPjlD}(LJ$+8E^HB+$wM2^;+G@ap-)o^58B9W||(} zv+e!9O&J3Ogz;mr{mBuS2u2R0i-U}*2>)x~#4mSO3=-n2cQF<+ed(j7zlF93tj=}T zERPydEF<)2%cn>>Xv2cI6DzkuGD~>_W!{s@T|2mx9GHqi%^X30>~s5WK+PNqL$*Gz z+<*Pl1;~c!n#i#sck>s2rCrd_V4ef~1q!dQ7hMIN#1Dl&!) zhlP1Zh>Uij)Id|E_REJ<3xyxEA0gbqlJuN^l_H)80pCokzJ`4sBs@2jj`@9N@wT$# zvQPiI5*3;_)e=PcWGF9zIK&+pIigs0H=vX@*PG6l)k8j}LC*Fip&wBCNETug--A>H zAA?%XUbiOe+WEYP--;s#WJFV$_ zl2y!KPrEFK$1?W+rkLc;KxF*B6(@-Wv6!w9{wbA&^43#EU*BOo+M)jZ_D)mVi zh>+-e%e=y?O8?6(BGm=)PJZIM&enYTlIi5L>kyl_?f~S!$4z3L$*x;h{+@TU>1&bO z`{O?#T-*W|y_bJXnomok+3zTi`rg70&{J4yx{~acJAlh^RGnD_Q`&T{z{dKL9c3I) zqnSrJKebT#Gqim+W;`s@6Qh*m{#-pqIIK?S(>Ru{QI(N3Lk~2;Ij1g7(|6S_SZb52 zehy50i9K>Ut0-4;S_{F$;BcSRKOfV&oN}-q1j*!1K#uaV6FNZWn^4)M`uqx6`R)>U zMkADbP}S=wD#C|dW;6Mvt+vU5>j4EJt=BmispFY{*EOIx>zpFRRcvy!pk3aCtW*W z0z=pT8$UiUd60A5piPaO%#&YeG~Ml-8)kuz-n%9*b`VeJ4ev^tK}Ft8t|0PJqJ?Z^ zI)ay$5@t=LOZFrAfsUmi2D$6dc&JrreL-fL#}qIK;=zVggtk)+hgr(ZDF_tE{aaVT z!OGmnYS@Xm1WAHiZ(aNv17s*`LE=*sF8YRa?=5ip*6)F!jAES%BiSja4zd!+dxP*R>rGUk7{)J!7Kv=#a?B|7LaD&wLpC{3H3Y!3O@i2^!l4D9zRVttAsiwgEq zIUj;~9$G{*&nq~YX4uCNf4rw4y$7O3SeCzmyF~+w7@N{rElsAA>37FSy9vAr8Ap6B^v!s33n^2wWu^|kjue8~{hGg=$GFt|;o@4h|=b?CX9kFMCHA&x96{IEb}}cj_sdnVbsyil+_{u3S9wfLVeyIUUk&%)5jkE(^MVa_0e>6WDQ}oiYa?|6bBd7O&^c9>M z@F?i!98UBu38#M0jB0~HY{Ew(<3zX~!xdZU`PeNF1q{3r_uZlUymQSG~GPum(X!aRybAW$*_e3X!O zCro8oa3OA6l#GR^D9LVD=zTGK2DZ%isIoe{Gwb3#6@F0yA38^sQn#bd!)eSSb&Ffy ztWvWs@0g>9AQF3s3OuSq`wvJ-hdY&kKFy3iEkk5-sN*H}LfM!dgDA@6HYU~-?Z#um zw?(5}lYbU~u*94%m}z}v=`u+3D+n6?;dGjxo`fM5oA$a*=ME}qlOLv*lSWnzW2m2` zr=FPc9K<>ajX1z9G5Lc}+pa3G5Ny=kMmm;#$Ara!dp%5zAv7cZVeD&ppFDA??9HHJ zZ$#cp+U;R;%OZqT+57p3sw35t0TZykfj}!(gW+alXw!0=7+LqxMm|d8fP;{(;SD+@ zFcV7*gT{@%pP`$9K5ftr zy#CCTv~Pz;b@R_rfn7YxX`du$#)Y?V8vEtLCE!psJscOz*HW%^t_y>_hpd>AI%_D= zWF$LR5?1h1&a^eG2BJ)ZUU{Jk57#D7KgMF=u<2sAr#z~ve((a6jCdVy{-F9)1OWk$ z2zT7C#6pl$N<){+GUm@`uUP;inwDP<{<_Q2!dIx!pma5HG6L0Yba&u<^1&zS9z#%= zB>xgVGm#Y!JxC%s8I9$zG%plV!tSx0-`Jj|=@4B?2lGNAIMn5(sC;Uu2=~6$oEI5= zoojpz*TJ}+1meCW-^;mrkcBDr>+QEF!Kop>Ym<$~-skUjX;WUFf6}MRpc_38mx;xg zDzY2%Tb~Ggmhv9aTb|HSIOcS&U?!^rZ#dp{_`M`835ntGxjtX(drg=@$>&#@_IJ^Ia&F4r0Li*6TcKj0YG*b@+MQ zd%4;Vh%iyT_=9#P32u4SOiK5NGVxj5oF)e+wmZn5 zOW_+gDH&(e-1+U6IZRe%8Mj@v2&%{&OOc9KnIVBikY^F z-0^eO?vlTFW_OL!f?9}Lfe~LhH(ZIQUg#eNjAyP7DQ7tr zjLMBTTv5F=N=Qg!RHI6LFX>+~WdS82dbFnqjtX@23tJtt?%U+0wEbYJk`j`uhP0F# zBh{*bF%r_FZ;Q2KO1-$MeV6j#Fxieg@o@Kv{>4*Ht&m_+UTiHgpXAD$p_@z9iqpp< z{y6OJKmNoO7{iguxYxka=#$0>^RtZ`9d?f5_5~q*bWmgAg5zs(uTv`Pm+Etoe&KRK ztL93H?WD&$#&?ORA)jd27jB`;FFy>eZRBZ}|Nh8$e83MW{#LyBYtplarba!_TQIt% zf6}nWcS*{?B-JIKc~eAp-e*#G;?*;5JRIW~v^4wi(`&nF!npK{+-|y8si67L&PblT zTkQV!i0I9#-E~16I!#t0mDU=0F3+!N0htLSE3%*{bxuOG5JBF_$HBGLI0h~C`-~J{ z{d;RDD#jJ8x8Ii~=fDp}YdeBDv0*;Bje;S!BJy@INHLB@$)Eqq#M+8sbxfqBDYfp@ zuc@snac*~@JgKYeWe{jk&ADBJ*K6gC6)ddec-Vg@)|*evU-z8C{r%RrvbWuNHyc8c zf39{qszeyD-(vIxEVnp2(4ZZnZ5B~oUd9AWZW2e}_XKFKNU(0Y2&ax8usvs6!OjGl zfUT*jVkb7c6ZPxGLg}gOq?Wp0ie4ax^^~pYXw=AplaYmJb=$;?4C^-Bh*R1~(>;DA zm6h=Km_I>d+jdz~{rO}?jm^xLXmb8!C-=k#{XTv;d0Yz(cJUS@7Uh3bAIuyPpFVQ9%x z#G0xeE3|F^93UbKuJOcT$a{DLj-}I!`Zg9KPOxU+%GfNRoM)%Zulfkk3VTYsxty>yj2{^`*TVXVb{v`?bfDh>9 z$*k`0#ZPs^bl`zw0p_x@5cWC*_z^VEdQGh`ayfsOR*MngQP0jhWV=)+q&sYPEy3p^ zM-OU1EYQ4DXG^O-5brc@mg=H7L(Ycp<4&smh!I(EpUlEZ5t}>WUdX?6_6TF?eREWI z!!UG&{ilzCy>`^*gWJE6TU?%n-F0MQvTiv@pxZXM21(Ng#B&fN%u(6NFzp=i{WmkC zaH<1tSFLrz$1;F~SICAB%>A5KN ze1qcxb{6z0}MNnbqKF6kw{pxcu=glK1u?ua;h-NOUS7;H~d2)y) z?6fv~#T90^sK2L;I(Op(P6hVS)}tYJ6Nbzi&GVyy%SOZ=d7lLLFntJ%y*yj;BaK@D z5@(C)jFyGY);GNv+FwjP1w<4GJq&1BWb)(N(;DH2_Pw0B6r@fm(qsRnMDcTyC%-0% zv`KN4rd(%flpeKg-2n=6UPRnwBwr+g=Wrko8p_AbeH;$R+~TlJFbl95jzSn?WD4vhl6@UVW7O?;HMHdQc=d z%9Devlt}njs`zmC{Rz3)>*CU((U~DYOtuk=+&Tk@)W^smq>Sm?LEyf%Ttik}VEt+e zlWMSIvWbUPG&l*6nn{8UYrKat8_L{;?Kp4U{fY zc6cfP;_reQ&R|--V44Bi9Ezx4YmLC~S-MAJLbu3vcVvUdB?n6CSTUwGAh|2N>`vEl z!S#O@=^LbNLl$SfZNoAr0a&&EyHJqLklUcsSN80PThASx#U~5X`WL;LIRq4t?P=AP zDnUS2%lpD2#xr@h(VC9Z$Kf0B-fZYWakF-@9~Yg+R7}-wutr!qEr(M9rdn&Gs#%r@ zN~ZcbKw(oU<#635oYKJzPgpY?XkgczVR5-Dh za9a2gz><~Uboezw8QNq};)M1N5di#@R46SqslF2V|E8qe{NmmQF0 zu`=qDUP1oV*yUurZUuBRmJdgiF1&ZG^mlX}IvzC2Ttv$xCKhS$PXaIhrA8Ig^`@d} zN7zjp<9fpg-t6TweP+ZXu zkIf@iKDSp}qAyxz*DPq)oer1Vx}+D}`*^C4fpzAKUCnJ=2YOCz%P=4T{b`u`f@&;dzU*}o)6wa=nkMtywTc2?h{%K zP?e|GH|Sfh3likbL)iR?@Gdyv3fsSQ1+Yf(MYhMR@2CG_!M4>7-%kafM7PCp!?v6X z0xU~LvSh@EQz0`|Pm7DJjxtHcq!YzH-I!qh?-}1D04{6vYn8_gz*o)Vjv)S1Nz{>K zN1q24+N$K?<6LEK_{BwJpZ69siPD2{pa3HjyZH!)w1oftX51k@W%MSZ< zx1-(zz`fak&Ao4R>td z>WgDNwPk|ZDX|%2s6ZcZ){TweX3_N2EYJ!&n|kR8fy@3U|MyV$_>edOsU3-?>bE%H zSSv%JAbpVdCHdvjJk!k|ra0Cnrpu`=i5=5HFX!ZxiJSJ~85o+gkE?lSscg z1P0xSuwANHJ3Zxv#$mgEOycT#Z?o*q=NGT%f0@_u{AlHxU6=y5PaM?W&<#k3nj^kp zU9A2R-a@_ip^exU|Jkqj#=^gZGTXsH4hP*;5~oHo`^}KPrn6HKO0#T_m1X1gX!=b- zXdzkF89l&C&rYV}`eMDITU?pN^M9XZ<&Uh_u@lT=@FRBD&}@KKwtLjP9PI#p^AjC@ zP&6Rjtrkn_ixcB@S?yi_7C&M4Nz+*)`ay&y^Fr$NbZRWmgW)GxWnwEhl5HK3xZA(x zGF(ibLOpY_LlB%z966t+m2Y->YiyawqMgYSKBn$x(SD*;<||p|Jio(L3*LBuoW^og zO!e)KnE*AiY;-B#RYi;`nRH|J0o7$M#m~UtImSu?(i@Uy>dfIthiP^)3Xddk0yJ{m{|(AgrMmi*{XB32xGcG}bjA zWf{Wgnd{5kmw|g0Z~|%ua@r3~e4rZ2PM*a>NH?f&_auyWQS^MGy6Ig*f0xS8r%%4- z7k~ovz=A)O-c&sxbQh9)0Yvj*#PyE8ey@M^n0ol{7^zh=5%lM424lH^)n;b;;U)z= zv$7(tG4434$hraS8vQSRza%-{0G{S-Lumm>W2W5`TaA3Xcyihh7AfauBD2 z(4^VRYC@ z!izXH-G~x@weUI0TsQYs@;C95IG4$jnQeT}%7$@cZP7fXC;ul{V5^UpBt)r@EkkLK z6tl0`<)C*0or7e5vGxHB+mF(i3#gAL4FAIawr#!#g_pknz5|>1KGA?GQSGiSiUjuf5QV{HAcy{O234%*ryU7; z(US4<-|P+jySV^Px;oyD-g>Gu{j4Ha%4TkrD2d4nBeoCRce>4vI`g0&H6SZb@nT`0 z#y2x%fAj+NoKYpo>#Tl}hbx^CI8O^$0XWs+glD1g5R*rkPIuT(Zyi%ZJG+9_l#@d9 z2cB}?Y+sba;rqMVNY_{X9j&?&TT@yK_VxG(xNm`5gMJ+ z`Bk;dgrK0s^RUIB!o!BwBzY_i>y3cqsCPYbV+gTu1t!=?EKxfTQQ|BHoHk)z%tGz6#DM2u;iN_!pEy4k{mW*L_n!p&qJZt-{1t+&1rQ)drVa!kM2ASdInG4@TrE!l zF;dum^FmfL8)K%d1n4dcH3+{HZyhghl=DS+0`*m?4T+i(6b-EBqE`WXGLIB|3P%*{ zZd$4CXTBZRHSgG4$78oYL*R#1mF_4YUoy}k+0qWSZ@tS-L8iOhmn~}aS=r@q`!HAL z+7b*`!M8Nt-r7?0c4zO+wsnh!?E9#JUVHjQ_+Uy+;%AsXAlFRBq0(5hhGmsv^r2?} zsApYH&pbVJ`)EVv)@}OdCV-mWod;6N#uD%Ao#*Svu9#ZvjXQnooRLc{bH`)@ro7(;~+q)hR2Wp6JfEg><8kQro;)6Y4zOBUb?cfz4#Xnu2Oa7hRH;{5wLJXuk~ zCc80NF^lQo=qkvSxp^*S31+V13v!0beQ&tWHX!Pvn&QES49u}H{%hO>jEN}RwqHwq z6AkaNcO?=v zU~^@t4j{0r7I)KiN{bO%oGqpxo-@6P7uhjQr0h~cx#_EX+t!VCwx11uLEur87uTeaY;+EL%*jD+=83D1^+)ZuYQ z_UkGE3+nuVQIi!}EkbzqxWbrrc4-ZyQ*+g>^IR8Sa~03K{eOflg!-(bx%hK>8J}={ z;M{$c^TIleqCYmcVf9QsK)lp{T`Y2Eo8oLQ^{#IkZb-Z!Vs2xYDfXc*UjInZG@{b4 z*84Hr1j%q}(6^>-fM~?Uq~;04iUmX)-GVwBbHLib?`-!iMM7y1F4p|C7H){3-1oGH)HAO>Y_uNIzD>5c=$$tF#FE7=PXHS2$V^*dhtP=;YC$;?Jk+bt0j7ABbNZJqO5Q_=YB0InMUmVn|mQp30l0vb*W8y!Z_0Oe3zJzX1Jb;yancbFW*K3Tbg@CO>eh(bQ=0 zF4S|$EfUkcQDIlVKlNP;^JGTU?N>*{jch_((;MBJKGc;G8cQ`y0TSTqj7@B(LC4>c z2~%fzbVsjl;&97|o?kEW?&9MbtS0_@C0L(Izy)s$5DzC2fNRV(bz+_m2ddKBn5@yC zO2Ts;>p5`U4md%)>y%5Yb#C2p5^%3`UOfL9rW0n!O_h_K>>3>Ls_)8PRwo^0A=!D) z6Ed$=KdA|~?LSW1Q|hAJwA>3`NjXsb5%PKC!)#{TNYv&FXXBN!3aPQ%;a#!nmtFOv z$0W4vYr%W;I!ez6YS_JClkBu~vHnsfcxleRuAyCr|N3zTDT4p2rC;=ed$Woj!jBKw zDq|v}?!JV!hFi%v5fom5oh^Pff>uP^o|344!y^#kvtMwcfS=%MHH#9|!6|GR^Wq$Z zsyOEe2PmP~iSqOw{&45fp!6_>W8X9bm7c%!e{VPlZvKu=`S_8Qt5r!Ci8;x;<6bktzH#w zgv9Cdu{`A!y(b(2wi{PECLU{sF#9zvWOZR_s_(nCwwz}yb7PByjDCdGk$w+wWDP3- z3VK%Zrd-3{K?y_p7sGla4=^j8J=0l#OAoj!^`M(<7iNR~={N-Ul{amz-h5n9gsIM2GTY8EW`pr;( zn_)Bk#;;^qkf4yZG;3aH@-#0uYt{>Ih+&&)Q#F_r{p0(_GH4vzlKpjCB{d4(8HL!Z z|83+752j=k02<0u1Phav13Pvn=?lsR34hO5WNucB#Qu1WTT#LN7+q*|1i>QJ!i@9M zAc_1Qt}Dai8LHzXr$Nrk-ORCe(xpCJVwR~!H-g^2HWghkMA<@FVK@~;2EWw<5I<|x z5*}ke7XE8KOgR2gjY$fqqr(fE5l!UZ^FD85T9BZrYMRztTa8xZHMlQ@gb_Um0q(FTeNVd>QNd6TJu|R6Ra7^h?efpO&y>9p=fq!alLQbaL~D+p~&#g+47*M0?*#cZiZR$KJqC)L+M@`bZp~T zSJtLrZVn39xwcG#IQK=D<6h5JCjwObT%#R()jEJa;BM!AIg8l&aYHg3PHCzAFw;B3 z`_(uEJuLAGg)p;Oz8Bky&8O6r|;r}&6LLf1G$UY!n)s0co^NtZ;qEYXJ zmcvtcA58pg2lsQv%1U%TY)+D=C0KsrHq<8M(agdWCY=ALpKuZVnQZd$9a5^VqIUL| zbu)yq4-H<`1xNaKt|Ya{rf;m_z%@5yKc8@d+-X3~_|Z=&TrGtjdC-5;tabG$jF_9^ zRhU0W>e;^o-5-7Czl#rl@<)aSZKPfjD#4}v7u;{a0%`p)d;Yptr;n-h>TmfF@1mCM zkGo8>rIB_9wBI*)ESz9-e=V)$6?7=;`UzbK+JV&+?vC@?JRVa+?*guBrlBLQrU^_W z?KNiBf<8FHhh;DWanM(iQrg70;=2Y*HH||)x@hm)OC=_2B3zRZt58qfSaIj{p&Fou zzpQ@F0dmyMcJ<17sxotdXF@hZA4n2OOQ|jBQeFHZ(gmSDrN( zTgOsLO?-|OD5EUk7N>9^8$ksrR+F~_nxR92gChgr^TSEAj%>|~7tjr63^O8-O1yDc zTPAN{tUSAo4QI|t%fRmyyK^S*gxqKh5YR$Xf!=jQ>PDh5_9O<i$;I7i|$eNGVvz}5?&pE`~2}iILGcyGZj>sE+5cB7|scp%6{j! zw~S6mr@o1uweQ>gB5lalFw}Nz21(ws8%vlNC@RL0K-%l=o=wEciX{=Qi;=v)+>-d( zyyUhL#=`jVen6R~>z6{5-XB&*fnP`&kdv$`6(mu>t+Vv)(2NOaeI+Y?E>b2%U;0i@ zH!A{wCwd%kE2c_Y>G{=-bgIx=6pWnK_mZCEM}P=GFFe)X-x@ZIl#Fwk)O`rZvZ|rK zlel!vP;hP&=L{Cva_Ny+%%tl`vegPxq6hRnR$hPQGbOeGz(8 z5{}1q*XPVC;ZLmTPh)D1otj~^Vg&o2&yw~OP?W!R&sNM9ux+dVH^VYs2XzemdX|!e zeD`|eou=-)m|iRbm?N3!!HnPJ|E~q0{5r)?5Gqz_8c4dY@qYI8#+&D9xHwwi(gvxI znJo{ySsrvKiUpOx{)2b?5sFB_dI)md?4?e42uGX|G)Fx&hy_JBUWP-6_P?2j9=&OC zGT`L*;oiS0FL?x7ka7l%xm81aJ)Jce1Ym-&I0)QQ{;t*?4>N9o?WFjEz^)wOCa#l^ z7v}hBtX&r7t6_pmfzrq_ICj{24eN=IZ~^y%^;YS0r`fFwf-RT)r56sJF1sX-=E~=E zJ%76H8h&UM-bYY0(0{HCd)vt(*iufudHhqe@n@(&TffqWPNCUEYQiGh(xI|oI-Rf6 zRiRhJLPLf}sJ;r~`ocs#M}tcGG`eAp2tbzy#{$x!(mC=jOz4<4@Gd;M^4J2Cc#0P(?g7Gp65}$`~?YH79)D>OKblAylt&{h8OBRC{qpEgls62H%c|7&d^dF~Gj~U%FczzDQq$1aE zfXj>yI$>kUA~p$V+L2Mhu9R{RghH@0n{=5!l+(UyVQP-sc>SQqegS}O5$w0n`5kue zXaCifw@{*L7Poh)l?CJW>IA$DHWizlFrGoito7E;Y@9=~4DNueuJHBpSjY)Mhf`Mc zc>~m-XMZ^HlEBTG;b%n6udiRRlURypq3JJr6d7Jdu!p;^=Q{xkJttTPhr_1h-ON4y z=@xRDsXy~X5ryH+YA`5%L5M$o(n!h|URr4jql73bd>~I2d^V3W63DQscv@%x2h@Eu zK~8R45RbVdY;lR7=_NPEduTlcOUypbkQq?~bMP2KB{nm;4bMDiw<(e_%DYpFEN;|w zPtz@GZ$Fv7rNw-*0d+%SXk3EI2mS^+cle)Ao=gi6O7|}Ug>_=TL<1<6@N_X+ve0@oV zud$^Oh)8cct{V{u4KLEoGPVLnu&73Af>Hb|ofO(Mh}vh_(pRL6ey|}$aOr7udhsGe zOkkB-HM9!UU;kRPnLZN#<0$FW(c4-27sCs9Tg%XC7KLUpu6M@|8>D-L`AnkaN2uWJ zZ3$xg93`1et92i43e088Zy;I)dVqeFfT%^VZ`|6Z)~klKQG%Ivg%auUUOI9Z8<~py zDWQc$?yjB+T@&c@F}iaoT2&$+DZcBvObb6JHE9Mp0EeSw<&IpBb(_b1$4Jxs=3NjvC$YQB+$R2TmH^rPUX(&?UBGOw;z(oL1|$g`@M@m@AJUX>Jf*N zy9;aLdx>Y8R?8(;RUb8;dVUecvKAZ%q(?=!+1IjV%HI)6B{P_GaL*?&+PovY?7rEM zbzBK-3Y6wSYk6`EuA9k66yZfqqC2-WDr!V`Y@}9SN>z4goaJy5z0uTI$nKSRnPj=F zG%5kZR&X#qv)BLID9`DGe116eUlLjjvJ9Ux#mLBt`-l6!(lEIqEYoxX_xrDCoYj>(NLkab2Voz%WK02Mperx=U3mM_#zRObzA!&oAuha2azUfn zVTl_r*|%f?w6{&nfs=*vQDfMLbphhOgy1-hImKd2sPXo|_dyMDzua@AK`%n_LD1$d zr!4^qP}^QKone_oY9QwAi>p?B*#~-{xttcIprpixN5&SK0fxA+VNK8yO?DN+h<^L+ zYKzVp;2Sjkvax$UH?G;NhAAD1gmkvkjK z6+Tr>eR^ZRJ#xAw=KLN_UEsmx&&ySOdCV+_!N@a9)CZ&xb%%Q?+^UG4*^nS5*fae* z5xOY+&C_`j&tl<&KGD<((OLB#P5N-|W3%v=AZNEN7MtL1!vID;HRBHMfa30vQa8z$ zKd9d)8IGlFsX(#8Io$pA6KoYovxbzLlY5wTuCY)31IRV)rSlZ>Gh^O-#x?I`e;yI< ze;whLeekrJk7=#4h#oW#ezK6R{^pLRRnX9zcS}45NnJb;rp5h9Puak71cf^i!e>83 z1B!9&^?*Bm?W@5r{Zk}H;qtn{b?zt}zE{A7T45x0!4F*0g=YF0{?T<<<0C~~f(6R| z6ZMP3vjdw-J9i_#UkQDkv#a)#8>4rDIp@n%eClhKYo`3JNxSX%jhOritk4$d**lt5 z%*+I~Y_62Ct%H6ZDk^HaRlJcVl5F~HJv#c6N$=C2DE%V@V0g5^HV-rzXH6fi#NS>N z6s3W*hX;2s)WUqxEDdfaN1vifB5vO1)G!IxbotFqXq}G!PK|cm+FVq*f45Wmp~Wd8^6=SJx2VOL7s`NW zvBtp|9FIJaPsVp&Nf88kKC6Gk>x}w94v+ML8~SA|H=*)u`>pq)3;Pyk#@D-Fn5k+RD1mMLj17IHY>}=f(r>(|Fu#NMv9bp-&l|K6B`JJM2ROK0o7g6eIfUPKx=x zImj2tl6d&A3xh9gLL~gjVSLHwP0yiuMYYwEFbC_qm`zzWgDsu+24(T>E~!!`SHm)R zVRA#e2{PA&x4E<*>DcqU za^gj26oU-z**d7I>lJ2yrF*aGjWW<3l3Q#GFwvX zl=WsGe|!(or9gREZ6jF-CFjdiIwDr*g$EFiw4lIIMuK-qJy5*fqwnS^^(1(+A|3rM z4LcuYTp4;3U@Us8_l6#&Ua=#jR~ zFx4gtwo(=jy^2_CIt2>8v>YJ^5~C4kf$0X?z4QTX*e zdou3F9Yni288P$xaOC)x0a=5F2_#*@r2&Q$Z|S30vrel_qrG3qbY+LU%brtEPRG_GfN zy&ZEpVUUAGdox`vK?&y<9`K!JMZlGkvw4W?z_bS+E@qE;V>>^j&`^PodrNrtfhHKQ zFgIJgRgaSh`h_I3-}~jryC-s#`;pLHpslF6-M?CM7%A@Xx5rlAKxA#}^@V0zxrez2 z{hf>PYJK|_yyuERUT?i+NlsKiW`ZKTwWCcvjZLp3?dGd%D7$JW0$IW@d)NO4UAAaX zh6lY}6b*kVnNm3xWQ7i558{%#E|*T=d8R+!S?THdRC+X`!qO6M9_Q=O_f!H<81Nc- zAil^gY`pt;%|hhr9iOuZABET}+4q0)hmg7;1_rXU_xbB=m3|1dNW6E*-H3<24 zCrNR6He)DRH25F3k~`#+ZKeDD1re_8NHym?P|sK zU@SUcRj}z~Iu2psdBK?|oXDkbiZzG{(fD-hariO-{0DA5T0_#i!Jq!v=7o$zBZO=M ze?qixkEXd(W`*s;s((o@hxbs?1XvX!ztWb`K6vK3 zm3YC1H6A<8Sx7g#Z&l$OQj>h4fjd?a)9eO&79tUg#!6ek?Igj?{$!>Pm^LH$*leuj z_5mD5WdoOLvfV0^xhekRNFL8W@{mxF@0aX_kh4kmIH1vH5>Z8 zRaaaO+(PhgdbW^?w>GRZ`O=hdRkog4v2(iPHMdi7S;n61Qvl#j@%^K)Mt;d;)d@o< zd~)T3)a$|cQPw#3rg4Nz81F<}0}}vUJXR_X&-MHc9~XtFoivWxH)D@!n)F9jxVjiOG?q4R+E7Rj>z7V|zWce)X-p;%Zj1Hd@Jsg(===qyF zkY@={w-HN{JfRn?*(6IKB5h^u;3ofvuY=Q+g;YJS@R6-3S-AgR@+o`93+hpx9S1a3S$s4 z2;2!)fIACajiLnv5*Jxs9 z+UH_A9H?PE!YV|q=k*w-I_>5v#~Ks?+D zO^f9LIl1KB6bVPOH6;NOzj)k2Tg^y)-ZUzH-_%0^0Uy$pI~J7VfgV1B#|e2l#}zxD z1?s9;mLsa-4_78t=#CXn8LWs#gHNfP3!KhiA$}hvD}>zRf{Dn%+rfom6+`I^B#utDHe|csBtbbfcqC2U=*yF+FyaiKJZGCfQRuwun3wuBRs!rkm_h#np&K4? zPff-o)R+_;;ugD-Y-JqjHG58Z5U=D~w&8L;aZ;^yghH_Mg)n*pen}1yq6#?-nFF zbl()D2$*Cp(3BmTcH_{vi9&qbws92TNk_u?gZcW7er$*G?~zf>68Pzte#a-vL8QW^ zh11kHn|K5|mAL2lzL`;zzp)?!q%Bv%aM#W~ev-T>%1rX2P@7rgR3|Qf|7;(wEvUb@ zxKIQC_;jUYWBJwOw@+7&&o00-n+h^3G5b&OYM0#okW6?eEbs_rfjYPL1#L1d zT)HIApbqZ6f>_xg;y!wzaNMl)@>PXI1|LQp~cwjA!FYgpXeR zY&E5@NI3F2dZ(}3pgX7hG%I=RfpU@#7y`nFfB*8>8zV9ab}tsb)edHIX zXrUh2>(czNmzSBxgsz|TClkyHwfMRAQs0c5KwPj%d*qAaahR9mYUW`KOJFlMYJGCrUYBS8x> zQpka+&fW%+o_wBan$|hGBt5YO z%yAbQQC@Qs55O9V+kzyrFOd&dn;&y{elvv5t#CpUASu!t;zKr}9f~Gk8(HK_C$bM~ zoN#DbTQ{yWL<>~=UNBPW7(_Q`Zj$Mr{wY`pLZKT)B92Q2{*=vXBS@2GA5-42zlloA zY@$L*v7j5;kwohN^!X5*edvIx4wfGjyAY1@mAIQgV8L*)`h+2grGifx>8)xK4K2qUX=-WVWt< zy_!4%Pr7k`SGkt;%)ly5&i8717Wc3Svf<7CPzwRTIUsG3&Vt58qh*S5sa`;2V&XX$oM30AR zs~ih-=58a(GEZO3`jR!yXeyIHRhIL|@~kkub!PJ9yhcRXn)#v*^qs8HeNE+Z)q?rO zJU|zA2^)DM{Wg)pFY!xRnQ6^{VoOXs?MO6FV{V|T_&E(V`}|RoCei*Nk&@yuJJ(vs z8OH{ZG=Um-;9f>X!+$gza<-XQN9q%?y$8$(2gIcH9J75R5w--<5eG`N%&`I+dNJ%N zLf;wWulwyVc%{TOTh~inj&2)AYvN=!g7B}4Tz4bvOpY$X;{Q5%UFbJ5cpV)yLQxmV z3-ycM?@668NBDDOGnSn$-Q`xD2Ji|TXE$$%g2&P{R9A}Wz4`=)=rnG7_pF^Tp64?E ziyl*ATz5u?;m0Xw4QQ_{Cbwrv9h+yxe%0}$Gv6yp*Fb!FFZgxqRk=ok8vxhZ(Kn3d z#|Y8oPcqKA67Y30-&4#5GivR@uIOmIYsKV})9Tw?&Y>WxXrq&(4zEdPK1y@?vk_9{ zf!uq<94RemSXOy;p4ShyPNlyE!8r_H2_);uwCdwYh zzOL}zPp&Zk2-sn_;02}U$R~?Lf7kt?j*FYD5t{ryJ7>`+E+~24rowi`L*BR{hRS%e=!c_I9;oCv+}0zuK+c;Nse}YP@w6z7xD7!XzIu7Jh4w z&x#!;cfdaBsTyC5%P@y-5bDYyT-APvTIMh<$w8<|Gi!OvT^r3&1}`A~so(Ugi(<-p zUUq&SsHxxExXrC@o1_i8swX{5IY6LhXIH!nccijwZw-w9UKbu1y%E(&l{3}VPZ8K3 z=i;+)%+S}jAkw_lJNlMmMZ)zmnZJFMyDqi z)}eKO@$leVgIZI>abZD_bK|S!My?XuZP<$k|sgi(N41B%z@m~_hsq!x!auWnN19e#=nsk zycFY|p0Ug(XIoWeqFQ9%R#fXYfDz89aCE|3e4en;T%Lp-?4_BzJ+s(gI@5H2|K8^1 z%nX5q;M9Rv$MKZAKe)KfBdwE=IWXPv`QzeJW@| z@2EECfPb72iW_qjCz{i+E>gl-2a-~dY1HB$bbR8#p>(-Y(<@KaOlLDNH5Xp4HRI-e ziUkx0?P#q$75g518G`5ov!bjvifcYS+%l-#IxyHSF;-O)vFUt5iB6reVvGv~WRrVSEiI@{9qs|RS) z=3J2MtN?4bX#)v4K?Mgz2c2;z=Es}6Z zkGx}+8>g80<3Z*I1G%oaSa`U2E%p}2IQ>fpILzJj`i(I_Y zr=2NyHGn0SDj#oozrTOnC@tTeQ&4Ss7bB0Koga~Q8!hLUYd29X_9I3}Z}SIqx1KWd ztqy-;7JAg2RAEt_q~aAh_0QC5s3fV;+9+d(oMcL|$jcC3RvAL+o})%Q?)IDv;_^Q?+`JUt#)5iQ_oyftd(8EN))@mE~ZP+P`WH1LL%ex zRf;#gKYU>F^(A>bs!R%;`tb3oGXa4mnl){S^pDRzL`!oq`6uKyW9e8FniUcp!YLXb zI95Q9no)~wcSIHgq846{st3XzkJmPOb6Pf25Iiw%u`?j{GympXoTW_TdO{y0Vb*;( zV@OXhgRCk^7Tf8g0sMnrdmhseff)0Us+Z<7#W--c_IrvH%`d!}Z60>|I7Z!ew)%8k z|GWs=AVp1DBd6fkqT;mil?Ucmbr;m8#@0V?#l|BPWT=+tR$}6-Z`8a8k$BSEW7%uE zzgp}~+qXqe_&O;)dzpPgqqvwa0KKG-olMQ40;^cF2loQlDZpzP)z@(h$2WLX(gPud zl@32A>3O34II#06v#2&c^TLAK}4TAStqgAe#110`jlzn6~PpM+T&uKQ~ zjtt+xY2tnW)4bhmXK|JweL}93E!ABfsgy>W$UM~0rxO0myW*q7>zaA;9|EW-mp={z z6ED=fdZQ7j>HVciHJSxn_g4)kYLbunfNpOLJDhR%zspR(s;r73l3r}fICUmpP*L(X zV~=IlNN@2`d@pyXpVy($XhCvnLr;08{(iXYtZ=8~UUlk;lA?CygSy1UB%1O0PYSC_ zds~~UM#IeBo@C;0!@u9;ksszSe(j<4e2!e*qoL|H+$T=mcE-o%XtGtT()0-g}plMrj(Ek`{8N}AdCj_w_ick4hZf2?I#OJ{XI~*Xd@r-}{jQy7;lTkwOM3FF zr11&qh1`!CAkx+G)@^1ii|~>pN*oU>gj@xo(`1I9Hz-nWepnuUmqiYvb#)U$?b#cg zu*kah@{7b+IJ+o5_+qX=q`K->M9-`g1BYrmebL~t5lk=?q|cibXZe=izi&RJvuzvq z@mk#ij_&Ahx`Ztv^L?E&7?U~qW9)_2-~TGKgZ<~$2TtLXH9^WV+g>l2)RgjQR&a1- z$3L=e1zE@*m?f$Ag4trIOjqZZy!J;fPtY&L_c|(_+KHPrBxJ)kBUX}^)IKWG9(aA6 z+&GoIC%QN3kX^v268aJT#~%-AI0d=~5+ouEpoN6B{#@X&AKVsf5YHOpyRMmxxJk>r z?!U$~c8rFf&2GS}J!MLuE1;W>bT5W&q;)_Pqst2tq!1zTof<&M@_{yv@F(^^41~w^ z0Kech0;dU$y1nU0`z=%9(^y#FvVG?^lVrTFIo++~Y)7#%k_#1-(^)-HTQROc+>~$p z;^z(7c?$?y1+wi3aqW#zTs)gY5TUgEK@=~}m`{kFfvG`vvQ1m@nyt7;&%|_C?+Q=o zU*^pLqv@YG{N0*NJl$$c_*tJp*3@~ju?e{2U8q0Ab{xziZtOpLtzgVYm{y*!u3B?8 zf;mMhy@o9nyC>JV0ugd0=<(D~biD?@G-zl195-b~tYPDf`_OaVp)oa?#(e_*OkyxJ zR0yMIxmggFElU=7mbG!FhuD30Zlt``Wej8I>a5+M^LK`Wx;h09cCuo zX4F?;*RCSU-W3j8cThoCet#F(V-U`=)YMOD>yXl*;-U?vP5CVf5$N-lc1~n z=5snk+`qtw&`UIR6?PSzq)4?}e|C%U!OF^+nUNE;2d) z%0-{9nx>BgPl_;0AZUs(+~&Kov!eL4)hcGAkalQ}qI-I|g(k+)VUte4?Y@Ujc$0^8 zJ$B^K??M-*YQDl>dPD(9bE6#UQhskH3LvU$jX;NAGPL;nInw2M^!GIKIGSS^Q9_|{ z5DFCnsqCRqv9IEp-*B4m7`8)NiN=eGkCE>fr+hJl&++nJr>D0=9>5_14kf&qGQsB$ z=|&$?g}=f-PLJLoT8qi}HOwIps`i>5@BgFft)rrf+Q#jHp}QLfkWxCNV`z~UC8Qff zx@O3sJCv3#5h;=GZlo1Oxi=pILc98=@oadwI%@kk3f6{!3*|1A(e6@YqShvmC7neu`;x z#lySns?8!EgfxOMuUcA`9>x3J5kBw4L@N2kP?)sb!%9=Yn7*YS6$=dFVnzpltjwI6 z!c-N+v$oyIPaEs}eL5dhNu>#Ty$u7Z*^Loy<_Yz95tA{Jx&`(n8#e2Cig}O^lss4|@CvQPA2)7gh;Cy+HU1Pq z?y7unnty(fM2AQ2%{lMTPXDa%ozko77qljy_}9KgaLV_4gWl|e=q268Q(m4E-l{yOH zTqn4@EO8nn8`on@64#lvdUdtB@WLT|%WKY2y8mS3K)Ll({dG1>BmO6Zex``Tpg2>B zh<7o@TQbJiv-OgGe3kAZyI59FP}Zh}xgIaaVLS(AGRR`pj-BoNlyw``wMbYy! z;pAdbAL*Ps#8he{S?8sx;TEr><81H8H^ReZIdu}8T9PtAG3f|qYFq=;!1Mi{`#9+< zoU!$ogJ{Uu@Q)|r2tWT1(l7rMzDKqT?VcJ{v4t}H@S3c;S{8yw-lnSRwDzqQkejAV z6Q4Gh>|b?0zd`I5{@_j>vk+47$r)(5EKW#00IcS7Ei2))IL;_#cKVKAPTo4w^!V`F zMG4T3oEj3i6+r~UKR)q{?qE`Wj*krRcP8l=67dfU(T}|w4h>inh4b$)Y}BG2JTi%k zcG)es;u_y;<=p_3Kcj2-VN?jF@zWEQ%PhEbD!A+rpCMDjq#_ZFficXtcD`^J;H|Bo zo35awNpLGdps?k{xE#gQx+Qh^6s#=qy^*Z^(cO3PxW|A8T~(mjzl6655X?kB3kj(< z&k(oH`A$8uM)e>m|A<#bh3Vko?HwO94F_(k;RA%!3KuD=utxVstL#BwVH{7A&n-f8IIyYzELHd820%+o_;vG zyfIBI8v}nVWSCgqB_HJ_T9PW6B90Ysj#bJ9%ejs~af)R|;7sOMJ;dCblCU zV~S~Q;c^tG7R@ewbd?J$Da->^F~%AAz3JK+y{y5RM&F^5tfiD0l7Ey{-g@WzKVc&5$l+*jnw z$gN2eW7R79`2os9oXVcYI0xIEL79Uc(>-Y9>6!(+%Q38zR(jv|y1fiu zuRo_ z@l(Hd;E)ulbTDKKbHrDitv>F1N9ThhHqE~63z~;od*Whk$f{uqt&lgXw#*;T4*ZoK zzVXm7d}8c>_OtB~AG#g{*_STP9{W~_RHVoo1b=0EsEBKq3MM_jE?nb6Ky2WTJVbd`sYNtjIcAOe)4~%%;3T*F!cqb8U5?>G zo^vthq$kpJL0_E}e|Ghn+t^-3$7gX?q|m_cVuCeZSnWcf$-?arm^g6V6<5E8EH9WJ zyFB-U{E}e$%NBQP4zfKfNifui4_EB@HlqzGQe{ zweS^JM%cL4y0(+W=zVv+S^kl>513`ERAC77wbnY`%%Yw%nal-8@1iRoh%3ytGjs}q zvuDAYnCb!QCSyILKjTRlbgsS_aD~JTM`;8x7Y5=bV4e!(`GJ0T317b0!RG}7=R8cj z{#Yy8mq2-ih8YQ2a$bNA0T5RUC<4!NI9c3m=EQ2X)GQO^7;LWKx(T-sh$?8apZ@%@ zvKV2PABlXcKZ~%F4zP8gUuNG(IH~AohWalLdf~2Wz6uYn!mHs<&`3|Dzi6Qw6m2gvbu{qL6T423wNG&S_}o+W zpw`p#bLC)0vn^5kq5mQ0cVD0)JJ~1hoVV(Q?S!ALRF3fHs-q8Fqy)|Y(8m`j2A!IV zUWi;@`{eW1?)q~>+rDGE$#sB$)&;o$b**&obk14C)I`ZnE(Z2>7gbW0V}2Kr@jJB- zrrd?Dj;$w#kk62mRB5~F+TIHJBAum}CC~Y5?~X0ig&5m2c&XnWO!cmx(7#&PhCE5y zpaa{g{O|8q_INkc{$0K4w?P~0!D_s^ke*?P%pW1Bpc_Y^8C%n~o_Ei%_w$`EBDFNL z01M^5G`8)rGBpMaZmxTZD2H9^uGQVtjm8$Lp(yo-)`|COf(Wy*{Shw^jAAnn4!(BN zJU^w{mKxOugqnDBzGE$7(!iZ8jM}2AOh2YoM@#P5&QmJWQqK3tUngAVm-(<6TUOsn z>Jj)8wEY(Jp>+A4sv@MK+%!K6$;W+&`!M`Z7Rr%$>#jYMG>A6@Aez`kvwXS^KV_y2Wqi7+Y^NNiV9j^48Yn zVBBFeoVNa7M(YVCl^Q!Ud}P=0X}~Hg->c2vbIR%cl0qMy7`j3q5qIQxr7UH0wN(VJ;k7Zr8O0Vu0239&3OF z`m(>pmO;}$VSYzrh?2Q zn@~&-z%9t!U@j03%D-e695ECAR#@*PEcT+W%lDN37nKEoX#OIMK67CLvr(qi;eiX_ z%*a?jv>p08|H&Dkz(C0o#Q4c?DNoZ%_K2Y<7|rlNc_%1<;9q?~X-buVhXHN(o;Y41 z3~Z$Oa@h@AkLo*O)&w0d6aKL3`tC6d>Lw%ATtcH&Bg~R=DFor&7Sh=TrR!%Q*h_gtS_(1e;K)pfxOUx^99` z^>7z*_O~+!4ztJ-}F@+{_LPsb*>h@JK*)w z^c5sPv7}nqMTLZ(OYH9jc;J(iEct$bWK3+)w#=0AlYOBP;yyokdyvE_E8VXp5Q(v- ze=6vp5=eGf@Ta#}SR?$7;>c90zj3>{GU@t)V>?=CG9nkDM4Ed_AVhN`hBo01+JFY; z<+)9GmcJE(zf3C-1WD|muXkNm?9&SV{D746b4(+!Va@S4#UCbAm|;&hEAa*?!(n?4 z+oYG_(zgW?bb+_X4UuIWW00yij#T$E5q~hDH34D&K>(B0kQnR`4{9su3mIgUGR*ww zFur#gu$Os+SPRM1EQqKJ@m|2>xTg28pFQ9VoOkGu7tm>@XS4oD*3u<|WEOe_&Lp7v6A8caMF zd@4qRIX+le|BlPV?$p$cWKCgTlbO=Im~TIxEhDKl87V zKwRF<>@^+V3sF2aGOEt#Jv@gkkTlH~YIHnIXr*8e;?)4(aw%$gaqr55#jrO=?=|{M zNKfPk@NMERi$m@=IG*YT9|1hNNc-}0izIbok-^#; z{3Y*C+HIGgawd=+^8W)Zlil)(Qyr19!gUy^;rY($_X&t zg&{r4ODY#Uh4G_wW>ZYh0XvYH&TjxXkU*A@u8t2QKx{6oXEc9g7w1537hmSjnxi7# zgF2D%U07s&r2R#g!$Xbz;MQ->JXMT4$K4;|yP?Q zo8v&bI0BqO*u^o;ilC@>KqPEdnV)N2RvgSQ5us9<34G8iC^XIfYI*l*s6#*eU(Wq_ zH3>DN!!_DnytLNDxC*B62~n{3ZYR|S4WVxLfcc5a9gh$KB@Y{6eAyefiD-os^!RsPYds~lK&{*7IJ^r65Py&?sn%EYf zbh}*zaRF>#Rw}R=Nla^lq9q-J_B=4IW2;{;OyU+PR~0)<(r#mTpf?jZTb@O;i3dxn zk52R<&h5bM)8yM4FjZ52hMi1AGUrCweK?Qu)|7V!mrI_ZR)pP?!*!zcmZqzmh#G9L zWPvL7erxf*)ecbLuK917l9NTdu|YtAqxd&-2Dm&JR9l zyAFn)t}EZmFQuhQXZ}kUve3!%OqHWtw@bc#EqzDa^5J&Jd6fYkl^3_S2k3y_9c39= z0Z65_PCvWvcP|$weQD0iy#ez3olzpWQc!oeoG2wa#GDB^Nt`1n2}f)s3k_3jOxBZk zs=qAog@($`iEYHm5@8&?!*_ve3!&8Ep)-h)y%5rT%Y-Mb^_uGa9-w1Hk}m(F?&~Kp zK>%uGsea~-Z~;6s+u!_ytpknP!nY-nU=QjnZOyZ$1p-N2pSg~{K6yp~MYVufd5oY_ zl+#%3m*U`V_i*zLXg*llFnCz?rTBT;X12T+1&w^_fKObRVI zbjVR>FW8b7kA4<5q)!B}%|vp1O?@tLd#zbs0VgKu`upVMAwAh2iprDlwmlOq`$^jQ zf}8QumHk)SSkJApO5RW*dsf-ov94)7u^V5dNtPj~Kve!H%UKO@1Vi0F!kAQhHOz+s zdJyJZlBoly&Xa}&)J+G~E#e8~%%8j&>FBTD-CVQv=LM9RGJ1FB&b`3P*r5z8^__~w z9*S(3t6jabm-My{1|1HeEqCf$NC@Pmv_lYI7bA>iCaZRQQA)2VzFU)EZi|0e6g==r z@G|2EIe-M0iS^YkWI0{5d}n1QMdCsh!JlZ?;<$$q5A`u8 z`N%}V!zs#bLV}GF%@6cuO@0;yi-O{y02N%SjXYdX50bZ2DUXGSRR5hEgYj`{RRFfJ z2lm?8OSze_0E2+3yn!k63Ff?{dl*pYM0P)6yp49v4&{-48X>?#UbH23`{8r@PpYTR zOm+s(;>@LKy#wy-ltaLWgCMad5o{_mOe4vhKCNByRo;`gF|6!rkZ$Pm7fh+Ne z=a<%fffhEw$q^KhiJz7zQpUJ zH4W;9joS6)+du>vjLEFZgTXNuzLA z{+PN3-*%`t#5GygOrK9Fh$dOWOc*^k$flITC)mq=W~cWqFC#x+8cCpju*L)q?kc|@ zV*5s>A8dQ6v!xyR?(QVVkD7>}V*$6PgxsGw&8}PKxI!}aNzPxnqteXczeyc&BV@-_g#E0XQ}N)*p^7JzpNznTC5!X!&T!)r~dNvUr7(JS*ruD)2qxB5Ga`X7CZ%o%H)LKAT9$!s4Rcnj5q8&s<73vcvdM_AT0B~y}bV4Hs4O3`T&JP|J7NXB}(3$==)&Q zsvXi(#@S)u)p3Y~h?8b3T%96!hf#zsT-Ec~7KYHC8RKdq7Cms8`lp(?uH{Zql-l;V z7>s3ejw??toaU9wZqI7Uuutg@lU#jX7p>t@HTXpT8;PK7bB6yG4s|j!y*}dF*P9(i zVAfZn969h-(=)%lX{-4ppf)YB%MCM`uBZ%}pnDVh!3s5Sl~LwBfg9tD8;i#F$Aj7N zT}g_Yg?QdWaVfY1AydzK*uNx4_;d?uURvtK*&CE=-c;wsLut$zXEaYk(*ggBIRUb9 zfV=WV>|3+;R}H4X|Dw0W))BI)$Ae|T4Qvxpn3Nj}3{&TdHS;k*f7ubyCmgP?Yo zp-uKj7A*fV=_9r!v9)&3OIRJgbo2Us-{W5^&`;$Ys{lFe((BLLTHMmMka6n&;4k&` zKr%1rAfMR_bPcGBAu<&-N0~@;s>E=IMO>IM<`t-qx+K6?8UA#u zKCap*-Ik%J+x<#hA{7Uq2oCTvHvYsL=6w-@(p5Ne&L6Aj+i6I^eIy#&Ivl5XuTny= zzjytIC14H3(%4Y2NsgeIjw;vAYA`=rvaCgR*P?ENIqewK4x%b&XauB1BW`v#cY;2q z7tOON&RmK6i7k~s&)$u6OkE^rx;h@e^tV?AD%htr=RT`IdZvvs^~O=G&dNptE_4mY z%s?zG!-xp9{t5tKf>p<0}dxYB`5I5GD}mgZSSYx>s0dac)m@E>|Spb zKs%Zv_t_!t_uw2XSNJ>ge(AIEUrnPn8s+U52wBo^o$iMdN#%2eF$ygnUL2OEl8V1Phu|Zx^%DQJ}v^;>MxV?(=Wbp-439(%XxEM0q(%Rfy)>pc|T^!+@vS`COC4tXMQPLGT}9F_!g= zlfKK{LrhgCv0WSkL;u+iMk~5ko#>LE6KA__Kz@1XG!r*%P$(T6 z!_jf<6TZlyGf|)h>$6*5h$mq96)Ss73<;#_O8tEN>D4A5Niyef=&$B-SL3Ca+YpH7 z726qY74q3GspGIpzxs=rjZCsQ&-QJm6FZiKIhMJUo*M0XW&0#m(1!|=LB^2tM@i9=dl7-G(Bq|I4f!Y zOM$MC^nvnvr!ZpZn+~yZGrA;ct=Y?WnFRi^L!T-mwkcbLb(ali@(G_#X^PriDxqMa znC0EiiI(Iz`l4FRK-`57T>c#y<6Z}m+vyG{*$dmH`*s{kwxM9%(v#=Fk=U7g^H!F{JEpCpR$Z)rw8PjBz%hN!_oFDJv5Mw zVJ16dk<^jWLx~I0}t`K=TFONyTz%rC-QxHDSC6vTDyk;!0))1nh zq<0=LA0zZ1i2MH^@Wr_dLXQzp+*w_*#HGMs?u%vr={i8KcaHs;O2DGI$Rj_@LKR+o zDS$)8hZ}cH)p>-YU%mNdJD;pD40FcR;x^7PBs&++3#;zB(aC_Pw$!KK^-x51HA2~a z)VuT0+n|K-2w8;2p+I!d#&(_Gkiqh5Z4`Oob2m^)U0XEx^c%-GKzV2c>n1%+zLds}CLaxk}% zdNTyvuUVG%SfA1kwul8`&=kf}PqynlA+=KBvq>*n>2i=1@){badO^AC6Zw6hiSu-| zs+vNyG-n&*lxR(~b5@}dxp2AiR*GsXnrYwqcbI8aM~w1%?&ct84Ex zj0SF87sh&?xBd3EjqO{<`=)LilM>9MEYd_0gZnIDn(+=uQCLo-RSxfg zb@(j8&d~1Eq&Wljp##=$Z8(cETHMq=Nqm-l{08%KBhq_U()Oz1c#0CSdis8eu%If5 zcK@#xK5qc$Pf=o5X5{?2^zI^s*c4;SB_J@J>4 zojJd#KL1erh;7W?0&T<|ANIsT4f;XffJ1=X`hG##-xcjiS0#6X^Fxkb$i{9&K{XjD z&+$sIe5Eyfp0<%0MhjZzXa}dY>TrCsEBICr71Tp|0{~n&Tv6Iq=zF(I=21VP48(ed z=eQmB7t{)J1WlhGO96DpX!bX-xsln$AIbh_EfZ9)Ko~gf4j84kliSqN{LeO6A*zy- z_*vc%tY;99C>_il(4Qe(grSikdtNU85EV+^h_W#`?)iGA54Avr>oX!nZpl6ya0>qEWSE2va&5r!rD5@PazpXKKt z(Tx%!-Mb#cGc}DtgbIe|tYT-D5&!69a!e@Yjrb~FObxk~g-ki&)`CT2e)4dt8 zfN-j4Unm)$au{N7i7fwW$*C4AcFHF%kr9{T*lB-UWU>IZC=(aeTb-m8<88rHs0rgtFSUh_CkZD=j_qm7jO$eN^S1zlXv8f~m<-;r31Yj@At zZFC~MdgQzR?gwTa4M^s||MK4z#h@9VvF6I=Kia6`1A#T?H9~wjX+jYlSADzC?3IL$(PN1{)(Cy1Z2#r! zyxrDf2rsYkYjY@o8nwPs3dbU1*stuyfV)pJ=ZVdh) zzF`)gJPLg$T*kYIfL;nHo5h*)G)aLJ*PChLNfc3R(8V_XTwOIG z-+M)Dvrig@&LRZIGw8wKX?3S`G(t^}wPufPOb@DSqP8wk>#2ozLd{hUIX(S){&@6q zk{&2pDDkTk_}jLf4g#25ij>Vgmgxc3nX%Iv5*4Djp#&@bd6%1;h1FBV%J~^%>a!o% zr7*En)AFktd8HV7J>H6E2ml*{+n%Rv&7do9w!^yjUvIty?o37nl^D_>-W;hlZE&~2 zao34Dj#A6hYx$W}>t}eyz-~#HZpzApS6Y-^djQa-aFF^+d+VCc{-bh7A1inrW4w)> z9zcBORG;NEC$e8@K>}1Y{oD?6yyUBnr6VXw8T;kkcBJ37$qOfkl~06#&aNwY&cG*Z z&IXI?&{@%M>Jqy&)+(An=~yD&HVBi$F;;H4t%3`dsWk?tIB_zs)w<+B$mykUd1$|Q zTGvU|j|Rpx6OTeLyyW#c`dwFpNV~gyL48iZsz*KN`m`T7R)FVo(YlobhS8OQRvG#} zb<@vIrC^2#*l}h&e=t%VXA7C zXn=~P1;PDcg$H)}D7(}-f#zjzD2?&8YV8N6D2VwVuoY-{)1)#b9@W~yA|H}$Dq~hK z@|A=%Lh`j^6F5}70;LqTQF>y}Im>80F8YuS-px=0wZKxq#lj+aEE&sBaa`)8g4*4i z7`YF@P)BuJaL%bjP}-k#(e#Z@3I_1N9Lpp;eYPS~&MR*agQK<3;BvH#+>%0`Zt9P^qjFd~&kSE4)C}pv)@x4+oqH6g85-kKbplO~!B>cPY(^{BhT5mi+;)Ov+Cl zt6(Q^P+(ZjZDCRliDqeg<}jnl%7&)<-ZfGqA?_R@a2=zJ3H0N%JX0HO%}bv0;!RQ;7a4 z^i(fd)}#@kxcIV_6L!KEn+zwhHn7j(j9TJ7V7!=fott&ketU5BcC}&J$+ThGtL=5x zdCfJTlG-8g;a%0-9c4tNNpv=OZ}@3rre11?#Ec->yvV?7>=KO+RDf)B+*S`xh+P0_ z`iYWE42PV4&UAWacKvtpyuJdeBGZ(P_M>oE$N103elO7aklX#)-mGSMu93ai&jkKq z=`Lr#HCoj+e}Fm4L(0gO9P3yw6v>S<$aFMJ^}$?NEH44Y14+V4di_$s{4f6UXRk5b z1$WTb`x&d8db8kcWj<#&UnvJt$&c!dHvt$6cE)Sjc&%Bf?I<$I>hOPi1ROsPSAr%| z+_ux`w*iQldZvSA9t=6!6u>)7Hm=G9x!YQRF*h~?>hGDF5u?$B!b zmFsl8jQAxI#z23lNL{2Ckta5GCfE^I;z+YAiZsdEvqeh4w&r0GU-H zU7HYSa~~5c@k1lr?&qj||8q(|I(0&~^R0^KE=fi5LYNIM)}rXSpGUbfl+ok`aF%0w z80Gsu=m>=J2#`e${#};K#1iT%Qz|f=djT=SIqKz;r_KAWieeU&W={5ZmLlECFd^*o z;kV}g^_8M4v3EJtnD6&3oox_=O;dUrI!!}VpsXw;o{(3mDyVed3x;_tj!E@JXYFmQ zW8tRrXBjbqM003${SSW77i`d$JvLE8GYgy6Ddt=-Ilu0{=zGpS(gg~P`p)G#C))5i z3AkIap~2X-6=~V2)W$LYFlbaSb!yvkKy;RueLPfj)-@IATxY&j0lyC6{#DtS-?9cO zRYd=Gf%Med6G_VqPfPmK13=ai6?v9TC-0WZrs+u%mycqe z)F$1X-M$A+C4RSRS8jCa{cla^e^}9^V@p!{<@VLx4WVQ);f~FPq$SX@5M7A3&d1<)AR7&|LABB-um(?mFs=HfERFt70@{;P&ISjFyk1q`QN>2ePSkZfYD_=%8 z`;Lc4`5Zc*`h*LO1Kd+6!f)WO6)WP2t$0EJ<7qg6ay<)DyhhENXK5umR1lwbW7~r* zm&Y|+@pEr%(_}MAu!Rp1v^VQ%maCPGdc@5p71&bjI#Ueum0pkgC4o=6A4k37&ox<( zqh0}>8HX4MD>KvIIB28(S+&vWW8sLI7~wyABd7VQiC9y}rU1%vp;W5b30Vz`=?nDl zklACCSuvg|9%5bkCBMi zs`~zM}955H& z(Kl{fxRfs))0BDAG9}@m@Uu<%=V?Ss4mwr6^ds zUssdu3#_$paxp`4rDbswVO@XnO^-jp-`B(x07#=leNQr%j9S-iiSe|Ze0~4cELbV~ z;nnP&)n&yN$_(eVi3+e_Vyr+1$I1s|kL|YTutZ;A^+(x3LXKG0@cM(+cg*~T_5b%y zxCurvP-nPv*q%Q=w`3R4;MyfJA#fhB{{HoO^ogU4P;0PfxyS)dkK1Rly-%P1`fepa zq>jo*;$J(UjJ0om16Vye=1qlzVu5^_57VM!Gt6cW^!NMGw%0$vnNZA~Ks%oT8Snee zIal|So^7MH&vm+H>g)8w%2Qtdq{ws8c0hAK60nJQcr5cgr8VXqcsat|9yN8zc+7<= zJzNyKtC8L(kQ%(v@!Zvn;;$DU%XNcG^q??N*3ctl@_$unJGvkLgL)t3#-s`&l(}^| z1V+T&Y=EF)O-sS}D;Ka3T*LzX;$iN??H1|NwDaDcg?9voQdgjz|Jd}}CAj@@bFR()YDnzQNNhY* zd*@8*zP7(#oSTb8LrLLqlV#arWDHi2B_^xGUw9X>Q*Y}N=kR$}Z9KxD#-<|U`I7Bf z#C^lUSZuzbAsHUW3nDZX)MtzaLmvwnse3wdD3|{BA?N*GWuN@@{#hYT?gjh$Zg#Qb z?w4VL3xR+KP3DsKeMaqbe3SfvtXep; zN>Asv0;LMylAsV1{pa(A00n#VXMT~t0u(Je9u)AlnrpzZ$9-H%@1TaxVF++}aGDv! zJC9iGbG+h_zKwFAFp=#i@xD6VSQ-5AMA2}wpe6E6UMKT@z>HnLLlnL#iAj0f28WV`SR zI?f6C3Sx5lp#RUs{r3`qZw?GplF^tIuCI8^jHu6<5&H3Oo;#u6A4SN7CP-g%Br6`! zx`-gG#Vo{DwfHfIBfdml_6__afJoDAj^{c{M-wBXCtua3kCzO4Jw;JC0Tecwxi0hJ z<;E@b^TU#9v%>aFzpIRzrleD&d=plBA|kx@=UBhT%gztTFGB!8Y+jym@eY*YZN0?-233Y zb5gl}b$6&crH6k`1vBwOnrVQ2`v}~3ZG9+d1rRY4JW-CXO$hid_4L<dpB+sl72nM87ih z=>Py{wA8&#J^`d(e6u30T{>z(TS1zAuX$1w(G-@MQ93&Od3b;;{+|yB=&8*FKtFvM z4Pxf!p}DkM_@Z5?_7BJ{;tYrmmO5n0O*^CkA5#|>Yc&D)IpRws`oTBzp3!$6fW&Nb z)_hlZX)fe@s1^f`|yUVqNAJ?B`#W9MN zVyVni59js%sL2>^L=VK+X)N&DPqC1WmkQea_+-5#pZ8v+Nygwyp(}`BtVqQFe4xfXg`oJR9Nre;d+T{tVWFcA4=Gau$8|xJk|5*4h!XX=K^)7({Q}U?qbC zH$yP+zl(V6&Q>{xIO4tJ(@#7_B-UFF#2@`#x);6$1Ryc^ek^Tj!yQK@y`-jF>!tbA?o?gG&|!L zbM3K+E1%PC^HrMEkAR2@8|XAO9x-iep@bHGoCyq4b4L1p$3yHZz&F`s%g+0LT&ej= zRQbqpu@z1wVD*9{Mmj6Dc=O+%nx1CQz4BO23t9(-v5OfA{49_e6yZFuhVivBZ3n}5m%*8%gUiOUGJ7T%CY2w-#oM6U!}iK zaRx?3mxsHHOj=wn*QJ0PNB63upL(W4b6?jFKCV9HV;0lhd>rSgh?^9yfbEqZX=3-I z2AkL8$yhJ5WwTUMxv>3Sr*8npnlNPscrVv{sc*C**A`Abm-Mo*d3UJGe%sfC8R=YGqdSc0yz=QF|zHIy9uVFZFcyV}8`uzXilVYXi=vSQIhB;B0 z6CtP`FNyMr3yHrG#q$_7nq@TYek~itNrJrhM#)_I7ZJurPkpYOXHI|bZjei3^v@=6 ziBNVx@tL@M18C+BERHs@{G^NV91!*X_CeRBtQqrK~-n5q_WkTfD4sEcWG-|LL4p>_^6R zPaoi~9z#cTB1leH+MoFAQCu9NZy-hgV5U@Od$Fy>--Dzp%AbHv)a=(H@DdSLysZ;Z z{Q*R?jLwDc)*}UZmQ?04&(QN2B6NCtB(G*doJt%O&CM9O_6eQS9fN%HM<(SWYz?1t6OcvP zKrT6TC#!ys6%V^VMcKth(=;m7B9sJtfnKIh-jf1*WVYQLBS>+?(i`-pToln>w@_Gn z1mcZ^1)vwV{4pznB+CSB-&6`k8a7bWhohMu{iB+AadagWPKh&fgLRwD$9!jBtW$vL zezcnWnjPpFFHmPxW{myN?mO=HI?C~8{;HIfkzku&vF`HAwt z!JZdDE4+7`gkKyiuzkjZNK4*sE6WH>zB}3cVYA`~i1tDO?0}@l_C!~&=dULoYvO2K zQj4Ws&*=HH2z8mePl7tsGlkL1XNiAcH@@if<`2fa4|VNYMRj>*iNy z3YaNvulP%ED0`e?-MC8!TmY?dl8(aZY0S^!U@Vq6(1o~X7eG`rH`og40GySCnlg8_ z_?51KYp+;Ls$Q4G&eP)E^LHv_K>}~90lB$gMV`T4-^ zZ@uFsVAHfX^O;g`uLS21<9&3Ck8w?`O{`?vP((YH?ZnjKpUaD%%JjGr#eJ!R26P|b zgOh|nJL#6>9Zta_qel^E#j4N`m}UY%LW%y7gxa*Gf68<#1RBsS`(vy1EL}5#ixWygM)z41D#Z*N3?`57;Vs;C$)PjSEM(@ML^XAG!!&vt*9e{o>UuDvUTB(nP6{J(5QFr1B zq^cMgBH!vYJ=MX2=)@dWs76bXoBIAwf~6mBPtGTo9wT^t^BOR{t!Xg+BKUh&{&wzM zJb10NglfeDDBD+|CwGlNcjmY8m`ec_80nXqo)g(f(r{6NpY<|SPP$p zyQ|WGD?={p5^HZcJAmkuLb`l^em%660?}@$KyIllSWHd+Ur_qP!q_F5=|SEAjDN&O zV<_~*3gnb+aD0k1PJLU8xJIW1anfdZ97N<+2cAQyDE?Mx5e8T287Mg0{>0~ip_Is z&4zSM?$K}M|CTyBqE>Hhw7@HL7eYWQnLzbJ3vpwz2xZv&MwP5pg;6TCD{VuK{n7-f zB77+@=1lyMecyb}-qEVH0280lVw@X+zADFuN)*rGa)X-gy{ZnU3-pj6<7N+RnR{e{ zI@bWZQ|SU_YkqJsn^X2B1!|>%34DXR%-J;}sh3FRLS?iAn7}Y=Jx5vCdLuZ+@INi1 z^$v|CJ#CI!53a~Kjz(1E-z`V2jqR{8U80%U82ns>VqHEQ5BrkK^t)A5+plr2^5?`w zMAnipcFY*}_%3EY3~xUuo^X`uZno~IL=(|wE@OkLc~ zZoj2-Rgn%#U1@b(b^gAL5$(*nTp8t%IfS`$DUAw#r^Y3SHIPTwJ`8^na>(OCczLJe z`n9hmtoc)e>vDJJnxS)>b+DN8#qw9SdnHZeE3Kjb28jRr{;0Ox@>S=fYGG>(0*fUW z#M~6~5=%4kO(aL7irrLE6!xlq@Y-=mg2z~n)ZgAY^PcbwXjo%B;|2;7oop7&7f6|O z*CPcR-$K82af6s8bayWm2cqd^BkojY*(ncQsi&(v%9UW`nIj#7UN@{L^y> zn@XAAsT$=Lwd@ zP?vrVKHlK5+5}vq+dzjwG0z6AvBUmh(kp+l;O@)+rwfyedg23jzn%=DC!H?`mY)RA znTB}VFT2H`K2eJ0gLFn#N0MR9UG^&(nFp4WSL0N1FupT8y9HFanJ+Y28+D;aO7n^{ z-HM&D)PmirylrHu+=e#vg;Kuf`pVdxFK*q|(k?rLz-&j$1wod7>l`RI$FgZZ1G$hK zb?QN{UUXLlQcMQVjW4oO%zCv|g^RbZtUzdZ)Qo<%W8C1pOCmQ7BWgE=MC0IZws@8g z<GS=+wrA4LY)rR=5b60{(d@%rWAL0R8cqgipBzurPGVI&+Be6U zcjAC$Tbg&hrld9Mzt0?cj{SmfT-vf2Xow?j6mfPnELMHrmY<@K`1LhktabgPrtN5$ z1|X=zDuvv{$`|W|{1oAlkEHF&M38eh854O%rNt9ThhMp{>TY1QM~cw(NcVOP?0!=k zIK%0vX50`p>TL`w^)47J;T<^9!5K;Z%O1*)=J8;Rv!O?=8UthiU^%2imUUP(c4tI$ zN^>${!7(@1?qacO8iw?sn!o8Fp4d{WSdTaDptr$k{|B^_=<$a7NGJ(5<3~Zp@Ea*S z-b;#j+)Wj(gvXHl-{+Yq2Fzdt2VU_5cllZtgbbZECeD4duwii35q~E(gfC?8BM~EH zx0#kJahWNN`s7OqXIMx6*QeOVEnd$#nd(VJcKB@}y>9ayvA~>00Um0?R$omxqMtk+ z#LT-MBW5u^kuUEkvFD;Lx7_#&ANjo`r{R@#fg2?+O6A3+Cqu`5Ov8*PFSX}0=QS-} zou!isu=J{=?O4_kAW5IuU67-Pody|hN|KTR5hgkfbRimIh4X&{Eb}rDNhPpTP zUV?&X_8ie1DcNApQ}r&cB3VWx%j2e3tNB%wbSc*nxx|3E zr;~8v`02l4=d~K5IZ5{ABFU;cy~4?JhhO?q4=1?~^G3#R{5e&K|onL+xbSf>BK_1srI6P5jawA{)2?igk0qZu*z`l^g%7XV&owFJqwLmXg*kw%~QF@cveF)^1 z(WucS|5kq7hk>m7k2Zrs-T_*x)2ccpo)A_e=IAdejaKQOQWw$r6WA`5Yf7SAwtI8W zJSc{9A7d{|3Xq2uX<{aOxlp`cHW|(L|U0I{zP4R~;2~*L8^zK|mNl zx{*+jmK-{jHV8pdO6iaqN@@Tp=`IC>MkS@YhLY|Yq`SfIjy~`EKHpli<_}%WFYdkP zp0m&1`|bnd9{zBrKqu9$C5&CjKLDeLE~$`G>Oqm=8&*?Ib*zS;Gk9Y2~2i-PWejT6~rdk=R9 zquA=Rg3C9tzyrssm{6j1cRj1P%6O+j{tp38iWu|$M9!w>=sa#B?dFYj(M$-}nbngr zhDlv*C1%TfWju5sk{-d;9NoWnqoC(HAvKwLy4Wq?^8KU4eb=;nF0bg=Qf7o-D;&|8 zm?8HKBVSIQkqL4^f9#Q0t~QylEwVH3F_U2cQY!D_kb{r=tL+@OGyRU$exdgKU(Mu9 zfGn~Sz|fnUdO$vOV^=g0esnx%}yu`&dFD}NTz;{ zd~GO!OtDPx6J}eCP>?5U0+X*fn2@Fk_+D8g2our>3C=E`#J<(fwiHqdQ~Og+FzP_8 zuo{d+H~)c55TBz?O(RM!eN)aaT?rFo-(6 zI00uuHVuAI>Agl^Z#V$l(I8kqRmu#TkUiE-PRh8HBvan+2B~oO2 z`^4vik}4~@!1G^%g6gi;S$mOIV6YcE_4+rdg3HD}xti}pb_h_3~wArPeZa$8^cw%XUdYTSWzZxyNpZOh>hviW|rlA7x69-c5N}pSQ)1!B4erF8xh{pap@ADMIPGbE9D;%8J zZQ&To$^E@OlcyOQ8K0ezKZuR zZy`P|7}oCol6NYALUnytz(L+ORdG=jQ%>RDiJt&A_w*9_MBi^=hF%U%hb1ftD}Zvw)US98TT)SjLX{9@((4bB@?Tgs5|daYaU z^W(J4ZXictXLRUuSoglH0UVB>4IeKv_g7?RSUde$>~jlbH?JQomgT6snOl2ga89=G zzY{B79Shrw@r3jJ4))ha_G9|-j4lIYs%OyR>f#jB_dBG(wk?v5W(!1%{tqe?dWjVK zLD^+sqQ}u<%I|`#AUIntWa7GXX{EeDgBt$kn+ce2`L!6!L>jtidpzB7=0bM`Mqh|k zrc>tQV}~fqGf8&CGsA;#AcJF4!RK_Acb@r5b^m4DPhSbogOcr&xya}_SEBawrMQ#A zXm6~U3KrJB>zGGA=?5K(LxayF;B$Ji$>^ZQxJah?qd5r z2HN)56c(p+$Z~sePzi2^xQZ8tloL(D7ew}&J&EcfMAyB;?Af1xZ|`mr>p}!cOwP?Q zlos4oj}q^6=3%K7vERuuIo)k%NZo#RF}jzBF)5l~n}-aM@j6^HXTUq$k9~pDRr6##BN^=Z?EAG(#YbV4ryb-G+zG!<2*D;J_6fEFy^{&L-8%qP3y!1{pOB*u zx3#BzFVs@Y*IDY~;26=gZUJzv3Ro)*#iw-DJ^!avgxBRgAWx2V=N!+sPtt7b=my3W zoF(Z)8GG%q%vA|}Xd@Ud%vi?W5nX)rJgu$uv4Ke>;Qd;#_#td4LA2{iTVsjLC}E+F zn{!pcxy#}8)R9DK@J-^KX-3=tG6T+3VcQ!j_1*%;0C#EgkHk(0u8l;#I!k-3dsg*; z`ueiYpGV4a!h&g(vbE>)*y2r<8JkCPa?iZx3~UgHAd1}!rf`Ko%TWEH9uE1#~*DQ_~nB}C=3r1zPwgwmwjN>B1p-tyLTP4mI8p=OX{t7_&K z2M6&laBjX>Q-K^0`;gk&b9A56h?3pW>Gus zlPva23rXExejH=U$bo!|er#cJ0Kb$AXI|Q3N--#4Y6hVJWPcLo9i4Awfj6GNR9MbN z2GFWFaHSX5Y>B;sGd(OLpe%@Ei_&L4S3fh@K$iCS6%D#>g0=N08ZLb*LoQ zIouo~d16T)_3|dvXHz&A+MwJ;~MwUc!Y)y^_-@m^6L8eDZ*9y@T><;6>Jzt{07a>}rD5Vo- zF3FmISSuK4-0@o8k$RdyomY$8XFuMB`sw@o7Cn^ZEB3=5wM4wnzP5%2!K|U|cIJb8 zJ8-gzH0MREDq()_tib&nkHwgD<9Nair4%T~8PXY!KA3bGcdriQ7Vb-jV0_2F9 zcNJay4I0Cd7)Z4>FL^6@mJDd6MdQP$?>Lhe<3=@PYd&O1p$RcGj`zd#cev37Ga6a! z@~T@it||)MDX?Yap$V*2QxQ+Dqr4>M>SR|-d4aX@j#Z_g zSG67^d|!ALK6Iu@$-4WRH(2c;Q6xbUyZgXo@Q{Xs=8}yK%etN2$dV>}hhh16jw?wV zJ3~ell3Cs(S$j>*H>!@w`)R^2XQ%Z3?GHp_{ zYo{{qN~;+HmXW8Sxup*>I@p;7=37EZS4L^;$8>{Ce)hC0bP8yjxWIRLLRe`c825Su zqtU5E@{R@zE)@P`Nh4YvCM(n=Wc`>MVk|bWlqiEGZ@5eA+SSyJslw+MFvq^q z1yuE$$@3VDGUv%72ajP4TiVi1GD5b4h_T3Uc--;ZzGK&zWeF6M;mV0qHOxBwPmGl= z!_!2ffdL$K3eX(J{!G9fg)kuR&+BlT^H7>Z9AO0E3r1<;I z8Cf&y8TWy$RldN}Czhodl7##2j8PdHnfZ26zx!64Ud||YkMbRxk9gHBIvX)iwCX8U zeVP@WRi*T8ObNYWo}Dz!k&^R$SI5caA!NcEG(hC7&(+D-XPT8Lj8SVZPaWuT!EHC- z`=^`ge;W;)2bRZW!}j{~n%l3FL+zn&p#D_%EalVG?_%+u4@65>T1XILv^>8E8q%DL zx2Coc?6CSeuZZoRq}cRk)r_5|{-TU-vfkI1v>U2s?Z7BhGxu;YpbcM}M3-*=2DA34 zpco<+hxpW39h-dM^c0m@A>uwAae#;=S}6esH{%dSXvn+BdrJ~@i4_RfYANKt3%-Mm zff#kjGKdQZCLy;YAdhB{x1s*5J7gx)u)ZkxUO5Wh=nN1G0-T3Am7pJeWB1x{EupWf3+iaKCqn>oF=L-#vh|NDz4E7&nz=Vv-l#S}Dkd z3GSrIb9v1YpK{oHnXSTLo-`&KtNr|CmGyT?`s+h=NnmOkD$al3)>8KBN&BUnMXE zBn%VsHG!74*@zX``h}=ot)(0*ssG?@>Dh+KhN*Qf6m$Q|q4DODy~6fU%ze?zSkNIH zPpvx;{YW!S$mWfI=Ac^J_)y|uJyv#_B$@Xuwlinknu23V;M8V&65&XE`Mq)g|1=cU zy`N~eby?*y$zR%c4%zilQrPwU&7I-a_o05^?La*K;(?Tja%NnkCP78g+*>R4dR9ZB zv8I9R_gB~B6)q{xK0C&w)q0#|9Ea`cbF5!h;c!xVUg5L)24xVf-Js-;2gV>zD$2K! zbj9s!1Htn^uJ{lP9WCxTTD$Uej^1`lx*Zf&^(j&D_#E~5wjQc;1ogomR->H zeP$N4xj{iB=tQ}kZz5*^yy+459}zEr{rLHnpHke)q2eAgkWxc5xdemJa8C_81czvZ z3NbE7UutKmep-6v15Q`8;l+GK80a>L(uFv5b^8eBSiN9+{rG0VVCfNK{mJbm5#51~ zoKgi(hd07t_gTi>Te>!%7EE{A`E(~N*_nW+i?q%hzR^DHrAjnJlAw*5(5p{y2(6hhYr;p;8 zYrYxfC+mm8-`p3&Etxmp!Cx$kY7W^gEDQ-QS5|nwaMzbz-IWq#2uB z`r7)-KALEvAGeZ#ae*4bLbq}tzWR!<3X|+od}impy?inv@rrzRpf5i|L0(BU-$*ty z)U^lNKOVK1G(=(2Fu3U8pFX=3dzGUxUQ1oeAWQwe+y0E1`=xW;s$0KOc5#SB!lt$Qovg<>i%+7l9 zZ9ahQ0oc^2oF$xk8~aa}#WhYJGT<17G$@Q}^&%monxBl6?&l}eu-r~UMYovjF_?HNADS3Z%R!JlyMq|F?c{YSvOYbzFj0`yWizDa z6AbIv?n@UhBxsSDutA9|SS*OJjSM+735!K=M&_5c@=sa`GUXvt?sqRVx8g6ea8nm9 zMFropj@DRUmU1b1ka`KbmtdmQOq(dEn?H0y1Lb3<6GIuk#*S(AeIG!n?jibnhqWHk zhmD#*ibtq+wSCFqiUo0`3zVsgRdtjQW8;9vYXz5+@`MLCmdKFi$y}l$yJ_;s3LARr zn=~XvF9%Od^>X3=6d-Z*m>Cf;QV+W+pYu(&qaAGu@_3lWDq@*)r?$;>A0_p>N`j~~ zkUiQG5Q+SkNVDiv=>q1vz?nj?Ueye}?p0#sQ9WS5<$65YUu?1vp*yjm`nB;_e!qQg zVIRM(bD#;=hh(eba(b=gD*}7}K_wP-jc{{{i`rsYefq|XP;-Jel(q64N7YecsTcji zUD{c15z}#)Y%|4s!U^+pZ$!MIXFLZ=Nvk2H=?#v35wmfzp;zR~PG8_SioX&c8J^{- z#zkEkFQJXyN|sLth^>-1$k)^U97gw(VUA5w~X(a6rI-4Fpn zerlEi%e}et0mkL%J-Laui-nW-uMoeLEjg_J}UfO+0=LBfQO( z|LFkQZOy@S62%}TS$j0`YUOj#XzQsvd%9PJE~5_)10q(W1V@GpSlAyPiB6e9zT9{ZCL8> zHd9MHCn{5eIY8@M>mu!uuusFsLV;{{xrWkfcYX`sQC+Ia2brd(0cd&=hkOdaok4c! z-1+2&E17?_FKzfMl`+fgqnuaj=CH=sAwHL;j!wdK;;Ig zn33$yY^aYo6>Q>a><=-VK3Ar`Jw_Tpf_XCwrnGcID;}R)vm}AtmxVdp76*%iC<03( zz(ph&Mk*Dd*kVdeh&xBXHfIs2c)Oi9Mu=PAA^*)&5u?IPG1t$b^Kle0NumBYV&{;B ze3>>si1oV!=QsGw4Y^U0FE^mQj+g=6G&tLiLjvcZSB^!HiranD0L_^E$&o>EQB? zu-Ka-7ODOFndh$Z={Oo|+(fNDbRMGk-rS9~{YcH=JteaMns*Il!jd1Ch~I*w`vtSu z8y@Wel|=6Q_}K6_i#XSG?n3T%oa_o_Sv?u>lT~dMrr&zHqtNv#LeK_x#vzw~&OLLe z!)!cAdx713xzS#%2e;>{)d`-;JTch7@;owsH^jK_i@9z9lU@7b;Q<1v6p&mlW&UA6 z$)_}7<*DLb4r=X<=8ijr_ZH+~E4c}9%D510>wzS*63hew$$niHBM5bB%QbfD^cJOO zy5EM3v$;zAsfri-IS%(JaO5EmislU#X|xa|&6yl{7qXz+fkou1%9mK{=riu7Euj%JV<1I%oe~I5$H94XlZ5*L>p~nJA`>CDh zIlGd0j-nJA>{pZB?P-T~Q4EQ3ud&56z3vp(B6P)oOc+WykZ6U#tF)!@t4 zme6HJYw5@~&6W(o1*Lm7{oIUyfb1r&i1=3~gFxuQ;p0uypIXVxAoM@JjW0-VCrvG> zi*@qK6A3&9DI^+ZCEmfy%%@Ke6*5tNLw)H_mp{~ZdiAJ zI{$^{n1GV;7j&+wT_6QDfPk_z0ngD_NY%P~P33WxpQXYdMnyjS)B1}iMT|zxu%U1# zR(I#lJtn7p1&;Bw;u|C-TNe?>*bD0dKQ=CzkSx39J)T5Mt=+56vq;qLA;7-WqrQ-> zz*Ia^zZ4YoRL0*8QO$TdRel))iXKu$`Uw5k8y-nv-W(5LA#Bc-X_8$k-vZ#+!W zqu!!$w8*t+p_KEKyEIU$7;=AVe~JOOxd#9F1bsWca7(d5I3<}snMvxy=nzg>zBhsk zeDzd%w{ITF)UM2{C?zhIbj<(`8s6F_OAK}9FTt1Ym*H9u3^I1Kc;sIlv8e3@V?<_c z73W3^k@qb{xsYbA^B|LV&tQ zS)6=!XE#RT$Q2)7-mYM^)Pemu{msR;2muOi8#xe`aN*8P*Kcwp+bTeeE8fcJo- zm@G~Rgjs~NHOX96Qx+dM0P35EA~K1OWjoy~BWi}}0A*pF!4&@t5*F(>yBHVgRhzW6 z@2E0ooSYD zY;>6JdFulClNGunjy!8(T^r+1yc6^?lEc*Ac;qRQTfwH>aT`H8aCYe;~je0*E2X;Hl%ksBRg-@Hh z=wveC(F{14B_v8oN`a`Pc6zTK$^J(By@z$miM3VrPsBD5i@G7@47-e;sy+jx6akXO!rYnOU%I zPZUVVq)ohMcB(D^w>7KB+EAWhDLjjvO}N*D&oib}yIr|>(~CU+J7HN$vhETLzkCn& z_CvM;$eu5t{SrQR&WF#q4~4yk>^p+yQn)u&oWB^|XKkqJ{(1VvdgfiV?;XSZuj5Hh zT12-Gt3O)TsqBhu zR3M+;Xb(sbJ!jND7Yhqkev1(?Nz!5yc^t{TULqKkzg#?28tEM>e!qlpp1Okd{R3ko zq72EANdgSwu}F*v_ukh?Y%C$!N`~e--5gSw?S)bd4-fl-NV}-;h}RjHz?k1Tl|j3E ztq<3)M|OLyy9Mlw?-CVY>80EDH2XVIa8h_i2p7kLzj-1FTZ3~viZx&)ocvx{MZH67 z+p(mT&WV7C4#Mg>??*L#9KOs(J9JgCz}#>dfqC-~|G27RDEv!YGEc-AelkeWC|$3d z9@MxG4Z1$5MQO9vD1%jxXMpD-oot1hhrb)fUANJGBIZ-RLF(u+#npbiKg*5PUg04L zr#o*MJ@s7JX%IVNlGrXe^C_?#_iabrS{^?>_8#Yj2gJss$x3}6yvDftt zDckzxZ!MmC)c(^MM@Zj+&WUzpiQA{ocLBV)PW;W6`#4uW)MD`25LT7yi>X|dZ&C%V z-|5wPrLs_XXi|e2G0xKPjXGYusAvagK>O})?wZeTS+`Z3I>Dw-uKp?diTxk3G?YWNXx*#<<*ihB z%poM$>T=vNb9{g5dQo{V&bm{b?;9ys_^18pTe3xi$k(T2fS)d1Qla$SewN+>nPGb@ z8esM>M#NveK-t5Fpm|lccM&22@%6HO`gvU+-T(F=taUVj)umNu`hdLQb|%Jg<<|aErc(Uh*C7qze*i|6&H66)pJ@Y-e>C_n z25sm{+b*6<2-*&zg}L+g0M;2ih19Vq@LoX=4`5%D7Sa6SZiHjF;%y~fNTM?SAMLy& zRMq`%qlaz+d4Cl@Ol2JJ5V79D97Mxl*V}=z z;#Fs~@SnNK*x(i;_IiVIs|rQGZ_>vvbL`=mW)v^`XE3E9YqH?(pRfMX{rBC!C2hEm zU)KXV64d0}238DnX#$q&t5a3ZR>SgT)t+6J+ZMxm{FGVwI%IuJ zP2Syc>yXcQ21lER=A*hUY6?>SG_&{Qz-4?YIiLRfG7oW$e@28uE^ZLf!$cU9%*+8+ zQE|PX$Jyjijk-!XpM+rIHt1lPW8JSq@}v=&{8VD%a%T_3zRrv(L4t&ee6X`PRgG6Z zAZXxa9IDdCHQU0?etu4Jr!wpgx-b4uHbJzk_|KrCpLk9mt5U%lt$_eEc)i5!=^Ei$ zkT_*4MbT#$6mYR?uT;KdR2KPt^xldMOF=Pf;#kDBanvYiXS|3w<9Gc|Z>AysY?=sH za&1RZh%Cp=W}d|3UsMwAIY6P%-gs3Jz#kMXzWk^8_%Hx;KL$mRcR&2ew*EGTBQf51 zpjpX@+|QeAE2haor_Mrq1$VWIxTpSXjgxe_GX%2V@?5g31oz1UjMB98G(Vb;cenS8 znb&8VLuALwg-MVsS^}PvtOlnN`UycE@s;1?TyY&FGS2w}1uVI5_NBpmw0JEWR`|(W zz)VLk5(6~ej^FiA?qBaAOB+J`MIXps6r@Km|oO`rPe#+kWJ&8FWJyttScu zkbkhz<(kesXv`-XXE+XqX+rjt=44;=)_5MS?fkx){f*D{dr|7jj8ect^-v%%Bh-QZ zr$!@}f3A7On<5{s`;!6@?hna{Pw^y)8+cy-*x2iCI_3^nov)`@=6m+~%cjeAADr%| z7e!w>zLC$V`HP^7ry^b}HX^QPPfpIdJMQ0dRKnUCceZS%04}wt(;e&kZh~ z_+G(#XA3@>(TD9i6md?q@ns=X9nhOtE4o~8OxKA2*8&Fu%{QXyg8-@xkqN-j%V((I z0y!!X;F@;0d;Eg1aL|f6wXhZRqT^B+AWEKodYs?X7;s&US1JIxnt~Ij196_J&T1pk zP5dRDrgecBnT3;vHgK`z8InJ!i=m7>Nfr8ORgWK|>B)T;)oCT>edZ$Kbz~#rvG-z$ z`CL_m0&nR;QPrHrb>^2y*Yk>9~tnf_=k`nIERKSZ{545$nZdjWPQx% z@TZPaH(`!&9PxND!ev!3FvpdftvODDJG?J{SG7VaxznP7P?vo0!`rmkDrZzMpjVB6 zQ(xJS{Xl)(&plr)&-MG4hA;J2&9-DX?#NGS+Rx9+#2$jl8UpANDB|z${;k~n2s9k| zK_@dLg9yMc4>eX8v&=G!NMC0S^A5Dp=(s6+}d(^vQZ+80}3 zwntPNd2_yGUNWTD3kB-556|8AqD~@k`wl=)E)jo`J*QDJ(B1&KV{tP8+eX*~Mxt5D zmcQ|S`H{=mDMostXq`Q+$Ry%vrcr{rpC1L75=BJ-7H&2ixLK*&=lJ(}xX zc1K{|)R1QdYIIP!{c@xrSq-snJ2^v)w|S2lv+k1w`&eXjp8&SW4=^i^#M2qhi*sBa zv|Gua?tHyBZ*%ztCwGFa_jLKf%%o2m5BsK$;>g8~`|J&e9nz%Sla)A8iUG=hrkv8; zV7~2cyawyP-w+2D7UC9yPI%fSHgxHP-}iXhO|IVO^5u$RHZ_+rzSeRujUBHeo{0?D z7qTjG8D0L!wlw@y$mO449s3r_(M7t-h;bXlQcyUL_oedawq_!@en0^VB4uYf4dya7 z=^^vhd*Z%3q~~66LvDHFgz3!tId`~95v75In8;CXt+?UJt%X6P2S;Lej$6?awp#2c*i!Z ztgLL0-%&!-@W>zNzTmzzFghO+Db;smFl5{puj8$GOQ~}`tD(H!RvG`mc-G9ROEEzA z@^;^CrP%ahr*`i^QE5#uM@yhtt?k-;e*vYxRDc)J7&+Iq1JF_pPVE9*G}*R0;T}G^ zu8d*femBgj2e}&wxs53EiIHeAymJ#{+%l z{L}|hS=ss(e+ki086b&D}v-dvgUQv2m8zr)-)TfVnt{@wKqOo*KHd9_tmp-dQ!mL9&UKz3hOc zM{#lI5zfDJK1M%Yf4?C>+xA2Bepx=@XArW{r35Tg*JsbiKH>8%NQ*Sxh>CRh-jZFt zq}2`pgW(7aX7(*{N(YvuJ;{7S$C!w-;IF39E2c!SRAjv*;VpHD z$<#GRWp}Di$cqXT6St3-54^8#Bazx*Sp9is=WIs(u*jmvzU{`khpBj)vGn*eP;D6E z@NmkqtCq^Tb$dgceVlJfo-UD=&mFsZ9c|zNjh!Q!r~B4dPf(<>J>NsTvG9nhgVnt* z{Tr&Zf5m}0N>cgB>gm1PK51q}O!X@_?$~#s)$tZ+2?3WzE8Bm_;vl?6D`2@iDov9w zXuyK^rHRgiLvrzPS8|o2`mDN0p!{KvV?w1Hq0%BQ1H6;Ta#s7&)w+@_ZQMq6Ff_$_ z#q`EJ?L4{6%>jxEM)(+|Kke>u^?DI!rWvLh-EwSr*89bUlKf5~qezs0S$ScpFEaQ5LZ}HKL%!w~kz&cI zNDU@pu)FFPwR}JSODDk+=_QF;Lhh#^;kWN?%1RYKSp0MmCY_BI)PJ;8JpFehRaU`R zf53IF7)eVdWH&{5axwx+kBxYb78h5GA=9CUU}RxX={Ofhk}r;R^}K*RRO0gM5veKE za{RhXT7@NXXE(Eog`{d{pt-25y#cMIZzf`LU1DW8<^x&n#x8%pe*0@F3LDGh2L7u~ zN4d?v3d5x)Adv=0jO!|afs+`7_znoazpg4UN!veU;E1{ zPms2jctGlv66g4@xrxc{SLx4*1t((R`T>a179UOrlo*n~Wp3+?ikjXS3BhH!70TnP zCf5#v;!>0!e&3Rx7fkc=WGWY1!uG@FBC`)uoyVbqpIzLs6?bAG?Z~&|$0?`!VS^EOyv~4$FqFNy{;yHCD zPM2)+0Sh@8kCZL2=y^?wSJFnj{69hY1zMtMGx&7sFWmL_TaU>v$v^xJIG^<%<1^#Z zD)-%mhW2rObMwd~S+PI6-ve;d+*ua6aRk-#0_Ol!!{zyL@F+>jx9*g^{rV~3@Y`OM zOy|%Fa}J9xg)S!57-ww4Epj&=gbMTt^-Tr2FbKAGF$iT&2x%ihzDrj-ETA;3-_UJGRu>>Zra&0dd%rX#78uI17g*kv-RTCepjHld`9gGoqLr|MF$XrXh*E6?vwmB63RsNseK@l z_am%}Ny7i&s+Q#9P}8Kul1!}rNyBF^HZzd>d39XP2BU<=5Xx0ezuhuH`l7bwngh-g z$86`z!_idnbnU`tL3bYKjeR==-Q-CRJ5c^w2;`2T%M)oxnflybL4MZ1YD09@f)8Hv zj(MbXBxeGUo{W9j1<&ADI$enM>K&||UexJ!EwJjasH%It`Aoc4$nodx=bLCXy*Sns zsuGzb?pCDdV?SFbksDgWQE%Q&UM?p2dP8@0w%#`h=23>Xf5MR;DGN8>daNjr-BBjn zfQ(0qhSak9)sSB#648kbbmJtQS^HfZI%2WH|wXbyxL?LR@b{6z(k`OOTdZVw()3XyU)H}?}iuhc(kA1a%JodxO&BN2C*!J>?fh9}ThtN!^1b~WIZRrAs78@i{ zi_>vzv46}`hUKyW$RQ7KJ3>5ur8~#}>s5q>VdPZlC-0w~1Lb^g7aYiY{fWwd3cAB% z+0<^J`QySjT9`_6^KO5u$0g&&G_Z03Qy2!$2W2yMTN}Rf>R3c}4J-t#t(+bE=}N~% zfCbnCXzY9C>Sxb9hhi$E-UYlg(z$K!?v|S&s1&`f7?Wn*7bAQSD}U!Z9t-HTnXs#l z-}*0l8Famnm7_jk)KLz5D9@$#powAIj}mKl zmYyB1TV6^YpM6qGrD0atU$pSA1sYXeknGS5n;8?mI>sTSrX-^bIYZ>iWn@R#v zdLAdzpNw3(J9Z7;!!-7_&>8RJ>HaS|JYQ-Y@PVqeqM=>pR4Cv&%Q3d7fV?)tP4!WR07)ko{oiGl{B z4$$p+o_wuaj4Yi$nhZ&eKSk@S*)!oJK;=PB&SOZMB4}+|*ZZlwi&=+HlIi_bjJ7&O ze*i8{@3))w!Vy#gY*X<4UXV^swrW&6{fCFU0?ba3v?{Ftiu?=Upksg%7A~Ex`O)_5 zto*)?p$98S9+3sY?5at|lHpL9hgu?Ci`H6$;18gf#a`5cR+PZq1v*h-&w!S%0MYzf z$%{@BQa#7TQjosTtS$z$o(Q)uTmT0y{=~<)D3DI!3r{%&R9zWI^FD82@cgXdvL;b{ z4$s&cdtdB~2{;frJ)d&iYYR@8QOyPDL=$&0EgZ106ZGR2eU?FOzG7booU^h)5PoeK ze3R~F!g3mEhE(75K2$>v?QsM?olMbJc&TX@ocaHIGVmXo>KK?{LJP%QLv0>?h86co z0h7<_VD4^JEi}c)mrc)pO(Cq?rTAKiTqe?6F zdLzlP?ZXWQ(`B>t%H5`V*gEk*D2>$dPOE2HV>non;|(^T`03e(jr@MElHqRW?gQGO z_J6mPB{V#m=KLWI-EaNR2MAmT@EZ}Mi|B*tcZMHp1Obgy5V%?}+8N7!{BBN3FM=T+ zPyJ`8+&TCwG9iD1S=9TGq^>1QM;ahe7%%G}#1HUBROpHc%!L&1BPGwdXr+8$z%+q^ zEzc}Iaz_OY=ZLHBWtTTtXv!CwBpyHQO{<1FQeakoKh!t)c*Ot0Nj(|YQSoO4Ln*b? zqAI&ahG1mn6MwpTXqQR1Gz1QOMqZI7EE6y{K!hrH76>tL80RJcOFgw@D`XLEBy;XT zZ?bTDzqkiGPPnBQB{qEY9}j4h5BiB z2eyhN^q^gosu5?aK-RmWlzOpnOk2TJphT;Pu_DYYMvBQd+UhRvVi3F&alOT`E0prB z$i~P(kDk({a$QmFUYC>qw1V-P_K{geVS`7bTV*1)(1qfLMc)y~NRon10J60qysmeh zsDeC%0-F%AkY*|3$hE5_lsOl{(Km->&+ZBY?~>JBAvzwByd4cKKD-g(OD!nW0F`-o z_`=^!s^cCAu~o~3!++n;-|sS2jCm}T#V@ETCIP0;#N;=c?uS>s|$D9y=JRYi23%qm*t+nG6KCkiGR7`VN1 zS<~PehK?oID8EgL9EczJ`*+o!qIYfIjrg+vO@4%Q z0xqKkqaY!z2rulQU+(Daa^l-7YzC~>Fyff^%;C~46kHUib2JI52#vo$Y(tQ)F?37__paO-CFmvRIr%kGeiv$WU^9?f#aa+9u)ZRLE&U%F`fK}d=v`bH#x@WO%@ zgNatfkpW>mzPRJW&)#zC>EUfBzWS#+Lr80pg1T}zUnhw4zgdX?ASiUsq-~+1l`F@^qZYUG^+-Y88jTisvI`zlgo)r`L+~@ z1D-FL({vlwRc7_gH2^Y@nt|o`bc%6n#Lw?Fk|xQRa)GXc{zO=?1@R`h>l+mNH?DX( z6}^&^ysbPNh(R|a+-y3sqqjv;H?YPba;Mj4%OZFdcdx;2?A_JrX+I>^9pj-ti8u+o zCAI)%jsPxd(vDDRg4VT4&{eR=?&W9tCfV`g(>jUU#n=fymOZSbSHXy42pZ#ff8ji@ zPjsf$wSr5DOi33d>{v$UPbbXV^MzEoeV+vBY~If7z~aIZ<{T-K;W<^fu4f_Pyl~*| ztv*$k;Za*q*CUz1lMt%%w_=lqEP#@&-P%g;UnzRlN8r`R7`D$;fv!YB=@kSvmmsub zX*NLhR&s^EH6T=hjT9l;aiz*^e$fk}N0aw%&>pJHhPieva2#mW0%tE)gxNBf)5;~S znKDo({L2phNmUxVhr5^04USzM{il;#7WZAplVYzUL)tT|BWo*@w$1m=V3|H#&Oc`z zssF>94+ELsXI$&QI>x`jUOESlF~_)6#JIta*KVrvPhkiUx8UXO#1K$Frz|sHb17{- z25G~WP}MR?)q`I;D+-MFFd+iJ` zl;`%CxnjfqowqPzB@#<7&9gY@Mxc4ff_76Pz~E5`-AKNOV-sO?M0UARN_XG{I_ zY`_};cdj>8m0O) z6V{_%TD{Hj-yigU$0nT`PUN=-MP8?OZ@5U=hxY3jdEegvK5G*%966m6%;z)*6W`u2 zS^_$DKyYT|w+sdd@nFtcAaOmQr4k*UHw8zAd6aQ*o-@uYnU6ni4TtDT*=&|5gp((~N&pSnVBiEdIYZErg{kA&Q(U|E@Z8FZZBQGX zUkAzg)lnadHdWPQ8i-UqR{!r#wukPIk#`R5`vA5M@jgEa0XY(_nn@_ef;P7$Oynzt0!`6rr{-M zy`ph|)6Nz;{NghY2S+$4tuVi^MeZ)b8PEnD|9yrsIltFBAdP4OCSZj>j?QO)OL7@h z_Rrv4oY(*M67ZZOYy?j2fRZ5wvh+S{%>w3VVJzJQvwlHf^yZ;T%_lr>iH9WQfPsthD@fGZB);TXV|ZvCt9sSLzG z%FSTZ0x+>T%J;(TT;v?|0?7@%51*bN^#D>vLE-E9rXZC_mAxo;eSx~PCH_0xRqUDs z3;d%}l08-R$-3I;?*BiZKi77ZHX|8;-DgZ{m*SGMjJO=!O!X}A373A3>rlA6%U>jg zhjL~5WcK&{hxub5fosDouuhmSZKuc2S5LnItVeF5T^Z@bUlV{qxq%k==CD1#EzkI# z@S!M&+hF%&f>EQvu4qp})_cEqF9zv>4We@;@Cx6kTOIxsyA>+Jc<-OKkTf3Hs56#h zjkJ^k4|i>AUT@w2F#?XpjtY9p;p$}Gxa{g_xM;69m6Q`#4x#^r9^$C=pI;~dGGm>v z0+2a;&{5*`T>%DLxS+puhax4mt-Y`#`LTK)NE8EsgnLry7s!=qB56f)jx4d%ZSp2; z4DfxPyO%CH=5qrVSuQ{j=);LW;n)K1l(r!rw7ZuQb+VbUmT#4+U4EG^gX2Wf%*AiV zJpG=`;RF4M=YM-t{QjN%BalLNNP$W10lWU>sP-y^XCKtjIELr(K_2uP%6P01u2fro z=)T-&Rt@f5Cdl2-|11RqSq!G+Z6srgnPXTquzP0+0hL9HF4`E;DTdEFhr(8UO3gT{ zQ5s`;qQpe*8tfn(?V-_Guq~3~wYE!6`9{IkKdRSiXu_~zwi8JINBch zFgDJMdD^I^TY&G*=CWr#L6M3xAK@`#0MxqvvJ9gLP}8jEsU zlq78f0z>8LN-B1f2T9kH)t=aBsg_-^AEweM&*aQ`@^Eqr?BdDMvrQEX0#j!}tNsQY z?aHIkLVXJDA_HrC_8o1mfCr%6ZsI=Yf4wQ)ZHDSug`I7yM#`$ZPiz9}wt;1rnIoyR zRb+T?oRAMgYejyM8-JbT{S0@5x`}xs!;Z$q_}~n`NhGSU=fZhpDa9Njb9# zCb^uqMh$?BAJB`zB5dDRZ|(VL#eK27T$=Ro+5e(1iJadNRl38JhVfHi+ z%*iTXXrSs>yW($XO<^^#pL^t?cQ}Lgp{eRy<7|f@E$7?hDVjHy89m`%7U` z!2J0HctlvWpYyqPSw#}%!K=F5m_{f>+X|ORR2#N$g+*^wHNIvma{gZawsQ0Q{BV2y zbaUM%KhC!<0!3!8c)UF;WdMrWY_NYFr5!CkUNrkrybs??opU^s)Gc}}yzs8~SgkYI z2UP!Y9O&sGx%*_w&;upP`LIqN&s{E8KD^lUpP1F)hTCu5Mt98~Q*yV3bUx)(5#N9Z zusUIrH>;9xTIzW7t&7d~gxweTxvFj`!5G9}+qf^qC^Z3#Wh^^OgPs%h`ENNKc)%`v z`wVy{a`&W(4#{)L?*Z??e#w{NH01K1TrNo_!U$+fk-(FNx{Epd0KNAn8hUn0GJ|P{ zObt&2#*>0B8)E|tj#{7lO_*1h5@L+HGUMPBVmiykB6tqIJMj5Q3OH>IU_BnemmYkHf>>TABX+b%NWLX(AV(+VGqpzde`LLPG~50E|6ilTPEezW zL?|k1)s7vb!)RSCYEz>{ji9kt5W7l^TBTars!?iH%@A9a+BIUesJ-X!<+|RV_xt*u z-_bvu(?8@SuRNcRalha0Lm08U!(iS%Q7T{g6qkt)^qY70o?YuJJ%_bj*>&I{RS}7p z_!An75(w|Xl_$udy_Hd0;_PI&nn%#J_fVnUWurIDAurFjdWJTflR2HnmpmeJuPXfa z3W5|O$P{{wGiPe&fM)uQ*Ri`s)ABTB(Kkqr`9ma5h9~cWO&cI~Xo(T4cU10zP_3Si zUe3v0%8t?Nje*x-tm^g4*!P}T0ZO?(Gm7eknU{^9GMz3sxl=w-CUs1M;eopA<3~dE z**g{vf@$FS07FPL6^U~1F+z$2XMXro6Y@m*$L=h0QhLq3IoQS@Rtbi_okrTQ3u`Y z4xDfg^md2=nt~AQPlq#JzQ^!41Y1P3x)9Uv1TMstbO&Bk@6{mZz zP4a*%&SB&vIo@S|;LiC|t5wMNm$%Zs%mP&Kvzz> z82#V_f3*W-W&O9FMNMBBUUb`k_kUYZ$vf}cc>g+^_={73n&S~4l{%S^cCcHS*-*s~ zt(U)eYM^Q0LU7tSIGl?T7(_oe@4~mxZ=ikN|^lhvn{$IF&y3IJ~9Ex7yx01mUDS$Cd=5`DSIG7>1Zbm=Ev z8xXCsdZs0(bYK%eeCn0kwgJ7WW5B#!BS$%)NN+lhO?kwpPiZgt3w6s9w}ij;wwg6n z83%4wzPnL6Pwtu_nDFxwU7z_Tny1B^#IG{Z0QzP8X6rtKx_rv=99I{1Gt8>jot~eW zI?PYmEzPL?=J{S}8YmORstxn#|DWAf(uFKX!u6uh>80wgDH2|c1EhOharE`Sfve2% zpRZ*FAs4`;dM#Y+TUw3$!g5|X%aAi6{Ig=-g%pBuzqNFvT<7w7ww5lgpLVO5AqpNm zP52B16a*6*JAjq3`4f}XVNDm>IU=bNG>o1=R6D5Mwr!-U63y3M4{%D-lS!DVZ@POa zmx+w=Da-kHWeRX-P~5pZEc7I+XAkV=6Z%GW9Ed};D!wxzE5|mAQmN^yCa&}7p$0jD zf=otMwEE7BPmGsB-+Msn+qVQp4!@qVErM?lj(tKNo-BI^R*$u#h{89x| zx#c{ktnDwTgnJsVjnckN{782sbY91+UIlK5cj)uk=YN}&S93Jqq7w8y>wdqgdiSY6r!X2vE3T>ac4vb(b*?PNE@xdV9D){UxdrDed>S%DS-i_#bi2SG98`Y$?q zzBfRW8r(C-#`BESb~n+>t$z_B*5htrjssN3hs6p z4B8p>DwZc+MDnjE8u zvVS8V6-*Cy0h?|941cKv#x6`93JSD?oIu2}E>qMZk24Ndl_{2yTH0N`qB)b91(v?8}4Y= z8xR2^P7k>TZgFYooSs1S?cPsGBMefPtL%jzt(k#e={1FBRNXhWhtW%9)W#${^5~m& z;J&p1j3?5b^|-DI<`hmrddpVr0TCnDhWl4A?;c`@>iz3AlmmU` zFXc?U?_^hx5QhzAH*}u^b^1$#-bgi0evz1>$&J?Daalsi--Duez`I+(-2cyV=%4QS zeD<`M-s=q{F%(JVT&?w8e5t26G*Pz{w=)Ki-MVYN{rJsFqL3~@7;XNS8877J5apXX zs%HRq1Tj@~)a`Zg^bLe^Jtkg+UEu^zZQPm|QDE5LVP%QFfBco?8)h^DIo#m+<9^T6 zIrn1$l3ov1Kg|Vozc{sSELHpJZW0PZPtN4LuH7eZBM#39xxVqDtMTIT4VJ`$yvEZx zhK4C1*D1Eg{nk3~*#_^GmA^dA?3SaFoNR#Wo{YEI4P9;HG4t-ztW$a1GHYUV6N5js ztxxAT8-C$<73dSRM1MIY5j~06Y)%8%x*~MK=EcsvpD`P=H+@D+9vy39cx*blk~?ID zZbrL$sz|k8J}I|ZTvnK~Xq>0@lG?e{C$c0%i7Be$biNux>3AH6z;4AcEn%{?2gB(p znZX0;*m^!2;7^D7#mb^MRP7s|1r+L^Z_amQhDiGwi;6sf++neUkMR4q{il;qcMCjF z4L)1ZvslLUnLAV|H8SJcPi(x=*YeYh?Ss>OVV3?3@u$EA#mWtw#TTF6Oosoj!~}<^ z!wgIo(_KKn@um1g5IO#0iEakl!q=lGSiw(nu0AU|`-`?jjq(r0vC%A1?)|5hS|!nb zMBd>`o`(<)a4!)^637dq`+{eE$7gYseOHZDHIskZm*Ru`rjg0%XsYcjSy_zj)tgBt zB4Y&#?gz@*vLu_$;2l^PEBQn!2(N2eAJfu1CU5Ho$B>KQj}o#`)50W z++Fux5IKDhxtEtwuF8TglxYPyfnyT~$8&-zWi zz|rdNTq2|tBh;hPM?BxbNBaWmq5Henzu0?T^3hyr4rOVAhIESRI~m5u+82kmThT3g zHkGhcj(_Ir5jjb71!A|Hc{0Iw%D@%?o6&sS|5h%N*-+r+@U{)UDEpmj-q8|Q@qlCL z&a!L$)e^j=gidl=Ch5sHg4>8a%hs3xhb*0szcq*FcxRxSWG%Of?T@;Qz4z zfY1DY6z%_=e<9@X=};59QEOf_pIOC(e4xKj3}uUbB3Ak2^Kf&Wz^#^(of5{L%Vv*w z4(SpvR(q`=;Ay=ma`G&H$}5E6^HN>^UeVBR0*kDAXbjD}11qH=PoN%_aZ<*4Q#x3< zIGEncI*sns_K_P(kS!Cx35X1qYWFL#%CqioR$27JY_W0BL`pKsnU;^`0V;h7#5H%r z`vfi=q=?mhUfrq&;C!}m=0>V=zw>cWau@f32K35=7W61zx)=}a63OU#$?}+*1|G({ ztu$>V=-}E{$e9-8IUOfPtrkLJ93 zHSGUGFo!kKFNfz!ulB9pyC_V!^a?kDG`NJ`|JZPdc&ro;VWf{=;WHzLJ>}k8r zM7@J0_T={&O6DAWAzW@+~RahUrYq$vXBQ;RR??cQq3tYa2D!mJ8?Dfqn@Cn+wc zemPTcbKY+EQ&X5MIFn_g+L2YiB+rg?g<;YRrUKH+zn3I;HTsLfvV7{bht@NFrsQVB z$zh+~+8SkHF`tS2J61l{p))9nJU|q4_L487%18f9RfSw|1EZ zi}>izb6KX=_DFGA_MN^Xu5(?XoJ#eA8ZQ#d;+LU`nNA6qzRyG@uBpCR>$7N*P0oJc z%5v_f>Z@Z~oTy%GpvcXZfllH?@q3{LCz$}|h2xEcTs3~mDdrl%8o0+6uKo52m0@UV zculGpm(KzN*<^m6vbDjtzKk5x@A;E?Pues&!(LEs5`1?Z#0LNhOXWb-(H+50OksHL z*~N%WzBB~hZ~fDz3spjrEy&K!t}=dyeqyE})Axt@D|b;hm&2`yVTMJH(T{XE^Et3M9@YA^|6;5TRr?!qjlxoW`?;rZehNr0883*2S z)elXoQOuIR>sU|>@ft!4FWd=w7F@glRn{aEfvPshxOxaXBsJr9sw32j_^ZEJso0$9 zFeP441IKv#f8o0e_#uOXy~KwCj4Am%;kBEzN+qSUyB8s>BI?LPuz*hFwtG|Xub9GS zWqgbArgb8|G4zc@o{Y`4c{PI>IpooFTYXVvSPb-G#xY-`)HBt~^k9Sglw1U+K1H^k zeHFnhVDLKX-5K!$!Pl36Bk6V{CYRd?UL620$zPs_pBW>b8xcGwE*i7}BW05VHXvDj*# z&(=aZFSf|xkK)Dt03~q`(f++b)3t>vPxn_#qU~03}KV&^%4Fs(vxGpr)!vqqw(b;4U4w)izdojRWAylr3spkPpRA!=3Jr3 z<;RM{S^y0mcB|st?~DEWslR+9n71$PgiQBz_tA1r-B)V)tKO7lK}^=5?jpwG;eG6#rT2{3d=KxK9i7ZA{MNWCZIW!mo%Nnj|> z&hB%AOI@s)63!{~v|EB$T^i?h)65k21pOHG1=8q~$erD>oq%>a*fw_qqdJ(?G8R z3(I|XoLg@TeN&wF65wQ6G7g5FJB>h_BdyJimyFD20??G2rS=lRsH2tg1Rl?CYLm+J zShE~Wy5@;>=39rc3@XPnOj(-VPhdmSCSA=`b>M0f<5#-Qz%21>RjkYQPMH5lpBDZ? z-SlrkuIv?YwQWD3u{n7}LD&JTM?Z65F#Sq%>K6x*@=biS-p!tlo1HjK4F5_fg{`WJ z3Afcoa2ZV*>k-{s=|vj9yPqHHT6q6HdXGJ@4)9`io-P+G%V6*>ON{Z+9idvF-@-n4 zSpaCp%m0ko@*HXTsx&cz<_RzKww`!h9j_1Q(RVu3QCh(qnL4g0JDHqlPj}kT$DokL z5(~&@;#KJmBWf9m3CrpJW`5mwlrbMSw?X={VCAsX3Xg0}ielc*&28EMYt#m;4Uy{W z@V3CTI>F7-rnn5uW}`G@D7YtBthp;8Ym9qEADdo+bZ+Hi5=)*7L@Q2AIfiW_N!HW{ zW$DY=%j~AcaxE4XI#F>=A%3)b=tFQ!Ohv17lNSTUijZBjpI80Cv+*bQY|JhVWcU4U zn1AHH5XZmTdov)A?`P_&uKss7-ux+~n}_SHug3bDDLA*bUkygQs;AB0p?fqh&fiWR zkgL?jJ1j@Zd?N0f8GSui>W0qxawz#$lmpSBuF%CA<+Nqc0^nV6$&y(d%UuI*ewRqq zM`sEZJn>8K{=H%s;S2u8nk&CP-F14Qq$K0GbWy-%h%up^|C%v1S9R0jZX6=MY_+e! zu}ZJHX;LRoERW*PLqmV;V(HLpBw?|?G|V(Bw37M6@*ts{Q;;vRDk=q24vdkO2}F zHC`12;I895P!}O{Wz=PtOP)~5wlUApZX}%=2#Wb|W!FVOyn~L$r)odq9YIKtKFE1c zEscFl6{RU>2dA*_rZH||Jq2L~i4tu%3G_3xqXT!@PO_oihWi;&<_JvB z*J1FhSgUT7-4kM${6TjT4U*ekbn_v_fr$|520im1iFlNJ(@wjgxEY<@Q9JEKMi4~-;JRR+WbbPAM=e#ovTcc$$JCc255#)6yBPz zkS3YMoaSm5Zm_N8<$=g_VS771NRtWlx)`H@9-Quk? z@vVXK3(`jfsY~9yWjdWxl&hkEIO0Qs*q(*Odd8)`*B{eFEPS|*BK_oJ+inseR~V|z+4AfN_zjZg8EytG{>xJJg>b2v|?splBON@E<%fK zv#4umR{bKMcF*{DkyC1KWov1Va|Y|#{-kh{k`tZ-{uu33udRr`j~_ZWs`|MhR;OK* z6@0qkn@H0X3))n)N#HudSCe~aLgDr5O7+ z*k8B>+C<-X7bjFd0JX@#M)2twTtQ0{u60s3{NkE|{P=XzGsaH*ElAFL@LOayj0f+E z4;|6-!{1$jXz1Ym=H5VM4kMo`{OnZ&kL*Hh9%yH$hq0 zW9_M->j&vPF5mUp#HO_Zka`ABG5lj{uyFPOtvc=LSqwg6XFL(DLD^POGkQFreB!KY z+MRW{T&P05MG^9%;3`KFz|&W>jkgsE2oaouEGMbdl!SQw%Czx#N-}-%Ro*Y&+-p{M zcOh)R+3f*ZtqARG|77MO_fB%O$_|3k<4>75K{`aI+npqXnF8HOcgiY*l^KUywIACT zjUe)qaz_5(hr9iXS}biSWfBKV@m6JTA#>XunP^oEpxv09ApFSw z_rYqyqUzivvOBr`OEL6vt~A&kle9c0`IS~gv0Ww19*aCcww{= zKl~CHU|N{hVIN(+|N8y2svgs>PTjcsC75>_!@b+X8jUQY%jitkrJTD6^UD{XDN{MDZ*-A=3J{P0%+mcEl(*Uas^20mlP zaZ$VNwr8K-@H~I^^-ABEcZPf8EUW&b1!=ykB*|U(6U;}xo6lYC#=yuZk*~g*v{l~| zokU%JPdA72j?!TkN4p+zM{V_?1iC5Ikuf*A&(>`$O*6{p!w?u@v?bdxR`TM}hMeF8 z1oKi8PU^(;NJfJ1)x?9J-Plf)CvrRJNiM$s@F4)u(NbB=BZb_uzEQ1+LllP?mtXD2~-dvt`tdn#v=(TqUe;xYI#GU%DsH0K2g@#on{S zv0urZE6SLNKUK~(A9V(n9Ny9BG= zlYG`FdEE3ZIrv!1Q3e1EmDr+&zqnXXEd$nrl>Fu*N$TH+0ek_q41fHL`=90rKXR0u z_L(tO9k-winb8M3y-Tg9G8&s!L~VFRTpX4nLOwi7ZI$axj6`Qu@QTI7)sj#?a&1p# zs`)_W>jYA_i9^0pc3CBHG_akM#dMPkpV^DX?H z-^;x&#ix~D?-#IND;V7!YQ9)$)lZbxluAmdz2Po~_oMY|i;{3q<6{fqFwHieF)K5S z6P4b}FD)2xxt?DiQPFeUYEfx5^sM%UHn6310W=1aGDP#Q|0w1D83YzaULcX1f@F3c zieY{2eC4M@=Y-{Ceey_Nx7B~~#nNUI@zbE2*Z^iN3No4*yb_TnSJk4fvLm4#7b!Ve z=9B@Zfn=?JL;t0ITU{ql_fW5B3g4k?L3srw2$^4Rk4wG;YY%gf&dwaU)le8S3%cje zxbbom#L17NSJpNMDdD(O*IT5vMJx}Z7h7T^DKSenF&nfqIHg^!fDzR+4oWk0eJPps zv(ssT80yFOZ|5@BC*@@qPBF3VNBUPp{luSuq6b5=KpmwaQgGyYew=q%dLPIF_q=!e*!{H5ViRFeHRo9RinrL!PCIUO@YnTV*Eah2UI=Rvx&O2)pg&Oa5sOIxa@ zMyVJ_2`O)!uBbV_H_5q5eG6^Z7Eu%Y>w&rXA6tTSaO$;CkB^SfW!6XUZ4km8mA?b> zMO=I?8@8p$XjfrT63J-H+PvWQWG@_kW+#zwL~XIUowH#duPuLg+v4sZ5ems(R%8op z*XI+@O6qM;yLO;6-ry!^&}=pqzD6;ftzkalx}LHK?evP&R*a!u$Jg5O z?v(Uo*G`8xCPb>(-nCrHw0LrKU}RaTr=Nc@e*1dME9lpv;UivOOOGylmvD{JH#ysh z8&*&ceZ;9^z`U6R%O4wJ^d)30(u>P6tiCOcz5J`cSHJ55!uEEai*x&CN6Lp~tW1LP zd69L25?BbZ8vbjk0APpQ$U!L}@wz(ZT!ec3^*@o`%&gA~wM+;}6pS2BrUk|eO3$a` zg7Lu`rkNVkv=ZE8bc*0Sp_u(fs%Sz~b!ePnyX6cc}NBr@6^Wyg6KViw&~k@FQ%&gb) z#K-cuTf?M~$e_{tZ_2JE=QV_H66Bo;U7U&OeRar`gC2*vXEb31ubC~_rRJErfU&WA zKkyB|d+HylcKhzE*eSZ!ykNLRKF@Xoxey2uOL&6}y>1(?YPNz7n$bN0?Ut(O3Sr_Qr-b_n;_IA7yuE%r+@$}MGSvPdevX`E;$)pQz){u&Z4<5xMWwB)PSh_OP-C2_TJPrpQNX? z$95JdZoE*ZawyP#lMG`?4(WhG-w_IB+7K~z5_EZl$P7;nT)2VhZf(lystYZz~ zT$g?0y-hruVge@X4gz`V4#z~4*{#j8R!^Lg3RMnGjFjfASJ!1DefsziF@H^9YD%C(cmTDce~h%Cw$!5u5@Zh8ROWK$yVx zm-#54QA^Oy(D&0?zw`vuZMC2Xd_rOCt?l*ZdB8T%h2vDks)N)~vl~GKB&9H=3)(Fn z?hPMjer%gFmxsv;E!oS` z98y_^0lZ^S%dD*m4Zkwg5C+g8LHtUa)QQ?dIF3RFu?z2=<%zetOcExSoy8eRPJmZ! zk0++3&j=%>1+Lhqe3ubObHGV#A(s7_(5fx&5D4eyTba4!_xnxikgq@sqdWB%9g_`r zT};l{Mg7ChG%E+($iZK2%>KCz`Nnc;`Oz2Q1QE8Ym!BU;-^d)`*Bc#`dyby*{95E} z#S=ajwM#MV~#0OiQdTfj_VnJ89j@6jZQcCZQ@@ zv3|LHr%74*+YMHv$-ZN~-F$8K@25#eD)a-R+gD>>7Cb79bjmy;r%S-kt!wZvDw>7* zjTn^v(aN6lonE^*o1m@YUo-5!wvgRYEy+84<%jH*{*rcLKzf5HlBy}xM=#p{_^Va! ziw2hDnFm&o8#m4+G)sI2edb?^Z4|Ek$ENYmv-&YrwO=!tHKX#@@U46^tJ%o^7oXLYnc|-24>uJiVI;v+1{6(fN=;9G8E`*Jm6^IiG zp%x_P4CT%<tH`Q2`-}y;q2HF(_nI$cdwpBp_-n|a@Oq>C$%Di-OgHp@{Er-6c zqS~zyT%bo#!?78|$jk0%ukL?q;Lsk^{N^AL*0+6xW6@i`c!5pOF)UT|yLKDX(Qkwc zd>^s&O@Gr$G1O9{)gE{`vr&Z{lS?B5W>Kvd*lu&&!rf{^5We`j=*<3!dUf_;KziD8 zKqXLV5I0p=6;EX!e2He90fvmB?S@XKs?{7H{I zC5Ej8{4-q}Kj^Pz<-i>k*7G%_&TI;4WA4^;(Sm6!`JnT2-{)2mTb_(h+Blvx7+xTi zlYVCQ6jUEd%j0ed3!yFtwkGiSZ^extzqxk5 z>ofa+Jplp3Z3i;JY${M$73-{emf6~b*kwuf_Y?S+K~5_-wj@uq+gUn^C(tYSxrwQ1 z%A9rn1Rrz5)1)9_%UHCfai-mq#*9i6=Q-%nT++aaa#uA$N$HU155{F~QbnHK+c3os zG+&c|S&LB+(mq%L>J@$ax`PZz-8;cepbQYjgw*%|##}!T?`g60_L`l<$U8>Bh1Utt z!|Oh^-5UCTtij@B?BDnMb12CJyOV5kPBN7i(l?L88XFS=&^MM^!$Q0JrdA#U8bpv)=b!#c+yirJG^!%i%mm}bNi@3%4nt*1#?8M3 zbE&Nn@rg{pPgos($DNT&PD`HPjPu+q58o#`tk@-_#euZ*80$n4D&F5^WqKjCRw4XJJZ;&+Bwb44i#v(St~?zJ$YMNxh;nt7;ZCTrySvsShdN{yx0^d}$m{J{r0 zP3e7ek96|%+LGl%);ls)C)5o(JzR9mbvrbs9GOb|ciqs))?$vB(4Dts8_8^GcSH0) zyp*IqdN#!oBRPzl`Gc)OU7XO^eVAMEQgHU+s9i)a)> z|8!KAN-ep3vW`RIzmCWQgHX#->!wJb@cnK>KG5X0z=H5F^;S-JDVSvx0mH;;1&CWW zg_p(o^rEiB(%HCxX0aIUpA)Iz*Bf#UcVe9Q*+RgQ@>&+o!4h7j#)O>?n|lOyL8jq; zh&K1k0rq)BpurV3lCjXV;vR@Cs+yW_qI?={z8!@b)Q}T8>>`U3Z#UQO$uAz`tydQAKWWyIX^U{TdEy~9XTcfuX|(>XzL^6z5xQC4B#==Wix>mbj4A%a=enC$gQmQ-e4>o;1CV9Q(J-G)$ zIHecT!9gLZ#$O2zt>x@wFf^|A%V7GYSF_kvWpqlRqDVg4!g&t$u3lkziHSIR`S~wiJ}9&tGze-1DdS&& zd@}`Gq5W6xrof4%icTyAzY(ppJJJ$vL}`Pb)q4~>rjn#i<+Wn< zfn!`c!<1NODz%qayf8+b2$7|;??x=B5KYV!R!Up^>6irz(gEQ-P{s2<-1 zK4VGhS${K(Svj&9_E!@12IrGZhXfu#-JhB|t9W37CUr25;hnNE-9MI=9`D%@EtVQn zIM|)FE@%sB70t`dm&Vloq8kUewm-c$*RxKwl_ruL9UPf6Az0xUhW-?;2YP4XFTUsC zWCNlodI4Yq=Q zqgqU(28?^3fYAFN>}sWnPShbePSL32IGd1%_P!6Iwz`ejj5xTOYruJnfG}+LqHH!4 zFX&?Imb;3T1gT}UGA}tGHuhqDX}ZEiM_Q+}d!?xN{W@6KQT;6ewAR_)v-O!>#qToz z(gRvxFRaLv-9(jm&5XG)vdKZ*S5+hs)>uI&Ia)V*oE&n?+_^Lgoj!rO z<6!RB2^&N_loTwDVc_C2H8OW$Rfr$CiS%25&nBcr=v+Dx`>T?>*IIC5UF!-fWXk+6 zUE=rrQXs|9^hGlmBH6~$x!|DLnX*LJJxaRV$cUb;Ro`^GhMrH`+E6+K6EQ?_<5qH$ zxda(ZSGaj*-%zVVZ!0T7SCXKneBA$db<4IUt(XGoLJWGcLoq026$KG0x5n^(})rwizsA`mT#OG1N%DGR0F=3ob-G`aN z*YTUn(QfkRTM@CjGDi(g_2>;&N=(!dwTs&npV%Il(p7nj^l<5|?rIWlbmkF$4Zs3E zVT!4&=s4oT+5mc%!UI6q`bVG33K8@HWY?mIEKCwasEhB#|KdGwHH`s6a?M-7x&4nR z`JXXAJUH@l6z_45fXt2oHeQg+D9a%F49|hD0`)%fjsgb&)svIA$J08LdRw*no zP?7Irqsa|pC<`G@$Vcwk;>$G+Krg(_st!{1PD8pmq+5n`sdu8go4rpbLf2f{_#kBWp7 zh2B`{1#$Y>aBcCa;;t^hF%*gK-=q&pLr3lno5d{`c;8O4N>jLNbk7ch0*oF+ak==# z)z?z)sQSg#S^t$;)D$Mcz4%+1|03;=)C`q9zvQ@p%!B}Lv1KJbp9irzXNQAM+LHx%87!#+N#dAv*f2(;Oc4t~EJIMnR4UeyI%Oy)$UBmUJJd^@CBf1UpAGJ#T%07}T( z3ajI0Q8|QM!nzB(oBC8_VaM>74%7v{QB%_5E`A@QrG&rKHiz7A5^>cy!+}zFp2XSq z#Dz#$t5G%`16!>xT_!KRhcfW{7;=G*>9<|(?**c$BW zW_8L8msZ}|V}oSJEi=1wxo3k#j&oN02cmdz^*uKTh0e%{UlFGxw}#G?j&ZjweZ6`nF`Muu$6knBO1Tb5QdW5btoGB zV4+qUS~1c?WrFdg0p()OC5Kgb;EeFi%xUn4=+XNUOQB>W?it%&z?`eNn29;Y2#>I~ zSp4YMqDhX_I~Cp^^+WBwbdJW9B^)vAp;2ucw2O8Gb_;B714r@}a-Z}Lx~IGAj2~UO z;m!y_W#jS~#_CA0)5i|x^DSvNqc5Tw==$3J)MaO-k!YNM+_;stPin$nj+eH6A9>dH6LY9!`3L6P}5=yb6;ueK%PKk@TxR4NY+O08{?h? zr3>5HND9(~G%P}V6c#YTdZ6K0psbuExYWxWZQE@r@V9eG8(4fOUsd>jj+wVKI1p>I zu6&ECkZL}E`~eV$ypTsOrlFArgVXD<3N1pA^)B5s%`xEj4&yFWUj0jz<4-VRkI~oe zf(&(Clsaec#5>6EAROT-h=>{K2dl!xYt_o?XpyI1GP;!a335^e(=Ba_{#=uF`1~d* zC&3-J>^Rn)#8lf+$$qMXyZ~j9&)j5fbP}1>Gx!mll}#?PEosA+ESmk4nLV?57B#AF zEQE7vekYy+_y_Y+Po~3|oB@OE=|fsEhyXp^EO4qgs8@q6V#AB#WX9TC@{~@t2X+DB zLH}7L+#=zE-x>r{J+|LVK$S;SGRB>kr=sA{GWCWMHnghWHG;NuBXF0J+%XXu(;_uQyVBtgWrR zZ|36sv*$&QQ;6|0wi)qlBYlt++GB^gRT1_L--vJQ2uKuXoK2 zs^?1j73=vdVr2Hnr;|PobQ-S+oefv(oQ z8+ufeT_KIVhDdgE&qTUC{N7+!K)iQhm=h-%)KtEWdzK@6wQhx_iMv00Z9D9l_)Ak4 z7nu(YeBZWSRN@X7+3MMkC9J?_IVt}fs zMs?$lYtZ=Y4Fnd(_Kk^#@?(&sR2wKB4~Ent;tHD*bn z)wiBpZcL&8O(5%GPxuU)A!^V4=&~Cp#C%2(B>ugZY1DH$KoE3HHWr3>*^*I1?x+3) z7B9uSaTx&0DIOmIr~Xdmj$OU}R(93x>rE;xCzzjHdAbuXq9Jz8P1?C zCI?HRVgHn_fqx5H!d?K_r9S~fqC+Iqx6tp8A^TcLz|*cs7e>-Tr?5d7ZtF~U zV6_>7s|mK^*<)B)Si34#Xm7Dz6{V!lHEkDf)mv@I6eBj+M9z>W@8$OKZeW7a?mEB+ ztLuie(5PuP>z%TF(xzXW)XZbde(*PY0yy!@fc@=}mS4U{gJn(P;rSNPzDzN0D`mu5M}qn-d}4RV$S)=JyJi%i*VqKUTB#}^u3VmdZsW2N z36C&CBZA)G_049x&q`|sy;EyXZMui5sl|t|n|e9QUu7zo_^yy6V#Kk(`65VfVt|p= zXt`4THUNcLQU48Ckbj2pO0divMT-=#%WGrf0hHsg<{ip@hisoVcFVVVSxXizf|+ii zF$;ABjs#COd!24eLjO%G<5+j0VTFCJu<0HS=J=>1nmuYZJ@9+^z7zgc_)1ML3EI3L z{)#8a7E-QJ9Uq8jbV$$)NVcnoKQm}wEh1t4)GL~di~P6B=>}7RaF^{GHUjF=4wt#!DnnFc$|WyT zd*{WXO5md>&o$=QI`5yMK64Ou`KYIT>a-a0K8xN+EaN4@6?FudJOoe+zP#R{;R&;s z>|6iqY%{fC#m1-7OaRF{{VE5s&iP@YFUTvAG|&xRda5---nO@78#Rkj{XyPKR8+s=SX;3 zCrOhUGVS%#YnXFJ{2HJp`~on)1Fiu_0^>E~=LzBLIAOE_?lxM#SqvQX@*v#G8`$0W zI$Ntd3w?%?BcH+U0@6AWaLMds@IHP|o+~FEkvXe~31b)CBxx;h5aru$K_ORZzBIFl z=Y!<{E8`qo884V1*V_e!j6$y0M9wh83U$R??q~_P`Oc0;yf}6dAm2UKC>4*Fc$?`U zlI+p97+Bz3Z_C~HG^Z;y)z%s4*M#aiX%?Gf)VF@9{k(a8hl2q@GBwS4fAzn&1*QSR zwe9z3$=}=ME#oAkDA7mjqQ#(Df^}p-y=zVI&>Z9|B6qies0r{TpL~-UZ3LHd*fz6F zU%WkGrOYz02Sj8^H`{=B$;C+Emtg&kUj8TY&}{65Ee|D&wsr?L({|G+;bOpMMOdz5 z?Y34gav&v2?P60LG;%2gn~%V=nFWmNP$D9WV?J41hvCk1AFY7F5K!-g4P#rTn%g2U zvgcoVSnR$=YR&!eTbK&<6EctA4NcW@?4%lH;=@wyl__ zY|ly)eICSDvi*wC1FD(pQh=MShu*B`-`4BDs6zH7c#b+t4_QQ7o>1zdma%un(1(!``HbK!@-v{(`voyG&TY>GtUoHV>fj`y2$r1Pi80ugjLM5E>L4S zw(doBUoX)8-(0y>=B%Y-R^gZ^mJh@8Nw#t4GW3?0jWn~+o_E0K*0ZnekhYq>Y3p8c zv%&E7-e&Hdxro|O#jV&`g5qKzfsc{+jzZsF8PQn|h{k;0ncChjy5sslY2ATa=Y=?% zwX49$SOw&cFfc>6IVVEge);~^zdq4dKftOWyc@-fC7(F)V2)O3?xTj2-EyV9iGiuDZ@`SpXV%) z7vYfdNMP?x+ObBXea_L`4RQ6`f}VYVVR|Wh`aH8lrKZ_t;)RjL@6basYVPU1oHU~* z#ijPo>HdSazKpuVC%OQv#={B-My9H$zKWi#4XJov)K;_Qf!*W}>AlPM!#se`r9uFx zj>f+`ekl2Gxe5HT!wkD7mR*TsZw%W6zHD1y3OxCXLt8~O!wRvhTp?WEiY_-Sz`vON znPi19d3RRY-pZ^3-u4A3OxZ1KEBT_BgeFc2pe!oIE1tZYtG{H6nocq|=r9u8gEYIZ z0F3|B?|OADkggy=e|~h!nr5w~3l+uFlScvrf-HO@8 zhv)^EG@%wa;l|*Bs7Kj-*_9~!rGMajs*lcL1Pn$$uKusM=QC)c1V3JXj4phnORftt zD09GrKYp4L!HzuiMSyU|>7viGs6RfMbU5P3`7Ofd>iqF*L;u?0uG{aSj~TA`8JxPc z8k`iNeUPT&Hp}hfW4#ElMt<4IH}Abt3qxfx#l!=Mx2pe-wfBx{YU{d(1tSJgA%KE{ zK8=~BieReubma*ylK&Hg^T-@ zpM$_H)Eh){4;59Sk57pUawp=5NFSjYH`Dlyq<(Y{C~PcrK>BvSGL?+AH*d7Or0(yg zu=lpCI>+lUdU=JWZHZKP`-7^e^vGT6SYGRxxR{ngT!(PlLzy;Z?e1?jfZ}5?7alaJ(X=Z3L?Y-AmqrqB0TFOh z4RVHjgqXb2CxBjLNGM__)x2a%xXxeu^m*DDLQw9=s)MxyPPm~fh?KROYbl~`aAcGp z7Wq#@e{$j_E^6}!oizJ_cp1`(KdKN*=Q^K7_x_ZZJ4gq0rhzV?HD33zj`gUPw@e?n z7ku?J9#PM&F1}I`;{x(EW$p|%f-~~w(Z3ZF*~!5+DMk4C=|9JVDCF^J0>~P{Ng^CL zE0-$ChVPMgJg(aYrSWfhveYp20FavEfBm&pstzf8PX7_+UMtCKKPh2g0k%auJ%nN7 zqaoIhM86IkY1x5(CLeRkoMQQaN*uQYw!CDAAfW9&vw~R!x(ge-_*|W*&F_{1Ds0fH zzXj+xPfo96|Ek%BzQ@l53e|&u%>?NaV&@0~Gf$8=UbEVMk_o+y5!IT0((QOid=p}Y zY<@QAsOHHr3zX3Z*J#>okxe9nRBN^YhS7r0HeFU!r&&@!`X1f$tmRG*4#=J#7{$3R z7A9pW8Lv(>(4C*c?>BCjlN4%YVt)_F{Rg00V7Bwc>0k7vzbYtDLb%m9%b^dqTg5vr z?WaK289N9(_qu`5d(|3p^*dac>4WsEwqE2;E9}cDr9&VDn1|0nKsI3jpbfhWJ8=Wp zt4Sc)lo`w(W1xL|E>5S9T{%dM(`PQS(SNW~BxHv@~*rK5PyKX+3N!jLfP%rAH? z)<6G}WeG6|W*sdm?u?Y?nEI#SYy);z7S1co#BIiY*|DD>6cEdNgQSrbz9&>%rYWG; zAXY$iK40axFfC|a#-qt(SEB#LV9kM`R-)g4V#XZHbG?+glEsyl3%NUVb+cSu+qaZV z=Iu|d*Y+fn|23yFognMi^cJ>1=N#E;t@=YaZ-LPV7(%e9-n*;e=9D_yB(E1j~82m4BMn6>MMT63-ub zkanH?GlE`ua0b$~ytt(7;|nA$=!8KZFb5dia{ys-zFvj5YLB#|3`Fvq#mZO>0*K^O zwe6%HyBMJElkR-1-~&F4AF|9U&1Z7$m5KA0I+UO3>QlM5iM)QQlex+GliI`bV^ftX zC-1eDlUHml?cM8YvmU-Y8Rzk&1sFk{<+7@%SKy9oxic;fc zg7)6J!l^)#MAg@Y4SkgjJ}Y--`9_sucGX%ssIPP=ba$CWz1lucfSv;~l6~g|=k-K( z8WaZ7`glBkS^NagxwZ~aB13K7C7DFkFfPU&T-Pv2 z5tcXDsk&ydQKfMEqw^$;044+Z;&$glWZv92@w<;vv){9lwZ5j~9WsEDO?prN%v?!w zck<1@=Z77BeMH@r$UlyOg`OiAJ15;dde_gaTHoODyFXe09CvRo4fb24q@@m)wmnvf z4Z3$Yr-pD3fty3g9>iFxT)u_)vuiQn-hZ4MWmU*&3Ga>Bg>1;6&kE?q9WSvPBfJy)D_ z8k8+J#=_FRSf6R);UbQlYPmy4nk1lpc? zwT6+6_Cbim9foG|HZ23)oq!DAU4~k>zI~}TX0|V=vbY|b)}m~6##QLXYt%EhJB2M- zmUr7;J{i4iz3Qk#m;1!$&*h2+;{!Ev8I^zGG(}KK7A*PrXzf}D#VM`nk3nWx zk}J)c8#XESe|^?>Juu=l0rxK*pVfVY1d_SHF`*Zz99rKpZqDR(|M>ucM-AGE+v36g zqu2|9)_5x(&gZoC;pwtHn}BVi4;l{Bo~5eCX;i)tnoZFD)UCi!%U@f%v867A4_Lgq zrA3eGhIeZp2_3h&QYRsVP5~jb9N^0_jF$_e>gtXcxB%b|b_qhQX&_sgi`Vad=O1!ZW}y{d*_5GH40JJ?UvHFZAwDgOTWLS)x3Z>I5O=<=kzF0D=mL7Y+~pmC zOYu>VX>%@|-(j@Fx?+L)ey-Qu;4j>T+?u5JL6{mf*T?k3yCA&))Fev6zI zi?tTsty2+G1^2K2-icGsffBzijmuSFF7)>#{yqyGG3zV$^rQjg5dSNJi(eDAlvYeN z2NhE!EW^Px!9o z${qrF+h}-0eF7t7SUU|QvbtD-d?nsiRj8~9(>N)4JT2_KA)2J-*? zoBh_-v3mp4^T75a8;?5-IZs~^+#Y#h2`ZYIaEX^+uWO8!Q07zS=vo%057>_Ffhyoa z#cBFnpuSqDUS`_P*bk&BEGO!N^8vh_Z>04*b$WzfKNkcYZXT~KeF%lzhjn}n-TYz& zo>VPN^!tgq~u3uK4Cc+WTVnJRsSv=t8Jrn@gDn zPiI7fmf6<)(U&y8PAbe4G_6O+HO-S1NfH;E54^Pco%$`ck#)C(9;TZ%*%D`pC874d zPWy_|jq$m*!hGq1_Z!$^*H2-SVq6*%6=JP`TvrxYUy?86 z2+ESC^*yJl%8+!H`vnO8(@TMLEGYkF?7S2`T&^PR2WCWWvOqOydQ|Nfqj2Q;Nw@X{ zki8EvZxpC37B+7H@Lh%PrPbX3J{-XkGrj2rw)}0wZRk}L-PON# zwU5vyXr$KkCrZB0k=G@)D{~*Qk=jYF@EF#j)Jf4A$QD@JibV|Yr7LH4Xn4c2*b<=V zj|;*C6$o&LYFU?VoaHuwIONj_xuA5=J4Fwnv-4GSi?)RaZ3pu1?bxDOo&Ir+$ZSWS z=-xG3P7BihN~_x25ZhTs5uI`1;pSkw4}!91i#B)!ZXkpfpxcu+b(*w^oaz>NU8;*a zSYCd(69T7q5+4H!=h;+5oe|95ocpcxE@}iz-pepG+K~7ctrR#{5I;G^Ciz-CoV6IA(G5?(Q2D^T&V|H!P=*%FjTKwpIF<8Jtt?zq+3qX zl46i6cHn6H%H!Lo!g1L;_3zS%DRtnrnrhM(qZ$oysJ2H&6Q2|hn8z1I#*OXgK`(eV zz7pjTfa+T(Hw-->a?;y0qrZcXb#X?}~s^CRA2 zC}4qL5h90J;i-u91hCr{^;!TaC#-=BA^Ntj(A!R; z61}j}B$YP@*}(4m!zkEoc%$Ch9S`ku2?+NY?uIkw-w-F^9<3^VJq~x8G!R|C%A7tm zHSKEH)X8?@VnSeT-=p*c)7vRd_p}y&6e^8z*vDlVm@981?&7KwYbQ+hFGrShSl-Sb z{$%ySJOZt_3j}(LZR$yQS4OxTq5+@&K!U#4wC&W)oiz?*%?iUXsNh)a!=!PQ4kGWv zkgd8v%-t1WWDB3dbZ$a)7nS`(s2nigNVznPn>4ELIo?cfR+ya&ThxWl(wxP8lNxdC z(d;?3&ShRby7_tGvchakZPVTUJZqEahrNz(s1@kmJ!j|^CSKQmliZAYceUbV+oI8D z`rP8H?z-R?kr(;g|D9*bOG**2*lAU%{*hSD0;+Yg%r6L`@I_j+W=I_aAW)&`%- ze@MU~RtkD-U(V=yvp#lwt5v}H1*4lnXY42sk@YTQ1x9?oSZ%-me*zs5diiXc{%Bj; z709Dm41Be9sw8IH9^cwmY?Yh%RRt^3eut((UzPk4C*fPf9vn#@Yy`xXxlk)O5yrEX z7YQ%L#P5&~S)fTlN&Ka&!Ve<4&lmxeOMIH6fZLz`SO zeh_&pS-nDFw)TJ;ElR}nBvv-xEp_I!KN-d&SIh(X#IIdTO+@b%8@oF(oL1GMkYm?N zkLVg-Rag-)Kd`sHRv`Ok!lG;S!sadscW<+r@#Tkl#gB;ObAvPJGvm*pAD{*LKG6H9 zx0C8nV!q!KNe)ARmRu1-@&i4I6KL+{dIb@(b?QAgzZ-NDNy8I$DX99_@Vp)Sy-MoW zo*>B-K7@+Wu(ZEc@7=k0a%Ri*p4-Q>bS3RX?#WGw)FM;Ro->B>H0YdqHonPxg4458 zE@MSwu4NuKK+_1j0?oSD9C6$&UZvCa2O!VSPsu3jTXO@8wEzS_g1qLVOP|vwo;LQw zwLi+gywDmu%_;R};<2)D@7oIo#kos*+7EGWJ>2*DzTXb&x8|*UcU|MDKUEi|iiC<4 z`_o}5g5N%I*poUe;+FYTQ}z=I@wGa$#gaF*^EDJr0dJ~ec%1c9~Ze<26jC0SKi*EJ!$B4c@)Qp;1jCwYdN2*+66Lcd6 zGim+F+}CyF)#SB~LQ+eeS&B-5X|+U<&iTIm+~$xZ;T!bX`bwGvd2K)2O1ZhOe7AY@ z;vN4JbL*KEKZ|IJRusz-h#{2WTr6bGvHNF4^_pX<=9X1=q4MfCg&sn2K7Z2BcA9U$ zrrqzk@%ifY%&dCFTSI1TcH%9U17fXLMScp0szTpum6Ung4(l(j#cN!o7zuL~>QYnC z6lrr0$P^j~E8MD50i}`;8+ek^{}9+?%%DM-T(pR2t|Y_m&{o-Wp7AL`2v8*E1*#o_ zKte+sa@Ukr*zIeyW4Fd)bPz-EfmjQ60R%NIU`is3K~SPSFxQcUb)USZ``MDoT-Z_% z2>lKi0&q96K$-^Nqj}Mw2>8;(q6|T1?75Mm+k55Nz^LVS--IB6MvwIb`a;n+3#VgY zpMcq%{4dJZE%6ij#HJ?|dfY4-9{#Xy)TgN21$!u-Z?j?o<||a|S2@{Z_a~Q!5)r{R zprGX4YU&}1Z#_k&rH5zD_*3Na0I$lu-ZqbSHID=lxhvr4>&_PwAxj)B1?hbEdO|+4 zzJn^x)$A;H$pmBM@^(l!d~&HTE#->cGJF&0`V90MY3MVFuQ6efV+WGe5+9_(UrK4G zsy`~+9`j!SPQ+^-3?3kc7!_7{+S1_mArP1>^#kb65|1{lNuB_#7*Dfm#a5%5EnqCr z5BAJi`!N1G#tKr;ZHazEtdKCvt9JU;obc7i;r&`X-%|otTf7Y-(9Khf5R-=(v(Sn} z*A9}a+(k2tu~oSh$dBiAys4czq&>(FSdV~u)vP}W${1kJ!A?K!v2}5gNfnyDC_H~H zouW^Vj?r`p#5A!*@8W@n9iH8`k}Ye2UeNcwVho%K?G`HU=bzhcM3&5FHzDVVy+y`l z+=JT!xz|aR*@&~v#xx%ul1y537N#yCFNQ^d5}y`Y8Vtm*!LZ z;XQ`DuuA!oJN>G{drQ?PKbKr>3A=5gn)q63pzU&3f?GuC56VMKSyyw&do9!e+e+~j z0}pAcMDtd11oM@-@al$jdLx04nYV=AWVY;^#$LQA;4~XvGF+@4>*$(>L?tiY=s9B-vpTdzFBn*vQ!V&4M;oo2}w;sfiO%7o*X3Zlh#_@(LNJ%n?* z)L8pIXoXsvMzGWDNSru3L~BA4X`oOj?srq^8BRs*5~Pqh#KnWERh3oD#&MKyK39&W zeR#JIv#${u&mGkimhJvQE8n7QpSEere}A6jQ$5vYCU9}PT&Xbir@gVOqVe^Wsr@ZOA zHu%H4-i0Gowr%42kg#)O4z%MA@rd>heo6bvhD_?7LVJVPi~6cHQSz3o3bvZNS-o}i ziEa1J`Kuyb>Uq@`08rX5R3aTf?tjf2Bk5&ja_r1D|Z@ zq{W8zl9XwYifY9e;}#ZlXn3wzIb@F*zq8%nmBK@nYpP)wM;5{NxWidX2 z>i7Xk8dTpHivoK98I0IR>}#kSN8h)Uue>|Xfu5{5?i2lv zc#|XzTF8rAfk*Fj!0eel@k=FzE4xXJgAn|Iy94MTJxnt z`}RB9yj&lPlUt`9ki?T*w1&)#h%wvM<;_sY>f39|+wazORUYR0v6L|qw>}7A6CiZ0 zdN|PpbMJ_!Xu0*UJ zI?K^NJQ(7ps_K|_pxzsN|9&>=oaOGuLd%{A?bC|TfIU60G!NG@m5uxQnL%{cPK}?9 zInD+p-HCc{z|ce2bJ{*;j%9dbFw~{r?$V;~{=wu>#U3dhk$Ox8Dj;d*$S#`WdkQ1D zC>ir4g`9n)U&*-^g|garW)>cK1YYkS3BC;AFwnkx!@A&z%Q)m$c+;-gV=bS8~>>U~dWtLo}CG{KDWO}ilwexDzy|ev`axsnf z6m`T?7bvSrn#rG~zb{q`)Hkc$+nTp!9((QA%f{bV-a|W0vn2fiB2AXm@vbsn-9$LU z=+(=dyFa>Lwd3M&SW-^Y)4gw(5acS7ca$vEqYmg|OBH($bL7dJI{Z~n zZ`=_aMdeT^WKn9B*7oI*H767o_E^#Ol>LOvmOi^(`t&U*L7IvY#sbj;F~iQ|kP*PG zDZFa;$*g3DW(*CRZX-k%Zs*GyuBmy@+Kd(+> zbY-gDdHU^Wayr^_zm(G7j@&R_Ej$1Doz6SMrvdI#??N@g?<<7hn6QzhdtSYmmS*9C z5EivV<^VQWiC2$?VogzIrhA$soylCEv|mwcSjWm8!>&6*5xHHM>Zd*HC@Gf;Zu`{W z)Y(}TD`06AZ8Qsn^S)NFqD%G}~?_UtTxcPd+hq4>YQAwE$HRN7vG>R3{5hffLU0+o^ z>hNRqo4g<^nTW^4MRV|Vi23@iziNHQ@)pq=av7fdG_^DR36FR|!B5A)Z>Gs##&%sk zYclP@t~>@-Mq#S`2T5yu4T>>0X(=QAV@tyq!D=Cm!jun4a!7=cg)jx{P7nFG$3Off zMflq*^WT}Tt#Wer1P5z_`?q`!J_K>2em4Htb^K1GL}A1-%~~Yk4LJis%{$V6|5@zw z7_Y14Ji$`<;rrkJT^9v4WD-%ekpJU7;75~q0{Tv%ALsE3_P=VMplLxkVEKK|ct+W}Fyj)x)RKcL6o=*RIF zkMVOH?+12#7{3$Zaj_E)cR2n%yoyASFOA=bB96Zo z=&u4Oxyz=W`q);4FEkpnDY8tK^hTtJ{qqBMW&Gec=skJ&@4FTyCQf~9nQ!NlmXB>w zP0d{w`}3@SUw7&Y>f^aJ{Tsx!ThqAvlYj57yE$2Xe@Tn%DaS|u z;Pk6t3}djbg8pN9Z;f19;DEy;=$ZMlz{g#qI6SokR*fum~t27zg1<_=af`RD^1~-;9v7w ziMYP1B$-bo;?H;f{t^BTuM=EM?8{==5nF;L*Tny-ErZ#LCqV231#%v*A^tTPy&73E zpZfIvV9DPdJZ0SUE016C_1nRsE+oEyj&OXS|A*HQ`<7pe z{(oEL;QKOQ-kV?k|M(zL0QqWl{FxnpkMw`Mz;k}y6aC+V!uT4XkUh8mj|;;8$V(F6 zW%=())PG+hGgyUp&i%(l{O6YpVAZ6X{dp$;+r{}khuF*gO@a7d#_a#kq4GK({~y=% z9RW>!oY#m07S7X}hOTnGZ?CC61}J>wBM;!kF~q!}+%wyiVm|luizSX?Y zcNq6PGd<1bMZr7OK?Vh%caPuxESuu~(OOEm+~j`@x2Z89{QFDjKC4dfXGfVbgnnIu170^pdzR{3aw-g|7(PXL8Yx^#0Q?M9o}ygG0-xUO3^ z;Ig$C&E|Xl_$CS#mveXlvwPlg+?0Wkh4ne_E$@9%_Ox}XY+Cw6*UC;wlkl}eS=L8B% zoqewA?j0K7k}Z0MF$p58y(^ z^)S|D_A}RG3agI*%aWT_Ln~ECAj#`wa%-k>V@N{o%1uzkd`e^WHBKA!`CHWi7dxQ` z1y+M#>*ZQB@&Jsdeqy#Lka7EDE9vIGP>hnJZEK<8sgn>*JOGs5&KS_Mf{l>($hR01 z+*z)yYsCYtctV?DZNQ3!{%PI#7A<{&GU*|l?-l5lg2+Lx5Fns&0o-Gr!MZF|sC7D1 zXg26(=Y=|)$y^Y!@1N7@IkZy&Y{auBA8G%!`tdm1JCW^?lUrrAIVHejG9UCE?edg- zj(jh^FZESB-y)(EkT$u*w}7>HGa{J?{El}NZ65hRSURUh*C9deR4Q(-7R8_eFGLp?oguu*c%Wrm|?5uMdNj8%p0v19{ftQ+)(^*=+ z@4K4|a|m+sm#d+0dZ*XEh5^Kg4v|^!>-psQ#&~dk4b`556HBHJKTCb%Th;gPp7YWH zCfJrJf1luy@hcr7$4BB6IL^2+(ro{Vxjx(^W+)odJ{?*rUJli}U&0t>)@TpYbZ zhcoZEC8;Z(yFW%%Zo@-G@E$h|ZNd!e?*0gymloL#{)LH_F1fc!WPV=+p0wt8<%fmB z=@>3XLpCGmnz%L|)jZg)6G1>s<_|!#=OX+`k<-G{NXz^P|5@(x{pALEKg#LPM2u1b*FRo5~s1G49G$we_^jLp$YjU)g}VUdq7 z`ok*v!|(m^bUN=`02Wk)3*y5D9__>){d8L00Ykd!Vsi1(an0`0SN*MDtp7c{IYGkR!0eRt zqQ+AW=*{E~k>A1#e;r|7-uUx}3@d+p+%Te(gqssE-t7z4B z>a8g))HpjL^+}r67TqpPHvnuL9Jv>dwy{SxZB7;+g^@p;vpB{Cy?KXLI~_aZczeO` zj@A)ph&MaU_X7UG0vLmpEPc25qZ_04i{UG!!2H^2)Q74MK&=a$WV1`4SWW|aWJzX1 z6OKd&1fKZ`Psn%hRwpiiI4 z?To$#rb>>Hbd1;FR$aZW8Y4F&FZZzIOs!_LVP`QgPwWL?$GkXSodFYsWT(E02Vy4a zAYMQw?z0w5rC`0P{{UedfR3JEkv;1S6?~Dzi_mu8=vo{MvJke&py}i;Na7jV)(+f-=uFq+Wl~3{4#22r$8LD zn#ma)E0oXB9OC=>D(d~XoV>?a7?TB{puIQO%`e*(HWc(fhng|u-t-B_!{e@|QOPe> zMtl--rG?9s6Vnf+`4{{pPP8_A;_9|<5y@RT-Kb1zcyDKbC%`Ea*(zP!ERv3W`7x8f z;P7Q=OBmaHkjHdWN2p=%;g52I-hN8WM|Xcc`wxzI%~gRqmC}C|0leMsP)sPGwa zy)*T>5-s3p2aDP6MM_Hvtx3|3vP0)rs9V=fWRH}?&Q0zc3GFJ+_ zLP{{h@=>e!NFomYZw2FshI{&LbR)99_=KaBmr=CtJJTXp}ZZYYAhD{Vs-Vs4!4arr22V z9`xKFrx)G=%^7;0z1GTe{&t;BLeCW+EbARoT=YA9W0#S|p7f^^eIl6!6jb4JDWn65f%1gOuO~LebC}t)fYa8!9fo(@1X8bTxV15Gqn6X#tE7t zE)+G+2RB2-J=G@G#1sqPu2E)Nd834E)0gJEq$BIntzps&xMFJk#@dcQQ7h0J9_TqS zb`#hO=WO84zU!|Z!BisijwOv+ z^wPNoIYHgy5(@ODkMNC#*p4F*Ja@(<(C*YLXl!;?Wy!b&?KJNsM>?tVF=#0wf_Zy_ z=65Vy_04k|_-8+6poQYGzqd6%QAxad^|| zpC?Ns`*A-wIr6%$IFz-y0pG;UowMJL^?35D*Q8TT+wm>FiR#dxwWzU2h0R^u6fYsa zKN(ZyOWGEew@K|pw&w;qr*G>-@#|MY=K2dVM#X{v$(r%*8o4ssycKrqpz&J9dYJW_ z)7&_+AF$123K*RjEMpX?Uu!tJ?67e^DCI3xU-u^Avcc-j?tmj}Waay3UR?u=<%g_^ zdw6nFF`%z3fSwHdr4c_0w!Z5H17rCir?avi8Ai0_=ewM3wH1QDx`jO&KHBpX7KWDN z?Fe|L{9dHx_ciLiZ2C%hnWmM;I(tv6Oi6meaOe;OD=RlS>z`!2ozJ7H>s`x*cq7+B zcV|v-1ZVGTH}A;ww)Gm~oVCf?Fr5L83v2&32BPNCY0N1{bALz?m0Yd!6lSrc}HI z!n*UqWPExi48E;azWWvrLvIcr9Sn2b_gu7(UCU&`e1{n`uSyH$)=l%tm!etVXKMj{ z*%+UHEZ`Fq;HcX>9>Y`*1e*ATm7PEkx`{V_RQ*L*um;u@i&?IK z5%N`SyUkB*BZ^uRJm<(9h;Y<_hxmi&EcHm`?H(;Ox!;ajtWRgvyN&!-5A15xbOTXI zt46>m2jrsvGOfwb%7ldD`B@curiSbuEI5u;Y9_`43D@ylV#`OlJXkuTpCe zH6G|5H115N@MihZX>sBFMetiOyy{9}e+o9CuS8XLB@)VSe~M3W`8y`WKk*N;PzDRQ zA}pIA!w6zRE^?Y|o2;G8ke@34QMbb?Tn8A_w~f0nMUCEhKn;6X?s4$6$<__Dl5pk2 zImeX(74JGjBp_OqGT&KwzQS^M^65gK_ za7D7vd%c@vE2h|0pngrU>LWUR3Wz*imoC;=(xQe>z^egXp9-F3#b9lnMRD#zlYbdYs$w?GnFZN}upLQ{jI-lFSXJ>Q}DK9-ILP zmU-Mc%-r4=#2u~1_{1HVOUz-dImAj)*U4i))3=lAhoc(tl4f-fX_;|8?;E3>WNpj) zLBjAM6Xmi^vW|M`v#Ui9%k1Eoobczbukcjy2Zc+I-P-7oM)v0TAptYAna-|)yh12) zMo`~iSv&~wlP#9XD55s2o`j^tLY0?uHx32mu~69UW1d-PAm9`Ho!1=?p6!+D3l? zde42^VsI?F!vn*&ksIIQVc^F2*493T$gc`BE_SHR!qG4T-cjt>WqmPd9IfCl4XVjj zgG(DsojK52Ldv0q%-tGpkI^x{O$sZJiE*ulC9gW%?TlDR`KAAjFD>)nQ&(2GE|z~C zN=C8_)x~a%+Hv;1T$58x2XxMBSEkkT7u=l1A#D3+bSA1rhd^mbE1)WeGWb0A;}Szy zE7=$Oxt2b%g)BDRG6a+7y!~7Pr;|}r2=83`dri+Mpy111&M zknOhw&4n}H73@Xs4wOKsQ*cwQ(Q`aYK+D?#l%9OA*nA@>4$>9hLt0%9uCP6CHS(RuLBNYjv=bey? z=(`3y{sQ5i_#Q+5QBF?yYw7-|X|p$g-%`=UR68Cae;H&2+(k-sg6DzM%k+yL{sRLh zZhs|Q{oTgRHqyT{D0W70x=5ivfBE<65=A_5CQU<=B>GxtM!Fr4o1<&miZ-ksxvP#> zs#$n%p1yK}4LXq&W%tT;nz0#PX>_VAHE@Mr_W($g2^VNhOro@>>y>+U25znhZ3cc8 zS)J-Xjdh!LNZld6Aw3K%*hwWSLtqKY&C7Vby~B@$-4j8ca1t=HM&F06swTbW43)ZF zn-l5t5ua>IEvCMfrHcs=bz>#s>*)Zz@Oi@9OVE*1`KTB6r9NbV=Q_opjFl(hPngt0 ze9`I(_4Jh_uwq(xloOvkJ5|gWw7*_K+eRJmnF}hssWyCO+MYqS*{h!ExVRo{^tj48 zB3Je6<4-o6;1E(W8V*Umg;!E^N@gh@pT7N{<6p%$VUd$g7QBshA>^SqQ;+vz1_ zoAHv6t8q(nu}N(BaH@6780!bUvaiJEx`gBacVuQDAeIp&Bv)rAgqEY(#Tdqu6DgZ1 zeN!HlSlxq=VPM&yolOUbdhM?QMR2KR>+03+J6VyTYt&*9FZP=K3FY=fCLda~7Ew+X zeMZY|LPgn4;`~tK>(e0j#ok~?OY5&aMtd9GcTM%({_ANhYe3z8 z<~+(z|NFt<@9z6HWQxV9#oVDYrCVy?OIxJcftzYxuyd3$?KHo^B}SyPzkoiy7#S&U zYBxK%zjNq~G{f|H?`3fAYl6OdpC36UZ9j^iE#YR~@{OYf^kY`iY{;IXzq3N?G|FRS zOQA!ZYm_0*s-=l((sa(um6u?9^cl}8Q*zBZSPjCN3_7r>67#D8B4wb{psuzARJWe! z6Fj#=Tj^JQ$9Gwz=Clfo62uIf9TfP1%U-n{XYe1L{AAfD*$543bqhfwbU^Z_LzM{B({BVY9O2i0;9x8vNn7pF2nZ zsQC9Wnr><3xC z3)L(5fV4dW6v&18jt-VQ=_6gNzK*{9q+RR;yKm%kK^0?7&FxS&7wy#d885j4Kcu&B z<~gO{0P#TVNpgySc8&XUk1=2x^~GEFQuVc&$7ID#1AU6i{!mo@xO3ZZBw*Mh!(nw% z>T_s+v5{%(N%J0|*f}rYHrW=$*Mw(_LQOE*p>3gKR^5}AV!oj2 z#?HBPsi^tr$$uph7gnUpP^nJ)68%Le1|AN_<@{ubr(xt5XIy!g^@9B_a(v!8&*RmT zs2p8nTXvUQ-xtoBKy!IO$}_38Fy1~*ug~q^n2m&VNdDnr}_QzzqI`j&cC8(QjOf$5{~C_K6JyjvY1Kz8kdraV)v&u!tnhHW1OVhVvZRnzmN1@js0Z4EdOi;G zS7W35h{{aVhM1tQk6jw)c(rv+=M&I2chjE+@&iz>E73UTwXp zpkyK{;u^=Ss`OiL>lluz;H|FguDRTy-*1h)ENXJ&m6LPsUvWM_|BY4V7*H z0=DGTYZPV&oKU6JNycJ2How}AkDdCiTiG0C*T?}A_m=IGc94Kdqu4^5neLD}uTqPQ zGa*UznJ|U2vYTo;&Maemq?c|oS4u_!Vm!?AQUy#Y!?Hhrl1*giUa#9)iv*;6@}1@x zBfo?FZzoC7d_jCN^?o3t&DC=G`N=2iTWF0Jz!wtc!QohORzWiaa*@r8GIq`fz&{+a zYHMGuxXQ~0yob1-x{~yzPyV%It}!i7_IYA9V|C%N|6aU-5)4)E8BB*k)eJ756HL$J z%O)R1O|xKGIQb5~$;ilbCg&%eI8WsUNNea;s{Ql>aDzD)4XMR05m002fW8H{q&+fg z<-Bw^|T!39_NB#JCPk-sn11^qg zSX~D#ycjI!QM1ENWrYg{H<&VEL@&v>Fhl5&ij#D4k%16c3+UpbWE9XDtyVWF2{=hW z8yvY}H)4tX5Of+zH@ZK}2(hfS0a!qdLDlYDhAzI#Z{%eizk#sMOn@Ixj?K51anKR4 zEm~UK-CpldkakBl=|5-3%2l=?LF)C%(K4hO)K8}?Vq8nQo`D2mdFYep8u?8c6QaJ3 zclQ~Da;Lk>r>6Amch<7+WSIE(4VCn*M*J1Z=|idC80nj8sQzpDN1cejS$S<#73w2S z{rY-7jirwq5@M{?Avx8={=307uN^fM7Aa0+&5X_*qouS_+%`TXR0Xoe|TtFo~9{hbu# zTJx<*fJI{E_+S|v3hFS)h14d|RXVZD1eaYcuD&YZjTu_#FTmcohJF+$>}TKSA5gyk z;(BEf*~?R6SCnYP-haOdT}DfbA(PG;2o6ny2d>=2ckl22B9v&qOkD?HauD&JdP_lC`v??-~0M77oS zwR(?Nq03fldg(mRsEjHRPN2g{_eauC`vdimITQW&h?%yepUMd~1+l*l&&*{AWiwSw z#GSvR8hoe>7z2G{EwX$tbT4QJdk274LGiRqC6-Y&=ml@nXal7)cQJ(>iE{p#aIDB7 z3R8nNs{qR=JakN@TaJ3dM7*AHqCtvEo3akbkiz2h$Rc2?etUAw4iEbpgg>5x*h{(_ z_}vP6>Y1hB+=`p;N>M|HT;jKTQmu-Y;gcA^V~+QS%iPB`HU$?<`n&c@YUg6m?#THS zqzR0Hm$9X+^8F@pF`;0iy`o6Q+F6nGOKO|Zh%W|w z$TF8wkiYw>=S^$D7bAuHnIk_F(*QADWD7BixYi=DfRV;fRF8id31xgF#M&pVX9b4klCT>} z>_(Ct@>|xk!<}jA?YF663)4a-79&@Jd=bFlZF)WUg$LPDq|J3ycn;Y&<3zwXF$MJE zHClM2B1iQlwn_DIW8tHZ-|k)6eQv@RvqA5+EQYZ%^GEJh#*4ee7z)g(MK%a@Av3~c z3HP5qm$L-i8ExE&FkO=yz7f;@S9!KpdoKAmPK{+(eHF3v{7j-SYu_?PxkoD-gX9-i z2(IHPhg_StnbLehO0+4YM|yfy_OBp|`Gc?Xn`O||j!$TRFCUiYCn{l;KbjwxEPyi# zY4AMOw)PUtq{Q*M2cQ03c_4&E=0|OHrkF#fCE^y_*qiYy9smaNoe5$H=;1eTagkBx z^|7Dr_OHP7qw<(q0VP?0CQc1PBGQ)az7a#ufVfb6O zac&`s+!HwjcyYtZlt^S{4%v&S?Qu}hdz>pJ$gi4C#X8#CN)(GvQ!q=uw01H9VT#wG zan%lN>mq-c=*}`e$x)~TZBG|zPUIAFZGj+IXaCBG zHKi-F;=5BRyP+kx4zE%Q9QOJ0qiboo4=uU{pVYsuB&y#Z^YJO%+4wwEw)K0tJt0@T zp?h;UCE?9KiG=zqVutsyO7uEt^LqKZWY}ZR#}`4Dfl;&w^7|$^Lk2NLHo|t6@3Tdb z3}pc%TdpFKV#x+#kN6?!Ccd4(N!NqQR8D%IDF-D-P=oF@`-E0d_tx@QiyRKUbjuiR z>hoO0akg`L#Sme1(!J?TTy< zevq0HXp|?2&@UxzBH$9;<*=>4Rf$9Um)2@;46{m1>#wMfEFG8z& z4Fzq(FRccPY6@D0As(MAhk44jQk%SKdT4QS{h@~RXh36vA9Af0V3*!Y-9g;a0Xf*% z;FHoX(YuEq2Q88gcLuzi;lHN&IXTgak~rPiXy7vN_cr)#m|V18>}+N6x5VIB`oQEJqM(U0Vv?1#ZXDr|=SPs#oJZVDeE~ z_PaDSv?g3@XII{R=ZuOt{r}qg@^`4$|9?ptgqcQ`FpMcnSwgmqeJq_UNgb33*=oub zMs@}xTO#`som41mq9*I0Y@GkD zV|$Kyj`^R|;!a2l^JwSHmbZD-$jGBD=fn}XOWSh)xPm0_J904}|(`T1_M5*93ZnWzvozEZnqeuD8o$I>nqNiQZaNbEiJ#$h~5o)icf4MQ;{tk&*^_`BAuIes2xiB`r`B>@$p z1nGfS&qXE0F3cg3oOMD7*;HQIGcuQZz7 z4vva=Yr>>Y?1kl&Hd^4~QdCPLSrW7f(G@y;qzX`&9FIv!9mO@iCsQy*6({#C0)3Gz zzEl7e@cJ0fB5XsTB=j)SffqIK82@>QFKPm~Ep59W%pD>rHS5@XZ50?K3obwJqS_Uya zsFh}O=`+xqloE8SgeWln@e$jO5pG*+i{6`*G^Y`Bgrt>inC{V$n?AGs3ffF>#S{x2 zc3XAMIMp#8$o}fD(;l?UEj$N?t=wu73f$!ZIv6~&>cTUSy=P0OCgWAk??}DfJE#z# z4hCwWZonDkqpyMpouEDttzt)JOfB)*#paEdIRW&X_xIIasb9#nsiFofDB*29Pp8af z)L41EISgAS)UpHNm^uF58tC-|gOn^*?}Or4jmJUg@$84XK{WRl@isew!D-Oe2p}zZ z;o2%}uK4h3Kr?C|2Nq_(O(R2-MbJix)$}P`MgFns#le4gVFl7m%mCh81w|Cal>zCF zQ^Ln-YZg0oSC#i?(>n``n=Jm-K~hq$8hyroTw*`&GYbzE;m^V`sMnR-r#3xEUkoK? zh6JNZT0-t&w4L?P4-tsZE^MU&S}_>JGBj3P=K>*E06nUy*}#pMBbX4*jv+rjsKThx zO2d>5f=bNio<$Lil(|VBBUQ6D=YL0rhh*i6BFW5O#vK7;93`Mp_QW$F!z zTbSGT*}s)@CJkq=_xViO!*feov(@Dht+dbGt4z2Z$bjU*Qlu1{@+R$`z0om!N&U;Z zRpSA0R=molfzi<63QjWYDGUq7x3xna;7g+KbrQ7>{sxuNk-5DfBXrQ*(%~+&qP(^jnaR4PBk4rm#l6QlUUyf&XvKauL)=7T zVqh7hsY<-Xr!kOGk|rxU)#chxHy~<#e)uXTt+QCu<87QPJzA1#pR;G%8aV^!&$78) z@<>ZCFio4k*Y<7nd_1qL2*-9Z?8q$)_6GMpXuIvn)>NZ?wTp*8?&}sp=PL(Ii7=Ovx@Yqxxxy(syjC-GA0VP+OIz4Jf~{q)T-{D z6L1*ZC7+f6bba!|(!^A1!SZ|WWb($^vxKcST1OwYz^Zb*x)d_@vYBm-Sz}d;M48wc z0U|quUlJWD#9z|p8;**ty4C9sZhmgXY)qA|vtum-4LG^GJgD`|k#(^Q@sWz|8*2~t zW}LN4yo$L@EON=I?$kE&>P(E+%o+SG`^|l8P|@1IjZkBX=+Qn}>fEfC4tc%V&j{NF ze5V7>?)}6^g?e>(r9`7T6Ye8Dl-dE5CUIzt<9eGXcL`mSU7be*p`lz`x{G?3M2W1~ zLz`2K!)tVeAco!bI+Gh7%PXARm>e6p@JZ`*B%cF-jt4CX!kpxr`0LycoO1$2D&YmU z!j-rchU&p+9P8{pU2c!aLu$Ze!kF6uw^s%bIsg zM7H+qrEFLMB!S!28H+zS3YE0o&yuWS?u1Wfz`P#*k3jE z&+=N64SD*2k=bieEF^FCocYBANVT#;iT?c=nY;rKB4HZaeaZlixj}AiE;8q`bYzwtR`!ffNH zdOJY7!^OWa;+3=2q5jY=TIGb6| zXMs!U+a;X4a?eAH=1+@N{C3?M%-9<~$4%n)e($I8=^i5bz*yt1c#n}TpsG+*hCJ6X z;O5>(hj)smEWfu$2UCMXeRtQQUrPV{O22=|2MG)!+{|`aCmE#@G|yS+#W;EzE2fbA z(kwnfx*+d>;NO_=6woMvNm~B8_YNbJ2i_jizB?4P1!z)CmVoOs_L@EY$j=Z&zrRqg zST{@d@Y<`h(_;z^DjBqq!^Swc;y8xsA;&cha5W>{Jjtd;xF!vt#VJKf@@PfQfB>BjprF`AQ$FD}#T6|SBNXqgxZ42@ z#B*11kL}BAvan>JR|O@RbpBSYxidjrHD@jpuc~m1KDeis#We(FW#J5Q zuBw6uh~alxG||>DHamI~xSRQTzACtph;F<1j*sasV== z?4v`DgJ-&$xHBRvpEijFfs-4>#A2s5YZbJ$G`yz<)LAWh4}j2LI^ez!9R}ISsCHM?Ep# zTvX_>lCK*28fR2vig60Ag??`}V>V)7sF>b-ty9^C4?Nl&E^b_}B2H0lywHkiE(zPKGBEb`)+A70`shV^Y zxfSRkxCb6Az+b{1KT-v3EVP#(X!tr3`4Xr%RoFd_vYbSbARhbtzWFEBq*T}j6{8zNPqc-2jLF0WD)w+LXlO9DAK=(Om+FR6^IMh#QWVLp z^`)c0Q1g4c1Lh0B%+!fxuxBhsLofM*MB1d`#z_4Dk}cke(gwuQFcEpg^w<}Wem2}^ zs0?j*C2S3k6tLxi@!0YfdtX=rBlcXl6UlZYcyvAxMDyJRiF_D7^dlA>?ZK#1 z)<%nLCccWbq>rG+HU&i93&JyF13vAlA+Na9_+As6Ng2c-7*X!#U+Di5yZuJKbtv4# z2KtWgsJ(@KtU}(7zY^(0OSC{jjw=l22YAUCqm!@|mZ;Dwqk!i2!DJvilfz%bdv1Rl z+GCP|`(U4gS3~d2`#&^U%;24i*^SKJ!-Sq2UT(A5E)xLg#XN_tW-W_@js`8uT*X)F zN1vh(PQ3YnG_~Ykn@C2OCQ<9PcKditZ=)AtN$(5>rP20>n3!*GohVs6qbCU|11B3S z*&Wjs^uon4PlDt*ZK|k@*>btWCv*m=?MSS8M4e;gN*9fd^ojNP?v=<1OO1_>pwe~y z;Q5T4W;3`_p{u?;zc&!n?2QX|(M%d=p3S+mdD`#4q(1!Q^Rufx)FUld(^jH|PgQYH zq-1MY)Iu$yd1DRYQcqnZCc{!;-k?htv>rt}jh<|X9;!FYbZyN6=`_d@kr$}f_WZe7 zyaQmalwFb*#_8KIixD636|5H4P{@DvorYaL$`~FRUinBrH8Jj4>h$7c^|v-tvU{;_ z)saT4iq3@qSAXEuVV3EE3!rVFO z{cu6vGG6^n9QW}_jUWwNvDHFmtqoxccu_2gSGkWF#Ps~UmyAF+xs$rJRBO!dC%J6> zD`vj`5mXbZcUF7y%o%|Q~1^EdXLIul1rs&6YlAE#I?rU74i zuQD*XBsQgP%<6mLX}gqOH}Moe_sri}mxP1&H&5#iukM2cr#XrL8?PAFNrC%F)~& z#8S@Zw~wk9FZgD^nHW-UJFPyaGNAx)rN<|d-rTfu;-eKvEG)E_V++sMo);@FH_px| zNFhIJ^h!R-|H1F+$+VR5a%^_FRez`Fl5aAycpZH$F~X4EpL?nfUJIMTUK`-?aP!Flzd6C|)ZT_pkRsOLxe%$b zY1^%Psi82@Z6B`LyU4dFzc7fHU^&of%8lHkUSO8|*o@^(kyk*Qt# zfL`JnE@=NoD*FoLgQhYweiEo63P1Y{28@Fnbf0Mk@4J{tDpy5REa+G@&=?j$S&y;T z)a~ZB12Sib7tCUc8S``S*ECo4Dq3D9qqQ|l)`l+nd^vD!^zlpJ>q{~MF-%V%s?7o# zyS0Sw!U+?aealR`sr&rMZ1XFM+Cq(|Rzw2zy1a&dvI8w!PQ^qy@pap#WKLy?eycYw zdr8?W4pW^;ZZ|)(^`fjHHg;pT-i!QydEH$I~~bs@}B8 z^(iXKeg2mXYmH`JCk?;;bcXA&XEtV=vX3E9`KhQ;6v!EhstGGs-O8Z8OOf5zA`6N z>Kk><(BF>9x%EMbONLckC_<_4O8b zn=yg2vCo0B68e3>sbK9=Qz)@w+!C*YSbypSo8{2V~Xl=dNW`JfeL z?J|M$(bR9E$6G;|C+he<5%m>_{eAuTxUdf1zEzn$^6PIDLivZK2@5ePzMm?u-OnMc zfRxYc_@H~7q}+%>kN^y`MLgnU=H>!kt%|A$p=e%uOkA!Fmgyv4lajN7_iwu@1G(5} z*E5cqxh8rnIZV$@f0|GaI`Fo;ya4yPd%<`7!%A_DWaady>rZQ!)|M)#=UfVFIv-vO z3iPI|5Nj^}(9qnQhioczT^e2bg7`a4X#~|wZ6%4O9l~IVEq%P4b6=ySUxuLYy1<7b5 zJ5hU18FwJNnbD6ucYnosmT%kPRu3F3@^Qo?du<9obptE;y`3?RMN<17Mp-9~Jj!Fh5S^p5D@Y%lcI_1SHaP>oGN3t#al#U=!6;G0C|Hg6b*EOv$!xo5Z;#r<8(@W zg-2Af2O)C{zg0xBGx;!i^wWoohM|^vW2nG4u@8=SMb_W#lEY;^e;P$OgdX!1!VSUI zBa=p>_;*|^7>WAa8mVH^p_08<4IKk_vo*J!`QGrB;OH#tyxWI@{~*M{Of_zdgepQ4 zVeHVizDxVxCT{|xu7VvFK5`l4%}LPjrzD+T-Q)R)JR1FCC+P}{?FHJGVuT$!_ky_o zMPfN@5LU%=b|z1wPYes}9c5rDgeJTk7%wGTxI|OAy2Iq78F92koZ}TNqsmGuv%pfW!9l+p(Zwrnqf~wi|Cnb#z?KEj zwWAD4z80= zZ#Ewa=)|ecG@s7my{o#7!0gCt8$*c7iPkvPX~6Tv zU^UJ#4i>#Iua60NvsQ+y?CBCQvjRMWLGvwQ0@dJ$fK^TFNUTSttt z3zyR+sVRo7!?KZq^rh+yi$M9n}A zY@8pkd2%i95$^5+4;+st-gGT+XZcG*SXk}RnJ0CF!QhOPrd?ioUPJ`R;h@1LN7fB) z_e@Q^&wujkbF0aivwIhlz2Y5Akxr7HvR+aRV9ih%mJFY#RU@}xk(qA7gpZIm0WeuY zhJGqf&}%2~6H+grkPMUR2TsYn;<7#a(aN&}(s5_l3gRaRI3*fLHI!uiMcH$$Rv4mVg8QX=Lac(Rw zgfJrrPzlmS6Wg2@OHsspTH!b@j-6>8T*AlKJ;1XbF3!r+Y~r_=u=@4T`s;>W{DC5w zs3LNz?ETLG)-M$gVU}|Vyrz%baYu*;X{u7mFYv_fU1W7}!RY_(MYZ^YKH3`U*`g~}t|iDl#L`aIgCblmj( zo9)-j+N6)t9TS8mFzdZXabSqFM_)m6shtBa;lS%e5I1iKb;}TPsZ5S*M6cqL(JsuFdLE|-R>DS8z2h^!*5cX)75Nz>P zk{jyi2uLc`^0fUNgX+Oo2`N*-%36#WF#1tWNM+POZqL z>&JLj!}l)0ky#kNz<~OZGZ7qi*QuFvAuD8g4hkY_F2bodS9+ygqm{aqpdWzsBo|AT zm}$6{YF#+g-;B6+JF)|tGSPcTNVWHnJ2^yfKbu8G6Q64DJ;bbsXuNmHsX<)!=pV+* zwKX~2;~B%50V9Ta*vfFJ*4qW66yG=nUq}8VvirFWT?AQNUwtcIn#FH1XIZ?(%WEd- z3ZS0ahd@`+^-;rj zW3?Izg+qP5zH@l09HB;hPM?ZL1L2qv;xUjQ(5~nb-7$yz0dGot0c2LKfcrU3&CEB# z`!&>3_r1Ld9z7P|hI7M;Xn%2QWFH6quXA{#HK04>9q4O}U}u~2B&qe!^%UFiow(tg zrsn5ONE}@2^l50z=n}&^z5r&t2&YZJ7qaHcKGGMmIHb9$&t-4QKz7qEs0(J|L^n&6 ze>{Ix*P23p_4wkkPQzZ&Lv6vkcH1p$k@(+DB8QQF<|iCK;x0KuDFh2x^TA*Fp6fU+$$znL*y@ z(9J9@YQ%xs{9ztNIzhHLPv^}8<*yDWta`8M(QBa9m(7Dmr%GRY1Ls0Zo=CI^$#Oz) zUg?gH-j40HKQZDG!`b+Co1Lr0^DU!)EALCX;hH$nO_uC-f)AZq*TFW(%e@jUY%pe3 z>(=RNl!xzPl}*K)i}Ukl15wqmOPoRYz076t$XH3T0<(8X)KF5M-~_SLyIjUHa`{o> zh`ZVNYt*^v-#<<8%>w>O?i*IA?XA+8ffl1#A(})+rM!?T#0eio*Gb?8>62|e+ZNET z&VLQMb+?-g_zAD&SdR*u!QU&tR?9-D)+b31#hCWz9k-2x7JpZ-H8`5yGSZ~;N2>~Z zYe|!zt34&Ng+*M6^UZV-_je6r_)(^d-04uqwR0_C#_A{!t0WYeiF$#mDL+lp{*eBx z!~jwa`B>2wY96tc>$;d=LjX!l71IdbX~^5n{*XOFm?=8@Uo3BddjV>yQCCPF6^uKB zEg}4=_1Xl{8PvB0sbfA8o?tIFg`_b7M9!SN6f_5w>?@QKlfFMlW z5|GL($2~kW^L*bPx)dp-SrmwF{)O)O?$OxZ{>BybZ=-Me=3vW;C$Od z;&@_C54sVtKiNbeqGMD;Y_{_htn+C%Cptn-z%C9RvG*k*u@UVG8Y`xL9l+ozxQGt*ccLfo>qIPACI-)8ba3^&C<;-Kwz z>~=@&6I~Pk-pP4oBL{VElx_Z9n4q4nk=V)$W$IgC^HU zw&9LZu|99?VR7?7Q|>N}G+RB)TDx2YGA$Ylv*tow5whulBFpl|frsUMX=AHPEVYZm zR_wWiNsR#-F2F1@khX6ap_StrWn*`mV>q)3|CyPA?A||=A|iPiX3QEQFt`2j$p!M( z@+zgYEg|aaI?Gk8NEhn$s8px6aGn}sx!2+I%tqY^Q|tMbsh7>kf z3b?0s=GE5v2BjC{u;YSbY)^~v>9_=Sh841eH7GzJdQMUxl`sM>9_!zt&$7bz)J7?4yk-^{H!5uQSnjMD}~_D(8DyomfKN z`^wzPU6-(mzk!rwkKVn=oSUNvqYc-o-j_k!hlDl3vfp#e9r(1Sg#5JdqX&uA1*mH2 zK+3CMeHV59EW#LMUMUQW2(`{8s7iQukTuWE3?GCMpiKD4Iutx4${*V)bM9J4F;?BC z>qPVmtIEOdv{I2}SbW-a+_PBqzDr#0McIuLeZ~3fR|a@a-CN&ItNMu~UGKUH~TQos{cYNE0&ddz43wsC!? z>p`TT369tSn~Z|T;bN3jSmJLiAIAN`y*>sI@@&<)`G#hljAi9XR4DoyQe^aGX9#pB zF=z7-X?y0vt@cbyD`8m#7U*T%DZ=iY^6Qbk#?^*ajk2#AEKk3_X(>u};cq64N^(Gh zvY?V$V(uRo!wcqJ&@b-dn`c?T)%Rd%SDhCh;Z9)3Wr+fzM=3RTCgDLc1zVl6aJ{{-gY^hg)usTv z7t`iGQ;fR4=V%z;s7_J7VcgxTG=sDjm07or5M^?`g1x-}-9{3_QXJUO2*%A@qxJf$X`2k=tE3Z%yZwsmJiDl zJ$lcuOX=P$5+u^D3;oLatXRFpZj*VkF_?A(@(wuNDCO=@R zC*Y;?N=Y7p-i+E_2wZ)}!5>E3 zvQK3H-RWaP?J>Qd^3IaWP`dnZ;Qsx;HT6Kg|9}u?aZZMy&^Nh1vs&C&x;`~|Jy+HQ z8m%(nW2Uwn5`V?+?&x>`(9wtUo)U-J-5Lb-9E=rT^%>IOQoOxUNO`z*CBx_y1aAlU zfxC3DpHSGTW_uP|0c$u(Si;$$^z7mFys(_z^#a8UV0aA1cf=2M#VJTR(Slc)^ajjO zK#a(ZYWs^KP7KUoDN#fI;1+Euwk->c+mm6HNY;8e+y5+@DvM2S&3}YVy~WVkd>^E? z905TnxAyF^tA!tlXkx}+Lr*_$+?$QJv^8a@&(sa$>HwBK#K0(FDkh_e3Ek=|Q}Tk6 ze${SQn9aAL*iK53n--}BuC~D+wj%WXKzFtPpcnhb)J)Z{NY8g#^DLvpbyVIRXaS3u z_6T3-8Zeg&X7_+%G3vH3qyj4*#Sr^^0Zg9f)0R~%U{-|QFKjqYep4u;-*+XC1MwD@ zHagr9k&8=43B$dk+@uI~gz?1$D#&rbC2|U>6Jri49HF^H-|QBK=x`3uZc#@>xfQK@2!%=dlZPc)a!*sG#hBQ#Cm{WJp85gck8FTv z+!h9suj}0a;_%U->JhI26*W%BKDuh8BDTeI`aE{_CM-pf#VFKig{}^U29%prS0DDG zjx~FxV|*JD(PutU7eKWZWgzXi{D^@i!w3OU_=%(dhs}#%U zcq*92-#;HWq0)RMkm9g~El&6g%Czi#vJHL*^9qJH0+GNC{&H|#ISE*yVs z|FfMC9eTR;RV7$bh;HTBR9_R_oPxb>ulqvIJz&0UKJ9B);RDV?jV>F~YV z7Wn%$$E}&5B|cx=J*UCK(axKhW@n%F8>oJ8dXszpd9`{%j^exP&#XU4k1OBt`Pc^b z?PkT@C6$NI{gVr261~yifpfrCbTty^2Xj%s9yh=6qu&85T}@5S1ziSH_`kN*Uu#A; z?}DmpNAT|g(7)qb`z@I0UWQ#5qksGyyf}BVW`MwK_!C#(KdUxwHSh)LlFvr}>s3R1 z_ZyU=$qSM*f9V(0mB68p!+(YU7uEL9kCYT@unjn;r00WyFX<`*^3H05o=(TJmUOK^eeExEOFCUygyzrr#|zLuaXN#xVuRPM9-D1i%0$1gE1=KD z)10y;=sMIjdXS8Gv2 z1ry<{0qpQZTnCtj1D+E%X>H>xwL1Z8U>ddQM{B>?veqMoNkOZTK@hFYC(j3t1g$m- zwE_%`-(eeM1L~#$g_Hu8KDKI{{;1!U>AwHv!0#5h!u{sDE8+ZiAN;iUoZM@&kk-&;g>)t#a-a)ysPlh0`(0TkNDlYKr-RWcOdn$ ztj{syu7p`W<|Xa{C+{&~ES*{jFT*`}FG@DJMX~8RRBHMCZ97BgOfl(dS>iw6bk`l0 zp|`Ql##LN+PwmIh&^@{QobH%w=;jogz%?iUM3;&qc~-%ibFa~@FCbRd=eKx-+Hb(2 z3Nq@D=e9va(L*axQSF&?Cw4^n8soT0G%&xaW&Fb6`tNJNV{NV%C%}goV-~z|x(`Sr zcb{AN_N5JIOw8#s&uPFuiJ44EZttr3>FPYs2=+uWm3t=a&%OQUTRwh#m*Pjv>w!yQ z?5%*CvHk`kUVD4t@iedlUtqyC60SU&v<9ph+LHR0CRV{DHJ!y~RA%aE5)xTbOgHU6 zF~)kJn_%s49s50Z4oe|KflmUkYD%|+DVVbRUkIQ zYEAfvXT|8m2^-^hMiPBMtWPFTv<{EQv1bPW zcM$S5D%HO6hbUwJ?K&chzso>;z>$9)mAC6QA;Q5ZEGiCO?hoin4t>=mWL_OiMp>q5@kf>3H}=+xq&mKEy*A&?aFm$ zZtW82Cv>>$Q$Y8(KuPoB8bH7(QpA>PT?wLG-+@B2Byqgf;!NdF4J7CllOI1hw&NRB z?XdEG;8$Puj~#X}lp*(b$U_>a`6E)+fLBRiU>Gt1nLcEf1<%O@^v8265>mg*ad#Iu zQ^ltC`v{oK7bqKNdTQ4~$!Pm@`}m!!CNI(VFKX8o0rKj>xx(LJD8Wyxl-qJYUyK4Q zJ^lm!{{P-kAfx4LN`z*ISGMe(JTHnlI|{czJg=VP^or;@wP`u6L7 Date: Tue, 16 Jun 2020 14:17:04 +0300 Subject: [PATCH 061/133] row compression (#798) make batch size parameter simplify creating jobId --- .../connectors/bigquery/common/TypeUtil.java | 113 +++++++++ .../compression/CompactFeatureRows.java | 72 ++++++ .../compression/FeatureRowsBatch.java | 228 ++++++++++++++++++ .../bigquery/writer/BigQueryWrite.java | 57 +++-- .../io/gcp/bigquery/BatchLoadsWithResult.java | 72 ++++-- .../bigquery/writer/BigQuerySinkTest.java | 73 +++++- 6 files changed, 559 insertions(+), 56 deletions(-) create mode 100644 storage/connectors/bigquery/src/main/java/feast/storage/connectors/bigquery/compression/CompactFeatureRows.java create mode 100644 storage/connectors/bigquery/src/main/java/feast/storage/connectors/bigquery/compression/FeatureRowsBatch.java diff --git a/storage/connectors/bigquery/src/main/java/feast/storage/connectors/bigquery/common/TypeUtil.java b/storage/connectors/bigquery/src/main/java/feast/storage/connectors/bigquery/common/TypeUtil.java index a8e4cda678d..9f6cff20e1d 100644 --- a/storage/connectors/bigquery/src/main/java/feast/storage/connectors/bigquery/common/TypeUtil.java +++ b/storage/connectors/bigquery/src/main/java/feast/storage/connectors/bigquery/common/TypeUtil.java @@ -17,9 +17,12 @@ package feast.storage.connectors.bigquery.common; import com.google.cloud.bigquery.StandardSQLTypeName; +import com.google.protobuf.ByteString; import feast.proto.types.ValueProto; import java.util.HashMap; +import java.util.List; import java.util.Map; +import java.util.stream.Collectors; public class TypeUtil { @@ -63,4 +66,114 @@ public class TypeUtil { public static StandardSQLTypeName toStandardSqlType(ValueProto.ValueType.Enum valueType) { return VALUE_TYPE_TO_STANDARD_SQL_TYPE.get(valueType); } + + public static Object protoValueToObject( + ValueProto.Value value, ValueProto.Value.ValCase valCase) { + switch (valCase) { + case BYTES_VAL: + return value.getBytesVal().toByteArray(); + case STRING_VAL: + return value.getStringVal(); + case INT32_VAL: + return value.getInt32Val(); + case INT64_VAL: + return value.getInt64Val(); + case DOUBLE_VAL: + return value.getDoubleVal(); + case FLOAT_VAL: + return value.getFloatVal(); + case BOOL_VAL: + return value.getBoolVal(); + case BYTES_LIST_VAL: + return value.getBytesListVal().getValList().stream() + .map(ByteString::toByteArray) + .collect(Collectors.toList()); + case STRING_LIST_VAL: + return value.getStringListVal().getValList(); + case INT32_LIST_VAL: + return value.getInt32ListVal().getValList(); + case INT64_LIST_VAL: + return value.getInt64ListVal().getValList(); + case DOUBLE_LIST_VAL: + return value.getDoubleListVal().getValList(); + case FLOAT_LIST_VAL: + return value.getFloatListVal().getValList(); + case BOOL_LIST_VAL: + return value.getBoolListVal().getValList(); + case VAL_NOT_SET: + break; + } + return null; + } + + public static ValueProto.Value objectToProtoValue( + Object value, ValueProto.Value.ValCase valCase) { + ValueProto.Value.Builder builder = ValueProto.Value.newBuilder(); + switch (valCase) { + case BYTES_VAL: + return builder.setBytesVal(ByteString.copyFrom((byte[]) value)).build(); + case STRING_VAL: + return builder.setStringVal((String) value).build(); + case INT32_VAL: + return builder.setInt32Val((Integer) value).build(); + case INT64_VAL: + return builder.setInt64Val((Long) value).build(); + case DOUBLE_VAL: + return builder.setDoubleVal((Double) value).build(); + case FLOAT_VAL: + return builder.setFloatVal((Float) value).build(); + case BOOL_VAL: + return builder.setBoolVal((Boolean) value).build(); + case BYTES_LIST_VAL: + return builder + .setBytesListVal( + ValueProto.BytesList.newBuilder() + .addAllVal( + ((List) value) + .stream().map(ByteString::copyFrom).collect(Collectors.toList())) + .build()) + .build(); + case STRING_LIST_VAL: + return builder + .setStringListVal( + ValueProto.StringList.newBuilder().addAllVal((List) value).build()) + .build(); + case INT32_LIST_VAL: + return builder + .setInt32ListVal( + ValueProto.Int32List.newBuilder().addAllVal((List) value).build()) + .build(); + case INT64_LIST_VAL: + return builder + .setInt64ListVal( + ValueProto.Int64List.newBuilder().addAllVal((List) value).build()) + .build(); + case DOUBLE_LIST_VAL: + return builder + .setDoubleListVal( + ValueProto.DoubleList.newBuilder().addAllVal((List) value).build()) + .build(); + case FLOAT_LIST_VAL: + return builder + .setFloatListVal( + ValueProto.FloatList.newBuilder().addAllVal((List) value).build()) + .build(); + case BOOL_LIST_VAL: + return builder + .setBoolListVal( + ValueProto.BoolList.newBuilder().addAllVal((List) value).build()) + .build(); + case VAL_NOT_SET: + break; + } + return null; + } + + public static Object protoValueToObject(ValueProto.Value value) { + return protoValueToObject(value, value.getValCase()); + } + + public static Object getDefaultProtoValue(ValueProto.Value.ValCase valCase) { + return protoValueToObject(ValueProto.Value.getDefaultInstance(), valCase); + } } diff --git a/storage/connectors/bigquery/src/main/java/feast/storage/connectors/bigquery/compression/CompactFeatureRows.java b/storage/connectors/bigquery/src/main/java/feast/storage/connectors/bigquery/compression/CompactFeatureRows.java new file mode 100644 index 00000000000..e53ac636fe1 --- /dev/null +++ b/storage/connectors/bigquery/src/main/java/feast/storage/connectors/bigquery/compression/CompactFeatureRows.java @@ -0,0 +1,72 @@ +/* + * SPDX-License-Identifier: Apache-2.0 + * Copyright 2018-2020 The Feast Authors + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package feast.storage.connectors.bigquery.compression; + +import feast.proto.types.FeatureRowProto; +import org.apache.beam.sdk.coders.KvCoder; +import org.apache.beam.sdk.coders.SnappyCoder; +import org.apache.beam.sdk.coders.StringUtf8Coder; +import org.apache.beam.sdk.transforms.DoFn; +import org.apache.beam.sdk.transforms.GroupIntoBatches; +import org.apache.beam.sdk.transforms.PTransform; +import org.apache.beam.sdk.transforms.ParDo; +import org.apache.beam.sdk.values.KV; +import org.apache.beam.sdk.values.PCollection; + +/** + * Collects batch of FeatureRow, converts them into columnar form: + * + *

List<FeatureRow> -> + * + *

- Feature1 List<Integer> - all values from all rows in one list + * + *

- Feature2 List<String> + * + *

- ... + * + *

FeatureRowsBatch is in column form, hence it is better compressible. For compression we use + * Snappy, since it's already available in Beam as SnappyCoder. + */ +public class CompactFeatureRows + extends PTransform< + PCollection>, + PCollection>> { + private final int batchSize; + + public CompactFeatureRows(int batchSize) { + this.batchSize = batchSize; + } + + @Override + public PCollection> expand( + PCollection> input) { + return input + .apply(GroupIntoBatches.ofSize(batchSize)) + .apply(ParDo.of(new FeatureRowToColumnar())) + .setCoder( + KvCoder.of( + StringUtf8Coder.of(), SnappyCoder.of(FeatureRowsBatch.FeatureRowsCoder.of()))); + } + + public static class FeatureRowToColumnar + extends DoFn>, KV> { + @ProcessElement + public void process(ProcessContext c) { + c.output(KV.of(c.element().getKey(), new FeatureRowsBatch(c.element().getValue()))); + } + } +} diff --git a/storage/connectors/bigquery/src/main/java/feast/storage/connectors/bigquery/compression/FeatureRowsBatch.java b/storage/connectors/bigquery/src/main/java/feast/storage/connectors/bigquery/compression/FeatureRowsBatch.java new file mode 100644 index 00000000000..67b48a4cd19 --- /dev/null +++ b/storage/connectors/bigquery/src/main/java/feast/storage/connectors/bigquery/compression/FeatureRowsBatch.java @@ -0,0 +1,228 @@ +/* + * SPDX-License-Identifier: Apache-2.0 + * Copyright 2018-2020 The Feast Authors + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package feast.storage.connectors.bigquery.compression; + +import static feast.proto.types.ValueProto.Value.ValCase.*; +import static feast.storage.connectors.bigquery.common.TypeUtil.*; + +import feast.proto.types.FeatureRowProto; +import feast.proto.types.FieldProto; +import feast.proto.types.ValueProto; +import java.io.IOException; +import java.io.InputStream; +import java.io.OutputStream; +import java.io.Serializable; +import java.util.*; +import java.util.stream.Collectors; +import java.util.stream.IntStream; +import org.apache.beam.sdk.coders.*; +import org.apache.beam.sdk.schemas.Schema; +import org.apache.beam.sdk.values.Row; + +/** + * Based on list of featureRows this class infers common schema (with all found features) and then + * transpose list of rows into list of values format (column-oriented). + * + *

getFeatureRows provides reverse transformation + */ +public class FeatureRowsBatch implements Serializable { + private final Schema schema; + private String featureSetReference; + private List values = new ArrayList<>(); + + public static Map protoToSchemaTypes = + new HashMap<>(); + public static Map schemaToProtoTypes; + public static Map defaultValues = new HashMap<>(); + + static { + protoToSchemaTypes.put(BYTES_VAL, Schema.FieldType.BYTES); + protoToSchemaTypes.put(STRING_VAL, Schema.FieldType.STRING); + protoToSchemaTypes.put(INT32_VAL, Schema.FieldType.INT32); + protoToSchemaTypes.put(INT64_VAL, Schema.FieldType.INT64); + protoToSchemaTypes.put(DOUBLE_VAL, Schema.FieldType.DOUBLE); + protoToSchemaTypes.put(FLOAT_VAL, Schema.FieldType.FLOAT); + protoToSchemaTypes.put(BOOL_VAL, Schema.FieldType.BOOLEAN); + protoToSchemaTypes.put(BYTES_LIST_VAL, Schema.FieldType.array(Schema.FieldType.BYTES)); + protoToSchemaTypes.put(STRING_LIST_VAL, Schema.FieldType.array(Schema.FieldType.STRING)); + protoToSchemaTypes.put(INT32_LIST_VAL, Schema.FieldType.array(Schema.FieldType.INT32)); + protoToSchemaTypes.put(INT64_LIST_VAL, Schema.FieldType.array(Schema.FieldType.INT64)); + protoToSchemaTypes.put(FLOAT_LIST_VAL, Schema.FieldType.array(Schema.FieldType.FLOAT)); + protoToSchemaTypes.put(BOOL_LIST_VAL, Schema.FieldType.array(Schema.FieldType.BOOLEAN)); + protoToSchemaTypes.put(DOUBLE_LIST_VAL, Schema.FieldType.array(Schema.FieldType.DOUBLE)); + + schemaToProtoTypes = + protoToSchemaTypes.entrySet().stream() + .collect(Collectors.toMap(Map.Entry::getValue, Map.Entry::getKey)); + } + + public FeatureRowsBatch(Iterable featureRows) { + this.schema = this.inferCommonSchema(featureRows); + this.initValues(); + this.toColumnar(featureRows); + } + + FeatureRowsBatch(Schema schema, List values) { + this.schema = schema; + this.values = values; + } + + private Schema inferCommonSchema(Iterable featureRows) { + Map types = new HashMap<>(); + List fieldsInOrder = new ArrayList<>(); + + featureRows.forEach( + row -> + row.getFieldsList() + .forEach( + f -> { + Schema.FieldType fieldType = + protoToSchemaTypes.get(f.getValue().getValCase()); + if (types.containsKey(f.getName())) { + if (!types.get(f.getName()).equals(fieldType)) { + throw new RuntimeException("schema cannot be inferred"); + } + + return; + } + + if (!defaultValues.containsKey(f.getName())) { + defaultValues.put( + f.getName(), getDefaultProtoValue(f.getValue().getValCase())); + } + + Schema.Field column = + Schema.Field.of(f.getName(), Schema.FieldType.array(fieldType)); + + types.put(f.getName(), fieldType); + fieldsInOrder.add(column); + + if (featureSetReference == null) { + featureSetReference = row.getFeatureSet(); + } + })); + Schema schema = Schema.builder().addFields(fieldsInOrder).build(); + schema.setUUID(UUID.randomUUID()); + return schema; + } + + private void initValues() { + IntStream.range(0, schema.getFieldCount()) + .forEach( + idx -> { + values.add(new ArrayList<>()); + }); + } + + private void toColumnar(Iterable featureRows) { + featureRows.forEach( + row -> { + Map rowValues = + row.getFieldsList().stream() + .collect(Collectors.toMap(FieldProto.Field::getName, FieldProto.Field::getValue)); + + IntStream.range(0, schema.getFieldCount()) + .forEach( + idx -> { + Schema.Field field = schema.getField(idx); + if (rowValues.containsKey(field.getName())) { + ((List) values.get(idx)) + .add(protoValueToObject(rowValues.get(field.getName()))); + } else { + ((List) values.get(idx)).add(defaultValues.get(field.getName())); + } + }); + }); + } + + public Schema getSchema() { + return this.schema; + } + + public String getFeatureSetReference() { + return this.featureSetReference; + } + + public FeatureRowsBatch withFeatureSetReference(String featureSetReference) { + this.featureSetReference = featureSetReference; + return this; + } + + public Row toRow() { + return Row.withSchema(schema).attachValues(values).build(); + } + + public static FeatureRowsBatch fromRow(Row row) { + return new FeatureRowsBatch(row.getSchema(), row.getValues()); + } + + public Iterator getFeatureRows() { + return IntStream.range(0, ((List) values.get(0)).size()) + .parallel() + .mapToObj( + rowIdx -> + FeatureRowProto.FeatureRow.newBuilder() + .setFeatureSet(getFeatureSetReference()) + .addAllFields( + IntStream.range(0, schema.getFieldCount()) + .mapToObj( + fieldIdx -> + FieldProto.Field.newBuilder() + .setName(schema.getField(fieldIdx).getName()) + .setValue( + objectToProtoValue( + ((List) values.get(fieldIdx)).get(rowIdx), + schemaToProtoTypes.get( + schema + .getField(fieldIdx) + .getType() + .getCollectionElementType()))) + .build()) + .collect(Collectors.toList())) + .build()) + .iterator(); + } + + public static class FeatureRowsCoder extends CustomCoder { + private final Coder schemaCoder = SerializableCoder.of(Schema.class); + private final Coder referenceCoder = StringUtf8Coder.of(); + + public static FeatureRowsCoder of() { + return new FeatureRowsCoder(); + } + + private Coder getDelegateCoder(Schema schema) { + return RowCoderGenerator.generate(schema); + } + + @Override + public void encode(FeatureRowsBatch value, OutputStream outStream) + throws CoderException, IOException { + schemaCoder.encode(value.getSchema(), outStream); + referenceCoder.encode(value.getFeatureSetReference(), outStream); + getDelegateCoder(value.getSchema()).encode(value.toRow(), outStream); + } + + @Override + public FeatureRowsBatch decode(InputStream inStream) throws CoderException, IOException { + Schema schema = schemaCoder.decode(inStream); + String reference = referenceCoder.decode(inStream); + return FeatureRowsBatch.fromRow(getDelegateCoder(schema).decode(inStream)) + .withFeatureSetReference(reference); + } + } +} diff --git a/storage/connectors/bigquery/src/main/java/feast/storage/connectors/bigquery/writer/BigQueryWrite.java b/storage/connectors/bigquery/src/main/java/feast/storage/connectors/bigquery/writer/BigQueryWrite.java index c8966bfa341..6916534e4f2 100644 --- a/storage/connectors/bigquery/src/main/java/feast/storage/connectors/bigquery/writer/BigQueryWrite.java +++ b/storage/connectors/bigquery/src/main/java/feast/storage/connectors/bigquery/writer/BigQueryWrite.java @@ -24,6 +24,8 @@ import feast.proto.types.FeatureRowProto.FeatureRow; import feast.storage.api.writer.FailedElement; import feast.storage.api.writer.WriteResult; +import feast.storage.connectors.bigquery.compression.CompactFeatureRows; +import feast.storage.connectors.bigquery.compression.FeatureRowsBatch; import java.util.List; import java.util.Map; import org.apache.beam.sdk.coders.KvCoder; @@ -52,6 +54,7 @@ public class BigQueryWrite extends PTransform, WriteResu private static final Duration BIGQUERY_JOB_MAX_EXPECTING_RESULT_TIME = Duration.standardHours(1); private static final int BIGQUERY_MAX_JOB_RETRIES = 20; + private static final int DEFAULT_COMPACTION_BATCH_SIZE = 10000; private DatasetId destination; private PCollectionView>> schemas; @@ -59,6 +62,7 @@ public class BigQueryWrite extends PTransform, WriteResu private Duration triggeringFrequency = BIGQUERY_DEFAULT_WRITE_TRIGGERING_FREQUENCY; private Duration expectingResultTime = BIGQUERY_JOB_MAX_EXPECTING_RESULT_TIME; private BigQueryServices testServices; + private int compactionBatchSize = DEFAULT_COMPACTION_BATCH_SIZE; public BigQueryWrite( DatasetId destination, PCollectionView>> schemas) { @@ -81,6 +85,11 @@ public BigQueryWrite withExpectingResultTime(Duration expectingResultTime) { return this; } + public BigQueryWrite withCompactionBatchSize(int batchSize) { + this.compactionBatchSize = batchSize; + return this; + } + /** * BigQuery writer 1. choose destination based on featureSetName {@link * FeatureDynamicDestinations} 2. dynamically pull destination's schema from schemas' view 3. @@ -117,7 +126,7 @@ public WriteResult expand(PCollection input) { input.apply( "BatchingInput", Window.into(FixedWindows.of(triggeringFrequency)) - .withAllowedLateness(expectingResultTime) + .withAllowedLateness(Duration.ZERO) .discardingFiredPanes()); PCollection> insertionResult = @@ -164,27 +173,30 @@ public void processElement(ProcessContext context) {} private PCollection mergeInputWithResult( PCollection inputInFixedWindow, PCollection> successful) { - final TupleTag inputTag = new TupleTag<>(); + final TupleTag inputTag = new TupleTag<>(); final TupleTag successTag = new TupleTag<>(); + PCollection> insertedRows = + inputInFixedWindow + .apply( + "MakeElementKey", + ParDo.of( + new DoFn>() { + @ProcessElement + public void process(ProcessContext c) { + FeatureRow element = c.element(); + c.output( + KV.of( + BigQuerySinkHelpers.getTableDestination( + destination, element.getFeatureSet()) + .getTableSpec(), + element)); + } + })) + .apply(new CompactFeatureRows(compactionBatchSize)); + PCollection> inputWithResult = - KeyedPCollectionTuple.of( - inputTag, - inputInFixedWindow.apply( - "MakeElementKey", - ParDo.of( - new DoFn>() { - @ProcessElement - public void process(ProcessContext c) { - FeatureRow element = c.element(); - c.output( - KV.of( - BigQuerySinkHelpers.getTableDestination( - destination, element.getFeatureSet()) - .getTableSpec(), - element)); - } - }))) + KeyedPCollectionTuple.of(inputTag, insertedRows) .and( successTag, successful.apply( @@ -208,7 +220,12 @@ public void process(ProcessContext c) { CoGbkResult result = c.element().getValue(); result .getAll(successTag) - .forEach(success -> result.getAll(inputTag).forEach(c::output)); + .forEach( + success -> + result + .getAll(inputTag) + .forEach( + rows -> rows.getFeatureRows().forEachRemaining(c::output))); } })); } diff --git a/storage/connectors/bigquery/src/main/java/org/apache/beam/sdk/io/gcp/bigquery/BatchLoadsWithResult.java b/storage/connectors/bigquery/src/main/java/org/apache/beam/sdk/io/gcp/bigquery/BatchLoadsWithResult.java index 89ce54b8219..c0c3f6f84c2 100644 --- a/storage/connectors/bigquery/src/main/java/org/apache/beam/sdk/io/gcp/bigquery/BatchLoadsWithResult.java +++ b/storage/connectors/bigquery/src/main/java/org/apache/beam/sdk/io/gcp/bigquery/BatchLoadsWithResult.java @@ -5,12 +5,17 @@ import com.google.api.services.bigquery.model.TableRow; import com.google.auto.value.AutoValue; + import java.util.Collections; import java.util.List; import java.util.Set; import javax.annotation.Nullable; + import org.apache.beam.sdk.coders.*; import org.apache.beam.sdk.options.ValueProvider; +import org.apache.beam.sdk.state.StateSpec; +import org.apache.beam.sdk.state.StateSpecs; +import org.apache.beam.sdk.state.ValueState; import org.apache.beam.sdk.transforms.*; import org.apache.beam.sdk.transforms.windowing.*; import org.apache.beam.sdk.values.*; @@ -23,7 +28,7 @@ @AutoValue public abstract class BatchLoadsWithResult extends PTransform< - PCollection>, PCollection>> { + PCollection>, PCollection>> { static final Logger LOG = LoggerFactory.getLogger(BatchLoadsWithResult.class); @VisibleForTesting @@ -39,6 +44,8 @@ public abstract class BatchLoadsWithResult static final int DEFAULT_MAX_RETRY_JOBS = 3; + static final int FILE_TRIGGERING_RECORD_COUNT = 500000; + @Nullable abstract BigQueryServices getBigQueryServices(); @@ -135,6 +142,12 @@ public PCollection> expand( PCollection> results = input + .apply("WindowWithTrigger", + Window.>configure() + .triggering( + Repeatedly.forever( + AfterPane.elementCountAtLeast(FILE_TRIGGERING_RECORD_COUNT))) + .discardingFiredPanes()) .apply( "PutAllRowsInSingleShard", ParDo.of( @@ -150,8 +163,8 @@ public void process(ProcessContext c) { .apply( "WriteGroupedRecords", ParDo.of( - new WriteGroupedRecordsToFiles<>( - tempFilePrefixView, DEFAULT_MAX_FILE_SIZE, getRowWriterFactory())) + new WriteGroupedRecordsToFiles<>( + tempFilePrefixView, DEFAULT_MAX_FILE_SIZE, getRowWriterFactory())) .withSideInputs(tempFilePrefixView)) .setCoder(WriteBundlesToFiles.ResultCoder.of(getDestinationCoder())); @@ -163,6 +176,10 @@ tempFilePrefixView, DEFAULT_MAX_FILE_SIZE, getRowWriterFactory())) // Copied from original BatchLoads PCollectionTuple partitions = results + .apply( + Window.>configure() + .triggering(DefaultTrigger.of()) + ) .apply("AttachSingletonKey", WithKeys.of((Void) null)) .setCoder( KvCoder.of( @@ -172,15 +189,15 @@ tempFilePrefixView, DEFAULT_MAX_FILE_SIZE, getRowWriterFactory())) .apply( "WritePartitionTriggered", ParDo.of( - new WritePartition<>( - false, - getDynamicDestinations(), - tempFilePrefixView, - DEFAULT_MAX_FILES_PER_PARTITION, - DEFAULT_MAX_BYTES_PER_PARTITION, - multiPartitionsTag, - singlePartitionTag, - getRowWriterFactory())) + new WritePartition<>( + false, + getDynamicDestinations(), + tempFilePrefixView, + DEFAULT_MAX_FILES_PER_PARTITION, + DEFAULT_MAX_BYTES_PER_PARTITION, + multiPartitionsTag, + singlePartitionTag, + getRowWriterFactory())) .withSideInputs(tempFilePrefixView) .withOutputTags(multiPartitionsTag, TupleTagList.of(singlePartitionTag))); @@ -203,31 +220,40 @@ private PCollection createLoadJobIdPrefixView( // So generated ids can be applied as side input return input .apply( - "JobIdWindow", - Window.>configure() - .triggering(Repeatedly.forever(AfterProcessingTime.pastFirstElementInPane())) - .discardingFiredPanes()) - .apply( - "JustDummyEvent", + "EraseKey", ParDo.of( - new DoFn, KV>() { + new DoFn, KV>() { @ProcessElement public void process(ProcessContext c) { - c.output(KV.of((Void) null, (Void) null)); + c.output(KV.of(null, c.element().getValue())); } })) - .apply("OneEventPerWindow", GroupByKey.create()) .apply( "CreateJobId", ParDo.of( - new DoFn>, String>() { + new DoFn, String>() { + @StateId("generatedForWindow") + private final StateSpec> generatedForWindow = StateSpecs.value(BooleanCoder.of()); + @ProcessElement - public void process(ProcessContext c, BoundedWindow w) { + public void process( + ProcessContext c, + BoundedWindow w, + @StateId("generatedForWindow") ValueState generatedForWindow) { + + if (generatedForWindow.read() != null) { + return; + } + + generatedForWindow.write(true); + c.output( String.format( "beam_load_%s_%s", c.getPipelineOptions().getJobName().replaceAll("-", ""), BigQueryHelpers.randomUUIDString())); + + LOG.info("Pane {}, start: {}, last: {}", c.pane().getIndex(), c.pane().isFirst(), c.pane().isLast()); LOG.info("[BQ] New window {}, {}", c.timestamp(), w.maxTimestamp()); } })); diff --git a/storage/connectors/bigquery/src/test/java/feast/storage/connectors/bigquery/writer/BigQuerySinkTest.java b/storage/connectors/bigquery/src/test/java/feast/storage/connectors/bigquery/writer/BigQuerySinkTest.java index e36e2313456..f4262fa81b6 100644 --- a/storage/connectors/bigquery/src/test/java/feast/storage/connectors/bigquery/writer/BigQuerySinkTest.java +++ b/storage/connectors/bigquery/src/test/java/feast/storage/connectors/bigquery/writer/BigQuerySinkTest.java @@ -16,6 +16,7 @@ */ package feast.storage.connectors.bigquery.writer; +import static feast.storage.common.testing.TestUtil.createRandomValue; import static feast.storage.common.testing.TestUtil.field; import static feast.storage.connectors.bigquery.writer.FeatureSetSpecToTableSchema.*; import static org.hamcrest.CoreMatchers.*; @@ -34,17 +35,20 @@ import feast.proto.core.FeatureSetProto.FeatureSetSpec; import feast.proto.core.FeatureSetProto.FeatureSpec; import feast.proto.types.FeatureRowProto.FeatureRow; +import feast.proto.types.FieldProto; import feast.proto.types.ValueProto; import feast.storage.api.writer.FeatureSink; import feast.storage.api.writer.WriteResult; +import feast.storage.connectors.bigquery.compression.CompactFeatureRows; +import feast.storage.connectors.bigquery.compression.FeatureRowsBatch; import java.io.IOException; import java.io.Serializable; -import java.util.ArrayList; -import java.util.Arrays; -import java.util.List; -import java.util.Random; +import java.util.*; import java.util.logging.ConsoleHandler; import java.util.logging.Logger; +import java.util.stream.Collectors; +import java.util.stream.IntStream; +import java.util.stream.Stream; import org.apache.beam.sdk.coders.KvCoder; import org.apache.beam.sdk.coders.StringUtf8Coder; import org.apache.beam.sdk.extensions.protobuf.ProtoCoder; @@ -57,8 +61,7 @@ import org.apache.beam.sdk.testing.PAssert; import org.apache.beam.sdk.testing.TestPipeline; import org.apache.beam.sdk.testing.TestStream; -import org.apache.beam.sdk.transforms.Create; -import org.apache.beam.sdk.transforms.PTransform; +import org.apache.beam.sdk.transforms.*; import org.apache.beam.sdk.transforms.windowing.AfterPane; import org.apache.beam.sdk.transforms.windowing.GlobalWindows; import org.apache.beam.sdk.transforms.windowing.Repeatedly; @@ -114,13 +117,25 @@ public static PipelineOptions makePipelineOptions() { return options; } - public FeatureRow generateRow(String featureSet) { - return FeatureRow.newBuilder() - .setFeatureSet(featureSet) - .addFields(field("entity", rd.nextInt(), ValueProto.ValueType.Enum.INT64)) - .addFields( - field("feature", String.format("%f", rd.nextFloat()), ValueProto.ValueType.Enum.STRING)) - .build(); + private FeatureRow generateRow(String featureSet) { + FeatureRow.Builder row = + FeatureRow.newBuilder() + .setFeatureSet(featureSet) + .addFields(field("entity", rd.nextInt(), ValueProto.ValueType.Enum.INT64)); + + for (ValueProto.ValueType.Enum type : ValueProto.ValueType.Enum.values()) { + if (type == ValueProto.ValueType.Enum.INVALID + || type == ValueProto.ValueType.Enum.UNRECOGNIZED) { + continue; + } + row.addFields( + FieldProto.Field.newBuilder() + .setName(String.format("feature_%d", type.getNumber())) + .setValue(createRandomValue(type, 5)) + .build()); + } + + return row.build(); } @Before @@ -396,6 +411,38 @@ public void updateSpecInFlight() { assertThat(loadConfiguration.getSchema().getFields(), is(expectedFields)); } + public static class ExtractKV extends DoFn> { + @ProcessElement + public void process(ProcessContext c) { + c.output(KV.of(c.element().getFeatureSet(), c.element())); + } + } + + public static class FlatMap extends DoFn, FeatureRow> { + @ProcessElement + public void process(ProcessContext c) { + c.element().getValue().getFeatureRows().forEachRemaining(c::output); + } + } + + @Test + public void featureRowCompressShouldPackAndUnpackSuccessfully() { + Stream stream1 = IntStream.range(0, 1000).mapToObj(i -> generateRow("project/fs")); + Stream stream2 = + IntStream.range(0, 1000).mapToObj(i -> generateRow("project/fs_2")); + + List input = Stream.concat(stream1, stream2).collect(Collectors.toList()); + + PCollection result = + p.apply(Create.of(input)) + .apply("KV", ParDo.of(new ExtractKV())) + .apply(new CompactFeatureRows(1000)) + .apply("Flat", ParDo.of(new FlatMap())); + + PAssert.that(result).containsInAnyOrder(input); + p.run(); + } + public static class TableAnswer implements Answer

, Serializable { TableId tableId; TableDefinition tableDefinition; From 61371022f127a5297f43ddae90307a1bddaa4978 Mon Sep 17 00:00:00 2001 From: Oleksii Moskalenko Date: Wed, 17 Jun 2020 04:19:06 +0300 Subject: [PATCH 062/133] FeatureSets are delivered to Ingestion Job through Kafka (#792) * intermediate table in m2m Job <-> FeatureSet with delivery status + FeatureSet version switch to spring-kafka (configs) specService send message to kafka & expect ack & update status accordingly jobs runner to send source & specs config (source + ack) ingestion job to read specs from kafka and send ack return featureSets in ingestionJob generate uniq topic name for each test run prevent listJobs from failing when job failed on start * fix listJobs with filter --- core/pom.xml | 4 +- .../feast/core/config/FeastProperties.java | 14 + .../core/config/FeatureStreamConfig.java | 127 ++++++--- .../java/feast/core/config/JobConfig.java | 38 ++- .../java/feast/core/dao/JobRepository.java | 8 +- .../java/feast/core/job/JobUpdateTask.java | 43 +-- .../core/job/dataflow/DataflowJobManager.java | 48 ++-- .../job/direct/DirectRunnerJobManager.java | 33 +-- .../java/feast/core/model/FeatureSet.java | 37 ++- .../feast/core/model/FeatureSetJobStatus.java | 64 +++++ core/src/main/java/feast/core/model/Job.java | 22 +- .../core/service/JobCoordinatorService.java | 38 +-- .../java/feast/core/service/JobService.java | 11 +- .../java/feast/core/service/SpecService.java | 109 +++++++- .../feast/core/util/KafkaSerialization.java | 68 +++++ .../main/resources/application-override.yaml | 0 core/src/main/resources/application.yml | 4 + .../feast/core/job/JobUpdateTaskTest.java | 16 +- .../job/dataflow/DataflowJobManagerTest.java | 45 ++-- .../direct/DirectRunnerJobManagerTest.java | 37 +-- .../service/JobCoordinatorServiceTest.java | 18 +- .../feast/core/service/JobServiceTest.java | 7 +- .../feast/core/service/SpecServiceTest.java | 188 +++++++++++++- .../feast/core/service/TestObjectFactory.java | 21 +- .../java/feast/core/util/ModelHelpers.java | 41 +++ .../scripts/test-end-to-end-batch-dataflow.sh | 5 +- .../values-end-to-end-batch-dataflow.yaml | 3 + ingestion/pom.xml | 7 + .../main/java/feast/ingestion/ImportJob.java | 155 +++++------ .../ingestion/options/ImportOptions.java | 23 +- .../transform/FeatureRowToStoreAllocator.java | 93 +++++++ .../ProcessAndValidateFeatureRows.java | 22 +- .../transform/fn/ValidateFeatureRowDoFn.java | 25 +- .../specs/FilterRelevantFunction.java | 52 ++++ .../specs/KafkaRecordToFeatureSetSpec.java | 53 ++++ .../transform/specs/ReadFeatureSetSpecs.java | 107 ++++++++ .../specs/WriteFeatureSetSpecAck.java | 85 ++++++ .../java/feast/ingestion/utils/SpecUtil.java | 97 ++++--- .../java/feast/ingestion/ImportJobTest.java | 67 +++-- .../FeatureRowToStoreAllocatorTest.java | 99 +++++++ .../ProcessAndValidateFeatureRowsTest.java | 26 +- .../specs/FeatureSetSpecReadAndWriteTest.java | 244 ++++++++++++++++++ .../transform/specs/FilterRelevantTest.java | 109 ++++++++ .../src/test/java/feast/test/TestUtil.java | 64 +++-- protos/feast/core/FeatureSet.proto | 9 + protos/feast/core/IngestionJob.proto | 15 ++ 46 files changed, 1981 insertions(+), 420 deletions(-) create mode 100644 core/src/main/java/feast/core/model/FeatureSetJobStatus.java create mode 100644 core/src/main/java/feast/core/util/KafkaSerialization.java delete mode 100644 core/src/main/resources/application-override.yaml create mode 100644 core/src/test/java/feast/core/util/ModelHelpers.java create mode 100644 ingestion/src/main/java/feast/ingestion/transform/FeatureRowToStoreAllocator.java create mode 100644 ingestion/src/main/java/feast/ingestion/transform/specs/FilterRelevantFunction.java create mode 100644 ingestion/src/main/java/feast/ingestion/transform/specs/KafkaRecordToFeatureSetSpec.java create mode 100644 ingestion/src/main/java/feast/ingestion/transform/specs/ReadFeatureSetSpecs.java create mode 100644 ingestion/src/main/java/feast/ingestion/transform/specs/WriteFeatureSetSpecAck.java create mode 100644 ingestion/src/test/java/feast/ingestion/transform/FeatureRowToStoreAllocatorTest.java create mode 100644 ingestion/src/test/java/feast/ingestion/transform/specs/FeatureSetSpecReadAndWriteTest.java create mode 100644 ingestion/src/test/java/feast/ingestion/transform/specs/FilterRelevantTest.java diff --git a/core/pom.xml b/core/pom.xml index f28a1fa28ba..81dcb144de7 100644 --- a/core/pom.xml +++ b/core/pom.xml @@ -171,8 +171,8 @@ - org.apache.kafka - kafka-clients + org.springframework.kafka + spring-kafka diff --git a/core/src/main/java/feast/core/config/FeastProperties.java b/core/src/main/java/feast/core/config/FeastProperties.java index 6dad2782429..b508de76e5d 100644 --- a/core/src/main/java/feast/core/config/FeastProperties.java +++ b/core/src/main/java/feast/core/config/FeastProperties.java @@ -139,6 +139,10 @@ public static class StreamProperties { /* Feature stream options */ @NotNull private FeatureStreamOptions options; + /* FeatureSetSpec stream options - communication channel between SpecService and IngestionJob + * to update Spec inside job w/o restart */ + @NotNull private FeatureSetSpecStreamProperties specsOptions; + /** Feature stream options */ @Getter @Setter @@ -159,6 +163,16 @@ public static class FeatureStreamOptions { /* Number of Kafka partitions to to use for managed feature stream. */ @Positive private int partitions = 1; } + + @Getter + @Setter + public static class FeatureSetSpecStreamProperties { + /* Kafka topic to send feature set spec to ingestion streaming job */ + @NotBlank private String specsTopic = "feast-feature-set-specs"; + + /* Kafka topic to receive acknowledgment from ingestion job on successful processing of new specs */ + @NotBlank private String specsAckTopic = "feast-feature-set-specs-ack"; + } } /** Feast population job metrics */ diff --git a/core/src/main/java/feast/core/config/FeatureStreamConfig.java b/core/src/main/java/feast/core/config/FeatureStreamConfig.java index c1982604c31..40647bb2e24 100644 --- a/core/src/main/java/feast/core/config/FeatureStreamConfig.java +++ b/core/src/main/java/feast/core/config/FeatureStreamConfig.java @@ -16,30 +16,119 @@ */ package feast.core.config; -import com.google.common.base.Strings; import feast.core.config.FeastProperties.StreamProperties; import feast.core.model.Source; +import feast.core.util.KafkaSerialization; +import feast.proto.core.FeatureSetProto; +import feast.proto.core.IngestionJobProto; import feast.proto.core.SourceProto.KafkaSourceConfig; import feast.proto.core.SourceProto.SourceType; -import java.util.Collections; import java.util.HashMap; import java.util.Map; -import java.util.concurrent.ExecutionException; import lombok.extern.slf4j.Slf4j; -import org.apache.kafka.clients.admin.AdminClient; import org.apache.kafka.clients.admin.AdminClientConfig; -import org.apache.kafka.clients.admin.CreateTopicsResult; import org.apache.kafka.clients.admin.NewTopic; -import org.apache.kafka.common.errors.TopicExistsException; +import org.apache.kafka.clients.consumer.ConsumerConfig; +import org.apache.kafka.clients.producer.ProducerConfig; +import org.apache.kafka.common.config.TopicConfig; +import org.apache.kafka.common.serialization.StringDeserializer; +import org.apache.kafka.common.serialization.StringSerializer; import org.springframework.beans.factory.annotation.Autowired; import org.springframework.context.annotation.Bean; import org.springframework.context.annotation.Configuration; +import org.springframework.kafka.core.*; @Slf4j @Configuration public class FeatureStreamConfig { String DEFAULT_KAFKA_REQUEST_TIMEOUT_MS_CONFIG = "15000"; + int DEFAULT_SPECS_TOPIC_PARTITIONING = 1; + short DEFAULT_SPECS_TOPIC_REPLICATION = 3; + + @Bean + public KafkaAdmin admin(FeastProperties feastProperties) { + String bootstrapServers = feastProperties.getStream().getOptions().getBootstrapServers(); + + Map configs = new HashMap<>(); + configs.put(AdminClientConfig.BOOTSTRAP_SERVERS_CONFIG, bootstrapServers); + configs.put( + AdminClientConfig.REQUEST_TIMEOUT_MS_CONFIG, DEFAULT_KAFKA_REQUEST_TIMEOUT_MS_CONFIG); + return new KafkaAdmin(configs); + } + + @Bean + public NewTopic featureRowsTopic(FeastProperties feastProperties) { + StreamProperties streamProperties = feastProperties.getStream(); + + return new NewTopic( + streamProperties.getOptions().getTopic(), + streamProperties.getOptions().getPartitions(), + streamProperties.getOptions().getReplicationFactor()); + } + + @Bean + public NewTopic featureSetSpecsTopic(FeastProperties feastProperties) { + StreamProperties streamProperties = feastProperties.getStream(); + Map configs = new HashMap<>(); + configs.put(TopicConfig.CLEANUP_POLICY_CONFIG, TopicConfig.CLEANUP_POLICY_COMPACT); + + NewTopic topic = + new NewTopic( + streamProperties.getSpecsOptions().getSpecsTopic(), + DEFAULT_SPECS_TOPIC_PARTITIONING, + DEFAULT_SPECS_TOPIC_REPLICATION); + + topic.configs(configs); + return topic; + } + + @Bean + public NewTopic featureSetSpecsAckTopic(FeastProperties feastProperties) { + StreamProperties streamProperties = feastProperties.getStream(); + + return new NewTopic( + streamProperties.getSpecsOptions().getSpecsAckTopic(), + DEFAULT_SPECS_TOPIC_PARTITIONING, + (short) 1); + } + + @Bean + public KafkaTemplate specKafkaTemplate( + FeastProperties feastProperties) { + StreamProperties streamProperties = feastProperties.getStream(); + Map props = new HashMap<>(); + + props.put( + ProducerConfig.BOOTSTRAP_SERVERS_CONFIG, + streamProperties.getOptions().getBootstrapServers()); + + KafkaTemplate t = + new KafkaTemplate<>( + new DefaultKafkaProducerFactory<>( + props, new StringSerializer(), new KafkaSerialization.ProtoSerializer<>())); + t.setDefaultTopic(streamProperties.getSpecsOptions().getSpecsTopic()); + return t; + } + + @Bean + public ConsumerFactory ackConsumerFactory(FeastProperties feastProperties) { + StreamProperties streamProperties = feastProperties.getStream(); + Map props = new HashMap<>(); + + props.put( + ConsumerConfig.BOOTSTRAP_SERVERS_CONFIG, + streamProperties.getOptions().getBootstrapServers()); + props.put( + ConsumerConfig.GROUP_ID_CONFIG, + String.format( + "core-service-%s", FeatureStreamConfig.class.getPackage().getImplementationVersion())); + + return new DefaultKafkaConsumerFactory<>( + props, + new StringDeserializer(), + new KafkaSerialization.ProtoDeserializer<>(IngestionJobProto.FeatureSetSpecAck.parser())); + } @Autowired @Bean @@ -50,31 +139,7 @@ public Source getDefaultSource(FeastProperties feastProperties) { case KAFKA: String bootstrapServers = streamProperties.getOptions().getBootstrapServers(); String topicName = streamProperties.getOptions().getTopic(); - Map map = new HashMap<>(); - map.put(AdminClientConfig.BOOTSTRAP_SERVERS_CONFIG, bootstrapServers); - map.put( - AdminClientConfig.REQUEST_TIMEOUT_MS_CONFIG, DEFAULT_KAFKA_REQUEST_TIMEOUT_MS_CONFIG); - AdminClient client = AdminClient.create(map); - - NewTopic newTopic = - new NewTopic( - topicName, - streamProperties.getOptions().getPartitions(), - streamProperties.getOptions().getReplicationFactor()); - CreateTopicsResult createTopicsResult = - client.createTopics(Collections.singleton(newTopic)); - try { - createTopicsResult.values().get(topicName).get(); - } catch (InterruptedException | ExecutionException e) { - if (e.getCause().getClass().equals(TopicExistsException.class)) { - log.warn( - Strings.lenientFormat( - "Unable to create topic %s in the feature stream, topic already exists, using existing topic.", - topicName)); - } else { - throw new RuntimeException(e.getMessage(), e); - } - } + KafkaSourceConfig sourceConfig = KafkaSourceConfig.newBuilder() .setBootstrapServers(bootstrapServers) diff --git a/core/src/main/java/feast/core/config/JobConfig.java b/core/src/main/java/feast/core/config/JobConfig.java index 30023de0641..fd0dc6ef5cf 100644 --- a/core/src/main/java/feast/core/config/JobConfig.java +++ b/core/src/main/java/feast/core/config/JobConfig.java @@ -24,8 +24,10 @@ import feast.core.job.dataflow.DataflowJobManager; import feast.core.job.direct.DirectJobRegistry; import feast.core.job.direct.DirectRunnerJobManager; +import feast.proto.core.IngestionJobProto; import feast.proto.core.RunnerProto.DataflowRunnerConfigOptions; import feast.proto.core.RunnerProto.DirectRunnerConfigOptions; +import feast.proto.core.SourceProto; import java.util.Map; import lombok.extern.slf4j.Slf4j; import org.springframework.beans.factory.annotation.Autowired; @@ -38,6 +40,30 @@ public class JobConfig { private final Gson gson = new Gson(); + /** + * Create SpecsStreamingUpdateConfig, which is used to set up communications (bi-directional + * channel) to send new FeatureSetSpec to IngestionJob and receive acknowledgments. + * + * @param feastProperties feast config properties + */ + @Bean + public IngestionJobProto.SpecsStreamingUpdateConfig createSpecsStreamingUpdateConfig( + FeastProperties feastProperties) { + FeastProperties.StreamProperties streamProperties = feastProperties.getStream(); + + return IngestionJobProto.SpecsStreamingUpdateConfig.newBuilder() + .setSource( + SourceProto.KafkaSourceConfig.newBuilder() + .setBootstrapServers(streamProperties.getOptions().getBootstrapServers()) + .setTopic(streamProperties.getSpecsOptions().getSpecsTopic()) + .build()) + .setAck( + SourceProto.KafkaSourceConfig.newBuilder() + .setBootstrapServers(streamProperties.getOptions().getBootstrapServers()) + .setTopic(streamProperties.getSpecsOptions().getSpecsAckTopic())) + .build(); + } + /** * Get a JobManager according to the runner type and Dataflow configuration. * @@ -45,7 +71,9 @@ public class JobConfig { */ @Bean @Autowired - public JobManager getJobManager(FeastProperties feastProperties) + public JobManager getJobManager( + FeastProperties feastProperties, + IngestionJobProto.SpecsStreamingUpdateConfig specsStreamingUpdateConfig) throws InvalidProtocolBufferException { JobProperties jobProperties = feastProperties.getJobs(); @@ -60,13 +88,17 @@ public JobManager getJobManager(FeastProperties feastProperties) DataflowRunnerConfigOptions.Builder dataflowRunnerConfigOptions = DataflowRunnerConfigOptions.newBuilder(); JsonFormat.parser().merge(configJson, dataflowRunnerConfigOptions); - return new DataflowJobManager(dataflowRunnerConfigOptions.build(), metrics); + return new DataflowJobManager( + dataflowRunnerConfigOptions.build(), metrics, specsStreamingUpdateConfig); case DIRECT: DirectRunnerConfigOptions.Builder directRunnerConfigOptions = DirectRunnerConfigOptions.newBuilder(); JsonFormat.parser().merge(configJson, directRunnerConfigOptions); return new DirectRunnerJobManager( - directRunnerConfigOptions.build(), new DirectJobRegistry(), metrics); + directRunnerConfigOptions.build(), + new DirectJobRegistry(), + metrics, + specsStreamingUpdateConfig); default: throw new IllegalArgumentException("Unsupported runner: " + runner); } diff --git a/core/src/main/java/feast/core/dao/JobRepository.java b/core/src/main/java/feast/core/dao/JobRepository.java index c61f3eacc05..58af8c7ec5e 100644 --- a/core/src/main/java/feast/core/dao/JobRepository.java +++ b/core/src/main/java/feast/core/dao/JobRepository.java @@ -16,10 +16,8 @@ */ package feast.core.dao; -import feast.core.model.FeatureSet; +import feast.core.model.FeatureSetJobStatus; import feast.core.model.Job; -import feast.core.model.JobStatus; -import java.util.Collection; import java.util.List; import org.springframework.data.jpa.repository.JpaRepository; import org.springframework.stereotype.Repository; @@ -27,13 +25,11 @@ /** JPA repository supplying Job objects keyed by ID. */ @Repository public interface JobRepository extends JpaRepository { - List findByStatusNotIn(Collection statuses); - List findBySourceIdAndStoreNameOrderByLastUpdatedDesc(String sourceId, String storeName); // find jobs by feast store name List findByStoreName(String storeName); // find jobs by featureset - List findByFeatureSetsIn(List featureSets); + List findByFeatureSetJobStatusesIn(List featureSetsJobStatuses); } diff --git a/core/src/main/java/feast/core/job/JobUpdateTask.java b/core/src/main/java/feast/core/job/JobUpdateTask.java index 0c79bb620db..9abdfa3e455 100644 --- a/core/src/main/java/feast/core/job/JobUpdateTask.java +++ b/core/src/main/java/feast/core/job/JobUpdateTask.java @@ -20,12 +20,8 @@ import feast.core.log.Action; import feast.core.log.AuditLogger; import feast.core.log.Resource; -import feast.core.model.FeatureSet; -import feast.core.model.Job; -import feast.core.model.JobStatus; -import feast.core.model.Source; -import feast.core.model.Store; -import feast.proto.core.FeatureSetProto.FeatureSetStatus; +import feast.core.model.*; +import feast.proto.core.FeatureSetProto; import java.time.Instant; import java.util.List; import java.util.Optional; @@ -103,16 +99,12 @@ public Job call() { } boolean requiresUpdate(Job job) { - // If set of feature sets has changed - if (!Sets.newHashSet(featureSets).equals(Sets.newHashSet(job.getFeatureSets()))) { + // if store subscriptions have changed + if (!Sets.newHashSet(store.getSubscriptions()) + .equals(Sets.newHashSet(job.getStore().getSubscriptions()))) { return true; } - // If any of the incoming feature sets were updated - for (FeatureSet featureSet : featureSets) { - if (featureSet.getStatus() == FeatureSetStatus.STATUS_PENDING) { - return true; - } - } + return false; } @@ -123,10 +115,15 @@ private Job createJob() { /** Start or update the job to ingest data to the sink. */ private Job startJob(String jobId) { + Job job = new Job(); + job.setId(jobId); + job.setRunner(jobManager.getRunnerType()); + job.setSource(source); + job.setStore(store); + job.setStatus(JobStatus.PENDING); + + updateFeatureSets(job); - Job job = - new Job( - jobId, "", jobManager.getRunnerType(), source, store, featureSets, JobStatus.PENDING); try { logAudit(Action.SUBMIT, job, "Building graph and submitting to %s", runnerName); @@ -151,9 +148,19 @@ private Job startJob(String jobId) { } } + private void updateFeatureSets(Job job) { + for (FeatureSet fs : featureSets) { + FeatureSetJobStatus status = new FeatureSetJobStatus(); + status.setFeatureSet(fs); + status.setJob(job); + status.setDeliveryStatus(FeatureSetProto.FeatureSetJobDeliveryStatus.STATUS_IN_PROGRESS); + job.getFeatureSetJobStatuses().add(status); + } + } + /** Update the given job */ private Job updateJob(Job job) { - job.setFeatureSets(featureSets); + updateFeatureSets(job); job.setStore(store); logAudit(Action.UPDATE, job, "Updating job %s for runner %s", job.getId(), runnerName); return jobManager.updateJob(job); diff --git a/core/src/main/java/feast/core/job/dataflow/DataflowJobManager.java b/core/src/main/java/feast/core/job/dataflow/DataflowJobManager.java index 50a23224078..181b2339b3a 100644 --- a/core/src/main/java/feast/core/job/dataflow/DataflowJobManager.java +++ b/core/src/main/java/feast/core/job/dataflow/DataflowJobManager.java @@ -31,21 +31,16 @@ import feast.core.exception.JobExecutionException; import feast.core.job.JobManager; import feast.core.job.Runner; -import feast.core.job.option.FeatureSetJsonByteConverter; import feast.core.model.*; import feast.ingestion.ImportJob; -import feast.ingestion.options.BZip2Compressor; import feast.ingestion.options.ImportOptions; -import feast.ingestion.options.OptionCompressor; -import feast.proto.core.FeatureSetProto; +import feast.proto.core.IngestionJobProto; import feast.proto.core.RunnerProto.DataflowRunnerConfigOptions; import feast.proto.core.SourceProto; import feast.proto.core.StoreProto; import java.io.IOException; import java.security.GeneralSecurityException; -import java.util.ArrayList; import java.util.Collections; -import java.util.List; import lombok.extern.slf4j.Slf4j; import org.apache.beam.runners.dataflow.DataflowPipelineJob; import org.apache.beam.runners.dataflow.DataflowRunner; @@ -62,15 +57,19 @@ public class DataflowJobManager implements JobManager { private final Dataflow dataflow; private final DataflowRunnerConfig defaultOptions; private final MetricsProperties metrics; + private final IngestionJobProto.SpecsStreamingUpdateConfig specsStreamingUpdateConfig; public DataflowJobManager( - DataflowRunnerConfigOptions runnerConfigOptions, MetricsProperties metricsProperties) { - this(runnerConfigOptions, metricsProperties, getGoogleCredential()); + DataflowRunnerConfigOptions runnerConfigOptions, + MetricsProperties metricsProperties, + IngestionJobProto.SpecsStreamingUpdateConfig specsStreamingUpdateConfig) { + this(runnerConfigOptions, metricsProperties, specsStreamingUpdateConfig, getGoogleCredential()); } public DataflowJobManager( DataflowRunnerConfigOptions runnerConfigOptions, MetricsProperties metricsProperties, + IngestionJobProto.SpecsStreamingUpdateConfig specsStreamingUpdateConfig, Credential credential) { defaultOptions = new DataflowRunnerConfig(runnerConfigOptions); @@ -91,6 +90,7 @@ public DataflowJobManager( this.metrics = metricsProperties; this.projectId = defaultOptions.getProject(); this.location = defaultOptions.getRegion(); + this.specsStreamingUpdateConfig = specsStreamingUpdateConfig; } private static Credential getGoogleCredential() { @@ -112,17 +112,9 @@ public Runner getRunnerType() { @Override public Job startJob(Job job) { try { - List featureSetProtos = new ArrayList<>(); - for (FeatureSet featureSet : job.getFeatureSets()) { - featureSetProtos.add(featureSet.toProto()); - } String extId = submitDataflowJob( - job.getId(), - featureSetProtos, - job.getSource().toProto(), - job.getStore().toProto(), - false); + job.getId(), job.getSource().toProto(), job.getStore().toProto(), false); job.setExtId(extId); return job; @@ -223,13 +215,9 @@ public JobStatus getJobStatus(Job job) { } private String submitDataflowJob( - String jobName, - List featureSetProtos, - SourceProto.Source source, - StoreProto.Store sink, - boolean update) { + String jobName, SourceProto.Source source, StoreProto.Store sink, boolean update) { try { - ImportOptions pipelineOptions = getPipelineOptions(jobName, featureSetProtos, sink, update); + ImportOptions pipelineOptions = getPipelineOptions(jobName, source, sink, update); DataflowPipelineJob pipelineResult = runPipeline(pipelineOptions); String jobId = waitForJobToRun(pipelineResult); return jobId; @@ -240,19 +228,17 @@ private String submitDataflowJob( } private ImportOptions getPipelineOptions( - String jobName, - List featureSets, - StoreProto.Store sink, - boolean update) + String jobName, SourceProto.Source source, StoreProto.Store sink, boolean update) throws IOException, IllegalAccessException { ImportOptions pipelineOptions = PipelineOptionsFactory.fromArgs(defaultOptions.toArgs()).as(ImportOptions.class); - OptionCompressor> featureSetJsonCompressor = - new BZip2Compressor<>(new FeatureSetJsonByteConverter()); + JsonFormat.Printer jsonPrinter = JsonFormat.printer(); - pipelineOptions.setFeatureSetJson(featureSetJsonCompressor.compress(featureSets)); - pipelineOptions.setStoreJson(Collections.singletonList(JsonFormat.printer().print(sink))); + pipelineOptions.setSpecsStreamingUpdateConfigJson( + jsonPrinter.print(specsStreamingUpdateConfig)); + pipelineOptions.setSourceJson(jsonPrinter.print(source)); + pipelineOptions.setStoreJson(Collections.singletonList(jsonPrinter.print(sink))); pipelineOptions.setProject(projectId); pipelineOptions.setDefaultFeastProject(Project.DEFAULT_NAME); pipelineOptions.setUpdate(update); diff --git a/core/src/main/java/feast/core/job/direct/DirectRunnerJobManager.java b/core/src/main/java/feast/core/job/direct/DirectRunnerJobManager.java index 715adbdd430..242ac43496e 100644 --- a/core/src/main/java/feast/core/job/direct/DirectRunnerJobManager.java +++ b/core/src/main/java/feast/core/job/direct/DirectRunnerJobManager.java @@ -22,22 +22,15 @@ import feast.core.exception.JobExecutionException; import feast.core.job.JobManager; import feast.core.job.Runner; -import feast.core.job.option.FeatureSetJsonByteConverter; -import feast.core.model.FeatureSet; -import feast.core.model.Job; -import feast.core.model.JobStatus; -import feast.core.model.Project; +import feast.core.model.*; import feast.ingestion.ImportJob; -import feast.ingestion.options.BZip2Compressor; import feast.ingestion.options.ImportOptions; -import feast.ingestion.options.OptionCompressor; -import feast.proto.core.FeatureSetProto; +import feast.proto.core.IngestionJobProto; import feast.proto.core.RunnerProto.DirectRunnerConfigOptions; +import feast.proto.core.SourceProto; import feast.proto.core.StoreProto; import java.io.IOException; -import java.util.ArrayList; import java.util.Collections; -import java.util.List; import lombok.extern.slf4j.Slf4j; import org.apache.beam.runners.direct.DirectRunner; import org.apache.beam.sdk.PipelineResult; @@ -51,14 +44,17 @@ public class DirectRunnerJobManager implements JobManager { private DirectRunnerConfig defaultOptions; private final DirectJobRegistry jobs; private MetricsProperties metrics; + private final IngestionJobProto.SpecsStreamingUpdateConfig specsStreamingUpdateConfig; public DirectRunnerJobManager( DirectRunnerConfigOptions directRunnerConfigOptions, DirectJobRegistry jobs, - MetricsProperties metricsProperties) { + MetricsProperties metricsProperties, + IngestionJobProto.SpecsStreamingUpdateConfig specsStreamingUpdateConfig) { this.defaultOptions = new DirectRunnerConfig(directRunnerConfigOptions); this.jobs = jobs; this.metrics = metricsProperties; + this.specsStreamingUpdateConfig = specsStreamingUpdateConfig; } @Override @@ -74,12 +70,8 @@ public Runner getRunnerType() { @Override public Job startJob(Job job) { try { - List featureSetProtos = new ArrayList<>(); - for (FeatureSet featureSet : job.getFeatureSets()) { - featureSetProtos.add(featureSet.toProto()); - } ImportOptions pipelineOptions = - getPipelineOptions(job.getId(), featureSetProtos, job.getStore().toProto()); + getPipelineOptions(job.getId(), job.getSource().toProto(), job.getStore().toProto()); PipelineResult pipelineResult = runPipeline(pipelineOptions); DirectJob directJob = new DirectJob(job.getId(), pipelineResult); jobs.add(directJob); @@ -93,15 +85,14 @@ public Job startJob(Job job) { } private ImportOptions getPipelineOptions( - String jobName, List featureSets, StoreProto.Store sink) + String jobName, SourceProto.Source source, StoreProto.Store sink) throws IOException, IllegalAccessException { ImportOptions pipelineOptions = PipelineOptionsFactory.fromArgs(defaultOptions.toArgs()).as(ImportOptions.class); - OptionCompressor> featureSetJsonCompressor = - new BZip2Compressor<>(new FeatureSetJsonByteConverter()); - - pipelineOptions.setFeatureSetJson(featureSetJsonCompressor.compress(featureSets)); + pipelineOptions.setSpecsStreamingUpdateConfigJson( + JsonFormat.printer().print(specsStreamingUpdateConfig)); + pipelineOptions.setSourceJson(JsonFormat.printer().print(source)); pipelineOptions.setJobName(jobName); pipelineOptions.setStoreJson(Collections.singletonList(JsonFormat.printer().print(sink))); pipelineOptions.setRunner(DirectRunner.class); diff --git a/core/src/main/java/feast/core/model/FeatureSet.java b/core/src/main/java/feast/core/model/FeatureSet.java index f7b2dc7cd49..e1b50098b90 100644 --- a/core/src/main/java/feast/core/model/FeatureSet.java +++ b/core/src/main/java/feast/core/model/FeatureSet.java @@ -29,6 +29,7 @@ import lombok.Getter; import lombok.Setter; import org.apache.commons.lang3.builder.HashCodeBuilder; +import org.springframework.data.util.Pair; import org.tensorflow.metadata.v0.*; @Getter @@ -84,6 +85,12 @@ public class FeatureSet extends AbstractTimestampEntity { @Column(name = "labels", columnDefinition = "text") private String labels; + @Column(name = "version", columnDefinition = "integer default 0") + private int version; + + @OneToMany(mappedBy = "featureSet", cascade = CascadeType.ALL) + private Set jobStatuses = new HashSet<>(); + public FeatureSet() { super(); } @@ -125,6 +132,29 @@ public void setProject(Project project) { this.project = project; } + public String getReference() { + return String.format("%s/%s", getProjectName(), getName()); + } + + /** @return Pair <ProjectName, FeatureSetName> */ + public static Pair parseReference(String reference) { + String[] split = reference.split("/", 2); + if (split.length == 1) { + return Pair.of(Project.DEFAULT_NAME, split[0]); + } + + if (split.length > 2) { + throw new RuntimeException( + "FeatureSet reference must have the format /"); + } + + return Pair.of(split[0], split[1]); + } + + public int incVersion() { + return ++version; + } + public static FeatureSet fromProto(FeatureSetProto.FeatureSet featureSetProto) { FeatureSetSpec featureSetSpec = featureSetProto.getSpec(); Source source = Source.fromProto(featureSetSpec.getSource()); @@ -257,7 +287,8 @@ public FeatureSetProto.FeatureSet toProto() throws InvalidProtocolBufferExceptio .addAllEntities(entitySpecs) .addAllFeatures(featureSpecs) .putAllLabels(TypeConversion.convertJsonStringToMap(labels)) - .setSource(source.toProto()); + .setSource(source.toProto()) + .setVersion(version); return FeatureSetProto.FeatureSet.newBuilder().setMeta(meta).setSpec(spec).build(); } @@ -300,6 +331,10 @@ public boolean equals(Object obj) { return false; } + if (version != other.version) { + return false; + } + // Create a map of all fields in this feature set Map entitiesMap = new HashMap<>(); Map featuresMap = new HashMap<>(); diff --git a/core/src/main/java/feast/core/model/FeatureSetJobStatus.java b/core/src/main/java/feast/core/model/FeatureSetJobStatus.java new file mode 100644 index 00000000000..7f21c26335a --- /dev/null +++ b/core/src/main/java/feast/core/model/FeatureSetJobStatus.java @@ -0,0 +1,64 @@ +/* + * SPDX-License-Identifier: Apache-2.0 + * Copyright 2018-2020 The Feast Authors + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package feast.core.model; + +import feast.proto.core.FeatureSetProto.FeatureSetJobDeliveryStatus; +import java.io.Serializable; +import javax.persistence.*; +import javax.persistence.Entity; +import lombok.EqualsAndHashCode; +import lombok.Getter; +import lombok.Setter; + +@Getter +@Setter +@Entity +@Table( + name = "jobs_feature_sets", + indexes = { + @Index(name = "idx_jobs_feature_sets_job_id", columnList = "job_id"), + @Index(name = "idx_jobs_feature_sets_feature_sets_id", columnList = "feature_sets_id") + }) +public class FeatureSetJobStatus { + @Embeddable + @EqualsAndHashCode + public static class FeatureSetJobStatusKey implements Serializable { + public FeatureSetJobStatusKey() {} + + @Column(name = "job_id") + String jobId; + + @Column(name = "feature_sets_id") + long featureSetId; + } + + @EmbeddedId private FeatureSetJobStatusKey id = new FeatureSetJobStatusKey(); + + @ManyToOne + @MapsId("jobId") + @JoinColumn(name = "job_id") + private Job job; + + @ManyToOne + @MapsId("featureSetId") + @JoinColumn(name = "feature_sets_id") + private FeatureSet featureSet; + + @Enumerated(EnumType.STRING) + @Column(name = "delivery_status") + private FeatureSetJobDeliveryStatus deliveryStatus; +} diff --git a/core/src/main/java/feast/core/model/Job.java b/core/src/main/java/feast/core/model/Job.java index 5fce3dffbe4..ae54221b9c6 100644 --- a/core/src/main/java/feast/core/model/Job.java +++ b/core/src/main/java/feast/core/model/Job.java @@ -21,7 +21,9 @@ import feast.proto.core.FeatureSetProto; import feast.proto.core.IngestionJobProto; import java.util.ArrayList; +import java.util.HashSet; import java.util.List; +import java.util.Set; import javax.persistence.*; import javax.persistence.Entity; import lombok.AllArgsConstructor; @@ -59,17 +61,9 @@ public class Job extends AbstractTimestampEntity { @JoinColumn(name = "store_name") private Store store; - // FeatureSets populated by the job - @ManyToMany(cascade = CascadeType.ALL) - @JoinTable( - name = "jobs_feature_sets", - joinColumns = @JoinColumn(name = "job_id"), - inverseJoinColumns = @JoinColumn(name = "feature_sets_id"), - indexes = { - @Index(name = "idx_jobs_feature_sets_job_id", columnList = "job_id"), - @Index(name = "idx_jobs_feature_sets_feature_sets_id", columnList = "feature_sets_id") - }) - private List featureSets; + // FeatureSets populated by the job via intermediate FeatureSetJobStatus model + @OneToMany(mappedBy = "job", cascade = CascadeType.ALL) + private Set featureSetJobStatuses = new HashSet<>(); @Enumerated(EnumType.STRING) @Column(name = "status", length = 16) @@ -100,8 +94,8 @@ public IngestionJobProto.IngestionJob toProto() throws InvalidProtocolBufferExce // convert featuresets of job to protos List featureSetProtos = new ArrayList<>(); - for (FeatureSet featureSet : this.getFeatureSets()) { - featureSetProtos.add(featureSet.toProto()); + for (FeatureSetJobStatus featureSet : this.getFeatureSetJobStatuses()) { + featureSetProtos.add(featureSet.getFeatureSet().toProto()); } // build ingestion job proto with job data @@ -110,9 +104,9 @@ public IngestionJobProto.IngestionJob toProto() throws InvalidProtocolBufferExce .setId(this.getId()) .setExternalId(this.getExtId()) .setStatus(this.getStatus().toProto()) - .addAllFeatureSets(featureSetProtos) .setSource(this.getSource().toProto()) .setStore(this.getStore().toProto()) + .addAllFeatureSets(featureSetProtos) .build(); return ingestJob; diff --git a/core/src/main/java/feast/core/service/JobCoordinatorService.java b/core/src/main/java/feast/core/service/JobCoordinatorService.java index 90ee54ca166..69da3e507ed 100644 --- a/core/src/main/java/feast/core/service/JobCoordinatorService.java +++ b/core/src/main/java/feast/core/service/JobCoordinatorService.java @@ -23,13 +23,9 @@ import feast.core.dao.JobRepository; import feast.core.job.JobManager; import feast.core.job.JobUpdateTask; -import feast.core.model.FeatureSet; -import feast.core.model.Job; -import feast.core.model.Source; -import feast.core.model.Store; +import feast.core.model.*; import feast.proto.core.CoreServiceProto.ListStoresRequest.Filter; import feast.proto.core.CoreServiceProto.ListStoresResponse; -import feast.proto.core.FeatureSetProto.FeatureSetStatus; import feast.proto.core.StoreProto; import feast.proto.core.StoreProto.Store.Subscription; import java.util.ArrayList; @@ -121,9 +117,6 @@ public void Poll() throws InvalidProtocolBufferException { log.info("Creating/Updating {} jobs...", jobUpdateTasks.size()); startOrUpdateJobs(jobUpdateTasks); - - log.info("Updating feature set status"); - updateFeatureSetStatuses(jobUpdateTasks); } void startOrUpdateJobs(List tasks) { @@ -148,35 +141,6 @@ void startOrUpdateJobs(List tasks) { executorService.shutdown(); } - // TODO: make this more efficient - private void updateFeatureSetStatuses(List jobUpdateTasks) { - Set ready = new HashSet<>(); - Set pending = new HashSet<>(); - for (JobUpdateTask task : jobUpdateTasks) { - getJob(task.getSource(), task.getStore()) - .ifPresent( - job -> { - if (job.isRunning()) { - ready.addAll(job.getFeatureSets()); - } else { - pending.addAll(job.getFeatureSets()); - } - }); - } - ready.removeAll(pending); - ready.forEach( - fs -> { - fs.setStatus(FeatureSetStatus.STATUS_READY); - featureSetRepository.save(fs); - }); - pending.forEach( - fs -> { - fs.setStatus(FeatureSetStatus.STATUS_JOB_STARTING); - featureSetRepository.save(fs); - }); - featureSetRepository.flush(); - } - @Transactional public Optional getJob(Source source, Store store) { List jobs = diff --git a/core/src/main/java/feast/core/service/JobService.java b/core/src/main/java/feast/core/service/JobService.java index cc125305ec3..b0b7aeeb35a 100644 --- a/core/src/main/java/feast/core/service/JobService.java +++ b/core/src/main/java/feast/core/service/JobService.java @@ -122,7 +122,11 @@ public ListIngestionJobsResponse listJobs(ListIngestionJobsRequest request) .collect(Collectors.toList()); // find jobs for the matching featuresets - Collection matchingJobs = this.jobRepository.findByFeatureSetsIn(featureSets); + Collection matchingJobs = + this.jobRepository.findByFeatureSetJobStatusesIn( + featureSets.stream() + .flatMap(fs -> fs.getJobStatuses().stream()) + .collect(Collectors.toList())); List jobIds = matchingJobs.stream().map(Job::getId).collect(Collectors.toList()); matchingJobIds = this.mergeResults(matchingJobIds, jobIds); } @@ -137,6 +141,11 @@ public ListIngestionJobsResponse listJobs(ListIngestionJobsRequest request) List ingestJobs = new ArrayList<>(); for (String jobId : matchingJobIds) { Job job = this.jobRepository.findById(jobId).get(); + // job that failed on start won't be converted toProto successfully + // and they're irrelevant here + if (job.getStatus() == JobStatus.ERROR) { + continue; + } ingestJobs.add(job.toProto()); } diff --git a/core/src/main/java/feast/core/service/SpecService.java b/core/src/main/java/feast/core/service/SpecService.java index 01cd264c761..9000ee983ee 100644 --- a/core/src/main/java/feast/core/service/SpecService.java +++ b/core/src/main/java/feast/core/service/SpecService.java @@ -16,6 +16,7 @@ */ package feast.core.service; +import static feast.core.model.FeatureSet.parseReference; import static feast.core.validators.Matchers.checkValidCharacters; import static feast.core.validators.Matchers.checkValidCharactersAllowAsterisk; @@ -24,10 +25,7 @@ import feast.core.dao.ProjectRepository; import feast.core.dao.StoreRepository; import feast.core.exception.RetrievalException; -import feast.core.model.FeatureSet; -import feast.core.model.Project; -import feast.core.model.Source; -import feast.core.model.Store; +import feast.core.model.*; import feast.core.validators.FeatureSetValidator; import feast.proto.core.CoreServiceProto.ApplyFeatureSetResponse; import feast.proto.core.CoreServiceProto.ApplyFeatureSetResponse.Status; @@ -42,13 +40,19 @@ import feast.proto.core.CoreServiceProto.UpdateStoreResponse; import feast.proto.core.FeatureSetProto; import feast.proto.core.FeatureSetProto.FeatureSetStatus; +import feast.proto.core.IngestionJobProto; import feast.proto.core.SourceProto; import feast.proto.core.StoreProto; import feast.proto.core.StoreProto.Store.Subscription; import java.util.ArrayList; import java.util.List; +import java.util.concurrent.TimeUnit; import lombok.extern.slf4j.Slf4j; +import org.apache.kafka.clients.consumer.ConsumerRecord; import org.springframework.beans.factory.annotation.Autowired; +import org.springframework.data.util.Pair; +import org.springframework.kafka.annotation.KafkaListener; +import org.springframework.kafka.core.KafkaTemplate; import org.springframework.stereotype.Service; import org.springframework.transaction.annotation.Transactional; @@ -60,21 +64,26 @@ @Service public class SpecService { + private final int SPEC_PUBLISHING_TIMEOUT_SECONDS = 5; + private final FeatureSetRepository featureSetRepository; private final ProjectRepository projectRepository; private final StoreRepository storeRepository; private final Source defaultSource; + private final KafkaTemplate specPublisher; @Autowired public SpecService( FeatureSetRepository featureSetRepository, StoreRepository storeRepository, ProjectRepository projectRepository, - Source defaultSource) { + Source defaultSource, + KafkaTemplate specPublisher) { this.featureSetRepository = featureSetRepository; this.storeRepository = storeRepository; this.projectRepository = projectRepository; this.defaultSource = defaultSource; + this.specPublisher = specPublisher; } /** @@ -240,6 +249,7 @@ public ListStoresResponse listStores(ListStoresRequest.Filter filter) { * * @param newFeatureSet Feature set that will be created or updated. */ + @Transactional public ApplyFeatureSetResponse applyFeatureSet(FeatureSetProto.FeatureSet newFeatureSet) throws InvalidProtocolBufferException { // Autofill default project if not specified @@ -299,6 +309,39 @@ public ApplyFeatureSetResponse applyFeatureSet(FeatureSetProto.FeatureSet newFea status = Status.UPDATED; } + featureSet.incVersion(); + + // Sending latest version of FeatureSet to all currently running IngestionJobs (there's one + // topic for all sets). + // All related jobs would apply new FeatureSet on the fly. + // We wait for Kafka broker to ack that the message was added to topic before actually + // committing this FeatureSet. + // In case kafka doesn't respond within SPEC_PUBLISHING_TIMEOUT_SECONDS we abort current + // transaction and return error to client. + try { + specPublisher + .sendDefault(featureSet.getReference(), featureSet.toProto().getSpec()) + .get(SPEC_PUBLISHING_TIMEOUT_SECONDS, TimeUnit.SECONDS); + } catch (Exception e) { + throw io.grpc.Status.UNAVAILABLE + .withDescription( + String.format( + "Unable to publish FeatureSet to Kafka. Cause: %s", + e.getCause() != null ? e.getCause().getMessage() : "unknown")) + .withCause(e) + .asRuntimeException(); + } + + // Updating delivery status for related jobs (that are currently using this FeatureSet). + // We now set status to IN_PROGRESS, so listenAckFromJobs would be able to + // monitor delivery progress for each new version. + featureSet.getJobStatuses().stream() + .filter(s -> s.getJob().isRunning()) + .forEach( + s -> + s.setDeliveryStatus( + FeatureSetProto.FeatureSetJobDeliveryStatus.STATUS_IN_PROGRESS)); + // Persist the FeatureSet object featureSet.setStatus(FeatureSetStatus.STATUS_PENDING); project.addFeatureSet(featureSet); @@ -347,4 +390,60 @@ public UpdateStoreResponse updateStore(UpdateStoreRequest updateStoreRequest) .setStore(updateStoreRequest.getStore()) .build(); } + + /** + * Listener for ACK messages coming from IngestionJob when FeatureSetSpec is installed (in + * pipeline). + * + *

Updates FeatureSetJobStatus for respected FeatureSet (selected by reference) and Job (select + * by Id). + * + *

When all related (running) to FeatureSet jobs are updated - FeatureSet receives READY status + * + * @param record ConsumerRecord with key: FeatureSet reference and value: Ack message + */ + @KafkaListener(topics = {"${feast.stream.specsOptions.specsAckTopic}"}) + @Transactional + public void listenAckFromJobs( + ConsumerRecord record) { + String setReference = record.key(); + Pair projectAndSetName = parseReference(setReference); + FeatureSet featureSet = + featureSetRepository.findFeatureSetByNameAndProject_Name( + projectAndSetName.getSecond(), projectAndSetName.getFirst()); + if (featureSet == null) { + log.warn( + String.format("ACKListener received message for unknown FeatureSet %s", setReference)); + return; + } + + if (featureSet.getVersion() != record.value().getFeatureSetVersion()) { + log.warn( + String.format( + "ACKListener received outdated ack for %s. Current %d, Received %d", + setReference, featureSet.getVersion(), record.value().getFeatureSetVersion())); + return; + } + + featureSet.getJobStatuses().stream() + .filter(js -> js.getJob().getId().equals(record.value().getJobName())) + .findFirst() + .ifPresent( + featureSetJobStatus -> + featureSetJobStatus.setDeliveryStatus( + FeatureSetProto.FeatureSetJobDeliveryStatus.STATUS_DELIVERED)); + + boolean allDelivered = + featureSet.getJobStatuses().stream() + .filter(js -> js.getJob().isRunning()) + .allMatch( + js -> + js.getDeliveryStatus() + .equals(FeatureSetProto.FeatureSetJobDeliveryStatus.STATUS_DELIVERED)); + + if (allDelivered) { + featureSet.setStatus(FeatureSetStatus.STATUS_READY); + featureSetRepository.saveAndFlush(featureSet); + } + } } diff --git a/core/src/main/java/feast/core/util/KafkaSerialization.java b/core/src/main/java/feast/core/util/KafkaSerialization.java new file mode 100644 index 00000000000..881d8eb9cf8 --- /dev/null +++ b/core/src/main/java/feast/core/util/KafkaSerialization.java @@ -0,0 +1,68 @@ +/* + * SPDX-License-Identifier: Apache-2.0 + * Copyright 2018-2020 The Feast Authors + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package feast.core.util; + +import com.google.protobuf.GeneratedMessageV3; +import com.google.protobuf.InvalidProtocolBufferException; +import com.google.protobuf.Message; +import com.google.protobuf.Parser; +import java.io.ByteArrayOutputStream; +import java.io.IOException; +import org.apache.kafka.common.serialization.Deserializer; +import org.apache.kafka.common.serialization.Serializer; + +/* +Serializer & Deserializer implementation to write & read protobuf object from/to kafka + */ +public class KafkaSerialization { + public static class ProtoSerializer implements Serializer { + @Override + public byte[] serialize(String topic, T data) { + ByteArrayOutputStream stream = new ByteArrayOutputStream(); + try { + data.writeTo(stream); + } catch (IOException e) { + throw new RuntimeException( + String.format( + "Unable to serialize object of type %s. Reason: %s", + data.getClass().getName(), e.getCause().getMessage())); + } + + return stream.toByteArray(); + } + } + + public static class ProtoDeserializer implements Deserializer { + private Parser parser; + + public ProtoDeserializer(Parser parser) { + this.parser = parser; + } + + @Override + public T deserialize(String topic, byte[] data) { + try { + return parser.parseFrom(data); + } catch (InvalidProtocolBufferException e) { + throw new RuntimeException( + String.format( + "Unable to deserialize object from topic %s. Reason: %s", + topic, e.getCause().getMessage())); + } + } + } +} diff --git a/core/src/main/resources/application-override.yaml b/core/src/main/resources/application-override.yaml deleted file mode 100644 index e69de29bb2d..00000000000 diff --git a/core/src/main/resources/application.yml b/core/src/main/resources/application.yml index 5dc5698e889..9f02ed302db 100644 --- a/core/src/main/resources/application.yml +++ b/core/src/main/resources/application.yml @@ -78,6 +78,10 @@ feast: replicationFactor: 1 partitions: 1 + specsOptions: + specsTopic: feast-specs + specsAckTopic: feast-specs-ack + spring: jpa: properties.hibernate: diff --git a/core/src/test/java/feast/core/job/JobUpdateTaskTest.java b/core/src/test/java/feast/core/job/JobUpdateTaskTest.java index d1826738019..fc633ba15f8 100644 --- a/core/src/test/java/feast/core/job/JobUpdateTaskTest.java +++ b/core/src/test/java/feast/core/job/JobUpdateTaskTest.java @@ -24,11 +24,8 @@ import static org.mockito.Mockito.when; import static org.mockito.MockitoAnnotations.initMocks; -import feast.core.model.FeatureSet; -import feast.core.model.Job; -import feast.core.model.JobStatus; -import feast.core.model.Source; -import feast.core.model.Store; +import feast.core.model.*; +import feast.core.util.ModelHelpers; import feast.proto.core.FeatureSetProto; import feast.proto.core.FeatureSetProto.FeatureSetMeta; import feast.proto.core.FeatureSetProto.FeatureSetSpec; @@ -92,7 +89,14 @@ public void setUp() { } Job makeJob(String extId, List featureSets, JobStatus status) { - return new Job("job", extId, RUNNER, source, store, featureSets, status); + return new Job( + "job", + extId, + RUNNER, + source, + store, + ModelHelpers.makeFeatureSetJobStatus(featureSets), + status); } JobUpdateTask makeTask(List featureSets, Optional currentJob) { diff --git a/core/src/test/java/feast/core/job/dataflow/DataflowJobManagerTest.java b/core/src/test/java/feast/core/job/dataflow/DataflowJobManagerTest.java index ea9caa91ff6..50812882029 100644 --- a/core/src/test/java/feast/core/job/dataflow/DataflowJobManagerTest.java +++ b/core/src/test/java/feast/core/job/dataflow/DataflowJobManagerTest.java @@ -26,20 +26,19 @@ import com.google.api.client.googleapis.testing.auth.oauth2.MockGoogleCredential; import com.google.api.services.dataflow.Dataflow; import com.google.common.collect.Lists; +import com.google.common.collect.Sets; import com.google.protobuf.Duration; import com.google.protobuf.util.JsonFormat; import com.google.protobuf.util.JsonFormat.Printer; import feast.core.config.FeastProperties.MetricsProperties; import feast.core.exception.JobExecutionException; import feast.core.job.Runner; -import feast.core.job.option.FeatureSetJsonByteConverter; import feast.core.model.*; -import feast.ingestion.options.BZip2Compressor; import feast.ingestion.options.ImportOptions; -import feast.ingestion.options.OptionCompressor; import feast.proto.core.FeatureSetProto; import feast.proto.core.FeatureSetProto.FeatureSetMeta; import feast.proto.core.FeatureSetProto.FeatureSetSpec; +import feast.proto.core.IngestionJobProto; import feast.proto.core.RunnerProto.DataflowRunnerConfigOptions; import feast.proto.core.RunnerProto.DataflowRunnerConfigOptions.Builder; import feast.proto.core.SourceProto; @@ -50,8 +49,6 @@ import feast.proto.core.StoreProto.Store.StoreType; import feast.proto.core.StoreProto.Store.Subscription; import java.io.IOException; -import java.util.Collections; -import java.util.List; import org.apache.beam.runners.dataflow.DataflowPipelineJob; import org.apache.beam.runners.dataflow.DataflowRunner; import org.apache.beam.sdk.PipelineResult.State; @@ -71,6 +68,7 @@ public class DataflowJobManagerTest { @Mock private Dataflow dataflow; private DataflowRunnerConfigOptions defaults; + private IngestionJobProto.SpecsStreamingUpdateConfig specsStreamingUpdateConfig; private DataflowJobManager dfJobManager; @Before @@ -94,7 +92,17 @@ public void setUp() { e.printStackTrace(); } - dfJobManager = new DataflowJobManager(defaults, metricsProperties, credential); + specsStreamingUpdateConfig = + IngestionJobProto.SpecsStreamingUpdateConfig.newBuilder() + .setSource( + KafkaSourceConfig.newBuilder() + .setTopic("specs_topic") + .setBootstrapServers("servers:9092") + .build()) + .build(); + + dfJobManager = + new DataflowJobManager(defaults, metricsProperties, specsStreamingUpdateConfig, credential); dfJobManager = spy(dfJobManager); } @@ -142,11 +150,7 @@ public void shouldStartJobWithCorrectPipelineOptions() throws IOException { expectedPipelineOptions.setLabels(defaults.getLabelsMap()); expectedPipelineOptions.setJobName(jobName); expectedPipelineOptions.setStoreJson(Lists.newArrayList(printer.print(store))); - - OptionCompressor> featureSetJsonCompressor = - new BZip2Compressor<>(new FeatureSetJsonByteConverter()); - expectedPipelineOptions.setFeatureSetJson( - featureSetJsonCompressor.compress(Collections.singletonList(featureSet))); + expectedPipelineOptions.setSourceJson(printer.print(source)); ArgumentCaptor captor = ArgumentCaptor.forClass(ImportOptions.class); @@ -155,6 +159,10 @@ public void shouldStartJobWithCorrectPipelineOptions() throws IOException { when(mockPipelineResult.getJobId()).thenReturn(expectedExtJobId); doReturn(mockPipelineResult).when(dfJobManager).runPipeline(any()); + + FeatureSetJobStatus featureSetJobStatus = new FeatureSetJobStatus(); + featureSetJobStatus.setFeatureSet(FeatureSet.fromProto(featureSet)); + Job job = new Job( jobName, @@ -162,7 +170,7 @@ public void shouldStartJobWithCorrectPipelineOptions() throws IOException { Runner.DATAFLOW, Source.fromProto(source), Store.fromProto(store), - Lists.newArrayList(FeatureSet.fromProto(featureSet)), + Sets.newHashSet(featureSetJobStatus), JobStatus.PENDING); Job actual = dfJobManager.startJob(job); @@ -184,9 +192,6 @@ public void shouldStartJobWithCorrectPipelineOptions() throws IOException { // Assume the files that are staged are correct expectedPipelineOptions.setFilesToStage(actualPipelineOptions.getFilesToStage()); - assertThat( - actualPipelineOptions.getFeatureSetJson(), - equalTo(expectedPipelineOptions.getFeatureSetJson())); assertThat( actualPipelineOptions.getDeadLetterTableSpec(), equalTo(expectedPipelineOptions.getDeadLetterTableSpec())); @@ -197,6 +202,11 @@ public void shouldStartJobWithCorrectPipelineOptions() throws IOException { equalTo(expectedPipelineOptions.getMetricsExporterType())); assertThat( actualPipelineOptions.getStoreJson(), equalTo(expectedPipelineOptions.getStoreJson())); + assertThat( + actualPipelineOptions.getSourceJson(), equalTo(expectedPipelineOptions.getSourceJson())); + assertThat( + actualPipelineOptions.getSpecsStreamingUpdateConfigJson(), + equalTo(printer.print(specsStreamingUpdateConfig))); assertThat(actual.getExtId(), equalTo(expectedExtJobId)); } @@ -231,6 +241,9 @@ public void shouldThrowExceptionWhenJobStateTerminal() throws IOException { doReturn(mockPipelineResult).when(dfJobManager).runPipeline(any()); + FeatureSetJobStatus featureSetJobStatus = new FeatureSetJobStatus(); + featureSetJobStatus.setFeatureSet(FeatureSet.fromProto(featureSet)); + Job job = new Job( "job", @@ -238,7 +251,7 @@ public void shouldThrowExceptionWhenJobStateTerminal() throws IOException { Runner.DATAFLOW, Source.fromProto(source), Store.fromProto(store), - Lists.newArrayList(FeatureSet.fromProto(featureSet)), + Sets.newHashSet(featureSetJobStatus), JobStatus.PENDING); expectedException.expect(JobExecutionException.class); diff --git a/core/src/test/java/feast/core/job/direct/DirectRunnerJobManagerTest.java b/core/src/test/java/feast/core/job/direct/DirectRunnerJobManagerTest.java index 0128f5aa0b3..0e797dc64d4 100644 --- a/core/src/test/java/feast/core/job/direct/DirectRunnerJobManagerTest.java +++ b/core/src/test/java/feast/core/job/direct/DirectRunnerJobManagerTest.java @@ -16,6 +16,7 @@ */ package feast.core.job.direct; +import static feast.core.util.ModelHelpers.makeFeatureSetJobStatus; import static org.hamcrest.MatcherAssert.assertThat; import static org.hamcrest.Matchers.equalTo; import static org.mockito.ArgumentMatchers.any; @@ -31,17 +32,15 @@ import com.google.protobuf.util.JsonFormat.Printer; import feast.core.config.FeastProperties.MetricsProperties; import feast.core.job.Runner; -import feast.core.job.option.FeatureSetJsonByteConverter; import feast.core.model.FeatureSet; import feast.core.model.Job; import feast.core.model.JobStatus; import feast.core.model.Source; import feast.core.model.Store; -import feast.ingestion.options.BZip2Compressor; import feast.ingestion.options.ImportOptions; -import feast.ingestion.options.OptionCompressor; import feast.proto.core.FeatureSetProto; import feast.proto.core.FeatureSetProto.FeatureSetSpec; +import feast.proto.core.IngestionJobProto; import feast.proto.core.RunnerProto.DirectRunnerConfigOptions; import feast.proto.core.SourceProto; import feast.proto.core.SourceProto.KafkaSourceConfig; @@ -51,8 +50,6 @@ import feast.proto.core.StoreProto.Store.StoreType; import feast.proto.core.StoreProto.Store.Subscription; import java.io.IOException; -import java.util.Collections; -import java.util.List; import org.apache.beam.runners.direct.DirectRunner; import org.apache.beam.sdk.PipelineResult; import org.apache.beam.sdk.options.PipelineOptionsFactory; @@ -71,6 +68,7 @@ public class DirectRunnerJobManagerTest { private DirectRunnerJobManager drJobManager; private DirectRunnerConfigOptions defaults; + private IngestionJobProto.SpecsStreamingUpdateConfig specsStreamingUpdateConfig; @Before public void setUp() { @@ -79,7 +77,18 @@ public void setUp() { MetricsProperties metricsProperties = new MetricsProperties(); metricsProperties.setEnabled(false); - drJobManager = new DirectRunnerJobManager(defaults, directJobRegistry, metricsProperties); + specsStreamingUpdateConfig = + IngestionJobProto.SpecsStreamingUpdateConfig.newBuilder() + .setSource( + KafkaSourceConfig.newBuilder() + .setTopic("specs_topic") + .setBootstrapServers("servers:9092") + .build()) + .build(); + + drJobManager = + new DirectRunnerJobManager( + defaults, directJobRegistry, metricsProperties, specsStreamingUpdateConfig); drJobManager = Mockito.spy(drJobManager); } @@ -125,11 +134,7 @@ public void shouldStartDirectJobAndRegisterPipelineResult() throws IOException { expectedPipelineOptions.setTargetParallelism(1); expectedPipelineOptions.setStoreJson(Lists.newArrayList(printer.print(store))); expectedPipelineOptions.setProject(""); - - OptionCompressor> featureSetJsonCompressor = - new BZip2Compressor<>(new FeatureSetJsonByteConverter()); - expectedPipelineOptions.setFeatureSetJson( - featureSetJsonCompressor.compress(Collections.singletonList(featureSet))); + expectedPipelineOptions.setSourceJson(printer.print(source)); ArgumentCaptor pipelineOptionsCaptor = ArgumentCaptor.forClass(ImportOptions.class); @@ -145,7 +150,7 @@ public void shouldStartDirectJobAndRegisterPipelineResult() throws IOException { Runner.DIRECT, Source.fromProto(source), Store.fromProto(store), - Lists.newArrayList(FeatureSet.fromProto(featureSet)), + makeFeatureSetJobStatus(FeatureSet.fromProto(featureSet)), JobStatus.PENDING); Job actual = drJobManager.startJob(job); verify(drJobManager, times(1)).runPipeline(pipelineOptionsCaptor.capture()); @@ -156,9 +161,6 @@ public void shouldStartDirectJobAndRegisterPipelineResult() throws IOException { expectedPipelineOptions.setOptionsId( actualPipelineOptions.getOptionsId()); // avoid comparing this value - assertThat( - actualPipelineOptions.getFeatureSetJson(), - equalTo(expectedPipelineOptions.getFeatureSetJson())); assertThat( actualPipelineOptions.getDeadLetterTableSpec(), equalTo(expectedPipelineOptions.getDeadLetterTableSpec())); @@ -169,6 +171,11 @@ public void shouldStartDirectJobAndRegisterPipelineResult() throws IOException { equalTo(expectedPipelineOptions.getMetricsExporterType())); assertThat( actualPipelineOptions.getStoreJson(), equalTo(expectedPipelineOptions.getStoreJson())); + assertThat( + actualPipelineOptions.getSourceJson(), equalTo(expectedPipelineOptions.getSourceJson())); + assertThat( + actualPipelineOptions.getSpecsStreamingUpdateConfigJson(), + equalTo(printer.print(specsStreamingUpdateConfig))); assertThat(jobStarted.getPipelineResult(), equalTo(mockPipelineResult)); assertThat(jobStarted.getJobId(), equalTo(expectedJobId)); diff --git a/core/src/test/java/feast/core/service/JobCoordinatorServiceTest.java b/core/src/test/java/feast/core/service/JobCoordinatorServiceTest.java index 8386efb28f4..caec99f2287 100644 --- a/core/src/test/java/feast/core/service/JobCoordinatorServiceTest.java +++ b/core/src/test/java/feast/core/service/JobCoordinatorServiceTest.java @@ -34,9 +34,8 @@ import feast.core.job.JobManager; import feast.core.job.JobMatcher; import feast.core.job.Runner; -import feast.core.model.FeatureSet; -import feast.core.model.Job; -import feast.core.model.JobStatus; +import feast.core.model.*; +import feast.core.util.ModelHelpers; import feast.proto.core.CoreServiceProto.ListFeatureSetsRequest.Filter; import feast.proto.core.CoreServiceProto.ListFeatureSetsResponse; import feast.proto.core.CoreServiceProto.ListStoresResponse; @@ -50,7 +49,6 @@ import feast.proto.core.StoreProto.Store.RedisConfig; import feast.proto.core.StoreProto.Store.StoreType; import feast.proto.core.StoreProto.Store.Subscription; -import java.util.Arrays; import java.util.Collections; import java.util.List; import org.junit.Before; @@ -159,7 +157,7 @@ public void shouldGenerateAndSubmitJobsIfAny() throws InvalidProtocolBufferExcep Runner.DATAFLOW, feast.core.model.Source.fromProto(source), feast.core.model.Store.fromProto(store), - Arrays.asList(featureSet1, featureSet2), + ModelHelpers.makeFeatureSetJobStatus(featureSet1, featureSet2), JobStatus.PENDING); Job expected = @@ -169,7 +167,7 @@ public void shouldGenerateAndSubmitJobsIfAny() throws InvalidProtocolBufferExcep Runner.DATAFLOW, feast.core.model.Source.fromProto(source), feast.core.model.Store.fromProto(store), - Arrays.asList(featureSet1, featureSet2), + ModelHelpers.makeFeatureSetJobStatus(featureSet1, featureSet2), JobStatus.RUNNING); when(featureSetRepository.findAllByNameLikeAndProject_NameLikeOrderByNameAsc("%", "project1")) @@ -246,7 +244,7 @@ public void shouldGroupJobsBySource() throws InvalidProtocolBufferException { Runner.DATAFLOW, feast.core.model.Source.fromProto(source1), feast.core.model.Store.fromProto(store), - Arrays.asList(featureSet1), + ModelHelpers.makeFeatureSetJobStatus(featureSet1), JobStatus.PENDING); Job expected1 = @@ -256,7 +254,7 @@ public void shouldGroupJobsBySource() throws InvalidProtocolBufferException { Runner.DATAFLOW, feast.core.model.Source.fromProto(source1), feast.core.model.Store.fromProto(store), - Arrays.asList(featureSet1), + ModelHelpers.makeFeatureSetJobStatus(featureSet1), JobStatus.RUNNING); Job expectedInput2 = @@ -266,7 +264,7 @@ public void shouldGroupJobsBySource() throws InvalidProtocolBufferException { Runner.DATAFLOW, feast.core.model.Source.fromProto(source2), feast.core.model.Store.fromProto(store), - Arrays.asList(featureSet2), + ModelHelpers.makeFeatureSetJobStatus(featureSet2), JobStatus.PENDING); Job expected2 = @@ -276,7 +274,7 @@ public void shouldGroupJobsBySource() throws InvalidProtocolBufferException { Runner.DATAFLOW, feast.core.model.Source.fromProto(source2), feast.core.model.Store.fromProto(store), - Arrays.asList(featureSet2), + ModelHelpers.makeFeatureSetJobStatus(featureSet2), JobStatus.RUNNING); ArgumentCaptor> jobArgCaptor = ArgumentCaptor.forClass(List.class); diff --git a/core/src/test/java/feast/core/service/JobServiceTest.java b/core/src/test/java/feast/core/service/JobServiceTest.java index ff056287f9b..27509620ff2 100644 --- a/core/src/test/java/feast/core/service/JobServiceTest.java +++ b/core/src/test/java/feast/core/service/JobServiceTest.java @@ -16,6 +16,7 @@ */ package feast.core.service; +import static feast.core.util.ModelHelpers.makeFeatureSetJobStatus; import static org.hamcrest.CoreMatchers.equalTo; import static org.hamcrest.MatcherAssert.assertThat; import static org.junit.Assert.fail; @@ -25,6 +26,7 @@ import static org.mockito.Mockito.when; import static org.mockito.MockitoAnnotations.initMocks; +import com.google.common.collect.Lists; import com.google.protobuf.InvalidProtocolBufferException; import feast.core.dao.JobRepository; import feast.core.job.JobManager; @@ -126,7 +128,8 @@ public void setupJobRepository() { when(this.jobRepository.findById(this.job.getId())).thenReturn(Optional.of(this.job)); when(this.jobRepository.findByStoreName(this.dataStore.getName())) .thenReturn(Arrays.asList(this.job)); - when(this.jobRepository.findByFeatureSetsIn(Arrays.asList(this.featureSet))) + when(this.jobRepository.findByFeatureSetJobStatusesIn( + Lists.newArrayList((this.featureSet.getJobStatuses())))) .thenReturn(Arrays.asList(this.job)); when(this.jobRepository.findAll()).thenReturn(Arrays.asList(this.job)); } @@ -155,7 +158,7 @@ private Job newDummyJob(String id, String extId, JobStatus status) { Runner.DATAFLOW, this.dataSource, this.dataStore, - Arrays.asList(this.featureSet), + makeFeatureSetJobStatus(this.featureSet), status); } diff --git a/core/src/test/java/feast/core/service/SpecServiceTest.java b/core/src/test/java/feast/core/service/SpecServiceTest.java index b5fd03fc7fc..9ec219b6c97 100644 --- a/core/src/test/java/feast/core/service/SpecServiceTest.java +++ b/core/src/test/java/feast/core/service/SpecServiceTest.java @@ -16,12 +16,13 @@ */ package feast.core.service; +import static org.hamcrest.CoreMatchers.is; import static org.hamcrest.MatcherAssert.assertThat; import static org.hamcrest.core.IsEqual.equalTo; import static org.junit.Assert.assertEquals; -import static org.mockito.Mockito.times; -import static org.mockito.Mockito.verify; -import static org.mockito.Mockito.when; +import static org.mockito.ArgumentMatchers.any; +import static org.mockito.ArgumentMatchers.eq; +import static org.mockito.Mockito.*; import static org.mockito.MockitoAnnotations.initMocks; import com.google.api.client.util.Lists; @@ -45,16 +46,21 @@ import feast.proto.core.FeatureSetProto.EntitySpec; import feast.proto.core.FeatureSetProto.FeatureSetSpec; import feast.proto.core.FeatureSetProto.FeatureSpec; +import feast.proto.core.IngestionJobProto; import feast.proto.core.StoreProto; import feast.proto.core.StoreProto.Store.RedisConfig; import feast.proto.core.StoreProto.Store.StoreType; import feast.proto.core.StoreProto.Store.Subscription; import feast.proto.types.ValueProto.ValueType.Enum; +import io.grpc.StatusRuntimeException; import java.sql.Date; import java.time.Instant; import java.util.*; import java.util.Map.Entry; +import java.util.concurrent.CancellationException; import java.util.stream.Collectors; +import lombok.SneakyThrows; +import org.apache.kafka.clients.consumer.ConsumerRecord; import org.junit.Before; import org.junit.Rule; import org.junit.Test; @@ -62,6 +68,8 @@ import org.mockito.ArgumentCaptor; import org.mockito.ArgumentMatchers; import org.mockito.Mock; +import org.springframework.kafka.core.KafkaTemplate; +import org.springframework.scheduling.annotation.AsyncResult; import org.tensorflow.metadata.v0.BoolDomain; import org.tensorflow.metadata.v0.FeaturePresence; import org.tensorflow.metadata.v0.FeaturePresenceWithinGroup; @@ -86,6 +94,8 @@ public class SpecServiceTest { @Mock private ProjectRepository projectRepository; + @Mock private KafkaTemplate kafkaTemplate; + @Rule public final ExpectedException expectedException = ExpectedException.none(); private SpecService specService; @@ -143,8 +153,11 @@ public void setUp() { when(storeRepository.findById("SERVING")).thenReturn(Optional.of(store1)); when(storeRepository.findById("NOTFOUND")).thenReturn(Optional.empty()); + when(kafkaTemplate.sendDefault(any(), any())).thenReturn(new AsyncResult<>(null)); + specService = - new SpecService(featureSetRepository, storeRepository, projectRepository, defaultSource); + new SpecService( + featureSetRepository, storeRepository, projectRepository, defaultSource, kafkaTemplate); } @Test @@ -284,6 +297,7 @@ public void applyFeatureSetShouldApplyFeatureSetIfNotExists() .build(); assertThat(applyFeatureSetResponse.getStatus(), equalTo(Status.CREATED)); assertThat(applyFeatureSetResponse.getFeatureSet().getSpec(), equalTo(expected.getSpec())); + assertThat(applyFeatureSetResponse.getFeatureSet().getSpec().getVersion(), equalTo(1)); } @Test @@ -308,7 +322,12 @@ public void applyFeatureSetShouldUpdateAndSaveFeatureSetIfAlreadyExists() .toBuilder() .setMeta(incomingFeatureSet.getMeta().toBuilder().build()) .setSpec( - incomingFeatureSet.getSpec().toBuilder().setSource(defaultSource.toProto()).build()) + incomingFeatureSet + .getSpec() + .toBuilder() + .setVersion(2) + .setSource(defaultSource.toProto()) + .build()) .build(); ApplyFeatureSetResponse applyFeatureSetResponse = @@ -318,6 +337,74 @@ public void applyFeatureSetShouldUpdateAndSaveFeatureSetIfAlreadyExists() assertEquals( FeatureSet.fromProto(applyFeatureSetResponse.getFeatureSet()), FeatureSet.fromProto(expected)); + + assertThat(applyFeatureSetResponse.getFeatureSet().getSpec().getVersion(), equalTo(2)); + verify(kafkaTemplate) + .sendDefault(eq(featureSets.get(0).getReference()), any(FeatureSetSpec.class)); + } + + @Test + @SneakyThrows + public void applyFeatureSetShouldNotWorkWithoutKafkaAck() { + FeatureSet fsInTest = featureSets.get(1); + FeatureSetProto.FeatureSet incomingFeatureSet = fsInTest.toProto(); + CancellationException exc = new CancellationException(); + when(kafkaTemplate.sendDefault(eq(fsInTest.getReference()), any()).get()).thenThrow(exc); + + incomingFeatureSet = + incomingFeatureSet + .toBuilder() + .setMeta(incomingFeatureSet.getMeta()) + .setSpec( + incomingFeatureSet + .getSpec() + .toBuilder() + .addFeatures( + FeatureSpec.newBuilder().setName("feature2").setValueType(Enum.STRING)) + .build()) + .build(); + + expectedException.expect(StatusRuntimeException.class); + specService.applyFeatureSet(incomingFeatureSet); + verify(featureSetRepository, never()).saveAndFlush(ArgumentMatchers.any(FeatureSet.class)); + } + + @Test + @SneakyThrows + public void applyFeatureSetShouldUpdateDeliveryStatuses() { + FeatureSet fsInTest = featureSets.get(1); + FeatureSetJobStatus j1 = + newJob( + fsInTest, + JobStatus.RUNNING, + FeatureSetProto.FeatureSetJobDeliveryStatus.STATUS_DELIVERED); + FeatureSetJobStatus j2 = + newJob( + fsInTest, + JobStatus.ABORTED, + FeatureSetProto.FeatureSetJobDeliveryStatus.STATUS_DELIVERED); + + fsInTest.getJobStatuses().addAll(Arrays.asList(j1, j2)); + + FeatureSetProto.FeatureSet incomingFeatureSet = fsInTest.toProto(); + incomingFeatureSet = + incomingFeatureSet + .toBuilder() + .setMeta(incomingFeatureSet.getMeta()) + .setSpec( + incomingFeatureSet + .getSpec() + .toBuilder() + .addFeatures( + FeatureSpec.newBuilder().setName("feature2").setValueType(Enum.STRING)) + .build()) + .build(); + + specService.applyFeatureSet(incomingFeatureSet); + assertThat( + j1.getDeliveryStatus(), is(FeatureSetProto.FeatureSetJobDeliveryStatus.STATUS_IN_PROGRESS)); + assertThat( + j2.getDeliveryStatus(), is(FeatureSetProto.FeatureSetJobDeliveryStatus.STATUS_DELIVERED)); } @Test @@ -681,6 +768,70 @@ public void getOrListFeatureSetShouldUseDefaultProjectIfProjectUnspecified() assertThat(listResponse.getFeatureSetsList(), equalTo(Arrays.asList(expected.toProto()))); } + @Test + public void specAckListenerShouldDoNothingWhenMessageIsOutdated() { + FeatureSet fsInTest = featureSets.get(1); + FeatureSetJobStatus j1 = + newJob( + fsInTest, + JobStatus.RUNNING, + FeatureSetProto.FeatureSetJobDeliveryStatus.STATUS_IN_PROGRESS); + FeatureSetJobStatus j2 = + newJob( + fsInTest, + JobStatus.RUNNING, + FeatureSetProto.FeatureSetJobDeliveryStatus.STATUS_IN_PROGRESS); + + fsInTest.getJobStatuses().addAll(Arrays.asList(j1, j2)); + + specService.listenAckFromJobs(newAckMessage("project/invalid", 0, j1.getJob().getId())); + specService.listenAckFromJobs(newAckMessage(fsInTest.getReference(), 0, "")); + specService.listenAckFromJobs(newAckMessage(fsInTest.getReference(), -1, j1.getJob().getId())); + + assertThat( + j1.getDeliveryStatus(), is(FeatureSetProto.FeatureSetJobDeliveryStatus.STATUS_IN_PROGRESS)); + assertThat( + j2.getDeliveryStatus(), is(FeatureSetProto.FeatureSetJobDeliveryStatus.STATUS_IN_PROGRESS)); + } + + @Test + public void specAckListenerShouldUpdateFeatureSetStatus() { + FeatureSet fsInTest = featureSets.get(1); + fsInTest.setStatus(FeatureSetProto.FeatureSetStatus.STATUS_PENDING); + + FeatureSetJobStatus j1 = + newJob( + fsInTest, + JobStatus.RUNNING, + FeatureSetProto.FeatureSetJobDeliveryStatus.STATUS_IN_PROGRESS); + FeatureSetJobStatus j2 = + newJob( + fsInTest, + JobStatus.RUNNING, + FeatureSetProto.FeatureSetJobDeliveryStatus.STATUS_IN_PROGRESS); + FeatureSetJobStatus j3 = + newJob( + fsInTest, + JobStatus.ABORTED, + FeatureSetProto.FeatureSetJobDeliveryStatus.STATUS_IN_PROGRESS); + + fsInTest.getJobStatuses().addAll(Arrays.asList(j1, j2, j3)); + + specService.listenAckFromJobs( + newAckMessage(fsInTest.getReference(), fsInTest.getVersion(), j1.getJob().getId())); + + assertThat( + j1.getDeliveryStatus(), is(FeatureSetProto.FeatureSetJobDeliveryStatus.STATUS_DELIVERED)); + assertThat(fsInTest.getStatus(), is(FeatureSetProto.FeatureSetStatus.STATUS_PENDING)); + + specService.listenAckFromJobs( + newAckMessage(fsInTest.getReference(), fsInTest.getVersion(), j2.getJob().getId())); + + assertThat( + j2.getDeliveryStatus(), is(FeatureSetProto.FeatureSetJobDeliveryStatus.STATUS_DELIVERED)); + assertThat(fsInTest.getStatus(), is(FeatureSetProto.FeatureSetStatus.STATUS_READY)); + } + private FeatureSet newDummyFeatureSet(String name, String project) { FeatureSpec f1 = FeatureSpec.newBuilder() @@ -698,6 +849,33 @@ private FeatureSet newDummyFeatureSet(String name, String project) { return fs; } + private FeatureSetJobStatus newJob( + FeatureSet fs, JobStatus status, FeatureSetProto.FeatureSetJobDeliveryStatus deliveryStatus) { + Job job = new Job(); + job.setStatus(status); + job.setId(UUID.randomUUID().toString()); + + FeatureSetJobStatus featureSetJobStatus = new FeatureSetJobStatus(); + featureSetJobStatus.setJob(job); + featureSetJobStatus.setFeatureSet(fs); + featureSetJobStatus.setDeliveryStatus(deliveryStatus); + + return featureSetJobStatus; + } + + private ConsumerRecord newAckMessage( + String key, int version, String jobName) { + return new ConsumerRecord<>( + "topic", + 0, + 0, + key, + IngestionJobProto.FeatureSetSpecAck.newBuilder() + .setFeatureSetVersion(version) + .setJobName(jobName) + .build()); + } + private Store newDummyStore(String name) { // Add type to this method when we enable filtering by type Store store = new Store(); diff --git a/core/src/test/java/feast/core/service/TestObjectFactory.java b/core/src/test/java/feast/core/service/TestObjectFactory.java index 40c379d3cf7..a498d6c21c2 100644 --- a/core/src/test/java/feast/core/service/TestObjectFactory.java +++ b/core/src/test/java/feast/core/service/TestObjectFactory.java @@ -39,15 +39,18 @@ public class TestObjectFactory { public static FeatureSet CreateFeatureSet( String name, String project, List entities, List features) { - return new FeatureSet( - name, - project, - 100L, - entities, - features, - defaultSource, - new HashMap<>(), - FeatureSetProto.FeatureSetStatus.STATUS_READY); + FeatureSet fs = + new FeatureSet( + name, + project, + 100L, + entities, + features, + defaultSource, + new HashMap<>(), + FeatureSetProto.FeatureSetStatus.STATUS_READY); + fs.setVersion(1); + return fs; } public static Feature CreateFeature(String name, ValueProto.ValueType.Enum valueType) { diff --git a/core/src/test/java/feast/core/util/ModelHelpers.java b/core/src/test/java/feast/core/util/ModelHelpers.java new file mode 100644 index 00000000000..f864a5562de --- /dev/null +++ b/core/src/test/java/feast/core/util/ModelHelpers.java @@ -0,0 +1,41 @@ +/* + * SPDX-License-Identifier: Apache-2.0 + * Copyright 2018-2020 The Feast Authors + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package feast.core.util; + +import feast.core.model.FeatureSet; +import feast.core.model.FeatureSetJobStatus; +import java.util.List; +import java.util.Set; +import java.util.stream.Collectors; +import java.util.stream.Stream; + +public class ModelHelpers { + public static Set makeFeatureSetJobStatus(FeatureSet... featureSets) { + return Stream.of(featureSets) + .map( + fs -> { + FeatureSetJobStatus s = new FeatureSetJobStatus(); + s.setFeatureSet(fs); + return s; + }) + .collect(Collectors.toSet()); + } + + public static Set makeFeatureSetJobStatus(List featureSets) { + return makeFeatureSetJobStatus(featureSets.toArray(FeatureSet[]::new)); + } +} diff --git a/infra/scripts/test-end-to-end-batch-dataflow.sh b/infra/scripts/test-end-to-end-batch-dataflow.sh index 9ad29185570..e138a75bdea 100755 --- a/infra/scripts/test-end-to-end-batch-dataflow.sh +++ b/infra/scripts/test-end-to-end-batch-dataflow.sh @@ -14,6 +14,8 @@ test -z ${K8_CLUSTER_NAME} && K8_CLUSTER_NAME="feast-e2e-dataflow" test -z ${HELM_RELEASE_NAME} && HELM_RELEASE_NAME="pr-$PULL_NUMBER" test -z ${HELM_COMMON_NAME} && HELM_COMMON_NAME="deps" test -z ${DATASET_NAME} && DATASET_NAME=feast_e2e_$(date +%s) +test -z ${SPECS_TOPIC} && SPECS_TOPIC=feast-specs-$(date +%s) + feast_kafka_1_ip_name="feast-kafka-1" feast_kafka_2_ip_name="feast-kafka-2" @@ -209,8 +211,9 @@ export GCLOUD_SUBNET=$GCLOUD_SUBNET export GCLOUD_REGION=$GCLOUD_REGION export HELM_COMMON_NAME=$HELM_COMMON_NAME export IMAGE_TAG=${PULL_PULL_SHA:1} +export SPECS_TOPIC=$SPECS_TOPIC -envsubst $'$TEMP_BUCKET $DATASET_NAME $GCLOUD_PROJECT $GCLOUD_NETWORK \ +envsubst $'$TEMP_BUCKET $DATASET_NAME $GCLOUD_PROJECT $GCLOUD_NETWORK $SPECS_TOPIC \ $GCLOUD_SUBNET $GCLOUD_REGION $IMAGE_TAG $HELM_COMMON_NAME $feast_kafka_1_ip $feast_kafka_2_ip $feast_kafka_3_ip $feast_redis_ip $feast_statsd_ip' < $ORIGINAL_DIR/infra/scripts/test-templates/values-end-to-end-batch-dataflow.yaml > $ORIGINAL_DIR/infra/charts/feast/values-end-to-end-batch-dataflow-updated.yaml diff --git a/infra/scripts/test-templates/values-end-to-end-batch-dataflow.yaml b/infra/scripts/test-templates/values-end-to-end-batch-dataflow.yaml index 345d5234184..48231face69 100644 --- a/infra/scripts/test-templates/values-end-to-end-batch-dataflow.yaml +++ b/infra/scripts/test-templates/values-end-to-end-batch-dataflow.yaml @@ -15,6 +15,9 @@ feast-core: stream: options: bootstrapServers: $feast_kafka_1_ip:31090 + specsOptions: + specsTopic: $SPECS_TOPIC + specsAckTopic: $SPECS_TOPIC-ack jobs: active_runner: dataflow diff --git a/ingestion/pom.xml b/ingestion/pom.xml index 36d5cd6269b..3eadcdca252 100644 --- a/ingestion/pom.xml +++ b/ingestion/pom.xml @@ -230,6 +230,13 @@ test + + org.slf4j + slf4j-simple + 1.7.30 + test + + com.google.guava guava diff --git a/ingestion/src/main/java/feast/ingestion/ImportJob.java b/ingestion/src/main/java/feast/ingestion/ImportJob.java index 5cef8e57449..8dee52e08a2 100644 --- a/ingestion/src/main/java/feast/ingestion/ImportJob.java +++ b/ingestion/src/main/java/feast/ingestion/ImportJob.java @@ -16,20 +16,20 @@ */ package feast.ingestion; -import static feast.ingestion.utils.SpecUtil.getFeatureSetReference; import static feast.ingestion.utils.StoreUtil.getFeatureSink; import com.google.protobuf.InvalidProtocolBufferException; -import feast.ingestion.options.BZip2Decompressor; import feast.ingestion.options.ImportOptions; -import feast.ingestion.options.StringListStreamConverter; +import feast.ingestion.transform.FeatureRowToStoreAllocator; import feast.ingestion.transform.ProcessAndValidateFeatureRows; import feast.ingestion.transform.ReadFromSource; import feast.ingestion.transform.metrics.WriteFailureMetricsTransform; import feast.ingestion.transform.metrics.WriteSuccessMetricsTransform; +import feast.ingestion.transform.specs.ReadFeatureSetSpecs; +import feast.ingestion.transform.specs.WriteFeatureSetSpecAck; import feast.ingestion.utils.SpecUtil; -import feast.proto.core.FeatureSetProto.FeatureSet; import feast.proto.core.FeatureSetProto.FeatureSetSpec; +import feast.proto.core.IngestionJobProto.SpecsStreamingUpdateConfig; import feast.proto.core.SourceProto.Source; import feast.proto.core.StoreProto.Store; import feast.proto.types.FeatureRowProto.FeatureRow; @@ -39,15 +39,16 @@ import feast.storage.api.writer.WriteResult; import feast.storage.connectors.bigquery.writer.BigQueryDeadletterSink; import java.io.IOException; -import java.util.HashMap; import java.util.List; import java.util.Map; +import java.util.stream.Collectors; import org.apache.beam.sdk.Pipeline; import org.apache.beam.sdk.PipelineResult; import org.apache.beam.sdk.options.PipelineOptionsFactory; import org.apache.beam.sdk.options.PipelineOptionsValidator; import org.apache.beam.sdk.transforms.*; import org.apache.beam.sdk.values.*; +import org.apache.commons.lang3.tuple.Pair; import org.slf4j.Logger; public class ImportJob { @@ -73,11 +74,14 @@ public static void main(String[] args) throws IOException { public static PipelineResult runPipeline(ImportOptions options) throws IOException { /* * Steps: - * 1. Read messages from Feast Source as FeatureRow - * 2. Validate the feature rows to ensure the schema matches what is registered to the system - * 3. Write FeatureRow to the corresponding Store - * 4. Write elements that failed to be processed to a dead letter queue. - * 5. Write metrics to a metrics sink + * 1. Read FeatureSetSpec messages from kafka + * 2. Read messages from Feast Source as FeatureRow + * 3. Validate the feature rows to ensure the schema matches what is registered to the system + * 4. Distribute rows across stores by subscription + * 5. Write FeatureRow to the corresponding Store + * 6. Write elements that failed to be processed to a dead letter queue. + * 7. Write metrics to a metrics sink + * 8. Send ack on receiving FeatureSetSpec */ PipelineOptionsValidator.validate(ImportOptions.class, options); @@ -85,75 +89,71 @@ public static PipelineResult runPipeline(ImportOptions options) throws IOExcepti log.info("Starting import job with settings: \n{}", options.toString()); - BZip2Decompressor> decompressor = - new BZip2Decompressor<>(new StringListStreamConverter()); - List featureSetJson = decompressor.decompress(options.getFeatureSetJson()); - List featureSets = SpecUtil.parseFeatureSetSpecJsonList(featureSetJson); List stores = SpecUtil.parseStoreJsonList(options.getStoreJson()); + Source source = SpecUtil.parseSourceJson(options.getSourceJson()); + SpecsStreamingUpdateConfig specsStreamingUpdateConfig = + SpecUtil.parseSpecsStreamingUpdateConfig(options.getSpecsStreamingUpdateConfigJson()); + + // Step 1. Read FeatureSetSpecs from Spec source + PCollection> featureSetSpecs = + pipeline.apply( + "ReadFeatureSetSpecs", + ReadFeatureSetSpecs.newBuilder() + .setSource(source) + .setStores(stores) + .setSpecsStreamingUpdateConfig(specsStreamingUpdateConfig) + .build()); + + PCollectionView>> globalSpecView = + featureSetSpecs.apply("GlobalSpecView", View.asMultimap()); + + // Step 2. Read messages from Feast Source as FeatureRow. + PCollectionTuple convertedFeatureRows = + pipeline.apply( + "ReadFeatureRowFromSource", + ReadFromSource.newBuilder() + .setSource(source) + .setSuccessTag(FEATURE_ROW_OUT) + .setFailureTag(DEADLETTER_OUT) + .build()); + + // Step 3. Process and validate incoming FeatureRows + PCollectionTuple validatedRows = + convertedFeatureRows + .get(FEATURE_ROW_OUT) + .apply( + ProcessAndValidateFeatureRows.newBuilder() + .setDefaultProject(options.getDefaultFeastProject()) + .setFeatureSetSpecs(globalSpecView) + .setSuccessTag(FEATURE_ROW_OUT) + .setFailureTag(DEADLETTER_OUT) + .build()); + + Map> storeTags = + stores.stream() + .map(s -> Pair.of(s, new TupleTag())) + .collect(Collectors.toMap(Pair::getLeft, Pair::getRight)); + + // Step 4. Allocate validated rows to stores by store subscription + PCollectionTuple storeAllocatedRows = + validatedRows + .get(FEATURE_ROW_OUT) + .apply( + FeatureRowToStoreAllocator.newBuilder() + .setStores(stores) + .setStoreTags(storeTags) + .build()); for (Store store : stores) { - List subscribedFeatureSets = - SpecUtil.getSubscribedFeatureSets(store.getSubscriptionsList(), featureSets); - - // Generate tags by key - Map featureSetSpecsByKey = new HashMap<>(); - subscribedFeatureSets.stream() - .forEach( - fs -> { - String ref = getFeatureSetReference(fs.getSpec()); - featureSetSpecsByKey.put(ref, fs.getSpec()); - }); - - PCollection> staticSpecs = - pipeline - .apply(Create.of("")) - .apply( - "TemporarySpecSource", - ParDo.of( - new DoFn>() { - @ProcessElement - public void process(ProcessContext c) { - featureSetSpecsByKey.forEach((key, value) -> c.output(KV.of(key, value))); - } - })); - - FeatureSink featureSink = getFeatureSink(store, staticSpecs); - - // TODO: make the source part of the job initialisation options - Source source = subscribedFeatureSets.get(0).getSpec().getSource(); - - for (FeatureSet featureSet : subscribedFeatureSets) { - // Ensure Store has valid configuration and Feast can access it. - featureSink.prepareWrite(featureSet); - } + FeatureSink featureSink = getFeatureSink(store, featureSetSpecs); - // Step 1. Read messages from Feast Source as FeatureRow. - PCollectionTuple convertedFeatureRows = - pipeline.apply( - "ReadFeatureRowFromSource", - ReadFromSource.newBuilder() - .setSource(source) - .setSuccessTag(FEATURE_ROW_OUT) - .setFailureTag(DEADLETTER_OUT) - .build()); - - // Step 2. Process and validate incoming FeatureRows - PCollectionTuple validatedRows = - convertedFeatureRows - .get(FEATURE_ROW_OUT) - .apply( - ProcessAndValidateFeatureRows.newBuilder() - .setDefaultProject(options.getDefaultFeastProject()) - .setFeatureSetSpecs(featureSetSpecsByKey) - .setSuccessTag(FEATURE_ROW_OUT) - .setFailureTag(DEADLETTER_OUT) - .build()); - - // Step 3. Write FeatureRow to the corresponding Store. + // Step 5. Write FeatureRow to the corresponding Store. WriteResult writeFeatureRows = - validatedRows.get(FEATURE_ROW_OUT).apply("WriteFeatureRowToStore", featureSink.writer()); + storeAllocatedRows + .get(storeTags.get(store)) + .apply("WriteFeatureRowToStore", featureSink.writer()); - // Step 4. Write FailedElements to a dead letter table in BigQuery. + // Step 6. Write FailedElements to a dead letter table in BigQuery. if (options.getDeadLetterTableSpec() != null) { // TODO: make deadletter destination type configurable DeadletterSink deadletterSink = @@ -172,7 +172,7 @@ public void process(ProcessContext c) { .apply("WriteFailedElements_WriteFeatureRowToStore", deadletterSink.write()); } - // Step 5. Write metrics to a metrics sink. + // Step 7. Write metrics to a metrics sink. writeFeatureRows .getSuccessfulInserts() .apply("WriteSuccessMetrics", WriteSuccessMetricsTransform.create(store.getName())); @@ -182,6 +182,13 @@ public void process(ProcessContext c) { .apply("WriteFailureMetrics", WriteFailureMetricsTransform.create(store.getName())); } + // Step 8. Send ack that FeatureSetSpec state is updated + featureSetSpecs.apply( + "WriteAck", + WriteFeatureSetSpecAck.newBuilder() + .setSpecsStreamingUpdateConfig(specsStreamingUpdateConfig) + .build()); + return pipeline.run(); } } diff --git a/ingestion/src/main/java/feast/ingestion/options/ImportOptions.java b/ingestion/src/main/java/feast/ingestion/options/ImportOptions.java index e3a1b841c4a..9ee1ff891af 100644 --- a/ingestion/src/main/java/feast/ingestion/options/ImportOptions.java +++ b/ingestion/src/main/java/feast/ingestion/options/ImportOptions.java @@ -36,16 +36,29 @@ public interface ImportOptions extends PipelineOptions, DataflowPipelineOptions, @Required @Description( - "JSON string representation of the FeatureSet that the import job will process, in BZip2 binary format." - + "FeatureSet follows the format in feast.core.FeatureSet proto." - + "Mutliple FeatureSetSpec can be passed by specifying '--featureSet={...}' multiple times" + "JSON string representation of the SpecsPipe configuration." + + "Job will use this to know where read new FeatureSetSpec from (kafka broker & topic)" + + "and where send acknowledgment on successful update of job's state to." + + "SpecsStreamingUpdateConfig follows the format in feast.core.IngestionJob.SpecsStreamingUpdateConfig proto." + "The conversion of Proto message to JSON should follow this mapping:" + "https://developers.google.com/protocol-buffers/docs/proto3#json" + "Please minify and remove all insignificant whitespace such as newline in the JSON string" + "to prevent error when parsing the options") - byte[] getFeatureSetJson(); + String getSpecsStreamingUpdateConfigJson(); - void setFeatureSetJson(byte[] featureSetJson); + void setSpecsStreamingUpdateConfigJson(String json); + + @Required + @Description( + "JSON string representation of the Source that will be used to read FeatureRows from." + + "Source follows the format in featst.core.Source proto. Currently only kafka source is supported" + + "The conversion of Proto message to JSON should follow this mapping:" + + "https://developers.google.com/protocol-buffers/docs/proto3#json" + + "Please minify and remove all insignificant whitespace such as newline in the JSON string" + + "to prevent error when parsing the options") + String getSourceJson(); + + void setSourceJson(String json); @Required @Description( diff --git a/ingestion/src/main/java/feast/ingestion/transform/FeatureRowToStoreAllocator.java b/ingestion/src/main/java/feast/ingestion/transform/FeatureRowToStoreAllocator.java new file mode 100644 index 00000000000..48889773c49 --- /dev/null +++ b/ingestion/src/main/java/feast/ingestion/transform/FeatureRowToStoreAllocator.java @@ -0,0 +1,93 @@ +/* + * SPDX-License-Identifier: Apache-2.0 + * Copyright 2018-2020 The Feast Authors + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package feast.ingestion.transform; + +import static feast.ingestion.utils.SpecUtil.parseFeatureSetReference; + +import com.google.auto.value.AutoValue; +import feast.ingestion.utils.SpecUtil; +import feast.proto.core.StoreProto; +import feast.proto.types.FeatureRowProto; +import java.util.List; +import java.util.Map; +import java.util.stream.Collectors; +import org.apache.beam.sdk.transforms.DoFn; +import org.apache.beam.sdk.transforms.PTransform; +import org.apache.beam.sdk.transforms.ParDo; +import org.apache.beam.sdk.values.PCollection; +import org.apache.beam.sdk.values.PCollectionTuple; +import org.apache.beam.sdk.values.TupleTag; +import org.apache.beam.sdk.values.TupleTagList; +import org.apache.commons.lang3.tuple.Pair; + +/** + * For each incoming {@link FeatureRowProto.FeatureRow} allocator choose only stores that + * subscripted to its project and featureSet names. + * + *

Return PCollectionTuple with one {@link TupleTag} per {@link StoreProto.Store}. Tags must be + * generated in advance. + */ +@AutoValue +public abstract class FeatureRowToStoreAllocator + extends PTransform, PCollectionTuple> { + public abstract List getStores(); + + public abstract Map> getStoreTags(); + + public static Builder newBuilder() { + return new AutoValue_FeatureRowToStoreAllocator.Builder(); + } + + @AutoValue.Builder + public abstract static class Builder { + public abstract Builder setStores(List stores); + + public abstract Builder setStoreTags( + Map> tags); + + public abstract FeatureRowToStoreAllocator build(); + } + + @Override + public PCollectionTuple expand(PCollection input) { + return input.apply( + "AssignRowToStore", + ParDo.of( + new DoFn() { + @ProcessElement + public void process(ProcessContext c, @Element FeatureRowProto.FeatureRow row) { + Pair projectAndSetNames = + parseFeatureSetReference(row.getFeatureSet()); + getStores().stream() + .filter( + s -> + SpecUtil.isSubscribedToFeatureSet( + s.getSubscriptionsList(), + projectAndSetNames.getLeft(), + projectAndSetNames.getRight())) + .forEach(s -> c.output(getStoreTags().get(s), row)); + } + }) + .withOutputTags( + getStoreTags().get(getStores().get(0)), + TupleTagList.of( + getStores().stream() + .skip(1) + .map(getStoreTags()::get) + .collect(Collectors.toList())))); + } +} diff --git a/ingestion/src/main/java/feast/ingestion/transform/ProcessAndValidateFeatureRows.java b/ingestion/src/main/java/feast/ingestion/transform/ProcessAndValidateFeatureRows.java index 2ce8918ccf9..729bff69a7f 100644 --- a/ingestion/src/main/java/feast/ingestion/transform/ProcessAndValidateFeatureRows.java +++ b/ingestion/src/main/java/feast/ingestion/transform/ProcessAndValidateFeatureRows.java @@ -19,25 +19,20 @@ import com.google.auto.value.AutoValue; import feast.ingestion.transform.fn.ProcessFeatureRowDoFn; import feast.ingestion.transform.fn.ValidateFeatureRowDoFn; -import feast.ingestion.values.FeatureSet; import feast.proto.core.FeatureSetProto; import feast.proto.types.FeatureRowProto.FeatureRow; import feast.storage.api.writer.FailedElement; import java.util.Map; -import java.util.stream.Collectors; import org.apache.beam.sdk.transforms.PTransform; import org.apache.beam.sdk.transforms.ParDo; -import org.apache.beam.sdk.values.PCollection; -import org.apache.beam.sdk.values.PCollectionTuple; -import org.apache.beam.sdk.values.TupleTag; -import org.apache.beam.sdk.values.TupleTagList; -import org.apache.commons.lang3.tuple.Pair; +import org.apache.beam.sdk.values.*; @AutoValue public abstract class ProcessAndValidateFeatureRows extends PTransform, PCollectionTuple> { - public abstract Map getFeatureSetSpecs(); + public abstract PCollectionView>> + getFeatureSetSpecs(); public abstract String getDefaultProject(); @@ -53,7 +48,7 @@ public static Builder newBuilder() { public abstract static class Builder { public abstract Builder setFeatureSetSpecs( - Map featureSets); + PCollectionView>> featureSets); public abstract Builder setDefaultProject(String defaultProject); @@ -66,22 +61,17 @@ public abstract Builder setFeatureSetSpecs( @Override public PCollectionTuple expand(PCollection input) { - - Map featureSets = - getFeatureSetSpecs().entrySet().stream() - .map(e -> Pair.of(e.getKey(), new FeatureSet(e.getValue()))) - .collect(Collectors.toMap(Pair::getLeft, Pair::getRight)); - return input .apply("ProcessFeatureRows", ParDo.of(new ProcessFeatureRowDoFn(getDefaultProject()))) .apply( "ValidateFeatureRows", ParDo.of( ValidateFeatureRowDoFn.newBuilder() - .setFeatureSets(featureSets) + .setFeatureSets(getFeatureSetSpecs()) .setSuccessTag(getSuccessTag()) .setFailureTag(getFailureTag()) .build()) + .withSideInputs(getFeatureSetSpecs()) .withOutputTags(getSuccessTag(), TupleTagList.of(getFailureTag()))); } } diff --git a/ingestion/src/main/java/feast/ingestion/transform/fn/ValidateFeatureRowDoFn.java b/ingestion/src/main/java/feast/ingestion/transform/fn/ValidateFeatureRowDoFn.java index 856d94828ad..b89af891407 100644 --- a/ingestion/src/main/java/feast/ingestion/transform/fn/ValidateFeatureRowDoFn.java +++ b/ingestion/src/main/java/feast/ingestion/transform/fn/ValidateFeatureRowDoFn.java @@ -17,8 +17,10 @@ package feast.ingestion.transform.fn; import com.google.auto.value.AutoValue; +import com.google.common.collect.Iterators; import feast.ingestion.values.FeatureSet; import feast.ingestion.values.Field; +import feast.proto.core.FeatureSetProto; import feast.proto.types.FeatureRowProto.FeatureRow; import feast.proto.types.FieldProto; import feast.proto.types.ValueProto.Value.ValCase; @@ -27,12 +29,14 @@ import java.util.List; import java.util.Map; import org.apache.beam.sdk.transforms.DoFn; +import org.apache.beam.sdk.values.PCollectionView; import org.apache.beam.sdk.values.TupleTag; @AutoValue public abstract class ValidateFeatureRowDoFn extends DoFn { - public abstract Map getFeatureSets(); + public abstract PCollectionView>> + getFeatureSets(); public abstract TupleTag getSuccessTag(); @@ -45,7 +49,8 @@ public static Builder newBuilder() { @AutoValue.Builder public abstract static class Builder { - public abstract Builder setFeatureSets(Map featureSets); + public abstract Builder setFeatureSets( + PCollectionView>> featureSets); public abstract Builder setSuccessTag(TupleTag successTag); @@ -58,9 +63,14 @@ public abstract static class Builder { public void processElement(ProcessContext context) { String error = null; FeatureRow featureRow = context.element(); - FeatureSet featureSet = getFeatureSets().get(featureRow.getFeatureSet()); + Iterable featureSetSpecs = + context.sideInput(getFeatureSets()).get(featureRow.getFeatureSet()); + List fields = new ArrayList<>(); - if (featureSet != null) { + if (featureSetSpecs != null) { + FeatureSetProto.FeatureSetSpec latestSpec = Iterators.getLast(featureSetSpecs.iterator()); + FeatureSet featureSet = new FeatureSet(latestSpec); + for (FieldProto.Field field : featureRow.getFieldsList()) { Field fieldSpec = featureSet.getField(field.getName()); if (fieldSpec == null) { @@ -98,9 +108,10 @@ public void processElement(ProcessContext context) { .setJobName(context.getPipelineOptions().getJobName()) .setPayload(featureRow.toString()) .setErrorMessage(error); - if (featureSet != null) { - String[] split = featureSet.getReference().split("/"); - failedElement = failedElement.setProjectName(split[0]).setFeatureSetName(split[1]); + if (featureSetSpecs != null) { + FeatureSetProto.FeatureSetSpec spec = Iterators.getLast(featureSetSpecs.iterator()); + failedElement = + failedElement.setProjectName(spec.getProject()).setFeatureSetName(spec.getName()); } context.output(getFailureTag(), failedElement.build()); } else { diff --git a/ingestion/src/main/java/feast/ingestion/transform/specs/FilterRelevantFunction.java b/ingestion/src/main/java/feast/ingestion/transform/specs/FilterRelevantFunction.java new file mode 100644 index 00000000000..cb2c84e2cd0 --- /dev/null +++ b/ingestion/src/main/java/feast/ingestion/transform/specs/FilterRelevantFunction.java @@ -0,0 +1,52 @@ +/* + * SPDX-License-Identifier: Apache-2.0 + * Copyright 2018-2020 The Feast Authors + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package feast.ingestion.transform.specs; + +import feast.ingestion.utils.SpecUtil; +import feast.proto.core.FeatureSetProto; +import feast.proto.core.SourceProto; +import feast.proto.core.StoreProto; +import java.util.List; +import org.apache.beam.sdk.transforms.ProcessFunction; +import org.apache.beam.sdk.values.KV; + +/** + * Selects only those FeatureSetSpecs that have the same source as current Job and their featureSet + * reference matches to current Job Subscriptions (extracted from Stores) + */ +public class FilterRelevantFunction + implements ProcessFunction, Boolean> { + private final List stores; + private final SourceProto.Source source; + + public FilterRelevantFunction(SourceProto.Source source, List stores) { + this.source = source; + this.stores = stores; + } + + @Override + public Boolean apply(KV input) throws Exception { + return stores.stream() + .anyMatch( + s -> + SpecUtil.isSubscribedToFeatureSet( + s.getSubscriptionsList(), + input.getValue().getProject(), + input.getValue().getName())) + && input.getValue().getSource().equals(source); + } +} diff --git a/ingestion/src/main/java/feast/ingestion/transform/specs/KafkaRecordToFeatureSetSpec.java b/ingestion/src/main/java/feast/ingestion/transform/specs/KafkaRecordToFeatureSetSpec.java new file mode 100644 index 00000000000..47aefd0c0bb --- /dev/null +++ b/ingestion/src/main/java/feast/ingestion/transform/specs/KafkaRecordToFeatureSetSpec.java @@ -0,0 +1,53 @@ +/* + * SPDX-License-Identifier: Apache-2.0 + * Copyright 2018-2020 The Feast Authors + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package feast.ingestion.transform.specs; + +import com.google.protobuf.InvalidProtocolBufferException; +import feast.proto.core.FeatureSetProto; +import org.apache.beam.sdk.io.kafka.KafkaRecord; +import org.apache.beam.sdk.transforms.DoFn; +import org.apache.beam.sdk.values.KV; +import org.slf4j.Logger; +import org.slf4j.LoggerFactory; + +/** + * Parse FeatureSetSpec from KafkaRecord. We expect message with: + * + *

+ * + *

    + *
  • key: String - FeatureSet reference + *
  • value: FeatureSetSpec serialized in proto + *
+ */ +public class KafkaRecordToFeatureSetSpec + extends DoFn, KV> { + private static final Logger log = LoggerFactory.getLogger(KafkaRecordToFeatureSetSpec.class); + + @ProcessElement + public void process(ProcessContext c) { + try { + FeatureSetProto.FeatureSetSpec featureSetSpec = + FeatureSetProto.FeatureSetSpec.parseFrom(c.element().getKV().getValue()); + c.output(KV.of(new String(c.element().getKV().getKey()), featureSetSpec)); + } catch (InvalidProtocolBufferException e) { + log.error( + String.format( + "Unable to decode FeatureSetSpec with reference %s", c.element().getKV().getKey())); + } + } +} diff --git a/ingestion/src/main/java/feast/ingestion/transform/specs/ReadFeatureSetSpecs.java b/ingestion/src/main/java/feast/ingestion/transform/specs/ReadFeatureSetSpecs.java new file mode 100644 index 00000000000..c777438be6a --- /dev/null +++ b/ingestion/src/main/java/feast/ingestion/transform/specs/ReadFeatureSetSpecs.java @@ -0,0 +1,107 @@ +/* + * SPDX-License-Identifier: Apache-2.0 + * Copyright 2018-2020 The Feast Authors + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package feast.ingestion.transform.specs; + +import com.google.auto.value.AutoValue; +import com.google.common.collect.ImmutableMap; +import com.google.common.collect.Lists; +import feast.proto.core.FeatureSetProto.FeatureSetSpec; +import feast.proto.core.IngestionJobProto; +import feast.proto.core.SourceProto; +import feast.proto.core.StoreProto; +import java.util.ArrayList; +import java.util.Comparator; +import java.util.List; +import org.apache.beam.sdk.io.kafka.KafkaIO; +import org.apache.beam.sdk.transforms.*; +import org.apache.beam.sdk.transforms.windowing.AfterPane; +import org.apache.beam.sdk.transforms.windowing.GlobalWindows; +import org.apache.beam.sdk.transforms.windowing.Repeatedly; +import org.apache.beam.sdk.transforms.windowing.Window; +import org.apache.beam.sdk.values.KV; +import org.apache.beam.sdk.values.PBegin; +import org.apache.beam.sdk.values.PCollection; +import org.apache.kafka.clients.consumer.ConsumerConfig; +import org.joda.time.Duration; + +/** + * Source of {@link FeatureSetSpec} + * + *

Reads from Kafka topic (from the beginning) {@link FeatureSetSpec}. + * + *

Filters only relevant Specs by {@link feast.proto.core.StoreProto.Store} subscriptions and + * {@link feast.proto.core.SourceProto.Source}. + * + *

Compacts {@link FeatureSetSpec} by reference (if it was not yet compacted in Kafka). + */ +@AutoValue +public abstract class ReadFeatureSetSpecs + extends PTransform>> { + public abstract IngestionJobProto.SpecsStreamingUpdateConfig getSpecsStreamingUpdateConfig(); + + public abstract SourceProto.Source getSource(); + + public abstract List getStores(); + + public static Builder newBuilder() { + return new AutoValue_ReadFeatureSetSpecs.Builder(); + } + + @AutoValue.Builder + public abstract static class Builder { + public abstract Builder setSpecsStreamingUpdateConfig( + IngestionJobProto.SpecsStreamingUpdateConfig config); + + public abstract Builder setSource(SourceProto.Source source); + + public abstract Builder setStores(List stores); + + public abstract ReadFeatureSetSpecs build(); + } + + @Override + public PCollection> expand(PBegin input) { + return input + .apply( + KafkaIO.readBytes() + .withBootstrapServers( + getSpecsStreamingUpdateConfig().getSource().getBootstrapServers()) + .withTopic(getSpecsStreamingUpdateConfig().getSource().getTopic()) + .withConsumerConfigUpdates( + ImmutableMap.of( + ConsumerConfig.AUTO_OFFSET_RESET_CONFIG, + "earliest", + ConsumerConfig.ENABLE_AUTO_COMMIT_CONFIG, + false))) + .apply("ParseFeatureSetSpec", ParDo.of(new KafkaRecordToFeatureSetSpec())) + .apply("OnlyRelevantSpecs", Filter.by(new FilterRelevantFunction(getSource(), getStores()))) + .apply( + Window.>into(new GlobalWindows()) + .triggering(Repeatedly.forever(AfterPane.elementCountAtLeast(1))) + .accumulatingFiredPanes() + .withAllowedLateness(Duration.ZERO)) + .apply( + Combine.perKey( + (SerializableFunction, FeatureSetSpec>) + specs -> { + ArrayList featureSetSpecs = Lists.newArrayList(specs); + featureSetSpecs.sort( + Comparator.comparing(FeatureSetSpec::getVersion).reversed()); + return featureSetSpecs.get(0); + })); + } +} diff --git a/ingestion/src/main/java/feast/ingestion/transform/specs/WriteFeatureSetSpecAck.java b/ingestion/src/main/java/feast/ingestion/transform/specs/WriteFeatureSetSpecAck.java new file mode 100644 index 00000000000..b5d49b70766 --- /dev/null +++ b/ingestion/src/main/java/feast/ingestion/transform/specs/WriteFeatureSetSpecAck.java @@ -0,0 +1,85 @@ +/* + * SPDX-License-Identifier: Apache-2.0 + * Copyright 2018-2020 The Feast Authors + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package feast.ingestion.transform.specs; + +import com.google.auto.value.AutoValue; +import feast.proto.core.FeatureSetProto; +import feast.proto.core.IngestionJobProto; +import java.io.ByteArrayOutputStream; +import java.io.IOException; +import org.apache.beam.sdk.io.kafka.KafkaIO; +import org.apache.beam.sdk.transforms.DoFn; +import org.apache.beam.sdk.transforms.PTransform; +import org.apache.beam.sdk.transforms.ParDo; +import org.apache.beam.sdk.values.KV; +import org.apache.beam.sdk.values.PCollection; +import org.apache.beam.sdk.values.PDone; +import org.apache.kafka.common.serialization.ByteArraySerializer; +import org.apache.kafka.common.serialization.StringSerializer; + +/** + * Converts input {@link feast.proto.core.FeatureSetProto.FeatureSetSpec} into {@link + * feast.proto.core.IngestionJobProto.FeatureSetSpecAck} message and writes it to kafka (ack-topic). + */ +@AutoValue +public abstract class WriteFeatureSetSpecAck + extends PTransform>, PDone> { + public abstract IngestionJobProto.SpecsStreamingUpdateConfig getSpecsStreamingUpdateConfig(); + + public static Builder newBuilder() { + return new AutoValue_WriteFeatureSetSpecAck.Builder(); + } + + @AutoValue.Builder + public abstract static class Builder { + public abstract Builder setSpecsStreamingUpdateConfig( + IngestionJobProto.SpecsStreamingUpdateConfig config); + + public abstract WriteFeatureSetSpecAck build(); + } + + @Override + public PDone expand(PCollection> input) { + return input + .apply("FeatureSetSpecToAckMessage", ParDo.of(new BuildAckMessage())) + .apply( + "ToKafka", + KafkaIO.write() + .withBootstrapServers( + getSpecsStreamingUpdateConfig().getAck().getBootstrapServers()) + .withTopic(getSpecsStreamingUpdateConfig().getAck().getTopic()) + .withKeySerializer(StringSerializer.class) + .withValueSerializer(ByteArraySerializer.class)); + } + + private static class BuildAckMessage + extends DoFn, KV> { + @ProcessElement + public void process(ProcessContext c) throws IOException { + ByteArrayOutputStream encodedAck = new ByteArrayOutputStream(); + + IngestionJobProto.FeatureSetSpecAck.newBuilder() + .setFeatureSetReference(c.element().getKey()) + .setJobName(c.getPipelineOptions().getJobName()) + .setFeatureSetVersion(c.element().getValue().getVersion()) + .build() + .writeTo(encodedAck); + + c.output(KV.of(c.element().getKey(), encodedAck.toByteArray())); + } + } +} diff --git a/ingestion/src/main/java/feast/ingestion/utils/SpecUtil.java b/ingestion/src/main/java/feast/ingestion/utils/SpecUtil.java index a1356584b3f..79c81c162d1 100644 --- a/ingestion/src/main/java/feast/ingestion/utils/SpecUtil.java +++ b/ingestion/src/main/java/feast/ingestion/utils/SpecUtil.java @@ -20,9 +20,11 @@ import com.google.protobuf.util.JsonFormat; import feast.ingestion.values.Field; import feast.proto.core.FeatureSetProto.EntitySpec; -import feast.proto.core.FeatureSetProto.FeatureSet; import feast.proto.core.FeatureSetProto.FeatureSetSpec; import feast.proto.core.FeatureSetProto.FeatureSpec; +import feast.proto.core.IngestionJobProto; +import feast.proto.core.IngestionJobProto.SpecsStreamingUpdateConfig; +import feast.proto.core.SourceProto.Source; import feast.proto.core.StoreProto.Store; import feast.proto.core.StoreProto.Store.Subscription; import java.util.ArrayList; @@ -30,62 +32,60 @@ import java.util.List; import java.util.Map; import java.util.regex.Pattern; +import org.apache.commons.lang3.tuple.Pair; public class SpecUtil { + public static String PROJECT_DEFAULT_NAME = "default"; public static String getFeatureSetReference(FeatureSetSpec featureSetSpec) { return String.format("%s/%s", featureSetSpec.getProject(), featureSetSpec.getName()); } + public static Pair parseFeatureSetReference(String reference) { + String[] split = reference.split("/", 2); + if (split.length == 1) { + return Pair.of(PROJECT_DEFAULT_NAME, split[0]); + } else { + return Pair.of(split[0], split[1]); + } + } + /** Get only feature set specs that matches the subscription */ - public static List getSubscribedFeatureSets( - List subscriptions, List featureSets) { - List subscribed = new ArrayList<>(); - for (FeatureSet featureSet : featureSets) { - for (Subscription sub : subscriptions) { - // If configuration missing, fail - if (sub.getProject().isEmpty() || sub.getName().isEmpty()) { - throw new IllegalArgumentException( - String.format("Subscription is missing arguments: %s", sub.toString())); - } + public static boolean isSubscribedToFeatureSet( + List subscriptions, String projectName, String featureSetName) { - // If all wildcards, subscribe to everything - if (sub.getProject().equals("*") || sub.getName().equals("*")) { - subscribed.add(featureSet); - break; - } + for (Subscription sub : subscriptions) { + // If configuration missing, fail + if (sub.getProject().isEmpty() || sub.getName().isEmpty()) { + throw new IllegalArgumentException( + String.format("Subscription is missing arguments: %s", sub.toString())); + } - // Match project name - if (!featureSet.getSpec().getProject().equals(sub.getProject())) { - continue; - } + // If all wildcards, subscribe to everything + if (sub.getProject().equals("*") || sub.getName().equals("*")) { + return true; + } - // Convert wildcard to regex - String subName = sub.getName(); - if (!sub.getName().contains(".*")) { - subName = subName.replace("*", ".*"); - } + // Match project name + if (!projectName.equals(sub.getProject())) { + continue; + } - // Match feature set name to pattern - Pattern pattern = Pattern.compile(subName); - if (!pattern.matcher(featureSet.getSpec().getName()).matches()) { - continue; - } - subscribed.add(featureSet); + // Convert wildcard to regex + String subName = sub.getName(); + if (!sub.getName().contains(".*")) { + subName = subName.replace("*", ".*"); } - } - return subscribed; - } - public static List parseFeatureSetSpecJsonList(List jsonList) - throws InvalidProtocolBufferException { - List featureSets = new ArrayList<>(); - for (String json : jsonList) { - FeatureSetSpec.Builder builder = FeatureSetSpec.newBuilder(); - JsonFormat.parser().merge(json, builder); - featureSets.add(FeatureSet.newBuilder().setSpec(builder.build()).build()); + // Match feature set name to pattern + Pattern pattern = Pattern.compile(subName); + if (!pattern.matcher(featureSetName).matches()) { + continue; + } + return true; } - return featureSets; + + return false; } public static List parseStoreJsonList(List jsonList) @@ -99,6 +99,19 @@ public static List parseStoreJsonList(List jsonList) return stores; } + public static Source parseSourceJson(String jsonSource) throws InvalidProtocolBufferException { + Source.Builder builder = Source.newBuilder(); + JsonFormat.parser().merge(jsonSource, builder); + return builder.build(); + } + + public static IngestionJobProto.SpecsStreamingUpdateConfig parseSpecsStreamingUpdateConfig( + String jsonConfig) throws InvalidProtocolBufferException { + SpecsStreamingUpdateConfig.Builder builder = SpecsStreamingUpdateConfig.newBuilder(); + JsonFormat.parser().merge(jsonConfig, builder); + return builder.build(); + } + public static Map getFieldsByName(FeatureSetSpec featureSetSpec) { Map fieldByName = new HashMap<>(); for (EntitySpec entitySpec : featureSetSpec.getEntitiesList()) { diff --git a/ingestion/src/test/java/feast/ingestion/ImportJobTest.java b/ingestion/src/test/java/feast/ingestion/ImportJobTest.java index 39e42963782..47b5aa3f9b8 100644 --- a/ingestion/src/test/java/feast/ingestion/ImportJobTest.java +++ b/ingestion/src/test/java/feast/ingestion/ImportJobTest.java @@ -16,15 +16,18 @@ */ package feast.ingestion; +import static feast.ingestion.utils.SpecUtil.getFeatureSetReference; + +import com.google.common.collect.ImmutableList; import com.google.common.io.Files; import com.google.protobuf.InvalidProtocolBufferException; import com.google.protobuf.util.JsonFormat; -import feast.ingestion.options.BZip2Compressor; import feast.ingestion.options.ImportOptions; import feast.proto.core.FeatureSetProto.EntitySpec; import feast.proto.core.FeatureSetProto.FeatureSet; import feast.proto.core.FeatureSetProto.FeatureSetSpec; import feast.proto.core.FeatureSetProto.FeatureSpec; +import feast.proto.core.IngestionJobProto; import feast.proto.core.SourceProto.KafkaSourceConfig; import feast.proto.core.SourceProto.Source; import feast.proto.core.SourceProto.SourceType; @@ -53,6 +56,7 @@ import java.util.Map; import java.util.stream.Collectors; import java.util.stream.IntStream; +import org.apache.beam.repackaged.core.org.apache.commons.lang3.tuple.Pair; import org.apache.beam.sdk.PipelineResult; import org.apache.beam.sdk.PipelineResult.State; import org.apache.beam.sdk.options.PipelineOptionsFactory; @@ -74,6 +78,9 @@ public class ImportJobTest { private static final String KAFKA_BOOTSTRAP_SERVERS = KAFKA_HOST + ":" + KAFKA_PORT; private static final short KAFKA_REPLICATION_FACTOR = 1; private static final String KAFKA_TOPIC = "topic_1"; + private static final String KAFKA_SPECS_TOPIC = "topic_specs_1"; + private static final String KAFKA_SPECS_ACK_TOPIC = "topic_specs_ack_1"; + private static final long KAFKA_PUBLISH_TIMEOUT_SEC = 10; @SuppressWarnings("UnstableApiUsage") @@ -117,6 +124,30 @@ public static void tearDown() { @Test public void runPipeline_ShouldWriteToRedisCorrectlyGivenValidSpecAndFeatureRow() throws IOException, InterruptedException { + Source featureSource = + Source.newBuilder() + .setType(SourceType.KAFKA) + .setKafkaSourceConfig( + KafkaSourceConfig.newBuilder() + .setBootstrapServers(KAFKA_HOST + ":" + KAFKA_PORT) + .setTopic(KAFKA_TOPIC) + .build()) + .build(); + + IngestionJobProto.SpecsStreamingUpdateConfig specsStreamingUpdateConfig = + IngestionJobProto.SpecsStreamingUpdateConfig.newBuilder() + .setSource( + KafkaSourceConfig.newBuilder() + .setBootstrapServers(KAFKA_HOST + ":" + KAFKA_PORT) + .setTopic(KAFKA_SPECS_TOPIC) + .build()) + .setAck( + KafkaSourceConfig.newBuilder() + .setBootstrapServers(KAFKA_HOST + ":" + KAFKA_PORT) + .setTopic(KAFKA_SPECS_ACK_TOPIC) + .build()) + .build(); + FeatureSetSpec spec = FeatureSetSpec.newBuilder() .setName("feature_set") @@ -140,15 +171,7 @@ public void runPipeline_ShouldWriteToRedisCorrectlyGivenValidSpecAndFeatureRow() FeatureSpec.newBuilder().setName("feature_2").setValueType(Enum.STRING).build()) .addFeatures( FeatureSpec.newBuilder().setName("feature_3").setValueType(Enum.INT64).build()) - .setSource( - Source.newBuilder() - .setType(SourceType.KAFKA) - .setKafkaSourceConfig( - KafkaSourceConfig.newBuilder() - .setBootstrapServers(KAFKA_HOST + ":" + KAFKA_PORT) - .setTopic(KAFKA_TOPIC) - .build()) - .build()) + .setSource(featureSource) .build(); FeatureSet featureSet = FeatureSet.newBuilder().setSpec(spec).build(); @@ -167,20 +190,16 @@ public void runPipeline_ShouldWriteToRedisCorrectlyGivenValidSpecAndFeatureRow() .build(); ImportOptions options = PipelineOptionsFactory.create().as(ImportOptions.class); - BZip2Compressor compressor = - new BZip2Compressor<>( - option -> { - JsonFormat.Printer printer = - JsonFormat.printer().omittingInsignificantWhitespace().printingEnumsAsInts(); - return printer.print(option).getBytes(); - }); - options.setFeatureSetJson(compressor.compress(spec)); + + options.setSpecsStreamingUpdateConfigJson( + JsonFormat.printer().print(specsStreamingUpdateConfig)); + options.setSourceJson(JsonFormat.printer().print(featureSource)); options.setStoreJson(Collections.singletonList(JsonFormat.printer().print(redis))); options.setDefaultFeastProject("myproject"); options.setProject(""); options.setBlockOnRun(false); - List input = new ArrayList<>(); + List> input = new ArrayList<>(); Map expected = new HashMap<>(); LOGGER.info("Generating test data ..."); @@ -189,7 +208,7 @@ public void runPipeline_ShouldWriteToRedisCorrectlyGivenValidSpecAndFeatureRow() i -> { FeatureRow randomRow = TestUtil.createRandomFeatureRow(featureSet.getSpec()); RedisKey redisKey = TestUtil.createRedisKey(featureSet.getSpec(), randomRow); - input.add(randomRow); + input.add(Pair.of("", randomRow)); List fields = randomRow.getFieldsList().stream() .filter( @@ -216,7 +235,13 @@ public void runPipeline_ShouldWriteToRedisCorrectlyGivenValidSpecAndFeatureRow() Assert.assertEquals(pipelineResult.getState(), State.RUNNING); LOGGER.info("Publishing {} Feature Row messages to Kafka ...", input.size()); - TestUtil.publishFeatureRowsToKafka( + TestUtil.publishToKafka( + KAFKA_BOOTSTRAP_SERVERS, + KAFKA_SPECS_TOPIC, + ImmutableList.of(Pair.of(getFeatureSetReference(spec), spec)), + ByteArraySerializer.class, + KAFKA_PUBLISH_TIMEOUT_SEC); + TestUtil.publishToKafka( KAFKA_BOOTSTRAP_SERVERS, KAFKA_TOPIC, input, diff --git a/ingestion/src/test/java/feast/ingestion/transform/FeatureRowToStoreAllocatorTest.java b/ingestion/src/test/java/feast/ingestion/transform/FeatureRowToStoreAllocatorTest.java new file mode 100644 index 00000000000..0ba5d563690 --- /dev/null +++ b/ingestion/src/test/java/feast/ingestion/transform/FeatureRowToStoreAllocatorTest.java @@ -0,0 +1,99 @@ +/* + * SPDX-License-Identifier: Apache-2.0 + * Copyright 2018-2020 The Feast Authors + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package feast.ingestion.transform; + +import com.google.common.collect.ImmutableList; +import com.google.common.collect.ImmutableMap; +import feast.proto.core.StoreProto; +import feast.proto.types.FeatureRowProto; +import java.util.Map; +import org.apache.beam.sdk.extensions.protobuf.ProtoCoder; +import org.apache.beam.sdk.testing.PAssert; +import org.apache.beam.sdk.testing.TestPipeline; +import org.apache.beam.sdk.transforms.Count; +import org.apache.beam.sdk.transforms.Create; +import org.apache.beam.sdk.values.PCollectionTuple; +import org.apache.beam.sdk.values.TupleTag; +import org.junit.Rule; +import org.junit.Test; + +public class FeatureRowToStoreAllocatorTest { + @Rule public transient TestPipeline p = TestPipeline.create(); + + private StoreProto.Store newStore(String s) { + return StoreProto.Store.newBuilder() + .addSubscriptions( + StoreProto.Store.Subscription.newBuilder().setProject("project").setName(s).build()) + .build(); + } + + @Test + public void featureRowShouldBeAllocatedToStoreTagsAccordingToSubscription() { + StoreProto.Store bqOnlyStore = newStore("bq*"); + StoreProto.Store redisOnlyStore = newStore("redis*"); + StoreProto.Store anyStore = newStore("*"); + + Map> storeTags = + ImmutableMap.of( + bqOnlyStore, new TupleTag<>(), + redisOnlyStore, new TupleTag<>(), + anyStore, new TupleTag<>()); + + PCollectionTuple allocatedRows = + p.apply( + Create.of( + FeatureRowProto.FeatureRow.newBuilder().setFeatureSet("project/bq_1").build(), + FeatureRowProto.FeatureRow.newBuilder().setFeatureSet("project/bq_2").build(), + FeatureRowProto.FeatureRow.newBuilder() + .setFeatureSet("project/redis_1") + .build(), + FeatureRowProto.FeatureRow.newBuilder() + .setFeatureSet("project/redis_2") + .build(), + FeatureRowProto.FeatureRow.newBuilder() + .setFeatureSet("project/redis_3") + .build())) + .apply( + FeatureRowToStoreAllocator.newBuilder() + .setStoreTags(storeTags) + .setStores(ImmutableList.of(bqOnlyStore, redisOnlyStore, anyStore)) + .build()); + + PAssert.that( + allocatedRows + .get(storeTags.get(bqOnlyStore)) + .setCoder(ProtoCoder.of(FeatureRowProto.FeatureRow.class)) + .apply("CountBq", Count.globally())) + .containsInAnyOrder(2L); + + PAssert.that( + allocatedRows + .get(storeTags.get(redisOnlyStore)) + .setCoder(ProtoCoder.of(FeatureRowProto.FeatureRow.class)) + .apply("CountRedis", Count.globally())) + .containsInAnyOrder(3L); + + PAssert.that( + allocatedRows + .get(storeTags.get(anyStore)) + .setCoder(ProtoCoder.of(FeatureRowProto.FeatureRow.class)) + .apply("CountAll", Count.globally())) + .containsInAnyOrder(5L); + + p.run(); + } +} diff --git a/ingestion/src/test/java/feast/ingestion/transform/ProcessAndValidateFeatureRowsTest.java b/ingestion/src/test/java/feast/ingestion/transform/ProcessAndValidateFeatureRowsTest.java index 8c5d7bd8ed6..010a593ab87 100644 --- a/ingestion/src/test/java/feast/ingestion/transform/ProcessAndValidateFeatureRowsTest.java +++ b/ingestion/src/test/java/feast/ingestion/transform/ProcessAndValidateFeatureRowsTest.java @@ -34,7 +34,9 @@ import org.apache.beam.sdk.testing.TestPipeline; import org.apache.beam.sdk.transforms.Count; import org.apache.beam.sdk.transforms.Create; +import org.apache.beam.sdk.transforms.View; import org.apache.beam.sdk.values.PCollectionTuple; +import org.apache.beam.sdk.values.PCollectionView; import org.apache.beam.sdk.values.TupleTag; import org.junit.Rule; import org.junit.Test; @@ -104,6 +106,9 @@ public void shouldWriteSuccessAndFailureTagsCorrectly() { input.add(FeatureRow.newBuilder().setFeatureSet("invalid").build()); + PCollectionView>> specsView = + p.apply("StaticSpecs", Create.of(featureSetSpecs)).apply(View.asMultimap()); + PCollectionTuple output = p.apply(Create.of(input)) .setCoder(ProtoCoder.of(FeatureRow.class)) @@ -112,7 +117,7 @@ public void shouldWriteSuccessAndFailureTagsCorrectly() { .setDefaultProject("myproject") .setFailureTag(FAILURE_TAG) .setSuccessTag(SUCCESS_TAG) - .setFeatureSetSpecs(featureSetSpecs) + .setFeatureSetSpecs(specsView) .build()); PAssert.that(output.get(SUCCESS_TAG)).containsInAnyOrder(expected); @@ -154,6 +159,9 @@ public void shouldStripVersions() { randomRow = randomRow.toBuilder().setFeatureSet("myproject/feature_set:1").build(); input.add(randomRow); + PCollectionView>> specsView = + p.apply("StaticSpecs", Create.of(featureSetSpecs)).apply(View.asMultimap()); + PCollectionTuple output = p.apply(Create.of(input)) .setCoder(ProtoCoder.of(FeatureRow.class)) @@ -162,7 +170,7 @@ public void shouldStripVersions() { .setDefaultProject("myproject") .setFailureTag(FAILURE_TAG) .setSuccessTag(SUCCESS_TAG) - .setFeatureSetSpecs(featureSetSpecs) + .setFeatureSetSpecs(specsView) .build()); PAssert.that(output.get(SUCCESS_TAG)).containsInAnyOrder(expected); @@ -203,6 +211,9 @@ public void shouldApplyDefaultProject() { randomRow = randomRow.toBuilder().setFeatureSet("feature_set").build(); input.add(randomRow); + PCollectionView>> specsView = + p.apply("StaticSpecs", Create.of(featureSetSpecs)).apply(View.asMultimap()); + PCollectionTuple output = p.apply(Create.of(input)) .setCoder(ProtoCoder.of(FeatureRow.class)) @@ -211,7 +222,7 @@ public void shouldApplyDefaultProject() { .setDefaultProject("myproject") .setFailureTag(FAILURE_TAG) .setSuccessTag(SUCCESS_TAG) - .setFeatureSetSpecs(featureSetSpecs) + .setFeatureSetSpecs(specsView) .build()); PAssert.that(output.get(SUCCESS_TAG)).containsInAnyOrder(expected); @@ -241,8 +252,8 @@ public void shouldExcludeUnregisteredFields() { FeatureSpec.newBuilder().setName("feature_2").setValueType(Enum.INT64).build()) .build(); - Map featureSets = new HashMap<>(); - featureSets.put("myproject/feature_set", fs1); + Map featureSetSpecs = new HashMap<>(); + featureSetSpecs.put("myproject/feature_set", fs1); List input = new ArrayList<>(); List expected = new ArrayList<>(); @@ -258,6 +269,9 @@ public void shouldExcludeUnregisteredFields() { .setValue(Value.newBuilder().setStringVal("hello"))) .build()); + PCollectionView>> specsView = + p.apply("StaticSpecs", Create.of(featureSetSpecs)).apply(View.asMultimap()); + PCollectionTuple output = p.apply(Create.of(input)) .setCoder(ProtoCoder.of(FeatureRow.class)) @@ -266,7 +280,7 @@ public void shouldExcludeUnregisteredFields() { .setDefaultProject("myproject") .setFailureTag(FAILURE_TAG) .setSuccessTag(SUCCESS_TAG) - .setFeatureSetSpecs(featureSets) + .setFeatureSetSpecs(specsView) .build()); PAssert.that(output.get(SUCCESS_TAG)).containsInAnyOrder(expected); diff --git a/ingestion/src/test/java/feast/ingestion/transform/specs/FeatureSetSpecReadAndWriteTest.java b/ingestion/src/test/java/feast/ingestion/transform/specs/FeatureSetSpecReadAndWriteTest.java new file mode 100644 index 00000000000..b0cad59b5e0 --- /dev/null +++ b/ingestion/src/test/java/feast/ingestion/transform/specs/FeatureSetSpecReadAndWriteTest.java @@ -0,0 +1,244 @@ +/* + * SPDX-License-Identifier: Apache-2.0 + * Copyright 2018-2020 The Feast Authors + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package feast.ingestion.transform.specs; + +import static feast.ingestion.utils.SpecUtil.getFeatureSetReference; +import static org.hamcrest.CoreMatchers.hasItem; +import static org.hamcrest.MatcherAssert.assertThat; + +import com.google.common.collect.ImmutableList; +import com.google.common.collect.Lists; +import com.google.common.io.Files; +import com.google.protobuf.InvalidProtocolBufferException; +import feast.proto.core.FeatureSetProto; +import feast.proto.core.IngestionJobProto; +import feast.proto.core.SourceProto; +import feast.proto.core.StoreProto; +import feast.test.TestUtil; +import java.io.IOException; +import java.util.List; +import java.util.stream.Collectors; +import org.apache.beam.repackaged.core.org.apache.commons.lang3.tuple.Pair; +import org.apache.beam.runners.direct.DirectOptions; +import org.apache.beam.sdk.PipelineResult; +import org.apache.beam.sdk.options.PipelineOptions; +import org.apache.beam.sdk.options.PipelineOptionsFactory; +import org.apache.beam.sdk.testing.TestPipeline; +import org.apache.kafka.clients.consumer.ConsumerRecord; +import org.apache.kafka.clients.consumer.ConsumerRecords; +import org.apache.kafka.clients.consumer.KafkaConsumer; +import org.apache.kafka.common.serialization.ByteArraySerializer; +import org.apache.kafka.common.serialization.Deserializer; +import org.joda.time.Duration; +import org.junit.*; + +public class FeatureSetSpecReadAndWriteTest { + @Rule public transient TestPipeline p = TestPipeline.fromOptions(makePipelineOptions()); + + private static final String KAFKA_HOST = "localhost"; + private static final int KAFKA_PORT = 29092; + private static final String KAFKA_BOOTSTRAP_SERVERS = KAFKA_HOST + ":" + KAFKA_PORT; + private static final short KAFKA_REPLICATION_FACTOR = 1; + private static final String KAFKA_TOPIC = "topic"; + private static final String KAFKA_SPECS_TOPIC = "topic_specs"; + private static final String KAFKA_SPECS_ACK_TOPIC = "topic_specs_ack"; + + private static final long KAFKA_PUBLISH_TIMEOUT_SEC = 10; + private static final long KAFKA_POLL_TIMEOUT_SEC = 10; + + private KafkaConsumer consumer; + + @SuppressWarnings("UnstableApiUsage") + private static final String ZOOKEEPER_DATA_DIR = Files.createTempDir().getAbsolutePath(); + + private static final String ZOOKEEPER_HOST = "localhost"; + private static final int ZOOKEEPER_PORT = 2183; + + @BeforeClass + public static void setupClass() throws IOException, InterruptedException { + TestUtil.LocalKafka.start( + KAFKA_HOST, + KAFKA_PORT, + KAFKA_REPLICATION_FACTOR, + true, + ZOOKEEPER_HOST, + ZOOKEEPER_PORT, + ZOOKEEPER_DATA_DIR); + } + + @Before + public void setup() { + consumer = + TestUtil.makeKafkaConsumer( + KAFKA_BOOTSTRAP_SERVERS, KAFKA_SPECS_ACK_TOPIC, AckMessageDeserializer.class); + } + + @AfterClass + public static void tearDown() { + TestUtil.LocalKafka.stop(); + } + + public static PipelineOptions makePipelineOptions() { + DirectOptions options = PipelineOptionsFactory.as(DirectOptions.class); + options.setJobName("test_job"); + options.setBlockOnRun(false); + return options; + } + + @Test + public void pipelineShouldReadSpecsAndAcknowledge() { + SourceProto.Source source = + SourceProto.Source.newBuilder() + .setKafkaSourceConfig( + SourceProto.KafkaSourceConfig.newBuilder() + .setBootstrapServers(KAFKA_BOOTSTRAP_SERVERS) + .setTopic(KAFKA_TOPIC) + .build()) + .build(); + + StoreProto.Store store = + StoreProto.Store.newBuilder() + .addSubscriptions( + StoreProto.Store.Subscription.newBuilder() + .setProject("project") + .setName("*") + .build()) + .build(); + + IngestionJobProto.SpecsStreamingUpdateConfig specsStreamingUpdateConfig = + IngestionJobProto.SpecsStreamingUpdateConfig.newBuilder() + .setSource( + SourceProto.KafkaSourceConfig.newBuilder() + .setBootstrapServers(KAFKA_BOOTSTRAP_SERVERS) + .setTopic(KAFKA_SPECS_TOPIC) + .build()) + .setAck( + SourceProto.KafkaSourceConfig.newBuilder() + .setBootstrapServers(KAFKA_BOOTSTRAP_SERVERS) + .setTopic(KAFKA_SPECS_ACK_TOPIC) + .build()) + .build(); + + p.apply( + ReadFeatureSetSpecs.newBuilder() + .setSource(source) + .setStores(ImmutableList.of(store)) + .setSpecsStreamingUpdateConfig(specsStreamingUpdateConfig) + .build()) + .apply( + WriteFeatureSetSpecAck.newBuilder() + .setSpecsStreamingUpdateConfig(specsStreamingUpdateConfig) + .build()); + + // specs' history is being compacted on the initial read + publishSpecToKafka("project", "fs", 1, source); + publishSpecToKafka("project", "fs", 2, source); + publishSpecToKafka("project", "fs", 3, source); + publishSpecToKafka("project", "fs_2", 2, source); + + PipelineResult run = p.run(); + run.waitUntilFinish(Duration.standardSeconds(10)); + + List acks = getFeatureSetSpecAcks(); + + assertThat( + acks, + hasItem( + IngestionJobProto.FeatureSetSpecAck.newBuilder() + .setJobName("test_job") + .setFeatureSetVersion(3) + .setFeatureSetReference("project/fs") + .build())); + assertThat( + acks, + hasItem( + IngestionJobProto.FeatureSetSpecAck.newBuilder() + .setJobName("test_job") + .setFeatureSetVersion(2) + .setFeatureSetReference("project/fs_2") + .build())); + + // in-flight update 1 + publishSpecToKafka("project", "fs", 4, source); + + run.waitUntilFinish(Duration.standardSeconds(5)); + + assertThat( + getFeatureSetSpecAcks(), + hasItem( + IngestionJobProto.FeatureSetSpecAck.newBuilder() + .setJobName("test_job") + .setFeatureSetVersion(4) + .setFeatureSetReference("project/fs") + .build())); + + // in-flight update 2 + publishSpecToKafka("project", "fs_2", 3, source); + + run.waitUntilFinish(Duration.standardSeconds(5)); + + assertThat( + getFeatureSetSpecAcks(), + hasItem( + IngestionJobProto.FeatureSetSpecAck.newBuilder() + .setJobName("test_job") + .setFeatureSetVersion(3) + .setFeatureSetReference("project/fs_2") + .build())); + } + + private List getFeatureSetSpecAcks() { + ConsumerRecords consumerRecords = + consumer.poll(java.time.Duration.ofSeconds(KAFKA_POLL_TIMEOUT_SEC)); + + return Lists.newArrayList(consumerRecords.records(KAFKA_SPECS_ACK_TOPIC)).stream() + .map(ConsumerRecord::value) + .collect(Collectors.toList()); + } + + private void publishSpecToKafka( + String project, String name, int version, SourceProto.Source source) { + FeatureSetProto.FeatureSetSpec spec = + FeatureSetProto.FeatureSetSpec.newBuilder() + .setProject(project) + .setName(name) + .setVersion(version) + .setSource(source) + .build(); + + TestUtil.publishToKafka( + KAFKA_BOOTSTRAP_SERVERS, + KAFKA_SPECS_TOPIC, + ImmutableList.of(Pair.of(getFeatureSetReference(spec), spec)), + ByteArraySerializer.class, + KAFKA_PUBLISH_TIMEOUT_SEC); + } + + public static class AckMessageDeserializer + implements Deserializer { + + @Override + public IngestionJobProto.FeatureSetSpecAck deserialize(String topic, byte[] data) { + try { + return IngestionJobProto.FeatureSetSpecAck.parseFrom(data); + } catch (InvalidProtocolBufferException e) { + e.printStackTrace(); + return null; + } + } + } +} diff --git a/ingestion/src/test/java/feast/ingestion/transform/specs/FilterRelevantTest.java b/ingestion/src/test/java/feast/ingestion/transform/specs/FilterRelevantTest.java new file mode 100644 index 00000000000..33736dad41d --- /dev/null +++ b/ingestion/src/test/java/feast/ingestion/transform/specs/FilterRelevantTest.java @@ -0,0 +1,109 @@ +/* + * SPDX-License-Identifier: Apache-2.0 + * Copyright 2018-2020 The Feast Authors + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package feast.ingestion.transform.specs; + +import com.google.common.collect.ImmutableList; +import com.google.common.collect.ImmutableMap; +import feast.proto.core.FeatureSetProto; +import feast.proto.core.SourceProto; +import feast.proto.core.StoreProto; +import org.apache.beam.sdk.testing.PAssert; +import org.apache.beam.sdk.testing.TestPipeline; +import org.apache.beam.sdk.transforms.Count; +import org.apache.beam.sdk.transforms.Create; +import org.apache.beam.sdk.transforms.Filter; +import org.apache.beam.sdk.values.KV; +import org.apache.beam.sdk.values.PCollection; +import org.junit.Rule; +import org.junit.Test; + +public class FilterRelevantTest { + @Rule public transient TestPipeline p = TestPipeline.create(); + + @Test + public void onlySpecsThatMatchStoreSubscriptionShouldPass() { + SourceProto.Source source = + SourceProto.Source.newBuilder() + .setKafkaSourceConfig( + SourceProto.KafkaSourceConfig.newBuilder() + .setBootstrapServers("localhost") + .setTopic("topic") + .build()) + .build(); + + StoreProto.Store store1 = + StoreProto.Store.newBuilder() + .addSubscriptions( + StoreProto.Store.Subscription.newBuilder() + .setProject("project") + .setName("fs*") + .build()) + .build(); + + StoreProto.Store store2 = + StoreProto.Store.newBuilder() + .addSubscriptions( + StoreProto.Store.Subscription.newBuilder() + .setProject("project_2") + .setName("fs_1") + .build()) + .build(); + + SourceProto.Source irrelevantSource = + SourceProto.Source.newBuilder() + .setKafkaSourceConfig( + SourceProto.KafkaSourceConfig.newBuilder() + .setBootstrapServers("localhost") + .setTopic("other_topic") + .build()) + .build(); + + PCollection> filtered = + p.apply( + Create.of( + ImmutableMap.of( + "project/fs_1", makeSpecBuilder(source).build(), // pass + "project/fs_2", + makeSpecBuilder(irrelevantSource).build(), // different source + "invalid/fs_3", + makeSpecBuilder(source) + .setProject("invalid") + .build(), // invalid project + "project/invalid", + makeSpecBuilder(source).setName("invalid").build(), // invalid name + "project_2/fs_1", + makeSpecBuilder(source).setProject("project_2").build() // pass + ))) + .apply(Filter.by(new FilterRelevantFunction(source, ImmutableList.of(store1, store2)))); + + PAssert.that(filtered) + .containsInAnyOrder( + KV.of("project/fs_1", makeSpecBuilder(source).build()), + KV.of("project_2/fs_1", makeSpecBuilder(source).setProject("project_2").build())); + + PAssert.that(filtered.apply(Count.globally())).containsInAnyOrder(2L); + + p.run(); + } + + private FeatureSetProto.FeatureSetSpec.Builder makeSpecBuilder(SourceProto.Source source) { + return FeatureSetProto.FeatureSetSpec.newBuilder() + .setProject("project") + .setName("fs_1") + .setSource(source); + } +} diff --git a/ingestion/src/test/java/feast/test/TestUtil.java b/ingestion/src/test/java/feast/test/TestUtil.java index 3204d93bcda..91a8b9ba7ea 100644 --- a/ingestion/src/test/java/feast/test/TestUtil.java +++ b/ingestion/src/test/java/feast/test/TestUtil.java @@ -18,7 +18,10 @@ import static feast.ingestion.utils.SpecUtil.getFeatureSetReference; +import com.google.common.collect.ImmutableList; +import com.google.common.io.Files; import com.google.protobuf.ByteString; +import com.google.protobuf.Message; import com.google.protobuf.util.Timestamps; import feast.ingestion.transform.metrics.WriteSuccessMetricsTransform; import feast.proto.core.FeatureSetProto.FeatureSet; @@ -43,15 +46,20 @@ import java.util.concurrent.TimeoutException; import kafka.server.KafkaConfig; import kafka.server.KafkaServerStartable; +import org.apache.beam.repackaged.core.org.apache.commons.lang3.tuple.Pair; import org.apache.beam.sdk.PipelineResult; import org.apache.beam.sdk.PipelineResult.State; import org.apache.beam.sdk.metrics.MetricResult; import org.apache.beam.sdk.metrics.MetricResults; import org.apache.commons.lang3.RandomStringUtils; +import org.apache.kafka.clients.consumer.ConsumerConfig; +import org.apache.kafka.clients.consumer.KafkaConsumer; import org.apache.kafka.clients.producer.KafkaProducer; import org.apache.kafka.clients.producer.Producer; +import org.apache.kafka.clients.producer.ProducerConfig; import org.apache.kafka.clients.producer.ProducerRecord; -import org.apache.kafka.common.serialization.LongSerializer; +import org.apache.kafka.common.serialization.StringDeserializer; +import org.apache.kafka.common.serialization.StringSerializer; import org.apache.zookeeper.server.ServerConfig; import org.apache.zookeeper.server.ZooKeeperServerMain; import org.joda.time.Duration; @@ -113,16 +121,19 @@ public static void start( kafkaProp.put("zookeeper.connect", zookeeperHost + ":" + zookeeperPort); kafkaProp.put("host.name", kafkaHost); kafkaProp.put("port", kafkaPort); + kafkaProp.put("log.dir", Files.createTempDir().getAbsolutePath()); kafkaProp.put("offsets.topic.replication.factor", kafkaReplicationFactor); KafkaConfig kafkaConfig = new KafkaConfig(kafkaProp); server = new KafkaServerStartable(kafkaConfig); new Thread(server::startup).start(); + Thread.sleep(5000); } public static void stop() { if (server != null) { try { server.shutdown(); + LocalZookeeper.stop(); } catch (Exception e) { e.printStackTrace(); } @@ -139,23 +150,24 @@ public static void stop() { * @param valueSerializer in Feast this valueSerializer should be "ByteArraySerializer.class" * @param publishTimeoutSec duration to wait for publish operation (of each message) to succeed */ - public static void publishFeatureRowsToKafka( + public static void publishToKafka( String bootstrapServers, String topic, - List messages, + List> messages, Class valueSerializer, long publishTimeoutSec) { - Long defaultKey = 1L; + Properties prop = new Properties(); - prop.put("bootstrap.servers", bootstrapServers); - prop.put("key.serializer", LongSerializer.class); - prop.put("value.serializer", valueSerializer); - Producer producer = new KafkaProducer<>(prop); + prop.put(ProducerConfig.BOOTSTRAP_SERVERS_CONFIG, bootstrapServers); + prop.put(ProducerConfig.KEY_SERIALIZER_CLASS_CONFIG, StringSerializer.class); + prop.put(ProducerConfig.VALUE_SERIALIZER_CLASS_CONFIG, valueSerializer); + Producer producer = new KafkaProducer<>(prop); messages.forEach( featureRow -> { - ProducerRecord record = - new ProducerRecord<>(topic, defaultKey, featureRow.toByteArray()); + ProducerRecord record = + new ProducerRecord<>( + topic, featureRow.getLeft(), featureRow.getRight().toByteArray()); try { producer.send(record).get(publishTimeoutSec, TimeUnit.SECONDS); } catch (InterruptedException | ExecutionException | TimeoutException e) { @@ -164,6 +176,22 @@ public static void publishFeatureRowsToKafka( }); } + public static KafkaConsumer makeKafkaConsumer( + String bootstrapServers, String topic, Class valueDeserializer) { + Properties prop = new Properties(); + prop.put(ConsumerConfig.BOOTSTRAP_SERVERS_CONFIG, bootstrapServers); + prop.put(ConsumerConfig.KEY_DESERIALIZER_CLASS_CONFIG, StringDeserializer.class); + prop.put(ConsumerConfig.VALUE_DESERIALIZER_CLASS_CONFIG, valueDeserializer); + prop.put(ConsumerConfig.AUTO_OFFSET_RESET_CONFIG, "earliest"); + prop.put(ConsumerConfig.ENABLE_AUTO_COMMIT_CONFIG, false); + prop.put(ConsumerConfig.GROUP_ID_CONFIG, "test"); + + KafkaConsumer consumer = new KafkaConsumer<>(prop); + + consumer.subscribe(ImmutableList.of(topic)); + return consumer; + } + /** * Create a Feature Row with random value according to the FeatureSetSpec * @@ -315,20 +343,26 @@ public static RedisKey createRedisKey(FeatureSetSpec featureSetSpec, FeatureRow } private static class LocalZookeeper { + public static Thread thread; static void start(int zookeeperPort, String zookeeperDataDir) { - final ZooKeeperServerMain zookeeper = new ZooKeeperServerMain(); + ZooKeeperServerMain zookeeper = new ZooKeeperServerMain(); final ServerConfig serverConfig = new ServerConfig(); serverConfig.parse(new String[] {String.valueOf(zookeeperPort), zookeeperDataDir}); - new Thread( + thread = + new Thread( () -> { try { zookeeper.runFromConfig(serverConfig); - } catch (IOException e) { + } catch (Exception e) { e.printStackTrace(); } - }) - .start(); + }); + thread.start(); + } + + static void stop() { + thread.interrupt(); } } diff --git a/protos/feast/core/FeatureSet.proto b/protos/feast/core/FeatureSet.proto index b0b15276bcf..de22388ce76 100644 --- a/protos/feast/core/FeatureSet.proto +++ b/protos/feast/core/FeatureSet.proto @@ -63,6 +63,10 @@ message FeatureSetSpec { // User defined metadata map labels = 8; + + // Read-only self-incrementing version that increases monotonically + // when changes are made to a feature set + int32 version = 9; } message EntitySpec { @@ -150,3 +154,8 @@ enum FeatureSetStatus { STATUS_JOB_STARTING = 3; STATUS_READY = 2; } + +enum FeatureSetJobDeliveryStatus { + STATUS_IN_PROGRESS = 0; + STATUS_DELIVERED = 1; +} \ No newline at end of file diff --git a/protos/feast/core/IngestionJob.proto b/protos/feast/core/IngestionJob.proto index c63e573ffea..0ce0f9681c0 100644 --- a/protos/feast/core/IngestionJob.proto +++ b/protos/feast/core/IngestionJob.proto @@ -63,3 +63,18 @@ enum IngestionJobStatus { // job has been suspended SUSPENDED = 8; } + +// Config for bi-directional communication channel between Core Service and Ingestion Job +message SpecsStreamingUpdateConfig { + // out-channel for publishing new FeatureSetSpecs (by Core). + // IngestionJob use it as source of existing FeatureSetSpecs and new real-time updates + feast.core.KafkaSourceConfig source = 1; + // ack-channel for sending acknowledgments when new FeatureSetSpecs is installed in Job + feast.core.KafkaSourceConfig ack = 2; +} + +message FeatureSetSpecAck { + string feature_set_reference = 1; + int32 feature_set_version = 2; + string job_name = 3; +} \ No newline at end of file From a5bd8775b12ec7e2fde0e20a95c2ce32e12f57ea Mon Sep 17 00:00:00 2001 From: Terence Lim Date: Wed, 17 Jun 2020 18:31:04 +0800 Subject: [PATCH 063/133] Add API for listing feature sets using labels (#785) * Update CoreService proto * Update SpecService and add unit test * Update Python sdk and cli * Fix python lint error * Add more tests * Black formatting changes * Revert "Black formatting changes" This reverts commit d82200f25383fed9a81ebd3b973fb4797208e5fe. * Refactor FeatureSet labels filter logic * Set default labels to empty dict * Refactor feature_set_list cli * Refactor brittle test * Implement helper function and not convert to proto * Update cli * Add docstrings and improve variable naming * Use getter method instead * Fix java lint * Add description for labels filter proto Co-authored-by: Terence --- .../java/feast/core/model/FeatureSet.java | 21 +++++ .../java/feast/core/service/SpecService.java | 16 +++- .../feast/core/service/SpecServiceTest.java | 76 ++++++++++++++++++- protos/feast/core/CoreService.proto | 4 + sdk/python/feast/cli.py | 44 ++++++++++- sdk/python/feast/client.py | 6 +- sdk/python/tests/test_client.py | 62 +++++++++++++++ tests/e2e/redis/basic-ingest-redis-serving.py | 28 +++++++ 8 files changed, 248 insertions(+), 9 deletions(-) diff --git a/core/src/main/java/feast/core/model/FeatureSet.java b/core/src/main/java/feast/core/model/FeatureSet.java index e1b50098b90..67a21e0b250 100644 --- a/core/src/main/java/feast/core/model/FeatureSet.java +++ b/core/src/main/java/feast/core/model/FeatureSet.java @@ -128,6 +128,23 @@ private String getProjectName() { } } + /** + * Return a boolean to facilitate streaming elements on the basis of given predicate. + * + * @param labelsFilter labels contain key-value mapping for labels attached to the FeatureSet + * @return boolean True if FeatureSet contains all labels in the labelsFilter + */ + public boolean hasAllLabels(Map labelsFilter) { + Map featureSetLabelsMap = this.getLabelsMap(); + for (String key : labelsFilter.keySet()) { + if (!featureSetLabelsMap.containsKey(key) + || !featureSetLabelsMap.get(key).equals(labelsFilter.get(key))) { + return false; + } + } + return true; + } + public void setProject(Project project) { this.project = project; } @@ -293,6 +310,10 @@ public FeatureSetProto.FeatureSet toProto() throws InvalidProtocolBufferExceptio return FeatureSetProto.FeatureSet.newBuilder().setMeta(meta).setSpec(spec).build(); } + public Map getLabelsMap() { + return TypeConversion.convertJsonStringToMap(this.getLabels()); + } + @Override public int hashCode() { HashCodeBuilder hcb = new HashCodeBuilder(); diff --git a/core/src/main/java/feast/core/service/SpecService.java b/core/src/main/java/feast/core/service/SpecService.java index 9000ee983ee..a4f1202858d 100644 --- a/core/src/main/java/feast/core/service/SpecService.java +++ b/core/src/main/java/feast/core/service/SpecService.java @@ -46,7 +46,9 @@ import feast.proto.core.StoreProto.Store.Subscription; import java.util.ArrayList; import java.util.List; +import java.util.Map; import java.util.concurrent.TimeUnit; +import java.util.stream.Collectors; import lombok.extern.slf4j.Slf4j; import org.apache.kafka.clients.consumer.ConsumerRecord; import org.springframework.beans.factory.annotation.Autowired; @@ -123,9 +125,9 @@ public GetFeatureSetResponse getFeatureSet(GetFeatureSetRequest request) } /** - * Return a list of feature sets matching the feature set name and project provided in the filter. - * All fields are requried. Use '*' for all arguments in order to return all feature sets in all - * projects. + * Return a list of feature sets matching the feature set name, project and labels provided in the + * filter. All fields are required. Use '*' in feature set name and project, and empty map in + * labels in order to return all feature sets in all projects. * *

Project name can be explicitly provided, or an asterisk can be provided to match all * projects. It is not possible to provide a combination of asterisks/wildcards and text. If the @@ -135,6 +137,9 @@ public GetFeatureSetResponse getFeatureSet(GetFeatureSetRequest request) * sets will be returned. Regex is not supported. Explicitly defining a feature set name is not * possible if a project name is not set explicitly * + *

The labels in the filter accepts a map. All feature sets which contain every provided label + * will be returned. + * * @param filter filter containing the desired featureSet name * @return ListFeatureSetsResponse with list of featureSets found matching the filter */ @@ -142,6 +147,7 @@ public ListFeatureSetsResponse listFeatureSets(ListFeatureSetsRequest.Filter fil throws InvalidProtocolBufferException { String name = filter.getFeatureSetName(); String project = filter.getProject(); + Map labelsFilter = filter.getLabelsMap(); if (name.isEmpty()) { throw new IllegalArgumentException( @@ -197,6 +203,10 @@ public ListFeatureSetsResponse listFeatureSets(ListFeatureSetsRequest.Filter fil ListFeatureSetsResponse.Builder response = ListFeatureSetsResponse.newBuilder(); if (featureSets.size() > 0) { + featureSets = + featureSets.stream() + .filter(featureSet -> featureSet.hasAllLabels(labelsFilter)) + .collect(Collectors.toList()); for (FeatureSet featureSet : featureSets) { response.addFeatureSets(featureSet.toProto()); } diff --git a/core/src/test/java/feast/core/service/SpecServiceTest.java b/core/src/test/java/feast/core/service/SpecServiceTest.java index 9ec219b6c97..20a84edbdaa 100644 --- a/core/src/test/java/feast/core/service/SpecServiceTest.java +++ b/core/src/test/java/feast/core/service/SpecServiceTest.java @@ -106,7 +106,7 @@ public class SpecServiceTest { // TODO: Updates update features in place, so if tests follow the wrong order they might break. // Refactor this maybe? @Before - public void setUp() { + public void setUp() throws InvalidProtocolBufferException { initMocks(this); defaultSource = TestObjectFactory.defaultSource; @@ -121,7 +121,50 @@ public void setUp() { "f3", "project1", Arrays.asList(f3e1), Arrays.asList(f3f2, f3f1)); FeatureSet featureSet4 = newDummyFeatureSet("f4", Project.DEFAULT_NAME); - featureSets = Arrays.asList(featureSet1, featureSet2, featureSet3, featureSet4); + Map singleFeatureSetLabels = + new HashMap<>() { + { + put("fsLabel1", "fsValue1"); + } + }; + Map duoFeatureSetLabels = + new HashMap<>() { + { + put("fsLabel1", "fsValue1"); + put("fsLabel2", "fsValue2"); + } + }; + FeatureSet featureSet5 = newDummyFeatureSet("f5", Project.DEFAULT_NAME); + FeatureSet featureSet6 = newDummyFeatureSet("f6", Project.DEFAULT_NAME); + FeatureSetSpec featureSetSpec5 = featureSet5.toProto().getSpec().toBuilder().build(); + FeatureSetSpec featureSetSpec6 = featureSet6.toProto().getSpec().toBuilder().build(); + FeatureSetProto.FeatureSet fs5 = + FeatureSetProto.FeatureSet.newBuilder() + .setSpec( + featureSetSpec5 + .toBuilder() + .setSource(defaultSource.toProto()) + .putAllLabels(singleFeatureSetLabels) + .build()) + .build(); + FeatureSetProto.FeatureSet fs6 = + FeatureSetProto.FeatureSet.newBuilder() + .setSpec( + featureSetSpec6 + .toBuilder() + .setSource(defaultSource.toProto()) + .putAllLabels(duoFeatureSetLabels) + .build()) + .build(); + + featureSets = + Arrays.asList( + featureSet1, + featureSet2, + featureSet3, + featureSet4, + FeatureSet.fromProto(fs5), + FeatureSet.fromProto(fs6)); when(featureSetRepository.findAll()).thenReturn(featureSets); when(featureSetRepository.findAllByOrderByNameAsc()).thenReturn(featureSets); @@ -713,6 +756,35 @@ public void applyFeatureSetShouldAcceptFeatureSetLabels() throws InvalidProtocol assertEquals(featureSetLabels, appliedLabels); } + @Test + public void shouldFilterByFeatureSetLabels() throws InvalidProtocolBufferException { + List list = new ArrayList<>(); + ListFeatureSetsResponse actual1 = + specService.listFeatureSets( + Filter.newBuilder() + .setFeatureSetName("*") + .setProject("*") + .putLabels("fsLabel2", "fsValue2") + .build()); + list.add(featureSets.get(5).toProto()); + ListFeatureSetsResponse expected1 = + ListFeatureSetsResponse.newBuilder().addAllFeatureSets(list).build(); + + ListFeatureSetsResponse actual2 = + specService.listFeatureSets( + Filter.newBuilder() + .setFeatureSetName("*") + .setProject("*") + .putLabels("fsLabel1", "fsValue1") + .build()); + list.add(0, featureSets.get(4).toProto()); + ListFeatureSetsResponse expected2 = + ListFeatureSetsResponse.newBuilder().addAllFeatureSets(list).build(); + + assertThat(actual1, equalTo(expected1)); + assertThat(actual2, equalTo(expected2)); + } + @Test public void shouldUpdateStoreIfConfigChanges() throws InvalidProtocolBufferException { when(storeRepository.findById("SERVING")).thenReturn(Optional.of(stores.get(0))); diff --git a/protos/feast/core/CoreService.proto b/protos/feast/core/CoreService.proto index 05755b701d8..a6ac313b2a5 100644 --- a/protos/feast/core/CoreService.proto +++ b/protos/feast/core/CoreService.proto @@ -140,6 +140,10 @@ message ListFeatureSetsRequest { // - my-feature-set* can be used to match all features prefixed by "my-feature-set" // - my-feature-set-6 can be used to select a single feature set string feature_set_name = 1; + + // User defined metadata for feature set. + // Feature sets with all matching labels will be returned. + map labels = 4; } } diff --git a/sdk/python/feast/cli.py b/sdk/python/feast/cli.py index 4db41f92f18..34a3f600e15 100644 --- a/sdk/python/feast/cli.py +++ b/sdk/python/feast/cli.py @@ -120,15 +120,55 @@ def feature_set(): pass +def _get_labels_dict(label_str: str): + """ + Converts CLI input labels string to dictionary format if provided string is valid. + """ + labels_dict = {} + labels_kv = label_str.split(",") + if label_str == "": + return labels_dict + if len(labels_kv) % 2 == 1: + raise ValueError("Uneven key-value label pairs were entered") + for k, v in zip(labels_kv[0::2], labels_kv[1::2]): + labels_dict[k] = v + return labels_dict + + @feature_set.command(name="list") -def feature_set_list(): +@click.option( + "--project", + "-p", + help="Project that feature set belongs to", + type=click.STRING, + default="*", +) +@click.option( + "--name", + "-n", + help="Filters feature sets by name. Wildcards (*) may be included to match multiple feature sets", + type=click.STRING, + default="*", +) +@click.option( + "--labels", + "-l", + help="Labels to filter for feature sets", + type=click.STRING, + default="", +) +def feature_set_list(project: str, name: str, labels: str): """ List all feature sets """ feast_client = Client() # type: Client + labels_dict = _get_labels_dict(labels) + table = [] - for fs in feast_client.list_feature_sets(project="*", name="*"): + for fs in feast_client.list_feature_sets( + project=project, name=name, labels=labels_dict + ): table.append([fs.name, repr(fs)]) from tabulate import tabulate diff --git a/sdk/python/feast/client.py b/sdk/python/feast/client.py index 316573ce93d..2ec90e434db 100644 --- a/sdk/python/feast/client.py +++ b/sdk/python/feast/client.py @@ -425,7 +425,7 @@ def _apply_feature_set(self, feature_set: FeatureSet): feature_set._update_from_feature_set(applied_fs) def list_feature_sets( - self, project: str = None, name: str = None, + self, project: str = None, name: str = None, labels: Dict[str, str] = dict() ) -> List[FeatureSet]: """ Retrieve a list of feature sets from Feast Core @@ -448,7 +448,9 @@ def list_feature_sets( if name is None: name = "*" - filter = ListFeatureSetsRequest.Filter(project=project, feature_set_name=name) + filter = ListFeatureSetsRequest.Filter( + project=project, feature_set_name=name, labels=labels + ) # Get latest feature sets from Feast Core feature_set_protos = self._core_service_stub.ListFeatureSets( diff --git a/sdk/python/tests/test_client.py b/sdk/python/tests/test_client.py index ee89b83737a..85fea31dbb1 100644 --- a/sdk/python/tests/test_client.py +++ b/sdk/python/tests/test_client.py @@ -34,6 +34,7 @@ from feast.core.CoreService_pb2 import ( GetFeastCoreVersionResponse, GetFeatureSetResponse, + ListFeatureSetsResponse, ListIngestionJobsResponse, ) from feast.core.FeatureSet_pb2 import EntitySpec as EntitySpecProto @@ -321,6 +322,67 @@ def test_get_feature_set(self, mocked_client, mocker): and len(feature_set.entities) == 1 ) + @pytest.mark.parametrize( + "mocked_client", + [pytest.lazy_fixture("mock_client"), pytest.lazy_fixture("secure_mock_client")], + ) + def test_list_feature_sets(self, mocked_client, mocker): + mocker.patch.object( + mocked_client, + "_core_service_stub", + return_value=Core.CoreServiceStub(grpc.insecure_channel("")), + ) + + feature_set_1_proto = FeatureSetProto( + spec=FeatureSetSpecProto( + project="test", + name="driver_car", + max_age=Duration(seconds=3600), + labels={"key1": "val1", "key2": "val2"}, + features=[ + FeatureSpecProto( + name="feature_1", value_type=ValueProto.ValueType.FLOAT + ) + ], + ) + ) + feature_set_2_proto = FeatureSetProto( + spec=FeatureSetSpecProto( + project="test", + name="driver_ride", + max_age=Duration(seconds=3600), + labels={"key1": "val1"}, + features=[ + FeatureSpecProto( + name="feature_1", value_type=ValueProto.ValueType.FLOAT + ) + ], + ) + ) + + mocker.patch.object( + mocked_client._core_service_stub, + "ListFeatureSets", + return_value=ListFeatureSetsResponse( + feature_sets=[feature_set_1_proto, feature_set_2_proto] + ), + ) + + feature_sets = mocked_client.list_feature_sets(labels={"key1": "val1"}) + assert len(feature_sets) == 2 + + feature_set = feature_sets[0] + assert ( + feature_set.name == "driver_car" + and "key1" in feature_set.labels + and feature_set.labels["key1"] == "val1" + and "key2" in feature_set.labels + and feature_set.labels["key2"] == "val2" + and feature_set.fields["feature_1"].name == "feature_1" + and feature_set.fields["feature_1"].dtype == ValueType.FLOAT + and len(feature_set.features) == 1 + ) + @pytest.mark.parametrize( "mocked_client", [pytest.lazy_fixture("mock_client"), pytest.lazy_fixture("secure_mock_client")], diff --git a/tests/e2e/redis/basic-ingest-redis-serving.py b/tests/e2e/redis/basic-ingest-redis-serving.py index 63a3c794a5a..1424936f0c8 100644 --- a/tests/e2e/redis/basic-ingest-redis-serving.py +++ b/tests/e2e/redis/basic-ingest-redis-serving.py @@ -126,6 +126,34 @@ def test_basic_register_feature_set_success(client): project=PROJECT_NAME) assert cust_trans_fs_actual == cust_trans_fs_expected + # Register feature set with labels + driver_unlabelled_fs = FeatureSet( + "driver_unlabelled", + features=[ + Feature("rating", ValueType.FLOAT), + Feature("cost", ValueType.FLOAT) + ], + entities=[Entity("entity_id", ValueType.INT64)], + max_age=Duration(seconds=100) + ) + driver_labeled_fs_expected = FeatureSet( + "driver_labeled", + features=[ + Feature("rating", ValueType.FLOAT), + Feature("cost", ValueType.FLOAT) + ], + entities=[Entity("entity_id", ValueType.INT64)], + max_age=Duration(seconds=100), + labels={"key1":"val1"} + ) + client.set_project(PROJECT_NAME) + client.apply(driver_unlabelled_fs) + client.apply(driver_labeled_fs_expected) + driver_fs_actual = client.list_feature_sets( + project=PROJECT_NAME, labels={"key1": "val1"} + )[0] + assert driver_fs_actual == driver_labeled_fs_expected + # reset client's project for other tests client.set_project() From e343c2fc8d98d37473d78297726f8b1299d1998d Mon Sep 17 00:00:00 2001 From: Terence Lim Date: Thu, 18 Jun 2020 11:51:05 +0800 Subject: [PATCH 064/133] Add Discovery API for listing Features (#797) * Add feature and entity listing in feast core * Update Core protos * Update Python SDK * Add tests for new listing functionality * Fix python linting * Fix python linting * Fix e2e redis test * Remove check that relies on other feature set * Address PR comments * Address more PR comments * Revert ListEntities implementation * Apply spotless * Add description for ListFeaturesRequest proto * Fix comment * Apply spotless Co-authored-by: Terence Co-authored-by: Willem Pienaar <6728866+woop@users.noreply.github.com> --- .../java/feast/core/grpc/CoreServiceImpl.java | 26 +++ .../main/java/feast/core/model/Feature.java | 17 ++ .../java/feast/core/model/FeatureSet.java | 81 +++++++ .../java/feast/core/service/SpecService.java | 62 ++++++ .../feast/core/service/SpecServiceTest.java | 199 +++++++++++++++++- .../feast/core/service/TestObjectFactory.java | 11 + protos/feast/core/CoreService.proto | 29 +++ sdk/python/feast/cli.py | 75 +++++++ sdk/python/feast/client.py | 53 ++++- sdk/python/tests/test_client.py | 47 +++++ tests/e2e/redis/basic-ingest-redis-serving.py | 65 ++++++ 11 files changed, 662 insertions(+), 3 deletions(-) diff --git a/core/src/main/java/feast/core/grpc/CoreServiceImpl.java b/core/src/main/java/feast/core/grpc/CoreServiceImpl.java index 7a81690a582..8f17e78b001 100644 --- a/core/src/main/java/feast/core/grpc/CoreServiceImpl.java +++ b/core/src/main/java/feast/core/grpc/CoreServiceImpl.java @@ -107,6 +107,32 @@ public void listFeatureSets( } } + /** Retrieve a list of features */ + @Override + public void listFeatures( + ListFeaturesRequest request, StreamObserver responseObserver) { + try { + ListFeaturesResponse response = specService.listFeatures(request.getFilter()); + responseObserver.onNext(response); + responseObserver.onCompleted(); + } catch (IllegalArgumentException e) { + log.error("Illegal arguments provided to ListFeatures method: ", e); + responseObserver.onError( + Status.INVALID_ARGUMENT + .withDescription(e.getMessage()) + .withCause(e) + .asRuntimeException()); + } catch (RetrievalException e) { + log.error("Unable to fetch entities requested in ListFeatures method: ", e); + responseObserver.onError( + Status.NOT_FOUND.withDescription(e.getMessage()).withCause(e).asRuntimeException()); + } catch (Exception e) { + log.error("Exception has occurred in ListFeatures method: ", e); + responseObserver.onError( + Status.INTERNAL.withDescription(e.getMessage()).withCause(e).asRuntimeException()); + } + } + @Override public void getFeatureStatistics( GetFeatureStatisticsRequest request, diff --git a/core/src/main/java/feast/core/model/Feature.java b/core/src/main/java/feast/core/model/Feature.java index 0b457493792..87978e65ce3 100644 --- a/core/src/main/java/feast/core/model/Feature.java +++ b/core/src/main/java/feast/core/model/Feature.java @@ -91,6 +91,23 @@ private Feature(String name, ValueType.Enum type) { this.setType(type.toString()); } + /** + * Return a boolean to facilitate streaming elements on the basis of given predicate. + * + * @param labelsFilter contain labels that should be attached to Feature + * @return boolean True if Feature contains all labels in the labelsFilter + */ + public boolean hasAllLabels(Map labelsFilter) { + Map featureLabelsMap = this.getLabels(); + for (String key : labelsFilter.keySet()) { + if (!featureLabelsMap.containsKey(key) + || !featureLabelsMap.get(key).equals(labelsFilter.get(key))) { + return false; + } + } + return true; + } + public static Feature fromProto(FeatureSpec featureSpec) { Feature feature = new Feature(featureSpec.getName(), featureSpec.getValueType()); feature.labels = TypeConversion.convertMapToJsonString(featureSpec.getLabelsMap()); diff --git a/core/src/main/java/feast/core/model/FeatureSet.java b/core/src/main/java/feast/core/model/FeatureSet.java index 67a21e0b250..d1e8c5ee140 100644 --- a/core/src/main/java/feast/core/model/FeatureSet.java +++ b/core/src/main/java/feast/core/model/FeatureSet.java @@ -23,6 +23,7 @@ import feast.core.util.TypeConversion; import feast.proto.core.FeatureSetProto; import feast.proto.core.FeatureSetProto.*; +import feast.proto.serving.ServingAPIProto.FeatureReference; import java.util.*; import java.util.stream.Collectors; import javax.persistence.*; @@ -128,6 +129,86 @@ private String getProjectName() { } } + /** + * Return a boolean to facilitate streaming elements on the basis of given predicate. + * + * @param entitiesFilter contain entities that should be attached to the FeatureSet + * @return boolean True if FeatureSet contains all entities in the entitiesFilter + */ + public boolean hasAllEntities(List entitiesFilter) { + List allEntitiesName = + this.getEntities().stream().map(entity -> entity.getName()).collect(Collectors.toList()); + return allEntitiesName.equals(entitiesFilter); + } + + /** + * Returns a map of Feature references and Features if FeatureSet's Feature contains all labels in + * the labelsFilter + * + * @param labelsFilter contain labels that should be attached to FeatureSet's features + * @return Map of Feature references and Features + */ + public Map getFeaturesByRef(Map labelsFilter) { + Map validFeaturesMap = new HashMap<>(); + List validFeatures; + if (labelsFilter.size() > 0) { + validFeatures = filterFeaturesByAllLabels(this.getFeatures(), labelsFilter); + for (Feature feature : validFeatures) { + FeatureReference featureRef = + FeatureReference.newBuilder() + .setProject(this.getProjectName()) + .setFeatureSet(this.getName()) + .setName(feature.getName()) + .build(); + validFeaturesMap.put(renderFeatureRef(featureRef), feature); + } + return validFeaturesMap; + } + for (Feature feature : this.getFeatures()) { + FeatureReference featureRef = + FeatureReference.newBuilder() + .setProject(this.getProjectName()) + .setFeatureSet(this.getName()) + .setName(feature.getName()) + .build(); + validFeaturesMap.put(renderFeatureRef(featureRef), feature); + } + return validFeaturesMap; + } + + /** + * Returns a list of Features if FeatureSet's Feature contains all labels in labelsFilter + * + * @param labelsFilter contain labels that should be attached to FeatureSet's features + * @return List of Features + */ + public static List filterFeaturesByAllLabels( + Set features, Map labelsFilter) { + List validFeatures = + features.stream() + .filter(feature -> feature.hasAllLabels(labelsFilter)) + .collect(Collectors.toList()); + + return validFeatures; + } + + /** + * Render a feature reference as string. + * + * @param featureReference to render as string + * @return string representation of feature reference. + */ + public static String renderFeatureRef(FeatureReference featureReference) { + String refStr = + featureReference.getProject() + + "/" + + featureReference.getFeatureSet() + + ":" + + featureReference.getName(); + + return refStr; + } + /** * Return a boolean to facilitate streaming elements on the basis of given predicate. * diff --git a/core/src/main/java/feast/core/service/SpecService.java b/core/src/main/java/feast/core/service/SpecService.java index a4f1202858d..e60aeee57d3 100644 --- a/core/src/main/java/feast/core/service/SpecService.java +++ b/core/src/main/java/feast/core/service/SpecService.java @@ -33,6 +33,8 @@ import feast.proto.core.CoreServiceProto.GetFeatureSetResponse; import feast.proto.core.CoreServiceProto.ListFeatureSetsRequest; import feast.proto.core.CoreServiceProto.ListFeatureSetsResponse; +import feast.proto.core.CoreServiceProto.ListFeaturesRequest; +import feast.proto.core.CoreServiceProto.ListFeaturesResponse; import feast.proto.core.CoreServiceProto.ListStoresRequest; import feast.proto.core.CoreServiceProto.ListStoresResponse; import feast.proto.core.CoreServiceProto.ListStoresResponse.Builder; @@ -215,6 +217,65 @@ public ListFeatureSetsResponse listFeatureSets(ListFeatureSetsRequest.Filter fil return response.build(); } + /** + * Return a map of feature references and features matching the project, labels and entities + * provided in the filter. All fields are required. + * + *

Project name must be explicitly provided or if the project name is omitted, the default + * project would be used. A combination of asterisks/wildcards and text is not allowed. + * + *

The entities in the filter accepts a list. All matching features will be returned. Regex is + * not supported. If no entities are provided, features will not be filtered by entities. + * + *

The labels in the filter accepts a map. All matching features will be returned. Regex is not + * supported. If no labels are provided, features will not be filtered by labels. + * + * @param filter filter containing the desired project name, entities and labels + * @return ListEntitiesResponse with map of feature references and features found matching the + * filter + */ + public ListFeaturesResponse listFeatures(ListFeaturesRequest.Filter filter) { + try { + String project = filter.getProject(); + List entities = filter.getEntitiesList(); + Map labels = filter.getLabelsMap(); + + checkValidCharactersAllowAsterisk(project, "projectName"); + + // Autofill default project if project not specified + if (project.isEmpty()) { + project = Project.DEFAULT_NAME; + } + + // Currently defaults to all FeatureSets + List featureSets = + featureSetRepository.findAllByNameLikeAndProject_NameOrderByNameAsc("%", project); + + ListFeaturesResponse.Builder response = ListFeaturesResponse.newBuilder(); + if (entities.size() > 0) { + featureSets = + featureSets.stream() + .filter(featureSet -> featureSet.hasAllEntities(entities)) + .collect(Collectors.toList()); + } + + Map featuresMap; + for (FeatureSet featureSet : featureSets) { + featuresMap = featureSet.getFeaturesByRef(labels); + for (Map.Entry entry : featuresMap.entrySet()) { + response.putFeatures(entry.getKey(), entry.getValue().toProto()); + } + } + + return response.build(); + } catch (InvalidProtocolBufferException e) { + throw io.grpc.Status.NOT_FOUND + .withDescription("Unable to retrieve features") + .withCause(e) + .asRuntimeException(); + } + } + /** * Get stores matching the store name provided in the filter. If the store name is not provided, * the method will return all stores currently registered to Feast. @@ -242,6 +303,7 @@ public ListStoresResponse listStores(ListStoresRequest.Filter filter) { String.format("Store with name '%s' not found", name))); return ListStoresResponse.newBuilder().addStore(store.toProto()).build(); } catch (InvalidProtocolBufferException e) { + throw io.grpc.Status.NOT_FOUND .withDescription("Unable to retrieve stores") .withCause(e) diff --git a/core/src/test/java/feast/core/service/SpecServiceTest.java b/core/src/test/java/feast/core/service/SpecServiceTest.java index 20a84edbdaa..ae3e8786440 100644 --- a/core/src/test/java/feast/core/service/SpecServiceTest.java +++ b/core/src/test/java/feast/core/service/SpecServiceTest.java @@ -38,6 +38,8 @@ import feast.proto.core.CoreServiceProto.GetFeatureSetResponse; import feast.proto.core.CoreServiceProto.ListFeatureSetsRequest.Filter; import feast.proto.core.CoreServiceProto.ListFeatureSetsResponse; +import feast.proto.core.CoreServiceProto.ListFeaturesRequest; +import feast.proto.core.CoreServiceProto.ListFeaturesResponse; import feast.proto.core.CoreServiceProto.ListStoresRequest; import feast.proto.core.CoreServiceProto.ListStoresResponse; import feast.proto.core.CoreServiceProto.UpdateStoreRequest; @@ -100,6 +102,7 @@ public class SpecServiceTest { private SpecService specService; private List featureSets; + private List features; private List stores; private Source defaultSource; @@ -113,6 +116,11 @@ public void setUp() throws InvalidProtocolBufferException { FeatureSet featureSet1 = newDummyFeatureSet("f1", "project1"); FeatureSet featureSet2 = newDummyFeatureSet("f2", "project1"); + Map featureLabels1 = Map.ofEntries(Map.entry("key1", "val1")); + Map featureLabels2 = Map.ofEntries(Map.entry("key2", "val2")); + Map dummyLabels = Map.ofEntries(Map.entry("key", "value")); + + Feature dummyFeature = TestObjectFactory.CreateFeature("feature", Enum.STRING, dummyLabels); Feature f3f1 = TestObjectFactory.CreateFeature("f3f1", Enum.INT64); Feature f3f2 = TestObjectFactory.CreateFeature("f3f2", Enum.INT64); Entity f3e1 = TestObjectFactory.CreateEntity("f3e1", Enum.STRING); @@ -157,6 +165,21 @@ public void setUp() throws InvalidProtocolBufferException { .build()) .build(); + Entity f7e1 = TestObjectFactory.CreateEntity("f7e1", Enum.STRING); + Entity f9e1 = TestObjectFactory.CreateEntity("f9e1", Enum.STRING); + Feature f7f1 = TestObjectFactory.CreateFeature("f7f1", Enum.INT64, featureLabels1); + Feature f8f1 = TestObjectFactory.CreateFeature("f8f1", Enum.INT64, featureLabels2); + FeatureSet featureSet7 = + TestObjectFactory.CreateFeatureSet( + "f7", "project2", Arrays.asList(f7e1), Arrays.asList(f3f1, f3f2, f7f1)); + FeatureSet featureSet8 = + TestObjectFactory.CreateFeatureSet( + "f8", "project2", Arrays.asList(f7e1), Arrays.asList(f3f1, f8f1)); + FeatureSet featureSet9 = + TestObjectFactory.CreateFeatureSet( + "f9", "default", Arrays.asList(f9e1), Arrays.asList(f3f1, f8f1)); + features = Arrays.asList(dummyFeature, f3f1, f3f2, f7f1, f8f1); + featureSets = Arrays.asList( featureSet1, @@ -164,7 +187,10 @@ public void setUp() throws InvalidProtocolBufferException { featureSet3, featureSet4, FeatureSet.fromProto(fs5), - FeatureSet.fromProto(fs6)); + FeatureSet.fromProto(fs6), + featureSet7, + featureSet8, + featureSet9); when(featureSetRepository.findAll()).thenReturn(featureSets); when(featureSetRepository.findAllByOrderByNameAsc()).thenReturn(featureSets); @@ -174,6 +200,10 @@ public void setUp() throws InvalidProtocolBufferException { .thenReturn(featureSets.get(1)); when(featureSetRepository.findAllByNameLikeAndProject_NameOrderByNameAsc("f1", "project1")) .thenReturn(featureSets.subList(0, 1)); + when(featureSetRepository.findAllByNameLikeAndProject_NameOrderByNameAsc("%", "default")) + .thenReturn(featureSets.subList(8, 9)); + when(featureSetRepository.findAllByNameLikeAndProject_NameOrderByNameAsc("%", "project2")) + .thenReturn(featureSets.subList(6, 8)); when(featureSetRepository.findAllByNameLikeAndProject_NameOrderByNameAsc("asd", "project1")) .thenReturn(Lists.newArrayList()); when(featureSetRepository.findAllByNameLikeAndProject_NameOrderByNameAsc("f%", "project1")) @@ -757,6 +787,173 @@ public void applyFeatureSetShouldAcceptFeatureSetLabels() throws InvalidProtocol } @Test + public void shouldFilterFeaturesByEntitiesAndLabels() throws InvalidProtocolBufferException { + // Case 1: Only filter by entities + List entities = new ArrayList<>(); + String currentProject = "project2"; + + entities.add("f7e1"); + ListFeaturesResponse actual1 = + specService.listFeatures( + ListFeaturesRequest.Filter.newBuilder() + .setProject(currentProject) + .addAllEntities(entities) + .build()); + Map expectedMap1 = + Map.ofEntries( + Map.entry( + currentProject + + "/" + + featureSets.get(6).getName() + + ":" + + features.get(1).getName(), + features.get(1).toProto()), + Map.entry( + currentProject + + "/" + + featureSets.get(6).getName() + + ":" + + features.get(2).getName(), + features.get(2).toProto()), + Map.entry( + currentProject + + "/" + + featureSets.get(6).getName() + + ":" + + features.get(3).getName(), + features.get(3).toProto()), + Map.entry( + currentProject + + "/" + + featureSets.get(7).getName() + + ":" + + features.get(1).getName(), + features.get(1).toProto()), + Map.entry( + currentProject + + "/" + + featureSets.get(7).getName() + + ":" + + features.get(4).getName(), + features.get(4).toProto())); + ListFeaturesResponse expected1 = + ListFeaturesResponse.newBuilder().putAllFeatures(expectedMap1).build(); + + // Case 2: Filter by entities and labels + Map featureLabels1 = Map.ofEntries(Map.entry("key1", "val1")); + Map featureLabels2 = Map.ofEntries(Map.entry("key2", "val2")); + ListFeaturesResponse actual2 = + specService.listFeatures( + ListFeaturesRequest.Filter.newBuilder() + .setProject(currentProject) + .addAllEntities(entities) + .putAllLabels(featureLabels1) + .build()); + Map expectedMap2 = + Map.ofEntries( + Map.entry( + currentProject + + "/" + + featureSets.get(6).getName() + + ":" + + features.get(3).getName(), + features.get(3).toProto())); + ListFeaturesResponse expected2 = + ListFeaturesResponse.newBuilder().putAllFeatures(expectedMap2).build(); + + // Case 3: Filter by labels + ListFeaturesResponse actual3 = + specService.listFeatures( + ListFeaturesRequest.Filter.newBuilder() + .setProject(currentProject) + .putAllLabels(featureLabels2) + .build()); + Map expectedMap3 = + Map.ofEntries( + Map.entry( + currentProject + + "/" + + featureSets.get(7).getName() + + ":" + + features.get(4).getName(), + features.get(4).toProto())); + ListFeaturesResponse expected3 = + ListFeaturesResponse.newBuilder().putAllFeatures(expectedMap3).build(); + + // Case 4: Filter by nothing, except project + ListFeaturesResponse actual4 = + specService.listFeatures( + ListFeaturesRequest.Filter.newBuilder().setProject(currentProject).build()); + Map expectedMap4 = + Map.ofEntries( + Map.entry( + currentProject + + "/" + + featureSets.get(6).getName() + + ":" + + features.get(1).getName(), + features.get(1).toProto()), + Map.entry( + currentProject + + "/" + + featureSets.get(6).getName() + + ":" + + features.get(2).getName(), + features.get(2).toProto()), + Map.entry( + currentProject + + "/" + + featureSets.get(6).getName() + + ":" + + features.get(3).getName(), + features.get(3).toProto()), + Map.entry( + currentProject + + "/" + + featureSets.get(7).getName() + + ":" + + features.get(1).getName(), + features.get(1).toProto()), + Map.entry( + currentProject + + "/" + + featureSets.get(7).getName() + + ":" + + features.get(4).getName(), + features.get(4).toProto())); + ListFeaturesResponse expected4 = + ListFeaturesResponse.newBuilder().putAllFeatures(expectedMap4).build(); + + // Case 5: Filter by nothing; will use default project + currentProject = "default"; + ListFeaturesResponse actual5 = + specService.listFeatures(ListFeaturesRequest.Filter.newBuilder().build()); + Map expectedMap5 = + Map.ofEntries( + Map.entry( + currentProject + + "/" + + featureSets.get(8).getName() + + ":" + + features.get(1).getName(), + features.get(1).toProto()), + Map.entry( + currentProject + + "/" + + featureSets.get(8).getName() + + ":" + + features.get(4).getName(), + features.get(4).toProto())); + ListFeaturesResponse expected5 = + ListFeaturesResponse.newBuilder().putAllFeatures(expectedMap5).build(); + + assertThat(actual1, equalTo(expected1)); + assertThat(actual2, equalTo(expected2)); + assertThat(actual3, equalTo(expected3)); + assertThat(actual4, equalTo(expected4)); + assertThat(actual5, equalTo(expected5)); + } + public void shouldFilterByFeatureSetLabels() throws InvalidProtocolBufferException { List list = new ArrayList<>(); ListFeatureSetsResponse actual1 = diff --git a/core/src/test/java/feast/core/service/TestObjectFactory.java b/core/src/test/java/feast/core/service/TestObjectFactory.java index a498d6c21c2..53522e09ca4 100644 --- a/core/src/test/java/feast/core/service/TestObjectFactory.java +++ b/core/src/test/java/feast/core/service/TestObjectFactory.java @@ -25,6 +25,7 @@ import feast.proto.types.ValueProto; import java.util.HashMap; import java.util.List; +import java.util.Map; public class TestObjectFactory { @@ -53,6 +54,16 @@ public static FeatureSet CreateFeatureSet( return fs; } + public static Feature CreateFeature( + String name, ValueProto.ValueType.Enum valueType, Map labels) { + return Feature.fromProto( + FeatureSetProto.FeatureSpec.newBuilder() + .setName(name) + .setValueType(valueType) + .putAllLabels(labels) + .build()); + } + public static Feature CreateFeature(String name, ValueProto.ValueType.Enum valueType) { return Feature.fromProto( FeatureSetProto.FeatureSpec.newBuilder().setName(name).setValueType(valueType).build()); diff --git a/protos/feast/core/CoreService.proto b/protos/feast/core/CoreService.proto index a6ac313b2a5..a9c3f353732 100644 --- a/protos/feast/core/CoreService.proto +++ b/protos/feast/core/CoreService.proto @@ -43,6 +43,12 @@ service CoreService { // sets currently stored in the registry. rpc ListFeatureSets (ListFeatureSetsRequest) returns (ListFeatureSetsResponse); + // Returns all feature references and respective features matching that filter. If none are found + // an empty map will be returned + // If no filter is provided in the request, the response will contain all the features + // currently stored in the default project. + rpc ListFeatures (ListFeaturesRequest) returns (ListFeaturesResponse); + // Get feature statistics computed over the data in the batch stores. // // Returns a dataset containing TFDV statistics mapped to each valid historical store. @@ -151,6 +157,29 @@ message ListFeatureSetsResponse { repeated feast.core.FeatureSet feature_sets = 1; } +message ListFeaturesRequest { + message Filter { + // User defined metadata for feature. + // Features with all matching labels will be returned. + map labels = 1; + + // List of entities contained within the featureSet that the feature belongs to. + // Only feature sets with these entities will be searched for features. + repeated string entities = 2; + + // Name of project that the feature sets belongs to. Filtering on projects is disabled. + // It is NOT possible to provide an asterisk with a string in order to do pattern matching. + // If unspecified this field will default to the default project 'default'. + string project = 3; + } + + Filter filter = 1; +} + +message ListFeaturesResponse { + map features = 1; +} + message ListStoresRequest { message Filter { // Name of desired store. Regex is not supported in this query. diff --git a/sdk/python/feast/cli.py b/sdk/python/feast/cli.py index 34a3f600e15..68fee30c9b1 100644 --- a/sdk/python/feast/cli.py +++ b/sdk/python/feast/cli.py @@ -112,6 +112,81 @@ def config_set(prop, value): sys.exit(1) +@cli.group(name="features") +def feature(): + """ + Manage feature + """ + pass + + +def _get_features_labels_dict(label_str: str): + """ + Converts CLI input labels string to dictionary format if provided string is valid. + """ + labels_dict = {} + labels_kv = label_str.split(",") + if label_str == "": + return labels_dict + if len(labels_kv) % 2 == 1: + raise ValueError("Uneven key-value label pairs were entered") + for k, v in zip(labels_kv[0::2], labels_kv[1::2]): + labels_dict[k] = v + return labels_dict + + +def _get_features_entities(entities_str: str): + """ + Converts CLI input entities string to list format if provided string is valid. + """ + entities_list = [] + if entities_str == "": + return entities_list + return entities_str.split(",") + + +@feature.command(name="list") +@click.option( + "--project", + "-p", + help="Project that feature belongs to", + type=click.STRING, + default="*", +) +@click.option( + "--entities", + "-n", + help="Entities to filter for features", + type=click.STRING, + default="", +) +@click.option( + "--labels", + "-l", + help="Labels to filter for features", + type=click.STRING, + default="", +) +def feature_list(project: str, entities: str, labels: str): + """ + List all features + """ + feast_client = Client() # type: Client + + entities_list = _get_features_entities(entities) + labels_dict = _get_features_labels_dict(labels) + + table = [] + for feature_ref, feature in feast_client.list_features_by_ref( + project=project, entities=entities_list, labels=labels_dict + ).items(): + table.append([feature.name, feature.dtype, repr(feature_ref)]) + + from tabulate import tabulate + + print(tabulate(table, headers=["NAME", "DTYPE", "REFERENCE"], tablefmt="plain")) + + @cli.group(name="feature-sets") def feature_set(): """ diff --git a/sdk/python/feast/client.py b/sdk/python/feast/client.py index 2ec90e434db..8d85f07cd33 100644 --- a/sdk/python/feast/client.py +++ b/sdk/python/feast/client.py @@ -51,6 +51,8 @@ GetFeatureSetRequest, GetFeatureSetResponse, GetFeatureStatisticsRequest, + ListFeaturesRequest, + ListFeaturesResponse, ListFeatureSetsRequest, ListFeatureSetsResponse, ListIngestionJobsRequest, @@ -61,7 +63,7 @@ ) from feast.core.CoreService_pb2_grpc import CoreServiceStub from feast.core.FeatureSet_pb2 import FeatureSetStatus -from feast.feature import FeatureRef +from feast.feature import Feature, FeatureRef from feast.feature_set import Entity, FeatureSet, FeatureSetRef from feast.job import IngestJob, RetrievalJob from feast.loaders.abstract_producer import get_producer @@ -495,10 +497,57 @@ def get_feature_set( raise grpc.RpcError(e.details()) return FeatureSet.from_proto(get_feature_set_response.feature_set) + def list_features_by_ref( + self, + project: str = None, + entities: List[str] = list(), + labels: Dict[str, str] = dict(), + ) -> Dict[FeatureRef, Feature]: + """ + Returns a list of features based on filters provided. + + Args: + project: Feast project that these features belongs to + entities: Feast entity that these features are associated with + labels: Feast labels that these features are associated with + + Returns: + Dictionary of + + Examples: + >>> from feast import Client + >>> + >>> feast_client = Client(core_url="localhost:6565") + >>> features = list_features_by_ref(project="test_project", entities=["driver_id"], labels={"key1":"val1","key2":"val2"}) + >>> print(features) + """ + self._connect_core() + + if project is None: + if self.project is not None: + project = self.project + else: + project = "default" + + filter = ListFeaturesRequest.Filter( + project=project, entities=entities, labels=labels + ) + + feature_protos = self._core_service_stub.ListFeatures( + ListFeaturesRequest(filter=filter) + ) # type: ListFeaturesResponse + + features_dict = {} + for ref_str, feature_proto in feature_protos.features.items(): + feature_ref = FeatureRef.from_str(ref_str, ignore_project=True) + feature = Feature.from_proto(feature_proto) + features_dict[feature_ref] = feature + + return features_dict + def list_entities(self) -> Dict[str, Entity]: """ Returns a dictionary of entities across all feature sets - Returns: Dictionary of entities, indexed by name """ diff --git a/sdk/python/tests/test_client.py b/sdk/python/tests/test_client.py index 85fea31dbb1..04626f73cd5 100644 --- a/sdk/python/tests/test_client.py +++ b/sdk/python/tests/test_client.py @@ -34,6 +34,7 @@ from feast.core.CoreService_pb2 import ( GetFeastCoreVersionResponse, GetFeatureSetResponse, + ListFeaturesResponse, ListFeatureSetsResponse, ListIngestionJobsResponse, ) @@ -383,6 +384,52 @@ def test_list_feature_sets(self, mocked_client, mocker): and len(feature_set.features) == 1 ) + @pytest.mark.parametrize( + "mocked_client", + [pytest.lazy_fixture("mock_client"), pytest.lazy_fixture("secure_mock_client")], + ) + def test_list_features(self, mocked_client, mocker): + mocker.patch.object( + mocked_client, + "_core_service_stub", + return_value=Core.CoreServiceStub(grpc.insecure_channel("")), + ) + + feature1_proto = FeatureSpecProto( + name="feature_1", value_type=ValueProto.ValueType.FLOAT + ) + feature2_proto = FeatureSpecProto( + name="feature_2", value_type=ValueProto.ValueType.STRING + ) + + mocker.patch.object( + mocked_client._core_service_stub, + "ListFeatures", + return_value=ListFeaturesResponse( + features={ + "driver_car:feature_1": feature1_proto, + "driver_car:feature_2": feature2_proto, + } + ), + ) + + features = mocked_client.list_features_by_ref(project="test") + assert len(features) == 2 + + ref_str_list = [] + feature_name_list = [] + feature_dtype_list = [] + for ref_str, feature_proto in features.items(): + ref_str_list.append(ref_str) + feature_name_list.append(feature_proto.name) + feature_dtype_list.append(feature_proto.dtype) + + assert ( + set(ref_str_list) == set(["driver_car:feature_1", "driver_car:feature_2"]) + and set(feature_name_list) == set(["feature_1", "feature_2"]) + and set(feature_dtype_list) == set([ValueType.FLOAT, ValueType.STRING]) + ) + @pytest.mark.parametrize( "mocked_client", [pytest.lazy_fixture("mock_client"), pytest.lazy_fixture("secure_mock_client")], diff --git a/tests/e2e/redis/basic-ingest-redis-serving.py b/tests/e2e/redis/basic-ingest-redis-serving.py index 1424936f0c8..1d38cc924b4 100644 --- a/tests/e2e/redis/basic-ingest-redis-serving.py +++ b/tests/e2e/redis/basic-ingest-redis-serving.py @@ -3,6 +3,7 @@ import random import time import grpc +from collections import OrderedDict from feast.entity import Entity from feast.serving.ServingService_pb2 import ( GetOnlineFeaturesRequest, @@ -666,6 +667,70 @@ def test_all_types_infer_register_ingest_file_success(client, client.ingest(feature_set=all_types_fs, source=all_types_parquet_file) +@pytest.mark.timeout(200) +@pytest.mark.run(order=42) +def test_list_entities_and_features(client): + customer_entity = Entity("customer_id", ValueType.INT64) + driver_entity = Entity("driver_id", ValueType.INT64) + + customer_feature_rating = Feature(name="rating", dtype=ValueType.FLOAT, labels={"key1":"val1"}) + customer_feature_cost = Feature(name="cost", dtype=ValueType.FLOAT) + driver_feature_rating = Feature(name="rating", dtype=ValueType.FLOAT) + driver_feature_cost = Feature(name="cost", dtype=ValueType.FLOAT, labels={"key1":"val1"}) + + filter_by_project_entity_labels_expected = dict([ + ("customer:rating", customer_feature_rating) + ]) + + filter_by_project_entity_expected = dict([ + ("driver:cost", driver_feature_cost), + ("driver:rating", driver_feature_rating) + ]) + + filter_by_project_labels_expected = dict([ + ("customer:rating", customer_feature_rating), + ("driver:cost", driver_feature_cost) + ]) + + customer_fs = FeatureSet( + "customer", + features=[ + customer_feature_rating, + customer_feature_cost + ], + entities=[customer_entity], + max_age=Duration(seconds=100) + ) + + driver_fs = FeatureSet( + "driver", + features=[ + driver_feature_rating, + driver_feature_cost + ], + entities=[driver_entity], + max_age=Duration(seconds=100) + ) + + client.set_project(PROJECT_NAME) + client.apply(customer_fs) + client.apply(driver_fs) + + # Test for listing of features + # Case 1: Filter by: project, entities and labels + filter_by_project_entity_labels_actual = client.list_features_by_ref(project=PROJECT_NAME, entities=["customer_id"], labels={"key1":"val1"}) + + # Case 2: Filter by: project, entities + filter_by_project_entity_actual = client.list_features_by_ref(project=PROJECT_NAME, entities=["driver_id"]) + + # Case 3: Filter by: project, labels + filter_by_project_labels_actual = client.list_features_by_ref(project=PROJECT_NAME, labels={"key1":"val1"}) + + assert set(filter_by_project_entity_labels_expected) == set(filter_by_project_entity_labels_actual) + assert set(filter_by_project_entity_expected) == set(filter_by_project_entity_actual) + assert set(filter_by_project_labels_expected) == set(filter_by_project_labels_actual) + + # TODO: rewrite these using python SDK once the labels are implemented there class TestsBasedOnGrpc: GRPC_CONNECTION_TIMEOUT = 3 From d3acb8337324798f9a33de4e06d28a116efc3dd1 Mon Sep 17 00:00:00 2001 From: Oleksii Moskalenko Date: Thu, 18 Jun 2020 11:39:05 +0300 Subject: [PATCH 065/133] solve techdebt: move communication with job to JobCoordinator (#800) --- .../feast/core/config/FeastProperties.java | 4 + .../feast/core/dao/FeatureSetRepository.java | 4 + .../java/feast/core/job/JobUpdateTask.java | 17 ++ .../feast/core/model/FeatureSetJobStatus.java | 20 ++ .../core/service/JobCoordinatorService.java | 140 +++++++++++- .../java/feast/core/service/SpecService.java | 101 +------- core/src/main/resources/application.yml | 1 + .../feast/core/job/JobUpdateTaskTest.java | 58 +++++ .../service/JobCoordinatorServiceTest.java | 216 ++++++++++++++++-- .../feast/core/service/SpecServiceTest.java | 174 +------------- .../feast/core/service/TestObjectFactory.java | 21 +- 11 files changed, 457 insertions(+), 299 deletions(-) diff --git a/core/src/main/java/feast/core/config/FeastProperties.java b/core/src/main/java/feast/core/config/FeastProperties.java index b508de76e5d..07303107a43 100644 --- a/core/src/main/java/feast/core/config/FeastProperties.java +++ b/core/src/main/java/feast/core/config/FeastProperties.java @@ -172,6 +172,10 @@ public static class FeatureSetSpecStreamProperties { /* Kafka topic to receive acknowledgment from ingestion job on successful processing of new specs */ @NotBlank private String specsAckTopic = "feast-feature-set-specs-ack"; + + /* Notify jobs interval in millisecond. + How frequently Feast will check on Pending FeatureSets and publish them to kafka. */ + @Positive private long notifyIntervalMilliseconds; } } diff --git a/core/src/main/java/feast/core/dao/FeatureSetRepository.java b/core/src/main/java/feast/core/dao/FeatureSetRepository.java index b136650dfdf..38a690b0d6c 100644 --- a/core/src/main/java/feast/core/dao/FeatureSetRepository.java +++ b/core/src/main/java/feast/core/dao/FeatureSetRepository.java @@ -17,6 +17,7 @@ package feast.core.dao; import feast.core.model.FeatureSet; +import feast.proto.core.FeatureSetProto; import java.util.List; import org.springframework.data.jpa.repository.JpaRepository; @@ -37,4 +38,7 @@ public interface FeatureSetRepository extends JpaRepository // find all feature sets matching the given name pattern and project pattern List findAllByNameLikeAndProject_NameLikeOrderByNameAsc( String name, String project_name); + + // find all feature sets matching given status + List findAllByStatus(FeatureSetProto.FeatureSetStatus status); } diff --git a/core/src/main/java/feast/core/job/JobUpdateTask.java b/core/src/main/java/feast/core/job/JobUpdateTask.java index 9abdfa3e455..9a2e5e6c25e 100644 --- a/core/src/main/java/feast/core/job/JobUpdateTask.java +++ b/core/src/main/java/feast/core/job/JobUpdateTask.java @@ -24,6 +24,7 @@ import feast.proto.core.FeatureSetProto; import java.time.Instant; import java.util.List; +import java.util.Map; import java.util.Optional; import java.util.concurrent.Callable; import java.util.concurrent.ExecutionException; @@ -32,6 +33,7 @@ import java.util.concurrent.Future; import java.util.concurrent.TimeUnit; import java.util.concurrent.TimeoutException; +import java.util.stream.Collectors; import lombok.Getter; import lombok.extern.slf4j.Slf4j; import org.apache.beam.vendor.guava.v26_0_jre.com.google.common.hash.Hashing; @@ -149,10 +151,24 @@ private Job startJob(String jobId) { } private void updateFeatureSets(Job job) { + Map alreadyConnected = + job.getFeatureSetJobStatuses().stream() + .collect(Collectors.toMap(FeatureSetJobStatus::getFeatureSet, s -> s)); + for (FeatureSet fs : featureSets) { + if (alreadyConnected.containsKey(fs)) { + continue; + } + FeatureSetJobStatus status = new FeatureSetJobStatus(); status.setFeatureSet(fs); status.setJob(job); + if (fs.getStatus() == FeatureSetProto.FeatureSetStatus.STATUS_READY) { + // Feature Set was already delivered to previous generation of the job + // (another words, it exists in kafka) + // so we expect Job will ack latest version based on history from kafka topic + status.setVersion(fs.getVersion()); + } status.setDeliveryStatus(FeatureSetProto.FeatureSetJobDeliveryStatus.STATUS_IN_PROGRESS); job.getFeatureSetJobStatuses().add(status); } @@ -175,6 +191,7 @@ private Job updateStatus(Job job) { } job.setStatus(newStatus); + updateFeatureSets(job); return job; } diff --git a/core/src/main/java/feast/core/model/FeatureSetJobStatus.java b/core/src/main/java/feast/core/model/FeatureSetJobStatus.java index 7f21c26335a..f6320defdda 100644 --- a/core/src/main/java/feast/core/model/FeatureSetJobStatus.java +++ b/core/src/main/java/feast/core/model/FeatureSetJobStatus.java @@ -16,6 +16,7 @@ */ package feast.core.model; +import com.google.common.base.Objects; import feast.proto.core.FeatureSetProto.FeatureSetJobDeliveryStatus; import java.io.Serializable; import javax.persistence.*; @@ -61,4 +62,23 @@ public FeatureSetJobStatusKey() {} @Enumerated(EnumType.STRING) @Column(name = "delivery_status") private FeatureSetJobDeliveryStatus deliveryStatus; + + @Column(name = "version", columnDefinition = "integer default 0") + private int version; + + @Override + public boolean equals(Object o) { + if (this == o) return true; + if (o == null || getClass() != o.getClass()) return false; + FeatureSetJobStatus that = (FeatureSetJobStatus) o; + return version == that.version + && Objects.equal(job.getId(), that.job.getId()) + && Objects.equal(featureSet.getReference(), that.featureSet.getReference()) + && deliveryStatus == that.deliveryStatus; + } + + @Override + public int hashCode() { + return Objects.hashCode(job, featureSet, deliveryStatus, version); + } } diff --git a/core/src/main/java/feast/core/service/JobCoordinatorService.java b/core/src/main/java/feast/core/service/JobCoordinatorService.java index 69da3e507ed..19d5d65b618 100644 --- a/core/src/main/java/feast/core/service/JobCoordinatorService.java +++ b/core/src/main/java/feast/core/service/JobCoordinatorService.java @@ -16,6 +16,8 @@ */ package feast.core.service; +import static feast.core.model.FeatureSet.parseReference; + import com.google.protobuf.InvalidProtocolBufferException; import feast.core.config.FeastProperties; import feast.core.config.FeastProperties.JobProperties; @@ -26,6 +28,8 @@ import feast.core.model.*; import feast.proto.core.CoreServiceProto.ListStoresRequest.Filter; import feast.proto.core.CoreServiceProto.ListStoresResponse; +import feast.proto.core.FeatureSetProto; +import feast.proto.core.IngestionJobProto; import feast.proto.core.StoreProto; import feast.proto.core.StoreProto.Store.Subscription; import java.util.ArrayList; @@ -33,14 +37,15 @@ import java.util.List; import java.util.Optional; import java.util.Set; -import java.util.concurrent.ExecutionException; -import java.util.concurrent.ExecutorCompletionService; -import java.util.concurrent.ExecutorService; -import java.util.concurrent.Executors; +import java.util.concurrent.*; import java.util.stream.Collectors; import javax.validation.constraints.Positive; import lombok.extern.slf4j.Slf4j; +import org.apache.kafka.clients.consumer.ConsumerRecord; import org.springframework.beans.factory.annotation.Autowired; +import org.springframework.data.util.Pair; +import org.springframework.kafka.annotation.KafkaListener; +import org.springframework.kafka.core.KafkaTemplate; import org.springframework.scheduling.annotation.Scheduled; import org.springframework.stereotype.Service; import org.springframework.transaction.annotation.Transactional; @@ -49,11 +54,14 @@ @Service public class JobCoordinatorService { + private final int SPEC_PUBLISHING_TIMEOUT_SECONDS = 5; + private final JobRepository jobRepository; private final FeatureSetRepository featureSetRepository; private final SpecService specService; private final JobManager jobManager; private final JobProperties jobProperties; + private final KafkaTemplate specPublisher; @Autowired public JobCoordinatorService( @@ -61,12 +69,14 @@ public JobCoordinatorService( FeatureSetRepository featureSetRepository, SpecService specService, JobManager jobManager, - FeastProperties feastProperties) { + FeastProperties feastProperties, + KafkaTemplate specPublisher) { this.jobRepository = jobRepository; this.featureSetRepository = featureSetRepository; this.specService = specService; this.jobManager = jobManager; this.jobProperties = feastProperties.getJobs(); + this.specPublisher = specPublisher; } /** @@ -153,4 +163,124 @@ public Optional getJob(Source source, Store store) { // return the latest return Optional.of(jobs.get(0)); } + + @Transactional + @Scheduled(fixedDelayString = "${feast.stream.specsOptions.notifyIntervalMilliseconds}") + public void notifyJobsWhenFeatureSetUpdated() { + List pendingFeatureSets = + featureSetRepository.findAllByStatus(FeatureSetProto.FeatureSetStatus.STATUS_PENDING); + + pendingFeatureSets.stream() + .filter( + fs -> { + List runningJobs = + fs.getJobStatuses().stream() + .filter(jobStatus -> jobStatus.getJob().isRunning()) + .collect(Collectors.toList()); + + return runningJobs.size() > 0 + && runningJobs.stream() + .allMatch(jobStatus -> jobStatus.getVersion() < fs.getVersion()); + }) + .forEach( + fs -> { + log.info("Sending new FeatureSet {} to Ingestion", fs.getReference()); + + // Sending latest version of FeatureSet to all currently running IngestionJobs + // (there's one topic for all sets). + // All related jobs would apply new FeatureSet on the fly. + // In case kafka doesn't respond within SPEC_PUBLISHING_TIMEOUT_SECONDS we will try + // again later. + try { + specPublisher + .sendDefault(fs.getReference(), fs.toProto().getSpec()) + .get(SPEC_PUBLISHING_TIMEOUT_SECONDS, TimeUnit.SECONDS); + } catch (Exception e) { + log.error( + "Error occurred while sending FeatureSetSpec to kafka. Cause {}." + + " Will retry later", + e.getMessage()); + return; + } + + // Updating delivery status for related jobs (that are currently using this + // FeatureSet). + // We now set status to IN_PROGRESS, so listenAckFromJobs would be able to + // monitor delivery progress for each new version. + fs.getJobStatuses().stream() + .filter(s -> s.getJob().isRunning()) + .forEach( + jobStatus -> { + jobStatus.setDeliveryStatus( + FeatureSetProto.FeatureSetJobDeliveryStatus.STATUS_IN_PROGRESS); + jobStatus.setVersion(fs.getVersion()); + }); + featureSetRepository.saveAndFlush(fs); + }); + } + + /** + * Listener for ACK messages coming from IngestionJob when FeatureSetSpec is installed (in + * pipeline). + * + *

Updates FeatureSetJobStatus for respected FeatureSet (selected by reference) and Job (select + * by Id). + * + *

When all related (running) to FeatureSet jobs are updated - FeatureSet receives READY status + * + * @param record ConsumerRecord with key: FeatureSet reference and value: Ack message + */ + @KafkaListener(topics = {"${feast.stream.specsOptions.specsAckTopic}"}) + @Transactional + public void listenAckFromJobs( + ConsumerRecord record) { + String setReference = record.key(); + Pair projectAndSetName = parseReference(setReference); + FeatureSet featureSet = + featureSetRepository.findFeatureSetByNameAndProject_Name( + projectAndSetName.getSecond(), projectAndSetName.getFirst()); + if (featureSet == null) { + log.warn( + String.format("ACKListener received message for unknown FeatureSet %s", setReference)); + return; + } + + int ackVersion = record.value().getFeatureSetVersion(); + + if (featureSet.getVersion() != ackVersion) { + log.warn( + String.format( + "ACKListener received outdated ack for %s. Current %d, Received %d", + setReference, featureSet.getVersion(), ackVersion)); + return; + } + + log.info("Updating featureSet {} delivery statuses.", featureSet.getReference()); + + featureSet.getJobStatuses().stream() + .filter( + js -> + js.getJob().getId().equals(record.value().getJobName()) + && js.getVersion() == ackVersion) + .findFirst() + .ifPresent( + featureSetJobStatus -> + featureSetJobStatus.setDeliveryStatus( + FeatureSetProto.FeatureSetJobDeliveryStatus.STATUS_DELIVERED)); + + boolean allDelivered = + featureSet.getJobStatuses().stream() + .filter(js -> js.getJob().isRunning()) + .allMatch( + js -> + js.getDeliveryStatus() + .equals(FeatureSetProto.FeatureSetJobDeliveryStatus.STATUS_DELIVERED)); + + if (allDelivered) { + log.info("FeatureSet {} update is completely delivered", featureSet.getReference()); + + featureSet.setStatus(FeatureSetProto.FeatureSetStatus.STATUS_READY); + featureSetRepository.saveAndFlush(featureSet); + } + } } diff --git a/core/src/main/java/feast/core/service/SpecService.java b/core/src/main/java/feast/core/service/SpecService.java index e60aeee57d3..4365b8fd6c2 100644 --- a/core/src/main/java/feast/core/service/SpecService.java +++ b/core/src/main/java/feast/core/service/SpecService.java @@ -16,7 +16,6 @@ */ package feast.core.service; -import static feast.core.model.FeatureSet.parseReference; import static feast.core.validators.Matchers.checkValidCharacters; import static feast.core.validators.Matchers.checkValidCharactersAllowAsterisk; @@ -42,21 +41,15 @@ import feast.proto.core.CoreServiceProto.UpdateStoreResponse; import feast.proto.core.FeatureSetProto; import feast.proto.core.FeatureSetProto.FeatureSetStatus; -import feast.proto.core.IngestionJobProto; import feast.proto.core.SourceProto; import feast.proto.core.StoreProto; import feast.proto.core.StoreProto.Store.Subscription; import java.util.ArrayList; import java.util.List; import java.util.Map; -import java.util.concurrent.TimeUnit; import java.util.stream.Collectors; import lombok.extern.slf4j.Slf4j; -import org.apache.kafka.clients.consumer.ConsumerRecord; import org.springframework.beans.factory.annotation.Autowired; -import org.springframework.data.util.Pair; -import org.springframework.kafka.annotation.KafkaListener; -import org.springframework.kafka.core.KafkaTemplate; import org.springframework.stereotype.Service; import org.springframework.transaction.annotation.Transactional; @@ -68,26 +61,21 @@ @Service public class SpecService { - private final int SPEC_PUBLISHING_TIMEOUT_SECONDS = 5; - private final FeatureSetRepository featureSetRepository; private final ProjectRepository projectRepository; private final StoreRepository storeRepository; private final Source defaultSource; - private final KafkaTemplate specPublisher; @Autowired public SpecService( FeatureSetRepository featureSetRepository, StoreRepository storeRepository, ProjectRepository projectRepository, - Source defaultSource, - KafkaTemplate specPublisher) { + Source defaultSource) { this.featureSetRepository = featureSetRepository; this.storeRepository = storeRepository; this.projectRepository = projectRepository; this.defaultSource = defaultSource; - this.specPublisher = specPublisher; } /** @@ -383,37 +371,6 @@ public ApplyFeatureSetResponse applyFeatureSet(FeatureSetProto.FeatureSet newFea featureSet.incVersion(); - // Sending latest version of FeatureSet to all currently running IngestionJobs (there's one - // topic for all sets). - // All related jobs would apply new FeatureSet on the fly. - // We wait for Kafka broker to ack that the message was added to topic before actually - // committing this FeatureSet. - // In case kafka doesn't respond within SPEC_PUBLISHING_TIMEOUT_SECONDS we abort current - // transaction and return error to client. - try { - specPublisher - .sendDefault(featureSet.getReference(), featureSet.toProto().getSpec()) - .get(SPEC_PUBLISHING_TIMEOUT_SECONDS, TimeUnit.SECONDS); - } catch (Exception e) { - throw io.grpc.Status.UNAVAILABLE - .withDescription( - String.format( - "Unable to publish FeatureSet to Kafka. Cause: %s", - e.getCause() != null ? e.getCause().getMessage() : "unknown")) - .withCause(e) - .asRuntimeException(); - } - - // Updating delivery status for related jobs (that are currently using this FeatureSet). - // We now set status to IN_PROGRESS, so listenAckFromJobs would be able to - // monitor delivery progress for each new version. - featureSet.getJobStatuses().stream() - .filter(s -> s.getJob().isRunning()) - .forEach( - s -> - s.setDeliveryStatus( - FeatureSetProto.FeatureSetJobDeliveryStatus.STATUS_IN_PROGRESS)); - // Persist the FeatureSet object featureSet.setStatus(FeatureSetStatus.STATUS_PENDING); project.addFeatureSet(featureSet); @@ -462,60 +419,4 @@ public UpdateStoreResponse updateStore(UpdateStoreRequest updateStoreRequest) .setStore(updateStoreRequest.getStore()) .build(); } - - /** - * Listener for ACK messages coming from IngestionJob when FeatureSetSpec is installed (in - * pipeline). - * - *

Updates FeatureSetJobStatus for respected FeatureSet (selected by reference) and Job (select - * by Id). - * - *

When all related (running) to FeatureSet jobs are updated - FeatureSet receives READY status - * - * @param record ConsumerRecord with key: FeatureSet reference and value: Ack message - */ - @KafkaListener(topics = {"${feast.stream.specsOptions.specsAckTopic}"}) - @Transactional - public void listenAckFromJobs( - ConsumerRecord record) { - String setReference = record.key(); - Pair projectAndSetName = parseReference(setReference); - FeatureSet featureSet = - featureSetRepository.findFeatureSetByNameAndProject_Name( - projectAndSetName.getSecond(), projectAndSetName.getFirst()); - if (featureSet == null) { - log.warn( - String.format("ACKListener received message for unknown FeatureSet %s", setReference)); - return; - } - - if (featureSet.getVersion() != record.value().getFeatureSetVersion()) { - log.warn( - String.format( - "ACKListener received outdated ack for %s. Current %d, Received %d", - setReference, featureSet.getVersion(), record.value().getFeatureSetVersion())); - return; - } - - featureSet.getJobStatuses().stream() - .filter(js -> js.getJob().getId().equals(record.value().getJobName())) - .findFirst() - .ifPresent( - featureSetJobStatus -> - featureSetJobStatus.setDeliveryStatus( - FeatureSetProto.FeatureSetJobDeliveryStatus.STATUS_DELIVERED)); - - boolean allDelivered = - featureSet.getJobStatuses().stream() - .filter(js -> js.getJob().isRunning()) - .allMatch( - js -> - js.getDeliveryStatus() - .equals(FeatureSetProto.FeatureSetJobDeliveryStatus.STATUS_DELIVERED)); - - if (allDelivered) { - featureSet.setStatus(FeatureSetStatus.STATUS_READY); - featureSetRepository.saveAndFlush(featureSet); - } - } } diff --git a/core/src/main/resources/application.yml b/core/src/main/resources/application.yml index 9f02ed302db..c34af21495e 100644 --- a/core/src/main/resources/application.yml +++ b/core/src/main/resources/application.yml @@ -81,6 +81,7 @@ feast: specsOptions: specsTopic: feast-specs specsAckTopic: feast-specs-ack + notifyIntervalMilliseconds: 1000 spring: jpa: diff --git a/core/src/test/java/feast/core/job/JobUpdateTaskTest.java b/core/src/test/java/feast/core/job/JobUpdateTaskTest.java index fc633ba15f8..16e36274596 100644 --- a/core/src/test/java/feast/core/job/JobUpdateTaskTest.java +++ b/core/src/test/java/feast/core/job/JobUpdateTaskTest.java @@ -16,6 +16,8 @@ */ package feast.core.job; +import static feast.proto.core.FeatureSetProto.FeatureSetJobDeliveryStatus.STATUS_DELIVERED; +import static org.hamcrest.Matchers.containsInAnyOrder; import static org.hamcrest.core.Is.is; import static org.hamcrest.core.IsEqual.equalTo; import static org.junit.Assert.assertThat; @@ -57,6 +59,7 @@ public class JobUpdateTaskTest { private Store store; private Source source; private FeatureSet featureSet1; + private FeatureSet featureSet2; @Before public void setUp() { @@ -85,7 +88,11 @@ public void setUp() { featureSet1 = FeatureSet.fromProto(fsBuilder.setSpec(specBuilder.setName("featureSet1")).build()); + featureSet2 = + FeatureSet.fromProto(fsBuilder.setSpec(specBuilder.setName("featureSet2")).build()); featureSet1.setSource(source); + featureSet2.setStatus(FeatureSetProto.FeatureSetStatus.STATUS_READY); + featureSet2.setVersion(5); } Job makeJob(String extId, List featureSets, JobStatus status) { @@ -176,4 +183,55 @@ public void shouldTimeout() { Job actual = jobUpdateTask.call(); assertThat(actual, is(IsNull.nullValue())); } + + @Test + public void featureSetsShouldBeUpdated() { + Job job = makeJob("", Collections.emptyList(), JobStatus.RUNNING); + + when(jobManager.getJobStatus(job)).thenReturn(JobStatus.RUNNING); + + JobUpdateTask jobUpdateTask = + new JobUpdateTask( + Collections.singletonList(featureSet1), + source, + store, + Optional.of(job), + jobManager, + 0L); + + jobUpdateTask.call(); + + FeatureSetJobStatus expectedStatus1 = new FeatureSetJobStatus(); + expectedStatus1.setJob(job); + expectedStatus1.setFeatureSet(featureSet1); + expectedStatus1.setVersion(0); + expectedStatus1.setDeliveryStatus( + FeatureSetProto.FeatureSetJobDeliveryStatus.STATUS_IN_PROGRESS); + + assertThat(job.getFeatureSetJobStatuses(), containsInAnyOrder(expectedStatus1)); + + expectedStatus1.setDeliveryStatus(STATUS_DELIVERED); + job.getFeatureSetJobStatuses().forEach(j -> j.setDeliveryStatus(STATUS_DELIVERED)); + + JobUpdateTask jobUpdateTask2 = + new JobUpdateTask( + Arrays.asList(featureSet1, featureSet2), + source, + store, + Optional.of(job), + jobManager, + 0L); + + jobUpdateTask2.call(); + + FeatureSetJobStatus expectedStatus2 = new FeatureSetJobStatus(); + expectedStatus2.setJob(job); + expectedStatus2.setFeatureSet(featureSet2); + expectedStatus2.setVersion(featureSet2.getVersion()); + expectedStatus2.setDeliveryStatus( + FeatureSetProto.FeatureSetJobDeliveryStatus.STATUS_IN_PROGRESS); + + assertThat( + job.getFeatureSetJobStatuses(), containsInAnyOrder(expectedStatus1, expectedStatus2)); + } } diff --git a/core/src/test/java/feast/core/service/JobCoordinatorServiceTest.java b/core/src/test/java/feast/core/service/JobCoordinatorServiceTest.java index caec99f2287..9b8e47de751 100644 --- a/core/src/test/java/feast/core/service/JobCoordinatorServiceTest.java +++ b/core/src/test/java/feast/core/service/JobCoordinatorServiceTest.java @@ -17,14 +17,13 @@ package feast.core.service; import static org.hamcrest.CoreMatchers.equalTo; +import static org.hamcrest.CoreMatchers.is; import static org.hamcrest.MatcherAssert.assertThat; -import static org.mockito.ArgumentMatchers.any; -import static org.mockito.ArgumentMatchers.argThat; -import static org.mockito.Mockito.times; -import static org.mockito.Mockito.verify; -import static org.mockito.Mockito.when; +import static org.mockito.ArgumentMatchers.*; +import static org.mockito.Mockito.*; import static org.mockito.MockitoAnnotations.initMocks; +import com.google.common.collect.ImmutableList; import com.google.common.collect.Lists; import com.google.protobuf.InvalidProtocolBufferException; import feast.core.config.FeastProperties; @@ -42,6 +41,7 @@ import feast.proto.core.FeatureSetProto; import feast.proto.core.FeatureSetProto.FeatureSetMeta; import feast.proto.core.FeatureSetProto.FeatureSetSpec; +import feast.proto.core.IngestionJobProto; import feast.proto.core.SourceProto.KafkaSourceConfig; import feast.proto.core.SourceProto.Source; import feast.proto.core.SourceProto.SourceType; @@ -49,14 +49,20 @@ import feast.proto.core.StoreProto.Store.RedisConfig; import feast.proto.core.StoreProto.Store.StoreType; import feast.proto.core.StoreProto.Store.Subscription; +import java.util.Arrays; import java.util.Collections; import java.util.List; +import java.util.concurrent.CancellationException; +import lombok.SneakyThrows; +import org.apache.kafka.clients.consumer.ConsumerRecord; import org.junit.Before; import org.junit.Rule; import org.junit.Test; import org.junit.rules.ExpectedException; import org.mockito.ArgumentCaptor; import org.mockito.Mock; +import org.springframework.kafka.core.KafkaTemplate; +import org.springframework.scheduling.annotation.AsyncResult; public class JobCoordinatorServiceTest { @@ -65,8 +71,10 @@ public class JobCoordinatorServiceTest { @Mock JobManager jobManager; @Mock SpecService specService; @Mock FeatureSetRepository featureSetRepository; + @Mock private KafkaTemplate kafkaTemplate; private FeastProperties feastProperties; + private JobCoordinatorService jcs; @Before public void setUp() { @@ -75,14 +83,23 @@ public void setUp() { JobProperties jobProperties = new JobProperties(); jobProperties.setJobUpdateTimeoutSeconds(5); feastProperties.setJobs(jobProperties); + + jcs = + new JobCoordinatorService( + jobRepository, + featureSetRepository, + specService, + jobManager, + feastProperties, + kafkaTemplate); + + when(kafkaTemplate.sendDefault(any(), any())).thenReturn(new AsyncResult<>(null)); } @Test public void shouldDoNothingIfNoStoresFound() throws InvalidProtocolBufferException { when(specService.listStores(any())).thenReturn(ListStoresResponse.newBuilder().build()); - JobCoordinatorService jcs = - new JobCoordinatorService( - jobRepository, featureSetRepository, specService, jobManager, feastProperties); + jcs.Poll(); verify(jobRepository, times(0)).saveAndFlush(any()); } @@ -101,9 +118,7 @@ public void shouldDoNothingIfNoMatchingFeatureSetsFound() throws InvalidProtocol when(specService.listFeatureSets( Filter.newBuilder().setProject("*").setFeatureSetName("*").build())) .thenReturn(ListFeatureSetsResponse.newBuilder().build()); - JobCoordinatorService jcs = - new JobCoordinatorService( - jobRepository, featureSetRepository, specService, jobManager, feastProperties); + jcs.Poll(); verify(jobRepository, times(0)).saveAndFlush(any()); } @@ -178,9 +193,6 @@ public void shouldGenerateAndSubmitJobsIfAny() throws InvalidProtocolBufferExcep when(jobManager.startJob(argThat(new JobMatcher(expectedInput)))).thenReturn(expected); when(jobManager.getRunnerType()).thenReturn(Runner.DATAFLOW); - JobCoordinatorService jcs = - new JobCoordinatorService( - jobRepository, featureSetRepository, specService, jobManager, feastProperties); jcs.Poll(); verify(jobRepository, times(1)).saveAll(jobArgCaptor.capture()); List actual = jobArgCaptor.getValue(); @@ -288,9 +300,6 @@ public void shouldGroupJobsBySource() throws InvalidProtocolBufferException { when(jobManager.startJob(argThat(new JobMatcher(expectedInput2)))).thenReturn(expected2); when(jobManager.getRunnerType()).thenReturn(Runner.DATAFLOW); - JobCoordinatorService jcs = - new JobCoordinatorService( - jobRepository, featureSetRepository, specService, jobManager, feastProperties); jcs.Poll(); verify(jobRepository, times(1)).saveAll(jobArgCaptor.capture()); @@ -299,4 +308,177 @@ public void shouldGroupJobsBySource() throws InvalidProtocolBufferException { assertThat(actual.get(0), equalTo(expected1)); assertThat(actual.get(1), equalTo(expected2)); } + + @Test + public void shouldSendPendingFeatureSetToJobs() { + FeatureSet fs1 = + TestObjectFactory.CreateFeatureSet( + "fs_1", "project", Collections.emptyList(), Collections.emptyList()); + fs1.setVersion(2); + + FeatureSetJobStatus status1 = + TestObjectFactory.CreateFeatureSetJobStatusWithJob( + JobStatus.RUNNING, FeatureSetProto.FeatureSetJobDeliveryStatus.STATUS_DELIVERED, 1); + FeatureSetJobStatus status2 = + TestObjectFactory.CreateFeatureSetJobStatusWithJob( + JobStatus.RUNNING, FeatureSetProto.FeatureSetJobDeliveryStatus.STATUS_DELIVERED, 1); + FeatureSetJobStatus status3 = + TestObjectFactory.CreateFeatureSetJobStatusWithJob( + JobStatus.ABORTED, FeatureSetProto.FeatureSetJobDeliveryStatus.STATUS_DELIVERED, 2); + + // spec needs to be send + fs1.getJobStatuses().addAll(ImmutableList.of(status1, status2, status3)); + + FeatureSet fs2 = + TestObjectFactory.CreateFeatureSet( + "fs_2", "project", Collections.emptyList(), Collections.emptyList()); + fs2.setVersion(5); + + // spec already sent to kafka + fs2.getJobStatuses() + .addAll( + ImmutableList.of( + TestObjectFactory.CreateFeatureSetJobStatusWithJob( + JobStatus.RUNNING, + FeatureSetProto.FeatureSetJobDeliveryStatus.STATUS_IN_PROGRESS, + 5))); + + // feature set without running jobs attached + FeatureSet fs3 = + TestObjectFactory.CreateFeatureSet( + "fs_3", "project", Collections.emptyList(), Collections.emptyList()); + fs3.getJobStatuses() + .addAll( + ImmutableList.of( + TestObjectFactory.CreateFeatureSetJobStatusWithJob( + JobStatus.ABORTED, + FeatureSetProto.FeatureSetJobDeliveryStatus.STATUS_IN_PROGRESS, + 5))); + + when(featureSetRepository.findAllByStatus(FeatureSetProto.FeatureSetStatus.STATUS_PENDING)) + .thenReturn(ImmutableList.of(fs1, fs2, fs3)); + + jcs.notifyJobsWhenFeatureSetUpdated(); + + verify(kafkaTemplate).sendDefault(eq(fs1.getReference()), any(FeatureSetSpec.class)); + verify(kafkaTemplate, never()).sendDefault(eq(fs2.getReference()), any(FeatureSetSpec.class)); + verify(kafkaTemplate, never()).sendDefault(eq(fs3.getReference()), any(FeatureSetSpec.class)); + + assertThat(status1.getVersion(), is(2)); + assertThat( + status1.getDeliveryStatus(), + is(FeatureSetProto.FeatureSetJobDeliveryStatus.STATUS_IN_PROGRESS)); + + assertThat(status2.getVersion(), is(2)); + assertThat( + status2.getDeliveryStatus(), + is(FeatureSetProto.FeatureSetJobDeliveryStatus.STATUS_IN_PROGRESS)); + + assertThat(status3.getVersion(), is(2)); + assertThat( + status3.getDeliveryStatus(), + is(FeatureSetProto.FeatureSetJobDeliveryStatus.STATUS_DELIVERED)); + } + + @Test + @SneakyThrows + public void shouldNotUpdateJobStatusVersionWhenKafkaUnavailable() { + FeatureSet fsInTest = + TestObjectFactory.CreateFeatureSet( + "fs_1", "project", Collections.emptyList(), Collections.emptyList()); + fsInTest.setVersion(2); + + FeatureSetJobStatus featureSetJobStatus = + TestObjectFactory.CreateFeatureSetJobStatusWithJob( + JobStatus.RUNNING, FeatureSetProto.FeatureSetJobDeliveryStatus.STATUS_DELIVERED, 1); + fsInTest.getJobStatuses().add(featureSetJobStatus); + + CancellationException exc = new CancellationException(); + when(kafkaTemplate.sendDefault(eq(fsInTest.getReference()), any()).get()).thenThrow(exc); + when(featureSetRepository.findAllByStatus(FeatureSetProto.FeatureSetStatus.STATUS_PENDING)) + .thenReturn(ImmutableList.of(fsInTest)); + + jcs.notifyJobsWhenFeatureSetUpdated(); + assertThat(featureSetJobStatus.getVersion(), is(1)); + } + + @Test + public void specAckListenerShouldDoNothingWhenMessageIsOutdated() { + FeatureSet fsInTest = + TestObjectFactory.CreateFeatureSet( + "fs", "project", Collections.emptyList(), Collections.emptyList()); + FeatureSetJobStatus j1 = + TestObjectFactory.CreateFeatureSetJobStatusWithJob( + JobStatus.RUNNING, FeatureSetProto.FeatureSetJobDeliveryStatus.STATUS_IN_PROGRESS, 1); + FeatureSetJobStatus j2 = + TestObjectFactory.CreateFeatureSetJobStatusWithJob( + JobStatus.RUNNING, FeatureSetProto.FeatureSetJobDeliveryStatus.STATUS_IN_PROGRESS, 1); + + fsInTest.getJobStatuses().addAll(Arrays.asList(j1, j2)); + + when(featureSetRepository.findFeatureSetByNameAndProject_Name( + fsInTest.getName(), fsInTest.getProject().getName())) + .thenReturn(fsInTest); + + jcs.listenAckFromJobs(newAckMessage("project/invalid", 0, j1.getJob().getId())); + jcs.listenAckFromJobs(newAckMessage(fsInTest.getReference(), 0, "")); + jcs.listenAckFromJobs(newAckMessage(fsInTest.getReference(), -1, j1.getJob().getId())); + + assertThat( + j1.getDeliveryStatus(), is(FeatureSetProto.FeatureSetJobDeliveryStatus.STATUS_IN_PROGRESS)); + assertThat( + j2.getDeliveryStatus(), is(FeatureSetProto.FeatureSetJobDeliveryStatus.STATUS_IN_PROGRESS)); + } + + @Test + public void specAckListenerShouldUpdateFeatureSetStatus() { + FeatureSet fsInTest = + TestObjectFactory.CreateFeatureSet( + "fs", "project", Collections.emptyList(), Collections.emptyList()); + fsInTest.setStatus(FeatureSetProto.FeatureSetStatus.STATUS_PENDING); + + FeatureSetJobStatus j1 = + TestObjectFactory.CreateFeatureSetJobStatusWithJob( + JobStatus.RUNNING, FeatureSetProto.FeatureSetJobDeliveryStatus.STATUS_IN_PROGRESS, 1); + FeatureSetJobStatus j2 = + TestObjectFactory.CreateFeatureSetJobStatusWithJob( + JobStatus.RUNNING, FeatureSetProto.FeatureSetJobDeliveryStatus.STATUS_IN_PROGRESS, 1); + FeatureSetJobStatus j3 = + TestObjectFactory.CreateFeatureSetJobStatusWithJob( + JobStatus.ABORTED, FeatureSetProto.FeatureSetJobDeliveryStatus.STATUS_IN_PROGRESS, 1); + + fsInTest.getJobStatuses().addAll(Arrays.asList(j1, j2, j3)); + + when(featureSetRepository.findFeatureSetByNameAndProject_Name( + fsInTest.getName(), fsInTest.getProject().getName())) + .thenReturn(fsInTest); + + jcs.listenAckFromJobs( + newAckMessage(fsInTest.getReference(), fsInTest.getVersion(), j1.getJob().getId())); + + assertThat( + j1.getDeliveryStatus(), is(FeatureSetProto.FeatureSetJobDeliveryStatus.STATUS_DELIVERED)); + assertThat(fsInTest.getStatus(), is(FeatureSetProto.FeatureSetStatus.STATUS_PENDING)); + + jcs.listenAckFromJobs( + newAckMessage(fsInTest.getReference(), fsInTest.getVersion(), j2.getJob().getId())); + + assertThat( + j2.getDeliveryStatus(), is(FeatureSetProto.FeatureSetJobDeliveryStatus.STATUS_DELIVERED)); + + assertThat(fsInTest.getStatus(), is(FeatureSetProto.FeatureSetStatus.STATUS_READY)); + } + + private ConsumerRecord newAckMessage( + String key, int version, String jobName) { + return new ConsumerRecord<>( + "topic", + 0, + 0, + key, + IngestionJobProto.FeatureSetSpecAck.newBuilder() + .setFeatureSetVersion(version) + .setJobName(jobName) + .build()); + } } diff --git a/core/src/test/java/feast/core/service/SpecServiceTest.java b/core/src/test/java/feast/core/service/SpecServiceTest.java index ae3e8786440..dad04462ec8 100644 --- a/core/src/test/java/feast/core/service/SpecServiceTest.java +++ b/core/src/test/java/feast/core/service/SpecServiceTest.java @@ -16,12 +16,9 @@ */ package feast.core.service; -import static org.hamcrest.CoreMatchers.is; import static org.hamcrest.MatcherAssert.assertThat; import static org.hamcrest.core.IsEqual.equalTo; import static org.junit.Assert.assertEquals; -import static org.mockito.ArgumentMatchers.any; -import static org.mockito.ArgumentMatchers.eq; import static org.mockito.Mockito.*; import static org.mockito.MockitoAnnotations.initMocks; @@ -48,21 +45,16 @@ import feast.proto.core.FeatureSetProto.EntitySpec; import feast.proto.core.FeatureSetProto.FeatureSetSpec; import feast.proto.core.FeatureSetProto.FeatureSpec; -import feast.proto.core.IngestionJobProto; import feast.proto.core.StoreProto; import feast.proto.core.StoreProto.Store.RedisConfig; import feast.proto.core.StoreProto.Store.StoreType; import feast.proto.core.StoreProto.Store.Subscription; import feast.proto.types.ValueProto.ValueType.Enum; -import io.grpc.StatusRuntimeException; import java.sql.Date; import java.time.Instant; import java.util.*; import java.util.Map.Entry; -import java.util.concurrent.CancellationException; import java.util.stream.Collectors; -import lombok.SneakyThrows; -import org.apache.kafka.clients.consumer.ConsumerRecord; import org.junit.Before; import org.junit.Rule; import org.junit.Test; @@ -70,8 +62,6 @@ import org.mockito.ArgumentCaptor; import org.mockito.ArgumentMatchers; import org.mockito.Mock; -import org.springframework.kafka.core.KafkaTemplate; -import org.springframework.scheduling.annotation.AsyncResult; import org.tensorflow.metadata.v0.BoolDomain; import org.tensorflow.metadata.v0.FeaturePresence; import org.tensorflow.metadata.v0.FeaturePresenceWithinGroup; @@ -96,8 +86,6 @@ public class SpecServiceTest { @Mock private ProjectRepository projectRepository; - @Mock private KafkaTemplate kafkaTemplate; - @Rule public final ExpectedException expectedException = ExpectedException.none(); private SpecService specService; @@ -226,11 +214,8 @@ public void setUp() throws InvalidProtocolBufferException { when(storeRepository.findById("SERVING")).thenReturn(Optional.of(store1)); when(storeRepository.findById("NOTFOUND")).thenReturn(Optional.empty()); - when(kafkaTemplate.sendDefault(any(), any())).thenReturn(new AsyncResult<>(null)); - specService = - new SpecService( - featureSetRepository, storeRepository, projectRepository, defaultSource, kafkaTemplate); + new SpecService(featureSetRepository, storeRepository, projectRepository, defaultSource); } @Test @@ -412,72 +397,6 @@ public void applyFeatureSetShouldUpdateAndSaveFeatureSetIfAlreadyExists() FeatureSet.fromProto(expected)); assertThat(applyFeatureSetResponse.getFeatureSet().getSpec().getVersion(), equalTo(2)); - verify(kafkaTemplate) - .sendDefault(eq(featureSets.get(0).getReference()), any(FeatureSetSpec.class)); - } - - @Test - @SneakyThrows - public void applyFeatureSetShouldNotWorkWithoutKafkaAck() { - FeatureSet fsInTest = featureSets.get(1); - FeatureSetProto.FeatureSet incomingFeatureSet = fsInTest.toProto(); - CancellationException exc = new CancellationException(); - when(kafkaTemplate.sendDefault(eq(fsInTest.getReference()), any()).get()).thenThrow(exc); - - incomingFeatureSet = - incomingFeatureSet - .toBuilder() - .setMeta(incomingFeatureSet.getMeta()) - .setSpec( - incomingFeatureSet - .getSpec() - .toBuilder() - .addFeatures( - FeatureSpec.newBuilder().setName("feature2").setValueType(Enum.STRING)) - .build()) - .build(); - - expectedException.expect(StatusRuntimeException.class); - specService.applyFeatureSet(incomingFeatureSet); - verify(featureSetRepository, never()).saveAndFlush(ArgumentMatchers.any(FeatureSet.class)); - } - - @Test - @SneakyThrows - public void applyFeatureSetShouldUpdateDeliveryStatuses() { - FeatureSet fsInTest = featureSets.get(1); - FeatureSetJobStatus j1 = - newJob( - fsInTest, - JobStatus.RUNNING, - FeatureSetProto.FeatureSetJobDeliveryStatus.STATUS_DELIVERED); - FeatureSetJobStatus j2 = - newJob( - fsInTest, - JobStatus.ABORTED, - FeatureSetProto.FeatureSetJobDeliveryStatus.STATUS_DELIVERED); - - fsInTest.getJobStatuses().addAll(Arrays.asList(j1, j2)); - - FeatureSetProto.FeatureSet incomingFeatureSet = fsInTest.toProto(); - incomingFeatureSet = - incomingFeatureSet - .toBuilder() - .setMeta(incomingFeatureSet.getMeta()) - .setSpec( - incomingFeatureSet - .getSpec() - .toBuilder() - .addFeatures( - FeatureSpec.newBuilder().setName("feature2").setValueType(Enum.STRING)) - .build()) - .build(); - - specService.applyFeatureSet(incomingFeatureSet); - assertThat( - j1.getDeliveryStatus(), is(FeatureSetProto.FeatureSetJobDeliveryStatus.STATUS_IN_PROGRESS)); - assertThat( - j2.getDeliveryStatus(), is(FeatureSetProto.FeatureSetJobDeliveryStatus.STATUS_DELIVERED)); } @Test @@ -1037,70 +956,6 @@ public void getOrListFeatureSetShouldUseDefaultProjectIfProjectUnspecified() assertThat(listResponse.getFeatureSetsList(), equalTo(Arrays.asList(expected.toProto()))); } - @Test - public void specAckListenerShouldDoNothingWhenMessageIsOutdated() { - FeatureSet fsInTest = featureSets.get(1); - FeatureSetJobStatus j1 = - newJob( - fsInTest, - JobStatus.RUNNING, - FeatureSetProto.FeatureSetJobDeliveryStatus.STATUS_IN_PROGRESS); - FeatureSetJobStatus j2 = - newJob( - fsInTest, - JobStatus.RUNNING, - FeatureSetProto.FeatureSetJobDeliveryStatus.STATUS_IN_PROGRESS); - - fsInTest.getJobStatuses().addAll(Arrays.asList(j1, j2)); - - specService.listenAckFromJobs(newAckMessage("project/invalid", 0, j1.getJob().getId())); - specService.listenAckFromJobs(newAckMessage(fsInTest.getReference(), 0, "")); - specService.listenAckFromJobs(newAckMessage(fsInTest.getReference(), -1, j1.getJob().getId())); - - assertThat( - j1.getDeliveryStatus(), is(FeatureSetProto.FeatureSetJobDeliveryStatus.STATUS_IN_PROGRESS)); - assertThat( - j2.getDeliveryStatus(), is(FeatureSetProto.FeatureSetJobDeliveryStatus.STATUS_IN_PROGRESS)); - } - - @Test - public void specAckListenerShouldUpdateFeatureSetStatus() { - FeatureSet fsInTest = featureSets.get(1); - fsInTest.setStatus(FeatureSetProto.FeatureSetStatus.STATUS_PENDING); - - FeatureSetJobStatus j1 = - newJob( - fsInTest, - JobStatus.RUNNING, - FeatureSetProto.FeatureSetJobDeliveryStatus.STATUS_IN_PROGRESS); - FeatureSetJobStatus j2 = - newJob( - fsInTest, - JobStatus.RUNNING, - FeatureSetProto.FeatureSetJobDeliveryStatus.STATUS_IN_PROGRESS); - FeatureSetJobStatus j3 = - newJob( - fsInTest, - JobStatus.ABORTED, - FeatureSetProto.FeatureSetJobDeliveryStatus.STATUS_IN_PROGRESS); - - fsInTest.getJobStatuses().addAll(Arrays.asList(j1, j2, j3)); - - specService.listenAckFromJobs( - newAckMessage(fsInTest.getReference(), fsInTest.getVersion(), j1.getJob().getId())); - - assertThat( - j1.getDeliveryStatus(), is(FeatureSetProto.FeatureSetJobDeliveryStatus.STATUS_DELIVERED)); - assertThat(fsInTest.getStatus(), is(FeatureSetProto.FeatureSetStatus.STATUS_PENDING)); - - specService.listenAckFromJobs( - newAckMessage(fsInTest.getReference(), fsInTest.getVersion(), j2.getJob().getId())); - - assertThat( - j2.getDeliveryStatus(), is(FeatureSetProto.FeatureSetJobDeliveryStatus.STATUS_DELIVERED)); - assertThat(fsInTest.getStatus(), is(FeatureSetProto.FeatureSetStatus.STATUS_READY)); - } - private FeatureSet newDummyFeatureSet(String name, String project) { FeatureSpec f1 = FeatureSpec.newBuilder() @@ -1118,33 +973,6 @@ private FeatureSet newDummyFeatureSet(String name, String project) { return fs; } - private FeatureSetJobStatus newJob( - FeatureSet fs, JobStatus status, FeatureSetProto.FeatureSetJobDeliveryStatus deliveryStatus) { - Job job = new Job(); - job.setStatus(status); - job.setId(UUID.randomUUID().toString()); - - FeatureSetJobStatus featureSetJobStatus = new FeatureSetJobStatus(); - featureSetJobStatus.setJob(job); - featureSetJobStatus.setFeatureSet(fs); - featureSetJobStatus.setDeliveryStatus(deliveryStatus); - - return featureSetJobStatus; - } - - private ConsumerRecord newAckMessage( - String key, int version, String jobName) { - return new ConsumerRecord<>( - "topic", - 0, - 0, - key, - IngestionJobProto.FeatureSetSpecAck.newBuilder() - .setFeatureSetVersion(version) - .setJobName(jobName) - .build()); - } - private Store newDummyStore(String name) { // Add type to this method when we enable filtering by type Store store = new Store(); diff --git a/core/src/test/java/feast/core/service/TestObjectFactory.java b/core/src/test/java/feast/core/service/TestObjectFactory.java index 53522e09ca4..99e1cf9e9f2 100644 --- a/core/src/test/java/feast/core/service/TestObjectFactory.java +++ b/core/src/test/java/feast/core/service/TestObjectFactory.java @@ -16,16 +16,14 @@ */ package feast.core.service; -import feast.core.model.Entity; -import feast.core.model.Feature; -import feast.core.model.FeatureSet; -import feast.core.model.Source; +import feast.core.model.*; import feast.proto.core.FeatureSetProto; import feast.proto.core.SourceProto; import feast.proto.types.ValueProto; import java.util.HashMap; import java.util.List; import java.util.Map; +import java.util.UUID; public class TestObjectFactory { @@ -73,4 +71,19 @@ public static Entity CreateEntity(String name, ValueProto.ValueType.Enum valueTy return Entity.fromProto( FeatureSetProto.EntitySpec.newBuilder().setName(name).setValueType(valueType).build()); } + + public static FeatureSetJobStatus CreateFeatureSetJobStatusWithJob( + JobStatus status, FeatureSetProto.FeatureSetJobDeliveryStatus deliveryStatus, int version) { + Job job = new Job(); + job.setStatus(status); + job.setId(UUID.randomUUID().toString()); + + FeatureSetJobStatus featureSetJobStatus = new FeatureSetJobStatus(); + featureSetJobStatus.setJob(job); + + featureSetJobStatus.setDeliveryStatus(deliveryStatus); + featureSetJobStatus.setVersion(version); + + return featureSetJobStatus; + } } From f117b657956560d57a8e3048dbe3343131030e54 Mon Sep 17 00:00:00 2001 From: Ches Martin Date: Thu, 18 Jun 2020 23:38:05 +0700 Subject: [PATCH 066/133] Fix Optional#get() and string comparison bugs in JobService (#804) * Fix Optional#get() and string comparison bugs in JobService Clears a FIXME left from #650, get() uses that will always fail. Also fixes a bunch of == comparisons on strings, reference equality is not what you want there. * Hide Maven transfer progress, for CI It makes trying to read output in GitHub Actions horrible. Seeing failures in red is kind of nice, so `--no-transfer-progress` keeps colors unlike `--batch-mode`. --- Makefile | 6 +- .../java/feast/core/service/JobService.java | 75 +++++++++---------- .../feast/core/service/JobServiceTest.java | 28 +++---- 3 files changed, 54 insertions(+), 55 deletions(-) diff --git a/Makefile b/Makefile index 1f41ad09049..7abbda0a1b2 100644 --- a/Makefile +++ b/Makefile @@ -41,13 +41,13 @@ format-java: mvn spotless:apply lint-java: - mvn spotless:check + mvn --no-transfer-progress spotless:check test-java: - mvn test + mvn --no-transfer-progress test test-java-with-coverage: - mvn test jacoco:report-aggregate + mvn --no-transfer-progress test jacoco:report-aggregate build-java: mvn clean verify diff --git a/core/src/main/java/feast/core/service/JobService.java b/core/src/main/java/feast/core/service/JobService.java index b0b7aeeb35a..a671a0f68ea 100644 --- a/core/src/main/java/feast/core/service/JobService.java +++ b/core/src/main/java/feast/core/service/JobService.java @@ -71,7 +71,8 @@ public JobService( } } - /* Job Service API */ + // region Job Service API + /** * List Ingestion Jobs in Feast matching the given request. See CoreService protobuf documentation * for more detailed documentation. @@ -88,16 +89,16 @@ public ListIngestionJobsResponse listJobs(ListIngestionJobsRequest request) // check that filter specified and not empty if (request.hasFilter() - && !(request.getFilter().getId() == "" - && request.getFilter().getStoreName() == "" - && request.getFilter().hasFeatureSetReference() == false)) { + && !(request.getFilter().getId().isEmpty() + && request.getFilter().getStoreName().isEmpty() + && !request.getFilter().hasFeatureSetReference())) { // filter jobs based on request filter ListIngestionJobsRequest.Filter filter = request.getFilter(); // for proto3, default value for missing values: // - numeric values (ie int) is zero // - strings is empty string - if (filter.getId() != "") { + if (!filter.getId().isEmpty()) { // get by id: no more filters required: found job Optional job = this.jobRepository.findById(filter.getId()); if (job.isPresent()) { @@ -105,7 +106,7 @@ public ListIngestionJobsResponse listJobs(ListIngestionJobsRequest request) } } else { // multiple filters can apply together in an 'and' operation - if (filter.getStoreName() != "") { + if (!filter.getStoreName().isEmpty()) { // find jobs by name List jobs = this.jobRepository.findByStoreName(filter.getStoreName()); Set jobIds = jobs.stream().map(Job::getId).collect(Collectors.toSet()); @@ -140,7 +141,7 @@ public ListIngestionJobsResponse listJobs(ListIngestionJobsRequest request) // convert matching job models to ingestion job protos List ingestJobs = new ArrayList<>(); for (String jobId : matchingJobIds) { - Job job = this.jobRepository.findById(jobId).get(); + Job job = this.jobRepository.findById(jobId).orElseThrow(); // job that failed on start won't be converted toProto successfully // and they're irrelevant here if (job.getStatus() == JobStatus.ERROR) { @@ -160,21 +161,20 @@ public ListIngestionJobsResponse listJobs(ListIngestionJobsRequest request) * @param request restart ingestion job request specifying which job to stop * @throws NoSuchElementException when restart job request requests to restart a nonexistent job. * @throws UnsupportedOperationException when job to be restarted is in an unsupported status - * @throws InvalidProtocolBufferException on error when constructing response protobuf */ @Transactional - public RestartIngestionJobResponse restartJob(RestartIngestionJobRequest request) - throws InvalidProtocolBufferException { - // check job exists - Optional getJob = this.jobRepository.findById(request.getId()); - if (getJob.isEmpty()) { - // FIXME: if getJob.isEmpty then constructing this error message will always throw an error... - throw new NoSuchElementException( - "Attempted to stop nonexistent job with id: " + getJob.get().getId()); - } + public RestartIngestionJobResponse restartJob(RestartIngestionJobRequest request) { + String jobId = request.getId(); + + Job job = + this.jobRepository + .findById(jobId) + .orElseThrow( + () -> + new NoSuchElementException( + "Attempted to restart nonexistent job with id: " + jobId)); // check job status is valid for restarting - Job job = getJob.get(); JobStatus status = job.getStatus(); if (status.isTransitional() || status.isTerminal() || status == JobStatus.UNKNOWN) { throw new UnsupportedOperationException( @@ -202,20 +202,20 @@ public RestartIngestionJobResponse restartJob(RestartIngestionJobRequest request * @param request stop ingestion job request specifying which job to stop * @throws NoSuchElementException when stop job request requests to stop a nonexistent job. * @throws UnsupportedOperationException when job to be stopped is in an unsupported status - * @throws InvalidProtocolBufferException on error when constructing response protobuf */ @Transactional - public StopIngestionJobResponse stopJob(StopIngestionJobRequest request) - throws InvalidProtocolBufferException { - // check job exists - Optional getJob = this.jobRepository.findById(request.getId()); - if (getJob.isEmpty()) { - throw new NoSuchElementException( - "Attempted to stop nonexistent job with id: " + getJob.get().getId()); - } + public StopIngestionJobResponse stopJob(StopIngestionJobRequest request) { + String jobId = request.getId(); + + Job job = + this.jobRepository + .findById(jobId) + .orElseThrow( + () -> + new NoSuchElementException( + "Attempted to stop nonexistent job with id: " + jobId)); // check job status is valid for stopping - Job job = getJob.get(); JobStatus status = job.getStatus(); if (status.isTerminal()) { // do nothing - job is already stopped @@ -240,7 +240,9 @@ public StopIngestionJobResponse stopJob(StopIngestionJobRequest request) return StopIngestionJobResponse.newBuilder().build(); } - /* Private Utility Methods */ + // endregion + // region Private Utility Methods + private Set mergeResults(Set results, Collection newResults) { if (results.size() <= 0) { // no existing results: copy over new results @@ -252,7 +254,7 @@ private Set mergeResults(Set results, Collection newResults) { return results; } - // converts feature set reference to a list feature set filter + /** converts feature set reference to a list feature set filter */ private ListFeatureSetsRequest.Filter toListFeatureSetFilter(FeatureSetReference fsReference) { // match featuresets using contents of featureset reference String fsName = fsReference.getName(); @@ -262,16 +264,13 @@ private ListFeatureSetsRequest.Filter toListFeatureSetFilter(FeatureSetReference // for proto3, default value for missing values: // - numeric values (ie int) is zero // - strings is empty string - ListFeatureSetsRequest.Filter filter = - ListFeatureSetsRequest.Filter.newBuilder() - .setFeatureSetName((fsName != "") ? fsName : "*") - .setProject((fsProject != "") ? fsProject : "*") - .build(); - - return filter; + return ListFeatureSetsRequest.Filter.newBuilder() + .setFeatureSetName(fsName.isEmpty() ? "*" : fsName) + .setProject(fsProject.isEmpty() ? "*" : fsProject) + .build(); } - // sync job status using job manager + /** sync job status using job manager */ private Job syncJobStatus(JobManager jobManager, Job job) { JobStatus newStatus = jobManager.getJobStatus(job); // log job status transition diff --git a/core/src/test/java/feast/core/service/JobServiceTest.java b/core/src/test/java/feast/core/service/JobServiceTest.java index 27509620ff2..6020db7b668 100644 --- a/core/src/test/java/feast/core/service/JobServiceTest.java +++ b/core/src/test/java/feast/core/service/JobServiceTest.java @@ -46,11 +46,7 @@ import feast.proto.core.StoreProto.Store.StoreType; import feast.proto.types.ValueProto.ValueType.Enum; import java.time.Instant; -import java.util.ArrayList; -import java.util.Arrays; -import java.util.Date; -import java.util.List; -import java.util.Optional; +import java.util.*; import org.junit.Before; import org.junit.Test; import org.mockito.Mock; @@ -87,7 +83,7 @@ public void setup() { "*:*:*"); // fake featureset & job - this.featureSet = this.newDummyFeatureSet("food", 2, "hunger"); + this.featureSet = this.newDummyFeatureSet("food", "hunger"); this.job = this.newDummyJob("kafka-to-redis", "job-1111", JobStatus.PENDING); try { this.ingestionJob = this.job.toProto(); @@ -140,7 +136,7 @@ public void setupJobManager() { .thenReturn(this.newDummyJob(this.job.getId(), this.job.getExtId(), JobStatus.PENDING)); } - private FeatureSet newDummyFeatureSet(String name, int version, String project) { + private FeatureSet newDummyFeatureSet(String name, String project) { Feature feature = TestObjectFactory.CreateFeature(name + "_feature", Enum.INT64); Entity entity = TestObjectFactory.CreateEntity(name + "_entity", Enum.STRING); @@ -242,7 +238,7 @@ public void testListJobsByStoreName() { @Test public void testListIngestionJobByFeatureSetReference() { - // list job by feature set reference: name and version and project + // list job by feature set reference: name and project ListIngestionJobsRequest.Filter filter = ListIngestionJobsRequest.Filter.newBuilder() .setFeatureSetReference(this.fsReferences.get(0)) @@ -281,7 +277,7 @@ private StopIngestionJobResponse tryStopJob( fail("Expected exception, but none was thrown"); } } catch (Exception e) { - if (expectError != true) { + if (!expectError) { // unexpected exception e.printStackTrace(); fail("Caught Unexpected exception trying to restart job"); @@ -330,8 +326,7 @@ public void testStopUnsupportedError() { // check for UnsupportedOperationException when trying to stop jobs are // in an in unknown or in a transitional state JobStatus prevStatus = this.job.getStatus(); - List unsupportedStatuses = new ArrayList<>(); - unsupportedStatuses.addAll(JobStatus.getTransitionalStates()); + List unsupportedStatuses = new ArrayList<>(JobStatus.getTransitionalStates()); unsupportedStatuses.add(JobStatus.UNKNOWN); for (JobStatus status : unsupportedStatuses) { @@ -345,6 +340,12 @@ public void testStopUnsupportedError() { this.job.setStatus(prevStatus); } + @Test(expected = NoSuchElementException.class) + public void testStopJobForUnknownId() { + var request = StopIngestionJobRequest.newBuilder().setId("bogusJobId").build(); + jobService.stopJob(request); + } + // restart jobs private RestartIngestionJobResponse tryRestartJob( RestartIngestionJobRequest request, boolean expectError) { @@ -356,7 +357,7 @@ private RestartIngestionJobResponse tryRestartJob( fail("Expected exception, but none was thrown"); } } catch (Exception e) { - if (expectError != true) { + if (!expectError) { // unexpected exception e.printStackTrace(); fail("Caught Unexpected exception trying to stop job"); @@ -392,8 +393,7 @@ public void testRestartUnsupportedError() { // check for UnsupportedOperationException when trying to restart jobs are // in an in unknown or in a transitional state JobStatus prevStatus = this.job.getStatus(); - List unsupportedStatuses = new ArrayList<>(); - unsupportedStatuses.addAll(JobStatus.getTransitionalStates()); + List unsupportedStatuses = new ArrayList<>(JobStatus.getTransitionalStates()); unsupportedStatuses.add(JobStatus.UNKNOWN); for (JobStatus status : unsupportedStatuses) { From a74a4847ad766f762f9dbc840e8d0f0f2f08ac16 Mon Sep 17 00:00:00 2001 From: Thi Nguyen Date: Thu, 18 Jun 2020 20:54:05 -0400 Subject: [PATCH 067/133] Use latest instead of dev as the default image tag in helm charts (#806) Co-authored-by: Thi Nguyen --- infra/charts/feast/charts/feast-core/README.md | 2 +- infra/charts/feast/charts/feast-core/values.yaml | 2 +- infra/charts/feast/charts/feast-serving/README.md | 2 +- infra/charts/feast/charts/feast-serving/values.yaml | 2 +- 4 files changed, 4 insertions(+), 4 deletions(-) diff --git a/infra/charts/feast/charts/feast-core/README.md b/infra/charts/feast/charts/feast-core/README.md index f85b4223c84..b64afbc5982 100644 --- a/infra/charts/feast/charts/feast-core/README.md +++ b/infra/charts/feast/charts/feast-core/README.md @@ -23,7 +23,7 @@ Current chart version is `0.5.0-alpha.1` | gcpServiceAccount.existingSecret.name | string | `"feast-gcp-service-account"` | Name of the existing secret containing the service account | | image.pullPolicy | string | `"IfNotPresent"` | Image pull policy | | image.repository | string | `"gcr.io/kf-feast/feast-core"` | Docker image repository | -| image.tag | string | `"dev"` | Image tag | +| image.tag | string | `"latest"` | Image tag | | ingress.grpc.annotations | object | `{}` | Extra annotations for the ingress | | ingress.grpc.auth.enabled | bool | `false` | Flag to enable auth | | ingress.grpc.class | string | `"nginx"` | Which ingress controller to use | diff --git a/infra/charts/feast/charts/feast-core/values.yaml b/infra/charts/feast/charts/feast-core/values.yaml index d5ad50ea156..25fee869155 100644 --- a/infra/charts/feast/charts/feast-core/values.yaml +++ b/infra/charts/feast/charts/feast-core/values.yaml @@ -5,7 +5,7 @@ image: # image.repository -- Docker image repository repository: gcr.io/kf-feast/feast-core # image.tag -- Image tag - tag: dev + tag: latest # image.pullPolicy -- Image pull policy pullPolicy: IfNotPresent diff --git a/infra/charts/feast/charts/feast-serving/README.md b/infra/charts/feast/charts/feast-serving/README.md index eb638359e7d..cd3506ca338 100644 --- a/infra/charts/feast/charts/feast-serving/README.md +++ b/infra/charts/feast/charts/feast-serving/README.md @@ -23,7 +23,7 @@ Current chart version is `0.5.0-alpha.1` | gcpServiceAccount.existingSecret.name | string | `"feast-gcp-service-account"` | Name of the existing secret containing the service account | | image.pullPolicy | string | `"IfNotPresent"` | Image pull policy | | image.repository | string | `"gcr.io/kf-feast/feast-serving"` | Docker image repository | -| image.tag | string | `"dev"` | Image tag | +| image.tag | string | `"latest"` | Image tag | | ingress.grpc.annotations | object | `{}` | Extra annotations for the ingress | | ingress.grpc.auth.enabled | bool | `false` | Flag to enable auth | | ingress.grpc.class | string | `"nginx"` | Which ingress controller to use | diff --git a/infra/charts/feast/charts/feast-serving/values.yaml b/infra/charts/feast/charts/feast-serving/values.yaml index 18069c28634..0bc845a7143 100644 --- a/infra/charts/feast/charts/feast-serving/values.yaml +++ b/infra/charts/feast/charts/feast-serving/values.yaml @@ -5,7 +5,7 @@ image: # image.repository -- Docker image repository repository: gcr.io/kf-feast/feast-serving # image.tag -- Image tag - tag: dev + tag: latest # image.pullPolicy -- Image pull policy pullPolicy: IfNotPresent From 00698c58a4a2ed5e3f744e34248bf34ca7039a01 Mon Sep 17 00:00:00 2001 From: Willem Pienaar Date: Fri, 19 Jun 2020 05:00:34 +0000 Subject: [PATCH 068/133] GitBook: [master] 3 pages modified --- docs/contributing/adding-a-new-store.md | 3 +- .../deploying-feast/docker-compose.md | 62 +++++++------------ .../deploying-feast/kubernetes.md | 2 +- 3 files changed, 24 insertions(+), 43 deletions(-) diff --git a/docs/contributing/adding-a-new-store.md b/docs/contributing/adding-a-new-store.md index 56c06b9b4b6..d59b82e6463 100644 --- a/docs/contributing/adding-a-new-store.md +++ b/docs/contributing/adding-a-new-store.md @@ -82,6 +82,5 @@ Historical storage Online storage -* [Redis](https://github.com/gojek/feast/tree/master/storage/connectors/redis) -* [Redis Cluster](https://github.com/gojek/feast/tree/master/storage/connectors/rediscluster) +* [Redis & Redis Cluster](https://github.com/gojek/feast/tree/master/storage/connectors/redis) diff --git a/docs/getting-started/deploying-feast/docker-compose.md b/docs/getting-started/deploying-feast/docker-compose.md index cb174d767d3..1608c2bf6f5 100644 --- a/docs/getting-started/deploying-feast/docker-compose.md +++ b/docs/getting-started/deploying-feast/docker-compose.md @@ -4,26 +4,16 @@ This guide will bring Feast up using Docker Compose. This will allow you to: -* Create, register, and manage feature sets +* Create, register, and manage [feature sets](../../user-guide/feature-sets.md) * Ingest feature data into Feast * Retrieve features for online serving * Retrieve features for batch serving \(only if using Google Cloud Platform\) -This guide is split into three parts: - -1. Setting up your environment -2. Starting Feast with **online serving support only** \(does not require GCP\). -3. Starting Feast with support for **both online and batch** serving \(requires GCP\) - -{% hint style="info" %} -The docker compose setup uses Direct Runner for the Apache Beam jobs that populate data stores. Running Beam with the Direct Runner means it does not need a dedicated runner like Flink or Dataflow, but this comes at the cost of performance. We recommend the use of a dedicated runner when running Feast with very large workloads. -{% endhint %} - ### 0. Requirements -* [Docker compose](https://docs.docker.com/compose/install/) must be installed. -* \(for batch serving only\) For batch serving you will also need a [GCP service account key](https://cloud.google.com/iam/docs/creating-managing-service-account-keys) that has access to [Google Cloud Storage](https://cloud.google.com/storage) and [BigQuery](https://cloud.google.com/bigquery). -* \(for batch serving only\) [Google Cloud SDK ](https://cloud.google.com/sdk/install)installed, authenticated, and configured to the GCP project you will use. +* [Docker Compose](https://docs.docker.com/compose/install/) should be installed +* [GCP service account](https://cloud.google.com/iam/docs/creating-managing-service-account-keys) that has access to [Google Cloud Storage](https://cloud.google.com/storage) and [BigQuery](https://cloud.google.com/bigquery) \(for historical serving only\). +* [Google Cloud SDK ](https://cloud.google.com/sdk/install)installed, authenticated, and configured to the GCP project you want to use \(for historical serving only\). ## 1. Set up environment @@ -35,13 +25,7 @@ Make a copy of the `.env.sample` file: cp .env.sample .env ``` -The following command will bring up a Feast deployment with a single Feast Core instance without any online stores: - -```javascript -docker-compose up -d -``` - -## 2. Docker Compose for Online Serving +## 2. Start Feast for Online Serving Run the following command if you would like to start both Feast Core and Feast Serving at the same time. The Feast Serving online deployment will use Redis as its database. @@ -52,13 +36,7 @@ docker-compose \ up -d ``` -If you already have Feast Core running \(externally or from the earlier section\), then you can optionally bring up only the Feast Serving online deployment for Redis. - -```javascript -docker-compose -f docker-compose.online.yml up -d -``` - -Your Feast deployment should now be starting up. Have a look at the docker logs, especially that of the Jupyter container: +Your Feast deployment should now be starting up. Have a look at the Jupyter docker logs to know when your notebook is ready to be used: ```text docker logs feast_jupyter_1 @@ -71,17 +49,19 @@ Once it is ready you should be able to connect to a local notebook that contains [I 05:50:22.991 NotebookApp] http://localhost:8888/ ``` +Navigate to the Jupyter Notebook container at the following URL when the container is ready. + {% embed url="http://localhost:8888/tree/feast/examples" caption="" %} -## 3. Docker Compose for Online and Batch Serving +## 3. Start Feast for Training and Online Serving {% hint style="info" %} -Batch serving requires Google Cloud Storage to function, specifically Google Cloud Storage \(GCP\) and BigQuery. +Batch serving requires Google Cloud Platform to function, specifically Google Cloud Storage \(GCP\) and BigQuery. {% endhint %} ### 3.1 Set up Google Cloud Platform -Create a [service account ](https://cloud.google.com/iam/docs/creating-managing-service-accounts)from the GCP console and copy it to the `infra/docker-compose/gcp-service-accounts` folder: +Create a [service account ](https://cloud.google.com/iam/docs/creating-managing-service-accounts)and copy it to the `infra/docker-compose/gcp-service-accounts` folder: ```javascript cp my-service-account.json ${FEAST_HOME_DIR}/infra/docker-compose/gcp-service-accounts @@ -99,19 +79,19 @@ Configure the `.env` file based on your environment. At the very least you have | Parameter | Description | | :--- | :--- | -| FEAST\_CORE\_GCP\_SERVICE\_ACCOUNT\_KEY | This should be your service account file name, for example `key.json`. | -| FEAST\_BATCH\_SERVING\_GCP\_SERVICE\_ACCOUNT\_KEY | This should be your service account file name, for example `key.json` | -| FEAST\_JUPYTER\_GCP\_SERVICE\_ACCOUNT\_KEY | This should be your service account file name, for example `key.json` | -| FEAST\_JOB\_STAGING\_LOCATION | Google Cloud Storage bucket that Feast will use to stage data exports and batch retrieval requests, for example `gs://your-gcs-bucket/staging` | +| `FEAST_CORE_GCP_SERVICE_ACCOUNT_KEY` | This should be your service account file name, for example `key.json`. | +| `FEAST_BATCH_SERVING_GCP_SERVICE_ACCOUNT_KEY` | This should be your service account file name, for example `key.json` | +| `FEAST_JUPYTER_GCP_SERVICE_ACCOUNT_KEY` | This should be your service account file name, for example `key.json` | -### 3.3 Configure .bq-store.yml +### 3.3 Configure Historical Serving -We will also need to configure the `bq-store.yml` file inside `infra/docker-compose/serving/` to configure the BigQuery storage configuration as well as the feature sets that the store subscribes to. At a minimum you will need to set: +We will also need to configure the `batch-serving.yml` file inside `infra/docker-compose/serving/`. This configuration is used to retrieve training datasets from Feast. At a minimum you will need to set: | Parameter | Description | | :--- | :--- | -| bigquery\_config.project\_id | This is you [GCP project Id](https://cloud.google.com/resource-manager/docs/creating-managing-projects). | -| bigquery\_config.dataset\_id | This is the name of the BigQuery dataset that tables will be created in. Each feature set will have one table in BigQuery. | +| `feast.stores.config.project_id` | This is your [GCP project Id](https://cloud.google.com/resource-manager/docs/creating-managing-projects). | +| `feast.stores.config.dataset_id` | This is the name of the BigQuery dataset that feature data will be stored in. | +| `feast.stores.config.staging_location` | This is the staging location on Google Cloud Storage for retrieval of training datasets | ### 3.4 Start Feast \(with batch retrieval support\) @@ -125,7 +105,9 @@ docker-compose \ up -d ``` -A Jupyter Notebook environment should become available within a few minutes: +A Jupyter Notebook should become available within a few minutes: {% embed url="http://localhost:8888/tree/feast/examples" %} + + diff --git a/docs/getting-started/deploying-feast/kubernetes.md b/docs/getting-started/deploying-feast/kubernetes.md index a7432836f3a..51ec954e19f 100644 --- a/docs/getting-started/deploying-feast/kubernetes.md +++ b/docs/getting-started/deploying-feast/kubernetes.md @@ -20,7 +20,7 @@ This guide requires [Google Cloud Platform](https://cloud.google.com/) for insta 1. [Google Cloud SDK ](https://cloud.google.com/sdk/install)installed, authenticated, and configured to the project you will use. 2. [Kubectl](https://kubernetes.io/docs/tasks/tools/install-kubectl/) installed. -3. [Helm](https://helm.sh/3) \(2.16.0 or greater\) installed on your local machine with Tiller installed in your cluster. Helm 3 has not been tested yet. +3. [Helm](https://helm.sh/) \(2.16.0 or greater\) installed on your local machine with Tiller installed in your cluster. Helm 3 has not been tested yet. ## 1. Set up GCP From 49dca68d7cb51c06f3b07e0d5ed7cc1ac45d03b5 Mon Sep 17 00:00:00 2001 From: Jayanth Kumar M J Date: Fri, 19 Jun 2020 01:26:45 -0400 Subject: [PATCH 069/133] Add authentication support for end-to-end tests. (#807) * test end to end with auth. * removed multiple initialization of client. --- .prow/config.yaml | 14 +++++++ infra/scripts/test-end-to-end.sh | 41 ++++++++++++++++++- tests/e2e/conftest.py | 1 + tests/e2e/redis/basic-ingest-redis-serving.py | 13 +++++- 4 files changed, 66 insertions(+), 3 deletions(-) diff --git a/.prow/config.yaml b/.prow/config.yaml index 7035d5cc9be..1c92884c802 100644 --- a/.prow/config.yaml +++ b/.prow/config.yaml @@ -154,6 +154,20 @@ presubmits: memory: "6144Mi" skip_branches: - ^v0\.(3|4)-branch$ + + - name: test-end-to-end-auth + decorate: true + always_run: false + spec: + containers: + - image: maven:3.6-jdk-11 + command: ["infra/scripts/test-end-to-end.sh", "True"] + resources: + requests: + cpu: "6" + memory: "6144Mi" + skip_branches: + - ^v0\.(3|4)-branch$ - name: test-end-to-end-redis-cluster decorate: true diff --git a/infra/scripts/test-end-to-end.sh b/infra/scripts/test-end-to-end.sh index c37ad20eb48..34bcba54103 100755 --- a/infra/scripts/test-end-to-end.sh +++ b/infra/scripts/test-end-to-end.sh @@ -2,6 +2,11 @@ set -e set -o pipefail +ENABLE_AUTH="False" +if [[ -n $1 ]]; then + ENABLE_AUTH=$1 +fi +echo "Authenication enabled : ${ENABLE_AUTH}" test -z ${GOOGLE_APPLICATION_CREDENTIALS} && GOOGLE_APPLICATION_CREDENTIALS="/etc/service-account/service-account.json" test -z ${SKIP_BUILD_JARS} && SKIP_BUILD_JARS="false" @@ -40,7 +45,39 @@ else echo "[DEBUG] Skipping building jars" fi -start_feast_core +echo " +============================================================ +Starting Feast Core +============================================================ +" +# Start Feast Core with auth if enabled +cat < /tmp/core.warehouse.application.yml +feast: + jobs: + polling_interval_milliseconds: 10000 + active_runner: direct + runners: + - name: direct + type: DirectRunner + options:{} + security: + authentication: + enabled: true + provider: jwt + authorization: + enabled: false + provider: none +EOF + +if [[ ${ENABLE_AUTH} = "True" ]]; + then + print_banner "Starting 'Feast core with auth'." + start_feast_core /tmp/core.warehouse.application.yml + else + print_banner "Starting 'Feast core without auth'." + start_feast_core +fi + start_feast_serving install_python_with_miniconda_and_feast_sdk @@ -53,7 +90,7 @@ ORIGINAL_DIR=$(pwd) cd tests/e2e set +e -pytest redis/* --junitxml=${LOGS_ARTIFACT_PATH}/python-sdk-test-report.xml +pytest redis/* --enable_auth=${ENABLE_AUTH} --junitxml=${LOGS_ARTIFACT_PATH}/python-sdk-test-report.xml TEST_EXIT_CODE=$? if [[ ${TEST_EXIT_CODE} != 0 ]]; then diff --git a/tests/e2e/conftest.py b/tests/e2e/conftest.py index 8ea472b6620..89b08023347 100644 --- a/tests/e2e/conftest.py +++ b/tests/e2e/conftest.py @@ -3,3 +3,4 @@ def pytest_addoption(parser): parser.addoption("--serving_url", action="store", default="localhost:6566") parser.addoption("--allow_dirty", action="store", default="False") parser.addoption("--gcs_path", action="store", default="gs://feast-templocation-kf-feast/") + parser.addoption("--enable_auth", action="store", default="False") diff --git a/tests/e2e/redis/basic-ingest-redis-serving.py b/tests/e2e/redis/basic-ingest-redis-serving.py index 1d38cc924b4..a6183b2dbc4 100644 --- a/tests/e2e/redis/basic-ingest-redis-serving.py +++ b/tests/e2e/redis/basic-ingest-redis-serving.py @@ -49,10 +49,21 @@ def allow_dirty(pytestconfig): "allow_dirty").lower() == "true" else False +@pytest.fixture(scope='module') +def enable_auth(pytestconfig): + return True if pytestconfig.getoption( + "enable_auth").lower() == "true" else False + + @pytest.fixture(scope='module') def client(core_url, serving_url, allow_dirty): # Get client for core and serving - client = Client(core_url=core_url, serving_url=serving_url) + # if enable_auth is True, Google Id token will be + # passed in the metadata for authentication. + client = Client(core_url=core_url, + serving_url=serving_url, + core_enable_auth=enable_auth, + core_auth_provider="google") client.create_project(PROJECT_NAME) # Ensure Feast core is active, but empty From f425d7de64392217bbd65a4f07ad2848751a122d Mon Sep 17 00:00:00 2001 From: Willem Pienaar <6728866+woop@users.noreply.github.com> Date: Fri, 19 Jun 2020 13:49:05 +0800 Subject: [PATCH 070/133] Refactor Source & Job data model and Stop Duplicate Ingestion Jobs (#685) * Rebase & squash on master * Fix java unit tests * Rebase on JobService PR fix * Update JobUpdateTask's createJobId() to generate id based on source config instead of id. * Fix wrong conflict resolve in FeatureStreamConfig Co-authored-by: Zhu Zhanyan --- core/pom.xml | 2 +- .../core/config/FeatureStreamConfig.java | 8 +- .../java/feast/core/dao/JobRepository.java | 18 +- .../java/feast/core/dao/SourceRepository.java | 5 +- .../main/java/feast/core/job/JobManager.java | 10 +- .../java/feast/core/job/JobUpdateTask.java | 90 +++- .../core/job/dataflow/DataflowJobManager.java | 18 +- .../job/direct/DirectRunnerJobManager.java | 22 +- .../java/feast/core/model/FeatureSet.java | 2 +- core/src/main/java/feast/core/model/Job.java | 60 ++- .../main/java/feast/core/model/Source.java | 198 +++---- .../src/main/java/feast/core/model/Store.java | 24 + .../core/service/JobCoordinatorService.java | 185 +++++-- .../java/feast/core/service/JobService.java | 38 +- .../test/java/feast/core/job/JobMatcher.java | 1 + .../feast/core/job/JobUpdateTaskTest.java | 88 +-- .../job/dataflow/DataflowJobManagerTest.java | 35 +- .../direct/DirectRunnerJobManagerTest.java | 41 +- .../service/JobCoordinatorServiceTest.java | 507 ++++++++++++------ .../feast/core/service/JobServiceTest.java | 63 +-- .../feast/core/service/SpecServiceTest.java | 60 +-- .../feast/core/service/TestObjectFactory.java | 89 --- .../java/feast/core/util/ModelHelpers.java | 41 -- .../test/java/feast/core/util/TestUtil.java | 147 +++++ pom.xml | 2 +- sdk/python/feast/cli.py | 2 +- sdk/python/feast/client.py | 4 +- sdk/python/feast/feature_set.py | 6 +- sdk/python/feast/loaders/ingest.py | 2 +- sdk/python/feast/type_map.py | 4 +- tests/e2e/redis/basic-ingest-redis-serving.py | 47 +- 31 files changed, 1148 insertions(+), 671 deletions(-) delete mode 100644 core/src/test/java/feast/core/service/TestObjectFactory.java delete mode 100644 core/src/test/java/feast/core/util/ModelHelpers.java create mode 100644 core/src/test/java/feast/core/util/TestUtil.java diff --git a/core/pom.xml b/core/pom.xml index 81dcb144de7..c4b3c0f4aa5 100644 --- a/core/pom.xml +++ b/core/pom.xml @@ -175,7 +175,7 @@ spring-kafka - + org.projectlombok lombok diff --git a/core/src/main/java/feast/core/config/FeatureStreamConfig.java b/core/src/main/java/feast/core/config/FeatureStreamConfig.java index 40647bb2e24..32445456048 100644 --- a/core/src/main/java/feast/core/config/FeatureStreamConfig.java +++ b/core/src/main/java/feast/core/config/FeatureStreamConfig.java @@ -21,6 +21,7 @@ import feast.core.util.KafkaSerialization; import feast.proto.core.FeatureSetProto; import feast.proto.core.IngestionJobProto; +import feast.proto.core.SourceProto; import feast.proto.core.SourceProto.KafkaSourceConfig; import feast.proto.core.SourceProto.SourceType; import java.util.HashMap; @@ -145,7 +146,12 @@ public Source getDefaultSource(FeastProperties feastProperties) { .setBootstrapServers(bootstrapServers) .setTopic(topicName) .build(); - return new Source(featureStreamType, sourceConfig, true); + SourceProto.Source source = + SourceProto.Source.newBuilder() + .setType(featureStreamType) + .setKafkaSourceConfig(sourceConfig) + .build(); + return Source.fromProto(source, true); default: throw new RuntimeException("Unsupported source stream, only [KAFKA] is supported"); } diff --git a/core/src/main/java/feast/core/dao/JobRepository.java b/core/src/main/java/feast/core/dao/JobRepository.java index 58af8c7ec5e..d2849a8eda5 100644 --- a/core/src/main/java/feast/core/dao/JobRepository.java +++ b/core/src/main/java/feast/core/dao/JobRepository.java @@ -18,18 +18,28 @@ import feast.core.model.FeatureSetJobStatus; import feast.core.model.Job; +import feast.core.model.JobStatus; +import feast.proto.core.SourceProto; +import java.util.Collection; import java.util.List; +import java.util.Optional; import org.springframework.data.jpa.repository.JpaRepository; import org.springframework.stereotype.Repository; /** JPA repository supplying Job objects keyed by ID. */ @Repository public interface JobRepository extends JpaRepository { - List findBySourceIdAndStoreNameOrderByLastUpdatedDesc(String sourceId, String storeName); + Optional + findFirstBySourceTypeAndSourceConfigAndStoreNameAndStatusNotInOrderByLastUpdatedDesc( + SourceProto.SourceType sourceType, + String sourceConfig, + String storeName, + Collection statuses); - // find jobs by feast store name - List findByStoreName(String storeName); + List findByStatus(JobStatus status); - // find jobs by featureset List findByFeatureSetJobStatusesIn(List featureSetsJobStatuses); + + // find jobs by feast store name + List findByStoreName(String storeName); } diff --git a/core/src/main/java/feast/core/dao/SourceRepository.java b/core/src/main/java/feast/core/dao/SourceRepository.java index 09214a9b684..1cf02dbcf82 100644 --- a/core/src/main/java/feast/core/dao/SourceRepository.java +++ b/core/src/main/java/feast/core/dao/SourceRepository.java @@ -17,7 +17,10 @@ package feast.core.dao; import feast.core.model.Source; +import feast.proto.core.SourceProto.SourceType; import org.springframework.data.jpa.repository.JpaRepository; /** JPA repository supplying Source objects keyed by id. */ -public interface SourceRepository extends JpaRepository {} +public interface SourceRepository extends JpaRepository { + Source findFirstByTypeAndConfigOrderByIdAsc(SourceType type, String config); +} diff --git a/core/src/main/java/feast/core/job/JobManager.java b/core/src/main/java/feast/core/job/JobManager.java index eda211b7574..20b5a861084 100644 --- a/core/src/main/java/feast/core/job/JobManager.java +++ b/core/src/main/java/feast/core/job/JobManager.java @@ -29,7 +29,7 @@ public interface JobManager { Runner getRunnerType(); /** - * Start an import job. + * Start an import job. Start should change the status of the Job from PENDING to RUNNING. * * @param job job to start * @return Job @@ -45,11 +45,13 @@ public interface JobManager { Job updateJob(Job job); /** - * Abort a job given runner-specific job ID. + * Abort a job given runner-specific job ID. Abort should change the status of the Job from + * RUNNING to ABORTING. * - * @param extId runner specific job id. + * @param job to abort. + * @return The aborted Job */ - void abortJob(String extId); + Job abortJob(Job job); /** * Restart an job. If job is an terminated state, will simply start the job. Might cause data to diff --git a/core/src/main/java/feast/core/job/JobUpdateTask.java b/core/src/main/java/feast/core/job/JobUpdateTask.java index 9a2e5e6c25e..b5ab782634f 100644 --- a/core/src/main/java/feast/core/job/JobUpdateTask.java +++ b/core/src/main/java/feast/core/job/JobUpdateTask.java @@ -23,8 +23,10 @@ import feast.core.model.*; import feast.proto.core.FeatureSetProto; import java.time.Instant; +import java.util.HashSet; import java.util.List; import java.util.Map; +import java.util.Objects; import java.util.Optional; import java.util.concurrent.Callable; import java.util.concurrent.ExecutionException; @@ -36,11 +38,10 @@ import java.util.stream.Collectors; import lombok.Getter; import lombok.extern.slf4j.Slf4j; -import org.apache.beam.vendor.guava.v26_0_jre.com.google.common.hash.Hashing; /** * JobUpdateTask is a callable that starts or updates a job given a set of featureSetSpecs, as well - * as their source and sink. + * their source and sink to transition to targetStatus. * *

When complete, the JobUpdateTask returns the updated Job object to be pushed to the db. */ @@ -48,9 +49,19 @@ @Getter public class JobUpdateTask implements Callable { + /** + * JobTargetStatus enum defines the possible target statuses that JobUpdateTask can transition a + * Job to. + */ + public enum JobTargetStatus { + RUNNING, + ABORTED + } + private final List featureSets; private final Source source; private final Store store; + private final JobTargetStatus targetStatus; private final Optional currentJob; private final JobManager jobManager; private final long jobUpdateTimeoutSeconds; @@ -62,8 +73,8 @@ public JobUpdateTask( Store store, Optional currentJob, JobManager jobManager, - long jobUpdateTimeoutSeconds) { - + long jobUpdateTimeoutSeconds, + JobTargetStatus targetStatus) { this.featureSets = featureSets; this.source = source; this.store = store; @@ -71,6 +82,7 @@ public JobUpdateTask( this.jobManager = jobManager; this.jobUpdateTimeoutSeconds = jobUpdateTimeoutSeconds; this.runnerName = jobManager.getRunnerType().toString(); + this.targetStatus = targetStatus; } @Override @@ -78,22 +90,27 @@ public Job call() { ExecutorService executorService = Executors.newSingleThreadExecutor(); Future submittedJob; - if (currentJob.isEmpty()) { + if (this.targetStatus.equals(JobTargetStatus.RUNNING) && currentJob.isEmpty()) { submittedJob = executorService.submit(this::createJob); + } else if (this.targetStatus.equals(JobTargetStatus.RUNNING) + && currentJob.isPresent() + && requiresUpdate(currentJob.get())) { + submittedJob = executorService.submit(() -> updateJob(currentJob.get())); + } else if (this.targetStatus.equals(JobTargetStatus.ABORTED) + && currentJob.isPresent() + && currentJob.get().getStatus() == JobStatus.RUNNING) { + submittedJob = executorService.submit(() -> stopJob(currentJob.get())); + } else if (this.targetStatus.equals(JobTargetStatus.ABORTED) && currentJob.isEmpty()) { + throw new IllegalArgumentException("Cannot abort an nonexistent ingestion job."); } else { - Job job = currentJob.get(); - - if (requiresUpdate(job)) { - submittedJob = executorService.submit(() -> updateJob(job)); - } else { - return updateStatus(job); - } + return this.updateStatus(currentJob.get()); } try { return submittedJob.get(getJobUpdateTimeoutSeconds(), TimeUnit.SECONDS); } catch (InterruptedException | ExecutionException | TimeoutException e) { - log.warn("Unable to start job for source {} and sink {}: {}", source, store, e.getMessage()); + log.warn("Unable to start job for source {} and sink {}:", source, store); + e.printStackTrace(); return null; } finally { executorService.shutdownNow(); @@ -111,24 +128,38 @@ boolean requiresUpdate(Job job) { } private Job createJob() { - String jobId = createJobId(source.getId(), store.getName()); + String jobId = createJobId(source, store.getName()); return startJob(jobId); } /** Start or update the job to ingest data to the sink. */ private Job startJob(String jobId) { - Job job = new Job(); - job.setId(jobId); - job.setRunner(jobManager.getRunnerType()); - job.setSource(source); - job.setStore(store); - job.setStatus(JobStatus.PENDING); + Job job = + Job.builder() + .setId(jobId) + .setRunner(jobManager.getRunnerType()) + .setSource(source) + .setStore(store) + .setStatus(JobStatus.PENDING) + .setFeatureSetJobStatuses(new HashSet<>()) + .build(); updateFeatureSets(job); try { logAudit(Action.SUBMIT, job, "Building graph and submitting to %s", runnerName); + System.out.println( + job.equals( + Job.builder() + .setId("job") + .setExtId("") + .setRunner(Runner.DATAFLOW) + .setSource(source) + .setStore(store) + .setStatus(JobStatus.PENDING) + .build())); + job = jobManager.startJob(job); var extId = job.getExtId(); if (extId.isEmpty()) { @@ -182,6 +213,12 @@ private Job updateJob(Job job) { return jobManager.updateJob(job); } + /** Stop the given job */ + private Job stopJob(Job job) { + logAudit(Action.ABORT, job, "Aborting job %s for runner %s", job.getId(), runnerName); + return jobManager.abortJob(job); + } + private Job updateStatus(Job job) { JobStatus currentStatus = job.getStatus(); JobStatus newStatus = jobManager.getJobStatus(job); @@ -195,14 +232,13 @@ private Job updateStatus(Job job) { return job; } - String createJobId(String sourceId, String storeName) { + String createJobId(Source source, String storeName) { String dateSuffix = String.valueOf(Instant.now().toEpochMilli()); - String[] sourceParts = sourceId.split("/", 2); - String sourceType = sourceParts[0].toLowerCase(); - String sourceHash = - Hashing.murmur3_128().hashUnencodedChars(sourceParts[1]).toString().substring(0, 10); - String jobId = String.format("%s-%s-to-%s-%s", sourceType, sourceHash, storeName, dateSuffix); - return jobId.replaceAll("_", "-"); + String jobId = + String.format( + "%s-%d-to-%s-%s", + source.getTypeString(), Objects.hashCode(source.getConfig()), storeName, dateSuffix); + return jobId.replaceAll("_store", "-"); } private void logAudit(Action action, Job job, String detail, Object... args) { diff --git a/core/src/main/java/feast/core/job/dataflow/DataflowJobManager.java b/core/src/main/java/feast/core/job/dataflow/DataflowJobManager.java index 181b2339b3a..8b1c03d546e 100644 --- a/core/src/main/java/feast/core/job/dataflow/DataflowJobManager.java +++ b/core/src/main/java/feast/core/job/dataflow/DataflowJobManager.java @@ -116,6 +116,7 @@ public Job startJob(Job job) { submitDataflowJob( job.getId(), job.getSource().toProto(), job.getStore().toProto(), false); job.setExtId(extId); + job.setStatus(JobStatus.RUNNING); return job; } catch (InvalidProtocolBufferException e) { @@ -136,25 +137,27 @@ public Job startJob(Job job) { */ @Override public Job updateJob(Job job) { - abortJob(job.getExtId()); + abortJob(job); return job; } /** * Abort an existing Dataflow job. Streaming Dataflow jobs are always drained, not cancelled. * - * @param dataflowJobId Dataflow-specific job id (not the job name) + * @param job to abort. + * @return The aborted Job. */ @Override - public void abortJob(String dataflowJobId) { + public Job abortJob(Job job) { + String dataflowJobId = job.getExtId(); try { - com.google.api.services.dataflow.model.Job job = + com.google.api.services.dataflow.model.Job dataflowJob = dataflow.projects().locations().jobs().get(projectId, location, dataflowJobId).execute(); com.google.api.services.dataflow.model.Job content = new com.google.api.services.dataflow.model.Job(); - if (job.getType().equals(DataflowJobType.JOB_TYPE_BATCH.toString())) { + if (dataflowJob.getType().equals(DataflowJobType.JOB_TYPE_BATCH.toString())) { content.setRequestedState(DataflowJobState.JOB_STATE_CANCELLED.toString()); - } else if (job.getType().equals(DataflowJobType.JOB_TYPE_STREAMING.toString())) { + } else if (dataflowJob.getType().equals(DataflowJobType.JOB_TYPE_STREAMING.toString())) { content.setRequestedState(DataflowJobState.JOB_STATE_DRAINING.toString()); } dataflow @@ -168,6 +171,9 @@ public void abortJob(String dataflowJobId) { throw new RuntimeException( Strings.lenientFormat("Unable to drain job with id: %s", dataflowJobId), e); } + + job.setStatus(JobStatus.ABORTING); + return job; } /** diff --git a/core/src/main/java/feast/core/job/direct/DirectRunnerJobManager.java b/core/src/main/java/feast/core/job/direct/DirectRunnerJobManager.java index 242ac43496e..b8b909c02a5 100644 --- a/core/src/main/java/feast/core/job/direct/DirectRunnerJobManager.java +++ b/core/src/main/java/feast/core/job/direct/DirectRunnerJobManager.java @@ -121,30 +121,32 @@ private ImportOptions getPipelineOptions( */ @Override public Job updateJob(Job job) { - String jobId = job.getExtId(); - abortJob(jobId); try { - return startJob(job); + return startJob(abortJob(job)); } catch (JobExecutionException e) { throw new JobExecutionException(String.format("Error running ingestion job: %s", e), e); } } /** - * Abort the direct runner job with the given id, then remove it from the direct jobs registry. + * Abort the direct runner job,removing it from the direct jobs registry. * - * @param extId runner specific job id. + * @param job to abort. + * @return The aborted Job */ @Override - public void abortJob(String extId) { - DirectJob job = jobs.get(extId); + public Job abortJob(Job job) { + DirectJob directJob = jobs.get(job.getExtId()); try { - job.abort(); + directJob.abort(); } catch (IOException e) { throw new RuntimeException( - Strings.lenientFormat("Unable to abort DirectRunner job %s", extId), e); + Strings.lenientFormat("Unable to abort DirectRunner job %s", job.getExtId(), e)); } - jobs.remove(extId); + jobs.remove(job.getExtId()); + + job.setStatus(JobStatus.ABORTING); + return job; } public PipelineResult runPipeline(ImportOptions pipelineOptions) throws IOException { diff --git a/core/src/main/java/feast/core/model/FeatureSet.java b/core/src/main/java/feast/core/model/FeatureSet.java index d1e8c5ee140..e032c712fce 100644 --- a/core/src/main/java/feast/core/model/FeatureSet.java +++ b/core/src/main/java/feast/core/model/FeatureSet.java @@ -30,7 +30,7 @@ import lombok.Getter; import lombok.Setter; import org.apache.commons.lang3.builder.HashCodeBuilder; -import org.springframework.data.util.Pair; +import org.apache.commons.lang3.tuple.Pair; import org.tensorflow.metadata.v0.*; @Getter diff --git a/core/src/main/java/feast/core/model/Job.java b/core/src/main/java/feast/core/model/Job.java index ae54221b9c6..de2508d671b 100644 --- a/core/src/main/java/feast/core/model/Job.java +++ b/core/src/main/java/feast/core/model/Job.java @@ -20,23 +20,37 @@ import feast.core.job.Runner; import feast.proto.core.FeatureSetProto; import feast.proto.core.IngestionJobProto; +import feast.proto.core.SourceProto.SourceType; import java.util.ArrayList; import java.util.HashSet; import java.util.List; +import java.util.Objects; import java.util.Set; +import java.util.stream.Collectors; import javax.persistence.*; import javax.persistence.Entity; +import lombok.AccessLevel; import lombok.AllArgsConstructor; +import lombok.Builder; import lombok.Getter; import lombok.Setter; /** Contains information about a run job. */ -@AllArgsConstructor @Getter @Setter @Entity @Table(name = "jobs") +@Builder(setterPrefix = "set", toBuilder = true) +@AllArgsConstructor(access = AccessLevel.PRIVATE) public class Job extends AbstractTimestampEntity { + public static class JobBuilder { + public JobBuilder setSource(Source source) { + // map source fields into equavilent inline job fields. + this.sourceType = source.getType(); + this.sourceConfig = source.getConfig(); + return this; + } + } // Internal job name. Generated by feast ingestion upon invocation. @Id private String id; @@ -51,10 +65,12 @@ public class Job extends AbstractTimestampEntity { @Column(name = "runner") private Runner runner; - // Source id - @ManyToOne - @JoinColumn(name = "source_id") - private Source source; + // Source type and config, derieved from job's source and stored as inline fields. + @Column(name = "source_type") + private SourceType sourceType; + + @Column(name = "source_config") + private String sourceConfig; // Sink id @ManyToOne @@ -85,6 +101,19 @@ public String getSinkName() { return store.getName(); } + public List getFeatureSets() { + return this.featureSetJobStatuses.stream() + .map(FeatureSetJobStatus::getFeatureSet) + .collect(Collectors.toList()); + } + + public Source getSource() { + Source source = new Source(); + source.setType(this.sourceType); + source.setConfig(this.sourceConfig); + return source; + } + /** * Convert a job model to ingestion job proto * @@ -111,4 +140,25 @@ public IngestionJobProto.IngestionJob toProto() throws InvalidProtocolBufferExce return ingestJob; } + + @Override + public int hashCode() { + return Objects.hash(getSource(), this.store, this.runner); + } + + @Override + public boolean equals(Object obj) { + if (this == obj) return true; + if (!super.equals(obj)) return false; + if (getClass() != obj.getClass()) return false; + Job other = (Job) obj; + if (!runner.equals(other.runner)) { + return false; + } else if (!getSource().equals(other.getSource())) { + return false; + } else if (!store.equals(other.store)) { + return false; + } + return true; + } } diff --git a/core/src/main/java/feast/core/model/Source.java b/core/src/main/java/feast/core/model/Source.java index d199a1b676a..0b9f2a3e4c7 100644 --- a/core/src/main/java/feast/core/model/Source.java +++ b/core/src/main/java/feast/core/model/Source.java @@ -16,43 +16,37 @@ */ package feast.core.model; -import com.google.common.collect.Sets; -import com.google.protobuf.Message; +import com.google.protobuf.TextFormat; import feast.proto.core.SourceProto; import feast.proto.core.SourceProto.KafkaSourceConfig; import feast.proto.core.SourceProto.Source.Builder; import feast.proto.core.SourceProto.SourceType; import io.grpc.Status; import java.util.Objects; -import java.util.Set; -import javax.persistence.Column; +import javax.persistence.*; import javax.persistence.Entity; -import javax.persistence.Id; -import javax.persistence.Table; +import lombok.AllArgsConstructor; +import lombok.Getter; import lombok.Setter; -@Setter @Entity +@Getter +@Setter +@AllArgsConstructor @Table(name = "sources") public class Source { - private static final Set KAFKA_OPTIONS = Sets.newHashSet("bootstrapServers"); + /** Source Id. Internal use only, do not use to identify the source. */ + @Id @GeneratedValue private long id; - @Id - @Column(name = "id", updatable = false, nullable = false) - private String id; - - // Type of the source. Should map to feast.types.Source.SourceType + /** Type of the source */ + @Enumerated(EnumType.STRING) @Column(name = "type", nullable = false) - private String type; - - // Bootstrap servers, comma delimited. Used by kafka sources. - @Column(name = "bootstrap_servers") - private String bootstrapServers; + private SourceType type; - // Topics to listen to, comma delimited. Used by kafka sources. - @Column(name = "topics") - private String topics; + /** Configuration object specific to each source type */ + @Column(name = "config") + private String config; @Column(name = "is_default") private boolean isDefault; @@ -61,144 +55,101 @@ public Source() { super(); } - public Source(SourceType type, KafkaSourceConfig config, boolean isDefault) { - if (config.getBootstrapServers().isEmpty() || config.getTopic().isEmpty()) { - throw Status.INVALID_ARGUMENT - .withDescription( - "Unsupported source options. Kafka source requires bootstrap servers and topic to be specified.") - .asRuntimeException(); - } - this.type = type.toString(); - this.bootstrapServers = config.getBootstrapServers(); - this.topics = config.getTopic(); - this.isDefault = isDefault; - this.id = generateId(); - } - /** * Construct a source facade object from a given proto object. * - * @param sourceProto SourceProto.Source object + * @param sourceSpec SourceProto.Source object + * @param isDefault Whether to return the default source object if the source was not defined by + * the user * @return Source facade object */ - public static Source fromProto(SourceProto.Source sourceProto) { - if (sourceProto.equals(SourceProto.Source.getDefaultInstance())) { + public static Source fromProto(SourceProto.Source sourceSpec, boolean isDefault) { + + if (sourceSpec.equals(SourceProto.Source.getDefaultInstance())) { Source source = new Source(); - source.isDefault = true; + source.setDefault(true); return source; } - switch (sourceProto.getType()) { + Source source = new Source(); + source.setType(sourceSpec.getType()); + + switch (sourceSpec.getType()) { case KAFKA: - return new Source(sourceProto.getType(), sourceProto.getKafkaSourceConfig(), false); + if (sourceSpec.getKafkaSourceConfig().getBootstrapServers().isEmpty() + || sourceSpec.getKafkaSourceConfig().getTopic().isEmpty()) { + throw Status.INVALID_ARGUMENT + .withDescription( + "Unsupported source options. Kafka source requires bootstrap servers and topic to be specified.") + .asRuntimeException(); + } + source.setConfig(sourceSpec.getKafkaSourceConfig().toString()); + break; case UNRECOGNIZED: default: throw Status.INVALID_ARGUMENT .withDescription("Unsupported source type. Only [KAFKA] is supported.") .asRuntimeException(); } - } - /** - * Convert this object to its equivalent proto object. - * - * @return SourceProto.Source - */ - public SourceProto.Source toProto() { - Builder builder = SourceProto.Source.newBuilder().setType(SourceType.valueOf(type)); - switch (SourceType.valueOf(type)) { - case KAFKA: - KafkaSourceConfig config = - KafkaSourceConfig.newBuilder() - .setBootstrapServers(bootstrapServers) - .setTopic(topics) - .build(); - return builder.setKafkaSourceConfig(config).build(); - case UNRECOGNIZED: - default: - throw new RuntimeException("Unable to convert source to proto"); - } + source.setDefault(isDefault); + return source; } /** - * Get the id for this feature source + * Construct a source facade object from a given proto object. * - * @return feature source id in the format TYPE/options + * @param sourceSpec SourceProto.Source object + * @return Source facade object */ - public String getId() { - return id; + public static Source fromProto(SourceProto.Source sourceSpec) { + return fromProto(sourceSpec, false); } /** - * Get the options for this feature source + * Convert this object to its equivalent proto object. * - * @return feature source options + * @return SourceProto.Source */ - public Message getOptions() { - switch (SourceType.valueOf(type)) { + public SourceProto.Source toProto() { + Builder builder = SourceProto.Source.newBuilder().setType(this.getType()); + + switch (this.getType()) { case KAFKA: - return KafkaSourceConfig.newBuilder() - .setBootstrapServers(bootstrapServers) - .setTopic(topics) - .build(); + KafkaSourceConfig.Builder kafkaSourceConfig = KafkaSourceConfig.newBuilder(); + try { + com.google.protobuf.TextFormat.getParser().merge(this.getConfig(), kafkaSourceConfig); + } catch (TextFormat.ParseException e) { + throw new RuntimeException( + String.format( + "Unable to deserialize source configuration from String to KafkaSourceConfig: %s", + this.getConfig()), + e); + } + return builder.setKafkaSourceConfig(kafkaSourceConfig).build(); + case INVALID: case UNRECOGNIZED: default: - throw new RuntimeException("Unable to convert source to proto"); + throw new RuntimeException( + String.format( + "Unable to build Source from configuration and type: %s %s", + this.getConfig(), this.getType())); } } /** - * Get the type of source. - * - * @return SourceType of this feature source - */ - public SourceType getType() { - return SourceType.valueOf(type); - } - - /** - * Indicate whether to use the system defaults or not. - * - * @return boolean indicating whether this feature set source uses defaults. - */ - public boolean isDefault() { - return isDefault; - } - - /** - * Override equality for sources. isDefault is always compared first; if both sources are using - * the default feature source, they will be equal. If not they will be compared based on their - * type-specific options. + * Override equality for sources. Sources are compared based on their type and type-specific + * options. * * @param other other Source * @return boolean equal */ public boolean equalTo(Source other) { - if (other.isDefault && isDefault || (type == null && other.type == null)) { - return true; - } - - if ((type == null || !type.equals(other.type))) { + if ((this.getType() == null || !this.getType().equals(other.getType()))) { return false; } - switch (SourceType.valueOf(type)) { - case KAFKA: - return bootstrapServers.equals(other.bootstrapServers) && topics.equals(other.topics); - case UNRECOGNIZED: - default: - return false; - } - } - - private String generateId() { - switch (SourceType.valueOf(type)) { - case KAFKA: - return String.format("KAFKA/%s/%s", bootstrapServers, topics); - default: - // should not occur - return ""; - } + return this.getConfig().equals(other.getConfig()); } @Override @@ -210,11 +161,20 @@ public boolean equals(Object o) { return false; } Source source = (Source) o; - return id.equals(source.id); + return this.equalTo(source); } @Override public int hashCode() { - return Objects.hash(id); + return Objects.hash(getType(), getConfig()); + } + + /** + * Returns the type of this Source in String format + * + * @return Source type in String format + */ + public String getTypeString() { + return this.getType().getValueDescriptor().getName(); } } diff --git a/core/src/main/java/feast/core/model/Store.java b/core/src/main/java/feast/core/model/Store.java index 1f7c373bdc5..55614330128 100644 --- a/core/src/main/java/feast/core/model/Store.java +++ b/core/src/main/java/feast/core/model/Store.java @@ -28,6 +28,7 @@ import java.util.ArrayList; import java.util.Arrays; import java.util.List; +import java.util.Objects; import java.util.stream.Collectors; import javax.persistence.Column; import javax.persistence.Entity; @@ -139,4 +140,27 @@ private Subscription convertStringToSubscription(String sub) { String[] split = sub.split(":", 2); return Subscription.newBuilder().setProject(split[0]).setName(split[1]).build(); } + + @Override + public int hashCode() { + return Objects.hash(this.name, this.type, this.subscriptions) ^ Arrays.hashCode(this.config); + } + + @Override + public boolean equals(Object obj) { + if (this == obj) return true; + if (getClass() != obj.getClass()) return false; + Store other = (Store) obj; + + if (!name.equals(other.name)) { + return false; + } else if (!type.equals(other.type)) { + return false; + } else if (!Arrays.equals(config, config)) { + return false; + } else if (!subscriptions.equals(other.subscriptions)) { + return false; + } + return true; + } } diff --git a/core/src/main/java/feast/core/service/JobCoordinatorService.java b/core/src/main/java/feast/core/service/JobCoordinatorService.java index 19d5d65b618..83ffe075c72 100644 --- a/core/src/main/java/feast/core/service/JobCoordinatorService.java +++ b/core/src/main/java/feast/core/service/JobCoordinatorService.java @@ -23,27 +23,32 @@ import feast.core.config.FeastProperties.JobProperties; import feast.core.dao.FeatureSetRepository; import feast.core.dao.JobRepository; +import feast.core.dao.SourceRepository; import feast.core.job.JobManager; import feast.core.job.JobUpdateTask; +import feast.core.job.JobUpdateTask.JobTargetStatus; import feast.core.model.*; +import feast.core.model.FeatureSet; +import feast.core.model.Job; +import feast.core.model.JobStatus; +import feast.core.model.Source; +import feast.core.model.Store; import feast.proto.core.CoreServiceProto.ListStoresRequest.Filter; import feast.proto.core.CoreServiceProto.ListStoresResponse; import feast.proto.core.FeatureSetProto; import feast.proto.core.IngestionJobProto; -import feast.proto.core.StoreProto; -import feast.proto.core.StoreProto.Store.Subscription; import java.util.ArrayList; -import java.util.HashSet; +import java.util.Collection; +import java.util.LinkedList; import java.util.List; +import java.util.Map; import java.util.Optional; -import java.util.Set; import java.util.concurrent.*; import java.util.stream.Collectors; -import javax.validation.constraints.Positive; import lombok.extern.slf4j.Slf4j; +import org.apache.commons.lang3.tuple.Pair; import org.apache.kafka.clients.consumer.ConsumerRecord; import org.springframework.beans.factory.annotation.Autowired; -import org.springframework.data.util.Pair; import org.springframework.kafka.annotation.KafkaListener; import org.springframework.kafka.core.KafkaTemplate; import org.springframework.scheduling.annotation.Scheduled; @@ -58,6 +63,7 @@ public class JobCoordinatorService { private final JobRepository jobRepository; private final FeatureSetRepository featureSetRepository; + private final SourceRepository sourceRepository; private final SpecService specService; private final JobManager jobManager; private final JobProperties jobProperties; @@ -67,12 +73,14 @@ public class JobCoordinatorService { public JobCoordinatorService( JobRepository jobRepository, FeatureSetRepository featureSetRepository, + SourceRepository sourceRepository, SpecService specService, JobManager jobManager, FeastProperties feastProperties, KafkaTemplate specPublisher) { this.jobRepository = jobRepository; this.featureSetRepository = featureSetRepository; + this.sourceRepository = sourceRepository; this.specService = specService; this.jobManager = jobManager; this.jobProperties = feastProperties.getJobs(); @@ -84,7 +92,7 @@ public JobCoordinatorService( * *

1) Checks DB and extracts jobs that have to run based on the specs available * - *

2) Does a diff with the current set of jobs, starts/updates job(s) if necessary + *

2) Does a diff with the current set of jobs, starts/updates/stops job(s) if necessary * *

3) Updates job object in DB with status, feature sets * @@ -94,32 +102,9 @@ public JobCoordinatorService( @Scheduled(fixedDelayString = "${feast.jobs.polling_interval_milliseconds}") public void Poll() throws InvalidProtocolBufferException { log.info("Polling for new jobs..."); - @Positive long updateTimeout = jobProperties.getJobUpdateTimeoutSeconds(); - List jobUpdateTasks = new ArrayList<>(); - ListStoresResponse listStoresResponse = specService.listStores(Filter.newBuilder().build()); - - for (StoreProto.Store storeSpec : listStoresResponse.getStoreList()) { - Set featureSets = new HashSet<>(); - Store store = Store.fromProto(storeSpec); - - for (Subscription subscription : store.getSubscriptions()) { - List featureSetsForSub = - featureSetRepository.findAllByNameLikeAndProject_NameLikeOrderByNameAsc( - subscription.getName().replace('*', '%'), - subscription.getProject().replace('*', '%')); - featureSets.addAll(featureSetsForSub); - } + List> sourceStoreMappings = getSourceToStoreMappings(); + List jobUpdateTasks = makeJobUpdateTasks(sourceStoreMappings); - featureSets.stream() - .collect(Collectors.groupingBy(FeatureSet::getSource)) - .forEach( - (source, setsForSource) -> { - Optional originalJob = getJob(source, store); - jobUpdateTasks.add( - new JobUpdateTask( - setsForSource, source, store, originalJob, jobManager, updateTimeout)); - }); - } if (jobUpdateTasks.isEmpty()) { log.info("No jobs found."); return; @@ -151,17 +136,133 @@ void startOrUpdateJobs(List tasks) { executorService.shutdown(); } + /** + * Makes Job Update Tasks required to reconcile the current ingestion jobs with the given source + * to store map. Compares the current ingestion jobs and source to store mapping to determine + * which whether jobs have started/stopped/updated in terms of Job Update tasks. Only tries to + * stop ingestion jobs its the required ingestions to maintained ingestion jobs are already + * RUNNING. + * + * @param sourceStoreMappings a list of source to store pairs where ingestion jobs would have to + * be maintained for ingestion to work correctly. + * @return list of job update tasks required to reconcile the current ingestion jobs to the state + * that is defined by sourceStoreMap. + */ + private List makeJobUpdateTasks(List> sourceStoreMappings) { + List jobUpdateTasks = new LinkedList<>(); + // Ensure a running job for each source to store mapping + long updateTimeout = jobProperties.getJobUpdateTimeoutSeconds(); + List requiredJobs = new LinkedList<>(); + boolean isSafeToStopJobs = true; + for (Pair mapping : sourceStoreMappings) { + Source source = mapping.getLeft(); + Store store = mapping.getRight(); + + Optional sourceToStoreJob = getJob(source, store); + if (sourceToStoreJob.isPresent()) { + Job job = sourceToStoreJob.get(); + // Record the job as required to safeguard it from getting stopped + requiredJobs.add(sourceToStoreJob.get()); + if (!job.isRunning()) { + // Mark that it is not safe to stop jobs without disrupting ingestion + isSafeToStopJobs = false; + } + } + + jobUpdateTasks.add( + new JobUpdateTask( + getFeatureSetsForStore(store), + source, + store, + sourceToStoreJob, + jobManager, + updateTimeout, + JobTargetStatus.RUNNING)); + } + // Stop extra jobs that are not required to mantain ingestion when safe + if (isSafeToStopJobs) { + getExtraJobs(requiredJobs) + .forEach( + extraJob -> { + jobUpdateTasks.add( + new JobUpdateTask( + extraJob.getFeatureSets(), + extraJob.getSource(), + extraJob.getStore(), + Optional.of(extraJob), + jobManager, + updateTimeout, + JobTargetStatus.ABORTED)); + }); + } + + return jobUpdateTasks; + } + + /** Get the non terminated ingestion job ingesting between source and source if exists. */ @Transactional public Optional getJob(Source source, Store store) { - List jobs = - jobRepository.findBySourceIdAndStoreNameOrderByLastUpdatedDesc( - source.getId(), store.getName()); - jobs = jobs.stream().filter(job -> !job.hasTerminated()).collect(Collectors.toList()); - if (jobs.isEmpty()) { - return Optional.empty(); - } - // return the latest - return Optional.of(jobs.get(0)); + return jobRepository + .findFirstBySourceTypeAndSourceConfigAndStoreNameAndStatusNotInOrderByLastUpdatedDesc( + source.getType(), source.getConfig(), store.getName(), JobStatus.getTerminalStates()); + } + + /** Get running extra ingestion jobs that have ids not in keepJobs */ + @Transactional + private Collection getExtraJobs(List keepJobs) { + List runningJobs = jobRepository.findByStatus(JobStatus.RUNNING); + Map extraJobMap = + runningJobs.stream().collect(Collectors.toMap(job -> job.getId(), job -> job)); + keepJobs.forEach(job -> extraJobMap.remove(job.getId())); + return extraJobMap.values(); + } + + /** + * Generate a source to store mapping. The mapping maps the source/store pairs in which ingestion + * jobs would have to be maintained for ingestion to work correctly. + * + * @return a list of pairs signifying a mapping from source to store. + */ + private List> getSourceToStoreMappings() { + ListStoresResponse listStoresResponse = specService.listStores(Filter.newBuilder().build()); + List stores = + listStoresResponse.getStoreList().stream() + .map(storeSpec -> Store.fromProto(storeSpec)) + .collect(Collectors.toList()); + + // build mapping from source to store. + // compile a set of sources via subscribed FeatureSets of stores. + List> sourceStoreMappings = + stores.stream() + .flatMap( + store -> + getFeatureSetsForStore(store).stream() + .map(featureSet -> featureSet.getSource()) + .map(source -> Pair.of(source, store))) + .distinct() + .collect(Collectors.toList()); + + return sourceStoreMappings; + } + + /** + * Get the FeatureSets that the given store subscribes to. + * + * @param store to get subscribed FeatureSets for + * @return list of FeatureSets that the store subscribes to. + */ + @Transactional + private List getFeatureSetsForStore(Store store) { + return store.getSubscriptions().stream() + .flatMap( + subscription -> + featureSetRepository + .findAllByNameLikeAndProject_NameLikeOrderByNameAsc( + subscription.getName().replace('*', '%'), + subscription.getProject().replace('*', '%')) + .stream()) + .distinct() + .collect(Collectors.toList()); } @Transactional @@ -238,7 +339,7 @@ public void listenAckFromJobs( Pair projectAndSetName = parseReference(setReference); FeatureSet featureSet = featureSetRepository.findFeatureSetByNameAndProject_Name( - projectAndSetName.getSecond(), projectAndSetName.getFirst()); + projectAndSetName.getRight(), projectAndSetName.getLeft()); if (featureSet == null) { log.warn( String.format("ACKListener received message for unknown FeatureSet %s", setReference)); diff --git a/core/src/main/java/feast/core/service/JobService.java b/core/src/main/java/feast/core/service/JobService.java index a671a0f68ea..b300959cd95 100644 --- a/core/src/main/java/feast/core/service/JobService.java +++ b/core/src/main/java/feast/core/service/JobService.java @@ -188,8 +188,8 @@ public RestartIngestionJobResponse restartJob(RestartIngestionJobRequest request String.format( "Restarted job (id: %s, extId: %s runner: %s)", job.getId(), job.getExtId(), job.getRunner())); - // sync job status & update job model in job repository - job = this.syncJobStatus(jobManager, job); + this.logStatusChange(job, status, job.getStatus()); + // update job model in job repository this.jobRepository.saveAndFlush(job); return RestartIngestionJobResponse.newBuilder().build(); @@ -224,17 +224,17 @@ public StopIngestionJobResponse stopJob(StopIngestionJobRequest request) { throw new UnsupportedOperationException( "Stopping a job with a transitional or unknown status is unsupported"); } + this.logStatusChange(job, status, job.getStatus()); // stop job with job manager JobManager jobManager = this.jobManagers.get(job.getRunner()); - jobManager.abortJob(job.getExtId()); + job = jobManager.abortJob(job); log.info( String.format( - "Restarted job (id: %s, extId: %s runner: %s)", + "Aborted job (id: %s, extId: %s runner: %s)", job.getId(), job.getExtId(), job.getRunner())); - - // sync job status & update job model in job repository - job = this.syncJobStatus(jobManager, job); + this.logStatusChange(job, status, job.getStatus()); + // update job model in job repository this.jobRepository.saveAndFlush(job); return StopIngestionJobResponse.newBuilder().build(); @@ -270,20 +270,14 @@ private ListFeatureSetsRequest.Filter toListFeatureSetFilter(FeatureSetReference .build(); } - /** sync job status using job manager */ - private Job syncJobStatus(JobManager jobManager, Job job) { - JobStatus newStatus = jobManager.getJobStatus(job); - // log job status transition - if (newStatus != job.getStatus()) { - AuditLogger.log( - Resource.JOB, - job.getId(), - Action.STATUS_CHANGE, - "Job status transition: changed from %s to %s", - job.getStatus(), - newStatus); - job.setStatus(newStatus); - } - return job; + /** log job status using job manager */ + private void logStatusChange(Job job, JobStatus oldStatus, JobStatus newStatus) { + AuditLogger.log( + Resource.JOB, + job.getId(), + Action.STATUS_CHANGE, + "Job status transition: changed from %s to %s", + oldStatus, + newStatus); } } diff --git a/core/src/test/java/feast/core/job/JobMatcher.java b/core/src/test/java/feast/core/job/JobMatcher.java index 87be05668fa..9c897248578 100644 --- a/core/src/test/java/feast/core/job/JobMatcher.java +++ b/core/src/test/java/feast/core/job/JobMatcher.java @@ -19,6 +19,7 @@ import feast.core.model.Job; import org.mockito.ArgumentMatcher; +/** Custom Job ArgumentMatcher that disregards the Job id when matching jobs */ public class JobMatcher implements ArgumentMatcher { private Job left; diff --git a/core/src/test/java/feast/core/job/JobUpdateTaskTest.java b/core/src/test/java/feast/core/job/JobUpdateTaskTest.java index 16e36274596..a9f19ba6d51 100644 --- a/core/src/test/java/feast/core/job/JobUpdateTaskTest.java +++ b/core/src/test/java/feast/core/job/JobUpdateTaskTest.java @@ -23,11 +23,14 @@ import static org.junit.Assert.assertThat; import static org.mockito.Mockito.doReturn; import static org.mockito.Mockito.spy; +import static org.mockito.Mockito.times; +import static org.mockito.Mockito.verify; import static org.mockito.Mockito.when; import static org.mockito.MockitoAnnotations.initMocks; +import feast.core.job.JobUpdateTask.JobTargetStatus; import feast.core.model.*; -import feast.core.util.ModelHelpers; +import feast.core.util.TestUtil; import feast.proto.core.FeatureSetProto; import feast.proto.core.FeatureSetProto.FeatureSetMeta; import feast.proto.core.FeatureSetProto.FeatureSetSpec; @@ -96,18 +99,21 @@ public void setUp() { } Job makeJob(String extId, List featureSets, JobStatus status) { - return new Job( - "job", - extId, - RUNNER, - source, - store, - ModelHelpers.makeFeatureSetJobStatus(featureSets), - status); + return Job.builder() + .setId("job") + .setExtId(extId) + .setRunner(RUNNER) + .setSource(source) + .setStore(store) + .setFeatureSetJobStatuses(TestUtil.makeFeatureSetJobStatus(featureSets)) + .setStatus(status) + .build(); } - JobUpdateTask makeTask(List featureSets, Optional currentJob) { - return new JobUpdateTask(featureSets, source, store, currentJob, jobManager, 100L); + JobUpdateTask makeTask( + List featureSets, Optional currentJob, JobTargetStatus targetStatus) { + return new JobUpdateTask( + featureSets, source, store, currentJob, jobManager, 100L, targetStatus); } @Test @@ -118,7 +124,8 @@ public void shouldUpdateJobIfPresent() { List newFeatureSetsPopulatedByJob = Arrays.asList(featureSet1, featureSet2); Job originalJob = makeJob("old_ext", existingFeatureSetsPopulatedByJob, JobStatus.RUNNING); - JobUpdateTask jobUpdateTask = makeTask(newFeatureSetsPopulatedByJob, Optional.of(originalJob)); + JobUpdateTask jobUpdateTask = + makeTask(newFeatureSetsPopulatedByJob, Optional.of(originalJob), JobTargetStatus.RUNNING); Job submittedJob = makeJob("old_ext", newFeatureSetsPopulatedByJob, JobStatus.RUNNING); Job expected = makeJob("new_ext", newFeatureSetsPopulatedByJob, JobStatus.PENDING); @@ -131,11 +138,12 @@ public void shouldUpdateJobIfPresent() { @Test public void shouldCreateJobIfNotPresent() { var featureSets = Collections.singletonList(featureSet1); - JobUpdateTask jobUpdateTask = spy(makeTask(featureSets, Optional.empty())); - doReturn("job").when(jobUpdateTask).createJobId("KAFKA/servers:9092/topic", "test"); + JobUpdateTask jobUpdateTask = + spy(makeTask(featureSets, Optional.empty(), JobTargetStatus.RUNNING)); + doReturn("job").when(jobUpdateTask).createJobId(source, "test"); Job expectedInput = makeJob("", featureSets, JobStatus.PENDING); - Job expected = makeJob("ext", featureSets, JobStatus.PENDING); + Job expected = makeJob("ext", featureSets, JobStatus.RUNNING); when(jobManager.startJob(expectedInput)).thenReturn(expected); @@ -147,7 +155,8 @@ public void shouldCreateJobIfNotPresent() { public void shouldUpdateJobStatusIfNotCreateOrUpdate() { var featureSets = Collections.singletonList(featureSet1); Job originalJob = makeJob("ext", featureSets, JobStatus.RUNNING); - JobUpdateTask jobUpdateTask = makeTask(featureSets, Optional.of(originalJob)); + JobUpdateTask jobUpdateTask = + makeTask(featureSets, Optional.of(originalJob), JobTargetStatus.RUNNING); when(jobManager.getJobStatus(originalJob)).thenReturn(JobStatus.ABORTING); Job updated = jobUpdateTask.call(); @@ -158,8 +167,9 @@ public void shouldUpdateJobStatusIfNotCreateOrUpdate() { @Test public void shouldReturnJobWithErrorStatusIfFailedToSubmit() { var featureSets = Collections.singletonList(featureSet1); - JobUpdateTask jobUpdateTask = spy(makeTask(featureSets, Optional.empty())); - doReturn("job").when(jobUpdateTask).createJobId("KAFKA/servers:9092/topic", "test"); + JobUpdateTask jobUpdateTask = + spy(makeTask(featureSets, Optional.empty(), JobTargetStatus.RUNNING)); + doReturn("job").when(jobUpdateTask).createJobId(source, "test"); Job expectedInput = makeJob("", featureSets, JobStatus.PENDING); Job expected = makeJob("", featureSets, JobStatus.ERROR); @@ -178,7 +188,13 @@ public void shouldTimeout() { JobUpdateTask jobUpdateTask = spy( new JobUpdateTask( - featureSets, source, store, Optional.empty(), jobManager, timeoutSeconds)); + featureSets, + source, + store, + Optional.empty(), + jobManager, + timeoutSeconds, + JobTargetStatus.RUNNING)); Job actual = jobUpdateTask.call(); assertThat(actual, is(IsNull.nullValue())); @@ -191,13 +207,7 @@ public void featureSetsShouldBeUpdated() { when(jobManager.getJobStatus(job)).thenReturn(JobStatus.RUNNING); JobUpdateTask jobUpdateTask = - new JobUpdateTask( - Collections.singletonList(featureSet1), - source, - store, - Optional.of(job), - jobManager, - 0L); + makeTask(List.of(featureSet1), Optional.of(job), JobTargetStatus.RUNNING); jobUpdateTask.call(); @@ -214,14 +224,7 @@ public void featureSetsShouldBeUpdated() { job.getFeatureSetJobStatuses().forEach(j -> j.setDeliveryStatus(STATUS_DELIVERED)); JobUpdateTask jobUpdateTask2 = - new JobUpdateTask( - Arrays.asList(featureSet1, featureSet2), - source, - store, - Optional.of(job), - jobManager, - 0L); - + makeTask(List.of(featureSet1, featureSet2), Optional.of(job), JobTargetStatus.RUNNING); jobUpdateTask2.call(); FeatureSetJobStatus expectedStatus2 = new FeatureSetJobStatus(); @@ -234,4 +237,21 @@ public void featureSetsShouldBeUpdated() { assertThat( job.getFeatureSetJobStatuses(), containsInAnyOrder(expectedStatus1, expectedStatus2)); } + + @Test + public void shouldStopJobIfTargetStatusIsAbort() { + var featureSets = Collections.singletonList(featureSet1); + Job originalJob = makeJob("ext", featureSets, JobStatus.RUNNING); + JobUpdateTask jobUpdateTask = + makeTask(featureSets, Optional.of(originalJob), JobTargetStatus.ABORTED); + + Job expected = makeJob("ext", featureSets, JobStatus.ABORTING); + + when(jobManager.getJobStatus(originalJob)).thenReturn(JobStatus.ABORTING); + when(jobManager.abortJob(originalJob)).thenReturn(expected); + + Job actual = jobUpdateTask.call(); + verify(jobManager, times(1)).abortJob(originalJob); + assertThat(actual, equalTo(expected)); + } } diff --git a/core/src/test/java/feast/core/job/dataflow/DataflowJobManagerTest.java b/core/src/test/java/feast/core/job/dataflow/DataflowJobManagerTest.java index 50812882029..5f8403f7979 100644 --- a/core/src/test/java/feast/core/job/dataflow/DataflowJobManagerTest.java +++ b/core/src/test/java/feast/core/job/dataflow/DataflowJobManagerTest.java @@ -164,14 +164,15 @@ public void shouldStartJobWithCorrectPipelineOptions() throws IOException { featureSetJobStatus.setFeatureSet(FeatureSet.fromProto(featureSet)); Job job = - new Job( - jobName, - "", - Runner.DATAFLOW, - Source.fromProto(source), - Store.fromProto(store), - Sets.newHashSet(featureSetJobStatus), - JobStatus.PENDING); + Job.builder() + .setId(jobName) + .setExtId("") + .setRunner(Runner.DATAFLOW) + .setSource(Source.fromProto(source)) + .setStore(Store.fromProto(store)) + .setFeatureSetJobStatuses(Sets.newHashSet(featureSetJobStatus)) + .setStatus(JobStatus.PENDING) + .build(); Job actual = dfJobManager.startJob(job); verify(dfJobManager, times(1)).runPipeline(captor.capture()); @@ -208,6 +209,7 @@ public void shouldStartJobWithCorrectPipelineOptions() throws IOException { actualPipelineOptions.getSpecsStreamingUpdateConfigJson(), equalTo(printer.print(specsStreamingUpdateConfig))); assertThat(actual.getExtId(), equalTo(expectedExtJobId)); + assertThat(actual.getStatus(), equalTo(JobStatus.RUNNING)); } @Test @@ -245,14 +247,15 @@ public void shouldThrowExceptionWhenJobStateTerminal() throws IOException { featureSetJobStatus.setFeatureSet(FeatureSet.fromProto(featureSet)); Job job = - new Job( - "job", - "", - Runner.DATAFLOW, - Source.fromProto(source), - Store.fromProto(store), - Sets.newHashSet(featureSetJobStatus), - JobStatus.PENDING); + Job.builder() + .setId("job") + .setExtId("") + .setRunner(Runner.DATAFLOW) + .setSource(Source.fromProto(source)) + .setStore(Store.fromProto(store)) + .setFeatureSetJobStatuses(Sets.newHashSet(featureSetJobStatus)) + .setStatus(JobStatus.PENDING) + .build(); expectedException.expect(JobExecutionException.class); dfJobManager.startJob(job); diff --git a/core/src/test/java/feast/core/job/direct/DirectRunnerJobManagerTest.java b/core/src/test/java/feast/core/job/direct/DirectRunnerJobManagerTest.java index 0e797dc64d4..5b70c59d692 100644 --- a/core/src/test/java/feast/core/job/direct/DirectRunnerJobManagerTest.java +++ b/core/src/test/java/feast/core/job/direct/DirectRunnerJobManagerTest.java @@ -16,7 +16,7 @@ */ package feast.core.job.direct; -import static feast.core.util.ModelHelpers.makeFeatureSetJobStatus; +import static feast.core.util.TestUtil.makeFeatureSetJobStatus; import static org.hamcrest.MatcherAssert.assertThat; import static org.hamcrest.Matchers.equalTo; import static org.mockito.ArgumentMatchers.any; @@ -144,17 +144,21 @@ public void shouldStartDirectJobAndRegisterPipelineResult() throws IOException { doReturn(mockPipelineResult).when(drJobManager).runPipeline(any()); Job job = - new Job( - expectedJobId, - "", - Runner.DIRECT, - Source.fromProto(source), - Store.fromProto(store), - makeFeatureSetJobStatus(FeatureSet.fromProto(featureSet)), - JobStatus.PENDING); + Job.builder() + .setId(expectedJobId) + .setExtId("") + .setRunner(Runner.DIRECT) + .setSource(Source.fromProto(source)) + .setStore(Store.fromProto(store)) + .setFeatureSetJobStatuses(makeFeatureSetJobStatus(FeatureSet.fromProto(featureSet))) + .setStatus(JobStatus.PENDING) + .build(); + Job actual = drJobManager.startJob(job); + verify(drJobManager, times(1)).runPipeline(pipelineOptionsCaptor.capture()); verify(directJobRegistry, times(1)).add(directJobCaptor.capture()); + assertThat(actual.getStatus(), equalTo(JobStatus.RUNNING)); ImportOptions actualPipelineOptions = pipelineOptionsCaptor.getValue(); DirectJob jobStarted = directJobCaptor.getValue(); @@ -184,10 +188,19 @@ public void shouldStartDirectJobAndRegisterPipelineResult() throws IOException { @Test public void shouldAbortJobThenRemoveFromRegistry() throws IOException { - DirectJob job = Mockito.mock(DirectJob.class); - when(directJobRegistry.get("job")).thenReturn(job); - drJobManager.abortJob("job"); - verify(job, times(1)).abort(); - verify(directJobRegistry, times(1)).remove("job"); + Job job = + Job.builder() + .setId("id") + .setExtId("ext1") + .setRunner(Runner.DIRECT) + .setStatus(JobStatus.RUNNING) + .build(); + + DirectJob directJob = Mockito.mock(DirectJob.class); + when(directJobRegistry.get("ext1")).thenReturn(directJob); + job = drJobManager.abortJob(job); + verify(directJob, times(1)).abort(); + verify(directJobRegistry, times(1)).remove("ext1"); + assertThat(job.getStatus(), equalTo(JobStatus.ABORTING)); } } diff --git a/core/src/test/java/feast/core/service/JobCoordinatorServiceTest.java b/core/src/test/java/feast/core/service/JobCoordinatorServiceTest.java index 9b8e47de751..3f7267671ab 100644 --- a/core/src/test/java/feast/core/service/JobCoordinatorServiceTest.java +++ b/core/src/test/java/feast/core/service/JobCoordinatorServiceTest.java @@ -16,9 +16,10 @@ */ package feast.core.service; -import static org.hamcrest.CoreMatchers.equalTo; import static org.hamcrest.CoreMatchers.is; import static org.hamcrest.MatcherAssert.assertThat; +import static org.hamcrest.Matchers.containsInAnyOrder; +import static org.junit.Assert.assertTrue; import static org.mockito.ArgumentMatchers.*; import static org.mockito.Mockito.*; import static org.mockito.MockitoAnnotations.initMocks; @@ -30,11 +31,11 @@ import feast.core.config.FeastProperties.JobProperties; import feast.core.dao.FeatureSetRepository; import feast.core.dao.JobRepository; +import feast.core.dao.SourceRepository; import feast.core.job.JobManager; -import feast.core.job.JobMatcher; import feast.core.job.Runner; import feast.core.model.*; -import feast.core.util.ModelHelpers; +import feast.core.util.TestUtil; import feast.proto.core.CoreServiceProto.ListFeatureSetsRequest.Filter; import feast.proto.core.CoreServiceProto.ListFeatureSetsResponse; import feast.proto.core.CoreServiceProto.ListStoresResponse; @@ -42,16 +43,16 @@ import feast.proto.core.FeatureSetProto.FeatureSetMeta; import feast.proto.core.FeatureSetProto.FeatureSetSpec; import feast.proto.core.IngestionJobProto; +import feast.proto.core.SourceProto; import feast.proto.core.SourceProto.KafkaSourceConfig; -import feast.proto.core.SourceProto.Source; import feast.proto.core.SourceProto.SourceType; import feast.proto.core.StoreProto; -import feast.proto.core.StoreProto.Store.RedisConfig; -import feast.proto.core.StoreProto.Store.StoreType; import feast.proto.core.StoreProto.Store.Subscription; +import java.util.ArrayList; import java.util.Arrays; import java.util.Collections; import java.util.List; +import java.util.Optional; import java.util.concurrent.CancellationException; import lombok.SneakyThrows; import org.apache.kafka.clients.consumer.ConsumerRecord; @@ -72,6 +73,7 @@ public class JobCoordinatorServiceTest { @Mock SpecService specService; @Mock FeatureSetRepository featureSetRepository; @Mock private KafkaTemplate kafkaTemplate; + @Mock SourceRepository sourceRepository; private FeastProperties feastProperties; private JobCoordinatorService jcs; @@ -88,6 +90,7 @@ public void setUp() { new JobCoordinatorService( jobRepository, featureSetRepository, + sourceRepository, specService, jobManager, feastProperties, @@ -99,41 +102,34 @@ public void setUp() { @Test public void shouldDoNothingIfNoStoresFound() throws InvalidProtocolBufferException { when(specService.listStores(any())).thenReturn(ListStoresResponse.newBuilder().build()); - jcs.Poll(); verify(jobRepository, times(0)).saveAndFlush(any()); } @Test public void shouldDoNothingIfNoMatchingFeatureSetsFound() throws InvalidProtocolBufferException { - StoreProto.Store store = - StoreProto.Store.newBuilder() - .setName("test") - .setType(StoreType.REDIS) - .setRedisConfig(RedisConfig.newBuilder().build()) - .addSubscriptions(Subscription.newBuilder().setProject("*").setName("*").build()) - .build(); + Store store = + TestUtil.createStore( + "test", List.of(Subscription.newBuilder().setName("*").setProject("*").build())); + StoreProto.Store storeSpec = store.toProto(); + when(specService.listStores(any())) - .thenReturn(ListStoresResponse.newBuilder().addStore(store).build()); + .thenReturn(ListStoresResponse.newBuilder().addStore(storeSpec).build()); when(specService.listFeatureSets( Filter.newBuilder().setProject("*").setFeatureSetName("*").build())) .thenReturn(ListFeatureSetsResponse.newBuilder().build()); - jcs.Poll(); verify(jobRepository, times(0)).saveAndFlush(any()); } @Test public void shouldGenerateAndSubmitJobsIfAny() throws InvalidProtocolBufferException { - StoreProto.Store store = - StoreProto.Store.newBuilder() - .setName("test") - .setType(StoreType.REDIS) - .setRedisConfig(RedisConfig.newBuilder().build()) - .addSubscriptions(Subscription.newBuilder().setProject("project1").setName("*").build()) - .build(); - Source source = - Source.newBuilder() + Store store = + TestUtil.createStore( + "test", List.of(Subscription.newBuilder().setName("*").setProject("project1").build())); + StoreProto.Store storeSpec = store.toProto(); + SourceProto.Source sourceSpec = + SourceProto.Source.newBuilder() .setType(SourceType.KAFKA) .setKafkaSourceConfig( KafkaSourceConfig.newBuilder() @@ -141,196 +137,407 @@ public void shouldGenerateAndSubmitJobsIfAny() throws InvalidProtocolBufferExcep .setBootstrapServers("servers:9092") .build()) .build(); + Source source = Source.fromProto(sourceSpec); - FeatureSetProto.FeatureSet featureSetProto1 = + FeatureSetProto.FeatureSet featureSetSpec1 = FeatureSetProto.FeatureSet.newBuilder() .setSpec( FeatureSetSpec.newBuilder() - .setSource(source) + .setSource(sourceSpec) .setProject("project1") .setName("features1")) .setMeta(FeatureSetMeta.newBuilder()) .build(); - FeatureSet featureSet1 = FeatureSet.fromProto(featureSetProto1); - FeatureSetProto.FeatureSet featureSetProto2 = + FeatureSet featureSet1 = FeatureSet.fromProto(featureSetSpec1); + FeatureSetProto.FeatureSet featureSetSpec2 = FeatureSetProto.FeatureSet.newBuilder() .setSpec( FeatureSetSpec.newBuilder() - .setSource(source) + .setSource(sourceSpec) .setProject("project1") .setName("features2")) .setMeta(FeatureSetMeta.newBuilder()) .build(); - FeatureSet featureSet2 = FeatureSet.fromProto(featureSetProto2); - String extId = "ext"; + FeatureSet featureSet2 = FeatureSet.fromProto(featureSetSpec2); ArgumentCaptor> jobArgCaptor = ArgumentCaptor.forClass(List.class); Job expectedInput = - new Job( - "", - "", - Runner.DATAFLOW, - feast.core.model.Source.fromProto(source), - feast.core.model.Store.fromProto(store), - ModelHelpers.makeFeatureSetJobStatus(featureSet1, featureSet2), - JobStatus.PENDING); + Job.builder() + .setId("id") + .setExtId("") + .setRunner(Runner.DATAFLOW) + .setSource(source) + .setStore(store) + .setFeatureSetJobStatuses(TestUtil.makeFeatureSetJobStatus(featureSet1, featureSet2)) + .setStatus(JobStatus.PENDING) + .build(); Job expected = - new Job( - "some_id", - extId, - Runner.DATAFLOW, - feast.core.model.Source.fromProto(source), - feast.core.model.Store.fromProto(store), - ModelHelpers.makeFeatureSetJobStatus(featureSet1, featureSet2), - JobStatus.RUNNING); + expectedInput.toBuilder().setExtId("extid1").setStatus(JobStatus.RUNNING).build(); when(featureSetRepository.findAllByNameLikeAndProject_NameLikeOrderByNameAsc("%", "project1")) .thenReturn(Lists.newArrayList(featureSet1, featureSet2)); + when(sourceRepository.findFirstByTypeAndConfigOrderByIdAsc( + source.getType(), source.getConfig())) + .thenReturn(source); when(specService.listStores(any())) - .thenReturn(ListStoresResponse.newBuilder().addStore(store).build()); + .thenReturn(ListStoresResponse.newBuilder().addStore(storeSpec).build()); - when(jobManager.startJob(argThat(new JobMatcher(expectedInput)))).thenReturn(expected); + when(jobManager.startJob(expectedInput)).thenReturn(expected); when(jobManager.getRunnerType()).thenReturn(Runner.DATAFLOW); + when(jobRepository.findByStatus(JobStatus.RUNNING)).thenReturn(List.of()); + when(jobRepository + .findFirstBySourceTypeAndSourceConfigAndStoreNameAndStatusNotInOrderByLastUpdatedDesc( + source.getType(), + source.getConfig(), + store.getName(), + JobStatus.getTerminalStates())) + .thenReturn(Optional.empty()); + jcs.Poll(); verify(jobRepository, times(1)).saveAll(jobArgCaptor.capture()); List actual = jobArgCaptor.getValue(); - assertThat(actual, equalTo(Collections.singletonList(expected))); + assertThat(actual, containsInAnyOrder(expected)); } @Test public void shouldGroupJobsBySource() throws InvalidProtocolBufferException { - StoreProto.Store store = - StoreProto.Store.newBuilder() - .setName("test") - .setType(StoreType.REDIS) - .setRedisConfig(RedisConfig.newBuilder().build()) - .addSubscriptions(Subscription.newBuilder().setProject("project1").setName("*").build()) - .build(); - Source source1 = - Source.newBuilder() - .setType(SourceType.KAFKA) - .setKafkaSourceConfig( - KafkaSourceConfig.newBuilder() - .setTopic("topic") - .setBootstrapServers("servers:9092") - .build()) - .build(); - Source source2 = - Source.newBuilder() - .setType(SourceType.KAFKA) - .setKafkaSourceConfig( - KafkaSourceConfig.newBuilder() - .setTopic("topic") - .setBootstrapServers("other.servers:9092") - .build()) - .build(); + Store store = + TestUtil.createStore( + "test", List.of(Subscription.newBuilder().setName("*").setProject("project1").build())); + StoreProto.Store storeSpec = store.toProto(); - FeatureSetProto.FeatureSet featureSetProto1 = - FeatureSetProto.FeatureSet.newBuilder() - .setSpec( - FeatureSetSpec.newBuilder() - .setSource(source1) - .setProject("project1") - .setName("features1")) - .setMeta(FeatureSetMeta.newBuilder()) - .build(); - FeatureSet featureSet1 = FeatureSet.fromProto(featureSetProto1); + Source source1 = TestUtil.createKafkaSource("servers:9092", "topic", false); + Source source2 = TestUtil.createKafkaSource("others.servers:9092", "topic", false); - FeatureSetProto.FeatureSet featureSetProto2 = - FeatureSetProto.FeatureSet.newBuilder() - .setSpec( - FeatureSetSpec.newBuilder() - .setSource(source2) - .setProject("project1") - .setName("features2")) - .setMeta(FeatureSetMeta.newBuilder()) - .build(); - FeatureSet featureSet2 = FeatureSet.fromProto(featureSetProto2); + FeatureSet featureSet1 = TestUtil.createEmptyFeatureSet("features1", source1); + FeatureSet featureSet2 = TestUtil.createEmptyFeatureSet("features2", source2); Job expectedInput1 = - new Job( - "name1", - "", - Runner.DATAFLOW, - feast.core.model.Source.fromProto(source1), - feast.core.model.Store.fromProto(store), - ModelHelpers.makeFeatureSetJobStatus(featureSet1), - JobStatus.PENDING); + Job.builder() + .setId("id1") + .setExtId("") + .setRunner(Runner.DATAFLOW) + .setSource(source1) + .setStore(store) + .setFeatureSetJobStatuses(TestUtil.makeFeatureSetJobStatus(featureSet1)) + .setStatus(JobStatus.PENDING) + .build(); Job expected1 = - new Job( - "name1", - "extId1", - Runner.DATAFLOW, - feast.core.model.Source.fromProto(source1), - feast.core.model.Store.fromProto(store), - ModelHelpers.makeFeatureSetJobStatus(featureSet1), - JobStatus.RUNNING); + expectedInput1.toBuilder().setExtId("extid1").setStatus(JobStatus.RUNNING).build(); Job expectedInput2 = - new Job( - "", - "extId2", - Runner.DATAFLOW, - feast.core.model.Source.fromProto(source2), - feast.core.model.Store.fromProto(store), - ModelHelpers.makeFeatureSetJobStatus(featureSet2), - JobStatus.PENDING); + Job.builder() + .setId("id2") + .setExtId("") + .setRunner(Runner.DATAFLOW) + .setSource(source2) + .setStore(store) + .setFeatureSetJobStatuses(TestUtil.makeFeatureSetJobStatus(featureSet2)) + .setStatus(JobStatus.PENDING) + .build(); Job expected2 = - new Job( - "name2", - "extId2", - Runner.DATAFLOW, - feast.core.model.Source.fromProto(source2), - feast.core.model.Store.fromProto(store), - ModelHelpers.makeFeatureSetJobStatus(featureSet2), - JobStatus.RUNNING); + expectedInput2.toBuilder().setExtId("extid2").setStatus(JobStatus.RUNNING).build(); + + when(featureSetRepository.findAllByNameLikeAndProject_NameLikeOrderByNameAsc("%", "project1")) + .thenReturn(Lists.newArrayList(featureSet1, featureSet2)); + when(sourceRepository.findFirstByTypeAndConfigOrderByIdAsc( + source1.getType(), source1.getConfig())) + .thenReturn(source1); + when(sourceRepository.findFirstByTypeAndConfigOrderByIdAsc( + source2.getType(), source2.getConfig())) + .thenReturn(source2); + when(specService.listStores(any())) + .thenReturn(ListStoresResponse.newBuilder().addStore(storeSpec).build()); + + when(jobManager.startJob(expectedInput1)).thenReturn(expected1); + when(jobManager.startJob(expectedInput2)).thenReturn(expected2); + when(jobManager.getRunnerType()).thenReturn(Runner.DATAFLOW); + + when(jobRepository.findByStatus(JobStatus.RUNNING)).thenReturn(List.of()); + when(jobRepository + .findFirstBySourceTypeAndSourceConfigAndStoreNameAndStatusNotInOrderByLastUpdatedDesc( + source1.getType(), + source1.getConfig(), + storeSpec.getName(), + JobStatus.getTerminalStates())) + .thenReturn(Optional.empty()); + when(jobRepository + .findFirstBySourceTypeAndSourceConfigAndStoreNameAndStatusNotInOrderByLastUpdatedDesc( + source2.getType(), + source2.getConfig(), + storeSpec.getName(), + JobStatus.getTerminalStates())) + .thenReturn(Optional.empty()); + + jcs.Poll(); + ArgumentCaptor> jobArgCaptor = ArgumentCaptor.forClass(List.class); + verify(jobRepository, times(1)).saveAll(jobArgCaptor.capture()); + List actual = jobArgCaptor.getValue(); + + for (Job expectedJob : List.of(expected1, expected2)) { + assertTrue(actual.contains(expectedJob)); + } + } + + @Test + public void shouldGroupJobsBySourceAndIgnoreDuplicateSourceObjects() + throws InvalidProtocolBufferException { + Store store = + TestUtil.createStore( + "test", List.of(Subscription.newBuilder().setName("*").setProject("project1").build())); + StoreProto.Store storeSpec = store.toProto(); + + // simulate duplicate source objects: create source objects from the same spec but with + // different ids + Source source1 = TestUtil.createKafkaSource("servers:9092", "topic", false); + source1.setId(1); + Source source2 = TestUtil.createKafkaSource("servers:9092", "topic", false); + source2.setId(2); + + FeatureSet featureSet1 = TestUtil.createEmptyFeatureSet("features1", source1); + FeatureSet featureSet2 = TestUtil.createEmptyFeatureSet("features2", source2); + + Job expectedInput = + Job.builder() + .setId("id") + .setExtId("") + .setSource(source1) + .setStore(store) + .setRunner(Runner.DATAFLOW) + .setFeatureSetJobStatuses(TestUtil.makeFeatureSetJobStatus(featureSet1, featureSet2)) + .setStatus(JobStatus.PENDING) + .build(); + + Job expected = + expectedInput.toBuilder().setExtId("extid1").setStatus(JobStatus.RUNNING).build(); when(featureSetRepository.findAllByNameLikeAndProject_NameLikeOrderByNameAsc("%", "project1")) .thenReturn(Lists.newArrayList(featureSet1, featureSet2)); + when(sourceRepository.findFirstByTypeAndConfigOrderByIdAsc( + source1.getType(), source1.getConfig())) + .thenReturn(source1); + when(sourceRepository.findFirstByTypeAndConfigOrderByIdAsc( + source2.getType(), source2.getConfig())) + .thenReturn(source1); + when(specService.listStores(any())) + .thenReturn(ListStoresResponse.newBuilder().addStore(storeSpec).build()); + + when(jobManager.startJob(expectedInput)).thenReturn(expected); + when(jobManager.getRunnerType()).thenReturn(Runner.DATAFLOW); + + when(jobRepository.findByStatus(JobStatus.RUNNING)).thenReturn(List.of()); + when(jobRepository + .findFirstBySourceTypeAndSourceConfigAndStoreNameAndStatusNotInOrderByLastUpdatedDesc( + source1.getType(), + source1.getConfig(), + storeSpec.getName(), + JobStatus.getTerminalStates())) + .thenReturn(Optional.empty()); + + ArgumentCaptor> jobArgCaptor = ArgumentCaptor.forClass(List.class); + + jcs.Poll(); + verify(jobRepository, times(1)).saveAll(jobArgCaptor.capture()); + List actual = jobArgCaptor.getValue(); + assertThat(actual, containsInAnyOrder(expected)); + } + + @Test + public void shouldStopDuplicateJobsForSource() throws InvalidProtocolBufferException { + Store store = + TestUtil.createStore( + "test", List.of(Subscription.newBuilder().setName("*").setProject("project1").build())); + StoreProto.Store storeSpec = store.toProto(); + + Source source = TestUtil.createKafkaSource("servers:9092", "topic", false); + + FeatureSet featureSet = TestUtil.createEmptyFeatureSet("features2", source); + + // simulate 3 running jobs, all serving the same source to store pairing. + // JobCoordinatorService should abort 2 the extra jobs. + List inputJobs = new ArrayList<>(); + List expectedJobs = new ArrayList<>(); + List extraJobs = new ArrayList(); + for (int i = 0; i < 3; i++) { + inputJobs.add( + Job.builder() + .setId(String.format("id%d", i)) + .setExtId(String.format("extid%d", i)) + .setSource(source) + .setStore(store) + .setRunner(Runner.DATAFLOW) + .setFeatureSetJobStatuses(TestUtil.makeFeatureSetJobStatus(featureSet)) + .setStatus(JobStatus.RUNNING) + .build()); + + JobStatus targetStatus = (i >= 1) ? JobStatus.ABORTED : JobStatus.RUNNING; + expectedJobs.add(inputJobs.get(i).toBuilder().setStatus(targetStatus).build()); + + if (targetStatus == JobStatus.ABORTED) { + when(jobManager.abortJob(inputJobs.get(i))).thenReturn(expectedJobs.get(i)); + extraJobs.add(inputJobs.get(i)); + } + } + + when(featureSetRepository.findAllByNameLikeAndProject_NameLikeOrderByNameAsc("%", "project1")) + .thenReturn(Lists.newArrayList(featureSet)); + when(sourceRepository.findFirstByTypeAndConfigOrderByIdAsc( + source.getType(), source.getConfig())) + .thenReturn(source); + when(specService.listStores(any())) + .thenReturn(ListStoresResponse.newBuilder().addStore(storeSpec).build()); + + when(jobManager.getJobStatus(inputJobs.get(0))).thenReturn(JobStatus.RUNNING); + when(jobManager.getRunnerType()).thenReturn(Runner.DATAFLOW); + when(jobRepository + .findFirstBySourceTypeAndSourceConfigAndStoreNameAndStatusNotInOrderByLastUpdatedDesc( + source.getType(), + source.getConfig(), + store.getName(), + JobStatus.getTerminalStates())) + .thenReturn(Optional.of(inputJobs.get(0))); + when(jobRepository.findByStatus(JobStatus.RUNNING)).thenReturn(inputJobs); + + jcs.Poll(); + + ArgumentCaptor jobCaptor = ArgumentCaptor.forClass(Job.class); + + verify(jobManager, times(2)).abortJob(jobCaptor.capture()); + List abortedJobs = jobCaptor.getAllValues(); + for (Job extraJob : extraJobs) { + assertTrue(abortedJobs.contains(extraJob)); + } + + ArgumentCaptor> jobListCaptor = ArgumentCaptor.forClass(List.class); + verify(jobRepository, times(1)).saveAll(jobListCaptor.capture()); + List actual = jobListCaptor.getValue(); + for (Job expectedJob : expectedJobs) { + assertTrue(actual.contains(expectedJob)); + } + } + + @Test + public void shouldUseStoreSubscriptionToMapStore() throws InvalidProtocolBufferException { + Store store1 = + TestUtil.createStore( + "test", + List.of(Subscription.newBuilder().setName("features1").setProject("*").build())); + StoreProto.Store store1Spec = store1.toProto(); + + Store store2 = + TestUtil.createStore( + "test", + List.of(Subscription.newBuilder().setName("features2").setProject("*").build())); + StoreProto.Store store2Spec = store2.toProto(); + + Source source1 = TestUtil.createKafkaSource("servers:9092", "topic", false); + Source source2 = TestUtil.createKafkaSource("other.servers:9092", "topic", false); + + FeatureSet featureSet1 = TestUtil.createEmptyFeatureSet("feature1", source1); + FeatureSet featureSet2 = TestUtil.createEmptyFeatureSet("feature2", source2); + + Job expectedInput1 = + Job.builder() + .setId("id1") + .setExtId("") + .setRunner(Runner.DATAFLOW) + .setSource(source1) + .setStore(store1) + .setFeatureSetJobStatuses(TestUtil.makeFeatureSetJobStatus(featureSet1)) + .setStatus(JobStatus.PENDING) + .build(); + + Job expected1 = + expectedInput1.toBuilder().setExtId("extid1").setStatus(JobStatus.RUNNING).build(); + + Job expectedInput2 = + Job.builder() + .setId("id2") + .setExtId("") + .setRunner(Runner.DATAFLOW) + .setSource(source2) + .setStore(store2) + .setFeatureSetJobStatuses(TestUtil.makeFeatureSetJobStatus(featureSet2)) + .setStatus(JobStatus.PENDING) + .build(); + + Job expected2 = + expectedInput2.toBuilder().setExtId("extid2").setStatus(JobStatus.RUNNING).build(); + + ArgumentCaptor> jobArgCaptor = ArgumentCaptor.forClass(List.class); + + when(featureSetRepository.findAllByNameLikeAndProject_NameLikeOrderByNameAsc("features1", "%")) + .thenReturn(Lists.newArrayList(featureSet1)); + when(featureSetRepository.findAllByNameLikeAndProject_NameLikeOrderByNameAsc("features2", "%")) + .thenReturn(Lists.newArrayList(featureSet2)); + when(sourceRepository.findFirstByTypeAndConfigOrderByIdAsc( + source1.getType(), source1.getConfig())) + .thenReturn(source1); + when(sourceRepository.findFirstByTypeAndConfigOrderByIdAsc( + source2.getType(), source2.getConfig())) + .thenReturn(source2); when(specService.listStores(any())) - .thenReturn(ListStoresResponse.newBuilder().addStore(store).build()); + .thenReturn( + ListStoresResponse.newBuilder().addStore(store1Spec).addStore(store2Spec).build()); - when(jobManager.startJob(argThat(new JobMatcher(expectedInput1)))).thenReturn(expected1); - when(jobManager.startJob(argThat(new JobMatcher(expectedInput2)))).thenReturn(expected2); + when(jobManager.startJob(expectedInput1)).thenReturn(expected1); + when(jobManager.startJob(expectedInput2)).thenReturn(expected2); when(jobManager.getRunnerType()).thenReturn(Runner.DATAFLOW); + when(jobRepository.findByStatus(JobStatus.RUNNING)).thenReturn(List.of()); + when(jobRepository + .findFirstBySourceTypeAndSourceConfigAndStoreNameAndStatusNotInOrderByLastUpdatedDesc( + source1.getType(), + source1.getConfig(), + store1.getName(), + JobStatus.getTerminalStates())) + .thenReturn(Optional.empty()); + when(jobRepository + .findFirstBySourceTypeAndSourceConfigAndStoreNameAndStatusNotInOrderByLastUpdatedDesc( + source2.getType(), + source1.getConfig(), + store2.getName(), + JobStatus.getTerminalStates())) + .thenReturn(Optional.empty()); + jcs.Poll(); verify(jobRepository, times(1)).saveAll(jobArgCaptor.capture()); List actual = jobArgCaptor.getValue(); - assertThat(actual.get(0), equalTo(expected1)); - assertThat(actual.get(1), equalTo(expected2)); + for (Job expectedJob : List.of(expected1, expected2)) { + assertTrue(actual.contains(expectedJob)); + } } @Test public void shouldSendPendingFeatureSetToJobs() { FeatureSet fs1 = - TestObjectFactory.CreateFeatureSet( + TestUtil.CreateFeatureSet( "fs_1", "project", Collections.emptyList(), Collections.emptyList()); fs1.setVersion(2); FeatureSetJobStatus status1 = - TestObjectFactory.CreateFeatureSetJobStatusWithJob( + TestUtil.CreateFeatureSetJobStatusWithJob( JobStatus.RUNNING, FeatureSetProto.FeatureSetJobDeliveryStatus.STATUS_DELIVERED, 1); FeatureSetJobStatus status2 = - TestObjectFactory.CreateFeatureSetJobStatusWithJob( + TestUtil.CreateFeatureSetJobStatusWithJob( JobStatus.RUNNING, FeatureSetProto.FeatureSetJobDeliveryStatus.STATUS_DELIVERED, 1); FeatureSetJobStatus status3 = - TestObjectFactory.CreateFeatureSetJobStatusWithJob( + TestUtil.CreateFeatureSetJobStatusWithJob( JobStatus.ABORTED, FeatureSetProto.FeatureSetJobDeliveryStatus.STATUS_DELIVERED, 2); // spec needs to be send fs1.getJobStatuses().addAll(ImmutableList.of(status1, status2, status3)); FeatureSet fs2 = - TestObjectFactory.CreateFeatureSet( + TestUtil.CreateFeatureSet( "fs_2", "project", Collections.emptyList(), Collections.emptyList()); fs2.setVersion(5); @@ -338,19 +545,19 @@ public void shouldSendPendingFeatureSetToJobs() { fs2.getJobStatuses() .addAll( ImmutableList.of( - TestObjectFactory.CreateFeatureSetJobStatusWithJob( + TestUtil.CreateFeatureSetJobStatusWithJob( JobStatus.RUNNING, FeatureSetProto.FeatureSetJobDeliveryStatus.STATUS_IN_PROGRESS, 5))); // feature set without running jobs attached FeatureSet fs3 = - TestObjectFactory.CreateFeatureSet( + TestUtil.CreateFeatureSet( "fs_3", "project", Collections.emptyList(), Collections.emptyList()); fs3.getJobStatuses() .addAll( ImmutableList.of( - TestObjectFactory.CreateFeatureSetJobStatusWithJob( + TestUtil.CreateFeatureSetJobStatusWithJob( JobStatus.ABORTED, FeatureSetProto.FeatureSetJobDeliveryStatus.STATUS_IN_PROGRESS, 5))); @@ -384,12 +591,12 @@ public void shouldSendPendingFeatureSetToJobs() { @SneakyThrows public void shouldNotUpdateJobStatusVersionWhenKafkaUnavailable() { FeatureSet fsInTest = - TestObjectFactory.CreateFeatureSet( + TestUtil.CreateFeatureSet( "fs_1", "project", Collections.emptyList(), Collections.emptyList()); fsInTest.setVersion(2); FeatureSetJobStatus featureSetJobStatus = - TestObjectFactory.CreateFeatureSetJobStatusWithJob( + TestUtil.CreateFeatureSetJobStatusWithJob( JobStatus.RUNNING, FeatureSetProto.FeatureSetJobDeliveryStatus.STATUS_DELIVERED, 1); fsInTest.getJobStatuses().add(featureSetJobStatus); @@ -405,13 +612,13 @@ public void shouldNotUpdateJobStatusVersionWhenKafkaUnavailable() { @Test public void specAckListenerShouldDoNothingWhenMessageIsOutdated() { FeatureSet fsInTest = - TestObjectFactory.CreateFeatureSet( + TestUtil.CreateFeatureSet( "fs", "project", Collections.emptyList(), Collections.emptyList()); FeatureSetJobStatus j1 = - TestObjectFactory.CreateFeatureSetJobStatusWithJob( + TestUtil.CreateFeatureSetJobStatusWithJob( JobStatus.RUNNING, FeatureSetProto.FeatureSetJobDeliveryStatus.STATUS_IN_PROGRESS, 1); FeatureSetJobStatus j2 = - TestObjectFactory.CreateFeatureSetJobStatusWithJob( + TestUtil.CreateFeatureSetJobStatusWithJob( JobStatus.RUNNING, FeatureSetProto.FeatureSetJobDeliveryStatus.STATUS_IN_PROGRESS, 1); fsInTest.getJobStatuses().addAll(Arrays.asList(j1, j2)); @@ -433,18 +640,18 @@ public void specAckListenerShouldDoNothingWhenMessageIsOutdated() { @Test public void specAckListenerShouldUpdateFeatureSetStatus() { FeatureSet fsInTest = - TestObjectFactory.CreateFeatureSet( + TestUtil.CreateFeatureSet( "fs", "project", Collections.emptyList(), Collections.emptyList()); fsInTest.setStatus(FeatureSetProto.FeatureSetStatus.STATUS_PENDING); FeatureSetJobStatus j1 = - TestObjectFactory.CreateFeatureSetJobStatusWithJob( + TestUtil.CreateFeatureSetJobStatusWithJob( JobStatus.RUNNING, FeatureSetProto.FeatureSetJobDeliveryStatus.STATUS_IN_PROGRESS, 1); FeatureSetJobStatus j2 = - TestObjectFactory.CreateFeatureSetJobStatusWithJob( + TestUtil.CreateFeatureSetJobStatusWithJob( JobStatus.RUNNING, FeatureSetProto.FeatureSetJobDeliveryStatus.STATUS_IN_PROGRESS, 1); FeatureSetJobStatus j3 = - TestObjectFactory.CreateFeatureSetJobStatusWithJob( + TestUtil.CreateFeatureSetJobStatusWithJob( JobStatus.ABORTED, FeatureSetProto.FeatureSetJobDeliveryStatus.STATUS_IN_PROGRESS, 1); fsInTest.getJobStatuses().addAll(Arrays.asList(j1, j2, j3)); diff --git a/core/src/test/java/feast/core/service/JobServiceTest.java b/core/src/test/java/feast/core/service/JobServiceTest.java index 6020db7b668..02ebe611b3c 100644 --- a/core/src/test/java/feast/core/service/JobServiceTest.java +++ b/core/src/test/java/feast/core/service/JobServiceTest.java @@ -16,7 +16,7 @@ */ package feast.core.service; -import static feast.core.util.ModelHelpers.makeFeatureSetJobStatus; +import static feast.core.util.TestUtil.makeFeatureSetJobStatus; import static org.hamcrest.CoreMatchers.equalTo; import static org.hamcrest.MatcherAssert.assertThat; import static org.junit.Assert.fail; @@ -32,6 +32,7 @@ import feast.core.job.JobManager; import feast.core.job.Runner; import feast.core.model.*; +import feast.core.util.TestUtil; import feast.proto.core.CoreServiceProto.ListFeatureSetsRequest; import feast.proto.core.CoreServiceProto.ListFeatureSetsResponse; import feast.proto.core.CoreServiceProto.ListIngestionJobsRequest; @@ -73,7 +74,7 @@ public void setup() { // create mock objects for testing // fake data source - this.dataSource = TestObjectFactory.defaultSource; + this.dataSource = TestUtil.defaultSource; // fake data store this.dataStore = new Store( @@ -113,7 +114,6 @@ public void setupSpecService() { when(this.specService.listFeatureSets(this.listFilters.get(1))).thenReturn(response); - when(this.specService.listFeatureSets(this.listFilters.get(2))).thenReturn(response); } catch (InvalidProtocolBufferException e) { e.printStackTrace(); fail("Unexpected exception"); @@ -137,60 +137,48 @@ public void setupJobManager() { } private FeatureSet newDummyFeatureSet(String name, String project) { - Feature feature = TestObjectFactory.CreateFeature(name + "_feature", Enum.INT64); - Entity entity = TestObjectFactory.CreateEntity(name + "_entity", Enum.STRING); + Feature feature = TestUtil.CreateFeature(name + "_feature", Enum.INT64); + Entity entity = TestUtil.CreateEntity(name + "_entity", Enum.STRING); FeatureSet fs = - TestObjectFactory.CreateFeatureSet( - name, project, Arrays.asList(entity), Arrays.asList(feature)); + TestUtil.CreateFeatureSet(name, project, Arrays.asList(entity), Arrays.asList(feature)); fs.setCreated(Date.from(Instant.ofEpochSecond(10L))); return fs; } private Job newDummyJob(String id, String extId, JobStatus status) { - return new Job( - id, - extId, - Runner.DATAFLOW, - this.dataSource, - this.dataStore, - makeFeatureSetJobStatus(this.featureSet), - status); + return job.builder() + .setId(id) + .setExtId(extId) + .setRunner(Runner.DATAFLOW) + .setSource(this.dataSource) + .setStore(this.dataStore) + .setFeatureSetJobStatuses(makeFeatureSetJobStatus(this.featureSet)) + .setStatus(status) + .build(); } private List newDummyFeatureSetReferences() { return Arrays.asList( - // all provided: name, version and project - FeatureSetReference.newBuilder() - .setName(this.featureSet.getName()) - .setProject(this.featureSet.getProject().toString()) - .build(), - // name and project FeatureSetReference.newBuilder() .setName(this.featureSet.getName()) .setProject(this.featureSet.getProject().toString()) .build(), - // name and version + // name only FeatureSetReference.newBuilder().setName(this.featureSet.getName()).build()); } private List newDummyListRequestFilters() { return Arrays.asList( - // all provided: name, version and project - ListFeatureSetsRequest.Filter.newBuilder() - .setFeatureSetName(this.featureSet.getName()) - .setProject(this.featureSet.getProject().toString()) - .build(), - // name and project ListFeatureSetsRequest.Filter.newBuilder() .setFeatureSetName(this.featureSet.getName()) .setProject(this.featureSet.getProject().toString()) .build(), - // name and project + // name only ListFeatureSetsRequest.Filter.newBuilder() .setFeatureSetName(this.featureSet.getName()) .setProject("*") @@ -248,7 +236,7 @@ public void testListIngestionJobByFeatureSetReference() { ListIngestionJobsRequest.newBuilder().setFilter(filter).build(); assertThat(this.tryListJobs(request).getJobs(0), equalTo(this.ingestionJob)); - // list job by feature set reference: name and version + // list job by feature set reference: name filter = ListIngestionJobsRequest.Filter.newBuilder() .setFeatureSetReference(this.fsReferences.get(1)) @@ -256,15 +244,6 @@ public void testListIngestionJobByFeatureSetReference() { .build(); request = ListIngestionJobsRequest.newBuilder().setFilter(filter).build(); assertThat(this.tryListJobs(request).getJobs(0), equalTo(this.ingestionJob)); - - // list job by feature set reference: name and project - filter = - ListIngestionJobsRequest.Filter.newBuilder() - .setFeatureSetReference(this.fsReferences.get(2)) - .setId(this.job.getId()) - .build(); - request = ListIngestionJobsRequest.newBuilder().setFilter(filter).build(); - assertThat(this.tryListJobs(request).getJobs(0), equalTo(this.ingestionJob)); } private StopIngestionJobResponse tryStopJob( @@ -294,8 +273,10 @@ public void testStopJobForId() { StopIngestionJobRequest request = StopIngestionJobRequest.newBuilder().setId(this.job.getId()).build(); + when(this.jobManager.abortJob(this.job)) + .thenReturn(this.newDummyJob(job.getId(), job.getExtId(), JobStatus.ABORTING)); this.tryStopJob(request, false); - verify(this.jobManager).abortJob(this.job.getExtId()); + verify(this.jobManager).abortJob(this.job); // TODO: check that for job status change in featureset source @@ -315,7 +296,7 @@ public void testStopAlreadyStopped() { StopIngestionJobRequest.newBuilder().setId(this.job.getId()).build(); this.tryStopJob(request, false); - verify(this.jobManager, never()).abortJob(this.job.getExtId()); + verify(this.jobManager, never()).abortJob(this.job); } this.job.setStatus(prevStatus); diff --git a/core/src/test/java/feast/core/service/SpecServiceTest.java b/core/src/test/java/feast/core/service/SpecServiceTest.java index dad04462ec8..9c457379c4d 100644 --- a/core/src/test/java/feast/core/service/SpecServiceTest.java +++ b/core/src/test/java/feast/core/service/SpecServiceTest.java @@ -29,6 +29,7 @@ import feast.core.dao.StoreRepository; import feast.core.exception.RetrievalException; import feast.core.model.*; +import feast.core.util.TestUtil; import feast.proto.core.CoreServiceProto.ApplyFeatureSetResponse; import feast.proto.core.CoreServiceProto.ApplyFeatureSetResponse.Status; import feast.proto.core.CoreServiceProto.GetFeatureSetRequest; @@ -99,7 +100,7 @@ public class SpecServiceTest { @Before public void setUp() throws InvalidProtocolBufferException { initMocks(this); - defaultSource = TestObjectFactory.defaultSource; + defaultSource = TestUtil.defaultSource; FeatureSet featureSet1 = newDummyFeatureSet("f1", "project1"); FeatureSet featureSet2 = newDummyFeatureSet("f2", "project1"); @@ -108,13 +109,12 @@ public void setUp() throws InvalidProtocolBufferException { Map featureLabels2 = Map.ofEntries(Map.entry("key2", "val2")); Map dummyLabels = Map.ofEntries(Map.entry("key", "value")); - Feature dummyFeature = TestObjectFactory.CreateFeature("feature", Enum.STRING, dummyLabels); - Feature f3f1 = TestObjectFactory.CreateFeature("f3f1", Enum.INT64); - Feature f3f2 = TestObjectFactory.CreateFeature("f3f2", Enum.INT64); - Entity f3e1 = TestObjectFactory.CreateEntity("f3e1", Enum.STRING); + Feature dummyFeature = TestUtil.CreateFeature("feature", Enum.STRING, dummyLabels); + Feature f3f1 = TestUtil.CreateFeature("f3f1", Enum.INT64); + Feature f3f2 = TestUtil.CreateFeature("f3f2", Enum.INT64); + Entity f3e1 = TestUtil.CreateEntity("f3e1", Enum.STRING); FeatureSet featureSet3 = - TestObjectFactory.CreateFeatureSet( - "f3", "project1", Arrays.asList(f3e1), Arrays.asList(f3f2, f3f1)); + TestUtil.CreateFeatureSet("f3", "project1", Arrays.asList(f3e1), Arrays.asList(f3f2, f3f1)); FeatureSet featureSet4 = newDummyFeatureSet("f4", Project.DEFAULT_NAME); Map singleFeatureSetLabels = @@ -153,19 +153,17 @@ public void setUp() throws InvalidProtocolBufferException { .build()) .build(); - Entity f7e1 = TestObjectFactory.CreateEntity("f7e1", Enum.STRING); - Entity f9e1 = TestObjectFactory.CreateEntity("f9e1", Enum.STRING); - Feature f7f1 = TestObjectFactory.CreateFeature("f7f1", Enum.INT64, featureLabels1); - Feature f8f1 = TestObjectFactory.CreateFeature("f8f1", Enum.INT64, featureLabels2); + Entity f7e1 = TestUtil.CreateEntity("f7e1", Enum.STRING); + Entity f9e1 = TestUtil.CreateEntity("f9e1", Enum.STRING); + Feature f7f1 = TestUtil.CreateFeature("f7f1", Enum.INT64, featureLabels1); + Feature f8f1 = TestUtil.CreateFeature("f8f1", Enum.INT64, featureLabels2); FeatureSet featureSet7 = - TestObjectFactory.CreateFeatureSet( + TestUtil.CreateFeatureSet( "f7", "project2", Arrays.asList(f7e1), Arrays.asList(f3f1, f3f2, f7f1)); FeatureSet featureSet8 = - TestObjectFactory.CreateFeatureSet( - "f8", "project2", Arrays.asList(f7e1), Arrays.asList(f3f1, f8f1)); + TestUtil.CreateFeatureSet("f8", "project2", Arrays.asList(f7e1), Arrays.asList(f3f1, f8f1)); FeatureSet featureSet9 = - TestObjectFactory.CreateFeatureSet( - "f9", "default", Arrays.asList(f9e1), Arrays.asList(f3f1, f8f1)); + TestUtil.CreateFeatureSet("f9", "default", Arrays.asList(f9e1), Arrays.asList(f3f1, f8f1)); features = Arrays.asList(dummyFeature, f3f1, f3f2, f7f1, f8f1); featureSets = @@ -568,12 +566,11 @@ public void applyFeatureSetShouldUpdateFeatureSetWhenConstraintsAreUpdated() @Test public void applyFeatureSetShouldCreateProjectWhenNotAlreadyExists() throws InvalidProtocolBufferException { - Feature f3f1 = TestObjectFactory.CreateFeature("f3f1", Enum.INT64); - Feature f3f2 = TestObjectFactory.CreateFeature("f3f2", Enum.INT64); - Entity f3e1 = TestObjectFactory.CreateEntity("f3e1", Enum.STRING); + Feature f3f1 = TestUtil.CreateFeature("f3f1", Enum.INT64); + Feature f3f2 = TestUtil.CreateFeature("f3f2", Enum.INT64); + Entity f3e1 = TestUtil.CreateEntity("f3e1", Enum.STRING); FeatureSetProto.FeatureSet incomingFeatureSet = - TestObjectFactory.CreateFeatureSet( - "f3", "project", Arrays.asList(f3e1), Arrays.asList(f3f2, f3f1)) + TestUtil.CreateFeatureSet("f3", "project", Arrays.asList(f3e1), Arrays.asList(f3f2, f3f1)) .toProto(); ApplyFeatureSetResponse applyFeatureSetResponse = @@ -587,13 +584,13 @@ public void applyFeatureSetShouldCreateProjectWhenNotAlreadyExists() @Test public void applyFeatureSetShouldUsedDefaultProjectIfUnspecified() throws InvalidProtocolBufferException { - Feature f3f1 = TestObjectFactory.CreateFeature("f3f1", Enum.INT64); - Feature f3f2 = TestObjectFactory.CreateFeature("f3f2", Enum.INT64); - Entity f3e1 = TestObjectFactory.CreateEntity("f3e1", Enum.STRING); + Feature f3f1 = TestUtil.CreateFeature("f3f1", Enum.INT64); + Feature f3f2 = TestUtil.CreateFeature("f3f2", Enum.INT64); + Entity f3e1 = TestUtil.CreateEntity("f3e1", Enum.STRING); // In protov3, unspecified project defaults to "" FeatureSetProto.FeatureSet incomingFeatureSet = - TestObjectFactory.CreateFeatureSet("f3", "", Arrays.asList(f3e1), Arrays.asList(f3f2, f3f1)) + TestUtil.CreateFeatureSet("f3", "", Arrays.asList(f3e1), Arrays.asList(f3f2, f3f1)) .toProto(); ApplyFeatureSetResponse applyFeatureSetResponse = specService.applyFeatureSet(incomingFeatureSet); @@ -607,11 +604,11 @@ public void applyFeatureSetShouldUsedDefaultProjectIfUnspecified() @Test public void applyFeatureSetShouldFailWhenProjectIsArchived() throws InvalidProtocolBufferException { - Feature f3f1 = TestObjectFactory.CreateFeature("f3f1", Enum.INT64); - Feature f3f2 = TestObjectFactory.CreateFeature("f3f2", Enum.INT64); - Entity f3e1 = TestObjectFactory.CreateEntity("f3e1", Enum.STRING); + Feature f3f1 = TestUtil.CreateFeature("f3f1", Enum.INT64); + Feature f3f2 = TestUtil.CreateFeature("f3f2", Enum.INT64); + Entity f3e1 = TestUtil.CreateEntity("f3e1", Enum.STRING); FeatureSetProto.FeatureSet incomingFeatureSet = - TestObjectFactory.CreateFeatureSet( + TestUtil.CreateFeatureSet( "f3", "archivedproject", Arrays.asList(f3e1), Arrays.asList(f3f2, f3f1)) .toProto(); @@ -964,11 +961,10 @@ private FeatureSet newDummyFeatureSet(String name, String project) { .putLabels("key", "value") .build(); Feature feature = Feature.fromProto(f1); - Entity entity = TestObjectFactory.CreateEntity("entity", Enum.STRING); + Entity entity = TestUtil.CreateEntity("entity", Enum.STRING); FeatureSet fs = - TestObjectFactory.CreateFeatureSet( - name, project, Arrays.asList(entity), Arrays.asList(feature)); + TestUtil.CreateFeatureSet(name, project, Arrays.asList(entity), Arrays.asList(feature)); fs.setCreated(Date.from(Instant.ofEpochSecond(10L))); return fs; } diff --git a/core/src/test/java/feast/core/service/TestObjectFactory.java b/core/src/test/java/feast/core/service/TestObjectFactory.java deleted file mode 100644 index 99e1cf9e9f2..00000000000 --- a/core/src/test/java/feast/core/service/TestObjectFactory.java +++ /dev/null @@ -1,89 +0,0 @@ -/* - * SPDX-License-Identifier: Apache-2.0 - * Copyright 2018-2020 The Feast Authors - * - * Licensed under the Apache License, Version 2.0 (the "License"); - * you may not use this file except in compliance with the License. - * You may obtain a copy of the License at - * - * https://www.apache.org/licenses/LICENSE-2.0 - * - * Unless required by applicable law or agreed to in writing, software - * distributed under the License is distributed on an "AS IS" BASIS, - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - * See the License for the specific language governing permissions and - * limitations under the License. - */ -package feast.core.service; - -import feast.core.model.*; -import feast.proto.core.FeatureSetProto; -import feast.proto.core.SourceProto; -import feast.proto.types.ValueProto; -import java.util.HashMap; -import java.util.List; -import java.util.Map; -import java.util.UUID; - -public class TestObjectFactory { - - public static Source defaultSource = - new Source( - SourceProto.SourceType.KAFKA, - SourceProto.KafkaSourceConfig.newBuilder() - .setBootstrapServers("kafka:9092") - .setTopic("my-topic") - .build(), - true); - - public static FeatureSet CreateFeatureSet( - String name, String project, List entities, List features) { - FeatureSet fs = - new FeatureSet( - name, - project, - 100L, - entities, - features, - defaultSource, - new HashMap<>(), - FeatureSetProto.FeatureSetStatus.STATUS_READY); - fs.setVersion(1); - return fs; - } - - public static Feature CreateFeature( - String name, ValueProto.ValueType.Enum valueType, Map labels) { - return Feature.fromProto( - FeatureSetProto.FeatureSpec.newBuilder() - .setName(name) - .setValueType(valueType) - .putAllLabels(labels) - .build()); - } - - public static Feature CreateFeature(String name, ValueProto.ValueType.Enum valueType) { - return Feature.fromProto( - FeatureSetProto.FeatureSpec.newBuilder().setName(name).setValueType(valueType).build()); - } - - public static Entity CreateEntity(String name, ValueProto.ValueType.Enum valueType) { - return Entity.fromProto( - FeatureSetProto.EntitySpec.newBuilder().setName(name).setValueType(valueType).build()); - } - - public static FeatureSetJobStatus CreateFeatureSetJobStatusWithJob( - JobStatus status, FeatureSetProto.FeatureSetJobDeliveryStatus deliveryStatus, int version) { - Job job = new Job(); - job.setStatus(status); - job.setId(UUID.randomUUID().toString()); - - FeatureSetJobStatus featureSetJobStatus = new FeatureSetJobStatus(); - featureSetJobStatus.setJob(job); - - featureSetJobStatus.setDeliveryStatus(deliveryStatus); - featureSetJobStatus.setVersion(version); - - return featureSetJobStatus; - } -} diff --git a/core/src/test/java/feast/core/util/ModelHelpers.java b/core/src/test/java/feast/core/util/ModelHelpers.java deleted file mode 100644 index f864a5562de..00000000000 --- a/core/src/test/java/feast/core/util/ModelHelpers.java +++ /dev/null @@ -1,41 +0,0 @@ -/* - * SPDX-License-Identifier: Apache-2.0 - * Copyright 2018-2020 The Feast Authors - * - * Licensed under the Apache License, Version 2.0 (the "License"); - * you may not use this file except in compliance with the License. - * You may obtain a copy of the License at - * - * https://www.apache.org/licenses/LICENSE-2.0 - * - * Unless required by applicable law or agreed to in writing, software - * distributed under the License is distributed on an "AS IS" BASIS, - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - * See the License for the specific language governing permissions and - * limitations under the License. - */ -package feast.core.util; - -import feast.core.model.FeatureSet; -import feast.core.model.FeatureSetJobStatus; -import java.util.List; -import java.util.Set; -import java.util.stream.Collectors; -import java.util.stream.Stream; - -public class ModelHelpers { - public static Set makeFeatureSetJobStatus(FeatureSet... featureSets) { - return Stream.of(featureSets) - .map( - fs -> { - FeatureSetJobStatus s = new FeatureSetJobStatus(); - s.setFeatureSet(fs); - return s; - }) - .collect(Collectors.toSet()); - } - - public static Set makeFeatureSetJobStatus(List featureSets) { - return makeFeatureSetJobStatus(featureSets.toArray(FeatureSet[]::new)); - } -} diff --git a/core/src/test/java/feast/core/util/TestUtil.java b/core/src/test/java/feast/core/util/TestUtil.java new file mode 100644 index 00000000000..7a88ccdf77e --- /dev/null +++ b/core/src/test/java/feast/core/util/TestUtil.java @@ -0,0 +1,147 @@ +/* + * SPDX-License-Identifier: Apache-2.0 + * Copyright 2018-2020 The Feast Authors + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package feast.core.util; + +import feast.core.model.Entity; +import feast.core.model.Feature; +import feast.core.model.FeatureSet; +import feast.core.model.FeatureSetJobStatus; +import feast.core.model.Job; +import feast.core.model.JobStatus; +import feast.core.model.Project; +import feast.core.model.Source; +import feast.core.model.Store; +import feast.proto.core.FeatureSetProto; +import feast.proto.core.FeatureSetProto.FeatureSetMeta; +import feast.proto.core.FeatureSetProto.FeatureSetSpec; +import feast.proto.core.SourceProto; +import feast.proto.core.StoreProto; +import feast.proto.core.StoreProto.Store.RedisConfig; +import feast.proto.core.StoreProto.Store.StoreType; +import feast.proto.core.StoreProto.Store.Subscription; +import feast.proto.types.ValueProto; +import java.util.HashMap; +import java.util.List; +import java.util.Map; +import java.util.Set; +import java.util.UUID; +import java.util.stream.Collectors; +import java.util.stream.Stream; + +public class TestUtil { + public static Set makeFeatureSetJobStatus(FeatureSet... featureSets) { + return Stream.of(featureSets) + .map( + fs -> { + FeatureSetJobStatus s = new FeatureSetJobStatus(); + s.setFeatureSet(fs); + return s; + }) + .collect(Collectors.toSet()); + } + + public static Set makeFeatureSetJobStatus(List featureSets) { + return makeFeatureSetJobStatus(featureSets.toArray(FeatureSet[]::new)); + } + + public static Source createKafkaSource(String boostrapServers, String topic, boolean isDefault) { + return Source.fromProto( + SourceProto.Source.newBuilder() + .setType(SourceProto.SourceType.KAFKA) + .setKafkaSourceConfig( + SourceProto.KafkaSourceConfig.newBuilder() + .setBootstrapServers("kafka:9092") + .setTopic("my-topic") + .build()) + .build(), + true); + } + + public static Source defaultSource = createKafkaSource("kafka:9092", "topic", true); + + public static Store createStore(String name, List subscriptions) { + return Store.fromProto( + StoreProto.Store.newBuilder() + .setName(name) + .setType(StoreType.REDIS) + .setRedisConfig(RedisConfig.newBuilder().build()) + .addAllSubscriptions(subscriptions) + .build()); + } + + public static FeatureSet CreateFeatureSet( + String name, String project, List entities, List features) { + FeatureSet fs = + new FeatureSet( + name, + project, + 100L, + entities, + features, + defaultSource, + new HashMap<>(), + FeatureSetProto.FeatureSetStatus.STATUS_READY); + fs.setVersion(1); + return fs; + } + + public static FeatureSet createEmptyFeatureSet(String name, Source source) { + return FeatureSet.fromProto( + FeatureSetProto.FeatureSet.newBuilder() + .setSpec( + FeatureSetSpec.newBuilder() + .setSource(source.toProto()) + .setProject(Project.DEFAULT_NAME) + .setName(name)) + .setMeta(FeatureSetMeta.newBuilder().build()) + .build()); + } + + public static Feature CreateFeature(String name, ValueProto.ValueType.Enum valueType) { + return CreateFeature(name, valueType, Map.of()); + } + + public static Feature CreateFeature( + String name, ValueProto.ValueType.Enum valueType, Map labels) { + return Feature.fromProto( + FeatureSetProto.FeatureSpec.newBuilder() + .setName(name) + .setValueType(valueType) + .putAllLabels(labels) + .build()); + } + + public static Entity CreateEntity(String name, ValueProto.ValueType.Enum valueType) { + return Entity.fromProto( + FeatureSetProto.EntitySpec.newBuilder().setName(name).setValueType(valueType).build()); + } + + public static FeatureSetJobStatus CreateFeatureSetJobStatusWithJob( + JobStatus status, FeatureSetProto.FeatureSetJobDeliveryStatus deliveryStatus, int version) { + Job job = new Job(); + job.setStatus(status); + job.setId(UUID.randomUUID().toString()); + + FeatureSetJobStatus featureSetJobStatus = new FeatureSetJobStatus(); + featureSetJobStatus.setJob(job); + + featureSetJobStatus.setDeliveryStatus(deliveryStatus); + featureSetJobStatus.setVersion(version); + + return featureSetJobStatus; + } +} diff --git a/pom.xml b/pom.xml index 4d1be1f2a6c..38ff13cf71d 100644 --- a/pom.xml +++ b/pom.xml @@ -227,7 +227,7 @@ org.projectlombok lombok - 1.18.2 + 1.18.12 provided diff --git a/sdk/python/feast/cli.py b/sdk/python/feast/cli.py index 68fee30c9b1..9eb356034a8 100644 --- a/sdk/python/feast/cli.py +++ b/sdk/python/feast/cli.py @@ -18,8 +18,8 @@ import click import pkg_resources - import yaml + from feast.client import Client from feast.config import Config from feast.core.IngestionJob_pb2 import IngestionJobStatus diff --git a/sdk/python/feast/client.py b/sdk/python/feast/client.py index 8d85f07cd33..ff6cf91b3b5 100644 --- a/sdk/python/feast/client.py +++ b/sdk/python/feast/client.py @@ -26,10 +26,10 @@ import grpc import pandas as pd -from google.protobuf.timestamp_pb2 import Timestamp - import pyarrow as pa import pyarrow.parquet as pq +from google.protobuf.timestamp_pb2 import Timestamp + from feast.config import Config from feast.constants import ( CONFIG_CORE_SECURE_KEY, diff --git a/sdk/python/feast/feature_set.py b/sdk/python/feast/feature_set.py index f4397f6cbd5..ee06463bc6f 100644 --- a/sdk/python/feast/feature_set.py +++ b/sdk/python/feast/feature_set.py @@ -16,14 +16,15 @@ from typing import Dict, List, MutableMapping, Optional import pandas as pd +import pyarrow as pa +import yaml from google.protobuf import json_format from google.protobuf.duration_pb2 import Duration from google.protobuf.json_format import MessageToDict, MessageToJson from google.protobuf.message import Message from pandas.api.types import is_datetime64_ns_dtype +from pyarrow.lib import TimestampType -import pyarrow as pa -import yaml from feast.core.FeatureSet_pb2 import FeatureSet as FeatureSetProto from feast.core.FeatureSet_pb2 import FeatureSetMeta as FeatureSetMetaProto from feast.core.FeatureSet_pb2 import FeatureSetSpec as FeatureSetSpecProto @@ -39,7 +40,6 @@ pa_to_feast_value_type, python_type_to_feast_value_type, ) -from pyarrow.lib import TimestampType from tensorflow_metadata.proto.v0 import schema_pb2 diff --git a/sdk/python/feast/loaders/ingest.py b/sdk/python/feast/loaders/ingest.py index 5e8e5da4d41..b439dbd3027 100644 --- a/sdk/python/feast/loaders/ingest.py +++ b/sdk/python/feast/loaders/ingest.py @@ -4,8 +4,8 @@ from typing import Iterable, List import pandas as pd - import pyarrow.parquet as pq + from feast.constants import DATETIME_COLUMN from feast.feature_set import FeatureSet from feast.type_map import ( diff --git a/sdk/python/feast/type_map.py b/sdk/python/feast/type_map.py index 7a483a6c2a0..85def25fcb9 100644 --- a/sdk/python/feast/type_map.py +++ b/sdk/python/feast/type_map.py @@ -17,9 +17,10 @@ import numpy as np import pandas as pd +import pyarrow as pa from google.protobuf.timestamp_pb2 import Timestamp +from pyarrow.lib import TimestampType -import pyarrow as pa from feast.constants import DATETIME_COLUMN from feast.types import FeatureRow_pb2 as FeatureRowProto from feast.types import Field_pb2 as FieldProto @@ -35,7 +36,6 @@ from feast.types.Value_pb2 import Value as ProtoValue from feast.types.Value_pb2 import ValueType as ProtoValueType from feast.value_type import ValueType -from pyarrow.lib import TimestampType def python_type_to_feast_value_type( diff --git a/tests/e2e/redis/basic-ingest-redis-serving.py b/tests/e2e/redis/basic-ingest-redis-serving.py index a6183b2dbc4..778323c9cb5 100644 --- a/tests/e2e/redis/basic-ingest-redis-serving.py +++ b/tests/e2e/redis/basic-ingest-redis-serving.py @@ -5,6 +5,7 @@ import grpc from collections import OrderedDict from feast.entity import Entity +from feast.source import KafkaSource from feast.serving.ServingService_pb2 import ( GetOnlineFeaturesRequest, GetOnlineFeaturesResponse, @@ -12,7 +13,7 @@ from feast.core.IngestionJob_pb2 import IngestionJobStatus from feast.core.CoreService_pb2_grpc import CoreServiceStub from feast.core import CoreService_pb2 -from feast.types.Value_pb2 import Value as Value +from feast.types.Value_pb2 import Value from feast.client import Client from feast.feature_set import FeatureSet, FeatureSetRef from feast.type_map import ValueType @@ -741,6 +742,50 @@ def test_list_entities_and_features(client): assert set(filter_by_project_entity_expected) == set(filter_by_project_entity_actual) assert set(filter_by_project_labels_expected) == set(filter_by_project_labels_actual) +@pytest.mark.timeout(900) +@pytest.mark.run(order=50) +def test_sources_deduplicate_ingest_jobs(client): + source = KafkaSource("localhost:9092", "feast-features") + alt_source = KafkaSource("localhost:9092", "feast-data") + + def get_running_jobs(): + return [ job for job in client.list_ingest_jobs() if job.status == IngestionJobStatus.RUNNING ] + + # stop all ingest jobs + ingest_jobs = client.list_ingest_jobs() + for ingest_job in ingest_jobs: + client.stop_ingest_job(ingest_job) + for ingest_job in ingest_jobs: + ingest_job.wait(IngestionJobStatus.ABORTED) + + # register multiple featuresets with the same source + # only one ingest job should spawned due to test ingest job deduplication + cust_trans_fs = FeatureSet.from_yaml(f"{DIR_PATH}/basic/cust_trans_fs.yaml") + driver_fs = FeatureSet.from_yaml(f"{DIR_PATH}/basic/driver_fs.yaml") + cust_trans_fs.source, driver_fs.source = source, source + client.apply(cust_trans_fs) + client.apply(driver_fs) + + while len(get_running_jobs()) != 1: + assert 0 <= len(get_running_jobs()) <= 1 + time.sleep(1) + + # update feature sets with different sources, should spawn 2 ingest jobs + driver_fs.source = alt_source + client.apply(driver_fs) + + while len(get_running_jobs()) != 2: + assert 1 <= len(get_running_jobs()) <= 2 + time.sleep(1) + + # update feature sets with same source again, should spawn only 1 ingest job + driver_fs.source = source + client.apply(driver_fs) + + while len(get_running_jobs()) != 1: + assert 1 <= len(get_running_jobs()) <= 2 + time.sleep(1) + # TODO: rewrite these using python SDK once the labels are implemented there class TestsBasedOnGrpc: From fbb7a022aaa735eb5ffee1fad4299653d6066251 Mon Sep 17 00:00:00 2001 From: David Heryanto Date: Fri, 19 Jun 2020 14:11:43 +0800 Subject: [PATCH 071/133] Fix broken link in helm installation docs (#808) --- infra/charts/feast/README.md | 9 +++++++-- infra/charts/feast/README.md.gotmpl | 9 +++++++-- 2 files changed, 14 insertions(+), 4 deletions(-) diff --git a/infra/charts/feast/README.md b/infra/charts/feast/README.md index 3d868b2fd13..c6d75c392f2 100644 --- a/infra/charts/feast/README.md +++ b/infra/charts/feast/README.md @@ -116,8 +116,13 @@ training, access to BigQuery is required. First, create a [service account](http will provide the credentials to access BigQuery. Grant the service account `editor` role so it has write permissions to BigQuery and Cloud Storage. -> In production, it is advised to give only the required [permissions](foo-feast-batch-serving-test) for the -> the service account, versus `editor` role which is very permissive. +> In production, it is advised to grant these specific roles, versus `editor` +> role which is very permissive: +> - **Dataflow Admin**: Permissions to create and manage Dataflow jobs +> - **Service Account User**: Permissions to set a service account on Dataflow workers +> - **Storage Admin**: Permissions to write files into Google Cloud Storage +> - **BigQuery Data Editor**: Permissions to write data into BigQuery +> - **BigQuery Job User**: Permisssions to run query in BigQuery Create a Kubernetes secret for the service account JSON file: ```bash diff --git a/infra/charts/feast/README.md.gotmpl b/infra/charts/feast/README.md.gotmpl index 69d40fbb25e..75fc6661c80 100644 --- a/infra/charts/feast/README.md.gotmpl +++ b/infra/charts/feast/README.md.gotmpl @@ -91,8 +91,13 @@ training, access to BigQuery is required. First, create a [service account](http will provide the credentials to access BigQuery. Grant the service account `editor` role so it has write permissions to BigQuery and Cloud Storage. -> In production, it is advised to give only the required [permissions](foo-feast-batch-serving-test) for the -> the service account, versus `editor` role which is very permissive. +> In production, it is advised to grant these specific roles, versus `editor` +> role which is very permissive: +> - **Dataflow Admin**: Permissions to create and manage Dataflow jobs +> - **Service Account User**: Permissions to set a service account on Dataflow workers +> - **Storage Admin**: Permissions to write files into Google Cloud Storage +> - **BigQuery Data Editor**: Permissions to write data into BigQuery +> - **BigQuery Job User**: Permisssions to run query in BigQuery Create a Kubernetes secret for the service account JSON file: ```bash From eebf4587ad2ae0cfca00f7def888f8ff10396e5f Mon Sep 17 00:00:00 2001 From: Zhu Zhan Yan Date: Fri, 19 Jun 2020 14:45:05 +0800 Subject: [PATCH 072/133] Add Field Status metadata to Online Serving (#658) * Update ServingService protobuf with additions to support online metadata retrieval. Additions are outlined in detail in this RFC: https://docs.google.com/document/d/1VQngwBcx-yWgGpAbsFVdth9GnjL8q-ZgUNBGv57R0Fk/ * Change error thrown by sendMultiGet() to Unknown as it describes the fault more accurately * Return boolean primitive instead of Boolean object type in FeatureRowDecoder This removes the unneeded possiblity of a null being returning from the the decoder. * Config sendAndProcessMultiGet() to throw an exception on feature row decoding failure. Instead of the current behaviour to silently fail with and return an empty feature row. * Refactor sendMultiGetAndProcess() to getFeaturesForFeatureSet(). Rename as the original name described the implementation instead what the goal of the method is. Construct and pass the decoder outside method in getFeaturesForFeatureSet() instead of inside method in sendMultiGetAndProcess() as constructing it inside added uncessary arguments and code (ie getting the string ref out a redis key seemed out of place.) Changed getFeaturesForFeatureSet() to return null instead of a nullFeatureRow. when retrieving data from redis using sendMultiGet() using a key missing in Redis. This makes the missing key condition easier to detect. Removed unncessary arguments passed to sendMultiGetAndProcess() - featureReference used to contruct the nullFeatureRow - featureSetSpec used to contruct the decoder. * Update ResponseJSONMapper to use new GetOnlineFeatures protobuf * Remove references to FieldValues in OnlineServingService & Tests * Update java sdk's FieldClient to use the new GetOnlineFeatures protobuf * Update OnlineRetriever to return null on missing retrieval (such as missing key for redis). * Refactor out unnecessary nesting of returned list in OnlineRetriever's getOnlineFeatures() This refactor is done to improve code readablity as it is difficult to conceptualise what each nesting of the list corresponds to: - developers have to keep in mind that the outer nest corresponds to faetures requests and the inner nesting corresponds to entity rows. The nesting is unnecessary as: - inside RedisOnlineRetriever we have a loop over feature set requests. - outside in OnlineServingService where this is used, we also loop over feature set requests. * Add online feature metadata to OnlineServingService's getOnlineFeatures() Updated getOnlineFeatures() to return metadata on each response Field. - Added sanity check with exception to ensure that the feature rows retrieved from the OnlineRetriever matches the no. of entity rows pass to it. Refactor code out of getOnlineFeatures() to smaller methods to reduce method size and improve readabilty: - added unpackFields(entityRow) to unpack EntityRows into response Fields - added unpackFields(featureRow) to unpack FeatureRows into response Fields - added attachMetadata(field) to add metadata to response Fields. - moved loop over feature references into populateStaleKeyCountMetrics() * Update OnlineServingServiceTest to check that online metadata is set correctly. * Added online metadata support java sdk's FeastClient. * Changed go sdk's Row to store a map of fields instead of just Values * Update response.go to use updated Row and take into account FieldStatus * Added Field() to go sdk's types.go to construct fields from a Value. * Update go sdk's request.go to use updated Row and Add GetOnlineFeatures flags Added missing boolean flags for GetOnlineFeatures's IncludeMetadataInResponse and OmitEntitiesInResponse flags. * Update go protobuf generated code for updated protobuf defintion * Update python sdk's client to document new fieldstatus metadata, added flags Added missing boolean flags for GetOnlineFeaturesRequest's omit_entities_in_response & include_metadata_in_response flags. Update unit tests with changes to GetOnlineFeaturesResponse protobuff * Fixed issue where getOnlineFeatures() returned Record in a non deterministic order. Enforce that getOnlineFeatures() returns Records in the same order as the user provides the EntityRow * Update e2e ingestion tests to support new get_online_features api and check for metadata * Fixed NullPointerException when feature row is null. * Fixed issue in which unpackFields(entityRow) was not responding to includeMetadata flag * Check FieldStatus in end to end tests to wait for ingestion of values to complete. * Moved test checks out of polling loop in e2e online ingestion tests * Apply spotless java formatting * Fixed misplaced break in end to end tests * Fixed typos in e2e ingestion tests. * Apply changes on RedisOnlineRetriever to RedisClusterOnlineRetriever Changes * Removed unnessary nesting of list in FeaturRows returned * Refactored sendMultiGetAndProcess() to getOnlineFeaturesForFeatureSet() Necessary due to code duplciation between RedisOnlineRetriever and RedisOnlineRetriever. * Rename getFeaturesForFeatureSet() to getFeaturesFromRedis() for better sementics * Update Serving application.yml to include example on how to config RedisCluster * Correct missing include_meta flag in e2e tests get_online_features() calls * Correct minor comment typo in serving's application.yml * Correct another minor typo in e2e test: missing .status * Fix Feast core and serving compilation/test failures after rebase * Fix go SDK tests after rebase * Update ServingService protobuf to include metadata without breaking changes. * Revert Feast Serving's ServingService & ResponseJSONMapper to FieldValues * Revert serving's OnlineServingService to backwards compatible FieldValues API * Revert Java SDK's FeastClient to backwards compatible FieldStatus API. * Revert Go SDK's client.go to backwards compatible FieldStatus API. * Fix Java SDK unit tests * Added .Statuses() method to GO Sdk's OnlineFeaturesResponse to get metadata * Fixed Java SDK not stripping project from field values and statuses * Fixed Go SDK's client.go not stripping project from field values and statuses * Fixed Go SDK's Unit tests * Revert Python SDK to backward compatible FieldValues API & fix tests * Move hardcoded constants from job.py to constants.py in Python SDK to consolidate constants * Added wait backoff implementation in Python SDK: wait_retry_backoff() * Update Python SDK's job.py to use shared backoff wait implemetation * Clarify that maximum range in ServingService proto to max age * Fixed online serving e2e tests * Fix python lint. * Collect entity refs in Python SDK as a single line. * Use guard style if statements in Go Sdk's types.go * Make strip field values part of Python SDK's get_online_features() more functional * Fix go unit tests * Remove include_metadata_in_response flag from ServingService proto as its redundant. * Readd missing span log call to log retrieved feature rows. * Use more modern Streams.zip() to iterate entity rows and feature rows in OnlineServingService * Change utility methods in OnlineServingService to static. * Use "outside max age" instead "stale" term in OnlineServingService as "stale" is a overloaded term. * Refactor serving's OnlineRetriever to return a empty Optional instead of returning null. This allows the code to more semenatic and readable compared to dealing with nullable FeatureRow values * Fixed typo in basic redis e2e tests * Fixed typo OnlineServingService's logFeatureRowsTrace() Co-authored-by: Zhu Zhanyan --- go.mod | 3 +- go.sum | 8 + protos/feast/serving/ServingService.proto | 47 +- sdk/go/client.go | 18 +- sdk/go/client_test.go | 13 + sdk/go/protos/feast/core/CoreService.pb.go | 546 +++++++++++++----- sdk/go/protos/feast/core/Runner.pb.go | 130 +++-- sdk/go/protos/feast/core/Store.pb.go | 87 +-- .../protos/feast/serving/ServingService.pb.go | 326 +++++++---- sdk/go/request.go | 15 +- sdk/go/response.go | 34 +- sdk/go/response_test.go | 30 + sdk/go/types.go | 2 +- .../java/com/gojek/feast/FeastClient.java | 28 +- .../src/main/java/com/gojek/feast/Row.java | 17 + .../java/com/gojek/feast/FeastClientTest.java | 38 +- sdk/python/feast/client.py | 44 +- sdk/python/feast/constants.py | 5 + sdk/python/feast/job.py | 61 +- sdk/python/feast/wait.py | 56 ++ sdk/python/tests/test_client.py | 21 +- .../serving/service/OnlineServingService.java | 308 +++++++--- .../util/mappers/ResponseJSONMapper.java | 2 +- serving/src/main/resources/application.yml | 15 +- .../service/OnlineServingServiceTest.java | 357 ++++++++---- .../api/retriever/OnlineRetriever.java | 27 +- .../redis/retriever/FeatureRowDecoder.java | 4 +- .../RedisClusterOnlineRetriever.java | 113 ++-- .../redis/retriever/RedisOnlineRetriever.java | 111 ++-- .../RedisClusterOnlineRetrieverTest.java | 37 +- .../retriever/RedisOnlineRetrieverTest.java | 38 +- tests/e2e/redis/basic-ingest-redis-serving.py | 256 ++++---- tests/e2e/redis/basic/cust_trans_fs.yaml | 2 + 33 files changed, 1825 insertions(+), 974 deletions(-) create mode 100644 sdk/python/feast/wait.py diff --git a/go.mod b/go.mod index d5917cb33e3..f116fffa831 100644 --- a/go.mod +++ b/go.mod @@ -25,8 +25,7 @@ require ( golang.org/x/lint v0.0.0-20200302205851-738671d3881b // indirect golang.org/x/net v0.0.0-20200513185701-a91f0712d120 golang.org/x/sys v0.0.0-20200515095857-1151b9dac4a9 // indirect - golang.org/x/tools v0.0.0-20200521211927-2b542361a4fc // indirect - google.golang.org/genproto v0.0.0-20200515170657-fc4c6c6a6587 // indirect + golang.org/x/tools v0.0.0-20200604042327-9b20fe4cabe8 // indirect google.golang.org/grpc v1.29.1 google.golang.org/protobuf v1.24.0 // indirect gopkg.in/russross/blackfriday.v2 v2.0.0 // indirect diff --git a/go.sum b/go.sum index 90a517da7e4..4be66277bcf 100644 --- a/go.sum +++ b/go.sum @@ -162,6 +162,7 @@ github.com/google/go-cmp v0.3.0 h1:crn/baboCvb5fXaQ0IJ1SGTsTVrWpDsCWC8EGETZijY= github.com/google/go-cmp v0.3.0/go.mod h1:8QqcDgzrUqlUb/G2PQTWiueGozuR1884gddMywk6iLU= github.com/google/go-cmp v0.3.1 h1:Xye71clBPdm5HgqGwUkwhbynsUJZhDbS20FvLhQ2izg= github.com/google/go-cmp v0.3.1/go.mod h1:8QqcDgzrUqlUb/G2PQTWiueGozuR1884gddMywk6iLU= +github.com/google/go-cmp v0.4.0 h1:xsAVV57WRhGj6kEIi8ReJzQlHHqcBYCElAvkovg3B/4= github.com/google/go-cmp v0.4.0/go.mod h1:v8dTdLbMG2kIc/vJvl+f65V22dbkXbowE6jgT/gNBxE= github.com/google/gofuzz v0.0.0-20161122191042-44d81051d367/go.mod h1:HP5RmnzzSNb993RKQDq4+1A4ia9nllfqcQFTQJedwGI= github.com/google/gofuzz v0.0.0-20170612174753-24818f796faf/go.mod h1:HP5RmnzzSNb993RKQDq4+1A4ia9nllfqcQFTQJedwGI= @@ -319,6 +320,7 @@ github.com/stretchr/testify v1.2.0/go.mod h1:a8OnRcib4nhh0OaRAV+Yts87kKdq0PP7pXf github.com/stretchr/testify v1.2.2 h1:bSDNvY7ZPG5RlJ8otE/7V6gMiyenm9RtJ7IUVIAoJ1w= github.com/stretchr/testify v1.2.2/go.mod h1:a8OnRcib4nhh0OaRAV+Yts87kKdq0PP7pXfy6kDkUVs= github.com/stretchr/testify v1.3.0/go.mod h1:M5WIy9Dh21IEIfnGCwXGc5bZfKNJtfHm1UVUgZn+9EI= +github.com/stretchr/testify v1.4.0 h1:2E4SXV/wtOkTonXsotYi4li6zVWxYlZuYNCXe9XRJyk= github.com/stretchr/testify v1.4.0/go.mod h1:j7eGeouHqKxXV5pUuKE4zz7dFj8WfuZ+81PSLYec5m4= github.com/tmc/grpc-websocket-proxy v0.0.0-20170815181823-89b8d40f7ca8/go.mod h1:ncp9v5uamzpCO7NfCPTXjqaC+bZgJeR0sMTm6dMHP7U= github.com/tmc/grpc-websocket-proxy v0.0.0-20190109142713-0ad062ec5ee5/go.mod h1:ncp9v5uamzpCO7NfCPTXjqaC+bZgJeR0sMTm6dMHP7U= @@ -466,8 +468,12 @@ golang.org/x/tools v0.0.0-20200519205726-57a9e4404bf7 h1:nm4zDh9WvH4jiuUpMY5RUsv golang.org/x/tools v0.0.0-20200519205726-57a9e4404bf7/go.mod h1:EkVYQZoAsY45+roYkvgYkIh4xh/qjgUK9TdY2XT94GE= golang.org/x/tools v0.0.0-20200521211927-2b542361a4fc h1:6m2YO+AmBApbUOmhsghW+IfRyZOY4My4UYvQQrEpHfY= golang.org/x/tools v0.0.0-20200521211927-2b542361a4fc/go.mod h1:EkVYQZoAsY45+roYkvgYkIh4xh/qjgUK9TdY2XT94GE= +golang.org/x/tools v0.0.0-20200601175630-2caf76543d99/go.mod h1:EkVYQZoAsY45+roYkvgYkIh4xh/qjgUK9TdY2XT94GE= +golang.org/x/tools v0.0.0-20200604042327-9b20fe4cabe8 h1:8Xr1qwxn90MXYKftwNxIO2g4J+26naghxFS5rYiTZww= +golang.org/x/tools v0.0.0-20200604042327-9b20fe4cabe8/go.mod h1:EkVYQZoAsY45+roYkvgYkIh4xh/qjgUK9TdY2XT94GE= golang.org/x/xerrors v0.0.0-20190717185122-a985d3407aa7/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0= golang.org/x/xerrors v0.0.0-20191011141410-1b5146add898/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0= +golang.org/x/xerrors v0.0.0-20191204190536-9bdfabe68543 h1:E7g+9GITq07hpfrRu66IVDexMakfv52eLZ2CXBWiKr4= golang.org/x/xerrors v0.0.0-20191204190536-9bdfabe68543/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0= gonum.org/v1/gonum v0.0.0-20190331200053-3d26580ed485/go.mod h1:2ltnJ7xHfj0zHS40VVPYEAAMTa3ZGguvHGBSJeRWqE0= gonum.org/v1/netlib v0.0.0-20190313105609-8cb42192e0e0/go.mod h1:wa6Ws7BG/ESfp6dHfk7C6KdzKA7wR7u/rKwOGE66zvw= @@ -491,6 +497,7 @@ google.golang.org/genproto v0.0.0-20200319113533-08878b785e9c h1:5aI3/f/3eCZps9x google.golang.org/genproto v0.0.0-20200319113533-08878b785e9c/go.mod h1:55QSHmfGQM9UVYDPBsyGGes0y52j32PQ3BqQfXhyH3c= google.golang.org/genproto v0.0.0-20200515170657-fc4c6c6a6587 h1:1Ym+vvUpq1ZHvxzn34gENJX8U4aKO+vhy2P/2+Xl6qQ= google.golang.org/genproto v0.0.0-20200515170657-fc4c6c6a6587/go.mod h1:YsZOwe1myG/8QRHRsmBRE1LrgQY60beZKjly0O1fX9U= +google.golang.org/genproto v0.0.0-20200526211855-cb27e3aa2013 h1:+kGHl1aib/qcwaRi1CbqBZ1rk19r85MNUf8HaBghugY= google.golang.org/genproto v0.0.0-20200526211855-cb27e3aa2013/go.mod h1:NbSheEEYHJ7i3ixzK3sjbqSGDJWnxyFXZblF3eUsNvo= google.golang.org/grpc v1.13.0/go.mod h1:yo6s7OP7yaDglbqo1J04qKzAhqBH6lvTonzMVmEdcZw= google.golang.org/grpc v1.19.0/go.mod h1:mqu4LbDTu4XGKhr4mRzUsmM4RtVoemTSY81AxZiDr8c= @@ -540,6 +547,7 @@ gopkg.in/yaml.v2 v2.0.0-20170812160011-eb3733d160e7/go.mod h1:JAlM8MvJe8wmxCU4Bl gopkg.in/yaml.v2 v2.2.1/go.mod h1:hI93XBmqTisBFMUTm0b8Fm+jr3Dg1NNxqwp+5A1VGuI= gopkg.in/yaml.v2 v2.2.2 h1:ZCJp+EgiOT7lHqUV2J862kp8Qj64Jo6az82+3Td9dZw= gopkg.in/yaml.v2 v2.2.2/go.mod h1:hI93XBmqTisBFMUTm0b8Fm+jr3Dg1NNxqwp+5A1VGuI= +gopkg.in/yaml.v2 v2.2.4 h1:/eiJrUcujPVeJ3xlSWaiNi3uSVmDGBK1pDHUHAnao1I= gopkg.in/yaml.v2 v2.2.4/go.mod h1:hI93XBmqTisBFMUTm0b8Fm+jr3Dg1NNxqwp+5A1VGuI= gotest.tools v2.2.0+incompatible/go.mod h1:DsYFclhRJ6vuDpmuTbkuFWG+y2sxOXAzmJt81HFBacw= honnef.co/go/tools v0.0.0-20190102054323-c2f93a96b099/go.mod h1:rf3lG4BRIbNafJWhAfAdb/ePZxsR/4RtNHQocxwk9r4= diff --git a/protos/feast/serving/ServingService.proto b/protos/feast/serving/ServingService.proto index cd7d51bd59c..8e36d459d66 100644 --- a/protos/feast/serving/ServingService.proto +++ b/protos/feast/serving/ServingService.proto @@ -91,8 +91,8 @@ message GetOnlineFeaturesRequest { bool omit_entities_in_response = 3; message EntityRow { - // Request timestamp of this row. This value will be used, together with maxAge, - // to determine feature staleness. + // Request timestamp of this row. This value will be used, + // together with maxAge, to determine feature staleness. google.protobuf.Timestamp entity_timestamp = 1; // Map containing mapping of entity name to entity value. @@ -100,15 +100,6 @@ message GetOnlineFeaturesRequest { } } -message GetBatchFeaturesRequest { - // List of features that are being retrieved - repeated FeatureReference features = 3; - - // Source of the entity dataset containing the timestamps and entity keys to retrieve - // features for. - DatasetSource dataset_source = 2; -} - message GetOnlineFeaturesResponse { // Feature values retrieved from feast. repeated FieldValues field_values = 1; @@ -116,10 +107,42 @@ message GetOnlineFeaturesResponse { message FieldValues { // Map of feature or entity name to feature/entity values. // Timestamps are not returned in this response. - map fields = 1; + map fields = 1; + // Map of feature or entity name to feature/entity statuses/metadata. + map statuses = 2; + } + + enum FieldStatus { + // Status is unset for this field. + INVALID = 0; + + // Field value is present for this field and age is within max age. + PRESENT = 1; + + // Values could be found for entity key and age is within max age, but + // this field value is assigned a value on ingestion into feast. + NULL_VALUE = 2; + + // Entity key did not return any values as they do not exist in Feast. + // This could suggest that the feature values have not yet been ingested + // into feast or the ingestion failed. + NOT_FOUND = 3; + + // Values could be found for entity key, but field values are outside the maximum + // allowable range. + OUTSIDE_MAX_AGE = 4; } } +message GetBatchFeaturesRequest { + // List of features that are being retrieved + repeated FeatureReference features = 3; + + // Source of the entity dataset containing the timestamps and entity keys to retrieve + // features for. + DatasetSource dataset_source = 2; +} + message GetBatchFeaturesResponse { Job job = 1; } diff --git a/sdk/go/client.go b/sdk/go/client.go index 30b3dc4bf09..617c37a6f10 100644 --- a/sdk/go/client.go +++ b/sdk/go/client.go @@ -59,21 +59,25 @@ func (fc *GrpcClient) GetOnlineFeatures(ctx context.Context, req *OnlineFeatures } } - // strip projects from to projects - for _, fieldValue := range resp.GetFieldValues() { - stripFields := make(map[string]*types.Value, len(fieldValue.Fields)) - for refStr, value := range fieldValue.Fields { + // strip projects from feature refs recieved + for _, fieldValues := range resp.GetFieldValues() { + stripFields := make(map[string]*types.Value, len(fieldValues.Fields)) + stripStatuses := make(map[string]serving.GetOnlineFeaturesResponse_FieldStatus, len(fieldValues.Statuses)) + for refStr, _ := range fieldValues.Fields { _, isEntity := entityRefs[refStr] + stripRefStr := refStr if !isEntity { // is feature ref featureRef, err := parseFeatureRef(refStr, true) if err != nil { return nil, err } - refStr = toFeatureRefStr(featureRef) + stripRefStr = toFeatureRefStr(featureRef) } - stripFields[refStr] = value + + stripFields[stripRefStr] = fieldValues.Fields[refStr] + stripStatuses[stripRefStr] = fieldValues.Statuses[refStr] } - fieldValue.Fields = stripFields + fieldValues.Fields, fieldValues.Statuses = stripFields, stripStatuses } return &OnlineFeaturesResponse{RawResponse: resp}, err diff --git a/sdk/go/client_test.go b/sdk/go/client_test.go index 1ad6629e9af..49fbe83266c 100644 --- a/sdk/go/client_test.go +++ b/sdk/go/client_test.go @@ -27,6 +27,7 @@ func TestGetOnlineFeatures(t *testing.T) { Features: []string{ "driver:rating", "rating", + "null_value", }, Entities: []Row{ {"driver_id": Int64Val(1)}, @@ -41,6 +42,12 @@ func TestGetOnlineFeatures(t *testing.T) { Fields: map[string]*types.Value{ "driver_project/driver:rating": Int64Val(1), "driver_project/rating": Int64Val(1), + "driver_project/null_value": {}, + }, + Statuses: map[string]serving.GetOnlineFeaturesResponse_FieldStatus{ + "driver_project/driver:rating": serving.GetOnlineFeaturesResponse_PRESENT, + "driver_project/rating": serving.GetOnlineFeaturesResponse_PRESENT, + "driver_project/null_value": serving.GetOnlineFeaturesResponse_NULL_VALUE, }, }, }, @@ -53,6 +60,12 @@ func TestGetOnlineFeatures(t *testing.T) { Fields: map[string]*types.Value{ "driver:rating": Int64Val(1), "rating": Int64Val(1), + "null_value": {}, + }, + Statuses: map[string]serving.GetOnlineFeaturesResponse_FieldStatus{ + "driver:rating": serving.GetOnlineFeaturesResponse_PRESENT, + "rating": serving.GetOnlineFeaturesResponse_PRESENT, + "null_value": serving.GetOnlineFeaturesResponse_NULL_VALUE, }, }, }, diff --git a/sdk/go/protos/feast/core/CoreService.pb.go b/sdk/go/protos/feast/core/CoreService.pb.go index 4f22ae4c373..a6d1dee6651 100644 --- a/sdk/go/protos/feast/core/CoreService.pb.go +++ b/sdk/go/protos/feast/core/CoreService.pb.go @@ -24,7 +24,9 @@ package core import ( context "context" + v0 "github.com/feast-dev/feast/sdk/go/protos/tensorflow_metadata/proto/v0" proto "github.com/golang/protobuf/proto" + timestamp "github.com/golang/protobuf/ptypes/timestamp" grpc "google.golang.org/grpc" codes "google.golang.org/grpc/codes" status "google.golang.org/grpc/status" @@ -1273,6 +1275,168 @@ func (*StopIngestionJobResponse) Descriptor() ([]byte, []int) { return file_feast_core_CoreService_proto_rawDescGZIP(), []int{23} } +type GetFeatureStatisticsRequest struct { + state protoimpl.MessageState + sizeCache protoimpl.SizeCache + unknownFields protoimpl.UnknownFields + + // Feature set to retrieve the statistics for. A fully qualified feature set + // id in the format of project/feature_set must be provided. + FeatureSetId string `protobuf:"bytes,1,opt,name=feature_set_id,json=featureSetId,proto3" json:"feature_set_id,omitempty"` + // Optional filter which filters returned statistics by selected features. These + // features must be present in the data that is being processed. + Features []string `protobuf:"bytes,2,rep,name=features,proto3" json:"features,omitempty"` + // Optional filter to select store over which the statistics will retrieved. + // Only historical stores are allowed. + Store string `protobuf:"bytes,3,opt,name=store,proto3" json:"store,omitempty"` + // Optional start and end dates over which to filter statistical data + // Start date is inclusive, but end date is not. + // Only dates are supported, not times. + // Cannot be used with dataset_ids. + // If this period spans multiple days, unaggregatable statistics will be dropped. + StartDate *timestamp.Timestamp `protobuf:"bytes,4,opt,name=start_date,json=startDate,proto3" json:"start_date,omitempty"` + EndDate *timestamp.Timestamp `protobuf:"bytes,5,opt,name=end_date,json=endDate,proto3" json:"end_date,omitempty"` + // Optional list of ingestion Ids by which to filter data before + // retrieving statistics. + // Cannot be used with the date ranges + // If multiple dataset ids are provided, unaggregatable statistics will be dropped. + IngestionIds []string `protobuf:"bytes,6,rep,name=ingestion_ids,json=ingestionIds,proto3" json:"ingestion_ids,omitempty"` + // Setting this flag to true will force a recalculation of statistics and overwrite results currently in the + // cache, if any. + ForceRefresh bool `protobuf:"varint,7,opt,name=force_refresh,json=forceRefresh,proto3" json:"force_refresh,omitempty"` +} + +func (x *GetFeatureStatisticsRequest) Reset() { + *x = GetFeatureStatisticsRequest{} + if protoimpl.UnsafeEnabled { + mi := &file_feast_core_CoreService_proto_msgTypes[24] + ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x)) + ms.StoreMessageInfo(mi) + } +} + +func (x *GetFeatureStatisticsRequest) String() string { + return protoimpl.X.MessageStringOf(x) +} + +func (*GetFeatureStatisticsRequest) ProtoMessage() {} + +func (x *GetFeatureStatisticsRequest) ProtoReflect() protoreflect.Message { + mi := &file_feast_core_CoreService_proto_msgTypes[24] + if protoimpl.UnsafeEnabled && x != nil { + ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x)) + if ms.LoadMessageInfo() == nil { + ms.StoreMessageInfo(mi) + } + return ms + } + return mi.MessageOf(x) +} + +// Deprecated: Use GetFeatureStatisticsRequest.ProtoReflect.Descriptor instead. +func (*GetFeatureStatisticsRequest) Descriptor() ([]byte, []int) { + return file_feast_core_CoreService_proto_rawDescGZIP(), []int{24} +} + +func (x *GetFeatureStatisticsRequest) GetFeatureSetId() string { + if x != nil { + return x.FeatureSetId + } + return "" +} + +func (x *GetFeatureStatisticsRequest) GetFeatures() []string { + if x != nil { + return x.Features + } + return nil +} + +func (x *GetFeatureStatisticsRequest) GetStore() string { + if x != nil { + return x.Store + } + return "" +} + +func (x *GetFeatureStatisticsRequest) GetStartDate() *timestamp.Timestamp { + if x != nil { + return x.StartDate + } + return nil +} + +func (x *GetFeatureStatisticsRequest) GetEndDate() *timestamp.Timestamp { + if x != nil { + return x.EndDate + } + return nil +} + +func (x *GetFeatureStatisticsRequest) GetIngestionIds() []string { + if x != nil { + return x.IngestionIds + } + return nil +} + +func (x *GetFeatureStatisticsRequest) GetForceRefresh() bool { + if x != nil { + return x.ForceRefresh + } + return false +} + +type GetFeatureStatisticsResponse struct { + state protoimpl.MessageState + sizeCache protoimpl.SizeCache + unknownFields protoimpl.UnknownFields + + // Contains statistics for the requested data. + // Due to the limitations of TFDV and Facets, only a single dataset can be returned in, + // despite the message being of list type. + DatasetFeatureStatisticsList *v0.DatasetFeatureStatisticsList `protobuf:"bytes,1,opt,name=dataset_feature_statistics_list,json=datasetFeatureStatisticsList,proto3" json:"dataset_feature_statistics_list,omitempty"` +} + +func (x *GetFeatureStatisticsResponse) Reset() { + *x = GetFeatureStatisticsResponse{} + if protoimpl.UnsafeEnabled { + mi := &file_feast_core_CoreService_proto_msgTypes[25] + ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x)) + ms.StoreMessageInfo(mi) + } +} + +func (x *GetFeatureStatisticsResponse) String() string { + return protoimpl.X.MessageStringOf(x) +} + +func (*GetFeatureStatisticsResponse) ProtoMessage() {} + +func (x *GetFeatureStatisticsResponse) ProtoReflect() protoreflect.Message { + mi := &file_feast_core_CoreService_proto_msgTypes[25] + if protoimpl.UnsafeEnabled && x != nil { + ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x)) + if ms.LoadMessageInfo() == nil { + ms.StoreMessageInfo(mi) + } + return ms + } + return mi.MessageOf(x) +} + +// Deprecated: Use GetFeatureStatisticsResponse.ProtoReflect.Descriptor instead. +func (*GetFeatureStatisticsResponse) Descriptor() ([]byte, []int) { + return file_feast_core_CoreService_proto_rawDescGZIP(), []int{25} +} + +func (x *GetFeatureStatisticsResponse) GetDatasetFeatureStatisticsList() *v0.DatasetFeatureStatisticsList { + if x != nil { + return x.DatasetFeatureStatisticsList + } + return nil +} + type ListFeatureSetsRequest_Filter struct { state protoimpl.MessageState sizeCache protoimpl.SizeCache @@ -1299,7 +1463,7 @@ type ListFeatureSetsRequest_Filter struct { func (x *ListFeatureSetsRequest_Filter) Reset() { *x = ListFeatureSetsRequest_Filter{} if protoimpl.UnsafeEnabled { - mi := &file_feast_core_CoreService_proto_msgTypes[24] + mi := &file_feast_core_CoreService_proto_msgTypes[26] ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x)) ms.StoreMessageInfo(mi) } @@ -1312,7 +1476,7 @@ func (x *ListFeatureSetsRequest_Filter) String() string { func (*ListFeatureSetsRequest_Filter) ProtoMessage() {} func (x *ListFeatureSetsRequest_Filter) ProtoReflect() protoreflect.Message { - mi := &file_feast_core_CoreService_proto_msgTypes[24] + mi := &file_feast_core_CoreService_proto_msgTypes[26] if protoimpl.UnsafeEnabled && x != nil { ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x)) if ms.LoadMessageInfo() == nil { @@ -1354,7 +1518,7 @@ type ListStoresRequest_Filter struct { func (x *ListStoresRequest_Filter) Reset() { *x = ListStoresRequest_Filter{} if protoimpl.UnsafeEnabled { - mi := &file_feast_core_CoreService_proto_msgTypes[25] + mi := &file_feast_core_CoreService_proto_msgTypes[27] ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x)) ms.StoreMessageInfo(mi) } @@ -1367,7 +1531,7 @@ func (x *ListStoresRequest_Filter) String() string { func (*ListStoresRequest_Filter) ProtoMessage() {} func (x *ListStoresRequest_Filter) ProtoReflect() protoreflect.Message { - mi := &file_feast_core_CoreService_proto_msgTypes[25] + mi := &file_feast_core_CoreService_proto_msgTypes[27] if protoimpl.UnsafeEnabled && x != nil { ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x)) if ms.LoadMessageInfo() == nil { @@ -1406,7 +1570,7 @@ type ListIngestionJobsRequest_Filter struct { func (x *ListIngestionJobsRequest_Filter) Reset() { *x = ListIngestionJobsRequest_Filter{} if protoimpl.UnsafeEnabled { - mi := &file_feast_core_CoreService_proto_msgTypes[26] + mi := &file_feast_core_CoreService_proto_msgTypes[28] ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x)) ms.StoreMessageInfo(mi) } @@ -1419,7 +1583,7 @@ func (x *ListIngestionJobsRequest_Filter) String() string { func (*ListIngestionJobsRequest_Filter) ProtoMessage() {} func (x *ListIngestionJobsRequest_Filter) ProtoReflect() protoreflect.Message { - mi := &file_feast_core_CoreService_proto_msgTypes[26] + mi := &file_feast_core_CoreService_proto_msgTypes[28] if protoimpl.UnsafeEnabled && x != nil { ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x)) if ms.LoadMessageInfo() == nil { @@ -1461,7 +1625,12 @@ var File_feast_core_CoreService_proto protoreflect.FileDescriptor var file_feast_core_CoreService_proto_rawDesc = []byte{ 0x0a, 0x1c, 0x66, 0x65, 0x61, 0x73, 0x74, 0x2f, 0x63, 0x6f, 0x72, 0x65, 0x2f, 0x43, 0x6f, 0x72, 0x65, 0x53, 0x65, 0x72, 0x76, 0x69, 0x63, 0x65, 0x2e, 0x70, 0x72, 0x6f, 0x74, 0x6f, 0x12, 0x0a, - 0x66, 0x65, 0x61, 0x73, 0x74, 0x2e, 0x63, 0x6f, 0x72, 0x65, 0x1a, 0x1b, 0x66, 0x65, 0x61, 0x73, + 0x66, 0x65, 0x61, 0x73, 0x74, 0x2e, 0x63, 0x6f, 0x72, 0x65, 0x1a, 0x1f, 0x67, 0x6f, 0x6f, 0x67, + 0x6c, 0x65, 0x2f, 0x70, 0x72, 0x6f, 0x74, 0x6f, 0x62, 0x75, 0x66, 0x2f, 0x74, 0x69, 0x6d, 0x65, + 0x73, 0x74, 0x61, 0x6d, 0x70, 0x2e, 0x70, 0x72, 0x6f, 0x74, 0x6f, 0x1a, 0x2d, 0x74, 0x65, 0x6e, + 0x73, 0x6f, 0x72, 0x66, 0x6c, 0x6f, 0x77, 0x5f, 0x6d, 0x65, 0x74, 0x61, 0x64, 0x61, 0x74, 0x61, + 0x2f, 0x70, 0x72, 0x6f, 0x74, 0x6f, 0x2f, 0x76, 0x30, 0x2f, 0x73, 0x74, 0x61, 0x74, 0x69, 0x73, + 0x74, 0x69, 0x63, 0x73, 0x2e, 0x70, 0x72, 0x6f, 0x74, 0x6f, 0x1a, 0x1b, 0x66, 0x65, 0x61, 0x73, 0x74, 0x2f, 0x63, 0x6f, 0x72, 0x65, 0x2f, 0x46, 0x65, 0x61, 0x74, 0x75, 0x72, 0x65, 0x53, 0x65, 0x74, 0x2e, 0x70, 0x72, 0x6f, 0x74, 0x6f, 0x1a, 0x16, 0x66, 0x65, 0x61, 0x73, 0x74, 0x2f, 0x63, 0x6f, 0x72, 0x65, 0x2f, 0x53, 0x74, 0x6f, 0x72, 0x65, 0x2e, 0x70, 0x72, 0x6f, 0x74, 0x6f, 0x1a, @@ -1586,82 +1755,118 @@ var file_feast_core_CoreService_proto_rawDesc = []byte{ 0x65, 0x73, 0x74, 0x69, 0x6f, 0x6e, 0x4a, 0x6f, 0x62, 0x52, 0x65, 0x71, 0x75, 0x65, 0x73, 0x74, 0x12, 0x0e, 0x0a, 0x02, 0x69, 0x64, 0x18, 0x01, 0x20, 0x01, 0x28, 0x09, 0x52, 0x02, 0x69, 0x64, 0x22, 0x1a, 0x0a, 0x18, 0x53, 0x74, 0x6f, 0x70, 0x49, 0x6e, 0x67, 0x65, 0x73, 0x74, 0x69, 0x6f, - 0x6e, 0x4a, 0x6f, 0x62, 0x52, 0x65, 0x73, 0x70, 0x6f, 0x6e, 0x73, 0x65, 0x32, 0xcb, 0x08, 0x0a, - 0x0b, 0x43, 0x6f, 0x72, 0x65, 0x53, 0x65, 0x72, 0x76, 0x69, 0x63, 0x65, 0x12, 0x66, 0x0a, 0x13, - 0x47, 0x65, 0x74, 0x46, 0x65, 0x61, 0x73, 0x74, 0x43, 0x6f, 0x72, 0x65, 0x56, 0x65, 0x72, 0x73, - 0x69, 0x6f, 0x6e, 0x12, 0x26, 0x2e, 0x66, 0x65, 0x61, 0x73, 0x74, 0x2e, 0x63, 0x6f, 0x72, 0x65, - 0x2e, 0x47, 0x65, 0x74, 0x46, 0x65, 0x61, 0x73, 0x74, 0x43, 0x6f, 0x72, 0x65, 0x56, 0x65, 0x72, - 0x73, 0x69, 0x6f, 0x6e, 0x52, 0x65, 0x71, 0x75, 0x65, 0x73, 0x74, 0x1a, 0x27, 0x2e, 0x66, 0x65, - 0x61, 0x73, 0x74, 0x2e, 0x63, 0x6f, 0x72, 0x65, 0x2e, 0x47, 0x65, 0x74, 0x46, 0x65, 0x61, 0x73, - 0x74, 0x43, 0x6f, 0x72, 0x65, 0x56, 0x65, 0x72, 0x73, 0x69, 0x6f, 0x6e, 0x52, 0x65, 0x73, 0x70, - 0x6f, 0x6e, 0x73, 0x65, 0x12, 0x54, 0x0a, 0x0d, 0x47, 0x65, 0x74, 0x46, 0x65, 0x61, 0x74, 0x75, - 0x72, 0x65, 0x53, 0x65, 0x74, 0x12, 0x20, 0x2e, 0x66, 0x65, 0x61, 0x73, 0x74, 0x2e, 0x63, 0x6f, - 0x72, 0x65, 0x2e, 0x47, 0x65, 0x74, 0x46, 0x65, 0x61, 0x74, 0x75, 0x72, 0x65, 0x53, 0x65, 0x74, - 0x52, 0x65, 0x71, 0x75, 0x65, 0x73, 0x74, 0x1a, 0x21, 0x2e, 0x66, 0x65, 0x61, 0x73, 0x74, 0x2e, + 0x6e, 0x4a, 0x6f, 0x62, 0x52, 0x65, 0x73, 0x70, 0x6f, 0x6e, 0x73, 0x65, 0x22, 0xb1, 0x02, 0x0a, + 0x1b, 0x47, 0x65, 0x74, 0x46, 0x65, 0x61, 0x74, 0x75, 0x72, 0x65, 0x53, 0x74, 0x61, 0x74, 0x69, + 0x73, 0x74, 0x69, 0x63, 0x73, 0x52, 0x65, 0x71, 0x75, 0x65, 0x73, 0x74, 0x12, 0x24, 0x0a, 0x0e, + 0x66, 0x65, 0x61, 0x74, 0x75, 0x72, 0x65, 0x5f, 0x73, 0x65, 0x74, 0x5f, 0x69, 0x64, 0x18, 0x01, + 0x20, 0x01, 0x28, 0x09, 0x52, 0x0c, 0x66, 0x65, 0x61, 0x74, 0x75, 0x72, 0x65, 0x53, 0x65, 0x74, + 0x49, 0x64, 0x12, 0x1a, 0x0a, 0x08, 0x66, 0x65, 0x61, 0x74, 0x75, 0x72, 0x65, 0x73, 0x18, 0x02, + 0x20, 0x03, 0x28, 0x09, 0x52, 0x08, 0x66, 0x65, 0x61, 0x74, 0x75, 0x72, 0x65, 0x73, 0x12, 0x14, + 0x0a, 0x05, 0x73, 0x74, 0x6f, 0x72, 0x65, 0x18, 0x03, 0x20, 0x01, 0x28, 0x09, 0x52, 0x05, 0x73, + 0x74, 0x6f, 0x72, 0x65, 0x12, 0x39, 0x0a, 0x0a, 0x73, 0x74, 0x61, 0x72, 0x74, 0x5f, 0x64, 0x61, + 0x74, 0x65, 0x18, 0x04, 0x20, 0x01, 0x28, 0x0b, 0x32, 0x1a, 0x2e, 0x67, 0x6f, 0x6f, 0x67, 0x6c, + 0x65, 0x2e, 0x70, 0x72, 0x6f, 0x74, 0x6f, 0x62, 0x75, 0x66, 0x2e, 0x54, 0x69, 0x6d, 0x65, 0x73, + 0x74, 0x61, 0x6d, 0x70, 0x52, 0x09, 0x73, 0x74, 0x61, 0x72, 0x74, 0x44, 0x61, 0x74, 0x65, 0x12, + 0x35, 0x0a, 0x08, 0x65, 0x6e, 0x64, 0x5f, 0x64, 0x61, 0x74, 0x65, 0x18, 0x05, 0x20, 0x01, 0x28, + 0x0b, 0x32, 0x1a, 0x2e, 0x67, 0x6f, 0x6f, 0x67, 0x6c, 0x65, 0x2e, 0x70, 0x72, 0x6f, 0x74, 0x6f, + 0x62, 0x75, 0x66, 0x2e, 0x54, 0x69, 0x6d, 0x65, 0x73, 0x74, 0x61, 0x6d, 0x70, 0x52, 0x07, 0x65, + 0x6e, 0x64, 0x44, 0x61, 0x74, 0x65, 0x12, 0x23, 0x0a, 0x0d, 0x69, 0x6e, 0x67, 0x65, 0x73, 0x74, + 0x69, 0x6f, 0x6e, 0x5f, 0x69, 0x64, 0x73, 0x18, 0x06, 0x20, 0x03, 0x28, 0x09, 0x52, 0x0c, 0x69, + 0x6e, 0x67, 0x65, 0x73, 0x74, 0x69, 0x6f, 0x6e, 0x49, 0x64, 0x73, 0x12, 0x23, 0x0a, 0x0d, 0x66, + 0x6f, 0x72, 0x63, 0x65, 0x5f, 0x72, 0x65, 0x66, 0x72, 0x65, 0x73, 0x68, 0x18, 0x07, 0x20, 0x01, + 0x28, 0x08, 0x52, 0x0c, 0x66, 0x6f, 0x72, 0x63, 0x65, 0x52, 0x65, 0x66, 0x72, 0x65, 0x73, 0x68, + 0x22, 0x9b, 0x01, 0x0a, 0x1c, 0x47, 0x65, 0x74, 0x46, 0x65, 0x61, 0x74, 0x75, 0x72, 0x65, 0x53, + 0x74, 0x61, 0x74, 0x69, 0x73, 0x74, 0x69, 0x63, 0x73, 0x52, 0x65, 0x73, 0x70, 0x6f, 0x6e, 0x73, + 0x65, 0x12, 0x7b, 0x0a, 0x1f, 0x64, 0x61, 0x74, 0x61, 0x73, 0x65, 0x74, 0x5f, 0x66, 0x65, 0x61, + 0x74, 0x75, 0x72, 0x65, 0x5f, 0x73, 0x74, 0x61, 0x74, 0x69, 0x73, 0x74, 0x69, 0x63, 0x73, 0x5f, + 0x6c, 0x69, 0x73, 0x74, 0x18, 0x01, 0x20, 0x01, 0x28, 0x0b, 0x32, 0x34, 0x2e, 0x74, 0x65, 0x6e, + 0x73, 0x6f, 0x72, 0x66, 0x6c, 0x6f, 0x77, 0x2e, 0x6d, 0x65, 0x74, 0x61, 0x64, 0x61, 0x74, 0x61, + 0x2e, 0x76, 0x30, 0x2e, 0x44, 0x61, 0x74, 0x61, 0x73, 0x65, 0x74, 0x46, 0x65, 0x61, 0x74, 0x75, + 0x72, 0x65, 0x53, 0x74, 0x61, 0x74, 0x69, 0x73, 0x74, 0x69, 0x63, 0x73, 0x4c, 0x69, 0x73, 0x74, + 0x52, 0x1c, 0x64, 0x61, 0x74, 0x61, 0x73, 0x65, 0x74, 0x46, 0x65, 0x61, 0x74, 0x75, 0x72, 0x65, + 0x53, 0x74, 0x61, 0x74, 0x69, 0x73, 0x74, 0x69, 0x63, 0x73, 0x4c, 0x69, 0x73, 0x74, 0x32, 0xb6, + 0x09, 0x0a, 0x0b, 0x43, 0x6f, 0x72, 0x65, 0x53, 0x65, 0x72, 0x76, 0x69, 0x63, 0x65, 0x12, 0x66, + 0x0a, 0x13, 0x47, 0x65, 0x74, 0x46, 0x65, 0x61, 0x73, 0x74, 0x43, 0x6f, 0x72, 0x65, 0x56, 0x65, + 0x72, 0x73, 0x69, 0x6f, 0x6e, 0x12, 0x26, 0x2e, 0x66, 0x65, 0x61, 0x73, 0x74, 0x2e, 0x63, 0x6f, + 0x72, 0x65, 0x2e, 0x47, 0x65, 0x74, 0x46, 0x65, 0x61, 0x73, 0x74, 0x43, 0x6f, 0x72, 0x65, 0x56, + 0x65, 0x72, 0x73, 0x69, 0x6f, 0x6e, 0x52, 0x65, 0x71, 0x75, 0x65, 0x73, 0x74, 0x1a, 0x27, 0x2e, + 0x66, 0x65, 0x61, 0x73, 0x74, 0x2e, 0x63, 0x6f, 0x72, 0x65, 0x2e, 0x47, 0x65, 0x74, 0x46, 0x65, + 0x61, 0x73, 0x74, 0x43, 0x6f, 0x72, 0x65, 0x56, 0x65, 0x72, 0x73, 0x69, 0x6f, 0x6e, 0x52, 0x65, + 0x73, 0x70, 0x6f, 0x6e, 0x73, 0x65, 0x12, 0x54, 0x0a, 0x0d, 0x47, 0x65, 0x74, 0x46, 0x65, 0x61, + 0x74, 0x75, 0x72, 0x65, 0x53, 0x65, 0x74, 0x12, 0x20, 0x2e, 0x66, 0x65, 0x61, 0x73, 0x74, 0x2e, 0x63, 0x6f, 0x72, 0x65, 0x2e, 0x47, 0x65, 0x74, 0x46, 0x65, 0x61, 0x74, 0x75, 0x72, 0x65, 0x53, - 0x65, 0x74, 0x52, 0x65, 0x73, 0x70, 0x6f, 0x6e, 0x73, 0x65, 0x12, 0x5a, 0x0a, 0x0f, 0x4c, 0x69, - 0x73, 0x74, 0x46, 0x65, 0x61, 0x74, 0x75, 0x72, 0x65, 0x53, 0x65, 0x74, 0x73, 0x12, 0x22, 0x2e, - 0x66, 0x65, 0x61, 0x73, 0x74, 0x2e, 0x63, 0x6f, 0x72, 0x65, 0x2e, 0x4c, 0x69, 0x73, 0x74, 0x46, - 0x65, 0x61, 0x74, 0x75, 0x72, 0x65, 0x53, 0x65, 0x74, 0x73, 0x52, 0x65, 0x71, 0x75, 0x65, 0x73, - 0x74, 0x1a, 0x23, 0x2e, 0x66, 0x65, 0x61, 0x73, 0x74, 0x2e, 0x63, 0x6f, 0x72, 0x65, 0x2e, 0x4c, - 0x69, 0x73, 0x74, 0x46, 0x65, 0x61, 0x74, 0x75, 0x72, 0x65, 0x53, 0x65, 0x74, 0x73, 0x52, 0x65, - 0x73, 0x70, 0x6f, 0x6e, 0x73, 0x65, 0x12, 0x4b, 0x0a, 0x0a, 0x4c, 0x69, 0x73, 0x74, 0x53, 0x74, - 0x6f, 0x72, 0x65, 0x73, 0x12, 0x1d, 0x2e, 0x66, 0x65, 0x61, 0x73, 0x74, 0x2e, 0x63, 0x6f, 0x72, - 0x65, 0x2e, 0x4c, 0x69, 0x73, 0x74, 0x53, 0x74, 0x6f, 0x72, 0x65, 0x73, 0x52, 0x65, 0x71, 0x75, - 0x65, 0x73, 0x74, 0x1a, 0x1e, 0x2e, 0x66, 0x65, 0x61, 0x73, 0x74, 0x2e, 0x63, 0x6f, 0x72, 0x65, - 0x2e, 0x4c, 0x69, 0x73, 0x74, 0x53, 0x74, 0x6f, 0x72, 0x65, 0x73, 0x52, 0x65, 0x73, 0x70, 0x6f, - 0x6e, 0x73, 0x65, 0x12, 0x5a, 0x0a, 0x0f, 0x41, 0x70, 0x70, 0x6c, 0x79, 0x46, 0x65, 0x61, 0x74, - 0x75, 0x72, 0x65, 0x53, 0x65, 0x74, 0x12, 0x22, 0x2e, 0x66, 0x65, 0x61, 0x73, 0x74, 0x2e, 0x63, - 0x6f, 0x72, 0x65, 0x2e, 0x41, 0x70, 0x70, 0x6c, 0x79, 0x46, 0x65, 0x61, 0x74, 0x75, 0x72, 0x65, - 0x53, 0x65, 0x74, 0x52, 0x65, 0x71, 0x75, 0x65, 0x73, 0x74, 0x1a, 0x23, 0x2e, 0x66, 0x65, 0x61, - 0x73, 0x74, 0x2e, 0x63, 0x6f, 0x72, 0x65, 0x2e, 0x41, 0x70, 0x70, 0x6c, 0x79, 0x46, 0x65, 0x61, - 0x74, 0x75, 0x72, 0x65, 0x53, 0x65, 0x74, 0x52, 0x65, 0x73, 0x70, 0x6f, 0x6e, 0x73, 0x65, 0x12, - 0x4e, 0x0a, 0x0b, 0x55, 0x70, 0x64, 0x61, 0x74, 0x65, 0x53, 0x74, 0x6f, 0x72, 0x65, 0x12, 0x1e, - 0x2e, 0x66, 0x65, 0x61, 0x73, 0x74, 0x2e, 0x63, 0x6f, 0x72, 0x65, 0x2e, 0x55, 0x70, 0x64, 0x61, - 0x74, 0x65, 0x53, 0x74, 0x6f, 0x72, 0x65, 0x52, 0x65, 0x71, 0x75, 0x65, 0x73, 0x74, 0x1a, 0x1f, - 0x2e, 0x66, 0x65, 0x61, 0x73, 0x74, 0x2e, 0x63, 0x6f, 0x72, 0x65, 0x2e, 0x55, 0x70, 0x64, 0x61, - 0x74, 0x65, 0x53, 0x74, 0x6f, 0x72, 0x65, 0x52, 0x65, 0x73, 0x70, 0x6f, 0x6e, 0x73, 0x65, 0x12, - 0x54, 0x0a, 0x0d, 0x43, 0x72, 0x65, 0x61, 0x74, 0x65, 0x50, 0x72, 0x6f, 0x6a, 0x65, 0x63, 0x74, - 0x12, 0x20, 0x2e, 0x66, 0x65, 0x61, 0x73, 0x74, 0x2e, 0x63, 0x6f, 0x72, 0x65, 0x2e, 0x43, 0x72, - 0x65, 0x61, 0x74, 0x65, 0x50, 0x72, 0x6f, 0x6a, 0x65, 0x63, 0x74, 0x52, 0x65, 0x71, 0x75, 0x65, - 0x73, 0x74, 0x1a, 0x21, 0x2e, 0x66, 0x65, 0x61, 0x73, 0x74, 0x2e, 0x63, 0x6f, 0x72, 0x65, 0x2e, - 0x43, 0x72, 0x65, 0x61, 0x74, 0x65, 0x50, 0x72, 0x6f, 0x6a, 0x65, 0x63, 0x74, 0x52, 0x65, 0x73, - 0x70, 0x6f, 0x6e, 0x73, 0x65, 0x12, 0x57, 0x0a, 0x0e, 0x41, 0x72, 0x63, 0x68, 0x69, 0x76, 0x65, - 0x50, 0x72, 0x6f, 0x6a, 0x65, 0x63, 0x74, 0x12, 0x21, 0x2e, 0x66, 0x65, 0x61, 0x73, 0x74, 0x2e, + 0x65, 0x74, 0x52, 0x65, 0x71, 0x75, 0x65, 0x73, 0x74, 0x1a, 0x21, 0x2e, 0x66, 0x65, 0x61, 0x73, + 0x74, 0x2e, 0x63, 0x6f, 0x72, 0x65, 0x2e, 0x47, 0x65, 0x74, 0x46, 0x65, 0x61, 0x74, 0x75, 0x72, + 0x65, 0x53, 0x65, 0x74, 0x52, 0x65, 0x73, 0x70, 0x6f, 0x6e, 0x73, 0x65, 0x12, 0x5a, 0x0a, 0x0f, + 0x4c, 0x69, 0x73, 0x74, 0x46, 0x65, 0x61, 0x74, 0x75, 0x72, 0x65, 0x53, 0x65, 0x74, 0x73, 0x12, + 0x22, 0x2e, 0x66, 0x65, 0x61, 0x73, 0x74, 0x2e, 0x63, 0x6f, 0x72, 0x65, 0x2e, 0x4c, 0x69, 0x73, + 0x74, 0x46, 0x65, 0x61, 0x74, 0x75, 0x72, 0x65, 0x53, 0x65, 0x74, 0x73, 0x52, 0x65, 0x71, 0x75, + 0x65, 0x73, 0x74, 0x1a, 0x23, 0x2e, 0x66, 0x65, 0x61, 0x73, 0x74, 0x2e, 0x63, 0x6f, 0x72, 0x65, + 0x2e, 0x4c, 0x69, 0x73, 0x74, 0x46, 0x65, 0x61, 0x74, 0x75, 0x72, 0x65, 0x53, 0x65, 0x74, 0x73, + 0x52, 0x65, 0x73, 0x70, 0x6f, 0x6e, 0x73, 0x65, 0x12, 0x69, 0x0a, 0x14, 0x47, 0x65, 0x74, 0x46, + 0x65, 0x61, 0x74, 0x75, 0x72, 0x65, 0x53, 0x74, 0x61, 0x74, 0x69, 0x73, 0x74, 0x69, 0x63, 0x73, + 0x12, 0x27, 0x2e, 0x66, 0x65, 0x61, 0x73, 0x74, 0x2e, 0x63, 0x6f, 0x72, 0x65, 0x2e, 0x47, 0x65, + 0x74, 0x46, 0x65, 0x61, 0x74, 0x75, 0x72, 0x65, 0x53, 0x74, 0x61, 0x74, 0x69, 0x73, 0x74, 0x69, + 0x63, 0x73, 0x52, 0x65, 0x71, 0x75, 0x65, 0x73, 0x74, 0x1a, 0x28, 0x2e, 0x66, 0x65, 0x61, 0x73, + 0x74, 0x2e, 0x63, 0x6f, 0x72, 0x65, 0x2e, 0x47, 0x65, 0x74, 0x46, 0x65, 0x61, 0x74, 0x75, 0x72, + 0x65, 0x53, 0x74, 0x61, 0x74, 0x69, 0x73, 0x74, 0x69, 0x63, 0x73, 0x52, 0x65, 0x73, 0x70, 0x6f, + 0x6e, 0x73, 0x65, 0x12, 0x4b, 0x0a, 0x0a, 0x4c, 0x69, 0x73, 0x74, 0x53, 0x74, 0x6f, 0x72, 0x65, + 0x73, 0x12, 0x1d, 0x2e, 0x66, 0x65, 0x61, 0x73, 0x74, 0x2e, 0x63, 0x6f, 0x72, 0x65, 0x2e, 0x4c, + 0x69, 0x73, 0x74, 0x53, 0x74, 0x6f, 0x72, 0x65, 0x73, 0x52, 0x65, 0x71, 0x75, 0x65, 0x73, 0x74, + 0x1a, 0x1e, 0x2e, 0x66, 0x65, 0x61, 0x73, 0x74, 0x2e, 0x63, 0x6f, 0x72, 0x65, 0x2e, 0x4c, 0x69, + 0x73, 0x74, 0x53, 0x74, 0x6f, 0x72, 0x65, 0x73, 0x52, 0x65, 0x73, 0x70, 0x6f, 0x6e, 0x73, 0x65, + 0x12, 0x5a, 0x0a, 0x0f, 0x41, 0x70, 0x70, 0x6c, 0x79, 0x46, 0x65, 0x61, 0x74, 0x75, 0x72, 0x65, + 0x53, 0x65, 0x74, 0x12, 0x22, 0x2e, 0x66, 0x65, 0x61, 0x73, 0x74, 0x2e, 0x63, 0x6f, 0x72, 0x65, + 0x2e, 0x41, 0x70, 0x70, 0x6c, 0x79, 0x46, 0x65, 0x61, 0x74, 0x75, 0x72, 0x65, 0x53, 0x65, 0x74, + 0x52, 0x65, 0x71, 0x75, 0x65, 0x73, 0x74, 0x1a, 0x23, 0x2e, 0x66, 0x65, 0x61, 0x73, 0x74, 0x2e, + 0x63, 0x6f, 0x72, 0x65, 0x2e, 0x41, 0x70, 0x70, 0x6c, 0x79, 0x46, 0x65, 0x61, 0x74, 0x75, 0x72, + 0x65, 0x53, 0x65, 0x74, 0x52, 0x65, 0x73, 0x70, 0x6f, 0x6e, 0x73, 0x65, 0x12, 0x4e, 0x0a, 0x0b, + 0x55, 0x70, 0x64, 0x61, 0x74, 0x65, 0x53, 0x74, 0x6f, 0x72, 0x65, 0x12, 0x1e, 0x2e, 0x66, 0x65, + 0x61, 0x73, 0x74, 0x2e, 0x63, 0x6f, 0x72, 0x65, 0x2e, 0x55, 0x70, 0x64, 0x61, 0x74, 0x65, 0x53, + 0x74, 0x6f, 0x72, 0x65, 0x52, 0x65, 0x71, 0x75, 0x65, 0x73, 0x74, 0x1a, 0x1f, 0x2e, 0x66, 0x65, + 0x61, 0x73, 0x74, 0x2e, 0x63, 0x6f, 0x72, 0x65, 0x2e, 0x55, 0x70, 0x64, 0x61, 0x74, 0x65, 0x53, + 0x74, 0x6f, 0x72, 0x65, 0x52, 0x65, 0x73, 0x70, 0x6f, 0x6e, 0x73, 0x65, 0x12, 0x54, 0x0a, 0x0d, + 0x43, 0x72, 0x65, 0x61, 0x74, 0x65, 0x50, 0x72, 0x6f, 0x6a, 0x65, 0x63, 0x74, 0x12, 0x20, 0x2e, + 0x66, 0x65, 0x61, 0x73, 0x74, 0x2e, 0x63, 0x6f, 0x72, 0x65, 0x2e, 0x43, 0x72, 0x65, 0x61, 0x74, + 0x65, 0x50, 0x72, 0x6f, 0x6a, 0x65, 0x63, 0x74, 0x52, 0x65, 0x71, 0x75, 0x65, 0x73, 0x74, 0x1a, + 0x21, 0x2e, 0x66, 0x65, 0x61, 0x73, 0x74, 0x2e, 0x63, 0x6f, 0x72, 0x65, 0x2e, 0x43, 0x72, 0x65, + 0x61, 0x74, 0x65, 0x50, 0x72, 0x6f, 0x6a, 0x65, 0x63, 0x74, 0x52, 0x65, 0x73, 0x70, 0x6f, 0x6e, + 0x73, 0x65, 0x12, 0x57, 0x0a, 0x0e, 0x41, 0x72, 0x63, 0x68, 0x69, 0x76, 0x65, 0x50, 0x72, 0x6f, + 0x6a, 0x65, 0x63, 0x74, 0x12, 0x21, 0x2e, 0x66, 0x65, 0x61, 0x73, 0x74, 0x2e, 0x63, 0x6f, 0x72, + 0x65, 0x2e, 0x41, 0x72, 0x63, 0x68, 0x69, 0x76, 0x65, 0x50, 0x72, 0x6f, 0x6a, 0x65, 0x63, 0x74, + 0x52, 0x65, 0x71, 0x75, 0x65, 0x73, 0x74, 0x1a, 0x22, 0x2e, 0x66, 0x65, 0x61, 0x73, 0x74, 0x2e, 0x63, 0x6f, 0x72, 0x65, 0x2e, 0x41, 0x72, 0x63, 0x68, 0x69, 0x76, 0x65, 0x50, 0x72, 0x6f, 0x6a, - 0x65, 0x63, 0x74, 0x52, 0x65, 0x71, 0x75, 0x65, 0x73, 0x74, 0x1a, 0x22, 0x2e, 0x66, 0x65, 0x61, - 0x73, 0x74, 0x2e, 0x63, 0x6f, 0x72, 0x65, 0x2e, 0x41, 0x72, 0x63, 0x68, 0x69, 0x76, 0x65, 0x50, - 0x72, 0x6f, 0x6a, 0x65, 0x63, 0x74, 0x52, 0x65, 0x73, 0x70, 0x6f, 0x6e, 0x73, 0x65, 0x12, 0x51, - 0x0a, 0x0c, 0x4c, 0x69, 0x73, 0x74, 0x50, 0x72, 0x6f, 0x6a, 0x65, 0x63, 0x74, 0x73, 0x12, 0x1f, - 0x2e, 0x66, 0x65, 0x61, 0x73, 0x74, 0x2e, 0x63, 0x6f, 0x72, 0x65, 0x2e, 0x4c, 0x69, 0x73, 0x74, - 0x50, 0x72, 0x6f, 0x6a, 0x65, 0x63, 0x74, 0x73, 0x52, 0x65, 0x71, 0x75, 0x65, 0x73, 0x74, 0x1a, - 0x20, 0x2e, 0x66, 0x65, 0x61, 0x73, 0x74, 0x2e, 0x63, 0x6f, 0x72, 0x65, 0x2e, 0x4c, 0x69, 0x73, - 0x74, 0x50, 0x72, 0x6f, 0x6a, 0x65, 0x63, 0x74, 0x73, 0x52, 0x65, 0x73, 0x70, 0x6f, 0x6e, 0x73, - 0x65, 0x12, 0x60, 0x0a, 0x11, 0x4c, 0x69, 0x73, 0x74, 0x49, 0x6e, 0x67, 0x65, 0x73, 0x74, 0x69, - 0x6f, 0x6e, 0x4a, 0x6f, 0x62, 0x73, 0x12, 0x24, 0x2e, 0x66, 0x65, 0x61, 0x73, 0x74, 0x2e, 0x63, - 0x6f, 0x72, 0x65, 0x2e, 0x4c, 0x69, 0x73, 0x74, 0x49, 0x6e, 0x67, 0x65, 0x73, 0x74, 0x69, 0x6f, - 0x6e, 0x4a, 0x6f, 0x62, 0x73, 0x52, 0x65, 0x71, 0x75, 0x65, 0x73, 0x74, 0x1a, 0x25, 0x2e, 0x66, - 0x65, 0x61, 0x73, 0x74, 0x2e, 0x63, 0x6f, 0x72, 0x65, 0x2e, 0x4c, 0x69, 0x73, 0x74, 0x49, 0x6e, - 0x67, 0x65, 0x73, 0x74, 0x69, 0x6f, 0x6e, 0x4a, 0x6f, 0x62, 0x73, 0x52, 0x65, 0x73, 0x70, 0x6f, - 0x6e, 0x73, 0x65, 0x12, 0x66, 0x0a, 0x13, 0x52, 0x65, 0x73, 0x74, 0x61, 0x72, 0x74, 0x49, 0x6e, - 0x67, 0x65, 0x73, 0x74, 0x69, 0x6f, 0x6e, 0x4a, 0x6f, 0x62, 0x12, 0x26, 0x2e, 0x66, 0x65, 0x61, - 0x73, 0x74, 0x2e, 0x63, 0x6f, 0x72, 0x65, 0x2e, 0x52, 0x65, 0x73, 0x74, 0x61, 0x72, 0x74, 0x49, - 0x6e, 0x67, 0x65, 0x73, 0x74, 0x69, 0x6f, 0x6e, 0x4a, 0x6f, 0x62, 0x52, 0x65, 0x71, 0x75, 0x65, - 0x73, 0x74, 0x1a, 0x27, 0x2e, 0x66, 0x65, 0x61, 0x73, 0x74, 0x2e, 0x63, 0x6f, 0x72, 0x65, 0x2e, - 0x52, 0x65, 0x73, 0x74, 0x61, 0x72, 0x74, 0x49, 0x6e, 0x67, 0x65, 0x73, 0x74, 0x69, 0x6f, 0x6e, - 0x4a, 0x6f, 0x62, 0x52, 0x65, 0x73, 0x70, 0x6f, 0x6e, 0x73, 0x65, 0x12, 0x5d, 0x0a, 0x10, 0x53, - 0x74, 0x6f, 0x70, 0x49, 0x6e, 0x67, 0x65, 0x73, 0x74, 0x69, 0x6f, 0x6e, 0x4a, 0x6f, 0x62, 0x12, - 0x23, 0x2e, 0x66, 0x65, 0x61, 0x73, 0x74, 0x2e, 0x63, 0x6f, 0x72, 0x65, 0x2e, 0x53, 0x74, 0x6f, - 0x70, 0x49, 0x6e, 0x67, 0x65, 0x73, 0x74, 0x69, 0x6f, 0x6e, 0x4a, 0x6f, 0x62, 0x52, 0x65, 0x71, - 0x75, 0x65, 0x73, 0x74, 0x1a, 0x24, 0x2e, 0x66, 0x65, 0x61, 0x73, 0x74, 0x2e, 0x63, 0x6f, 0x72, - 0x65, 0x2e, 0x53, 0x74, 0x6f, 0x70, 0x49, 0x6e, 0x67, 0x65, 0x73, 0x74, 0x69, 0x6f, 0x6e, 0x4a, - 0x6f, 0x62, 0x52, 0x65, 0x73, 0x70, 0x6f, 0x6e, 0x73, 0x65, 0x42, 0x59, 0x0a, 0x10, 0x66, 0x65, - 0x61, 0x73, 0x74, 0x2e, 0x70, 0x72, 0x6f, 0x74, 0x6f, 0x2e, 0x63, 0x6f, 0x72, 0x65, 0x42, 0x10, - 0x43, 0x6f, 0x72, 0x65, 0x53, 0x65, 0x72, 0x76, 0x69, 0x63, 0x65, 0x50, 0x72, 0x6f, 0x74, 0x6f, - 0x5a, 0x33, 0x67, 0x69, 0x74, 0x68, 0x75, 0x62, 0x2e, 0x63, 0x6f, 0x6d, 0x2f, 0x66, 0x65, 0x61, - 0x73, 0x74, 0x2d, 0x64, 0x65, 0x76, 0x2f, 0x66, 0x65, 0x61, 0x73, 0x74, 0x2f, 0x73, 0x64, 0x6b, - 0x2f, 0x67, 0x6f, 0x2f, 0x70, 0x72, 0x6f, 0x74, 0x6f, 0x73, 0x2f, 0x66, 0x65, 0x61, 0x73, 0x74, - 0x2f, 0x63, 0x6f, 0x72, 0x65, 0x62, 0x06, 0x70, 0x72, 0x6f, 0x74, 0x6f, 0x33, + 0x65, 0x63, 0x74, 0x52, 0x65, 0x73, 0x70, 0x6f, 0x6e, 0x73, 0x65, 0x12, 0x51, 0x0a, 0x0c, 0x4c, + 0x69, 0x73, 0x74, 0x50, 0x72, 0x6f, 0x6a, 0x65, 0x63, 0x74, 0x73, 0x12, 0x1f, 0x2e, 0x66, 0x65, + 0x61, 0x73, 0x74, 0x2e, 0x63, 0x6f, 0x72, 0x65, 0x2e, 0x4c, 0x69, 0x73, 0x74, 0x50, 0x72, 0x6f, + 0x6a, 0x65, 0x63, 0x74, 0x73, 0x52, 0x65, 0x71, 0x75, 0x65, 0x73, 0x74, 0x1a, 0x20, 0x2e, 0x66, + 0x65, 0x61, 0x73, 0x74, 0x2e, 0x63, 0x6f, 0x72, 0x65, 0x2e, 0x4c, 0x69, 0x73, 0x74, 0x50, 0x72, + 0x6f, 0x6a, 0x65, 0x63, 0x74, 0x73, 0x52, 0x65, 0x73, 0x70, 0x6f, 0x6e, 0x73, 0x65, 0x12, 0x60, + 0x0a, 0x11, 0x4c, 0x69, 0x73, 0x74, 0x49, 0x6e, 0x67, 0x65, 0x73, 0x74, 0x69, 0x6f, 0x6e, 0x4a, + 0x6f, 0x62, 0x73, 0x12, 0x24, 0x2e, 0x66, 0x65, 0x61, 0x73, 0x74, 0x2e, 0x63, 0x6f, 0x72, 0x65, + 0x2e, 0x4c, 0x69, 0x73, 0x74, 0x49, 0x6e, 0x67, 0x65, 0x73, 0x74, 0x69, 0x6f, 0x6e, 0x4a, 0x6f, + 0x62, 0x73, 0x52, 0x65, 0x71, 0x75, 0x65, 0x73, 0x74, 0x1a, 0x25, 0x2e, 0x66, 0x65, 0x61, 0x73, + 0x74, 0x2e, 0x63, 0x6f, 0x72, 0x65, 0x2e, 0x4c, 0x69, 0x73, 0x74, 0x49, 0x6e, 0x67, 0x65, 0x73, + 0x74, 0x69, 0x6f, 0x6e, 0x4a, 0x6f, 0x62, 0x73, 0x52, 0x65, 0x73, 0x70, 0x6f, 0x6e, 0x73, 0x65, + 0x12, 0x66, 0x0a, 0x13, 0x52, 0x65, 0x73, 0x74, 0x61, 0x72, 0x74, 0x49, 0x6e, 0x67, 0x65, 0x73, + 0x74, 0x69, 0x6f, 0x6e, 0x4a, 0x6f, 0x62, 0x12, 0x26, 0x2e, 0x66, 0x65, 0x61, 0x73, 0x74, 0x2e, + 0x63, 0x6f, 0x72, 0x65, 0x2e, 0x52, 0x65, 0x73, 0x74, 0x61, 0x72, 0x74, 0x49, 0x6e, 0x67, 0x65, + 0x73, 0x74, 0x69, 0x6f, 0x6e, 0x4a, 0x6f, 0x62, 0x52, 0x65, 0x71, 0x75, 0x65, 0x73, 0x74, 0x1a, + 0x27, 0x2e, 0x66, 0x65, 0x61, 0x73, 0x74, 0x2e, 0x63, 0x6f, 0x72, 0x65, 0x2e, 0x52, 0x65, 0x73, + 0x74, 0x61, 0x72, 0x74, 0x49, 0x6e, 0x67, 0x65, 0x73, 0x74, 0x69, 0x6f, 0x6e, 0x4a, 0x6f, 0x62, + 0x52, 0x65, 0x73, 0x70, 0x6f, 0x6e, 0x73, 0x65, 0x12, 0x5d, 0x0a, 0x10, 0x53, 0x74, 0x6f, 0x70, + 0x49, 0x6e, 0x67, 0x65, 0x73, 0x74, 0x69, 0x6f, 0x6e, 0x4a, 0x6f, 0x62, 0x12, 0x23, 0x2e, 0x66, + 0x65, 0x61, 0x73, 0x74, 0x2e, 0x63, 0x6f, 0x72, 0x65, 0x2e, 0x53, 0x74, 0x6f, 0x70, 0x49, 0x6e, + 0x67, 0x65, 0x73, 0x74, 0x69, 0x6f, 0x6e, 0x4a, 0x6f, 0x62, 0x52, 0x65, 0x71, 0x75, 0x65, 0x73, + 0x74, 0x1a, 0x24, 0x2e, 0x66, 0x65, 0x61, 0x73, 0x74, 0x2e, 0x63, 0x6f, 0x72, 0x65, 0x2e, 0x53, + 0x74, 0x6f, 0x70, 0x49, 0x6e, 0x67, 0x65, 0x73, 0x74, 0x69, 0x6f, 0x6e, 0x4a, 0x6f, 0x62, 0x52, + 0x65, 0x73, 0x70, 0x6f, 0x6e, 0x73, 0x65, 0x42, 0x59, 0x0a, 0x10, 0x66, 0x65, 0x61, 0x73, 0x74, + 0x2e, 0x70, 0x72, 0x6f, 0x74, 0x6f, 0x2e, 0x63, 0x6f, 0x72, 0x65, 0x42, 0x10, 0x43, 0x6f, 0x72, + 0x65, 0x53, 0x65, 0x72, 0x76, 0x69, 0x63, 0x65, 0x50, 0x72, 0x6f, 0x74, 0x6f, 0x5a, 0x33, 0x67, + 0x69, 0x74, 0x68, 0x75, 0x62, 0x2e, 0x63, 0x6f, 0x6d, 0x2f, 0x66, 0x65, 0x61, 0x73, 0x74, 0x2d, + 0x64, 0x65, 0x76, 0x2f, 0x66, 0x65, 0x61, 0x73, 0x74, 0x2f, 0x73, 0x64, 0x6b, 0x2f, 0x67, 0x6f, + 0x2f, 0x70, 0x72, 0x6f, 0x74, 0x6f, 0x73, 0x2f, 0x66, 0x65, 0x61, 0x73, 0x74, 0x2f, 0x63, 0x6f, + 0x72, 0x65, 0x62, 0x06, 0x70, 0x72, 0x6f, 0x74, 0x6f, 0x33, } var ( @@ -1677,7 +1882,7 @@ func file_feast_core_CoreService_proto_rawDescGZIP() []byte { } var file_feast_core_CoreService_proto_enumTypes = make([]protoimpl.EnumInfo, 2) -var file_feast_core_CoreService_proto_msgTypes = make([]protoimpl.MessageInfo, 27) +var file_feast_core_CoreService_proto_msgTypes = make([]protoimpl.MessageInfo, 29) var file_feast_core_CoreService_proto_goTypes = []interface{}{ (ApplyFeatureSetResponse_Status)(0), // 0: feast.core.ApplyFeatureSetResponse.Status (UpdateStoreResponse_Status)(0), // 1: feast.core.UpdateStoreResponse.Status @@ -1705,58 +1910,67 @@ var file_feast_core_CoreService_proto_goTypes = []interface{}{ (*RestartIngestionJobResponse)(nil), // 23: feast.core.RestartIngestionJobResponse (*StopIngestionJobRequest)(nil), // 24: feast.core.StopIngestionJobRequest (*StopIngestionJobResponse)(nil), // 25: feast.core.StopIngestionJobResponse - (*ListFeatureSetsRequest_Filter)(nil), // 26: feast.core.ListFeatureSetsRequest.Filter - (*ListStoresRequest_Filter)(nil), // 27: feast.core.ListStoresRequest.Filter - (*ListIngestionJobsRequest_Filter)(nil), // 28: feast.core.ListIngestionJobsRequest.Filter - (*FeatureSet)(nil), // 29: feast.core.FeatureSet - (*Store)(nil), // 30: feast.core.Store - (*IngestionJob)(nil), // 31: feast.core.IngestionJob - (*FeatureSetReference)(nil), // 32: feast.core.FeatureSetReference + (*GetFeatureStatisticsRequest)(nil), // 26: feast.core.GetFeatureStatisticsRequest + (*GetFeatureStatisticsResponse)(nil), // 27: feast.core.GetFeatureStatisticsResponse + (*ListFeatureSetsRequest_Filter)(nil), // 28: feast.core.ListFeatureSetsRequest.Filter + (*ListStoresRequest_Filter)(nil), // 29: feast.core.ListStoresRequest.Filter + (*ListIngestionJobsRequest_Filter)(nil), // 30: feast.core.ListIngestionJobsRequest.Filter + (*FeatureSet)(nil), // 31: feast.core.FeatureSet + (*Store)(nil), // 32: feast.core.Store + (*IngestionJob)(nil), // 33: feast.core.IngestionJob + (*timestamp.Timestamp)(nil), // 34: google.protobuf.Timestamp + (*v0.DatasetFeatureStatisticsList)(nil), // 35: tensorflow.metadata.v0.DatasetFeatureStatisticsList + (*FeatureSetReference)(nil), // 36: feast.core.FeatureSetReference } var file_feast_core_CoreService_proto_depIdxs = []int32{ - 29, // 0: feast.core.GetFeatureSetResponse.feature_set:type_name -> feast.core.FeatureSet - 26, // 1: feast.core.ListFeatureSetsRequest.filter:type_name -> feast.core.ListFeatureSetsRequest.Filter - 29, // 2: feast.core.ListFeatureSetsResponse.feature_sets:type_name -> feast.core.FeatureSet - 27, // 3: feast.core.ListStoresRequest.filter:type_name -> feast.core.ListStoresRequest.Filter - 30, // 4: feast.core.ListStoresResponse.store:type_name -> feast.core.Store - 29, // 5: feast.core.ApplyFeatureSetRequest.feature_set:type_name -> feast.core.FeatureSet - 29, // 6: feast.core.ApplyFeatureSetResponse.feature_set:type_name -> feast.core.FeatureSet + 31, // 0: feast.core.GetFeatureSetResponse.feature_set:type_name -> feast.core.FeatureSet + 28, // 1: feast.core.ListFeatureSetsRequest.filter:type_name -> feast.core.ListFeatureSetsRequest.Filter + 31, // 2: feast.core.ListFeatureSetsResponse.feature_sets:type_name -> feast.core.FeatureSet + 29, // 3: feast.core.ListStoresRequest.filter:type_name -> feast.core.ListStoresRequest.Filter + 32, // 4: feast.core.ListStoresResponse.store:type_name -> feast.core.Store + 31, // 5: feast.core.ApplyFeatureSetRequest.feature_set:type_name -> feast.core.FeatureSet + 31, // 6: feast.core.ApplyFeatureSetResponse.feature_set:type_name -> feast.core.FeatureSet 0, // 7: feast.core.ApplyFeatureSetResponse.status:type_name -> feast.core.ApplyFeatureSetResponse.Status - 30, // 8: feast.core.UpdateStoreRequest.store:type_name -> feast.core.Store - 30, // 9: feast.core.UpdateStoreResponse.store:type_name -> feast.core.Store + 32, // 8: feast.core.UpdateStoreRequest.store:type_name -> feast.core.Store + 32, // 9: feast.core.UpdateStoreResponse.store:type_name -> feast.core.Store 1, // 10: feast.core.UpdateStoreResponse.status:type_name -> feast.core.UpdateStoreResponse.Status - 28, // 11: feast.core.ListIngestionJobsRequest.filter:type_name -> feast.core.ListIngestionJobsRequest.Filter - 31, // 12: feast.core.ListIngestionJobsResponse.jobs:type_name -> feast.core.IngestionJob - 32, // 13: feast.core.ListIngestionJobsRequest.Filter.feature_set_reference:type_name -> feast.core.FeatureSetReference - 10, // 14: feast.core.CoreService.GetFeastCoreVersion:input_type -> feast.core.GetFeastCoreVersionRequest - 2, // 15: feast.core.CoreService.GetFeatureSet:input_type -> feast.core.GetFeatureSetRequest - 4, // 16: feast.core.CoreService.ListFeatureSets:input_type -> feast.core.ListFeatureSetsRequest - 6, // 17: feast.core.CoreService.ListStores:input_type -> feast.core.ListStoresRequest - 8, // 18: feast.core.CoreService.ApplyFeatureSet:input_type -> feast.core.ApplyFeatureSetRequest - 12, // 19: feast.core.CoreService.UpdateStore:input_type -> feast.core.UpdateStoreRequest - 14, // 20: feast.core.CoreService.CreateProject:input_type -> feast.core.CreateProjectRequest - 16, // 21: feast.core.CoreService.ArchiveProject:input_type -> feast.core.ArchiveProjectRequest - 18, // 22: feast.core.CoreService.ListProjects:input_type -> feast.core.ListProjectsRequest - 20, // 23: feast.core.CoreService.ListIngestionJobs:input_type -> feast.core.ListIngestionJobsRequest - 22, // 24: feast.core.CoreService.RestartIngestionJob:input_type -> feast.core.RestartIngestionJobRequest - 24, // 25: feast.core.CoreService.StopIngestionJob:input_type -> feast.core.StopIngestionJobRequest - 11, // 26: feast.core.CoreService.GetFeastCoreVersion:output_type -> feast.core.GetFeastCoreVersionResponse - 3, // 27: feast.core.CoreService.GetFeatureSet:output_type -> feast.core.GetFeatureSetResponse - 5, // 28: feast.core.CoreService.ListFeatureSets:output_type -> feast.core.ListFeatureSetsResponse - 7, // 29: feast.core.CoreService.ListStores:output_type -> feast.core.ListStoresResponse - 9, // 30: feast.core.CoreService.ApplyFeatureSet:output_type -> feast.core.ApplyFeatureSetResponse - 13, // 31: feast.core.CoreService.UpdateStore:output_type -> feast.core.UpdateStoreResponse - 15, // 32: feast.core.CoreService.CreateProject:output_type -> feast.core.CreateProjectResponse - 17, // 33: feast.core.CoreService.ArchiveProject:output_type -> feast.core.ArchiveProjectResponse - 19, // 34: feast.core.CoreService.ListProjects:output_type -> feast.core.ListProjectsResponse - 21, // 35: feast.core.CoreService.ListIngestionJobs:output_type -> feast.core.ListIngestionJobsResponse - 23, // 36: feast.core.CoreService.RestartIngestionJob:output_type -> feast.core.RestartIngestionJobResponse - 25, // 37: feast.core.CoreService.StopIngestionJob:output_type -> feast.core.StopIngestionJobResponse - 26, // [26:38] is the sub-list for method output_type - 14, // [14:26] is the sub-list for method input_type - 14, // [14:14] is the sub-list for extension type_name - 14, // [14:14] is the sub-list for extension extendee - 0, // [0:14] is the sub-list for field type_name + 30, // 11: feast.core.ListIngestionJobsRequest.filter:type_name -> feast.core.ListIngestionJobsRequest.Filter + 33, // 12: feast.core.ListIngestionJobsResponse.jobs:type_name -> feast.core.IngestionJob + 34, // 13: feast.core.GetFeatureStatisticsRequest.start_date:type_name -> google.protobuf.Timestamp + 34, // 14: feast.core.GetFeatureStatisticsRequest.end_date:type_name -> google.protobuf.Timestamp + 35, // 15: feast.core.GetFeatureStatisticsResponse.dataset_feature_statistics_list:type_name -> tensorflow.metadata.v0.DatasetFeatureStatisticsList + 36, // 16: feast.core.ListIngestionJobsRequest.Filter.feature_set_reference:type_name -> feast.core.FeatureSetReference + 10, // 17: feast.core.CoreService.GetFeastCoreVersion:input_type -> feast.core.GetFeastCoreVersionRequest + 2, // 18: feast.core.CoreService.GetFeatureSet:input_type -> feast.core.GetFeatureSetRequest + 4, // 19: feast.core.CoreService.ListFeatureSets:input_type -> feast.core.ListFeatureSetsRequest + 26, // 20: feast.core.CoreService.GetFeatureStatistics:input_type -> feast.core.GetFeatureStatisticsRequest + 6, // 21: feast.core.CoreService.ListStores:input_type -> feast.core.ListStoresRequest + 8, // 22: feast.core.CoreService.ApplyFeatureSet:input_type -> feast.core.ApplyFeatureSetRequest + 12, // 23: feast.core.CoreService.UpdateStore:input_type -> feast.core.UpdateStoreRequest + 14, // 24: feast.core.CoreService.CreateProject:input_type -> feast.core.CreateProjectRequest + 16, // 25: feast.core.CoreService.ArchiveProject:input_type -> feast.core.ArchiveProjectRequest + 18, // 26: feast.core.CoreService.ListProjects:input_type -> feast.core.ListProjectsRequest + 20, // 27: feast.core.CoreService.ListIngestionJobs:input_type -> feast.core.ListIngestionJobsRequest + 22, // 28: feast.core.CoreService.RestartIngestionJob:input_type -> feast.core.RestartIngestionJobRequest + 24, // 29: feast.core.CoreService.StopIngestionJob:input_type -> feast.core.StopIngestionJobRequest + 11, // 30: feast.core.CoreService.GetFeastCoreVersion:output_type -> feast.core.GetFeastCoreVersionResponse + 3, // 31: feast.core.CoreService.GetFeatureSet:output_type -> feast.core.GetFeatureSetResponse + 5, // 32: feast.core.CoreService.ListFeatureSets:output_type -> feast.core.ListFeatureSetsResponse + 27, // 33: feast.core.CoreService.GetFeatureStatistics:output_type -> feast.core.GetFeatureStatisticsResponse + 7, // 34: feast.core.CoreService.ListStores:output_type -> feast.core.ListStoresResponse + 9, // 35: feast.core.CoreService.ApplyFeatureSet:output_type -> feast.core.ApplyFeatureSetResponse + 13, // 36: feast.core.CoreService.UpdateStore:output_type -> feast.core.UpdateStoreResponse + 15, // 37: feast.core.CoreService.CreateProject:output_type -> feast.core.CreateProjectResponse + 17, // 38: feast.core.CoreService.ArchiveProject:output_type -> feast.core.ArchiveProjectResponse + 19, // 39: feast.core.CoreService.ListProjects:output_type -> feast.core.ListProjectsResponse + 21, // 40: feast.core.CoreService.ListIngestionJobs:output_type -> feast.core.ListIngestionJobsResponse + 23, // 41: feast.core.CoreService.RestartIngestionJob:output_type -> feast.core.RestartIngestionJobResponse + 25, // 42: feast.core.CoreService.StopIngestionJob:output_type -> feast.core.StopIngestionJobResponse + 30, // [30:43] is the sub-list for method output_type + 17, // [17:30] is the sub-list for method input_type + 17, // [17:17] is the sub-list for extension type_name + 17, // [17:17] is the sub-list for extension extendee + 0, // [0:17] is the sub-list for field type_name } func init() { file_feast_core_CoreService_proto_init() } @@ -2058,7 +2272,7 @@ func file_feast_core_CoreService_proto_init() { } } file_feast_core_CoreService_proto_msgTypes[24].Exporter = func(v interface{}, i int) interface{} { - switch v := v.(*ListFeatureSetsRequest_Filter); i { + switch v := v.(*GetFeatureStatisticsRequest); i { case 0: return &v.state case 1: @@ -2070,7 +2284,7 @@ func file_feast_core_CoreService_proto_init() { } } file_feast_core_CoreService_proto_msgTypes[25].Exporter = func(v interface{}, i int) interface{} { - switch v := v.(*ListStoresRequest_Filter); i { + switch v := v.(*GetFeatureStatisticsResponse); i { case 0: return &v.state case 1: @@ -2082,6 +2296,30 @@ func file_feast_core_CoreService_proto_init() { } } file_feast_core_CoreService_proto_msgTypes[26].Exporter = func(v interface{}, i int) interface{} { + switch v := v.(*ListFeatureSetsRequest_Filter); i { + case 0: + return &v.state + case 1: + return &v.sizeCache + case 2: + return &v.unknownFields + default: + return nil + } + } + file_feast_core_CoreService_proto_msgTypes[27].Exporter = func(v interface{}, i int) interface{} { + switch v := v.(*ListStoresRequest_Filter); i { + case 0: + return &v.state + case 1: + return &v.sizeCache + case 2: + return &v.unknownFields + default: + return nil + } + } + file_feast_core_CoreService_proto_msgTypes[28].Exporter = func(v interface{}, i int) interface{} { switch v := v.(*ListIngestionJobsRequest_Filter); i { case 0: return &v.state @@ -2100,7 +2338,7 @@ func file_feast_core_CoreService_proto_init() { GoPackagePath: reflect.TypeOf(x{}).PkgPath(), RawDescriptor: file_feast_core_CoreService_proto_rawDesc, NumEnums: 2, - NumMessages: 27, + NumMessages: 29, NumExtensions: 0, NumServices: 1, }, @@ -2138,6 +2376,10 @@ type CoreServiceClient interface { // If no filter is provided in the request, the response will contain all the feature // sets currently stored in the registry. ListFeatureSets(ctx context.Context, in *ListFeatureSetsRequest, opts ...grpc.CallOption) (*ListFeatureSetsResponse, error) + // Get feature statistics computed over the data in the batch stores. + // + // Returns a dataset containing TFDV statistics mapped to each valid historical store. + GetFeatureStatistics(ctx context.Context, in *GetFeatureStatisticsRequest, opts ...grpc.CallOption) (*GetFeatureStatisticsResponse, error) // Retrieve store details given a filter. // // Returns all stores matching that filter. If none are found, an empty list will be returned. @@ -2220,6 +2462,15 @@ func (c *coreServiceClient) ListFeatureSets(ctx context.Context, in *ListFeature return out, nil } +func (c *coreServiceClient) GetFeatureStatistics(ctx context.Context, in *GetFeatureStatisticsRequest, opts ...grpc.CallOption) (*GetFeatureStatisticsResponse, error) { + out := new(GetFeatureStatisticsResponse) + err := c.cc.Invoke(ctx, "/feast.core.CoreService/GetFeatureStatistics", in, out, opts...) + if err != nil { + return nil, err + } + return out, nil +} + func (c *coreServiceClient) ListStores(ctx context.Context, in *ListStoresRequest, opts ...grpc.CallOption) (*ListStoresResponse, error) { out := new(ListStoresResponse) err := c.cc.Invoke(ctx, "/feast.core.CoreService/ListStores", in, out, opts...) @@ -2314,6 +2565,10 @@ type CoreServiceServer interface { // If no filter is provided in the request, the response will contain all the feature // sets currently stored in the registry. ListFeatureSets(context.Context, *ListFeatureSetsRequest) (*ListFeatureSetsResponse, error) + // Get feature statistics computed over the data in the batch stores. + // + // Returns a dataset containing TFDV statistics mapped to each valid historical store. + GetFeatureStatistics(context.Context, *GetFeatureStatisticsRequest) (*GetFeatureStatisticsResponse, error) // Retrieve store details given a filter. // // Returns all stores matching that filter. If none are found, an empty list will be returned. @@ -2374,6 +2629,9 @@ func (*UnimplementedCoreServiceServer) GetFeatureSet(context.Context, *GetFeatur func (*UnimplementedCoreServiceServer) ListFeatureSets(context.Context, *ListFeatureSetsRequest) (*ListFeatureSetsResponse, error) { return nil, status.Errorf(codes.Unimplemented, "method ListFeatureSets not implemented") } +func (*UnimplementedCoreServiceServer) GetFeatureStatistics(context.Context, *GetFeatureStatisticsRequest) (*GetFeatureStatisticsResponse, error) { + return nil, status.Errorf(codes.Unimplemented, "method GetFeatureStatistics not implemented") +} func (*UnimplementedCoreServiceServer) ListStores(context.Context, *ListStoresRequest) (*ListStoresResponse, error) { return nil, status.Errorf(codes.Unimplemented, "method ListStores not implemented") } @@ -2460,6 +2718,24 @@ func _CoreService_ListFeatureSets_Handler(srv interface{}, ctx context.Context, return interceptor(ctx, in, info, handler) } +func _CoreService_GetFeatureStatistics_Handler(srv interface{}, ctx context.Context, dec func(interface{}) error, interceptor grpc.UnaryServerInterceptor) (interface{}, error) { + in := new(GetFeatureStatisticsRequest) + if err := dec(in); err != nil { + return nil, err + } + if interceptor == nil { + return srv.(CoreServiceServer).GetFeatureStatistics(ctx, in) + } + info := &grpc.UnaryServerInfo{ + Server: srv, + FullMethod: "/feast.core.CoreService/GetFeatureStatistics", + } + handler := func(ctx context.Context, req interface{}) (interface{}, error) { + return srv.(CoreServiceServer).GetFeatureStatistics(ctx, req.(*GetFeatureStatisticsRequest)) + } + return interceptor(ctx, in, info, handler) +} + func _CoreService_ListStores_Handler(srv interface{}, ctx context.Context, dec func(interface{}) error, interceptor grpc.UnaryServerInterceptor) (interface{}, error) { in := new(ListStoresRequest) if err := dec(in); err != nil { @@ -2638,6 +2914,10 @@ var _CoreService_serviceDesc = grpc.ServiceDesc{ MethodName: "ListFeatureSets", Handler: _CoreService_ListFeatureSets_Handler, }, + { + MethodName: "GetFeatureStatistics", + Handler: _CoreService_GetFeatureStatistics_Handler, + }, { MethodName: "ListStores", Handler: _CoreService_ListStores_Handler, diff --git a/sdk/go/protos/feast/core/Runner.pb.go b/sdk/go/protos/feast/core/Runner.pb.go index b1255e52ba6..74dcf9fad77 100644 --- a/sdk/go/protos/feast/core/Runner.pb.go +++ b/sdk/go/protos/feast/core/Runner.pb.go @@ -53,6 +53,9 @@ type DirectRunnerConfigOptions struct { TargetParallelism int32 `protobuf:"varint,1,opt,name=targetParallelism,proto3" json:"targetParallelism,omitempty"` // BigQuery table specification, e.g. PROJECT_ID:DATASET_ID.PROJECT_ID DeadLetterTableSpec string `protobuf:"bytes,2,opt,name=deadLetterTableSpec,proto3" json:"deadLetterTableSpec,omitempty"` + // A pipeline level default location for storing temporary files. + // Support Google Cloud Storage locations or local path + TempLocation string `protobuf:"bytes,3,opt,name=tempLocation,proto3" json:"tempLocation,omitempty"` } func (x *DirectRunnerConfigOptions) Reset() { @@ -101,6 +104,13 @@ func (x *DirectRunnerConfigOptions) GetDeadLetterTableSpec() string { return "" } +func (x *DirectRunnerConfigOptions) GetTempLocation() string { + if x != nil { + return x.TempLocation + } + return "" +} + type DataflowRunnerConfigOptions struct { state protoimpl.MessageState sizeCache protoimpl.SizeCache @@ -131,6 +141,8 @@ type DataflowRunnerConfigOptions struct { MaxNumWorkers int32 `protobuf:"varint,11,opt,name=maxNumWorkers,proto3" json:"maxNumWorkers,omitempty"` // BigQuery table specification, e.g. PROJECT_ID:DATASET_ID.PROJECT_ID DeadLetterTableSpec string `protobuf:"bytes,12,opt,name=deadLetterTableSpec,proto3" json:"deadLetterTableSpec,omitempty"` + // Labels to apply to the dataflow job + Labels map[string]string `protobuf:"bytes,13,rep,name=labels,proto3" json:"labels,omitempty" protobuf_key:"bytes,1,opt,name=key,proto3" protobuf_val:"bytes,2,opt,name=value,proto3"` } func (x *DataflowRunnerConfigOptions) Reset() { @@ -249,54 +261,72 @@ func (x *DataflowRunnerConfigOptions) GetDeadLetterTableSpec() string { return "" } +func (x *DataflowRunnerConfigOptions) GetLabels() map[string]string { + if x != nil { + return x.Labels + } + return nil +} + var File_feast_core_Runner_proto protoreflect.FileDescriptor var file_feast_core_Runner_proto_rawDesc = []byte{ 0x0a, 0x17, 0x66, 0x65, 0x61, 0x73, 0x74, 0x2f, 0x63, 0x6f, 0x72, 0x65, 0x2f, 0x52, 0x75, 0x6e, 0x6e, 0x65, 0x72, 0x2e, 0x70, 0x72, 0x6f, 0x74, 0x6f, 0x12, 0x0a, 0x66, 0x65, 0x61, 0x73, 0x74, - 0x2e, 0x63, 0x6f, 0x72, 0x65, 0x22, 0x7b, 0x0a, 0x19, 0x44, 0x69, 0x72, 0x65, 0x63, 0x74, 0x52, - 0x75, 0x6e, 0x6e, 0x65, 0x72, 0x43, 0x6f, 0x6e, 0x66, 0x69, 0x67, 0x4f, 0x70, 0x74, 0x69, 0x6f, - 0x6e, 0x73, 0x12, 0x2c, 0x0a, 0x11, 0x74, 0x61, 0x72, 0x67, 0x65, 0x74, 0x50, 0x61, 0x72, 0x61, - 0x6c, 0x6c, 0x65, 0x6c, 0x69, 0x73, 0x6d, 0x18, 0x01, 0x20, 0x01, 0x28, 0x05, 0x52, 0x11, 0x74, - 0x61, 0x72, 0x67, 0x65, 0x74, 0x50, 0x61, 0x72, 0x61, 0x6c, 0x6c, 0x65, 0x6c, 0x69, 0x73, 0x6d, - 0x12, 0x30, 0x0a, 0x13, 0x64, 0x65, 0x61, 0x64, 0x4c, 0x65, 0x74, 0x74, 0x65, 0x72, 0x54, 0x61, - 0x62, 0x6c, 0x65, 0x53, 0x70, 0x65, 0x63, 0x18, 0x02, 0x20, 0x01, 0x28, 0x09, 0x52, 0x13, 0x64, - 0x65, 0x61, 0x64, 0x4c, 0x65, 0x74, 0x74, 0x65, 0x72, 0x54, 0x61, 0x62, 0x6c, 0x65, 0x53, 0x70, - 0x65, 0x63, 0x22, 0xc7, 0x03, 0x0a, 0x1b, 0x44, 0x61, 0x74, 0x61, 0x66, 0x6c, 0x6f, 0x77, 0x52, - 0x75, 0x6e, 0x6e, 0x65, 0x72, 0x43, 0x6f, 0x6e, 0x66, 0x69, 0x67, 0x4f, 0x70, 0x74, 0x69, 0x6f, - 0x6e, 0x73, 0x12, 0x18, 0x0a, 0x07, 0x70, 0x72, 0x6f, 0x6a, 0x65, 0x63, 0x74, 0x18, 0x01, 0x20, - 0x01, 0x28, 0x09, 0x52, 0x07, 0x70, 0x72, 0x6f, 0x6a, 0x65, 0x63, 0x74, 0x12, 0x16, 0x0a, 0x06, - 0x72, 0x65, 0x67, 0x69, 0x6f, 0x6e, 0x18, 0x02, 0x20, 0x01, 0x28, 0x09, 0x52, 0x06, 0x72, 0x65, - 0x67, 0x69, 0x6f, 0x6e, 0x12, 0x12, 0x0a, 0x04, 0x7a, 0x6f, 0x6e, 0x65, 0x18, 0x03, 0x20, 0x01, - 0x28, 0x09, 0x52, 0x04, 0x7a, 0x6f, 0x6e, 0x65, 0x12, 0x26, 0x0a, 0x0e, 0x73, 0x65, 0x72, 0x76, - 0x69, 0x63, 0x65, 0x41, 0x63, 0x63, 0x6f, 0x75, 0x6e, 0x74, 0x18, 0x04, 0x20, 0x01, 0x28, 0x09, - 0x52, 0x0e, 0x73, 0x65, 0x72, 0x76, 0x69, 0x63, 0x65, 0x41, 0x63, 0x63, 0x6f, 0x75, 0x6e, 0x74, - 0x12, 0x18, 0x0a, 0x07, 0x6e, 0x65, 0x74, 0x77, 0x6f, 0x72, 0x6b, 0x18, 0x05, 0x20, 0x01, 0x28, - 0x09, 0x52, 0x07, 0x6e, 0x65, 0x74, 0x77, 0x6f, 0x72, 0x6b, 0x12, 0x1e, 0x0a, 0x0a, 0x73, 0x75, - 0x62, 0x6e, 0x65, 0x74, 0x77, 0x6f, 0x72, 0x6b, 0x18, 0x06, 0x20, 0x01, 0x28, 0x09, 0x52, 0x0a, - 0x73, 0x75, 0x62, 0x6e, 0x65, 0x74, 0x77, 0x6f, 0x72, 0x6b, 0x12, 0x2c, 0x0a, 0x11, 0x77, 0x6f, - 0x72, 0x6b, 0x65, 0x72, 0x4d, 0x61, 0x63, 0x68, 0x69, 0x6e, 0x65, 0x54, 0x79, 0x70, 0x65, 0x18, - 0x07, 0x20, 0x01, 0x28, 0x09, 0x52, 0x11, 0x77, 0x6f, 0x72, 0x6b, 0x65, 0x72, 0x4d, 0x61, 0x63, - 0x68, 0x69, 0x6e, 0x65, 0x54, 0x79, 0x70, 0x65, 0x12, 0x32, 0x0a, 0x14, 0x61, 0x75, 0x74, 0x6f, - 0x73, 0x63, 0x61, 0x6c, 0x69, 0x6e, 0x67, 0x41, 0x6c, 0x67, 0x6f, 0x72, 0x69, 0x74, 0x68, 0x6d, - 0x18, 0x08, 0x20, 0x01, 0x28, 0x09, 0x52, 0x14, 0x61, 0x75, 0x74, 0x6f, 0x73, 0x63, 0x61, 0x6c, - 0x69, 0x6e, 0x67, 0x41, 0x6c, 0x67, 0x6f, 0x72, 0x69, 0x74, 0x68, 0x6d, 0x12, 0x22, 0x0a, 0x0c, - 0x75, 0x73, 0x65, 0x50, 0x75, 0x62, 0x6c, 0x69, 0x63, 0x49, 0x70, 0x73, 0x18, 0x09, 0x20, 0x01, - 0x28, 0x08, 0x52, 0x0c, 0x75, 0x73, 0x65, 0x50, 0x75, 0x62, 0x6c, 0x69, 0x63, 0x49, 0x70, 0x73, - 0x12, 0x22, 0x0a, 0x0c, 0x74, 0x65, 0x6d, 0x70, 0x4c, 0x6f, 0x63, 0x61, 0x74, 0x69, 0x6f, 0x6e, - 0x18, 0x0a, 0x20, 0x01, 0x28, 0x09, 0x52, 0x0c, 0x74, 0x65, 0x6d, 0x70, 0x4c, 0x6f, 0x63, 0x61, - 0x74, 0x69, 0x6f, 0x6e, 0x12, 0x24, 0x0a, 0x0d, 0x6d, 0x61, 0x78, 0x4e, 0x75, 0x6d, 0x57, 0x6f, - 0x72, 0x6b, 0x65, 0x72, 0x73, 0x18, 0x0b, 0x20, 0x01, 0x28, 0x05, 0x52, 0x0d, 0x6d, 0x61, 0x78, - 0x4e, 0x75, 0x6d, 0x57, 0x6f, 0x72, 0x6b, 0x65, 0x72, 0x73, 0x12, 0x30, 0x0a, 0x13, 0x64, 0x65, + 0x2e, 0x63, 0x6f, 0x72, 0x65, 0x22, 0x9f, 0x01, 0x0a, 0x19, 0x44, 0x69, 0x72, 0x65, 0x63, 0x74, + 0x52, 0x75, 0x6e, 0x6e, 0x65, 0x72, 0x43, 0x6f, 0x6e, 0x66, 0x69, 0x67, 0x4f, 0x70, 0x74, 0x69, + 0x6f, 0x6e, 0x73, 0x12, 0x2c, 0x0a, 0x11, 0x74, 0x61, 0x72, 0x67, 0x65, 0x74, 0x50, 0x61, 0x72, + 0x61, 0x6c, 0x6c, 0x65, 0x6c, 0x69, 0x73, 0x6d, 0x18, 0x01, 0x20, 0x01, 0x28, 0x05, 0x52, 0x11, + 0x74, 0x61, 0x72, 0x67, 0x65, 0x74, 0x50, 0x61, 0x72, 0x61, 0x6c, 0x6c, 0x65, 0x6c, 0x69, 0x73, + 0x6d, 0x12, 0x30, 0x0a, 0x13, 0x64, 0x65, 0x61, 0x64, 0x4c, 0x65, 0x74, 0x74, 0x65, 0x72, 0x54, + 0x61, 0x62, 0x6c, 0x65, 0x53, 0x70, 0x65, 0x63, 0x18, 0x02, 0x20, 0x01, 0x28, 0x09, 0x52, 0x13, + 0x64, 0x65, 0x61, 0x64, 0x4c, 0x65, 0x74, 0x74, 0x65, 0x72, 0x54, 0x61, 0x62, 0x6c, 0x65, 0x53, + 0x70, 0x65, 0x63, 0x12, 0x22, 0x0a, 0x0c, 0x74, 0x65, 0x6d, 0x70, 0x4c, 0x6f, 0x63, 0x61, 0x74, + 0x69, 0x6f, 0x6e, 0x18, 0x03, 0x20, 0x01, 0x28, 0x09, 0x52, 0x0c, 0x74, 0x65, 0x6d, 0x70, 0x4c, + 0x6f, 0x63, 0x61, 0x74, 0x69, 0x6f, 0x6e, 0x22, 0xcf, 0x04, 0x0a, 0x1b, 0x44, 0x61, 0x74, 0x61, + 0x66, 0x6c, 0x6f, 0x77, 0x52, 0x75, 0x6e, 0x6e, 0x65, 0x72, 0x43, 0x6f, 0x6e, 0x66, 0x69, 0x67, + 0x4f, 0x70, 0x74, 0x69, 0x6f, 0x6e, 0x73, 0x12, 0x18, 0x0a, 0x07, 0x70, 0x72, 0x6f, 0x6a, 0x65, + 0x63, 0x74, 0x18, 0x01, 0x20, 0x01, 0x28, 0x09, 0x52, 0x07, 0x70, 0x72, 0x6f, 0x6a, 0x65, 0x63, + 0x74, 0x12, 0x16, 0x0a, 0x06, 0x72, 0x65, 0x67, 0x69, 0x6f, 0x6e, 0x18, 0x02, 0x20, 0x01, 0x28, + 0x09, 0x52, 0x06, 0x72, 0x65, 0x67, 0x69, 0x6f, 0x6e, 0x12, 0x12, 0x0a, 0x04, 0x7a, 0x6f, 0x6e, + 0x65, 0x18, 0x03, 0x20, 0x01, 0x28, 0x09, 0x52, 0x04, 0x7a, 0x6f, 0x6e, 0x65, 0x12, 0x26, 0x0a, + 0x0e, 0x73, 0x65, 0x72, 0x76, 0x69, 0x63, 0x65, 0x41, 0x63, 0x63, 0x6f, 0x75, 0x6e, 0x74, 0x18, + 0x04, 0x20, 0x01, 0x28, 0x09, 0x52, 0x0e, 0x73, 0x65, 0x72, 0x76, 0x69, 0x63, 0x65, 0x41, 0x63, + 0x63, 0x6f, 0x75, 0x6e, 0x74, 0x12, 0x18, 0x0a, 0x07, 0x6e, 0x65, 0x74, 0x77, 0x6f, 0x72, 0x6b, + 0x18, 0x05, 0x20, 0x01, 0x28, 0x09, 0x52, 0x07, 0x6e, 0x65, 0x74, 0x77, 0x6f, 0x72, 0x6b, 0x12, + 0x1e, 0x0a, 0x0a, 0x73, 0x75, 0x62, 0x6e, 0x65, 0x74, 0x77, 0x6f, 0x72, 0x6b, 0x18, 0x06, 0x20, + 0x01, 0x28, 0x09, 0x52, 0x0a, 0x73, 0x75, 0x62, 0x6e, 0x65, 0x74, 0x77, 0x6f, 0x72, 0x6b, 0x12, + 0x2c, 0x0a, 0x11, 0x77, 0x6f, 0x72, 0x6b, 0x65, 0x72, 0x4d, 0x61, 0x63, 0x68, 0x69, 0x6e, 0x65, + 0x54, 0x79, 0x70, 0x65, 0x18, 0x07, 0x20, 0x01, 0x28, 0x09, 0x52, 0x11, 0x77, 0x6f, 0x72, 0x6b, + 0x65, 0x72, 0x4d, 0x61, 0x63, 0x68, 0x69, 0x6e, 0x65, 0x54, 0x79, 0x70, 0x65, 0x12, 0x32, 0x0a, + 0x14, 0x61, 0x75, 0x74, 0x6f, 0x73, 0x63, 0x61, 0x6c, 0x69, 0x6e, 0x67, 0x41, 0x6c, 0x67, 0x6f, + 0x72, 0x69, 0x74, 0x68, 0x6d, 0x18, 0x08, 0x20, 0x01, 0x28, 0x09, 0x52, 0x14, 0x61, 0x75, 0x74, + 0x6f, 0x73, 0x63, 0x61, 0x6c, 0x69, 0x6e, 0x67, 0x41, 0x6c, 0x67, 0x6f, 0x72, 0x69, 0x74, 0x68, + 0x6d, 0x12, 0x22, 0x0a, 0x0c, 0x75, 0x73, 0x65, 0x50, 0x75, 0x62, 0x6c, 0x69, 0x63, 0x49, 0x70, + 0x73, 0x18, 0x09, 0x20, 0x01, 0x28, 0x08, 0x52, 0x0c, 0x75, 0x73, 0x65, 0x50, 0x75, 0x62, 0x6c, + 0x69, 0x63, 0x49, 0x70, 0x73, 0x12, 0x22, 0x0a, 0x0c, 0x74, 0x65, 0x6d, 0x70, 0x4c, 0x6f, 0x63, + 0x61, 0x74, 0x69, 0x6f, 0x6e, 0x18, 0x0a, 0x20, 0x01, 0x28, 0x09, 0x52, 0x0c, 0x74, 0x65, 0x6d, + 0x70, 0x4c, 0x6f, 0x63, 0x61, 0x74, 0x69, 0x6f, 0x6e, 0x12, 0x24, 0x0a, 0x0d, 0x6d, 0x61, 0x78, + 0x4e, 0x75, 0x6d, 0x57, 0x6f, 0x72, 0x6b, 0x65, 0x72, 0x73, 0x18, 0x0b, 0x20, 0x01, 0x28, 0x05, + 0x52, 0x0d, 0x6d, 0x61, 0x78, 0x4e, 0x75, 0x6d, 0x57, 0x6f, 0x72, 0x6b, 0x65, 0x72, 0x73, 0x12, + 0x30, 0x0a, 0x13, 0x64, 0x65, 0x61, 0x64, 0x4c, 0x65, 0x74, 0x74, 0x65, 0x72, 0x54, 0x61, 0x62, + 0x6c, 0x65, 0x53, 0x70, 0x65, 0x63, 0x18, 0x0c, 0x20, 0x01, 0x28, 0x09, 0x52, 0x13, 0x64, 0x65, 0x61, 0x64, 0x4c, 0x65, 0x74, 0x74, 0x65, 0x72, 0x54, 0x61, 0x62, 0x6c, 0x65, 0x53, 0x70, 0x65, - 0x63, 0x18, 0x0c, 0x20, 0x01, 0x28, 0x09, 0x52, 0x13, 0x64, 0x65, 0x61, 0x64, 0x4c, 0x65, 0x74, - 0x74, 0x65, 0x72, 0x54, 0x61, 0x62, 0x6c, 0x65, 0x53, 0x70, 0x65, 0x63, 0x42, 0x54, 0x0a, 0x10, - 0x66, 0x65, 0x61, 0x73, 0x74, 0x2e, 0x70, 0x72, 0x6f, 0x74, 0x6f, 0x2e, 0x63, 0x6f, 0x72, 0x65, - 0x42, 0x0b, 0x52, 0x75, 0x6e, 0x6e, 0x65, 0x72, 0x50, 0x72, 0x6f, 0x74, 0x6f, 0x5a, 0x33, 0x67, - 0x69, 0x74, 0x68, 0x75, 0x62, 0x2e, 0x63, 0x6f, 0x6d, 0x2f, 0x66, 0x65, 0x61, 0x73, 0x74, 0x2d, - 0x64, 0x65, 0x76, 0x2f, 0x66, 0x65, 0x61, 0x73, 0x74, 0x2f, 0x73, 0x64, 0x6b, 0x2f, 0x67, 0x6f, - 0x2f, 0x70, 0x72, 0x6f, 0x74, 0x6f, 0x73, 0x2f, 0x66, 0x65, 0x61, 0x73, 0x74, 0x2f, 0x63, 0x6f, - 0x72, 0x65, 0x62, 0x06, 0x70, 0x72, 0x6f, 0x74, 0x6f, 0x33, + 0x63, 0x12, 0x4b, 0x0a, 0x06, 0x6c, 0x61, 0x62, 0x65, 0x6c, 0x73, 0x18, 0x0d, 0x20, 0x03, 0x28, + 0x0b, 0x32, 0x33, 0x2e, 0x66, 0x65, 0x61, 0x73, 0x74, 0x2e, 0x63, 0x6f, 0x72, 0x65, 0x2e, 0x44, + 0x61, 0x74, 0x61, 0x66, 0x6c, 0x6f, 0x77, 0x52, 0x75, 0x6e, 0x6e, 0x65, 0x72, 0x43, 0x6f, 0x6e, + 0x66, 0x69, 0x67, 0x4f, 0x70, 0x74, 0x69, 0x6f, 0x6e, 0x73, 0x2e, 0x4c, 0x61, 0x62, 0x65, 0x6c, + 0x73, 0x45, 0x6e, 0x74, 0x72, 0x79, 0x52, 0x06, 0x6c, 0x61, 0x62, 0x65, 0x6c, 0x73, 0x1a, 0x39, + 0x0a, 0x0b, 0x4c, 0x61, 0x62, 0x65, 0x6c, 0x73, 0x45, 0x6e, 0x74, 0x72, 0x79, 0x12, 0x10, 0x0a, + 0x03, 0x6b, 0x65, 0x79, 0x18, 0x01, 0x20, 0x01, 0x28, 0x09, 0x52, 0x03, 0x6b, 0x65, 0x79, 0x12, + 0x14, 0x0a, 0x05, 0x76, 0x61, 0x6c, 0x75, 0x65, 0x18, 0x02, 0x20, 0x01, 0x28, 0x09, 0x52, 0x05, + 0x76, 0x61, 0x6c, 0x75, 0x65, 0x3a, 0x02, 0x38, 0x01, 0x42, 0x54, 0x0a, 0x10, 0x66, 0x65, 0x61, + 0x73, 0x74, 0x2e, 0x70, 0x72, 0x6f, 0x74, 0x6f, 0x2e, 0x63, 0x6f, 0x72, 0x65, 0x42, 0x0b, 0x52, + 0x75, 0x6e, 0x6e, 0x65, 0x72, 0x50, 0x72, 0x6f, 0x74, 0x6f, 0x5a, 0x33, 0x67, 0x69, 0x74, 0x68, + 0x75, 0x62, 0x2e, 0x63, 0x6f, 0x6d, 0x2f, 0x66, 0x65, 0x61, 0x73, 0x74, 0x2d, 0x64, 0x65, 0x76, + 0x2f, 0x66, 0x65, 0x61, 0x73, 0x74, 0x2f, 0x73, 0x64, 0x6b, 0x2f, 0x67, 0x6f, 0x2f, 0x70, 0x72, + 0x6f, 0x74, 0x6f, 0x73, 0x2f, 0x66, 0x65, 0x61, 0x73, 0x74, 0x2f, 0x63, 0x6f, 0x72, 0x65, 0x62, + 0x06, 0x70, 0x72, 0x6f, 0x74, 0x6f, 0x33, } var ( @@ -311,17 +341,19 @@ func file_feast_core_Runner_proto_rawDescGZIP() []byte { return file_feast_core_Runner_proto_rawDescData } -var file_feast_core_Runner_proto_msgTypes = make([]protoimpl.MessageInfo, 2) +var file_feast_core_Runner_proto_msgTypes = make([]protoimpl.MessageInfo, 3) var file_feast_core_Runner_proto_goTypes = []interface{}{ (*DirectRunnerConfigOptions)(nil), // 0: feast.core.DirectRunnerConfigOptions (*DataflowRunnerConfigOptions)(nil), // 1: feast.core.DataflowRunnerConfigOptions + nil, // 2: feast.core.DataflowRunnerConfigOptions.LabelsEntry } var file_feast_core_Runner_proto_depIdxs = []int32{ - 0, // [0:0] is the sub-list for method output_type - 0, // [0:0] is the sub-list for method input_type - 0, // [0:0] is the sub-list for extension type_name - 0, // [0:0] is the sub-list for extension extendee - 0, // [0:0] is the sub-list for field type_name + 2, // 0: feast.core.DataflowRunnerConfigOptions.labels:type_name -> feast.core.DataflowRunnerConfigOptions.LabelsEntry + 1, // [1:1] is the sub-list for method output_type + 1, // [1:1] is the sub-list for method input_type + 1, // [1:1] is the sub-list for extension type_name + 1, // [1:1] is the sub-list for extension extendee + 0, // [0:1] is the sub-list for field type_name } func init() { file_feast_core_Runner_proto_init() } @@ -361,7 +393,7 @@ func file_feast_core_Runner_proto_init() { GoPackagePath: reflect.TypeOf(x{}).PkgPath(), RawDescriptor: file_feast_core_Runner_proto_rawDesc, NumEnums: 0, - NumMessages: 2, + NumMessages: 3, NumExtensions: 0, NumServices: 0, }, diff --git a/sdk/go/protos/feast/core/Store.pb.go b/sdk/go/protos/feast/core/Store.pb.go index a36c6afc6ab..829e2e7c180 100644 --- a/sdk/go/protos/feast/core/Store.pb.go +++ b/sdk/go/protos/feast/core/Store.pb.go @@ -377,11 +377,12 @@ type Store_BigQueryConfig struct { sizeCache protoimpl.SizeCache unknownFields protoimpl.UnknownFields - ProjectId string `protobuf:"bytes,1,opt,name=project_id,json=projectId,proto3" json:"project_id,omitempty"` - DatasetId string `protobuf:"bytes,2,opt,name=dataset_id,json=datasetId,proto3" json:"dataset_id,omitempty"` - StagingLocation string `protobuf:"bytes,3,opt,name=staging_location,json=stagingLocation,proto3" json:"staging_location,omitempty"` - InitialRetryDelaySeconds int32 `protobuf:"varint,4,opt,name=initial_retry_delay_seconds,json=initialRetryDelaySeconds,proto3" json:"initial_retry_delay_seconds,omitempty"` - TotalTimeoutSeconds int32 `protobuf:"varint,5,opt,name=total_timeout_seconds,json=totalTimeoutSeconds,proto3" json:"total_timeout_seconds,omitempty"` + ProjectId string `protobuf:"bytes,1,opt,name=project_id,json=projectId,proto3" json:"project_id,omitempty"` + DatasetId string `protobuf:"bytes,2,opt,name=dataset_id,json=datasetId,proto3" json:"dataset_id,omitempty"` + StagingLocation string `protobuf:"bytes,3,opt,name=staging_location,json=stagingLocation,proto3" json:"staging_location,omitempty"` + InitialRetryDelaySeconds int32 `protobuf:"varint,4,opt,name=initial_retry_delay_seconds,json=initialRetryDelaySeconds,proto3" json:"initial_retry_delay_seconds,omitempty"` + TotalTimeoutSeconds int32 `protobuf:"varint,5,opt,name=total_timeout_seconds,json=totalTimeoutSeconds,proto3" json:"total_timeout_seconds,omitempty"` + WriteTriggeringFrequencySeconds int32 `protobuf:"varint,6,opt,name=write_triggering_frequency_seconds,json=writeTriggeringFrequencySeconds,proto3" json:"write_triggering_frequency_seconds,omitempty"` } func (x *Store_BigQueryConfig) Reset() { @@ -451,6 +452,13 @@ func (x *Store_BigQueryConfig) GetTotalTimeoutSeconds() int32 { return 0 } +func (x *Store_BigQueryConfig) GetWriteTriggeringFrequencySeconds() int32 { + if x != nil { + return x.WriteTriggeringFrequencySeconds + } + return 0 +} + type Store_CassandraConfig struct { state protoimpl.MessageState sizeCache protoimpl.SizeCache @@ -643,7 +651,7 @@ var File_feast_core_Store_proto protoreflect.FileDescriptor var file_feast_core_Store_proto_rawDesc = []byte{ 0x0a, 0x16, 0x66, 0x65, 0x61, 0x73, 0x74, 0x2f, 0x63, 0x6f, 0x72, 0x65, 0x2f, 0x53, 0x74, 0x6f, 0x72, 0x65, 0x2e, 0x70, 0x72, 0x6f, 0x74, 0x6f, 0x12, 0x0a, 0x66, 0x65, 0x61, 0x73, 0x74, 0x2e, - 0x63, 0x6f, 0x72, 0x65, 0x22, 0xb4, 0x09, 0x0a, 0x05, 0x53, 0x74, 0x6f, 0x72, 0x65, 0x12, 0x12, + 0x63, 0x6f, 0x72, 0x65, 0x22, 0x81, 0x0a, 0x0a, 0x05, 0x53, 0x74, 0x6f, 0x72, 0x65, 0x12, 0x12, 0x0a, 0x04, 0x6e, 0x61, 0x6d, 0x65, 0x18, 0x01, 0x20, 0x01, 0x28, 0x09, 0x52, 0x04, 0x6e, 0x61, 0x6d, 0x65, 0x12, 0x2f, 0x0a, 0x04, 0x74, 0x79, 0x70, 0x65, 0x18, 0x02, 0x20, 0x01, 0x28, 0x0e, 0x32, 0x1b, 0x2e, 0x66, 0x65, 0x61, 0x73, 0x74, 0x2e, 0x63, 0x6f, 0x72, 0x65, 0x2e, 0x53, 0x74, @@ -680,7 +688,7 @@ var file_feast_core_Store_proto_rawDesc = []byte{ 0x6d, 0x73, 0x18, 0x03, 0x20, 0x01, 0x28, 0x05, 0x52, 0x10, 0x69, 0x6e, 0x69, 0x74, 0x69, 0x61, 0x6c, 0x42, 0x61, 0x63, 0x6b, 0x6f, 0x66, 0x66, 0x4d, 0x73, 0x12, 0x1f, 0x0a, 0x0b, 0x6d, 0x61, 0x78, 0x5f, 0x72, 0x65, 0x74, 0x72, 0x69, 0x65, 0x73, 0x18, 0x04, 0x20, 0x01, 0x28, 0x05, 0x52, - 0x0a, 0x6d, 0x61, 0x78, 0x52, 0x65, 0x74, 0x72, 0x69, 0x65, 0x73, 0x1a, 0xec, 0x01, 0x0a, 0x0e, + 0x0a, 0x6d, 0x61, 0x78, 0x52, 0x65, 0x74, 0x72, 0x69, 0x65, 0x73, 0x1a, 0xb9, 0x02, 0x0a, 0x0e, 0x42, 0x69, 0x67, 0x51, 0x75, 0x65, 0x72, 0x79, 0x43, 0x6f, 0x6e, 0x66, 0x69, 0x67, 0x12, 0x1d, 0x0a, 0x0a, 0x70, 0x72, 0x6f, 0x6a, 0x65, 0x63, 0x74, 0x5f, 0x69, 0x64, 0x18, 0x01, 0x20, 0x01, 0x28, 0x09, 0x52, 0x09, 0x70, 0x72, 0x6f, 0x6a, 0x65, 0x63, 0x74, 0x49, 0x64, 0x12, 0x1d, 0x0a, @@ -695,36 +703,41 @@ var file_feast_core_Store_proto_rawDesc = []byte{ 0x65, 0x63, 0x6f, 0x6e, 0x64, 0x73, 0x12, 0x32, 0x0a, 0x15, 0x74, 0x6f, 0x74, 0x61, 0x6c, 0x5f, 0x74, 0x69, 0x6d, 0x65, 0x6f, 0x75, 0x74, 0x5f, 0x73, 0x65, 0x63, 0x6f, 0x6e, 0x64, 0x73, 0x18, 0x05, 0x20, 0x01, 0x28, 0x05, 0x52, 0x13, 0x74, 0x6f, 0x74, 0x61, 0x6c, 0x54, 0x69, 0x6d, 0x65, - 0x6f, 0x75, 0x74, 0x53, 0x65, 0x63, 0x6f, 0x6e, 0x64, 0x73, 0x1a, 0x39, 0x0a, 0x0f, 0x43, 0x61, - 0x73, 0x73, 0x61, 0x6e, 0x64, 0x72, 0x61, 0x43, 0x6f, 0x6e, 0x66, 0x69, 0x67, 0x12, 0x12, 0x0a, - 0x04, 0x68, 0x6f, 0x73, 0x74, 0x18, 0x01, 0x20, 0x01, 0x28, 0x09, 0x52, 0x04, 0x68, 0x6f, 0x73, - 0x74, 0x12, 0x12, 0x0a, 0x04, 0x70, 0x6f, 0x72, 0x74, 0x18, 0x02, 0x20, 0x01, 0x28, 0x05, 0x52, - 0x04, 0x70, 0x6f, 0x72, 0x74, 0x1a, 0x90, 0x01, 0x0a, 0x12, 0x52, 0x65, 0x64, 0x69, 0x73, 0x43, - 0x6c, 0x75, 0x73, 0x74, 0x65, 0x72, 0x43, 0x6f, 0x6e, 0x66, 0x69, 0x67, 0x12, 0x2b, 0x0a, 0x11, - 0x63, 0x6f, 0x6e, 0x6e, 0x65, 0x63, 0x74, 0x69, 0x6f, 0x6e, 0x5f, 0x73, 0x74, 0x72, 0x69, 0x6e, - 0x67, 0x18, 0x01, 0x20, 0x01, 0x28, 0x09, 0x52, 0x10, 0x63, 0x6f, 0x6e, 0x6e, 0x65, 0x63, 0x74, - 0x69, 0x6f, 0x6e, 0x53, 0x74, 0x72, 0x69, 0x6e, 0x67, 0x12, 0x2c, 0x0a, 0x12, 0x69, 0x6e, 0x69, - 0x74, 0x69, 0x61, 0x6c, 0x5f, 0x62, 0x61, 0x63, 0x6b, 0x6f, 0x66, 0x66, 0x5f, 0x6d, 0x73, 0x18, - 0x02, 0x20, 0x01, 0x28, 0x05, 0x52, 0x10, 0x69, 0x6e, 0x69, 0x74, 0x69, 0x61, 0x6c, 0x42, 0x61, - 0x63, 0x6b, 0x6f, 0x66, 0x66, 0x4d, 0x73, 0x12, 0x1f, 0x0a, 0x0b, 0x6d, 0x61, 0x78, 0x5f, 0x72, - 0x65, 0x74, 0x72, 0x69, 0x65, 0x73, 0x18, 0x03, 0x20, 0x01, 0x28, 0x05, 0x52, 0x0a, 0x6d, 0x61, - 0x78, 0x52, 0x65, 0x74, 0x72, 0x69, 0x65, 0x73, 0x1a, 0x42, 0x0a, 0x0c, 0x53, 0x75, 0x62, 0x73, - 0x63, 0x72, 0x69, 0x70, 0x74, 0x69, 0x6f, 0x6e, 0x12, 0x18, 0x0a, 0x07, 0x70, 0x72, 0x6f, 0x6a, - 0x65, 0x63, 0x74, 0x18, 0x03, 0x20, 0x01, 0x28, 0x09, 0x52, 0x07, 0x70, 0x72, 0x6f, 0x6a, 0x65, - 0x63, 0x74, 0x12, 0x12, 0x0a, 0x04, 0x6e, 0x61, 0x6d, 0x65, 0x18, 0x01, 0x20, 0x01, 0x28, 0x09, - 0x52, 0x04, 0x6e, 0x61, 0x6d, 0x65, 0x4a, 0x04, 0x08, 0x02, 0x10, 0x03, 0x22, 0x53, 0x0a, 0x09, - 0x53, 0x74, 0x6f, 0x72, 0x65, 0x54, 0x79, 0x70, 0x65, 0x12, 0x0b, 0x0a, 0x07, 0x49, 0x4e, 0x56, - 0x41, 0x4c, 0x49, 0x44, 0x10, 0x00, 0x12, 0x09, 0x0a, 0x05, 0x52, 0x45, 0x44, 0x49, 0x53, 0x10, - 0x01, 0x12, 0x0c, 0x0a, 0x08, 0x42, 0x49, 0x47, 0x51, 0x55, 0x45, 0x52, 0x59, 0x10, 0x02, 0x12, - 0x0d, 0x0a, 0x09, 0x43, 0x41, 0x53, 0x53, 0x41, 0x4e, 0x44, 0x52, 0x41, 0x10, 0x03, 0x12, 0x11, - 0x0a, 0x0d, 0x52, 0x45, 0x44, 0x49, 0x53, 0x5f, 0x43, 0x4c, 0x55, 0x53, 0x54, 0x45, 0x52, 0x10, - 0x04, 0x42, 0x08, 0x0a, 0x06, 0x63, 0x6f, 0x6e, 0x66, 0x69, 0x67, 0x42, 0x53, 0x0a, 0x10, 0x66, - 0x65, 0x61, 0x73, 0x74, 0x2e, 0x70, 0x72, 0x6f, 0x74, 0x6f, 0x2e, 0x63, 0x6f, 0x72, 0x65, 0x42, - 0x0a, 0x53, 0x74, 0x6f, 0x72, 0x65, 0x50, 0x72, 0x6f, 0x74, 0x6f, 0x5a, 0x33, 0x67, 0x69, 0x74, - 0x68, 0x75, 0x62, 0x2e, 0x63, 0x6f, 0x6d, 0x2f, 0x66, 0x65, 0x61, 0x73, 0x74, 0x2d, 0x64, 0x65, - 0x76, 0x2f, 0x66, 0x65, 0x61, 0x73, 0x74, 0x2f, 0x73, 0x64, 0x6b, 0x2f, 0x67, 0x6f, 0x2f, 0x70, - 0x72, 0x6f, 0x74, 0x6f, 0x73, 0x2f, 0x66, 0x65, 0x61, 0x73, 0x74, 0x2f, 0x63, 0x6f, 0x72, 0x65, - 0x62, 0x06, 0x70, 0x72, 0x6f, 0x74, 0x6f, 0x33, + 0x6f, 0x75, 0x74, 0x53, 0x65, 0x63, 0x6f, 0x6e, 0x64, 0x73, 0x12, 0x4b, 0x0a, 0x22, 0x77, 0x72, + 0x69, 0x74, 0x65, 0x5f, 0x74, 0x72, 0x69, 0x67, 0x67, 0x65, 0x72, 0x69, 0x6e, 0x67, 0x5f, 0x66, + 0x72, 0x65, 0x71, 0x75, 0x65, 0x6e, 0x63, 0x79, 0x5f, 0x73, 0x65, 0x63, 0x6f, 0x6e, 0x64, 0x73, + 0x18, 0x06, 0x20, 0x01, 0x28, 0x05, 0x52, 0x1f, 0x77, 0x72, 0x69, 0x74, 0x65, 0x54, 0x72, 0x69, + 0x67, 0x67, 0x65, 0x72, 0x69, 0x6e, 0x67, 0x46, 0x72, 0x65, 0x71, 0x75, 0x65, 0x6e, 0x63, 0x79, + 0x53, 0x65, 0x63, 0x6f, 0x6e, 0x64, 0x73, 0x1a, 0x39, 0x0a, 0x0f, 0x43, 0x61, 0x73, 0x73, 0x61, + 0x6e, 0x64, 0x72, 0x61, 0x43, 0x6f, 0x6e, 0x66, 0x69, 0x67, 0x12, 0x12, 0x0a, 0x04, 0x68, 0x6f, + 0x73, 0x74, 0x18, 0x01, 0x20, 0x01, 0x28, 0x09, 0x52, 0x04, 0x68, 0x6f, 0x73, 0x74, 0x12, 0x12, + 0x0a, 0x04, 0x70, 0x6f, 0x72, 0x74, 0x18, 0x02, 0x20, 0x01, 0x28, 0x05, 0x52, 0x04, 0x70, 0x6f, + 0x72, 0x74, 0x1a, 0x90, 0x01, 0x0a, 0x12, 0x52, 0x65, 0x64, 0x69, 0x73, 0x43, 0x6c, 0x75, 0x73, + 0x74, 0x65, 0x72, 0x43, 0x6f, 0x6e, 0x66, 0x69, 0x67, 0x12, 0x2b, 0x0a, 0x11, 0x63, 0x6f, 0x6e, + 0x6e, 0x65, 0x63, 0x74, 0x69, 0x6f, 0x6e, 0x5f, 0x73, 0x74, 0x72, 0x69, 0x6e, 0x67, 0x18, 0x01, + 0x20, 0x01, 0x28, 0x09, 0x52, 0x10, 0x63, 0x6f, 0x6e, 0x6e, 0x65, 0x63, 0x74, 0x69, 0x6f, 0x6e, + 0x53, 0x74, 0x72, 0x69, 0x6e, 0x67, 0x12, 0x2c, 0x0a, 0x12, 0x69, 0x6e, 0x69, 0x74, 0x69, 0x61, + 0x6c, 0x5f, 0x62, 0x61, 0x63, 0x6b, 0x6f, 0x66, 0x66, 0x5f, 0x6d, 0x73, 0x18, 0x02, 0x20, 0x01, + 0x28, 0x05, 0x52, 0x10, 0x69, 0x6e, 0x69, 0x74, 0x69, 0x61, 0x6c, 0x42, 0x61, 0x63, 0x6b, 0x6f, + 0x66, 0x66, 0x4d, 0x73, 0x12, 0x1f, 0x0a, 0x0b, 0x6d, 0x61, 0x78, 0x5f, 0x72, 0x65, 0x74, 0x72, + 0x69, 0x65, 0x73, 0x18, 0x03, 0x20, 0x01, 0x28, 0x05, 0x52, 0x0a, 0x6d, 0x61, 0x78, 0x52, 0x65, + 0x74, 0x72, 0x69, 0x65, 0x73, 0x1a, 0x42, 0x0a, 0x0c, 0x53, 0x75, 0x62, 0x73, 0x63, 0x72, 0x69, + 0x70, 0x74, 0x69, 0x6f, 0x6e, 0x12, 0x18, 0x0a, 0x07, 0x70, 0x72, 0x6f, 0x6a, 0x65, 0x63, 0x74, + 0x18, 0x03, 0x20, 0x01, 0x28, 0x09, 0x52, 0x07, 0x70, 0x72, 0x6f, 0x6a, 0x65, 0x63, 0x74, 0x12, + 0x12, 0x0a, 0x04, 0x6e, 0x61, 0x6d, 0x65, 0x18, 0x01, 0x20, 0x01, 0x28, 0x09, 0x52, 0x04, 0x6e, + 0x61, 0x6d, 0x65, 0x4a, 0x04, 0x08, 0x02, 0x10, 0x03, 0x22, 0x53, 0x0a, 0x09, 0x53, 0x74, 0x6f, + 0x72, 0x65, 0x54, 0x79, 0x70, 0x65, 0x12, 0x0b, 0x0a, 0x07, 0x49, 0x4e, 0x56, 0x41, 0x4c, 0x49, + 0x44, 0x10, 0x00, 0x12, 0x09, 0x0a, 0x05, 0x52, 0x45, 0x44, 0x49, 0x53, 0x10, 0x01, 0x12, 0x0c, + 0x0a, 0x08, 0x42, 0x49, 0x47, 0x51, 0x55, 0x45, 0x52, 0x59, 0x10, 0x02, 0x12, 0x0d, 0x0a, 0x09, + 0x43, 0x41, 0x53, 0x53, 0x41, 0x4e, 0x44, 0x52, 0x41, 0x10, 0x03, 0x12, 0x11, 0x0a, 0x0d, 0x52, + 0x45, 0x44, 0x49, 0x53, 0x5f, 0x43, 0x4c, 0x55, 0x53, 0x54, 0x45, 0x52, 0x10, 0x04, 0x42, 0x08, + 0x0a, 0x06, 0x63, 0x6f, 0x6e, 0x66, 0x69, 0x67, 0x42, 0x53, 0x0a, 0x10, 0x66, 0x65, 0x61, 0x73, + 0x74, 0x2e, 0x70, 0x72, 0x6f, 0x74, 0x6f, 0x2e, 0x63, 0x6f, 0x72, 0x65, 0x42, 0x0a, 0x53, 0x74, + 0x6f, 0x72, 0x65, 0x50, 0x72, 0x6f, 0x74, 0x6f, 0x5a, 0x33, 0x67, 0x69, 0x74, 0x68, 0x75, 0x62, + 0x2e, 0x63, 0x6f, 0x6d, 0x2f, 0x66, 0x65, 0x61, 0x73, 0x74, 0x2d, 0x64, 0x65, 0x76, 0x2f, 0x66, + 0x65, 0x61, 0x73, 0x74, 0x2f, 0x73, 0x64, 0x6b, 0x2f, 0x67, 0x6f, 0x2f, 0x70, 0x72, 0x6f, 0x74, + 0x6f, 0x73, 0x2f, 0x66, 0x65, 0x61, 0x73, 0x74, 0x2f, 0x63, 0x6f, 0x72, 0x65, 0x62, 0x06, 0x70, + 0x72, 0x6f, 0x74, 0x6f, 0x33, } var ( diff --git a/sdk/go/protos/feast/serving/ServingService.pb.go b/sdk/go/protos/feast/serving/ServingService.pb.go index fc2748d5d84..c8b09a352b5 100644 --- a/sdk/go/protos/feast/serving/ServingService.pb.go +++ b/sdk/go/protos/feast/serving/ServingService.pb.go @@ -243,6 +243,70 @@ func (DataFormat) EnumDescriptor() ([]byte, []int) { return file_feast_serving_ServingService_proto_rawDescGZIP(), []int{3} } +type GetOnlineFeaturesResponse_FieldStatus int32 + +const ( + // Status is unset for this field. + GetOnlineFeaturesResponse_INVALID GetOnlineFeaturesResponse_FieldStatus = 0 + // Field value is present for this field and age is within max age. + GetOnlineFeaturesResponse_PRESENT GetOnlineFeaturesResponse_FieldStatus = 1 + // Values could be found for entity key and age is within max age, but + // this field value is assigned a value on ingestion into feast. + GetOnlineFeaturesResponse_NULL_VALUE GetOnlineFeaturesResponse_FieldStatus = 2 + // Entity key did not return any values as they do not exist in Feast. + // This could suggest that the feature values have not yet been ingested + // into feast or the ingestion failed. + GetOnlineFeaturesResponse_NOT_FOUND GetOnlineFeaturesResponse_FieldStatus = 3 + // Values could be found for entity key, but field values are outside the maximum + // allowable range. + GetOnlineFeaturesResponse_OUTSIDE_MAX_AGE GetOnlineFeaturesResponse_FieldStatus = 4 +) + +// Enum value maps for GetOnlineFeaturesResponse_FieldStatus. +var ( + GetOnlineFeaturesResponse_FieldStatus_name = map[int32]string{ + 0: "INVALID", + 1: "PRESENT", + 2: "NULL_VALUE", + 3: "NOT_FOUND", + 4: "OUTSIDE_MAX_AGE", + } + GetOnlineFeaturesResponse_FieldStatus_value = map[string]int32{ + "INVALID": 0, + "PRESENT": 1, + "NULL_VALUE": 2, + "NOT_FOUND": 3, + "OUTSIDE_MAX_AGE": 4, + } +) + +func (x GetOnlineFeaturesResponse_FieldStatus) Enum() *GetOnlineFeaturesResponse_FieldStatus { + p := new(GetOnlineFeaturesResponse_FieldStatus) + *p = x + return p +} + +func (x GetOnlineFeaturesResponse_FieldStatus) String() string { + return protoimpl.X.EnumStringOf(x.Descriptor(), protoreflect.EnumNumber(x)) +} + +func (GetOnlineFeaturesResponse_FieldStatus) Descriptor() protoreflect.EnumDescriptor { + return file_feast_serving_ServingService_proto_enumTypes[4].Descriptor() +} + +func (GetOnlineFeaturesResponse_FieldStatus) Type() protoreflect.EnumType { + return &file_feast_serving_ServingService_proto_enumTypes[4] +} + +func (x GetOnlineFeaturesResponse_FieldStatus) Number() protoreflect.EnumNumber { + return protoreflect.EnumNumber(x) +} + +// Deprecated: Use GetOnlineFeaturesResponse_FieldStatus.Descriptor instead. +func (GetOnlineFeaturesResponse_FieldStatus) EnumDescriptor() ([]byte, []int) { + return file_feast_serving_ServingService_proto_rawDescGZIP(), []int{4, 0} +} + type GetFeastServingInfoRequest struct { state protoimpl.MessageState sizeCache protoimpl.SizeCache @@ -486,20 +550,17 @@ func (x *GetOnlineFeaturesRequest) GetOmitEntitiesInResponse() bool { return false } -type GetBatchFeaturesRequest struct { +type GetOnlineFeaturesResponse struct { state protoimpl.MessageState sizeCache protoimpl.SizeCache unknownFields protoimpl.UnknownFields - // List of features that are being retrieved - Features []*FeatureReference `protobuf:"bytes,3,rep,name=features,proto3" json:"features,omitempty"` - // Source of the entity dataset containing the timestamps and entity keys to retrieve - // features for. - DatasetSource *DatasetSource `protobuf:"bytes,2,opt,name=dataset_source,json=datasetSource,proto3" json:"dataset_source,omitempty"` + // Feature values retrieved from feast. + FieldValues []*GetOnlineFeaturesResponse_FieldValues `protobuf:"bytes,1,rep,name=field_values,json=fieldValues,proto3" json:"field_values,omitempty"` } -func (x *GetBatchFeaturesRequest) Reset() { - *x = GetBatchFeaturesRequest{} +func (x *GetOnlineFeaturesResponse) Reset() { + *x = GetOnlineFeaturesResponse{} if protoimpl.UnsafeEnabled { mi := &file_feast_serving_ServingService_proto_msgTypes[4] ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x)) @@ -507,13 +568,13 @@ func (x *GetBatchFeaturesRequest) Reset() { } } -func (x *GetBatchFeaturesRequest) String() string { +func (x *GetOnlineFeaturesResponse) String() string { return protoimpl.X.MessageStringOf(x) } -func (*GetBatchFeaturesRequest) ProtoMessage() {} +func (*GetOnlineFeaturesResponse) ProtoMessage() {} -func (x *GetBatchFeaturesRequest) ProtoReflect() protoreflect.Message { +func (x *GetOnlineFeaturesResponse) ProtoReflect() protoreflect.Message { mi := &file_feast_serving_ServingService_proto_msgTypes[4] if protoimpl.UnsafeEnabled && x != nil { ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x)) @@ -525,36 +586,32 @@ func (x *GetBatchFeaturesRequest) ProtoReflect() protoreflect.Message { return mi.MessageOf(x) } -// Deprecated: Use GetBatchFeaturesRequest.ProtoReflect.Descriptor instead. -func (*GetBatchFeaturesRequest) Descriptor() ([]byte, []int) { +// Deprecated: Use GetOnlineFeaturesResponse.ProtoReflect.Descriptor instead. +func (*GetOnlineFeaturesResponse) Descriptor() ([]byte, []int) { return file_feast_serving_ServingService_proto_rawDescGZIP(), []int{4} } -func (x *GetBatchFeaturesRequest) GetFeatures() []*FeatureReference { - if x != nil { - return x.Features - } - return nil -} - -func (x *GetBatchFeaturesRequest) GetDatasetSource() *DatasetSource { +func (x *GetOnlineFeaturesResponse) GetFieldValues() []*GetOnlineFeaturesResponse_FieldValues { if x != nil { - return x.DatasetSource + return x.FieldValues } return nil } -type GetOnlineFeaturesResponse struct { +type GetBatchFeaturesRequest struct { state protoimpl.MessageState sizeCache protoimpl.SizeCache unknownFields protoimpl.UnknownFields - // Feature values retrieved from feast. - FieldValues []*GetOnlineFeaturesResponse_FieldValues `protobuf:"bytes,1,rep,name=field_values,json=fieldValues,proto3" json:"field_values,omitempty"` + // List of features that are being retrieved + Features []*FeatureReference `protobuf:"bytes,3,rep,name=features,proto3" json:"features,omitempty"` + // Source of the entity dataset containing the timestamps and entity keys to retrieve + // features for. + DatasetSource *DatasetSource `protobuf:"bytes,2,opt,name=dataset_source,json=datasetSource,proto3" json:"dataset_source,omitempty"` } -func (x *GetOnlineFeaturesResponse) Reset() { - *x = GetOnlineFeaturesResponse{} +func (x *GetBatchFeaturesRequest) Reset() { + *x = GetBatchFeaturesRequest{} if protoimpl.UnsafeEnabled { mi := &file_feast_serving_ServingService_proto_msgTypes[5] ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x)) @@ -562,13 +619,13 @@ func (x *GetOnlineFeaturesResponse) Reset() { } } -func (x *GetOnlineFeaturesResponse) String() string { +func (x *GetBatchFeaturesRequest) String() string { return protoimpl.X.MessageStringOf(x) } -func (*GetOnlineFeaturesResponse) ProtoMessage() {} +func (*GetBatchFeaturesRequest) ProtoMessage() {} -func (x *GetOnlineFeaturesResponse) ProtoReflect() protoreflect.Message { +func (x *GetBatchFeaturesRequest) ProtoReflect() protoreflect.Message { mi := &file_feast_serving_ServingService_proto_msgTypes[5] if protoimpl.UnsafeEnabled && x != nil { ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x)) @@ -580,14 +637,21 @@ func (x *GetOnlineFeaturesResponse) ProtoReflect() protoreflect.Message { return mi.MessageOf(x) } -// Deprecated: Use GetOnlineFeaturesResponse.ProtoReflect.Descriptor instead. -func (*GetOnlineFeaturesResponse) Descriptor() ([]byte, []int) { +// Deprecated: Use GetBatchFeaturesRequest.ProtoReflect.Descriptor instead. +func (*GetBatchFeaturesRequest) Descriptor() ([]byte, []int) { return file_feast_serving_ServingService_proto_rawDescGZIP(), []int{5} } -func (x *GetOnlineFeaturesResponse) GetFieldValues() []*GetOnlineFeaturesResponse_FieldValues { +func (x *GetBatchFeaturesRequest) GetFeatures() []*FeatureReference { if x != nil { - return x.FieldValues + return x.Features + } + return nil +} + +func (x *GetBatchFeaturesRequest) GetDatasetSource() *DatasetSource { + if x != nil { + return x.DatasetSource } return nil } @@ -899,8 +963,8 @@ type GetOnlineFeaturesRequest_EntityRow struct { sizeCache protoimpl.SizeCache unknownFields protoimpl.UnknownFields - // Request timestamp of this row. This value will be used, together with maxAge, - // to determine feature staleness. + // Request timestamp of this row. This value will be used, + // together with maxAge, to determine feature staleness. EntityTimestamp *timestamp.Timestamp `protobuf:"bytes,1,opt,name=entity_timestamp,json=entityTimestamp,proto3" json:"entity_timestamp,omitempty"` // Map containing mapping of entity name to entity value. Fields map[string]*types.Value `protobuf:"bytes,2,rep,name=fields,proto3" json:"fields,omitempty" protobuf_key:"bytes,1,opt,name=key,proto3" protobuf_val:"bytes,2,opt,name=value,proto3"` @@ -960,6 +1024,8 @@ type GetOnlineFeaturesResponse_FieldValues struct { // Map of feature or entity name to feature/entity values. // Timestamps are not returned in this response. Fields map[string]*types.Value `protobuf:"bytes,1,rep,name=fields,proto3" json:"fields,omitempty" protobuf_key:"bytes,1,opt,name=key,proto3" protobuf_val:"bytes,2,opt,name=value,proto3"` + // Map of feature or entity name to feature/entity statuses/metadata. + Statuses map[string]GetOnlineFeaturesResponse_FieldStatus `protobuf:"bytes,2,rep,name=statuses,proto3" json:"statuses,omitempty" protobuf_key:"bytes,1,opt,name=key,proto3" protobuf_val:"varint,2,opt,name=value,proto3,enum=feast.serving.GetOnlineFeaturesResponse_FieldStatus"` } func (x *GetOnlineFeaturesResponse_FieldValues) Reset() { @@ -991,7 +1057,7 @@ func (x *GetOnlineFeaturesResponse_FieldValues) ProtoReflect() protoreflect.Mess // Deprecated: Use GetOnlineFeaturesResponse_FieldValues.ProtoReflect.Descriptor instead. func (*GetOnlineFeaturesResponse_FieldValues) Descriptor() ([]byte, []int) { - return file_feast_serving_ServingService_proto_rawDescGZIP(), []int{5, 0} + return file_feast_serving_ServingService_proto_rawDescGZIP(), []int{4, 0} } func (x *GetOnlineFeaturesResponse_FieldValues) GetFields() map[string]*types.Value { @@ -1001,6 +1067,13 @@ func (x *GetOnlineFeaturesResponse_FieldValues) GetFields() map[string]*types.Va return nil } +func (x *GetOnlineFeaturesResponse_FieldValues) GetStatuses() map[string]GetOnlineFeaturesResponse_FieldStatus { + if x != nil { + return x.Statuses + } + return nil +} + type DatasetSource_FileSource struct { state protoimpl.MessageState sizeCache protoimpl.SizeCache @@ -1017,7 +1090,7 @@ type DatasetSource_FileSource struct { func (x *DatasetSource_FileSource) Reset() { *x = DatasetSource_FileSource{} if protoimpl.UnsafeEnabled { - mi := &file_feast_serving_ServingService_proto_msgTypes[15] + mi := &file_feast_serving_ServingService_proto_msgTypes[16] ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x)) ms.StoreMessageInfo(mi) } @@ -1030,7 +1103,7 @@ func (x *DatasetSource_FileSource) String() string { func (*DatasetSource_FileSource) ProtoMessage() {} func (x *DatasetSource_FileSource) ProtoReflect() protoreflect.Message { - mi := &file_feast_serving_ServingService_proto_msgTypes[15] + mi := &file_feast_serving_ServingService_proto_msgTypes[16] if protoimpl.UnsafeEnabled && x != nil { ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x)) if ms.LoadMessageInfo() == nil { @@ -1119,6 +1192,44 @@ var file_feast_serving_ServingService_proto_rawDesc = []byte{ 0x79, 0x12, 0x28, 0x0a, 0x05, 0x76, 0x61, 0x6c, 0x75, 0x65, 0x18, 0x02, 0x20, 0x01, 0x28, 0x0b, 0x32, 0x12, 0x2e, 0x66, 0x65, 0x61, 0x73, 0x74, 0x2e, 0x74, 0x79, 0x70, 0x65, 0x73, 0x2e, 0x56, 0x61, 0x6c, 0x75, 0x65, 0x52, 0x05, 0x76, 0x61, 0x6c, 0x75, 0x65, 0x3a, 0x02, 0x38, 0x01, 0x22, + 0xdd, 0x04, 0x0a, 0x19, 0x47, 0x65, 0x74, 0x4f, 0x6e, 0x6c, 0x69, 0x6e, 0x65, 0x46, 0x65, 0x61, + 0x74, 0x75, 0x72, 0x65, 0x73, 0x52, 0x65, 0x73, 0x70, 0x6f, 0x6e, 0x73, 0x65, 0x12, 0x57, 0x0a, + 0x0c, 0x66, 0x69, 0x65, 0x6c, 0x64, 0x5f, 0x76, 0x61, 0x6c, 0x75, 0x65, 0x73, 0x18, 0x01, 0x20, + 0x03, 0x28, 0x0b, 0x32, 0x34, 0x2e, 0x66, 0x65, 0x61, 0x73, 0x74, 0x2e, 0x73, 0x65, 0x72, 0x76, + 0x69, 0x6e, 0x67, 0x2e, 0x47, 0x65, 0x74, 0x4f, 0x6e, 0x6c, 0x69, 0x6e, 0x65, 0x46, 0x65, 0x61, + 0x74, 0x75, 0x72, 0x65, 0x73, 0x52, 0x65, 0x73, 0x70, 0x6f, 0x6e, 0x73, 0x65, 0x2e, 0x46, 0x69, + 0x65, 0x6c, 0x64, 0x56, 0x61, 0x6c, 0x75, 0x65, 0x73, 0x52, 0x0b, 0x66, 0x69, 0x65, 0x6c, 0x64, + 0x56, 0x61, 0x6c, 0x75, 0x65, 0x73, 0x1a, 0x89, 0x03, 0x0a, 0x0b, 0x46, 0x69, 0x65, 0x6c, 0x64, + 0x56, 0x61, 0x6c, 0x75, 0x65, 0x73, 0x12, 0x58, 0x0a, 0x06, 0x66, 0x69, 0x65, 0x6c, 0x64, 0x73, + 0x18, 0x01, 0x20, 0x03, 0x28, 0x0b, 0x32, 0x40, 0x2e, 0x66, 0x65, 0x61, 0x73, 0x74, 0x2e, 0x73, + 0x65, 0x72, 0x76, 0x69, 0x6e, 0x67, 0x2e, 0x47, 0x65, 0x74, 0x4f, 0x6e, 0x6c, 0x69, 0x6e, 0x65, + 0x46, 0x65, 0x61, 0x74, 0x75, 0x72, 0x65, 0x73, 0x52, 0x65, 0x73, 0x70, 0x6f, 0x6e, 0x73, 0x65, + 0x2e, 0x46, 0x69, 0x65, 0x6c, 0x64, 0x56, 0x61, 0x6c, 0x75, 0x65, 0x73, 0x2e, 0x46, 0x69, 0x65, + 0x6c, 0x64, 0x73, 0x45, 0x6e, 0x74, 0x72, 0x79, 0x52, 0x06, 0x66, 0x69, 0x65, 0x6c, 0x64, 0x73, + 0x12, 0x5e, 0x0a, 0x08, 0x73, 0x74, 0x61, 0x74, 0x75, 0x73, 0x65, 0x73, 0x18, 0x02, 0x20, 0x03, + 0x28, 0x0b, 0x32, 0x42, 0x2e, 0x66, 0x65, 0x61, 0x73, 0x74, 0x2e, 0x73, 0x65, 0x72, 0x76, 0x69, + 0x6e, 0x67, 0x2e, 0x47, 0x65, 0x74, 0x4f, 0x6e, 0x6c, 0x69, 0x6e, 0x65, 0x46, 0x65, 0x61, 0x74, + 0x75, 0x72, 0x65, 0x73, 0x52, 0x65, 0x73, 0x70, 0x6f, 0x6e, 0x73, 0x65, 0x2e, 0x46, 0x69, 0x65, + 0x6c, 0x64, 0x56, 0x61, 0x6c, 0x75, 0x65, 0x73, 0x2e, 0x53, 0x74, 0x61, 0x74, 0x75, 0x73, 0x65, + 0x73, 0x45, 0x6e, 0x74, 0x72, 0x79, 0x52, 0x08, 0x73, 0x74, 0x61, 0x74, 0x75, 0x73, 0x65, 0x73, + 0x1a, 0x4d, 0x0a, 0x0b, 0x46, 0x69, 0x65, 0x6c, 0x64, 0x73, 0x45, 0x6e, 0x74, 0x72, 0x79, 0x12, + 0x10, 0x0a, 0x03, 0x6b, 0x65, 0x79, 0x18, 0x01, 0x20, 0x01, 0x28, 0x09, 0x52, 0x03, 0x6b, 0x65, + 0x79, 0x12, 0x28, 0x0a, 0x05, 0x76, 0x61, 0x6c, 0x75, 0x65, 0x18, 0x02, 0x20, 0x01, 0x28, 0x0b, + 0x32, 0x12, 0x2e, 0x66, 0x65, 0x61, 0x73, 0x74, 0x2e, 0x74, 0x79, 0x70, 0x65, 0x73, 0x2e, 0x56, + 0x61, 0x6c, 0x75, 0x65, 0x52, 0x05, 0x76, 0x61, 0x6c, 0x75, 0x65, 0x3a, 0x02, 0x38, 0x01, 0x1a, + 0x71, 0x0a, 0x0d, 0x53, 0x74, 0x61, 0x74, 0x75, 0x73, 0x65, 0x73, 0x45, 0x6e, 0x74, 0x72, 0x79, + 0x12, 0x10, 0x0a, 0x03, 0x6b, 0x65, 0x79, 0x18, 0x01, 0x20, 0x01, 0x28, 0x09, 0x52, 0x03, 0x6b, + 0x65, 0x79, 0x12, 0x4a, 0x0a, 0x05, 0x76, 0x61, 0x6c, 0x75, 0x65, 0x18, 0x02, 0x20, 0x01, 0x28, + 0x0e, 0x32, 0x34, 0x2e, 0x66, 0x65, 0x61, 0x73, 0x74, 0x2e, 0x73, 0x65, 0x72, 0x76, 0x69, 0x6e, + 0x67, 0x2e, 0x47, 0x65, 0x74, 0x4f, 0x6e, 0x6c, 0x69, 0x6e, 0x65, 0x46, 0x65, 0x61, 0x74, 0x75, + 0x72, 0x65, 0x73, 0x52, 0x65, 0x73, 0x70, 0x6f, 0x6e, 0x73, 0x65, 0x2e, 0x46, 0x69, 0x65, 0x6c, + 0x64, 0x53, 0x74, 0x61, 0x74, 0x75, 0x73, 0x52, 0x05, 0x76, 0x61, 0x6c, 0x75, 0x65, 0x3a, 0x02, + 0x38, 0x01, 0x22, 0x5b, 0x0a, 0x0b, 0x46, 0x69, 0x65, 0x6c, 0x64, 0x53, 0x74, 0x61, 0x74, 0x75, + 0x73, 0x12, 0x0b, 0x0a, 0x07, 0x49, 0x4e, 0x56, 0x41, 0x4c, 0x49, 0x44, 0x10, 0x00, 0x12, 0x0b, + 0x0a, 0x07, 0x50, 0x52, 0x45, 0x53, 0x45, 0x4e, 0x54, 0x10, 0x01, 0x12, 0x0e, 0x0a, 0x0a, 0x4e, + 0x55, 0x4c, 0x4c, 0x5f, 0x56, 0x41, 0x4c, 0x55, 0x45, 0x10, 0x02, 0x12, 0x0d, 0x0a, 0x09, 0x4e, + 0x4f, 0x54, 0x5f, 0x46, 0x4f, 0x55, 0x4e, 0x44, 0x10, 0x03, 0x12, 0x13, 0x0a, 0x0f, 0x4f, 0x55, + 0x54, 0x53, 0x49, 0x44, 0x45, 0x5f, 0x4d, 0x41, 0x58, 0x5f, 0x41, 0x47, 0x45, 0x10, 0x04, 0x22, 0x9b, 0x01, 0x0a, 0x17, 0x47, 0x65, 0x74, 0x42, 0x61, 0x74, 0x63, 0x68, 0x46, 0x65, 0x61, 0x74, 0x75, 0x72, 0x65, 0x73, 0x52, 0x65, 0x71, 0x75, 0x65, 0x73, 0x74, 0x12, 0x3b, 0x0a, 0x08, 0x66, 0x65, 0x61, 0x74, 0x75, 0x72, 0x65, 0x73, 0x18, 0x03, 0x20, 0x03, 0x28, 0x0b, 0x32, 0x1f, 0x2e, @@ -1128,26 +1239,7 @@ var file_feast_serving_ServingService_proto_rawDesc = []byte{ 0x73, 0x65, 0x74, 0x5f, 0x73, 0x6f, 0x75, 0x72, 0x63, 0x65, 0x18, 0x02, 0x20, 0x01, 0x28, 0x0b, 0x32, 0x1c, 0x2e, 0x66, 0x65, 0x61, 0x73, 0x74, 0x2e, 0x73, 0x65, 0x72, 0x76, 0x69, 0x6e, 0x67, 0x2e, 0x44, 0x61, 0x74, 0x61, 0x73, 0x65, 0x74, 0x53, 0x6f, 0x75, 0x72, 0x63, 0x65, 0x52, 0x0d, - 0x64, 0x61, 0x74, 0x61, 0x73, 0x65, 0x74, 0x53, 0x6f, 0x75, 0x72, 0x63, 0x65, 0x22, 0xad, 0x02, - 0x0a, 0x19, 0x47, 0x65, 0x74, 0x4f, 0x6e, 0x6c, 0x69, 0x6e, 0x65, 0x46, 0x65, 0x61, 0x74, 0x75, - 0x72, 0x65, 0x73, 0x52, 0x65, 0x73, 0x70, 0x6f, 0x6e, 0x73, 0x65, 0x12, 0x57, 0x0a, 0x0c, 0x66, - 0x69, 0x65, 0x6c, 0x64, 0x5f, 0x76, 0x61, 0x6c, 0x75, 0x65, 0x73, 0x18, 0x01, 0x20, 0x03, 0x28, - 0x0b, 0x32, 0x34, 0x2e, 0x66, 0x65, 0x61, 0x73, 0x74, 0x2e, 0x73, 0x65, 0x72, 0x76, 0x69, 0x6e, - 0x67, 0x2e, 0x47, 0x65, 0x74, 0x4f, 0x6e, 0x6c, 0x69, 0x6e, 0x65, 0x46, 0x65, 0x61, 0x74, 0x75, - 0x72, 0x65, 0x73, 0x52, 0x65, 0x73, 0x70, 0x6f, 0x6e, 0x73, 0x65, 0x2e, 0x46, 0x69, 0x65, 0x6c, - 0x64, 0x56, 0x61, 0x6c, 0x75, 0x65, 0x73, 0x52, 0x0b, 0x66, 0x69, 0x65, 0x6c, 0x64, 0x56, 0x61, - 0x6c, 0x75, 0x65, 0x73, 0x1a, 0xb6, 0x01, 0x0a, 0x0b, 0x46, 0x69, 0x65, 0x6c, 0x64, 0x56, 0x61, - 0x6c, 0x75, 0x65, 0x73, 0x12, 0x58, 0x0a, 0x06, 0x66, 0x69, 0x65, 0x6c, 0x64, 0x73, 0x18, 0x01, - 0x20, 0x03, 0x28, 0x0b, 0x32, 0x40, 0x2e, 0x66, 0x65, 0x61, 0x73, 0x74, 0x2e, 0x73, 0x65, 0x72, - 0x76, 0x69, 0x6e, 0x67, 0x2e, 0x47, 0x65, 0x74, 0x4f, 0x6e, 0x6c, 0x69, 0x6e, 0x65, 0x46, 0x65, - 0x61, 0x74, 0x75, 0x72, 0x65, 0x73, 0x52, 0x65, 0x73, 0x70, 0x6f, 0x6e, 0x73, 0x65, 0x2e, 0x46, - 0x69, 0x65, 0x6c, 0x64, 0x56, 0x61, 0x6c, 0x75, 0x65, 0x73, 0x2e, 0x46, 0x69, 0x65, 0x6c, 0x64, - 0x73, 0x45, 0x6e, 0x74, 0x72, 0x79, 0x52, 0x06, 0x66, 0x69, 0x65, 0x6c, 0x64, 0x73, 0x1a, 0x4d, - 0x0a, 0x0b, 0x46, 0x69, 0x65, 0x6c, 0x64, 0x73, 0x45, 0x6e, 0x74, 0x72, 0x79, 0x12, 0x10, 0x0a, - 0x03, 0x6b, 0x65, 0x79, 0x18, 0x01, 0x20, 0x01, 0x28, 0x09, 0x52, 0x03, 0x6b, 0x65, 0x79, 0x12, - 0x28, 0x0a, 0x05, 0x76, 0x61, 0x6c, 0x75, 0x65, 0x18, 0x02, 0x20, 0x01, 0x28, 0x0b, 0x32, 0x12, - 0x2e, 0x66, 0x65, 0x61, 0x73, 0x74, 0x2e, 0x74, 0x79, 0x70, 0x65, 0x73, 0x2e, 0x56, 0x61, 0x6c, - 0x75, 0x65, 0x52, 0x05, 0x76, 0x61, 0x6c, 0x75, 0x65, 0x3a, 0x02, 0x38, 0x01, 0x22, 0x40, 0x0a, + 0x64, 0x61, 0x74, 0x61, 0x73, 0x65, 0x74, 0x53, 0x6f, 0x75, 0x72, 0x63, 0x65, 0x22, 0x40, 0x0a, 0x18, 0x47, 0x65, 0x74, 0x42, 0x61, 0x74, 0x63, 0x68, 0x46, 0x65, 0x61, 0x74, 0x75, 0x72, 0x65, 0x73, 0x52, 0x65, 0x73, 0x70, 0x6f, 0x6e, 0x73, 0x65, 0x12, 0x24, 0x0a, 0x03, 0x6a, 0x6f, 0x62, 0x18, 0x01, 0x20, 0x01, 0x28, 0x0b, 0x32, 0x12, 0x2e, 0x66, 0x65, 0x61, 0x73, 0x74, 0x2e, 0x73, @@ -1254,65 +1346,69 @@ func file_feast_serving_ServingService_proto_rawDescGZIP() []byte { return file_feast_serving_ServingService_proto_rawDescData } -var file_feast_serving_ServingService_proto_enumTypes = make([]protoimpl.EnumInfo, 4) -var file_feast_serving_ServingService_proto_msgTypes = make([]protoimpl.MessageInfo, 16) +var file_feast_serving_ServingService_proto_enumTypes = make([]protoimpl.EnumInfo, 5) +var file_feast_serving_ServingService_proto_msgTypes = make([]protoimpl.MessageInfo, 17) var file_feast_serving_ServingService_proto_goTypes = []interface{}{ - (FeastServingType)(0), // 0: feast.serving.FeastServingType - (JobType)(0), // 1: feast.serving.JobType - (JobStatus)(0), // 2: feast.serving.JobStatus - (DataFormat)(0), // 3: feast.serving.DataFormat - (*GetFeastServingInfoRequest)(nil), // 4: feast.serving.GetFeastServingInfoRequest - (*GetFeastServingInfoResponse)(nil), // 5: feast.serving.GetFeastServingInfoResponse - (*FeatureReference)(nil), // 6: feast.serving.FeatureReference - (*GetOnlineFeaturesRequest)(nil), // 7: feast.serving.GetOnlineFeaturesRequest - (*GetBatchFeaturesRequest)(nil), // 8: feast.serving.GetBatchFeaturesRequest + (FeastServingType)(0), // 0: feast.serving.FeastServingType + (JobType)(0), // 1: feast.serving.JobType + (JobStatus)(0), // 2: feast.serving.JobStatus + (DataFormat)(0), // 3: feast.serving.DataFormat + (GetOnlineFeaturesResponse_FieldStatus)(0), // 4: feast.serving.GetOnlineFeaturesResponse.FieldStatus + (*GetFeastServingInfoRequest)(nil), // 5: feast.serving.GetFeastServingInfoRequest + (*GetFeastServingInfoResponse)(nil), // 6: feast.serving.GetFeastServingInfoResponse + (*FeatureReference)(nil), // 7: feast.serving.FeatureReference + (*GetOnlineFeaturesRequest)(nil), // 8: feast.serving.GetOnlineFeaturesRequest (*GetOnlineFeaturesResponse)(nil), // 9: feast.serving.GetOnlineFeaturesResponse - (*GetBatchFeaturesResponse)(nil), // 10: feast.serving.GetBatchFeaturesResponse - (*GetJobRequest)(nil), // 11: feast.serving.GetJobRequest - (*GetJobResponse)(nil), // 12: feast.serving.GetJobResponse - (*Job)(nil), // 13: feast.serving.Job - (*DatasetSource)(nil), // 14: feast.serving.DatasetSource - (*GetOnlineFeaturesRequest_EntityRow)(nil), // 15: feast.serving.GetOnlineFeaturesRequest.EntityRow - nil, // 16: feast.serving.GetOnlineFeaturesRequest.EntityRow.FieldsEntry - (*GetOnlineFeaturesResponse_FieldValues)(nil), // 17: feast.serving.GetOnlineFeaturesResponse.FieldValues - nil, // 18: feast.serving.GetOnlineFeaturesResponse.FieldValues.FieldsEntry - (*DatasetSource_FileSource)(nil), // 19: feast.serving.DatasetSource.FileSource - (*timestamp.Timestamp)(nil), // 20: google.protobuf.Timestamp - (*types.Value)(nil), // 21: feast.types.Value + (*GetBatchFeaturesRequest)(nil), // 10: feast.serving.GetBatchFeaturesRequest + (*GetBatchFeaturesResponse)(nil), // 11: feast.serving.GetBatchFeaturesResponse + (*GetJobRequest)(nil), // 12: feast.serving.GetJobRequest + (*GetJobResponse)(nil), // 13: feast.serving.GetJobResponse + (*Job)(nil), // 14: feast.serving.Job + (*DatasetSource)(nil), // 15: feast.serving.DatasetSource + (*GetOnlineFeaturesRequest_EntityRow)(nil), // 16: feast.serving.GetOnlineFeaturesRequest.EntityRow + nil, // 17: feast.serving.GetOnlineFeaturesRequest.EntityRow.FieldsEntry + (*GetOnlineFeaturesResponse_FieldValues)(nil), // 18: feast.serving.GetOnlineFeaturesResponse.FieldValues + nil, // 19: feast.serving.GetOnlineFeaturesResponse.FieldValues.FieldsEntry + nil, // 20: feast.serving.GetOnlineFeaturesResponse.FieldValues.StatusesEntry + (*DatasetSource_FileSource)(nil), // 21: feast.serving.DatasetSource.FileSource + (*timestamp.Timestamp)(nil), // 22: google.protobuf.Timestamp + (*types.Value)(nil), // 23: feast.types.Value } var file_feast_serving_ServingService_proto_depIdxs = []int32{ 0, // 0: feast.serving.GetFeastServingInfoResponse.type:type_name -> feast.serving.FeastServingType - 6, // 1: feast.serving.GetOnlineFeaturesRequest.features:type_name -> feast.serving.FeatureReference - 15, // 2: feast.serving.GetOnlineFeaturesRequest.entity_rows:type_name -> feast.serving.GetOnlineFeaturesRequest.EntityRow - 6, // 3: feast.serving.GetBatchFeaturesRequest.features:type_name -> feast.serving.FeatureReference - 14, // 4: feast.serving.GetBatchFeaturesRequest.dataset_source:type_name -> feast.serving.DatasetSource - 17, // 5: feast.serving.GetOnlineFeaturesResponse.field_values:type_name -> feast.serving.GetOnlineFeaturesResponse.FieldValues - 13, // 6: feast.serving.GetBatchFeaturesResponse.job:type_name -> feast.serving.Job - 13, // 7: feast.serving.GetJobRequest.job:type_name -> feast.serving.Job - 13, // 8: feast.serving.GetJobResponse.job:type_name -> feast.serving.Job + 7, // 1: feast.serving.GetOnlineFeaturesRequest.features:type_name -> feast.serving.FeatureReference + 16, // 2: feast.serving.GetOnlineFeaturesRequest.entity_rows:type_name -> feast.serving.GetOnlineFeaturesRequest.EntityRow + 18, // 3: feast.serving.GetOnlineFeaturesResponse.field_values:type_name -> feast.serving.GetOnlineFeaturesResponse.FieldValues + 7, // 4: feast.serving.GetBatchFeaturesRequest.features:type_name -> feast.serving.FeatureReference + 15, // 5: feast.serving.GetBatchFeaturesRequest.dataset_source:type_name -> feast.serving.DatasetSource + 14, // 6: feast.serving.GetBatchFeaturesResponse.job:type_name -> feast.serving.Job + 14, // 7: feast.serving.GetJobRequest.job:type_name -> feast.serving.Job + 14, // 8: feast.serving.GetJobResponse.job:type_name -> feast.serving.Job 1, // 9: feast.serving.Job.type:type_name -> feast.serving.JobType 2, // 10: feast.serving.Job.status:type_name -> feast.serving.JobStatus 3, // 11: feast.serving.Job.data_format:type_name -> feast.serving.DataFormat - 19, // 12: feast.serving.DatasetSource.file_source:type_name -> feast.serving.DatasetSource.FileSource - 20, // 13: feast.serving.GetOnlineFeaturesRequest.EntityRow.entity_timestamp:type_name -> google.protobuf.Timestamp - 16, // 14: feast.serving.GetOnlineFeaturesRequest.EntityRow.fields:type_name -> feast.serving.GetOnlineFeaturesRequest.EntityRow.FieldsEntry - 21, // 15: feast.serving.GetOnlineFeaturesRequest.EntityRow.FieldsEntry.value:type_name -> feast.types.Value - 18, // 16: feast.serving.GetOnlineFeaturesResponse.FieldValues.fields:type_name -> feast.serving.GetOnlineFeaturesResponse.FieldValues.FieldsEntry - 21, // 17: feast.serving.GetOnlineFeaturesResponse.FieldValues.FieldsEntry.value:type_name -> feast.types.Value - 3, // 18: feast.serving.DatasetSource.FileSource.data_format:type_name -> feast.serving.DataFormat - 4, // 19: feast.serving.ServingService.GetFeastServingInfo:input_type -> feast.serving.GetFeastServingInfoRequest - 7, // 20: feast.serving.ServingService.GetOnlineFeatures:input_type -> feast.serving.GetOnlineFeaturesRequest - 8, // 21: feast.serving.ServingService.GetBatchFeatures:input_type -> feast.serving.GetBatchFeaturesRequest - 11, // 22: feast.serving.ServingService.GetJob:input_type -> feast.serving.GetJobRequest - 5, // 23: feast.serving.ServingService.GetFeastServingInfo:output_type -> feast.serving.GetFeastServingInfoResponse - 9, // 24: feast.serving.ServingService.GetOnlineFeatures:output_type -> feast.serving.GetOnlineFeaturesResponse - 10, // 25: feast.serving.ServingService.GetBatchFeatures:output_type -> feast.serving.GetBatchFeaturesResponse - 12, // 26: feast.serving.ServingService.GetJob:output_type -> feast.serving.GetJobResponse - 23, // [23:27] is the sub-list for method output_type - 19, // [19:23] is the sub-list for method input_type - 19, // [19:19] is the sub-list for extension type_name - 19, // [19:19] is the sub-list for extension extendee - 0, // [0:19] is the sub-list for field type_name + 21, // 12: feast.serving.DatasetSource.file_source:type_name -> feast.serving.DatasetSource.FileSource + 22, // 13: feast.serving.GetOnlineFeaturesRequest.EntityRow.entity_timestamp:type_name -> google.protobuf.Timestamp + 17, // 14: feast.serving.GetOnlineFeaturesRequest.EntityRow.fields:type_name -> feast.serving.GetOnlineFeaturesRequest.EntityRow.FieldsEntry + 23, // 15: feast.serving.GetOnlineFeaturesRequest.EntityRow.FieldsEntry.value:type_name -> feast.types.Value + 19, // 16: feast.serving.GetOnlineFeaturesResponse.FieldValues.fields:type_name -> feast.serving.GetOnlineFeaturesResponse.FieldValues.FieldsEntry + 20, // 17: feast.serving.GetOnlineFeaturesResponse.FieldValues.statuses:type_name -> feast.serving.GetOnlineFeaturesResponse.FieldValues.StatusesEntry + 23, // 18: feast.serving.GetOnlineFeaturesResponse.FieldValues.FieldsEntry.value:type_name -> feast.types.Value + 4, // 19: feast.serving.GetOnlineFeaturesResponse.FieldValues.StatusesEntry.value:type_name -> feast.serving.GetOnlineFeaturesResponse.FieldStatus + 3, // 20: feast.serving.DatasetSource.FileSource.data_format:type_name -> feast.serving.DataFormat + 5, // 21: feast.serving.ServingService.GetFeastServingInfo:input_type -> feast.serving.GetFeastServingInfoRequest + 8, // 22: feast.serving.ServingService.GetOnlineFeatures:input_type -> feast.serving.GetOnlineFeaturesRequest + 10, // 23: feast.serving.ServingService.GetBatchFeatures:input_type -> feast.serving.GetBatchFeaturesRequest + 12, // 24: feast.serving.ServingService.GetJob:input_type -> feast.serving.GetJobRequest + 6, // 25: feast.serving.ServingService.GetFeastServingInfo:output_type -> feast.serving.GetFeastServingInfoResponse + 9, // 26: feast.serving.ServingService.GetOnlineFeatures:output_type -> feast.serving.GetOnlineFeaturesResponse + 11, // 27: feast.serving.ServingService.GetBatchFeatures:output_type -> feast.serving.GetBatchFeaturesResponse + 13, // 28: feast.serving.ServingService.GetJob:output_type -> feast.serving.GetJobResponse + 25, // [25:29] is the sub-list for method output_type + 21, // [21:25] is the sub-list for method input_type + 21, // [21:21] is the sub-list for extension type_name + 21, // [21:21] is the sub-list for extension extendee + 0, // [0:21] is the sub-list for field type_name } func init() { file_feast_serving_ServingService_proto_init() } @@ -1370,7 +1466,7 @@ func file_feast_serving_ServingService_proto_init() { } } file_feast_serving_ServingService_proto_msgTypes[4].Exporter = func(v interface{}, i int) interface{} { - switch v := v.(*GetBatchFeaturesRequest); i { + switch v := v.(*GetOnlineFeaturesResponse); i { case 0: return &v.state case 1: @@ -1382,7 +1478,7 @@ func file_feast_serving_ServingService_proto_init() { } } file_feast_serving_ServingService_proto_msgTypes[5].Exporter = func(v interface{}, i int) interface{} { - switch v := v.(*GetOnlineFeaturesResponse); i { + switch v := v.(*GetBatchFeaturesRequest); i { case 0: return &v.state case 1: @@ -1477,7 +1573,7 @@ func file_feast_serving_ServingService_proto_init() { return nil } } - file_feast_serving_ServingService_proto_msgTypes[15].Exporter = func(v interface{}, i int) interface{} { + file_feast_serving_ServingService_proto_msgTypes[16].Exporter = func(v interface{}, i int) interface{} { switch v := v.(*DatasetSource_FileSource); i { case 0: return &v.state @@ -1498,8 +1594,8 @@ func file_feast_serving_ServingService_proto_init() { File: protoimpl.DescBuilder{ GoPackagePath: reflect.TypeOf(x{}).PkgPath(), RawDescriptor: file_feast_serving_ServingService_proto_rawDesc, - NumEnums: 4, - NumMessages: 16, + NumEnums: 5, + NumMessages: 17, NumExtensions: 0, NumServices: 1, }, diff --git a/sdk/go/request.go b/sdk/go/request.go index 683f17c16ed..a11999ae890 100644 --- a/sdk/go/request.go +++ b/sdk/go/request.go @@ -25,6 +25,9 @@ type OnlineFeaturesRequest struct { // Project specifies the project would contain the feature sets where the requested features belong to. Project string + + // whether to omit the entities fields in the response. + OmitEntities bool } // Builds the feast-specified request payload from the wrapper. @@ -34,16 +37,18 @@ func (r OnlineFeaturesRequest) buildRequest() (*serving.GetOnlineFeaturesRequest return nil, err } + // build request entity rows from native entities entityRows := make([]*serving.GetOnlineFeaturesRequest_EntityRow, len(r.Entities)) - - for i := range r.Entities { + for i, entity := range r.Entities { entityRows[i] = &serving.GetOnlineFeaturesRequest_EntityRow{ - Fields: r.Entities[i], + Fields: entity, } } + return &serving.GetOnlineFeaturesRequest{ - Features: featureRefs, - EntityRows: entityRows, + Features: featureRefs, + EntityRows: entityRows, + OmitEntitiesInResponse: r.OmitEntities, }, nil } diff --git a/sdk/go/response.go b/sdk/go/response.go index 1bc80af9fbb..7fa50761b69 100644 --- a/sdk/go/response.go +++ b/sdk/go/response.go @@ -25,12 +25,22 @@ type OnlineFeaturesResponse struct { // Rows retrieves the result of the request as a list of Rows. func (r OnlineFeaturesResponse) Rows() []Row { rows := make([]Row, len(r.RawResponse.FieldValues)) - for i, val := range r.RawResponse.FieldValues { - rows[i] = val.Fields + for i, fieldValues := range r.RawResponse.FieldValues { + rows[i] = fieldValues.Fields } return rows } +// Statuses retrieves field level status metadata for each row in Rows(). +// Each status map returned maps status 1:1 to each returned row from Rows() +func (r OnlineFeaturesResponse) Statuses() []map[string]serving.GetOnlineFeaturesResponse_FieldStatus { + statuses := make([]map[string]serving.GetOnlineFeaturesResponse_FieldStatus, len(r.RawResponse.FieldValues)) + for i, fieldValues := range r.RawResponse.FieldValues { + statuses[i] = fieldValues.Statuses + } + return statuses +} + // Int64Arrays retrieves the result of the request as a list of int64 slices. Any missing values will be filled // with the missing values provided. func (r OnlineFeaturesResponse) Int64Arrays(order []string, fillNa []int64) ([][]int64, error) { @@ -38,17 +48,17 @@ func (r OnlineFeaturesResponse) Int64Arrays(order []string, fillNa []int64) ([][ if len(fillNa) != len(order) { return nil, fmt.Errorf(ErrLengthMismatch, len(fillNa), len(order)) } - for i, val := range r.RawResponse.FieldValues { + for i, fieldValues := range r.RawResponse.FieldValues { rows[i] = make([]int64, len(order)) for j, fname := range order { - fValue, exists := val.Fields[fname] + value, exists := fieldValues.Fields[fname] if !exists { return nil, fmt.Errorf(ErrFeatureNotFound, fname) } - val := fValue.GetVal() - if val == nil { + valType := value.GetVal() + if valType == nil { rows[i][j] = fillNa[j] - } else if int64Val, ok := val.(*types.Value_Int64Val); ok { + } else if int64Val, ok := valType.(*types.Value_Int64Val); ok { rows[i][j] = int64Val.Int64Val } else { return nil, fmt.Errorf(ErrTypeMismatch, "int64") @@ -65,17 +75,17 @@ func (r OnlineFeaturesResponse) Float64Arrays(order []string, fillNa []float64) if len(fillNa) != len(order) { return nil, fmt.Errorf(ErrLengthMismatch, len(fillNa), len(order)) } - for i, val := range r.RawResponse.FieldValues { + for i, records := range r.RawResponse.FieldValues { rows[i] = make([]float64, len(order)) for j, fname := range order { - fValue, exists := val.Fields[fname] + value, exists := records.Fields[fname] if !exists { return nil, fmt.Errorf(ErrFeatureNotFound, fname) } - val := fValue.GetVal() - if val == nil { + valType := value.GetVal() + if valType == nil { rows[i][j] = fillNa[j] - } else if doubleVal, ok := val.(*types.Value_DoubleVal); ok { + } else if doubleVal, ok := valType.(*types.Value_DoubleVal); ok { rows[i][j] = doubleVal.DoubleVal } else { return nil, fmt.Errorf(ErrTypeMismatch, "float64") diff --git a/sdk/go/response_test.go b/sdk/go/response_test.go index 6a7c4211187..0949f24d679 100644 --- a/sdk/go/response_test.go +++ b/sdk/go/response_test.go @@ -16,12 +16,20 @@ var response = OnlineFeaturesResponse{ "project1/feature1": Int64Val(1), "project1/feature2": {}, }, + Statuses: map[string]serving.GetOnlineFeaturesResponse_FieldStatus{ + "project1/feature1": serving.GetOnlineFeaturesResponse_PRESENT, + "project1/feature2": serving.GetOnlineFeaturesResponse_NULL_VALUE, + }, }, { Fields: map[string]*types.Value{ "project1/feature1": Int64Val(2), "project1/feature2": Int64Val(2), }, + Statuses: map[string]serving.GetOnlineFeaturesResponse_FieldStatus{ + "project1/feature1": serving.GetOnlineFeaturesResponse_PRESENT, + "project1/feature2": serving.GetOnlineFeaturesResponse_PRESENT, + }, }, }, }, @@ -43,6 +51,28 @@ func TestOnlineFeaturesResponseToRow(t *testing.T) { } } +func TestOnlineFeaturesResponseoToStatuses(t *testing.T) { + actual := response.Statuses() + expected := []map[string]serving.GetOnlineFeaturesResponse_FieldStatus{ + { + "project1/feature1": serving.GetOnlineFeaturesResponse_PRESENT, + "project1/feature2": serving.GetOnlineFeaturesResponse_NULL_VALUE, + }, + { + "project1/feature1": serving.GetOnlineFeaturesResponse_PRESENT, + "project1/feature2": serving.GetOnlineFeaturesResponse_PRESENT, + }, + } + if len(expected) != len(actual) { + t.Errorf("expected: %v, got: %v", expected, actual) + } + for i := range expected { + if !cmp.Equal(expected[i], actual[i]) { + t.Errorf("expected: %v, got: %v", expected, actual) + } + } +} + func TestOnlineFeaturesResponseToInt64Array(t *testing.T) { type args struct { order []string diff --git a/sdk/go/types.go b/sdk/go/types.go index dd6553163bd..600af0b658a 100644 --- a/sdk/go/types.go +++ b/sdk/go/types.go @@ -5,7 +5,7 @@ import ( "github.com/golang/protobuf/proto" ) -// Row map of entity values +// Row is a map of fields type Row map[string]*types.Value func (r Row) equalTo(other Row) bool { diff --git a/sdk/java/src/main/java/com/gojek/feast/FeastClient.java b/sdk/java/src/main/java/com/gojek/feast/FeastClient.java index a81fdab21a2..11b4f352f64 100644 --- a/sdk/java/src/main/java/com/gojek/feast/FeastClient.java +++ b/sdk/java/src/main/java/com/gojek/feast/FeastClient.java @@ -24,7 +24,6 @@ import feast.proto.serving.ServingAPIProto.GetOnlineFeaturesResponse; import feast.proto.serving.ServingServiceGrpc; import feast.proto.serving.ServingServiceGrpc.ServingServiceBlockingStub; -import feast.proto.types.ValueProto.Value; import io.grpc.ManagedChannel; import io.grpc.ManagedChannelBuilder; import java.util.HashSet; @@ -143,20 +142,21 @@ public List getOnlineFeatures( return response.getFieldValuesList().stream() .map( - field -> { + fieldValues -> { Row row = Row.create(); - field - .getFieldsMap() - .forEach( - (String name, Value value) -> { - // Strip project from string Feature References from returned from serving - if (!entityRefs.contains(name)) { - FeatureReference featureRef = - RequestUtil.parseFeatureRef(name, true).build(); - name = RequestUtil.renderFeatureRef(featureRef); - } - row.set(name, value); - }); + for (String fieldName : fieldValues.getFieldsMap().keySet()) { + String stripFieldName = fieldName; + if (!entityRefs.contains(fieldName)) { + // Strip project from string Feature References from returned from serving + FeatureReference featureRef = + RequestUtil.parseFeatureRef(fieldName, true).build(); + stripFieldName = RequestUtil.renderFeatureRef(featureRef); + row.set( + stripFieldName, + fieldValues.getFieldsMap().get(fieldName), + fieldValues.getStatusesMap().get(fieldName)); + } + } return row; }) .collect(Collectors.toList()); diff --git a/sdk/java/src/main/java/com/gojek/feast/Row.java b/sdk/java/src/main/java/com/gojek/feast/Row.java index 4a3035f8f19..51f820e320a 100644 --- a/sdk/java/src/main/java/com/gojek/feast/Row.java +++ b/sdk/java/src/main/java/com/gojek/feast/Row.java @@ -19,6 +19,7 @@ import com.google.protobuf.ByteString; import com.google.protobuf.Timestamp; import com.google.protobuf.util.Timestamps; +import feast.proto.serving.ServingAPIProto.GetOnlineFeaturesResponse.FieldStatus; import feast.proto.types.ValueProto.Value; import feast.proto.types.ValueProto.Value.ValCase; import java.time.Instant; @@ -32,11 +33,13 @@ public class Row { private Timestamp entity_timestamp; private HashMap fields; + private HashMap fieldStatuses; public static Row create() { Row row = new Row(); row.entity_timestamp = Timestamps.fromMillis(System.currentTimeMillis()); row.fields = new HashMap<>(); + row.fieldStatuses = new HashMap<>(); return row; } @@ -55,6 +58,10 @@ public Row setEntityTimestamp(String dateTime) { } public Row set(String fieldName, Object value) { + return this.set(fieldName, value, FieldStatus.PRESENT); + } + + public Row set(String fieldName, Object value, FieldStatus status) { String valueType = value.getClass().getCanonicalName(); switch (valueType) { case "java.lang.Integer": @@ -85,6 +92,8 @@ public Row set(String fieldName, Object value) { "Type '%s' is unsupported in Feast. Please use one of these value types: Integer, Long, Float, Double, String, byte[].", valueType)); } + + fieldStatuses.put(fieldName, status); return this; } @@ -116,6 +125,14 @@ public byte[] getByte(String fieldName) { return getValue(fieldName).map(Value::getBytesVal).map(ByteString::toByteArray).orElse(null); } + public Map getStatuses() { + return fieldStatuses; + } + + public FieldStatus getStatus(String fieldName) { + return fieldStatuses.get(fieldName); + } + @Override public String toString() { List parts = new ArrayList<>(); diff --git a/sdk/java/src/test/java/com/gojek/feast/FeastClientTest.java b/sdk/java/src/test/java/com/gojek/feast/FeastClientTest.java index 717792244ee..5cb756f8de6 100644 --- a/sdk/java/src/test/java/com/gojek/feast/FeastClientTest.java +++ b/sdk/java/src/test/java/com/gojek/feast/FeastClientTest.java @@ -25,6 +25,7 @@ import feast.proto.serving.ServingAPIProto.GetOnlineFeaturesRequest; import feast.proto.serving.ServingAPIProto.GetOnlineFeaturesRequest.EntityRow; import feast.proto.serving.ServingAPIProto.GetOnlineFeaturesResponse; +import feast.proto.serving.ServingAPIProto.GetOnlineFeaturesResponse.FieldStatus; import feast.proto.serving.ServingAPIProto.GetOnlineFeaturesResponse.FieldValues; import feast.proto.serving.ServingServiceGrpc.ServingServiceImplBase; import feast.proto.types.ValueProto.Value; @@ -53,8 +54,9 @@ public class FeastClientTest { public void getOnlineFeatures( GetOnlineFeaturesRequest request, StreamObserver responseObserver) { + if (!request.equals(FeastClientTest.getFakeRequest())) { - responseObserver.onError(Status.UNKNOWN.asRuntimeException()); + responseObserver.onError(Status.FAILED_PRECONDITION.asRuntimeException()); } responseObserver.onNext(FeastClientTest.getFakeResponse()); @@ -86,7 +88,7 @@ public void setup() throws Exception { public void shouldGetOnlineFeatures() { List rows = this.client.getOnlineFeatures( - Arrays.asList("driver:name", "rating"), + Arrays.asList("driver:name", "rating", "null_value"), Arrays.asList( Row.create().set("driver_id", 1).setEntityTimestamp(Instant.ofEpochSecond(100))), "driver_project"); @@ -98,6 +100,17 @@ public void shouldGetOnlineFeatures() { put("driver_id", intValue(1)); put("driver:name", strValue("david")); put("rating", intValue(3)); + put("null_value", Value.newBuilder().build()); + } + }); + assertEquals( + rows.get(0).getStatuses(), + new HashMap() { + { + put("driver_id", FieldStatus.PRESENT); + put("driver:name", FieldStatus.PRESENT); + put("rating", FieldStatus.PRESENT); + put("null_value", FieldStatus.NULL_VALUE); } }); } @@ -113,6 +126,11 @@ private static GetOnlineFeaturesRequest getFakeRequest() { .build()) .addFeatures( FeatureReference.newBuilder().setProject("driver_project").setName("rating").build()) + .addFeatures( + FeatureReference.newBuilder() + .setProject("driver_project") + .setName("null_value") + .build()) .addEntityRows( EntityRow.newBuilder() .setEntityTimestamp(Timestamp.newBuilder().setSeconds(100)) @@ -124,14 +142,14 @@ private static GetOnlineFeaturesResponse getFakeResponse() { return GetOnlineFeaturesResponse.newBuilder() .addFieldValues( FieldValues.newBuilder() - .putAllFields( - new HashMap() { - { - put("driver_id", intValue(1)); - put("driver:name", strValue("david")); - put("rating", intValue(3)); - } - }) + .putFields("driver_project/driver_id", intValue(1)) + .putStatuses("driver_project/driver_id", FieldStatus.PRESENT) + .putFields("driver_project/driver:name", strValue("david")) + .putStatuses("driver_project/driver:name", FieldStatus.PRESENT) + .putFields("driver_project/rating", intValue(3)) + .putStatuses("driver_project/rating", FieldStatus.PRESENT) + .putFields("driver_project/null_value", Value.newBuilder().build()) + .putStatuses("driver_project/null_value", FieldStatus.NULL_VALUE) .build()) .build(); } diff --git a/sdk/python/feast/client.py b/sdk/python/feast/client.py index ff6cf91b3b5..c30e4265df7 100644 --- a/sdk/python/feast/client.py +++ b/sdk/python/feast/client.py @@ -671,6 +671,7 @@ def get_online_features( feature_refs: List[str], entity_rows: List[GetOnlineFeaturesRequest.EntityRow], project: Optional[str] = None, + omit_entities: bool = False, ) -> GetOnlineFeaturesResponse: """ Retrieves the latest online feature data from Feast Serving @@ -686,16 +687,19 @@ def get_online_features( retrieval. All entity types within a feature project: Specifies the project which contain the FeatureSets which the requested features belong to. + omit_entities: If true will omit entity values in the returned feature data. Returns: - Returns a list of maps where each item in the list contains the - latest feature values for the provided entities + GetOnlineFeaturesResponse containing the feature data in records. + Each EntityRow provided will yield one record, which contains + data fields with data value and field status metadata (if included). """ self._connect_serving() try: response = self._serving_service_stub.GetOnlineFeatures( GetOnlineFeaturesRequest( + omit_entities_in_response=omit_entities, features=_build_feature_references( feature_ref_strs=feature_refs, project=project if project is not None else self.project, @@ -703,25 +707,33 @@ def get_online_features( entity_rows=entity_rows, ) ) - # collect entity row refs - entity_refs = set() - for entity_row in entity_rows: - entity_refs.update(entity_row.fields.keys()) + entity_refs = { + key for entity_row in entity_rows for key in entity_row.fields.keys() + } + # strip the project part the string feature references returned from serving + strip = ( + lambda ref: repr(FeatureRef.from_str(ref, ignore_project=True)) + if ref not in entity_refs + else ref + ) strip_field_values = [] for field_value in response.field_values: - # strip the project part the string feature references returned from serving - strip_fields = {} - for ref_str, value in field_value.fields.items(): - if ref_str not in entity_refs: - ref_str = repr( - FeatureRef.from_str(ref_str, ignore_project=True) - ) - strip_fields[ref_str] = value + keys, fields, statuses = ( + field_value.fields.keys(), + field_value.fields, + field_value.statuses, + ) + fields_and_statuses = [ + (strip(key), fields[key], statuses[key]) for key in keys + ] + keys, fields, statuses = zip(*fields_and_statuses) strip_field_values.append( - GetOnlineFeaturesResponse.FieldValues(fields=strip_fields) + GetOnlineFeaturesResponse.FieldValues( + fields=dict(zip(keys, fields)), + statuses=dict(zip(keys, statuses)), + ) ) - del response.field_values[:] response.field_values.extend(strip_field_values) diff --git a/sdk/python/feast/constants.py b/sdk/python/feast/constants.py index c4bde75404a..4b22a8e975d 100644 --- a/sdk/python/feast/constants.py +++ b/sdk/python/feast/constants.py @@ -35,6 +35,9 @@ "batch_feature_request_wait_time_seconds" ) +CONFIG_TIMEOUT_KEY = "timeout" +CONFIG_MAX_WAIT_INTERVAL_KEY = "max_wait_interval" + # Configuration option default values FEAST_DEFAULT_OPTIONS = { CONFIG_PROJECT_KEY: "default", @@ -45,4 +48,6 @@ CONFIG_GRPC_CONNECTION_TIMEOUT_DEFAULT_KEY: "3", CONFIG_GRPC_CONNECTION_TIMEOUT_APPLY_KEY: "600", CONFIG_BATCH_FEATURE_REQUEST_WAIT_TIME_SECONDS_KEY: "600", + CONFIG_TIMEOUT_KEY: "21600", + CONFIG_MAX_WAIT_INTERVAL_KEY: "60", } diff --git a/sdk/python/feast/job.py b/sdk/python/feast/job.py index 21b08224bad..829b4f79319 100644 --- a/sdk/python/feast/job.py +++ b/sdk/python/feast/job.py @@ -1,6 +1,4 @@ import tempfile -import time -from datetime import datetime, timedelta from typing import List from urllib.parse import urlparse @@ -9,6 +7,8 @@ from google.cloud import storage from google.protobuf.json_format import MessageToJson +from feast.constants import CONFIG_TIMEOUT_KEY +from feast.constants import FEAST_DEFAULT_OPTIONS as defaults from feast.core.CoreService_pb2 import ListIngestionJobsRequest from feast.core.CoreService_pb2_grpc import CoreServiceStub from feast.core.IngestionJob_pb2 import IngestionJob as IngestJobProto @@ -23,13 +23,7 @@ from feast.serving.ServingService_pb2 import Job as JobProto from feast.serving.ServingService_pb2_grpc import ServingServiceStub from feast.source import Source - -# Maximum no of seconds to wait until the retrieval jobs status is DONE in Feast -# Currently set to the maximum query execution time limit in BigQuery -DEFAULT_TIMEOUT_SEC: int = 21600 - -# Maximum no of seconds to wait before reloading the job status in Feast -MAX_WAIT_INTERVAL_SEC: int = 60 +from feast.wait import wait_retry_backoff class RetrievalJob: @@ -71,7 +65,7 @@ def reload(self): """ self.job_proto = self.serving_stub.GetJob(GetJobRequest(job=self.job_proto)).job - def get_avro_files(self, timeout_sec: int = DEFAULT_TIMEOUT_SEC): + def get_avro_files(self, timeout_sec: int = int(defaults[CONFIG_TIMEOUT_KEY])): """ Wait until job is done to get the file uri to Avro result files on Google Cloud Storage. @@ -84,21 +78,17 @@ def get_avro_files(self, timeout_sec: int = DEFAULT_TIMEOUT_SEC): Returns: str: Google Cloud Storage file uris of the returned Avro files. """ - max_wait_datetime = datetime.now() + timedelta(seconds=timeout_sec) - wait_duration_sec = 2 - - while self.status != JOB_STATUS_DONE: - if datetime.now() > max_wait_datetime: - raise Exception( - "Timeout exceeded while waiting for result. Please retry " - "this method or use a longer timeout value." - ) + def try_retrieve(): self.reload() - time.sleep(wait_duration_sec) + return None, self.status == JOB_STATUS_DONE - # Backoff the wait duration exponentially up till MAX_WAIT_INTERVAL_SEC - wait_duration_sec = min(wait_duration_sec * 2, MAX_WAIT_INTERVAL_SEC) + wait_retry_backoff( + retry_fn=try_retrieve, + timeout_secs=timeout_sec, + timeout_msg="Timeout exceeded while waiting for result. Please retry " + "this method or use a longer timeout value.", + ) if self.job_proto.error: raise Exception(self.job_proto.error) @@ -111,7 +101,7 @@ def get_avro_files(self, timeout_sec: int = DEFAULT_TIMEOUT_SEC): return [urlparse(uri) for uri in self.job_proto.file_uris] - def result(self, timeout_sec: int = DEFAULT_TIMEOUT_SEC): + def result(self, timeout_sec: int = int(defaults[CONFIG_TIMEOUT_KEY])): """ Wait until job is done to get an iterable rows of result. The row can only represent an Avro row in Feast 0.3. @@ -142,7 +132,9 @@ def result(self, timeout_sec: int = DEFAULT_TIMEOUT_SEC): for record in avro_reader: yield record - def to_dataframe(self, timeout_sec: int = DEFAULT_TIMEOUT_SEC) -> pd.DataFrame: + def to_dataframe( + self, timeout_sec: int = int(defaults[CONFIG_TIMEOUT_KEY]) + ) -> pd.DataFrame: """ Wait until a job is done to get an iterable rows of result. This method will split the response into chunked DataFrame of a specified size to @@ -164,7 +156,9 @@ def to_dataframe(self, timeout_sec: int = DEFAULT_TIMEOUT_SEC) -> pd.DataFrame: return pd.DataFrame.from_records(records) def to_chunked_dataframe( - self, max_chunk_size: int = -1, timeout_sec: int = DEFAULT_TIMEOUT_SEC + self, + max_chunk_size: int = -1, + timeout_sec: int = int(defaults[CONFIG_TIMEOUT_KEY]), ) -> pd.DataFrame: """ Wait until a job is done to get an iterable rows of result. This method @@ -281,18 +275,11 @@ def wait(self, status: IngestionJobStatus, timeout_secs: float = 300): timeout_secs: Maximum seconds to wait before timing out. """ # poll & wait for job status to transition - wait_begin = time.time() - wait_secs = 2 - elapsed_secs = 0 - while self.status != status and elapsed_secs <= timeout_secs: - time.sleep(wait_secs) - # back off wait duration exponentially, capped at MAX_WAIT_INTERVAL_SEC - wait_secs = min(wait_secs * 2, MAX_WAIT_INTERVAL_SEC) - elapsed_secs = time.time() - wait_begin - - # raise error if timeout - if elapsed_secs > timeout_secs: - raise TimeoutError("Wait for IngestJob's status to transition timed out") + wait_retry_backoff( + retry_fn=(lambda: (None, self.status == status)), + timeout_secs=timeout_secs, + timeout_msg="Wait for IngestJob's status to transition timed out", + ) def __str__(self): # render the contents of ingest job as human readable string diff --git a/sdk/python/feast/wait.py b/sdk/python/feast/wait.py new file mode 100644 index 00000000000..f25036d76ea --- /dev/null +++ b/sdk/python/feast/wait.py @@ -0,0 +1,56 @@ +# Copyright 2019 The Feast Authors +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# https://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +import time +from typing import Any, Callable, Optional, Tuple + +from feast.constants import CONFIG_MAX_WAIT_INTERVAL_KEY +from feast.constants import FEAST_DEFAULT_OPTIONS as defaults + + +def wait_retry_backoff( + retry_fn: Callable[[], Tuple[Any, bool]], + timeout_secs: Optional[int] = None, + timeout_msg: Optional[str] = "Timeout while waiting for retry_fn() to return True", + max_interval_secs: Optional[int] = int(defaults[CONFIG_MAX_WAIT_INTERVAL_KEY]), +) -> Any: + """ + Repeatedly try calling given retry_fn until it returns a True boolean success flag. + Waits with a exponential backoff between retries until timeout when it throws TimeoutError. + Args: + retry_fn: Callable that returns a result and a boolean success flag. + timeout_secs: timeout in seconds to give up retrying and throw TimeoutError, + or None to retry perpectually. + timeout_msg: Message to use when throwing TimeoutError. + max_interval_secs: max wait in seconds to wait between retries. + Returns: + Returned Result from retry_fn() if success flag is True. + """ + wait_secs, elapsed_secs = 1, 0 + result, is_success = retry_fn() + wait_begin = time.time() + while not is_success and elapsed_secs <= timeout_secs: + # back off wait duration exponentially, capped at MAX_WAIT_INTERVAL_SEC + elapsed_secs = time.time() - wait_begin + till_timeout_secs = timeout_secs - elapsed_secs + wait_secs = min(wait_secs * 2, max_interval_secs, till_timeout_secs) + time.sleep(wait_secs) + # retry call + result, is_success = retry_fn() + elapsed_secs = time.time() - wait_begin + + if not is_success and elapsed_secs > timeout_secs: + raise TimeoutError(timeout_msg) + + return result diff --git a/sdk/python/tests/test_client.py b/sdk/python/tests/test_client.py index 04626f73cd5..9437ef735d1 100644 --- a/sdk/python/tests/test_client.py +++ b/sdk/python/tests/test_client.py @@ -220,6 +220,7 @@ def int_val(x): project="driver_project", feature_set="driver", name="age" ), FeatureRefProto(project="driver_project", name="rating"), + FeatureRefProto(project="driver_project", name="null_value"), ] ) recieve_response = GetOnlineFeaturesResponse() @@ -234,7 +235,14 @@ def int_val(x): "driver_id": int_val(row_number), "driver_project/driver:age": int_val(1), "driver_project/rating": int_val(9), - } + "driver_project/null_value": ValueProto.Value(), + }, + statuses={ + "driver_id": GetOnlineFeaturesResponse.FieldStatus.PRESENT, + "driver_project/driver:age": GetOnlineFeaturesResponse.FieldStatus.PRESENT, + "driver_project/rating": GetOnlineFeaturesResponse.FieldStatus.PRESENT, + "driver_project/null_value": GetOnlineFeaturesResponse.FieldStatus.NULL_VALUE, + }, ) recieve_response.field_values.append(field_values) @@ -245,7 +253,7 @@ def int_val(x): ) got_response = mocked_client.get_online_features( entity_rows=request.entity_rows, - feature_refs=["driver:age", "rating"], + feature_refs=["driver:age", "rating", "null_value"], project="driver_project", ) # type: GetOnlineFeaturesResponse mocked_client._serving_service_stub.GetOnlineFeatures.assert_called_with( @@ -253,10 +261,19 @@ def int_val(x): ) got_fields = got_response.field_values[0].fields + got_statuses = got_response.field_values[0].statuses assert ( got_fields["driver_id"] == int_val(1) + and got_statuses["driver_id"] + == GetOnlineFeaturesResponse.FieldStatus.PRESENT and got_fields["driver:age"] == int_val(1) + and got_statuses["driver:age"] + == GetOnlineFeaturesResponse.FieldStatus.PRESENT and got_fields["rating"] == int_val(9) + and got_statuses["rating"] == GetOnlineFeaturesResponse.FieldStatus.PRESENT + and got_fields["null_value"] == ValueProto.Value() + and got_statuses["null_value"] + == GetOnlineFeaturesResponse.FieldStatus.NULL_VALUE ) @pytest.mark.parametrize( diff --git a/serving/src/main/java/feast/serving/service/OnlineServingService.java b/serving/src/main/java/feast/serving/service/OnlineServingService.java index bb73e34f51c..6d581f6c710 100644 --- a/serving/src/main/java/feast/serving/service/OnlineServingService.java +++ b/serving/src/main/java/feast/serving/service/OnlineServingService.java @@ -17,12 +17,13 @@ package feast.serving.service; import com.google.common.collect.ImmutableMap; -import com.google.common.collect.Maps; import com.google.protobuf.Duration; import feast.proto.serving.ServingAPIProto.*; import feast.proto.serving.ServingAPIProto.GetOnlineFeaturesRequest.EntityRow; +import feast.proto.serving.ServingAPIProto.GetOnlineFeaturesResponse.FieldStatus; import feast.proto.serving.ServingAPIProto.GetOnlineFeaturesResponse.FieldValues; import feast.proto.types.FeatureRowProto.FeatureRow; +import feast.proto.types.FieldProto.Field; import feast.proto.types.ValueProto.Value; import feast.serving.specs.CachedSpecService; import feast.serving.util.Metrics; @@ -32,9 +33,10 @@ import io.grpc.Status; import io.opentracing.Scope; import io.opentracing.Tracer; -import java.util.List; -import java.util.Map; +import java.util.*; import java.util.stream.Collectors; +import org.apache.beam.vendor.grpc.v1p21p0.com.google.common.collect.Streams; +import org.apache.commons.lang3.tuple.Pair; import org.slf4j.Logger; public class OnlineServingService implements ServingService { @@ -64,79 +66,239 @@ public GetFeastServingInfoResponse getFeastServingInfo( @Override public GetOnlineFeaturesResponse getOnlineFeatures(GetOnlineFeaturesRequest request) { try (Scope scope = tracer.buildSpan("getOnlineFeatures").startActive(true)) { - GetOnlineFeaturesResponse.Builder getOnlineFeaturesResponseBuilder = - GetOnlineFeaturesResponse.newBuilder(); - List featureSetRequests = - specService.getFeatureSets(request.getFeaturesList()); List entityRows = request.getEntityRowsList(); - Map> featureValuesMap = - entityRows.stream() - .collect(Collectors.toMap(row -> row, row -> Maps.newHashMap(row.getFieldsMap()))); - // Get all feature rows from the retriever. Each feature row list corresponds to a single - // feature set request. - List> featureRows = - retriever.getOnlineFeatures(entityRows, featureSetRequests); - if (scope != null) { - scope.span().log(ImmutableMap.of("event", "featureRows", "value", featureRows)); + // Collect the feature/entity value for each entity row in entityValueMap + Map> entityValuesMap = + entityRows.stream().collect(Collectors.toMap(row -> row, row -> new HashMap<>())); + // Collect the feature/entity status metadata for each entity row in entityValueMap + Map> entityStatusesMap = + entityRows.stream().collect(Collectors.toMap(row -> row, row -> new HashMap<>())); + // Collect featureRows retrieved for logging/tracing + List>> logFeatureRows = new LinkedList<>(); + + if (!request.getOmitEntitiesInResponse()) { + // Add entity row's fields as response fields + entityRows.forEach( + entityRow -> { + Map valueMap = entityRow.getFieldsMap(); + entityValuesMap.get(entityRow).putAll(valueMap); + entityStatusesMap.get(entityRow).putAll(getMetadataMap(valueMap, false, false)); + }); } - // For each feature set request, read the feature rows returned by the retriever, and - // populate the featureValuesMap with the feature values corresponding to that entity row. - for (var fsIdx = 0; fsIdx < featureRows.size(); fsIdx++) { - List featureRowsForFs = featureRows.get(fsIdx); - FeatureSetRequest featureSetRequest = featureSetRequests.get(fsIdx); - - String project = featureSetRequest.getSpec().getProject(); - - // In order to return values containing the same feature references provided by the user, - // we reuse the feature references in the request as the keys in the featureValuesMap - Map refsByName = featureSetRequest.getFeatureRefsByName(); - - // Each feature row returned (per feature set request) corresponds to a given entity row. - // For each feature row, update the featureValuesMap. - for (var entityRowIdx = 0; entityRowIdx < entityRows.size(); entityRowIdx++) { - FeatureRow featureRow = featureRowsForFs.get(entityRowIdx); - EntityRow entityRow = entityRows.get(entityRowIdx); - - // If the row is stale, put an empty value into the featureValuesMap. - if (isStale(featureSetRequest, entityRow, featureRow)) { - featureSetRequest - .getFeatureReferences() - .forEach( - ref -> { - populateStaleKeyCountMetrics(project, ref); - featureValuesMap - .get(entityRow) - .put(RefUtil.generateFeatureStringRef(ref), Value.newBuilder().build()); - }); - - } else { - populateRequestCountMetrics(featureSetRequest); - - // Else populate the featureValueMap at this entityRow with the values in the feature - // row. - featureRow.getFieldsList().stream() - .filter(field -> refsByName.containsKey(field.getName())) - .forEach( - field -> { - FeatureReference ref = refsByName.get(field.getName()); - String id = RefUtil.generateFeatureStringRef(ref); - featureValuesMap.get(entityRow).put(id, field.getValue()); - }); - } + List featureSetRequests = + specService.getFeatureSets(request.getFeaturesList()); + for (FeatureSetRequest featureSetRequest : featureSetRequests) { + // Pull feature rows for given entity rows from the feature/featureset specified in feature + // set request. + // from the configured online + List> featureRows = + retriever.getOnlineFeatures(entityRows, featureSetRequest); + // Check that feature row returned corresponds to a given entity row. + if (featureRows.size() != entityRows.size()) { + throw Status.INTERNAL + .withDescription( + "The no. of FeatureRow obtained from OnlineRetriever" + + "does not match no. of entityRow passed.") + .asRuntimeException(); } + + Streams.zip(entityRows.stream(), featureRows.stream(), Pair::of) + .forEach( + entityFeaturePair -> { + EntityRow entityRow = entityFeaturePair.getLeft(); + Optional featureRow = entityFeaturePair.getRight(); + // Unpack feature field values and merge into entityValueMap + boolean isOutsideMaxAge = + checkOutsideMaxAge(featureSetRequest, entityRow, featureRow); + Map valueMap = + unpackValueMap(featureRow, featureSetRequest, isOutsideMaxAge); + entityValuesMap.get(entityRow).putAll(valueMap); + + // Generate metadata for feature values and merge into entityFieldsMap + boolean isNotFound = featureRow.isEmpty(); + Map statusMap = + getMetadataMap(valueMap, isNotFound, isOutsideMaxAge); + entityStatusesMap.get(entityRow).putAll(statusMap); + + // Populate metrics/log request + populateStaleKeyCountMetrics(statusMap, featureSetRequest); + }); + populateRequestCountMetrics(featureSetRequest); + logFeatureRows.add(featureRows); + } + if (scope != null) { + logFeatureRowsTrace(scope, logFeatureRows, featureSetRequests); } - List fieldValues = - featureValuesMap.values().stream() - .map(valueMap -> FieldValues.newBuilder().putAllFields(valueMap).build()) + // Build response field values from entityValuesMap and entityStatusesMap + // Reponse field values should be in the same order as the entityRows provided by the user. + List fieldValuesList = + entityRows.stream() + .map( + entityRow -> { + return FieldValues.newBuilder() + .putAllFields(entityValuesMap.get(entityRow)) + .putAllStatuses(entityStatusesMap.get(entityRow)) + .build(); + }) .collect(Collectors.toList()); - return getOnlineFeaturesResponseBuilder.addAllFieldValues(fieldValues).build(); + return GetOnlineFeaturesResponse.newBuilder().addAllFieldValues(fieldValuesList).build(); + } + } + + /** + * Unpack feature values using data from the given feature row for features specified in the given + * feature set request. + * + * @param featureRow optional to unpack for feature values. + * @param featureSetRequest for which the feature row was retrieved. + * @param isOutsideMaxAge whether which the feature row contains values that is outside max age. + * @return valueMap mapping string feature name to feature value for the given feature set + * request. + */ + private static Map unpackValueMap( + Optional featureRow, + FeatureSetRequest featureSetRequest, + boolean isOutsideMaxAge) { + Map valueMap = new HashMap<>(); + // In order to return values containing the same feature references provided by the user, + // we reuse the feature references in the request as the keys in field builder map + Map nameRefMap = featureSetRequest.getFeatureRefsByName(); + if (featureRow.isPresent()) { + // unpack feature row's feature values and populate value map + Map featureValueMap = + featureRow.get().getFieldsList().stream() + .filter(featureRowField -> nameRefMap.containsKey(featureRowField.getName())) + .collect( + Collectors.toMap( + featureRowField -> { + FeatureReference featureRef = nameRefMap.get(featureRowField.getName()); + return RefUtil.generateFeatureStringRef(featureRef); + }, + featureRowField -> { + // drop feature values with an age outside feature set's max age. + return (isOutsideMaxAge) + ? Value.newBuilder().build() + : featureRowField.getValue(); + })); + valueMap.putAll(featureValueMap); } + // create empty values for features specified in request but not present in feature row. + Set missingFeatures = + nameRefMap.values().stream() + .map(ref -> RefUtil.generateFeatureStringRef(ref)) + .collect(Collectors.toSet()); + missingFeatures.removeAll(valueMap.keySet()); + missingFeatures.forEach(refString -> valueMap.put(refString, Value.newBuilder().build())); + + return valueMap; } - private void populateStaleKeyCountMetrics(String project, FeatureReference ref) { - Metrics.staleKeyCount.labels(project, ref.getName()).inc(); + /** + * Generate Field level Status metadata for the given valueMap. + * + * @param valueMap map of field name to value to generate metadata for. + * @param isNotFound whether the given valueMap represents values that were not found in the + * online retriever. + * @param isOutsideMaxAge whether the given valueMap contains values with age outside feature + * set's max age. + * @return a 1:1 map keyed by field name containing field status metadata instead of values in the + * given valueMap. + */ + private static Map getMetadataMap( + Map valueMap, boolean isNotFound, boolean isOutsideMaxAge) { + return valueMap.entrySet().stream() + .collect( + Collectors.toMap( + es -> es.getKey(), + es -> { + Value fieldValue = es.getValue(); + if (isNotFound) { + return FieldStatus.NOT_FOUND; + } else if (isOutsideMaxAge) { + return FieldStatus.OUTSIDE_MAX_AGE; + } else if (fieldValue.getValCase().equals(Value.ValCase.VAL_NOT_SET)) { + return FieldStatus.NULL_VALUE; + } + return FieldStatus.PRESENT; + })); + } + + /** + * Determine if the feature data in the given feature row is outside maxAge. Data is outside + * maxAge to be when the difference ingestion time set in feature row and the retrieval time set + * in entity row exceeds featureset max age. + * + * @param featureSetRequest contains the spec where feature's max age is extracted. + * @param entityRow contains the retrieval timing of when features are pulled. + * @param featureRow contains the ingestion timing and feature data. + */ + private static boolean checkOutsideMaxAge( + FeatureSetRequest featureSetRequest, EntityRow entityRow, Optional featureRow) { + Duration maxAge = featureSetRequest.getSpec().getMaxAge(); + if (featureRow.isEmpty()) { // no data to consider + return false; + } + if (maxAge.equals(Duration.getDefaultInstance())) { // max age is not set + return false; + } + + long givenTimestamp = entityRow.getEntityTimestamp().getSeconds(); + if (givenTimestamp == 0) { + givenTimestamp = System.currentTimeMillis() / 1000; + } + long timeDifference = givenTimestamp - featureRow.get().getEventTimestamp().getSeconds(); + return timeDifference > maxAge.getSeconds(); + } + + private void logFeatureRowsTrace( + Scope scope, + List>> logFeatureRows, + List featureSetRequests) { + List> loggableFeatureRows = + Streams.zip( + logFeatureRows.stream(), + featureSetRequests.stream(), + (featureRows, featureSetRequest) -> { + FeatureRow.Builder nullFeatureRowBuilder = + FeatureRow.newBuilder() + .setFeatureSet( + RefUtil.generateFeatureSetStringRef(featureSetRequest.getSpec())); + for (FeatureReference featureReference : + featureSetRequest.getFeatureReferences()) { + nullFeatureRowBuilder.addFields( + Field.newBuilder().setName(featureReference.getName())); + } + + // log null feature row when feature row is empty + return featureRows.stream() + .map( + featureRow -> { + return (featureRow.isEmpty()) + ? nullFeatureRowBuilder.build() + : featureRow.get(); + }) + .collect(Collectors.toList()); + }) + .collect(Collectors.toList()); + + scope.span().log(ImmutableMap.of("event", "featureRows", "value", loggableFeatureRows)); + } + + private void populateStaleKeyCountMetrics( + Map statusMap, FeatureSetRequest featureSetRequest) { + String project = featureSetRequest.getSpec().getProject(); + statusMap + .entrySet() + .forEach( + es -> { + String featureRefString = es.getKey(); + FieldStatus status = es.getValue(); + if (status == FieldStatus.OUTSIDE_MAX_AGE) { + Metrics.staleKeyCount.labels(project, featureRefString).inc(); + } + }); } private void populateRequestCountMetrics(FeatureSetRequest featureSetRequest) { @@ -156,18 +318,4 @@ public GetBatchFeaturesResponse getBatchFeatures(GetBatchFeaturesRequest getFeat public GetJobResponse getJob(GetJobRequest getJobRequest) { throw Status.UNIMPLEMENTED.withDescription("Method not implemented").asRuntimeException(); } - - private boolean isStale( - FeatureSetRequest featureSetRequest, EntityRow entityRow, FeatureRow featureRow) { - Duration maxAge = featureSetRequest.getSpec().getMaxAge(); - if (maxAge.equals(Duration.getDefaultInstance())) { - return false; - } - long givenTimestamp = entityRow.getEntityTimestamp().getSeconds(); - if (givenTimestamp == 0) { - givenTimestamp = System.currentTimeMillis() / 1000; - } - long timeDifference = givenTimestamp - featureRow.getEventTimestamp().getSeconds(); - return timeDifference > maxAge.getSeconds(); - } } diff --git a/serving/src/main/java/feast/serving/util/mappers/ResponseJSONMapper.java b/serving/src/main/java/feast/serving/util/mappers/ResponseJSONMapper.java index 6aba17ac431..238df549513 100644 --- a/serving/src/main/java/feast/serving/util/mappers/ResponseJSONMapper.java +++ b/serving/src/main/java/feast/serving/util/mappers/ResponseJSONMapper.java @@ -29,7 +29,7 @@ public class ResponseJSONMapper { public static List> mapGetOnlineFeaturesResponse( GetOnlineFeaturesResponse response) { return response.getFieldValuesList().stream() - .map(fieldValue -> convertFieldValuesToMap(fieldValue)) + .map(fieldValues -> convertFieldValuesToMap(fieldValues)) .collect(Collectors.toList()); } diff --git a/serving/src/main/resources/application.yml b/serving/src/main/resources/application.yml index b5eff65391b..2399d132ef9 100644 --- a/serving/src/main/resources/application.yml +++ b/serving/src/main/resources/application.yml @@ -10,10 +10,9 @@ feast: # List of store configurations stores: - # Below are two store configurations. One for Redis and one for BigQuery. # Please see https://api.docs.feast.dev/grpc/feast.core.pb.html#Store for configuration options - name: online # Name of the store (referenced by active_store) - type: REDIS # Type of the store. REDIS, BIGQUERY are available options + type: REDIS # Type of the store. REDIS, REDIS_CLUSTER, BIGQUERY are available options config: # Store specific configuration. See host: localhost port: 6379 @@ -22,7 +21,15 @@ feast: # Wildcards match all options. No filtering is done. - name: "*" project: "*" - + - name: online_cluster + type: REDIS_CLUSTER + config: # Store specific configuration. + # Connection string specifies the host:port of Redis instances in the redis cluster. + connection_string: "localhost:7000,localhost:7001,localhost:7002,localhost:7003,localhost:7004,localhost:7005" + subscriptions: + - name: "*" + project: "*" + version: "*" - name: historical type: BIGQUERY config: # Store specific configuration. @@ -77,4 +84,4 @@ server: # The port number on which the Tomcat webserver that serves REST API endpoints should listen # It is set by default to 8081 so it does not conflict with Tomcat webserver on Feast Core # if both Feast Core and Serving are running on the same machine - port: ${SERVER_PORT:8081} \ No newline at end of file + port: ${SERVER_PORT:8081} diff --git a/serving/src/test/java/feast/serving/service/OnlineServingServiceTest.java b/serving/src/test/java/feast/serving/service/OnlineServingServiceTest.java index 6358460a070..9ab892a0c44 100644 --- a/serving/src/test/java/feast/serving/service/OnlineServingServiceTest.java +++ b/serving/src/test/java/feast/serving/service/OnlineServingServiceTest.java @@ -17,7 +17,7 @@ package feast.serving.service; import static org.hamcrest.MatcherAssert.assertThat; -import static org.hamcrest.Matchers.containsInAnyOrder; +import static org.hamcrest.Matchers.equalTo; import static org.mockito.Mockito.when; import static org.mockito.MockitoAnnotations.initMocks; @@ -30,9 +30,10 @@ import feast.proto.serving.ServingAPIProto.GetOnlineFeaturesRequest; import feast.proto.serving.ServingAPIProto.GetOnlineFeaturesRequest.EntityRow; import feast.proto.serving.ServingAPIProto.GetOnlineFeaturesResponse; +import feast.proto.serving.ServingAPIProto.GetOnlineFeaturesResponse.FieldStatus; import feast.proto.serving.ServingAPIProto.GetOnlineFeaturesResponse.FieldValues; import feast.proto.types.FeatureRowProto.FeatureRow; -import feast.proto.types.FieldProto.Field; +import feast.proto.types.FieldProto; import feast.proto.types.ValueProto.Value; import feast.serving.specs.CachedSpecService; import feast.storage.api.retriever.FeatureSetRequest; @@ -41,8 +42,7 @@ import io.opentracing.Tracer.SpanBuilder; import java.util.Collections; import java.util.List; -import java.util.Map; -import java.util.stream.Collectors; +import java.util.Optional; import org.junit.Before; import org.junit.Test; import org.mockito.ArgumentMatchers; @@ -66,11 +66,13 @@ public void setUp() { } @Test - public void shouldReturnResponseWithValuesIfKeysPresent() { + public void shouldReturnResponseWithValuesAndMetadataIfKeysPresent() { GetOnlineFeaturesRequest request = GetOnlineFeaturesRequest.newBuilder() + .setOmitEntitiesInResponse(false) .addFeatures(FeatureReference.newBuilder().setName("feature1").build()) - .addFeatures(FeatureReference.newBuilder().setName("feature2").build()) + .addFeatures( + FeatureReference.newBuilder().setName("feature2").setProject("project").build()) .addEntityRows( EntityRow.newBuilder() .setEntityTimestamp(Timestamp.newBuilder().setSeconds(100)) @@ -83,26 +85,54 @@ public void shouldReturnResponseWithValuesIfKeysPresent() { .putFields("entity2", strValue("b"))) .build(); - List featureRows = + List> featureRows = Lists.newArrayList( - FeatureRow.newBuilder() - .setEventTimestamp(Timestamp.newBuilder().setSeconds(100)) - .addAllFields( - Lists.newArrayList( - Field.newBuilder().setName("entity1").setValue(intValue(1)).build(), - Field.newBuilder().setName("entity2").setValue(strValue("a")).build(), - Field.newBuilder().setName("feature1").setValue(intValue(1)).build(), - Field.newBuilder().setName("feature2").setValue(intValue(1)).build())) - .build(), - FeatureRow.newBuilder() - .setEventTimestamp(Timestamp.newBuilder().setSeconds(100)) - .addAllFields( - Lists.newArrayList( - Field.newBuilder().setName("entity1").setValue(intValue(2)).build(), - Field.newBuilder().setName("entity2").setValue(strValue("b")).build(), - Field.newBuilder().setName("feature1").setValue(intValue(2)).build(), - Field.newBuilder().setName("feature2").setValue(intValue(2)).build())) - .build()); + Optional.of( + FeatureRow.newBuilder() + .setEventTimestamp(Timestamp.newBuilder().setSeconds(100)) + .addAllFields( + Lists.newArrayList( + FieldProto.Field.newBuilder() + .setName("entity1") + .setValue(intValue(1)) + .build(), + FieldProto.Field.newBuilder() + .setName("entity2") + .setValue(strValue("a")) + .build(), + FieldProto.Field.newBuilder() + .setName("feature1") + .setValue(intValue(1)) + .build(), + FieldProto.Field.newBuilder() + .setName("feature2") + .setValue(intValue(1)) + .build())) + .setFeatureSet("featureSet") + .build()), + Optional.of( + FeatureRow.newBuilder() + .setEventTimestamp(Timestamp.newBuilder().setSeconds(100)) + .addAllFields( + Lists.newArrayList( + FieldProto.Field.newBuilder() + .setName("entity1") + .setValue(intValue(2)) + .build(), + FieldProto.Field.newBuilder() + .setName("entity2") + .setValue(strValue("b")) + .build(), + FieldProto.Field.newBuilder() + .setName("feature1") + .setValue(intValue(2)) + .build(), + FieldProto.Field.newBuilder() + .setName("feature2") + .setValue(intValue(2)) + .build())) + .setFeatureSet("featureSet") + .build())); FeatureSetRequest featureSetRequest = FeatureSetRequest.newBuilder() @@ -112,9 +142,8 @@ public void shouldReturnResponseWithValuesIfKeysPresent() { when(specService.getFeatureSets(request.getFeaturesList())) .thenReturn(Collections.singletonList(featureSetRequest)); - when(retriever.getOnlineFeatures( - request.getEntityRowsList(), Collections.singletonList(featureSetRequest))) - .thenReturn(Collections.singletonList(featureRows)); + when(retriever.getOnlineFeatures(request.getEntityRowsList(), featureSetRequest)) + .thenReturn(featureRows); when(tracer.buildSpan(ArgumentMatchers.any())).thenReturn(Mockito.mock(SpanBuilder.class)); GetOnlineFeaturesResponse expected = @@ -122,28 +151,38 @@ public void shouldReturnResponseWithValuesIfKeysPresent() { .addFieldValues( FieldValues.newBuilder() .putFields("entity1", intValue(1)) + .putStatuses("entity1", FieldStatus.PRESENT) .putFields("entity2", strValue("a")) + .putStatuses("entity2", FieldStatus.PRESENT) .putFields("feature1", intValue(1)) - .putFields("feature2", intValue(1))) + .putStatuses("feature1", FieldStatus.PRESENT) + .putFields("project/feature2", intValue(1)) + .putStatuses("project/feature2", FieldStatus.PRESENT) + .build()) .addFieldValues( FieldValues.newBuilder() .putFields("entity1", intValue(2)) + .putStatuses("entity1", FieldStatus.PRESENT) .putFields("entity2", strValue("b")) + .putStatuses("entity2", FieldStatus.PRESENT) .putFields("feature1", intValue(2)) - .putFields("feature2", intValue(2))) + .putStatuses("feature1", FieldStatus.PRESENT) + .putFields("project/feature2", intValue(2)) + .putStatuses("project/feature2", FieldStatus.PRESENT) + .build()) .build(); GetOnlineFeaturesResponse actual = onlineServingService.getOnlineFeatures(request); - assertThat( - responseToMapList(actual), containsInAnyOrder(responseToMapList(expected).toArray())); + assertThat(actual, equalTo(expected)); } @Test - public void shouldReturnResponseWithUnsetValuesIfKeysNotPresent() { + public void shouldReturnResponseWithUnsetValuesAndMetadataIfKeysNotPresent() { // some keys not present, should have empty values GetOnlineFeaturesRequest request = GetOnlineFeaturesRequest.newBuilder() .addFeatures(FeatureReference.newBuilder().setName("feature1").build()) - .addFeatures(FeatureReference.newBuilder().setName("feature2").build()) + .addFeatures( + FeatureReference.newBuilder().setName("feature2").setProject("project").build()) .addEntityRows( EntityRow.newBuilder() .setEntityTimestamp(Timestamp.newBuilder().setSeconds(100)) @@ -162,29 +201,29 @@ public void shouldReturnResponseWithUnsetValuesIfKeysNotPresent() { .setSpec(getFeatureSetSpec()) .build(); - List featureRows = + List> featureRows = Lists.newArrayList( - FeatureRow.newBuilder() - .setEventTimestamp(Timestamp.newBuilder().setSeconds(100)) - .setFeatureSet("project/featureSet") - .addAllFields( - Lists.newArrayList( - Field.newBuilder().setName("feature1").setValue(intValue(1)).build(), - Field.newBuilder().setName("feature2").setValue(intValue(1)).build())) - .build(), - FeatureRow.newBuilder() - .setFeatureSet("project/featureSet") - .addAllFields( - Lists.newArrayList( - Field.newBuilder().setName("feature1").build(), - Field.newBuilder().setName("feature2").build())) - .build()); + Optional.of( + FeatureRow.newBuilder() + .setEventTimestamp(Timestamp.newBuilder().setSeconds(100)) + .setFeatureSet("project/featureSet") + .addAllFields( + Lists.newArrayList( + FieldProto.Field.newBuilder() + .setName("feature1") + .setValue(intValue(1)) + .build(), + FieldProto.Field.newBuilder() + .setName("feature2") + .setValue(intValue(1)) + .build())) + .build()), + Optional.empty()); when(specService.getFeatureSets(request.getFeaturesList())) .thenReturn(Collections.singletonList(featureSetRequest)); - when(retriever.getOnlineFeatures( - request.getEntityRowsList(), Collections.singletonList(featureSetRequest))) - .thenReturn(Collections.singletonList(featureRows)); + when(retriever.getOnlineFeatures(request.getEntityRowsList(), featureSetRequest)) + .thenReturn(featureRows); when(tracer.buildSpan(ArgumentMatchers.any())).thenReturn(Mockito.mock(SpanBuilder.class)); GetOnlineFeaturesResponse expected = @@ -192,28 +231,39 @@ public void shouldReturnResponseWithUnsetValuesIfKeysNotPresent() { .addFieldValues( FieldValues.newBuilder() .putFields("entity1", intValue(1)) + .putStatuses("entity1", FieldStatus.PRESENT) .putFields("entity2", strValue("a")) + .putStatuses("entity2", FieldStatus.PRESENT) .putFields("feature1", intValue(1)) - .putFields("feature2", intValue(1))) + .putStatuses("feature1", FieldStatus.PRESENT) + .putFields("project/feature2", intValue(1)) + .putStatuses("project/feature2", FieldStatus.PRESENT) + .build()) .addFieldValues( FieldValues.newBuilder() .putFields("entity1", intValue(2)) + .putStatuses("entity1", FieldStatus.PRESENT) .putFields("entity2", strValue("b")) + .putStatuses("entity2", FieldStatus.PRESENT) .putFields("feature1", Value.newBuilder().build()) - .putFields("feature2", Value.newBuilder().build())) + .putStatuses("feature1", FieldStatus.NOT_FOUND) + .putFields("project/feature2", Value.newBuilder().build()) + .putStatuses("project/feature2", FieldStatus.NOT_FOUND) + .build()) .build(); GetOnlineFeaturesResponse actual = onlineServingService.getOnlineFeatures(request); - assertThat( - responseToMapList(actual), containsInAnyOrder(responseToMapList(expected).toArray())); + assertThat(actual, equalTo(expected)); } @Test - public void shouldReturnResponseWithUnsetValuesIfMaxAgeIsExceeded() { - // keys present, but too stale comp. to maxAge + public void shouldReturnResponseWithUnsetValuesAndMetadataIfMaxAgeIsExceeded() { + // keys present, but considered stale when compared to maxAge GetOnlineFeaturesRequest request = GetOnlineFeaturesRequest.newBuilder() .addFeatures(FeatureReference.newBuilder().setName("feature1").build()) - .addFeatures(FeatureReference.newBuilder().setName("feature2").build()) + .addFeatures( + FeatureReference.newBuilder().setName("feature2").setProject("project").build()) + .setOmitEntitiesInResponse(false) .addEntityRows( EntityRow.newBuilder() .setEntityTimestamp(Timestamp.newBuilder().setSeconds(100)) @@ -226,29 +276,55 @@ public void shouldReturnResponseWithUnsetValuesIfMaxAgeIsExceeded() { .putFields("entity2", strValue("b"))) .build(); - List featureRows = + List> featureRows = Lists.newArrayList( - FeatureRow.newBuilder() - .setEventTimestamp(Timestamp.newBuilder().setSeconds(100)) - .addAllFields( - Lists.newArrayList( - Field.newBuilder().setName("entity1").setValue(intValue(1)).build(), - Field.newBuilder().setName("entity2").setValue(strValue("a")).build(), - Field.newBuilder().setName("feature1").setValue(intValue(1)).build(), - Field.newBuilder().setName("feature2").setValue(intValue(1)).build())) - .setFeatureSet("project/featureSet") - .build(), - FeatureRow.newBuilder() - .setEventTimestamp( - Timestamp.newBuilder().setSeconds(50)) // this value should be nulled - .addAllFields( - Lists.newArrayList( - Field.newBuilder().setName("entity1").setValue(intValue(2)).build(), - Field.newBuilder().setName("entity2").setValue(strValue("b")).build(), - Field.newBuilder().setName("feature1").setValue(intValue(2)).build(), - Field.newBuilder().setName("feature2").setValue(intValue(2)).build())) - .setFeatureSet("project/featureSet") - .build()); + Optional.of( + FeatureRow.newBuilder() + .setEventTimestamp(Timestamp.newBuilder().setSeconds(100)) + .addAllFields( + Lists.newArrayList( + FieldProto.Field.newBuilder() + .setName("entity1") + .setValue(intValue(1)) + .build(), + FieldProto.Field.newBuilder() + .setName("entity2") + .setValue(strValue("a")) + .build(), + FieldProto.Field.newBuilder() + .setName("feature1") + .setValue(intValue(1)) + .build(), + FieldProto.Field.newBuilder() + .setName("feature2") + .setValue(intValue(1)) + .build())) + .setFeatureSet("project/featureSet") + .build()), + Optional.of( + FeatureRow.newBuilder() + .setEventTimestamp( + Timestamp.newBuilder().setSeconds(50)) // this value should be nulled + .addAllFields( + Lists.newArrayList( + FieldProto.Field.newBuilder() + .setName("entity1") + .setValue(intValue(2)) + .build(), + FieldProto.Field.newBuilder() + .setName("entity2") + .setValue(strValue("b")) + .build(), + FieldProto.Field.newBuilder() + .setName("feature1") + .setValue(intValue(2)) + .build(), + FieldProto.Field.newBuilder() + .setName("project/feature2") + .setValue(intValue(2)) + .build())) + .setFeatureSet("project/featureSet") + .build())); FeatureSetSpec spec = getFeatureSetSpec().toBuilder().setMaxAge(Duration.newBuilder().setSeconds(1)).build(); @@ -260,9 +336,8 @@ public void shouldReturnResponseWithUnsetValuesIfMaxAgeIsExceeded() { when(specService.getFeatureSets(request.getFeaturesList())) .thenReturn(Collections.singletonList(featureSetRequest)); - when(retriever.getOnlineFeatures( - request.getEntityRowsList(), Collections.singletonList(featureSetRequest))) - .thenReturn(Collections.singletonList(featureRows)); + when(retriever.getOnlineFeatures(request.getEntityRowsList(), featureSetRequest)) + .thenReturn(featureRows); when(tracer.buildSpan(ArgumentMatchers.any())).thenReturn(Mockito.mock(SpanBuilder.class)); GetOnlineFeaturesResponse expected = @@ -270,19 +345,28 @@ public void shouldReturnResponseWithUnsetValuesIfMaxAgeIsExceeded() { .addFieldValues( FieldValues.newBuilder() .putFields("entity1", intValue(1)) + .putStatuses("entity1", FieldStatus.PRESENT) .putFields("entity2", strValue("a")) + .putStatuses("entity2", FieldStatus.PRESENT) .putFields("feature1", intValue(1)) - .putFields("feature2", intValue(1))) + .putStatuses("feature1", FieldStatus.PRESENT) + .putFields("project/feature2", intValue(1)) + .putStatuses("project/feature2", FieldStatus.PRESENT) + .build()) .addFieldValues( FieldValues.newBuilder() .putFields("entity1", intValue(2)) + .putStatuses("entity1", FieldStatus.PRESENT) .putFields("entity2", strValue("b")) + .putStatuses("entity2", FieldStatus.PRESENT) .putFields("feature1", Value.newBuilder().build()) - .putFields("feature2", Value.newBuilder().build())) + .putStatuses("feature1", FieldStatus.OUTSIDE_MAX_AGE) + .putFields("project/feature2", Value.newBuilder().build()) + .putStatuses("project/feature2", FieldStatus.OUTSIDE_MAX_AGE) + .build()) .build(); GetOnlineFeaturesResponse actual = onlineServingService.getOnlineFeatures(request); - assertThat( - responseToMapList(actual), containsInAnyOrder(responseToMapList(expected).toArray())); + assertThat(actual, equalTo(expected)); } @Test @@ -290,6 +374,7 @@ public void shouldFilterOutUndesiredRows() { // requested rows less than the rows available in the featureset GetOnlineFeaturesRequest request = GetOnlineFeaturesRequest.newBuilder() + .setOmitEntitiesInResponse(false) .addFeatures(FeatureReference.newBuilder().setName("feature1").build()) .addEntityRows( EntityRow.newBuilder() @@ -303,26 +388,54 @@ public void shouldFilterOutUndesiredRows() { .putFields("entity2", strValue("b"))) .build(); - List featureRows = + List> featureRows = Lists.newArrayList( - FeatureRow.newBuilder() - .setEventTimestamp(Timestamp.newBuilder().setSeconds(100)) - .addAllFields( - Lists.newArrayList( - Field.newBuilder().setName("entity1").setValue(intValue(1)).build(), - Field.newBuilder().setName("entity2").setValue(strValue("a")).build(), - Field.newBuilder().setName("feature1").setValue(intValue(1)).build(), - Field.newBuilder().setName("feature2").setValue(intValue(1)).build())) - .build(), - FeatureRow.newBuilder() - .setEventTimestamp(Timestamp.newBuilder().setSeconds(100)) - .addAllFields( - Lists.newArrayList( - Field.newBuilder().setName("entity1").setValue(intValue(2)).build(), - Field.newBuilder().setName("entity2").setValue(strValue("b")).build(), - Field.newBuilder().setName("feature1").setValue(intValue(2)).build(), - Field.newBuilder().setName("feature2").setValue(intValue(2)).build())) - .build()); + Optional.of( + FeatureRow.newBuilder() + .setEventTimestamp(Timestamp.newBuilder().setSeconds(100)) + .addAllFields( + Lists.newArrayList( + FieldProto.Field.newBuilder() + .setName("entity1") + .setValue(intValue(1)) + .build(), + FieldProto.Field.newBuilder() + .setName("entity2") + .setValue(strValue("a")) + .build(), + FieldProto.Field.newBuilder() + .setName("feature1") + .setValue(intValue(1)) + .build(), + FieldProto.Field.newBuilder() + .setName("feature2") + .setValue(intValue(1)) + .build())) + .setFeatureSet("featureSet") + .build()), + Optional.of( + FeatureRow.newBuilder() + .setEventTimestamp(Timestamp.newBuilder().setSeconds(100)) + .addAllFields( + Lists.newArrayList( + FieldProto.Field.newBuilder() + .setName("entity1") + .setValue(intValue(2)) + .build(), + FieldProto.Field.newBuilder() + .setName("entity2") + .setValue(strValue("b")) + .build(), + FieldProto.Field.newBuilder() + .setName("feature1") + .setValue(intValue(2)) + .build(), + FieldProto.Field.newBuilder() + .setName("feature2") + .setValue(intValue(2)) + .build())) + .setFeatureSet("featureSet") + .build())); FeatureSetRequest featureSetRequest = FeatureSetRequest.newBuilder() @@ -332,9 +445,8 @@ public void shouldFilterOutUndesiredRows() { when(specService.getFeatureSets(request.getFeaturesList())) .thenReturn(Collections.singletonList(featureSetRequest)); - when(retriever.getOnlineFeatures( - request.getEntityRowsList(), Collections.singletonList(featureSetRequest))) - .thenReturn(Collections.singletonList(featureRows)); + when(retriever.getOnlineFeatures(request.getEntityRowsList(), featureSetRequest)) + .thenReturn(featureRows); when(tracer.buildSpan(ArgumentMatchers.any())).thenReturn(Mockito.mock(SpanBuilder.class)); GetOnlineFeaturesResponse expected = @@ -342,27 +454,28 @@ public void shouldFilterOutUndesiredRows() { .addFieldValues( FieldValues.newBuilder() .putFields("entity1", intValue(1)) + .putStatuses("entity1", FieldStatus.PRESENT) .putFields("entity2", strValue("a")) - .putFields("feature1", intValue(1))) + .putStatuses("entity2", FieldStatus.PRESENT) + .putFields("feature1", intValue(1)) + .putStatuses("feature1", FieldStatus.PRESENT) + .build()) .addFieldValues( FieldValues.newBuilder() .putFields("entity1", intValue(2)) + .putStatuses("entity1", FieldStatus.PRESENT) .putFields("entity2", strValue("b")) - .putFields("feature1", intValue(2))) + .putStatuses("entity2", FieldStatus.PRESENT) + .putFields("feature1", intValue(2)) + .putStatuses("feature1", FieldStatus.PRESENT) + .build()) .build(); GetOnlineFeaturesResponse actual = onlineServingService.getOnlineFeatures(request); - assertThat( - responseToMapList(actual), containsInAnyOrder(responseToMapList(expected).toArray())); - } - - private List> responseToMapList(GetOnlineFeaturesResponse response) { - return response.getFieldValuesList().stream() - .map(FieldValues::getFieldsMap) - .collect(Collectors.toList()); + assertThat(actual, equalTo(expected)); } private Value intValue(int val) { - return Value.newBuilder().setInt64Val(val).build(); + return Value.newBuilder().setInt32Val(val).build(); } private Value strValue(String val) { diff --git a/storage/api/src/main/java/feast/storage/api/retriever/OnlineRetriever.java b/storage/api/src/main/java/feast/storage/api/retriever/OnlineRetriever.java index 45d6b352632..f56cf6ca022 100644 --- a/storage/api/src/main/java/feast/storage/api/retriever/OnlineRetriever.java +++ b/storage/api/src/main/java/feast/storage/api/retriever/OnlineRetriever.java @@ -19,22 +19,25 @@ import feast.proto.serving.ServingAPIProto.GetOnlineFeaturesRequest.EntityRow; import feast.proto.types.FeatureRowProto.FeatureRow; import java.util.List; +import java.util.Optional; -/** - * An online retriever is a feature retriever that retrieves the latest feature data corresponding - * to provided entities. - */ +/** An online retriever is a feature retriever that retrieves the latest feature data. */ public interface OnlineRetriever { /** - * Get all values corresponding to the request. + * Get online features for the given entity rows using data retrieved from the feature/featureset + * specified in feature set request. + * + *

Each {@link FeatureRow} optional in the returned list then corresponds to an {@link + * EntityRow} provided by the user. If feature for a given entity row is not found, will return an + * empty optional instead. The no. of {@link FeatureRow} returned should match the no. of given + * {@link EntityRow}s * - * @param entityRows list of entity rows in the feature request - * @param featureSetRequests List of {@link FeatureSetRequest} to feature references in the - * request tied to that feature set. - * @return list of lists of {@link FeatureRow}s corresponding to each feature set request and - * entity row. + * @param entityRows list of entity rows to request features for. + * @param featureSetRequest specifies the features/feature set to retrieve data from + * @return list of {@link FeatureRow}s corresponding to data retrieved for each entity row from + * feature/featureset specified in featureset request. */ - List> getOnlineFeatures( - List entityRows, List featureSetRequests); + List> getOnlineFeatures( + List entityRows, FeatureSetRequest featureSetRequest); } diff --git a/storage/connectors/redis/src/main/java/feast/storage/connectors/redis/retriever/FeatureRowDecoder.java b/storage/connectors/redis/src/main/java/feast/storage/connectors/redis/retriever/FeatureRowDecoder.java index fd9556841e4..aad3147f710 100644 --- a/storage/connectors/redis/src/main/java/feast/storage/connectors/redis/retriever/FeatureRowDecoder.java +++ b/storage/connectors/redis/src/main/java/feast/storage/connectors/redis/retriever/FeatureRowDecoder.java @@ -43,7 +43,7 @@ public FeatureRowDecoder(String featureSetRef, FeatureSetSpec spec) { * @param featureRow Feature row * @return boolean */ - public Boolean isEncoded(FeatureRow featureRow) { + public boolean isEncoded(FeatureRow featureRow) { return featureRow.getFeatureSet().isEmpty() && featureRow.getFieldsList().stream().allMatch(field -> field.getName().isEmpty()); } @@ -54,7 +54,7 @@ public Boolean isEncoded(FeatureRow featureRow) { * @param featureRow Feature row * @return boolean */ - public Boolean isEncodingValid(FeatureRow featureRow) { + public boolean isEncodingValid(FeatureRow featureRow) { return featureRow.getFieldsList().size() == spec.getFeaturesList().size(); } diff --git a/storage/connectors/redis/src/main/java/feast/storage/connectors/redis/retriever/RedisClusterOnlineRetriever.java b/storage/connectors/redis/src/main/java/feast/storage/connectors/redis/retriever/RedisClusterOnlineRetriever.java index 18619252c27..c006149cd51 100644 --- a/storage/connectors/redis/src/main/java/feast/storage/connectors/redis/retriever/RedisClusterOnlineRetriever.java +++ b/storage/connectors/redis/src/main/java/feast/storage/connectors/redis/retriever/RedisClusterOnlineRetriever.java @@ -20,7 +20,6 @@ import com.google.protobuf.InvalidProtocolBufferException; import feast.proto.core.FeatureSetProto.EntitySpec; import feast.proto.core.FeatureSetProto.FeatureSetSpec; -import feast.proto.serving.ServingAPIProto.FeatureReference; import feast.proto.serving.ServingAPIProto.GetOnlineFeaturesRequest.EntityRow; import feast.proto.storage.RedisProto.RedisKey; import feast.proto.types.FeatureRowProto.FeatureRow; @@ -38,9 +37,11 @@ import java.util.Arrays; import java.util.List; import java.util.Map; +import java.util.Optional; import java.util.concurrent.ExecutionException; import java.util.stream.Collectors; +/** Defines a storage retriever */ public class RedisClusterOnlineRetriever implements OnlineRetriever { private final RedisAdvancedClusterCommands syncCommands; @@ -69,36 +70,24 @@ public static OnlineRetriever create(StatefulRedisClusterConnection> getOnlineFeatures( - List entityRows, List featureSetRequests) { - - List> featureRows = new ArrayList<>(); - for (FeatureSetRequest featureSetRequest : featureSetRequests) { - List redisKeys = buildRedisKeys(entityRows, featureSetRequest.getSpec()); - try { - List featureRowsForFeatureSet = - sendAndProcessMultiGet( - redisKeys, - featureSetRequest.getSpec(), - featureSetRequest.getFeatureReferences().asList()); - featureRows.add(featureRowsForFeatureSet); - } catch (InvalidProtocolBufferException | ExecutionException e) { - throw Status.INTERNAL - .withDescription("Unable to parse protobuf while retrieving feature") - .withCause(e) - .asRuntimeException(); - } + public List> getOnlineFeatures( + List entityRows, FeatureSetRequest featureSetRequest) { + + // get features for this features/featureset in featureset request + FeatureSetSpec featureSetSpec = featureSetRequest.getSpec(); + List redisKeys = buildRedisKeys(entityRows, featureSetSpec); + FeatureRowDecoder decoder = + new FeatureRowDecoder(generateFeatureSetStringRef(featureSetSpec), featureSetSpec); + List> featureRows = new ArrayList<>(); + try { + featureRows = getFeaturesFromRedis(redisKeys, decoder); + } catch (InvalidProtocolBufferException | ExecutionException e) { + throw Status.INTERNAL + .withDescription("Unable to parse protobuf while retrieving feature") + .withCause(e) + .asRuntimeException(); } return featureRows; } @@ -147,49 +136,51 @@ private RedisKey makeRedisKey( return builder.build(); } - private List sendAndProcessMultiGet( - List redisKeys, - FeatureSetSpec featureSetSpec, - List featureReferences) + /** + * Get features from data pulled from the Redis for a specific featureset. + * + * @param redisKeys keys used to retrieve data from Redis for a specific featureset. + * @param decoder used to decode the data retrieved from Redis for a specific featureset. + * @return List of {@link FeatureRow} optionals + */ + private List> getFeaturesFromRedis( + List redisKeys, FeatureRowDecoder decoder) throws InvalidProtocolBufferException, ExecutionException { + // pull feature row data bytes from redis using given redis keys + List featureRowsBytes = sendMultiGet(redisKeys); + List> featureRows = new ArrayList<>(); - List values = sendMultiGet(redisKeys); - List featureRows = new ArrayList<>(); - - FeatureRow.Builder nullFeatureRowBuilder = - FeatureRow.newBuilder().setFeatureSet(generateFeatureSetStringRef(featureSetSpec)); - for (FeatureReference featureReference : featureReferences) { - nullFeatureRowBuilder.addFields(Field.newBuilder().setName(featureReference.getName())); - } - - for (int i = 0; i < values.size(); i++) { - - byte[] value = values.get(i); - if (value == null) { - featureRows.add(nullFeatureRowBuilder.build()); + for (byte[] featureRowBytes : featureRowsBytes) { + if (featureRowBytes == null) { + featureRows.add(Optional.empty()); continue; } - FeatureRow featureRow = FeatureRow.parseFrom(value); - String featureSetRef = redisKeys.get(i).getFeatureSet(); - FeatureRowDecoder decoder = new FeatureRowDecoder(featureSetRef, featureSetSpec); - if (decoder.isEncodingValid(featureRow)) { - featureRow = decoder.decode(featureRow); - } else { - featureRows.add(nullFeatureRowBuilder.build()); - continue; + // decode feature rows from data bytes using decoder. + FeatureRow featureRow = FeatureRow.parseFrom(featureRowBytes); + if (decoder.isEncoded(featureRow)) { + if (decoder.isEncodingValid(featureRow)) { + featureRow = decoder.decode(featureRow); + } else { + // decoding feature row failed: data corruption could have occurred + throw Status.DATA_LOSS + .withDescription( + "Failed to decode FeatureRow from bytes retrieved from redis" + + ": Possible data corruption") + .asRuntimeException(); + } } - - featureRows.add(featureRow); + featureRows.add(Optional.of(featureRow)); } return featureRows; } /** - * Send a list of get request as an mget + * Pull the data stored in Redis at the given keys as bytes using the mget command. If no data is + * stored at a given key in Redis, will subsitute the data with null. * - * @param keys list of {@link RedisKey} - * @return list of {@link FeatureRow} in primitive byte representation for each {@link RedisKey} + * @param keys list of {@link RedisKey} to pull from redis. + * @return list of data bytes or null pulled from redis for each given key. */ private List sendMultiGet(List keys) { try { diff --git a/storage/connectors/redis/src/main/java/feast/storage/connectors/redis/retriever/RedisOnlineRetriever.java b/storage/connectors/redis/src/main/java/feast/storage/connectors/redis/retriever/RedisOnlineRetriever.java index 0db4837c069..2b09f7346bb 100644 --- a/storage/connectors/redis/src/main/java/feast/storage/connectors/redis/retriever/RedisOnlineRetriever.java +++ b/storage/connectors/redis/src/main/java/feast/storage/connectors/redis/retriever/RedisOnlineRetriever.java @@ -20,7 +20,6 @@ import com.google.protobuf.InvalidProtocolBufferException; import feast.proto.core.FeatureSetProto.EntitySpec; import feast.proto.core.FeatureSetProto.FeatureSetSpec; -import feast.proto.serving.ServingAPIProto.FeatureReference; import feast.proto.serving.ServingAPIProto.GetOnlineFeaturesRequest.EntityRow; import feast.proto.storage.RedisProto.RedisKey; import feast.proto.types.FeatureRowProto.FeatureRow; @@ -37,6 +36,7 @@ import java.util.ArrayList; import java.util.List; import java.util.Map; +import java.util.Optional; import java.util.concurrent.ExecutionException; import java.util.stream.Collectors; @@ -62,37 +62,26 @@ public static OnlineRetriever create(StatefulRedisConnection con return new RedisOnlineRetriever(connection); } - /** - * Gets online features from redis. This method returns a list of {@link FeatureRow}s - * corresponding to each feature set spec. Each feature row in the list then corresponds to an - * {@link EntityRow} provided by the user. - * - * @param entityRows list of entity rows in the feature request - * @param featureSetRequests Map of {@link feast.proto.core.FeatureSetProto.FeatureSetSpec} to - * feature references in the request tied to that feature set. - * @return List of List of {@link FeatureRow} - */ + /** {@inheritDoc} */ @Override - public List> getOnlineFeatures( - List entityRows, List featureSetRequests) { - - List> featureRows = new ArrayList<>(); - for (FeatureSetRequest featureSetRequest : featureSetRequests) { - List redisKeys = buildRedisKeys(entityRows, featureSetRequest.getSpec()); - try { - List featureRowsForFeatureSet = - sendAndProcessMultiGet( - redisKeys, - featureSetRequest.getSpec(), - featureSetRequest.getFeatureReferences().asList()); - featureRows.add(featureRowsForFeatureSet); - } catch (InvalidProtocolBufferException | ExecutionException e) { - throw Status.INTERNAL - .withDescription("Unable to parse protobuf while retrieving feature") - .withCause(e) - .asRuntimeException(); - } + public List> getOnlineFeatures( + List entityRows, FeatureSetRequest featureSetRequest) { + + // get features for this features/featureset in featureset request + FeatureSetSpec featureSetSpec = featureSetRequest.getSpec(); + List redisKeys = buildRedisKeys(entityRows, featureSetSpec); + FeatureRowDecoder decoder = + new FeatureRowDecoder(generateFeatureSetStringRef(featureSetSpec), featureSetSpec); + List> featureRows = new ArrayList<>(); + try { + featureRows = getFeaturesFromRedis(redisKeys, decoder); + } catch (InvalidProtocolBufferException | ExecutionException e) { + throw Status.INTERNAL + .withDescription("Unable to parse protobuf while retrieving feature") + .withCause(e) + .asRuntimeException(); } + return featureRows; } @@ -140,51 +129,51 @@ private RedisKey makeRedisKey( return builder.build(); } - private List sendAndProcessMultiGet( - List redisKeys, - FeatureSetSpec featureSetSpec, - List featureReferences) + /** + * Get features from data pulled from the Redis for a specific featureset. + * + * @param redisKeys keys used to retrieve data from Redis for a specific featureset. + * @param decoder used to decode the data retrieved from Redis for a specific featureset. + * @return List of {@link FeatureRow} optionals + */ + private List> getFeaturesFromRedis( + List redisKeys, FeatureRowDecoder decoder) throws InvalidProtocolBufferException, ExecutionException { + // pull feature row data bytes from redis using given redis keys + List featureRowsBytes = sendMultiGet(redisKeys); + List> featureRows = new ArrayList<>(); - List values = sendMultiGet(redisKeys); - List featureRows = new ArrayList<>(); - - FeatureRow.Builder nullFeatureRowBuilder = - FeatureRow.newBuilder().setFeatureSet(generateFeatureSetStringRef(featureSetSpec)); - for (FeatureReference featureReference : featureReferences) { - nullFeatureRowBuilder.addFields(Field.newBuilder().setName(featureReference.getName())); - } - - for (int i = 0; i < values.size(); i++) { - - byte[] value = values.get(i); - if (value == null) { - featureRows.add(nullFeatureRowBuilder.build()); + for (byte[] featureRowBytes : featureRowsBytes) { + if (featureRowBytes == null) { + featureRows.add(Optional.empty()); continue; } - FeatureRow featureRow = FeatureRow.parseFrom(value); - String featureSetRef = redisKeys.get(i).getFeatureSet(); - FeatureRowDecoder decoder = new FeatureRowDecoder(featureSetRef, featureSetSpec); + // decode feature rows from data bytes using decoder. + FeatureRow featureRow = FeatureRow.parseFrom(featureRowBytes); if (decoder.isEncoded(featureRow)) { if (decoder.isEncodingValid(featureRow)) { featureRow = decoder.decode(featureRow); } else { - featureRows.add(nullFeatureRowBuilder.build()); - continue; + // decoding feature row failed: data corruption could have occurred + throw Status.DATA_LOSS + .withDescription( + "Failed to decode FeatureRow from bytes retrieved from redis" + + ": Possible data corruption") + .asRuntimeException(); } } - - featureRows.add(featureRow); + featureRows.add(Optional.of(featureRow)); } return featureRows; } /** - * Send a list of get request as an mget + * Pull the data stored in Redis at the given keys as bytes using the mget command. If no data is + * stored at a given key in Redis, will subsitute the data with null. * - * @param keys list of {@link RedisKey} - * @return list of {@link FeatureRow} in primitive byte representation for each {@link RedisKey} + * @param keys list of {@link RedisKey} to pull from redis. + * @return list of data bytes or null pulled from redis for each given key. */ private List sendMultiGet(List keys) { try { @@ -203,14 +192,14 @@ private List sendMultiGet(List keys) { }) .collect(Collectors.toList()); } catch (Exception e) { - throw Status.NOT_FOUND - .withDescription("Unable to retrieve feature from Redis") + throw Status.UNKNOWN + .withDescription("Unexpected error when pulling data from from Redis.") .withCause(e) .asRuntimeException(); } } - // TODO: Refactor this out to common package? + // TODO: Refactor this out to common package private static String generateFeatureSetStringRef(FeatureSetSpec featureSetSpec) { String ref = String.format("%s/%s", featureSetSpec.getProject(), featureSetSpec.getName()); return ref; diff --git a/storage/connectors/redis/src/test/java/feast/storage/connectors/redis/retriever/RedisClusterOnlineRetrieverTest.java b/storage/connectors/redis/src/test/java/feast/storage/connectors/redis/retriever/RedisClusterOnlineRetrieverTest.java index 45842cef6ee..76e22d07c54 100644 --- a/storage/connectors/redis/src/test/java/feast/storage/connectors/redis/retriever/RedisClusterOnlineRetrieverTest.java +++ b/storage/connectors/redis/src/test/java/feast/storage/connectors/redis/retriever/RedisClusterOnlineRetrieverTest.java @@ -132,9 +132,9 @@ public void shouldReturnResponseWithValuesIfKeysPresent() { when(connection.sync()).thenReturn(syncCommands); when(syncCommands.mget(redisKeyList)).thenReturn(featureRowBytes); - List> expected = - ImmutableList.of( - Lists.newArrayList( + List> expected = + Lists.newArrayList( + Optional.of( FeatureRow.newBuilder() .setEventTimestamp(Timestamp.newBuilder().setSeconds(100)) .setFeatureSet("project/featureSet") @@ -142,7 +142,8 @@ public void shouldReturnResponseWithValuesIfKeysPresent() { Lists.newArrayList( Field.newBuilder().setName("feature1").setValue(intValue(1)).build(), Field.newBuilder().setName("feature2").setValue(intValue(1)).build())) - .build(), + .build()), + Optional.of( FeatureRow.newBuilder() .setEventTimestamp(Timestamp.newBuilder().setSeconds(100)) .setFeatureSet("project/featureSet") @@ -152,14 +153,13 @@ public void shouldReturnResponseWithValuesIfKeysPresent() { Field.newBuilder().setName("feature2").setValue(intValue(2)).build())) .build())); - List> actual = - redisClusterOnlineRetriever.getOnlineFeatures( - entityRows, ImmutableList.of(featureSetRequest)); + List> actual = + redisClusterOnlineRetriever.getOnlineFeatures(entityRows, featureSetRequest); assertThat(actual, equalTo(expected)); } @Test - public void shouldReturnResponseWithUnsetValuesIfKeysNotPresent() { + public void shouldReturnNullIfKeysNotPresent() { FeatureSetRequest featureSetRequest = FeatureSetRequest.newBuilder() .setSpec(getFeatureSetSpec()) @@ -201,9 +201,9 @@ public void shouldReturnResponseWithUnsetValuesIfKeysNotPresent() { when(connection.sync()).thenReturn(syncCommands); when(syncCommands.mget(redisKeyList)).thenReturn(featureRowBytes); - List> expected = - ImmutableList.of( - Lists.newArrayList( + List> expected = + Lists.newArrayList( + Optional.of( FeatureRow.newBuilder() .setEventTimestamp(Timestamp.newBuilder().setSeconds(100)) .setFeatureSet("project/featureSet") @@ -211,18 +211,11 @@ public void shouldReturnResponseWithUnsetValuesIfKeysNotPresent() { Lists.newArrayList( Field.newBuilder().setName("feature1").setValue(intValue(1)).build(), Field.newBuilder().setName("feature2").setValue(intValue(1)).build())) - .build(), - FeatureRow.newBuilder() - .setFeatureSet("project/featureSet") - .addAllFields( - Lists.newArrayList( - Field.newBuilder().setName("feature1").build(), - Field.newBuilder().setName("feature2").build())) - .build())); + .build()), + Optional.empty()); - List> actual = - redisClusterOnlineRetriever.getOnlineFeatures( - entityRows, ImmutableList.of(featureSetRequest)); + List> actual = + redisClusterOnlineRetriever.getOnlineFeatures(entityRows, featureSetRequest); assertThat(actual, equalTo(expected)); } diff --git a/storage/connectors/redis/src/test/java/feast/storage/connectors/redis/retriever/RedisOnlineRetrieverTest.java b/storage/connectors/redis/src/test/java/feast/storage/connectors/redis/retriever/RedisOnlineRetrieverTest.java index adacacb941e..1292f4ab0dc 100644 --- a/storage/connectors/redis/src/test/java/feast/storage/connectors/redis/retriever/RedisOnlineRetrieverTest.java +++ b/storage/connectors/redis/src/test/java/feast/storage/connectors/redis/retriever/RedisOnlineRetrieverTest.java @@ -132,9 +132,9 @@ public void shouldReturnResponseWithValuesIfKeysPresent() { when(connection.sync()).thenReturn(syncCommands); when(syncCommands.mget(redisKeyList)).thenReturn(featureRowBytes); - List> expected = - ImmutableList.of( - Lists.newArrayList( + List> expected = + Lists.newArrayList( + Optional.of( FeatureRow.newBuilder() .setEventTimestamp(Timestamp.newBuilder().setSeconds(100)) .setFeatureSet("project/featureSet") @@ -142,7 +142,8 @@ public void shouldReturnResponseWithValuesIfKeysPresent() { Lists.newArrayList( Field.newBuilder().setName("feature1").setValue(intValue(1)).build(), Field.newBuilder().setName("feature2").setValue(intValue(1)).build())) - .build(), + .build()), + Optional.of( FeatureRow.newBuilder() .setEventTimestamp(Timestamp.newBuilder().setSeconds(100)) .setFeatureSet("project/featureSet") @@ -152,13 +153,13 @@ public void shouldReturnResponseWithValuesIfKeysPresent() { Field.newBuilder().setName("feature2").setValue(intValue(2)).build())) .build())); - List> actual = - redisOnlineRetriever.getOnlineFeatures(entityRows, ImmutableList.of(featureSetRequest)); + List> actual = + redisOnlineRetriever.getOnlineFeatures(entityRows, featureSetRequest); assertThat(actual, equalTo(expected)); } @Test - public void shouldReturnResponseWithUnsetValuesIfKeysNotPresent() { + public void shouldReturnNullIfKeysNotPresent() { FeatureSetRequest featureSetRequest = FeatureSetRequest.newBuilder() .setSpec(getFeatureSetSpec()) @@ -192,7 +193,7 @@ public void shouldReturnResponseWithUnsetValuesIfKeysNotPresent() { List> featureRowBytes = featureRows.stream() - .map(x -> KeyValue.from(new byte[1], Optional.of(x.toByteArray()))) + .map(row -> KeyValue.from(new byte[1], Optional.of(row.toByteArray()))) .collect(Collectors.toList()); featureRowBytes.add(null); @@ -200,9 +201,9 @@ public void shouldReturnResponseWithUnsetValuesIfKeysNotPresent() { when(connection.sync()).thenReturn(syncCommands); when(syncCommands.mget(redisKeyList)).thenReturn(featureRowBytes); - List> expected = - ImmutableList.of( - Lists.newArrayList( + List> expected = + Lists.newArrayList( + Optional.of( FeatureRow.newBuilder() .setEventTimestamp(Timestamp.newBuilder().setSeconds(100)) .setFeatureSet("project/featureSet") @@ -210,17 +211,10 @@ public void shouldReturnResponseWithUnsetValuesIfKeysNotPresent() { Lists.newArrayList( Field.newBuilder().setName("feature1").setValue(intValue(1)).build(), Field.newBuilder().setName("feature2").setValue(intValue(1)).build())) - .build(), - FeatureRow.newBuilder() - .setFeatureSet("project/featureSet") - .addAllFields( - Lists.newArrayList( - Field.newBuilder().setName("feature1").build(), - Field.newBuilder().setName("feature2").build())) - .build())); - - List> actual = - redisOnlineRetriever.getOnlineFeatures(entityRows, ImmutableList.of(featureSetRequest)); + .build()), + Optional.empty()); + List> actual = + redisOnlineRetriever.getOnlineFeatures(entityRows, featureSetRequest); assertThat(actual, equalTo(expected)); } diff --git a/tests/e2e/redis/basic-ingest-redis-serving.py b/tests/e2e/redis/basic-ingest-redis-serving.py index 778323c9cb5..5facfe7ee39 100644 --- a/tests/e2e/redis/basic-ingest-redis-serving.py +++ b/tests/e2e/redis/basic-ingest-redis-serving.py @@ -17,6 +17,7 @@ from feast.client import Client from feast.feature_set import FeatureSet, FeatureSetRef from feast.type_map import ValueType +from feast.wait import wait_retry_backoff from feast.constants import FEAST_DEFAULT_OPTIONS, CONFIG_PROJECT_KEY from google.protobuf.duration_pb2 import Duration from datetime import datetime @@ -33,6 +34,56 @@ PROJECT_NAME = 'basic_' + uuid.uuid4().hex.upper()[0:6] DIR_PATH = os.path.dirname(os.path.realpath(__file__)) +def basic_dataframe(entities, features, ingest_time, n_size, null_features=[]): + """ + Generate a basic feast-ingestable dataframe for testing. + Entity value incrementally increase from 1 to n_size + Features values are randomlly generated floats. + entities - names of entities + features - names of the features + ingest_time - ingestion timestamp + n_size - no. of rows in the generated dataframe. + null_features - names of features that contain null values + Returns the generated dataframe + """ + offset = random.randint(1000, 100000) # ensure a unique key space is used + df_dict = { + "datetime": [ingest_time.replace(tzinfo=pytz.utc) for _ in + range(n_size)], + } + for entity_name in entities: + df_dict[entity_name] = list(range(1, n_size + 1)) + for feature_name in features: + df_dict[feature_name] = [np.random.rand() for _ in range(n_size)] + for null_feature_name in null_features: + df_dict[null_feature_name] = [None for _ in range(n_size)] + return pd.DataFrame(df_dict) + +def check_online_response(feature_ref, ingest_df, response): + """ + Check the feature value and status in the given online serving response. + feature_refs - string feature ref used to access feature in response + ingest_df - dataframe of ingested values + response - response to extract retrieved feature value and metadata + Returns True if given response has expected feature value and metadata, otherwise False. + """ + feature_ref_splits = feature_ref.split(":") + if len(feature_ref_splits) == 1: + feature_name = feature_ref + else: + _, feature_name = feature_ref_splits + + returned_status = response.field_values[0].statuses[feature_ref] + if ingest_df.loc[0, feature_name] is None: + return returned_status == GetOnlineFeaturesResponse.FieldStatus.NULL_VALUE + else: + sent_value = float(ingest_df.iloc[0][feature_name]) + returned_value = float(response.field_values[0].fields[feature_ref].float_val) + return ( + math.isclose(sent_value, returned_value, abs_tol=FLOAT_TOLERANCE) + and returned_status == GetOnlineFeaturesResponse.FieldStatus.PRESENT + ) + @pytest.fixture(scope='module') def core_url(pytestconfig): @@ -78,19 +129,6 @@ def client(core_url, serving_url, allow_dirty): return client -def basic_dataframe(entities, features, ingest_time, n_size): - offset = random.randint(1000, 100000) # ensure a unique key space is used - df_dict = { - "datetime": [ingest_time.replace(tzinfo=pytz.utc) for _ in - range(n_size)], - } - for entity_name in entities: - df_dict[entity_name] = list(range(1, n_size + 1)) - for feature_name in features: - df_dict[feature_name] = [np.random.rand() for _ in range(n_size)] - return pd.DataFrame(df_dict) - - @pytest.fixture(scope="module") def ingest_time(): return datetime.utcnow() @@ -100,10 +138,10 @@ def ingest_time(): def cust_trans_df(ingest_time): return basic_dataframe(entities=["customer_id"], features=["daily_transactions", "total_transactions"], + null_features=["null_values"], ingest_time=ingest_time, n_size=5) - @pytest.fixture(scope="module") def driver_df(ingest_time): return basic_dataframe(entities=["driver_id"], @@ -111,7 +149,6 @@ def driver_df(ingest_time): ingest_time=ingest_time, n_size=5) - def test_version_returns_results(client): version_info = client.version() assert not version_info['core'] is 'not configured' @@ -186,9 +223,13 @@ def test_basic_ingest_success(client, cust_trans_df, driver_df): @pytest.mark.timeout(90) @pytest.mark.run(order=12) def test_basic_retrieve_online_success(client, cust_trans_df): + feature_refs=[ + "daily_transactions", + "total_transactions", + "null_values" + ] # Poll serving for feature values until the correct values are returned - while True: - time.sleep(1) + def try_get_features(): response = client.get_online_features( entity_rows=[ GetOnlineFeaturesRequest.EntityRow( @@ -199,45 +240,28 @@ def test_basic_retrieve_online_success(client, cust_trans_df): } ) ], - # Test retrieve with different variations of the string feature refs - feature_refs=[ - "daily_transactions", - "total_transactions", - ] - ) # type: GetOnlineFeaturesResponse - - if response is None: - continue - - returned_daily_transactions = float( - response.field_values[0] - .fields["daily_transactions"] - .float_val - ) - sent_daily_transactions = float( - cust_trans_df.iloc[0]["daily_transactions"]) - - if math.isclose( - sent_daily_transactions, - returned_daily_transactions, - abs_tol=FLOAT_TOLERANCE, - ): - break + feature_refs=feature_refs) + # type: GetOnlineFeaturesResponse + is_ok = all([check_online_response(ref, cust_trans_df, response) for ref in feature_refs]) + return response, is_ok + wait_retry_backoff(retry_fn=try_get_features, + timeout_secs=90, + timeout_msg="Timed out trying to get online feature values") @pytest.mark.timeout(90) @pytest.mark.run(order=13) def test_basic_retrieve_online_multiple_featureset(client, cust_trans_df, driver_df): + # Test retrieve with different variations of the string feature refs + # ie feature set inference for feature refs without specified feature set + feature_ref_df_mapping = [ + ("customer_transactions:daily_transactions", cust_trans_df), + ("driver:rating", driver_df), + ("total_transactions", cust_trans_df), + ] # Poll serving for feature values until the correct values are returned - while True: - time.sleep(1) - # Test retrieve with different variations of the string feature refs - # ie feature set inference for feature refs without specified feature set - feature_ref_df_mapping = [ - ("customer_transactions:daily_transactions", cust_trans_df), - ("driver:rating", driver_df), - ("total_transactions", cust_trans_df), - ] + def try_get_features(): + feature_refs = [mapping[0] for mapping in feature_ref_df_mapping] response = client.get_online_features( entity_rows=[ GetOnlineFeaturesRequest.EntityRow( @@ -251,35 +275,15 @@ def test_basic_retrieve_online_multiple_featureset(client, cust_trans_df, driver } ) ], - feature_refs=[mapping[0] for mapping in feature_ref_df_mapping], + feature_refs=feature_refs, ) # type: GetOnlineFeaturesResponse + is_ok = all([check_online_response(ref, df, response) + for ref, df in feature_ref_df_mapping]) + return response, is_ok - if response is None: - continue - - def check_response(ingest_df, response, feature_ref): - returned_value = float( - response.field_values[0] - .fields[feature_ref] - .float_val - ) - feature_ref_splits = feature_ref.split(":") - if len(feature_ref_splits) == 1: - feature_name = feature_ref - else: - _, feature_name = feature_ref_splits - - sent_value = float( - ingest_df.iloc[0][feature_name]) - - return math.isclose( - sent_value, - returned_value, - abs_tol=FLOAT_TOLERANCE, - ) - - if all([check_response(df, response, ref) for ref, df in feature_ref_df_mapping]): - break + wait_retry_backoff(retry_fn=try_get_features, + timeout_secs=90, + timeout_msg="Timed out trying to get online feature values") @pytest.mark.timeout(300) @@ -418,10 +422,23 @@ def test_all_types_ingest_success(client, all_types_dataframe): @pytest.mark.timeout(90) @pytest.mark.run(order=22) def test_all_types_retrieve_online_success(client, all_types_dataframe): - # Poll serving for feature values until the correct values are returned - while True: - time.sleep(1) - + # Poll serving for feature values until the correct values are returned_float_list + feature_refs = [ + "float_feature", + "int64_feature", + "int32_feature", + "double_feature", + "string_feature", + "bool_feature", + "bytes_feature", + "float_list_feature", + "int64_list_feature", + "int32_list_feature", + "string_list_feature", + "bytes_list_feature", + "double_list_feature", + ] + def try_get_features(): response = client.get_online_features( entity_rows=[ GetOnlineFeaturesRequest.EntityRow( @@ -429,39 +446,22 @@ def test_all_types_retrieve_online_success(client, all_types_dataframe): int64_val=all_types_dataframe.iloc[0]["user_id"])} ) ], - feature_refs=[ - "float_feature", - "int64_feature", - "int32_feature", - "string_feature", - "bytes_feature", - "bool_feature", - "double_feature", - "float_list_feature", - "int64_list_feature", - "int32_list_feature", - "string_list_feature", - "bytes_list_feature", - "double_list_feature", - ], + feature_refs=feature_refs, ) # type: GetOnlineFeaturesResponse + is_ok = check_online_response("float_feature", all_types_dataframe, response) + return response, is_ok - if response is None: - continue - - returned_float_list = ( - response.field_values[0] - .fields["float_list_feature"] - .float_list_val.val - ) - - sent_float_list = all_types_dataframe.iloc[0]["float_list_feature"] - - if math.isclose( - returned_float_list[0], sent_float_list[0], abs_tol=FLOAT_TOLERANCE - ): - break + response = wait_retry_backoff(retry_fn=try_get_features, + timeout_secs=90, + timeout_msg="Timed out trying to get online feature values") + # check returned values + returned_float_list = response.field_values[0].fields["float_list_feature"].float_list_val.val + sent_float_list = all_types_dataframe.iloc[0]["float_list_feature"] + assert math.isclose(returned_float_list[0], sent_float_list[0], abs_tol=FLOAT_TOLERANCE) + # check returned metadata + assert (response.field_values[0].statuses["float_list_feature"] + == GetOnlineFeaturesResponse.FieldStatus.PRESENT) @pytest.mark.timeout(300) @pytest.mark.run(order=29) @@ -544,9 +544,11 @@ def test_large_volume_ingest_success(client, large_volume_dataframe): @pytest.mark.run(order=32) def test_large_volume_retrieve_online_success(client, large_volume_dataframe): # Poll serving for feature values until the correct values are returned + feature_refs=[ + "daily_transactions_large", + "total_transactions_large", + ] while True: - time.sleep(1) - response = client.get_online_features( entity_rows=[ GetOnlineFeaturesRequest.EntityRow( @@ -558,30 +560,14 @@ def test_large_volume_retrieve_online_success(client, large_volume_dataframe): } ) ], - feature_refs=[ - "daily_transactions_large", - "total_transactions_large", - ], + feature_refs=feature_refs, ) # type: GetOnlineFeaturesResponse + is_ok = all([check_online_response(ref, large_volume_dataframe, response) for ref in feature_refs]) + return None, is_ok - if response is None: - continue - - returned_daily_transactions = float( - response.field_values[0] - .fields["daily_transactions_large"] - .float_val - ) - sent_daily_transactions = float( - large_volume_dataframe.iloc[0]["daily_transactions_large"]) - - if math.isclose( - sent_daily_transactions, - returned_daily_transactions, - abs_tol=FLOAT_TOLERANCE, - ): - break - + wait_retry_backoff(retry_fn=try_get_features, + timeout_secs=90, + timeout_msg="Timed out trying to get online feature values") @pytest.fixture(scope='module') def all_types_parquet_file(): diff --git a/tests/e2e/redis/basic/cust_trans_fs.yaml b/tests/e2e/redis/basic/cust_trans_fs.yaml index 14d46794a6d..941037670d3 100644 --- a/tests/e2e/redis/basic/cust_trans_fs.yaml +++ b/tests/e2e/redis/basic/cust_trans_fs.yaml @@ -9,4 +9,6 @@ spec: valueType: FLOAT - name: total_transactions valueType: FLOAT + - name: null_values + valueType: FLOAT maxAge: 3600s From 64e5bc430f50de4cfe17bef0e6ea16ee37416aa9 Mon Sep 17 00:00:00 2001 From: Willem Pienaar <6728866+woop@users.noreply.github.com> Date: Fri, 19 Jun 2020 16:37:05 +0800 Subject: [PATCH 073/133] Build Feast Jupyter image and clean up examples (#803) * Build base Jupyter image and clean up Docker Compose * Clean up Basic example * Automatically build new Feast Jupyter Docker image * Remove startup script from Feast Jupyter image * Clean up and fix bugs in Telco notebook * Re-order Dockerfile to optimize installation --- .prow/config.yaml | 8 + examples/basic/basic.ipynb | 673 ++++------ ... Prediction (with Feast and XGBoost).ipynb | 1108 +++++++++++++---- infra/docker-compose/basic.ipynb | 730 +++++++++++ infra/docker-compose/docker-compose.batch.yml | 1 - infra/docker-compose/docker-compose.yml | 10 +- infra/docker-compose/jupyter/startup.sh | 18 - infra/docker/jupyter/Dockerfile | 26 + 8 files changed, 1900 insertions(+), 674 deletions(-) create mode 100644 infra/docker-compose/basic.ipynb delete mode 100755 infra/docker-compose/jupyter/startup.sh create mode 100644 infra/docker/jupyter/Dockerfile diff --git a/.prow/config.yaml b/.prow/config.yaml index 1c92884c802..632cf591465 100644 --- a/.prow/config.yaml +++ b/.prow/config.yaml @@ -465,6 +465,11 @@ postsubmits: --file infra/docker/serving/Dockerfile \ --google-service-account-file /etc/gcloud/service-account.json + infra/scripts/publish-docker-image.sh \ + --repository gcr.io/kf-feast/feast-jupyter \ + --tag ${PULL_BASE_REF:1} \ + --file infra/docker/jupyter/Dockerfile \ + --google-service-account-file /etc/gcloud/service-account.json HIGHEST_SEMVER_TAG=$(git tag -l --sort -version:refname | head -n 1) echo "Only push to latest tag if tag is the highest semver version $HIGHEST_SEMVER_TAG" @@ -476,6 +481,9 @@ postsubmits: docker tag gcr.io/kf-feast/feast-serving:${PULL_BASE_REF:1} gcr.io/kf-feast/feast-serving:latest docker push gcr.io/kf-feast/feast-serving:latest + + docker tag gcr.io/kf-feast/feast-jupyter:${PULL_BASE_REF:1} gcr.io/kf-feast/feast-jupyter:latest + docker push gcr.io/kf-feast/feast-jupyter:latest fi fi diff --git a/examples/basic/basic.ipynb b/examples/basic/basic.ipynb index a56121328c5..3d6bb3cc2ab 100644 --- a/examples/basic/basic.ipynb +++ b/examples/basic/basic.ipynb @@ -11,7 +11,9 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "This is a minimal example of using Feast. In this example we will\n", + "This is a minimal example of using Feast. The point is to show users how to get data into Feast and how to retrieve features for online serving and model training.\n", + "\n", + "In this example we will\n", "1. Create a synthetic customer feature dataset\n", "2. Register a feature set to represent these features in Feast\n", "3. Ingest these features into Feast\n", @@ -60,55 +62,9 @@ }, { "cell_type": "code", - "execution_count": 2, + "execution_count": null, "metadata": {}, - "outputs": [ - { - "name": "stdout", - "output_type": "stream", - "text": [ - "Requirement already satisfied: feast in /home/zzy/.conda/envs/feast-ml/lib/python3.7/site-packages (0.5.0.post0)\n", - "Requirement already satisfied: google in /home/zzy/.conda/envs/feast-ml/lib/python3.7/site-packages (from feast) (2.0.3)\n", - "Requirement already satisfied: tabulate==0.8.* in /home/zzy/.conda/envs/feast-ml/lib/python3.7/site-packages (from feast) (0.8.7)\n", - "Requirement already satisfied: pandavro==1.5.* in /home/zzy/.conda/envs/feast-ml/lib/python3.7/site-packages (from feast) (1.5.1)\n", - "Requirement already satisfied: pandas==0.* in /home/zzy/.local/lib/python3.7/site-packages (from feast) (0.25.0)\n", - "Requirement already satisfied: google-cloud-core==1.0.* in /home/zzy/.conda/envs/feast-ml/lib/python3.7/site-packages (from feast) (1.0.3)\n", - "Requirement already satisfied: grpcio==1.* in /home/zzy/.conda/envs/feast-ml/lib/python3.7/site-packages (from feast) (1.29.0)\n", - "Requirement already satisfied: fastavro<0.23,>=0.22.11 in /home/zzy/.conda/envs/feast-ml/lib/python3.7/site-packages (from feast) (0.22.13)\n", - "Requirement already satisfied: PyYAML==5.1.* in /home/zzy/.conda/envs/feast-ml/lib/python3.7/site-packages (from feast) (5.1.2)\n", - "Requirement already satisfied: googleapis-common-protos==1.* in /home/zzy/.conda/envs/feast-ml/lib/python3.7/site-packages (from feast) (1.51.0)\n", - "Requirement already satisfied: tqdm==4.* in /home/zzy/.conda/envs/feast-ml/lib/python3.7/site-packages (from feast) (4.46.0)\n", - "Requirement already satisfied: numpy in /home/zzy/.local/lib/python3.7/site-packages (from feast) (1.17.4)\n", - "Requirement already satisfied: confluent-kafka in /home/zzy/.conda/envs/feast-ml/lib/python3.7/site-packages (from feast) (1.4.2)\n", - "Requirement already satisfied: google-cloud-bigquery-storage==0.7.* in /home/zzy/.conda/envs/feast-ml/lib/python3.7/site-packages (from feast) (0.7.0)\n", - "Requirement already satisfied: toml==0.10.* in /home/zzy/.conda/envs/feast-ml/lib/python3.7/site-packages (from feast) (0.10.1)\n", - "Requirement already satisfied: protobuf>=3.10 in /home/zzy/.conda/envs/feast-ml/lib/python3.7/site-packages (from feast) (3.12.1)\n", - "Requirement already satisfied: google-cloud-bigquery==1.18.* in /home/zzy/.conda/envs/feast-ml/lib/python3.7/site-packages (from feast) (1.18.1)\n", - "Requirement already satisfied: pyarrow>=0.15.1 in /home/zzy/.conda/envs/feast-ml/lib/python3.7/site-packages (from feast) (0.17.1)\n", - "Requirement already satisfied: google-api-core==1.14.* in /home/zzy/.conda/envs/feast-ml/lib/python3.7/site-packages (from feast) (1.14.3)\n", - "Requirement already satisfied: google-cloud-storage==1.20.* in /home/zzy/.conda/envs/feast-ml/lib/python3.7/site-packages (from feast) (1.20.0)\n", - "Requirement already satisfied: google-auth==1.6.* in /home/zzy/.conda/envs/feast-ml/lib/python3.7/site-packages (from feast) (1.6.3)\n", - "Requirement already satisfied: kafka-python==1.* in /home/zzy/.conda/envs/feast-ml/lib/python3.7/site-packages (from feast) (1.4.7)\n", - "Requirement already satisfied: Click==7.* in /home/zzy/.conda/envs/feast-ml/lib/python3.7/site-packages (from feast) (7.1.2)\n", - "Requirement already satisfied: beautifulsoup4 in /home/zzy/.conda/envs/feast-ml/lib/python3.7/site-packages (from google->feast) (4.9.1)\n", - "Requirement already satisfied: six>=1.9 in /home/zzy/.conda/envs/feast-ml/lib/python3.7/site-packages (from pandavro==1.5.*->feast) (1.14.0)\n", - "Requirement already satisfied: python-dateutil>=2.6.1 in /home/zzy/.local/lib/python3.7/site-packages (from pandas==0.*->feast) (2.8.0)\n", - "Requirement already satisfied: pytz>=2017.2 in /home/zzy/.local/lib/python3.7/site-packages (from pandas==0.*->feast) (2019.1)\n", - "Requirement already satisfied: setuptools in /home/zzy/.local/lib/python3.7/site-packages (from protobuf>=3.10->feast) (41.6.0)\n", - "Requirement already satisfied: google-resumable-media<0.5.0dev,>=0.3.1 in /home/zzy/.conda/envs/feast-ml/lib/python3.7/site-packages (from google-cloud-bigquery==1.18.*->feast) (0.4.1)\n", - "Requirement already satisfied: requests<3.0.0dev,>=2.18.0 in /home/zzy/.conda/envs/feast-ml/lib/python3.7/site-packages (from google-api-core==1.14.*->feast) (2.23.0)\n", - "Requirement already satisfied: rsa>=3.1.4 in /home/zzy/.conda/envs/feast-ml/lib/python3.7/site-packages (from google-auth==1.6.*->feast) (4.0)\n", - "Requirement already satisfied: cachetools>=2.0.0 in /home/zzy/.conda/envs/feast-ml/lib/python3.7/site-packages (from google-auth==1.6.*->feast) (4.1.0)\n", - "Requirement already satisfied: pyasn1-modules>=0.2.1 in /home/zzy/.conda/envs/feast-ml/lib/python3.7/site-packages (from google-auth==1.6.*->feast) (0.2.8)\n", - "Requirement already satisfied: soupsieve>1.2 in /home/zzy/.conda/envs/feast-ml/lib/python3.7/site-packages (from beautifulsoup4->google->feast) (2.0.1)\n", - "Requirement already satisfied: chardet<4,>=3.0.2 in /home/zzy/.conda/envs/feast-ml/lib/python3.7/site-packages (from requests<3.0.0dev,>=2.18.0->google-api-core==1.14.*->feast) (3.0.4)\n", - "Requirement already satisfied: idna<3,>=2.5 in /home/zzy/.conda/envs/feast-ml/lib/python3.7/site-packages (from requests<3.0.0dev,>=2.18.0->google-api-core==1.14.*->feast) (2.9)\n", - "Requirement already satisfied: urllib3!=1.25.0,!=1.25.1,<1.26,>=1.21.1 in /home/zzy/.conda/envs/feast-ml/lib/python3.7/site-packages (from requests<3.0.0dev,>=2.18.0->google-api-core==1.14.*->feast) (1.25.8)\n", - "Requirement already satisfied: certifi>=2017.4.17 in /home/zzy/.local/lib/python3.7/site-packages (from requests<3.0.0dev,>=2.18.0->google-api-core==1.14.*->feast) (2019.6.16)\n", - "Requirement already satisfied: pyasn1>=0.1.3 in /home/zzy/.conda/envs/feast-ml/lib/python3.7/site-packages (from rsa>=3.1.4->google-auth==1.6.*->feast) (0.4.8)\n" - ] - } - ], + "outputs": [], "source": [ "!pip install feast" ] @@ -122,7 +78,7 @@ }, { "cell_type": "code", - "execution_count": 3, + "execution_count": 2, "metadata": {}, "outputs": [], "source": [ @@ -149,7 +105,7 @@ }, { "cell_type": "code", - "execution_count": 4, + "execution_count": 3, "metadata": {}, "outputs": [], "source": [ @@ -172,165 +128,82 @@ }, { "cell_type": "code", - "execution_count": 5, + "execution_count": 4, "metadata": {}, "outputs": [], "source": [ "days = [datetime.utcnow().replace(hour=0, minute=0, second=0, microsecond=0).replace(tzinfo=utc) \\\n", - " - timedelta(day) for day in range(3)][::-1]\n", + " - timedelta(day) for day in range(10)][::-1]\n", "\n", "customers = [1001, 1002, 1003, 1004, 1005]" ] }, { "cell_type": "code", - "execution_count": 6, + "execution_count": 5, "metadata": {}, "outputs": [ { - "data": { - "text/html": [ - "

\n", - "\n", - "
\n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - "
datetimecustomer_iddaily_transactionstotal_transactions
02020-05-25 00:00:00+00:0010018.70480277
12020-05-25 00:00:00+00:0010027.16388731
22020-05-25 00:00:00+00:0010039.93597668
32020-05-25 00:00:00+00:0010041.10798078
42020-05-25 00:00:00+00:0010058.30738136
52020-05-26 00:00:00+00:0010012.41681114
62020-05-26 00:00:00+00:0010024.8177359
72020-05-26 00:00:00+00:0010034.40971495
82020-05-26 00:00:00+00:0010046.6173176
92020-05-26 00:00:00+00:0010051.03252586
\n", - "" - ], - "text/plain": [ - " datetime customer_id daily_transactions \\\n", - "0 2020-05-25 00:00:00+00:00 1001 8.704802 \n", - "1 2020-05-25 00:00:00+00:00 1002 7.163887 \n", - "2 2020-05-25 00:00:00+00:00 1003 9.935976 \n", - "3 2020-05-25 00:00:00+00:00 1004 1.107980 \n", - "4 2020-05-25 00:00:00+00:00 1005 8.307381 \n", - "5 2020-05-26 00:00:00+00:00 1001 2.416811 \n", - "6 2020-05-26 00:00:00+00:00 1002 4.817735 \n", - "7 2020-05-26 00:00:00+00:00 1003 4.409714 \n", - "8 2020-05-26 00:00:00+00:00 1004 6.617317 \n", - "9 2020-05-26 00:00:00+00:00 1005 1.032525 \n", - "\n", - " total_transactions \n", - "0 77 \n", - "1 31 \n", - "2 68 \n", - "3 78 \n", - "4 36 \n", - "5 14 \n", - "6 9 \n", - "7 95 \n", - "8 6 \n", - "9 86 " - ] - }, - "execution_count": 6, - "metadata": {}, - "output_type": "execute_result" + "name": "stdout", + "output_type": "stream", + "text": [ + " datetime customer_id daily_transactions \\\n", + "0 2020-06-09 00:00:00+00:00 1001 0.564751 \n", + "1 2020-06-09 00:00:00+00:00 1002 3.945566 \n", + "2 2020-06-09 00:00:00+00:00 1003 7.291928 \n", + "3 2020-06-09 00:00:00+00:00 1004 6.690477 \n", + "4 2020-06-09 00:00:00+00:00 1005 6.415899 \n", + "5 2020-06-10 00:00:00+00:00 1001 0.347294 \n", + "6 2020-06-10 00:00:00+00:00 1002 5.363853 \n", + "7 2020-06-10 00:00:00+00:00 1003 0.538129 \n", + "8 2020-06-10 00:00:00+00:00 1004 4.755425 \n", + "9 2020-06-10 00:00:00+00:00 1005 2.867527 \n", + "10 2020-06-11 00:00:00+00:00 1001 9.493098 \n", + "11 2020-06-11 00:00:00+00:00 1002 5.130665 \n", + "12 2020-06-11 00:00:00+00:00 1003 1.794191 \n", + "13 2020-06-11 00:00:00+00:00 1004 4.698504 \n", + "14 2020-06-11 00:00:00+00:00 1005 2.908603 \n", + "15 2020-06-12 00:00:00+00:00 1001 9.857894 \n", + "16 2020-06-12 00:00:00+00:00 1002 5.416553 \n", + "17 2020-06-12 00:00:00+00:00 1003 5.374058 \n", + "18 2020-06-12 00:00:00+00:00 1004 9.834441 \n", + "19 2020-06-12 00:00:00+00:00 1005 0.480373 \n", + "\n", + " total_transactions \n", + "0 73 \n", + "1 75 \n", + "2 95 \n", + "3 50 \n", + "4 65 \n", + "5 28 \n", + "6 76 \n", + "7 42 \n", + "8 53 \n", + "9 61 \n", + "10 86 \n", + "11 31 \n", + "12 69 \n", + "13 9 \n", + "14 51 \n", + "15 23 \n", + "16 1 \n", + "17 34 \n", + "18 13 \n", + "19 50 \n" + ] } ], "source": [ "customer_features = pd.DataFrame(\n", " {\n", - " \"datetime\": [day for day in days for customer in customers],\n", - " \"customer_id\": [customer for day in days for customer in customers],\n", - " \"daily_transactions\": [np.random.rand() * 10 for _ in range(len(days) * len(customers))],\n", - " \"total_transactions\": [np.random.randint(100) for _ in range(len(days) * len(customers))],\n", + " \"datetime\": [day for day in days for customer in customers], # Datetime is required\n", + " \"customer_id\": [customer for day in days for customer in customers], # Customer is the entity\n", + " \"daily_transactions\": [np.random.rand() * 10 for _ in range(len(days) * len(customers))], # Feature 1\n", + " \"total_transactions\": [np.random.randint(100) for _ in range(len(days) * len(customers))], # Feature 2\n", " }\n", ")\n", "\n", - "customer_features.head(10)" + "print(customer_features.head(20))" ] }, { @@ -350,14 +223,13 @@ }, { "cell_type": "code", - "execution_count": 7, + "execution_count": 6, "metadata": {}, "outputs": [], "source": [ "customer_fs = FeatureSet(\n", " \"customer_transactions\",\n", - " entities=[Entity(name='customer_id', dtype=ValueType.INT64)],\n", - " max_age=Duration(seconds=432000) \n", + " entities=[Entity(name='customer_id', dtype=ValueType.INT64)]\n", ")" ] }, @@ -370,7 +242,7 @@ }, { "cell_type": "code", - "execution_count": 8, + "execution_count": 7, "metadata": {}, "outputs": [ { @@ -398,7 +270,24 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "The apply() method will register the provided feature set with Feast core, allowing users to retrieve features from this feature set" + "The apply() method will register the provided feature set with Feast Core (the feature registry)." + ] + }, + { + "cell_type": "code", + "execution_count": 8, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Feature set updated: \"customer_transactions\"\n" + ] + } + ], + "source": [ + "client.apply(customer_fs)" ] }, { @@ -410,7 +299,6 @@ "name": "stdout", "output_type": "stream", "text": [ - "Feature set created: \"customer_transactions\"\n", "{\n", " \"spec\": {\n", " \"name\": \"customer_transactions\",\n", @@ -422,15 +310,15 @@ " ],\n", " \"features\": [\n", " {\n", - " \"name\": \"daily_transactions\",\n", - " \"valueType\": \"DOUBLE\"\n", - " },\n", - " {\n", " \"name\": \"total_transactions\",\n", " \"valueType\": \"INT64\"\n", + " },\n", + " {\n", + " \"name\": \"daily_transactions\",\n", + " \"valueType\": \"DOUBLE\"\n", " }\n", " ],\n", - " \"maxAge\": \"432000s\",\n", + " \"maxAge\": \"0s\",\n", " \"source\": {\n", " \"type\": \"KAFKA\",\n", " \"kafkaSourceConfig\": {\n", @@ -441,15 +329,14 @@ " \"project\": \"default\"\n", " },\n", " \"meta\": {\n", - " \"createdTimestamp\": \"2020-05-27T03:58:07Z\",\n", - " \"status\": \"STATUS_PENDING\"\n", + " \"createdTimestamp\": \"2020-06-18T12:04:08Z\",\n", + " \"status\": \"STATUS_READY\"\n", " }\n", "}\n" ] } ], "source": [ - "client.apply(customer_fs)\n", "customer_fs = client.get_feature_set(\"customer_transactions\")\n", "print(customer_fs)" ] @@ -461,11 +348,26 @@ "### 7. Ingest data into Feast for a feature set" ] }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Next we will ingest/load data into Feast. This process populates all registered stores (BigQuery, Redis) with your feature data." + ] + }, { "cell_type": "code", - "execution_count": 10, + "execution_count": 11, "metadata": {}, "outputs": [ + { + "name": "stderr", + "output_type": "stream", + "text": [ + "\r", + " 0%| | 0/50 [00:00\n", - "\n", - "\n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - "
datetimecustomer_id
02020-05-25 00:00:00+00:001001
12020-05-25 00:00:00+00:001002
22020-05-25 00:00:00+00:001003
32020-05-25 00:00:00+00:001004
42020-05-25 00:00:00+00:001005
52020-05-26 00:00:00+00:001001
62020-05-26 00:00:00+00:001002
72020-05-26 00:00:00+00:001003
82020-05-26 00:00:00+00:001004
92020-05-26 00:00:00+00:001005
\n", - "" - ], - "text/plain": [ - " datetime customer_id\n", - "0 2020-05-25 00:00:00+00:00 1001\n", - "1 2020-05-25 00:00:00+00:00 1002\n", - "2 2020-05-25 00:00:00+00:00 1003\n", - "3 2020-05-25 00:00:00+00:00 1004\n", - "4 2020-05-25 00:00:00+00:00 1005\n", - "5 2020-05-26 00:00:00+00:00 1001\n", - "6 2020-05-26 00:00:00+00:00 1002\n", - "7 2020-05-26 00:00:00+00:00 1003\n", - "8 2020-05-26 00:00:00+00:00 1004\n", - "9 2020-05-26 00:00:00+00:00 1005" - ] - }, - "execution_count": 19, - "metadata": {}, - "output_type": "execute_result" + "name": "stdout", + "output_type": "stream", + "text": [ + " datetime customer_id\n", + "0 2020-06-15 10:35:10.918716+00:00 1001\n", + "1 2020-06-15 14:00:10.918758+00:00 1002\n", + "2 2020-06-17 08:59:10.918767+00:00 1003\n", + "3 2020-06-13 16:51:10.918774+00:00 1004\n", + "4 2020-06-17 06:14:10.918780+00:00 1005\n", + "5 2020-06-17 14:33:10.918786+00:00 1001\n", + "6 2020-06-14 23:15:10.918792+00:00 1002\n", + "7 2020-06-15 11:25:10.918798+00:00 1003\n", + "8 2020-06-18 09:04:10.918804+00:00 1004\n", + "9 2020-06-16 10:27:10.918810+00:00 1005\n" + ] } ], "source": [ + "event_timestamps = [datetime.utcnow().replace(tzinfo=utc) - timedelta(days=randrange(5), hours=randrange(24), minutes=randrange(60)) for day in range(30)]\n", + "\n", "entity_rows = pd.DataFrame(\n", " {\n", - " \"datetime\": [day for day in days for customer in customers],\n", - " \"customer_id\": [customer for day in days for customer in customers],\n", + " \"datetime\": event_timestamps,\n", + " \"customer_id\": [customers[idx % len(customers)] for idx in range(len(event_timestamps))],\n", " }\n", ")\n", "\n", - "entity_rows.head(10)" + "print(entity_rows.head(10))" ] }, { @@ -741,12 +556,12 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "Next we will create a new client object, but this time we will configure it to connect to the Batch Serving Service. This service will allow us to retrieve historical feature data." + "Next we will create a new client object, but this time we will configure it to connect to the Feast Batch Serving. This service will allow us to retrieve historical feature data." ] }, { "cell_type": "code", - "execution_count": 20, + "execution_count": 15, "metadata": {}, "outputs": [], "source": [ @@ -757,12 +572,14 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "By calling the `get_batch_features` method we are able to retrieve a `job` object for the exporting of feature data. For every entity and timestamp combination in `entity_rows` we will be receiving a row with feature values joined to it." + "By calling the `get_batch_features` method we are able to retrieve a `job` object. This object can be used to retrieve the resulting training dataset that is exported by Feast. \n", + "\n", + "The dataset that is returned will contain feature values for each entity and timestamp combination in `entity_rows`." ] }, { "cell_type": "code", - "execution_count": 21, + "execution_count": 16, "metadata": { "scrolled": true }, @@ -786,108 +603,114 @@ }, { "cell_type": "code", - "execution_count": 22, + "execution_count": 17, + "metadata": {}, + "outputs": [], + "source": [ + "df = job.to_dataframe()" + ] + }, + { + "cell_type": "code", + "execution_count": 18, "metadata": {}, "outputs": [ { - "data": { - "text/html": [ - "

\n", - "\n", - "\n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - "
event_timestampcustomer_iddaily_transactionstotal_transactions
02020-05-26 00:00:00+00:0010012.41681114
12020-05-26 00:00:00+00:0010046.6173176
22020-05-26 00:00:00+00:0010034.40971495
32020-05-26 00:00:00+00:0010051.03252586
42020-05-26 00:00:00+00:0010024.8177359
\n", - "
" - ], - "text/plain": [ - " event_timestamp customer_id daily_transactions \\\n", - "0 2020-05-26 00:00:00+00:00 1001 2.416811 \n", - "1 2020-05-26 00:00:00+00:00 1004 6.617317 \n", - "2 2020-05-26 00:00:00+00:00 1003 4.409714 \n", - "3 2020-05-26 00:00:00+00:00 1005 1.032525 \n", - "4 2020-05-26 00:00:00+00:00 1002 4.817735 \n", - "\n", - " total_transactions \n", - "0 14 \n", - "1 6 \n", - "2 95 \n", - "3 86 \n", - "4 9 " - ] - }, - "execution_count": 22, - "metadata": {}, - "output_type": "execute_result" + "name": "stdout", + "output_type": "stream", + "text": [ + " event_timestamp customer_id daily_transactions \\\n", + "0 2020-06-13 23:45:10.918874+00:00 1001 1.879220 \n", + "1 2020-06-18 12:11:10.918845+00:00 1001 5.122846 \n", + "2 2020-06-17 20:46:10.918903+00:00 1001 2.145294 \n", + "3 2020-06-18 02:50:10.918816+00:00 1001 5.122846 \n", + "4 2020-06-15 10:35:10.918716+00:00 1001 5.758472 \n", + "5 2020-06-17 14:33:10.918786+00:00 1001 2.145294 \n", + "6 2020-06-14 23:15:10.918792+00:00 1002 5.467141 \n", + "7 2020-06-14 07:22:10.918851+00:00 1002 5.467141 \n", + "8 2020-06-17 23:40:10.918880+00:00 1002 3.338614 \n", + "9 2020-06-15 14:00:10.918758+00:00 1002 4.921264 \n", + "10 2020-06-15 18:22:10.918909+00:00 1002 4.921264 \n", + "11 2020-06-16 21:10:10.918822+00:00 1002 1.838186 \n", + "12 2020-06-18 05:47:10.918886+00:00 1003 2.702916 \n", + "13 2020-06-17 08:59:10.918767+00:00 1003 0.211125 \n", + "14 2020-06-15 11:25:10.918798+00:00 1003 4.476252 \n", + "15 2020-06-16 09:56:10.918857+00:00 1003 9.123597 \n", + "16 2020-06-14 11:39:10.918915+00:00 1003 6.353373 \n", + "17 2020-06-15 03:21:10.918828+00:00 1003 4.476252 \n", + "18 2020-06-18 09:04:10.918804+00:00 1004 8.756623 \n", + "19 2020-06-14 14:18:10.918834+00:00 1004 8.647374 \n", + "20 2020-06-17 03:10:10.918863+00:00 1004 2.377199 \n", + "21 2020-06-13 16:51:10.918774+00:00 1004 6.362085 \n", + "22 2020-06-15 03:54:10.918892+00:00 1004 8.235070 \n", + "23 2020-06-17 19:01:10.918921+00:00 1004 2.377199 \n", + "24 2020-06-17 06:14:10.918780+00:00 1005 9.958688 \n", + "25 2020-06-16 08:23:10.918839+00:00 1005 0.006388 \n", + "26 2020-06-16 00:30:10.918927+00:00 1005 0.006388 \n", + "27 2020-06-16 10:27:10.918810+00:00 1005 0.006388 \n", + "28 2020-06-17 01:50:10.918869+00:00 1005 9.958688 \n", + "29 2020-06-17 08:42:10.918897+00:00 1005 9.958688 \n", + "\n", + " total_transactions \n", + "0 7 \n", + "1 96 \n", + "2 63 \n", + "3 96 \n", + "4 85 \n", + "5 63 \n", + "6 10 \n", + "7 10 \n", + "8 50 \n", + "9 55 \n", + "10 55 \n", + "11 83 \n", + "12 50 \n", + "13 96 \n", + "14 61 \n", + "15 85 \n", + "16 69 \n", + "17 61 \n", + "18 84 \n", + "19 95 \n", + "20 25 \n", + "21 2 \n", + "22 58 \n", + "23 25 \n", + "24 6 \n", + "25 36 \n", + "26 36 \n", + "27 36 \n", + "28 6 \n", + "29 6 \n" + ] } ], "source": [ - "df = job.to_dataframe()\n", - "df.head()" + "print(df.head(50))" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "The training dataset is staged on Google Cloud Storage and can be accessed directly if it is too large to load into memory" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "job.get_avro_files()" ] } ], "metadata": { "kernelspec": { - "display_name": "feast-ml-py374", + "display_name": "Python 3", "language": "python", - "name": "feast-ml-py374" + "name": "python3" }, "language_info": { "codemirror_mode": { @@ -899,9 +722,9 @@ "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", - "version": "3.7.4" + "version": "3.7.6" } }, "nbformat": 4, - "nbformat_minor": 4 + "nbformat_minor": 2 } diff --git a/examples/feast-xgboost-churn-prediction-tutorial/Telecom Customer Churn Prediction (with Feast and XGBoost).ipynb b/examples/feast-xgboost-churn-prediction-tutorial/Telecom Customer Churn Prediction (with Feast and XGBoost).ipynb index c29c01efffb..8bc1f1edc12 100644 --- a/examples/feast-xgboost-churn-prediction-tutorial/Telecom Customer Churn Prediction (with Feast and XGBoost).ipynb +++ b/examples/feast-xgboost-churn-prediction-tutorial/Telecom Customer Churn Prediction (with Feast and XGBoost).ipynb @@ -11,12 +11,11 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "This tutorial will demonstrate the use of Feast in productionising a churn model. The tutorial is broken down into two sections\n", + "This tutorial will demonstrate the use of Feast in productionizing a churn model. The tutorial will walk through:\n", "\n", "1. Churn Modelling (without Feast): In this section we explore the data, refine it, train a model, and evaluate its performance.\n", "2. Churn Modelling (with Feast): In this section we introduce Feast for feature storage, management, as well as serving.\n", "\n", - "\n", "This tutorial is an extension of [this](https://www.kaggle.com/pavanraj159/telecom-customer-churn-prediction/comments#6.-Model-Performances) Kaggle notebook" ] }, @@ -29,9 +28,36 @@ "## 1. Churn Modelling (without Feast)" ] }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Install dependencies for tutorial" + ] + }, { "cell_type": "code", "execution_count": null, + "metadata": { + "pycharm": { + "name": "#%%\n" + } + }, + "outputs": [], + "source": [ + "!pip install sklearn python-slugify plotly statsmodels yellowbrick xgboost" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Import dependencies" + ] + }, + { + "cell_type": "code", + "execution_count": 2, "metadata": { "_cell_guid": "b1076dfc-b9ad-4769-8c92-a6c4dae69d19", "_kg_hide-input": false, @@ -39,27 +65,20 @@ }, "outputs": [], "source": [ - "import numpy as np\n", - "import pandas as pd\n", "import os\n", - "import matplotlib.pyplot as plt\n", - "from PIL import Image\n", - "%matplotlib inline\n", + "import numpy as np\n", "import pandas as pd\n", - "import seaborn as sns\n", - "import itertools\n", "import warnings\n", "warnings.filterwarnings(\"ignore\")\n", - "import io\n", "import plotly.offline as py\n", - "py.init_notebook_mode(connected=True)\n", "import plotly.graph_objs as go\n", "import plotly.tools as tls\n", - "import plotly.figure_factory as ff\n", - "import statsmodels, yellowbrick\n", - "import sklearn # Tested with 0.22.1\n", - "import imblearn\n", - "from slugify import slugify" + "from slugify import slugify\n", + "from sklearn.preprocessing import LabelEncoder\n", + "from sklearn.preprocessing import StandardScaler\n", + "from xgboost import XGBClassifier\n", + "from feast import Client, FeatureSet, Entity, ValueType\n", + "import datetime as dt" ] }, { @@ -69,9 +88,16 @@ "### 1.1 Data" ] }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Retrieve the raw telco dataset" + ] + }, { "cell_type": "code", - "execution_count": 2, + "execution_count": 3, "metadata": { "_cell_guid": "79c7e3d0-c299-4dcb-8224-4455121ee9b0", "_uuid": "d629ff2d2480ee46fbb7e2d37f6b5fab8052498a" @@ -279,7 +305,7 @@ "[5 rows x 21 columns]" ] }, - "execution_count": 2, + "execution_count": 3, "metadata": {}, "output_type": "execute_result" } @@ -300,7 +326,7 @@ }, { "cell_type": "code", - "execution_count": 3, + "execution_count": 4, "metadata": { "_uuid": "8b10c13086dff7182e399b849e31bc03df54a14e" }, @@ -350,7 +376,7 @@ "target_col = [\"Churn\"]\n", "cat_cols = telcom.nunique()[telcom.nunique() < 6].keys().tolist()\n", "cat_cols = [x for x in cat_cols if x not in target_col]\n", - "num_cols = [x for x in telcom.columns if x not in cat_cols + target_col + Id_col]\n" + "num_cols = [x for x in telcom.columns if x not in cat_cols + target_col + Id_col]" ] }, { @@ -364,7 +390,7 @@ }, { "cell_type": "code", - "execution_count": 4, + "execution_count": 5, "metadata": { "_uuid": "8921591320c5e336ec5a2e1efc5ed3cb0f9ec1b2" }, @@ -585,15 +611,12 @@ "[5 rows x 35 columns]" ] }, - "execution_count": 4, + "execution_count": 5, "metadata": {}, "output_type": "execute_result" } ], "source": [ - "from sklearn.preprocessing import LabelEncoder\n", - "from sklearn.preprocessing import StandardScaler\n", - "\n", "# Customer id col\n", "Id_col = ['customer_id']\n", "\n", @@ -647,7 +670,7 @@ }, { "cell_type": "code", - "execution_count": 5, + "execution_count": 6, "metadata": {}, "outputs": [ { @@ -689,7 +712,7 @@ " customer_id\n", " 7032\n", " 7032\n", - " 0835-DUUIQ\n", + " 0256-LTHVJ\n", " 1\n", " NaN\n", " NaN\n", @@ -1181,7 +1204,7 @@ ], "text/plain": [ " count unique top freq \\\n", - "customer_id 7032 7032 0835-DUUIQ 1 \n", + "customer_id 7032 7032 0256-LTHVJ 1 \n", "gender 7032 NaN NaN NaN \n", "seniorcitizen 7032 NaN NaN NaN \n", "partner 7032 NaN NaN NaN \n", @@ -1292,7 +1315,7 @@ "totalcharges -0.830249 -0.390815 0.666827 2.82426 " ] }, - "execution_count": 5, + "execution_count": 6, "metadata": {}, "output_type": "execute_result" } @@ -1312,11 +1335,93 @@ }, { "cell_type": "code", - "execution_count": 6, + "execution_count": 7, "metadata": { "_uuid": "b52cf9c7f402ed706e82221e3f8601fdeea9ab27" }, "outputs": [ + { + "data": { + "text/html": [ + " \n", + " " + ] + }, + "metadata": {}, + "output_type": "display_data" + }, { "data": { "application/vnd.plotly.v1+json": { @@ -1457,7 +1562,7 @@ -0.007514979909200033, -0.01632782307070617, -0.013092839264555001, - -0.0008067457759124324, + -8.067457759124324E-4, -0.008507162405232782, -0.0071243969867245535, -0.010105418366566195, @@ -1474,16 +1579,16 @@ -0.003603167413572989, -0.015973079031173835, 0.001631872518613598, - 0.0008437084888753327, + 8.437084888753327E-4, 0.01319936726545174, -0.0010503798619876774, - -0.0006494991207074467, + -6.494991207074467E-4, -0.010516394125351592, -0.004318975744551275, 0.01627881894278637, 0.005285371870295646, -0.013779327268354416, - 4.783950839776602e-05 + 4.783950839776602E-5 ], [ -0.001819390613419179, @@ -1540,7 +1645,7 @@ 0.1425612874681736, -0.0010430787434336079, 0.0012346095228208073, - -0.0002855204740384597, + -2.855204740384597E-4, -0.2802019157901561, 0.08306706395255747, 0.24733370647615796, @@ -1690,7 +1795,7 @@ 0.11139068731904943, 0.0869415675760231, 0.09045518641091457, - -0.00036426636786500763, + -3.6426636786500763E-4, -0.17407470231312427, -0.26736609150996915, -0.08408097138202993, @@ -1702,7 +1807,7 @@ 0.510100290145439 ], [ - -0.0008067457759124324, + -8.067457759124324E-4, 0.059513871482029225, 0.1535564364182745, 0.013899668260943368, @@ -2054,8 +2159,8 @@ 0.04275388869901973, -0.0014704079413737543, -0.013428683581908657, - -0.0004678565419111244, - -0.00079001062411148, + -4.678565419111244E-4, + -7.9001062411148E-4, 0.014665913237832425, 0.013786269825793156, -0.16136793538251534, @@ -2100,7 +2205,7 @@ [ 0.004744965758849955, -0.18251949495535458, - -0.0002855204740384597, + -2.855204740384597E-4, 0.13838288994798562, 0.1718171065699321, -0.33279949932167546, @@ -2195,7 +2300,7 @@ 0.05762874929825631, 0.06758968145792751, -0.10954646682258033, - 0.0001971241890499786, + 1.971241890499786E-4, -0.2512993032332009, -0.017196458507178478, 0.15389327309228798, @@ -2314,13 +2419,13 @@ 0.1826633671546002 ], [ - 0.0008437084888753327, + 8.437084888753327E-4, 0.17132216591713703, -0.08320661736633733, -0.1492739811934336, 0.0027471183312986857, -0.11229466175861408, - -0.00036426636786500763, + -3.6426636786500763E-4, -0.003308493511411254, -0.11480726996437085, 0.1447470086556032, @@ -2370,7 +2475,7 @@ -0.3059839224841771, 0.3196937439459745, 0.006208692442055045, - 0.0001971241890499786, + 1.971241890499786E-4, -0.007422540118618717, -0.28809669563791657, -0.28558254792863036, @@ -2422,7 +2527,7 @@ -0.5924430690900127 ], [ - -0.0006494991207074467, + -6.494991207074467E-4, 0.0018604411671567017, -0.048481275955609554, -0.0014594010172321779, @@ -2474,7 +2579,7 @@ -0.03215707580783382, 0.014777815951004133, 0.023690157563696343, - -0.0004678565419111244, + -4.678565419111244E-4, 0.005613761063888361, -0.006230482005525746, -0.05215631834144274, @@ -2510,7 +2615,7 @@ -0.08063023581160907, 0.009750281447949528, 0.07573985650934924, - -0.00079001062411148, + -7.9001062411148E-4, 0.01746632456448475, -0.020153136101165186, -0.19870886170688193, @@ -2638,7 +2743,7 @@ 0.6510648032262032 ], [ - 4.783950839776602e-05, + 4.783950839776602E-5, 0.10241060539532633, 0.31907236323857324, 0.0646532494217739, @@ -3510,20 +3615,20 @@ "
\n", " \n", " \n", - "
\n", + "
\n", "