Skip to content

chore: Fix integration test commands for contrib plugins + local integration tests. #3017

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
feast-ci-bot merged 2 commits into from
Aug 5, 2022
Merged

chore: Fix integration test commands for contrib plugins + local integration tests. #3017

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
e5e3298
chore: Fix integration test commands for contrib plugins. Fix local i…
adchia Aug 5, 2022
43467b2
title
adchia Aug 5, 2022
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
62 changes: 58 additions & 4 deletions CONTRIBUTING.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,6 +1,42 @@
# Development Guide: Main Feast Repository
<h1>Development Guide: Main Feast Repository</h1>

> Please see [Development Guide](https://docs.feast.dev/project/development-guide) for project level development instructions.

<h2>Table of Contents</h2>

- [Overview](#overview)
- [Community](#community)
- [Making a pull request](#making-a-pull-request)
- [Pull request checklist](#pull-request-checklist)
- [Forking the repo](#forking-the-repo)
- [Pre-commit Hooks](#pre-commit-hooks)
- [Signing off commits](#signing-off-commits)
- [Incorporating upstream changes from master](#incorporating-upstream-changes-from-master)
- [Feast Python SDK / CLI](#feast-python-sdk--cli)
- [Environment Setup](#environment-setup)
- [Code Style & Linting](#code-style--linting)
- [Unit Tests](#unit-tests)
- [Integration Tests](#integration-tests)
- [Local integration tests](#local-integration-tests)
- [(Advanced) Full integration tests](#advanced-full-integration-tests)
- [(Advanced) Running specific provider tests or running your test against specific online or offline stores](#advanced-running-specific-provider-tests-or-running-your-test-against-specific-online-or-offline-stores)
- [(Experimental) Run full integration tests against containerized services](#experimental-run-full-integration-tests-against-containerized-services)
- [Contrib integration tests](#contrib-integration-tests)
- [(Contrib) Running tests for Spark offline store](#contrib-running-tests-for-spark-offline-store)
- [(Contrib) Running tests for Trino offline store](#contrib-running-tests-for-trino-offline-store)
- [(Contrib) Running tests for Postgres offline store](#contrib-running-tests-for-postgres-offline-store)
- [(Contrib) Running tests for Postgres online store](#contrib-running-tests-for-postgres-online-store)
- [(Contrib) Running tests for HBase online store](#contrib-running-tests-for-hbase-online-store)
- [(Experimental) Feast UI](#experimental-feast-ui)
- [Feast Java Serving](#feast-java-serving)
- [Feast Go Client](#feast-go-client)
- [Environment Setup](#environment-setup-1)
- [Building](#building)
- [Code Style & Linting](#code-style--linting-1)
- [Unit Tests](#unit-tests-1)
- [Testing with Github Actions workflows](#testing-with-github-actions-workflows)
- [Issues](#issues)

## Overview
This guide is targeted at developers looking to contribute to Feast components in
the main Feast repository:
Expand Down Expand Up @@ -147,7 +183,7 @@ These tests create new temporary tables / datasets locally only, and they are cl
make test-python-integration-local
```

#### Full integration tests
#### (Advanced) Full integration tests
To test across clouds, on top of setting up Redis, you also need GCP / AWS / Snowflake setup.

> Note: you can manually control what tests are run today by inspecting
Expand Down Expand Up @@ -211,7 +247,7 @@ export SNOWFLAKE_CI_WAREHOUSE='[your warehouse]'

Then run `make test-python-integration`. Note that for Snowflake / GCP / AWS, this will create new temporary tables / datasets.

#### Running specific provider tests or running your test against specific online or offline stores
#### (Advanced) Running specific provider tests or running your test against specific online or offline stores

1. If you don't need to have your test run against all of the providers(`gcp`, `aws`, and `snowflake`) or don't need to run against all of the online stores, you can tag your test with specific providers or stores that you need(`@pytest.mark.universal_online_stores` or `@pytest.mark.universal_online_stores` with the `only` parameter). The `only` parameter selects specific offline providers and online stores that your test will test against. Example:

Expand Down Expand Up @@ -245,7 +281,25 @@ The services with containerized replacements currently implemented are:

You can run `make test-python-integration-container` to run tests against the containerized versions of dependencies.

## Feast UI
### Contrib integration tests
#### (Contrib) Running tests for Spark offline store
You can run `make test-python-universal-spark` to run all tests against the Spark offline store. (Note: you'll have to run `pip install -e ".[dev]"` first).

Not all tests are passing yet

#### (Contrib) Running tests for Trino offline store
You can run `make test-python-universal-trino` to run all tests against the Trino offline store. (Note: you'll have to run `pip install -e ".[dev]"` first)

#### (Contrib) Running tests for Postgres offline store
TODO

#### (Contrib) Running tests for Postgres online store
TODO

#### (Contrib) Running tests for HBase online store
TODO

## (Experimental) Feast UI
See [Feast contributing guide](ui/CONTRIBUTING.md)

## Feast Java Serving
Expand Down
44 changes: 35 additions & 9 deletions Makefile
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -79,13 +79,9 @@ test-python-integration-local:
FEAST_IS_LOCAL_TEST=True \
FEAST_LOCAL_ONLINE_CONTAINER=True \
python -m pytest -n 8 --integration \
-k "not test_apply_entity_integration and \
not test_apply_feature_view_integration and \
not test_apply_data_source_integration and \
not test_lambda_materialization and \
not test_feature_view_inference_success and \
not test_update_file_data_source_with_inferred_event_timestamp_col and \
not test_nullable_online_store" \
-k "not gcs_registry and \
not s3_registry and \
not test_lambda_materialization" \
sdk/python/tests \
) || echo "This script uses Docker, and it isn't running - please start the Docker Daemon and try again!";

Expand All @@ -97,9 +93,32 @@ test-python-integration-container:
python -m pytest -n 8 --integration sdk/python/tests \
) || echo "This script uses Docker, and it isn't running - please start the Docker Daemon and try again!";

test-python-universal-contrib:
test-python-universal-spark:
PYTHONPATH='.' \
FULL_REPO_CONFIGS_MODULE=sdk.python.feast.infra.offline_stores.contrib.contrib_repo_configuration \
FULL_REPO_CONFIGS_MODULE=sdk.python.feast.infra.offline_stores.contrib.spark_repo_configuration \
PYTEST_PLUGINS=feast.infra.offline_stores.contrib.spark_offline_store.tests \
FEAST_USAGE=False IS_TEST=True \
python -m pytest -n 8 --integration \
-k "not test_historical_retrieval_fails_on_validation and \
not test_historical_retrieval_with_validation and \
not test_historical_features_persisting and \
not test_historical_retrieval_fails_on_validation and \
not test_universal_cli and \
not test_go_feature_server and \
not test_feature_logging and \
not test_reorder_columns and \
not test_logged_features_validation and \
not test_lambda_materialization_consistency and \
not test_offline_write and \
not test_push_features_to_offline_store.py and \
not gcs_registry and \
not s3_registry and \
not test_universal_types" \
sdk/python/tests

test-python-universal-trino:
PYTHONPATH='.' \
FULL_REPO_CONFIGS_MODULE=sdk.python.feast.infra.offline_stores.contrib.trino_repo_configuration \
PYTEST_PLUGINS=feast.infra.offline_stores.contrib.trino_offline_store.tests \
FEAST_USAGE=False IS_TEST=True \
python -m pytest -n 8 --integration \
Expand All @@ -110,6 +129,13 @@ test-python-universal-contrib:
not test_universal_cli and \
not test_go_feature_server and \
not test_feature_logging and \
not test_reorder_columns and \
not test_logged_features_validation and \
not test_lambda_materialization_consistency and \
not test_offline_write and \
not test_push_features_to_offline_store.py and \
not gcs_registry and \
not s3_registry and \
not test_universal_types" \
sdk/python/tests

Expand Down
4 changes: 0 additions & 4 deletions ...res/contrib/contrib_repo_configuration.py → ...tores/contrib/spark_repo_configuration.py
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,17 +1,13 @@
from feast.infra.offline_stores.contrib.spark_offline_store.tests.data_source import (
SparkDataSourceCreator,
)
from feast.infra.offline_stores.contrib.trino_offline_store.tests.data_source import (
TrinoSourceCreator,
)
from tests.integration.feature_repos.repo_configuration import REDIS_CONFIG
from tests.integration.feature_repos.universal.online_store.redis import (
RedisOnlineStoreCreator,
)

AVAILABLE_OFFLINE_STORES = [
("local", SparkDataSourceCreator),
("local", TrinoSourceCreator),
]

AVAILABLE_ONLINE_STORES = {"redis": (REDIS_CONFIG, RedisOnlineStoreCreator)}
13 changes: 13 additions & 0 deletions sdk/python/feast/infra/offline_stores/contrib/trino_repo_configuration.py
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,13 @@
from feast.infra.offline_stores.contrib.trino_offline_store.tests.data_source import (
TrinoSourceCreator,
)
from tests.integration.feature_repos.repo_configuration import REDIS_CONFIG
from tests.integration.feature_repos.universal.online_store.redis import (
RedisOnlineStoreCreator,
)

AVAILABLE_OFFLINE_STORES = [
("local", TrinoSourceCreator),
]

AVAILABLE_ONLINE_STORES = {"redis": (REDIS_CONFIG, RedisOnlineStoreCreator)}
41 changes: 3 additions & 38 deletions sdk/python/tests/integration/registration/test_feature_store.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -30,11 +30,7 @@
from feast.infra.online_stores.sqlite import SqliteOnlineStoreConfig
from feast.repo_config import RepoConfig
from feast.types import Array, Bytes, Float64, Int64, String
from tests.utils.data_source_test_creator import (
prep_file_source,
simple_bq_source_using_query_arg,
simple_bq_source_using_table_arg,
)
from tests.utils.data_source_test_creator import prep_file_source


@pytest.mark.integration
Expand Down Expand Up @@ -96,50 +92,19 @@ def test_feature_view_inference_success(test_feature_store, dataframe_source):
tags={},
)

fv2 = FeatureView(
name="fv2",
entities=[entity],
ttl=timedelta(minutes=5),
online=True,
batch_source=simple_bq_source_using_table_arg(dataframe_source, "ts_1"),
tags={},
)

fv3 = FeatureView(
name="fv3",
entities=[entity],
ttl=timedelta(minutes=5),
online=True,
batch_source=simple_bq_source_using_query_arg(dataframe_source, "ts_1"),
tags={},
)

test_feature_store.apply([entity, fv1, fv2, fv3]) # Register Feature Views
test_feature_store.apply([entity, fv1]) # Register Feature Views
feature_view_1 = test_feature_store.list_feature_views()[0]
feature_view_2 = test_feature_store.list_feature_views()[1]
feature_view_3 = test_feature_store.list_feature_views()[2]

actual_file_source = {
(feature.name, feature.dtype) for feature in feature_view_1.features
}
actual_bq_using_table_arg_source = {
(feature.name, feature.dtype) for feature in feature_view_2.features
}
actual_bq_using_query_arg_source = {
(feature.name, feature.dtype) for feature in feature_view_3.features
}
expected = {
("float_col", Float64),
("int64_col", Int64),
("string_col", String),
}

assert (
expected
== actual_file_source
== actual_bq_using_table_arg_source
== actual_bq_using_query_arg_source
)
assert expected == actual_file_source

test_feature_store.teardown()

Expand Down
10 changes: 2 additions & 8 deletions sdk/python/tests/integration/registration/test_inference.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -5,11 +5,7 @@
from feast import RepoConfig
from feast.errors import RegistryInferenceFailure
from feast.inference import update_data_sources_with_inferred_event_timestamp_col
from tests.utils.data_source_test_creator import (
prep_file_source,
simple_bq_source_using_query_arg,
simple_bq_source_using_table_arg,
)
from tests.utils.data_source_test_creator import prep_file_source


@pytest.mark.integration
Expand All @@ -20,8 +16,6 @@ def test_update_file_data_source_with_inferred_event_timestamp_col(simple_datase
with prep_file_source(df=simple_dataset_1) as file_source:
data_sources = [
file_source,
simple_bq_source_using_table_arg(simple_dataset_1),
simple_bq_source_using_query_arg(simple_dataset_1),
]
update_data_sources_with_inferred_event_timestamp_col(
data_sources, RepoConfig(provider="local", project="test")
Expand All @@ -30,7 +24,7 @@ def test_update_file_data_source_with_inferred_event_timestamp_col(simple_datase
source.timestamp_field for source in data_sources
]

assert actual_event_timestamp_cols == ["ts_1", "ts_1", "ts_1"]
assert actual_event_timestamp_cols == ["ts_1"]

with prep_file_source(df=df_with_two_viable_timestamp_cols) as file_source:
with pytest.raises(RegistryInferenceFailure):
Expand Down
3 changes: 2 additions & 1 deletion sdk/python/tests/utils/e2e_test_validation.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -199,7 +199,8 @@ def make_feature_store_yaml(project, test_repo_config, repo_dir_name: Path):
),
]

if os.getenv("FEAST_IS_LOCAL_TEST", "False") == "True":
# Only test if this is NOT a local test
if os.getenv("FEAST_IS_LOCAL_TEST", "False") != "True":
NULLABLE_ONLINE_STORE_CONFIGS.extend(
[
IntegrationTestRepoConfig(
Expand Down