You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
* Make test_sql_registry tests unit tests
Signed-off-by: Felix Wang <wangfelix98@gmail.com>
* Update testing docs
Signed-off-by: Felix Wang <wangfelix98@gmail.com>
* Fix
Signed-off-by: Felix Wang <wangfelix98@gmail.com>
Copy file name to clipboardExpand all lines: docs/how-to-guides/adding-or-reusing-tests.md
+15-13Lines changed: 15 additions & 13 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -6,11 +6,13 @@ This guide will go over:
6
6
7
7
1. how Feast tests are setup
8
8
2. how to extend the test suite to test new functionality
9
-
3. how to use the existing test suite to test a new custom offline / online store.
9
+
3. how to use the existing test suite to test a new custom offline / online store
10
10
11
11
## Test suite overview
12
12
13
-
Let's inspect the test setup in `sdk/python/tests/integration`:
13
+
Unit tests are contained in `sdk/python/tests/unit`.
14
+
Integration tests are contained in `sdk/python/tests/integration`.
15
+
Let's inspect the structure of `sdk/python/tests/integration`:
14
16
15
17
```bash
16
18
$ tree
@@ -51,38 +53,38 @@ $ tree
51
53
│ ├── test_s3_custom_endpoint.py
52
54
│ └── test_universal_historical_retrieval.py
53
55
├── online_store
54
-
│ ├── test_online_retrieval.py
55
56
│ ├── test_push_features_to_online_store.py
56
57
│ └── test_universal_online.py
57
58
└── registration
58
59
├── test_feature_store.py
59
60
├── test_inference.py
60
61
├── test_registry.py
61
-
├── test_sql_registry.py
62
62
├── test_universal_cli.py
63
63
├── test_universal_odfv_feature_inference.py
64
64
└── test_universal_types.py
65
65
66
66
```
67
67
68
68
*`feature_repos` has setup files for most tests in the test suite.
69
-
*`conftest.py` and some of the individual test files contain fixtures which can be used to on different offline stores, online stores, etc. and thus abstract away store specific implementations so we don't need to rewrite the same test implementation for different stores.
69
+
*`conftest.py` (in the parent directory) contains the most common [fixtures](https://docs.pytest.org/en/6.2.x/fixture.html), which are designed as an abstraction on top of specific offline/online stores, so tests do not need to be rewritten for different stores. Individual test files also contain more specific fixtures.
70
+
* The tests are organized by which Feast component(s) they test.
70
71
71
72
## Structure of the test suite
72
73
73
-
### What is the universal test suite?
74
+
### Universal feature repo
74
75
75
-
The universal test suite verifies that crucial Feast functions (e.g `get_historical_features`, `get_online_features` etc.) have the correct behavior for each of the different environments that Feast could be used in. These environments are combinations of an offline store, online store, and provider and the universal test suite serves to run basic functional verification against all of these different permutations.
76
+
The universal feature repo refers to a set of fixtures (e.g. `environment` and `universal_data_sources`) that can be parametrized to cover various combinations of offline stores, online stores, and providers.
77
+
This allows tests to run against all these various combinations without requiring excess code.
78
+
The universal feature repo is constructed by fixtures in `conftest.py` with help from the various files in `feature_repos`.
76
79
77
-
We use pytest [fixtures](https://docs.pytest.org/en/6.2.x/fixture.html) to accomplish this without writing excess code.
80
+
### Integration vs. unit tests
78
81
79
82
Tests in Feast are split into integration and unit tests.
83
+
If a test requires external resources (e.g. cloud resources on GCP or AWS), it is an integration test.
84
+
If a test can be run purely locally (where locally includes Docker resources), it is a unit test.
80
85
81
-
### Is it an integration or unit test?
82
-
83
-
* Integration tests test non local Feast behavior. Integration tests mainly involve testing of Feast components that connect to services outside of Feast(e.g connecting to gcp or aws clients).
84
-
* Generally if the test requires the initialization of a feature store in an external environment in order to test (i.e using our universal test fixtures), it is probably an integration test.
85
-
* Unit tests, on the other hand, unit tests primarily test local and class level behavior that does not require spinning up an external service. If your test can be run locally without using any other services besides pytest, it is a unit test.
86
+
* Integration tests test non-local Feast behavior. For example, tests that require reading data from BigQuery or materializing data to DynamoDB are integration tests. Integration tests also tend to involve more complex Feast functionality.
87
+
* Unit tests test local Feast behavior. For example, tests that only require registering feature views are unit tests. Unit tests tend to only involve simple Feast functionality.
0 commit comments