Skip to content

feat: Postgres supported as Registry, Online store, and Offline store #2401

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
feast-ci-bot merged 24 commits into from
Apr 20, 2022
Merged

feat: Postgres supported as Registry, Online store, and Offline store #2401

Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
24 commits
Select commit Hold shift + click to select a range
ed18f7e
Feast-postgres added to Feast repo
nossrannug Mar 13, 2022
bc8a630
Add a template for postgres to allow feast init -t postgres
nossrannug Mar 20, 2022
1a705e3
Split contrib repo configuration to separate file for each
nossrannug Mar 20, 2022
643f911
Moved the postgres type maps into feast.type_map
nossrannug Mar 22, 2022
15e71e7
Upgrade to latest feast, get postgres tests running, pip lock files
nossrannug Apr 19, 2022
953c3e7
Fix setup.py
kevjumba Apr 20, 2022
66c4699
Fix the ci requirements
kevjumba Apr 20, 2022
d0b5081
Fix repo_config for rest of sources
kevjumba Apr 20, 2022
c4de36e
Make backwards compatible with python 3.8-
kevjumba Apr 20, 2022
979e326
Fix
kevjumba Apr 20, 2022
199850e
Fix
kevjumba Apr 20, 2022
885d789
address comments
adchia Apr 20, 2022
ad947a9
update lock files
adchia Apr 20, 2022
1f246ad
fix
adchia Apr 20, 2022
f625910
lock files
adchia Apr 20, 2022
c6d93cd
fix
adchia Apr 20, 2022
e6c39ff
cleanup
adchia Apr 20, 2022
caad6c0
missed refactor name
adchia Apr 20, 2022
a0f9476
Remove snowflake references from test.py for postgres
nossrannug Apr 20, 2022
7a792a5
update repo to match new API
adchia Apr 20, 2022
8e1509d
Add documentation
adchia Apr 20, 2022
8e6ec1e
rename utils
adchia Apr 20, 2022
3522e23
update roadmap docs
adchia Apr 20, 2022
72e9e3e
update documentation
adchia Apr 20, 2022
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
Split contrib repo configuration to separate file for each 
Signed-off-by: Gunnar Sv Sigurbjörnsson <gunnar.sigurbjornsson@gmail.com>
  • Loading branch information
@nossrannug @adchia
nossrannug authored and adchia committed Apr 20, 2022
commit 1a705e362f0ca4027f0b8e2aa93ee4435b1d49d9
5 changes: 4 additions & 1 deletion Makefile
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -74,9 +74,12 @@ test-python-integration-container:
test-python-universal-contrib:
Comment thread
adchia marked this conversation as resolved.
Outdated
PYTHONPATH='.' FULL_REPO_CONFIGS_MODULE=sdk.python.feast.infra.offline_stores.contrib.contrib_repo_configuration FEAST_USAGE=False IS_TEST=True python -m pytest -n 8 --integration --universal sdk/python/tests

test-python-universal-spark:
PYTHONPATH='.' FULL_REPO_CONFIGS_MODULE=sdk.python.feast.infra.offline_stores.contrib.spark_repo_configuration FEAST_USAGE=False IS_TEST=True python -m pytest -n 8 --integration --universal sdk/python/tests

test-python-universal-postgres:
PYTHONPATH='.' \
FULL_REPO_CONFIGS_MODULE=sdk.python.feast.infra.offline_stores.contrib.contrib_repo_configuration \
FULL_REPO_CONFIGS_MODULE=sdk.python.feast.infra.offline_stores.contrib.postgres_repo_configuration \
FEAST_USAGE=False \
IS_TEST=True \
python -m pytest --integration --universal \
Expand Down
11 changes: 1 addition & 10 deletions ...res/contrib/contrib_repo_configuration.py → ...es/contrib/postgres_repo_configuration.py
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,15 +1,9 @@
from tests.integration.feature_repos.integration_test_repo_config import (
IntegrationTestRepoConfig,
)
from tests.integration.feature_repos.universal.data_sources.postgres_data_source import (
from tests.integration.feature_repos.universal.data_sources.postgres import (
PostgreSQLDataSourceCreator,
)
from tests.integration.feature_repos.universal.data_sources.spark_data_source_creator import (
SparkDataSourceCreator,
)
from tests.integration.feature_repos.universal.data_sources.trino import (
TrinoSourceCreator,
)

POSTGRES_ONLINE_CONFIG = {
Comment thread
adchia marked this conversation as resolved.
"type": "postgres",
Expand All @@ -21,10 +15,7 @@
"password": "docker",
}


FULL_REPO_CONFIGS = [
IntegrationTestRepoConfig(offline_store_creator=SparkDataSourceCreator),
IntegrationTestRepoConfig(offline_store_creator=TrinoSourceCreator),
IntegrationTestRepoConfig(
provider="local",
offline_store_creator=PostgreSQLDataSourceCreator,
Expand Down
10 changes: 10 additions & 0 deletions sdk/python/feast/infra/offline_stores/contrib/spark_repo_configuration.py
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,10 @@
from tests.integration.feature_repos.integration_test_repo_config import (
IntegrationTestRepoConfig,
)
from tests.integration.feature_repos.universal.data_sources.spark_data_source_creator import (
SparkDataSourceCreator,
)

FULL_REPO_CONFIGS = [
IntegrationTestRepoConfig(offline_store_creator=SparkDataSourceCreator),
]
Empty file added sdk/python/tests/integration/feature_repos/universal/data_sources/__init__.py
View file
Open in desktop
Empty file.
1 change: 0 additions & 1 deletion ...rsal/data_sources/postgres_data_source.py → ..._repos/universal/data_sources/postgres.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,6 @@


class PostgreSQLDataSourceCreator(DataSourceCreator):

tables: List[str] = []

def __init__(self, project_name: str):
Expand Down