Skip to content

Commit f2bc411

Browse files
sfc-gh-madkinsdmillepyalexmichelle-rascati-spjudahrand
authored
Merge feast-snowflake plugin into main repo with documentation (feast-dev#2193)
* Add backticks to left_table_query_string (feast-dev#2250) Signed-off-by: david <davidmiller252@gmail.com> Signed-off-by: sfc-gh-madkins <miles.adkins@snowflake.com> * Delete entity key from Redis only when all attached feature views are gone (feast-dev#2240) * Delete entity from redis when the last attached feature view is deleted Signed-off-by: pyalex <moskalenko.alexey@gmail.com> * Delete entity key from Redis only when all attached feature views are gone Signed-off-by: pyalex <moskalenko.alexey@gmail.com> * make lint happy Signed-off-by: pyalex <moskalenko.alexey@gmail.com> * make lint happy Signed-off-by: pyalex <moskalenko.alexey@gmail.com> * one more try with mypy Signed-off-by: pyalex <moskalenko.alexey@gmail.com> Signed-off-by: sfc-gh-madkins <miles.adkins@snowflake.com> * historical_field_mappings2 merge for one sign off commit (feast-dev#2252) Signed-off-by: Michelle Rascati <michelle.rascati@sailpoint.com> Signed-off-by: sfc-gh-madkins <miles.adkins@snowflake.com> * Correct inconsistent dependency (feast-dev#2255) Signed-off-by: Judah Rand <17158624+judahrand@users.noreply.github.com> Signed-off-by: sfc-gh-madkins <miles.adkins@snowflake.com> * Add snowflake environment variables to allow testing on snowflake infra (feast-dev#2258) * add snowflake environment vars to test framework Signed-off-by: sfc-gh-madkins <miles.adkins@snowflake.com> * add snowflake environment vars to test framework Signed-off-by: sfc-gh-madkins <miles.adkins@snowflake.com> * Return `UNIX_TIMESTAMP` as Python `datetime` (feast-dev#2244) * Refactor `UNIX_TIMESTAMP` conversion Signed-off-by: Judah Rand <17158624+judahrand@users.noreply.github.com> * Return `UNIX_TIMESTAMP` types as `datetime` to user Signed-off-by: Judah Rand <17158624+judahrand@users.noreply.github.com> * Fix linting errors Signed-off-by: Judah Rand <17158624+judahrand@users.noreply.github.com> * Rename variable to something more sensible Signed-off-by: Judah Rand <17158624+judahrand@users.noreply.github.com> Signed-off-by: sfc-gh-madkins <miles.adkins@snowflake.com> * Feast plan clean up (feast-dev#2256) * Run validation and inference on views and entities during plan Signed-off-by: Felix Wang <wangfelix98@gmail.com> * Do not log objects that are unchanged Signed-off-by: Felix Wang <wangfelix98@gmail.com> * Rename Fco to FeastObject Signed-off-by: Felix Wang <wangfelix98@gmail.com> * Remove useless method Signed-off-by: Felix Wang <wangfelix98@gmail.com> * Lint Signed-off-by: Felix Wang <wangfelix98@gmail.com> * Always initialize registry during feature store initialization Signed-off-by: Felix Wang <wangfelix98@gmail.com> * Fix usage test Signed-off-by: Felix Wang <wangfelix98@gmail.com> * Remove print statements Signed-off-by: Felix Wang <wangfelix98@gmail.com> Signed-off-by: sfc-gh-madkins <miles.adkins@snowflake.com> * Squash commits Signed-off-by: Danny Chiao <danny@tecton.ai> Signed-off-by: sfc-gh-madkins <miles.adkins@snowflake.com> * Add error type and refactor query execution to have retries Signed-off-by: Danny Chiao <danny@tecton.ai> Signed-off-by: sfc-gh-madkins <miles.adkins@snowflake.com> * Handle more snowflake errors Signed-off-by: Danny Chiao <danny@tecton.ai> Signed-off-by: sfc-gh-madkins <miles.adkins@snowflake.com> * Fix lint errors Signed-off-by: Danny Chiao <danny@tecton.ai> Signed-off-by: sfc-gh-madkins <miles.adkins@snowflake.com> * Fix lint errors Signed-off-by: Danny Chiao <danny@tecton.ai> Signed-off-by: sfc-gh-madkins <miles.adkins@snowflake.com> * Fix lint errors Signed-off-by: Danny Chiao <danny@tecton.ai> Signed-off-by: sfc-gh-madkins <miles.adkins@snowflake.com> * Fix wrong import Signed-off-by: Danny Chiao <danny@tecton.ai> Signed-off-by: sfc-gh-madkins <miles.adkins@snowflake.com> * modify registry.db s3 object initialization to work in S3 subdirectory with Java Feast Server (feast-dev#2259) Signed-off-by: NalinGHub <nalinm01@gmail.com> Signed-off-by: sfc-gh-madkins <miles.adkins@snowflake.com> * clean up docs Signed-off-by: sfc-gh-madkins <miles.adkins@snowflake.com> * lint-python Signed-off-by: sfc-gh-madkins <miles.adkins@snowflake.com> * fixed historical test Signed-off-by: sfc-gh-madkins <miles.adkins@snowflake.com> * fixed historical test Signed-off-by: sfc-gh-madkins <miles.adkins@snowflake.com> Co-authored-by: David Miller <david@patagona.ca> Co-authored-by: Oleksii Moskalenko <moskalenko.alexey@gmail.com> Co-authored-by: Michelle Rascati <44408275+michelle-rascati-sp@users.noreply.github.com> Co-authored-by: Judah Rand <17158624+judahrand@users.noreply.github.com> Co-authored-by: Felix Wang <wangfelix98@gmail.com> Co-authored-by: Danny Chiao <danny@tecton.ai> Co-authored-by: Nalin Mehra <37969183+NalinGHub@users.noreply.github.com>
1 parent 2e4f3e5 commit f2bc411

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

45 files changed

+2005
-54
lines changed

‎README.md‎

Lines changed: 4 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -136,24 +136,24 @@ The list below contains the functionality that contributors are planning to deve
136136
* Want to speak to a Feast contributor? We are more than happy to jump on a call. Please schedule a time using [Calendly](https://calendly.com/d/x2ry-g5bb/meet-with-feast-team).
137137

138138
* **Data Sources**
139+
* [x] [Snowflake source](https://docs.feast.dev/reference/data-sources/snowflake)
139140
* [x] [Redshift source](https://docs.feast.dev/reference/data-sources/redshift)
140141
* [x] [BigQuery source](https://docs.feast.dev/reference/data-sources/bigquery)
141142
* [x] [Parquet file source](https://docs.feast.dev/reference/data-sources/file)
142143
* [x] [Synapse source (community plugin)](https://github.com/Azure/feast-azure)
143144
* [x] [Hive (community plugin)](https://github.com/baineng/feast-hive)
144145
* [x] [Postgres (community plugin)](https://github.com/nossrannug/feast-postgres)
145146
* [x] Kafka source (with [push support into the online store](reference/alpha-stream-ingestion.md))
146-
* [x] [Snowflake source (community plugin)](https://github.com/sfc-gh-madkins/feast-snowflake)
147147
* [ ] HTTP source
148148
* **Offline Stores**
149+
* [x] [Snowflake](https://docs.feast.dev/reference/offline-stores/snowflake)
149150
* [x] [Redshift](https://docs.feast.dev/reference/offline-stores/redshift)
150151
* [x] [BigQuery](https://docs.feast.dev/reference/offline-stores/bigquery)
151152
* [x] [Synapse (community plugin)](https://github.com/Azure/feast-azure)
152153
* [x] [Hive (community plugin)](https://github.com/baineng/feast-hive)
153154
* [x] [Postgres (community plugin)](https://github.com/nossrannug/feast-postgres)
154155
* [x] [In-memory / Pandas](https://docs.feast.dev/reference/offline-stores/file)
155156
* [x] [Custom offline store support](https://docs.feast.dev/how-to-guides/adding-a-new-offline-store)
156-
* [x] [Snowflake (community plugin)](https://github.com/sfc-gh-madkins/feast-snowflake)
157157
* [x] [Trino (communiuty plugin)](https://github.com/Shopify/feast-trino)
158158
* **Online Stores**
159159
* [x] [DynamoDB](https://docs.feast.dev/reference/online-stores/dynamodb)
@@ -208,7 +208,7 @@ The list below contains the functionality that contributors are planning to deve
208208
Please refer to the official documentation at [Documentation](https://docs.feast.dev/)
209209
* [Quickstart](https://docs.feast.dev/getting-started/quickstart)
210210
* [Tutorials](https://docs.feast.dev/tutorials/tutorials-overview)
211-
* [Running Feast with GCP/AWS](https://docs.feast.dev/how-to-guides/feast-gcp-aws)
211+
* [Running Feast with Snowflake/GCP/AWS](https://docs.feast.dev/how-to-guides/feast-snowflake-gcp-aws)
212212
* [Change Log](https://github.com/feast-dev/feast/blob/master/CHANGELOG.md)
213213
* [Slack (#Feast)](https://slack.feast.dev/)
214214

@@ -224,4 +224,4 @@ Thanks goes to these incredible people:
224224

225225
<a href="https://github.com/feast-dev/feast/graphs/contributors">
226226
<img src="https://contrib.rocks/image?repo=feast-dev/feast" />
227-
</a>
227+
</a>

‎docs/README.md‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -52,6 +52,6 @@ Explore the following resources to get started with Feast:
5252
* [Concepts](getting-started/concepts/) describes all important Feast API concepts
5353
* [Architecture](getting-started/architecture-and-components/) describes Feast's overall architecture.
5454
* [Tutorials](tutorials/tutorials-overview.md) shows full examples of using Feast in machine learning applications.
55-
* [Running Feast with GCP/AWS](how-to-guides/feast-gcp-aws/) provides a more in-depth guide to using Feast.
55+
* [Running Feast with Snowflake/GCP/AWS](how-to-guides/feast-snowflake-gcp-aws/) provides a more in-depth guide to using Feast.
5656
* [Reference](reference/feast-cli-commands.md) contains detailed API and design documents.
5757
* [Contributing](project/contributing.md) contains resources for anyone who wants to contribute to Feast.

‎docs/SUMMARY.md‎

Lines changed: 4 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -33,10 +33,11 @@
3333
* [Driver ranking](tutorials/driver-ranking-with-feast.md)
3434
* [Fraud detection on GCP](tutorials/fraud-detection.md)
3535
* [Real-time credit scoring on AWS](tutorials/real-time-credit-scoring-on-aws.md)
36+
* [Driver Stats using Snowflake](tutorials/driver-stats-using-snowflake.md)
3637

3738
## How-to Guides
3839

39-
* [Running Feast with GCP/AWS](how-to-guides/feast-gcp-aws/README.md)
40+
* [Running Feast with Snowflake/GCP/AWS](how-to-guides/feast-snowflake-gcp-aws/README.md)
4041
* [Install Feast](how-to-guides/feast-gcp-aws/install-feast.md)
4142
* [Create a feature repository](how-to-guides/feast-gcp-aws/create-a-feature-repository.md)
4243
* [Deploy a feature store](how-to-guides/feast-gcp-aws/deploy-a-feature-store.md)
@@ -54,10 +55,12 @@
5455

5556
* [Data sources](reference/data-sources/README.md)
5657
* [File](reference/data-sources/file.md)
58+
* [Snowflake](reference/data-sources/snowflake.md)
5759
* [BigQuery](reference/data-sources/bigquery.md)
5860
* [Redshift](reference/data-sources/redshift.md)
5961
* [Offline stores](reference/offline-stores/README.md)
6062
* [File](reference/offline-stores/file.md)
63+
* [Snowflake](reference/offline-stores/snowflake.md)
6164
* [BigQuery](reference/offline-stores/bigquery.md)
6265
* [Redshift](reference/offline-stores/redshift.md)
6366
* [Online stores](reference/online-stores/README.md)

‎docs/getting-started/third-party-integrations.md‎

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -13,26 +13,26 @@ Don't see your offline store or online store of choice here? Check out our guide
1313

1414
### **Data Sources**
1515

16+
* [x] [Snowflake source](https://docs.feast.dev/reference/data-sources/snowflake)
1617
* [x] [Redshift source](https://docs.feast.dev/reference/data-sources/redshift)
1718
* [x] [BigQuery source](https://docs.feast.dev/reference/data-sources/bigquery)
1819
* [x] [Parquet file source](https://docs.feast.dev/reference/data-sources/file)
1920
* [x] [Synapse source (community plugin)](https://github.com/Azure/feast-azure)
2021
* [x] [Hive (community plugin)](https://github.com/baineng/feast-hive)
2122
* [x] [Postgres (community plugin)](https://github.com/nossrannug/feast-postgres)
2223
* [x] Kafka source (with [push support into the online store](https://docs.feast.dev/reference/alpha-stream-ingestion))
23-
* [x] [Snowflake source (community plugin)](https://github.com/sfc-gh-madkins/feast-snowflake)
2424
* [ ] HTTP source
2525

2626
### Offline Stores
2727

28+
* [x] [Snowflake](https://docs.feast.dev/reference/offline-stores/snowflake)
2829
* [x] [Redshift](https://docs.feast.dev/reference/offline-stores/redshift)
2930
* [x] [BigQuery](https://docs.feast.dev/reference/offline-stores/bigquery)
3031
* [x] [Synapse (community plugin)](https://github.com/Azure/feast-azure)
3132
* [x] [Hive (community plugin)](https://github.com/baineng/feast-hive)
3233
* [x] [Postgres (community plugin)](https://github.com/nossrannug/feast-postgres)
3334
* [x] [In-memory / Pandas](https://docs.feast.dev/reference/offline-stores/file)
3435
* [x] [Custom offline store support](https://docs.feast.dev/how-to-guides/adding-a-new-offline-store)
35-
* [x] [Snowflake source (community plugin)](https://github.com/sfc-gh-madkins/feast-snowflake)
3636
* [x] [Trino (communiuty plugin)](https://github.com/Shopify/feast-trino)
3737

3838
### Online Stores

docs/how-to-guides/feast-gcp-aws/README.md renamed to docs/how-to-guides/feast-snowflake-gcp-aws/README.md

File renamed without changes.

docs/how-to-guides/feast-gcp-aws/build-a-training-dataset.md renamed to docs/how-to-guides/feast-snowflake-gcp-aws/build-a-training-dataset.md

File renamed without changes.

docs/how-to-guides/feast-gcp-aws/create-a-feature-repository.md renamed to docs/how-to-guides/feast-snowflake-gcp-aws/create-a-feature-repository.md

Lines changed: 16 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -13,6 +13,21 @@ Creating a new Feast repository in /<...>/tiny_pika.
1313
```
1414
{% endtab %}
1515

16+
{% tabs %}
17+
{% tab title="Snowflake template" %}
18+
```bash
19+
feast init -t snowflake
20+
Snowflake Deployment URL: ...
21+
Snowflake User Name: ...
22+
Snowflake Password: ...
23+
Snowflake Role Name: ...
24+
Snowflake Warehouse Name: ...
25+
Snowflake Database Name: ...
26+
27+
Creating a new Feast repository in /<...>/tiny_pika.
28+
```
29+
{% endtab %}
30+
1631
{% tab title="GCP template" %}
1732
```text
1833
feast init -t gcp
@@ -30,7 +45,7 @@ Redshift Database Name: ...
3045
Redshift User Name: ...
3146
Redshift S3 Staging Location (s3://*): ...
3247
Redshift IAM Role for S3 (arn:aws:iam::*:role/*): ...
33-
Should I upload example data to Redshift (overwriting 'feast_driver_hourly_stats' table)? (Y/n):
48+
Should I upload example data to Redshift (overwriting 'feast_driver_hourly_stats' table)? (Y/n):
3449
3550
Creating a new Feast repository in /<...>/tiny_pika.
3651
```
@@ -63,4 +78,3 @@ You can now use this feature repository for development. You can try the followi
6378
* Run `feast apply` to apply these definitions to Feast.
6479
* Edit the example feature definitions in `example.py` and run `feast apply` again to change feature definitions.
6580
* Initialize a git repository in the same directory and checking the feature repository into version control.
66-

docs/how-to-guides/feast-gcp-aws/deploy-a-feature-store.md renamed to docs/how-to-guides/feast-snowflake-gcp-aws/deploy-a-feature-store.md

File renamed without changes.

docs/how-to-guides/feast-gcp-aws/install-feast.md renamed to docs/how-to-guides/feast-snowflake-gcp-aws/install-feast.md

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -6,6 +6,12 @@ Install Feast using [pip](https://pip.pypa.io):
66
pip install feast
77
```
88

9+
Install Feast with Snowflake dependencies (required when using Snowflake):
10+
11+
```
12+
pip install 'feast[snowflake]'
13+
```
14+
915
Install Feast with GCP dependencies (required when using BigQuery or Firestore):
1016

1117
```

docs/how-to-guides/feast-gcp-aws/load-data-into-the-online-store.md renamed to docs/how-to-guides/feast-snowflake-gcp-aws/load-data-into-the-online-store.md

File renamed without changes.

0 commit comments

Comments
 (0)