Skip to content
Merged
Show file tree
Hide file tree
Changes from 1 commit
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
docs + nits2
Signed-off-by: Miles Adkins <miles.adkins@snowflake.com>
  • Loading branch information
sfc-gh-madkins committed Jul 28, 2022
commit c38fd29b573cc2b35933683d4a59fbcf74f431c0
1 change: 1 addition & 0 deletions docs/SUMMARY.md
Original file line number Diff line number Diff line change
Expand Up @@ -79,6 +79,7 @@
* [PostgreSQL (contrib)](reference/offline-stores/postgres.md)
* [Online stores](reference/online-stores/README.md)
* [SQLite](reference/online-stores/sqlite.md)
* [Snowflake](reference/online-stores/snowflake.md)
* [Redis](reference/online-stores/redis.md)
* [Datastore](reference/online-stores/datastore.md)
* [DynamoDB](reference/online-stores/dynamodb.md)
Expand Down
25 changes: 25 additions & 0 deletions docs/reference/online-stores/snowflake.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,25 @@
# Snowflake

## Description

The [Snowflake](https://trial.snowflake.com) online store provides support for materializing feature values into an Snowflake Transient Table for serving online features.

* Only the latest feature values are persisted

## Example

{% code title="feature_store.yaml" %}
```yaml
project: my_feature_repo
registry: data/registry.db
provider: local
online_store:
type: snowflake.online
account: SNOWFLAKE_DEPLOYMENT_URL
user: SNOWFLAKE_USER
password: SNOWFLAKE_PASSWORD
role: SNOWFLAKE_ROLE
warehouse: SNOWFLAKE_WAREHOUSE
database: SNOWFLAKE_DATABASE
```
{% endcode %}
3 changes: 2 additions & 1 deletion docs/roadmap.md
Original file line number Diff line number Diff line change
Expand Up @@ -27,6 +27,7 @@ The list below contains the functionality that contributors are planning to deve
* [x] [In-memory / Pandas](https://docs.feast.dev/reference/offline-stores/file)
* [x] [Custom offline store support](https://docs.feast.dev/how-to-guides/adding-a-new-offline-store)
* **Online Stores**
* [x] [Snowflake](https://docs.feast.dev/reference/online-stores/snowflake)
* [x] [DynamoDB](https://docs.feast.dev/reference/online-stores/dynamodb)
* [x] [Redis](https://docs.feast.dev/reference/online-stores/redis)
* [x] [Datastore](https://docs.feast.dev/reference/online-stores/datastore)
Expand Down Expand Up @@ -59,4 +60,4 @@ The list below contains the functionality that contributors are planning to deve
* [x] Model-centric feature tracking (feature services)
* [x] Amundsen integration (see [Feast extractor](https://github.com/amundsen-io/amundsen/blob/main/databuilder/databuilder/extractor/feast_extractor.py))
* [x] DataHub integration (see [DataHub Feast docs](https://datahubproject.io/docs/generated/ingestion/sources/feast/))
* [x] Feast Web UI (Alpha release. See [docs](https://docs.feast.dev/reference/alpha-web-ui))
* [x] Feast Web UI (Alpha release. See [docs](https://docs.feast.dev/reference/alpha-web-ui))
2 changes: 1 addition & 1 deletion sdk/python/docs/source/conf.py
Original file line number Diff line number Diff line change
Expand Up @@ -115,7 +115,7 @@
# Add any paths that contain custom static files (such as style sheets) here,
# relative to this directory. They are copied after the builtin static files,
# so a file named "default.css" will overwrite the builtin "default.css".
html_static_path = []
html_static_path = ["_static"]


# -- Options for HTMLHelp output ------------------------------------------
Expand Down
8 changes: 8 additions & 0 deletions sdk/python/docs/source/feast.infra.online_stores.rst
Original file line number Diff line number Diff line change
Expand Up @@ -52,6 +52,14 @@ feast.infra.online\_stores.redis module
:undoc-members:
:show-inheritance:

feast.infra.online\_stores.snowflake module
-------------------------------------------

.. automodule:: feast.infra.online_stores.snowflake
:members:
:undoc-members:
:show-inheritance:

feast.infra.online\_stores.sqlite module
----------------------------------------

Expand Down
3 changes: 3 additions & 0 deletions sdk/python/docs/source/index.rst
Original file line number Diff line number Diff line change
Expand Up @@ -250,18 +250,21 @@ Sqlite Online Store

.. automodule:: feast.infra.online_stores.sqlite
:members:
:noindex:

Datastore Online Store
----------------------

.. automodule:: feast.infra.online_stores.datastore
:members:
:noindex:

DynamoDB Online Store
---------------------

.. automodule:: feast.infra.online_stores.dynamodb
:members:
:noindex:

Redis Online Store
------------------
Expand Down
8 changes: 5 additions & 3 deletions sdk/python/feast/infra/online_stores/snowflake.py
Original file line number Diff line number Diff line change
Expand Up @@ -21,7 +21,7 @@


class SnowflakeOnlineStoreConfig(FeastConfigBaseModel):
""" Online store config for Snowflake """
"""Online store config for Snowflake"""

type: Literal["snowflake.online"] = "snowflake.online"
""" Online store type selector"""
Expand Down Expand Up @@ -102,9 +102,11 @@ def online_write_batch(
if dfs:
agg_df = pd.concat(dfs)

#This combines both the data upload plus the overwrite in the same transaction
# This combines both the data upload plus the overwrite in the same transaction
with get_snowflake_conn(config.online_store, autocommit=False) as conn:
write_pandas_binary(conn, agg_df, f"[online-transient] {config.project}_{table.name}") #special function for writing binary to snowflake
write_pandas_binary(
conn, agg_df, f"[online-transient] {config.project}_{table.name}"
) # special function for writing binary to snowflake

query = f"""
INSERT OVERWRITE INTO "{config.online_store.database}"."{config.online_store.schema_}"."[online-transient] {config.project}_{table.name}"
Expand Down