feat: Feast Spark Offline Store - #2349
Merged
Merged
Conversation
kevjumba
force-pushed
the
feast_offline_store
branch
from
March 1, 2022 02:01
cd74282 to
4eeef08
Compare
kevjumba
commented
Mar 1, 2022
adchia
reviewed
Mar 1, 2022
kevjumba
force-pushed
the
feast_offline_store
branch
from
March 2, 2022 22:19
733d6bb to
0e76808
Compare
|
Codecov Report❌ Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## master #2349 +/- ##
==========================================
- Coverage 58.41% 58.27% -0.15%
==========================================
Files 117 118 +1
Lines 9764 9885 +121
==========================================
+ Hits 5704 5760 +56
- Misses 4060 4125 +65
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Sentry. 🚀 New features to boost your workflow:
|
kevjumba
force-pushed
the
feast_offline_store
branch
from
March 3, 2022 00:28
3279f1f to
49c7dbf
Compare
pyalex
reviewed
Mar 3, 2022
adchia
reviewed
Mar 3, 2022
kevjumba
commented
Mar 3, 2022
adchia
reviewed
Mar 3, 2022
kevjumba
force-pushed
the
feast_offline_store
branch
from
March 3, 2022 23:24
cbbc57f to
12eb130
Compare
Signed-off-by: Kevin Zhang <kzhang@tecton.ai>
Signed-off-by: Kevin Zhang <kzhang@tecton.ai>
Signed-off-by: Kevin Zhang <kzhang@tecton.ai>
kevjumba
force-pushed
the
feast_offline_store
branch
from
March 4, 2022 20:05
9c6f6fa to
b43417e
Compare
Signed-off-by: Kevin Zhang <kzhang@tecton.ai>
Collaborator
|
[APPROVALNOTIFIER] This PR is APPROVED This pull-request has been approved by: adchia, kevjumba The full list of commands accepted by this bot can be found here. The pull request process is described here DetailsNeeds approval from an approver in each of these files:
Approvers can indicate their approval by writing |
woop
pushed a commit
that referenced
this pull request
Mar 5, 2022
# [0.19.0](v0.18.0...v0.19.0) (2022-03-05) ### Bug Fixes * Added additional value types to UI parser and removed references to registry-bq.json ([#2361](#2361)) ([d202d51](d202d51)) * Fix Redshift bug that stops waiting on statements after 5 minutes ([#2363](#2363)) ([74f887f](74f887f)) * Method _should_use_plan only returns true for local sqlite provider ([#2344](#2344)) ([fdb5f21](fdb5f21)) * Remove redis service to prevent more conflicts and add redis node to master_only ([#2354](#2354)) ([993616f](993616f)) * Rollback Redis-py to Redis-py-cluster ([#2347](#2347)) ([1ba86fb](1ba86fb)) * Update github workflow to prevent redis from overlapping ports. ([#2350](#2350)) ([c2a6c6c](c2a6c6c)) ### Features * Add owner field to Entity and rename labels to tags ([412d625](412d625)) * Allow all snowflake python connector connection methods to be available to Feast ([#2356](#2356)) ([ec7385c](ec7385c)) * Allowing password based authentication and SSL for Redis in Java feature server ([0af8adb](0af8adb)) * Event timestamps response ([#2355](#2355)) ([5481caf](5481caf)) * Feast Spark Offline Store ([#2349](#2349)) ([98b8d8d](98b8d8d)) * Initial merge of Web UI logic ([#2352](#2352)) ([ce3bc59](ce3bc59)) * Key ttl setting for redis online store ([#2341](#2341)) ([236a108](236a108)) * Metadata changes & making data sources top level objects to power Feast UI ([#2336](#2336)) ([43da230](43da230))
Member
|
🎉 This PR is included in version 0.19.0 🎉 The release is available on GitHub release Your semantic-release bot 📦🚀 |
rehevkor5
reviewed
May 13, 2024
| SELECT {field_string}, | ||
| ROW_NUMBER() OVER({partition_by_join_key_string} ORDER BY {timestamp_desc_string}) AS feast_row_ | ||
| FROM {from_expression} t1 | ||
| WHERE {event_timestamp_column} BETWEEN TIMESTAMP('{start_date_str}') AND TIMESTAMP('{end_date_str}') |
There was a problem hiding this comment.
I'm not sure if it was intentional that both the start and the end are inclusive, but IMO the starting point should be inclusive and the ending point should be exclusive. That way, if you have ranges of tuples like (a, b) and (b, c), they are contiguous and non-overlapping. By using BETWEEN the ranges will instead overlap at b.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
7 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What this PR does / why we need it:
Add spark offline store plugin from Adyen Spark to spark offline store. https://github.com/Adyen/feast-spark-offline-store
Currently 159/194 integration tests for spark source work.
Which issue(s) this PR fixes:
Fixes #
Does this PR introduce a user-facing change?: