Skip to content

feat: Add an experimental lambda-based materialization engine #2923

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
feast-ci-bot merged 10 commits into from
Jul 14, 2022
Merged

feat: Add an experimental lambda-based materialization engine #2923

Changes from 1 commit
Commits
Show all changes
10 commits
Select commit Hold shift + click to select a range
d790a74
feat: Add an experimental lambda-based materialization engine
achals Jul 8, 2022
36d0f4b
setup and teardown lambda func
achals Jul 8, 2022
88b625b
actually get the test working correctly
achals Jul 12, 2022
f2927e9
actually get the test working correctly
achals Jul 12, 2022
20f8a54
parallelize with threads
achals Jul 12, 2022
23e67b4
super call
achals Jul 12, 2022
b12e89d
fix bugs
achals Jul 12, 2022
2523540
fix tests
achals Jul 13, 2022
0f93f30
fix tests
achals Jul 13, 2022
37aae59
undo unintended changes
achals Jul 13, 2022
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
fix tests 
Signed-off-by: Achal Shah <achals@gmail.com>
  • Loading branch information
@achals
achals committed Jul 14, 2022
commit 0f93f306c9dac3a82656564695f070db49d2cfbd
4 changes: 3 additions & 1 deletion sdk/python/feast/infra/materialization/lambda/lambda_engine.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -113,7 +113,7 @@ def teardown_infra(
entities: Sequence[Entity],
):
# This should be tearing down the lambda function.
logger.info("Tearing down lambda %s: %s", self.lambda_name)
logger.info("Tearing down lambda %s", self.lambda_name)
r = self.lambda_client.delete_function(FunctionName=self.lambda_name)
logger.info("Finished tearing down lambda %s: %s", self.lambda_name, r)

Expand All @@ -139,6 +139,8 @@ def __init__(
)

self.lambda_name = f"feast-materialize-{self.repo_config.project}"
if len(self.lambda_name) > 64:
self.lambda_name = self.lambda_name[:64]
self.lambda_client = boto3.client("lambda")

def materialize(
Expand Down