Skip to content

feat: Add interfaces for batch materialization engine #2901

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
feast-ci-bot merged 11 commits into from
Jul 6, 2022
Merged

feat: Add interfaces for batch materialization engine #2901

Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
11 commits
Select commit Hold shift + click to select a range
6cc7e75
feat: Add scaffolding for batch materialization engine
achals Jul 1, 2022
1656436
fix tests
achals Jul 5, 2022
c16734b
fix tests
achals Jul 5, 2022
ceedbab
a little better
achals Jul 5, 2022
37fa081
a little better
achals Jul 5, 2022
795e65a
docs
achals Jul 5, 2022
5833556
more api updates'
achals Jul 5, 2022
5d1af33
fix typos
achals Jul 6, 2022
ff680a9
make engine importable
achals Jul 6, 2022
11b6da0
style stuff
achals Jul 6, 2022
2e9e7a8
style stuff
achals Jul 6, 2022
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
fix tests 
Signed-off-by: Achal Shah <achals@gmail.com>
  • Loading branch information
@achals
achals committed Jul 5, 2022
commit 165643675ae866a700e88d57d7d811a7f35c0d81
4 changes: 2 additions & 2 deletions sdk/python/feast/infra/materialization/__init__.py
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,9 +1,9 @@
from batch_materialization_engine import (
from .batch_materialization_engine import (
BatchMaterializationEngine,
MaterializationJob,
MaterializationTask,
)
from local_engine import LocalMaterializationEngine
from .local_engine import LocalMaterializationEngine

__all__ = [
Comment thread
felixwang9817 marked this conversation as resolved.
"MaterializationJob",
Expand Down
2 changes: 0 additions & 2 deletions sdk/python/feast/infra/materialization/batch_materialization_engine.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -46,13 +46,11 @@ def __init__(
self,
*,
repo_config: RepoConfig,
registry: BaseRegistry,
offline_store: OfflineStore,
online_store: OnlineStore,
**kwargs,
):
self.repo_config = repo_config
self.registry = registry
self.offline_store = offline_store
self.online_store = online_store

Expand Down
5 changes: 3 additions & 2 deletions sdk/python/feast/infra/materialization/local_engine.py
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,14 +1,14 @@
from datetime import datetime
from typing import Callable, Dict, List, Literal, Optional, Tuple, Union

import dask as dd
import dask.dataframe as dd
import pandas as pd
import pyarrow as pa
from tqdm import tqdm

from feast import Entity, FeatureView, RepoConfig, ValueType
from feast.feature_view import DUMMY_ENTITY_ID
from feast.infra.materialization import (
from .batch_materialization_engine import (
BatchMaterializationEngine,
MaterializationJob,
MaterializationTask,
Expand All @@ -19,6 +19,7 @@
from feast.protos.feast.types.Value_pb2 import Value as ValueProto
from feast.repo_config import FeastConfigBaseModel
from feast.type_map import python_values_to_proto_values
from ...registry import BaseRegistry


class LocalMaterializationJob(MaterializationJob):
Expand Down