Skip to content

fix: Fix broken proto conversion methods for data sources #2603

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
feast-ci-bot merged 15 commits into from
Apr 24, 2022
Merged

fix: Fix broken proto conversion methods for data sources #2603

Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
15 commits
Select commit Hold shift + click to select a range
da33eb4
Fix Snowflake proto conversion and add test
felixwang9817 Apr 21, 2022
c68f53f
Add proto conversion test for FileSource
felixwang9817 Apr 23, 2022
a2fd043
Fix Redshift proto conversion and add test
felixwang9817 Apr 23, 2022
99413ca
Add proto conversion test for BigQuerySource
felixwang9817 Apr 23, 2022
3587993
Fix tests to use DataSource.from_proto
felixwang9817 Apr 23, 2022
d0175da
Add proto conversion test for KafkaSource
felixwang9817 Apr 23, 2022
56384fa
Add proto conversion test for KinesisSource
felixwang9817 Apr 23, 2022
74734a5
Add proto conversion test for PushSource
felixwang9817 Apr 23, 2022
2d45843
Add proto conversion test for PushSource
felixwang9817 Apr 23, 2022
7abf502
Add name and other fixes
felixwang9817 Apr 23, 2022
f4f6252
Fix proto conversion tests
felixwang9817 Apr 23, 2022
f1caec3
Add tags to test
felixwang9817 Apr 23, 2022
696b10a
Fix BigQuerySource bug
felixwang9817 Apr 24, 2022
4bffc5c
Fix bug in RedshiftSource and TrinoSource
felixwang9817 Apr 24, 2022
0a0cc58
Remove references to event_timestamp_column
felixwang9817 Apr 24, 2022
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
Add proto conversion test for KafkaSource 
Signed-off-by: Felix Wang <wangfelix98@gmail.com>
  • Loading branch information
@felixwang9817
felixwang9817 committed Apr 23, 2022
commit d0175dafafae1751a8602e8d0c4a9db2f8e5f440
2 changes: 1 addition & 1 deletion sdk/python/feast/data_source.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -409,7 +409,7 @@ def __init__(

if _message_format is None:
raise ValueError("Message format must be specified for Kafka source")
print("Asdfasdf")

super().__init__(
event_timestamp_column=_event_timestamp_column,
created_timestamp_column=created_timestamp_column,
Expand Down
15 changes: 15 additions & 0 deletions sdk/python/tests/unit/test_data_sources.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -201,3 +201,18 @@ def test_proto_conversion():
assert DataSource.from_proto(file_source.to_proto()) == file_source
assert DataSource.from_proto(redshift_source.to_proto()) == redshift_source
assert DataSource.from_proto(snowflake_source.to_proto()) == snowflake_source

kafka_source = KafkaSource(
name="test_source",
bootstrap_servers="test_servers",
message_format=ProtoFormat("class_path"),
topic="test_topic",
timestamp_field="event_timestamp",
created_timestamp_column="created_timestamp",
field_mapping={"foo": "bar"},
description="test description",
owner="test@gmail.com",
batch_source=file_source,
)

assert DataSource.from_proto(kafka_source.to_proto()) == kafka_source