Skip to content

Replace Data Source specific format options with DataFormat message - #1049

Merged
feast-ci-bot merged 24 commits into
feast-dev:masterfrom
mrzzy:data-format-proto
Oct 19, 2020
Merged

Replace Data Source specific format options with DataFormat message#1049
feast-ci-bot merged 24 commits into
feast-dev:masterfrom
mrzzy:data-format-proto

Conversation

@mrzzy

@mrzzy mrzzy commented Oct 13, 2020

Copy link
Copy Markdown
Collaborator

What this PR does / why we need it:
Add data format messages to represent as data formats:

  • FileFormat will replace file_format in FileOptions
  • StreamFormat class_path` in Kafka and KinesisOptions.
  • This would allow us to support multiple data formats for each SourceType (ie AvroFormat and ProtoFormat for Kafka).
  • Added AvroFormat Protobuf message

Which issue(s) this PR fixes:

Fixes #

Does this PR introduce a user-facing change?:

Add `FileFormat` to replace `file_format` in FileOptions and `StreamFormat` to replace `class_path` in Kafka and KinesisOptions.

@mrzzy mrzzy added this to the v0.8.0 milestone Oct 13, 2020
@mrzzy mrzzy changed the title Protobuf: Replace Data Source specific format options with DataFormat message Replace Data Source specific format options with DataFormat message Oct 13, 2020
Comment thread core/src/main/java/feast/core/model/DataSource.java Outdated
Comment thread protos/feast/core/DataSource.proto Outdated
@mrzzy mrzzy changed the title Replace Data Source specific format options with DataFormat message WIP: Replace Data Source specific format options with DataFormat message Oct 14, 2020
@mrzzy mrzzy changed the title WIP: Replace Data Source specific format options with DataFormat message Replace Data Source specific format options with DataFormat message Oct 14, 2020
@mrzzy

mrzzy commented Oct 16, 2020

Copy link
Copy Markdown
Collaborator Author

/hold due to Protobuf API change

mrzzy added 20 commits October 18, 2020 17:32
Signed-off-by: Zhu Zhanyan <program.nom@gmail.com>
…ve objects.

Signed-off-by: Zhu Zhanyan <program.nom@gmail.com>
Signed-off-by: Zhu Zhanyan <program.nom@gmail.com>
…, DataFormat specs

Signed-off-by: Zhu Zhanyan <program.nom@gmail.com>
…ables

Signed-off-by: Zhu Zhanyan <program.nom@gmail.com>
Signed-off-by: Zhu Zhanyan <program.nom@gmail.com>
Signed-off-by: Zhu Zhanyan <program.nom@gmail.com>
Signed-off-by: Zhu Zhanyan <program.nom@gmail.com>
…e current API

Signed-off-by: Zhu Zhanyan <program.nom@gmail.com>
Signed-off-by: Zhu Zhanyan <program.nom@gmail.com>
Signed-off-by: Zhu Zhanyan <program.nom@gmail.com>
…ages

Signed-off-by: Zhu Zhanyan <program.nom@gmail.com>
Signed-off-by: Zhu Zhanyan <program.nom@gmail.com>
DataFormat.proto

Signed-off-by: Zhu Zhanyan <program.nom@gmail.com>
messages

Signed-off-by: Zhu Zhanyan <program.nom@gmail.com>
Signed-off-by: Zhu Zhanyan <program.nom@gmail.com>
…d by git

Signed-off-by: Zhu Zhanyan <program.nom@gmail.com>
Signed-off-by: Zhu Zhanyan <program.nom@gmail.com>
Signed-off-by: Zhu Zhanyan <program.nom@gmail.com>
… instead of expected str

Signed-off-by: Zhu Zhanyan <program.nom@gmail.com>
@feast-ci-bot

Copy link
Copy Markdown
Collaborator

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: mrzzy, pyalex

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Details Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@pyalex

pyalex commented Oct 19, 2020

Copy link
Copy Markdown
Collaborator

/lgtm

@pyalex

pyalex commented Oct 19, 2020

Copy link
Copy Markdown
Collaborator

/unhold

@feast-ci-bot
feast-ci-bot merged commit b9e6597 into feast-dev:master Oct 19, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants