Improve & Update Configuration Documentation by mrzzy · Pull Request #985 · feast-dev/feast

mrzzy · 2020-09-06T10:28:21Z

What this PR does / why we need it:
Improve and update configuration documentation where necessary to clarify their function:

Adds comments to application.yml files to better document configuration properties.
Update helm chart documentation with helm-docs.
Document that Feast means dataset name instead of dataset id in Historical Serving's feast.stores.config.dataset_id.
Users are directed from documentation to these comments to figure out what each configuration properly does.

Which issue(s) this PR fixes:

Fixes #995

Does this PR introduce a user-facing change?:

Adds comments to `application.yml` files to better document configuration properties.
Helm documentation updated with helm-docs.

feast-ci-bot · 2020-09-06T10:28:25Z

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by: mrzzy
To complete the pull request process, please assign davidheryanto
You can assign the PR to them by writing /assign @davidheryanto in a comment when ready.

The full list of commands accepted by this bot can be found here.

Details

Needs approval from an approver in each of these files:

OWNERS

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

mrzzy · 2020-09-10T03:00:22Z

/hold

including reference to community-provided Node.js SDK

* Fix push semver versioned ci image not being pushed if has semver suffix. * Add comments to document semver regex. * Replace regular expression matcher with grep -P for Perl regex style matching.

* Update version lint script to detect maven, docker image and stable version . * Allow users to lint versions against their merge branch of choice by branch TARGET_MERGE_BRANCH * Update development image tag version from 'dev' to 'develop' to make it easier to lint version. * Update lint version script to only echo file contents when version is wrong. * Update lint script to flag all failures instead of just one. * Remove file contents output from lint version script. * Update lint version script to make output more concise and readable. * Use develop images instead of stable images in master deployments. * Update outdated documentation in serving readme. * Fix outdated version in datatypes java README.md and add version lint check. * Add version lint checks to documentation. * Fix semver regex in github actions workflow not matching suffixless version tags. * Add version lint check for latest stable version in changelog. * Update lint-version script use tool agnostic variable names. * Fix typo.

mrzzy · 2020-09-13T04:06:21Z

/unhold

* Fix Go SDK outputing extra ":" in metadata header for authentication causing authentication failure. * Fix go test

* Fixes issue where lint-version sorts prerelease versions higher than actual stable version. * Refactor out version parsing from tags into get_tag_release common function. * Use get_tag_release in master docker push workflow.

* Update version references in docs to v0.7.0 * Bump master versions to 0.8-SNAPSHOT * Update 0.7 release changelog with hotfix * Delete duplicate entries in changelog

* Add entity data generator * Update core protos * Add entities as higher-level concept * Update python sdk * Add tests * Update to use scalar entity * Address PR comments * Update proto and tests * Remove redis pytest * Remove wildcard filters * Update golang generated files * Update tests * Rename constraints * Update list entities method * Update go protos

* Reorganise existing protos in CoreService by type. Signed-off-by: Terence <terencelimxp@gmail.com> * Add new FeatureTables API to Core Protobuf definitions Signed-off-by: Terence <terencelimxp@gmail.com> * Fix name collision in proto java outer classname with message name Signed-off-by: Terence <terencelimxp@gmail.com> * Add missing max age field to Feature Table proto. Signed-off-by: Terence <terencelimxp@gmail.com> * Add Flyway DB migration to add Feature Table API. Signed-off-by: Terence <terencelimxp@gmail.com> * Rename options field to options_json and change type to text. * Options to be stored as Protobuf JSON. * Change from varchar to text to remove char limit Signed-off-by: Terence <terencelimxp@gmail.com> * FeatureTable: Rename entity_names to entities Signed-off-by: Terence <terencelimxp@gmail.com> * Revert Reorganise existing protos in CoreService by type as it make it hard for reviewers to review changes Signed-off-by: Terence <terencelimxp@gmail.com> * Add FeatureSource entity for native representation of FeatureSource protobuf Signed-off-by: Terence <terencelimxp@gmail.com> * Add missing nullable annotation on FeatureSource entity. Signed-off-by: Terence <terencelimxp@gmail.com> * Update ListFeatureTablesRequest's Filter to follow naming convention. Signed-off-by: Terence <terencelimxp@gmail.com> * Add missing serialization code for FeatureSource's field mapping. Signed-off-by: Terence <terencelimxp@gmail.com> * Split Feature proto from FeatureTable proto. Signed-off-by: Terence <terencelimxp@gmail.com> * Update FeatureTable entity_names field to entities Signed-off-by: Terence <terencelimxp@gmail.com> * Revert putting project in feature table spec Signed-off-by: Terence <terencelimxp@gmail.com> * Update ListFeatureTable Proto to return full FeatureTable objects and limit to listing from one Project. Signed-off-by: Terence <terencelimxp@gmail.com> * Fix typo in CoreService proto Signed-off-by: Terence <terencelimxp@gmail.com> * Add FeatureV2 core model to store FeatureSpecV2 Signed-off-by: Terence <terencelimxp@gmail.com> * Add FeatureTable core model to store FeatureTable protos Signed-off-by: Terence <terencelimxp@gmail.com> * Fix naming grammar in CoreService proto Signed-off-by: Terence <terencelimxp@gmail.com> * Standardise naming of specifying projects in CoreService proto Signed-off-by: Terence <terencelimxp@gmail.com> * Rename FeatureSource proto to FeatureSourceSpec for compatiblity. Signed-off-by: Terence <terencelimxp@gmail.com> * Update FeatureSource model to store type specific options as seperate columns instead of JSON. Signed-off-by: Terence <terencelimxp@gmail.com> * Add FeatureTableTest unit test to test FeatureTable core model Signed-off-by: Terence <terencelimxp@gmail.com> * Add FeatureTableValidator to validate FeatureTableSpec protobufs Signed-off-by: Terence <terencelimxp@gmail.com> * Add listFeatureTables(), applyFeatureTable() & getFeatureTable() to Core's SpecService Signed-off-by: Terence <terencelimxp@gmail.com> * Add FeatureTableRepository to save & retrieve FeatureTables in database. Signed-off-by: Terence <terencelimxp@gmail.com> * Fix hibernate errors on Feast Core boot. Signed-off-by: Terence <terencelimxp@gmail.com> * Implement listFeatureTables() , applyFeatureTable(), and getFeatureTable() in CoreServiceImpl Signed-off-by: Terence <terencelimxp@gmail.com> * Add applyFeatureSet integration tests SpecServiceIT Signed-off-by: Terence <terencelimxp@gmail.com> * Various fixes for creating FeatureTabes with applyFeatureTable Signed-off-by: Terence <terencelimxp@gmail.com> * Fixed bug with updating FeatureTable Signed-off-by: Terence <terencelimxp@gmail.com> * Update ListFeatureTables Signed-off-by: Terence <terencelimxp@gmail.com> * Add Python SDK Signed-off-by: Terence <terencelimxp@gmail.com> * Update GetFeatureTable Signed-off-by: Terence <terencelimxp@gmail.com> * Remove unused proto imports and generate go protos Signed-off-by: Terence <terencelimxp@gmail.com> * Fix ListFeatureTables IT Signed-off-by: Terence <terencelimxp@gmail.com> * Update comment to generalize FeatureSource's field mapping to all fields instead of just for feature. Signed-off-by: Terence <terencelimxp@gmail.com> * Fix feature table validator condition Signed-off-by: Terence <terencelimxp@gmail.com> * Fix feature table unit tests Signed-off-by: Terence <terencelimxp@gmail.com> * Update feature source proto Signed-off-by: Terence <terencelimxp@gmail.com> * Address PR comments Signed-off-by: Terence <terencelimxp@gmail.com> * Replace test with IT Signed-off-by: Terence <terencelimxp@gmail.com> * Update IT config Signed-off-by: Terence <terencelimxp@gmail.com> * Fix removal of entity check Signed-off-by: Terence <terencelimxp@gmail.com> * Fix test sort issue Signed-off-by: Terence <terencelimxp@gmail.com> * Store source options as json Signed-off-by: Terence <terencelimxp@gmail.com> * Update go protos Signed-off-by: Terence <terencelimxp@gmail.com> * Remove go FeatureSource proto Signed-off-by: Terence <terencelimxp@gmail.com> * Increase IT max pool size Signed-off-by: Terence <terencelimxp@gmail.com> * Reduce pool size instead Signed-off-by: Terence <terencelimxp@gmail.com> * Replace mutablemapping with dict Signed-off-by: Terence <terencelimxp@gmail.com> * Standardize use of timestamp_column instead of ts_column Signed-off-by: Terence <terencelimxp@gmail.com> Co-authored-by: Terence Lim <terencelimxp@gmail.com>

Signed-off-by: Zhu Zhanyan <program.nom@gmail.com>

…used. Signed-off-by: Zhu Zhanyan <program.nom@gmail.com>

Signed-off-by: Zhu Zhanyan <program.nom@gmail.com>

…t being used." This reverts commit d82d072e41ad1fee4a88fa4560936fd8db19a71c. Signed-off-by: Zhu Zhanyan <program.nom@gmail.com>

Signed-off-by: Zhu Zhanyan <program.nom@gmail.com>

…name not dataset id Signed-off-by: Zhu Zhanyan <program.nom@gmail.com>

feast-ci-bot · 2020-10-06T03:11:08Z

@mrzzy: The following tests failed, say /retest to rerun them all:

Test name	Commit	Details	Rerun command
test-end-to-end-auth	`59e668d`	link	`/test test-end-to-end-auth`
test-end-to-end-redis-cluster	`59e668d`	link	`/test test-end-to-end-redis-cluster`
test-end-to-end	`59e668d`	link	`/test test-end-to-end`

Full PR test history

Details

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

mrzzy added area/documentation kind/techdebt labels Sep 6, 2020

mrzzy requested review from davidheryanto, khorshuheng, pyalex, woop and zhilingc as code owners September 6, 2020 10:28

feast-ci-bot added the size/L label Sep 6, 2020

mrzzy changed the title ~~Improve and update configuration documentation~~ Improve and Update Config Documentation Sep 6, 2020

mrzzy removed the kind/techdebt label Sep 6, 2020

feast-ci-bot added the needs-kind label Sep 6, 2020

GitBook: [master] 34 pages modified

bbcbb20

feast-ci-bot added the do-not-merge/hold label Sep 10, 2020

MichaelHirn and others added 4 commits September 10, 2020 13:43

Add community SDK section to API docs (#992)

e0dfb67

including reference to community-provided Node.js SDK

Fix Github Actions versioned image push (#994)

239e191

* Fix push semver versioned ci image not being pushed if has semver suffix. * Add comments to document semver regex. * Replace regular expression matcher with grep -P for Perl regex style matching.

Fix lint version not pulling tags. (#999)

4930978

mrzzy mentioned this pull request Sep 13, 2020

Helm chart values-batch-serving.yaml incorrectly asks for dataset_id when it actually means dataset name #995

Closed

mrzzy changed the title ~~Improve and Update Config Documentation~~ WIP: Improve and Update Config Documentation Sep 13, 2020

feast-ci-bot added the do-not-merge/work-in-progress label Sep 13, 2020

feast-ci-bot removed the do-not-merge/hold label Sep 13, 2020

mrzzy and others added 6 commits September 14, 2020 08:51

Fix Go SDK extra colon in metadata header for Authentication (#1001)

4809a96

* Fix Go SDK outputing extra ":" in metadata header for authentication causing authentication failure. * Fix go test

Add job-controller to test containers (#1006)

3b1e606

GitBook: [master] 34 pages modified

53e08ca

GitBook: [master] 36 pages modified

0738e6f

GitBook: [master] 2 pages and 4 assets modified

9ca4c58

GitBook: [master] 3 pages modified

3ec1cc2

terryyylim and others added 13 commits September 16, 2020 16:29

Probe for available port (#1004)

e35d215

GitBook: [master] 37 pages and 7 assets modified

535a0d5

Remove and update deprecation functions (#996)

b777904

Add validation for dataflow region (#1005)

4b48253

Fix py sdk historical retrieval conversion (#1002)

413c4f1

Include user identity in fluentd audit logs (#1011)

4a14a05

Update tests with bool list (#1012)

d887b37

Forward port lint version fixes: (#1008)

01d2661

* Fixes issue where lint-version sorts prerelease versions higher than actual stable version. * Refactor out version parsing from tags into get_tag_release common function. * Use get_tag_release in master docker push workflow.

Release v0.7.0 and Bump Versions (#1007)

0a322f0

* Update version references in docs to v0.7.0 * Bump master versions to 0.8-SNAPSHOT * Update 0.7 release changelog with hotfix * Delete duplicate entries in changelog

GitBook: [master] 37 pages modified

f5a5640

GitBook: [master] 37 pages modified

dbd8b71

mrzzy changed the title ~~WIP: Improve and Update Config Documentation~~ Improve and Update Config Documentation Oct 5, 2020

feast-ci-bot removed the do-not-merge/work-in-progress label Oct 5, 2020

mrzzy added 9 commits October 5, 2020 11:32

Add comments to document to core's application.yml

8f373d7

Signed-off-by: Zhu Zhanyan <program.nom@gmail.com>

Add comments to document job controller's application.yml

170a5e8

Signed-off-by: Zhu Zhanyan <program.nom@gmail.com>

Add documentation on serving's application.yml.

73dce3a

Signed-off-by: Zhu Zhanyan <program.nom@gmail.com>

Remove FeatureStreamOptions from Core's FeastProperties as not being …

6a69d75

…used. Signed-off-by: Zhu Zhanyan <program.nom@gmail.com>

Update helm documentaion with helm-docs

5d934d1

Signed-off-by: Zhu Zhanyan <program.nom@gmail.com>

Revert "Remove FeatureStreamOptions from Core's FeastProperties as no…

6bac954

…t being used." This reverts commit d82d072e41ad1fee4a88fa4560936fd8db19a71c. Signed-off-by: Zhu Zhanyan <program.nom@gmail.com>

Allow users to configure Feast Core's webserver port.

12becb0

Signed-off-by: Zhu Zhanyan <program.nom@gmail.com>

Fix misleading metrics docs in application.yml

828bcc6

Signed-off-by: Zhu Zhanyan <program.nom@gmail.com>

Document that BigQuery's store's dataset_id option refers to dataset …

59e668d

…name not dataset id Signed-off-by: Zhu Zhanyan <program.nom@gmail.com>

mrzzy changed the title ~~Improve and Update Config Documentation~~ Improve & Update Configuration Documentation Oct 6, 2020

mrzzy changed the base branch from master to v0.7-branch October 6, 2020 03:10

feast-ci-bot added size/XXL and removed size/L labels Oct 6, 2020

woop closed this Oct 27, 2020

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Improve & Update Configuration Documentation#985

Improve & Update Configuration Documentation#985
mrzzy wants to merge 36 commits into
feast-dev:v0.7-branchfrom
mrzzy:configuration-docs

mrzzy commented Sep 6, 2020 •

edited

Loading

Uh oh!

feast-ci-bot commented Sep 6, 2020

Uh oh!

mrzzy commented Sep 10, 2020

Uh oh!

mrzzy commented Sep 13, 2020

Uh oh!

feast-ci-bot commented Oct 6, 2020

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

6 participants

Conversation

mrzzy commented Sep 6, 2020 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

feast-ci-bot commented Sep 6, 2020

Uh oh!

mrzzy commented Sep 10, 2020

Uh oh!

mrzzy commented Sep 13, 2020

Uh oh!

feast-ci-bot commented Oct 6, 2020

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

6 participants

mrzzy commented Sep 6, 2020 •

edited

Loading