Introducing Flyway as DB schema migration manager - #809

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by: pyalex
To complete the pull request process, please assign davidheryanto
You can assign the PR to them by writing /assign @davidheryanto in a comment when ready.

The full list of commands accepted by this bot can be found here.

/retest

/hold

@pyalex We probably need a new page underneath Administration (link) for Upgrading Feast. We can just keep it simple for now, but what I was thinking is just having a release specific process (steps to upgrade from 0.4 to 0.5), and a section for common tasks like applying a migration. This PR should probably introduce the latter.

@woop flyway is integrated with spring-boot and all migrations will be happen automatically with start of core. There's no need for any manual steps. However such page in documentation still maybe useful to explain what was changed and what issues may occur

Any comment on why this was closed, change of plans?

Superseded by #810?

flyway is integrated with spring-boot and all migrations will be happen automatically with start of core. There's no need for any manual steps.

Some organizations will have database operations managed by dedicated DBA teams, production DDL needs to run through formal change management processes, etc. (Speaking from experience here). Flyway can still help prepare scripts for such procedures, and Feast docs don't need to duplicate things that Flyway docs cover about how this might work, but I absolutely agree that Feast docs should cover what will happen, ideally with some acknowledgement that implicit auto will not be usable by everyone and what they might do instead.

This ought to cover initial deployment of Feast in the future too IMO, where environments again may not allow automatic creation of schema by Hibernate or Flyway (these kinds of things are all disabled by Hibernate's defaults, Feast config sets Spring's hibernate.ddl-auto: update—which is something that might need to be reviewed given Flyway should own management of schema now).

@ches I believe the idea was that the backport wasn't necessary since Spring ignores the baseline migration if a schema already exists, so folks coming from 0.5 would just get the diffs in 0.6.

You have a valid point though, in that not all organizations have the freedom to use implicit auto. This seems to only be a problem if you are baselined prior to 0.6.

What would be an acceptable minimum here? Should the migrations start from an earlier version? Should we document a means of upgrading versions (without migrations, perhaps import/export of schemas)?