Skip to content

Commit ff95d68

Browse files
woopgitbook-bot
woop
authored and
gitbook-bot
committed
GitBook: [master] 33 pages modified
1 parent 7db6ad6 commit ff95d68

5 files changed

Lines changed: 84 additions & 24 deletions

File tree

‎docs/SUMMARY.md‎

Lines changed: 3 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -35,7 +35,7 @@
3535
## Administration
3636

3737
* [Troubleshooting](administration/troubleshooting.md)
38-
* [Feast Upgrading](administration/upgrading.md)
38+
* [Upgrading Feast](administration/upgrading.md)
3939

4040
## Reference
4141

@@ -47,6 +47,7 @@
4747
* [Go SDK](https://godoc.org/github.com/feast-dev/feast/sdk/go)
4848
* [Java SDK](https://javadoc.io/doc/dev.feast/feast-sdk)
4949
* [Python SDK](https://api.docs.feast.dev/python/)
50+
* [Limitations](reference/limitations.md)
5051

5152
## Contributing
5253

@@ -55,4 +56,5 @@
5556
* [Style Guide](contributing/style-guide.md)
5657
* [Release Process](contributing/release-process.md)
5758
* [Adding a New Store](contributing/adding-a-new-store.md)
59+
* [Design Decisions](contributing/design-decisions.md)
5860

‎docs/administration/upgrading.md‎

Lines changed: 30 additions & 22 deletions
Original file line numberDiff line numberDiff line change
@@ -1,18 +1,16 @@
11
# Upgrading Feast
22

3+
## Migration 0.5 -> 0.6
34

4-
## Migration 0.5 -> 0.6
55
### Database schema
6-
In Release 0.6 we introduced [Flyway](https://flywaydb.org/) to handle schema migrations in PostgreSQL.
7-
Flyway is integrated into `core` and for now on all migrations will be run automatically on
8-
`core` start. It uses table `flyway_schema_history` in the same database (also created automatically) to keep track of already applied migrations.
9-
So no specific maintenance should be needed.
6+
7+
In Release 0.6 we introduced [Flyway](https://flywaydb.org/) to handle schema migrations in PostgreSQL. Flyway is integrated into `core` and for now on all migrations will be run automatically on `core` start. It uses table `flyway_schema_history` in the same database \(also created automatically\) to keep track of already applied migrations. So no specific maintenance should be needed.
108

119
If you already have existing deployment of feast 0.5 - Flyway will detect existing tables and omit first baseline migration.
1210

13-
After `core` started you should have `flyway_schema_history` look
14-
like this
15-
```
11+
After `core` started you should have `flyway_schema_history` look like this
12+
13+
```text
1614
>> select version, description, script, checksum from flyway_schema_history
1715
1816
version | description | script | checksum
@@ -24,34 +22,44 @@ version | description | scr
2422
In this release next major schema changes were done:
2523

2624
* Source is not shared between FeatureSets anymore. It's changed to 1:1 relation
27-
and source's primary key is now auto-incremented number.
25+
26+
and source's primary key is now auto-incremented number.
27+
2828
* Due to generalization of Source `sources.topics` & `sources.bootstrap_servers` columns were deprecated.
29-
They will be replaced with `sources.config`. Data migration handled by code when respected Source is used.
30-
`topics` and `bootstrap_servers` will be deleted in the next release.
31-
* Job (table `jobs`) is no longer connected to `Source` (table `sources`) since it uses consolidated source for optimization purposes.
32-
All data required by Job would be embedded in its table.
3329

34-
New Models (tables):
35-
* feature_statistics
30+
They will be replaced with `sources.config`. Data migration handled by code when respected Source is used.
31+
32+
`topics` and `bootstrap_servers` will be deleted in the next release.
33+
34+
* Job \(table `jobs`\) is no longer connected to `Source` \(table `sources`\) since it uses consolidated source for optimization purposes.
35+
36+
All data required by Job would be embedded in its table.
37+
38+
New Models \(tables\):
39+
40+
* feature\_statistics
3641

3742
Minor changes:
38-
* FeatureSet has new column version (see [proto](https://github.com/feast-dev/feast/blob/master/protos/feast/core/FeatureSet.proto) for details)
43+
44+
* FeatureSet has new column version \(see [proto](https://github.com/feast-dev/feast/blob/master/protos/feast/core/FeatureSet.proto) for details\)
3945
* Connecting table `jobs_feature_sets` in many-to-many relation between jobs & feature sets
40-
has now `version` and `delivery_status`.
4146

42-
## Migration 0.4 -> 0.6
47+
has now `version` and `delivery_status`.
48+
49+
## Migration 0.4 -> 0.6
4350

4451
### Database
45-
For all versions earlier than 0.5 seamless migration is not feasible due to earlier breaking changes and
46-
creation of new database will be required.
4752

48-
Since database will be empty - first (baseline) migration would be applied:
53+
For all versions earlier than 0.5 seamless migration is not feasible due to earlier breaking changes and creation of new database will be required.
54+
55+
Since database will be empty - first \(baseline\) migration would be applied:
4956

50-
```
57+
```text
5158
>> select version, description, script, checksum from flyway_schema_history
5259
5360
version | description | script | checksum
5461
--------+-----------------------------------------+-----------------------------------------+------------
5562
1 | Baseline | V1__Baseline.sql | 1091472110
5663
2 | RELEASE 0.6 Generalizing Source AND ... | V2__RELEASE_0.6_Generalizing_Source_... | 1537500232
5764
```
65+

‎docs/contributing/design-decisions.md‎

Lines changed: 8 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,8 @@
1+
# Design Decisions
2+
3+
## Job Management
4+
5+
* We currently support [optional consolidation](https://github.com/feast-dev/feast/pull/825/files#diff-11e36b0d5cfc6742aa51adf2812b664eR85) of jobs by source. This allows one job to populate many stores if all the store share a source. The upside is we need to provision less jobs. The downside is we need to managing the ingestion of data into stores independently becomes a lot more complicated. For the time being we will allow a flag to switch between both consolidation strategies, but we may remove consolidation in the future if it introduces too much complexity in the overall design.
6+
7+
8+

‎docs/reference/limitations.md‎

Lines changed: 43 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,43 @@
1+
# Limitations
2+
3+
## Feast API
4+
5+
<table>
6+
<thead>
7+
<tr>
8+
<th style="text-align:left">Limitation</th>
9+
<th style="text-align:left">Motivation</th>
10+
</tr>
11+
</thead>
12+
<tbody>
13+
<tr>
14+
<td style="text-align:left">Features names and entity names cannot overlap in feature set specifications</td>
15+
<td
16+
style="text-align:left">Features and entities become columns in historical stores which may cause
17+
conflicts</td>
18+
</tr>
19+
<tr>
20+
<td style="text-align:left">
21+
<p>The following field names are reserved in feature sets and FeatureRow
22+
messages</p>
23+
<ul>
24+
<li><code>event_timestamp</code>
25+
</li>
26+
<li><code>datetime</code>
27+
</li>
28+
<li><code>created_timestamp</code>
29+
</li>
30+
<li><code>ingestion_id</code>
31+
</li>
32+
<li><code>job_id</code>
33+
</li>
34+
</ul>
35+
</td>
36+
<td style="text-align:left">These key words are used for column names when persisting metadata in
37+
historical stores</td>
38+
</tr>
39+
</tbody>
40+
</table>
41+
42+
43+

‎docs/user-guide/feature-retrieval.md‎

Lines changed: 0 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -138,7 +138,6 @@ for feature_set in feature_sets:
138138
anomalies = tfdv.validate_statistics(statistics=stats, schema=schema)
139139
```
140140

141-
142141
## Online feature retrieval
143142

144143
Online feature retrieval works in much the same way as batch retrieval, with one important distinction: Online stores only maintain the current state of features. No historical data is served.

0 commit comments

Comments
 (0)