You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: docs/concepts/feature-tables.md
+37-78Lines changed: 37 additions & 78 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -4,13 +4,20 @@
4
4
5
5
Feature tables are both a schema and a logical means of grouping features, data [sources](sources.md), and other related metadata.
6
6
7
-
Data typically comes in the form of flat files, dataframes, tables in a database, or events on a stream. Thus, the data is presented in multiple columns and/or fields in multiple rows and/or events.
7
+
Feature tables serve the following purposes:
8
8
9
-
Feature tables define the unique properties of these data [sources](sources.md), including how Feast interprets and sources them. Feature tables also enable Feast to[ ingest](../user-guide/data-ingestion.md) and [store](../advanced/stores.md) groups of fields from these data sources. Additionally, feature tables introduce efficient storage and logical namespacing to data within [stores](../advanced/stores.md).
9
+
* They are a means for defining the location and properties of data [sources](sources.md).
10
+
* They are used to create within Feast a database-level structure for the storage of feature values.
11
+
* The data sources described within feature tables enable Feast to ingest and store features within Feast.
12
+
* They ensure data is efficiently stored during [ingestion](../user-guide/data-ingestion.md).
13
+
14
+
{% hint style="info" %}
15
+
Feast does not yet apply feature transformations. Transformations are currently expected to happen before data is ingested into Feast. The data sources described within feature tables should reference feature values in their already computed form.
16
+
{% endhint %}
10
17
11
18
### Features
12
19
13
-
A feature is an individual measurable property or characteristic of an observable phenomenon. For example, in a bank a feature could be `total_foreign_transactions_24h` for a specific class of credit cards the bank issues. Feature data is the input both for training models, and for models served in production.
20
+
A feature is an individual measurable property or characteristic of an observable phenomenon. For example, in a bank, a feature could be `total_foreign_transactions_24h` for a specific class of credit cards the bank issues. Feature data is the input both for training models, and for models served in production.
14
21
15
22
{% hint style="info" %}
16
23
Features are the most important concepts within a feature store.
@@ -24,32 +31,26 @@ You define a feature by providing a name and value type. In our example, we use
Visit [FeatureSpec](https://api.docs.feast.dev/grpc/feast.core.pb.html#FeatureSpecV2) for the complete feature specification API.
34
+
Features act purely as a schema within feature tables. Feature tables and features act as normal database tables and columns.
28
35
29
-
{% hint style="warning" %}
30
-
Feature tables are **not** a source for retrieving features. Rather, you can retrieve feature values from feature tables.
31
-
{% endhint %}
36
+
Visit [FeatureSpec](https://api.docs.feast.dev/grpc/feast.core.pb.html#FeatureSpecV2) for the complete feature specification API.
32
37
33
38
## Structure of a Feature Table
34
39
35
-
When you register a feature table, at a minimum specify a batch source to populate the feature table.
36
-
37
-
If your use case includes a stream source, specify the stream source when you register the feature table. However, because a stream source may not be available in all use cases, Feast does not require specifying one.
40
+
Feature tables contain the following fields:
38
41
39
-
When you create a feature-table specification, include the following fields:
42
+
***Name:** Name of feature table. This name must be unique within a project.
43
+
***Entities:** List of [entities](entities.md) to associate with the features defined in this feature table. Entities are used as lookup keys when retrieving features from a feature table.
44
+
***Features:** List of features within this feature table.
45
+
***Labels:** Labels are arbitrary key-value properties that can be defined by users.
46
+
***Max age:** Max age affect the retrieval of features from a feature table. Age is measured as the duration of time between the event timestamp of a feature and the lookup time on an entity key used to retrieve the feature. Feature values outside max age will be returned as unset values. Max age allows for eviction of keys from online stores and limits the amount of scanning for historical feature values during retrieval.
47
+
***Batch Source:** The batch data source from which you can ingest feature values into Feast. Visit [Sources](sources.md) to learn more about them.
48
+
***Stream Source:** The streaming data source from which you can ingest streaming feature values into Feast. Visit [Sources](sources.md) to learn more about them.
40
49
41
-
***name:** Name of feature table. This name must be unique.
42
-
***entities:** List of names of entities to associate with the features defined in this feature table. Visit [Entities](entities.md) to learn more about them.
43
-
***features:** List of feature specifications for each feature defined in this feature table.
44
-
***labels:** User-defined metadata.
45
-
***max\_age:** Max age is measured as the duration of time between the feature's event timestamp and its retrieval. Feature values outside max age will be returned as unset values and indicated to you.
46
-
***batch\_source:** The batch or offline data source from which you create batch source feature data. Visit [Sources](sources.md) to learn more about them.
47
-
***stream\_source:** The stream or online data source from which you create stream or online feature data. See [Sources](sources.md) page for more details.
48
-
49
-
Here is an example of a valid feature-table specification. In our example, we again use a ride-hailing company:
50
+
Here is a ride-hailing example of a valid feature-table specification:
from feast import BigQuerySource, FeatureTable, Feature, ValueType
55
56
@@ -78,27 +79,20 @@ driver_ft = FeatureTable(
78
79
{% endtab %}
79
80
{% endtabs %}
80
81
81
-
By default, Feast assumes that features specified in the feature-table specification corresponds 1 - 1 to the fields found in the batch source.
82
+
When you register a feature table, at a minimum specify a batch source to populate the featuretable. Stream sources are optional. They are used to stream feature values into online stores.
82
83
83
-
However, if the names of the fields in the batch source are different from the names of features, you can use `field_mappings` to ensure the names correspond.
84
+
By default, Feast assumes that features specified in the feature-table specification corresponds one-to-one to the fields found in the sources. All features defined in a feature table should be available in the defined sources.
84
85
85
-
In the example feature-specification table above, we use `field_mappings` to ensure the field named `rating` in the batch source is mapped to the feature named `driver_rating`.
86
-
87
-
88
-
{% hint style="info" %}
89
-
When applying a feature table without specifying a project, Feast either creates or updates the feature table in the `default` project.
86
+
However, if the names of the fields in the batch source are different from the names of features, you can use `field_mappings` to ensure the names correspond.
90
87
91
-
To create a feature table for another project, specify the project of choice in the `apply_feature_table` call.
92
-
{% endhint %}
88
+
In the example feature-specification table above, we use `field_mappings` to ensure the field named `rating` in the batch source is mapped to the feature named `driver_rating`.
Feature table definitions may need to change over time to reflect more accurately your use case. In our ride-hailing example below, we update the max age:
108
102
109
103
```python
110
-
driver_ft = FeatureTable(..., max_age=7200, ...)
104
+
driver_ft = FeatureTable()
105
+
106
+
client.apply_feature_table(driver_ft)
107
+
108
+
driver_ft.labels = {"team": "marketplace"}
109
+
111
110
client.apply_feature_table(driver_ft)
112
111
```
113
112
114
-
Feast currently supports the following:
113
+
Feast currently supports the following changes to feature tables:
115
114
116
115
* Adding new features.
117
116
* Deleting existing features
118
117
* Changing the feature table's source, max age, and labels.
119
118
120
119
{% hint style="warning" %}
121
-
Deleted features are archived, rather than removed completely. Importantly, new features **cannot** use the names of these deleted features.
120
+
Deleted features are archived, rather than removed completely. Importantly, new features cannot use the names of these deleted features.
122
121
{% endhint %}
123
122
124
-
Feast currently does **not** support the following:
123
+
Feast currently does not support the following changes to feature tables:
125
124
126
125
* Changes to the project or name of a feature table.
127
126
* Changes to entities related to a feature table.
@@ -130,46 +129,6 @@ Feast currently does **not** support the following:
130
129
#### Deleting a Feature Table
131
130
132
131
{% hint style="danger" %}
133
-
Feast currently does not support deleting a feature table.
0 commit comments