You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: docs/advanced/stores.md
+1-1Lines changed: 1 addition & 1 deletion
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -29,7 +29,7 @@ Data is persisted in historical stores like BigQuery in log format. Repeated ing
29
29
30
30
Online stores maintain only the latest values for a specific feature. Feast currently supports Redis as an online store. Online stores are meant for very high throughput writes from ingestion jobs and very low latency access to features during online serving.
31
31
32
-
Please continue to the [feature retrieval](feature-retrieval.md) section for more details on retrieving data from online storage.
32
+
Please continue to the [feature retrieval](../user-guide/feature-retrieval.md) section for more details on retrieving data from online storage.
Copy file name to clipboardExpand all lines: docs/concepts/architecture.md
+2-2Lines changed: 2 additions & 2 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -22,11 +22,11 @@ Streaming systems can also ingest data into Feast. This is done by publishing to
22
22
23
23
Stores are nothing more than databases used to store feature data. Feast loads data into stores through an ingestion process, after which the data can be served through the Feast Serving API. Stores are documented in the following section.
24
24
25
-
{% page-ref page="../user-guide/stores.md" %}
25
+
{% page-ref page="../advanced/stores.md" %}
26
26
27
27
### **Feast Serving**
28
28
29
-
`Feast Serving` is the data access layer through which end users and production systems retrieve feature data. Each `Serving` instance is backed by a [store](../user-guide/stores.md).
29
+
`Feast Serving` is the data access layer through which end users and production systems retrieve feature data. Each `Serving` instance is backed by a [store](../advanced/stores.md).
30
30
31
31
Since Feast supports multiple store types \(online, historical\) it is common to have two instances of Feast Serving deployed, one for online serving and one for historical serving. However, Feast allows for any number of `Feast Serving` deployments, meaning it is possible to deploy a `Feast Serving` deployment per production system, with its own stores and population jobs.
Copy file name to clipboardExpand all lines: docs/concepts/entities.md
+41-4Lines changed: 41 additions & 4 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -1,17 +1,26 @@
1
1
# Entities
2
2
3
-
An entity is any domain object that can be modelled and that information can be stored about. Entities are usually recognisable concepts, either concrete or abstract, such as persons, places, things, or events which have relevance to the modelled system.
3
+
## Overview
4
4
5
-
More formally, an entity is an instance of an entity type. An entity type is the class of entities where entities are the instances.
5
+
An entity is any domain object that can be modelled and that information can be stored about. Entities are usually recognisable concepts, either concrete or abstract, such as persons, places, things, or events which have relevance to the modelled system.
6
6
7
7
* Examples of entity types in the context of ride-hailing and food delivery: `customer`, `order`, `driver`, `restaurant`, `dish`, `area`.
8
8
* A specific driver, for example a driver with ID `D011234` would be an entity of the entity type `driver`
9
9
10
-
An entity is the object on which features are observed. For example we could have a feature `total_trips_24h` on the driver `D01123` with a feature value of `11`.
10
+
An entity is the object on which features are observed. For example, we could have a feature `total_trips_24h` on the driver `D01123` with a feature value of `11`.
11
11
12
12
In the context of Feast, entities are important because they are used as keys when looking up feature values. Entities are also used when joining feature values between different feature tables in order to build one large data set to train a model, or to serve a model.
13
13
14
-
Entities can be created through the [Feast SDK](../getting-started/connecting-to-feast-1/connecting-to-feast.md) as follows:
14
+
## Structure of an Entity
15
+
16
+
When creating an entity specification, the following fields should be considered:
17
+
18
+
***name**: Name of the entity.
19
+
***description**: Description of the entity.
20
+
***value\_type**: Value type of the entity.
21
+
***labels**: User-defined metadata.
22
+
23
+
A valid entity specification is shown below:
15
24
16
25
```python
17
26
from feast import Entity, ValueType
@@ -24,5 +33,33 @@ customer = Entity(
24
33
)
25
34
```
26
35
36
+
## Working with an Entity
37
+
38
+
#### Creating an Entity
39
+
40
+
```python
41
+
# Create a customer entity
42
+
customer_entity = Entity(name="customer_id", description="ID of car customer")
In Feast, each feature can be uniquely addressed through a feature reference. A feature reference is composed of the following components:
6
+
7
+
* Feature Table name
8
+
* Feature name
9
+
10
+
## Structure of a Feature Reference
11
+
12
+
A string based feature reference takes on the following format:
13
+
14
+
`<feature-table-name>:<feature-name>`
15
+
16
+
```python
17
+
# Feature references
18
+
feature_refs = [
19
+
"driver_trips:average_daily_rides",
20
+
"driver_trips:maximum_daily_rides",
21
+
"driver_trips:rating",
22
+
]
23
+
```
24
+
25
+
Feature references only apply to a single `project`. Features cannot be retrieved across projects in a single request.
26
+
27
+
## Working with a Feature Reference
28
+
29
+
#### Feature Retrieval
30
+
31
+
Feature retrieval \(or serving\) is the process of retrieving either historical features or online features from Feast, for the purposes of training or serving a model.
32
+
33
+
Feast attempts to unify the process of retrieving features in both the historical and online case. It does this through the creation of feature references. One of the major advantages of using Feast is that you have a single semantic reference to a feature. These feature references can then be stored alongside your model and loaded into a serving layer where it can be used for online feature retrieval.
34
+
35
+
More information about how to perform feature retrieval for historical and online features can be found in the sections under **User Guide**.
Copy file name to clipboardExpand all lines: docs/concepts/feature-tables.md
+97-10Lines changed: 97 additions & 10 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -1,16 +1,14 @@
1
1
# Feature Tables
2
2
3
+
## Overview
4
+
3
5
Feature tables are both a schema and a means of identifying data sources for features.
4
6
5
7
Data typically comes in the form of flat files, dataframes, tables in a database, or events on a stream. Thus, the data occurs with multiple columns/fields in multiple rows/events.
6
8
7
-
Feature tables are a way for defining the unique properties of these data sources, how Feast should interpret them, and how Feast should source them. Feature tables allow for groups of fields in these data sources to be [ingested](../user-guide/data-ingestion.md) and [stored](../user-guide/stores.md) together. Feature tables allow for efficient storage and logical namespacing of data within [stores](../user-guide/stores.md).
8
-
9
-
{% hint style="info" %}
10
-
Feature tables are a grouping of features based on how they are loaded into Feast. They ensure that data is efficiently stored during ingestion. Feature tables are not a grouping of features for retrieval of features. During retrieval it is possible to retrieve feature values from any number of feature tables.
11
-
{% endhint %}
9
+
Feature tables are a way for defining the unique properties of these data sources, how Feast should interpret them, and how Feast should source them. Feature tables allow for groups of fields in these data sources to be [ingested](../user-guide/data-ingestion.md) and [stored](../advanced/stores.md) together. Feature tables allow for efficient storage and logical namespacing of data within [stores](../advanced/stores.md).
12
10
13
-
## Features
11
+
####Features
14
12
15
13
A feature is an individual measurable property or characteristic of a phenomenon being observed. Features are the most important concepts within a feature store. Feature data is used both as input to models during training and when models are served in production.
Please see the [FeatureSpec](https://api.docs.feast.dev/grpc/feast.core.pb.html#FeatureSpecV2) for the complete feature specification API.
26
24
27
-
## Driver Trips Example
25
+
{% hint style="info" %}
26
+
Feature tables are a grouping of features based on how they are loaded into Feast. They ensure that data is efficiently stored during ingestion. Feature tables are not a grouping of features for retrieval of features. During retrieval it is possible to retrieve feature values from any number of feature tables.
27
+
{% endhint %}
28
+
29
+
## Structure of a Feature Table
30
+
31
+
Users are allowed to exclude a stream source when registering a Feature table, since stream sources might not necessarily exist. However, at minimal, a batch source must be specified since data needs to be available somewhere for ingestion into Feast.
32
+
33
+
When creating a feature table specification, the following fields should be considered:
28
34
29
-
Below is an example specification of a basic `driver trips` feature table created using the Python SDK:
35
+
***name:** Name of feature table. Must be unique.
36
+
***entities:** List of names of entities to associate with the features defined in this feature table. See [Entities](entities.md) page for more details.
37
+
***features:** List of feature specifications for each feature defined with this feature table.
38
+
***labels:** User-defined metadata.
39
+
***max\_age:** Max age is measured as the duration of time between the feature's event timestamp when when the feature is retrieved. Feature values outside max age will be returned as unset values and indicated to end user.
40
+
***batch\_source:** Batch/Offline data source to source batch/offline feature data. See [Sources](sources.md) page for more details.
41
+
***stream\_source:** Stream/Online data source to source stream/online feature data. See [Sources](sources.md) page for more details.
42
+
43
+
A valid feature table specification is shown below:
30
44
31
45
{% tabs %}
32
46
{% tab title="driver\_feature\_table.py" %}
@@ -41,6 +55,10 @@ driver_ft = FeatureTable(
41
55
Feature("average_daily_rides", ValueType.FLOAT),
42
56
Feature("rating", ValueType.FLOAT)
43
57
],
58
+
max_age=14400,
59
+
labels={
60
+
"team": "driver_matching"
61
+
},
44
62
batch_source=BigQuerySource(
45
63
table_ref="gcp_project:bq_dataset.bq_table",
46
64
event_timestamp_column="datetime",
@@ -54,11 +72,80 @@ driver_ft = FeatureTable(
54
72
{% endtab %}
55
73
{% endtabs %}
56
74
57
-
Shown in the YAML specifications above, the entity `driver_id` relates to an entity that has already been registered with Feast.
75
+
Shown in the feature table specification above, the entity `driver_id` relates to an entity that has already been registered with Feast.
58
76
59
-
By default, Feast assumes that the features specified in the YAML specification is a 1-1 mapping to the fields found in the batch source. However, if fields in the batch source are named differently to the feature names, `field_mappings` can be utilised as shown above \(i.e `rating` field in batch source is to be mapped to `driver_rating` feature.
77
+
By default, Feast assumes that the features specified in the feature table specification is a 1-1 mapping to the fields found in the batch source. However, if fields in the batch source are named differently to the feature names, `field_mappings` can be utilised as shown above \(i.e `rating` field in batch source is to be mapped to `driver_rating` feature.
60
78
61
79
{% hint style="info" %}
62
80
When applying a Feature Table without specifying a project, Feast creates/updates the Feature Table in the`default` project. To create a Feature Table in another project, specify the project of choice in the`apply_feature_table` call.
63
81
{% endhint %}
64
82
83
+
## Working with a Feature Table
84
+
85
+
There are various functionality that comes with a Feature Table.
86
+
87
+
#### Creating a Feature Table
88
+
89
+
```python
90
+
driver_ft = FeatureTable(..., max_age=14400, ...)
91
+
client.apply_feature_table(driver_ft)
92
+
```
93
+
94
+
#### Updating a Feature Table
95
+
96
+
In order to facilitate the need for feature table definitions to change over time, a limited set of changes can be made to existing feature tables.
97
+
98
+
```python
99
+
driver_ft = FeatureTable(..., max_age=7200, ...)
100
+
client.apply_feature_table(driver_ft)
101
+
```
102
+
103
+
Permitted changes include:
104
+
105
+
* Adding new features.
106
+
* Deleting existing features \(note that features are tombstoned and remain on record, rather than removed completely; as a result, new features will not be able to take the names of these deleted features\).
107
+
* Changing the feature table's source, max age and labels.
108
+
109
+
Note that the following are **not** allowed:
110
+
111
+
* Changes to project or name of feature table.
112
+
* Changes to entities related to the feature table.
113
+
* Changes to names and types of existing features.
0 commit comments