Skip to content

Commit 2626b50

Browse files
franciscojavierarceofranciscojavierarceo
authored
Merge branch 'master' into resolve_pyarrow
2 parents 17ac7df + 5482a0e commit 2626b50

File tree

11 files changed

+522
-25
lines changed

11 files changed

+522
-25
lines changed

‎.gitignore‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -235,6 +235,7 @@ infra/feast-operator/bin
235235
# ignore the website build directory
236236
infra/website/node_modules/
237237
infra/website/.astro/
238+
infra/website/dist/
238239

239240
# offline builds
240241
offline_build/

‎docs/getting-started/components/registry.md‎

Lines changed: 58 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -11,6 +11,64 @@ automatically stays synced with the registry. Users will often also want multipl
1111
different environments (e.g. dev vs staging vs prod), with staging and production registries with locked down write
1212
access since they can impact real user traffic. See [Running Feast in Production](../../how-to-guides/running-feast-in-production.md#1.-automatically-deploying-changes-to-your-feature-definitions) for details on how to set this up.
1313

14+
## Deleting objects from the registry
15+
16+
{% hint style="warning" %}
17+
Simply removing a feature definition from your code and running `feast apply` or `FeatureStore.apply()` **does not** delete the object from the registry. You must explicitly delete objects using the dedicated delete methods or CLI commands.
18+
{% endhint %}
19+
20+
### Using the CLI
21+
22+
The simplest way to delete objects is using the `feast delete` command:
23+
24+
```bash
25+
# Delete any Feast object by name
26+
feast delete my_feature_view
27+
feast delete my_entity
28+
feast delete my_feature_service
29+
```
30+
31+
See the [CLI documentation](../../reference/feast-cli-commands.md#delete) for more details.
32+
33+
### Using the Python SDK
34+
35+
To delete objects programmatically, use the explicit delete methods provided by the `FeatureStore` class:
36+
37+
#### Deleting feature views
38+
```python
39+
from feast import FeatureStore
40+
41+
store = FeatureStore(repo_path=".")
42+
store.delete_feature_view("my_feature_view")
43+
```
44+
45+
#### Deleting feature services
46+
```python
47+
store.delete_feature_service("my_feature_service")
48+
```
49+
50+
#### Deleting entities, data sources, and other objects
51+
52+
For entities, data sources, and other registry objects, you can use the registry methods directly:
53+
54+
```python
55+
# Delete an entity
56+
store._registry.delete_entity("my_entity", project=store.project)
57+
58+
# Delete a data source
59+
store._registry.delete_data_source("my_data_source", project=store.project)
60+
61+
# Delete a saved dataset
62+
store._registry.delete_saved_dataset("my_saved_dataset", project=store.project)
63+
64+
# Delete a validation reference
65+
store._registry.delete_validation_reference("my_validation_reference", project=store.project)
66+
```
67+
68+
{% hint style="info" %}
69+
When using `feast apply` via the CLI, you can also use the `objects_to_delete` parameter with `partial=False` to delete objects as part of the apply operation. However, this is less common and typically used in automated deployment scenarios.
70+
{% endhint %}
71+
1472
## Accessing the registry from clients
1573

1674
Users can specify the registry through a `feature_store.yaml` config file, or programmatically. We often see teams

‎docs/reference/beta-on-demand-feature-view.md‎

Lines changed: 40 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -309,4 +309,44 @@ There are new CLI commands to manage on demand feature views:
309309
feast on-demand-feature-views list: Lists all registered on demand feature views after feast apply is run.
310310
feast on-demand-feature-views describe [NAME]: Describes the definition of an on demand feature view.
311311

312+
## Troubleshooting
313+
314+
### Validation Issues with Complex Transformations
315+
316+
When defining On Demand Feature Views with complex transformations, you may encounter validation errors during `feast apply`. Feast validates ODFVs by constructing random inputs and running the transformation function to infer the output schema. This validation can sometimes be overly strict or fail for complex transformation logic.
317+
318+
If you encounter validation errors that you believe are incorrect, you can skip feature view validation:
319+
320+
**Python SDK:**
321+
```python
322+
from feast import FeatureStore
323+
324+
store = FeatureStore(repo_path=".")
325+
store.apply([my_odfv], skip_feature_view_validation=True)
326+
```
327+
328+
**CLI:**
329+
```bash
330+
feast apply --skip-feature-view-validation
331+
```
332+
333+
{% hint style="warning" %}
334+
Skipping validation bypasses important checks. Use this option only when the validation system is being overly strict. We encourage you to report validation issues on the [Feast GitHub repository](https://github.com/feast-dev/feast/issues) so the team can improve the validation system.
335+
{% endhint %}
336+
337+
**When to use skip_feature_view_validation:**
338+
- Your ODFV transformation uses complex logic that fails the random input validation
339+
- You've verified your transformation works correctly with real data
340+
- The validation error doesn't reflect an actual issue with your transformation
341+
342+
**What validation is skipped:**
343+
- Feature view name uniqueness checks
344+
- ODFV transformation validation via `_construct_random_input()`
345+
- Schema inference for features
346+
347+
**What is NOT skipped:**
348+
- Data source validation (use `--skip-source-validation` to skip)
349+
- Registry operations
350+
- Infrastructure updates
351+
312352

‎docs/reference/feast-cli-commands.md‎

Lines changed: 59 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -20,6 +20,7 @@ Options:
2020
Commands:
2121
apply Create or update a feature store deployment
2222
configuration Display Feast configuration
23+
delete Delete a Feast object from the registry
2324
entities Access entities
2425
feature-views Access feature views
2526
init Create a new Feast repository
@@ -51,17 +52,40 @@ Creates or updates a feature store deployment
5152
feast apply
5253
```
5354

55+
**Options:**
56+
* `--skip-source-validation`: Skip validation of data sources (don't check if tables exist)
57+
* `--skip-feature-view-validation`: Skip validation of feature views. Use with caution as this skips important checks
58+
59+
```bash
60+
# Skip only data source validation
61+
feast apply --skip-source-validation
62+
63+
# Skip only feature view validation
64+
feast apply --skip-feature-view-validation
65+
66+
# Skip both validations
67+
feast apply --skip-source-validation --skip-feature-view-validation
68+
```
69+
5470
**What does Feast apply do?**
5571

5672
1. Feast will scan Python files in your feature repository and find all Feast object definitions, such as feature views, entities, and data sources.
57-
2. Feast will validate your feature definitions (e.g. for uniqueness of features)
73+
2. Feast will validate your feature definitions (e.g. for uniqueness of features). This validation can be skipped using the `--skip-feature-view-validation` flag if the type/validation system is being overly strict.
5874
3. Feast will sync the metadata about Feast objects to the registry. If a registry does not exist, then it will be instantiated. The standard registry is a simple protobuf binary file that is stored on disk \(locally or in an object store\).
5975
4. Feast CLI will create all necessary feature store infrastructure. The exact infrastructure that is deployed or configured depends on the `provider` configuration that you have set in `feature_store.yaml`. For example, setting `local` as your provider will result in a `sqlite` online store being created.
6076

77+
{% hint style="info" %}
78+
The `--skip-feature-view-validation` flag is particularly useful for On-Demand Feature Views (ODFVs) with complex transformations that may fail validation. However, use it with caution and please report any validation issues to the Feast team on GitHub.
79+
{% endhint %}
80+
6181
{% hint style="warning" %}
6282
`feast apply` \(when configured to use cloud provider like `gcp` or `aws`\) will create cloud infrastructure. This may incur costs.
6383
{% endhint %}
6484

85+
{% hint style="info" %}
86+
**Important:** `feast apply` only registers or updates objects found in your Python files. It does **not** delete objects that you've removed from your code. To delete objects from the registry, you must use the `feast delete` command or explicit delete methods in the Python SDK. See the [Delete command](#delete) below and the [Registry documentation](../getting-started/components/registry.md#deleting-objects-from-the-registry) for details.
87+
{% endhint %}
88+
6589
## Configuration
6690

6791
Display the actual configuration being used by Feast, including both user-provided configurations and default configurations applied by Feast.
@@ -84,6 +108,40 @@ auth:
84108
type: no_auth
85109
```
86110
111+
## Delete
112+
113+
Delete a Feast object from the registry by its name.
114+
115+
```bash
116+
feast delete <OBJECT_NAME>
117+
```
118+
119+
**What does feast delete do?**
120+
121+
The `feast delete` command removes a Feast object (such as a feature view, entity, data source, feature service, etc.) from the registry. The command will:
122+
123+
1. Search for the object by name across all object types (entities, feature views, feature services, data sources, saved datasets, validation references, etc.)
124+
2. Delete the first matching object found
125+
3. Remove any associated infrastructure
126+
127+
**Example:**
128+
129+
```bash
130+
# Delete a feature view named "driver_hourly_stats"
131+
feast delete driver_hourly_stats
132+
133+
# Delete an entity named "driver"
134+
feast delete driver
135+
```
136+
137+
{% hint style="warning" %}
138+
The delete operation is permanent and will remove the object from the registry. Make sure you want to delete the object before running this command.
139+
{% endhint %}
140+
141+
{% hint style="info" %}
142+
If multiple objects have the same name across different types, `feast delete` will delete the first one it finds. For programmatic deletion with more control, use the Python SDK methods like `store.delete_feature_view()`, `store.delete_feature_service()`, etc.
143+
{% endhint %}
144+
87145
## Entities
88146

89147
List all registered entities

‎docs/reference/type-system.md‎

Lines changed: 139 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -3,13 +3,149 @@
33
## Motivation
44

55
Feast uses an internal type system to provide guarantees on training and serving data.
6-
Feast currently supports eight primitive types - `INT32`, `INT64`, `FLOAT32`, `FLOAT64`, `STRING`, `BYTES`, `BOOL`, and `UNIX_TIMESTAMP` - and the corresponding array types.
7-
Map type is also supported using a key of `STRING` type and any supported feast type as a value.
6+
Feast supports primitive types, array types, and map types for feature values.
87
Null types are not supported, although the `UNIX_TIMESTAMP` type is nullable.
98
The type system is controlled by [`Value.proto`](https://github.com/feast-dev/feast/blob/master/protos/feast/types/Value.proto) in protobuf and by [`types.py`](https://github.com/feast-dev/feast/blob/master/sdk/python/feast/types.py) in Python.
109
Type conversion logic can be found in [`type_map.py`](https://github.com/feast-dev/feast/blob/master/sdk/python/feast/type_map.py).
1110

12-
## Examples
11+
## Supported Types
12+
13+
Feast supports the following data types:
14+
15+
### Primitive Types
16+
17+
| Feast Type | Python Type | Description |
18+
|------------|-------------|-------------|
19+
| `Int32` | `int` | 32-bit signed integer |
20+
| `Int64` | `int` | 64-bit signed integer |
21+
| `Float32` | `float` | 32-bit floating point |
22+
| `Float64` | `float` | 64-bit floating point |
23+
| `String` | `str` | String/text value |
24+
| `Bytes` | `bytes` | Binary data |
25+
| `Bool` | `bool` | Boolean value |
26+
| `UnixTimestamp` | `datetime` | Unix timestamp (nullable) |
27+
28+
### Array Types
29+
30+
All primitive types have corresponding array (list) types:
31+
32+
| Feast Type | Python Type | Description |
33+
|------------|-------------|-------------|
34+
| `Array(Int32)` | `List[int]` | List of 32-bit integers |
35+
| `Array(Int64)` | `List[int]` | List of 64-bit integers |
36+
| `Array(Float32)` | `List[float]` | List of 32-bit floats |
37+
| `Array(Float64)` | `List[float]` | List of 64-bit floats |
38+
| `Array(String)` | `List[str]` | List of strings |
39+
| `Array(Bytes)` | `List[bytes]` | List of binary data |
40+
| `Array(Bool)` | `List[bool]` | List of booleans |
41+
| `Array(UnixTimestamp)` | `List[datetime]` | List of timestamps |
42+
43+
### Map Types
44+
45+
Map types allow storing dictionary-like data structures:
46+
47+
| Feast Type | Python Type | Description |
48+
|------------|-------------|-------------|
49+
| `Map` | `Dict[str, Any]` | Dictionary with string keys and any supported Feast type as values (including nested maps) |
50+
| `Array(Map)` | `List[Dict[str, Any]]` | List of dictionaries |
51+
52+
**Note:** Map keys must always be strings. Map values can be any supported Feast type, including primitives, arrays, or nested maps.
53+
54+
## Complete Feature View Example
55+
56+
Below is a complete example showing how to define a feature view with all supported types:
57+
58+
```python
59+
from datetime import timedelta
60+
from feast import Entity, FeatureView, Field, FileSource
61+
from feast.types import (
62+
Int32, Int64, Float32, Float64, String, Bytes, Bool, UnixTimestamp,
63+
Array, Map
64+
)
65+
66+
# Define a data source
67+
user_features_source = FileSource(
68+
path="data/user_features.parquet",
69+
timestamp_field="event_timestamp",
70+
)
71+
72+
# Define an entity
73+
user = Entity(
74+
name="user_id",
75+
description="User identifier",
76+
)
77+
78+
# Define a feature view with all supported types
79+
user_features = FeatureView(
80+
name="user_features",
81+
entities=[user],
82+
ttl=timedelta(days=1),
83+
schema=[
84+
# Primitive types
85+
Field(name="age", dtype=Int32),
86+
Field(name="account_balance", dtype=Int64),
87+
Field(name="transaction_amount", dtype=Float32),
88+
Field(name="credit_score", dtype=Float64),
89+
Field(name="username", dtype=String),
90+
Field(name="profile_picture", dtype=Bytes),
91+
Field(name="is_active", dtype=Bool),
92+
Field(name="last_login", dtype=UnixTimestamp),
93+
94+
# Array types
95+
Field(name="daily_steps", dtype=Array(Int32)),
96+
Field(name="transaction_history", dtype=Array(Int64)),
97+
Field(name="ratings", dtype=Array(Float32)),
98+
Field(name="portfolio_values", dtype=Array(Float64)),
99+
Field(name="favorite_items", dtype=Array(String)),
100+
Field(name="document_hashes", dtype=Array(Bytes)),
101+
Field(name="notification_settings", dtype=Array(Bool)),
102+
Field(name="login_timestamps", dtype=Array(UnixTimestamp)),
103+
104+
# Map types
105+
Field(name="user_preferences", dtype=Map),
106+
Field(name="metadata", dtype=Map),
107+
Field(name="activity_log", dtype=Array(Map)),
108+
],
109+
source=user_features_source,
110+
)
111+
```
112+
113+
### Map Type Usage Examples
114+
115+
Maps can store complex nested data structures:
116+
117+
```python
118+
# Simple map
119+
user_preferences = {
120+
"theme": "dark",
121+
"language": "en",
122+
"notifications_enabled": True,
123+
"font_size": 14
124+
}
125+
126+
# Nested map
127+
metadata = {
128+
"profile": {
129+
"bio": "Software engineer",
130+
"location": "San Francisco"
131+
},
132+
"stats": {
133+
"followers": 1000,
134+
"posts": 250
135+
}
136+
}
137+
138+
# List of maps
139+
activity_log = [
140+
{"action": "login", "timestamp": "2024-01-01T10:00:00", "ip": "192.168.1.1"},
141+
{"action": "purchase", "timestamp": "2024-01-01T11:30:00", "amount": 99.99},
142+
{"action": "logout", "timestamp": "2024-01-01T12:00:00"}
143+
]
144+
```
145+
146+
## Type System in Practice
147+
148+
The sections below explain how Feast uses its type system in different contexts.
13149

14150
### Feature inference
15151

0 commit comments

Comments
 (0)