You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: docs/getting-started/concepts/feast-types.md
+17-8Lines changed: 17 additions & 8 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -11,29 +11,38 @@ Feast supports the following categories of data types:
11
11
-**Array types**: ordered lists of any primitive type, e.g. `Array(Int64)`, `Array(String)`.
12
12
-**Set types**: unordered collections of unique values for any primitive type, e.g. `Set(String)`, `Set(Int64)`.
13
13
-**Map types**: dictionary-like structures with string keys and values that can be any supported Feast type (including nested maps), e.g. `Map`, `Array(Map)`.
14
+
-**JSON type**: opaque JSON data stored as a string at the proto level but semantically distinct from `String` — backends use native JSON types (`jsonb`, `VARIANT`, etc.), e.g. `Json`, `Array(Json)`.
15
+
-**Struct type**: schema-aware structured type with named, typed fields. Unlike `Map` (which is schema-free), a `Struct` declares its field names and their types, enabling schema validation, e.g. `Struct({"name": String, "age": Int32})`.
14
16
15
17
For a complete reference with examples, see [Type System](../../reference/type-system.md).
16
18
17
19
Each feature or schema field in Feast is associated with a data type, which is stored in Feast's [registry](registry.md). These types are also used to ensure that Feast operates on values correctly (e.g. making sure that timestamp columns used for [point-in-time correct joins](point-in-time-joins.md) actually have the timestamp type).
18
20
19
21
As a result, each system that Feast interacts with needs a way to translate data types from the native platform into a Feast type. E.g., Snowflake SQL types are converted to Feast types [here](https://rtd.feast.dev/en/master/feast.html#feast.type_map.snowflake_python_type_to_feast_value_type). The onus is therefore on authors of offline or online store connectors to make sure that this type mapping happens correctly.
20
22
21
-
### Backend Type Mapping for Maps
23
+
### Backend Type Mapping for Complex Types
22
24
23
-
Map types are supported across all major Feast backends:
25
+
Map, JSON, and Struct types are supported across all major Feast backends:
**Note**: When the backend native type is ambiguous (e.g., `jsonb` could be `Map`, `Json`, or `Struct`), the **schema-declared Feast type takes precedence**. The backend-to-Feast type mappings above are only used for schema inference when no explicit type is provided.
38
47
39
48
**Note**: Feast currently does *not* support a null type in its type system.
**JSON vs Map vs Struct**: These three complex types serve different purposes:
192
+
-**`Map`**: Schema-free dictionary with string keys. Use when the keys and values are dynamic.
193
+
-**`Json`**: Opaque JSON data stored as a string. Backends use native JSON types (`jsonb`, `VARIANT`). Use for configuration blobs or API responses where you don't need field-level typing.
194
+
-**`Struct`**: Schema-aware structured type with named, typed fields. Persisted through the registry via Field tags. Use when you know the exact structure and want type safety.
195
+
189
196
Validation is supported in all compute engines (Local, Spark, and Ray). When a required column is missing, a `ValueError` is raised. Type mismatches are logged as warnings but do not block execution, allowing for safe gradual adoption.
190
197
191
198
The `enable_validation` parameter is also available on `BatchFeatureView` and `StreamFeatureView`, as well as their respective decorators (`@batch_feature_view` and `@stream_feature_view`).
Note that this mapping is non-injective, that is more than one Pandas type may corresponds to one Feast type (but not vice versa). In these cases, when converting Feast values to Pandas, the **first** Pandas type in the table above is used.
56
60
@@ -82,6 +86,10 @@ Here's how Feast types map to BigQuery types when using BigQuery for offline sto
82
86
| BOOL\_LIST |`ARRAY<BOOL>`|
83
87
| MAP |`JSON` / `STRUCT`|
84
88
| MAP\_LIST |`ARRAY<JSON>` / `ARRAY<STRUCT>`|
89
+
| JSON |`JSON`|
90
+
| JSON\_LIST |`ARRAY<JSON>`|
91
+
| STRUCT |`STRUCT` / `RECORD`|
92
+
| STRUCT\_LIST |`ARRAY<STRUCT>`|
85
93
86
94
Values that are not specified by the table above will cause an error on conversion.
Here's how Feast types map to Redshift types when using Redshift for offline storage:
@@ -114,5 +123,6 @@ Here's how Feast types map to Redshift types when using Redshift for offline sto
114
123
| FLOAT |`FLOAT4` / `REAL`|
115
124
| BOOL |`BOOL`|
116
125
| MAP |`SUPER`|
126
+
| JSON |`json` / `SUPER`|
117
127
118
128
Note: Redshift's `SUPER` type stores semi-structured JSON data. During materialization, Feast automatically handles `SUPER` columns that are exported as JSON strings by parsing them back into Python dictionaries before converting to `MAP` proto values.
0 commit comments