Skip to content

Commit dbaf799

Browse files
docs: add max_read_workers config documentation for DynamoDB online store
Add Configuration section documenting: - Configuration options table with max_read_workers, batch_size, consistent_reads, etc. - Performance tuning guidance for parallel batch reads - Example feature_store.yaml with tuning options Signed-off-by: abhijeet-dhumal <abhijeetdhumal652@gmail.com> Made-with: Cursor
1 parent f4eebb4 commit dbaf799

File tree

1 file changed

+41
-0
lines changed

1 file changed

+41
-0
lines changed

docs/reference/online-stores/dynamodb.md

Lines changed: 41 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -22,6 +22,47 @@ online_store:
2222
2323
The full set of configuration options is available in [DynamoDBOnlineStoreConfig](https://rtd.feast.dev/en/master/#feast.infra.online_stores.dynamodb.DynamoDBOnlineStoreConfig).
2424
25+
## Configuration
26+
27+
Below is a example with performance tuning options:
28+
29+
{% code title="feature_store.yaml" %}
30+
```yaml
31+
project: my_feature_repo
32+
registry: data/registry.db
33+
provider: aws
34+
online_store:
35+
type: dynamodb
36+
region: us-west-2
37+
batch_size: 100
38+
max_read_workers: 10
39+
consistent_reads: false
40+
```
41+
{% endcode %}
42+
43+
### Configuration Options
44+
45+
| Option | Type | Default | Description |
46+
| ------ | ---- | ------- | ----------- |
47+
| `region` | string | | AWS region for DynamoDB |
48+
| `table_name_template` | string | `{project}.{table_name}` | Template for table names |
49+
| `batch_size` | int | `40` | Number of items per BatchGetItem/BatchWriteItem request (max 100) |
50+
| `max_read_workers` | int | `10` | Maximum parallel threads for batch read operations. Higher values improve throughput for large batch reads but increase resource usage |
51+
| `consistent_reads` | bool | `false` | Whether to use strongly consistent reads (higher latency, guaranteed latest data) |
52+
| `tags` | dict | `null` | AWS resource tags added to each table |
53+
| `session_based_auth` | bool | `false` | Use AWS session-based client authentication |
54+
55+
### Performance Tuning
56+
57+
**Parallel Batch Reads**: When reading features for many entities, DynamoDB's BatchGetItem is limited to 100 items per request. For 500 entities, this requires 5 batch requests. The `max_read_workers` option controls how many of these batches execute in parallel:
58+
59+
- **Sequential (old behavior)**: 5 batches × 10ms = 50ms total
60+
- **Parallel (with `max_read_workers: 10`)**: 5 batches in parallel ≈ 10ms total
61+
62+
For high-throughput workloads with large entity counts, increase `max_read_workers` (up to 20-30) based on your DynamoDB capacity and network conditions.
63+
64+
**Batch Size**: Increase `batch_size` up to 100 to reduce the number of API calls. However, larger batches may hit DynamoDB's 16MB response limit for tables with large feature values.
65+
2566
## Permissions
2667

2768
Feast requires the following permissions in order to execute commands for DynamoDB online store:

0 commit comments

Comments
 (0)