Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
18 commits
Select commit Hold shift + click to select a range
d82dcf2
feat: scaffold Aerospike online store
vkagamlyk Apr 20, 2026
2fa7ba0
feat: implement Aerospike online_write_batch
vkagamlyk Apr 20, 2026
4173e0a
feat: implement Aerospike online_read
vkagamlyk Apr 20, 2026
534b935
feat: implement Aerospike update and teardown
vkagamlyk Apr 20, 2026
fb4dff7
test: add Aerospike unit and integration tests
vkagamlyk Apr 20, 2026
53d5972
feat: add async online_read/write and lifecycle hooks for Aerospike
vkagamlyk Apr 20, 2026
17b1dcb
docs: add Aerospike online store reference and tuning guide
vkagamlyk Apr 20, 2026
f552a94
fix: use bytearray keys and zip-based batch mapping for Aerospike reads
vkagamlyk Apr 20, 2026
ebb3d60
docs: clarify Aerospike auth and TLS sections are Enterprise-only
vkagamlyk Apr 20, 2026
d86dcbf
feat: add aerospike to feast-operator supported online stores
vkagamlyk Apr 20, 2026
5d9e38f
fix(aerospike): project requested_features server-side and surface pe…
vkagamlyk Apr 22, 2026
5553367
feat(aerospike)!: rename total_timeout_ms -> batch_total_timeout_ms a…
vkagamlyk Apr 22, 2026
953b9d1
refactor(aerospike): use MAP_KEY_ORDERED, KEY_DIGEST, and instance-sc…
vkagamlyk Apr 22, 2026
0c8c184
feat(aerospike): add per-FV namespace/set overrides and prewriting hook
vkagamlyk Apr 30, 2026
6628607
test: update aerospike image tag
vkagamlyk Jun 16, 2026
9e76c2f
chore: sync README template and secrets baseline after master merge
vkagamlyk Jun 16, 2026
ea9da18
chore: fix secrets baseline line number for v1 operator types
vkagamlyk Jun 16, 2026
2c37c0f
docs: update aerospike docs
vkagamlyk Jun 16, 2026
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 3 additions & 3 deletions .secrets.baseline
Original file line number Diff line number Diff line change
Expand Up @@ -957,7 +957,7 @@
"filename": "infra/feast-operator/api/v1/featurestore_types.go",
"hashed_secret": "44e17306b837162269a410204daaa5ecee4ec22c",
"is_verified": false,
"line_number": 906
"line_number": 907
}
],
"infra/feast-operator/api/v1/zz_generated.deepcopy.go": [
Expand Down Expand Up @@ -989,7 +989,7 @@
"filename": "infra/feast-operator/api/v1alpha1/featurestore_types.go",
"hashed_secret": "44e17306b837162269a410204daaa5ecee4ec22c",
"is_verified": false,
"line_number": 649
"line_number": 650
}
],
"infra/feast-operator/api/v1alpha1/zz_generated.deepcopy.go": [
Expand Down Expand Up @@ -1555,5 +1555,5 @@
}
]
},
"generated_at": "2026-06-11T15:45:28Z"
"generated_at": "2026-06-16T21:53:17Z"
}
1 change: 1 addition & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -227,6 +227,7 @@ The list below contains the functionality that contributors are planning to deve
* [x] [SingleStore](https://docs.feast.dev/reference/online-stores/singlestore)
* [x] [Couchbase](https://docs.feast.dev/reference/online-stores/couchbase)
* [x] [MongoDB](https://docs.feast.dev/reference/online-stores/mongodb)
* [x] [Aerospike](https://docs.feast.dev/reference/online-stores/aerospike)
* [x] [Qdrant (vector store)](https://docs.feast.dev/reference/online-stores/qdrant)
* [x] [Milvus (vector store)](https://docs.feast.dev/reference/online-stores/milvus)
* [x] [Faiss (vector store)](https://docs.feast.dev/reference/online-stores/faiss)
Expand Down
1 change: 1 addition & 0 deletions docs/SUMMARY.md
Original file line number Diff line number Diff line change
Expand Up @@ -164,6 +164,7 @@
* [SingleStore](reference/online-stores/singlestore.md)
* [Milvus](reference/online-stores/milvus.md)
* [MongoDB](reference/online-stores/mongodb.md)
* [Aerospike](reference/online-stores/aerospike.md)
* [Elasticsearch](reference/online-stores/elasticsearch.md)
* [Qdrant](reference/online-stores/qdrant.md)
* [Faiss](reference/online-stores/faiss.md)
Expand Down
31 changes: 31 additions & 0 deletions docs/how-to-guides/online-server-performance-tuning.md
Original file line number Diff line number Diff line change
Expand Up @@ -278,6 +278,7 @@ The online store is the single largest factor in `get_online_features()` latency
| **DynamoDB** | 2–5 ms | Yes | Serverless, auto-scaling on AWS | Pay-per-request cost; batch API limits (100 items) |
| **PostgreSQL** | 3–10 ms | No (threadpool) | Teams with existing Postgres infra | Connection pooling needed at scale |
| **MongoDB** | 2–5 ms | Yes | Flexible schema, async-native | Requires index tuning for large datasets |
| **Aerospike** | < 1 ms | No (threadpool) | Ultra-low latency, hybrid memory (RAM + SSD), large datasets | Namespace must be pre-configured on the cluster |
| **Bigtable** | 3–8 ms | No (threadpool) | Large-scale GCP workloads | Row-key design affects read performance |
| **Cassandra / ScyllaDB** | 2–5 ms | No (threadpool) | Multi-region, write-heavy | Tunable consistency; requires DC-aware routing |
| **Remote** | Varies | No (threadpool) | Centralized feature server architecture | Adds an HTTP hop; tune connection pool |
Expand Down Expand Up @@ -305,6 +306,7 @@ The feature server can read from the online store using either an **async** or *
| **MongoDB** | Yes | Yes | Uses `motor` (async MongoDB driver) |
| **PostgreSQL** | Implemented | No | Has `online_read_async` but does not yet advertise via `async_supported`; uses sync/threadpool path |
| **Redis** | Implemented | **Yes** | `online_read_async` and `online_write_batch_async` both implemented; uses sync/threadpool path for `get_online_features` (overridden with batched single pipeline) |
| **Aerospike** | Implemented | No | Async methods wrap the blocking C client via `run_in_executor`; does not yet advertise via `async_supported`, so the server still uses the threadpool path |
| All others | No | No | Fall back to sync with `run_in_threadpool()` |

**When async matters most:**
Expand Down Expand Up @@ -467,6 +469,34 @@ online_store:
- **`connectTimeoutMS` / `socketTimeoutMS`**: Tighter timeouts improve p99 by failing fast on slow connections.
- MongoDB is one of the stores with **full async support** (read and write), so it benefits from concurrent feature view reads via `asyncio.gather()`.

### Aerospike tuning

Aerospike offers sub-millisecond reads thanks to its hybrid-memory architecture (primary index in RAM, data on SSD or RAM). Tune the per-call policies in the Feast config and rely on the Aerospike cluster's own tuning for everything else:

```yaml
online_store:
type: aerospike
hosts:
- ["aerospike-1.internal", 3000]
- ["aerospike-2.internal", 3000]
namespace: feast
read_timeout_ms: 150 # hard deadline for a single-record get
write_timeout_ms: 300 # hard deadline for a single-record put/operate
batch_total_timeout_ms: 500 # hard deadline for online_read / online_write_batch
socket_timeout_ms: 50 # per-attempt deadline so max_retries can actually fire
max_retries: 2
ttl_seconds: 86400 # record-level TTL; omit to use the namespace default
client_kwargs: # escape hatch for any client-config field not surfaced above
policies:
batch:
concurrent_nodes: 0 # 0 = parallel to every node (lowest latency on multi-node clusters)
```

- **`*_timeout_ms` (total)** vs **`socket_timeout_ms` (per-attempt)**: `*_timeout_ms` is the hard deadline for a whole call *including* retries; `socket_timeout_ms` is the per-attempt deadline that allows `max_retries` to actually fire within that budget. Without `socket_timeout_ms`, a single slow attempt can consume the entire total deadline and retries never run.
- **`hosts`**: List every seed node. The Aerospike client discovers the rest of the cluster automatically and opens one connection pool per node.
- **`ttl_seconds: 0`** means "never expire"; omit the key to inherit the namespace's `default-ttl`. Expiry is enforced by the server's `nsup` thread — nothing to delete on the client side.
- Co-locate the feature server in the **same availability zone / rack** as the Aerospike cluster; sub-millisecond reads are bandwidth- and RTT-sensitive.

### Remote online store tuning

The Remote online store connects to a Feast feature server over HTTP. Connection pooling is critical:
Expand Down Expand Up @@ -700,6 +730,7 @@ This applies to every connection-oriented online store:
| **DynamoDB** | `max_pool_connections` (HTTP pool) | 10 | No hard limit, but AWS SDK has per-process pool caps; monitor throttling |
| **Redis** | Connection per worker | 1 | `maxclients` on the Redis server (default: 10,000) |
| **MongoDB** | `maxPoolSize` (in `client_kwargs`) | 100 | Server's `net.maxIncomingConnections` |
| **Aerospike** | Driver manages pool per seed node | Auto | `proto-fd-max` (default 15000) on each Aerospike node |
| **Cassandra** | Driver manages pool per node | Auto | `native_transport_max_threads` on each Cassandra node |
| **Remote** | `connection_pool_size` (HTTP pool) | 50 | The target feature server's worker capacity |

Expand Down
4 changes: 4 additions & 0 deletions docs/reference/online-stores/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -58,6 +58,10 @@ Please see [Online Store](../../getting-started/components/online-store.md) for
[mongodb.md](mongodb.md)
{% endcontent-ref %}

{% content-ref url="aerospike.md" %}
[aerospike.md](aerospike.md)
{% endcontent-ref %}

{% content-ref url="hazelcast.md" %}
[hazelcast.md](hazelcast.md)
{% endcontent-ref %}
Expand Down
Loading