feat: Add OnlineStore for MongoDB#6025
Conversation
5a815be to
6581159
Compare
|
@caseyclements please check the errors in the tests :) |
6aa5578 to
91dc75c
Compare
|
@shuchu Would you please help me investigate the CI timeout? I can run the universal tests for mongodb locally, like so: |
|
@caseyclements Please rebase with master, it would solve the CI issue |
91dc75c to
bcaa548
Compare
|
Thanks @ntkathole , @shuchu . Would you please approve further workflows if applicable. |
|
Hi @shuchu, @ntkathole, @oavdeev, @ankurs. All checks are now passing on approved workflows. Would you please kick off the others? Of course, your personal feedback would be great as well. In the meantime, I am working on an offline store. |
|
Hi @shuchu, @ntkathole, @oavdeev, @ankurs. I made a small change in |
Need to add mongodb in operator as well |
@ntkathole Thanks. The go files weren't on my radar. It looks like we're almost there. I updated a few small pieces based on feedback from our product team. Would you please kick off the workflows? |
|
@caseyclements seems linting issue with operator code, you can run make lint in feast-operator dir to fix |
I believe this might be fixed by updating the GOLANGCI_LINT version. This fixes the issue which I could reproduce locally. No actual changes were needed though. It was a tooling issue. |
|
@ntkathole Can I get a rerun of the workflows please? Is there someone on the west coast that can do another run? |
| We remove any feature views named in tables_to_delete. | ||
| The Entities are serialized in the _id. No schema needs be adjusted. | ||
| """ | ||
| if config.online_store.type != "mongodb": |
There was a problem hiding this comment.
| if config.online_store.type != "mongodb": | |
| if not isinstance(config.online_store, MongoDBOnlineStoreConfig): |
Good to be consistent across
| _client_async: Optional[AsyncMongoClient] = None | ||
| _collection_async: Optional[AsyncCollection] = None | ||
|
|
||
| def online_write_batch( |
There was a problem hiding this comment.
online_write_batch and online_write_batch_async has duplicated logic written in two different styles, we can extract the shared logic into a static helper that both paths call
|
HI @ntkathole I'll make the suggested changes. Is the one failing test a red-herring? It appears to be an unrelated timeout. |
yes, unrelated |
|
Please rebase with master, seems there are some conflicts |
Can I merge, or is only rebase permitted? |
| async def close(self) -> None: | ||
| """Close the async MongoDB client and release its resources.""" | ||
| if self._client_async is not None: | ||
| await self._client_async.close() | ||
| self._client_async = None | ||
| self._collection_async = None |
There was a problem hiding this comment.
🟡 Resource leak: teardown() never closes async client, close() never closes sync client
The MongoDBOnlineStore maintains two independent clients: _client (sync MongoClient) and _client_async (async AsyncMongoClient). The teardown() method (line 273-276) only closes the sync _client and leaves _client_async open. Conversely, the close() method (line 280-283) only closes _client_async and leaves the sync _client open. If both clients are created during the lifetime of a store instance (e.g., sync writes via online_write_batch followed by async reads via online_read_async, or vice versa), the other client's connection pool will leak when cleanup runs through either path.
| async def close(self) -> None: | |
| """Close the async MongoDB client and release its resources.""" | |
| if self._client_async is not None: | |
| await self._client_async.close() | |
| self._client_async = None | |
| self._collection_async = None | |
| async def close(self) -> None: | |
| """Close both async and sync MongoDB clients and release their resources.""" | |
| if self._client_async is not None: | |
| await self._client_async.close() | |
| self._client_async = None | |
| self._collection_async = None | |
| if self._client is not None: | |
| self._client.close() | |
| self._client = None | |
| self._collection = None |
Was this helpful? React with 👍 or 👎 to provide feedback.
…t.toml Any change to pyproject.toml invalidates the pixi.lock manifest hash, causing 'pixi install --locked' to fail in CI even when the changed section (mongodb optional extras) is not used by any pixi environment. Regenerated with: pixi install (v0.63.2) Signed-off-by: Casey Clements <casey.clements@mongodb.com>
Stores like mongodb, redis, and dynamodb raise FeastExtrasDependencyImportError at import time when their optional Python extras are not installed. test_all.py only caught ModuleNotFoundError, so any such import caused the entire test_docstrings() function to abort rather than gracefully skipping the module. Extend the except clause to also catch FeastExtrasDependencyImportError so the doctest run completes for all other modules when an optional extra is absent in the test environment. Signed-off-by: Casey Clements <casey.clements@mongodb.com>
Directory was renamed from tests/integration/feature_repos to tests/universal/feature_repos in upstream/master. Update the PYTEST_PLUGINS module path in test-python-universal-mongodb-online to match the new location. Signed-off-by: Casey Clements <casey.clements@mongodb.com>
…eError and other third-party import errors qdrant_client raises TypeError at import time when a gRPC EnumTypeWrapper is used with the | union operator on Python < 3.12. The previous fix only caught ModuleNotFoundError and FeastExtrasDependencyImportError, leaving the test runner vulnerable to any other exception raised by third-party libraries during pkgutil.walk_packages. Changes: - Catch bare Exception in the import try/except block so any import-time error from an optional dependency causes a graceful skip rather than an abort of the entire test run. - Initialize temp_module = None before the try block and add a continue guard so that a failed import never leaves a stale module reference to be used in the subsequent doctest execution block. Signed-off-by: Casey Clements <casey.clements@mongodb.com>
…r package import failures FeastExtrasDependencyImportError inherits from FeastError -> Exception, not from ImportError. pkgutil.walk_packages calls __import__ internally when recursing into sub-packages, and only silently swallows ImportError subclasses; any other exception is re-raised, crashing the entire test run. Passing onerror=lambda _: None tells walk_packages to skip any package that fails to import during the walk phase, regardless of the exception type. The inner importlib.import_module try/except already handles the same errors for the explicit import step used to collect doctests. Signed-off-by: Casey Clements <casey.clements@mongodb.com>
…iversal.feature_repos The upstream directory tests/integration/feature_repos was renamed to tests/universal/feature_repos. Three files still referenced the old path: - mongodb_repo_configuration.py: IntegrationTestRepoConfig and MongoDBOnlineStoreCreator imports - tests/.../online_store/mongodb.py: OnlineStoreCreator import - tests/unit/online_store/test_mongodb_online_retrieval.py: TAGS import Updating all three unblocks test collection and allows make test-python-universal-mongodb-online to run locally. Signed-off-by: Casey Clements <casey.clements@mongodb.com>
…-operator The Python ONLINE_STORE_CLASS_FOR_TYPE now includes 'mongodb', so the operator's parity check (test-datasources) fails unless the Go API also lists it. Add 'mongodb' to ValidOnlineStoreDBStorePersistenceTypes in both api/v1 and api/v1alpha1. Signed-off-by: Casey Clements <casey.clements@mongodb.com>
Add DriverInfo with the Feast name and version to both the sync MongoClient and async AsyncMongoClient so that MongoDB can identify traffic originating from a Feast AI integration. Signed-off-by: Casey Clements <casey.clements@mongodb.com>
Signed-off-by: Casey Clements <casey.clements@mongodb.com>
…rePersistence The +kubebuilder:validation:Enum annotation on the Type field of OnlineStoreDBStorePersistence was not updated when mongodb was added to ValidOnlineStoreDBStorePersistenceTypes. This annotation drives CRD OpenAPI schema validation at Kubernetes admission time, so any FeatureStore CR specifying type: mongodb would be rejected by the API server. Updated both api/v1 and api/v1alpha1. Signed-off-by: Casey Clements <casey.clements@mongodb.com>
Signed-off-by: Casey Clements <casey.clements@mongodb.com>
…onsistent with sync counterpart Signed-off-by: Casey Clements <casey.clements@mongodb.com>
Signed-off-by: ntkathole <nikhilkathole2683@gmail.com>
Signed-off-by: Casey Clements <casey.clements@mongodb.com>
…ially identical. Signed-off-by: Casey Clements <casey.clements@mongodb.com>
cb9e1d6 to
e761e10
Compare
@ntkathole I've rebased all my commits from the latest master, and run tests locally. Please approve workflows and have a look. |
# [0.61.0](v0.60.0...v0.61.0) (2026-03-10) ### Bug Fixes * Add grpcio dependency group to transformation server Dockerfile ([2c2150a](2c2150a)) * Add https readiness check for rest-registry tests ([ea85e63](ea85e63)) * Add website build check for PRs and fix blog frontmatter YAML error ([#6079](#6079)) ([30a3a43](30a3a43)) * Added MLflow metric charts across feature selection ([#6080](#6080)) ([a403361](a403361)) * Check duplicate names for feature view across types ([#5999](#5999)) ([95b9af8](95b9af8)) * Fix integration tests ([#6046](#6046)) ([02d5548](02d5548)) * Fix non-specific label selector on metrics service ([a1a160d](a1a160d)) * Fixed IntegrityError on SqlRegistry ([#6047](#6047)) ([325e148](325e148)) * Fixed pre-commit check ([114b7db](114b7db)) * Fixed uv cache permission error for docker build on mac ([ad807be](ad807be)) * Fixes a `PydanticDeprecatedSince20` warning for trino_offline_store ([#5991](#5991)) ([abfd18a](abfd18a)) * Integration test failures ([#6040](#6040)) ([9165870](9165870)) * Ray offline store tests are duplicated across 3 workflows ([54f705a](54f705a)) * Reenable tests ([#6036](#6036)) ([82ee7f8](82ee7f8)) * Use commitlint pre-commit hook instead of a separate action ([35a81e7](35a81e7)) ### Features * Add complex type support (Map, JSON, Struct) with schema validation ([#5974](#5974)) ([1200dbf](1200dbf)) * Add materialization, feature freshness, request latency, and push metrics to feature server ([2c6be18](2c6be18)) * Add non-entity retrieval support for ClickHouse offline store ([4d08ddc](4d08ddc)), closes [#5835](#5835) * Add OnlineStore for MongoDB ([#6025](#6025)) ([bf4e3fa](bf4e3fa)), closes [golang/go#74462](golang/go#74462) * Added CodeQL SAST scanning and detect-secrets pre-commit hook ([547b516](547b516)) * Adding optional name to Aggregation (feast-dev[#5994](#5994)) ([#6083](#6083)) ([56469f7](56469f7)) * Feature Server High-Availability on Kubernetes ([#6028](#6028)) ([9c07b4c](9c07b4c)), closes [Hi#Availability](https://github.com/Hi/issues/Availability) [Hi#Availability](https://github.com/Hi/issues/Availability) * **go:** Implement metrics and tracing for http and grpc servers ([#5925](#5925)) ([2b4ec9a](2b4ec9a)) * Horizontal scaling support to the Feast operator ([#6000](#6000)) ([3ec13e6](3ec13e6)) * Making feature view source optional (feast-dev[#6074](#6074)) ([#6075](#6075)) ([76917b7](76917b7)) * Support arm docker build ([#6061](#6061)) ([1e1f5d9](1e1f5d9)) * Use orjson for faster JSON serialization in feature server ([6f5203a](6f5203a)) ### Performance Improvements * Optimize protobuf parsing in Redis online store ([#6023](#6023)) ([59dfdb8](59dfdb8)) * Optimize timestamp conversion in _convert_rows_to_protobuf ([33a2e95](33a2e95)) * Parallelize DynamoDB batch reads in sync online_read ([#6024](#6024)) ([9699944](9699944)) * Remove redundant entity key serialization in online_read ([d87283f](d87283f))
* Initial commit on INTPYTHON-297-MongoDB-Feast-Integration
Signed-off-by: Casey Clements <casey.clements@mongodb.com>
* Added mongodb to project.optional-dependencies in pyproject.toml. Now pymongo is found as extra
Signed-off-by: Casey Clements <casey.clements@mongodb.com>
* Checkpoint. Passing tests.unit.online_store.test_online_writes.TestOnlineWrites with default args of MongoDBOnlineStoreConfig. Lots to do.
Signed-off-by: Casey Clements <casey.clements@mongodb.com>
* Handle Nan in dfs for test_online_writes.py. Now all tests in the module pass
Signed-off-by: Casey Clements <casey.clements@mongodb.com>
* Removed suffix of implementation: mongodb_openai -> mongodb
Signed-off-by: Casey Clements <casey.clements@mongodb.com>
* Moved MongoDBOnlineStore to feast.infra.online_store.contrib
Signed-off-by: Casey Clements <casey.clements@mongodb.com>
* Formatting
Signed-off-by: Casey Clements <casey.clements@mongodb.com>
* Refactor online_read that converts bson to proto. Left two methods for now. simply and transformming
Signed-off-by: Casey Clements <casey.clements@mongodb.com>
* Remove file added early during discovery
Signed-off-by: Casey Clements <casey.clements@mongodb.com>
* Formatting
Signed-off-by: Casey Clements <casey.clements@mongodb.com>
* Added version of test that uses testcontainers mongodb instead of assuming one is running
Signed-off-by: Casey Clements <casey.clements@mongodb.com>
* Create Make target for universal tests
Signed-off-by: Casey Clements <casey.clements@mongodb.com>
* Cleanup
Signed-off-by: Casey Clements <casey.clements@mongodb.com>
* Removed temporary integration tests requiring one to spin up own mongodb server.
Signed-off-by: Casey Clements <casey.clements@mongodb.com>
* Format
Signed-off-by: Casey Clements <casey.clements@mongodb.com>
* Typing
Signed-off-by: Casey Clements <casey.clements@mongodb.com>
* Implemented ASync API and Tests
Signed-off-by: Casey Clements <casey.clements@mongodb.com>
* Removed offline store stubs. The first PR will only contain the OnlineStore
Signed-off-by: Casey Clements <casey.clements@mongodb.com>
* Moved mongodb_online_store out of cobtrib package.
Signed-off-by: Casey Clements <casey.clements@mongodb.com>
* Add documentation.
Signed-off-by: Casey Clements <casey.clements@mongodb.com>
* Cleanups and docstrings
Signed-off-by: Casey Clements <casey.clements@mongodb.com>
* Fixed another reference to contrib dir
Signed-off-by: Casey Clements <casey.clements@mongodb.com>
* Typos
Signed-off-by: Casey Clements <casey.clements@mongodb.com>
* Made _convert_raw_docs_to_proto staticmethods private
Signed-off-by: Casey Clements <casey.clements@mongodb.com>
* After benchmarking two alogithm for conevrting read results from bson to proto, removed the naive one. It was outcompeted 3X across dimensions
Signed-off-by: Casey Clements <casey.clements@mongodb.com>
* Add extra unit tests
Signed-off-by: Casey Clements <casey.clements@mongodb.com>
* Formatting
Signed-off-by: Casey Clements <casey.clements@mongodb.com>
* Fixes in pyproject.toml
Signed-off-by: Casey Clements <casey.clements@mongodb.com>
* Fixed Detect secrets false positives.
Signed-off-by: Casey Clements <casey.clements@mongodb.com>
* Update sdk/python/feast/infra/online_stores/mongodb_online_store/mongodb.py
Set _client and _collection to None upon synchronous teardown.
Co-authored-by: devin-ai-integration[bot] <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Signed-off-by: Casey Clements <casey.clements@mongodb.com>
* Fix CI: guard pymongo imports and skip test module when pymongo unavailable
In the unit-test-python CI job, pymongo is not installed (it is an optional
extra). This caused ModuleNotFoundError during pytest collection.
Two changes:
1. mongodb.py: wrap pymongo imports in try/except -> FeastExtrasDependencyImportError
(consistent with redis.py, dynamodb.py, datastore.py pattern)
2. test_mongodb_online_retrieval.py: add pytest.importorskip(\pymongo\) so
the entire test module is skipped gracefully when pymongo is absent
Signed-off-by: Casey Clements <casey.clements@mongodb.com>
* Fix: return (None, None) when entity doc exists but feature view was never written
MongoDB stores all feature views for an entity in a single document.
If FV 'driver_stats' is written, an entity doc exists for driver_1.
A subsequent read for FV 'pricing' (never written) was previously
returning (None, {all_features: ValueProto()}) - a truthy feature dict
with all-empty protos - instead of the correct (None, None) sentinel.
Feast and downstream callers use (None, None) to signal entity absence.
A non-None feature dict means 'entity found, values are null', which
causes different behaviour in the feature retrieval pipeline.
Fix: after confirming the entity doc exists, also check that the
feature view key is present in doc['features']. If absent, return
(None, None) rather than a dict of empty protos.
Adds test_convert_raw_docs_entity_exists_but_fv_not_written to cover
the multi-feature-view colocation scenario identified in code review.
Signed-off-by: Casey Clements <casey.clements@mongodb.com>
* Update pixi.lock after adding mongodb optional dependency to pyproject.toml
Any change to pyproject.toml invalidates the pixi.lock manifest hash,
causing 'pixi install --locked' to fail in CI even when the changed
section (mongodb optional extras) is not used by any pixi environment.
Regenerated with: pixi install (v0.63.2)
Signed-off-by: Casey Clements <casey.clements@mongodb.com>
* fix: catch FeastExtrasDependencyImportError in doctest runner
Stores like mongodb, redis, and dynamodb raise FeastExtrasDependencyImportError
at import time when their optional Python extras are not installed.
test_all.py only caught ModuleNotFoundError, so any such import caused the
entire test_docstrings() function to abort rather than gracefully skipping
the module.
Extend the except clause to also catch FeastExtrasDependencyImportError so
the doctest run completes for all other modules when an optional extra is
absent in the test environment.
Signed-off-by: Casey Clements <casey.clements@mongodb.com>
* fix: update PYTEST_PLUGINS path for mongodb online store tests
Directory was renamed from tests/integration/feature_repos to
tests/universal/feature_repos in upstream/master. Update the
PYTEST_PLUGINS module path in test-python-universal-mongodb-online
to match the new location.
Signed-off-by: Casey Clements <casey.clements@mongodb.com>
* fix: broaden import exception handling in doctest runner to catch TypeError and other third-party import errors
qdrant_client raises TypeError at import time when a gRPC EnumTypeWrapper
is used with the | union operator on Python < 3.12. The previous fix only
caught ModuleNotFoundError and FeastExtrasDependencyImportError, leaving
the test runner vulnerable to any other exception raised by third-party
libraries during pkgutil.walk_packages.
Changes:
- Catch bare Exception in the import try/except block so any import-time
error from an optional dependency causes a graceful skip rather than an
abort of the entire test run.
- Initialize temp_module = None before the try block and add a continue
guard so that a failed import never leaves a stale module reference to
be used in the subsequent doctest execution block.
Signed-off-by: Casey Clements <casey.clements@mongodb.com>
* fix: pass onerror to pkgutil.walk_packages to suppress non-ImportError package import failures
FeastExtrasDependencyImportError inherits from FeastError -> Exception, not
from ImportError. pkgutil.walk_packages calls __import__ internally when
recursing into sub-packages, and only silently swallows ImportError subclasses;
any other exception is re-raised, crashing the entire test run.
Passing onerror=lambda _: None tells walk_packages to skip any package that
fails to import during the walk phase, regardless of the exception type. The
inner importlib.import_module try/except already handles the same errors for
the explicit import step used to collect doctests.
Signed-off-by: Casey Clements <casey.clements@mongodb.com>
* fix: update stale tests.integration.feature_repos imports to tests.universal.feature_repos
The upstream directory tests/integration/feature_repos was renamed to
tests/universal/feature_repos. Three files still referenced the old path:
- mongodb_repo_configuration.py: IntegrationTestRepoConfig and
MongoDBOnlineStoreCreator imports
- tests/.../online_store/mongodb.py: OnlineStoreCreator import
- tests/unit/online_store/test_mongodb_online_retrieval.py: TAGS import
Updating all three unblocks test collection and allows
make test-python-universal-mongodb-online to run locally.
Signed-off-by: Casey Clements <casey.clements@mongodb.com>
* feat: add mongodb to ValidOnlineStoreDBStorePersistenceTypes in feast-operator
The Python ONLINE_STORE_CLASS_FOR_TYPE now includes 'mongodb', so the
operator's parity check (test-datasources) fails unless the Go API also
lists it. Add 'mongodb' to ValidOnlineStoreDBStorePersistenceTypes in
both api/v1 and api/v1alpha1.
Signed-off-by: Casey Clements <casey.clements@mongodb.com>
* feat: add Feast driver metadata to MongoDB client instantiations
Add DriverInfo with the Feast name and version to both the sync
MongoClient and async AsyncMongoClient so that MongoDB can identify
traffic originating from a Feast AI integration.
Signed-off-by: Casey Clements <casey.clements@mongodb.com>
* docs: update MongoDB online store status from alpha to preview
Signed-off-by: Casey Clements <casey.clements@mongodb.com>
* fix: add mongodb to kubebuilder Enum annotations for OnlineStoreDBStorePersistence
The +kubebuilder:validation:Enum annotation on the Type field of
OnlineStoreDBStorePersistence was not updated when mongodb was added to
ValidOnlineStoreDBStorePersistenceTypes. This annotation drives CRD
OpenAPI schema validation at Kubernetes admission time, so any
FeatureStore CR specifying type: mongodb would be rejected by the API
server. Updated both api/v1 and api/v1alpha1.
Signed-off-by: Casey Clements <casey.clements@mongodb.com>
* Update +GOLANGCI_LINT_VERSION to fix upstream issue golang/go#74462
Signed-off-by: Casey Clements <casey.clements@mongodb.com>
* fix: raise ValueError in _get_client_async for invalid config type, consistent with sync counterpart
Signed-off-by: Casey Clements <casey.clements@mongodb.com>
* fix: Added mongodb to operator yamls
Signed-off-by: ntkathole <nikhilkathole2683@gmail.com>
* Small change suggested by ntkathole
Signed-off-by: Casey Clements <casey.clements@mongodb.com>
* Factor out write logic into utility function making sync/async essentially identical.
Signed-off-by: Casey Clements <casey.clements@mongodb.com>
---------
Signed-off-by: Casey Clements <casey.clements@mongodb.com>
Signed-off-by: ntkathole <nikhilkathole2683@gmail.com>
Co-authored-by: devin-ai-integration[bot] <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Co-authored-by: ntkathole <nikhilkathole2683@gmail.com>
# [0.61.0](feast-dev/feast@v0.60.0...v0.61.0) (2026-03-10) ### Bug Fixes * Add grpcio dependency group to transformation server Dockerfile ([2c2150a](feast-dev@2c2150a)) * Add https readiness check for rest-registry tests ([ea85e63](feast-dev@ea85e63)) * Add website build check for PRs and fix blog frontmatter YAML error ([feast-dev#6079](feast-dev#6079)) ([30a3a43](feast-dev@30a3a43)) * Added MLflow metric charts across feature selection ([feast-dev#6080](feast-dev#6080)) ([a403361](feast-dev@a403361)) * Check duplicate names for feature view across types ([feast-dev#5999](feast-dev#5999)) ([95b9af8](feast-dev@95b9af8)) * Fix integration tests ([feast-dev#6046](feast-dev#6046)) ([02d5548](feast-dev@02d5548)) * Fix non-specific label selector on metrics service ([a1a160d](feast-dev@a1a160d)) * Fixed IntegrityError on SqlRegistry ([feast-dev#6047](feast-dev#6047)) ([325e148](feast-dev@325e148)) * Fixed pre-commit check ([114b7db](feast-dev@114b7db)) * Fixed uv cache permission error for docker build on mac ([ad807be](feast-dev@ad807be)) * Fixes a `PydanticDeprecatedSince20` warning for trino_offline_store ([feast-dev#5991](feast-dev#5991)) ([abfd18a](feast-dev@abfd18a)) * Integration test failures ([feast-dev#6040](feast-dev#6040)) ([9165870](feast-dev@9165870)) * Ray offline store tests are duplicated across 3 workflows ([54f705a](feast-dev@54f705a)) * Reenable tests ([feast-dev#6036](feast-dev#6036)) ([82ee7f8](feast-dev@82ee7f8)) * Use commitlint pre-commit hook instead of a separate action ([35a81e7](feast-dev@35a81e7)) ### Features * Add complex type support (Map, JSON, Struct) with schema validation ([feast-dev#5974](feast-dev#5974)) ([1200dbf](feast-dev@1200dbf)) * Add materialization, feature freshness, request latency, and push metrics to feature server ([2c6be18](feast-dev@2c6be18)) * Add non-entity retrieval support for ClickHouse offline store ([4d08ddc](feast-dev@4d08ddc)), closes [feast-dev#5835](feast-dev#5835) * Add OnlineStore for MongoDB ([feast-dev#6025](feast-dev#6025)) ([bf4e3fa](feast-dev@bf4e3fa)), closes [golang/go#74462](golang/go#74462) * Added CodeQL SAST scanning and detect-secrets pre-commit hook ([547b516](feast-dev@547b516)) * Adding optional name to Aggregation (feast-dev[feast-dev#5994](feast-dev#5994)) ([feast-dev#6083](feast-dev#6083)) ([56469f7](feast-dev@56469f7)) * Feature Server High-Availability on Kubernetes ([feast-dev#6028](feast-dev#6028)) ([9c07b4c](feast-dev@9c07b4c)), closes [Hi#Availability](https://github.com/Hi/issues/Availability) [Hi#Availability](https://github.com/Hi/issues/Availability) * **go:** Implement metrics and tracing for http and grpc servers ([feast-dev#5925](feast-dev#5925)) ([2b4ec9a](feast-dev@2b4ec9a)) * Horizontal scaling support to the Feast operator ([feast-dev#6000](feast-dev#6000)) ([3ec13e6](feast-dev@3ec13e6)) * Making feature view source optional (feast-dev[feast-dev#6074](feast-dev#6074)) ([feast-dev#6075](feast-dev#6075)) ([76917b7](feast-dev@76917b7)) * Support arm docker build ([feast-dev#6061](feast-dev#6061)) ([1e1f5d9](feast-dev@1e1f5d9)) * Use orjson for faster JSON serialization in feature server ([6f5203a](feast-dev@6f5203a)) ### Performance Improvements * Optimize protobuf parsing in Redis online store ([feast-dev#6023](feast-dev#6023)) ([59dfdb8](feast-dev@59dfdb8)) * Optimize timestamp conversion in _convert_rows_to_protobuf ([33a2e95](feast-dev@33a2e95)) * Parallelize DynamoDB batch reads in sync online_read ([feast-dev#6024](feast-dev#6024)) ([9699944](feast-dev@9699944)) * Remove redundant entity key serialization in online_read ([d87283f](feast-dev@d87283f))
* Initial commit on INTPYTHON-297-MongoDB-Feast-Integration
Signed-off-by: Casey Clements <casey.clements@mongodb.com>
* Added mongodb to project.optional-dependencies in pyproject.toml. Now pymongo is found as extra
Signed-off-by: Casey Clements <casey.clements@mongodb.com>
* Checkpoint. Passing tests.unit.online_store.test_online_writes.TestOnlineWrites with default args of MongoDBOnlineStoreConfig. Lots to do.
Signed-off-by: Casey Clements <casey.clements@mongodb.com>
* Handle Nan in dfs for test_online_writes.py. Now all tests in the module pass
Signed-off-by: Casey Clements <casey.clements@mongodb.com>
* Removed suffix of implementation: mongodb_openai -> mongodb
Signed-off-by: Casey Clements <casey.clements@mongodb.com>
* Moved MongoDBOnlineStore to feast.infra.online_store.contrib
Signed-off-by: Casey Clements <casey.clements@mongodb.com>
* Formatting
Signed-off-by: Casey Clements <casey.clements@mongodb.com>
* Refactor online_read that converts bson to proto. Left two methods for now. simply and transformming
Signed-off-by: Casey Clements <casey.clements@mongodb.com>
* Remove file added early during discovery
Signed-off-by: Casey Clements <casey.clements@mongodb.com>
* Formatting
Signed-off-by: Casey Clements <casey.clements@mongodb.com>
* Added version of test that uses testcontainers mongodb instead of assuming one is running
Signed-off-by: Casey Clements <casey.clements@mongodb.com>
* Create Make target for universal tests
Signed-off-by: Casey Clements <casey.clements@mongodb.com>
* Cleanup
Signed-off-by: Casey Clements <casey.clements@mongodb.com>
* Removed temporary integration tests requiring one to spin up own mongodb server.
Signed-off-by: Casey Clements <casey.clements@mongodb.com>
* Format
Signed-off-by: Casey Clements <casey.clements@mongodb.com>
* Typing
Signed-off-by: Casey Clements <casey.clements@mongodb.com>
* Implemented ASync API and Tests
Signed-off-by: Casey Clements <casey.clements@mongodb.com>
* Removed offline store stubs. The first PR will only contain the OnlineStore
Signed-off-by: Casey Clements <casey.clements@mongodb.com>
* Moved mongodb_online_store out of cobtrib package.
Signed-off-by: Casey Clements <casey.clements@mongodb.com>
* Add documentation.
Signed-off-by: Casey Clements <casey.clements@mongodb.com>
* Cleanups and docstrings
Signed-off-by: Casey Clements <casey.clements@mongodb.com>
* Fixed another reference to contrib dir
Signed-off-by: Casey Clements <casey.clements@mongodb.com>
* Typos
Signed-off-by: Casey Clements <casey.clements@mongodb.com>
* Made _convert_raw_docs_to_proto staticmethods private
Signed-off-by: Casey Clements <casey.clements@mongodb.com>
* After benchmarking two alogithm for conevrting read results from bson to proto, removed the naive one. It was outcompeted 3X across dimensions
Signed-off-by: Casey Clements <casey.clements@mongodb.com>
* Add extra unit tests
Signed-off-by: Casey Clements <casey.clements@mongodb.com>
* Formatting
Signed-off-by: Casey Clements <casey.clements@mongodb.com>
* Fixes in pyproject.toml
Signed-off-by: Casey Clements <casey.clements@mongodb.com>
* Fixed Detect secrets false positives.
Signed-off-by: Casey Clements <casey.clements@mongodb.com>
* Update sdk/python/feast/infra/online_stores/mongodb_online_store/mongodb.py
Set _client and _collection to None upon synchronous teardown.
Co-authored-by: devin-ai-integration[bot] <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Signed-off-by: Casey Clements <casey.clements@mongodb.com>
* Fix CI: guard pymongo imports and skip test module when pymongo unavailable
In the unit-test-python CI job, pymongo is not installed (it is an optional
extra). This caused ModuleNotFoundError during pytest collection.
Two changes:
1. mongodb.py: wrap pymongo imports in try/except -> FeastExtrasDependencyImportError
(consistent with redis.py, dynamodb.py, datastore.py pattern)
2. test_mongodb_online_retrieval.py: add pytest.importorskip(\pymongo\) so
the entire test module is skipped gracefully when pymongo is absent
Signed-off-by: Casey Clements <casey.clements@mongodb.com>
* Fix: return (None, None) when entity doc exists but feature view was never written
MongoDB stores all feature views for an entity in a single document.
If FV 'driver_stats' is written, an entity doc exists for driver_1.
A subsequent read for FV 'pricing' (never written) was previously
returning (None, {all_features: ValueProto()}) - a truthy feature dict
with all-empty protos - instead of the correct (None, None) sentinel.
Feast and downstream callers use (None, None) to signal entity absence.
A non-None feature dict means 'entity found, values are null', which
causes different behaviour in the feature retrieval pipeline.
Fix: after confirming the entity doc exists, also check that the
feature view key is present in doc['features']. If absent, return
(None, None) rather than a dict of empty protos.
Adds test_convert_raw_docs_entity_exists_but_fv_not_written to cover
the multi-feature-view colocation scenario identified in code review.
Signed-off-by: Casey Clements <casey.clements@mongodb.com>
* Update pixi.lock after adding mongodb optional dependency to pyproject.toml
Any change to pyproject.toml invalidates the pixi.lock manifest hash,
causing 'pixi install --locked' to fail in CI even when the changed
section (mongodb optional extras) is not used by any pixi environment.
Regenerated with: pixi install (v0.63.2)
Signed-off-by: Casey Clements <casey.clements@mongodb.com>
* fix: catch FeastExtrasDependencyImportError in doctest runner
Stores like mongodb, redis, and dynamodb raise FeastExtrasDependencyImportError
at import time when their optional Python extras are not installed.
test_all.py only caught ModuleNotFoundError, so any such import caused the
entire test_docstrings() function to abort rather than gracefully skipping
the module.
Extend the except clause to also catch FeastExtrasDependencyImportError so
the doctest run completes for all other modules when an optional extra is
absent in the test environment.
Signed-off-by: Casey Clements <casey.clements@mongodb.com>
* fix: update PYTEST_PLUGINS path for mongodb online store tests
Directory was renamed from tests/integration/feature_repos to
tests/universal/feature_repos in upstream/master. Update the
PYTEST_PLUGINS module path in test-python-universal-mongodb-online
to match the new location.
Signed-off-by: Casey Clements <casey.clements@mongodb.com>
* fix: broaden import exception handling in doctest runner to catch TypeError and other third-party import errors
qdrant_client raises TypeError at import time when a gRPC EnumTypeWrapper
is used with the | union operator on Python < 3.12. The previous fix only
caught ModuleNotFoundError and FeastExtrasDependencyImportError, leaving
the test runner vulnerable to any other exception raised by third-party
libraries during pkgutil.walk_packages.
Changes:
- Catch bare Exception in the import try/except block so any import-time
error from an optional dependency causes a graceful skip rather than an
abort of the entire test run.
- Initialize temp_module = None before the try block and add a continue
guard so that a failed import never leaves a stale module reference to
be used in the subsequent doctest execution block.
Signed-off-by: Casey Clements <casey.clements@mongodb.com>
* fix: pass onerror to pkgutil.walk_packages to suppress non-ImportError package import failures
FeastExtrasDependencyImportError inherits from FeastError -> Exception, not
from ImportError. pkgutil.walk_packages calls __import__ internally when
recursing into sub-packages, and only silently swallows ImportError subclasses;
any other exception is re-raised, crashing the entire test run.
Passing onerror=lambda _: None tells walk_packages to skip any package that
fails to import during the walk phase, regardless of the exception type. The
inner importlib.import_module try/except already handles the same errors for
the explicit import step used to collect doctests.
Signed-off-by: Casey Clements <casey.clements@mongodb.com>
* fix: update stale tests.integration.feature_repos imports to tests.universal.feature_repos
The upstream directory tests/integration/feature_repos was renamed to
tests/universal/feature_repos. Three files still referenced the old path:
- mongodb_repo_configuration.py: IntegrationTestRepoConfig and
MongoDBOnlineStoreCreator imports
- tests/.../online_store/mongodb.py: OnlineStoreCreator import
- tests/unit/online_store/test_mongodb_online_retrieval.py: TAGS import
Updating all three unblocks test collection and allows
make test-python-universal-mongodb-online to run locally.
Signed-off-by: Casey Clements <casey.clements@mongodb.com>
* feat: add mongodb to ValidOnlineStoreDBStorePersistenceTypes in feast-operator
The Python ONLINE_STORE_CLASS_FOR_TYPE now includes 'mongodb', so the
operator's parity check (test-datasources) fails unless the Go API also
lists it. Add 'mongodb' to ValidOnlineStoreDBStorePersistenceTypes in
both api/v1 and api/v1alpha1.
Signed-off-by: Casey Clements <casey.clements@mongodb.com>
* feat: add Feast driver metadata to MongoDB client instantiations
Add DriverInfo with the Feast name and version to both the sync
MongoClient and async AsyncMongoClient so that MongoDB can identify
traffic originating from a Feast AI integration.
Signed-off-by: Casey Clements <casey.clements@mongodb.com>
* docs: update MongoDB online store status from alpha to preview
Signed-off-by: Casey Clements <casey.clements@mongodb.com>
* fix: add mongodb to kubebuilder Enum annotations for OnlineStoreDBStorePersistence
The +kubebuilder:validation:Enum annotation on the Type field of
OnlineStoreDBStorePersistence was not updated when mongodb was added to
ValidOnlineStoreDBStorePersistenceTypes. This annotation drives CRD
OpenAPI schema validation at Kubernetes admission time, so any
FeatureStore CR specifying type: mongodb would be rejected by the API
server. Updated both api/v1 and api/v1alpha1.
Signed-off-by: Casey Clements <casey.clements@mongodb.com>
* Update +GOLANGCI_LINT_VERSION to fix upstream issue golang/go#74462
Signed-off-by: Casey Clements <casey.clements@mongodb.com>
* fix: raise ValueError in _get_client_async for invalid config type, consistent with sync counterpart
Signed-off-by: Casey Clements <casey.clements@mongodb.com>
* fix: Added mongodb to operator yamls
Signed-off-by: ntkathole <nikhilkathole2683@gmail.com>
* Small change suggested by ntkathole
Signed-off-by: Casey Clements <casey.clements@mongodb.com>
* Factor out write logic into utility function making sync/async essentially identical.
Signed-off-by: Casey Clements <casey.clements@mongodb.com>
---------
Signed-off-by: Casey Clements <casey.clements@mongodb.com>
Signed-off-by: ntkathole <nikhilkathole2683@gmail.com>
Co-authored-by: devin-ai-integration[bot] <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Co-authored-by: ntkathole <nikhilkathole2683@gmail.com>
# [0.61.0](feast-dev/feast@v0.60.0...v0.61.0) (2026-03-10) ### Bug Fixes * Add grpcio dependency group to transformation server Dockerfile ([2c2150a](feast-dev@2c2150a)) * Add https readiness check for rest-registry tests ([ea85e63](feast-dev@ea85e63)) * Add website build check for PRs and fix blog frontmatter YAML error ([feast-dev#6079](feast-dev#6079)) ([30a3a43](feast-dev@30a3a43)) * Added MLflow metric charts across feature selection ([feast-dev#6080](feast-dev#6080)) ([a403361](feast-dev@a403361)) * Check duplicate names for feature view across types ([feast-dev#5999](feast-dev#5999)) ([95b9af8](feast-dev@95b9af8)) * Fix integration tests ([feast-dev#6046](feast-dev#6046)) ([02d5548](feast-dev@02d5548)) * Fix non-specific label selector on metrics service ([a1a160d](feast-dev@a1a160d)) * Fixed IntegrityError on SqlRegistry ([feast-dev#6047](feast-dev#6047)) ([325e148](feast-dev@325e148)) * Fixed pre-commit check ([114b7db](feast-dev@114b7db)) * Fixed uv cache permission error for docker build on mac ([ad807be](feast-dev@ad807be)) * Fixes a `PydanticDeprecatedSince20` warning for trino_offline_store ([feast-dev#5991](feast-dev#5991)) ([abfd18a](feast-dev@abfd18a)) * Integration test failures ([feast-dev#6040](feast-dev#6040)) ([9165870](feast-dev@9165870)) * Ray offline store tests are duplicated across 3 workflows ([54f705a](feast-dev@54f705a)) * Reenable tests ([feast-dev#6036](feast-dev#6036)) ([82ee7f8](feast-dev@82ee7f8)) * Use commitlint pre-commit hook instead of a separate action ([35a81e7](feast-dev@35a81e7)) ### Features * Add complex type support (Map, JSON, Struct) with schema validation ([feast-dev#5974](feast-dev#5974)) ([1200dbf](feast-dev@1200dbf)) * Add materialization, feature freshness, request latency, and push metrics to feature server ([2c6be18](feast-dev@2c6be18)) * Add non-entity retrieval support for ClickHouse offline store ([4d08ddc](feast-dev@4d08ddc)), closes [feast-dev#5835](feast-dev#5835) * Add OnlineStore for MongoDB ([feast-dev#6025](feast-dev#6025)) ([bf4e3fa](feast-dev@bf4e3fa)), closes [golang/go#74462](golang/go#74462) * Added CodeQL SAST scanning and detect-secrets pre-commit hook ([547b516](feast-dev@547b516)) * Adding optional name to Aggregation (feast-dev[feast-dev#5994](feast-dev#5994)) ([feast-dev#6083](feast-dev#6083)) ([56469f7](feast-dev@56469f7)) * Feature Server High-Availability on Kubernetes ([feast-dev#6028](feast-dev#6028)) ([9c07b4c](feast-dev@9c07b4c)), closes [Hi#Availability](https://github.com/Hi/issues/Availability) [Hi#Availability](https://github.com/Hi/issues/Availability) * **go:** Implement metrics and tracing for http and grpc servers ([feast-dev#5925](feast-dev#5925)) ([2b4ec9a](feast-dev@2b4ec9a)) * Horizontal scaling support to the Feast operator ([feast-dev#6000](feast-dev#6000)) ([3ec13e6](feast-dev@3ec13e6)) * Making feature view source optional (feast-dev[feast-dev#6074](feast-dev#6074)) ([feast-dev#6075](feast-dev#6075)) ([76917b7](feast-dev@76917b7)) * Support arm docker build ([feast-dev#6061](feast-dev#6061)) ([1e1f5d9](feast-dev@1e1f5d9)) * Use orjson for faster JSON serialization in feature server ([6f5203a](feast-dev@6f5203a)) ### Performance Improvements * Optimize protobuf parsing in Redis online store ([feast-dev#6023](feast-dev#6023)) ([59dfdb8](feast-dev@59dfdb8)) * Optimize timestamp conversion in _convert_rows_to_protobuf ([33a2e95](feast-dev@33a2e95)) * Parallelize DynamoDB batch reads in sync online_read ([feast-dev#6024](feast-dev#6024)) ([9699944](feast-dev@9699944)) * Remove redundant entity key serialization in online_read ([d87283f](feast-dev@d87283f))
* Initial commit on INTPYTHON-297-MongoDB-Feast-Integration
Signed-off-by: Casey Clements <casey.clements@mongodb.com>
* Added mongodb to project.optional-dependencies in pyproject.toml. Now pymongo is found as extra
Signed-off-by: Casey Clements <casey.clements@mongodb.com>
* Checkpoint. Passing tests.unit.online_store.test_online_writes.TestOnlineWrites with default args of MongoDBOnlineStoreConfig. Lots to do.
Signed-off-by: Casey Clements <casey.clements@mongodb.com>
* Handle Nan in dfs for test_online_writes.py. Now all tests in the module pass
Signed-off-by: Casey Clements <casey.clements@mongodb.com>
* Removed suffix of implementation: mongodb_openai -> mongodb
Signed-off-by: Casey Clements <casey.clements@mongodb.com>
* Moved MongoDBOnlineStore to feast.infra.online_store.contrib
Signed-off-by: Casey Clements <casey.clements@mongodb.com>
* Formatting
Signed-off-by: Casey Clements <casey.clements@mongodb.com>
* Refactor online_read that converts bson to proto. Left two methods for now. simply and transformming
Signed-off-by: Casey Clements <casey.clements@mongodb.com>
* Remove file added early during discovery
Signed-off-by: Casey Clements <casey.clements@mongodb.com>
* Formatting
Signed-off-by: Casey Clements <casey.clements@mongodb.com>
* Added version of test that uses testcontainers mongodb instead of assuming one is running
Signed-off-by: Casey Clements <casey.clements@mongodb.com>
* Create Make target for universal tests
Signed-off-by: Casey Clements <casey.clements@mongodb.com>
* Cleanup
Signed-off-by: Casey Clements <casey.clements@mongodb.com>
* Removed temporary integration tests requiring one to spin up own mongodb server.
Signed-off-by: Casey Clements <casey.clements@mongodb.com>
* Format
Signed-off-by: Casey Clements <casey.clements@mongodb.com>
* Typing
Signed-off-by: Casey Clements <casey.clements@mongodb.com>
* Implemented ASync API and Tests
Signed-off-by: Casey Clements <casey.clements@mongodb.com>
* Removed offline store stubs. The first PR will only contain the OnlineStore
Signed-off-by: Casey Clements <casey.clements@mongodb.com>
* Moved mongodb_online_store out of cobtrib package.
Signed-off-by: Casey Clements <casey.clements@mongodb.com>
* Add documentation.
Signed-off-by: Casey Clements <casey.clements@mongodb.com>
* Cleanups and docstrings
Signed-off-by: Casey Clements <casey.clements@mongodb.com>
* Fixed another reference to contrib dir
Signed-off-by: Casey Clements <casey.clements@mongodb.com>
* Typos
Signed-off-by: Casey Clements <casey.clements@mongodb.com>
* Made _convert_raw_docs_to_proto staticmethods private
Signed-off-by: Casey Clements <casey.clements@mongodb.com>
* After benchmarking two alogithm for conevrting read results from bson to proto, removed the naive one. It was outcompeted 3X across dimensions
Signed-off-by: Casey Clements <casey.clements@mongodb.com>
* Add extra unit tests
Signed-off-by: Casey Clements <casey.clements@mongodb.com>
* Formatting
Signed-off-by: Casey Clements <casey.clements@mongodb.com>
* Fixes in pyproject.toml
Signed-off-by: Casey Clements <casey.clements@mongodb.com>
* Fixed Detect secrets false positives.
Signed-off-by: Casey Clements <casey.clements@mongodb.com>
* Update sdk/python/feast/infra/online_stores/mongodb_online_store/mongodb.py
Set _client and _collection to None upon synchronous teardown.
Co-authored-by: devin-ai-integration[bot] <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Signed-off-by: Casey Clements <casey.clements@mongodb.com>
* Fix CI: guard pymongo imports and skip test module when pymongo unavailable
In the unit-test-python CI job, pymongo is not installed (it is an optional
extra). This caused ModuleNotFoundError during pytest collection.
Two changes:
1. mongodb.py: wrap pymongo imports in try/except -> FeastExtrasDependencyImportError
(consistent with redis.py, dynamodb.py, datastore.py pattern)
2. test_mongodb_online_retrieval.py: add pytest.importorskip(\pymongo\) so
the entire test module is skipped gracefully when pymongo is absent
Signed-off-by: Casey Clements <casey.clements@mongodb.com>
* Fix: return (None, None) when entity doc exists but feature view was never written
MongoDB stores all feature views for an entity in a single document.
If FV 'driver_stats' is written, an entity doc exists for driver_1.
A subsequent read for FV 'pricing' (never written) was previously
returning (None, {all_features: ValueProto()}) - a truthy feature dict
with all-empty protos - instead of the correct (None, None) sentinel.
Feast and downstream callers use (None, None) to signal entity absence.
A non-None feature dict means 'entity found, values are null', which
causes different behaviour in the feature retrieval pipeline.
Fix: after confirming the entity doc exists, also check that the
feature view key is present in doc['features']. If absent, return
(None, None) rather than a dict of empty protos.
Adds test_convert_raw_docs_entity_exists_but_fv_not_written to cover
the multi-feature-view colocation scenario identified in code review.
Signed-off-by: Casey Clements <casey.clements@mongodb.com>
* Update pixi.lock after adding mongodb optional dependency to pyproject.toml
Any change to pyproject.toml invalidates the pixi.lock manifest hash,
causing 'pixi install --locked' to fail in CI even when the changed
section (mongodb optional extras) is not used by any pixi environment.
Regenerated with: pixi install (v0.63.2)
Signed-off-by: Casey Clements <casey.clements@mongodb.com>
* fix: catch FeastExtrasDependencyImportError in doctest runner
Stores like mongodb, redis, and dynamodb raise FeastExtrasDependencyImportError
at import time when their optional Python extras are not installed.
test_all.py only caught ModuleNotFoundError, so any such import caused the
entire test_docstrings() function to abort rather than gracefully skipping
the module.
Extend the except clause to also catch FeastExtrasDependencyImportError so
the doctest run completes for all other modules when an optional extra is
absent in the test environment.
Signed-off-by: Casey Clements <casey.clements@mongodb.com>
* fix: update PYTEST_PLUGINS path for mongodb online store tests
Directory was renamed from tests/integration/feature_repos to
tests/universal/feature_repos in upstream/master. Update the
PYTEST_PLUGINS module path in test-python-universal-mongodb-online
to match the new location.
Signed-off-by: Casey Clements <casey.clements@mongodb.com>
* fix: broaden import exception handling in doctest runner to catch TypeError and other third-party import errors
qdrant_client raises TypeError at import time when a gRPC EnumTypeWrapper
is used with the | union operator on Python < 3.12. The previous fix only
caught ModuleNotFoundError and FeastExtrasDependencyImportError, leaving
the test runner vulnerable to any other exception raised by third-party
libraries during pkgutil.walk_packages.
Changes:
- Catch bare Exception in the import try/except block so any import-time
error from an optional dependency causes a graceful skip rather than an
abort of the entire test run.
- Initialize temp_module = None before the try block and add a continue
guard so that a failed import never leaves a stale module reference to
be used in the subsequent doctest execution block.
Signed-off-by: Casey Clements <casey.clements@mongodb.com>
* fix: pass onerror to pkgutil.walk_packages to suppress non-ImportError package import failures
FeastExtrasDependencyImportError inherits from FeastError -> Exception, not
from ImportError. pkgutil.walk_packages calls __import__ internally when
recursing into sub-packages, and only silently swallows ImportError subclasses;
any other exception is re-raised, crashing the entire test run.
Passing onerror=lambda _: None tells walk_packages to skip any package that
fails to import during the walk phase, regardless of the exception type. The
inner importlib.import_module try/except already handles the same errors for
the explicit import step used to collect doctests.
Signed-off-by: Casey Clements <casey.clements@mongodb.com>
* fix: update stale tests.integration.feature_repos imports to tests.universal.feature_repos
The upstream directory tests/integration/feature_repos was renamed to
tests/universal/feature_repos. Three files still referenced the old path:
- mongodb_repo_configuration.py: IntegrationTestRepoConfig and
MongoDBOnlineStoreCreator imports
- tests/.../online_store/mongodb.py: OnlineStoreCreator import
- tests/unit/online_store/test_mongodb_online_retrieval.py: TAGS import
Updating all three unblocks test collection and allows
make test-python-universal-mongodb-online to run locally.
Signed-off-by: Casey Clements <casey.clements@mongodb.com>
* feat: add mongodb to ValidOnlineStoreDBStorePersistenceTypes in feast-operator
The Python ONLINE_STORE_CLASS_FOR_TYPE now includes 'mongodb', so the
operator's parity check (test-datasources) fails unless the Go API also
lists it. Add 'mongodb' to ValidOnlineStoreDBStorePersistenceTypes in
both api/v1 and api/v1alpha1.
Signed-off-by: Casey Clements <casey.clements@mongodb.com>
* feat: add Feast driver metadata to MongoDB client instantiations
Add DriverInfo with the Feast name and version to both the sync
MongoClient and async AsyncMongoClient so that MongoDB can identify
traffic originating from a Feast AI integration.
Signed-off-by: Casey Clements <casey.clements@mongodb.com>
* docs: update MongoDB online store status from alpha to preview
Signed-off-by: Casey Clements <casey.clements@mongodb.com>
* fix: add mongodb to kubebuilder Enum annotations for OnlineStoreDBStorePersistence
The +kubebuilder:validation:Enum annotation on the Type field of
OnlineStoreDBStorePersistence was not updated when mongodb was added to
ValidOnlineStoreDBStorePersistenceTypes. This annotation drives CRD
OpenAPI schema validation at Kubernetes admission time, so any
FeatureStore CR specifying type: mongodb would be rejected by the API
server. Updated both api/v1 and api/v1alpha1.
Signed-off-by: Casey Clements <casey.clements@mongodb.com>
* Update +GOLANGCI_LINT_VERSION to fix upstream issue golang/go#74462
Signed-off-by: Casey Clements <casey.clements@mongodb.com>
* fix: raise ValueError in _get_client_async for invalid config type, consistent with sync counterpart
Signed-off-by: Casey Clements <casey.clements@mongodb.com>
* fix: Added mongodb to operator yamls
Signed-off-by: ntkathole <nikhilkathole2683@gmail.com>
* Small change suggested by ntkathole
Signed-off-by: Casey Clements <casey.clements@mongodb.com>
* Factor out write logic into utility function making sync/async essentially identical.
Signed-off-by: Casey Clements <casey.clements@mongodb.com>
---------
Signed-off-by: Casey Clements <casey.clements@mongodb.com>
Signed-off-by: ntkathole <nikhilkathole2683@gmail.com>
Co-authored-by: devin-ai-integration[bot] <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Co-authored-by: ntkathole <nikhilkathole2683@gmail.com>
Signed-off-by: Shizoqua <hr.lanreshittu@gmail.com>
# [0.61.0](feast-dev/feast@v0.60.0...v0.61.0) (2026-03-10) ### Bug Fixes * Add grpcio dependency group to transformation server Dockerfile ([2c2150a](feast-dev@2c2150a)) * Add https readiness check for rest-registry tests ([ea85e63](feast-dev@ea85e63)) * Add website build check for PRs and fix blog frontmatter YAML error ([feast-dev#6079](feast-dev#6079)) ([30a3a43](feast-dev@30a3a43)) * Added MLflow metric charts across feature selection ([feast-dev#6080](feast-dev#6080)) ([a403361](feast-dev@a403361)) * Check duplicate names for feature view across types ([feast-dev#5999](feast-dev#5999)) ([95b9af8](feast-dev@95b9af8)) * Fix integration tests ([feast-dev#6046](feast-dev#6046)) ([02d5548](feast-dev@02d5548)) * Fix non-specific label selector on metrics service ([a1a160d](feast-dev@a1a160d)) * Fixed IntegrityError on SqlRegistry ([feast-dev#6047](feast-dev#6047)) ([325e148](feast-dev@325e148)) * Fixed pre-commit check ([114b7db](feast-dev@114b7db)) * Fixed uv cache permission error for docker build on mac ([ad807be](feast-dev@ad807be)) * Fixes a `PydanticDeprecatedSince20` warning for trino_offline_store ([feast-dev#5991](feast-dev#5991)) ([abfd18a](feast-dev@abfd18a)) * Integration test failures ([feast-dev#6040](feast-dev#6040)) ([9165870](feast-dev@9165870)) * Ray offline store tests are duplicated across 3 workflows ([54f705a](feast-dev@54f705a)) * Reenable tests ([feast-dev#6036](feast-dev#6036)) ([82ee7f8](feast-dev@82ee7f8)) * Use commitlint pre-commit hook instead of a separate action ([35a81e7](feast-dev@35a81e7)) ### Features * Add complex type support (Map, JSON, Struct) with schema validation ([feast-dev#5974](feast-dev#5974)) ([1200dbf](feast-dev@1200dbf)) * Add materialization, feature freshness, request latency, and push metrics to feature server ([2c6be18](feast-dev@2c6be18)) * Add non-entity retrieval support for ClickHouse offline store ([4d08ddc](feast-dev@4d08ddc)), closes [feast-dev#5835](feast-dev#5835) * Add OnlineStore for MongoDB ([feast-dev#6025](feast-dev#6025)) ([bf4e3fa](feast-dev@bf4e3fa)), closes [golang/go#74462](golang/go#74462) * Added CodeQL SAST scanning and detect-secrets pre-commit hook ([547b516](feast-dev@547b516)) * Adding optional name to Aggregation (feast-dev[feast-dev#5994](feast-dev#5994)) ([feast-dev#6083](feast-dev#6083)) ([56469f7](feast-dev@56469f7)) * Feature Server High-Availability on Kubernetes ([feast-dev#6028](feast-dev#6028)) ([9c07b4c](feast-dev@9c07b4c)), closes [Hi#Availability](https://github.com/Hi/issues/Availability) [Hi#Availability](https://github.com/Hi/issues/Availability) * **go:** Implement metrics and tracing for http and grpc servers ([feast-dev#5925](feast-dev#5925)) ([2b4ec9a](feast-dev@2b4ec9a)) * Horizontal scaling support to the Feast operator ([feast-dev#6000](feast-dev#6000)) ([3ec13e6](feast-dev@3ec13e6)) * Making feature view source optional (feast-dev[feast-dev#6074](feast-dev#6074)) ([feast-dev#6075](feast-dev#6075)) ([76917b7](feast-dev@76917b7)) * Support arm docker build ([feast-dev#6061](feast-dev#6061)) ([1e1f5d9](feast-dev@1e1f5d9)) * Use orjson for faster JSON serialization in feature server ([6f5203a](feast-dev@6f5203a)) ### Performance Improvements * Optimize protobuf parsing in Redis online store ([feast-dev#6023](feast-dev#6023)) ([59dfdb8](feast-dev@59dfdb8)) * Optimize timestamp conversion in _convert_rows_to_protobuf ([33a2e95](feast-dev@33a2e95)) * Parallelize DynamoDB batch reads in sync online_read ([feast-dev#6024](feast-dev#6024)) ([9699944](feast-dev@9699944)) * Remove redundant entity key serialization in online_read ([d87283f](feast-dev@d87283f)) Signed-off-by: Shizoqua <hr.lanreshittu@gmail.com>
* Initial commit on INTPYTHON-297-MongoDB-Feast-Integration
Signed-off-by: Casey Clements <casey.clements@mongodb.com>
* Added mongodb to project.optional-dependencies in pyproject.toml. Now pymongo is found as extra
Signed-off-by: Casey Clements <casey.clements@mongodb.com>
* Checkpoint. Passing tests.unit.online_store.test_online_writes.TestOnlineWrites with default args of MongoDBOnlineStoreConfig. Lots to do.
Signed-off-by: Casey Clements <casey.clements@mongodb.com>
* Handle Nan in dfs for test_online_writes.py. Now all tests in the module pass
Signed-off-by: Casey Clements <casey.clements@mongodb.com>
* Removed suffix of implementation: mongodb_openai -> mongodb
Signed-off-by: Casey Clements <casey.clements@mongodb.com>
* Moved MongoDBOnlineStore to feast.infra.online_store.contrib
Signed-off-by: Casey Clements <casey.clements@mongodb.com>
* Formatting
Signed-off-by: Casey Clements <casey.clements@mongodb.com>
* Refactor online_read that converts bson to proto. Left two methods for now. simply and transformming
Signed-off-by: Casey Clements <casey.clements@mongodb.com>
* Remove file added early during discovery
Signed-off-by: Casey Clements <casey.clements@mongodb.com>
* Formatting
Signed-off-by: Casey Clements <casey.clements@mongodb.com>
* Added version of test that uses testcontainers mongodb instead of assuming one is running
Signed-off-by: Casey Clements <casey.clements@mongodb.com>
* Create Make target for universal tests
Signed-off-by: Casey Clements <casey.clements@mongodb.com>
* Cleanup
Signed-off-by: Casey Clements <casey.clements@mongodb.com>
* Removed temporary integration tests requiring one to spin up own mongodb server.
Signed-off-by: Casey Clements <casey.clements@mongodb.com>
* Format
Signed-off-by: Casey Clements <casey.clements@mongodb.com>
* Typing
Signed-off-by: Casey Clements <casey.clements@mongodb.com>
* Implemented ASync API and Tests
Signed-off-by: Casey Clements <casey.clements@mongodb.com>
* Removed offline store stubs. The first PR will only contain the OnlineStore
Signed-off-by: Casey Clements <casey.clements@mongodb.com>
* Moved mongodb_online_store out of cobtrib package.
Signed-off-by: Casey Clements <casey.clements@mongodb.com>
* Add documentation.
Signed-off-by: Casey Clements <casey.clements@mongodb.com>
* Cleanups and docstrings
Signed-off-by: Casey Clements <casey.clements@mongodb.com>
* Fixed another reference to contrib dir
Signed-off-by: Casey Clements <casey.clements@mongodb.com>
* Typos
Signed-off-by: Casey Clements <casey.clements@mongodb.com>
* Made _convert_raw_docs_to_proto staticmethods private
Signed-off-by: Casey Clements <casey.clements@mongodb.com>
* After benchmarking two alogithm for conevrting read results from bson to proto, removed the naive one. It was outcompeted 3X across dimensions
Signed-off-by: Casey Clements <casey.clements@mongodb.com>
* Add extra unit tests
Signed-off-by: Casey Clements <casey.clements@mongodb.com>
* Formatting
Signed-off-by: Casey Clements <casey.clements@mongodb.com>
* Fixes in pyproject.toml
Signed-off-by: Casey Clements <casey.clements@mongodb.com>
* Fixed Detect secrets false positives.
Signed-off-by: Casey Clements <casey.clements@mongodb.com>
* Update sdk/python/feast/infra/online_stores/mongodb_online_store/mongodb.py
Set _client and _collection to None upon synchronous teardown.
Co-authored-by: devin-ai-integration[bot] <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Signed-off-by: Casey Clements <casey.clements@mongodb.com>
* Fix CI: guard pymongo imports and skip test module when pymongo unavailable
In the unit-test-python CI job, pymongo is not installed (it is an optional
extra). This caused ModuleNotFoundError during pytest collection.
Two changes:
1. mongodb.py: wrap pymongo imports in try/except -> FeastExtrasDependencyImportError
(consistent with redis.py, dynamodb.py, datastore.py pattern)
2. test_mongodb_online_retrieval.py: add pytest.importorskip(\pymongo\) so
the entire test module is skipped gracefully when pymongo is absent
Signed-off-by: Casey Clements <casey.clements@mongodb.com>
* Fix: return (None, None) when entity doc exists but feature view was never written
MongoDB stores all feature views for an entity in a single document.
If FV 'driver_stats' is written, an entity doc exists for driver_1.
A subsequent read for FV 'pricing' (never written) was previously
returning (None, {all_features: ValueProto()}) - a truthy feature dict
with all-empty protos - instead of the correct (None, None) sentinel.
Feast and downstream callers use (None, None) to signal entity absence.
A non-None feature dict means 'entity found, values are null', which
causes different behaviour in the feature retrieval pipeline.
Fix: after confirming the entity doc exists, also check that the
feature view key is present in doc['features']. If absent, return
(None, None) rather than a dict of empty protos.
Adds test_convert_raw_docs_entity_exists_but_fv_not_written to cover
the multi-feature-view colocation scenario identified in code review.
Signed-off-by: Casey Clements <casey.clements@mongodb.com>
* Update pixi.lock after adding mongodb optional dependency to pyproject.toml
Any change to pyproject.toml invalidates the pixi.lock manifest hash,
causing 'pixi install --locked' to fail in CI even when the changed
section (mongodb optional extras) is not used by any pixi environment.
Regenerated with: pixi install (v0.63.2)
Signed-off-by: Casey Clements <casey.clements@mongodb.com>
* fix: catch FeastExtrasDependencyImportError in doctest runner
Stores like mongodb, redis, and dynamodb raise FeastExtrasDependencyImportError
at import time when their optional Python extras are not installed.
test_all.py only caught ModuleNotFoundError, so any such import caused the
entire test_docstrings() function to abort rather than gracefully skipping
the module.
Extend the except clause to also catch FeastExtrasDependencyImportError so
the doctest run completes for all other modules when an optional extra is
absent in the test environment.
Signed-off-by: Casey Clements <casey.clements@mongodb.com>
* fix: update PYTEST_PLUGINS path for mongodb online store tests
Directory was renamed from tests/integration/feature_repos to
tests/universal/feature_repos in upstream/master. Update the
PYTEST_PLUGINS module path in test-python-universal-mongodb-online
to match the new location.
Signed-off-by: Casey Clements <casey.clements@mongodb.com>
* fix: broaden import exception handling in doctest runner to catch TypeError and other third-party import errors
qdrant_client raises TypeError at import time when a gRPC EnumTypeWrapper
is used with the | union operator on Python < 3.12. The previous fix only
caught ModuleNotFoundError and FeastExtrasDependencyImportError, leaving
the test runner vulnerable to any other exception raised by third-party
libraries during pkgutil.walk_packages.
Changes:
- Catch bare Exception in the import try/except block so any import-time
error from an optional dependency causes a graceful skip rather than an
abort of the entire test run.
- Initialize temp_module = None before the try block and add a continue
guard so that a failed import never leaves a stale module reference to
be used in the subsequent doctest execution block.
Signed-off-by: Casey Clements <casey.clements@mongodb.com>
* fix: pass onerror to pkgutil.walk_packages to suppress non-ImportError package import failures
FeastExtrasDependencyImportError inherits from FeastError -> Exception, not
from ImportError. pkgutil.walk_packages calls __import__ internally when
recursing into sub-packages, and only silently swallows ImportError subclasses;
any other exception is re-raised, crashing the entire test run.
Passing onerror=lambda _: None tells walk_packages to skip any package that
fails to import during the walk phase, regardless of the exception type. The
inner importlib.import_module try/except already handles the same errors for
the explicit import step used to collect doctests.
Signed-off-by: Casey Clements <casey.clements@mongodb.com>
* fix: update stale tests.integration.feature_repos imports to tests.universal.feature_repos
The upstream directory tests/integration/feature_repos was renamed to
tests/universal/feature_repos. Three files still referenced the old path:
- mongodb_repo_configuration.py: IntegrationTestRepoConfig and
MongoDBOnlineStoreCreator imports
- tests/.../online_store/mongodb.py: OnlineStoreCreator import
- tests/unit/online_store/test_mongodb_online_retrieval.py: TAGS import
Updating all three unblocks test collection and allows
make test-python-universal-mongodb-online to run locally.
Signed-off-by: Casey Clements <casey.clements@mongodb.com>
* feat: add mongodb to ValidOnlineStoreDBStorePersistenceTypes in feast-operator
The Python ONLINE_STORE_CLASS_FOR_TYPE now includes 'mongodb', so the
operator's parity check (test-datasources) fails unless the Go API also
lists it. Add 'mongodb' to ValidOnlineStoreDBStorePersistenceTypes in
both api/v1 and api/v1alpha1.
Signed-off-by: Casey Clements <casey.clements@mongodb.com>
* feat: add Feast driver metadata to MongoDB client instantiations
Add DriverInfo with the Feast name and version to both the sync
MongoClient and async AsyncMongoClient so that MongoDB can identify
traffic originating from a Feast AI integration.
Signed-off-by: Casey Clements <casey.clements@mongodb.com>
* docs: update MongoDB online store status from alpha to preview
Signed-off-by: Casey Clements <casey.clements@mongodb.com>
* fix: add mongodb to kubebuilder Enum annotations for OnlineStoreDBStorePersistence
The +kubebuilder:validation:Enum annotation on the Type field of
OnlineStoreDBStorePersistence was not updated when mongodb was added to
ValidOnlineStoreDBStorePersistenceTypes. This annotation drives CRD
OpenAPI schema validation at Kubernetes admission time, so any
FeatureStore CR specifying type: mongodb would be rejected by the API
server. Updated both api/v1 and api/v1alpha1.
Signed-off-by: Casey Clements <casey.clements@mongodb.com>
* Update +GOLANGCI_LINT_VERSION to fix upstream issue golang/go#74462
Signed-off-by: Casey Clements <casey.clements@mongodb.com>
* fix: raise ValueError in _get_client_async for invalid config type, consistent with sync counterpart
Signed-off-by: Casey Clements <casey.clements@mongodb.com>
* fix: Added mongodb to operator yamls
Signed-off-by: ntkathole <nikhilkathole2683@gmail.com>
* Small change suggested by ntkathole
Signed-off-by: Casey Clements <casey.clements@mongodb.com>
* Factor out write logic into utility function making sync/async essentially identical.
Signed-off-by: Casey Clements <casey.clements@mongodb.com>
---------
Signed-off-by: Casey Clements <casey.clements@mongodb.com>
Signed-off-by: ntkathole <nikhilkathole2683@gmail.com>
Co-authored-by: devin-ai-integration[bot] <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Co-authored-by: ntkathole <nikhilkathole2683@gmail.com>
Signed-off-by: Aniket Paluskar <apaluska@redhat.com>
# [0.61.0](feast-dev/feast@v0.60.0...v0.61.0) (2026-03-10) ### Bug Fixes * Add grpcio dependency group to transformation server Dockerfile ([2c2150a](feast-dev@2c2150a)) * Add https readiness check for rest-registry tests ([ea85e63](feast-dev@ea85e63)) * Add website build check for PRs and fix blog frontmatter YAML error ([feast-dev#6079](feast-dev#6079)) ([30a3a43](feast-dev@30a3a43)) * Added MLflow metric charts across feature selection ([feast-dev#6080](feast-dev#6080)) ([a403361](feast-dev@a403361)) * Check duplicate names for feature view across types ([feast-dev#5999](feast-dev#5999)) ([95b9af8](feast-dev@95b9af8)) * Fix integration tests ([feast-dev#6046](feast-dev#6046)) ([02d5548](feast-dev@02d5548)) * Fix non-specific label selector on metrics service ([a1a160d](feast-dev@a1a160d)) * Fixed IntegrityError on SqlRegistry ([feast-dev#6047](feast-dev#6047)) ([325e148](feast-dev@325e148)) * Fixed pre-commit check ([114b7db](feast-dev@114b7db)) * Fixed uv cache permission error for docker build on mac ([ad807be](feast-dev@ad807be)) * Fixes a `PydanticDeprecatedSince20` warning for trino_offline_store ([feast-dev#5991](feast-dev#5991)) ([abfd18a](feast-dev@abfd18a)) * Integration test failures ([feast-dev#6040](feast-dev#6040)) ([9165870](feast-dev@9165870)) * Ray offline store tests are duplicated across 3 workflows ([54f705a](feast-dev@54f705a)) * Reenable tests ([feast-dev#6036](feast-dev#6036)) ([82ee7f8](feast-dev@82ee7f8)) * Use commitlint pre-commit hook instead of a separate action ([35a81e7](feast-dev@35a81e7)) ### Features * Add complex type support (Map, JSON, Struct) with schema validation ([feast-dev#5974](feast-dev#5974)) ([1200dbf](feast-dev@1200dbf)) * Add materialization, feature freshness, request latency, and push metrics to feature server ([2c6be18](feast-dev@2c6be18)) * Add non-entity retrieval support for ClickHouse offline store ([4d08ddc](feast-dev@4d08ddc)), closes [feast-dev#5835](feast-dev#5835) * Add OnlineStore for MongoDB ([feast-dev#6025](feast-dev#6025)) ([bf4e3fa](feast-dev@bf4e3fa)), closes [golang/go#74462](golang/go#74462) * Added CodeQL SAST scanning and detect-secrets pre-commit hook ([547b516](feast-dev@547b516)) * Adding optional name to Aggregation (feast-dev[feast-dev#5994](feast-dev#5994)) ([feast-dev#6083](feast-dev#6083)) ([56469f7](feast-dev@56469f7)) * Feature Server High-Availability on Kubernetes ([feast-dev#6028](feast-dev#6028)) ([9c07b4c](feast-dev@9c07b4c)), closes [Hi#Availability](https://github.com/Hi/issues/Availability) [Hi#Availability](https://github.com/Hi/issues/Availability) * **go:** Implement metrics and tracing for http and grpc servers ([feast-dev#5925](feast-dev#5925)) ([2b4ec9a](feast-dev@2b4ec9a)) * Horizontal scaling support to the Feast operator ([feast-dev#6000](feast-dev#6000)) ([3ec13e6](feast-dev@3ec13e6)) * Making feature view source optional (feast-dev[feast-dev#6074](feast-dev#6074)) ([feast-dev#6075](feast-dev#6075)) ([76917b7](feast-dev@76917b7)) * Support arm docker build ([feast-dev#6061](feast-dev#6061)) ([1e1f5d9](feast-dev@1e1f5d9)) * Use orjson for faster JSON serialization in feature server ([6f5203a](feast-dev@6f5203a)) ### Performance Improvements * Optimize protobuf parsing in Redis online store ([feast-dev#6023](feast-dev#6023)) ([59dfdb8](feast-dev@59dfdb8)) * Optimize timestamp conversion in _convert_rows_to_protobuf ([33a2e95](feast-dev@33a2e95)) * Parallelize DynamoDB batch reads in sync online_read ([feast-dev#6024](feast-dev#6024)) ([9699944](feast-dev@9699944)) * Remove redundant entity key serialization in online_read ([d87283f](feast-dev@d87283f)) Signed-off-by: Aniket Paluskar <apaluska@redhat.com>
* Initial commit on INTPYTHON-297-MongoDB-Feast-Integration
Signed-off-by: Casey Clements <casey.clements@mongodb.com>
* Added mongodb to project.optional-dependencies in pyproject.toml. Now pymongo is found as extra
Signed-off-by: Casey Clements <casey.clements@mongodb.com>
* Checkpoint. Passing tests.unit.online_store.test_online_writes.TestOnlineWrites with default args of MongoDBOnlineStoreConfig. Lots to do.
Signed-off-by: Casey Clements <casey.clements@mongodb.com>
* Handle Nan in dfs for test_online_writes.py. Now all tests in the module pass
Signed-off-by: Casey Clements <casey.clements@mongodb.com>
* Removed suffix of implementation: mongodb_openai -> mongodb
Signed-off-by: Casey Clements <casey.clements@mongodb.com>
* Moved MongoDBOnlineStore to feast.infra.online_store.contrib
Signed-off-by: Casey Clements <casey.clements@mongodb.com>
* Formatting
Signed-off-by: Casey Clements <casey.clements@mongodb.com>
* Refactor online_read that converts bson to proto. Left two methods for now. simply and transformming
Signed-off-by: Casey Clements <casey.clements@mongodb.com>
* Remove file added early during discovery
Signed-off-by: Casey Clements <casey.clements@mongodb.com>
* Formatting
Signed-off-by: Casey Clements <casey.clements@mongodb.com>
* Added version of test that uses testcontainers mongodb instead of assuming one is running
Signed-off-by: Casey Clements <casey.clements@mongodb.com>
* Create Make target for universal tests
Signed-off-by: Casey Clements <casey.clements@mongodb.com>
* Cleanup
Signed-off-by: Casey Clements <casey.clements@mongodb.com>
* Removed temporary integration tests requiring one to spin up own mongodb server.
Signed-off-by: Casey Clements <casey.clements@mongodb.com>
* Format
Signed-off-by: Casey Clements <casey.clements@mongodb.com>
* Typing
Signed-off-by: Casey Clements <casey.clements@mongodb.com>
* Implemented ASync API and Tests
Signed-off-by: Casey Clements <casey.clements@mongodb.com>
* Removed offline store stubs. The first PR will only contain the OnlineStore
Signed-off-by: Casey Clements <casey.clements@mongodb.com>
* Moved mongodb_online_store out of cobtrib package.
Signed-off-by: Casey Clements <casey.clements@mongodb.com>
* Add documentation.
Signed-off-by: Casey Clements <casey.clements@mongodb.com>
* Cleanups and docstrings
Signed-off-by: Casey Clements <casey.clements@mongodb.com>
* Fixed another reference to contrib dir
Signed-off-by: Casey Clements <casey.clements@mongodb.com>
* Typos
Signed-off-by: Casey Clements <casey.clements@mongodb.com>
* Made _convert_raw_docs_to_proto staticmethods private
Signed-off-by: Casey Clements <casey.clements@mongodb.com>
* After benchmarking two alogithm for conevrting read results from bson to proto, removed the naive one. It was outcompeted 3X across dimensions
Signed-off-by: Casey Clements <casey.clements@mongodb.com>
* Add extra unit tests
Signed-off-by: Casey Clements <casey.clements@mongodb.com>
* Formatting
Signed-off-by: Casey Clements <casey.clements@mongodb.com>
* Fixes in pyproject.toml
Signed-off-by: Casey Clements <casey.clements@mongodb.com>
* Fixed Detect secrets false positives.
Signed-off-by: Casey Clements <casey.clements@mongodb.com>
* Update sdk/python/feast/infra/online_stores/mongodb_online_store/mongodb.py
Set _client and _collection to None upon synchronous teardown.
Co-authored-by: devin-ai-integration[bot] <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Signed-off-by: Casey Clements <casey.clements@mongodb.com>
* Fix CI: guard pymongo imports and skip test module when pymongo unavailable
In the unit-test-python CI job, pymongo is not installed (it is an optional
extra). This caused ModuleNotFoundError during pytest collection.
Two changes:
1. mongodb.py: wrap pymongo imports in try/except -> FeastExtrasDependencyImportError
(consistent with redis.py, dynamodb.py, datastore.py pattern)
2. test_mongodb_online_retrieval.py: add pytest.importorskip(\pymongo\) so
the entire test module is skipped gracefully when pymongo is absent
Signed-off-by: Casey Clements <casey.clements@mongodb.com>
* Fix: return (None, None) when entity doc exists but feature view was never written
MongoDB stores all feature views for an entity in a single document.
If FV 'driver_stats' is written, an entity doc exists for driver_1.
A subsequent read for FV 'pricing' (never written) was previously
returning (None, {all_features: ValueProto()}) - a truthy feature dict
with all-empty protos - instead of the correct (None, None) sentinel.
Feast and downstream callers use (None, None) to signal entity absence.
A non-None feature dict means 'entity found, values are null', which
causes different behaviour in the feature retrieval pipeline.
Fix: after confirming the entity doc exists, also check that the
feature view key is present in doc['features']. If absent, return
(None, None) rather than a dict of empty protos.
Adds test_convert_raw_docs_entity_exists_but_fv_not_written to cover
the multi-feature-view colocation scenario identified in code review.
Signed-off-by: Casey Clements <casey.clements@mongodb.com>
* Update pixi.lock after adding mongodb optional dependency to pyproject.toml
Any change to pyproject.toml invalidates the pixi.lock manifest hash,
causing 'pixi install --locked' to fail in CI even when the changed
section (mongodb optional extras) is not used by any pixi environment.
Regenerated with: pixi install (v0.63.2)
Signed-off-by: Casey Clements <casey.clements@mongodb.com>
* fix: catch FeastExtrasDependencyImportError in doctest runner
Stores like mongodb, redis, and dynamodb raise FeastExtrasDependencyImportError
at import time when their optional Python extras are not installed.
test_all.py only caught ModuleNotFoundError, so any such import caused the
entire test_docstrings() function to abort rather than gracefully skipping
the module.
Extend the except clause to also catch FeastExtrasDependencyImportError so
the doctest run completes for all other modules when an optional extra is
absent in the test environment.
Signed-off-by: Casey Clements <casey.clements@mongodb.com>
* fix: update PYTEST_PLUGINS path for mongodb online store tests
Directory was renamed from tests/integration/feature_repos to
tests/universal/feature_repos in upstream/master. Update the
PYTEST_PLUGINS module path in test-python-universal-mongodb-online
to match the new location.
Signed-off-by: Casey Clements <casey.clements@mongodb.com>
* fix: broaden import exception handling in doctest runner to catch TypeError and other third-party import errors
qdrant_client raises TypeError at import time when a gRPC EnumTypeWrapper
is used with the | union operator on Python < 3.12. The previous fix only
caught ModuleNotFoundError and FeastExtrasDependencyImportError, leaving
the test runner vulnerable to any other exception raised by third-party
libraries during pkgutil.walk_packages.
Changes:
- Catch bare Exception in the import try/except block so any import-time
error from an optional dependency causes a graceful skip rather than an
abort of the entire test run.
- Initialize temp_module = None before the try block and add a continue
guard so that a failed import never leaves a stale module reference to
be used in the subsequent doctest execution block.
Signed-off-by: Casey Clements <casey.clements@mongodb.com>
* fix: pass onerror to pkgutil.walk_packages to suppress non-ImportError package import failures
FeastExtrasDependencyImportError inherits from FeastError -> Exception, not
from ImportError. pkgutil.walk_packages calls __import__ internally when
recursing into sub-packages, and only silently swallows ImportError subclasses;
any other exception is re-raised, crashing the entire test run.
Passing onerror=lambda _: None tells walk_packages to skip any package that
fails to import during the walk phase, regardless of the exception type. The
inner importlib.import_module try/except already handles the same errors for
the explicit import step used to collect doctests.
Signed-off-by: Casey Clements <casey.clements@mongodb.com>
* fix: update stale tests.integration.feature_repos imports to tests.universal.feature_repos
The upstream directory tests/integration/feature_repos was renamed to
tests/universal/feature_repos. Three files still referenced the old path:
- mongodb_repo_configuration.py: IntegrationTestRepoConfig and
MongoDBOnlineStoreCreator imports
- tests/.../online_store/mongodb.py: OnlineStoreCreator import
- tests/unit/online_store/test_mongodb_online_retrieval.py: TAGS import
Updating all three unblocks test collection and allows
make test-python-universal-mongodb-online to run locally.
Signed-off-by: Casey Clements <casey.clements@mongodb.com>
* feat: add mongodb to ValidOnlineStoreDBStorePersistenceTypes in feast-operator
The Python ONLINE_STORE_CLASS_FOR_TYPE now includes 'mongodb', so the
operator's parity check (test-datasources) fails unless the Go API also
lists it. Add 'mongodb' to ValidOnlineStoreDBStorePersistenceTypes in
both api/v1 and api/v1alpha1.
Signed-off-by: Casey Clements <casey.clements@mongodb.com>
* feat: add Feast driver metadata to MongoDB client instantiations
Add DriverInfo with the Feast name and version to both the sync
MongoClient and async AsyncMongoClient so that MongoDB can identify
traffic originating from a Feast AI integration.
Signed-off-by: Casey Clements <casey.clements@mongodb.com>
* docs: update MongoDB online store status from alpha to preview
Signed-off-by: Casey Clements <casey.clements@mongodb.com>
* fix: add mongodb to kubebuilder Enum annotations for OnlineStoreDBStorePersistence
The +kubebuilder:validation:Enum annotation on the Type field of
OnlineStoreDBStorePersistence was not updated when mongodb was added to
ValidOnlineStoreDBStorePersistenceTypes. This annotation drives CRD
OpenAPI schema validation at Kubernetes admission time, so any
FeatureStore CR specifying type: mongodb would be rejected by the API
server. Updated both api/v1 and api/v1alpha1.
Signed-off-by: Casey Clements <casey.clements@mongodb.com>
* Update +GOLANGCI_LINT_VERSION to fix upstream issue golang/go#74462
Signed-off-by: Casey Clements <casey.clements@mongodb.com>
* fix: raise ValueError in _get_client_async for invalid config type, consistent with sync counterpart
Signed-off-by: Casey Clements <casey.clements@mongodb.com>
* fix: Added mongodb to operator yamls
Signed-off-by: ntkathole <nikhilkathole2683@gmail.com>
* Small change suggested by ntkathole
Signed-off-by: Casey Clements <casey.clements@mongodb.com>
* Factor out write logic into utility function making sync/async essentially identical.
Signed-off-by: Casey Clements <casey.clements@mongodb.com>
---------
Signed-off-by: Casey Clements <casey.clements@mongodb.com>
Signed-off-by: ntkathole <nikhilkathole2683@gmail.com>
Co-authored-by: devin-ai-integration[bot] <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Co-authored-by: ntkathole <nikhilkathole2683@gmail.com>
Signed-off-by: yuanjun220 <1069645408@qq.com>
# [0.61.0](feast-dev/feast@v0.60.0...v0.61.0) (2026-03-10) ### Bug Fixes * Add grpcio dependency group to transformation server Dockerfile ([2c2150a](feast-dev@2c2150a)) * Add https readiness check for rest-registry tests ([ea85e63](feast-dev@ea85e63)) * Add website build check for PRs and fix blog frontmatter YAML error ([feast-dev#6079](feast-dev#6079)) ([30a3a43](feast-dev@30a3a43)) * Added MLflow metric charts across feature selection ([feast-dev#6080](feast-dev#6080)) ([a403361](feast-dev@a403361)) * Check duplicate names for feature view across types ([feast-dev#5999](feast-dev#5999)) ([95b9af8](feast-dev@95b9af8)) * Fix integration tests ([feast-dev#6046](feast-dev#6046)) ([02d5548](feast-dev@02d5548)) * Fix non-specific label selector on metrics service ([a1a160d](feast-dev@a1a160d)) * Fixed IntegrityError on SqlRegistry ([feast-dev#6047](feast-dev#6047)) ([325e148](feast-dev@325e148)) * Fixed pre-commit check ([114b7db](feast-dev@114b7db)) * Fixed uv cache permission error for docker build on mac ([ad807be](feast-dev@ad807be)) * Fixes a `PydanticDeprecatedSince20` warning for trino_offline_store ([feast-dev#5991](feast-dev#5991)) ([abfd18a](feast-dev@abfd18a)) * Integration test failures ([feast-dev#6040](feast-dev#6040)) ([9165870](feast-dev@9165870)) * Ray offline store tests are duplicated across 3 workflows ([54f705a](feast-dev@54f705a)) * Reenable tests ([feast-dev#6036](feast-dev#6036)) ([82ee7f8](feast-dev@82ee7f8)) * Use commitlint pre-commit hook instead of a separate action ([35a81e7](feast-dev@35a81e7)) ### Features * Add complex type support (Map, JSON, Struct) with schema validation ([feast-dev#5974](feast-dev#5974)) ([1200dbf](feast-dev@1200dbf)) * Add materialization, feature freshness, request latency, and push metrics to feature server ([2c6be18](feast-dev@2c6be18)) * Add non-entity retrieval support for ClickHouse offline store ([4d08ddc](feast-dev@4d08ddc)), closes [feast-dev#5835](feast-dev#5835) * Add OnlineStore for MongoDB ([feast-dev#6025](feast-dev#6025)) ([bf4e3fa](feast-dev@bf4e3fa)), closes [golang/go#74462](golang/go#74462) * Added CodeQL SAST scanning and detect-secrets pre-commit hook ([547b516](feast-dev@547b516)) * Adding optional name to Aggregation (feast-dev[feast-dev#5994](feast-dev#5994)) ([feast-dev#6083](feast-dev#6083)) ([56469f7](feast-dev@56469f7)) * Feature Server High-Availability on Kubernetes ([feast-dev#6028](feast-dev#6028)) ([9c07b4c](feast-dev@9c07b4c)), closes [Hi#Availability](https://github.com/Hi/issues/Availability) [Hi#Availability](https://github.com/Hi/issues/Availability) * **go:** Implement metrics and tracing for http and grpc servers ([feast-dev#5925](feast-dev#5925)) ([2b4ec9a](feast-dev@2b4ec9a)) * Horizontal scaling support to the Feast operator ([feast-dev#6000](feast-dev#6000)) ([3ec13e6](feast-dev@3ec13e6)) * Making feature view source optional (feast-dev[feast-dev#6074](feast-dev#6074)) ([feast-dev#6075](feast-dev#6075)) ([76917b7](feast-dev@76917b7)) * Support arm docker build ([feast-dev#6061](feast-dev#6061)) ([1e1f5d9](feast-dev@1e1f5d9)) * Use orjson for faster JSON serialization in feature server ([6f5203a](feast-dev@6f5203a)) ### Performance Improvements * Optimize protobuf parsing in Redis online store ([feast-dev#6023](feast-dev#6023)) ([59dfdb8](feast-dev@59dfdb8)) * Optimize timestamp conversion in _convert_rows_to_protobuf ([33a2e95](feast-dev@33a2e95)) * Parallelize DynamoDB batch reads in sync online_read ([feast-dev#6024](feast-dev#6024)) ([9699944](feast-dev@9699944)) * Remove redundant entity key serialization in online_read ([d87283f](feast-dev@d87283f)) Signed-off-by: yuanjun220 <1069645408@qq.com>
# [0.61.0](v0.60.0...v0.61.0) (2026-04-07) ### Bug Fixes * Add grpcio dependency group to transformation server Dockerfile ([2c2150a](2c2150a)) * Add https readiness check for rest-registry tests ([ea85e63](ea85e63)) * Add website build check for PRs and fix blog frontmatter YAML error ([#6079](#6079)) ([30a3a43](30a3a43)) * Added missing jackc/pgx/v5 entries ([94ad0e7](94ad0e7)) * Added MLflow metric charts across feature selection ([#6080](#6080)) ([a403361](a403361)) * Check duplicate names for feature view across types ([#5999](#5999)) ([95b9af8](95b9af8)) * Fix integration tests ([#6046](#6046)) ([02d5548](02d5548)) * Fix missing error handling for resource_counts endpoint ([d9706ce](d9706ce)) * Fix non-specific label selector on metrics service ([a1a160d](a1a160d)) * fix path feature_definitions.py ([7d7df68](7d7df68)) * Fix regstry Rest API tests intermittent failure ([d53a339](d53a339)) * Fixed IntegrityError on SqlRegistry ([#6047](#6047)) ([325e148](325e148)) * Fixed intermittent failures in get_historical_features ([c335ec7](c335ec7)) * Fixed pre-commit check ([114b7db](114b7db)) * Fixed the intermittent FeatureViewNotFoundException ([661ecc7](661ecc7)) * Fixed uv cache permission error for docker build on mac ([ad807be](ad807be)) * Fixes a `PydanticDeprecatedSince20` warning for trino_offline_store ([#5991](#5991)) ([abfd18a](abfd18a)) * Handle existing RBAC role gracefully in namespace registry ([b46a62b](b46a62b)) * Ignore ipynb files during apply ([#6151](#6151)) ([4ea123d](4ea123d)) * Integration test failures ([#6040](#6040)) ([9165870](9165870)) * Mount TLS volumes for init container ([080a9b5](080a9b5)) * **postgres:** Use end_date in synthetic entity_df for non-entity retrieval ([#6110](#6110)) ([088a802](088a802)), closes [#6066](#6066) * Ray offline store tests are duplicated across 3 workflows ([54f705a](54f705a)) * Reenable tests ([#6036](#6036)) ([82ee7f8](82ee7f8)) * SSL/TLS mode by default for postgres connection ([4844488](4844488)) * Use commitlint pre-commit hook instead of a separate action ([35a81e7](35a81e7)) ### Features * Add Claude Code agent skills for Feast ([#6081](#6081)) ([1e5b60f](1e5b60f)), closes [#5976](#5976) [#6007](#6007) * Add complex type support (Map, JSON, Struct) with schema validation ([#5974](#5974)) ([1200dbf](1200dbf)) * Add decimal to supported feature types ([#6029](#6029)) ([#6226](#6226)) ([cff6fbf](cff6fbf)) * Add feast apply init container to automate registry population on pod start ([#6106](#6106)) ([6b31a43](6b31a43)) * Add feature view versioning support to PostgreSQL and MySQL online stores ([#6193](#6193)) ([940e0f0](940e0f0)), closes [#6168](#6168) [#6169](#6169) [#2728](#2728) * Add materialization, feature freshness, request latency, and push metrics to feature server ([2c6be18](2c6be18)) * Add metadata statistics to registry api ([ef1d4fc](ef1d4fc)) * Add non-entity retrieval support for ClickHouse offline store ([4d08ddc](4d08ddc)), closes [#5835](#5835) * Add OnlineStore for MongoDB ([#6025](#6025)) ([bf4e3fa](bf4e3fa)), closes [golang/go#74462](golang/go#74462) * Add Oracle DB as Offline store in python sdk & operator ([#6017](#6017)) ([9d35368](9d35368)) * Add RBAC aggregation labels to FeatureStore ClusterRoles ([daf77c6](daf77c6)) * Add ServiceMonitor auto-generation for Prometheus discovery ([#6126](#6126)) ([56e6d21](56e6d21)) * Add typed_features field to grpc write request (([#6117](#6117)) ([#6118](#6118)) ([eeaa6db](eeaa6db)), closes [#6116](#6116) * Add UUID and TIME_UUID as feature types ([#5885](#5885)) ([#5951](#5951)) ([5d6e311](5d6e311)) * Add version indicators to lineage graph nodes ([#6187](#6187)) ([73805d3](73805d3)) * Add version tracking to FeatureView ([#6101](#6101)) ([ed4a4f2](ed4a4f2)) * Added Agent skills for AI Agents ([#6007](#6007)) ([99008c8](99008c8)) * Added CodeQL SAST scanning and detect-secrets pre-commit hook ([547b516](547b516)) * Added odfv transformations metrics ([8b5a526](8b5a526)) * Adding optional name to Aggregation (feast-dev[#5994](#5994)) ([#6083](#6083)) ([56469f7](56469f7)) * Created DocEmbedder class ([#5973](#5973)) ([0719c06](0719c06)) * Extended OIDC support to extract groups & namespaces and token injection with multiple methods ([#6089](#6089)) ([7c04026](7c04026)) * Feature Server High-Availability on Kubernetes ([#6028](#6028)) ([9c07b4c](9c07b4c)), closes [Hi#Availability](https://github.com/Hi/issues/Availability) [Hi#Availability](https://github.com/Hi/issues/Availability) * **go:** Implement metrics and tracing for http and grpc servers ([#5925](#5925)) ([2b4ec9a](2b4ec9a)) * Horizontal scaling support to the Feast operator ([#6000](#6000)) ([3ec13e6](3ec13e6)) * Making feature view source optional (feast-dev[#6074](#6074)) ([#6075](#6075)) ([76917b7](76917b7)) * Replace ORJSONResponse with Pydantic response models for faster JSON serialization ([65cf03c](65cf03c)) * Support arm docker build ([#6061](#6061)) ([1e1f5d9](1e1f5d9)) * Support distinct count aggregation [[#6116](#6116)] ([3639570](3639570)) * Support HTTP in MCP ([#6109](#6109)) ([e72b983](e72b983)) * Support nested collection types (Array/Set of Array/Set) ([#5947](#5947)) ([#6132](#6132)) ([ab61642](ab61642)) * Support podAnnotations on Deployment pod template ([1b3cdc1](1b3cdc1)) * Use orjson for faster JSON serialization in feature server ([6f5203a](6f5203a)) * Utilize date partition column in BigQuery ([#6076](#6076)) ([4ea9b32](4ea9b32)) ### Performance Improvements * Online feature response construction in a single pass over read rows ([113fb04](113fb04)) * Optimize protobuf parsing in Redis online store ([#6023](#6023)) ([59dfdb8](59dfdb8)) * Optimize timestamp conversion in _convert_rows_to_protobuf ([33a2e95](33a2e95)) * Parallelize DynamoDB batch reads in sync online_read ([#6024](#6024)) ([9699944](9699944)) * Remove redundant entity key serialization in online_read ([d87283f](d87283f))
What this PR does / why we need it:
Adds a first-class MongoDB online store integration at
feast.infra.online_stores.mongodb_online_store.Schema
Each entity is stored as a single MongoDB document keyed by its serialized entity key:
{ "_id": "<serialized_entity_key>", "features": { "<feature_view>": { "<feature>": <value> } }, "event_timestamps": { "<feature_view>": "<datetime>" }, "created_timestamp": "<datetime>" }Because MongoDB has a loose schema and supports upsert semantics natively,
update()requires nopre-creation — it only removes feature views named in
tables_to_deletevia$unset. Multiplefeature views for the same entity share one document.
Implementation highlights
online_write_batch,online_read,online_write_batch_async,online_read_async, andasync def close().async_supportedreturnsread=True, write=True.requested_featuresprojection — both read paths build a MongoDB field projection so onlythe requested feature columns are returned from the server.
_convert_raw_docs_to_prototransforms column-wise tominimise calls to
python_values_to_proto_values(one call per feature across all entities,rather than one call per entity × feature). Benchmarking confirmed this is ~3–4× faster than a
naïve row-wise approach across all scaling dimensions (entities, features, feature views).
serialize_entity_key, so compositekeys (e.g.
customer_id + driver_id) are handled without schema changes.Tests
test_mongodb_online_features) — spins up a real MongoDB containervia
testcontainers, writes to three feature views (single int key, single string key, compositekey), and asserts correct retrieval including type coercion and missing-entity handling.
test_convert_raw_docs_missing_entity— entity absent from query results →(None, None)test_convert_raw_docs_partial_doc— entity present but a feature key missing → emptyValueProto(schema-migration safety)test_convert_raw_docs_ordering— result order follows the requestedidslist regardlessof MongoDB cursor order
All tests pass. Code is clean under
mypy,ruff check, andruff format.