Skip to content

Commit 9a06044

Browse files
franciscojavierarceoredhatHameed
authored andcommitted
feat: Adding vector search for sqlite (feast-dev#4176)
* feat: Adding vector search for sqlite Signed-off-by: Francisco Javier Arceo <farceo@redhat.com> * adding the sqlite_vss dependency Signed-off-by: Francisco Javier Arceo <farceo@redhat.com> * linter Signed-off-by: Francisco Javier Arceo <farceo@redhat.com> * latest progress Signed-off-by: Francisco Javier Arceo <farceo@redhat.com> * uploading latest progress Signed-off-by: Francisco Javier Arceo <farceo@redhat.com> * updated function Signed-off-by: Francisco Javier Arceo <farceo@redhat.com> * adding configuration Signed-off-by: Francisco Javier Arceo <farceo@redhat.com> * adding current progress Signed-off-by: Francisco Javier Arceo <farceo@redhat.com> * updating requirements files Signed-off-by: Francisco Javier Arceo <farceo@redhat.com> * moving to sqlite-vec Signed-off-by: Francisco Javier Arceo <farceo@redhat.com> * updating sqlite.py Signed-off-by: Francisco Javier Arceo <farceo@redhat.com> * checking in progress Signed-off-by: Francisco Javier Arceo <farceo@redhat.com> * updated test type Signed-off-by: Francisco Javier Arceo <farceo@redhat.com> * got the initialization working, nice Signed-off-by: Francisco Javier Arceo <farceo@redhat.com> * checking in progress from last night Signed-off-by: Francisco Javier Arceo <farceo@redhat.com> * removing unnecessary stuff Signed-off-by: Francisco Javier Arceo <farceo@redhat.com> * fixing merge conflicts Signed-off-by: Francisco Javier Arceo <farceo@redhat.com> * removing files changed accidentally] Signed-off-by: Francisco Javier Arceo <farceo@redhat.com> * uploading current progress...things run but need to update the virtual table insertion Signed-off-by: Francisco Javier Arceo <farceo@redhat.com> * linted Signed-off-by: Francisco Javier Arceo <farceo@redhat.com> * adding working notes Signed-off-by: Francisco Javier Arceo <farceo@redhat.com> * found a bug, original feature_store.py was only grabbing first feature view, adjusted Signed-off-by: Francisco Javier Arceo <farceo@redhat.com> * cant use a string have to verify it is a proper FeatureView object Signed-off-by: Francisco Javier Arceo <farceo@redhat.com> * updated got it working, need to fix some other stuff still Signed-off-by: Francisco Javier Arceo <farceo@redhat.com> * working Signed-off-by: Francisco Javier Arceo <farceo@redhat.com> * linter Signed-off-by: Francisco Javier Arceo <farceo@redhat.com> * fixing some type issues Signed-off-by: Francisco Javier Arceo <farceo@redhat.com> * fixed typing and lint issues Signed-off-by: Francisco Javier Arceo <farceo@redhat.com> * updated dependencies Signed-off-by: Francisco Javier Arceo <farceo@redhat.com> * fix for pixi and updating requirements Signed-off-by: Francisco Javier Arceo <farceo@redhat.com> * fixed type Signed-off-by: Francisco Javier Arceo <farceo@redhat.com> * linter Signed-off-by: Francisco Javier Arceo <farceo@redhat.com> * testing sqlite_vec import Signed-off-by: Francisco Javier Arceo <farceo@redhat.com> * adding minimal example test Signed-off-by: Francisco Javier Arceo <farceo@redhat.com> * lint Signed-off-by: Francisco Javier Arceo <farceo@redhat.com> * testing raw sqlite Signed-off-by: Francisco Javier Arceo <farceo@redhat.com> * Printing package version * printing version Signed-off-by: Francisco Javier Arceo <farceo@redhat.com> * updated requirements * rebuilding requirments Signed-off-by: Francisco Javier Arceo <farceo@redhat.com> * only going to run this on 3.10 for now Signed-off-by: Francisco Javier Arceo <farceo@redhat.com> * updated docs for sqlite caveats Signed-off-by: Francisco Javier Arceo <farceo@redhat.com> * adding reason Signed-off-by: Francisco Javier Arceo <farceo@redhat.com> * skipping Signed-off-by: Francisco Javier Arceo <farceo@redhat.com> * updated tests Signed-off-by: Francisco Javier Arceo <farceo@redhat.com> * removing print Signed-off-by: Francisco Javier Arceo <farceo@redhat.com> * added method call Signed-off-by: Francisco Javier Arceo <farceo@redhat.com> * added prubt Signed-off-by: Francisco Javier Arceo <farceo@redhat.com> * added print Signed-off-by: Francisco Javier Arceo <farceo@redhat.com> * removing print Signed-off-by: Francisco Javier Arceo <farceo@redhat.com> * adding check in sqlite Signed-off-by: Francisco Javier Arceo <farceo@redhat.com> * missed an = Signed-off-by: Francisco Javier Arceo <farceo@redhat.com> * still running on 3.11 Signed-off-by: Francisco Javier Arceo <farceo@redhat.com> * typo Signed-off-by: Francisco Javier Arceo <farceo@redhat.com> * fix Signed-off-by: Francisco Javier Arceo <farceo@redhat.com> * fix Signed-off-by: Francisco Javier Arceo <farceo@redhat.com> * updated setup and docs Signed-off-by: Francisco Javier Arceo <farceo@redhat.com> * renamed things Signed-off-by: Francisco Javier Arceo <farceo@redhat.com> --------- Signed-off-by: Francisco Javier Arceo <farceo@redhat.com> Signed-off-by: Abdul Hameed <ahameed@redhat.com>
1 parent 36a574d commit 9a06044

15 files changed

Lines changed: 851 additions & 161 deletions

File tree

docs/reference/alpha-vector-database.md

Lines changed: 18 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -13,7 +13,9 @@ Below are supported vector databases and implemented features:
1313
| Elasticsearch | [x] | [x] |
1414
| Milvus | [ ] | [ ] |
1515
| Faiss | [ ] | [ ] |
16+
| SQLite | [x] | [ ] |
1617

18+
Note: SQLite is in limited access and only working on Python 3.10. It will be updated as [sqlite_vec](https://github.com/asg017/sqlite-vec/) progresses.
1719

1820
## Example
1921

@@ -108,4 +110,20 @@ def print_online_features(features):
108110
print(key, " : ", value)
109111

110112
print_online_features(features)
113+
```
114+
115+
### Configuration
116+
We offer two Online Store options for Vector Databases. PGVector and SQLite.
117+
118+
#### Installation with SQLite
119+
If you are using `pyenv` to manage your Python versions, you can install the SQLite extension with the following command:
120+
```bash
121+
PYTHON_CONFIGURE_OPTS="--enable-loadable-sqlite-extensions" \
122+
LDFLAGS="-L/opt/homebrew/opt/sqlite/lib" \
123+
CPPFLAGS="-I/opt/homebrew/opt/sqlite/include" \
124+
pyenv install 3.10.14
125+
```
126+
And you can the Feast install package via:
127+
```bash
128+
pip install feast[sqlite_vec]
111129
```

0 commit comments

Comments
 (0)