Skip to content

feat: Adding vector search for sqlite #4176

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
franciscojavierarceo merged 60 commits into from
Jun 10, 2024
Merged

feat: Adding vector search for sqlite #4176

Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
60 commits
Select commit Hold shift + click to select a range
a9be27d
feat: Adding vector search for sqlite
franciscojavierarceo May 5, 2024
2addd45
adding the sqlite_vss dependency
franciscojavierarceo May 5, 2024
4f9070b
linter
franciscojavierarceo May 6, 2024
7599770
latest progress
franciscojavierarceo May 9, 2024
3f9c0b5
uploading latest progress
franciscojavierarceo May 15, 2024
4f1d01b
updated function
franciscojavierarceo May 15, 2024
2a33c69
adding configuration
franciscojavierarceo May 16, 2024
81e4a27
adding current progress
franciscojavierarceo May 16, 2024
ee300d9
updating requirements files
franciscojavierarceo May 17, 2024
995a9c3
moving to sqlite-vec
franciscojavierarceo May 19, 2024
ceb7395
updating sqlite.py
franciscojavierarceo May 19, 2024
2cf4020
checking in progress
franciscojavierarceo May 20, 2024
494e12f
updated test type
franciscojavierarceo May 20, 2024
714b9c7
got the initialization working, nice
franciscojavierarceo May 22, 2024
cd41bf0
checking in progress from last night
franciscojavierarceo May 22, 2024
ae40c81
removing unnecessary stuff
franciscojavierarceo May 23, 2024
dd5b7e9
fixing merge conflicts
franciscojavierarceo May 27, 2024
59e8b40
removing files changed accidentally]
franciscojavierarceo May 27, 2024
642b68e
uploading current progress...things run but need to update the virtua…
franciscojavierarceo May 28, 2024
2a3dfd2
linted
franciscojavierarceo May 28, 2024
345526b
adding working notes
franciscojavierarceo May 28, 2024
4872d2d
found a bug, original feature_store.py was only grabbing first featur…
franciscojavierarceo May 28, 2024
5f8ff90
cant use a string have to verify it is a proper FeatureView object
franciscojavierarceo May 29, 2024
2a98513
updated got it working, need to fix some other stuff still
franciscojavierarceo May 30, 2024
1db8446
working
franciscojavierarceo May 31, 2024
d48ce6d
linter
franciscojavierarceo May 31, 2024
3fd1979
fixing some type issues
franciscojavierarceo May 31, 2024
773e581
fixed typing and lint issues
franciscojavierarceo May 31, 2024
9716f3c
Merge branch 'master' into get-online-documents-tests
franciscojavierarceo May 31, 2024
ccf5277
updated dependencies
franciscojavierarceo May 31, 2024
4b55f38
fix for pixi and updating requirements
franciscojavierarceo May 31, 2024
c0fcb06
fixed type
franciscojavierarceo May 31, 2024
34e4d29
linter
franciscojavierarceo May 31, 2024
21bbb7d
testing sqlite_vec import
franciscojavierarceo Jun 1, 2024
482df7e
adding minimal example test
franciscojavierarceo Jun 1, 2024
1d8601f
lint
franciscojavierarceo Jun 1, 2024
75b04c1
testing raw sqlite
franciscojavierarceo Jun 2, 2024
13da55d
Printing package version
franciscojavierarceo Jun 2, 2024
1b9a3bb
printing version
franciscojavierarceo Jun 3, 2024
2eedda8
updated requirements
franciscojavierarceo Jun 3, 2024
2d43200
Merge branch 'master' into get-online-documents-tests
franciscojavierarceo Jun 8, 2024
bc90a1c
rebuilding requirments
franciscojavierarceo Jun 8, 2024
c50f1d2
only going to run this on 3.10 for now
franciscojavierarceo Jun 8, 2024
c4eafab
updated docs for sqlite caveats
franciscojavierarceo Jun 8, 2024
eb37121
adding reason
franciscojavierarceo Jun 8, 2024
cfd7611
skipping
franciscojavierarceo Jun 8, 2024
58d595a
updated tests
franciscojavierarceo Jun 8, 2024
8db355c
removing print
franciscojavierarceo Jun 9, 2024
9335fa5
added method call
franciscojavierarceo Jun 10, 2024
511291a
added prubt
franciscojavierarceo Jun 10, 2024
b544527
added print
franciscojavierarceo Jun 10, 2024
efad816
removing print
franciscojavierarceo Jun 10, 2024
436b89a
adding check in sqlite
franciscojavierarceo Jun 10, 2024
263ec43
missed an =
franciscojavierarceo Jun 10, 2024
908b7f3
still running on 3.11
franciscojavierarceo Jun 10, 2024
65952db
typo
franciscojavierarceo Jun 10, 2024
39493b7
fix
franciscojavierarceo Jun 10, 2024
834e4f6
fix
franciscojavierarceo Jun 10, 2024
1c14c97
updated setup and docs
franciscojavierarceo Jun 10, 2024
ec60ea3
renamed things
franciscojavierarceo Jun 10, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
updated requirements
  • Loading branch information
@franciscojavierarceo
franciscojavierarceo committed Jun 3, 2024
commit 2eedda8a82fb4e796832f491b862dcf88b25acb6
2 changes: 1 addition & 1 deletion sdk/python/requirements/py3.10-ci-requirements.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -864,7 +864,7 @@ sqlalchemy-views==0.3.2
# via ibis-framework
sqlglot==20.11.0
# via ibis-framework
sqlite-vec==0.0.1a9
sqlite-vec==0.0.1a10
# via feast (setup.py)
stack-data==0.6.3
# via ipython
Expand Down
2 changes: 1 addition & 1 deletion sdk/python/requirements/py3.10-requirements.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -166,7 +166,7 @@ sniffio==1.3.1
# httpx
sqlalchemy[mypy]==2.0.30
# via feast (setup.py)
sqlite-vec==0.0.1a9
sqlite-vec==0.0.1a10
# via feast (setup.py)
starlette==0.37.2
# via fastapi
Expand Down
2 changes: 1 addition & 1 deletion sdk/python/requirements/py3.11-ci-requirements.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -857,7 +857,7 @@ sqlalchemy-views==0.3.2
# via ibis-framework
sqlglot==20.11.0
# via ibis-framework
sqlite-vec==0.0.1a9
sqlite-vec==0.0.1a10
# via feast (setup.py)
stack-data==0.6.3
# via ipython
Expand Down
2 changes: 1 addition & 1 deletion sdk/python/requirements/py3.11-requirements.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -164,7 +164,7 @@ sniffio==1.3.1
# httpx
sqlalchemy[mypy]==2.0.30
# via feast (setup.py)
sqlite-vec==0.0.1a9
sqlite-vec==0.0.1a10
# via feast (setup.py)
starlette==0.37.2
# via fastapi
Expand Down
2 changes: 1 addition & 1 deletion sdk/python/requirements/py3.9-ci-requirements.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -875,7 +875,7 @@ sqlalchemy-views==0.3.2
# via ibis-framework
sqlglot==20.11.0
# via ibis-framework
sqlite-vec==0.0.1a9
sqlite-vec==0.0.1a10
# via feast (setup.py)
stack-data==0.6.3
# via ipython
Expand Down
2 changes: 1 addition & 1 deletion sdk/python/requirements/py3.9-requirements.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -168,7 +168,7 @@ sniffio==1.3.1
# httpx
sqlalchemy[mypy]==2.0.30
# via feast (setup.py)
sqlite-vec==0.0.1a9
sqlite-vec==0.0.1a10
# via feast (setup.py)
starlette==0.37.2
# via fastapi
Expand Down
2 changes: 1 addition & 1 deletion setup.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -56,7 +56,7 @@
"pygments>=2.12.0,<3",
"PyYAML>=5.4.0,<7",
"requests",
"sqlite-vec",
"sqlite-vec==v0.0.1-alpha.10",

@tokoko tokoko Jun 10, 2024

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

How about we put this under sqlite extra?

@franciscojavierarceo franciscojavierarceo Jun 10, 2024

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍 put it under sqlite_vec extra

"SQLAlchemy[mypy]>1",
"tabulate>=0.8.0,<1",
"tenacity>=7,<9",
Expand Down