Skip to content

feat: add config for pca in annlite#606

Merged
JoanFM merged 2 commits into
mainfrom
feat-annlite-pca-config
Oct 11, 2022
Merged

feat: add config for pca in annlite#606
JoanFM merged 2 commits into
mainfrom
feat-annlite-pca-config

Conversation

@jemmyshin

@jemmyshin jemmyshin commented Oct 11, 2022

Copy link
Copy Markdown
Contributor

Goals: Enable loading indexer with PCA in DocumentArray AnnLite backend.

In AnnLite we load the model by their params_hash which compute by hashing n_dim, n_components...
So in order to successfully load the dumped indexer, we need to pass the n_components to AnnLite.

  • add a new parameter n_components, which is the output dimension of PCA, in annlite/backend.py AnnliteConfig
  • check and update documentation, if required. See guide

@JoanFM JoanFM left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

what about the annlite version? is it compatible?

@alaeddine-13 alaeddine-13 left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

can you make sure it's documented here ?
https://docarray.jina.ai/advanced/document-store/annlite/#config
also double check if we miss other params

@jemmyshin

Copy link
Copy Markdown
Contributor Author

No need to change the version, we already have this n_components in AnnLite long time ago

@codecov

codecov Bot commented Oct 11, 2022

Copy link
Copy Markdown

Codecov Report

Merging #606 (9d4c1a0) into main (262b2d1) will increase coverage by 0.00%.
The diff coverage is 100.00%.

@@           Coverage Diff           @@
##             main     #606   +/-   ##
=======================================
  Coverage   84.91%   84.91%           
=======================================
  Files         133      133           
  Lines        6714     6715    +1     
=======================================
+ Hits         5701     5702    +1     
  Misses       1013     1013           
Flag Coverage Δ
docarray 84.91% <100.00%> (+<0.01%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files Coverage Δ
docarray/array/storage/annlite/backend.py 96.05% <100.00%> (+0.05%) ⬆️

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

@JoanFM JoanFM merged commit e99cb44 into main Oct 11, 2022
@JoanFM JoanFM deleted the feat-annlite-pca-config branch October 11, 2022 10:45
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants