You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@lucene.apache.org by GitBox <gi...@apache.org> on 2022/11/09 18:49:36 UTC
[GitHub] [lucene] rmuir opened a new issue, #11911: improve checkindex to be more thorough for vectors (e.g. test seeking)
rmuir opened a new issue, #11911:
URL: https://github.com/apache/lucene/issues/11911
### Description
Currently checkindex only `next()'s` through the vectors. We should do some seeking as well. It doesnt have to be intense, e.g. we could do 64 seeks (if there's 20,000,000 docs in the index, seek to 0, 312500, ..., something like that).
This would have saved a lot of time with #11905 but will also improve testing in general. CheckIndex gets called on every index created by every test, so checks here are very powerful.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For additional commands, e-mail: issues-help@lucene.apache.org
[GitHub] [lucene] jtibshirani commented on issue #11911: improve checkindex to be more thorough for vectors (e.g. test seeking)
Posted by GitBox <gi...@apache.org>.
jtibshirani commented on issue #11911:
URL: https://github.com/apache/lucene/issues/11911#issuecomment-1309460506
This is a good idea! As a note -- in order to exercise all parts of the file format, we'll have to perform a kNN search too through `KnnVectorsReader#search`. If we only load `VectorValues` and iterating through them (even with skipping), then we won't exercise the HNSW graph part which lives in the `.vex` file.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For additional commands, e-mail: issues-help@lucene.apache.org
[GitHub] [lucene] rmuir closed issue #11911: improve checkindex to be more thorough for vectors (e.g. test seeking)
Posted by GitBox <gi...@apache.org>.
rmuir closed issue #11911: improve checkindex to be more thorough for vectors (e.g. test seeking)
URL: https://github.com/apache/lucene/issues/11911
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For additional commands, e-mail: issues-help@lucene.apache.org
[GitHub] [lucene] rmuir commented on issue #11911: improve checkindex to be more thorough for vectors (e.g. test seeking)
Posted by GitBox <gi...@apache.org>.
rmuir commented on issue #11911:
URL: https://github.com/apache/lucene/issues/11911#issuecomment-1310217848
"read every byte of the index" is the promise that checkindex makes. So this bug is really important.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For additional commands, e-mail: issues-help@lucene.apache.org