You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@lucene.apache.org by "benwtrent (via GitHub)" <gi...@apache.org> on 2023/12/14 14:13:50 UTC
[PR] Fix flaky tests that are caused by small float vectors [lucene]
benwtrent opened a new pull request, #12943:
URL: https://github.com/apache/lucene/pull/12943
While quantization generally works well, when the number of dimensions is tiny (just two like in our tests), and we are indexing a circle, and we have random merge policies, we can end up getting unexpected ordering on the resulting vectors.
closes: https://github.com/apache/lucene/issues/12940
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For additional commands, e-mail: issues-help@lucene.apache.org
Re: [PR] Fix flaky tests that are caused by small float vectors [lucene]
Posted by "benwtrent (via GitHub)" <gi...@apache.org>.
benwtrent commented on code in PR #12943:
URL: https://github.com/apache/lucene/pull/12943#discussion_r1426839005
##########
lucene/core/src/test/org/apache/lucene/index/TestKnnGraph.java:
##########
@@ -100,22 +100,18 @@ public KnnVectorsFormat getKnnVectorsFormatForField(String field) {
}
};
- if (vectorEncoding == VectorEncoding.FLOAT32) {
- float32Codec = codec;
Review Comment:
It could be, I was going to look into that one next.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For additional commands, e-mail: issues-help@lucene.apache.org
Re: [PR] Fix flaky tests that are caused by small float vectors [lucene]
Posted by "benwtrent (via GitHub)" <gi...@apache.org>.
benwtrent commented on code in PR #12943:
URL: https://github.com/apache/lucene/pull/12943#discussion_r1426804219
##########
lucene/core/src/test/org/apache/lucene/index/TestKnnGraph.java:
##########
@@ -100,22 +100,18 @@ public KnnVectorsFormat getKnnVectorsFormatForField(String field) {
}
};
- if (vectorEncoding == VectorEncoding.FLOAT32) {
- float32Codec = codec;
Review Comment:
@msokolov yeah, two tests are really relying on perfect scores & things being float. I mistakenly turned on quantization which adds some error bands to the scores (obviously, because its lossy) and thus its flaky.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For additional commands, e-mail: issues-help@lucene.apache.org
Re: [PR] Fix flaky tests that are caused by small float vectors [lucene]
Posted by "msokolov (via GitHub)" <gi...@apache.org>.
msokolov commented on code in PR #12943:
URL: https://github.com/apache/lucene/pull/12943#discussion_r1426801907
##########
lucene/core/src/test/org/apache/lucene/index/TestKnnGraph.java:
##########
@@ -100,22 +100,18 @@ public KnnVectorsFormat getKnnVectorsFormatForField(String field) {
}
};
- if (vectorEncoding == VectorEncoding.FLOAT32) {
- float32Codec = codec;
Review Comment:
so this was relying on the default codec being a FLOAT32 codec? TBH I haven't kept up with how we now select whether or not to use quantization. Is it the default? Do we need to override the codec to select it?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For additional commands, e-mail: issues-help@lucene.apache.org
Re: [PR] Fix flaky tests that are caused by small float vectors [lucene]
Posted by "msokolov (via GitHub)" <gi...@apache.org>.
msokolov commented on code in PR #12943:
URL: https://github.com/apache/lucene/pull/12943#discussion_r1426843989
##########
lucene/core/src/test/org/apache/lucene/index/TestKnnGraph.java:
##########
@@ -100,22 +100,18 @@ public KnnVectorsFormat getKnnVectorsFormatForField(String field) {
}
};
- if (vectorEncoding == VectorEncoding.FLOAT32) {
- float32Codec = codec;
Review Comment:
OK that does repro for me, but also on branch_9_8, so I think it must be a different issue - I'll open a new one to track
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For additional commands, e-mail: issues-help@lucene.apache.org
Re: [PR] Fix flaky tests that are caused by small float vectors [lucene]
Posted by "msokolov (via GitHub)" <gi...@apache.org>.
msokolov commented on code in PR #12943:
URL: https://github.com/apache/lucene/pull/12943#discussion_r1426817545
##########
lucene/core/src/test/org/apache/lucene/index/TestKnnGraph.java:
##########
@@ -100,22 +100,18 @@ public KnnVectorsFormat getKnnVectorsFormatForField(String field) {
}
};
- if (vectorEncoding == VectorEncoding.FLOAT32) {
- float32Codec = codec;
Review Comment:
there was just another failure on the dev list
FAILED: org.apache.lucene.util.hnsw.TestHnswFloatVectorGraph.testSortedAndUnsortedIndicesReturnSameResults
Error Message:
java.lang.AssertionError: expected:<[43, 199, 163, 3, 180]> but was:<[270, 43, 199, 163, 269]>
I wonder if it could be a similar root cause?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For additional commands, e-mail: issues-help@lucene.apache.org
Re: [PR] Fix flaky tests that are caused by small float vectors [lucene]
Posted by "benwtrent (via GitHub)" <gi...@apache.org>.
benwtrent merged PR #12943:
URL: https://github.com/apache/lucene/pull/12943
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For additional commands, e-mail: issues-help@lucene.apache.org
Re: [PR] Fix flaky tests that are caused by small float vectors [lucene]
Posted by "msokolov (via GitHub)" <gi...@apache.org>.
msokolov commented on code in PR #12943:
URL: https://github.com/apache/lucene/pull/12943#discussion_r1426843989
##########
lucene/core/src/test/org/apache/lucene/index/TestKnnGraph.java:
##########
@@ -100,22 +100,18 @@ public KnnVectorsFormat getKnnVectorsFormatForField(String field) {
}
};
- if (vectorEncoding == VectorEncoding.FLOAT32) {
- float32Codec = codec;
Review Comment:
OK that does repro for me, but also on branch_9_8, so I think it must be a different issue - I'll open a new one to track. Umm correction it does not repro on branch_9_8, but it is in any case a different issue: https://github.com/apache/lucene/issues/12945
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For additional commands, e-mail: issues-help@lucene.apache.org