[I] Scalar quantization extreme edge case of uniform vector values [lucene]

["benwtrent (via GitHub)" <gi...@apache.org>]

benwtrent opened a new issue, #13480:
URL: https://github.com/apache/lucene/issues/13480

   ### Description
   
   When quantizing vectors that have a uniform value, the quantiles can get really weird. Meaning both min and max quantiles are actually equivalent. Additionally, the scoring can be unexpected.
   
   Here is a test indicating this weirdness:
   
   ```
     public void testWeirdEdgeCase() throws IOException {
       float[] uniformVector = new float[128];
       for (int i = 0; i < uniformVector.length; i++) {
         uniformVector[i] = 1;
       }
       float[] query = new float[128];
       for (int i = 0; i < query.length; i++) {
         query[i] = 2;
       }
       ScalarQuantizer scalarQuantizer = ScalarQuantizer.fromVectors(fromFloats(new float[][] {uniformVector}),
         0.99f, 1, (byte) 7);
       byte[] quantized = new byte[128];
       float offset = scalarQuantizer.quantize(query, quantized, VectorSimilarityFunction.EUCLIDEAN);
       byte[] quantizedVector = new byte[128];
       float offsetVector = scalarQuantizer.quantize(uniformVector, quantizedVector, VectorSimilarityFunction.EUCLIDEAN);
       ScalarQuantizedVectorSimilarity quantizedSimilarity =
         ScalarQuantizedVectorSimilarity.fromVectorSimilarity(
           VectorSimilarityFunction.EUCLIDEAN, scalarQuantizer.getConstantMultiplier(), scalarQuantizer.getBits());
       float score1 = quantizedSimilarity.score(quantized, offset, quantizedVector, offsetVector);
   
       for (int i = 0; i < query.length; i++) {
         query[i] = 5;
       }
       offset = scalarQuantizer.quantize(query, quantized, VectorSimilarityFunction.EUCLIDEAN);
       float score2 = quantizedSimilarity.score(quantized, offset, quantizedVector, offsetVector);
       assertNotEquals(score1, score2);
     }
   ```
   
   ### Version and environment details
   
   _No response_


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For additional commands, e-mail: issues-help@lucene.apache.org