You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@lucene.apache.org by "kaivalnp (via GitHub)" <gi...@apache.org> on 2026/08/25 16:27:11 UTC

[I] Should we move vector bulk-scoring APIs to VectorUtilSupport? [lucene]

kaivalnp opened a new issue, #16556:
URL: https://github.com/apache/lucene/issues/16556

   ### Description
   
   Off-heap float vector bulk-scoring was introduced in #14980, and showed some nice performance improvements! (vectors are scored in chunks of a specific machine-dependent size to leverage SIMD instructions, and if I understand correctly, this optimization amortizes the cost of loading chunks of the query vector _across N documents_, instead of loading it _once per document_).
   
   There is currently an open issue to enable off-heap bulk-scoring for other vectors (byte vectors and quantized versions): #15155.
   
   I wonder if we should move the bulk-scoring APIs one level lower (from [`RandomVectorScorer`](https://github.com/apache/lucene/blob/2822b28b5ab042672d6cb569589128e5dcafa648/lucene/core/src/java/org/apache/lucene/util/hnsw/RandomVectorScorer.java#L37-L54) to [`VectorUtilSupport`](https://github.com/apache/lucene/blob/2822b28b5ab042672d6cb569589128e5dcafa648/lucene/core/src/java/org/apache/lucene/internal/vectorization/VectorUtilSupport.java#L25)) so that the native vectorization provider (#15508) can also benefit from it? (which I think was added to make use of specific instructions that the JVM cannot compile to)


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For additional commands, e-mail: issues-help@lucene.apache.org


Re: [I] Should we move vector bulk-scoring APIs to VectorUtilSupport? [lucene]

Posted by "Tony-X (via GitHub)" <gi...@apache.org>.
Tony-X commented on issue #16556:
URL: https://github.com/apache/lucene/issues/16556#issuecomment-5502246997

   >  this optimization amortizes the cost of loading chunks of the query vector across N documents, instead of loading it once per document
   
   Yes, we should leverage it more!
   
   Instead of moving, it'd be simpler to think about extending [VectorUtilSupport](https://github.com/apache/lucene/blob/2822b28b5ab042672d6cb569589128e5dcafa648/lucene/core/src/java/org/apache/lucene/internal/vectorization/VectorUtilSupport.java#L25) with methods that target single-query-many-doc scoring. 
   
   Then the [RandomVectorScorer](https://github.com/apache/lucene/blob/2822b28b5ab042672d6cb569589128e5dcafa648/lucene/core/src/java/org/apache/lucene/util/hnsw/RandomVectorScorer.java#L37-L54) implementors can start consuming those new methods. 
   
   This way we minimize the API surface churn. 


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For additional commands, e-mail: issues-help@lucene.apache.org


Re: [I] Should we move vector bulk-scoring APIs to VectorUtilSupport? [lucene]

Posted by "kaivalnp (via GitHub)" <gi...@apache.org>.
kaivalnp commented on issue #16556:
URL: https://github.com/apache/lucene/issues/16556#issuecomment-5542363590

   Thanks @Tony-X @CH-Abhinav, this is exactly what I had in mind..
   Please feel free to open a PR!


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For additional commands, e-mail: issues-help@lucene.apache.org


Re: [I] Should we move vector bulk-scoring APIs to VectorUtilSupport? [lucene]

Posted by "CH-Abhinav (via GitHub)" <gi...@apache.org>.
CH-Abhinav commented on issue #16556:
URL: https://github.com/apache/lucene/issues/16556#issuecomment-5557795812

   created a draft PR @Tony-X @kaivalnp it contains only `dotProductBulk()` for now once see the changes and based on the feedback i will continue writing other methods.
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For additional commands, e-mail: issues-help@lucene.apache.org


Re: [I] Should we move vector bulk-scoring APIs to VectorUtilSupport? [lucene]

Posted by "CH-Abhinav (via GitHub)" <gi...@apache.org>.
CH-Abhinav commented on issue #16556:
URL: https://github.com/apache/lucene/issues/16556#issuecomment-5536029891

   Yeah it would be better if we have method in `VectorUtilSupport` which can be used further in `RandomVectorScorer`. something like a bulkdotproduct which will simplify the work in upper APIs which depend on vectorutils.
   
   If no one is working on this I will try to make a PR on this issue and let's see results.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For additional commands, e-mail: issues-help@lucene.apache.org