[GitHub] [lucene] contrebande-labs commented on issue #11507: Increase the number of dims for KNN vectors to 2048 [LUCENE-10471]

["contrebande-labs (via GitHub)" <gi...@apache.org>]

contrebande-labs commented on issue #11507:
URL: https://github.com/apache/lucene/issues/11507#issuecomment-1552208982

   > [...] it's challenging to envision Lucene emerging as the leading solution for large-scale vector similarity search [...] 
   
   For anyone out there who knows IR's SOTA  _still_ is either BM25 or a combination of BM25 and HNSW similarity search, _no it isn't_. For most of _us_, actually, Lucene is ideally positionned to remain the IR leader, _especially_ if BM25+HNSW is supported on the same level as BM25-only use cases. Having Java Vector API BM25 _and_ HNSW implementations certainly won't hurt. But letting Lucene users decide and benchmark their own dense vector size (as all Lucene competitors do) is a _must_.
   
   And I hereby volonteer to help for either or all of the Java Vector API implementations, BM25+HNSW combination workflow helpers, and HNSW support of arbitrarily-sized vectors. Not limiting my help to just design and programming, but also benchmarking, documenting and whatnot. Who and where should I talk to about that?
   
   As for Rust, it's just not on par with Java as far as concurrency goes. Rust is _no one's_  language of choice to build IR distributed web services.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For additional commands, e-mail: issues-help@lucene.apache.org