You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@solr.apache.org by Rajani Maski <ra...@gmail.com> on 2023/05/25 18:42:16 UTC
Vector Search Parser(!vp) : calculate cosine
Hi Solr Users,
Is there a vector search parser that allows computing distance between a
document's vector field and query vector passed as query param? The jiras
SOLR-14397 <https://issues.apache.org/jira/browse/SOLR-14397> SOLR-12890
<https://issues.apache.org/jira/browse/SOLR-12890> suggests vector search
parser "!vp" and the other "cosine" as functions but I couldn't get it to
work on solr 9.1.1 version, is this supported yet, one of the jira state is
still "open"?
What are other alternatives to calculate cosine between vector fields in
solr vs query vector sent as query param?
Thanks,
Rajani
Re: Vector Search Parser(!vp) : calculate cosine
Posted by rajani m <ra...@gmail.com>.
Hi Alessandro,
Thank you, I will look forward to those features.
On Fri, May 26, 2023 at 12:10 PM Alessandro Benedetti <a....@sease.io>
wrote:
> Hi Rajani,
> the issues you mentioned are very old and superseded by more recent works
> in the vector-based search domain.
> The 'knn' query parser that Kumar detailed is the official way of doing
> vector search in Solr.
> Function queries for vector similarities and the possibility of better
> Hybrid search with Learning To Rank and vector-similarity features are
> coming soon!
>
> Some blogs that could help:
> - https://sease.io/2023/01/apache-solr-neural-search-tutorial.html
> - https://sease.io/2022/01/apache-solr-neural-search.html
> - https://sease.io/2022/01/apache-solr-neural-search-knn-benchmark.html
>
> Cheers
>
> --------------------------
> *Alessandro Benedetti*
> Director @ Sease Ltd.
> *Apache Lucene/Solr Committer*
> *Apache Solr PMC Member*
>
> e-mail: a.benedetti@sease.io
>
>
> *Sease* - Information Retrieval Applied
> Consulting | Training | Open Source
>
> Website: Sease.io <http://sease.io/>
> LinkedIn <https://linkedin.com/company/sease-ltd> | Twitter
> <https://twitter.com/seaseltd> | Youtube
> <https://www.youtube.com/channel/UCDx86ZKLYNpI3gzMercM7BQ> | Github
> <https://github.com/seaseltd>
>
>
> On Thu, 25 May 2023 at 22:12, rajani m <ra...@gmail.com> wrote:
>
> > That also invokes knn, correct? I just need the vector math response, a
> > cosine value of the query vector and document vector. For example, the
> > streaming api endpoint vector-math.html#dot-product-and-cosine-similarity
> > <
> >
> https://solr.apache.org/guide/7_5/vector-math.html#dot-product-and-cosine-similarity
> > >
> > looks like one however it is a streaming api, it would have been nice if
> > this was also supported by standard query parser or a function query.
> >
> >
> >
> > On Thu, May 25, 2023 at 3:34 PM kumar gaurav <kg...@gmail.com> wrote:
> >
> > > HI Rajani
> > >
> > > Read here :
> > >
> > >
> >
> https://solr.apache.org/guide/solr/latest/query-guide/dense-vector-search.html
> > >
> > > The knn query parser can be used .
> > >
> > > &q={!knn f=vector topK=10}[1.0, 2.0, 3.0, 4.0]
> > >
> > >
> > > Thanks
> > > Kumar Gaurav
> > >
> > >
> > >
> > > On Fri, 26 May 2023 at 00:14, Rajani Maski <ra...@gmail.com>
> > wrote:
> > >
> > > > Hi Solr Users,
> > > >
> > > > Is there a vector search parser that allows computing distance
> > > between a
> > > > document's vector field and query vector passed as query param? The
> > > jiras
> > > > SOLR-14397 <https://issues.apache.org/jira/browse/SOLR-14397>
> > > SOLR-12890
> > > > <https://issues.apache.org/jira/browse/SOLR-12890> suggests vector
> > > search
> > > > parser "!vp" and the other "cosine" as functions but I couldn't get
> it
> > to
> > > > work on solr 9.1.1 version, is this supported yet, one of the jira
> > state
> > > is
> > > > still "open"?
> > > >
> > > > What are other alternatives to calculate cosine between vector fields
> > in
> > > > solr vs query vector sent as query param?
> > > >
> > > >
> > > > Thanks,
> > > > Rajani
> > > >
> > >
> >
>
Re: Vector Search Parser(!vp) : calculate cosine
Posted by Alessandro Benedetti <a....@sease.io>.
Hi Rajani,
the issues you mentioned are very old and superseded by more recent works
in the vector-based search domain.
The 'knn' query parser that Kumar detailed is the official way of doing
vector search in Solr.
Function queries for vector similarities and the possibility of better
Hybrid search with Learning To Rank and vector-similarity features are
coming soon!
Some blogs that could help:
- https://sease.io/2023/01/apache-solr-neural-search-tutorial.html
- https://sease.io/2022/01/apache-solr-neural-search.html
- https://sease.io/2022/01/apache-solr-neural-search-knn-benchmark.html
Cheers
--------------------------
*Alessandro Benedetti*
Director @ Sease Ltd.
*Apache Lucene/Solr Committer*
*Apache Solr PMC Member*
e-mail: a.benedetti@sease.io
*Sease* - Information Retrieval Applied
Consulting | Training | Open Source
Website: Sease.io <http://sease.io/>
LinkedIn <https://linkedin.com/company/sease-ltd> | Twitter
<https://twitter.com/seaseltd> | Youtube
<https://www.youtube.com/channel/UCDx86ZKLYNpI3gzMercM7BQ> | Github
<https://github.com/seaseltd>
On Thu, 25 May 2023 at 22:12, rajani m <ra...@gmail.com> wrote:
> That also invokes knn, correct? I just need the vector math response, a
> cosine value of the query vector and document vector. For example, the
> streaming api endpoint vector-math.html#dot-product-and-cosine-similarity
> <
> https://solr.apache.org/guide/7_5/vector-math.html#dot-product-and-cosine-similarity
> >
> looks like one however it is a streaming api, it would have been nice if
> this was also supported by standard query parser or a function query.
>
>
>
> On Thu, May 25, 2023 at 3:34 PM kumar gaurav <kg...@gmail.com> wrote:
>
> > HI Rajani
> >
> > Read here :
> >
> >
> https://solr.apache.org/guide/solr/latest/query-guide/dense-vector-search.html
> >
> > The knn query parser can be used .
> >
> > &q={!knn f=vector topK=10}[1.0, 2.0, 3.0, 4.0]
> >
> >
> > Thanks
> > Kumar Gaurav
> >
> >
> >
> > On Fri, 26 May 2023 at 00:14, Rajani Maski <ra...@gmail.com>
> wrote:
> >
> > > Hi Solr Users,
> > >
> > > Is there a vector search parser that allows computing distance
> > between a
> > > document's vector field and query vector passed as query param? The
> > jiras
> > > SOLR-14397 <https://issues.apache.org/jira/browse/SOLR-14397>
> > SOLR-12890
> > > <https://issues.apache.org/jira/browse/SOLR-12890> suggests vector
> > search
> > > parser "!vp" and the other "cosine" as functions but I couldn't get it
> to
> > > work on solr 9.1.1 version, is this supported yet, one of the jira
> state
> > is
> > > still "open"?
> > >
> > > What are other alternatives to calculate cosine between vector fields
> in
> > > solr vs query vector sent as query param?
> > >
> > >
> > > Thanks,
> > > Rajani
> > >
> >
>
Re: Vector Search Parser(!vp) : calculate cosine
Posted by rajani m <ra...@gmail.com>.
That also invokes knn, correct? I just need the vector math response, a
cosine value of the query vector and document vector. For example, the
streaming api endpoint vector-math.html#dot-product-and-cosine-similarity
<https://solr.apache.org/guide/7_5/vector-math.html#dot-product-and-cosine-similarity>
looks like one however it is a streaming api, it would have been nice if
this was also supported by standard query parser or a function query.
On Thu, May 25, 2023 at 3:34 PM kumar gaurav <kg...@gmail.com> wrote:
> HI Rajani
>
> Read here :
>
> https://solr.apache.org/guide/solr/latest/query-guide/dense-vector-search.html
>
> The knn query parser can be used .
>
> &q={!knn f=vector topK=10}[1.0, 2.0, 3.0, 4.0]
>
>
> Thanks
> Kumar Gaurav
>
>
>
> On Fri, 26 May 2023 at 00:14, Rajani Maski <ra...@gmail.com> wrote:
>
> > Hi Solr Users,
> >
> > Is there a vector search parser that allows computing distance
> between a
> > document's vector field and query vector passed as query param? The
> jiras
> > SOLR-14397 <https://issues.apache.org/jira/browse/SOLR-14397>
> SOLR-12890
> > <https://issues.apache.org/jira/browse/SOLR-12890> suggests vector
> search
> > parser "!vp" and the other "cosine" as functions but I couldn't get it to
> > work on solr 9.1.1 version, is this supported yet, one of the jira state
> is
> > still "open"?
> >
> > What are other alternatives to calculate cosine between vector fields in
> > solr vs query vector sent as query param?
> >
> >
> > Thanks,
> > Rajani
> >
>
Re: Vector Search Parser(!vp) : calculate cosine
Posted by kumar gaurav <kg...@gmail.com>.
HI Rajani
Read here :
https://solr.apache.org/guide/solr/latest/query-guide/dense-vector-search.html
The knn query parser can be used .
&q={!knn f=vector topK=10}[1.0, 2.0, 3.0, 4.0]
Thanks
Kumar Gaurav
On Fri, 26 May 2023 at 00:14, Rajani Maski <ra...@gmail.com> wrote:
> Hi Solr Users,
>
> Is there a vector search parser that allows computing distance between a
> document's vector field and query vector passed as query param? The jiras
> SOLR-14397 <https://issues.apache.org/jira/browse/SOLR-14397> SOLR-12890
> <https://issues.apache.org/jira/browse/SOLR-12890> suggests vector search
> parser "!vp" and the other "cosine" as functions but I couldn't get it to
> work on solr 9.1.1 version, is this supported yet, one of the jira state is
> still "open"?
>
> What are other alternatives to calculate cosine between vector fields in
> solr vs query vector sent as query param?
>
>
> Thanks,
> Rajani
>