You are viewing a plain text version of this content. The canonical link for it is here.
Posted to solr-user@lucene.apache.org by Shawn Heisey <so...@elyograg.org> on 2014/04/04 04:42:35 UTC
Does sorting skip everything having to do with relevancy?
If I provide a sort parameter, will Solr (4.6.1) skip score/boost
processing?
In particular I would like to know what happens if I have a boost
parameter (with a complex function) for edismax search, but I include a
sort parameter on one of my fields. I am using distributed search.
I do know that if debugQuery is used, the score IS calculated, but I'm
talking about when debugQuery is not used.
Thanks,
Shawn
Re: Does sorting skip everything having to do with relevancy?
Posted by Mikhail Khludnev <mk...@griddynamics.com>.
Arhgh... It seems like Functions Queries (obviously) never throw an
exception.
I have to scratch my own which throws always.
Here is the proof that boost is lazy
https://gist.github.com/m-khl/10010541
On Sun, Apr 6, 2014 at 12:54 AM, Shawn Heisey <so...@elyograg.org> wrote:
> On 4/5/2014 1:21 PM, Mikhail Khludnev wrote:
> > I suppose "e" yields syntax error. Therefore, this case doesn't prove
> > anything yet.
> > Haven't you tried sqrt(-1) or log(-1) ?
>
> Using boost=sqrt(-1) is error-free whether I include the sort parameter
> or not. That seems like a bug.
>
> Thanks,
> Shawn
>
>
--
Sincerely yours
Mikhail Khludnev
Principal Engineer,
Grid Dynamics
<http://www.griddynamics.com>
<mk...@griddynamics.com>
Re: Does sorting skip everything having to do with relevancy?
Posted by Shawn Heisey <so...@elyograg.org>.
On 4/5/2014 1:21 PM, Mikhail Khludnev wrote:
> I suppose "e" yields syntax error. Therefore, this case doesn't prove
> anything yet.
> Haven't you tried sqrt(-1) or log(-1) ?
Using boost=sqrt(-1) is error-free whether I include the sort parameter
or not. That seems like a bug.
Thanks,
Shawn
Re: Does sorting skip everything having to do with relevancy?
Posted by Mikhail Khludnev <mk...@griddynamics.com>.
Shawn,
I suppose "e" yields syntax error. Therefore, this case doesn't prove
anything yet.
Haven't you tried sqrt(-1) or log(-1) ?
On Sat, Apr 5, 2014 at 1:47 AM, Shawn Heisey <so...@elyograg.org> wrote:
> On 4/4/2014 3:13 PM, Mikhail Khludnev wrote:
>
>> I suppose SolrIndexSearcher.buildTopDocsCollector() doesn't create a
>> Collector which calls score() in this case. Hence, it shouldn't waste CPU.
>> Just my impression.
>> Haven't you tried to check it supplying some weird formula, which throws
>> exception?
>>
>
> I didn't think of that. That's a good idea -- as long as there's not
> independent code that checks the function in addition to the code that
> actually runs it.
>
> With the following parameters added to an edismax query that otherwise
> works, I get an exception. It works if I change the "e" to 5.
>
> &sort=registered_date asc&boost=sum(5,"e")
>
> I will take Alvaro's suggestion and add "boost=1" to queries that use a
> sort parameter. It's probably a good idea to file that Jira.
>
> Thanks,
> Shawn
>
>
--
Sincerely yours
Mikhail Khludnev
Principal Engineer,
Grid Dynamics
<http://www.griddynamics.com>
<mk...@griddynamics.com>
Re: Does sorting skip everything having to do with relevancy?
Posted by Shawn Heisey <so...@elyograg.org>.
On 4/4/2014 3:13 PM, Mikhail Khludnev wrote:
> I suppose SolrIndexSearcher.buildTopDocsCollector() doesn't create a
> Collector which calls score() in this case. Hence, it shouldn't waste CPU.
> Just my impression.
> Haven't you tried to check it supplying some weird formula, which throws
> exception?
I didn't think of that. That's a good idea -- as long as there's not
independent code that checks the function in addition to the code that
actually runs it.
With the following parameters added to an edismax query that otherwise
works, I get an exception. It works if I change the "e" to 5.
&sort=registered_date asc&boost=sum(5,"e")
I will take Alvaro's suggestion and add "boost=1" to queries that use a
sort parameter. It's probably a good idea to file that Jira.
Thanks,
Shawn
Re: Does sorting skip everything having to do with relevancy?
Posted by Mikhail Khludnev <mk...@griddynamics.com>.
Hello Shawn,
I suppose SolrIndexSearcher.buildTopDocsCollector() doesn't create a
Collector which calls score() in this case. Hence, it shouldn't waste CPU.
Just my impression.
Haven't you tried to check it supplying some weird formula, which throws
exception?
On Sat, Apr 5, 2014 at 12:02 AM, Shawn Heisey <so...@elyograg.org> wrote:
> On 4/4/2014 1:48 PM, Alvaro Cabrerizo wrote:
>
>> If you dont want to waste your cpu time, then comment the boost parameter
>> in the query parser defined in your solrconfig.xml. If you cant do that,
>> then you can overwrite it sending the boost parameter for example using
>> the
>> constant function (e.g. http://..../...&boost=1&sort=your_sort). The
>> parameter boost will be overwritten if it is not defined as an invariant.
>>
>
> Thank you for responding. I know how I can override the behavior, what I
> want to find out is whether or not it's necessary to do so -- if it's not
> necessary because Solr skips it, then everything is good. If it is
> necessary, I can open an issue in Jira asking for Solr to get smarter.
> That way everyone benefits and they don't have to do anything except
> upgrade Solr.
>
> Thanks,
> Shawn
>
>
--
Sincerely yours
Mikhail Khludnev
Principal Engineer,
Grid Dynamics
<http://www.griddynamics.com>
<mk...@griddynamics.com>
Re: Does sorting skip everything having to do with relevancy?
Posted by Shawn Heisey <so...@elyograg.org>.
On 4/4/2014 1:48 PM, Alvaro Cabrerizo wrote:
> If you dont want to waste your cpu time, then comment the boost parameter
> in the query parser defined in your solrconfig.xml. If you cant do that,
> then you can overwrite it sending the boost parameter for example using the
> constant function (e.g. http://..../...&boost=1&sort=your_sort). The
> parameter boost will be overwritten if it is not defined as an invariant.
Thank you for responding. I know how I can override the behavior, what
I want to find out is whether or not it's necessary to do so -- if it's
not necessary because Solr skips it, then everything is good. If it is
necessary, I can open an issue in Jira asking for Solr to get smarter.
That way everyone benefits and they don't have to do anything except
upgrade Solr.
Thanks,
Shawn
Re: Does sorting skip everything having to do with relevancy?
Posted by Alvaro Cabrerizo <to...@gmail.com>.
Hi,
If you dont want to waste your cpu time, then comment the boost parameter
in the query parser defined in your solrconfig.xml. If you cant do that,
then you can overwrite it sending the boost parameter for example using the
constant function (e.g. http://..../...&boost=1&sort=your_sort). The
parameter boost will be overwritten if it is not defined as an invariant.
Regards.
On Fri, Apr 4, 2014 at 4:12 PM, Shawn Heisey <so...@elyograg.org> wrote:
> On 4/4/2014 12:48 AM, Alvaro Cabrerizo wrote:
> > By default solr is using the sort parameter over the "score field". So if
> > you overwrite it using other sort field, yes solr will use the parameter
> > you've provided. Remember, you can use multiple fields for
> > sorting<http://wiki.apache.org/solr/CommonQueryParameters#sort> so
> > you can make something like: sort score desc, your_field1 asc,
> your_field2
> > desc....
> >
> > The score of documents is calculated on every query (it does not depend
> on
> > the sort parameter or the debugQueryParameter) and the debubQuery is
> only a
> > mechanism for showing (or hidding) how score was calculated. If you want
> to
> > see a document score for a particular query (apart from the debugQuery)
> you
> > can ask for it in the solr response adding the parameter *fl=*,score* to
> > your request.
>
> These are things that I already know.
>
> What I want to know is whether Solr has code in place that will avoid
> wasting CPU cycles calculating the score that will never be displayed or
> used, *especially* the complex boost parameter that's in the request
> handler definition (solrconfig.xml).
>
> <str
>
> name="boost">min(recip(abs(ms(NOW/HOUR,registered_date)),1.92901e-10,1.5,1.5),0.85)</str>
>
> Do I need to send 'boost=' as a parameter (along with my sort) to get it
> to avoid that calculation?
>
> Thanks,
> Shawn
>
>
Re: Does sorting skip everything having to do with relevancy?
Posted by Shawn Heisey <so...@elyograg.org>.
On 4/4/2014 12:48 AM, Alvaro Cabrerizo wrote:
> By default solr is using the sort parameter over the "score field". So if
> you overwrite it using other sort field, yes solr will use the parameter
> you've provided. Remember, you can use multiple fields for
> sorting<http://wiki.apache.org/solr/CommonQueryParameters#sort> so
> you can make something like: sort score desc, your_field1 asc, your_field2
> desc....
>
> The score of documents is calculated on every query (it does not depend on
> the sort parameter or the debugQueryParameter) and the debubQuery is only a
> mechanism for showing (or hidding) how score was calculated. If you want to
> see a document score for a particular query (apart from the debugQuery) you
> can ask for it in the solr response adding the parameter *fl=*,score* to
> your request.
These are things that I already know.
What I want to know is whether Solr has code in place that will avoid
wasting CPU cycles calculating the score that will never be displayed or
used, *especially* the complex boost parameter that's in the request
handler definition (solrconfig.xml).
<str
name="boost">min(recip(abs(ms(NOW/HOUR,registered_date)),1.92901e-10,1.5,1.5),0.85)</str>
Do I need to send 'boost=' as a parameter (along with my sort) to get it
to avoid that calculation?
Thanks,
Shawn
Re: Does sorting skip everything having to do with relevancy?
Posted by Alvaro Cabrerizo <to...@gmail.com>.
Hi,
By default solr is using the sort parameter over the "score field". So if
you overwrite it using other sort field, yes solr will use the parameter
you've provided. Remember, you can use multiple fields for
sorting<http://wiki.apache.org/solr/CommonQueryParameters#sort> so
you can make something like: sort score desc, your_field1 asc, your_field2
desc....
The score of documents is calculated on every query (it does not depend on
the sort parameter or the debugQueryParameter) and the debubQuery is only a
mechanism for showing (or hidding) how score was calculated. If you want to
see a document score for a particular query (apart from the debugQuery) you
can ask for it in the solr response adding the parameter *fl=*,score* to
your request.
Regards.
On Fri, Apr 4, 2014 at 4:42 AM, Shawn Heisey <so...@elyograg.org> wrote:
> If I provide a sort parameter, will Solr (4.6.1) skip score/boost
> processing?
>
> In particular I would like to know what happens if I have a boost
> parameter (with a complex function) for edismax search, but I include a
> sort parameter on one of my fields. I am using distributed search.
>
> I do know that if debugQuery is used, the score IS calculated, but I'm
> talking about when debugQuery is not used.
>
> Thanks,
> Shawn
>
>