You are viewing a plain text version of this content. The canonical link for it is here.
Posted to solr-user@lucene.apache.org by Taisuke Miyazaki <mi...@lifull.com> on 2020/10/22 08:05:27 UTC
When are the score values evaluated?
Hi,
If you use a high value for the score, the values on the smaller scale are
ignored.
Example :
bq = foo:(1.0)^1.0
bf = sum(2000000)
When I do this, the additional score for "foo" at 1.0 does not affect the
sort order.
I'm assuming this is an issue with the precision of the score floating
point, is that correct?
As a test, if we change the query as follows, the order will change as you
would expect, reflecting the additional score of "foo" when it is 1.0
bq = foo:(1.0)^10
bf = sum(2000000)
How can I avoid this?
The idea I'm thinking of at the moment is to divide the whole thing by an
appropriate number, such as bf= div(sum(2000000),100).
However, this may or may not work as expected depending on when the
floating point operations are done and rounded off.
At what point are score's floats rounded?
1. when sorting
2. when calculating the score
3. when evaluating each function for each bq and bf
Regards,
Taisuke
Re: When are the score values evaluated?
Posted by Taisuke Miyazaki <mi...@lifull.com>.
This was my mistake.
Thank you.
Taisuke
2020年10月23日(金) 15:02 Taisuke Miyazaki <mi...@lifull.com>:
> Thanks.
>
> I analyzed it as explain=true and this is what I found.
> Why does this behave this way?
>
> fq=foo:1
> bq=foo:(1)^1
> bf=sum(2000000)
>
> If you do this, the score will be boosted by bq.
> However, if you remove fq, the score will not be boosted by bq.
> However, if you change the boost value of bq to 2, bq will be boosted
> regardless of whether you have fq or not.
>
> This behavior seems very strange to me. (I'm not familiar with the
> internals of Solr or Lucene).
>
> By the way, this doesn't happen if you change the sum number to a value
> that doesn't need to be expressed as an exponent. (20,000,000 is marked as
> 2.0E7 on EXPLAIN.)
>
> Regards,
> Taisuke
>
> 2020年10月22日(木) 21:41 Erick Erickson <er...@gmail.com>:
>
>> You’d get a much better idea of what goes on
>> if you added &explain=true and analyzed the
>> output. That’d show you exactly what is
>> calculated when.
>>
>> Best,
>> Erick
>>
>> > On Oct 22, 2020, at 4:05 AM, Taisuke Miyazaki <
>> miyazakitaisuke@lifull.com> wrote:
>> >
>> > Hi,
>> >
>> > If you use a high value for the score, the values on the smaller scale
>> are
>> > ignored.
>> >
>> > Example :
>> > bq = foo:(1.0)^1.0
>> > bf = sum(2000000)
>> >
>> > When I do this, the additional score for "foo" at 1.0 does not affect
>> the
>> > sort order.
>> >
>> > I'm assuming this is an issue with the precision of the score floating
>> > point, is that correct?
>> >
>> > As a test, if we change the query as follows, the order will change as
>> you
>> > would expect, reflecting the additional score of "foo" when it is 1.0
>> > bq = foo:(1.0)^10
>> > bf = sum(2000000)
>> >
>> > How can I avoid this?
>> > The idea I'm thinking of at the moment is to divide the whole thing by
>> an
>> > appropriate number, such as bf= div(sum(2000000),100).
>> > However, this may or may not work as expected depending on when the
>> > floating point operations are done and rounded off.
>> >
>> > At what point are score's floats rounded?
>> >
>> > 1. when sorting
>> > 2. when calculating the score
>> > 3. when evaluating each function for each bq and bf
>> >
>> > Regards,
>> > Taisuke
>>
>>
Re: When are the score values evaluated?
Posted by Taisuke Miyazaki <mi...@lifull.com>.
Thanks.
I analyzed it as explain=true and this is what I found.
Why does this behave this way?
fq=foo:1
bq=foo:(1)^1
bf=sum(2000000)
If you do this, the score will be boosted by bq.
However, if you remove fq, the score will not be boosted by bq.
However, if you change the boost value of bq to 2, bq will be boosted
regardless of whether you have fq or not.
This behavior seems very strange to me. (I'm not familiar with the
internals of Solr or Lucene).
By the way, this doesn't happen if you change the sum number to a value
that doesn't need to be expressed as an exponent. (20,000,000 is marked as
2.0E7 on EXPLAIN.)
Regards,
Taisuke
2020年10月22日(木) 21:41 Erick Erickson <er...@gmail.com>:
> You’d get a much better idea of what goes on
> if you added &explain=true and analyzed the
> output. That’d show you exactly what is
> calculated when.
>
> Best,
> Erick
>
> > On Oct 22, 2020, at 4:05 AM, Taisuke Miyazaki <
> miyazakitaisuke@lifull.com> wrote:
> >
> > Hi,
> >
> > If you use a high value for the score, the values on the smaller scale
> are
> > ignored.
> >
> > Example :
> > bq = foo:(1.0)^1.0
> > bf = sum(2000000)
> >
> > When I do this, the additional score for "foo" at 1.0 does not affect the
> > sort order.
> >
> > I'm assuming this is an issue with the precision of the score floating
> > point, is that correct?
> >
> > As a test, if we change the query as follows, the order will change as
> you
> > would expect, reflecting the additional score of "foo" when it is 1.0
> > bq = foo:(1.0)^10
> > bf = sum(2000000)
> >
> > How can I avoid this?
> > The idea I'm thinking of at the moment is to divide the whole thing by an
> > appropriate number, such as bf= div(sum(2000000),100).
> > However, this may or may not work as expected depending on when the
> > floating point operations are done and rounded off.
> >
> > At what point are score's floats rounded?
> >
> > 1. when sorting
> > 2. when calculating the score
> > 3. when evaluating each function for each bq and bf
> >
> > Regards,
> > Taisuke
>
>
Re: When are the score values evaluated?
Posted by Erick Erickson <er...@gmail.com>.
You’d get a much better idea of what goes on
if you added &explain=true and analyzed the
output. That’d show you exactly what is
calculated when.
Best,
Erick
> On Oct 22, 2020, at 4:05 AM, Taisuke Miyazaki <mi...@lifull.com> wrote:
>
> Hi,
>
> If you use a high value for the score, the values on the smaller scale are
> ignored.
>
> Example :
> bq = foo:(1.0)^1.0
> bf = sum(2000000)
>
> When I do this, the additional score for "foo" at 1.0 does not affect the
> sort order.
>
> I'm assuming this is an issue with the precision of the score floating
> point, is that correct?
>
> As a test, if we change the query as follows, the order will change as you
> would expect, reflecting the additional score of "foo" when it is 1.0
> bq = foo:(1.0)^10
> bf = sum(2000000)
>
> How can I avoid this?
> The idea I'm thinking of at the moment is to divide the whole thing by an
> appropriate number, such as bf= div(sum(2000000),100).
> However, this may or may not work as expected depending on when the
> floating point operations are done and rounded off.
>
> At what point are score's floats rounded?
>
> 1. when sorting
> 2. when calculating the score
> 3. when evaluating each function for each bq and bf
>
> Regards,
> Taisuke