You are viewing a plain text version of this content. The canonical link for it is here.
Posted to solr-user@lucene.apache.org by Zheng Lin Edwin Yeo <ed...@gmail.com> on 2015/05/26 07:34:32 UTC
Solr relevancy score in percentage
Hi,
Would like to check, does the new version of Solr allows this function of
display the relevancy score in percentage?
I understand from the older version that it is not able to, and the only
way is to take the highest score and use that as 100%, and calculate other
percentage from that number (For example if the max score is 10 and the
next result has a score of 5, you would do (5 / 10) * 100 = 50%)
Is there a better way to do this now? I'm using Solr 5.1
Regards,
Edwin
Re: Solr relevancy score in percentage
Posted by Erick Erickson <er...@gmail.com>.
This is one of those things that is, IMO, strictly a "feel good" thing
that's sometimes insisted upon
by the product manager and all the information in the world about
"this is really meaningless" falls
on deaf ears.
If you simply have no choice (a position I've been because it wasn't
worth the argument), you can
do the star thing. That is, display 5 stars for percentages between
80-100, 4 stars for 60-80 etc. and
not display the percentages or raw scores at all.
But as others have said, it really isn't providing any additional
information, and IMO misleading
the user...
Best,
Erick
On Tue, May 26, 2015 at 8:31 AM, Alessandro Benedetti
<be...@gmail.com> wrote:
> Honeslty the only case where the score in percentage could make sense, is
> for the More Like This.
> In that case Solr should provide that feature as we perfectly know that the
> 100 % similar score is a copy of the seed document.
>
> If I am right, because of the MLT implementation, not taking care of the
> identity score, we are getting there weird scores as well.
> Maybe in there is the only place I would prefer a percentage.
>
> Cheers
>
> 2015-05-26 16:23 GMT+01:00 Zheng Lin Edwin Yeo <ed...@gmail.com>:
>
>> Currently I've take the score that I get from Solr, and divide it by the
>> maxScore, and multiply it by 100 to get the percentage. All these are done
>> on the coding for the UI. The user will only see the percentage and will
>> not know anything about the score. Since the score by itself is
>> meaningless, so I don't think I should display that score of like 1.7 or
>> 0.2 on the UI, which could further confuse the user and raise alot more
>> questions.
>>
>> Regards,
>> Edwin
>>
>>
>>
>> On 26 May 2015 at 23:07, Shawn Heisey <ap...@elyograg.org> wrote:
>>
>> > On 5/26/2015 8:10 AM, Zheng Lin Edwin Yeo wrote:
>> > > We want the user to see how relevant the result is with respect to the
>> > > search query entered, and not how "good" the results are.
>> > > But I suspect a problem is that the 1st record will always be 100%,
>> > > regardless of what is the score, as the 1st record score will always be
>> > > equals to the maxScore.
>> >
>> > If you want to give your users *something* then simply display the score
>> > that you get from Solr. I recommend that you DON'T give them maxScore,
>> > because they will be tempted to make the percentage calculation
>> > themselves to try and find meaning where there is none. A clever user
>> > will be able to figure out maxScore for themselves simply by sorting on
>> > relevance and looking at the score on the top doc.
>> >
>> > When you get questions about what the number means, and you *WILL* get
>> > those questions, you can tell them that the number itself is meaningless
>> > and what matters is how the scores within a single result compare to
>> > each other -- exactly what you have been told here.
>> >
>> > Thanks,
>> > Shawn
>> >
>> >
>>
>
>
>
> --
> --------------------------
>
> Benedetti Alessandro
> Visiting card : http://about.me/alessandro_benedetti
>
> "Tyger, tyger burning bright
> In the forests of the night,
> What immortal hand or eye
> Could frame thy fearful symmetry?"
>
> William Blake - Songs of Experience -1794 England
Re: Solr relevancy score in percentage
Posted by Alessandro Benedetti <be...@gmail.com>.
Honeslty the only case where the score in percentage could make sense, is
for the More Like This.
In that case Solr should provide that feature as we perfectly know that the
100 % similar score is a copy of the seed document.
If I am right, because of the MLT implementation, not taking care of the
identity score, we are getting there weird scores as well.
Maybe in there is the only place I would prefer a percentage.
Cheers
2015-05-26 16:23 GMT+01:00 Zheng Lin Edwin Yeo <ed...@gmail.com>:
> Currently I've take the score that I get from Solr, and divide it by the
> maxScore, and multiply it by 100 to get the percentage. All these are done
> on the coding for the UI. The user will only see the percentage and will
> not know anything about the score. Since the score by itself is
> meaningless, so I don't think I should display that score of like 1.7 or
> 0.2 on the UI, which could further confuse the user and raise alot more
> questions.
>
> Regards,
> Edwin
>
>
>
> On 26 May 2015 at 23:07, Shawn Heisey <ap...@elyograg.org> wrote:
>
> > On 5/26/2015 8:10 AM, Zheng Lin Edwin Yeo wrote:
> > > We want the user to see how relevant the result is with respect to the
> > > search query entered, and not how "good" the results are.
> > > But I suspect a problem is that the 1st record will always be 100%,
> > > regardless of what is the score, as the 1st record score will always be
> > > equals to the maxScore.
> >
> > If you want to give your users *something* then simply display the score
> > that you get from Solr. I recommend that you DON'T give them maxScore,
> > because they will be tempted to make the percentage calculation
> > themselves to try and find meaning where there is none. A clever user
> > will be able to figure out maxScore for themselves simply by sorting on
> > relevance and looking at the score on the top doc.
> >
> > When you get questions about what the number means, and you *WILL* get
> > those questions, you can tell them that the number itself is meaningless
> > and what matters is how the scores within a single result compare to
> > each other -- exactly what you have been told here.
> >
> > Thanks,
> > Shawn
> >
> >
>
--
--------------------------
Benedetti Alessandro
Visiting card : http://about.me/alessandro_benedetti
"Tyger, tyger burning bright
In the forests of the night,
What immortal hand or eye
Could frame thy fearful symmetry?"
William Blake - Songs of Experience -1794 England
Re: Solr relevancy score in percentage
Posted by Zheng Lin Edwin Yeo <ed...@gmail.com>.
Currently I've take the score that I get from Solr, and divide it by the
maxScore, and multiply it by 100 to get the percentage. All these are done
on the coding for the UI. The user will only see the percentage and will
not know anything about the score. Since the score by itself is
meaningless, so I don't think I should display that score of like 1.7 or
0.2 on the UI, which could further confuse the user and raise alot more
questions.
Regards,
Edwin
On 26 May 2015 at 23:07, Shawn Heisey <ap...@elyograg.org> wrote:
> On 5/26/2015 8:10 AM, Zheng Lin Edwin Yeo wrote:
> > We want the user to see how relevant the result is with respect to the
> > search query entered, and not how "good" the results are.
> > But I suspect a problem is that the 1st record will always be 100%,
> > regardless of what is the score, as the 1st record score will always be
> > equals to the maxScore.
>
> If you want to give your users *something* then simply display the score
> that you get from Solr. I recommend that you DON'T give them maxScore,
> because they will be tempted to make the percentage calculation
> themselves to try and find meaning where there is none. A clever user
> will be able to figure out maxScore for themselves simply by sorting on
> relevance and looking at the score on the top doc.
>
> When you get questions about what the number means, and you *WILL* get
> those questions, you can tell them that the number itself is meaningless
> and what matters is how the scores within a single result compare to
> each other -- exactly what you have been told here.
>
> Thanks,
> Shawn
>
>
Re: Solr relevancy score in percentage
Posted by Shawn Heisey <ap...@elyograg.org>.
On 5/26/2015 8:10 AM, Zheng Lin Edwin Yeo wrote:
> We want the user to see how relevant the result is with respect to the
> search query entered, and not how "good" the results are.
> But I suspect a problem is that the 1st record will always be 100%,
> regardless of what is the score, as the 1st record score will always be
> equals to the maxScore.
If you want to give your users *something* then simply display the score
that you get from Solr. I recommend that you DON'T give them maxScore,
because they will be tempted to make the percentage calculation
themselves to try and find meaning where there is none. A clever user
will be able to figure out maxScore for themselves simply by sorting on
relevance and looking at the score on the top doc.
When you get questions about what the number means, and you *WILL* get
those questions, you can tell them that the number itself is meaningless
and what matters is how the scores within a single result compare to
each other -- exactly what you have been told here.
Thanks,
Shawn
Re: Solr relevancy score in percentage
Posted by Zheng Lin Edwin Yeo <ed...@gmail.com>.
Thank you everyone for your comments and recommendations. Will consider all
these points in my implementation.
Regards,
Edwin
On 27 May 2015 at 05:15, Walter Underwood <wu...@wunderwood.org> wrote:
> On May 26, 2015, at 7:10 AM, Zheng Lin Edwin Yeo <ed...@gmail.com>
> wrote:
>
> > We want the user to see how relevant the result is with respect to the
> > search query entered, and not how "good" the results are.
>
> That is the meaning of the score from a probabilistic model search engine.
> Solr is not a probabilistic engine, it is vector space engine. The scores
> are fundamentally different. Treating it as a probability of relevance
> will not work.
>
> wunder
> Walter Underwood
> wunder@wunderwood.org
> http://observer.wunderwood.org/ (my blog)
>
>
Re: Solr relevancy score in percentage
Posted by Walter Underwood <wu...@wunderwood.org>.
On May 26, 2015, at 7:10 AM, Zheng Lin Edwin Yeo <ed...@gmail.com> wrote:
> We want the user to see how relevant the result is with respect to the
> search query entered, and not how "good" the results are.
That is the meaning of the score from a probabilistic model search engine. Solr is not a probabilistic engine, it is vector space engine. The scores are fundamentally different. Treating it as a probability of relevance will not work.
wunder
Walter Underwood
wunder@wunderwood.org
http://observer.wunderwood.org/ (my blog)
Re: Solr relevancy score in percentage
Posted by Zheng Lin Edwin Yeo <ed...@gmail.com>.
We want the user to see how relevant the result is with respect to the
search query entered, and not how "good" the results are.
But I suspect a problem is that the 1st record will always be 100%,
regardless of what is the score, as the 1st record score will always be
equals to the maxScore.
Regards,
Edwin
On 26 May 2015 at 19:36, Daniel Collins <da...@gmail.com> wrote:
> The question is more why do you want your users to see the scores?
>
> If they are wanting to affect ranking, what you want is the ability to run
> the same query with different boosting and see the difference (2 result
> sets), then see if the new ordering is better or worse. What the
> actual/raw score is is irrelevant to that, what is important is ordering?
> If you want to show how "good" your results are, then as the link shows,
> that is very difficult to measure (and very subjective!)
>
> On 26 May 2015 at 09:37, Upayavira <uv...@odoko.co.uk> wrote:
>
> > Correct. The relevancy score simply states that we think result #1 is
> > more relevant than result #2. It doesn't say that #1 is relevant.
> >
> > The score doesn't have any validity across queries either, as, for
> > example, a different number of query terms will cause the score to
> > change.
> >
> > Upayavira
> >
> > On Tue, May 26, 2015, at 08:57 AM, Zheng Lin Edwin Yeo wrote:
> > > Hi Arslan,
> > >
> > > Thank you for the link. That means we are not advisable to show
> anything
> > > that's related to the relevancy score, even though the default sorting
> of
> > > the result is by relevancy score? Since showing the raw relevancy score
> > > does not make any sense to the user since they won't understand what it
> > > means too.
> > >
> > >
> > > Regards,
> > > Edwin
> > >
> > >
> > >
> > > On 26 May 2015 at 14:16, Ahmet Arslan <io...@yahoo.com.invalid>
> wrote:
> > >
> > > > Hi Edwin,
> > > >
> > > > Somehow, it is not recommended to display the relevancy score in
> > > > percentage:
> > > > https://wiki.apache.org/lucene-java/ScoresAsPercentages
> > > >
> > > > Ahmet
> > > >
> > > >
> > > >
> > > > On Tuesday, May 26, 2015 8:34 AM, Zheng Lin Edwin Yeo <
> > > > edwinyeozl@gmail.com> wrote:
> > > > Hi,
> > > >
> > > > Would like to check, does the new version of Solr allows this
> function
> > of
> > > > display the relevancy score in percentage?
> > > > I understand from the older version that it is not able to, and the
> > only
> > > > way is to take the highest score and use that as 100%, and calculate
> > other
> > > > percentage from that number (For example if the max score is 10 and
> the
> > > > next result has a score of 5, you would do (5 / 10) * 100 = 50%)
> > > >
> > > > Is there a better way to do this now? I'm using Solr 5.1
> > > >
> > > >
> > > > Regards,
> > > > Edwin
> > > >
> >
>
Re: Solr relevancy score in percentage
Posted by Daniel Collins <da...@gmail.com>.
The question is more why do you want your users to see the scores?
If they are wanting to affect ranking, what you want is the ability to run
the same query with different boosting and see the difference (2 result
sets), then see if the new ordering is better or worse. What the
actual/raw score is is irrelevant to that, what is important is ordering?
If you want to show how "good" your results are, then as the link shows,
that is very difficult to measure (and very subjective!)
On 26 May 2015 at 09:37, Upayavira <uv...@odoko.co.uk> wrote:
> Correct. The relevancy score simply states that we think result #1 is
> more relevant than result #2. It doesn't say that #1 is relevant.
>
> The score doesn't have any validity across queries either, as, for
> example, a different number of query terms will cause the score to
> change.
>
> Upayavira
>
> On Tue, May 26, 2015, at 08:57 AM, Zheng Lin Edwin Yeo wrote:
> > Hi Arslan,
> >
> > Thank you for the link. That means we are not advisable to show anything
> > that's related to the relevancy score, even though the default sorting of
> > the result is by relevancy score? Since showing the raw relevancy score
> > does not make any sense to the user since they won't understand what it
> > means too.
> >
> >
> > Regards,
> > Edwin
> >
> >
> >
> > On 26 May 2015 at 14:16, Ahmet Arslan <io...@yahoo.com.invalid> wrote:
> >
> > > Hi Edwin,
> > >
> > > Somehow, it is not recommended to display the relevancy score in
> > > percentage:
> > > https://wiki.apache.org/lucene-java/ScoresAsPercentages
> > >
> > > Ahmet
> > >
> > >
> > >
> > > On Tuesday, May 26, 2015 8:34 AM, Zheng Lin Edwin Yeo <
> > > edwinyeozl@gmail.com> wrote:
> > > Hi,
> > >
> > > Would like to check, does the new version of Solr allows this function
> of
> > > display the relevancy score in percentage?
> > > I understand from the older version that it is not able to, and the
> only
> > > way is to take the highest score and use that as 100%, and calculate
> other
> > > percentage from that number (For example if the max score is 10 and the
> > > next result has a score of 5, you would do (5 / 10) * 100 = 50%)
> > >
> > > Is there a better way to do this now? I'm using Solr 5.1
> > >
> > >
> > > Regards,
> > > Edwin
> > >
>
Re: Solr relevancy score in percentage
Posted by Upayavira <uv...@odoko.co.uk>.
Correct. The relevancy score simply states that we think result #1 is
more relevant than result #2. It doesn't say that #1 is relevant.
The score doesn't have any validity across queries either, as, for
example, a different number of query terms will cause the score to
change.
Upayavira
On Tue, May 26, 2015, at 08:57 AM, Zheng Lin Edwin Yeo wrote:
> Hi Arslan,
>
> Thank you for the link. That means we are not advisable to show anything
> that's related to the relevancy score, even though the default sorting of
> the result is by relevancy score? Since showing the raw relevancy score
> does not make any sense to the user since they won't understand what it
> means too.
>
>
> Regards,
> Edwin
>
>
>
> On 26 May 2015 at 14:16, Ahmet Arslan <io...@yahoo.com.invalid> wrote:
>
> > Hi Edwin,
> >
> > Somehow, it is not recommended to display the relevancy score in
> > percentage:
> > https://wiki.apache.org/lucene-java/ScoresAsPercentages
> >
> > Ahmet
> >
> >
> >
> > On Tuesday, May 26, 2015 8:34 AM, Zheng Lin Edwin Yeo <
> > edwinyeozl@gmail.com> wrote:
> > Hi,
> >
> > Would like to check, does the new version of Solr allows this function of
> > display the relevancy score in percentage?
> > I understand from the older version that it is not able to, and the only
> > way is to take the highest score and use that as 100%, and calculate other
> > percentage from that number (For example if the max score is 10 and the
> > next result has a score of 5, you would do (5 / 10) * 100 = 50%)
> >
> > Is there a better way to do this now? I'm using Solr 5.1
> >
> >
> > Regards,
> > Edwin
> >
Re: Solr relevancy score in percentage
Posted by Zheng Lin Edwin Yeo <ed...@gmail.com>.
Hi Arslan,
Thank you for the link. That means we are not advisable to show anything
that's related to the relevancy score, even though the default sorting of
the result is by relevancy score? Since showing the raw relevancy score
does not make any sense to the user since they won't understand what it
means too.
Regards,
Edwin
On 26 May 2015 at 14:16, Ahmet Arslan <io...@yahoo.com.invalid> wrote:
> Hi Edwin,
>
> Somehow, it is not recommended to display the relevancy score in
> percentage:
> https://wiki.apache.org/lucene-java/ScoresAsPercentages
>
> Ahmet
>
>
>
> On Tuesday, May 26, 2015 8:34 AM, Zheng Lin Edwin Yeo <
> edwinyeozl@gmail.com> wrote:
> Hi,
>
> Would like to check, does the new version of Solr allows this function of
> display the relevancy score in percentage?
> I understand from the older version that it is not able to, and the only
> way is to take the highest score and use that as 100%, and calculate other
> percentage from that number (For example if the max score is 10 and the
> next result has a score of 5, you would do (5 / 10) * 100 = 50%)
>
> Is there a better way to do this now? I'm using Solr 5.1
>
>
> Regards,
> Edwin
>
Re: Solr relevancy score in percentage
Posted by Ahmet Arslan <io...@yahoo.com.INVALID>.
Hi Edwin,
Somehow, it is not recommended to display the relevancy score in percentage:
https://wiki.apache.org/lucene-java/ScoresAsPercentages
Ahmet
On Tuesday, May 26, 2015 8:34 AM, Zheng Lin Edwin Yeo <ed...@gmail.com> wrote:
Hi,
Would like to check, does the new version of Solr allows this function of
display the relevancy score in percentage?
I understand from the older version that it is not able to, and the only
way is to take the highest score and use that as 100%, and calculate other
percentage from that number (For example if the max score is 10 and the
next result has a score of 5, you would do (5 / 10) * 100 = 50%)
Is there a better way to do this now? I'm using Solr 5.1
Regards,
Edwin