You are viewing a plain text version of this content. The canonical link for it is here.
Posted to solr-user@lucene.apache.org by "Tao, Jing" <jt...@webmd.net> on 2014/09/10 21:28:42 UTC
Inconsistent relevancy score between browser refreshes
I am seeing different relevancy scores for the same documents, between browser refreshes. Any ideas why? The query is the same, index is the same - why would score change?
Example:
First request returns:
<doc>
<str name="title">Stroke Anticoagulation and Prophylaxis</str>
<float name="score">3.463463</float>
</doc>
<doc>
<str name="title">Hemorrhagic Stroke</str>
<float name="score">3.463463</float>
</doc>
<doc>
<str name="title">Vertebrobasilar Stroke</str>
<float name="score">3.460521</float>
</doc>
Second request:
<doc>
<str name="title">Vertebrobasilar Stroke</str>
<float name="score">3.460521</float>
</doc>
<doc>
<str name="title">Hemorrhagic Stroke</str>
<float name="score">3.4484053</float>
</doc>
<doc>
<str name="title">Stroke Anticoagulation and Prophylaxis</str>
<float name="score">3.4484053</float>
</doc>
Third request:
<doc>
<str name="title">Stroke Anticoagulation and Prophylaxis</str>
<float name="score">3.463463</float>
</doc>
<doc>
<str name="title">Hemorrhagic Stroke</str>
<float name="score">3.463463</float>
</doc>
<doc>
<str name="title">Vertebrobasilar Stroke</str>
<float name="score">3.402718</float>
</doc>
Jing
RE: Inconsistent relevancy score between browser refreshes
Posted by "Tao, Jing" <jt...@webmd.net>.
Thanks Erick! After optimization, the scores don't change anymore. Now the only time order may change between browser refreshes is if the score is exactly the same between documents. But that's understandable.
First Request:
<doc>
<str name="id">refcenter_323409</str>
<str name="title">Vertebrobasilar Stroke</str>
<float name="score">3.4728973</float>
</doc>
<doc>
<str name="id">refcenter_1916662</str>
<str name="title">Hemorrhagic Stroke</str>
<float name="score">3.4613633</float>
</doc>
<doc>
<str name="id">refcenter_1160021</str>
<str name="title">Stroke Anticoagulation and Prophylaxis</str>
<float name="score">3.4613633</float>
</doc>
Second Request:
<doc>
<str name="id">refcenter_323409</str>
<str name="title">Vertebrobasilar Stroke</str>
<float name="score">3.4728973</float>
</doc>
<doc>
<str name="id">refcenter_1160021</str>
<str name="title">Stroke Anticoagulation and Prophylaxis</str>
<float name="score">3.4613633</float>
</doc>
<doc>
<str name="id">refcenter_1916662</str>
<str name="title">Hemorrhagic Stroke</str>
<float name="score">3.4613633</float>
</doc>
Jing
-----Original Message-----
From: Erick Erickson [mailto:erickerickson@gmail.com]
Sent: Thursday, September 11, 2014 5:35 PM
To: solr-user@lucene.apache.org
Subject: Re: Inconsistent relevancy score between browser refreshes
If you look at your shards individually, I'll be that you'll find a slight difference in the number of deleted docs. You can add &distrib=false to the query and it will be entirely served on a single node.
Indexes will be slightly different on various shards due to differing merges, and my theory is that you're seeing a very slight difference in score due to differing tf/idf statistics. Unfortunately, the stats include deleted docs which are purged on segment merge. Since the merges may be at different times on the various shards, you may have very slightly different scores.
A simple test would be to optimize. That should purge all the deleted docs' data.
Best,
Erick
On Wed, Sep 10, 2014 at 1:19 PM, Tao, Jing <jt...@webmd.net> wrote:
> 1) It is a SolrCloud setup on 4 servers, 4 shards, replication factor of 2.
> 2) There is no indexing going on.
> 3) No, I did not optimize.
> 4) Did not optimize between refreshes.
>
> Thanks,
> Jing
>
> -----Original Message-----
> From: Erick Erickson [mailto:erickerickson@gmail.com]
> Sent: Wednesday, September 10, 2014 4:09 PM
> To: solr-user@lucene.apache.org
> Subject: Re: Inconsistent relevancy score between browser refreshes
>
> More info please.
>
> 1> Are there replicas involved?
> 2> Is there any indexing going on?
> 3> If more than one node, did you optimize?
> 4> Did you optimize between refreshes?
>
> Best,
> Erick
>
> On Wed, Sep 10, 2014 at 12:28 PM, Tao, Jing <jt...@webmd.net> wrote:
>> I am seeing different relevancy scores for the same documents, between browser refreshes. Any ideas why? The query is the same, index is the same - why would score change?
>>
>> Example:
>> First request returns:
>> <doc>
>> <str name="title">Stroke Anticoagulation and Prophylaxis</str> <float
>> name="score">3.463463</float> </doc> <doc> <str
>> name="title">Hemorrhagic Stroke</str> <float
>> name="score">3.463463</float> </doc> <doc> <str
>> name="title">Vertebrobasilar Stroke</str> <float
>> name="score">3.460521</float> </doc>
>>
>> Second request:
>> <doc>
>> <str name="title">Vertebrobasilar Stroke</str> <float
>> name="score">3.460521</float> </doc> <doc> <str
>> name="title">Hemorrhagic Stroke</str> <float
>> name="score">3.4484053</float> </doc> <doc> <str name="title">Stroke
>> Anticoagulation and Prophylaxis</str> <float
>> name="score">3.4484053</float> </doc>
>>
>> Third request:
>> <doc>
>> <str name="title">Stroke Anticoagulation and Prophylaxis</str> <float
>> name="score">3.463463</float> </doc> <doc> <str
>> name="title">Hemorrhagic Stroke</str> <float
>> name="score">3.463463</float> </doc> <doc> <str
>> name="title">Vertebrobasilar Stroke</str> <float
>> name="score">3.402718</float> </doc>
>>
>>
>> Jing
Re: Inconsistent relevancy score between browser refreshes
Posted by Erick Erickson <er...@gmail.com>.
If you look at your shards individually, I'll be that you'll find a
slight difference in the number of deleted docs. You can add
&distrib=false to the query and it will be entirely served on a single
node.
Indexes will be slightly different on various shards due to differing
merges, and my theory is that you're seeing a very slight difference
in score due to differing tf/idf statistics. Unfortunately, the stats
include deleted docs which are purged on segment merge. Since the
merges may be at different times on the various shards, you may have
very slightly different scores.
A simple test would be to optimize. That should purge all the deleted
docs' data.
Best,
Erick
On Wed, Sep 10, 2014 at 1:19 PM, Tao, Jing <jt...@webmd.net> wrote:
> 1) It is a SolrCloud setup on 4 servers, 4 shards, replication factor of 2.
> 2) There is no indexing going on.
> 3) No, I did not optimize.
> 4) Did not optimize between refreshes.
>
> Thanks,
> Jing
>
> -----Original Message-----
> From: Erick Erickson [mailto:erickerickson@gmail.com]
> Sent: Wednesday, September 10, 2014 4:09 PM
> To: solr-user@lucene.apache.org
> Subject: Re: Inconsistent relevancy score between browser refreshes
>
> More info please.
>
> 1> Are there replicas involved?
> 2> Is there any indexing going on?
> 3> If more than one node, did you optimize?
> 4> Did you optimize between refreshes?
>
> Best,
> Erick
>
> On Wed, Sep 10, 2014 at 12:28 PM, Tao, Jing <jt...@webmd.net> wrote:
>> I am seeing different relevancy scores for the same documents, between browser refreshes. Any ideas why? The query is the same, index is the same - why would score change?
>>
>> Example:
>> First request returns:
>> <doc>
>> <str name="title">Stroke Anticoagulation and Prophylaxis</str> <float
>> name="score">3.463463</float> </doc> <doc> <str
>> name="title">Hemorrhagic Stroke</str> <float
>> name="score">3.463463</float> </doc> <doc> <str
>> name="title">Vertebrobasilar Stroke</str> <float
>> name="score">3.460521</float> </doc>
>>
>> Second request:
>> <doc>
>> <str name="title">Vertebrobasilar Stroke</str> <float
>> name="score">3.460521</float> </doc> <doc> <str
>> name="title">Hemorrhagic Stroke</str> <float
>> name="score">3.4484053</float> </doc> <doc> <str name="title">Stroke
>> Anticoagulation and Prophylaxis</str> <float
>> name="score">3.4484053</float> </doc>
>>
>> Third request:
>> <doc>
>> <str name="title">Stroke Anticoagulation and Prophylaxis</str> <float
>> name="score">3.463463</float> </doc> <doc> <str
>> name="title">Hemorrhagic Stroke</str> <float
>> name="score">3.463463</float> </doc> <doc> <str
>> name="title">Vertebrobasilar Stroke</str> <float
>> name="score">3.402718</float> </doc>
>>
>>
>> Jing
RE: Inconsistent relevancy score between browser refreshes
Posted by "Tao, Jing" <jt...@webmd.net>.
1) It is a SolrCloud setup on 4 servers, 4 shards, replication factor of 2.
2) There is no indexing going on.
3) No, I did not optimize.
4) Did not optimize between refreshes.
Thanks,
Jing
-----Original Message-----
From: Erick Erickson [mailto:erickerickson@gmail.com]
Sent: Wednesday, September 10, 2014 4:09 PM
To: solr-user@lucene.apache.org
Subject: Re: Inconsistent relevancy score between browser refreshes
More info please.
1> Are there replicas involved?
2> Is there any indexing going on?
3> If more than one node, did you optimize?
4> Did you optimize between refreshes?
Best,
Erick
On Wed, Sep 10, 2014 at 12:28 PM, Tao, Jing <jt...@webmd.net> wrote:
> I am seeing different relevancy scores for the same documents, between browser refreshes. Any ideas why? The query is the same, index is the same - why would score change?
>
> Example:
> First request returns:
> <doc>
> <str name="title">Stroke Anticoagulation and Prophylaxis</str> <float
> name="score">3.463463</float> </doc> <doc> <str
> name="title">Hemorrhagic Stroke</str> <float
> name="score">3.463463</float> </doc> <doc> <str
> name="title">Vertebrobasilar Stroke</str> <float
> name="score">3.460521</float> </doc>
>
> Second request:
> <doc>
> <str name="title">Vertebrobasilar Stroke</str> <float
> name="score">3.460521</float> </doc> <doc> <str
> name="title">Hemorrhagic Stroke</str> <float
> name="score">3.4484053</float> </doc> <doc> <str name="title">Stroke
> Anticoagulation and Prophylaxis</str> <float
> name="score">3.4484053</float> </doc>
>
> Third request:
> <doc>
> <str name="title">Stroke Anticoagulation and Prophylaxis</str> <float
> name="score">3.463463</float> </doc> <doc> <str
> name="title">Hemorrhagic Stroke</str> <float
> name="score">3.463463</float> </doc> <doc> <str
> name="title">Vertebrobasilar Stroke</str> <float
> name="score">3.402718</float> </doc>
>
>
> Jing
Re: Inconsistent relevancy score between browser refreshes
Posted by Erick Erickson <er...@gmail.com>.
More info please.
1> Are there replicas involved?
2> Is there any indexing going on?
3> If more than one node, did you optimize?
4> Did you optimize between refreshes?
Best,
Erick
On Wed, Sep 10, 2014 at 12:28 PM, Tao, Jing <jt...@webmd.net> wrote:
> I am seeing different relevancy scores for the same documents, between browser refreshes. Any ideas why? The query is the same, index is the same - why would score change?
>
> Example:
> First request returns:
> <doc>
> <str name="title">Stroke Anticoagulation and Prophylaxis</str>
> <float name="score">3.463463</float>
> </doc>
> <doc>
> <str name="title">Hemorrhagic Stroke</str>
> <float name="score">3.463463</float>
> </doc>
> <doc>
> <str name="title">Vertebrobasilar Stroke</str>
> <float name="score">3.460521</float>
> </doc>
>
> Second request:
> <doc>
> <str name="title">Vertebrobasilar Stroke</str>
> <float name="score">3.460521</float>
> </doc>
> <doc>
> <str name="title">Hemorrhagic Stroke</str>
> <float name="score">3.4484053</float>
> </doc>
> <doc>
> <str name="title">Stroke Anticoagulation and Prophylaxis</str>
> <float name="score">3.4484053</float>
> </doc>
>
> Third request:
> <doc>
> <str name="title">Stroke Anticoagulation and Prophylaxis</str>
> <float name="score">3.463463</float>
> </doc>
> <doc>
> <str name="title">Hemorrhagic Stroke</str>
> <float name="score">3.463463</float>
> </doc>
> <doc>
> <str name="title">Vertebrobasilar Stroke</str>
> <float name="score">3.402718</float>
> </doc>
>
>
> Jing