You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@solr.apache.org by Sergio García Maroto <ma...@gmail.com> on 2022/04/06 12:58:51 UTC
Solr 8.10.1 performance degradation vs Solr 6.6.1
Hi,
I am in the process of upgrading Solr 6.6.1 to Solr 8.10.1.
In general performance it´s almost the same or even a bit better when
running performance load testing.
There is a particular scenario where I see an important degradation.
That´s when I boost results base on a function. I boost results based on
two fields.
If I take the this part out both Solr 8 and Solr 6 are same
performance. *{!boost
b=sum(scale(PeopleTotalSD,1,2),scale(AssignmentsTotalSD,1,2))}*
Both servers have identical machines and data. Actually results coming
back are the same number.
q={!boost
b=sum(scale(PeopleTotalSD,1,2),scale(AssignmentsTotalSD,1,2))}((CompanyTypeSFD:("Private
Company") OR ((CompNameFreeTextS:(kaiku))^0.5 OR
(CompAliasFreeTextS:(kaiku))^0.5) OR NationalitySFD:(Algeria) OR
CompWebS:(http\:\/\/newCompanyWebsite)) AND ((*:*
-CompanyStatusSFD:("\*\*\*System Delete\*\*\*")) AND
type_level:(parent)))&start=0&rows=7&fl=CompanyID&sort=score desc
Any ideas if this is for some reason?
Regards,
Sergio
Re: Solr 8.10.1 performance degradation vs Solr 6.6.1
Posted by Sergio García Maroto <ma...@gmail.com>.
After testing different options. It seems the scale function has some
performance issue.
Other functions are fine. I had to replace it by log() to get similar
functionallity.
Regards,
Sergio Maroto
On Wed, 6 Apr 2022 at 17:18, Sergio García Maroto <ma...@gmail.com>
wrote:
> Thanks Mike.
> It seems like Solr 8 is using different parsing
>
> *Solr6*
> <str name="rawquerystring">{!boost
> b=sum(scale(PeopleTotalSD,1,2),scale(AssignmentsTotalSD,1,2))}((CompanyTypeSFD:("Private
> Company") OR ((CompNameFreeTextS:(BASF ))^0.5 OR (CompAliasFreeTextS:(BASF
> ))^0.5) OR NationalitySFD:(Algeria) OR
> CompWebS:(http\:\/\/newCompanyWebsite)) AND ((*:*
> -CompanyStatusSFD:("\*\*\*System Delete\*\*\*")) AND type_level:(parent)))
> </str> <str name="querystring">{!boost
> b=sum(scale(PeopleTotalSD,1,2),scale(AssignmentsTotalSD,1,2))}((CompanyTypeSFD:("Private
> Company") OR ((CompNameFreeTextS:(BASF ))^0.5 OR (CompAliasFreeTextS:(BASF
> ))^0.5) OR NationalitySFD:(Algeria) OR
> CompWebS:(http\:\/\/newCompanyWebsite)) AND ((*:*
> -CompanyStatusSFD:("\*\*\*System Delete\*\*\*")) AND type_level:(parent)))
> </str> <str name="parsedquery">BoostedQuery(boost(+(CompanyTypeSFD:Private
> Company ((CompNameFreeTextS:basf)^0.5 (CompAliasFreeTextS:basf)^0.5)
> NationalitySFD:Algeria CompWebS:http://newcompanywebsite) +(+(*:*
> -CompanyStatusSFD:***System Delete***)
> +type_level:parent),sum(scale(int(PeopleTotalSD),1.0,2.0),scale(int(AssignmentsTotalSD),1.0,2.0))))
> </str> <str name="parsedquery_toString">boost(+(CompanyTypeSFD:Private
> Company ((CompNameFreeTextS:basf)^0.5 (CompAliasFreeTextS:basf)^0.5)
> NationalitySFD:Algeria CompWebS:http://newcompanywebsite) +(+(*:*
> -CompanyStatusSFD:***System Delete***)
> +type_level:parent),sum(scale(int(PeopleTotalSD),1.0,2.0),scale(int(AssignmentsTotalSD),1.0,2.0)))
> </str>
>
> *Solr 8*
> <str name="rawquerystring">{!boost
> b=sum(scale(PeopleTotalSD,1,2),scale(AssignmentsTotalSD,1,2))}((CompanyTypeSFD:("Private
> Company") OR ((CompNameFreeTextS:(BASF ))^0.5 OR (CompAliasFreeTextS:(BASF
> ))^0.5) OR NationalitySFD:(Algeria) OR
> CompWebS:(http\:\/\/newCompanyWebsite)) AND ((*:*
> -CompanyStatusSFD:("\*\*\*System Delete\*\*\*")) AND type_level:(parent)))
> </str> <str name="querystring">{!boost
> b=sum(scale(PeopleTotalSD,1,2),scale(AssignmentsTotalSD,1,2))}((CompanyTypeSFD:("Private
> Company") OR ((CompNameFreeTextS:(BASF ))^0.5 OR (CompAliasFreeTextS:(BASF
> ))^0.5) OR NationalitySFD:(Algeria) OR
> CompWebS:(http\:\/\/newCompanyWebsite)) AND ((*:*
> -CompanyStatusSFD:("\*\*\*System Delete\*\*\*")) AND type_level:(parent)))
> </str> <str name="parsedquery">FunctionScoreQuery(FunctionScoreQuery(+(CompanyTypeSFD:Private
> Company ((CompNameFreeTextS:basf)^0.5 (CompAliasFreeTextS:basf)^0.5)
> NationalitySFD:Algeria CompWebS:http://newcompanywebsite) +(+(*:*
> -CompanyStatusSFD:***System Delete***) +type_level:parent), scored by
> boost(sum(scale(int(PeopleTotalSD),1.0,2.0),scale(int(AssignmentsTotalSD),1.0,2.0)))))
> </str> <str name="parsedquery_toString">FunctionScoreQuery(+(CompanyTypeSFD:Private
> Company ((CompNameFreeTextS:basf)^0.5 (CompAliasFreeTextS:basf)^0.5)
> NationalitySFD:Algeria CompWebS:http://newcompanywebsite) +(+(*:*
> -CompanyStatusSFD:***System Delete***) +type_level:parent), scored by
> boost(sum(scale(int(PeopleTotalSD),1.0,2.0),scale(int(AssignmentsTotalSD),1.0,2.0))))
> </str>
>
> On Wed, 6 Apr 2022 at 16:31, Mike Drob <md...@mdrob.com> wrote:
>
>> Can you try running with debug=query to see if the two are getting parsed
>> differently?
>>
>> On Wed, Apr 6, 2022 at 8:26 AM Sergio García Maroto <ma...@gmail.com>
>> wrote:
>>
>> > Forgot to mention.
>> > Solr 8 = 5 seconds
>> > Solr 6 = 1 second
>> >
>> > On Wed, 6 Apr 2022 at 14:58, Sergio García Maroto <ma...@gmail.com>
>> > wrote:
>> >
>> > > Hi,
>> > >
>> > > I am in the process of upgrading Solr 6.6.1 to Solr 8.10.1.
>> > > In general performance it´s almost the same or even a bit better when
>> > > running performance load testing.
>> > >
>> > > There is a particular scenario where I see an important degradation.
>> > > That´s when I boost results base on a function. I boost results based
>> on
>> > > two fields.
>> > > If I take the this part out both Solr 8 and Solr 6 are same
>> > performance. *{!boost
>> > > b=sum(scale(PeopleTotalSD,1,2),scale(AssignmentsTotalSD,1,2))}*
>> > > Both servers have identical machines and data. Actually results coming
>> > > back are the same number.
>> > >
>> > > q={!boost
>> > >
>> >
>> b=sum(scale(PeopleTotalSD,1,2),scale(AssignmentsTotalSD,1,2))}((CompanyTypeSFD:("Private
>> > > Company") OR ((CompNameFreeTextS:(kaiku))^0.5 OR
>> > > (CompAliasFreeTextS:(kaiku))^0.5) OR NationalitySFD:(Algeria) OR
>> > > CompWebS:(http\:\/\/newCompanyWebsite)) AND ((*:*
>> > > -CompanyStatusSFD:("\*\*\*System Delete\*\*\*")) AND
>> > > type_level:(parent)))&start=0&rows=7&fl=CompanyID&sort=score desc
>> > >
>> > > Any ideas if this is for some reason?
>> > >
>> > > Regards,
>> > > Sergio
>> > >
>> >
>>
>
Re: Solr 8.10.1 performance degradation vs Solr 6.6.1
Posted by Sergio García Maroto <ma...@gmail.com>.
Thanks Mike.
It seems like Solr 8 is using different parsing
*Solr6*
<str name="rawquerystring">{!boost
b=sum(scale(PeopleTotalSD,1,2),scale(AssignmentsTotalSD,1,2))}((CompanyTypeSFD:("Private
Company") OR ((CompNameFreeTextS:(BASF ))^0.5 OR (CompAliasFreeTextS:(BASF
))^0.5) OR NationalitySFD:(Algeria) OR
CompWebS:(http\:\/\/newCompanyWebsite)) AND ((*:*
-CompanyStatusSFD:("\*\*\*System Delete\*\*\*")) AND type_level:(parent)))</
str> <str name="querystring">{!boost
b=sum(scale(PeopleTotalSD,1,2),scale(AssignmentsTotalSD,1,2))}((CompanyTypeSFD:("Private
Company") OR ((CompNameFreeTextS:(BASF ))^0.5 OR (CompAliasFreeTextS:(BASF
))^0.5) OR NationalitySFD:(Algeria) OR
CompWebS:(http\:\/\/newCompanyWebsite)) AND ((*:*
-CompanyStatusSFD:("\*\*\*System Delete\*\*\*")) AND type_level:(parent)))</
str> <str name="parsedquery">BoostedQuery(boost(+(CompanyTypeSFD:Private
Company ((CompNameFreeTextS:basf)^0.5 (CompAliasFreeTextS:basf)^0.5)
NationalitySFD:Algeria CompWebS:http://newcompanywebsite) +(+(*:*
-CompanyStatusSFD:***System Delete***)
+type_level:parent),sum(scale(int(PeopleTotalSD),1.0,2.0),scale(int(AssignmentsTotalSD),1.0,2.0))))
</str> <str name="parsedquery_toString">boost(+(CompanyTypeSFD:Private
Company ((CompNameFreeTextS:basf)^0.5 (CompAliasFreeTextS:basf)^0.5)
NationalitySFD:Algeria CompWebS:http://newcompanywebsite) +(+(*:*
-CompanyStatusSFD:***System Delete***)
+type_level:parent),sum(scale(int(PeopleTotalSD),1.0,2.0),scale(int(AssignmentsTotalSD),1.0,2.0)))
</str>
*Solr 8*
<str name="rawquerystring">{!boost
b=sum(scale(PeopleTotalSD,1,2),scale(AssignmentsTotalSD,1,2))}((CompanyTypeSFD:("Private
Company") OR ((CompNameFreeTextS:(BASF ))^0.5 OR (CompAliasFreeTextS:(BASF
))^0.5) OR NationalitySFD:(Algeria) OR
CompWebS:(http\:\/\/newCompanyWebsite)) AND ((*:*
-CompanyStatusSFD:("\*\*\*System Delete\*\*\*")) AND type_level:(parent)))</
str> <str name="querystring">{!boost
b=sum(scale(PeopleTotalSD,1,2),scale(AssignmentsTotalSD,1,2))}((CompanyTypeSFD:("Private
Company") OR ((CompNameFreeTextS:(BASF ))^0.5 OR (CompAliasFreeTextS:(BASF
))^0.5) OR NationalitySFD:(Algeria) OR
CompWebS:(http\:\/\/newCompanyWebsite)) AND ((*:*
-CompanyStatusSFD:("\*\*\*System Delete\*\*\*")) AND type_level:(parent)))</
str> <str name="parsedquery">FunctionScoreQuery(FunctionScoreQuery(+(CompanyTypeSFD:Private
Company ((CompNameFreeTextS:basf)^0.5 (CompAliasFreeTextS:basf)^0.5)
NationalitySFD:Algeria CompWebS:http://newcompanywebsite) +(+(*:*
-CompanyStatusSFD:***System Delete***) +type_level:parent), scored by
boost(sum(scale(int(PeopleTotalSD),1.0,2.0),scale(int(AssignmentsTotalSD),1.0,2.0)))))
</str> <str name="parsedquery_toString">FunctionScoreQuery(+(CompanyTypeSFD:Private
Company ((CompNameFreeTextS:basf)^0.5 (CompAliasFreeTextS:basf)^0.5)
NationalitySFD:Algeria CompWebS:http://newcompanywebsite) +(+(*:*
-CompanyStatusSFD:***System Delete***) +type_level:parent), scored by
boost(sum(scale(int(PeopleTotalSD),1.0,2.0),scale(int(AssignmentsTotalSD),1.0,2.0))))
</str>
On Wed, 6 Apr 2022 at 16:31, Mike Drob <md...@mdrob.com> wrote:
> Can you try running with debug=query to see if the two are getting parsed
> differently?
>
> On Wed, Apr 6, 2022 at 8:26 AM Sergio García Maroto <ma...@gmail.com>
> wrote:
>
> > Forgot to mention.
> > Solr 8 = 5 seconds
> > Solr 6 = 1 second
> >
> > On Wed, 6 Apr 2022 at 14:58, Sergio García Maroto <ma...@gmail.com>
> > wrote:
> >
> > > Hi,
> > >
> > > I am in the process of upgrading Solr 6.6.1 to Solr 8.10.1.
> > > In general performance it´s almost the same or even a bit better when
> > > running performance load testing.
> > >
> > > There is a particular scenario where I see an important degradation.
> > > That´s when I boost results base on a function. I boost results based
> on
> > > two fields.
> > > If I take the this part out both Solr 8 and Solr 6 are same
> > performance. *{!boost
> > > b=sum(scale(PeopleTotalSD,1,2),scale(AssignmentsTotalSD,1,2))}*
> > > Both servers have identical machines and data. Actually results coming
> > > back are the same number.
> > >
> > > q={!boost
> > >
> >
> b=sum(scale(PeopleTotalSD,1,2),scale(AssignmentsTotalSD,1,2))}((CompanyTypeSFD:("Private
> > > Company") OR ((CompNameFreeTextS:(kaiku))^0.5 OR
> > > (CompAliasFreeTextS:(kaiku))^0.5) OR NationalitySFD:(Algeria) OR
> > > CompWebS:(http\:\/\/newCompanyWebsite)) AND ((*:*
> > > -CompanyStatusSFD:("\*\*\*System Delete\*\*\*")) AND
> > > type_level:(parent)))&start=0&rows=7&fl=CompanyID&sort=score desc
> > >
> > > Any ideas if this is for some reason?
> > >
> > > Regards,
> > > Sergio
> > >
> >
>
Re: Solr 8.10.1 performance degradation vs Solr 6.6.1
Posted by Mike Drob <md...@mdrob.com>.
Can you try running with debug=query to see if the two are getting parsed
differently?
On Wed, Apr 6, 2022 at 8:26 AM Sergio García Maroto <ma...@gmail.com>
wrote:
> Forgot to mention.
> Solr 8 = 5 seconds
> Solr 6 = 1 second
>
> On Wed, 6 Apr 2022 at 14:58, Sergio García Maroto <ma...@gmail.com>
> wrote:
>
> > Hi,
> >
> > I am in the process of upgrading Solr 6.6.1 to Solr 8.10.1.
> > In general performance it´s almost the same or even a bit better when
> > running performance load testing.
> >
> > There is a particular scenario where I see an important degradation.
> > That´s when I boost results base on a function. I boost results based on
> > two fields.
> > If I take the this part out both Solr 8 and Solr 6 are same
> performance. *{!boost
> > b=sum(scale(PeopleTotalSD,1,2),scale(AssignmentsTotalSD,1,2))}*
> > Both servers have identical machines and data. Actually results coming
> > back are the same number.
> >
> > q={!boost
> >
> b=sum(scale(PeopleTotalSD,1,2),scale(AssignmentsTotalSD,1,2))}((CompanyTypeSFD:("Private
> > Company") OR ((CompNameFreeTextS:(kaiku))^0.5 OR
> > (CompAliasFreeTextS:(kaiku))^0.5) OR NationalitySFD:(Algeria) OR
> > CompWebS:(http\:\/\/newCompanyWebsite)) AND ((*:*
> > -CompanyStatusSFD:("\*\*\*System Delete\*\*\*")) AND
> > type_level:(parent)))&start=0&rows=7&fl=CompanyID&sort=score desc
> >
> > Any ideas if this is for some reason?
> >
> > Regards,
> > Sergio
> >
>
Re: Solr 8.10.1 performance degradation vs Solr 6.6.1
Posted by Sergio García Maroto <ma...@gmail.com>.
Forgot to mention.
Solr 8 = 5 seconds
Solr 6 = 1 second
On Wed, 6 Apr 2022 at 14:58, Sergio García Maroto <ma...@gmail.com>
wrote:
> Hi,
>
> I am in the process of upgrading Solr 6.6.1 to Solr 8.10.1.
> In general performance it´s almost the same or even a bit better when
> running performance load testing.
>
> There is a particular scenario where I see an important degradation.
> That´s when I boost results base on a function. I boost results based on
> two fields.
> If I take the this part out both Solr 8 and Solr 6 are same performance. *{!boost
> b=sum(scale(PeopleTotalSD,1,2),scale(AssignmentsTotalSD,1,2))}*
> Both servers have identical machines and data. Actually results coming
> back are the same number.
>
> q={!boost
> b=sum(scale(PeopleTotalSD,1,2),scale(AssignmentsTotalSD,1,2))}((CompanyTypeSFD:("Private
> Company") OR ((CompNameFreeTextS:(kaiku))^0.5 OR
> (CompAliasFreeTextS:(kaiku))^0.5) OR NationalitySFD:(Algeria) OR
> CompWebS:(http\:\/\/newCompanyWebsite)) AND ((*:*
> -CompanyStatusSFD:("\*\*\*System Delete\*\*\*")) AND
> type_level:(parent)))&start=0&rows=7&fl=CompanyID&sort=score desc
>
> Any ideas if this is for some reason?
>
> Regards,
> Sergio
>