You are viewing a plain text version of this content. The canonical link for it is here.
Posted to solr-user@lucene.apache.org by Walter Underwood <wu...@netflix.com> on 2007/10/22 19:06:40 UTC
Performance when indexing or cold cache
We've had some performance problems while Solr is indexing and also when it
starts with a cold cache. I'm still digging through our own logs, but I'd
like to get more info about this, so any ideas or info are welcome.
We have four Solr servers on dual CPU PowerPC machines, 2G of heap, about
100-300 queries/second, 250K docs, Tomcat 6.0.10, not fronted by Apache.
We don't use facets, we sort by score. In general use, there are six
different request handlers called to build a page. Here is one, they
are all very similar.
<requestHandler name="movies_people" class="solr.DisMaxRequestHandler" >
<lst name="defaults">
<float name="tie">0.01</float>
<str name="qf">
exact^8.0 exact_alt^6.0 exact_base^8.0 title^4.0 title_alt^3.0
title_base^4.0 phonetic_hi^1.0
</str>
<str name="pf">
exact^12.0 exact_alt^9.0 exact_base^12.0 title^6.0 title_alt^4.0
title_base^6.0 phonetic_hi^1.5
</str>
<str name="bf">
popularity^2.0
</str>
<str name="fl">
id,type,movieid,personid,genreid,score
</str>
<str name="mm">1</str>
<int name="ps">100</int>
</lst>
<lst name="appends">
<str name="fq">(pushstatus:A AND (type:movie OR type:person))</str>
</lst>
</requestHandler>
wunder
Re: Performance when indexing or cold cache
Posted by sunnyfr <jo...@gmail.com>.
Hi Walter,
Did you find a way to sort out your issue, I would be very interested.
Thanks a lot,
Walter Underwood wrote:
>
> We've had some performance problems while Solr is indexing and also when
> it
> starts with a cold cache. I'm still digging through our own logs, but I'd
> like to get more info about this, so any ideas or info are welcome.
>
> We have four Solr servers on dual CPU PowerPC machines, 2G of heap, about
> 100-300 queries/second, 250K docs, Tomcat 6.0.10, not fronted by Apache.
> We don't use facets, we sort by score. In general use, there are six
> different request handlers called to build a page. Here is one, they
> are all very similar.
>
> <requestHandler name="movies_people" class="solr.DisMaxRequestHandler" >
> <lst name="defaults">
> <float name="tie">0.01</float>
> <str name="qf">
> exact^8.0 exact_alt^6.0 exact_base^8.0 title^4.0 title_alt^3.0
> title_base^4.0 phonetic_hi^1.0
> </str>
> <str name="pf">
> exact^12.0 exact_alt^9.0 exact_base^12.0 title^6.0 title_alt^4.0
> title_base^6.0 phonetic_hi^1.5
> </str>
> <str name="bf">
> popularity^2.0
> </str>
> <str name="fl">
> id,type,movieid,personid,genreid,score
> </str>
> <str name="mm">1</str>
> <int name="ps">100</int>
> </lst>
> <lst name="appends">
> <str name="fq">(pushstatus:A AND (type:movie OR type:person))</str>
> </lst>
> </requestHandler>
>
> wunder
>
>
>
>
--
View this message in context: http://www.nabble.com/Performance-when-indexing-or-cold-cache-tp13348420p22984912.html
Sent from the Solr - User mailing list archive at Nabble.com.
Re: Performance when indexing or cold cache
Posted by Walter Underwood <wu...@netflix.com>.
We do an optimize after indexing, so the number of segments
isn't an issue. We have the default autowarming settings.
wunder
On 10/22/07 11:00 AM, "Yonik Seeley" <yo...@apache.org> wrote:
> On 10/22/07, Walter Underwood <wu...@netflix.com> wrote:
>> <lst name="appends">
>> <str name="fq">(pushstatus:A AND (type:movie OR type:person))</str>
>> </lst>
>> </requestHandler>
>
> Perhaps try setting up a static warming query for this filter and any
> other common filters?
>
> Also look for correlations between when slow queries happen and the
> number of segments in the index (and perhaps lower mergeFactor to
> compensate if possible).
>
> -Yonik
Re: Performance when indexing or cold cache
Posted by Yonik Seeley <yo...@apache.org>.
On 10/22/07, Walter Underwood <wu...@netflix.com> wrote:
> <lst name="appends">
> <str name="fq">(pushstatus:A AND (type:movie OR type:person))</str>
> </lst>
> </requestHandler>
Perhaps try setting up a static warming query for this filter and any
other common filters?
Also look for correlations between when slow queries happen and the
number of segments in the index (and perhaps lower mergeFactor to
compensate if possible).
-Yonik
Re: Performance when indexing or cold cache
Posted by Walter Underwood <wu...@netflix.com>.
Solr 1.1. --wunder
On 10/22/07 10:06 AM, "Walter Underwood" <wu...@netflix.com> wrote:
> We've had some performance problems while Solr is indexing and also when it
> starts with a cold cache. I'm still digging through our own logs, but I'd
> like to get more info about this, so any ideas or info are welcome.
>
> We have four Solr servers on dual CPU PowerPC machines, 2G of heap, about
> 100-300 queries/second, 250K docs, Tomcat 6.0.10, not fronted by Apache.
> We don't use facets, we sort by score. In general use, there are six
> different request handlers called to build a page. Here is one, they
> are all very similar.
>
> <requestHandler name="movies_people" class="solr.DisMaxRequestHandler" >
> <lst name="defaults">
> <float name="tie">0.01</float>
> <str name="qf">
> exact^8.0 exact_alt^6.0 exact_base^8.0 title^4.0 title_alt^3.0
> title_base^4.0 phonetic_hi^1.0
> </str>
> <str name="pf">
> exact^12.0 exact_alt^9.0 exact_base^12.0 title^6.0 title_alt^4.0
> title_base^6.0 phonetic_hi^1.5
> </str>
> <str name="bf">
> popularity^2.0
> </str>
> <str name="fl">
> id,type,movieid,personid,genreid,score
> </str>
> <str name="mm">1</str>
> <int name="ps">100</int>
> </lst>
> <lst name="appends">
> <str name="fq">(pushstatus:A AND (type:movie OR type:person))</str>
> </lst>
> </requestHandler>
>
> wunder
>
>