You are viewing a plain text version of this content. The canonical link for it is here.
Posted to solr-user@lucene.apache.org by Sreeram Vaidyanathan <nv...@live.com> on 2009/09/04 01:47:31 UTC
Re: Sorting performance + replication of index between cores
Did u guys find a solution?
I am having a similar issue.
Setup:
One indexer box & 2 searcher box. Each having 6 different solr-cores
We have a lot of updates (in the range of a couple thousand items every few
mins).
The Snappuller/Snapinstaller pulls and commits every 5 mins.
Query response time peaks to 60+ seconds when a new searcher is being
prepared.
I have disabled the caches (filter, query & document).
We have a strict requirement of response time < 10 secs all the time.
Thanks
Sreeram
sunnyfr wrote:
>
> Hi Christophe,
>
> Did you find a way to fix up your problem, cuz even with replication will
> have this problem, lot of update means clear cache and manage that.
> I've the same issue, I just wondering if I won't turn off servers during
> update ???
> How did you fix that ?
>
> Thanks,
> sunny
>
>
> christophe-2 wrote:
>>
>> Hi,
>>
>> After fully reloading my index, using another field than a Data does not
>> help that much.
>> Using a warmup query avoids having the first request slow, but:
>> - Frequents commits means that the Searcher is reloaded frequently
>> and, as the warmup takes time, the clients must wait.
>> - Having warmup slows down the index process (I guess this is
>> because after a commit, the Searchers are recreated)
>>
>> So I'm considering, as suggested, to have two instances: one for
>> indexing and one for searching.
>> I was wondering if there are simple ways to replicate the index in a
>> single Solr server running two cores ? Any such config already tested ?
>> I guess that the standard replication based on rsync can be simplified a
>> lot in this case as the two indexes are on the same server.
>>
>> Thanks
>> Christophe
>>
>> Beniamin Janicki wrote:
>>> :so you can send your updates anytime you want, and as long as you only
>>> :commit every 5 minutes (or commit on a master as often as you want, but
>>> :only run snappuller/snapinstaller on your slaves every 5 minutes) your
>>> :results will be at most 5minutes + warming time stale.
>>>
>>> This is what I do as well ( commits are done once per 5 minutes ). I've
>>> got
>>> master - slave configuration. Master has turned off all caches
>>> (commented in
>>> solrconfig.cml) and setup only 2 maxWarmingSearchers. Index size has 5GB
>>> ,Xmx= 1GB and committing takes around 10 secs ( on default configuration
>>> with warming it took from 30 mins up to 2 hours).
>>>
>>> Slave caches are configured to have autowarmCount="0" and
>>> maxWarmingSearchers=1 , and I have new data 1 second after snapshoot is
>>> done. I haven't noticed any huge delays while serving search request.
>>> Try to use those values - may be they'll help in your case too.
>>>
>>> Ben Janicki
>>>
>>>
>>> -----Original Message-----
>>> From: Chris Hostetter [mailto:hossman_lucene@fucit.org]
>>> Sent: 22 October 2008 04:56
>>> To: solr-user@lucene.apache.org
>>> Subject: Re: Sorting performance
>>>
>>>
>>> : The problem is that I will have hundreds of users doing queries, and a
>>> : continuous flow of document coming in.
>>> : So a delay in warming up a cache "could" be acceptable if I do it a
>>> few
>>> times
>>> : per day. But not on a too regular basis (right now, the first query
>>> that
>>> loads
>>> : the cache takes 150s).
>>> :
>>> : However: I'm not sure why it looks not to be a good idea to update the
>>> caches
>>>
>>> you can refresh the caches automaticly after updating, the "newSearcher"
>>> event is fired whenever a searcher is opened (but before it's used by
>>> clients) so you can configure warming queries for it -- it doesn't have
>>> to
>>> be done manually (or by the first user to use that reader)
>>>
>>> so you can send your updates anytime you want, and as long as you only
>>> commit every 5 minutes (or commit on a master as often as you want, but
>>> only run snappuller/snapinstaller on your slaves every 5 minutes) your
>>> results will be at most 5minutes + warming time stale.
>>>
>>>
>>> -Hoss
>>>
>>>
>>
>>
>
>
--
View this message in context: http://www.nabble.com/Sorting-performance-tp20037712p25286018.html
Sent from the Solr - User mailing list archive at Nabble.com.