You are viewing a plain text version of this content. The canonical link for it is here.
Posted to solr-user@lucene.apache.org by Learner <bb...@gmail.com> on 2013/06/19 02:19:15 UTC
Merge tool based on mergefactor
We have SOLR master, primarily for indexing and SOLR slave primarily for
searching. I see that the merge factor plays a key factor in Indexing as
well as searching. I would like to have a high merge factor for my master
instance and low merge factor for slave.
As of now since I just replicate the data from master, the number of
segments remain the same as that of master. May be its already there ... but
I feel that it would be great if theres a tool with which we can merge the
segments in slave based on merge factor specified in slave (config) after
replication..Is there any work around to change the number of segments
without doing complete indexing?
--
View this message in context: http://lucene.472066.n3.nabble.com/Merge-tool-based-on-mergefactor-tp4071506.html
Sent from the Solr - User mailing list archive at Nabble.com.
Re: Merge tool based on mergefactor
Posted by Otis Gospodnetic <ot...@gmail.com>.
Hi,
On Wed, Jun 19, 2013 at 3:52 AM, Cosimo Streppone <co...@streppone.it> wrote:
> On 06/19/2013 03:21 AM, Otis Gospodnetic wrote:
>
>> You could call the optimize command directly on slaves, but specify
>> the target number of segments, e.g.
>> ..../solr/update?optimize=true&maxSegments=10
>>
>> Not sure I recommend doing this on slaves, but you could - maybe you
>> have spare capacity. You may also want to consider not doing it on
>> all your slaves at the same time...
>
> IIUC this assumes your slaves do not replicate too often,
> otherwise replication would reset the index to whatever
> number of segments the master has.
>
> You could still perform an optimize with maxSegments after
> every replication, if it's acceptable in the situation
> you are in.
>
> However, if you need slaves to update every 2-5 minutes,
> that would be impractical and wasteful.
>
> Is this correct?
Correct.
> If so, how to find a fair compromise/balance between
> master and slave merge factors if you need very frequent indexing
> of new documents (say continuous) on the master and up-to-date
> indexes on the slaves (say 2-5' pollInterval)?
If you need that you go to SolrCloud and start using softCommits.
Otis
--
Solr & ElasticSearch Support
http://sematext.com/
Re: Merge tool based on mergefactor
Posted by Cosimo Streppone <co...@streppone.it>.
On 06/19/2013 03:21 AM, Otis Gospodnetic wrote:
> You could call the optimize command directly on slaves, but specify
> the target number of segments, e.g.
> ..../solr/update?optimize=true&maxSegments=10
>
> Not sure I recommend doing this on slaves, but you could - maybe you
> have spare capacity. You may also want to consider not doing it on
> all your slaves at the same time...
IIUC this assumes your slaves do not replicate too often,
otherwise replication would reset the index to whatever
number of segments the master has.
You could still perform an optimize with maxSegments after
every replication, if it's acceptable in the situation
you are in.
However, if you need slaves to update every 2-5 minutes,
that would be impractical and wasteful.
Is this correct?
If so, how to find a fair compromise/balance between
master and slave merge factors if you need very frequent indexing
of new documents (say continuous) on the master and up-to-date
indexes on the slaves (say 2-5' pollInterval)?
--
Cosimo
Re: Merge tool based on mergefactor
Posted by Otis Gospodnetic <ot...@gmail.com>.
Hi,
You could call the optimize command directly on slaves, but specify
the target number of segments, e.g.
..../solr/update?optimize=true&maxSegments=10
Not sure I recommend doing this on slaves, but you could - maybe you
have spare capacity. You may also want to consider not doing it on
all your slaves at the same time...
Otis
--
Solr & ElasticSearch Support
http://sematext.com/
On Tue, Jun 18, 2013 at 8:19 PM, Learner <bb...@gmail.com> wrote:
> We have SOLR master, primarily for indexing and SOLR slave primarily for
> searching. I see that the merge factor plays a key factor in Indexing as
> well as searching. I would like to have a high merge factor for my master
> instance and low merge factor for slave.
>
> As of now since I just replicate the data from master, the number of
> segments remain the same as that of master. May be its already there ... but
> I feel that it would be great if theres a tool with which we can merge the
> segments in slave based on merge factor specified in slave (config) after
> replication..Is there any work around to change the number of segments
> without doing complete indexing?
>
>
>
> --
> View this message in context: http://lucene.472066.n3.nabble.com/Merge-tool-based-on-mergefactor-tp4071506.html
> Sent from the Solr - User mailing list archive at Nabble.com.