You are viewing a plain text version of this content. The canonical link for it is here.
Posted to solr-user@lucene.apache.org by Esteban Donato <es...@gmail.com> on 2011/06/24 10:11:40 UTC
multicore and replication cause OOM
Hi,
I have a Solr with 7 cores (~150MB each). All cores replicate at the
same time from a Solr master instance. Every time the replication
happens I get an OOM after experiencing long response times. This
Solr used to have 4 cores before and I've never got an OOM with that
configuration (replication occurs on daily basis).
My question is: could the new 3 cores be the cause of OOM? Does Solr
require considerable extra heap for performing the replication?.
Should I avoid replicating all the cores at the same time?
I'm using Solr 1.4 with the following mem configuration: -Xms512m
-Xmx512m -XX:NewSize=128M -XX:MaxNewSize=128M
Appreciate any help.
Regards,
Esteban
Re: multicore and replication cause OOM
Posted by Shalin Shekhar Mangar <sh...@gmail.com>.
On Sun, Jun 26, 2011 at 5:37 AM, Esteban Donato
<es...@gmail.com> wrote:
> thanks Shalin. One more question: is there any way to avoid multiple
> cores replicating at the same time? Like synchronizing the
> ReplicationHandler somehow?
>
Yes, just specify different poll intervals for each core. The
ReplicationHandler always tries to schedule pulls by rounding to the
nearest interval e.g. specifying an interval as "00:05:00" will cause
pulls to happen at 5 minutes past the hour, then 10 minutes past the
hour and so on.
--
Regards,
Shalin Shekhar Mangar.
Re: multicore and replication cause OOM
Posted by Esteban Donato <es...@gmail.com>.
thanks Shalin. One more question: is there any way to avoid multiple
cores replicating at the same time? Like synchronizing the
ReplicationHandler somehow?
On Fri, Jun 24, 2011 at 6:55 AM, Shalin Shekhar Mangar
<sh...@gmail.com> wrote:
> On Fri, Jun 24, 2011 at 1:41 PM, Esteban Donato
> <es...@gmail.com> wrote:
>> I have a Solr with 7 cores (~150MB each). All cores replicate at the
>> same time from a Solr master instance. Every time the replication
>> happens I get an OOM after experiencing long response times. This
>> Solr used to have 4 cores before and I've never got an OOM with that
>> configuration (replication occurs on daily basis).
>>
>> My question is: could the new 3 cores be the cause of OOM? Does Solr
>> require considerable extra heap for performing the replication?.
>
> Yes and no. Replication itself does not consume a lot of heap (I guess
> about a couple of MBs per ongoing replication). However, when the
> searchers are re-opened on the newly installed index, auto warming can
> cause memory usage to double for a core.
>
>> Should I avoid replicating all the cores at the same time?
>
> You should try that especially if you are so constrained for heap space.
>
>> I'm using Solr 1.4 with the following mem configuration: -Xms512m
>> -Xmx512m -XX:NewSize=128M -XX:MaxNewSize=128M
>
> That seems to be a small amount of RAM for indexing/querying seven
> 150MB indexes in parallel.
>
> --
> Regards,
> Shalin Shekhar Mangar.
>
Re: multicore and replication cause OOM
Posted by Shalin Shekhar Mangar <sh...@gmail.com>.
On Fri, Jun 24, 2011 at 1:41 PM, Esteban Donato
<es...@gmail.com> wrote:
> I have a Solr with 7 cores (~150MB each). All cores replicate at the
> same time from a Solr master instance. Every time the replication
> happens I get an OOM after experiencing long response times. This
> Solr used to have 4 cores before and I've never got an OOM with that
> configuration (replication occurs on daily basis).
>
> My question is: could the new 3 cores be the cause of OOM? Does Solr
> require considerable extra heap for performing the replication?.
Yes and no. Replication itself does not consume a lot of heap (I guess
about a couple of MBs per ongoing replication). However, when the
searchers are re-opened on the newly installed index, auto warming can
cause memory usage to double for a core.
> Should I avoid replicating all the cores at the same time?
You should try that especially if you are so constrained for heap space.
> I'm using Solr 1.4 with the following mem configuration: -Xms512m
> -Xmx512m -XX:NewSize=128M -XX:MaxNewSize=128M
That seems to be a small amount of RAM for indexing/querying seven
150MB indexes in parallel.
--
Regards,
Shalin Shekhar Mangar.