You are viewing a plain text version of this content. The canonical link for it is here.
Posted to solr-user@lucene.apache.org by Furkan KAMACI <fu...@gmail.com> on 2013/05/16 17:25:59 UTC
Zookeeper Ensemble Startup Parameters For SolrCloud?
I know that there have been many conversations about SolrCloud startup tips
i.e. which type of garbage collector to use etc. Also I know that there is
no an exact answer for this question. However I think that folks have some
tips about this question.
How do you start up your external Zookeeper, with which parameters and any
tips for it?
Re: Zookeeper Ensemble Startup Parameters For SolrCloud?
Posted by vsilgalis <vs...@gmail.com>.
I didn't change it and haven't seen any issues.
--
View this message in context: http://lucene.472066.n3.nabble.com/Zookeeper-Ensemble-Startup-Parameters-For-SolrCloud-tp4063905p4064654.html
Sent from the Solr - User mailing list archive at Nabble.com.
Re: Zookeeper Ensemble Startup Parameters For SolrCloud?
Posted by Furkan KAMACI <fu...@gmail.com>.
I have read that about zookeeper:
"Zookeeper servers have an active connections limit, which by default is
30." Do you define it higher than 30 for Solr?
2013/5/17 vsilgalis <vs...@gmail.com>
> As an example, I have 9 SOLR nodes (3 clusters of 3) using different
> versions
> of SOLR (4.1, 4.1, and 4.2.1), utilizing the same zookeeper ensemble (3
> servers), using chroot for the different configs across clusters.
>
> My zookeeper servers are just VMs, dual-core with 1GB of RAM and are only
> used for SOLRCloud
>
> JVM settings for zookeeper for heap size are start 256MB and max heap size
> of 512MB or: -Xms256m -Xmx512m
>
> I have never seen it use more than the specified start heap size of 256MB.
>
>
>
> --
> View this message in context:
> http://lucene.472066.n3.nabble.com/Zookeeper-Ensemble-Startup-Parameters-For-SolrCloud-tp4063905p4064279.html
> Sent from the Solr - User mailing list archive at Nabble.com.
>
Re: Zookeeper Ensemble Startup Parameters For SolrCloud?
Posted by vsilgalis <vs...@gmail.com>.
As an example, I have 9 SOLR nodes (3 clusters of 3) using different versions
of SOLR (4.1, 4.1, and 4.2.1), utilizing the same zookeeper ensemble (3
servers), using chroot for the different configs across clusters.
My zookeeper servers are just VMs, dual-core with 1GB of RAM and are only
used for SOLRCloud
JVM settings for zookeeper for heap size are start 256MB and max heap size
of 512MB or: -Xms256m -Xmx512m
I have never seen it use more than the specified start heap size of 256MB.
--
View this message in context: http://lucene.472066.n3.nabble.com/Zookeeper-Ensemble-Startup-Parameters-For-SolrCloud-tp4063905p4064279.html
Sent from the Solr - User mailing list archive at Nabble.com.
Re: Zookeeper Ensemble Startup Parameters For SolrCloud?
Posted by Furkan KAMACI <fu...@gmail.com>.
Hi Mark;
Thanks for the answer. Does Solr nodes holds the current state of cluster
(which Zookeeper ensemble knows) inside their cache/RAM?
2013/5/17 Mark Miller <ma...@gmail.com>
> The way Solr uses ZK, unless you are also using ZK with something else, I
> wouldn't worry about it at all. In a steady state, the cluster won't even
> really talk to ZK in any intensive manner at all.
>
> - Mark
>
> On May 16, 2013, at 5:07 PM, Furkan KAMACI <fu...@gmail.com> wrote:
>
> > Hi Shawn;
> >
> > I will have totally 18 Solr nodes at my current pre-prototype environment
> > over one collection and I don't have large config files. I know that best
> > and only recommend practice for estimating the heap size of my system
> needs
> > is to run load tests and I will.
> >
> > I asked this question because of an example at Zookeeper wiki:
> >
> > "You should take special care to set your Java max heap size correctly.
> In
> > particular, you should not create a situation in which ZooKeeper swaps to
> > disk. The disk is death to ZooKeeper. Everything is ordered, so if
> > processing one request swaps the disk, all other queued requests will
> > probably do the same. the disk. DON'T SWAP.
> >
> > Be conservative in your estimates: if you have 4G of RAM, do not set the
> > Java max heap size to 6G or even 4G. For example, it is more likely you
> > would use a 3G heap for a 4G machine, as the operating system and the
> cache
> > also need memory."
> >
> > This may be a more Zookeeper related question but one more question too.
> Is
> > there anything something like not to use Zookeeper on a virtual machine
> > because of performance issues or not?
> >
> >
> >
> >
> > 2013/5/16 Shawn Heisey <so...@elyograg.org>
> >
> >> On 5/16/2013 2:34 PM, Furkan KAMACI wrote:
> >>
> >>> You have some tips about JVM parameters starting a Solr node. What do
> you
> >>> have special for Solr when you start a Zookeeper ensemble. i.e. heap
> size?
> >>>
> >>
> >> I haven't given it any JVM options. The ZK process on my primary server
> >> has a 5GB virtual memory size and is using 131MB of system memory. If
> >> you're not going to be creating a large number of collection or replicas
> >> and you're not using super-large config files, you could probably limit
> the
> >> max heap to a pretty small number and be OK.
> >>
> >> Thanks,
> >> Shawn
> >>
> >>
>
>
Re: Zookeeper Ensemble Startup Parameters For SolrCloud?
Posted by Mark Miller <ma...@gmail.com>.
The way Solr uses ZK, unless you are also using ZK with something else, I wouldn't worry about it at all. In a steady state, the cluster won't even really talk to ZK in any intensive manner at all.
- Mark
On May 16, 2013, at 5:07 PM, Furkan KAMACI <fu...@gmail.com> wrote:
> Hi Shawn;
>
> I will have totally 18 Solr nodes at my current pre-prototype environment
> over one collection and I don't have large config files. I know that best
> and only recommend practice for estimating the heap size of my system needs
> is to run load tests and I will.
>
> I asked this question because of an example at Zookeeper wiki:
>
> "You should take special care to set your Java max heap size correctly. In
> particular, you should not create a situation in which ZooKeeper swaps to
> disk. The disk is death to ZooKeeper. Everything is ordered, so if
> processing one request swaps the disk, all other queued requests will
> probably do the same. the disk. DON'T SWAP.
>
> Be conservative in your estimates: if you have 4G of RAM, do not set the
> Java max heap size to 6G or even 4G. For example, it is more likely you
> would use a 3G heap for a 4G machine, as the operating system and the cache
> also need memory."
>
> This may be a more Zookeeper related question but one more question too. Is
> there anything something like not to use Zookeeper on a virtual machine
> because of performance issues or not?
>
>
>
>
> 2013/5/16 Shawn Heisey <so...@elyograg.org>
>
>> On 5/16/2013 2:34 PM, Furkan KAMACI wrote:
>>
>>> You have some tips about JVM parameters starting a Solr node. What do you
>>> have special for Solr when you start a Zookeeper ensemble. i.e. heap size?
>>>
>>
>> I haven't given it any JVM options. The ZK process on my primary server
>> has a 5GB virtual memory size and is using 131MB of system memory. If
>> you're not going to be creating a large number of collection or replicas
>> and you're not using super-large config files, you could probably limit the
>> max heap to a pretty small number and be OK.
>>
>> Thanks,
>> Shawn
>>
>>
Re: Zookeeper Ensemble Startup Parameters For SolrCloud?
Posted by Furkan KAMACI <fu...@gmail.com>.
Hi Shawn;
I will have totally 18 Solr nodes at my current pre-prototype environment
over one collection and I don't have large config files. I know that best
and only recommend practice for estimating the heap size of my system needs
is to run load tests and I will.
I asked this question because of an example at Zookeeper wiki:
"You should take special care to set your Java max heap size correctly. In
particular, you should not create a situation in which ZooKeeper swaps to
disk. The disk is death to ZooKeeper. Everything is ordered, so if
processing one request swaps the disk, all other queued requests will
probably do the same. the disk. DON'T SWAP.
Be conservative in your estimates: if you have 4G of RAM, do not set the
Java max heap size to 6G or even 4G. For example, it is more likely you
would use a 3G heap for a 4G machine, as the operating system and the cache
also need memory."
This may be a more Zookeeper related question but one more question too. Is
there anything something like not to use Zookeeper on a virtual machine
because of performance issues or not?
2013/5/16 Shawn Heisey <so...@elyograg.org>
> On 5/16/2013 2:34 PM, Furkan KAMACI wrote:
>
>> You have some tips about JVM parameters starting a Solr node. What do you
>> have special for Solr when you start a Zookeeper ensemble. i.e. heap size?
>>
>
> I haven't given it any JVM options. The ZK process on my primary server
> has a 5GB virtual memory size and is using 131MB of system memory. If
> you're not going to be creating a large number of collection or replicas
> and you're not using super-large config files, you could probably limit the
> max heap to a pretty small number and be OK.
>
> Thanks,
> Shawn
>
>
Re: Zookeeper Ensemble Startup Parameters For SolrCloud?
Posted by Shawn Heisey <so...@elyograg.org>.
On 5/16/2013 2:34 PM, Furkan KAMACI wrote:
> You have some tips about JVM parameters starting a Solr node. What do you
> have special for Solr when you start a Zookeeper ensemble. i.e. heap size?
I haven't given it any JVM options. The ZK process on my primary server
has a 5GB virtual memory size and is using 131MB of system memory. If
you're not going to be creating a large number of collection or replicas
and you're not using super-large config files, you could probably limit
the max heap to a pretty small number and be OK.
Thanks,
Shawn
Re: Zookeeper Ensemble Startup Parameters For SolrCloud?
Posted by Furkan KAMACI <fu...@gmail.com>.
Hi Shawn;
You have some tips about JVM parameters starting a Solr node. What do you
have special for Solr when you start a Zookeeper ensemble. i.e. heap size?
2013/5/16 Shawn Heisey <so...@elyograg.org>
> On 5/16/2013 9:25 AM, Furkan KAMACI wrote:
>
>> I know that there have been many conversations about SolrCloud startup
>> tips
>> i.e. which type of garbage collector to use etc. Also I know that there
>> is
>> no an exact answer for this question. However I think that folks have some
>> tips about this question.
>>
>> How do you start up your external Zookeeper, with which parameters and any
>> tips for it?
>>
>
> An external zookeeper is just that - external, not part of Solr. I
> followed the zookeeper docs, and used the normal zookeeper port, 2181:
>
> http://zookeeper.apache.org/doc/r3.4.5/
>
> Thanks,
> Shawn
>
>
Re: Zookeeper Ensemble Startup Parameters For SolrCloud?
Posted by Shawn Heisey <so...@elyograg.org>.
On 5/16/2013 9:25 AM, Furkan KAMACI wrote:
> I know that there have been many conversations about SolrCloud startup tips
> i.e. which type of garbage collector to use etc. Also I know that there is
> no an exact answer for this question. However I think that folks have some
> tips about this question.
>
> How do you start up your external Zookeeper, with which parameters and any
> tips for it?
An external zookeeper is just that - external, not part of Solr. I
followed the zookeeper docs, and used the normal zookeeper port, 2181:
http://zookeeper.apache.org/doc/r3.4.5/
Thanks,
Shawn