You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@hbase.apache.org by Rohit Kelkar <ro...@gmail.com> on 2013/04/18 17:19:04 UTC
zookeeper taking 15GB RAM
We have a 3 node cluster in production with the zookeeper instances running
on each node. I checked the memory usage for QuorumPeerMain using 'cat
/proc/<processid>/status' and it shows that the VMData is ~16GB.
The zookeeper is being used only by hbase. Is this usual for hbase to load
up the zookeeper memory so much or is it that we have missed out some
important maintenance activity on zookeeper?
- Rohit Kelkar
Re: zookeeper taking 15GB RAM
Posted by Rohit Kelkar <ro...@gmail.com>.
Thanks for the reply. I checked the default max heap size for java on the
nodes and it turns out its 16G. So now I have to start zookeeper with a
reasonable value for heapsize. What are the factors that would impact the
heap size of zookeeper? Is it more tables in hbase or is is the number of
regions?
- Rohit Kelkar
On Fri, Apr 19, 2013 at 11:22 AM, Arpit Gupta <ar...@hortonworks.com> wrote:
> Take a look at this https://issues.apache.org/jira/browse/ZOOKEEPER-1670
>
> When no xmx was set we noticed that zookeeper could take upto 1/4 of the
> memory available on the system with jdk 1.6
>
> --
> Arpit Gupta
> Hortonworks Inc.
> http://hortonworks.com/
>
> On Apr 19, 2013, at 9:15 AM, Rohit Kelkar <ro...@gmail.com> wrote:
>
> > Hi, any inputs on this issue? Is there some periodic cleanup that we need
> > to do?
> >
> > - Rohit Kelkar
> >
> >
> > On Thu, Apr 18, 2013 at 10:33 AM, Rohit Kelkar <rohitkelkar@gmail.com
> >wrote:
> >
> >> No. Just using the "bin/zkServer.sh start" command. Also each node has
> 48
> >> GB RAM
> >>
> >> - Rohit Kelkar
> >>
> >>
> >> On Thu, Apr 18, 2013 at 10:28 AM, Jean-Marc Spaggiari <
> >> jean-marc@spaggiari.org> wrote:
> >>
> >>> Hi Rohit,
> >>>
> >>> How are you starting your ZK servers? Are you passing any Xm*
> parameters?
> >>>
> >>> JM
> >>>
> >>> 2013/4/18 Rohit Kelkar <ro...@gmail.com>
> >>>
> >>>> We have a 3 node cluster in production with the zookeeper instances
> >>> running
> >>>> on each node. I checked the memory usage for QuorumPeerMain using 'cat
> >>>> /proc/<processid>/status' and it shows that the VMData is ~16GB.
> >>>> The zookeeper is being used only by hbase. Is this usual for hbase to
> >>> load
> >>>> up the zookeeper memory so much or is it that we have missed out some
> >>>> important maintenance activity on zookeeper?
> >>>>
> >>>> - Rohit Kelkar
> >>>>
> >>>
> >>
> >>
>
>
Re: zookeeper taking 15GB RAM
Posted by Arpit Gupta <ar...@hortonworks.com>.
Take a look at this https://issues.apache.org/jira/browse/ZOOKEEPER-1670
When no xmx was set we noticed that zookeeper could take upto 1/4 of the memory available on the system with jdk 1.6
--
Arpit Gupta
Hortonworks Inc.
http://hortonworks.com/
On Apr 19, 2013, at 9:15 AM, Rohit Kelkar <ro...@gmail.com> wrote:
> Hi, any inputs on this issue? Is there some periodic cleanup that we need
> to do?
>
> - Rohit Kelkar
>
>
> On Thu, Apr 18, 2013 at 10:33 AM, Rohit Kelkar <ro...@gmail.com>wrote:
>
>> No. Just using the "bin/zkServer.sh start" command. Also each node has 48
>> GB RAM
>>
>> - Rohit Kelkar
>>
>>
>> On Thu, Apr 18, 2013 at 10:28 AM, Jean-Marc Spaggiari <
>> jean-marc@spaggiari.org> wrote:
>>
>>> Hi Rohit,
>>>
>>> How are you starting your ZK servers? Are you passing any Xm* parameters?
>>>
>>> JM
>>>
>>> 2013/4/18 Rohit Kelkar <ro...@gmail.com>
>>>
>>>> We have a 3 node cluster in production with the zookeeper instances
>>> running
>>>> on each node. I checked the memory usage for QuorumPeerMain using 'cat
>>>> /proc/<processid>/status' and it shows that the VMData is ~16GB.
>>>> The zookeeper is being used only by hbase. Is this usual for hbase to
>>> load
>>>> up the zookeeper memory so much or is it that we have missed out some
>>>> important maintenance activity on zookeeper?
>>>>
>>>> - Rohit Kelkar
>>>>
>>>
>>
>>
Re: zookeeper taking 15GB RAM
Posted by Rohit Kelkar <ro...@gmail.com>.
Hi, any inputs on this issue? Is there some periodic cleanup that we need
to do?
- Rohit Kelkar
On Thu, Apr 18, 2013 at 10:33 AM, Rohit Kelkar <ro...@gmail.com>wrote:
> No. Just using the "bin/zkServer.sh start" command. Also each node has 48
> GB RAM
>
> - Rohit Kelkar
>
>
> On Thu, Apr 18, 2013 at 10:28 AM, Jean-Marc Spaggiari <
> jean-marc@spaggiari.org> wrote:
>
>> Hi Rohit,
>>
>> How are you starting your ZK servers? Are you passing any Xm* parameters?
>>
>> JM
>>
>> 2013/4/18 Rohit Kelkar <ro...@gmail.com>
>>
>> > We have a 3 node cluster in production with the zookeeper instances
>> running
>> > on each node. I checked the memory usage for QuorumPeerMain using 'cat
>> > /proc/<processid>/status' and it shows that the VMData is ~16GB.
>> > The zookeeper is being used only by hbase. Is this usual for hbase to
>> load
>> > up the zookeeper memory so much or is it that we have missed out some
>> > important maintenance activity on zookeeper?
>> >
>> > - Rohit Kelkar
>> >
>>
>
>
Re: zookeeper taking 15GB RAM
Posted by Rohit Kelkar <ro...@gmail.com>.
No. Just using the "bin/zkServer.sh start" command. Also each node has 48
GB RAM
- Rohit Kelkar
On Thu, Apr 18, 2013 at 10:28 AM, Jean-Marc Spaggiari <
jean-marc@spaggiari.org> wrote:
> Hi Rohit,
>
> How are you starting your ZK servers? Are you passing any Xm* parameters?
>
> JM
>
> 2013/4/18 Rohit Kelkar <ro...@gmail.com>
>
> > We have a 3 node cluster in production with the zookeeper instances
> running
> > on each node. I checked the memory usage for QuorumPeerMain using 'cat
> > /proc/<processid>/status' and it shows that the VMData is ~16GB.
> > The zookeeper is being used only by hbase. Is this usual for hbase to
> load
> > up the zookeeper memory so much or is it that we have missed out some
> > important maintenance activity on zookeeper?
> >
> > - Rohit Kelkar
> >
>
Re: zookeeper taking 15GB RAM
Posted by Jean-Marc Spaggiari <je...@spaggiari.org>.
Hi Rohit,
How are you starting your ZK servers? Are you passing any Xm* parameters?
JM
2013/4/18 Rohit Kelkar <ro...@gmail.com>
> We have a 3 node cluster in production with the zookeeper instances running
> on each node. I checked the memory usage for QuorumPeerMain using 'cat
> /proc/<processid>/status' and it shows that the VMData is ~16GB.
> The zookeeper is being used only by hbase. Is this usual for hbase to load
> up the zookeeper memory so much or is it that we have missed out some
> important maintenance activity on zookeeper?
>
> - Rohit Kelkar
>