You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@zookeeper.apache.org by Shivanshu Goswami <sg...@cs.stonybrook.edu> on 2017/02/22 13:51:06 UTC
Zookeeper scalability and memory consumption
Hi,
I want to know how Zookeeper scales when there is a high number of Znodes,
say 10 million. Is there some documentation on memory consumption?
--
Shivanshu Goswami
Re: Zookeeper scalability and memory consumption
Posted by Camille Fournier <ca...@apache.org>.
It depends on how much data you store in each node.
On Wed, Feb 22, 2017 at 11:21 AM, Shivanshu Goswami <
sgoswami@cs.stonybrook.edu> wrote:
> Thanks Jordan.
>
> My question is about the same. If it keeps the entire DB in memory,* is
> there a limit beyond which say Zookeeper server will stop working. *
>
> Eg. Say if 10 Gigs of RAM has been assigned for the Zookeeper server
> application.
>
> Also, if anyone has studied this before, is there an estimate of how much
> RAM will Zookeeper consume for some X znodes, X being say 100,000, 1
> million, 2 million and so on.
>
> On Wed, Feb 22, 2017 at 9:47 AM, Jordan Zimmerman <
> jordan@jordanzimmerman.com> wrote:
>
> > ZooKeeper keeps the entire database in memory. FYI
> >
> > ====================
> > Jordan Zimmerman
> >
> > > On Feb 22, 2017, at 10:51 AM, Shivanshu Goswami <
> > sgoswami@cs.stonybrook.edu> wrote:
> > >
> > > Hi,
> > >
> > > I want to know how Zookeeper scales when there is a high number of
> > Znodes,
> > > say 10 million. Is there some documentation on memory consumption?
> > >
> > > --
> > > Shivanshu Goswami
> >
>
>
>
> --
> Shivanshu Goswami,
> MS Computer Science 2016-17
> Stony Brook University
>
Re: Zookeeper scalability and memory consumption
Posted by Shivanshu Goswami <sg...@cs.stonybrook.edu>.
Thanks Jordan.
My question is about the same. If it keeps the entire DB in memory,* is
there a limit beyond which say Zookeeper server will stop working. *
Eg. Say if 10 Gigs of RAM has been assigned for the Zookeeper server
application.
Also, if anyone has studied this before, is there an estimate of how much
RAM will Zookeeper consume for some X znodes, X being say 100,000, 1
million, 2 million and so on.
On Wed, Feb 22, 2017 at 9:47 AM, Jordan Zimmerman <
jordan@jordanzimmerman.com> wrote:
> ZooKeeper keeps the entire database in memory. FYI
>
> ====================
> Jordan Zimmerman
>
> > On Feb 22, 2017, at 10:51 AM, Shivanshu Goswami <
> sgoswami@cs.stonybrook.edu> wrote:
> >
> > Hi,
> >
> > I want to know how Zookeeper scales when there is a high number of
> Znodes,
> > say 10 million. Is there some documentation on memory consumption?
> >
> > --
> > Shivanshu Goswami
>
--
Shivanshu Goswami,
MS Computer Science 2016-17
Stony Brook University
Re: Zookeeper scalability and memory consumption
Posted by Jordan Zimmerman <jo...@jordanzimmerman.com>.
ZooKeeper keeps the entire database in memory. FYI
====================
Jordan Zimmerman
> On Feb 22, 2017, at 10:51 AM, Shivanshu Goswami <sg...@cs.stonybrook.edu> wrote:
>
> Hi,
>
> I want to know how Zookeeper scales when there is a high number of Znodes,
> say 10 million. Is there some documentation on memory consumption?
>
> --
> Shivanshu Goswami