You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@zookeeper.apache.org by yosefarr <yo...@gmail.com> on 2011/08/11 11:19:06 UTC
ZooKeeper Limitation
Hi,
If i want to save users that connect to my system as zookeeper nodes and
each user has some proprties - connection id, user name and etc:
i need to know if there is a Limitation on ZooKeeper nodes numbers?
/server1
/user-1
/user-2
........
/user-n
Thanks,
Yosef.
--
View this message in context: http://zookeeper-user.578899.n2.nabble.com/ZooKeeper-Limitation-tp6675643p6675643.html
Sent from the zookeeper-user mailing list archive at Nabble.com.
Re: ZooKeeper Limitation
Posted by Patrick Hunt <ph...@apache.org>.
FYI: https://issues.apache.org/jira/browse/ZOOKEEPER-1162
We should probably try to mitigate this issue, it keeps coming up.
Patrick
On Sat, Aug 13, 2011 at 12:07 PM, Ted Dunning <te...@gmail.com> wrote:
> Yes. But getting a list of children can happen in other contexts. You should not be pushing things to the edge like that when there are such simple ways around the problem.
>
> Sent from my iPad
>
> On Aug 13, 2011, at 1:56 AM, yosefarr <yo...@gmail.com> wrote:
>
>> So I can have one node with 500000 sub nodes(Childs) and the get Childs will fail but get one child (getChild("/mainNode/child50")) will not fail because the request is less than 1M is that right?
>>
>> Sent from my iPhone
>>
>> On 12 באוג 2011, at 00:36, "Ted Dunning [via zookeeper-user]"<ml...@n2.nabble.com> wrote:
>>
>>> Same answer as before.
>>>
>>> Your request and your response have to fit into 1M byte. How many nodes
>>> that will be depends on their names.
>>>
>>> Hint: don't push the limit. Design in a way that is nowhere close to the
>>> limit.
>>>
>>> On Thu, Aug 11, 2011 at 2:16 PM, yosefarr <[hidden email]> wrote:
>>>
>>>> what will work, how much maximum childs in a single node
>>>>
>>>> On Fri, Aug 12, 2011 at 12:07 AM, Ted Dunning [via zookeeper-user] <
>>>> [hidden email]> wrote:
>>>>
>>>>> 1M byte is the limit.
>>>>>
>>>>> You need to put multiple levels of directory in there. 500,000 znodes in
>>>> a
>>>>>
>>>>> single directory won't work.
>>>>>
>>>>> On Thu, Aug 11, 2011 at 1:33 PM, yosefarr <[hidden email]<
>>>> http://user/SendEmail.jtp?type=node&node=6678116&i=0>>
>>>>> wrote:
>>>>>
>>>>>> Thanks you for your replay,
>>>>>> What do you mean that there is limit on data calls to ZK?what is the
>>>>> limit?
>>>>>> We will have too many direct childrens for a znode:
>>>>>> /znode
>>>>>> /child-1
>>>>>> /child-2
>>>>>> ........
>>>>>> /child-n
>>>>>> when "n" can be 500000 or more
>>>>>> in this case if i run getChildren("/znode") ,does this call will fail?
>>>>>> or getData("/znode/child-n") will fail?
>>>>>>
>>>>>> Thanks
>>>>>> Yosef.
>>>>>>
>>>>>>
>>>>>> On Thu, Aug 11, 2011 at 8:32 PM, Mahadev Konar-4 [via zookeeper-user] <
>>>>>> [hidden email] <http://user/SendEmail.jtp?type=node&node=6678116&i=1>>
>>>>> wrote:
>>>>>>
>>>>>>> Yosef,
>>>>>>> There isnt a limit on the number of ZK node numbers directly. We do
>>>>>>> have a limit on data calls to ZK. So in your case if you have too
>>>> many
>>>>>>> direct children of a single znode (say millions), the getChildren
>>>>>>> calls return an error. I'd advise against having too many direct
>>>>>>> chidlren for a znode. Bucketing the users might make sense for you if
>>>>>>> you have too many users.
>>>>>>>
>>>>>>> thanks
>>>>>>> mahadev
>>>>>>>
>>>>>>> On Thu, Aug 11, 2011 at 2:19 AM, yosefarr <[hidden email]<
>>>>>> http://user/SendEmail.jtp?type=node&node=6677260&i=0>>
>>>>>>> wrote:
>>>>>>>
>>>>>>>> Hi,
>>>>>>>> If i want to save users that connect to my system as zookeeper
>>>> nodes
>>>>>> and
>>>>>>>> each user has some proprties - connection id, user name and etc:
>>>>>>>> i need to know if there is a Limitation on ZooKeeper nodes numbers?
>>>>>>>>
>>>>>>>> /server1
>>>>>>>> /user-1
>>>>>>>> /user-2
>>>>>>>> ........
>>>>>>>> /user-n
>>>>>>>>
>>>>>>>> Thanks,
>>>>>>>> Yosef.
>>>>>>>>
>>>>>>>> --
>>>>>>>> View this message in context:
>>>>>>>
>>>>>>
>>>>>
>>>> http://zookeeper-user.578899.n2.nabble.com/ZooKeeper-Limitation-tp6675643p6675643.html
>>>>>>>> Sent from the zookeeper-user mailing list archive at Nabble.com.
>>>>>>>
>>>>>>>
>>>>>>> ------------------------------
>>>>>>> If you reply to this email, your message will be added to the
>>>>> discussion
>>>>>>> below:
>>>>>>>
>>>>>>>
>>>>>>
>>>>>
>>>> http://zookeeper-user.578899.n2.nabble.com/ZooKeeper-Limitation-tp6675643p6677260.html
>>>>>>> To unsubscribe from ZooKeeper Limitation, click here<
>>>>>>
>>>>>> .
>>>>>>>
>>>>>>>
>>>>>>
>>>>>>
>>>>>> --
>>>>>> View this message in context:
>>>>>>
>>>>>
>>>> http://zookeeper-user.578899.n2.nabble.com/ZooKeeper-Limitation-tp6675643p6677984.html
>>>>>
>>>>>> Sent from the zookeeper-user mailing list archive at Nabble.com.
>>>>>
>>>>>
>>>>> ------------------------------
>>>>> If you reply to this email, your message will be added to the discussion
>>>>> below:
>>>>>
>>>>>
>>>> http://zookeeper-user.578899.n2.nabble.com/ZooKeeper-Limitation-tp6675643p6678116.html
>>>>> To unsubscribe from ZooKeeper Limitation, click here<
>>>>> .
>>>>>
>>>>>
>>>>
>>>>
>>>> --
>>>> View this message in context:
>>>> http://zookeeper-user.578899.n2.nabble.com/ZooKeeper-Limitation-tp6675643p6678146.html
>>>> Sent from the zookeeper-user mailing list archive at Nabble.com.
>>>>
>>>
>>>
>>> If you reply to this email, your message will be added to the discussion below:
>>> http://zookeeper-user.578899.n2.nabble.com/ZooKeeper-Limitation-tp6675643p6678230.html
>>> To unsubscribe from ZooKeeper Limitation, click here.
>>
>>
>> --
>> View this message in context: http://zookeeper-user.578899.n2.nabble.com/ZooKeeper-Limitation-tp6675643p6682887.html
>> Sent from the zookeeper-user mailing list archive at Nabble.com.
>
Re: ZooKeeper Limitation
Posted by Ted Dunning <te...@gmail.com>.
Yes. But getting a list of children can happen in other contexts. You should not be pushing things to the edge like that when there are such simple ways around the problem.
Sent from my iPad
On Aug 13, 2011, at 1:56 AM, yosefarr <yo...@gmail.com> wrote:
> So I can have one node with 500000 sub nodes(Childs) and the get Childs will fail but get one child (getChild("/mainNode/child50")) will not fail because the request is less than 1M is that right?
>
> Sent from my iPhone
>
> On 12 באוג 2011, at 00:36, "Ted Dunning [via zookeeper-user]"<ml...@n2.nabble.com> wrote:
>
>> Same answer as before.
>>
>> Your request and your response have to fit into 1M byte. How many nodes
>> that will be depends on their names.
>>
>> Hint: don't push the limit. Design in a way that is nowhere close to the
>> limit.
>>
>> On Thu, Aug 11, 2011 at 2:16 PM, yosefarr <[hidden email]> wrote:
>>
>>> what will work, how much maximum childs in a single node
>>>
>>> On Fri, Aug 12, 2011 at 12:07 AM, Ted Dunning [via zookeeper-user] <
>>> [hidden email]> wrote:
>>>
>>>> 1M byte is the limit.
>>>>
>>>> You need to put multiple levels of directory in there. 500,000 znodes in
>>> a
>>>>
>>>> single directory won't work.
>>>>
>>>> On Thu, Aug 11, 2011 at 1:33 PM, yosefarr <[hidden email]<
>>> http://user/SendEmail.jtp?type=node&node=6678116&i=0>>
>>>> wrote:
>>>>
>>>>> Thanks you for your replay,
>>>>> What do you mean that there is limit on data calls to ZK?what is the
>>>> limit?
>>>>> We will have too many direct childrens for a znode:
>>>>> /znode
>>>>> /child-1
>>>>> /child-2
>>>>> ........
>>>>> /child-n
>>>>> when "n" can be 500000 or more
>>>>> in this case if i run getChildren("/znode") ,does this call will fail?
>>>>> or getData("/znode/child-n") will fail?
>>>>>
>>>>> Thanks
>>>>> Yosef.
>>>>>
>>>>>
>>>>> On Thu, Aug 11, 2011 at 8:32 PM, Mahadev Konar-4 [via zookeeper-user] <
>>>>> [hidden email] <http://user/SendEmail.jtp?type=node&node=6678116&i=1>>
>>>> wrote:
>>>>>
>>>>>> Yosef,
>>>>>> There isnt a limit on the number of ZK node numbers directly. We do
>>>>>> have a limit on data calls to ZK. So in your case if you have too
>>> many
>>>>>> direct children of a single znode (say millions), the getChildren
>>>>>> calls return an error. I'd advise against having too many direct
>>>>>> chidlren for a znode. Bucketing the users might make sense for you if
>>>>>> you have too many users.
>>>>>>
>>>>>> thanks
>>>>>> mahadev
>>>>>>
>>>>>> On Thu, Aug 11, 2011 at 2:19 AM, yosefarr <[hidden email]<
>>>>> http://user/SendEmail.jtp?type=node&node=6677260&i=0>>
>>>>>> wrote:
>>>>>>
>>>>>>> Hi,
>>>>>>> If i want to save users that connect to my system as zookeeper
>>> nodes
>>>>> and
>>>>>>> each user has some proprties - connection id, user name and etc:
>>>>>>> i need to know if there is a Limitation on ZooKeeper nodes numbers?
>>>>>>>
>>>>>>> /server1
>>>>>>> /user-1
>>>>>>> /user-2
>>>>>>> ........
>>>>>>> /user-n
>>>>>>>
>>>>>>> Thanks,
>>>>>>> Yosef.
>>>>>>>
>>>>>>> --
>>>>>>> View this message in context:
>>>>>>
>>>>>
>>>>
>>> http://zookeeper-user.578899.n2.nabble.com/ZooKeeper-Limitation-tp6675643p6675643.html
>>>>>>> Sent from the zookeeper-user mailing list archive at Nabble.com.
>>>>>>
>>>>>>
>>>>>> ------------------------------
>>>>>> If you reply to this email, your message will be added to the
>>>> discussion
>>>>>> below:
>>>>>>
>>>>>>
>>>>>
>>>>
>>> http://zookeeper-user.578899.n2.nabble.com/ZooKeeper-Limitation-tp6675643p6677260.html
>>>>>> To unsubscribe from ZooKeeper Limitation, click here<
>>>>>
>>>>> .
>>>>>>
>>>>>>
>>>>>
>>>>>
>>>>> --
>>>>> View this message in context:
>>>>>
>>>>
>>> http://zookeeper-user.578899.n2.nabble.com/ZooKeeper-Limitation-tp6675643p6677984.html
>>>>
>>>>> Sent from the zookeeper-user mailing list archive at Nabble.com.
>>>>
>>>>
>>>> ------------------------------
>>>> If you reply to this email, your message will be added to the discussion
>>>> below:
>>>>
>>>>
>>> http://zookeeper-user.578899.n2.nabble.com/ZooKeeper-Limitation-tp6675643p6678116.html
>>>> To unsubscribe from ZooKeeper Limitation, click here<
>>>> .
>>>>
>>>>
>>>
>>>
>>> --
>>> View this message in context:
>>> http://zookeeper-user.578899.n2.nabble.com/ZooKeeper-Limitation-tp6675643p6678146.html
>>> Sent from the zookeeper-user mailing list archive at Nabble.com.
>>>
>>
>>
>> If you reply to this email, your message will be added to the discussion below:
>> http://zookeeper-user.578899.n2.nabble.com/ZooKeeper-Limitation-tp6675643p6678230.html
>> To unsubscribe from ZooKeeper Limitation, click here.
>
>
> --
> View this message in context: http://zookeeper-user.578899.n2.nabble.com/ZooKeeper-Limitation-tp6675643p6682887.html
> Sent from the zookeeper-user mailing list archive at Nabble.com.
Re: ZooKeeper Limitation
Posted by yosefarr <yo...@gmail.com>.
So I can have one node with 500000 sub nodes(Childs) and the get Childs will fail but get one child (getChild("/mainNode/child50")) will not fail because the request is less than 1M is that right?
Sent from my iPhone
On 12 באוג 2011, at 00:36, "Ted Dunning [via zookeeper-user]"<ml...@n2.nabble.com> wrote:
> Same answer as before.
>
> Your request and your response have to fit into 1M byte. How many nodes
> that will be depends on their names.
>
> Hint: don't push the limit. Design in a way that is nowhere close to the
> limit.
>
> On Thu, Aug 11, 2011 at 2:16 PM, yosefarr <[hidden email]> wrote:
>
> > what will work, how much maximum childs in a single node
> >
> > On Fri, Aug 12, 2011 at 12:07 AM, Ted Dunning [via zookeeper-user] <
> > [hidden email]> wrote:
> >
> > > 1M byte is the limit.
> > >
> > > You need to put multiple levels of directory in there. 500,000 znodes in
> > a
> > >
> > > single directory won't work.
> > >
> > > On Thu, Aug 11, 2011 at 1:33 PM, yosefarr <[hidden email]<
> > http://user/SendEmail.jtp?type=node&node=6678116&i=0>>
> > > wrote:
> > >
> > > > Thanks you for your replay,
> > > > What do you mean that there is limit on data calls to ZK?what is the
> > > limit?
> > > > We will have too many direct childrens for a znode:
> > > > /znode
> > > > /child-1
> > > > /child-2
> > > > ........
> > > > /child-n
> > > > when "n" can be 500000 or more
> > > > in this case if i run getChildren("/znode") ,does this call will fail?
> > > > or getData("/znode/child-n") will fail?
> > > >
> > > > Thanks
> > > > Yosef.
> > > >
> > > >
> > > > On Thu, Aug 11, 2011 at 8:32 PM, Mahadev Konar-4 [via zookeeper-user] <
> > > > [hidden email] <http://user/SendEmail.jtp?type=node&node=6678116&i=1>>
> > > wrote:
> > > >
> > > > > Yosef,
> > > > > There isnt a limit on the number of ZK node numbers directly. We do
> > > > > have a limit on data calls to ZK. So in your case if you have too
> > many
> > > > > direct children of a single znode (say millions), the getChildren
> > > > > calls return an error. I'd advise against having too many direct
> > > > > chidlren for a znode. Bucketing the users might make sense for you if
> > > > > you have too many users.
> > > > >
> > > > > thanks
> > > > > mahadev
> > > > >
> > > > > On Thu, Aug 11, 2011 at 2:19 AM, yosefarr <[hidden email]<
> > > > http://user/SendEmail.jtp?type=node&node=6677260&i=0>>
> > > > > wrote:
> > > > >
> > > > > > Hi,
> > > > > > If i want to save users that connect to my system as zookeeper
> > nodes
> > > > and
> > > > > > each user has some proprties - connection id, user name and etc:
> > > > > > i need to know if there is a Limitation on ZooKeeper nodes numbers?
> > > > > >
> > > > > > /server1
> > > > > > /user-1
> > > > > > /user-2
> > > > > > ........
> > > > > > /user-n
> > > > > >
> > > > > > Thanks,
> > > > > > Yosef.
> > > > > >
> > > > > > --
> > > > > > View this message in context:
> > > > >
> > > >
> > >
> > http://zookeeper-user.578899.n2.nabble.com/ZooKeeper-Limitation-tp6675643p6675643.html
> > > > > > Sent from the zookeeper-user mailing list archive at Nabble.com.
> > > > >
> > > > >
> > > > > ------------------------------
> > > > > If you reply to this email, your message will be added to the
> > > discussion
> > > > > below:
> > > > >
> > > > >
> > > >
> > >
> > http://zookeeper-user.578899.n2.nabble.com/ZooKeeper-Limitation-tp6675643p6677260.html
> > > > > To unsubscribe from ZooKeeper Limitation, click here<
> > > >
> > > >.
> > > > >
> > > > >
> > > >
> > > >
> > > > --
> > > > View this message in context:
> > > >
> > >
> > http://zookeeper-user.578899.n2.nabble.com/ZooKeeper-Limitation-tp6675643p6677984.html
> > >
> > > > Sent from the zookeeper-user mailing list archive at Nabble.com.
> > >
> > >
> > > ------------------------------
> > > If you reply to this email, your message will be added to the discussion
> > > below:
> > >
> > >
> > http://zookeeper-user.578899.n2.nabble.com/ZooKeeper-Limitation-tp6675643p6678116.html
> > > To unsubscribe from ZooKeeper Limitation, click here<
> > >.
> > >
> > >
> >
> >
> > --
> > View this message in context:
> > http://zookeeper-user.578899.n2.nabble.com/ZooKeeper-Limitation-tp6675643p6678146.html
> > Sent from the zookeeper-user mailing list archive at Nabble.com.
> >
>
>
> If you reply to this email, your message will be added to the discussion below:
> http://zookeeper-user.578899.n2.nabble.com/ZooKeeper-Limitation-tp6675643p6678230.html
> To unsubscribe from ZooKeeper Limitation, click here.
--
View this message in context: http://zookeeper-user.578899.n2.nabble.com/ZooKeeper-Limitation-tp6675643p6682887.html
Sent from the zookeeper-user mailing list archive at Nabble.com.
Re: ZooKeeper Limitation
Posted by Ted Dunning <te...@gmail.com>.
Same answer as before.
Your request and your response have to fit into 1M byte. How many nodes
that will be depends on their names.
Hint: don't push the limit. Design in a way that is nowhere close to the
limit.
On Thu, Aug 11, 2011 at 2:16 PM, yosefarr <yo...@gmail.com> wrote:
> what will work, how much maximum childs in a single node
>
> On Fri, Aug 12, 2011 at 12:07 AM, Ted Dunning [via zookeeper-user] <
> ml-node+6678116-543095410-343064@n2.nabble.com> wrote:
>
> > 1M byte is the limit.
> >
> > You need to put multiple levels of directory in there. 500,000 znodes in
> a
> >
> > single directory won't work.
> >
> > On Thu, Aug 11, 2011 at 1:33 PM, yosefarr <[hidden email]<
> http://user/SendEmail.jtp?type=node&node=6678116&i=0>>
> > wrote:
> >
> > > Thanks you for your replay,
> > > What do you mean that there is limit on data calls to ZK?what is the
> > limit?
> > > We will have too many direct childrens for a znode:
> > > /znode
> > > /child-1
> > > /child-2
> > > ........
> > > /child-n
> > > when "n" can be 500000 or more
> > > in this case if i run getChildren("/znode") ,does this call will fail?
> > > or getData("/znode/child-n") will fail?
> > >
> > > Thanks
> > > Yosef.
> > >
> > >
> > > On Thu, Aug 11, 2011 at 8:32 PM, Mahadev Konar-4 [via zookeeper-user] <
> > > [hidden email] <http://user/SendEmail.jtp?type=node&node=6678116&i=1>>
> > wrote:
> > >
> > > > Yosef,
> > > > There isnt a limit on the number of ZK node numbers directly. We do
> > > > have a limit on data calls to ZK. So in your case if you have too
> many
> > > > direct children of a single znode (say millions), the getChildren
> > > > calls return an error. I'd advise against having too many direct
> > > > chidlren for a znode. Bucketing the users might make sense for you if
> > > > you have too many users.
> > > >
> > > > thanks
> > > > mahadev
> > > >
> > > > On Thu, Aug 11, 2011 at 2:19 AM, yosefarr <[hidden email]<
> > > http://user/SendEmail.jtp?type=node&node=6677260&i=0>>
> > > > wrote:
> > > >
> > > > > Hi,
> > > > > If i want to save users that connect to my system as zookeeper
> nodes
> > > and
> > > > > each user has some proprties - connection id, user name and etc:
> > > > > i need to know if there is a Limitation on ZooKeeper nodes numbers?
> > > > >
> > > > > /server1
> > > > > /user-1
> > > > > /user-2
> > > > > ........
> > > > > /user-n
> > > > >
> > > > > Thanks,
> > > > > Yosef.
> > > > >
> > > > > --
> > > > > View this message in context:
> > > >
> > >
> >
> http://zookeeper-user.578899.n2.nabble.com/ZooKeeper-Limitation-tp6675643p6675643.html
> > > > > Sent from the zookeeper-user mailing list archive at Nabble.com.
> > > >
> > > >
> > > > ------------------------------
> > > > If you reply to this email, your message will be added to the
> > discussion
> > > > below:
> > > >
> > > >
> > >
> >
> http://zookeeper-user.578899.n2.nabble.com/ZooKeeper-Limitation-tp6675643p6677260.html
> > > > To unsubscribe from ZooKeeper Limitation, click here<
> > >
> > >.
> > > >
> > > >
> > >
> > >
> > > --
> > > View this message in context:
> > >
> >
> http://zookeeper-user.578899.n2.nabble.com/ZooKeeper-Limitation-tp6675643p6677984.html
> >
> > > Sent from the zookeeper-user mailing list archive at Nabble.com.
> >
> >
> > ------------------------------
> > If you reply to this email, your message will be added to the discussion
> > below:
> >
> >
> http://zookeeper-user.578899.n2.nabble.com/ZooKeeper-Limitation-tp6675643p6678116.html
> > To unsubscribe from ZooKeeper Limitation, click here<
> http://zookeeper-user.578899.n2.nabble.com/template/NamlServlet.jtp?macro=unsubscribe_by_code&node=6675643&code=eW9zZWYuYXJyYWZAZ21haWwuY29tfDY2NzU2NDN8MTI4MTA4Mjk2NQ==
> >.
> >
> >
>
>
> --
> View this message in context:
> http://zookeeper-user.578899.n2.nabble.com/ZooKeeper-Limitation-tp6675643p6678146.html
> Sent from the zookeeper-user mailing list archive at Nabble.com.
>
Re: ZooKeeper Limitation
Posted by yosefarr <yo...@gmail.com>.
what will work, how much maximum childs in a single node
On Fri, Aug 12, 2011 at 12:07 AM, Ted Dunning [via zookeeper-user] <
ml-node+6678116-543095410-343064@n2.nabble.com> wrote:
> 1M byte is the limit.
>
> You need to put multiple levels of directory in there. 500,000 znodes in a
>
> single directory won't work.
>
> On Thu, Aug 11, 2011 at 1:33 PM, yosefarr <[hidden email]<http://user/SendEmail.jtp?type=node&node=6678116&i=0>>
> wrote:
>
> > Thanks you for your replay,
> > What do you mean that there is limit on data calls to ZK?what is the
> limit?
> > We will have too many direct childrens for a znode:
> > /znode
> > /child-1
> > /child-2
> > ........
> > /child-n
> > when "n" can be 500000 or more
> > in this case if i run getChildren("/znode") ,does this call will fail?
> > or getData("/znode/child-n") will fail?
> >
> > Thanks
> > Yosef.
> >
> >
> > On Thu, Aug 11, 2011 at 8:32 PM, Mahadev Konar-4 [via zookeeper-user] <
> > [hidden email] <http://user/SendEmail.jtp?type=node&node=6678116&i=1>>
> wrote:
> >
> > > Yosef,
> > > There isnt a limit on the number of ZK node numbers directly. We do
> > > have a limit on data calls to ZK. So in your case if you have too many
> > > direct children of a single znode (say millions), the getChildren
> > > calls return an error. I'd advise against having too many direct
> > > chidlren for a znode. Bucketing the users might make sense for you if
> > > you have too many users.
> > >
> > > thanks
> > > mahadev
> > >
> > > On Thu, Aug 11, 2011 at 2:19 AM, yosefarr <[hidden email]<
> > http://user/SendEmail.jtp?type=node&node=6677260&i=0>>
> > > wrote:
> > >
> > > > Hi,
> > > > If i want to save users that connect to my system as zookeeper nodes
> > and
> > > > each user has some proprties - connection id, user name and etc:
> > > > i need to know if there is a Limitation on ZooKeeper nodes numbers?
> > > >
> > > > /server1
> > > > /user-1
> > > > /user-2
> > > > ........
> > > > /user-n
> > > >
> > > > Thanks,
> > > > Yosef.
> > > >
> > > > --
> > > > View this message in context:
> > >
> >
> http://zookeeper-user.578899.n2.nabble.com/ZooKeeper-Limitation-tp6675643p6675643.html
> > > > Sent from the zookeeper-user mailing list archive at Nabble.com.
> > >
> > >
> > > ------------------------------
> > > If you reply to this email, your message will be added to the
> discussion
> > > below:
> > >
> > >
> >
> http://zookeeper-user.578899.n2.nabble.com/ZooKeeper-Limitation-tp6675643p6677260.html
> > > To unsubscribe from ZooKeeper Limitation, click here<
> >
> >.
> > >
> > >
> >
> >
> > --
> > View this message in context:
> >
> http://zookeeper-user.578899.n2.nabble.com/ZooKeeper-Limitation-tp6675643p6677984.html
>
> > Sent from the zookeeper-user mailing list archive at Nabble.com.
>
>
> ------------------------------
> If you reply to this email, your message will be added to the discussion
> below:
>
> http://zookeeper-user.578899.n2.nabble.com/ZooKeeper-Limitation-tp6675643p6678116.html
> To unsubscribe from ZooKeeper Limitation, click here<http://zookeeper-user.578899.n2.nabble.com/template/NamlServlet.jtp?macro=unsubscribe_by_code&node=6675643&code=eW9zZWYuYXJyYWZAZ21haWwuY29tfDY2NzU2NDN8MTI4MTA4Mjk2NQ==>.
>
>
--
View this message in context: http://zookeeper-user.578899.n2.nabble.com/ZooKeeper-Limitation-tp6675643p6678146.html
Sent from the zookeeper-user mailing list archive at Nabble.com.
Re: ZooKeeper Limitation
Posted by Ted Dunning <te...@gmail.com>.
1M byte is the limit.
You need to put multiple levels of directory in there. 500,000 znodes in a
single directory won't work.
On Thu, Aug 11, 2011 at 1:33 PM, yosefarr <yo...@gmail.com> wrote:
> Thanks you for your replay,
> What do you mean that there is limit on data calls to ZK?what is the limit?
> We will have too many direct childrens for a znode:
> /znode
> /child-1
> /child-2
> ........
> /child-n
> when "n" can be 500000 or more
> in this case if i run getChildren("/znode") ,does this call will fail?
> or getData("/znode/child-n") will fail?
>
> Thanks
> Yosef.
>
>
> On Thu, Aug 11, 2011 at 8:32 PM, Mahadev Konar-4 [via zookeeper-user] <
> ml-node+6677260-1581576647-343064@n2.nabble.com> wrote:
>
> > Yosef,
> > There isnt a limit on the number of ZK node numbers directly. We do
> > have a limit on data calls to ZK. So in your case if you have too many
> > direct children of a single znode (say millions), the getChildren
> > calls return an error. I'd advise against having too many direct
> > chidlren for a znode. Bucketing the users might make sense for you if
> > you have too many users.
> >
> > thanks
> > mahadev
> >
> > On Thu, Aug 11, 2011 at 2:19 AM, yosefarr <[hidden email]<
> http://user/SendEmail.jtp?type=node&node=6677260&i=0>>
> > wrote:
> >
> > > Hi,
> > > If i want to save users that connect to my system as zookeeper nodes
> and
> > > each user has some proprties - connection id, user name and etc:
> > > i need to know if there is a Limitation on ZooKeeper nodes numbers?
> > >
> > > /server1
> > > /user-1
> > > /user-2
> > > ........
> > > /user-n
> > >
> > > Thanks,
> > > Yosef.
> > >
> > > --
> > > View this message in context:
> >
> http://zookeeper-user.578899.n2.nabble.com/ZooKeeper-Limitation-tp6675643p6675643.html
> > > Sent from the zookeeper-user mailing list archive at Nabble.com.
> >
> >
> > ------------------------------
> > If you reply to this email, your message will be added to the discussion
> > below:
> >
> >
> http://zookeeper-user.578899.n2.nabble.com/ZooKeeper-Limitation-tp6675643p6677260.html
> > To unsubscribe from ZooKeeper Limitation, click here<
> http://zookeeper-user.578899.n2.nabble.com/template/NamlServlet.jtp?macro=unsubscribe_by_code&node=6675643&code=eW9zZWYuYXJyYWZAZ21haWwuY29tfDY2NzU2NDN8MTI4MTA4Mjk2NQ==
> >.
> >
> >
>
>
> --
> View this message in context:
> http://zookeeper-user.578899.n2.nabble.com/ZooKeeper-Limitation-tp6675643p6677984.html
> Sent from the zookeeper-user mailing list archive at Nabble.com.
Re: ZooKeeper Limitation
Posted by yosefarr <yo...@gmail.com>.
Thanks you for your replay,
What do you mean that there is limit on data calls to ZK?what is the limit?
We will have too many direct childrens for a znode:
/znode
/child-1
/child-2
........
/child-n
when "n" can be 500000 or more
in this case if i run getChildren("/znode") ,does this call will fail?
or getData("/znode/child-n") will fail?
Thanks
Yosef.
On Thu, Aug 11, 2011 at 8:32 PM, Mahadev Konar-4 [via zookeeper-user] <
ml-node+6677260-1581576647-343064@n2.nabble.com> wrote:
> Yosef,
> There isnt a limit on the number of ZK node numbers directly. We do
> have a limit on data calls to ZK. So in your case if you have too many
> direct children of a single znode (say millions), the getChildren
> calls return an error. I'd advise against having too many direct
> chidlren for a znode. Bucketing the users might make sense for you if
> you have too many users.
>
> thanks
> mahadev
>
> On Thu, Aug 11, 2011 at 2:19 AM, yosefarr <[hidden email]<http://user/SendEmail.jtp?type=node&node=6677260&i=0>>
> wrote:
>
> > Hi,
> > If i want to save users that connect to my system as zookeeper nodes and
> > each user has some proprties - connection id, user name and etc:
> > i need to know if there is a Limitation on ZooKeeper nodes numbers?
> >
> > /server1
> > /user-1
> > /user-2
> > ........
> > /user-n
> >
> > Thanks,
> > Yosef.
> >
> > --
> > View this message in context:
> http://zookeeper-user.578899.n2.nabble.com/ZooKeeper-Limitation-tp6675643p6675643.html
> > Sent from the zookeeper-user mailing list archive at Nabble.com.
>
>
> ------------------------------
> If you reply to this email, your message will be added to the discussion
> below:
>
> http://zookeeper-user.578899.n2.nabble.com/ZooKeeper-Limitation-tp6675643p6677260.html
> To unsubscribe from ZooKeeper Limitation, click here<http://zookeeper-user.578899.n2.nabble.com/template/NamlServlet.jtp?macro=unsubscribe_by_code&node=6675643&code=eW9zZWYuYXJyYWZAZ21haWwuY29tfDY2NzU2NDN8MTI4MTA4Mjk2NQ==>.
>
>
--
View this message in context: http://zookeeper-user.578899.n2.nabble.com/ZooKeeper-Limitation-tp6675643p6677984.html
Sent from the zookeeper-user mailing list archive at Nabble.com.
Re: ZooKeeper Limitation
Posted by Mahadev Konar <ma...@hortonworks.com>.
Yosef,
There isnt a limit on the number of ZK node numbers directly. We do
have a limit on data calls to ZK. So in your case if you have too many
direct children of a single znode (say millions), the getChildren
calls return an error. I'd advise against having too many direct
chidlren for a znode. Bucketing the users might make sense for you if
you have too many users.
thanks
mahadev
On Thu, Aug 11, 2011 at 2:19 AM, yosefarr <yo...@gmail.com> wrote:
> Hi,
> If i want to save users that connect to my system as zookeeper nodes and
> each user has some proprties - connection id, user name and etc:
> i need to know if there is a Limitation on ZooKeeper nodes numbers?
>
> /server1
> /user-1
> /user-2
> ........
> /user-n
>
> Thanks,
> Yosef.
>
> --
> View this message in context: http://zookeeper-user.578899.n2.nabble.com/ZooKeeper-Limitation-tp6675643p6675643.html
> Sent from the zookeeper-user mailing list archive at Nabble.com.