You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@curator.apache.org by Techy Teck <co...@gmail.com> on 2013/11/07 19:45:08 UTC
Re: Best Practice while creating znodes?
Can anyone help me on this?
Is there any best practices that we need to follow while creating the
znodes?
Meaning the znode name should be small not very large names or there
shouldn't be a dot or underscore within the word?
Anything like that?
I will be using Apache Curator.
On Wed, Nov 6, 2013 at 11:24 AM, Techy Teck <co...@gmail.com> wrote:
> Is there any best practices that we need to follow while creating the
> znodes?
>
> Meaning the znode name should be small not very large names or there
> shouldn't be a dot or underscore within the word?
>
> Anything like that?
>
>
Re: Best Practice while creating znodes?
Posted by Jordan Zimmerman <jo...@jordanzimmerman.com>.
Totally up to you.
On Nov 7, 2013, at 2:54 PM, Techy Teck <co...@gmail.com> wrote:
> Sure.. Thanks for the help..
>
> Apart from that like, znodes name should be small? Is that the case as
> well? Or it doesn't matter if it is long as well?
>
>
> On Thu, Nov 7, 2013 at 10:57 AM, Jordan Zimmerman <
> jordan@jordanzimmerman.com> wrote:
>
>> There are very few limitations on naming. You can't use a slash. Have a
>> look at:
>>
>> org.apache.zookeeper.common.PathUtils
>>
>> In terms of length, ZooKeeper has a 1MB limit per API call. So, for
>> example, if you call getChildren(), the entire length of all the children
>> can't exceed 1MB (unless you reconfigure ZK).
>>
>> -JZ
>>
>> On Nov 7, 2013, at 10:45 AM, Techy Teck <co...@gmail.com> wrote:
>>
>> Can anyone help me on this?
>>
>> Is there any best practices that we need to follow while creating the
>> znodes?
>>
>> Meaning the znode name should be small not very large names or there
>> shouldn't be a dot or underscore within the word?
>>
>> Anything like that?
>>
>> I will be using Apache Curator.
>>
>>
>>
>>
>> On Wed, Nov 6, 2013 at 11:24 AM, Techy Teck <co...@gmail.com>wrote:
>>
>>> Is there any best practices that we need to follow while creating the
>>> znodes?
>>>
>>> Meaning the znode name should be small not very large names or there
>>> shouldn't be a dot or underscore within the word?
>>>
>>> Anything like that?
>>>
>>>
>>
>>
Re: Best Practice while creating znodes?
Posted by Jordan Zimmerman <jo...@jordanzimmerman.com>.
Totally up to you.
On Nov 7, 2013, at 2:54 PM, Techy Teck <co...@gmail.com> wrote:
> Sure.. Thanks for the help..
>
> Apart from that like, znodes name should be small? Is that the case as
> well? Or it doesn't matter if it is long as well?
>
>
> On Thu, Nov 7, 2013 at 10:57 AM, Jordan Zimmerman <
> jordan@jordanzimmerman.com> wrote:
>
>> There are very few limitations on naming. You can't use a slash. Have a
>> look at:
>>
>> org.apache.zookeeper.common.PathUtils
>>
>> In terms of length, ZooKeeper has a 1MB limit per API call. So, for
>> example, if you call getChildren(), the entire length of all the children
>> can't exceed 1MB (unless you reconfigure ZK).
>>
>> -JZ
>>
>> On Nov 7, 2013, at 10:45 AM, Techy Teck <co...@gmail.com> wrote:
>>
>> Can anyone help me on this?
>>
>> Is there any best practices that we need to follow while creating the
>> znodes?
>>
>> Meaning the znode name should be small not very large names or there
>> shouldn't be a dot or underscore within the word?
>>
>> Anything like that?
>>
>> I will be using Apache Curator.
>>
>>
>>
>>
>> On Wed, Nov 6, 2013 at 11:24 AM, Techy Teck <co...@gmail.com>wrote:
>>
>>> Is there any best practices that we need to follow while creating the
>>> znodes?
>>>
>>> Meaning the znode name should be small not very large names or there
>>> shouldn't be a dot or underscore within the word?
>>>
>>> Anything like that?
>>>
>>>
>>
>>
Re: Best Practice while creating znodes?
Posted by Techy Teck <co...@gmail.com>.
Sure.. Thanks for the help..
Apart from that like, znodes name should be small? Is that the case as
well? Or it doesn't matter if it is long as well?
On Thu, Nov 7, 2013 at 10:57 AM, Jordan Zimmerman <
jordan@jordanzimmerman.com> wrote:
> There are very few limitations on naming. You can't use a slash. Have a
> look at:
>
> org.apache.zookeeper.common.PathUtils
>
> In terms of length, ZooKeeper has a 1MB limit per API call. So, for
> example, if you call getChildren(), the entire length of all the children
> can't exceed 1MB (unless you reconfigure ZK).
>
> -JZ
>
> On Nov 7, 2013, at 10:45 AM, Techy Teck <co...@gmail.com> wrote:
>
> Can anyone help me on this?
>
> Is there any best practices that we need to follow while creating the
> znodes?
>
> Meaning the znode name should be small not very large names or there
> shouldn't be a dot or underscore within the word?
>
> Anything like that?
>
> I will be using Apache Curator.
>
>
>
>
> On Wed, Nov 6, 2013 at 11:24 AM, Techy Teck <co...@gmail.com>wrote:
>
>> Is there any best practices that we need to follow while creating the
>> znodes?
>>
>> Meaning the znode name should be small not very large names or there
>> shouldn't be a dot or underscore within the word?
>>
>> Anything like that?
>>
>>
>
>
Re: Best Practice while creating znodes?
Posted by Check Peck <co...@gmail.com>.
By client you mean all those applications who are connecting to zookeeper?
I am using Curator, can we set it through that as well?
On Sun, Sep 13, 2015 at 2:14 PM, Jordan Zimmerman <
jordan@jordanzimmerman.com> wrote:
> If I can increase the jute.maxBuffer property, I do need to restart the
> whole cluster. Correct? Is there any problem if I increase this?
>
>
> Yes. Also, I think it has to be set on both the server and client.
> Frankly, I’ve never understand this 1MB limit. It seems unnecessary. At
> Netflix, I ran ZK with a larger number (I don’t remember what though).
>
> -Jordan
>
>
>
> On September 13, 2015 at 4:11:19 PM, Check Peck (comptechgeeky@gmail.com)
> wrote:
>
> Ok understood that part and I was correct on the 500 children nodes znode
> name example right?
>
> If I can increase the jute.maxBuffer property, I do need to restart the
> whole cluster. Correct? Is there any problem if I increase this?
>
> On Sun, Sep 13, 2015 at 2:08 PM, Jordan Zimmerman <
> jordan@jordanzimmerman.com> wrote:
>
>> No, the 1MB limit will hit you there as well. If you try to get your data
>> (via getData() call) it will fail. Note, you can increase the value via the
>> jute.maxBuffer property.
>>
>> -Jordan
>>
>> On September 13, 2015 at 4:06:48 PM, Check Peck (comptechgeeky@gmail.com)
>> wrote:
>>
>> What about data in those znode? They can be more than 1MB in each znode
>> correct?
>>
>>
>
Re: Best Practice while creating znodes?
Posted by Check Peck <co...@gmail.com>.
By client you mean all those applications who are connecting to zookeeper?
I am using Curator, can we set it through that as well?
On Sun, Sep 13, 2015 at 2:14 PM, Jordan Zimmerman <
jordan@jordanzimmerman.com> wrote:
> If I can increase the jute.maxBuffer property, I do need to restart the
> whole cluster. Correct? Is there any problem if I increase this?
>
>
> Yes. Also, I think it has to be set on both the server and client.
> Frankly, I’ve never understand this 1MB limit. It seems unnecessary. At
> Netflix, I ran ZK with a larger number (I don’t remember what though).
>
> -Jordan
>
>
>
> On September 13, 2015 at 4:11:19 PM, Check Peck (comptechgeeky@gmail.com)
> wrote:
>
> Ok understood that part and I was correct on the 500 children nodes znode
> name example right?
>
> If I can increase the jute.maxBuffer property, I do need to restart the
> whole cluster. Correct? Is there any problem if I increase this?
>
> On Sun, Sep 13, 2015 at 2:08 PM, Jordan Zimmerman <
> jordan@jordanzimmerman.com> wrote:
>
>> No, the 1MB limit will hit you there as well. If you try to get your data
>> (via getData() call) it will fail. Note, you can increase the value via the
>> jute.maxBuffer property.
>>
>> -Jordan
>>
>> On September 13, 2015 at 4:06:48 PM, Check Peck (comptechgeeky@gmail.com)
>> wrote:
>>
>> What about data in those znode? They can be more than 1MB in each znode
>> correct?
>>
>>
>
Re: Best Practice while creating znodes?
Posted by Jordan Zimmerman <jo...@jordanzimmerman.com>.
If I can increase the jute.maxBuffer property, I do need to restart the whole cluster. Correct? Is there any problem if I increase this?
Yes. Also, I think it has to be set on both the server and client. Frankly, I’ve never understand this 1MB limit. It seems unnecessary. At Netflix, I ran ZK with a larger number (I don’t remember what though).
-Jordan
On September 13, 2015 at 4:11:19 PM, Check Peck (comptechgeeky@gmail.com) wrote:
Ok understood that part and I was correct on the 500 children nodes znode name example right?
If I can increase the jute.maxBuffer property, I do need to restart the whole cluster. Correct? Is there any problem if I increase this?
On Sun, Sep 13, 2015 at 2:08 PM, Jordan Zimmerman <jo...@jordanzimmerman.com> wrote:
No, the 1MB limit will hit you there as well. If you try to get your data (via getData() call) it will fail. Note, you can increase the value via the jute.maxBuffer property.
-Jordan
On September 13, 2015 at 4:06:48 PM, Check Peck (comptechgeeky@gmail.com) wrote:
What about data in those znode? They can be more than 1MB in each znode correct?
Re: Best Practice while creating znodes?
Posted by Jordan Zimmerman <jo...@jordanzimmerman.com>.
If I can increase the jute.maxBuffer property, I do need to restart the whole cluster. Correct? Is there any problem if I increase this?
Yes. Also, I think it has to be set on both the server and client. Frankly, I’ve never understand this 1MB limit. It seems unnecessary. At Netflix, I ran ZK with a larger number (I don’t remember what though).
-Jordan
On September 13, 2015 at 4:11:19 PM, Check Peck (comptechgeeky@gmail.com) wrote:
Ok understood that part and I was correct on the 500 children nodes znode name example right?
If I can increase the jute.maxBuffer property, I do need to restart the whole cluster. Correct? Is there any problem if I increase this?
On Sun, Sep 13, 2015 at 2:08 PM, Jordan Zimmerman <jo...@jordanzimmerman.com> wrote:
No, the 1MB limit will hit you there as well. If you try to get your data (via getData() call) it will fail. Note, you can increase the value via the jute.maxBuffer property.
-Jordan
On September 13, 2015 at 4:06:48 PM, Check Peck (comptechgeeky@gmail.com) wrote:
What about data in those znode? They can be more than 1MB in each znode correct?
Re: Best Practice while creating znodes?
Posted by Chris Chen <cc...@twitter.com.INVALID>.
Tread carefully!
Yes it's configurable but it only takes one client that barfs on a large
payload to ruin your day (or at least make it really weird).
Jordan is right, it applies to all jute serialization calls; I've seen it
when a client is trying to send a setWatches packet during reconnection,
because the client has enough watches set to exceed the 1MB limit.
:\
cc
On Sun, Sep 13, 2015 at 2:24 PM, Jordan Zimmerman <
jordan@jordanzimmerman.com> wrote:
> Why all client has to set this limit. If on zookeeper side we have
> configured this then it should use that. No?
>
> My read of the code shows that the clients also use jute.maxBuffer.
> Hopefully, one of the ZK committers can comment on this. But, jute
> (see BinaryInputArchive.java) is used on both the client and the server.
>
> I don't see any methods in CuratorrFramework which takes this variable.
>
> Per the ZK docs:
>
>
> jute.maxbuffer:
>
> (Java system property:* jute.maxbuffer*)
>
> This option can only be set as a Java system property. There is no
> zookeeper prefix on it.
>
>
>
>
> On September 13, 2015 at 4:22:46 PM, Check Peck (comptechgeeky@gmail.com)
> wrote:
>
> Why all client has to set this limit. If on zookeeper side we have
> configured this then it should use that. No?
>
> Also how can we set this parameter through Curator? I don't see any
> methods in CuratorrFramework which takes this variable.
>
> On Sun, Sep 13, 2015 at 2:17 PM, Jordan Zimmerman <
> jordan@jordanzimmerman.com> wrote:
>
>> Yes, all clients. Curator just wraps ZooKeeper so all the same things
>> apply.
>>
>> -Jordan
>>
>>
>>
>> On September 13, 2015 at 4:14:17 PM, Check Peck (comptechgeeky@gmail.com)
>> wrote:
>>
>> I mean this code for getting children not earlier one, that is for data.
>>
>> List<String> children = client.getChildren().forPath(path);
>>
>> On Sun, Sep 13, 2015 at 2:10 PM, Check Peck <co...@gmail.com>
>> wrote:
>>
>>> Ok understood that part and I was correct on the 500 children nodes
>>> znode name example right?
>>>
>>> If I can increase the jute.maxBuffer property, I do need to restart the
>>> whole cluster. Correct? Is there any problem if I increase this?
>>>
>>> On Sun, Sep 13, 2015 at 2:08 PM, Jordan Zimmerman <
>>> jordan@jordanzimmerman.com> wrote:
>>>
>>>> No, the 1MB limit will hit you there as well. If you try to get your
>>>> data (via getData() call) it will fail. Note, you can increase the value
>>>> via the jute.maxBuffer property.
>>>>
>>>> -Jordan
>>>>
>>>> On September 13, 2015 at 4:06:48 PM, Check Peck (
>>>> comptechgeeky@gmail.com) wrote:
>>>>
>>>> What about data in those znode? They can be more than 1MB in each znode
>>>> correct?
>>>>
>>>>
>>>
>>
>
--
Christopher COORD Chen <cc...@twitter.com> @secretasianman
Re: Best Practice while creating znodes?
Posted by Marco Primi <ma...@gmail.com>.
A quick parenthesis while talking about hitting the buffer limit when
listing children:
https://issues.apache.org/jira/browse/ZOOKEEPER-2260 Paginated
getChildren call
It would be great to have some feedback on this patch.
On 9/14/15 2:40 AM, Karol Dudzinski wrote:
> Just had a very quick look at the code (on my phone so not that easy to read) and I think what I observed can be explained by the fact that BinaryInputArchive uses that property but BinaryOutputArchive does not. Therefore it sounds like a response of arbitrary size can be sent but it can only be received successfully if it's smaller than the buffer size. That would explain why I only needed the property on my client but if you want data larger than 1mb you'd need to set it on the server.
>
> Perhaps a committer (or someone not reading the code on a phone) can confirm.
>
> Karol
>
>> On 14 Sep 2015, at 09:34, Karol Dudzinski <ka...@gmail.com> wrote:
>>
>> I think there might be a bit more to this. I've not set the buffer size on the servers as I don't have a need to store anywhere near 1mb per node. However, I have hit the buffer size limit while calling getChildren. I set the buffer size only on the client calling getChildren and nowhere else and the exception went away so I'm not convinced the server uses that property for every response.
>>
>> I agree with Jordan that the docs are slightly confusing as they don't even mention that this property will affect getChildren calls for example. However, I can't say I've looked at this bit of code but that's certainly the behaviour I observed.
>>
>> Karol
>>
>>> On 13 Sep 2015, at 22:24, Jordan Zimmerman <jo...@jordanzimmerman.com> wrote:
>>>
>>> Why all client has to set this limit. If on zookeeper side we have configured this then it should use that. No?
>>> My read of the code shows that the clients also use jute.maxBuffer. Hopefully, one of the ZK committers can comment on this. But, jute (see BinaryInputArchive.java) is used on both the client and the server.
>>>
>>> I don't see any methods in CuratorrFramework which takes this variable.
>>> Per the ZK docs:
>>>
>>>
>>>
>>> jute.maxbuffer:
>>> (Java system property: jute.maxbuffer)
>>>
>>> This option can only be set as a Java system property. There is no zookeeper prefix on it.
>>>
>>>
>>>
>>>
>>>
>>> On September 13, 2015 at 4:22:46 PM, Check Peck (comptechgeeky@gmail.com) wrote:
>>>
>>> Why all client has to set this limit. If on zookeeper side we have configured this then it should use that. No?
>>>
>>> Also how can we set this parameter through Curator? I don't see any methods in CuratorrFramework which takes this variable.
>>>
>>> On Sun, Sep 13, 2015 at 2:17 PM, Jordan Zimmerman <jo...@jordanzimmerman.com> wrote:
>>> Yes, all clients. Curator just wraps ZooKeeper so all the same things apply.
>>>
>>> -Jordan
>>>
>>>
>>>
>>> On September 13, 2015 at 4:14:17 PM, Check Peck (comptechgeeky@gmail.com) wrote:
>>>
>>> I mean this code for getting children not earlier one, that is for data.
>>>
>>> List<String> children = client.getChildren().forPath(path);
>>>
>>> On Sun, Sep 13, 2015 at 2:10 PM, Check Peck <co...@gmail.com> wrote:
>>> Ok understood that part and I was correct on the 500 children nodes znode name example right?
>>>
>>> If I can increase the jute.maxBuffer property, I do need to restart the whole cluster. Correct? Is there any problem if I increase this?
>>>
>>> On Sun, Sep 13, 2015 at 2:08 PM, Jordan Zimmerman <jo...@jordanzimmerman.com> wrote:
>>> No, the 1MB limit will hit you there as well. If you try to get your data (via getData() call) it will fail. Note, you can increase the value via the jute.maxBuffer property.
>>>
>>> -Jordan
>>>
>>> On September 13, 2015 at 4:06:48 PM, Check Peck (comptechgeeky@gmail.com) wrote:
>>>
>>> What about data in those znode? They can be more than 1MB in each znode correct?
>>>
>>>
>>>
>>>
Re: Best Practice while creating znodes?
Posted by Karol Dudzinski <ka...@gmail.com>.
Just had a very quick look at the code (on my phone so not that easy to read) and I think what I observed can be explained by the fact that BinaryInputArchive uses that property but BinaryOutputArchive does not. Therefore it sounds like a response of arbitrary size can be sent but it can only be received successfully if it's smaller than the buffer size. That would explain why I only needed the property on my client but if you want data larger than 1mb you'd need to set it on the server.
Perhaps a committer (or someone not reading the code on a phone) can confirm.
Karol
> On 14 Sep 2015, at 09:34, Karol Dudzinski <ka...@gmail.com> wrote:
>
> I think there might be a bit more to this. I've not set the buffer size on the servers as I don't have a need to store anywhere near 1mb per node. However, I have hit the buffer size limit while calling getChildren. I set the buffer size only on the client calling getChildren and nowhere else and the exception went away so I'm not convinced the server uses that property for every response.
>
> I agree with Jordan that the docs are slightly confusing as they don't even mention that this property will affect getChildren calls for example. However, I can't say I've looked at this bit of code but that's certainly the behaviour I observed.
>
> Karol
>
>> On 13 Sep 2015, at 22:24, Jordan Zimmerman <jo...@jordanzimmerman.com> wrote:
>>
>> Why all client has to set this limit. If on zookeeper side we have configured this then it should use that. No?
>> My read of the code shows that the clients also use jute.maxBuffer. Hopefully, one of the ZK committers can comment on this. But, jute (see BinaryInputArchive.java) is used on both the client and the server.
>>
>> I don't see any methods in CuratorrFramework which takes this variable.
>> Per the ZK docs:
>>
>>
>>
>> jute.maxbuffer:
>> (Java system property: jute.maxbuffer)
>>
>> This option can only be set as a Java system property. There is no zookeeper prefix on it.
>>
>>
>>
>>
>>
>> On September 13, 2015 at 4:22:46 PM, Check Peck (comptechgeeky@gmail.com) wrote:
>>
>> Why all client has to set this limit. If on zookeeper side we have configured this then it should use that. No?
>>
>> Also how can we set this parameter through Curator? I don't see any methods in CuratorrFramework which takes this variable.
>>
>> On Sun, Sep 13, 2015 at 2:17 PM, Jordan Zimmerman <jo...@jordanzimmerman.com> wrote:
>> Yes, all clients. Curator just wraps ZooKeeper so all the same things apply.
>>
>> -Jordan
>>
>>
>>
>> On September 13, 2015 at 4:14:17 PM, Check Peck (comptechgeeky@gmail.com) wrote:
>>
>> I mean this code for getting children not earlier one, that is for data.
>>
>> List<String> children = client.getChildren().forPath(path);
>>
>> On Sun, Sep 13, 2015 at 2:10 PM, Check Peck <co...@gmail.com> wrote:
>> Ok understood that part and I was correct on the 500 children nodes znode name example right?
>>
>> If I can increase the jute.maxBuffer property, I do need to restart the whole cluster. Correct? Is there any problem if I increase this?
>>
>> On Sun, Sep 13, 2015 at 2:08 PM, Jordan Zimmerman <jo...@jordanzimmerman.com> wrote:
>> No, the 1MB limit will hit you there as well. If you try to get your data (via getData() call) it will fail. Note, you can increase the value via the jute.maxBuffer property.
>>
>> -Jordan
>>
>> On September 13, 2015 at 4:06:48 PM, Check Peck (comptechgeeky@gmail.com) wrote:
>>
>> What about data in those znode? They can be more than 1MB in each znode correct?
>>
>>
>>
>>
Re: Best Practice while creating znodes?
Posted by Karol Dudzinski <ka...@gmail.com>.
I think there might be a bit more to this. I've not set the buffer size on the servers as I don't have a need to store anywhere near 1mb per node. However, I have hit the buffer size limit while calling getChildren. I set the buffer size only on the client calling getChildren and nowhere else and the exception went away so I'm not convinced the server uses that property for every response.
I agree with Jordan that the docs are slightly confusing as they don't even mention that this property will affect getChildren calls for example. However, I can't say I've looked at this bit of code but that's certainly the behaviour I observed.
Karol
> On 13 Sep 2015, at 22:24, Jordan Zimmerman <jo...@jordanzimmerman.com> wrote:
>
> Why all client has to set this limit. If on zookeeper side we have configured this then it should use that. No?
> My read of the code shows that the clients also use jute.maxBuffer. Hopefully, one of the ZK committers can comment on this. But, jute (see BinaryInputArchive.java) is used on both the client and the server.
>
> I don't see any methods in CuratorrFramework which takes this variable.
> Per the ZK docs:
>
>
>
> jute.maxbuffer:
> (Java system property: jute.maxbuffer)
>
> This option can only be set as a Java system property. There is no zookeeper prefix on it.
>
>
>
>
>
> On September 13, 2015 at 4:22:46 PM, Check Peck (comptechgeeky@gmail.com) wrote:
>
> Why all client has to set this limit. If on zookeeper side we have configured this then it should use that. No?
>
> Also how can we set this parameter through Curator? I don't see any methods in CuratorrFramework which takes this variable.
>
> On Sun, Sep 13, 2015 at 2:17 PM, Jordan Zimmerman <jo...@jordanzimmerman.com> wrote:
> Yes, all clients. Curator just wraps ZooKeeper so all the same things apply.
>
> -Jordan
>
>
>
> On September 13, 2015 at 4:14:17 PM, Check Peck (comptechgeeky@gmail.com) wrote:
>
> I mean this code for getting children not earlier one, that is for data.
>
> List<String> children = client.getChildren().forPath(path);
>
> On Sun, Sep 13, 2015 at 2:10 PM, Check Peck <co...@gmail.com> wrote:
> Ok understood that part and I was correct on the 500 children nodes znode name example right?
>
> If I can increase the jute.maxBuffer property, I do need to restart the whole cluster. Correct? Is there any problem if I increase this?
>
> On Sun, Sep 13, 2015 at 2:08 PM, Jordan Zimmerman <jo...@jordanzimmerman.com> wrote:
> No, the 1MB limit will hit you there as well. If you try to get your data (via getData() call) it will fail. Note, you can increase the value via the jute.maxBuffer property.
>
> -Jordan
>
> On September 13, 2015 at 4:06:48 PM, Check Peck (comptechgeeky@gmail.com) wrote:
>
> What about data in those znode? They can be more than 1MB in each znode correct?
>
>
>
>
Re: Best Practice while creating znodes?
Posted by Chris Chen <cc...@twitter.com>.
Tread carefully!
Yes it's configurable but it only takes one client that barfs on a large
payload to ruin your day (or at least make it really weird).
Jordan is right, it applies to all jute serialization calls; I've seen it
when a client is trying to send a setWatches packet during reconnection,
because the client has enough watches set to exceed the 1MB limit.
:\
cc
On Sun, Sep 13, 2015 at 2:24 PM, Jordan Zimmerman <
jordan@jordanzimmerman.com> wrote:
> Why all client has to set this limit. If on zookeeper side we have
> configured this then it should use that. No?
>
> My read of the code shows that the clients also use jute.maxBuffer.
> Hopefully, one of the ZK committers can comment on this. But, jute
> (see BinaryInputArchive.java) is used on both the client and the server.
>
> I don't see any methods in CuratorrFramework which takes this variable.
>
> Per the ZK docs:
>
>
> jute.maxbuffer:
>
> (Java system property:* jute.maxbuffer*)
>
> This option can only be set as a Java system property. There is no
> zookeeper prefix on it.
>
>
>
>
> On September 13, 2015 at 4:22:46 PM, Check Peck (comptechgeeky@gmail.com)
> wrote:
>
> Why all client has to set this limit. If on zookeeper side we have
> configured this then it should use that. No?
>
> Also how can we set this parameter through Curator? I don't see any
> methods in CuratorrFramework which takes this variable.
>
> On Sun, Sep 13, 2015 at 2:17 PM, Jordan Zimmerman <
> jordan@jordanzimmerman.com> wrote:
>
>> Yes, all clients. Curator just wraps ZooKeeper so all the same things
>> apply.
>>
>> -Jordan
>>
>>
>>
>> On September 13, 2015 at 4:14:17 PM, Check Peck (comptechgeeky@gmail.com)
>> wrote:
>>
>> I mean this code for getting children not earlier one, that is for data.
>>
>> List<String> children = client.getChildren().forPath(path);
>>
>> On Sun, Sep 13, 2015 at 2:10 PM, Check Peck <co...@gmail.com>
>> wrote:
>>
>>> Ok understood that part and I was correct on the 500 children nodes
>>> znode name example right?
>>>
>>> If I can increase the jute.maxBuffer property, I do need to restart the
>>> whole cluster. Correct? Is there any problem if I increase this?
>>>
>>> On Sun, Sep 13, 2015 at 2:08 PM, Jordan Zimmerman <
>>> jordan@jordanzimmerman.com> wrote:
>>>
>>>> No, the 1MB limit will hit you there as well. If you try to get your
>>>> data (via getData() call) it will fail. Note, you can increase the value
>>>> via the jute.maxBuffer property.
>>>>
>>>> -Jordan
>>>>
>>>> On September 13, 2015 at 4:06:48 PM, Check Peck (
>>>> comptechgeeky@gmail.com) wrote:
>>>>
>>>> What about data in those znode? They can be more than 1MB in each znode
>>>> correct?
>>>>
>>>>
>>>
>>
>
--
Christopher COORD Chen <cc...@twitter.com> @secretasianman
Re: Best Practice while creating znodes?
Posted by Jordan Zimmerman <jo...@jordanzimmerman.com>.
Why all client has to set this limit. If on zookeeper side we have configured this then it should use that. No?
My read of the code shows that the clients also use jute.maxBuffer. Hopefully, one of the ZK committers can comment on this. But, jute (see BinaryInputArchive.java) is used on both the client and the server.
I don't see any methods in CuratorrFramework which takes this variable.
Per the ZK docs:
jute.maxbuffer:
(Java system property: jute.maxbuffer)
This option can only be set as a Java system property. There is no zookeeper prefix on it.
On September 13, 2015 at 4:22:46 PM, Check Peck (comptechgeeky@gmail.com) wrote:
Why all client has to set this limit. If on zookeeper side we have configured this then it should use that. No?
Also how can we set this parameter through Curator? I don't see any methods in CuratorrFramework which takes this variable.
On Sun, Sep 13, 2015 at 2:17 PM, Jordan Zimmerman <jo...@jordanzimmerman.com> wrote:
Yes, all clients. Curator just wraps ZooKeeper so all the same things apply.
-Jordan
On September 13, 2015 at 4:14:17 PM, Check Peck (comptechgeeky@gmail.com) wrote:
I mean this code for getting children not earlier one, that is for data.
List<String> children = client.getChildren().forPath(path);
On Sun, Sep 13, 2015 at 2:10 PM, Check Peck <co...@gmail.com> wrote:
Ok understood that part and I was correct on the 500 children nodes znode name example right?
If I can increase the jute.maxBuffer property, I do need to restart the whole cluster. Correct? Is there any problem if I increase this?
On Sun, Sep 13, 2015 at 2:08 PM, Jordan Zimmerman <jo...@jordanzimmerman.com> wrote:
No, the 1MB limit will hit you there as well. If you try to get your data (via getData() call) it will fail. Note, you can increase the value via the jute.maxBuffer property.
-Jordan
On September 13, 2015 at 4:06:48 PM, Check Peck (comptechgeeky@gmail.com) wrote:
What about data in those znode? They can be more than 1MB in each znode correct?
Re: Best Practice while creating znodes?
Posted by Jordan Zimmerman <jo...@jordanzimmerman.com>.
Why all client has to set this limit. If on zookeeper side we have configured this then it should use that. No?
My read of the code shows that the clients also use jute.maxBuffer. Hopefully, one of the ZK committers can comment on this. But, jute (see BinaryInputArchive.java) is used on both the client and the server.
I don't see any methods in CuratorrFramework which takes this variable.
Per the ZK docs:
jute.maxbuffer:
(Java system property: jute.maxbuffer)
This option can only be set as a Java system property. There is no zookeeper prefix on it.
On September 13, 2015 at 4:22:46 PM, Check Peck (comptechgeeky@gmail.com) wrote:
Why all client has to set this limit. If on zookeeper side we have configured this then it should use that. No?
Also how can we set this parameter through Curator? I don't see any methods in CuratorrFramework which takes this variable.
On Sun, Sep 13, 2015 at 2:17 PM, Jordan Zimmerman <jo...@jordanzimmerman.com> wrote:
Yes, all clients. Curator just wraps ZooKeeper so all the same things apply.
-Jordan
On September 13, 2015 at 4:14:17 PM, Check Peck (comptechgeeky@gmail.com) wrote:
I mean this code for getting children not earlier one, that is for data.
List<String> children = client.getChildren().forPath(path);
On Sun, Sep 13, 2015 at 2:10 PM, Check Peck <co...@gmail.com> wrote:
Ok understood that part and I was correct on the 500 children nodes znode name example right?
If I can increase the jute.maxBuffer property, I do need to restart the whole cluster. Correct? Is there any problem if I increase this?
On Sun, Sep 13, 2015 at 2:08 PM, Jordan Zimmerman <jo...@jordanzimmerman.com> wrote:
No, the 1MB limit will hit you there as well. If you try to get your data (via getData() call) it will fail. Note, you can increase the value via the jute.maxBuffer property.
-Jordan
On September 13, 2015 at 4:06:48 PM, Check Peck (comptechgeeky@gmail.com) wrote:
What about data in those znode? They can be more than 1MB in each znode correct?
Re: Best Practice while creating znodes?
Posted by Check Peck <co...@gmail.com>.
Why all client has to set this limit. If on zookeeper side we have
configured this then it should use that. No?
Also how can we set this parameter through Curator? I don't see any methods
in CuratorrFramework which takes this variable.
On Sun, Sep 13, 2015 at 2:17 PM, Jordan Zimmerman <
jordan@jordanzimmerman.com> wrote:
> Yes, all clients. Curator just wraps ZooKeeper so all the same things
> apply.
>
> -Jordan
>
>
>
> On September 13, 2015 at 4:14:17 PM, Check Peck (comptechgeeky@gmail.com)
> wrote:
>
> I mean this code for getting children not earlier one, that is for data.
>
> List<String> children = client.getChildren().forPath(path);
>
> On Sun, Sep 13, 2015 at 2:10 PM, Check Peck <co...@gmail.com>
> wrote:
>
>> Ok understood that part and I was correct on the 500 children nodes znode
>> name example right?
>>
>> If I can increase the jute.maxBuffer property, I do need to restart the
>> whole cluster. Correct? Is there any problem if I increase this?
>>
>> On Sun, Sep 13, 2015 at 2:08 PM, Jordan Zimmerman <
>> jordan@jordanzimmerman.com> wrote:
>>
>>> No, the 1MB limit will hit you there as well. If you try to get your
>>> data (via getData() call) it will fail. Note, you can increase the value
>>> via the jute.maxBuffer property.
>>>
>>> -Jordan
>>>
>>> On September 13, 2015 at 4:06:48 PM, Check Peck (comptechgeeky@gmail.com)
>>> wrote:
>>>
>>> What about data in those znode? They can be more than 1MB in each znode
>>> correct?
>>>
>>>
>>
>
Re: Best Practice while creating znodes?
Posted by Check Peck <co...@gmail.com>.
Why all client has to set this limit. If on zookeeper side we have
configured this then it should use that. No?
Also how can we set this parameter through Curator? I don't see any methods
in CuratorrFramework which takes this variable.
On Sun, Sep 13, 2015 at 2:17 PM, Jordan Zimmerman <
jordan@jordanzimmerman.com> wrote:
> Yes, all clients. Curator just wraps ZooKeeper so all the same things
> apply.
>
> -Jordan
>
>
>
> On September 13, 2015 at 4:14:17 PM, Check Peck (comptechgeeky@gmail.com)
> wrote:
>
> I mean this code for getting children not earlier one, that is for data.
>
> List<String> children = client.getChildren().forPath(path);
>
> On Sun, Sep 13, 2015 at 2:10 PM, Check Peck <co...@gmail.com>
> wrote:
>
>> Ok understood that part and I was correct on the 500 children nodes znode
>> name example right?
>>
>> If I can increase the jute.maxBuffer property, I do need to restart the
>> whole cluster. Correct? Is there any problem if I increase this?
>>
>> On Sun, Sep 13, 2015 at 2:08 PM, Jordan Zimmerman <
>> jordan@jordanzimmerman.com> wrote:
>>
>>> No, the 1MB limit will hit you there as well. If you try to get your
>>> data (via getData() call) it will fail. Note, you can increase the value
>>> via the jute.maxBuffer property.
>>>
>>> -Jordan
>>>
>>> On September 13, 2015 at 4:06:48 PM, Check Peck (comptechgeeky@gmail.com)
>>> wrote:
>>>
>>> What about data in those znode? They can be more than 1MB in each znode
>>> correct?
>>>
>>>
>>
>
Re: Best Practice while creating znodes?
Posted by Jordan Zimmerman <jo...@jordanzimmerman.com>.
Yes, all clients. Curator just wraps ZooKeeper so all the same things apply.
-Jordan
On September 13, 2015 at 4:14:17 PM, Check Peck (comptechgeeky@gmail.com) wrote:
I mean this code for getting children not earlier one, that is for data.
List<String> children = client.getChildren().forPath(path);
On Sun, Sep 13, 2015 at 2:10 PM, Check Peck <co...@gmail.com> wrote:
Ok understood that part and I was correct on the 500 children nodes znode name example right?
If I can increase the jute.maxBuffer property, I do need to restart the whole cluster. Correct? Is there any problem if I increase this?
On Sun, Sep 13, 2015 at 2:08 PM, Jordan Zimmerman <jo...@jordanzimmerman.com> wrote:
No, the 1MB limit will hit you there as well. If you try to get your data (via getData() call) it will fail. Note, you can increase the value via the jute.maxBuffer property.
-Jordan
On September 13, 2015 at 4:06:48 PM, Check Peck (comptechgeeky@gmail.com) wrote:
What about data in those znode? They can be more than 1MB in each znode correct?
Re: Best Practice while creating znodes?
Posted by Jordan Zimmerman <jo...@jordanzimmerman.com>.
Yes, all clients. Curator just wraps ZooKeeper so all the same things apply.
-Jordan
On September 13, 2015 at 4:14:17 PM, Check Peck (comptechgeeky@gmail.com) wrote:
I mean this code for getting children not earlier one, that is for data.
List<String> children = client.getChildren().forPath(path);
On Sun, Sep 13, 2015 at 2:10 PM, Check Peck <co...@gmail.com> wrote:
Ok understood that part and I was correct on the 500 children nodes znode name example right?
If I can increase the jute.maxBuffer property, I do need to restart the whole cluster. Correct? Is there any problem if I increase this?
On Sun, Sep 13, 2015 at 2:08 PM, Jordan Zimmerman <jo...@jordanzimmerman.com> wrote:
No, the 1MB limit will hit you there as well. If you try to get your data (via getData() call) it will fail. Note, you can increase the value via the jute.maxBuffer property.
-Jordan
On September 13, 2015 at 4:06:48 PM, Check Peck (comptechgeeky@gmail.com) wrote:
What about data in those znode? They can be more than 1MB in each znode correct?
Re: Best Practice while creating znodes?
Posted by Jordan Zimmerman <jo...@jordanzimmerman.com>.
If I can increase the jute.maxBuffer property, I do need to restart the whole cluster. Correct? Is there any problem if I increase this?
Yes. I believe the ZK docs are wrong. The docs say that the jute.maxBuffer only applies to node size. But, in my experience, it applies to every API call. You can see this in BinaryInputArchive.java which is used to read every response by the client.
-JZ
On September 13, 2015 at 4:14:17 PM, Check Peck (comptechgeeky@gmail.com) wrote:
I mean this code for getting children not earlier one, that is for data.
List<String> children = client.getChildren().forPath(path);
On Sun, Sep 13, 2015 at 2:10 PM, Check Peck <co...@gmail.com> wrote:
Ok understood that part and I was correct on the 500 children nodes znode name example right?
If I can increase the jute.maxBuffer property, I do need to restart the whole cluster. Correct? Is there any problem if I increase this?
On Sun, Sep 13, 2015 at 2:08 PM, Jordan Zimmerman <jo...@jordanzimmerman.com> wrote:
No, the 1MB limit will hit you there as well. If you try to get your data (via getData() call) it will fail. Note, you can increase the value via the jute.maxBuffer property.
-Jordan
On September 13, 2015 at 4:06:48 PM, Check Peck (comptechgeeky@gmail.com) wrote:
What about data in those znode? They can be more than 1MB in each znode correct?
Re: Best Practice while creating znodes?
Posted by Jordan Zimmerman <jo...@jordanzimmerman.com>.
If I can increase the jute.maxBuffer property, I do need to restart the whole cluster. Correct? Is there any problem if I increase this?
Yes. I believe the ZK docs are wrong. The docs say that the jute.maxBuffer only applies to node size. But, in my experience, it applies to every API call. You can see this in BinaryInputArchive.java which is used to read every response by the client.
-JZ
On September 13, 2015 at 4:14:17 PM, Check Peck (comptechgeeky@gmail.com) wrote:
I mean this code for getting children not earlier one, that is for data.
List<String> children = client.getChildren().forPath(path);
On Sun, Sep 13, 2015 at 2:10 PM, Check Peck <co...@gmail.com> wrote:
Ok understood that part and I was correct on the 500 children nodes znode name example right?
If I can increase the jute.maxBuffer property, I do need to restart the whole cluster. Correct? Is there any problem if I increase this?
On Sun, Sep 13, 2015 at 2:08 PM, Jordan Zimmerman <jo...@jordanzimmerman.com> wrote:
No, the 1MB limit will hit you there as well. If you try to get your data (via getData() call) it will fail. Note, you can increase the value via the jute.maxBuffer property.
-Jordan
On September 13, 2015 at 4:06:48 PM, Check Peck (comptechgeeky@gmail.com) wrote:
What about data in those znode? They can be more than 1MB in each znode correct?
Re: Best Practice while creating znodes?
Posted by Check Peck <co...@gmail.com>.
I mean this code for getting children not earlier one, that is for data.
List<String> children = client.getChildren().forPath(path);
On Sun, Sep 13, 2015 at 2:10 PM, Check Peck <co...@gmail.com> wrote:
> Ok understood that part and I was correct on the 500 children nodes znode
> name example right?
>
> If I can increase the jute.maxBuffer property, I do need to restart the
> whole cluster. Correct? Is there any problem if I increase this?
>
> On Sun, Sep 13, 2015 at 2:08 PM, Jordan Zimmerman <
> jordan@jordanzimmerman.com> wrote:
>
>> No, the 1MB limit will hit you there as well. If you try to get your data
>> (via getData() call) it will fail. Note, you can increase the value via the
>> jute.maxBuffer property.
>>
>> -Jordan
>>
>> On September 13, 2015 at 4:06:48 PM, Check Peck (comptechgeeky@gmail.com)
>> wrote:
>>
>> What about data in those znode? They can be more than 1MB in each znode
>> correct?
>>
>>
>
Re: Best Practice while creating znodes?
Posted by Check Peck <co...@gmail.com>.
I mean this code for getting children not earlier one, that is for data.
List<String> children = client.getChildren().forPath(path);
On Sun, Sep 13, 2015 at 2:10 PM, Check Peck <co...@gmail.com> wrote:
> Ok understood that part and I was correct on the 500 children nodes znode
> name example right?
>
> If I can increase the jute.maxBuffer property, I do need to restart the
> whole cluster. Correct? Is there any problem if I increase this?
>
> On Sun, Sep 13, 2015 at 2:08 PM, Jordan Zimmerman <
> jordan@jordanzimmerman.com> wrote:
>
>> No, the 1MB limit will hit you there as well. If you try to get your data
>> (via getData() call) it will fail. Note, you can increase the value via the
>> jute.maxBuffer property.
>>
>> -Jordan
>>
>> On September 13, 2015 at 4:06:48 PM, Check Peck (comptechgeeky@gmail.com)
>> wrote:
>>
>> What about data in those znode? They can be more than 1MB in each znode
>> correct?
>>
>>
>
Re: Best Practice while creating znodes?
Posted by Check Peck <co...@gmail.com>.
Ok understood that part and I was correct on the 500 children nodes znode
name example right?
If I can increase the jute.maxBuffer property, I do need to restart the
whole cluster. Correct? Is there any problem if I increase this?
On Sun, Sep 13, 2015 at 2:08 PM, Jordan Zimmerman <
jordan@jordanzimmerman.com> wrote:
> No, the 1MB limit will hit you there as well. If you try to get your data
> (via getData() call) it will fail. Note, you can increase the value via the
> jute.maxBuffer property.
>
> -Jordan
>
> On September 13, 2015 at 4:06:48 PM, Check Peck (comptechgeeky@gmail.com)
> wrote:
>
> What about data in those znode? They can be more than 1MB in each znode
> correct?
>
>
Re: Best Practice while creating znodes?
Posted by Check Peck <co...@gmail.com>.
Ok understood that part and I was correct on the 500 children nodes znode
name example right?
If I can increase the jute.maxBuffer property, I do need to restart the
whole cluster. Correct? Is there any problem if I increase this?
On Sun, Sep 13, 2015 at 2:08 PM, Jordan Zimmerman <
jordan@jordanzimmerman.com> wrote:
> No, the 1MB limit will hit you there as well. If you try to get your data
> (via getData() call) it will fail. Note, you can increase the value via the
> jute.maxBuffer property.
>
> -Jordan
>
> On September 13, 2015 at 4:06:48 PM, Check Peck (comptechgeeky@gmail.com)
> wrote:
>
> What about data in those znode? They can be more than 1MB in each znode
> correct?
>
>
Re: Best Practice while creating znodes?
Posted by Jordan Zimmerman <jo...@jordanzimmerman.com>.
No, the 1MB limit will hit you there as well. If you try to get your data (via getData() call) it will fail. Note, you can increase the value via the jute.maxBuffer property.
-Jordan
On September 13, 2015 at 4:06:48 PM, Check Peck (comptechgeeky@gmail.com) wrote:
What about data in those znode? They can be more than 1MB in each znode correct?
Re: Best Practice while creating znodes?
Posted by Jordan Zimmerman <jo...@jordanzimmerman.com>.
No, the 1MB limit will hit you there as well. If you try to get your data (via getData() call) it will fail. Note, you can increase the value via the jute.maxBuffer property.
-Jordan
On September 13, 2015 at 4:06:48 PM, Check Peck (comptechgeeky@gmail.com) wrote:
What about data in those znode? They can be more than 1MB in each znode correct?
Re: Best Practice while creating znodes?
Posted by Check Peck <co...@gmail.com>.
Ok so that means in the below code if I have 500 children in the list, and
znode names of all those 500 children is greater than 1 MB, then below get
call will fail. Am I correct?
List<String> children = client.getData().forPath("/root/clients/");
What about data in those znode? They can be more than 1MB in each znode
correct?
On Sun, Sep 13, 2015 at 1:57 PM, Jordan Zimmerman <
jordan@jordanzimmerman.com> wrote:
> The node names. Internally, ZooKeeper’s protocol is (by default) limited
> to 1MB per response.
>
> -JZ
>
>
>
> On September 13, 2015 at 3:44:10 PM, Check Peck (comptechgeeky@gmail.com)
> wrote:
>
> Hi Jordan,
>
> I was looking at this email which you sent me long time back. Here you
> mentioned - If I call getChildren(), the entire length of all the children
> can't exceed 1MB. Suppose, I have 500 clients as client_0, client_1 etc.
> And each client has 50-60 znodes with some data in it.
>
> List<String> children = client.getData().forPath("/root/clients/");
>
> So by length here you mean, actual znode name of all clients? Or something
> else?
>
> Actually I am confuse what does length the the entire length of all the
> children means? Does it mean data also within each node or just the
> zookeeper path of those node.
>
>
> On Thu, Nov 7, 2013 at 10:57 AM, Jordan Zimmerman <
> jordan@jordanzimmerman.com> wrote:
>
> > There are very few limitations on naming. You can't use a slash. Have a
> > look at:
> >
> > org.apache.zookeeper.common.PathUtils
> >
> > In terms of length, ZooKeeper has a 1MB limit per API call. So, for
> > example, if you call getChildren(), the entire length of all the
> children
> > can't exceed 1MB (unless you reconfigure ZK).
> >
> > -JZ
> >
> > On Nov 7, 2013, at 10:45 AM, Techy Teck <co...@gmail.com>
> wrote:
> >
> > Can anyone help me on this?
> >
> > Is there any best practices that we need to follow while creating the
> > znodes?
> >
> > Meaning the znode name should be small not very large names or there
> > shouldn't be a dot or underscore within the word?
> >
> > Anything like that?
> >
> > I will be using Apache Curator.
> >
> >
> >
> >
> > On Wed, Nov 6, 2013 at 11:24 AM, Techy Teck <co...@gmail.com>
> > wrote:
> >
> >> Is there any best practices that we need to follow while creating the
> >> znodes?
> >>
> >> Meaning the znode name should be small not very large names or there
> >> shouldn't be a dot or underscore within the word?
> >>
> >> Anything like that?
> >>
> >>
> >
> >
>
>
Re: Best Practice while creating znodes?
Posted by Check Peck <co...@gmail.com>.
Ok so that means in the below code if I have 500 children in the list, and
znode names of all those 500 children is greater than 1 MB, then below get
call will fail. Am I correct?
List<String> children = client.getData().forPath("/root/clients/");
What about data in those znode? They can be more than 1MB in each znode
correct?
On Sun, Sep 13, 2015 at 1:57 PM, Jordan Zimmerman <
jordan@jordanzimmerman.com> wrote:
> The node names. Internally, ZooKeeper’s protocol is (by default) limited
> to 1MB per response.
>
> -JZ
>
>
>
> On September 13, 2015 at 3:44:10 PM, Check Peck (comptechgeeky@gmail.com)
> wrote:
>
> Hi Jordan,
>
> I was looking at this email which you sent me long time back. Here you
> mentioned - If I call getChildren(), the entire length of all the children
> can't exceed 1MB. Suppose, I have 500 clients as client_0, client_1 etc.
> And each client has 50-60 znodes with some data in it.
>
> List<String> children = client.getData().forPath("/root/clients/");
>
> So by length here you mean, actual znode name of all clients? Or something
> else?
>
> Actually I am confuse what does length the the entire length of all the
> children means? Does it mean data also within each node or just the
> zookeeper path of those node.
>
>
> On Thu, Nov 7, 2013 at 10:57 AM, Jordan Zimmerman <
> jordan@jordanzimmerman.com> wrote:
>
> > There are very few limitations on naming. You can't use a slash. Have a
> > look at:
> >
> > org.apache.zookeeper.common.PathUtils
> >
> > In terms of length, ZooKeeper has a 1MB limit per API call. So, for
> > example, if you call getChildren(), the entire length of all the
> children
> > can't exceed 1MB (unless you reconfigure ZK).
> >
> > -JZ
> >
> > On Nov 7, 2013, at 10:45 AM, Techy Teck <co...@gmail.com>
> wrote:
> >
> > Can anyone help me on this?
> >
> > Is there any best practices that we need to follow while creating the
> > znodes?
> >
> > Meaning the znode name should be small not very large names or there
> > shouldn't be a dot or underscore within the word?
> >
> > Anything like that?
> >
> > I will be using Apache Curator.
> >
> >
> >
> >
> > On Wed, Nov 6, 2013 at 11:24 AM, Techy Teck <co...@gmail.com>
> > wrote:
> >
> >> Is there any best practices that we need to follow while creating the
> >> znodes?
> >>
> >> Meaning the znode name should be small not very large names or there
> >> shouldn't be a dot or underscore within the word?
> >>
> >> Anything like that?
> >>
> >>
> >
> >
>
>
Re: Best Practice while creating znodes?
Posted by Jordan Zimmerman <jo...@jordanzimmerman.com>.
The node names. Internally, ZooKeeper’s protocol is (by default) limited to 1MB per response.
-JZ
On September 13, 2015 at 3:44:10 PM, Check Peck (comptechgeeky@gmail.com) wrote:
Hi Jordan,
I was looking at this email which you sent me long time back. Here you
mentioned - If I call getChildren(), the entire length of all the children
can't exceed 1MB. Suppose, I have 500 clients as client_0, client_1 etc.
And each client has 50-60 znodes with some data in it.
List<String> children = client.getData().forPath("/root/clients/");
So by length here you mean, actual znode name of all clients? Or something
else?
Actually I am confuse what does length the the entire length of all the
children means? Does it mean data also within each node or just the
zookeeper path of those node.
On Thu, Nov 7, 2013 at 10:57 AM, Jordan Zimmerman <
jordan@jordanzimmerman.com> wrote:
> There are very few limitations on naming. You can't use a slash. Have a
> look at:
>
> org.apache.zookeeper.common.PathUtils
>
> In terms of length, ZooKeeper has a 1MB limit per API call. So, for
> example, if you call getChildren(), the entire length of all the children
> can't exceed 1MB (unless you reconfigure ZK).
>
> -JZ
>
> On Nov 7, 2013, at 10:45 AM, Techy Teck <co...@gmail.com> wrote:
>
> Can anyone help me on this?
>
> Is there any best practices that we need to follow while creating the
> znodes?
>
> Meaning the znode name should be small not very large names or there
> shouldn't be a dot or underscore within the word?
>
> Anything like that?
>
> I will be using Apache Curator.
>
>
>
>
> On Wed, Nov 6, 2013 at 11:24 AM, Techy Teck <co...@gmail.com>
> wrote:
>
>> Is there any best practices that we need to follow while creating the
>> znodes?
>>
>> Meaning the znode name should be small not very large names or there
>> shouldn't be a dot or underscore within the word?
>>
>> Anything like that?
>>
>>
>
>
Re: Best Practice while creating znodes?
Posted by Jordan Zimmerman <jo...@jordanzimmerman.com>.
The node names. Internally, ZooKeeper’s protocol is (by default) limited to 1MB per response.
-JZ
On September 13, 2015 at 3:44:10 PM, Check Peck (comptechgeeky@gmail.com) wrote:
Hi Jordan,
I was looking at this email which you sent me long time back. Here you
mentioned - If I call getChildren(), the entire length of all the children
can't exceed 1MB. Suppose, I have 500 clients as client_0, client_1 etc.
And each client has 50-60 znodes with some data in it.
List<String> children = client.getData().forPath("/root/clients/");
So by length here you mean, actual znode name of all clients? Or something
else?
Actually I am confuse what does length the the entire length of all the
children means? Does it mean data also within each node or just the
zookeeper path of those node.
On Thu, Nov 7, 2013 at 10:57 AM, Jordan Zimmerman <
jordan@jordanzimmerman.com> wrote:
> There are very few limitations on naming. You can't use a slash. Have a
> look at:
>
> org.apache.zookeeper.common.PathUtils
>
> In terms of length, ZooKeeper has a 1MB limit per API call. So, for
> example, if you call getChildren(), the entire length of all the children
> can't exceed 1MB (unless you reconfigure ZK).
>
> -JZ
>
> On Nov 7, 2013, at 10:45 AM, Techy Teck <co...@gmail.com> wrote:
>
> Can anyone help me on this?
>
> Is there any best practices that we need to follow while creating the
> znodes?
>
> Meaning the znode name should be small not very large names or there
> shouldn't be a dot or underscore within the word?
>
> Anything like that?
>
> I will be using Apache Curator.
>
>
>
>
> On Wed, Nov 6, 2013 at 11:24 AM, Techy Teck <co...@gmail.com>
> wrote:
>
>> Is there any best practices that we need to follow while creating the
>> znodes?
>>
>> Meaning the znode name should be small not very large names or there
>> shouldn't be a dot or underscore within the word?
>>
>> Anything like that?
>>
>>
>
>
Re: Best Practice while creating znodes?
Posted by Check Peck <co...@gmail.com>.
Hi Jordan,
I was looking at this email which you sent me long time back. Here you
mentioned - If I call getChildren(), the entire length of all the children
can't exceed 1MB. Suppose, I have 500 clients as client_0, client_1 etc.
And each client has 50-60 znodes with some data in it.
List<String> children = client.getData().forPath("/root/clients/");
So by length here you mean, actual znode name of all clients? Or something
else?
Actually I am confuse what does length the the entire length of all the
children means? Does it mean data also within each node or just the
zookeeper path of those node.
On Thu, Nov 7, 2013 at 10:57 AM, Jordan Zimmerman <
jordan@jordanzimmerman.com> wrote:
> There are very few limitations on naming. You can't use a slash. Have a
> look at:
>
> org.apache.zookeeper.common.PathUtils
>
> In terms of length, ZooKeeper has a 1MB limit per API call. So, for
> example, if you call getChildren(), the entire length of all the children
> can't exceed 1MB (unless you reconfigure ZK).
>
> -JZ
>
> On Nov 7, 2013, at 10:45 AM, Techy Teck <co...@gmail.com> wrote:
>
> Can anyone help me on this?
>
> Is there any best practices that we need to follow while creating the
> znodes?
>
> Meaning the znode name should be small not very large names or there
> shouldn't be a dot or underscore within the word?
>
> Anything like that?
>
> I will be using Apache Curator.
>
>
>
>
> On Wed, Nov 6, 2013 at 11:24 AM, Techy Teck <co...@gmail.com>
> wrote:
>
>> Is there any best practices that we need to follow while creating the
>> znodes?
>>
>> Meaning the znode name should be small not very large names or there
>> shouldn't be a dot or underscore within the word?
>>
>> Anything like that?
>>
>>
>
>
Re: Best Practice while creating znodes?
Posted by Techy Teck <co...@gmail.com>.
Sure.. Thanks for the help..
Apart from that like, znodes name should be small? Is that the case as
well? Or it doesn't matter if it is long as well?
On Thu, Nov 7, 2013 at 10:57 AM, Jordan Zimmerman <
jordan@jordanzimmerman.com> wrote:
> There are very few limitations on naming. You can't use a slash. Have a
> look at:
>
> org.apache.zookeeper.common.PathUtils
>
> In terms of length, ZooKeeper has a 1MB limit per API call. So, for
> example, if you call getChildren(), the entire length of all the children
> can't exceed 1MB (unless you reconfigure ZK).
>
> -JZ
>
> On Nov 7, 2013, at 10:45 AM, Techy Teck <co...@gmail.com> wrote:
>
> Can anyone help me on this?
>
> Is there any best practices that we need to follow while creating the
> znodes?
>
> Meaning the znode name should be small not very large names or there
> shouldn't be a dot or underscore within the word?
>
> Anything like that?
>
> I will be using Apache Curator.
>
>
>
>
> On Wed, Nov 6, 2013 at 11:24 AM, Techy Teck <co...@gmail.com>wrote:
>
>> Is there any best practices that we need to follow while creating the
>> znodes?
>>
>> Meaning the znode name should be small not very large names or there
>> shouldn't be a dot or underscore within the word?
>>
>> Anything like that?
>>
>>
>
>
Re: Best Practice while creating znodes?
Posted by Check Peck <co...@gmail.com>.
Hi Jordan,
I was looking at this email which you sent me long time back. Here you
mentioned - If I call getChildren(), the entire length of all the children
can't exceed 1MB. Suppose, I have 500 clients as client_0, client_1 etc.
And each client has 50-60 znodes with some data in it.
List<String> children = client.getData().forPath("/root/clients/");
So by length here you mean, actual znode name of all clients? Or something
else?
Actually I am confuse what does length the the entire length of all the
children means? Does it mean data also within each node or just the
zookeeper path of those node.
On Thu, Nov 7, 2013 at 10:57 AM, Jordan Zimmerman <
jordan@jordanzimmerman.com> wrote:
> There are very few limitations on naming. You can't use a slash. Have a
> look at:
>
> org.apache.zookeeper.common.PathUtils
>
> In terms of length, ZooKeeper has a 1MB limit per API call. So, for
> example, if you call getChildren(), the entire length of all the children
> can't exceed 1MB (unless you reconfigure ZK).
>
> -JZ
>
> On Nov 7, 2013, at 10:45 AM, Techy Teck <co...@gmail.com> wrote:
>
> Can anyone help me on this?
>
> Is there any best practices that we need to follow while creating the
> znodes?
>
> Meaning the znode name should be small not very large names or there
> shouldn't be a dot or underscore within the word?
>
> Anything like that?
>
> I will be using Apache Curator.
>
>
>
>
> On Wed, Nov 6, 2013 at 11:24 AM, Techy Teck <co...@gmail.com>
> wrote:
>
>> Is there any best practices that we need to follow while creating the
>> znodes?
>>
>> Meaning the znode name should be small not very large names or there
>> shouldn't be a dot or underscore within the word?
>>
>> Anything like that?
>>
>>
>
>
Re: Best Practice while creating znodes?
Posted by Jordan Zimmerman <jo...@jordanzimmerman.com>.
There are very few limitations on naming. You can't use a slash. Have a look at:
org.apache.zookeeper.common.PathUtils
In terms of length, ZooKeeper has a 1MB limit per API call. So, for example, if you call getChildren(), the entire length of all the children can't exceed 1MB (unless you reconfigure ZK).
-JZ
On Nov 7, 2013, at 10:45 AM, Techy Teck <co...@gmail.com> wrote:
> Can anyone help me on this?
>
> Is there any best practices that we need to follow while creating the znodes?
>
> Meaning the znode name should be small not very large names or there shouldn't be a dot or underscore within the word?
>
> Anything like that?
>
> I will be using Apache Curator.
>
>
>
>
> On Wed, Nov 6, 2013 at 11:24 AM, Techy Teck <co...@gmail.com> wrote:
> Is there any best practices that we need to follow while creating the znodes?
>
> Meaning the znode name should be small not very large names or there shouldn't be a dot or underscore within the word?
>
> Anything like that?
>
>
Re: Best Practice while creating znodes?
Posted by Jordan Zimmerman <jo...@jordanzimmerman.com>.
There are very few limitations on naming. You can't use a slash. Have a look at:
org.apache.zookeeper.common.PathUtils
In terms of length, ZooKeeper has a 1MB limit per API call. So, for example, if you call getChildren(), the entire length of all the children can't exceed 1MB (unless you reconfigure ZK).
-JZ
On Nov 7, 2013, at 10:45 AM, Techy Teck <co...@gmail.com> wrote:
> Can anyone help me on this?
>
> Is there any best practices that we need to follow while creating the znodes?
>
> Meaning the znode name should be small not very large names or there shouldn't be a dot or underscore within the word?
>
> Anything like that?
>
> I will be using Apache Curator.
>
>
>
>
> On Wed, Nov 6, 2013 at 11:24 AM, Techy Teck <co...@gmail.com> wrote:
> Is there any best practices that we need to follow while creating the znodes?
>
> Meaning the znode name should be small not very large names or there shouldn't be a dot or underscore within the word?
>
> Anything like that?
>
>