You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@zookeeper.apache.org by Edward Capriolo <ed...@huffingtonpost.com> on 2015/10/15 20:12:00 UTC
Documentation error in maxumum size of node
We are running zookeeper-3.4.5
byte[] bytes = new byte[1048576];
zookeeper.create("/test_max", bytes);
-> connection loss exception
byte[] bytes = new byte[1048576-100];
zookeeper.create("/test_max", bytes);
-> works
According to the documentation
http://zookeeper.apache.org/doc/r3.4.5/api/org/apache/zookeeper/ZooKeeper.html
The maximum allowable size of the data array is 1 MB (1,048,576 bytes).
Arrays larger than this will cause a KeeperExecption to be thrown.
Re: Documentation error in maxumum size of node
Posted by Edward Capriolo <ed...@huffingtonpost.com>.
It is not a blocker for us. We were simply trying to determine what the
exact limit was. Sorry to hyjack my own thread but another random question:
Session establishment is asynchronous. This constructor will initiate
connection to the server and return immediately - potentially (usually)
before the session is fully established. The watcher argument specifies the
watcher that will be notified of any changes in state. This notification
can come at any point before or after the constructor call has returned.
When I am using Zookeeper in a stateless way (like a one shot database
client) I put a watcher with a count down latch to know I am connected. I
do that here:
https://issues.apache.org/jira/secure/attachment/12655068/ZkSpaceMan.java
If you read this statement, it implies that the KeeperState.SyncConnected could
come before the constructor is complete, IF the constructor is not complete
will the watcher get that event?
Basically how does a Zookeeper know it is connected if the
KeeperState.SyncConnected
may come before one can watch for it?
On Thu, Oct 15, 2015 at 2:32 PM, Chris Nauroth <cn...@hortonworks.com>
wrote:
> Hello Edward,
>
> I suspect what you're seeing is caused by the fact that the maximum size
> enforcement happens at the client-server packet communication level, not
> on inspection of specific parts of the payload, such as the data buffer of
> the znode. Other parts of the payload, such as the path, would be a
> contributing factor too.
>
> If you're in a bind, then a potential workaround is to tune the
> configuration of jute.maxbuffer as described in the administrator guide.
> However, this usually comes with a disclaimer that ZooKeeper is better
> suited to smaller data storage, so you might want to consider if your
> application can be changed to do something different.
>
> http://zookeeper.apache.org/doc/r3.4.6/zookeeperAdmin.html
>
>
> The behavior of jute.maxbuffer is a common source of confusion. As you
> pointed out, the Javadoc makes it sound like the limit is enforced solely
> on the znode data contents. ZOOKEEPER-1295 is an open issue that tracks
> improving this documentation.
>
> https://issues.apache.org/jira/browse/ZOOKEEPER-1295
>
>
> I hope this helps.
>
> --Chris Nauroth
>
>
>
>
> On 10/15/15, 11:12 AM, "Edward Capriolo"
> <ed...@huffingtonpost.com> wrote:
>
> >We are running zookeeper-3.4.5
> >
> >byte[] bytes = new byte[1048576];
> > zookeeper.create("/test_max", bytes);
> >
> >-> connection loss exception
> >
> >byte[] bytes = new byte[1048576-100];
> > zookeeper.create("/test_max", bytes);
> >-> works
> >According to the documentation
> >http://zookeeper.apache.org/doc/r3.4.5/api/org/apache/zookeeper/ZooKeeper
> .
> >html
> >The maximum allowable size of the data array is 1 MB (1,048,576 bytes).
> >Arrays larger than this will cause a KeeperExecption to be thrown.
>
>
Re: Documentation error in maxumum size of node
Posted by Chris Nauroth <cn...@hortonworks.com>.
Hello Edward,
I suspect what you're seeing is caused by the fact that the maximum size
enforcement happens at the client-server packet communication level, not
on inspection of specific parts of the payload, such as the data buffer of
the znode. Other parts of the payload, such as the path, would be a
contributing factor too.
If you're in a bind, then a potential workaround is to tune the
configuration of jute.maxbuffer as described in the administrator guide.
However, this usually comes with a disclaimer that ZooKeeper is better
suited to smaller data storage, so you might want to consider if your
application can be changed to do something different.
http://zookeeper.apache.org/doc/r3.4.6/zookeeperAdmin.html
The behavior of jute.maxbuffer is a common source of confusion. As you
pointed out, the Javadoc makes it sound like the limit is enforced solely
on the znode data contents. ZOOKEEPER-1295 is an open issue that tracks
improving this documentation.
https://issues.apache.org/jira/browse/ZOOKEEPER-1295
I hope this helps.
--Chris Nauroth
On 10/15/15, 11:12 AM, "Edward Capriolo"
<ed...@huffingtonpost.com> wrote:
>We are running zookeeper-3.4.5
>
>byte[] bytes = new byte[1048576];
> zookeeper.create("/test_max", bytes);
>
>-> connection loss exception
>
>byte[] bytes = new byte[1048576-100];
> zookeeper.create("/test_max", bytes);
>-> works
>According to the documentation
>http://zookeeper.apache.org/doc/r3.4.5/api/org/apache/zookeeper/ZooKeeper.
>html
>The maximum allowable size of the data array is 1 MB (1,048,576 bytes).
>Arrays larger than this will cause a KeeperExecption to be thrown.