You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@zookeeper.apache.org by Yuhan Zhang <yz...@onescreen.com> on 2012/05/01 05:24:53 UTC
zooKeeper connectionloss on a folder
Hi all,
I'm using zooKeeper as a task queue; however, it appears that the folder
(zookeeper data node) that I used to store the tasks
(zooKeeper data nodes) are not accessible anymore - whenever I run an "ls"
through zkCli.sh, the app will quit with a connectionLoss.
The other folders are ok. and viewable using "ls".
Is the file corrupted? There are a lot of objects in the files. Is there a
limit for the number of object can be stored in a folder (node)?
Thank you.
Yuhan Zhang
Here are the error messages:
[zk: localhost:2181(CONNECTED) 1] ls /com.onescreen.viewer
2012-05-01 03:21:39,533 - WARN
[main-SendThread(localhost:2181):ClientCnxn$SendThread@1161] - Session
0x1370667c2210003 for server localhost/127.0.0.1:2181, unexpected error,
closing socket connection and attempting reconnect
java.io.IOException: Packet len4315700 is out of range!
at
org.apache.zookeeper.ClientCnxn$SendThread.readLength(ClientCnxn.java:710)
at org.apache.zookeeper.ClientCnxn$SendThread.doIO(ClientCnxn.java:869)
at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1130)
WATCHER::
WatchedEvent state:Disconnected type:None path:null
Exception in thread "main"
org.apache.zookeeper.KeeperException$ConnectionLossException:
KeeperErrorCode = ConnectionLoss for /com.onescreen.viewer
at org.apache.zookeeper.KeeperException.create(KeeperException.java:90)
at org.apache.zookeeper.KeeperException.create(KeeperException.java:42)
at org.apache.zookeeper.ZooKeeper.getChildren(ZooKeeper.java:1243)
at org.apache.zookeeper.ZooKeeper.getChildren(ZooKeeper.java:1271)
at
org.apache.zookeeper.ZooKeeperMain.processZKCmd(ZooKeeperMain.java:700)
at org.apache.zookeeper.ZooKeeperMain.processCmd(ZooKeeperMain.java:581)
at
org.apache.zookeeper.ZooKeeperMain.executeLine(ZooKeeperMain.java:353)
at org.apache.zookeeper.ZooKeeperMain.run(ZooKeeperMain.java:311)
at org.apache.zookeeper.ZooKeeperMain.main(ZooKeeperMain.java:270)
Re: zooKeeper connectionloss on a folder
Posted by Yuhan Zhang <yz...@onescreen.com>.
Hi Ted,
Thanks for the quick response. could you direct me to the part of the
zookeeper doc
for configuring this limit?
The Curator DistributedQueue watches for changes in the childNodes of a
node.
I'm depending on this to place items onto the queue, and make other
machines to
be notified and start processing. It would be a lot of work to modify the
queue to
watch on multiple dymanically-created nodes.. will think about this
approach later.
Thank you.
Yuhan
On Tue, May 1, 2012 at 11:22 AM, Ted Dunning <te...@gmail.com> wrote:
> There is a size limit for responses from the server. That is likely the
> problem you are seeing. This can be configured larger, but that isn't
> really a solution.
>
> It would be better to address this at the architectural level. Is there
> some way that you can go to a multi-level directory structure?
>
> On Tue, May 1, 2012 at 10:17 AM, Yuhan Zhang <yz...@onescreen.com> wrote:
>
> > More than 100000 objects under a node... I restarted last night with an
> > empty folder,
> > and the new folder reaches the same state. It kept happening..
> >
> > I'm using the curator library to add objects.
> > (com.netflix.curator.framework.recipes.queue.DistributedQueue) But
> feeling
> > it is a
> > problem of ZooKeeper.
> >
> > Thank you.
> >
> > Yuhan
> >
> > On Mon, Apr 30, 2012 at 10:50 PM, Ted Dunning <te...@gmail.com>
> > wrote:
> >
> > > How many objects are in that directory?
> > >
> > > Sent from my iPhone
> > >
> > > On Apr 30, 2012, at 8:24 PM, Yuhan Zhang <yz...@onescreen.com> wrote:
> > >
> > > > Hi all,
> > > >
> > > > I'm using zooKeeper as a task queue; however, it appears that the
> > folder
> > > > (zookeeper data node) that I used to store the tasks
> > > > (zooKeeper data nodes) are not accessible anymore - whenever I run an
> > > "ls"
> > > > through zkCli.sh, the app will quit with a connectionLoss.
> > > >
> > > > The other folders are ok. and viewable using "ls".
> > > >
> > > > Is the file corrupted? There are a lot of objects in the files. Is
> > there
> > > a
> > > > limit for the number of object can be stored in a folder (node)?
> > > >
> > > >
> > > > Thank you.
> > > >
> > > > Yuhan Zhang
> > > >
> > > > Here are the error messages:
> > > > [zk: localhost:2181(CONNECTED) 1] ls /com.onescreen.viewer
> > > > 2012-05-01 03:21:39,533 - WARN
> > > > [main-SendThread(localhost:2181):ClientCnxn$SendThread@1161] -
> Session
> > > > 0x1370667c2210003 for server localhost/127.0.0.1:2181, unexpected
> > error,
> > > > closing socket connection and attempting reconnect
> > > > java.io.IOException: Packet len4315700 is out of range!
> > > > at
> > > >
> > >
> >
> org.apache.zookeeper.ClientCnxn$SendThread.readLength(ClientCnxn.java:710)
> > > > at
> > > org.apache.zookeeper.ClientCnxn$SendThread.doIO(ClientCnxn.java:869)
> > > > at
> > > org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1130)
> > > >
> > > > WATCHER::
> > > >
> > > > WatchedEvent state:Disconnected type:None path:null
> > > > Exception in thread "main"
> > > > org.apache.zookeeper.KeeperException$ConnectionLossException:
> > > > KeeperErrorCode = ConnectionLoss for /com.onescreen.viewer
> > > > at
> > > org.apache.zookeeper.KeeperException.create(KeeperException.java:90)
> > > > at
> > > org.apache.zookeeper.KeeperException.create(KeeperException.java:42)
> > > > at org.apache.zookeeper.ZooKeeper.getChildren(ZooKeeper.java:1243)
> > > > at org.apache.zookeeper.ZooKeeper.getChildren(ZooKeeper.java:1271)
> > > > at
> > > >
> org.apache.zookeeper.ZooKeeperMain.processZKCmd(ZooKeeperMain.java:700)
> > > > at
> > > org.apache.zookeeper.ZooKeeperMain.processCmd(ZooKeeperMain.java:581)
> > > > at
> > > >
> org.apache.zookeeper.ZooKeeperMain.executeLine(ZooKeeperMain.java:353)
> > > > at org.apache.zookeeper.ZooKeeperMain.run(ZooKeeperMain.java:311)
> > > > at org.apache.zookeeper.ZooKeeperMain.main(ZooKeeperMain.java:270)
> > >
> >
> >
> >
> > --
> > Yuhan Zhang
> > Application Developer
> > OneScreen Inc.
> > yzhang@onescreen.com <eh...@onescreen.com>
> > www.onescreen.com
> >
> > The information contained in this e-mail is for the exclusive use of the
> > intended recipient(s) and may be confidential, proprietary, and/or
> legally
> > privileged. Inadvertent disclosure of this message does not constitute a
> > waiver of any privilege. If you receive this message in error, please do
> > not directly or indirectly print, copy, retransmit, disseminate, or
> > otherwise use the information. In addition, please delete this e-mail and
> > all copies and notify the sender.
> >
>
--
Yuhan Zhang
Application Developer
OneScreen Inc.
yzhang@onescreen.com <eh...@onescreen.com>
www.onescreen.com
The information contained in this e-mail is for the exclusive use of the
intended recipient(s) and may be confidential, proprietary, and/or legally
privileged. Inadvertent disclosure of this message does not constitute a
waiver of any privilege. If you receive this message in error, please do
not directly or indirectly print, copy, retransmit, disseminate, or
otherwise use the information. In addition, please delete this e-mail and
all copies and notify the sender.
Re: zooKeeper connectionloss on a folder
Posted by Ted Dunning <te...@gmail.com>.
There is a size limit for responses from the server. That is likely the
problem you are seeing. This can be configured larger, but that isn't
really a solution.
It would be better to address this at the architectural level. Is there
some way that you can go to a multi-level directory structure?
On Tue, May 1, 2012 at 10:17 AM, Yuhan Zhang <yz...@onescreen.com> wrote:
> More than 100000 objects under a node... I restarted last night with an
> empty folder,
> and the new folder reaches the same state. It kept happening..
>
> I'm using the curator library to add objects.
> (com.netflix.curator.framework.recipes.queue.DistributedQueue) But feeling
> it is a
> problem of ZooKeeper.
>
> Thank you.
>
> Yuhan
>
> On Mon, Apr 30, 2012 at 10:50 PM, Ted Dunning <te...@gmail.com>
> wrote:
>
> > How many objects are in that directory?
> >
> > Sent from my iPhone
> >
> > On Apr 30, 2012, at 8:24 PM, Yuhan Zhang <yz...@onescreen.com> wrote:
> >
> > > Hi all,
> > >
> > > I'm using zooKeeper as a task queue; however, it appears that the
> folder
> > > (zookeeper data node) that I used to store the tasks
> > > (zooKeeper data nodes) are not accessible anymore - whenever I run an
> > "ls"
> > > through zkCli.sh, the app will quit with a connectionLoss.
> > >
> > > The other folders are ok. and viewable using "ls".
> > >
> > > Is the file corrupted? There are a lot of objects in the files. Is
> there
> > a
> > > limit for the number of object can be stored in a folder (node)?
> > >
> > >
> > > Thank you.
> > >
> > > Yuhan Zhang
> > >
> > > Here are the error messages:
> > > [zk: localhost:2181(CONNECTED) 1] ls /com.onescreen.viewer
> > > 2012-05-01 03:21:39,533 - WARN
> > > [main-SendThread(localhost:2181):ClientCnxn$SendThread@1161] - Session
> > > 0x1370667c2210003 for server localhost/127.0.0.1:2181, unexpected
> error,
> > > closing socket connection and attempting reconnect
> > > java.io.IOException: Packet len4315700 is out of range!
> > > at
> > >
> >
> org.apache.zookeeper.ClientCnxn$SendThread.readLength(ClientCnxn.java:710)
> > > at
> > org.apache.zookeeper.ClientCnxn$SendThread.doIO(ClientCnxn.java:869)
> > > at
> > org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1130)
> > >
> > > WATCHER::
> > >
> > > WatchedEvent state:Disconnected type:None path:null
> > > Exception in thread "main"
> > > org.apache.zookeeper.KeeperException$ConnectionLossException:
> > > KeeperErrorCode = ConnectionLoss for /com.onescreen.viewer
> > > at
> > org.apache.zookeeper.KeeperException.create(KeeperException.java:90)
> > > at
> > org.apache.zookeeper.KeeperException.create(KeeperException.java:42)
> > > at org.apache.zookeeper.ZooKeeper.getChildren(ZooKeeper.java:1243)
> > > at org.apache.zookeeper.ZooKeeper.getChildren(ZooKeeper.java:1271)
> > > at
> > > org.apache.zookeeper.ZooKeeperMain.processZKCmd(ZooKeeperMain.java:700)
> > > at
> > org.apache.zookeeper.ZooKeeperMain.processCmd(ZooKeeperMain.java:581)
> > > at
> > > org.apache.zookeeper.ZooKeeperMain.executeLine(ZooKeeperMain.java:353)
> > > at org.apache.zookeeper.ZooKeeperMain.run(ZooKeeperMain.java:311)
> > > at org.apache.zookeeper.ZooKeeperMain.main(ZooKeeperMain.java:270)
> >
>
>
>
> --
> Yuhan Zhang
> Application Developer
> OneScreen Inc.
> yzhang@onescreen.com <eh...@onescreen.com>
> www.onescreen.com
>
> The information contained in this e-mail is for the exclusive use of the
> intended recipient(s) and may be confidential, proprietary, and/or legally
> privileged. Inadvertent disclosure of this message does not constitute a
> waiver of any privilege. If you receive this message in error, please do
> not directly or indirectly print, copy, retransmit, disseminate, or
> otherwise use the information. In addition, please delete this e-mail and
> all copies and notify the sender.
>
Re: zooKeeper connectionloss on a folder
Posted by Yuhan Zhang <yz...@onescreen.com>.
More than 100000 objects under a node... I restarted last night with an
empty folder,
and the new folder reaches the same state. It kept happening..
I'm using the curator library to add objects.
(com.netflix.curator.framework.recipes.queue.DistributedQueue) But feeling
it is a
problem of ZooKeeper.
Thank you.
Yuhan
On Mon, Apr 30, 2012 at 10:50 PM, Ted Dunning <te...@gmail.com> wrote:
> How many objects are in that directory?
>
> Sent from my iPhone
>
> On Apr 30, 2012, at 8:24 PM, Yuhan Zhang <yz...@onescreen.com> wrote:
>
> > Hi all,
> >
> > I'm using zooKeeper as a task queue; however, it appears that the folder
> > (zookeeper data node) that I used to store the tasks
> > (zooKeeper data nodes) are not accessible anymore - whenever I run an
> "ls"
> > through zkCli.sh, the app will quit with a connectionLoss.
> >
> > The other folders are ok. and viewable using "ls".
> >
> > Is the file corrupted? There are a lot of objects in the files. Is there
> a
> > limit for the number of object can be stored in a folder (node)?
> >
> >
> > Thank you.
> >
> > Yuhan Zhang
> >
> > Here are the error messages:
> > [zk: localhost:2181(CONNECTED) 1] ls /com.onescreen.viewer
> > 2012-05-01 03:21:39,533 - WARN
> > [main-SendThread(localhost:2181):ClientCnxn$SendThread@1161] - Session
> > 0x1370667c2210003 for server localhost/127.0.0.1:2181, unexpected error,
> > closing socket connection and attempting reconnect
> > java.io.IOException: Packet len4315700 is out of range!
> > at
> >
> org.apache.zookeeper.ClientCnxn$SendThread.readLength(ClientCnxn.java:710)
> > at
> org.apache.zookeeper.ClientCnxn$SendThread.doIO(ClientCnxn.java:869)
> > at
> org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1130)
> >
> > WATCHER::
> >
> > WatchedEvent state:Disconnected type:None path:null
> > Exception in thread "main"
> > org.apache.zookeeper.KeeperException$ConnectionLossException:
> > KeeperErrorCode = ConnectionLoss for /com.onescreen.viewer
> > at
> org.apache.zookeeper.KeeperException.create(KeeperException.java:90)
> > at
> org.apache.zookeeper.KeeperException.create(KeeperException.java:42)
> > at org.apache.zookeeper.ZooKeeper.getChildren(ZooKeeper.java:1243)
> > at org.apache.zookeeper.ZooKeeper.getChildren(ZooKeeper.java:1271)
> > at
> > org.apache.zookeeper.ZooKeeperMain.processZKCmd(ZooKeeperMain.java:700)
> > at
> org.apache.zookeeper.ZooKeeperMain.processCmd(ZooKeeperMain.java:581)
> > at
> > org.apache.zookeeper.ZooKeeperMain.executeLine(ZooKeeperMain.java:353)
> > at org.apache.zookeeper.ZooKeeperMain.run(ZooKeeperMain.java:311)
> > at org.apache.zookeeper.ZooKeeperMain.main(ZooKeeperMain.java:270)
>
--
Yuhan Zhang
Application Developer
OneScreen Inc.
yzhang@onescreen.com <eh...@onescreen.com>
www.onescreen.com
The information contained in this e-mail is for the exclusive use of the
intended recipient(s) and may be confidential, proprietary, and/or legally
privileged. Inadvertent disclosure of this message does not constitute a
waiver of any privilege. If you receive this message in error, please do
not directly or indirectly print, copy, retransmit, disseminate, or
otherwise use the information. In addition, please delete this e-mail and
all copies and notify the sender.
Re: zooKeeper connectionloss on a folder
Posted by Ted Dunning <te...@gmail.com>.
How many objects are in that directory?
Sent from my iPhone
On Apr 30, 2012, at 8:24 PM, Yuhan Zhang <yz...@onescreen.com> wrote:
> Hi all,
>
> I'm using zooKeeper as a task queue; however, it appears that the folder
> (zookeeper data node) that I used to store the tasks
> (zooKeeper data nodes) are not accessible anymore - whenever I run an "ls"
> through zkCli.sh, the app will quit with a connectionLoss.
>
> The other folders are ok. and viewable using "ls".
>
> Is the file corrupted? There are a lot of objects in the files. Is there a
> limit for the number of object can be stored in a folder (node)?
>
>
> Thank you.
>
> Yuhan Zhang
>
> Here are the error messages:
> [zk: localhost:2181(CONNECTED) 1] ls /com.onescreen.viewer
> 2012-05-01 03:21:39,533 - WARN
> [main-SendThread(localhost:2181):ClientCnxn$SendThread@1161] - Session
> 0x1370667c2210003 for server localhost/127.0.0.1:2181, unexpected error,
> closing socket connection and attempting reconnect
> java.io.IOException: Packet len4315700 is out of range!
> at
> org.apache.zookeeper.ClientCnxn$SendThread.readLength(ClientCnxn.java:710)
> at org.apache.zookeeper.ClientCnxn$SendThread.doIO(ClientCnxn.java:869)
> at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1130)
>
> WATCHER::
>
> WatchedEvent state:Disconnected type:None path:null
> Exception in thread "main"
> org.apache.zookeeper.KeeperException$ConnectionLossException:
> KeeperErrorCode = ConnectionLoss for /com.onescreen.viewer
> at org.apache.zookeeper.KeeperException.create(KeeperException.java:90)
> at org.apache.zookeeper.KeeperException.create(KeeperException.java:42)
> at org.apache.zookeeper.ZooKeeper.getChildren(ZooKeeper.java:1243)
> at org.apache.zookeeper.ZooKeeper.getChildren(ZooKeeper.java:1271)
> at
> org.apache.zookeeper.ZooKeeperMain.processZKCmd(ZooKeeperMain.java:700)
> at org.apache.zookeeper.ZooKeeperMain.processCmd(ZooKeeperMain.java:581)
> at
> org.apache.zookeeper.ZooKeeperMain.executeLine(ZooKeeperMain.java:353)
> at org.apache.zookeeper.ZooKeeperMain.run(ZooKeeperMain.java:311)
> at org.apache.zookeeper.ZooKeeperMain.main(ZooKeeperMain.java:270)