You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@zookeeper.apache.org by Johannes Zillmann <jz...@101tec.com> on 2008/09/12 11:49:06 UTC
ephemerals handling after restart
Hi all,
i have a question regarding ephemerals and it behavior on client crash/
restart.
We've a master/node cluster similar to a hadoop hdfs cluster but using
zk for management.
The nodes creates an ephemeral to announce there existence to the
master.
Now what i recognized is that after stopping the whole cluster and
starting only the master again, still some ephemeral nodes may exists
for some seconds.
That leads me to following questions.
What is if a node starts up again. Do it have to clean up it "old"
ephemeral node, or can it somehow acquire the old one ?
Just trying to find a best way how to deal with this, since on regular
restart of the cluster i often recognize something like this
master : node1, node2 connected
master : node2 disconnected
master : node2 connected again
Thanks for any help
Johannes
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
101tec GmbH
Halle (Saale), Saxony-Anhalt, Germany
http://www.101tec.com
Re: ephemerals handling after restart
Posted by Johannes Zillmann <jz...@101tec.com>.
Hi Ben,
thanks for your answer!
Is the session recoverable in case the zk server was restarted in
meantime ?
Johannes
On Sep 12, 2008, at 3:52 PM, Benjamin Reed wrote:
> If a application does not close the ZooKeeper session before shutting
> down, ZooKeeper will not cleanup the session until it times out. So
> when
> an application crashes and restarts, ZooKeeper doesn't know if the
> client is a restart of an old client or a new client.
>
> There is a way to alleviate this problem: you can actually maintain a
> session across client application restarts. If you save off the
> session
> id and password, when you restart you can try to reconnect to the
> session using the ZooKeeper constructor that takes the old session id
> and password. If the reconnect is successful you can then close the
> session and get everything to clean up immediately. (Or you could keep
> using the recovered session if you want to.)
>
> ben
>
> -----Original Message-----
> From: Johannes Zillmann [mailto:jz@101tec.com]
> Sent: Friday, September 12, 2008 2:49 AM
> To: zookeeper-user@hadoop.apache.org
> Subject: ephemerals handling after restart
>
> Hi all,
>
> i have a question regarding ephemerals and it behavior on client
> crash/
> restart.
> We've a master/node cluster similar to a hadoop hdfs cluster but using
> zk for management.
> The nodes creates an ephemeral to announce there existence to the
> master.
>
> Now what i recognized is that after stopping the whole cluster and
> starting only the master again, still some ephemeral nodes may exists
> for some seconds.
> That leads me to following questions.
> What is if a node starts up again. Do it have to clean up it "old"
> ephemeral node, or can it somehow acquire the old one ?
>
> Just trying to find a best way how to deal with this, since on regular
> restart of the cluster i often recognize something like this
> master : node1, node2 connected
> master : node2 disconnected
> master : node2 connected again
>
> Thanks for any help
> Johannes
>
> ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
> 101tec GmbH
> Halle (Saale), Saxony-Anhalt, Germany
> http://www.101tec.com
>
>
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
101tec GmbH
Halle (Saale), Saxony-Anhalt, Germany
http://www.101tec.com
RE: ephemerals handling after restart
Posted by Benjamin Reed <br...@yahoo-inc.com>.
If a application does not close the ZooKeeper session before shutting
down, ZooKeeper will not cleanup the session until it times out. So when
an application crashes and restarts, ZooKeeper doesn't know if the
client is a restart of an old client or a new client.
There is a way to alleviate this problem: you can actually maintain a
session across client application restarts. If you save off the session
id and password, when you restart you can try to reconnect to the
session using the ZooKeeper constructor that takes the old session id
and password. If the reconnect is successful you can then close the
session and get everything to clean up immediately. (Or you could keep
using the recovered session if you want to.)
ben
-----Original Message-----
From: Johannes Zillmann [mailto:jz@101tec.com]
Sent: Friday, September 12, 2008 2:49 AM
To: zookeeper-user@hadoop.apache.org
Subject: ephemerals handling after restart
Hi all,
i have a question regarding ephemerals and it behavior on client crash/
restart.
We've a master/node cluster similar to a hadoop hdfs cluster but using
zk for management.
The nodes creates an ephemeral to announce there existence to the
master.
Now what i recognized is that after stopping the whole cluster and
starting only the master again, still some ephemeral nodes may exists
for some seconds.
That leads me to following questions.
What is if a node starts up again. Do it have to clean up it "old"
ephemeral node, or can it somehow acquire the old one ?
Just trying to find a best way how to deal with this, since on regular
restart of the cluster i often recognize something like this
master : node1, node2 connected
master : node2 disconnected
master : node2 connected again
Thanks for any help
Johannes
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
101tec GmbH
Halle (Saale), Saxony-Anhalt, Germany
http://www.101tec.com