You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@zookeeper.apache.org by Jun Rao <ju...@gmail.com> on 2011/02/11 19:01:24 UTC
ephemeral node not deleted after client long gone
Hi,
I found an issue in zookeeper 3.3.0 where an ephemeral node didn't get
deleted after the client was long gone. This seems to be a rare event and
happens 1 out of 600 tries. Has there been a similar problem reported/fixed?
Thanks,
Jun
Re: ephemeral node not deleted after client long gone
Posted by Jun Rao <ju...@gmail.com>.
I saw the following entries on the session. It seems that the client that
created the ephemeral node has already been closed. Also, I was using ZK
3.2.1 server and ZK 3.3.0 client. Any issues like that related to ZK 3.2.1?
Thanks,
2011-02-11 04:39:07,350 - INFO
[NIOServerCxn.Factory:12913:NIOServerCnxn@615] - Creating new session
0x22cab09a7380273
2011-02-11 04:39:25,803 - INFO [CommitProcessor:0:NIOServerCnxn@833] -
closing session:0x22cab09a7380273 NIOServerCnxn:
java.nio.channels.SocketChannel[connected local=/172.17.135.39:12913remote=/
172.16.78.170:45385]
Jun
On Fri, Feb 11, 2011 at 1:32 PM, Fournier, Camille F. [Tech] <
Camille.Fournier@gs.com> wrote:
> There should be a log line somewhere associating that ephemeralOwner
> sessionID to a login, something like:
> 2010-12-07 02:04:26,824 - INFO [CommitProcessor:0:NIOServerCnxn@1580] -
> Established session 0x2cbe924f570000 with negotiated timeout 30000 for
> client /10.150.27.112:53673
>
> -----Original Message-----
> From: Jun Rao [mailto:junrao@gmail.com]
> Sent: Friday, February 11, 2011 2:58 PM
> To: user@zookeeper.apache.org
> Subject: Re: ephemeral node not deleted after client long gone
>
> Hmm, I am pretty sure the client that created the ephemeral node is gone.
> That client typically creates a bunch of ephemeral nodes. It seems that all
> nodes except one is gone. The hanging ephemeral node can be read from any
> ZK
> server and its info is listed below. Is there a way to get the information
> of the client who created an ephemeral node (host, processid, etc)?
>
> ctime = Fri Feb 11 04:39:25 PST 2011
> mZxid = 0x1f03f5ea35
> mtime = Fri Feb 11 04:39:25 PST 2011
> pZxid = 0x1f03f5ea35
> cversion = 0
> dataVersion = 0
> aclVersion = 0
> ephemeralOwner = 0x22cab09a7380273
> dataLength = 40
> numChildren = 0
>
> Thanks,
>
> Jun
>
> On Fri, Feb 11, 2011 at 10:54 AM, Mahadev Konar <ma...@apache.org>
> wrote:
>
> > Jun Rao,
> > No it cannot happen without a zookeeper restart.
> >
> > Are you sure you are shutting down the client?
> >
> > thanks
> > mahadev
> >
> > On Fri, Feb 11, 2011 at 10:43 AM, Jun Rao <ju...@gmail.com> wrote:
> > > Can the problems fixed in
> > > https://issues.apache.org/jira/browse/ZOOKEEPER-962 and
> > > https://issues.apache.org/<
> > https://issues.apache.org/jira/browse/ZOOKEEPER-919>
> > > jira/browse/ZOOKEEPER-919<
> > https://issues.apache.org/jira/browse/ZOOKEEPER-919>
> > > happen
> > > even when there is no restart in the ZK server ensemble? For the
> problem
> > > that I have seen, the ZK servers have always been up.
> > >
> > > Thanks,
> > >
> > > Jun
> > >
> > > On Fri, Feb 11, 2011 at 10:08 AM, Mahadev Konar <ma...@apache.org>
> > wrote:
> > >
> > >> Hi Jun,
> > >> Yes there was a bug reported:
> > >>
> > >> https://issues.apache.org/jira/browse/ZOOKEEPER-919
> > >>
> > >> Is this what you are seeing?
> > >>
> > >> thanks
> > >> mahadev
> > >>
> > >> On Fri, Feb 11, 2011 at 10:01 AM, Jun Rao <ju...@gmail.com> wrote:
> > >> > Hi,
> > >> >
> > >> > I found an issue in zookeeper 3.3.0 where an ephemeral node didn't
> get
> > >> > deleted after the client was long gone. This seems to be a rare
> event
> > and
> > >> > happens 1 out of 600 tries. Has there been a similar problem
> > >> reported/fixed?
> > >> > Thanks,
> > >> >
> > >> > Jun
> > >> >
> > >>
> > >
> >
>
RE: ephemeral node not deleted after client long gone
Posted by "Fournier, Camille F. [Tech]" <Ca...@gs.com>.
There should be a log line somewhere associating that ephemeralOwner sessionID to a login, something like:
2010-12-07 02:04:26,824 - INFO [CommitProcessor:0:NIOServerCnxn@1580] - Established session 0x2cbe924f570000 with negotiated timeout 30000 for client /10.150.27.112:53673
-----Original Message-----
From: Jun Rao [mailto:junrao@gmail.com]
Sent: Friday, February 11, 2011 2:58 PM
To: user@zookeeper.apache.org
Subject: Re: ephemeral node not deleted after client long gone
Hmm, I am pretty sure the client that created the ephemeral node is gone.
That client typically creates a bunch of ephemeral nodes. It seems that all
nodes except one is gone. The hanging ephemeral node can be read from any ZK
server and its info is listed below. Is there a way to get the information
of the client who created an ephemeral node (host, processid, etc)?
ctime = Fri Feb 11 04:39:25 PST 2011
mZxid = 0x1f03f5ea35
mtime = Fri Feb 11 04:39:25 PST 2011
pZxid = 0x1f03f5ea35
cversion = 0
dataVersion = 0
aclVersion = 0
ephemeralOwner = 0x22cab09a7380273
dataLength = 40
numChildren = 0
Thanks,
Jun
On Fri, Feb 11, 2011 at 10:54 AM, Mahadev Konar <ma...@apache.org> wrote:
> Jun Rao,
> No it cannot happen without a zookeeper restart.
>
> Are you sure you are shutting down the client?
>
> thanks
> mahadev
>
> On Fri, Feb 11, 2011 at 10:43 AM, Jun Rao <ju...@gmail.com> wrote:
> > Can the problems fixed in
> > https://issues.apache.org/jira/browse/ZOOKEEPER-962 and
> > https://issues.apache.org/<
> https://issues.apache.org/jira/browse/ZOOKEEPER-919>
> > jira/browse/ZOOKEEPER-919<
> https://issues.apache.org/jira/browse/ZOOKEEPER-919>
> > happen
> > even when there is no restart in the ZK server ensemble? For the problem
> > that I have seen, the ZK servers have always been up.
> >
> > Thanks,
> >
> > Jun
> >
> > On Fri, Feb 11, 2011 at 10:08 AM, Mahadev Konar <ma...@apache.org>
> wrote:
> >
> >> Hi Jun,
> >> Yes there was a bug reported:
> >>
> >> https://issues.apache.org/jira/browse/ZOOKEEPER-919
> >>
> >> Is this what you are seeing?
> >>
> >> thanks
> >> mahadev
> >>
> >> On Fri, Feb 11, 2011 at 10:01 AM, Jun Rao <ju...@gmail.com> wrote:
> >> > Hi,
> >> >
> >> > I found an issue in zookeeper 3.3.0 where an ephemeral node didn't get
> >> > deleted after the client was long gone. This seems to be a rare event
> and
> >> > happens 1 out of 600 tries. Has there been a similar problem
> >> reported/fixed?
> >> > Thanks,
> >> >
> >> > Jun
> >> >
> >>
> >
>
Re: ephemeral node not deleted after client long gone
Posted by Jun Rao <ju...@gmail.com>.
Hmm, I am pretty sure the client that created the ephemeral node is gone.
That client typically creates a bunch of ephemeral nodes. It seems that all
nodes except one is gone. The hanging ephemeral node can be read from any ZK
server and its info is listed below. Is there a way to get the information
of the client who created an ephemeral node (host, processid, etc)?
ctime = Fri Feb 11 04:39:25 PST 2011
mZxid = 0x1f03f5ea35
mtime = Fri Feb 11 04:39:25 PST 2011
pZxid = 0x1f03f5ea35
cversion = 0
dataVersion = 0
aclVersion = 0
ephemeralOwner = 0x22cab09a7380273
dataLength = 40
numChildren = 0
Thanks,
Jun
On Fri, Feb 11, 2011 at 10:54 AM, Mahadev Konar <ma...@apache.org> wrote:
> Jun Rao,
> No it cannot happen without a zookeeper restart.
>
> Are you sure you are shutting down the client?
>
> thanks
> mahadev
>
> On Fri, Feb 11, 2011 at 10:43 AM, Jun Rao <ju...@gmail.com> wrote:
> > Can the problems fixed in
> > https://issues.apache.org/jira/browse/ZOOKEEPER-962 and
> > https://issues.apache.org/<
> https://issues.apache.org/jira/browse/ZOOKEEPER-919>
> > jira/browse/ZOOKEEPER-919<
> https://issues.apache.org/jira/browse/ZOOKEEPER-919>
> > happen
> > even when there is no restart in the ZK server ensemble? For the problem
> > that I have seen, the ZK servers have always been up.
> >
> > Thanks,
> >
> > Jun
> >
> > On Fri, Feb 11, 2011 at 10:08 AM, Mahadev Konar <ma...@apache.org>
> wrote:
> >
> >> Hi Jun,
> >> Yes there was a bug reported:
> >>
> >> https://issues.apache.org/jira/browse/ZOOKEEPER-919
> >>
> >> Is this what you are seeing?
> >>
> >> thanks
> >> mahadev
> >>
> >> On Fri, Feb 11, 2011 at 10:01 AM, Jun Rao <ju...@gmail.com> wrote:
> >> > Hi,
> >> >
> >> > I found an issue in zookeeper 3.3.0 where an ephemeral node didn't get
> >> > deleted after the client was long gone. This seems to be a rare event
> and
> >> > happens 1 out of 600 tries. Has there been a similar problem
> >> reported/fixed?
> >> > Thanks,
> >> >
> >> > Jun
> >> >
> >>
> >
>
Re: ephemeral node not deleted after client long gone
Posted by Mahadev Konar <ma...@apache.org>.
Jun Rao,
No it cannot happen without a zookeeper restart.
Are you sure you are shutting down the client?
thanks
mahadev
On Fri, Feb 11, 2011 at 10:43 AM, Jun Rao <ju...@gmail.com> wrote:
> Can the problems fixed in
> https://issues.apache.org/jira/browse/ZOOKEEPER-962 and
> https://issues.apache.org/<https://issues.apache.org/jira/browse/ZOOKEEPER-919>
> jira/browse/ZOOKEEPER-919<https://issues.apache.org/jira/browse/ZOOKEEPER-919>
> happen
> even when there is no restart in the ZK server ensemble? For the problem
> that I have seen, the ZK servers have always been up.
>
> Thanks,
>
> Jun
>
> On Fri, Feb 11, 2011 at 10:08 AM, Mahadev Konar <ma...@apache.org> wrote:
>
>> Hi Jun,
>> Yes there was a bug reported:
>>
>> https://issues.apache.org/jira/browse/ZOOKEEPER-919
>>
>> Is this what you are seeing?
>>
>> thanks
>> mahadev
>>
>> On Fri, Feb 11, 2011 at 10:01 AM, Jun Rao <ju...@gmail.com> wrote:
>> > Hi,
>> >
>> > I found an issue in zookeeper 3.3.0 where an ephemeral node didn't get
>> > deleted after the client was long gone. This seems to be a rare event and
>> > happens 1 out of 600 tries. Has there been a similar problem
>> reported/fixed?
>> > Thanks,
>> >
>> > Jun
>> >
>>
>
Re: ephemeral node not deleted after client long gone
Posted by Jun Rao <ju...@gmail.com>.
Can the problems fixed in
https://issues.apache.org/jira/browse/ZOOKEEPER-962 and
https://issues.apache.org/<https://issues.apache.org/jira/browse/ZOOKEEPER-919>
jira/browse/ZOOKEEPER-919<https://issues.apache.org/jira/browse/ZOOKEEPER-919>
happen
even when there is no restart in the ZK server ensemble? For the problem
that I have seen, the ZK servers have always been up.
Thanks,
Jun
On Fri, Feb 11, 2011 at 10:08 AM, Mahadev Konar <ma...@apache.org> wrote:
> Hi Jun,
> Yes there was a bug reported:
>
> https://issues.apache.org/jira/browse/ZOOKEEPER-919
>
> Is this what you are seeing?
>
> thanks
> mahadev
>
> On Fri, Feb 11, 2011 at 10:01 AM, Jun Rao <ju...@gmail.com> wrote:
> > Hi,
> >
> > I found an issue in zookeeper 3.3.0 where an ephemeral node didn't get
> > deleted after the client was long gone. This seems to be a rare event and
> > happens 1 out of 600 tries. Has there been a similar problem
> reported/fixed?
> > Thanks,
> >
> > Jun
> >
>
Re: ephemeral node not deleted after client long gone
Posted by Mahadev Konar <ma...@apache.org>.
Hi Jun,
Yes there was a bug reported:
https://issues.apache.org/jira/browse/ZOOKEEPER-919
Is this what you are seeing?
thanks
mahadev
On Fri, Feb 11, 2011 at 10:01 AM, Jun Rao <ju...@gmail.com> wrote:
> Hi,
>
> I found an issue in zookeeper 3.3.0 where an ephemeral node didn't get
> deleted after the client was long gone. This seems to be a rare event and
> happens 1 out of 600 tries. Has there been a similar problem reported/fixed?
> Thanks,
>
> Jun
>