You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@zookeeper.apache.org by Stefan Zier <st...@sumologic.com> on 2011/12/07 17:07:06 UTC
3.3.2 -> 3.4.0 NodeExistsException when reading snapshot
Just upgraded one of our ensembles and am now hitting this exception. Any ideas how to recover?
2011-12-07 07:24:21,983 [myid:1] - WARN [main:DataTree@909] - Failed: Create transaction for /seq/y77mmizznsmdggqzx493-0030010980
org.apache.zookeeper.KeeperException$NodeExistsException: KeeperErrorCode = NodeExists
at org.apache.zookeeper.server.DataTree.createNode(DataTree.java:483)
at org.apache.zookeeper.server.DataTree.processTxn(DataTree.java:799)
at org.apache.zookeeper.server.persistence.FileTxnSnapLog.processTransaction(FileTxnSnapLog.java:197)
at org.apache.zookeeper.server.persistence.FileTxnSnapLog.restore(FileTxnSnapLog.java:150)
at org.apache.zookeeper.server.ZKDatabase.loadDataBase(ZKDatabase.java:223)
at org.apache.zookeeper.server.quorum.QuorumPeer.loadDataBase(QuorumPeer.java:418)
at org.apache.zookeeper.server.quorum.QuorumPeer.start(QuorumPeer.java:410)
at org.apache.zookeeper.server.quorum.QuorumPeerMain.runFromConfig(QuorumPeerMain.java:151)
at org.apache.zookeeper.server.quorum.QuorumPeerMain.initializeAndRun(QuorumPeerMain.java:111)
at org.apache.zookeeper.server.quorum.QuorumPeerMain.main(QuorumPeerMain.java:78)
Stefan Zier
Re: 3.3.2 -> 3.4.0 NodeExistsException when reading snapshot
Posted by Patrick Hunt <ph...@apache.org>.
We had an existing unit test for this, however the unit test itself
was broken and failed to find this. That test is fixed as part of
1319. Additionally I've added new unit tests (from a different
direction) to look for this as part of 1319.
If you want to add to the systest framework that would be a good idea as well.
Patrick
On Thu, Dec 8, 2011 at 9:38 AM, Neha Narkhede <ne...@gmail.com> wrote:
> Do we have some system tests like these in the zookeeper repository ? If
> not, maybe it is a good idea to add them ?
>
> Thanks,
> Neha
>
>
> On Thu, Dec 8, 2011 at 8:17 AM, Stefan Zier <st...@sumologic.com> wrote:
>>
>> Thanks for the responses, guys. In light of the subsequent announcement,
>> we reverted back to 3.3.4 for the time being. Just another data point: When
>> I did try zkCli.sh, it wouldn't even connect to the server. It looked like
>> this exception prevented it from starting up.
>>
>> Stefan Zier
>>
>>
>>
>> On Dec 7, 2011, at 10:27 AM, Patrick Hunt wrote:
>>
>> > small correction: zkCli.sh -server host:port
>> >
>> > On Wed, Dec 7, 2011 at 10:19 AM, Mahadev Konar <ma...@hortonworks.com>
>> > wrote:
>> >> Stefan,
>> >> It mostly looks like
>> >> https://issues.apache.org/jira/browse/ZOOKEEPER-1319, but just to
>> >> confirm can you do this -
>> >>
>> >> do a get on the same znode "/seq/y77mmizznsmdggqzx493-0030010980" on
>> >> each of the servers. You can just use the zkCli.sh to do that:
>> >>
>> >> zkClish.sh -server hostname1 port1
>> >> get /seq/y77mmizznsmdggqzx493-0030010980
>> >>
>> >> similar for other hosts and post the results?
>> >>
>> >> thanks
>> >> mahadev
>> >>
>> >> On Wed, Dec 7, 2011 at 8:07 AM, Stefan Zier <st...@sumologic.com>
>> >> wrote:
>> >>> Just upgraded one of our ensembles and am now hitting this exception.
>> >>> Any ideas how to recover?
>> >>>
>> >>> 2011-12-07 07:24:21,983 [myid:1] - WARN [main:DataTree@909] - Failed:
>> >>> Create transaction for /seq/y77mmizznsmdggqzx493-0030010980
>> >>> org.apache.zookeeper.KeeperException$NodeExistsException:
>> >>> KeeperErrorCode = NodeExists
>> >>> at
>> >>> org.apache.zookeeper.server.DataTree.createNode(DataTree.java:483)
>> >>> at
>> >>> org.apache.zookeeper.server.DataTree.processTxn(DataTree.java:799)
>> >>> at
>> >>> org.apache.zookeeper.server.persistence.FileTxnSnapLog.processTransaction(FileTxnSnapLog.java:197)
>> >>> at
>> >>> org.apache.zookeeper.server.persistence.FileTxnSnapLog.restore(FileTxnSnapLog.java:150)
>> >>> at
>> >>> org.apache.zookeeper.server.ZKDatabase.loadDataBase(ZKDatabase.java:223)
>> >>> at
>> >>> org.apache.zookeeper.server.quorum.QuorumPeer.loadDataBase(QuorumPeer.java:418)
>> >>> at
>> >>> org.apache.zookeeper.server.quorum.QuorumPeer.start(QuorumPeer.java:410)
>> >>> at
>> >>> org.apache.zookeeper.server.quorum.QuorumPeerMain.runFromConfig(QuorumPeerMain.java:151)
>> >>> at
>> >>> org.apache.zookeeper.server.quorum.QuorumPeerMain.initializeAndRun(QuorumPeerMain.java:111)
>> >>> at
>> >>> org.apache.zookeeper.server.quorum.QuorumPeerMain.main(QuorumPeerMain.java:78)
>> >>>
>> >>> Stefan Zier
>> >>>
>> >>>
>> >>>
>>
>
Re: 3.3.2 -> 3.4.0 NodeExistsException when reading snapshot
Posted by Neha Narkhede <ne...@gmail.com>.
Do we have some system tests like these in the zookeeper repository ? If
not, maybe it is a good idea to add them ?
Thanks,
Neha
On Thu, Dec 8, 2011 at 8:17 AM, Stefan Zier <st...@sumologic.com> wrote:
> Thanks for the responses, guys. In light of the subsequent announcement,
> we reverted back to 3.3.4 for the time being. Just another data point: When
> I did try zkCli.sh, it wouldn't even connect to the server. It looked like
> this exception prevented it from starting up.
>
> Stefan Zier
>
>
>
> On Dec 7, 2011, at 10:27 AM, Patrick Hunt wrote:
>
> > small correction: zkCli.sh -server host:port
> >
> > On Wed, Dec 7, 2011 at 10:19 AM, Mahadev Konar <ma...@hortonworks.com>
> wrote:
> >> Stefan,
> >> It mostly looks like
> >> https://issues.apache.org/jira/browse/ZOOKEEPER-1319, but just to
> >> confirm can you do this -
> >>
> >> do a get on the same znode "/seq/y77mmizznsmdggqzx493-0030010980" on
> >> each of the servers. You can just use the zkCli.sh to do that:
> >>
> >> zkClish.sh -server hostname1 port1
> >> get /seq/y77mmizznsmdggqzx493-0030010980
> >>
> >> similar for other hosts and post the results?
> >>
> >> thanks
> >> mahadev
> >>
> >> On Wed, Dec 7, 2011 at 8:07 AM, Stefan Zier <st...@sumologic.com>
> wrote:
> >>> Just upgraded one of our ensembles and am now hitting this exception.
> Any ideas how to recover?
> >>>
> >>> 2011-12-07 07:24:21,983 [myid:1] - WARN [main:DataTree@909] -
> Failed: Create transaction for /seq/y77mmizznsmdggqzx493-0030010980
> >>> org.apache.zookeeper.KeeperException$NodeExistsException:
> KeeperErrorCode = NodeExists
> >>> at
> org.apache.zookeeper.server.DataTree.createNode(DataTree.java:483)
> >>> at
> org.apache.zookeeper.server.DataTree.processTxn(DataTree.java:799)
> >>> at
> org.apache.zookeeper.server.persistence.FileTxnSnapLog.processTransaction(FileTxnSnapLog.java:197)
> >>> at
> org.apache.zookeeper.server.persistence.FileTxnSnapLog.restore(FileTxnSnapLog.java:150)
> >>> at
> org.apache.zookeeper.server.ZKDatabase.loadDataBase(ZKDatabase.java:223)
> >>> at
> org.apache.zookeeper.server.quorum.QuorumPeer.loadDataBase(QuorumPeer.java:418)
> >>> at
> org.apache.zookeeper.server.quorum.QuorumPeer.start(QuorumPeer.java:410)
> >>> at
> org.apache.zookeeper.server.quorum.QuorumPeerMain.runFromConfig(QuorumPeerMain.java:151)
> >>> at
> org.apache.zookeeper.server.quorum.QuorumPeerMain.initializeAndRun(QuorumPeerMain.java:111)
> >>> at
> org.apache.zookeeper.server.quorum.QuorumPeerMain.main(QuorumPeerMain.java:78)
> >>>
> >>> Stefan Zier
> >>>
> >>>
> >>>
>
>
Re: 3.3.2 -> 3.4.0 NodeExistsException when reading snapshot
Posted by Stefan Zier <st...@sumologic.com>.
Thanks for the responses, guys. In light of the subsequent announcement, we reverted back to 3.3.4 for the time being. Just another data point: When I did try zkCli.sh, it wouldn't even connect to the server. It looked like this exception prevented it from starting up.
Stefan Zier
On Dec 7, 2011, at 10:27 AM, Patrick Hunt wrote:
> small correction: zkCli.sh -server host:port
>
> On Wed, Dec 7, 2011 at 10:19 AM, Mahadev Konar <ma...@hortonworks.com> wrote:
>> Stefan,
>> It mostly looks like
>> https://issues.apache.org/jira/browse/ZOOKEEPER-1319, but just to
>> confirm can you do this -
>>
>> do a get on the same znode "/seq/y77mmizznsmdggqzx493-0030010980" on
>> each of the servers. You can just use the zkCli.sh to do that:
>>
>> zkClish.sh -server hostname1 port1
>> get /seq/y77mmizznsmdggqzx493-0030010980
>>
>> similar for other hosts and post the results?
>>
>> thanks
>> mahadev
>>
>> On Wed, Dec 7, 2011 at 8:07 AM, Stefan Zier <st...@sumologic.com> wrote:
>>> Just upgraded one of our ensembles and am now hitting this exception. Any ideas how to recover?
>>>
>>> 2011-12-07 07:24:21,983 [myid:1] - WARN [main:DataTree@909] - Failed: Create transaction for /seq/y77mmizznsmdggqzx493-0030010980
>>> org.apache.zookeeper.KeeperException$NodeExistsException: KeeperErrorCode = NodeExists
>>> at org.apache.zookeeper.server.DataTree.createNode(DataTree.java:483)
>>> at org.apache.zookeeper.server.DataTree.processTxn(DataTree.java:799)
>>> at org.apache.zookeeper.server.persistence.FileTxnSnapLog.processTransaction(FileTxnSnapLog.java:197)
>>> at org.apache.zookeeper.server.persistence.FileTxnSnapLog.restore(FileTxnSnapLog.java:150)
>>> at org.apache.zookeeper.server.ZKDatabase.loadDataBase(ZKDatabase.java:223)
>>> at org.apache.zookeeper.server.quorum.QuorumPeer.loadDataBase(QuorumPeer.java:418)
>>> at org.apache.zookeeper.server.quorum.QuorumPeer.start(QuorumPeer.java:410)
>>> at org.apache.zookeeper.server.quorum.QuorumPeerMain.runFromConfig(QuorumPeerMain.java:151)
>>> at org.apache.zookeeper.server.quorum.QuorumPeerMain.initializeAndRun(QuorumPeerMain.java:111)
>>> at org.apache.zookeeper.server.quorum.QuorumPeerMain.main(QuorumPeerMain.java:78)
>>>
>>> Stefan Zier
>>>
>>>
>>>
Re: 3.3.2 -> 3.4.0 NodeExistsException when reading snapshot
Posted by Patrick Hunt <ph...@apache.org>.
small correction: zkCli.sh -server host:port
On Wed, Dec 7, 2011 at 10:19 AM, Mahadev Konar <ma...@hortonworks.com> wrote:
> Stefan,
> It mostly looks like
> https://issues.apache.org/jira/browse/ZOOKEEPER-1319, but just to
> confirm can you do this -
>
> do a get on the same znode "/seq/y77mmizznsmdggqzx493-0030010980" on
> each of the servers. You can just use the zkCli.sh to do that:
>
> zkClish.sh -server hostname1 port1
> get /seq/y77mmizznsmdggqzx493-0030010980
>
> similar for other hosts and post the results?
>
> thanks
> mahadev
>
> On Wed, Dec 7, 2011 at 8:07 AM, Stefan Zier <st...@sumologic.com> wrote:
>> Just upgraded one of our ensembles and am now hitting this exception. Any ideas how to recover?
>>
>> 2011-12-07 07:24:21,983 [myid:1] - WARN [main:DataTree@909] - Failed: Create transaction for /seq/y77mmizznsmdggqzx493-0030010980
>> org.apache.zookeeper.KeeperException$NodeExistsException: KeeperErrorCode = NodeExists
>> at org.apache.zookeeper.server.DataTree.createNode(DataTree.java:483)
>> at org.apache.zookeeper.server.DataTree.processTxn(DataTree.java:799)
>> at org.apache.zookeeper.server.persistence.FileTxnSnapLog.processTransaction(FileTxnSnapLog.java:197)
>> at org.apache.zookeeper.server.persistence.FileTxnSnapLog.restore(FileTxnSnapLog.java:150)
>> at org.apache.zookeeper.server.ZKDatabase.loadDataBase(ZKDatabase.java:223)
>> at org.apache.zookeeper.server.quorum.QuorumPeer.loadDataBase(QuorumPeer.java:418)
>> at org.apache.zookeeper.server.quorum.QuorumPeer.start(QuorumPeer.java:410)
>> at org.apache.zookeeper.server.quorum.QuorumPeerMain.runFromConfig(QuorumPeerMain.java:151)
>> at org.apache.zookeeper.server.quorum.QuorumPeerMain.initializeAndRun(QuorumPeerMain.java:111)
>> at org.apache.zookeeper.server.quorum.QuorumPeerMain.main(QuorumPeerMain.java:78)
>>
>> Stefan Zier
>>
>>
>>
Re: 3.3.2 -> 3.4.0 NodeExistsException when reading snapshot
Posted by Mahadev Konar <ma...@hortonworks.com>.
Stefan,
It mostly looks like
https://issues.apache.org/jira/browse/ZOOKEEPER-1319, but just to
confirm can you do this -
do a get on the same znode "/seq/y77mmizznsmdggqzx493-0030010980" on
each of the servers. You can just use the zkCli.sh to do that:
zkClish.sh -server hostname1 port1
get /seq/y77mmizznsmdggqzx493-0030010980
similar for other hosts and post the results?
thanks
mahadev
On Wed, Dec 7, 2011 at 8:07 AM, Stefan Zier <st...@sumologic.com> wrote:
> Just upgraded one of our ensembles and am now hitting this exception. Any ideas how to recover?
>
> 2011-12-07 07:24:21,983 [myid:1] - WARN [main:DataTree@909] - Failed: Create transaction for /seq/y77mmizznsmdggqzx493-0030010980
> org.apache.zookeeper.KeeperException$NodeExistsException: KeeperErrorCode = NodeExists
> at org.apache.zookeeper.server.DataTree.createNode(DataTree.java:483)
> at org.apache.zookeeper.server.DataTree.processTxn(DataTree.java:799)
> at org.apache.zookeeper.server.persistence.FileTxnSnapLog.processTransaction(FileTxnSnapLog.java:197)
> at org.apache.zookeeper.server.persistence.FileTxnSnapLog.restore(FileTxnSnapLog.java:150)
> at org.apache.zookeeper.server.ZKDatabase.loadDataBase(ZKDatabase.java:223)
> at org.apache.zookeeper.server.quorum.QuorumPeer.loadDataBase(QuorumPeer.java:418)
> at org.apache.zookeeper.server.quorum.QuorumPeer.start(QuorumPeer.java:410)
> at org.apache.zookeeper.server.quorum.QuorumPeerMain.runFromConfig(QuorumPeerMain.java:151)
> at org.apache.zookeeper.server.quorum.QuorumPeerMain.initializeAndRun(QuorumPeerMain.java:111)
> at org.apache.zookeeper.server.quorum.QuorumPeerMain.main(QuorumPeerMain.java:78)
>
> Stefan Zier
>
>
>