You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@zookeeper.apache.org by siddhartha banik <si...@gmail.com> on 2010/10/27 15:10:27 UTC
Problem with Zookeeper cluster configuration
Hi,
I am trying to configure zookeeper cluster ... with 2 server instances.
zookeeper version : 3.2.2
Config files are :
*Server 1. zoo.cfg*
tickTime=2000
initLimit=10
syncLimit=5
dataDir=/home/xuser/zookeeper1/zookeeper-3.2.2/data/
clientPort=5181
server.1=3.7.192.142:5181:5888
server.2=3.7.192.145:5181:5888
*Server 2. zoo.cfg*
tickTime=2000
initLimit=10
syncLimit=5
dataDir=/home/xuser/zookeeper2/zookeeper-3.2.2/data/
clientPort=5181
server.1=3.7.192.142:5181:5888
server.2=3.7.192.145:5181:5888
I have also created "*myid*" files in respective data folders. Below are the
exception I am seeing :
*Server 1*
2010-10-27 07:43:43,411 - INFO [QuorumPeer:/0.0.0.0:5181:QuorumPeer@514] -
LOOKING
2010-10-27 07:43:43,418 - INFO [QuorumPeer:/0.0.0.0:5181
:FastLeaderElection@579] - New election: -1
2010-10-27 07:43:43,419 - INFO [QuorumPeer:/0.0.0.0:5181
:FastLeaderElection@618] - Notification: 1, -1, 382, 1, LOOKING, LOOKING, 1
2010-10-27 07:43:43,420 - INFO [QuorumPeer:/0.0.0.0:5181
:FastLeaderElection@642] - Adding vote
2010-10-27 07:43:43,436 - INFO [QuorumPeer:/0.0.0.0:5181
:FastLeaderElection@618] - Notification: 2, 0, 383, 1, LOOKING, LOOKING, 2
2010-10-27 07:43:43,442 - INFO [QuorumPeer:/0.0.0.0:5181
:FastLeaderElection@642] - Adding vote
2010-10-27 07:43:43,443 - INFO [QuorumPeer:/0.0.0.0:5181
:FastLeaderElection@618] - Notification: 2, 0, 383, 1, LOOKING, LOOKING, 1
2010-10-27 07:43:43,443 - INFO [QuorumPeer:/0.0.0.0:5181
:FastLeaderElection@642] - Adding vote
2010-10-27 07:43:43,444 - INFO [QuorumPeer:/0.0.0.0:5181:QuorumPeer@523] -
FOLLOWING
2010-10-27 07:43:43,445 - INFO [QuorumPeer:/0.0.0.0:5181
:ZooKeeperServer@160] - Created server
2010-10-27 07:43:43,447 - INFO [QuorumPeer:/0.0.0.0:5181:Follower@147] -
Following /3.7.192.145:5181
2010-10-27 07:43:43,461 - INFO [WorkerReceiver
Thread:FastLeaderElection$Messenger$WorkerReceiver@254] - Sending new
notification.
2010-10-27 07:43:43,462 - WARN [QuorumPeer:/0.0.0.0:5181:Follower@318] -
Exception when following the leader
java.io.EOFException
at java.io.DataInputStream.readInt(DataInputStream.java:375)
at
org.apache.jute.BinaryInputArchive.readInt(BinaryInputArchive.java:63)
at
org.apache.zookeeper.server.quorum.QuorumPacket.deserialize(QuorumPacket.java:66)
at
org.apache.jute.BinaryInputArchive.readRecord(BinaryInputArchive.java:108)
at
org.apache.zookeeper.server.quorum.Follower.readPacket(Follower.java:114)
at
org.apache.zookeeper.server.quorum.Follower.followLeader(Follower.java:193)
at
org.apache.zookeeper.server.quorum.QuorumPeer.run(QuorumPeer.java:525)
2010-10-27 07:43:43,464 - INFO [QuorumPeer:/0.0.0.0:5181:Follower@436] -
shutdown called
java.lang.Exception: shutdown Follower
at
org.apache.zookeeper.server.quorum.Follower.shutdown(Follower.java:436)
at
org.apache.zookeeper.server.quorum.QuorumPeer.run(QuorumPeer.java:529)
*Server 2
*010-10-27 07:59:22,387 - INFO [QuorumPeer:/0.0.0.0:5181:QuorumPeer@535] -
LEADING
2010-10-27 07:59:22,388 - INFO [QuorumPeer:/0.0.0.0:5181
:ZooKeeperServer@160] - Created server
2010-10-27 07:59:22,390 - ERROR [QuorumPeer:/0.0.0.0:5181:Leader@127] -
Couldn't bind to port 5181
java.net.BindException: Address already in use
at java.net.PlainSocketImpl.socketBind(Native Method)
at java.net.PlainSocketImpl.bind(PlainSocketImpl.java:359)
at java.net.ServerSocket.bind(ServerSocket.java:319)
at java.net.ServerSocket.<init>(ServerSocket.java:185)
at java.net.ServerSocket.<init>(ServerSocket.java:97)
at org.apache.zookeeper.server.quorum.Leader.<init>(Leader.java:125)
at
org.apache.zookeeper.server.quorum.QuorumPeer.makeLeader(QuorumPeer.java:417)
at
org.apache.zookeeper.server.quorum.QuorumPeer.run(QuorumPeer.java:537)
2010-10-27 07:59:22,392 - WARN [QuorumPeer:/0.0.0.0:5181:QuorumPeer@541] -
Unexpected exception
java.net.BindException: Address already in use
at java.net.PlainSocketImpl.socketBind(Native Method)
at java.net.PlainSocketImpl.bind(PlainSocketImpl.java:359)
at java.net.ServerSocket.bind(ServerSocket.java:319)
at java.net.ServerSocket.<init>(ServerSocket.java:185)
at java.net.ServerSocket.<init>(ServerSocket.java:97)
at org.apache.zookeeper.server.quorum.Leader.<init>(Leader.java:125)
at
org.apache.zookeeper.server.quorum.QuorumPeer.makeLeader(QuorumPeer.java:417)
at
org.apache.zookeeper.server.quorum.QuorumPeer.run(QuorumPeer.java:537)
2010-10-27 07:59:22,393 - INFO [WorkerReceiver
Thread:FastLeaderElection$Messenger$WorkerReceiver@254] - Sending new
notification.
2010-10-27 07:59:22,394 - INFO [QuorumPeer:/0.0.0.0:5181:QuorumPeer@514] -
LOOKING*
*
I have tried with netstat command also. No other process is using *5181 *port
other then zookeeper process.
Other thing I have tried is: using separate ports for server1 & server 2.
Surprise is after starting server 2, server 1 also starts to use the same
port as server 2 is using as client port. Does that matter , as server1 &
server 2 are running in different boxes.
Any help is appreciated.
Thanks
Siddhartha
Re: Problem with Zookeeper cluster configuration
Posted by Jared Cantwell <ja...@gmail.com>.
I think your issue is that the clientPort is the same port as the
leader communication port in the server.X entry. So the server begins
listening for clients on port 2181, but then cannot communicate with
the leader, or vice versa. Change the client port to something other
than 5181 and see if that works.
~Jared
On Wed, Oct 27, 2010 at 9:21 AM, siddhartha banik
<si...@gmail.com> wrote:
> Hi,
>
> I am trying to configure zookeeper cluster ... with 2 server instances.
> zookeeper version : 3.2.2
>
> Config files are :
>
> *Server 1. zoo.cfg*
>
> tickTime=2000
> initLimit=10
> syncLimit=5
> dataDir=/home/xuser/zookeeper1/zookeeper-3.2.2/data/
> clientPort=5181
> server.1=3.7.192.142:5181:5888
> server.2=3.7.192.145:5181:5888
>
> *Server 2. zoo.cfg*
>
> tickTime=2000
> initLimit=10
> syncLimit=5
> dataDir=/home/xuser/zookeeper2/zookeeper-3.2.2/data/
> clientPort=5181
> server.1=3.7.192.142:5181:5888
> server.2=3.7.192.145:5181:5888
>
> I have also created "*myid*" files in respective data folders. Below are the
> exception I am seeing :
>
> *Server 1*
>
> 2010-10-27 07:43:43,411 - INFO [QuorumPeer:/0.0.0.0:5181:QuorumPeer@514] -
> LOOKING
> 2010-10-27 07:43:43,418 - INFO [QuorumPeer:/0.0.0.0:5181
> :FastLeaderElection@579] - New election: -1
> 2010-10-27 07:43:43,419 - INFO [QuorumPeer:/0.0.0.0:5181
> :FastLeaderElection@618] - Notification: 1, -1, 382, 1, LOOKING, LOOKING, 1
> 2010-10-27 07:43:43,420 - INFO [QuorumPeer:/0.0.0.0:5181
> :FastLeaderElection@642] - Adding vote
> 2010-10-27 07:43:43,436 - INFO [QuorumPeer:/0.0.0.0:5181
> :FastLeaderElection@618] - Notification: 2, 0, 383, 1, LOOKING, LOOKING, 2
> 2010-10-27 07:43:43,442 - INFO [QuorumPeer:/0.0.0.0:5181
> :FastLeaderElection@642] - Adding vote
> 2010-10-27 07:43:43,443 - INFO [QuorumPeer:/0.0.0.0:5181
> :FastLeaderElection@618] - Notification: 2, 0, 383, 1, LOOKING, LOOKING, 1
> 2010-10-27 07:43:43,443 - INFO [QuorumPeer:/0.0.0.0:5181
> :FastLeaderElection@642] - Adding vote
> 2010-10-27 07:43:43,444 - INFO [QuorumPeer:/0.0.0.0:5181:QuorumPeer@523] -
> FOLLOWING
> 2010-10-27 07:43:43,445 - INFO [QuorumPeer:/0.0.0.0:5181
> :ZooKeeperServer@160] - Created server
> 2010-10-27 07:43:43,447 - INFO [QuorumPeer:/0.0.0.0:5181:Follower@147] -
> Following /3.7.192.145:5181
> 2010-10-27 07:43:43,461 - INFO [WorkerReceiver
> Thread:FastLeaderElection$Messenger$WorkerReceiver@254] - Sending new
> notification.
> 2010-10-27 07:43:43,462 - WARN [QuorumPeer:/0.0.0.0:5181:Follower@318] -
> Exception when following the leader
> java.io.EOFException
> at java.io.DataInputStream.readInt(DataInputStream.java:375)
> at
> org.apache.jute.BinaryInputArchive.readInt(BinaryInputArchive.java:63)
> at
> org.apache.zookeeper.server.quorum.QuorumPacket.deserialize(QuorumPacket.java:66)
> at
> org.apache.jute.BinaryInputArchive.readRecord(BinaryInputArchive.java:108)
> at
> org.apache.zookeeper.server.quorum.Follower.readPacket(Follower.java:114)
> at
> org.apache.zookeeper.server.quorum.Follower.followLeader(Follower.java:193)
> at
> org.apache.zookeeper.server.quorum.QuorumPeer.run(QuorumPeer.java:525)
> 2010-10-27 07:43:43,464 - INFO [QuorumPeer:/0.0.0.0:5181:Follower@436] -
> shutdown called
> java.lang.Exception: shutdown Follower
> at
> org.apache.zookeeper.server.quorum.Follower.shutdown(Follower.java:436)
> at
> org.apache.zookeeper.server.quorum.QuorumPeer.run(QuorumPeer.java:529)
>
> *Server 2
>
> *010-10-27 07:59:22,387 - INFO [QuorumPeer:/0.0.0.0:5181:QuorumPeer@535] -
> LEADING
> 2010-10-27 07:59:22,388 - INFO [QuorumPeer:/0.0.0.0:5181
> :ZooKeeperServer@160] - Created server
> 2010-10-27 07:59:22,390 - ERROR [QuorumPeer:/0.0.0.0:5181:Leader@127] -
> Couldn't bind to port 5181
> java.net.BindException: Address already in use
> at java.net.PlainSocketImpl.socketBind(Native Method)
> at java.net.PlainSocketImpl.bind(PlainSocketImpl.java:359)
> at java.net.ServerSocket.bind(ServerSocket.java:319)
> at java.net.ServerSocket.<init>(ServerSocket.java:185)
> at java.net.ServerSocket.<init>(ServerSocket.java:97)
> at org.apache.zookeeper.server.quorum.Leader.<init>(Leader.java:125)
> at
> org.apache.zookeeper.server.quorum.QuorumPeer.makeLeader(QuorumPeer.java:417)
> at
> org.apache.zookeeper.server.quorum.QuorumPeer.run(QuorumPeer.java:537)
> 2010-10-27 07:59:22,392 - WARN [QuorumPeer:/0.0.0.0:5181:QuorumPeer@541] -
> Unexpected exception
> java.net.BindException: Address already in use
> at java.net.PlainSocketImpl.socketBind(Native Method)
> at java.net.PlainSocketImpl.bind(PlainSocketImpl.java:359)
> at java.net.ServerSocket.bind(ServerSocket.java:319)
> at java.net.ServerSocket.<init>(ServerSocket.java:185)
> at java.net.ServerSocket.<init>(ServerSocket.java:97)
> at org.apache.zookeeper.server.quorum.Leader.<init>(Leader.java:125)
> at
> org.apache.zookeeper.server.quorum.QuorumPeer.makeLeader(QuorumPeer.java:417)
> at
> org.apache.zookeeper.server.quorum.QuorumPeer.run(QuorumPeer.java:537)
> 2010-10-27 07:59:22,393 - INFO [WorkerReceiver
> Thread:FastLeaderElection$Messenger$WorkerReceiver@254] - Sending new
> notification.
> 2010-10-27 07:59:22,394 - INFO [QuorumPeer:/0.0.0.0:5181:QuorumPeer@514] -
> LOOKING*
> *
>
> I have tried with netstat command also. No other process is using *5181 *port
> other then zookeeper process.
>
> Other thing I have tried is: using separate ports for server1 & server 2.
> Surprise is after starting server 2, server 1 also starts to use the same
> port as server 2 is using as client port. Does that matter , as server1 &
> server 2 are running in different boxes.
>
> Any help is appreciated.
>
>
> Thanks
> Siddhartha
>
Re: Problem with Zookeeper cluster configuration
Posted by Ted Dunning <te...@gmail.com>.
Why 2? Why not 3?
Even if this is just proof of concept, 2 servers is pretty weird.
That said your two instances should still be able to form a quorum. Have
you done the standard tests of back and forth connectivity on the ports in
question? Can you telnet to the ports?
Any chance you have left-over processes on those ports?
On Wed, Oct 27, 2010 at 6:21 AM, siddhartha banik <
siddhartha.banik@gmail.com> wrote:
> Hi,
>
> I am trying to configure zookeeper cluster ... with 2 server instances.
> zookeeper version : 3.2.2
>
> Config files are :
>
> *Server 1. zoo.cfg*
>
> tickTime=2000
> initLimit=10
> syncLimit=5
> dataDir=/home/xuser/zookeeper1/zookeeper-3.2.2/data/
> clientPort=5181
> server.1=3.7.192.142:5181:5888
> server.2=3.7.192.145:5181:5888
>
> *Server 2. zoo.cfg*
>
> tickTime=2000
> initLimit=10
> syncLimit=5
> dataDir=/home/xuser/zookeeper2/zookeeper-3.2.2/data/
> clientPort=5181
> server.1=3.7.192.142:5181:5888
> server.2=3.7.192.145:5181:5888
>
> I have also created "*myid*" files in respective data folders. Below are
> the
> exception I am seeing :
>
> *Server 1*
>
> 2010-10-27 07:43:43,411 - INFO [QuorumPeer:/0.0.0.0:5181:QuorumPeer@514]
> -
> LOOKING
> 2010-10-27 07:43:43,418 - INFO [QuorumPeer:/0.0.0.0:5181
> :FastLeaderElection@579] - New election: -1
> 2010-10-27 07:43:43,419 - INFO [QuorumPeer:/0.0.0.0:5181
> :FastLeaderElection@618] - Notification: 1, -1, 382, 1, LOOKING, LOOKING,
> 1
> 2010-10-27 07:43:43,420 - INFO [QuorumPeer:/0.0.0.0:5181
> :FastLeaderElection@642] - Adding vote
> 2010-10-27 07:43:43,436 - INFO [QuorumPeer:/0.0.0.0:5181
> :FastLeaderElection@618] - Notification: 2, 0, 383, 1, LOOKING, LOOKING, 2
> 2010-10-27 07:43:43,442 - INFO [QuorumPeer:/0.0.0.0:5181
> :FastLeaderElection@642] - Adding vote
> 2010-10-27 07:43:43,443 - INFO [QuorumPeer:/0.0.0.0:5181
> :FastLeaderElection@618] - Notification: 2, 0, 383, 1, LOOKING, LOOKING, 1
> 2010-10-27 07:43:43,443 - INFO [QuorumPeer:/0.0.0.0:5181
> :FastLeaderElection@642] - Adding vote
> 2010-10-27 07:43:43,444 - INFO [QuorumPeer:/0.0.0.0:5181:QuorumPeer@523]
> -
> FOLLOWING
> 2010-10-27 07:43:43,445 - INFO [QuorumPeer:/0.0.0.0:5181
> :ZooKeeperServer@160] - Created server
> 2010-10-27 07:43:43,447 - INFO [QuorumPeer:/0.0.0.0:5181:Follower@147] -
> Following /3.7.192.145:5181
> 2010-10-27 07:43:43,461 - INFO [WorkerReceiver
> Thread:FastLeaderElection$Messenger$WorkerReceiver@254] - Sending new
> notification.
> 2010-10-27 07:43:43,462 - WARN [QuorumPeer:/0.0.0.0:5181:Follower@318] -
> Exception when following the leader
> java.io.EOFException
> at java.io.DataInputStream.readInt(DataInputStream.java:375)
> at
> org.apache.jute.BinaryInputArchive.readInt(BinaryInputArchive.java:63)
> at
>
> org.apache.zookeeper.server.quorum.QuorumPacket.deserialize(QuorumPacket.java:66)
> at
> org.apache.jute.BinaryInputArchive.readRecord(BinaryInputArchive.java:108)
> at
> org.apache.zookeeper.server.quorum.Follower.readPacket(Follower.java:114)
> at
> org.apache.zookeeper.server.quorum.Follower.followLeader(Follower.java:193)
> at
> org.apache.zookeeper.server.quorum.QuorumPeer.run(QuorumPeer.java:525)
> 2010-10-27 07:43:43,464 - INFO [QuorumPeer:/0.0.0.0:5181:Follower@436] -
> shutdown called
> java.lang.Exception: shutdown Follower
> at
> org.apache.zookeeper.server.quorum.Follower.shutdown(Follower.java:436)
> at
> org.apache.zookeeper.server.quorum.QuorumPeer.run(QuorumPeer.java:529)
>
> *Server 2
>
> *010-10-27 07:59:22,387 - INFO [QuorumPeer:/0.0.0.0:5181:QuorumPeer@535]
> -
> LEADING
> 2010-10-27 07:59:22,388 - INFO [QuorumPeer:/0.0.0.0:5181
> :ZooKeeperServer@160] - Created server
> 2010-10-27 07:59:22,390 - ERROR [QuorumPeer:/0.0.0.0:5181:Leader@127] -
> Couldn't bind to port 5181
> java.net.BindException: Address already in use
> at java.net.PlainSocketImpl.socketBind(Native Method)
> at java.net.PlainSocketImpl.bind(PlainSocketImpl.java:359)
> at java.net.ServerSocket.bind(ServerSocket.java:319)
> at java.net.ServerSocket.<init>(ServerSocket.java:185)
> at java.net.ServerSocket.<init>(ServerSocket.java:97)
> at org.apache.zookeeper.server.quorum.Leader.<init>(Leader.java:125)
> at
>
> org.apache.zookeeper.server.quorum.QuorumPeer.makeLeader(QuorumPeer.java:417)
> at
> org.apache.zookeeper.server.quorum.QuorumPeer.run(QuorumPeer.java:537)
> 2010-10-27 07:59:22,392 - WARN [QuorumPeer:/0.0.0.0:5181:QuorumPeer@541]
> -
> Unexpected exception
> java.net.BindException: Address already in use
> at java.net.PlainSocketImpl.socketBind(Native Method)
> at java.net.PlainSocketImpl.bind(PlainSocketImpl.java:359)
> at java.net.ServerSocket.bind(ServerSocket.java:319)
> at java.net.ServerSocket.<init>(ServerSocket.java:185)
> at java.net.ServerSocket.<init>(ServerSocket.java:97)
> at org.apache.zookeeper.server.quorum.Leader.<init>(Leader.java:125)
> at
>
> org.apache.zookeeper.server.quorum.QuorumPeer.makeLeader(QuorumPeer.java:417)
> at
> org.apache.zookeeper.server.quorum.QuorumPeer.run(QuorumPeer.java:537)
> 2010-10-27 07:59:22,393 - INFO [WorkerReceiver
> Thread:FastLeaderElection$Messenger$WorkerReceiver@254] - Sending new
> notification.
> 2010-10-27 07:59:22,394 - INFO [QuorumPeer:/0.0.0.0:5181:QuorumPeer@514]
> -
> LOOKING*
> *
>
> I have tried with netstat command also. No other process is using *5181
> *port
> other then zookeeper process.
>
> Other thing I have tried is: using separate ports for server1 & server 2.
> Surprise is after starting server 2, server 1 also starts to use the same
> port as server 2 is using as client port. Does that matter , as server1 &
> server 2 are running in different boxes.
>
> Any help is appreciated.
>
>
> Thanks
> Siddhartha
>
Re: Problem with Zookeeper cluster configuration
Posted by siddhartha banik <si...@gmail.com>.
Yes, after changing the client port the problem in zookeeper cluster got
solved.
Thanks a LOT for all your help.
~ Siddhartha
On Wed, Oct 27, 2010 at 11:13 PM, Mahadev Konar <ma...@yahoo-inc.com>wrote:
> I think Jared pointed this out, given that your clientPort and quorum port
> are same:
>
> clientPort=5181
> server.1=3.7.192.142:5181:5888
> ^^^^
>
>
> The above 2 ports should be different.
>
> Thanks
> mahadev
>
> On 10/27/10 10:19 AM, "Ted Dunning" <te...@gmail.com> wrote:
>
> > Sorry, didn't see this last bit.
> >
> > Hmph. A real ZK person will have to answer this.
> >
> > On Wed, Oct 27, 2010 at 6:21 AM, siddhartha banik <
> > siddhartha.banik@gmail.com> wrote:
> >
> >> I have tried with netstat command also. No other process is using *5181
> >> *port
> >> other then zookeeper process.
> >>
> >> Other thing I have tried is: using separate ports for server1 & server
> 2.
> >> Surprise is after starting server 2, server 1 also starts to use the
> same
> >> port as server 2 is using as client port. Does that matter , as server1
> &
> >> server 2 are running in different boxes.
> >>
> >> Any help is appreciated.
> >>
> >>
> >> Thanks
> >> Siddhartha
> >>
> >
>
>
Re: Problem with Zookeeper cluster configuration
Posted by Mahadev Konar <ma...@yahoo-inc.com>.
I think Jared pointed this out, given that your clientPort and quorum port
are same:
clientPort=5181
server.1=3.7.192.142:5181:5888
^^^^
The above 2 ports should be different.
Thanks
mahadev
On 10/27/10 10:19 AM, "Ted Dunning" <te...@gmail.com> wrote:
> Sorry, didn't see this last bit.
>
> Hmph. A real ZK person will have to answer this.
>
> On Wed, Oct 27, 2010 at 6:21 AM, siddhartha banik <
> siddhartha.banik@gmail.com> wrote:
>
>> I have tried with netstat command also. No other process is using *5181
>> *port
>> other then zookeeper process.
>>
>> Other thing I have tried is: using separate ports for server1 & server 2.
>> Surprise is after starting server 2, server 1 also starts to use the same
>> port as server 2 is using as client port. Does that matter , as server1 &
>> server 2 are running in different boxes.
>>
>> Any help is appreciated.
>>
>>
>> Thanks
>> Siddhartha
>>
>
Re: Problem with Zookeeper cluster configuration
Posted by Ted Dunning <te...@gmail.com>.
Sorry, didn't see this last bit.
Hmph. A real ZK person will have to answer this.
On Wed, Oct 27, 2010 at 6:21 AM, siddhartha banik <
siddhartha.banik@gmail.com> wrote:
> I have tried with netstat command also. No other process is using *5181
> *port
> other then zookeeper process.
>
> Other thing I have tried is: using separate ports for server1 & server 2.
> Surprise is after starting server 2, server 1 also starts to use the same
> port as server 2 is using as client port. Does that matter , as server1 &
> server 2 are running in different boxes.
>
> Any help is appreciated.
>
>
> Thanks
> Siddhartha
>
Problem with Zookeeper cluster configuration
Posted by siddhartha banik <si...@gmail.com>.
Hi,
I am trying to configure zookeeper cluster ... with 2 server instances.
zookeeper version : 3.2.2
Config files are :
*Server 1. zoo.cfg*
tickTime=2000
initLimit=10
syncLimit=5
dataDir=/home/xuser/zookeeper1/zookeeper-3.2.2/data/
clientPort=5181
server.1=3.7.192.142:5181:5888
server.2=3.7.192.145:5181:5888
*Server 2. zoo.cfg*
tickTime=2000
initLimit=10
syncLimit=5
dataDir=/home/xuser/zookeeper2/zookeeper-3.2.2/data/
clientPort=5181
server.1=3.7.192.142:5181:5888
server.2=3.7.192.145:5181:5888
I have also created "*myid*" files in respective data folders. Below are the
exception I am seeing :
*Server 1*
2010-10-27 07:43:43,411 - INFO [QuorumPeer:/0.0.0.0:5181:QuorumPeer@514] -
LOOKING
2010-10-27 07:43:43,418 - INFO [QuorumPeer:/0.0.0.0:5181
:FastLeaderElection@579] - New election: -1
2010-10-27 07:43:43,419 - INFO [QuorumPeer:/0.0.0.0:5181
:FastLeaderElection@618] - Notification: 1, -1, 382, 1, LOOKING, LOOKING, 1
2010-10-27 07:43:43,420 - INFO [QuorumPeer:/0.0.0.0:5181
:FastLeaderElection@642] - Adding vote
2010-10-27 07:43:43,436 - INFO [QuorumPeer:/0.0.0.0:5181
:FastLeaderElection@618] - Notification: 2, 0, 383, 1, LOOKING, LOOKING, 2
2010-10-27 07:43:43,442 - INFO [QuorumPeer:/0.0.0.0:5181
:FastLeaderElection@642] - Adding vote
2010-10-27 07:43:43,443 - INFO [QuorumPeer:/0.0.0.0:5181
:FastLeaderElection@618] - Notification: 2, 0, 383, 1, LOOKING, LOOKING, 1
2010-10-27 07:43:43,443 - INFO [QuorumPeer:/0.0.0.0:5181
:FastLeaderElection@642] - Adding vote
2010-10-27 07:43:43,444 - INFO [QuorumPeer:/0.0.0.0:5181:QuorumPeer@523] -
FOLLOWING
2010-10-27 07:43:43,445 - INFO [QuorumPeer:/0.0.0.0:5181
:ZooKeeperServer@160] - Created server
2010-10-27 07:43:43,447 - INFO [QuorumPeer:/0.0.0.0:5181:Follower@147] -
Following /3.7.192.145:5181
2010-10-27 07:43:43,461 - INFO [WorkerReceiver
Thread:FastLeaderElection$Messenger$WorkerReceiver@254] - Sending new
notification.
2010-10-27 07:43:43,462 - WARN [QuorumPeer:/0.0.0.0:5181:Follower@318] -
Exception when following the leader
java.io.EOFException
at java.io.DataInputStream.readInt(DataInputStream.java:375)
at
org.apache.jute.BinaryInputArchive.readInt(BinaryInputArchive.java:63)
at
org.apache.zookeeper.server.quorum.QuorumPacket.deserialize(QuorumPacket.java:66)
at
org.apache.jute.BinaryInputArchive.readRecord(BinaryInputArchive.java:108)
at
org.apache.zookeeper.server.quorum.Follower.readPacket(Follower.java:114)
at
org.apache.zookeeper.server.quorum.Follower.followLeader(Follower.java:193)
at
org.apache.zookeeper.server.quorum.QuorumPeer.run(QuorumPeer.java:525)
2010-10-27 07:43:43,464 - INFO [QuorumPeer:/0.0.0.0:5181:Follower@436] -
shutdown called
java.lang.Exception: shutdown Follower
at
org.apache.zookeeper.server.quorum.Follower.shutdown(Follower.java:436)
at
org.apache.zookeeper.server.quorum.QuorumPeer.run(QuorumPeer.java:529)
*Server 2
*010-10-27 07:59:22,387 - INFO [QuorumPeer:/0.0.0.0:5181:QuorumPeer@535] -
LEADING
2010-10-27 07:59:22,388 - INFO [QuorumPeer:/0.0.0.0:5181
:ZooKeeperServer@160] - Created server
2010-10-27 07:59:22,390 - ERROR [QuorumPeer:/0.0.0.0:5181:Leader@127] -
Couldn't bind to port 5181
java.net.BindException: Address already in use
at java.net.PlainSocketImpl.socketBind(Native Method)
at java.net.PlainSocketImpl.bind(PlainSocketImpl.java:359)
at java.net.ServerSocket.bind(ServerSocket.java:319)
at java.net.ServerSocket.<init>(ServerSocket.java:185)
at java.net.ServerSocket.<init>(ServerSocket.java:97)
at org.apache.zookeeper.server.quorum.Leader.<init>(Leader.java:125)
at
org.apache.zookeeper.server.quorum.QuorumPeer.makeLeader(QuorumPeer.java:417)
at
org.apache.zookeeper.server.quorum.QuorumPeer.run(QuorumPeer.java:537)
2010-10-27 07:59:22,392 - WARN [QuorumPeer:/0.0.0.0:5181:QuorumPeer@541] -
Unexpected exception
java.net.BindException: Address already in use
at java.net.PlainSocketImpl.socketBind(Native Method)
at java.net.PlainSocketImpl.bind(PlainSocketImpl.java:359)
at java.net.ServerSocket.bind(ServerSocket.java:319)
at java.net.ServerSocket.<init>(ServerSocket.java:185)
at java.net.ServerSocket.<init>(ServerSocket.java:97)
at org.apache.zookeeper.server.quorum.Leader.<init>(Leader.java:125)
at
org.apache.zookeeper.server.quorum.QuorumPeer.makeLeader(QuorumPeer.java:417)
at
org.apache.zookeeper.server.quorum.QuorumPeer.run(QuorumPeer.java:537)
2010-10-27 07:59:22,393 - INFO [WorkerReceiver
Thread:FastLeaderElection$Messenger$WorkerReceiver@254] - Sending new
notification.
2010-10-27 07:59:22,394 - INFO [QuorumPeer:/0.0.0.0:5181:QuorumPeer@514] -
LOOKING*
*
I have tried with netstat command also. No other process is using *5181 *port
other then zookeeper process.
Other thing I have tried is: using separate ports for server1 & server 2.
Surprise is after starting server 2, server 1 also starts to use the same
port as server 2 is using as client port. Does that matter , as server1 &
server 2 are running in different boxes.
Any help is appreciated.
Thanks
Siddhartha