You are viewing a plain text version of this content. The canonical link for it is here.

Posted to user@zookeeper.apache.org by Martin Podval <ma...@podval.eu> on 2015/05/22 13:40:40 UTC

Dynamic construction of cluster

Hi All,

I'd like to programatically setup zookeeper cluster. My goal is to use
machines with CoreOS and dynamically deploy three nodes in form of docker
containers and setup them to one zoo cluster.

Except common setup in manual (/zookeeperReconfig.html) which shows how to
add another nodes to existing three nodes cluster, I found a conversation
which say how to do that from the beginning when I have no running nodes in
existing cluster. Unfortunately, this set of steps does not work for me.
I'm talking about
http://mail-archives.apache.org/mod_mbox/zookeeper-user/201408.mbox/%3CCAPSuxQipZFH2582bEMid2tCVBFk%3DC31hwCYjbFgSwChBF%2BZQVQ%40mail.gmail.com%3E

Here is a list of steps I did:

1. Run first node with standaloneEnabled=false and the only entry
in zoo.cfg.dynamic.
server.1=localhost:2381:2281:participant;0.0.0.0:2181

2. Run second node with following dynamic cfg:
server.1=localhost:2381:2281:participant;0.0.0.0:2181
server.2=localhost:2382:2282:observer;0.0.0.0:2182

Note that there is no difference in resulting behavior when I'd change
"observer" to "participant" for second node.

3. Now I have two running instances. I can use ./zkCli.sh to log into first
node. When I try to add second node using following command:

reconfig -add server.2=localhost:2382:2282:participant;0.0.0.0:2182

... it fails with:

> KeeperErrorCode = NewConfigNoQuorum for

However, after some research I found solution. But it's tricky and I don't
think that it's the only correct solution.

What is working for me? I can do step #3 on first node again but now with
"observer". This command causes that even first node knows about second
node. When I type 'config' to console in zkCli, it seems that it's working.
Next step is to log into second node using zkCli and than exec commands:

reconfig -remove 2 <- next step is not working w/o this removal
reconfig -add server.2=localhost:2382:2282:participant;0.0.0.0:2182

Well, now I have working cluster for two nodes. Finally, it's interesting
that now I can add third node using regular scenario I've mentioned in
first paragraph.

Do someone have some idea what I'm doing wrong?

Best regards,
Martin

Re: Dynamic construction of cluster

Posted by Alexander Shraer <sh...@gmail.com>.

Hi,

I just tried following your steps and everything worked for me. We also
have a test for a similar scenario which you're welcome to take a look at
(src/java/test/org/apache/zookeeper/server/quorum/StandaloneDisabledTest.java).

The error you're getting indicates that server 2 is not up or not connected
to server 1. To check that its up you can see if a dynamic config file was
created for it after you've started it.

Another option since you're running both servers in localhost is that
somehow your data directories get mixed up.

Here's how I do it.

create the following config files:

conf/zoo_replicated1.cfg:

dataDir=/home/shralex/zk-friday/zookeeper1
standaloneEnabled=false
syncLimit=2
initLimit=5
tickTime=2000
server.1=localhost:2381:2281:participant;0.0.0.0:2181

conf/zoo_replicated2.cfg:

standaloneEnabled=false
dataDir=/home/shralex/zk-friday/zookeeper2
syncLimit=2
tickTime=2000
initLimit=5
server.1=localhost:2381:2281:participant;0.0.0.0:2181
server.2=localhost:2382:2282:observer;0.0.0.0:2182

create the data directories zookeeper1 and zookeeper2 with the id files in
each one

shell window1:  export ZOOCFG=zoo_replicated1.cfg
shell window2: export ZOOCFG=zoo_replicated2.cfg

start server 1 in window 1 ./bin/zkServer.sh start

and continue as in your steps.

Cheers,
Alex


On Fri, May 22, 2015 at 4:40 AM, Martin Podval <ma...@podval.eu> wrote:

> Hi All,
>
> I'd like to programatically setup zookeeper cluster. My goal is to use
> machines with CoreOS and dynamically deploy three nodes in form of docker
> containers and setup them to one zoo cluster.
>
> Except common setup in manual (/zookeeperReconfig.html) which shows how to
> add another nodes to existing three nodes cluster, I found a conversation
> which say how to do that from the beginning when I have no running nodes in
> existing cluster. Unfortunately, this set of steps does not work for me.
> I'm talking about
>
> http://mail-archives.apache.org/mod_mbox/zookeeper-user/201408.mbox/%3CCAPSuxQipZFH2582bEMid2tCVBFk%3DC31hwCYjbFgSwChBF%2BZQVQ%40mail.gmail.com%3E
>
> Here is a list of steps I did:
>
> 1. Run first node with standaloneEnabled=false and the only entry
> in zoo.cfg.dynamic.
> server.1=localhost:2381:2281:participant;0.0.0.0:2181
>
> 2. Run second node with following dynamic cfg:
> server.1=localhost:2381:2281:participant;0.0.0.0:2181
> server.2=localhost:2382:2282:observer;0.0.0.0:2182
>
> Note that there is no difference in resulting behavior when I'd change
> "observer" to "participant" for second node.
>
> 3. Now I have two running instances. I can use ./zkCli.sh to log into first
> node. When I try to add second node using following command:
>
> reconfig -add server.2=localhost:2382:2282:participant;0.0.0.0:2182
>
> ... it fails with:
>
> > KeeperErrorCode = NewConfigNoQuorum for
>
> However, after some research I found solution. But it's tricky and I don't
> think that it's the only correct solution.
>
> What is working for me? I can do step #3 on first node again but now with
> "observer". This command causes that even first node knows about second
> node. When I type 'config' to console in zkCli, it seems that it's working.
> Next step is to log into second node using zkCli and than exec commands:
>
> reconfig -remove 2 <- next step is not working w/o this removal
> reconfig -add server.2=localhost:2382:2282:participant;0.0.0.0:2182
>
> Well, now I have working cluster for two nodes. Finally, it's interesting
> that now I can add third node using regular scenario I've mentioned in
> first paragraph.
>
> Do someone have some idea what I'm doing wrong?
>
> Best regards,
> Martin
>

Fwd: Dynamic construction of cluster

Posted by Martin Podval <ma...@podval.eu>.