You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@zookeeper.apache.org by Hartmut Lang <ha...@gmail.com> on 2012/09/01 21:31:10 UTC
Re: is it possible to run multiple servers on the same machine and
have data replication working?
Hi Peter,
running multiple instances on the same machine is no problem.
You just have to use different config-files (zoo.cfg), and set different
data-dirs in them (dataDir=...).
Hartmut
2012/8/31 Patrick Hunt <ph...@apache.org>
> Take a look at this: https://github.com/phunt/zkconf If you run it
> with --count=x (see the examples) it will generate the proper configs,
> as well as a start script to run all locally.
>
> Patrick
>
> On Fri, Aug 31, 2012 at 1:05 PM, Peter Litvak <pe...@gmail.com>
> wrote:
> > Hi All,
> >
> > There is an explanation on how to run multiple instances of the zookeeper
> > server on the same machine (in getting started) but that seems doesn't
> > allow data replication between those servers.
> > Is there a way to achieve a data replication among servers running on the
> > same machine?
> >
> > I suspect that the answer is NO, just wanted to make sure.
> >
> > Thank you,
> > Peter
>
Re: is it possible to run multiple servers on the same machine and
have data replication working?
Posted by Peter Litvak <pe...@gmail.com>.
Ok, so apparently I was using the wrong class to programmatically start the
server, when I use QuorumPeerMain the replication works.
I'll initiate another thread to ask for the right class to
use programmatically.
On Wed, Sep 5, 2012 at 5:13 PM, Peter Litvak <pe...@gmail.com> wrote:
> Thank you for the suggestions. All that is done and I'm still getting no
> replication.
> Basically here is a simple java test program, I'm using the ZooKeeper v
> 3.4.3.
> The tick is set to 2000 and init and sync to 2 ticks. Client ports and
> data dirs are different for each server, and myid is present and correct.
> --
> public class Test {
>
> private static final Logger log = Logger.getLogger(Test.class);
>
> public static void main(String[] agrs) throws Exception {
>
> startServer("src/main/resources/zoo_s1.cfg");
> startServer("src/main/resources/zoo_s2.cfg");
> startServer("src/main/resources/zoo_s3.cfg");
> Thread.sleep(30000);
>
> ZooKeeper zc1 = new ZooKeeper("localhost:2181", 100000, new
> MyWatcher());
> ZooKeeper zc2 = new ZooKeeper("localhost:2182", 100000, new
> MyWatcher());
> ZooKeeper zc3 = new ZooKeeper("localhost:2183", 100000, new
> MyWatcher());
>
> zc1.create("/ROOT", new byte[]{-1}, OPEN_ACL_UNSAFE, PERSISTENT);
> if (zc1.exists("/ROOT", false) != null) {
> log.info(">>>> ROOT was created!");
> } else {
> throw new RuntimeException("No ROOT !");
> }
>
> Thread.sleep(10000);
>
> if (zc2.exists("/ROOT", false) != null) {
> log.info(">>>> ROOT exists!");
> } else {
> log.info(">>>> No ROOT yet!");
> }
>
> Thread.sleep(10000);
>
> if (zc3.exists("/ROOT", false) != null) {
> log.info(">>>> ROOT exists!");
> } else {
> log.info(">>>> No ROOT yet!");
> }
> }
>
>
> private static Thread startServer(String configPath) throws
> QuorumPeerConfig.ConfigException {
>
> final ZooKeeperServerMain z1 = new ZooKeeperServerMain();
> final ServerConfig sc1 = new ServerConfig();
> sc1.parse(configPath);
> Thread t1 = new Thread(new Runnable() {
> @Override
> public void run() {
> try {
> z1.runFromConfig(sc1);
> } catch (IOException e) {
> log.error("!!!", e);
> }
> }
> });
>
> t1.start();
>
> return t1;
> }
>
> private static class MyWatcher implements Watcher {
>
> @Override
> public void process(WatchedEvent event) {
>
> log.info(event.toString());
> }
> }
> }
> --
>
> I expected the second and most definitely the third exists to succeed but
> they both print "No ROOT yet!".
>
> Any ideas on what could be wrong here?
>
>
> On Sun, Sep 2, 2012 at 8:48 AM, Hartmut Lang <ha...@gmail.com>wrote:
>
>> You also have to take care of the "clientPort".
>> You have to use different clientPorts per instance. Otherwise, the
>> instance
>> can not start up because the port is in use.
>> /Hartmut
>>
>> 2012/9/1 Peter Litvak <pe...@gmail.com>
>>
>> > Well, I actually had to create three server.x entries in the config
>> since
>> > it wouldn't start with just two.
>> > Maybe the problem is in that even though I have three entries I'm
>> starting
>> > only two server instances.
>> > So basically config has (as suggested by getting started guide):
>> > server.1=localhost:2887:3887
>> > server.2=localhost:2888:3888
>> > server.3=localhost:2889:3889
>> >
>> > I also have two different data dirs with myid (1 and 2).
>> > And then I start two servers using this config.
>> > I then have a program that starts one client connected to the first
>> > server which creates a node and immediately start the another client
>> that
>> > connects to the second server and checks for an existence of that node
>> and
>> > it fails.
>> >
>> > On Sat, Sep 1, 2012 at 4:51 PM, Jordan Zimmerman <
>> > jordan@jordanzimmerman.com
>> > > wrote:
>> >
>> > > Make sure both instances are in both config files. Also make sure the
>> > > myid files exist.
>> > >
>> > > ====================
>> > > Jordan Zimmerman
>> > >
>> > > On Sep 1, 2012, at 1:46 PM, Peter Litvak <pe...@gmail.com>
>> wrote:
>> > >
>> > > > Hmm, then something else is wrong, I don't get data replication
>> across
>> > > the
>> > > > servers. I'm wondering what could be the problem.
>> > > > On Sep 1, 2012 4:42 PM, "Hartmut Lang" <ha...@gmail.com>
>> wrote:
>> > > >
>> > > >> Hi Peter,
>> > > >>
>> > > >> running multiple instances on the same machine is no problem.
>> > > >> You just have to use different config-files (zoo.cfg), and set
>> > different
>> > > >> data-dirs in them (dataDir=...).
>> > > >>
>> > > >> Hartmut
>> > > >>
>> > > >>
>> > > >> 2012/8/31 Patrick Hunt <ph...@apache.org>
>> > > >>
>> > > >>> Take a look at this: https://github.com/phunt/zkconf If you run
>> it
>> > > >>> with --count=x (see the examples) it will generate the proper
>> > configs,
>> > > >>> as well as a start script to run all locally.
>> > > >>>
>> > > >>> Patrick
>> > > >>>
>> > > >>> On Fri, Aug 31, 2012 at 1:05 PM, Peter Litvak <
>> > peter.litvak@gmail.com>
>> > > >>> wrote:
>> > > >>>> Hi All,
>> > > >>>>
>> > > >>>> There is an explanation on how to run multiple instances of the
>> > > >> zookeeper
>> > > >>>> server on the same machine (in getting started) but that seems
>> > doesn't
>> > > >>>> allow data replication between those servers.
>> > > >>>> Is there a way to achieve a data replication among servers
>> running
>> > on
>> > > >> the
>> > > >>>> same machine?
>> > > >>>>
>> > > >>>> I suspect that the answer is NO, just wanted to make sure.
>> > > >>>>
>> > > >>>> Thank you,
>> > > >>>> Peter
>> > > >>>
>> > > >>
>> > >
>> >
>>
>
>
Re: is it possible to run multiple servers on the same machine and
have data replication working?
Posted by Peter Litvak <pe...@gmail.com>.
Thank you for the suggestions. All that is done and I'm still getting no
replication.
Basically here is a simple java test program, I'm using the ZooKeeper v
3.4.3.
The tick is set to 2000 and init and sync to 2 ticks. Client ports and data
dirs are different for each server, and myid is present and correct.
--
public class Test {
private static final Logger log = Logger.getLogger(Test.class);
public static void main(String[] agrs) throws Exception {
startServer("src/main/resources/zoo_s1.cfg");
startServer("src/main/resources/zoo_s2.cfg");
startServer("src/main/resources/zoo_s3.cfg");
Thread.sleep(30000);
ZooKeeper zc1 = new ZooKeeper("localhost:2181", 100000, new
MyWatcher());
ZooKeeper zc2 = new ZooKeeper("localhost:2182", 100000, new
MyWatcher());
ZooKeeper zc3 = new ZooKeeper("localhost:2183", 100000, new
MyWatcher());
zc1.create("/ROOT", new byte[]{-1}, OPEN_ACL_UNSAFE, PERSISTENT);
if (zc1.exists("/ROOT", false) != null) {
log.info(">>>> ROOT was created!");
} else {
throw new RuntimeException("No ROOT !");
}
Thread.sleep(10000);
if (zc2.exists("/ROOT", false) != null) {
log.info(">>>> ROOT exists!");
} else {
log.info(">>>> No ROOT yet!");
}
Thread.sleep(10000);
if (zc3.exists("/ROOT", false) != null) {
log.info(">>>> ROOT exists!");
} else {
log.info(">>>> No ROOT yet!");
}
}
private static Thread startServer(String configPath) throws
QuorumPeerConfig.ConfigException {
final ZooKeeperServerMain z1 = new ZooKeeperServerMain();
final ServerConfig sc1 = new ServerConfig();
sc1.parse(configPath);
Thread t1 = new Thread(new Runnable() {
@Override
public void run() {
try {
z1.runFromConfig(sc1);
} catch (IOException e) {
log.error("!!!", e);
}
}
});
t1.start();
return t1;
}
private static class MyWatcher implements Watcher {
@Override
public void process(WatchedEvent event) {
log.info(event.toString());
}
}
}
--
I expected the second and most definitely the third exists to succeed but
they both print "No ROOT yet!".
Any ideas on what could be wrong here?
On Sun, Sep 2, 2012 at 8:48 AM, Hartmut Lang <ha...@gmail.com> wrote:
> You also have to take care of the "clientPort".
> You have to use different clientPorts per instance. Otherwise, the instance
> can not start up because the port is in use.
> /Hartmut
>
> 2012/9/1 Peter Litvak <pe...@gmail.com>
>
> > Well, I actually had to create three server.x entries in the config since
> > it wouldn't start with just two.
> > Maybe the problem is in that even though I have three entries I'm
> starting
> > only two server instances.
> > So basically config has (as suggested by getting started guide):
> > server.1=localhost:2887:3887
> > server.2=localhost:2888:3888
> > server.3=localhost:2889:3889
> >
> > I also have two different data dirs with myid (1 and 2).
> > And then I start two servers using this config.
> > I then have a program that starts one client connected to the first
> > server which creates a node and immediately start the another client that
> > connects to the second server and checks for an existence of that node
> and
> > it fails.
> >
> > On Sat, Sep 1, 2012 at 4:51 PM, Jordan Zimmerman <
> > jordan@jordanzimmerman.com
> > > wrote:
> >
> > > Make sure both instances are in both config files. Also make sure the
> > > myid files exist.
> > >
> > > ====================
> > > Jordan Zimmerman
> > >
> > > On Sep 1, 2012, at 1:46 PM, Peter Litvak <pe...@gmail.com>
> wrote:
> > >
> > > > Hmm, then something else is wrong, I don't get data replication
> across
> > > the
> > > > servers. I'm wondering what could be the problem.
> > > > On Sep 1, 2012 4:42 PM, "Hartmut Lang" <ha...@gmail.com>
> wrote:
> > > >
> > > >> Hi Peter,
> > > >>
> > > >> running multiple instances on the same machine is no problem.
> > > >> You just have to use different config-files (zoo.cfg), and set
> > different
> > > >> data-dirs in them (dataDir=...).
> > > >>
> > > >> Hartmut
> > > >>
> > > >>
> > > >> 2012/8/31 Patrick Hunt <ph...@apache.org>
> > > >>
> > > >>> Take a look at this: https://github.com/phunt/zkconf If you run it
> > > >>> with --count=x (see the examples) it will generate the proper
> > configs,
> > > >>> as well as a start script to run all locally.
> > > >>>
> > > >>> Patrick
> > > >>>
> > > >>> On Fri, Aug 31, 2012 at 1:05 PM, Peter Litvak <
> > peter.litvak@gmail.com>
> > > >>> wrote:
> > > >>>> Hi All,
> > > >>>>
> > > >>>> There is an explanation on how to run multiple instances of the
> > > >> zookeeper
> > > >>>> server on the same machine (in getting started) but that seems
> > doesn't
> > > >>>> allow data replication between those servers.
> > > >>>> Is there a way to achieve a data replication among servers running
> > on
> > > >> the
> > > >>>> same machine?
> > > >>>>
> > > >>>> I suspect that the answer is NO, just wanted to make sure.
> > > >>>>
> > > >>>> Thank you,
> > > >>>> Peter
> > > >>>
> > > >>
> > >
> >
>
Re: is it possible to run multiple servers on the same machine and
have data replication working?
Posted by Hartmut Lang <ha...@gmail.com>.
You also have to take care of the "clientPort".
You have to use different clientPorts per instance. Otherwise, the instance
can not start up because the port is in use.
/Hartmut
2012/9/1 Peter Litvak <pe...@gmail.com>
> Well, I actually had to create three server.x entries in the config since
> it wouldn't start with just two.
> Maybe the problem is in that even though I have three entries I'm starting
> only two server instances.
> So basically config has (as suggested by getting started guide):
> server.1=localhost:2887:3887
> server.2=localhost:2888:3888
> server.3=localhost:2889:3889
>
> I also have two different data dirs with myid (1 and 2).
> And then I start two servers using this config.
> I then have a program that starts one client connected to the first
> server which creates a node and immediately start the another client that
> connects to the second server and checks for an existence of that node and
> it fails.
>
> On Sat, Sep 1, 2012 at 4:51 PM, Jordan Zimmerman <
> jordan@jordanzimmerman.com
> > wrote:
>
> > Make sure both instances are in both config files. Also make sure the
> > myid files exist.
> >
> > ====================
> > Jordan Zimmerman
> >
> > On Sep 1, 2012, at 1:46 PM, Peter Litvak <pe...@gmail.com> wrote:
> >
> > > Hmm, then something else is wrong, I don't get data replication across
> > the
> > > servers. I'm wondering what could be the problem.
> > > On Sep 1, 2012 4:42 PM, "Hartmut Lang" <ha...@gmail.com> wrote:
> > >
> > >> Hi Peter,
> > >>
> > >> running multiple instances on the same machine is no problem.
> > >> You just have to use different config-files (zoo.cfg), and set
> different
> > >> data-dirs in them (dataDir=...).
> > >>
> > >> Hartmut
> > >>
> > >>
> > >> 2012/8/31 Patrick Hunt <ph...@apache.org>
> > >>
> > >>> Take a look at this: https://github.com/phunt/zkconf If you run it
> > >>> with --count=x (see the examples) it will generate the proper
> configs,
> > >>> as well as a start script to run all locally.
> > >>>
> > >>> Patrick
> > >>>
> > >>> On Fri, Aug 31, 2012 at 1:05 PM, Peter Litvak <
> peter.litvak@gmail.com>
> > >>> wrote:
> > >>>> Hi All,
> > >>>>
> > >>>> There is an explanation on how to run multiple instances of the
> > >> zookeeper
> > >>>> server on the same machine (in getting started) but that seems
> doesn't
> > >>>> allow data replication between those servers.
> > >>>> Is there a way to achieve a data replication among servers running
> on
> > >> the
> > >>>> same machine?
> > >>>>
> > >>>> I suspect that the answer is NO, just wanted to make sure.
> > >>>>
> > >>>> Thank you,
> > >>>> Peter
> > >>>
> > >>
> >
>
Re: is it possible to run multiple servers on the same machine and
have data replication working?
Posted by Peter Litvak <pe...@gmail.com>.
Well, I actually had to create three server.x entries in the config since
it wouldn't start with just two.
Maybe the problem is in that even though I have three entries I'm starting
only two server instances.
So basically config has (as suggested by getting started guide):
server.1=localhost:2887:3887
server.2=localhost:2888:3888
server.3=localhost:2889:3889
I also have two different data dirs with myid (1 and 2).
And then I start two servers using this config.
I then have a program that starts one client connected to the first
server which creates a node and immediately start the another client that
connects to the second server and checks for an existence of that node and
it fails.
On Sat, Sep 1, 2012 at 4:51 PM, Jordan Zimmerman <jordan@jordanzimmerman.com
> wrote:
> Make sure both instances are in both config files. Also make sure the
> myid files exist.
>
> ====================
> Jordan Zimmerman
>
> On Sep 1, 2012, at 1:46 PM, Peter Litvak <pe...@gmail.com> wrote:
>
> > Hmm, then something else is wrong, I don't get data replication across
> the
> > servers. I'm wondering what could be the problem.
> > On Sep 1, 2012 4:42 PM, "Hartmut Lang" <ha...@gmail.com> wrote:
> >
> >> Hi Peter,
> >>
> >> running multiple instances on the same machine is no problem.
> >> You just have to use different config-files (zoo.cfg), and set different
> >> data-dirs in them (dataDir=...).
> >>
> >> Hartmut
> >>
> >>
> >> 2012/8/31 Patrick Hunt <ph...@apache.org>
> >>
> >>> Take a look at this: https://github.com/phunt/zkconf If you run it
> >>> with --count=x (see the examples) it will generate the proper configs,
> >>> as well as a start script to run all locally.
> >>>
> >>> Patrick
> >>>
> >>> On Fri, Aug 31, 2012 at 1:05 PM, Peter Litvak <pe...@gmail.com>
> >>> wrote:
> >>>> Hi All,
> >>>>
> >>>> There is an explanation on how to run multiple instances of the
> >> zookeeper
> >>>> server on the same machine (in getting started) but that seems doesn't
> >>>> allow data replication between those servers.
> >>>> Is there a way to achieve a data replication among servers running on
> >> the
> >>>> same machine?
> >>>>
> >>>> I suspect that the answer is NO, just wanted to make sure.
> >>>>
> >>>> Thank you,
> >>>> Peter
> >>>
> >>
>
Re: is it possible to run multiple servers on the same machine and
have data replication working?
Posted by Jordan Zimmerman <jo...@jordanzimmerman.com>.
Make sure both instances are in both config files. Also make sure the
myid files exist.
====================
Jordan Zimmerman
On Sep 1, 2012, at 1:46 PM, Peter Litvak <pe...@gmail.com> wrote:
> Hmm, then something else is wrong, I don't get data replication across the
> servers. I'm wondering what could be the problem.
> On Sep 1, 2012 4:42 PM, "Hartmut Lang" <ha...@gmail.com> wrote:
>
>> Hi Peter,
>>
>> running multiple instances on the same machine is no problem.
>> You just have to use different config-files (zoo.cfg), and set different
>> data-dirs in them (dataDir=...).
>>
>> Hartmut
>>
>>
>> 2012/8/31 Patrick Hunt <ph...@apache.org>
>>
>>> Take a look at this: https://github.com/phunt/zkconf If you run it
>>> with --count=x (see the examples) it will generate the proper configs,
>>> as well as a start script to run all locally.
>>>
>>> Patrick
>>>
>>> On Fri, Aug 31, 2012 at 1:05 PM, Peter Litvak <pe...@gmail.com>
>>> wrote:
>>>> Hi All,
>>>>
>>>> There is an explanation on how to run multiple instances of the
>> zookeeper
>>>> server on the same machine (in getting started) but that seems doesn't
>>>> allow data replication between those servers.
>>>> Is there a way to achieve a data replication among servers running on
>> the
>>>> same machine?
>>>>
>>>> I suspect that the answer is NO, just wanted to make sure.
>>>>
>>>> Thank you,
>>>> Peter
>>>
>>
Re: is it possible to run multiple servers on the same machine and
have data replication working?
Posted by Peter Litvak <pe...@gmail.com>.
Hmm, then something else is wrong, I don't get data replication across the
servers. I'm wondering what could be the problem.
On Sep 1, 2012 4:42 PM, "Hartmut Lang" <ha...@gmail.com> wrote:
> Hi Peter,
>
> running multiple instances on the same machine is no problem.
> You just have to use different config-files (zoo.cfg), and set different
> data-dirs in them (dataDir=...).
>
> Hartmut
>
>
> 2012/8/31 Patrick Hunt <ph...@apache.org>
>
> > Take a look at this: https://github.com/phunt/zkconf If you run it
> > with --count=x (see the examples) it will generate the proper configs,
> > as well as a start script to run all locally.
> >
> > Patrick
> >
> > On Fri, Aug 31, 2012 at 1:05 PM, Peter Litvak <pe...@gmail.com>
> > wrote:
> > > Hi All,
> > >
> > > There is an explanation on how to run multiple instances of the
> zookeeper
> > > server on the same machine (in getting started) but that seems doesn't
> > > allow data replication between those servers.
> > > Is there a way to achieve a data replication among servers running on
> the
> > > same machine?
> > >
> > > I suspect that the answer is NO, just wanted to make sure.
> > >
> > > Thank you,
> > > Peter
> >
>