You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@zookeeper.apache.org by mufc_fan <ra...@gmail.com> on 2014/08/13 10:46:15 UTC

Leader election in zookeeper

I have 15 machines connected through a network. out of 15, zookeeper servers
are running in three different machines and I have to select a leader for
this distributed systems. My doubts are

1. For the election, 3 are zookeeper servers here so whether they also
contest in this election.
2. How the election will be held. can u explain with each steps.
3. After electing a leader, how a node knows it is the leader.
4. Also can u explain how zookeeper co-ordinates this service simply.




--
View this message in context: http://zookeeper-user.578899.n2.nabble.com/Leader-election-in-zookeeper-tp7580147.html
Sent from the zookeeper-user mailing list archive at Nabble.com.

RE: Leader election in zookeeper

Posted by "Asta, Greg" <gr...@omnigon.com>.
Zookeeper has properties that make it good at these tasks.  It's designed specifically for these things.  Many other scenarios I've seen in the past were not really 'built for these activities' but instead - had these activities tacked onto something else that was maybe less than ideal.  Here's a few examples of where Zookeeper is "custom fit" for these activities:

- Automatic handling of disconnected clients (ephemeral nodes)
- Automatic failover and redundancy
- Callbacks that trigger real time on updates to watched nodes
- Data consistency/integrity on every server in the cluster - including 'counters' that are accurate across every node in the cluster

>From those basic features you can construct the more complicated functionality that people want like distributed locking and distributed "atomic" counters - in a fairly cheap and reliable fashion.

Now - you can build all those things yourself and you can make them work on some of the more traditional database tools as well but with varying degrees of labor.  For the users of Zookeeper - it's really avoiding that labor and headaches that drives them to use a product like this.

-Greg

-----Original Message-----
From: mufc_fan [mailto:rajeshkumarit8292@gmail.com] 
Sent: Thursday, August 14, 2014 12:02 AM
To: zookeeper-user@hadoop.apache.org
Subject: Re: Leader election in zookeeper

can u tell me why we go for zookeeper...Before zookeeper also companies used distributed environment in which services like leader election,lock and others are implemented and they works fine. why we go for zookeeper?



--
View this message in context: http://zookeeper-user.578899.n2.nabble.com/Leader-election-in-zookeeper-tp7580147p7580157.html
Sent from the zookeeper-user mailing list archive at Nabble.com.

Re: How to measure the load vs response time in zookeeper

Posted by Kuba Lekstan <ku...@gmail.com>.
Have you tried this: https://github.com/phunt/zk-smoketest?

Kuba.


2014-08-14 17:37 GMT+02:00 tbgu <tb...@cs.umass.edu>:

> Hi Folks:
>             I am trying to get the load vs response time graph about the
> use of zookeeper. I want to increase the number of requests sent per second
> and get the corresponding average response time for each request.  Is there
> any interface I can use to exactly know the arrival time of  each request
> response time? Thanks very much!
>
> Best
> Tianbo
>

How to measure the load vs response time in zookeeper

Posted by tbgu <tb...@cs.umass.edu>.
Hi Folks:
             I am trying to get the load vs response time graph about 
the use of zookeeper. I want to increase the number of requests sent per 
second and get the corresponding average response time for each 
request.  Is there any interface I can use to exactly know the arrival 
time of  each request response time? Thanks very much!

Best
Tianbo

Re: Leader election in zookeeper

Posted by Matt Narrell <ma...@gmail.com>.
So we don’t have to implement this on our own.

On Aug 13, 2014, at 10:01 PM, mufc_fan <ra...@gmail.com> wrote:

> can u tell me why we go for zookeeper...Before zookeeper also companies used
> distributed environment in which services like leader election,lock and
> others are implemented and they works fine. why we go for zookeeper?
> 
> 
> 
> --
> View this message in context: http://zookeeper-user.578899.n2.nabble.com/Leader-election-in-zookeeper-tp7580147p7580157.html
> Sent from the zookeeper-user mailing list archive at Nabble.com.


Re: Leader election in zookeeper

Posted by mufc_fan <ra...@gmail.com>.
can u tell me why we go for zookeeper...Before zookeeper also companies used
distributed environment in which services like leader election,lock and
others are implemented and they works fine. why we go for zookeeper?



--
View this message in context: http://zookeeper-user.578899.n2.nabble.com/Leader-election-in-zookeeper-tp7580147p7580157.html
Sent from the zookeeper-user mailing list archive at Nabble.com.

Re: Leader election in zookeeper

Posted by mufc_fan <ra...@gmail.com>.
Thank u...so, zookeeper is just to simplify distributed application
implementation.



--
View this message in context: http://zookeeper-user.578899.n2.nabble.com/Leader-election-in-zookeeper-tp7580147p7580156.html
Sent from the zookeeper-user mailing list archive at Nabble.com.

Re: Leader election in zookeeper

Posted by Alvaro Gareppe <ag...@gmail.com>.
I'm usign curator in my client apps.. so i'm not using that code.. But i
have tested it and works fine.. but I think the idea behind that is to give
an starting point to the developers.. but it should work as it is too


On Wed, Aug 13, 2014 at 3:34 PM, mufc_fan <ra...@gmail.com>
wrote:

> Thank u so much for ur patience...I also have one doubt...we have to use
> this
> election receipe as it is or we can modify as we wish
>
>
>
> --
> View this message in context:
> http://zookeeper-user.578899.n2.nabble.com/Leader-election-in-zookeeper-tp7580147p7580154.html
> Sent from the zookeeper-user mailing list archive at Nabble.com.
>



-- 
Ing. Alvaro Gareppe
agareppe@gmail.com

Re: Leader election in zookeeper

Posted by mufc_fan <ra...@gmail.com>.
Thank u so much for ur patience...I also have one doubt...we have to use this
election receipe as it is or we can modify as we wish



--
View this message in context: http://zookeeper-user.578899.n2.nabble.com/Leader-election-in-zookeeper-tp7580147p7580154.html
Sent from the zookeeper-user mailing list archive at Nabble.com.

Re: Leader election in zookeeper

Posted by Alvaro Gareppe <ag...@gmail.com>.
Yes it is like that..

The zookeeper leader election recipe is a "way to work" on the client side
to implement this kind of "selecto who process" logic in a dristributed
system.


On Wed, Aug 13, 2014 at 3:13 PM, mufc_fan <ra...@gmail.com>
wrote:

> From ur reply, i understand that we have to code these receipes by making
> use
> of client side libraries. thus when we have send message to all customers
> in
> db....first we have to elect a leader by leader election receipe logic and
> we have to code it using client side libraries. Also when leader fails,
> watch event triggers thus another election will be held ....correct me if
> am
> wrong...Thus we are creating znodes in zk server and we are making use of
> watch event ...I have to whether this refers as co-ordination of this
> service
>
>
>
> --
> View this message in context:
> http://zookeeper-user.578899.n2.nabble.com/Leader-election-in-zookeeper-tp7580147p7580152.html
> Sent from the zookeeper-user mailing list archive at Nabble.com.
>



-- 
Ing. Alvaro Gareppe
agareppe@gmail.com

Re: Leader election in zookeeper

Posted by mufc_fan <ra...@gmail.com>.
>From ur reply, i understand that we have to code these receipes by making use
of client side libraries. thus when we have send message to all customers in
db....first we have to elect a leader by leader election receipe logic and
we have to code it using client side libraries. Also when leader fails,
watch event triggers thus another election will be held ....correct me if am
wrong...Thus we are creating znodes in zk server and we are making use of
watch event ...I have to whether this refers as co-ordination of this
service



--
View this message in context: http://zookeeper-user.578899.n2.nabble.com/Leader-election-in-zookeeper-tp7580147p7580152.html
Sent from the zookeeper-user mailing list archive at Nabble.com.

Re: Leader election in zookeeper

Posted by Alvaro Gareppe <ag...@gmail.com>.
The idea (in the leader election scenario) for zookeeper is for you to be
able to coordinate a service in a distributed environment.

Consider the case where you have a service that takes all phones from the
DB and send a message to every one of them. and you have that service
running in 5 servers.. withour coordination each client will recieve 5
message, that is not what you need. Zookeeper is a possible solution to
that...

At the begining the service creates a zookeeper instance and offers itself
as leader..
if (leader) then process()
if not wait to be leader

Why is this helpful.. bezides coordination, to avoid havinf 2 servers
running the same process it also solves the problem "what if this server or
service is down?".. cause zookeeper will take care of changing the
leadership.

In this case a zookeeper node cam be used to track the last sent message so
the next leader (in case is needede) can continue from the last sent message

If you what to know how the leader election "protocol" is being implemented
or how to use it refer the link i sent you and the recipe classes .. it
should well documentd there

Regards


On Wed, Aug 13, 2014 at 1:23 PM, mufc_fan <ra...@gmail.com>
wrote:

> Thank you Alvaro for response...Now what i want to know whether I want to
> code receipes in java or C whatever...Also can u tell me why zookeeper is
> needed...I know it is to co-ordinate service but how... can u tell me with
> a
> simple example..plz I am waiting for ur response
>
>
>
> --
> View this message in context:
> http://zookeeper-user.578899.n2.nabble.com/Leader-election-in-zookeeper-tp7580147p7580150.html
> Sent from the zookeeper-user mailing list archive at Nabble.com.
>



-- 
Ing. Alvaro Gareppe
agareppe@gmail.com

Re: Leader election in zookeeper

Posted by mufc_fan <ra...@gmail.com>.
Thank you Alvaro for response...Now what i want to know whether I want to
code receipes in java or C whatever...Also can u tell me why zookeeper is
needed...I know it is to co-ordinate service but how... can u tell me with a
simple example..plz I am waiting for ur response



--
View this message in context: http://zookeeper-user.578899.n2.nabble.com/Leader-election-in-zookeeper-tp7580147p7580150.html
Sent from the zookeeper-user mailing list archive at Nabble.com.

Re: Leader election in zookeeper

Posted by Alvaro Gareppe <ag...@gmail.com>.
Taka a look into: <zookeeperfolder>/recipes/election in there you have a
leader election implementation.

The zookeepers server wont be part of the leader election of you service..
cause the idea is that you are selecting a leader to do some work.
Zookeeper also has a concept or leadership but that is internal of
zookeeper you dont need to take care of that.

If you take a look at that implementation then, after initializing
(creating connection and defining the node used for coodination), the only
thing you have to do is listen to the LeaderElectionAware event of the
LeaderElectionSupport class.


The idea behind that recipe is explained here:
http://zookeeper.apache.org/doc/trunk/recipes.html#sc_leaderElection

Other useful lib is that implements the leader election is "curator"



On Wed, Aug 13, 2014 at 5:46 AM, mufc_fan <ra...@gmail.com>
wrote:

> I have 15 machines connected through a network. out of 15, zookeeper
> servers
> are running in three different machines and I have to select a leader for
> this distributed systems. My doubts are
>
> 1. For the election, 3 are zookeeper servers here so whether they also
> contest in this election.
> 2. How the election will be held. can u explain with each steps.
> 3. After electing a leader, how a node knows it is the leader.
> 4. Also can u explain how zookeeper co-ordinates this service simply.
>
>
>
>
> --
> View this message in context:
> http://zookeeper-user.578899.n2.nabble.com/Leader-election-in-zookeeper-tp7580147.html
> Sent from the zookeeper-user mailing list archive at Nabble.com.
>



-- 
Ing. Alvaro Gareppe
agareppe@gmail.com