You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@zookeeper.apache.org by ch huang <ju...@gmail.com> on 2013/09/04 08:54:02 UTC
issue about write sync in majority followers
hi,all:
i am very confused about the write sync process,what i thank is
,client connect to one of follower ,send a update request to it,and the
follower persisted the update to it's disk ,then forward this request to
the leader,the leader broadcast it to all it's follower, if the leader get
responce of the majority of all follower say the update succeed, it commits
the update ,and the client will get success response (from who? the leader
or the follower who connected?)
so before leader persisted the change ,this change should be persisted on
majority of followers ?
what i think is right??
Re: issue about write sync in majority followers
Posted by Neha Narkhede <ne...@gmail.com>.
Ah, thanks for pointing that out, Kishore! That was my bad. There is no
relation between Kafka and Zookeeper replication
-Neha
On Wed, Sep 4, 2013 at 6:59 PM, ch huang <ju...@gmail.com> wrote:
> thanks for reply ,but any relation between kafka and zookeeper?
>
> On Wed, Sep 4, 2013 at 10:51 PM, Neha Narkhede <neha.narkhede@gmail.com
> >wrote:
>
> > Hi,
> >
> > We have attempted to describe the replication protocol here
> > http://kafka.apache.org/documentation.html#replication
> >
> > Please let us know if this does not answer your question.
> >
> > Thanks,
> > Neha
> > On Sep 3, 2013 11:54 PM, "ch huang" <ju...@gmail.com> wrote:
> >
> > > hi,all:
> > > i am very confused about the write sync process,what i thank
> > is
> > > ,client connect to one of follower ,send a update request to it,and the
> > > follower persisted the update to it's disk ,then forward this request
> to
> > > the leader,the leader broadcast it to all it's follower, if the leader
> > get
> > > responce of the majority of all follower say the update succeed, it
> > commits
> > > the update ,and the client will get success response (from who? the
> > leader
> > > or the follower who connected?)
> > >
> > > so before leader persisted the change ,this change should be persisted
> on
> > > majority of followers ?
> > >
> > > what i think is right??
> > >
> >
>
Re: issue about write sync in majority followers
Posted by ch huang <ju...@gmail.com>.
thanks for reply ,but any relation between kafka and zookeeper?
On Wed, Sep 4, 2013 at 10:51 PM, Neha Narkhede <ne...@gmail.com>wrote:
> Hi,
>
> We have attempted to describe the replication protocol here
> http://kafka.apache.org/documentation.html#replication
>
> Please let us know if this does not answer your question.
>
> Thanks,
> Neha
> On Sep 3, 2013 11:54 PM, "ch huang" <ju...@gmail.com> wrote:
>
> > hi,all:
> > i am very confused about the write sync process,what i thank
> is
> > ,client connect to one of follower ,send a update request to it,and the
> > follower persisted the update to it's disk ,then forward this request to
> > the leader,the leader broadcast it to all it's follower, if the leader
> get
> > responce of the majority of all follower say the update succeed, it
> commits
> > the update ,and the client will get success response (from who? the
> leader
> > or the follower who connected?)
> >
> > so before leader persisted the change ,this change should be persisted on
> > majority of followers ?
> >
> > what i think is right??
> >
>
Re: issue about write sync in majority followers
Posted by Neha Narkhede <ne...@gmail.com>.
Hi,
We have attempted to describe the replication protocol here
http://kafka.apache.org/documentation.html#replication
Please let us know if this does not answer your question.
Thanks,
Neha
On Sep 3, 2013 11:54 PM, "ch huang" <ju...@gmail.com> wrote:
> hi,all:
> i am very confused about the write sync process,what i thank is
> ,client connect to one of follower ,send a update request to it,and the
> follower persisted the update to it's disk ,then forward this request to
> the leader,the leader broadcast it to all it's follower, if the leader get
> responce of the majority of all follower say the update succeed, it commits
> the update ,and the client will get success response (from who? the leader
> or the follower who connected?)
>
> so before leader persisted the change ,this change should be persisted on
> majority of followers ?
>
> what i think is right??
>
Re: issue about write sync in majority followers
Posted by Edward Ribeiro <ed...@gmail.com>.
Huang,
AFAIK, the write sync process in your example is:
1. Client connect to a follower and send update request;
2. The follower redirects the request to leader (it doesn't persist the
state on disk yet!);
3. The leader atomically broadcast the requests, in order, to all followers;
4. Each follower receives the proposal, write it to disk (including the one
the client is connected to), and, if successful on writing, sends an
acknowledgement (ACK) message to the leader;
5. When the leader receives the ACK from majority of followers it
broadcasts a COMMIT message and deliver the message locally.
6. Each follower will deliver the message when they receive the COMMIT from
leader;
7. The follower in step 1 then notifies the client.
That is, all writing is routed through the leader, but reads go locally to
each follower.
Edward
On Fri, Sep 6, 2013 at 4:14 AM, ch huang <ju...@gmail.com> wrote:
> zookeeper use zad not paxos ,so here is discuss zad :)
>
> On Fri, Sep 6, 2013 at 2:29 PM, Flavio Junqueira <fpjunqueira@yahoo.com
> >wrote:
>
> > Did you mean to say zab instead of paxos?
> >
> > -Flavio
> >
> > On Sep 6, 2013, at 6:11 AM, Martin Kou <bi...@gmail.com> wrote:
> >
> > > The follower doesn't persist in the beginning - not until it's sure it
> > had the consensus of the quorum. Also, the client only talks to the node
> it
> > is connected to.
> > >
> > > The general algorithm you're looking for here is Paxos. You may want to
> > look that up.
> > >
> > > Sent from my iPhone
> > >
> > >> On Sep 3, 2013, at 11:54 PM, ch huang <ju...@gmail.com> wrote:
> > >>
> > >> hi,all:
> > >> i am very confused about the write sync process,what i thank
> is
> > >> ,client connect to one of follower ,send a update request to it,and
> the
> > >> follower persisted the update to it's disk ,then forward this request
> to
> > >> the leader,the leader broadcast it to all it's follower, if the leader
> > get
> > >> responce of the majority of all follower say the update succeed, it
> > commits
> > >> the update ,and the client will get success response (from who? the
> > leader
> > >> or the follower who connected?)
> > >>
> > >> so before leader persisted the change ,this change should be persisted
> > on
> > >> majority of followers ?
> > >>
> > >> what i think is right??
> >
> >
>
Re: issue about write sync in majority followers
Posted by ch huang <ju...@gmail.com>.
zookeeper use zad not paxos ,so here is discuss zad :)
On Fri, Sep 6, 2013 at 2:29 PM, Flavio Junqueira <fp...@yahoo.com>wrote:
> Did you mean to say zab instead of paxos?
>
> -Flavio
>
> On Sep 6, 2013, at 6:11 AM, Martin Kou <bi...@gmail.com> wrote:
>
> > The follower doesn't persist in the beginning - not until it's sure it
> had the consensus of the quorum. Also, the client only talks to the node it
> is connected to.
> >
> > The general algorithm you're looking for here is Paxos. You may want to
> look that up.
> >
> > Sent from my iPhone
> >
> >> On Sep 3, 2013, at 11:54 PM, ch huang <ju...@gmail.com> wrote:
> >>
> >> hi,all:
> >> i am very confused about the write sync process,what i thank is
> >> ,client connect to one of follower ,send a update request to it,and the
> >> follower persisted the update to it's disk ,then forward this request to
> >> the leader,the leader broadcast it to all it's follower, if the leader
> get
> >> responce of the majority of all follower say the update succeed, it
> commits
> >> the update ,and the client will get success response (from who? the
> leader
> >> or the follower who connected?)
> >>
> >> so before leader persisted the change ,this change should be persisted
> on
> >> majority of followers ?
> >>
> >> what i think is right??
>
>
Re: issue about write sync in majority followers
Posted by Flavio Junqueira <fp...@yahoo.com>.
Did you mean to say zab instead of paxos?
-Flavio
On Sep 6, 2013, at 6:11 AM, Martin Kou <bi...@gmail.com> wrote:
> The follower doesn't persist in the beginning - not until it's sure it had the consensus of the quorum. Also, the client only talks to the node it is connected to.
>
> The general algorithm you're looking for here is Paxos. You may want to look that up.
>
> Sent from my iPhone
>
>> On Sep 3, 2013, at 11:54 PM, ch huang <ju...@gmail.com> wrote:
>>
>> hi,all:
>> i am very confused about the write sync process,what i thank is
>> ,client connect to one of follower ,send a update request to it,and the
>> follower persisted the update to it's disk ,then forward this request to
>> the leader,the leader broadcast it to all it's follower, if the leader get
>> responce of the majority of all follower say the update succeed, it commits
>> the update ,and the client will get success response (from who? the leader
>> or the follower who connected?)
>>
>> so before leader persisted the change ,this change should be persisted on
>> majority of followers ?
>>
>> what i think is right??
Re: issue about write sync in majority followers
Posted by Martin Kou <bi...@gmail.com>.
The follower doesn't persist in the beginning - not until it's sure it had the consensus of the quorum. Also, the client only talks to the node it is connected to.
The general algorithm you're looking for here is Paxos. You may want to look that up.
Sent from my iPhone
> On Sep 3, 2013, at 11:54 PM, ch huang <ju...@gmail.com> wrote:
>
> hi,all:
> i am very confused about the write sync process,what i thank is
> ,client connect to one of follower ,send a update request to it,and the
> follower persisted the update to it's disk ,then forward this request to
> the leader,the leader broadcast it to all it's follower, if the leader get
> responce of the majority of all follower say the update succeed, it commits
> the update ,and the client will get success response (from who? the leader
> or the follower who connected?)
>
> so before leader persisted the change ,this change should be persisted on
> majority of followers ?
>
> what i think is right??