You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@zookeeper.apache.org by "bit1129@163.com" <bi...@163.com> on 2014/11/10 11:36:06 UTC
How does Zookeeper handle this situation
Hi zookeepers,
I got a question about leader election during ensemble starts up. Say, I got five servers, ABCDE, and start them one by one.
According to the election algorithm, ABC forms the majority, and then a leader is elected among ABC. When D, E start up, they will follow the leader. What if D or E have more recent data among ABCDE?
D,E's latest data will simply be discard? If this is case, this is a critical issue.
Very likely that I missed something how leader election works during startup, any explanation on this? thinks.
bit1129@163.com
Re: How does Zookeeper handle this situation
Posted by Eason Liao <li...@gmail.com>.
Here data means the transactions in log, not the data in ZooKeeper's
in-memory tree structure. The transaction must be committed first before
gets applied to ZooKeeper's in-memory state.
2014-11-10 2:41 GMT-08:00 Eason Liao <li...@gmail.com>:
> It's possible that D, E have more recent data, and the data of D, E will
> be discarded if they join leader. This is fine as long as the data is not
> COMMITTED.
>
> We consider the transaction is committed only if the transaction is
> persisted in the log of majority servers. In this case, if D, E have the
> committed data, then at least
> one of A, B, C will also have it. So the committed data won't get lost.
>
> Hope this answers your question.
>
>
>
> 2014-11-10 2:36 GMT-08:00 bit1129@163.com <bi...@163.com>:
>
>
>> Hi zookeepers,
>>
>> I got a question about leader election during ensemble starts up. Say,
>> I got five servers, ABCDE, and start them one by one.
>> According to the election algorithm, ABC forms the majority, and then a
>> leader is elected among ABC. When D, E start up, they will follow the
>> leader. What if D or E have more recent data among ABCDE?
>> D,E's latest data will simply be discard? If this is case, this is a
>> critical issue.
>>
>> Very likely that I missed something how leader election works during
>> startup, any explanation on this? thinks.
>>
>>
>>
>>
>> bit1129@163.com
>>
>
>
Re: Re: How does Zookeeper handle this situation
Posted by Eason Liao <li...@gmail.com>.
2. BCDE ack this proposal.
BCDE must write the proposal to log before acknowledging the leader.
After CDE get restarted, they will re-apply the transaction in log so the
committed transaction will be applied eventually.
2014-11-10 3:08 GMT-08:00 bit1129@163.com <bi...@163.com>:
> Thanks Eason, this makes sense, but the answer leads me for another
> question.
>
> Say, A is the leader, BCDE are the followers, consider the following
> scenario:
>
> 1. A sends a proposal to BCDE for a update.
> 2. BCDE ack this proposal.
> 3. A send the COMMIT request to BCDE.
> 4. B commit the request, but CDE dont' because of power off before B
> commits.
>
> At this point, A and B are alive but there is no majority any more, so the
> zookeeper service is actually down. The user notices this and first shut
> down A and B, and then restart CDEAB one by one.
>
> Because CDE don't have the latest update commit, is it the case that when
> A and B startup ,their last committed update will be discarded?
>
>
>
>
>
>
>
> bit1129@163.com
>
> From: Eason Liao
> Date: 2014-11-10 18:41
> To: user
> Subject: Re: How does Zookeeper handle this situation
> It's possible that D, E have more recent data, and the data of D, E will be
> discarded if they join leader. This is fine as long as the data is not
> COMMITTED.
>
> We consider the transaction is committed only if the transaction is
> persisted in the log of majority servers. In this case, if D, E have the
> committed data, then at least
> one of A, B, C will also have it. So the committed data won't get lost.
>
> Hope this answers your question.
>
>
>
> 2014-11-10 2:36 GMT-08:00 bit1129@163.com <bi...@163.com>:
>
> >
> > Hi zookeepers,
> >
> > I got a question about leader election during ensemble starts up. Say,
> I
> > got five servers, ABCDE, and start them one by one.
> > According to the election algorithm, ABC forms the majority, and then a
> > leader is elected among ABC. When D, E start up, they will follow the
> > leader. What if D or E have more recent data among ABCDE?
> > D,E's latest data will simply be discard? If this is case, this is a
> > critical issue.
> >
> > Very likely that I missed something how leader election works during
> > startup, any explanation on this? thinks.
> >
> >
> >
> >
> > bit1129@163.com
> >
>
Re: Re: How does Zookeeper handle this situation
Posted by "bit1129@163.com" <bi...@163.com>.
Thanks Eason, this makes sense, but the answer leads me for another question.
Say, A is the leader, BCDE are the followers, consider the following scenario:
1. A sends a proposal to BCDE for a update.
2. BCDE ack this proposal.
3. A send the COMMIT request to BCDE.
4. B commit the request, but CDE dont' because of power off before B commits.
At this point, A and B are alive but there is no majority any more, so the zookeeper service is actually down. The user notices this and first shut down A and B, and then restart CDEAB one by one.
Because CDE don't have the latest update commit, is it the case that when A and B startup ,their last committed update will be discarded?
bit1129@163.com
From: Eason Liao
Date: 2014-11-10 18:41
To: user
Subject: Re: How does Zookeeper handle this situation
It's possible that D, E have more recent data, and the data of D, E will be
discarded if they join leader. This is fine as long as the data is not
COMMITTED.
We consider the transaction is committed only if the transaction is
persisted in the log of majority servers. In this case, if D, E have the
committed data, then at least
one of A, B, C will also have it. So the committed data won't get lost.
Hope this answers your question.
2014-11-10 2:36 GMT-08:00 bit1129@163.com <bi...@163.com>:
>
> Hi zookeepers,
>
> I got a question about leader election during ensemble starts up. Say, I
> got five servers, ABCDE, and start them one by one.
> According to the election algorithm, ABC forms the majority, and then a
> leader is elected among ABC. When D, E start up, they will follow the
> leader. What if D or E have more recent data among ABCDE?
> D,E's latest data will simply be discard? If this is case, this is a
> critical issue.
>
> Very likely that I missed something how leader election works during
> startup, any explanation on this? thinks.
>
>
>
>
> bit1129@163.com
>
Re: How does Zookeeper handle this situation
Posted by Eason Liao <li...@gmail.com>.
It's possible that D, E have more recent data, and the data of D, E will be
discarded if they join leader. This is fine as long as the data is not
COMMITTED.
We consider the transaction is committed only if the transaction is
persisted in the log of majority servers. In this case, if D, E have the
committed data, then at least
one of A, B, C will also have it. So the committed data won't get lost.
Hope this answers your question.
2014-11-10 2:36 GMT-08:00 bit1129@163.com <bi...@163.com>:
>
> Hi zookeepers,
>
> I got a question about leader election during ensemble starts up. Say, I
> got five servers, ABCDE, and start them one by one.
> According to the election algorithm, ABC forms the majority, and then a
> leader is elected among ABC. When D, E start up, they will follow the
> leader. What if D or E have more recent data among ABCDE?
> D,E's latest data will simply be discard? If this is case, this is a
> critical issue.
>
> Very likely that I missed something how leader election works during
> startup, any explanation on this? thinks.
>
>
>
>
> bit1129@163.com
>