You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@zookeeper.apache.org by yuzhou li <ia...@gmail.com> on 2018/06/14 05:47:13 UTC
Dose client read dirty data in zk release-3.5.4 ?
Hi,everyone.
I git pull the tag release-3.5.4 from github. Before I read the
3.4.6 version code, I found 3.5.4 has some changes when the learner
synchronzied with leader.
If using DIFF way to synchronize histories, the follower or observer
will put the data in a queue, then start zkServer, then get all data
from the queue, and process the data as a normol request,after all
data is processd, the server is synchronized with leader completely.
But at zkServe start moment, the client request can be process by this
server,but this server has no data because the data is stored at the
queue currently, so the client will read dirty data.
Does this situation will happend?Or there is some point I has not find?
Re: Dose client read dirty data in zk release-3.5.4 ?
Posted by yuzhou li <ia...@gmail.com>.
Before learner receives the leader UPTODATE request, the loop run like this:
case Leader.PROPOSAL:
PacketInFlight pif = new PacketInFlight();
pif.hdr = new TxnHeader();
pif.rec = SerializeUtils.deserializeTxn(qp.getData(),
pif.hdr);//deserialize data
packetsNotCommitted.add(pif);//add to packetsNotCommitted queue
break;
case Leader.COMMIT:
pif = packetsNotCommitted.peekFirst();
if (!writeToTxnLog) {//only use SNAP synchronize way, writeToTxnLog
will be false
zk.processTxn(pif.hdr, pif.rec);//only update the memory data
tree,not write transcation log
packetsNotCommitted.remove();
} else {//use DIFF synchronize way
packetsCommitted.add(qp.getZxid());//add to packetsCommitted queue
}
break;
If use DIFF way to synchronize data,the real data synchronization has not
start before zk.startup
Is I understand something wrong?
Michael Han <ha...@apache.org> 于2018年6月18日周一 下午12:28写道：
> Data synchronization is already done if the execution hits the zk.startup
> (note the previous while loop will only break if learner receives leaders
> up to date message).
>
> On Wed, Jun 13, 2018 at 10:51 PM, yuzhou li <ia...@gmail.com> wrote:
>
> > The main code is at Learner.java syncWithLeader like this:
> > if (qp.getType() == Leader.DIFF) {
> > LOG.info("Getting a diff from the leader 0x{}",
> > Long.toHexString(qp.getZxid()));
> > snapshotNeeded = false;
> > }
> >
> > case Leader.COMMITANDACTIVATE:
> > if (!writeToTxnLog) {
> > if (pif.hdr.getZxid() != qp.getZxid()) {
> > LOG.warn("Committing " + qp.getZxid() + ", but next
> > proposal is " + pif.hdr.getZxid());
> > } else {
> > zk.processTxn(pif.hdr, pif.rec);
> > packetsNotCommitted.remove();
> > }
> > } else {
> > packetsCommitted.add(qp.getZxid());
> > }
> > break;
> >
> > zk.startup();//this will create the processor chain,and waiting
> > request will be processed after the zk startup
> >
> > if (zk instanceof FollowerZooKeeperServer) {
> > FollowerZooKeeperServer fzk = (FollowerZooKeeperServer)zk;
> > for(PacketInFlight p: packetsNotCommitted) {
> > fzk.logRequest(p.hdr, p.rec);
> > }
> > for(Long zxid: packetsCommitted) {
> > fzk.commit(zxid);
> > }
> > }
> >
> > yuzhou li <ia...@gmail.com> 于2018年6月14日周四 下午1:47写道：
> > >
> > > Hi,everyone.
> > > I git pull the tag release-3.5.4 from github. Before I read the
> > > 3.4.6 version code, I found 3.5.4 has some changes when the learner
> > > synchronzied with leader.
> > > If using DIFF way to synchronize histories, the follower or observer
> > > will put the data in a queue, then start zkServer, then get all data
> > > from the queue, and process the data as a normol request,after all
> > > data is processd, the server is synchronized with leader completely.
> > > But at zkServe start moment, the client request can be process by this
> > > server,but this server has no data because the data is stored at the
> > > queue currently, so the client will read dirty data.
> > > Does this situation will happend?Or there is some point I has not
> > find?
> >
>
Re: Dose client read dirty data in zk release-3.5.4 ?
Posted by Michael Han <ha...@apache.org>.
Data synchronization is already done if the execution hits the zk.startup
(note the previous while loop will only break if learner receives leaders
up to date message).
On Wed, Jun 13, 2018 at 10:51 PM, yuzhou li <ia...@gmail.com> wrote:
> The main code is at Learner.java syncWithLeader like this:
> if (qp.getType() == Leader.DIFF) {
> LOG.info("Getting a diff from the leader 0x{}",
> Long.toHexString(qp.getZxid()));
> snapshotNeeded = false;
> }
>
> case Leader.COMMITANDACTIVATE:
> if (!writeToTxnLog) {
> if (pif.hdr.getZxid() != qp.getZxid()) {
> LOG.warn("Committing " + qp.getZxid() + ", but next
> proposal is " + pif.hdr.getZxid());
> } else {
> zk.processTxn(pif.hdr, pif.rec);
> packetsNotCommitted.remove();
> }
> } else {
> packetsCommitted.add(qp.getZxid());
> }
> break;
>
> zk.startup();//this will create the processor chain,and waiting
> request will be processed after the zk startup
>
> if (zk instanceof FollowerZooKeeperServer) {
> FollowerZooKeeperServer fzk = (FollowerZooKeeperServer)zk;
> for(PacketInFlight p: packetsNotCommitted) {
> fzk.logRequest(p.hdr, p.rec);
> }
> for(Long zxid: packetsCommitted) {
> fzk.commit(zxid);
> }
> }
>
> yuzhou li <ia...@gmail.com> 于2018年6月14日周四 下午1:47写道：
> >
> > Hi,everyone.
> > I git pull the tag release-3.5.4 from github. Before I read the
> > 3.4.6 version code, I found 3.5.4 has some changes when the learner
> > synchronzied with leader.
> > If using DIFF way to synchronize histories, the follower or observer
> > will put the data in a queue, then start zkServer, then get all data
> > from the queue, and process the data as a normol request,after all
> > data is processd, the server is synchronized with leader completely.
> > But at zkServe start moment, the client request can be process by this
> > server,but this server has no data because the data is stored at the
> > queue currently, so the client will read dirty data.
> > Does this situation will happend?Or there is some point I has not
> find?
>
Re: Dose client read dirty data in zk release-3.5.4 ?
Posted by yuzhou li <ia...@gmail.com>.
The main code is at Learner.java syncWithLeader like this:
if (qp.getType() == Leader.DIFF) {
LOG.info("Getting a diff from the leader 0x{}",
Long.toHexString(qp.getZxid()));
snapshotNeeded = false;
}
case Leader.COMMITANDACTIVATE:
if (!writeToTxnLog) {
if (pif.hdr.getZxid() != qp.getZxid()) {
LOG.warn("Committing " + qp.getZxid() + ", but next
proposal is " + pif.hdr.getZxid());
} else {
zk.processTxn(pif.hdr, pif.rec);
packetsNotCommitted.remove();
}
} else {
packetsCommitted.add(qp.getZxid());
}
break;
zk.startup();//this will create the processor chain,and waiting
request will be processed after the zk startup
if (zk instanceof FollowerZooKeeperServer) {
FollowerZooKeeperServer fzk = (FollowerZooKeeperServer)zk;
for(PacketInFlight p: packetsNotCommitted) {
fzk.logRequest(p.hdr, p.rec);
}
for(Long zxid: packetsCommitted) {
fzk.commit(zxid);
}
}
yuzhou li <ia...@gmail.com> 于2018年6月14日周四 下午1:47写道：
>
> Hi,everyone.
> I git pull the tag release-3.5.4 from github. Before I read the
> 3.4.6 version code, I found 3.5.4 has some changes when the learner
> synchronzied with leader.
> If using DIFF way to synchronize histories, the follower or observer
> will put the data in a queue, then start zkServer, then get all data
> from the queue, and process the data as a normol request,after all
> data is processd, the server is synchronized with leader completely.
> But at zkServe start moment, the client request can be process by this
> server,but this server has no data because the data is stored at the
> queue currently, so the client will read dirty data.
> Does this situation will happend?Or there is some point I has not find?