You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@zookeeper.apache.org by "Flavio Junqueira (JIRA)" <ji...@apache.org> on 2012/10/06 15:47:03 UTC
[jira] [Created] (ZOOKEEPER-1559) Learner should not snapshot
uncommitted state
Flavio Junqueira created ZOOKEEPER-1559:
-------------------------------------------
Summary: Learner should not snapshot uncommitted state
Key: ZOOKEEPER-1559
URL: https://issues.apache.org/jira/browse/ZOOKEEPER-1559
Project: ZooKeeper
Issue Type: Sub-task
Reporter: Flavio Junqueira
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (ZOOKEEPER-1559) Learner should not snapshot
uncommitted state
Posted by "Jacky007 (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/ZOOKEEPER-1559?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13472070#comment-13472070 ]
Jacky007 commented on ZOOKEEPER-1559:
-------------------------------------
{quote}
Base on the current implementation, the leader start up and treat every txns in its txnlog as committed. It add those txn into committedLog and the follower get proposal/commit packet pair for those txns as part of the sync-up. Let me know what you think, this problem is quite complicate if we don't want to break compatibility.
{quote}
agree.
We have some discussions in ZOOKEEPER-1549. I think it is quite complicate even if we don't consider the compatibility.
> Learner should not snapshot uncommitted state
> ---------------------------------------------
>
> Key: ZOOKEEPER-1559
> URL: https://issues.apache.org/jira/browse/ZOOKEEPER-1559
> Project: ZooKeeper
> Issue Type: Sub-task
> Components: quorum
> Reporter: Flavio Junqueira
>
> The code in Learner.java is a bit entangled for backward compatibility reasons. We need to make sure that we can remove the calls to take a snapshot without breaking it.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (ZOOKEEPER-1559) Learner should not snapshot
uncommitted state
Posted by "Thawan Kooburat (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/ZOOKEEPER-1559?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13471724#comment-13471724 ]
Thawan Kooburat commented on ZOOKEEPER-1559:
--------------------------------------------
I want to add that the root of this problem is from the leader. Base on the current implementation, the leader start up and treat every txns in its txnlog as committed. It add those txn into committedLog and the follower get proposal/commit packet pair for those txns as part of the sync-up.
Let me know what you think, this problem is quite complicate if we don't want to break compatibility.
> Learner should not snapshot uncommitted state
> ---------------------------------------------
>
> Key: ZOOKEEPER-1559
> URL: https://issues.apache.org/jira/browse/ZOOKEEPER-1559
> Project: ZooKeeper
> Issue Type: Sub-task
> Components: quorum
> Reporter: Flavio Junqueira
>
> The code in Learner.java is a bit entangled for backward compatibility reasons. We need to make sure that we can remove the calls to take a snapshot without breaking it.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (ZOOKEEPER-1559) Learner should not snapshot
uncommitted state
Posted by "Flavio Junqueira (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/ZOOKEEPER-1559?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13471015#comment-13471015 ]
Flavio Junqueira commented on ZOOKEEPER-1559:
---------------------------------------------
This is the main bit of code I've been looking at and I believe we need to fix:
{code}
boolean snapshotTaken = false;
// we are now going to start getting transactions to apply followed by an UPTODATE
outerLoop:
while (self.isRunning()) {
readPacket(qp);
switch(qp.getType()) {
case Leader.PROPOSAL:
PacketInFlight pif = new PacketInFlight();
pif.hdr = new TxnHeader();
pif.rec = SerializeUtils.deserializeTxn(qp.getData(), pif.hdr);
if (pif.hdr.getZxid() != lastQueued + 1) {
LOG.warn("Got zxid 0x"
+ Long.toHexString(pif.hdr.getZxid())
+ " expected 0x"
+ Long.toHexString(lastQueued + 1));
}
lastQueued = pif.hdr.getZxid();
packetsNotCommitted.add(pif);
break;
case Leader.COMMIT:
if (!snapshotTaken) {
pif = packetsNotCommitted.peekFirst();
if (pif.hdr.getZxid() != qp.getZxid()) {
LOG.warn("Committing " + qp.getZxid() + ", but next proposal is " + pif.hdr.getZxid());
} else {
zk.processTxn(pif.hdr, pif.rec);
packetsNotCommitted.remove();
}
} else {
packetsCommitted.add(qp.getZxid());
}
break;
case Leader.INFORM:
TxnHeader hdr = new TxnHeader();
Record txn = SerializeUtils.deserializeTxn(qp.getData(), hdr);
zk.processTxn(hdr, txn);
break;
case Leader.UPTODATE:
if (!snapshotTaken) { // true for the pre v1.0 case
zk.takeSnapshot();
self.setCurrentEpoch(newEpoch);
}
self.cnxnFactory.setZooKeeperServer(zk);
break outerLoop;
case Leader.NEWLEADER: // it will be NEWLEADER in v1.0
zk.takeSnapshot();
self.setCurrentEpoch(newEpoch);
snapshotTaken = true;
writePacket(new QuorumPacket(Leader.ACK, newLeaderZxid, null, null), true);
break;
}
}
{code}
> Learner should not snapshot uncommitted state
> ---------------------------------------------
>
> Key: ZOOKEEPER-1559
> URL: https://issues.apache.org/jira/browse/ZOOKEEPER-1559
> Project: ZooKeeper
> Issue Type: Sub-task
> Components: quorum
> Reporter: Flavio Junqueira
>
> The code in Learner.java is a bit entangled for backward compatibility reasons. We need to make sure that we can remove the calls to take a snapshot without breaking it.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (ZOOKEEPER-1559) Learner should not snapshot
uncommitted state
Posted by "Flavio Junqueira (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/ZOOKEEPER-1559?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Flavio Junqueira updated ZOOKEEPER-1559:
----------------------------------------
Description: The code in Learner.java is a bit entangled for backward compatibility reasons. We need to make sure that we can remove the calls to take a snapshot without breaking it.
> Learner should not snapshot uncommitted state
> ---------------------------------------------
>
> Key: ZOOKEEPER-1559
> URL: https://issues.apache.org/jira/browse/ZOOKEEPER-1559
> Project: ZooKeeper
> Issue Type: Sub-task
> Components: quorum
> Reporter: Flavio Junqueira
>
> The code in Learner.java is a bit entangled for backward compatibility reasons. We need to make sure that we can remove the calls to take a snapshot without breaking it.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira