You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@zookeeper.apache.org by "Camille Fournier (Created) (JIRA)" <ji...@apache.org> on 2011/10/28 23:35:33 UTC
[jira] [Created] (ZOOKEEPER-1269) Multi deserialization issues
Multi deserialization issues
----------------------------
Key: ZOOKEEPER-1269
URL: https://issues.apache.org/jira/browse/ZOOKEEPER-1269
Project: ZooKeeper
Issue Type: Bug
Components: server
Affects Versions: 3.4.0
Reporter: Camille Fournier
>From the mailing list:
FileTxnSnapLog.restore contains a code block handling a NODEEXISTS failure during deserialization. The problem is explained there in a code comment. The code block however is only executed for a CREATE txn, not for a multiTxn containing a CREATE.
Even if the mentioned code block would also be executed for multi transactions, it needs adaption for multi transactions. What, if after the first failed transaction in a multi txn during deserialization, there would be subsequent transactions in the same multi that would also have failed?
We don't know, since the first failed transaction hides the information about the remaining transactions.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (ZOOKEEPER-1269) Multi deserialization issues
Posted by "Benjamin Reed (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/ZOOKEEPER-1269?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13166609#comment-13166609 ]
Benjamin Reed commented on ZOOKEEPER-1269:
------------------------------------------
+1 excellent work camille!
> Multi deserialization issues
> ----------------------------
>
> Key: ZOOKEEPER-1269
> URL: https://issues.apache.org/jira/browse/ZOOKEEPER-1269
> Project: ZooKeeper
> Issue Type: Bug
> Components: server
> Affects Versions: 3.4.0
> Reporter: Camille Fournier
> Assignee: Camille Fournier
> Fix For: 3.5.0, 3.4.1
>
> Attachments: ZOOKEEPER-1269.patch
>
>
> From the mailing list:
> FileTxnSnapLog.restore contains a code block handling a NODEEXISTS failure during deserialization. The problem is explained there in a code comment. The code block however is only executed for a CREATE txn, not for a multiTxn containing a CREATE.
> Even if the mentioned code block would also be executed for multi transactions, it needs adaption for multi transactions. What, if after the first failed transaction in a multi txn during deserialization, there would be subsequent transactions in the same multi that would also have failed?
> We don't know, since the first failed transaction hides the information about the remaining transactions.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (ZOOKEEPER-1269) Multi deserialization issues
Posted by "Hadoop QA (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/ZOOKEEPER-1269?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13139711#comment-13139711 ]
Hadoop QA commented on ZOOKEEPER-1269:
--------------------------------------
+1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12501375/ZOOKEEPER-1269.patch
against trunk revision 1190450.
+1 @author. The patch does not contain any @author tags.
+1 tests included. The patch appears to include 9 new or modified tests.
+1 javadoc. The javadoc tool did not generate any warning messages.
+1 javac. The applied patch does not increase the total number of javac compiler warnings.
+1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.
+1 release audit. The applied patch does not increase the total number of release audit warnings.
+1 core tests. The patch passed core unit tests.
+1 contrib tests. The patch passed contrib unit tests.
Test results: https://builds.apache.org/job/PreCommit-ZOOKEEPER-Build/740//testReport/
Findbugs warnings: https://builds.apache.org/job/PreCommit-ZOOKEEPER-Build/740//artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
Console output: https://builds.apache.org/job/PreCommit-ZOOKEEPER-Build/740//console
This message is automatically generated.
> Multi deserialization issues
> ----------------------------
>
> Key: ZOOKEEPER-1269
> URL: https://issues.apache.org/jira/browse/ZOOKEEPER-1269
> Project: ZooKeeper
> Issue Type: Bug
> Components: server
> Affects Versions: 3.4.0
> Reporter: Camille Fournier
> Attachments: ZOOKEEPER-1269.patch
>
>
> From the mailing list:
> FileTxnSnapLog.restore contains a code block handling a NODEEXISTS failure during deserialization. The problem is explained there in a code comment. The code block however is only executed for a CREATE txn, not for a multiTxn containing a CREATE.
> Even if the mentioned code block would also be executed for multi transactions, it needs adaption for multi transactions. What, if after the first failed transaction in a multi txn during deserialization, there would be subsequent transactions in the same multi that would also have failed?
> We don't know, since the first failed transaction hides the information about the remaining transactions.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (ZOOKEEPER-1269) Multi deserialization issues
Posted by "Ted Dunning (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/ZOOKEEPER-1269?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13138976#comment-13138976 ]
Ted Dunning commented on ZOOKEEPER-1269:
----------------------------------------
Once a multi makes it to the log, it is pretty much guaranteed that all of the elements of the multi will succeed or log replay will totally fail.
It may be a bit too cowboy to assume that this is really true, but the point of the idempotency property is to make this true.
> Multi deserialization issues
> ----------------------------
>
> Key: ZOOKEEPER-1269
> URL: https://issues.apache.org/jira/browse/ZOOKEEPER-1269
> Project: ZooKeeper
> Issue Type: Bug
> Components: server
> Affects Versions: 3.4.0
> Reporter: Camille Fournier
> Attachments: ZOOKEEPER-1269.patch
>
>
> From the mailing list:
> FileTxnSnapLog.restore contains a code block handling a NODEEXISTS failure during deserialization. The problem is explained there in a code comment. The code block however is only executed for a CREATE txn, not for a multiTxn containing a CREATE.
> Even if the mentioned code block would also be executed for multi transactions, it needs adaption for multi transactions. What, if after the first failed transaction in a multi txn during deserialization, there would be subsequent transactions in the same multi that would also have failed?
> We don't know, since the first failed transaction hides the information about the remaining transactions.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (ZOOKEEPER-1269) Multi deserialization issues
Posted by "Benjamin Reed (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/ZOOKEEPER-1269?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13139305#comment-13139305 ]
Benjamin Reed commented on ZOOKEEPER-1269:
------------------------------------------
we convert multi to idempotent transactions. by nature if an operation is a multi-op fails, there will not be any transaction there at all, so we should process the transactions resulting from multi-op in exactly the same way as normal transactions. the only difference is that we have to make sure that if one of the transactions get processed, they all do.
> Multi deserialization issues
> ----------------------------
>
> Key: ZOOKEEPER-1269
> URL: https://issues.apache.org/jira/browse/ZOOKEEPER-1269
> Project: ZooKeeper
> Issue Type: Bug
> Components: server
> Affects Versions: 3.4.0
> Reporter: Camille Fournier
> Attachments: ZOOKEEPER-1269.patch
>
>
> From the mailing list:
> FileTxnSnapLog.restore contains a code block handling a NODEEXISTS failure during deserialization. The problem is explained there in a code comment. The code block however is only executed for a CREATE txn, not for a multiTxn containing a CREATE.
> Even if the mentioned code block would also be executed for multi transactions, it needs adaption for multi transactions. What, if after the first failed transaction in a multi txn during deserialization, there would be subsequent transactions in the same multi that would also have failed?
> We don't know, since the first failed transaction hides the information about the remaining transactions.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (ZOOKEEPER-1269) Multi deserialization issues
Posted by "Hadoop QA (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/ZOOKEEPER-1269?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13138968#comment-13138968 ]
Hadoop QA commented on ZOOKEEPER-1269:
--------------------------------------
+1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12501375/ZOOKEEPER-1269.patch
against trunk revision 1190450.
+1 @author. The patch does not contain any @author tags.
+1 tests included. The patch appears to include 9 new or modified tests.
+1 javadoc. The javadoc tool did not generate any warning messages.
+1 javac. The applied patch does not increase the total number of javac compiler warnings.
+1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.
+1 release audit. The applied patch does not increase the total number of release audit warnings.
+1 core tests. The patch passed core unit tests.
+1 contrib tests. The patch passed contrib unit tests.
Test results: https://builds.apache.org/job/PreCommit-ZOOKEEPER-Build/726//testReport/
Findbugs warnings: https://builds.apache.org/job/PreCommit-ZOOKEEPER-Build/726//artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
Console output: https://builds.apache.org/job/PreCommit-ZOOKEEPER-Build/726//console
This message is automatically generated.
> Multi deserialization issues
> ----------------------------
>
> Key: ZOOKEEPER-1269
> URL: https://issues.apache.org/jira/browse/ZOOKEEPER-1269
> Project: ZooKeeper
> Issue Type: Bug
> Components: server
> Affects Versions: 3.4.0
> Reporter: Camille Fournier
> Attachments: ZOOKEEPER-1269.patch
>
>
> From the mailing list:
> FileTxnSnapLog.restore contains a code block handling a NODEEXISTS failure during deserialization. The problem is explained there in a code comment. The code block however is only executed for a CREATE txn, not for a multiTxn containing a CREATE.
> Even if the mentioned code block would also be executed for multi transactions, it needs adaption for multi transactions. What, if after the first failed transaction in a multi txn during deserialization, there would be subsequent transactions in the same multi that would also have failed?
> We don't know, since the first failed transaction hides the information about the remaining transactions.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (ZOOKEEPER-1269) Multi deserialization issues
Posted by "Mahadev konar (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/ZOOKEEPER-1269?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13141654#comment-13141654 ]
Mahadev konar commented on ZOOKEEPER-1269:
------------------------------------------
Camille, Should this go into 3.4 or just trunk?
> Multi deserialization issues
> ----------------------------
>
> Key: ZOOKEEPER-1269
> URL: https://issues.apache.org/jira/browse/ZOOKEEPER-1269
> Project: ZooKeeper
> Issue Type: Bug
> Components: server
> Affects Versions: 3.4.0
> Reporter: Camille Fournier
> Assignee: Camille Fournier
> Attachments: ZOOKEEPER-1269.patch
>
>
> From the mailing list:
> FileTxnSnapLog.restore contains a code block handling a NODEEXISTS failure during deserialization. The problem is explained there in a code comment. The code block however is only executed for a CREATE txn, not for a multiTxn containing a CREATE.
> Even if the mentioned code block would also be executed for multi transactions, it needs adaption for multi transactions. What, if after the first failed transaction in a multi txn during deserialization, there would be subsequent transactions in the same multi that would also have failed?
> We don't know, since the first failed transaction hides the information about the remaining transactions.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (ZOOKEEPER-1269) Multi deserialization issues
Posted by "Camille Fournier (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/ZOOKEEPER-1269?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13142225#comment-13142225 ]
Camille Fournier commented on ZOOKEEPER-1269:
---------------------------------------------
I think it should go into both, since it is a bug with multi.
> Multi deserialization issues
> ----------------------------
>
> Key: ZOOKEEPER-1269
> URL: https://issues.apache.org/jira/browse/ZOOKEEPER-1269
> Project: ZooKeeper
> Issue Type: Bug
> Components: server
> Affects Versions: 3.4.0
> Reporter: Camille Fournier
> Assignee: Camille Fournier
> Attachments: ZOOKEEPER-1269.patch
>
>
> From the mailing list:
> FileTxnSnapLog.restore contains a code block handling a NODEEXISTS failure during deserialization. The problem is explained there in a code comment. The code block however is only executed for a CREATE txn, not for a multiTxn containing a CREATE.
> Even if the mentioned code block would also be executed for multi transactions, it needs adaption for multi transactions. What, if after the first failed transaction in a multi txn during deserialization, there would be subsequent transactions in the same multi that would also have failed?
> We don't know, since the first failed transaction hides the information about the remaining transactions.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (ZOOKEEPER-1269) Multi deserialization issues
Posted by "Camille Fournier (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/ZOOKEEPER-1269?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13139004#comment-13139004 ]
Camille Fournier commented on ZOOKEEPER-1269:
---------------------------------------------
Ah yes, being in the log is enough for it to be true were snapshots taken in a frozen system state. But since they are not, you can have these operations fail in playback due to concurrency issues. Multi isn't a special case above the other zk ops, they all have this potential race.
> Multi deserialization issues
> ----------------------------
>
> Key: ZOOKEEPER-1269
> URL: https://issues.apache.org/jira/browse/ZOOKEEPER-1269
> Project: ZooKeeper
> Issue Type: Bug
> Components: server
> Affects Versions: 3.4.0
> Reporter: Camille Fournier
> Attachments: ZOOKEEPER-1269.patch
>
>
> From the mailing list:
> FileTxnSnapLog.restore contains a code block handling a NODEEXISTS failure during deserialization. The problem is explained there in a code comment. The code block however is only executed for a CREATE txn, not for a multiTxn containing a CREATE.
> Even if the mentioned code block would also be executed for multi transactions, it needs adaption for multi transactions. What, if after the first failed transaction in a multi txn during deserialization, there would be subsequent transactions in the same multi that would also have failed?
> We don't know, since the first failed transaction hides the information about the remaining transactions.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (ZOOKEEPER-1269) Multi deserialization issues
Posted by "Patrick Hunt (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/ZOOKEEPER-1269?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Patrick Hunt updated ZOOKEEPER-1269:
------------------------------------
Fix Version/s: 3.4.1
3.5.0
> Multi deserialization issues
> ----------------------------
>
> Key: ZOOKEEPER-1269
> URL: https://issues.apache.org/jira/browse/ZOOKEEPER-1269
> Project: ZooKeeper
> Issue Type: Bug
> Components: server
> Affects Versions: 3.4.0
> Reporter: Camille Fournier
> Assignee: Camille Fournier
> Fix For: 3.5.0, 3.4.1
>
> Attachments: ZOOKEEPER-1269.patch
>
>
> From the mailing list:
> FileTxnSnapLog.restore contains a code block handling a NODEEXISTS failure during deserialization. The problem is explained there in a code comment. The code block however is only executed for a CREATE txn, not for a multiTxn containing a CREATE.
> Even if the mentioned code block would also be executed for multi transactions, it needs adaption for multi transactions. What, if after the first failed transaction in a multi txn during deserialization, there would be subsequent transactions in the same multi that would also have failed?
> We don't know, since the first failed transaction hides the information about the remaining transactions.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (ZOOKEEPER-1269) Multi deserialization issues
Posted by "Camille Fournier (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/ZOOKEEPER-1269?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13141211#comment-13141211 ]
Camille Fournier commented on ZOOKEEPER-1269:
---------------------------------------------
Hey guys, someone want to review and commit this? Looks like we got the OK from the multi folks.
> Multi deserialization issues
> ----------------------------
>
> Key: ZOOKEEPER-1269
> URL: https://issues.apache.org/jira/browse/ZOOKEEPER-1269
> Project: ZooKeeper
> Issue Type: Bug
> Components: server
> Affects Versions: 3.4.0
> Reporter: Camille Fournier
> Assignee: Camille Fournier
> Attachments: ZOOKEEPER-1269.patch
>
>
> From the mailing list:
> FileTxnSnapLog.restore contains a code block handling a NODEEXISTS failure during deserialization. The problem is explained there in a code comment. The code block however is only executed for a CREATE txn, not for a multiTxn containing a CREATE.
> Even if the mentioned code block would also be executed for multi transactions, it needs adaption for multi transactions. What, if after the first failed transaction in a multi txn during deserialization, there would be subsequent transactions in the same multi that would also have failed?
> We don't know, since the first failed transaction hides the information about the remaining transactions.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (ZOOKEEPER-1269) Multi deserialization issues
Posted by "Hudson (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/ZOOKEEPER-1269?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13166828#comment-13166828 ]
Hudson commented on ZOOKEEPER-1269:
-----------------------------------
Integrated in ZooKeeper-trunk #1392 (See [https://builds.apache.org/job/ZooKeeper-trunk/1392/])
ZOOKEEPER-1269. Multi deserialization issues. (Camille Fournier via mahadev)
mahadev : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1212663
Files :
* /zookeeper/trunk/CHANGES.txt
* /zookeeper/trunk/src/java/main/org/apache/zookeeper/server/DataTree.java
* /zookeeper/trunk/src/java/main/org/apache/zookeeper/server/persistence/FileTxnSnapLog.java
* /zookeeper/trunk/src/java/test/org/apache/zookeeper/test/LoadFromLogTest.java
> Multi deserialization issues
> ----------------------------
>
> Key: ZOOKEEPER-1269
> URL: https://issues.apache.org/jira/browse/ZOOKEEPER-1269
> Project: ZooKeeper
> Issue Type: Bug
> Components: server
> Affects Versions: 3.4.0
> Reporter: Camille Fournier
> Assignee: Camille Fournier
> Fix For: 3.5.0, 3.4.1
>
> Attachments: ZOOKEEPER-1269.patch
>
>
> From the mailing list:
> FileTxnSnapLog.restore contains a code block handling a NODEEXISTS failure during deserialization. The problem is explained there in a code comment. The code block however is only executed for a CREATE txn, not for a multiTxn containing a CREATE.
> Even if the mentioned code block would also be executed for multi transactions, it needs adaption for multi transactions. What, if after the first failed transaction in a multi txn during deserialization, there would be subsequent transactions in the same multi that would also have failed?
> We don't know, since the first failed transaction hides the information about the remaining transactions.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (ZOOKEEPER-1269) Multi deserialization issues
Posted by "Camille Fournier (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/ZOOKEEPER-1269?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13162436#comment-13162436 ]
Camille Fournier commented on ZOOKEEPER-1269:
---------------------------------------------
This is a reasonably big bug to just leave outstanding for this long, can someone please review this and check it in?
> Multi deserialization issues
> ----------------------------
>
> Key: ZOOKEEPER-1269
> URL: https://issues.apache.org/jira/browse/ZOOKEEPER-1269
> Project: ZooKeeper
> Issue Type: Bug
> Components: server
> Affects Versions: 3.4.0
> Reporter: Camille Fournier
> Assignee: Camille Fournier
> Fix For: 3.5.0, 3.4.1
>
> Attachments: ZOOKEEPER-1269.patch
>
>
> From the mailing list:
> FileTxnSnapLog.restore contains a code block handling a NODEEXISTS failure during deserialization. The problem is explained there in a code comment. The code block however is only executed for a CREATE txn, not for a multiTxn containing a CREATE.
> Even if the mentioned code block would also be executed for multi transactions, it needs adaption for multi transactions. What, if after the first failed transaction in a multi txn during deserialization, there would be subsequent transactions in the same multi that would also have failed?
> We don't know, since the first failed transaction hides the information about the remaining transactions.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (ZOOKEEPER-1269) Multi deserialization issues
Posted by "Camille Fournier (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/ZOOKEEPER-1269?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13139348#comment-13139348 ]
Camille Fournier commented on ZOOKEEPER-1269:
---------------------------------------------
Right, ok. So I think the patch attached to this issue does exactly that, if someone would like to review it. What I'm not sure is whether the test I put in is particularly good, so would really appreciate one of the multi experts taking a gander there.
> Multi deserialization issues
> ----------------------------
>
> Key: ZOOKEEPER-1269
> URL: https://issues.apache.org/jira/browse/ZOOKEEPER-1269
> Project: ZooKeeper
> Issue Type: Bug
> Components: server
> Affects Versions: 3.4.0
> Reporter: Camille Fournier
> Attachments: ZOOKEEPER-1269.patch
>
>
> From the mailing list:
> FileTxnSnapLog.restore contains a code block handling a NODEEXISTS failure during deserialization. The problem is explained there in a code comment. The code block however is only executed for a CREATE txn, not for a multiTxn containing a CREATE.
> Even if the mentioned code block would also be executed for multi transactions, it needs adaption for multi transactions. What, if after the first failed transaction in a multi txn during deserialization, there would be subsequent transactions in the same multi that would also have failed?
> We don't know, since the first failed transaction hides the information about the remaining transactions.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (ZOOKEEPER-1269) Multi deserialization issues
Posted by "Ted Dunning (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/ZOOKEEPER-1269?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13138996#comment-13138996 ]
Ted Dunning commented on ZOOKEEPER-1269:
----------------------------------------
I am pretty sure that it SHOULD be true. Whether it is actually true is another matter. Being in the log is evidence that there were no reasons for the update to fail if the data store is actually working correctly.
Again, the assumption of correct operation at this scale may be too strong.
> Multi deserialization issues
> ----------------------------
>
> Key: ZOOKEEPER-1269
> URL: https://issues.apache.org/jira/browse/ZOOKEEPER-1269
> Project: ZooKeeper
> Issue Type: Bug
> Components: server
> Affects Versions: 3.4.0
> Reporter: Camille Fournier
> Attachments: ZOOKEEPER-1269.patch
>
>
> From the mailing list:
> FileTxnSnapLog.restore contains a code block handling a NODEEXISTS failure during deserialization. The problem is explained there in a code comment. The code block however is only executed for a CREATE txn, not for a multiTxn containing a CREATE.
> Even if the mentioned code block would also be executed for multi transactions, it needs adaption for multi transactions. What, if after the first failed transaction in a multi txn during deserialization, there would be subsequent transactions in the same multi that would also have failed?
> We don't know, since the first failed transaction hides the information about the remaining transactions.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (ZOOKEEPER-1269) Multi deserialization issues
Posted by "Camille Fournier (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/ZOOKEEPER-1269?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13138796#comment-13138796 ]
Camille Fournier commented on ZOOKEEPER-1269:
---------------------------------------------
The test here is a little bit vague, because I don't really understand how a "proper" but broken multitxn would look. Handling the error codes in FileTxnSnapLog is also a bit fuzzy. But I think the general refactor should fix the issue. Would be great if Marshall could take a look at this to verify.
> Multi deserialization issues
> ----------------------------
>
> Key: ZOOKEEPER-1269
> URL: https://issues.apache.org/jira/browse/ZOOKEEPER-1269
> Project: ZooKeeper
> Issue Type: Bug
> Components: server
> Affects Versions: 3.4.0
> Reporter: Camille Fournier
> Attachments: ZOOKEEPER-1269.patch
>
>
> From the mailing list:
> FileTxnSnapLog.restore contains a code block handling a NODEEXISTS failure during deserialization. The problem is explained there in a code comment. The code block however is only executed for a CREATE txn, not for a multiTxn containing a CREATE.
> Even if the mentioned code block would also be executed for multi transactions, it needs adaption for multi transactions. What, if after the first failed transaction in a multi txn during deserialization, there would be subsequent transactions in the same multi that would also have failed?
> We don't know, since the first failed transaction hides the information about the remaining transactions.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (ZOOKEEPER-1269) Multi deserialization issues
Posted by "Camille Fournier (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/ZOOKEEPER-1269?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13138984#comment-13138984 ]
Camille Fournier commented on ZOOKEEPER-1269:
---------------------------------------------
Are you sure about that given https://issues.apache.org/jira/browse/ZOOKEEPER-1046?
> Multi deserialization issues
> ----------------------------
>
> Key: ZOOKEEPER-1269
> URL: https://issues.apache.org/jira/browse/ZOOKEEPER-1269
> Project: ZooKeeper
> Issue Type: Bug
> Components: server
> Affects Versions: 3.4.0
> Reporter: Camille Fournier
> Attachments: ZOOKEEPER-1269.patch
>
>
> From the mailing list:
> FileTxnSnapLog.restore contains a code block handling a NODEEXISTS failure during deserialization. The problem is explained there in a code comment. The code block however is only executed for a CREATE txn, not for a multiTxn containing a CREATE.
> Even if the mentioned code block would also be executed for multi transactions, it needs adaption for multi transactions. What, if after the first failed transaction in a multi txn during deserialization, there would be subsequent transactions in the same multi that would also have failed?
> We don't know, since the first failed transaction hides the information about the remaining transactions.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (ZOOKEEPER-1269) Multi deserialization issues
Posted by "Camille Fournier (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/ZOOKEEPER-1269?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Camille Fournier updated ZOOKEEPER-1269:
----------------------------------------
Attachment: ZOOKEEPER-1269.patch
> Multi deserialization issues
> ----------------------------
>
> Key: ZOOKEEPER-1269
> URL: https://issues.apache.org/jira/browse/ZOOKEEPER-1269
> Project: ZooKeeper
> Issue Type: Bug
> Components: server
> Affects Versions: 3.4.0
> Reporter: Camille Fournier
> Attachments: ZOOKEEPER-1269.patch
>
>
> From the mailing list:
> FileTxnSnapLog.restore contains a code block handling a NODEEXISTS failure during deserialization. The problem is explained there in a code comment. The code block however is only executed for a CREATE txn, not for a multiTxn containing a CREATE.
> Even if the mentioned code block would also be executed for multi transactions, it needs adaption for multi transactions. What, if after the first failed transaction in a multi txn during deserialization, there would be subsequent transactions in the same multi that would also have failed?
> We don't know, since the first failed transaction hides the information about the remaining transactions.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (ZOOKEEPER-1269) Multi deserialization issues
Posted by "Marshall McMullen (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/ZOOKEEPER-1269?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13162496#comment-13162496 ]
Marshall McMullen commented on ZOOKEEPER-1269:
----------------------------------------------
I agree with Camille. I looked the code over a while back and from a Multi perspective looks correct.
> Multi deserialization issues
> ----------------------------
>
> Key: ZOOKEEPER-1269
> URL: https://issues.apache.org/jira/browse/ZOOKEEPER-1269
> Project: ZooKeeper
> Issue Type: Bug
> Components: server
> Affects Versions: 3.4.0
> Reporter: Camille Fournier
> Assignee: Camille Fournier
> Fix For: 3.5.0, 3.4.1
>
> Attachments: ZOOKEEPER-1269.patch
>
>
> From the mailing list:
> FileTxnSnapLog.restore contains a code block handling a NODEEXISTS failure during deserialization. The problem is explained there in a code comment. The code block however is only executed for a CREATE txn, not for a multiTxn containing a CREATE.
> Even if the mentioned code block would also be executed for multi transactions, it needs adaption for multi transactions. What, if after the first failed transaction in a multi txn during deserialization, there would be subsequent transactions in the same multi that would also have failed?
> We don't know, since the first failed transaction hides the information about the remaining transactions.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (ZOOKEEPER-1269) Multi deserialization issues
Posted by "Marshall McMullen (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/ZOOKEEPER-1269?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13139352#comment-13139352 ]
Marshall McMullen commented on ZOOKEEPER-1269:
----------------------------------------------
I agree with Benjamin's comment. In the PrepRequestProcessor we only apply the multi-op if we know all the contained ops will succeed. If any of them fail, the whole thing gets thrown away and we rollback. So there should be no multi-op specific handling that needs to be done in your patch. I looked over the patch and I don't see any problems.
> Multi deserialization issues
> ----------------------------
>
> Key: ZOOKEEPER-1269
> URL: https://issues.apache.org/jira/browse/ZOOKEEPER-1269
> Project: ZooKeeper
> Issue Type: Bug
> Components: server
> Affects Versions: 3.4.0
> Reporter: Camille Fournier
> Attachments: ZOOKEEPER-1269.patch
>
>
> From the mailing list:
> FileTxnSnapLog.restore contains a code block handling a NODEEXISTS failure during deserialization. The problem is explained there in a code comment. The code block however is only executed for a CREATE txn, not for a multiTxn containing a CREATE.
> Even if the mentioned code block would also be executed for multi transactions, it needs adaption for multi transactions. What, if after the first failed transaction in a multi txn during deserialization, there would be subsequent transactions in the same multi that would also have failed?
> We don't know, since the first failed transaction hides the information about the remaining transactions.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Assigned] (ZOOKEEPER-1269) Multi deserialization issues
Posted by "Patrick Hunt (Assigned) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/ZOOKEEPER-1269?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Patrick Hunt reassigned ZOOKEEPER-1269:
---------------------------------------
Assignee: Camille Fournier
> Multi deserialization issues
> ----------------------------
>
> Key: ZOOKEEPER-1269
> URL: https://issues.apache.org/jira/browse/ZOOKEEPER-1269
> Project: ZooKeeper
> Issue Type: Bug
> Components: server
> Affects Versions: 3.4.0
> Reporter: Camille Fournier
> Assignee: Camille Fournier
> Attachments: ZOOKEEPER-1269.patch
>
>
> From the mailing list:
> FileTxnSnapLog.restore contains a code block handling a NODEEXISTS failure during deserialization. The problem is explained there in a code comment. The code block however is only executed for a CREATE txn, not for a multiTxn containing a CREATE.
> Even if the mentioned code block would also be executed for multi transactions, it needs adaption for multi transactions. What, if after the first failed transaction in a multi txn during deserialization, there would be subsequent transactions in the same multi that would also have failed?
> We don't know, since the first failed transaction hides the information about the remaining transactions.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (ZOOKEEPER-1269) Multi deserialization issues
Posted by "Hadoop QA (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/ZOOKEEPER-1269?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13165532#comment-13165532 ]
Hadoop QA commented on ZOOKEEPER-1269:
--------------------------------------
+1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12501375/ZOOKEEPER-1269.patch
against trunk revision 1208979.
+1 @author. The patch does not contain any @author tags.
+1 tests included. The patch appears to include 9 new or modified tests.
+1 javadoc. The javadoc tool did not generate any warning messages.
+1 javac. The applied patch does not increase the total number of javac compiler warnings.
+1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.
+1 release audit. The applied patch does not increase the total number of release audit warnings.
+1 core tests. The patch passed core unit tests.
+1 contrib tests. The patch passed contrib unit tests.
Test results: https://builds.apache.org/job/PreCommit-ZOOKEEPER-Build/815//testReport/
Findbugs warnings: https://builds.apache.org/job/PreCommit-ZOOKEEPER-Build/815//artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
Console output: https://builds.apache.org/job/PreCommit-ZOOKEEPER-Build/815//console
This message is automatically generated.
> Multi deserialization issues
> ----------------------------
>
> Key: ZOOKEEPER-1269
> URL: https://issues.apache.org/jira/browse/ZOOKEEPER-1269
> Project: ZooKeeper
> Issue Type: Bug
> Components: server
> Affects Versions: 3.4.0
> Reporter: Camille Fournier
> Assignee: Camille Fournier
> Fix For: 3.5.0, 3.4.1
>
> Attachments: ZOOKEEPER-1269.patch
>
>
> From the mailing list:
> FileTxnSnapLog.restore contains a code block handling a NODEEXISTS failure during deserialization. The problem is explained there in a code comment. The code block however is only executed for a CREATE txn, not for a multiTxn containing a CREATE.
> Even if the mentioned code block would also be executed for multi transactions, it needs adaption for multi transactions. What, if after the first failed transaction in a multi txn during deserialization, there would be subsequent transactions in the same multi that would also have failed?
> We don't know, since the first failed transaction hides the information about the remaining transactions.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira