You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@zookeeper.apache.org by "Botond Hejj (Created) (JIRA)" <ji...@apache.org> on 2012/04/17 18:16:23 UTC
[jira] [Created] (ZOOKEEPER-1448) Node+Quota creation in
transaction log can crash leader startup
Node+Quota creation in transaction log can crash leader startup
---------------------------------------------------------------
Key: ZOOKEEPER-1448
URL: https://issues.apache.org/jira/browse/ZOOKEEPER-1448
Project: ZooKeeper
Issue Type: Bug
Components: server
Affects Versions: 3.3.5
Reporter: Botond Hejj
Fix For: 3.3.6
Hi,
I've found a bug in zookeeper related to quota creation which can shutdown zookeeper leader on startup.
Steps to reproduce:
1. create /quota_bug
2. setquota -n 10000 /quota_bug
3. stop the whole ensemble (the previous operations should be in the transaction log)
4. start all the servers
5. the elected leader will shutdown with an exception (Missing stat node for count /zookeeper/quota/quota_bug/zookeeper_
stats)
I've debugged a bit what happening and I found the following problem:
On startup each server loads the last snapshot and replays the last transaction log. While doing this it fills up the pTrie variable of the DataTree with the path of the nodes which have quota.
After the leader is elected the leader servers loads the snapshot and last transaction log but it doesn't clean up the pTrie variable. This means it still contains the "/quota_bug" path. Now when the "create /quota_bug" is processed from the transaction log the DataTree already thinks that the quota nodes ("/zookeeper/quota/quota_bug/zookeeper_limits" and "/zookeeper/quota/quota_bug/zookeeper_stats") are created but those node creation actually comes later in the transaction log. This leads to the missing stat node exception.
I think clearing the pTrie should solve this problem.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (ZOOKEEPER-1448) Node+Quota creation in
transaction log can crash leader startup
Posted by "Camille Fournier (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/ZOOKEEPER-1448?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13255749#comment-13255749 ]
Camille Fournier commented on ZOOKEEPER-1448:
---------------------------------------------
Good catch. Can you provide a patch for this?
> Node+Quota creation in transaction log can crash leader startup
> ---------------------------------------------------------------
>
> Key: ZOOKEEPER-1448
> URL: https://issues.apache.org/jira/browse/ZOOKEEPER-1448
> Project: ZooKeeper
> Issue Type: Bug
> Components: server
> Affects Versions: 3.3.5
> Reporter: Botond Hejj
> Fix For: 3.3.6
>
>
> Hi,
> I've found a bug in zookeeper related to quota creation which can shutdown zookeeper leader on startup.
> Steps to reproduce:
> 1. create /quota_bug
> 2. setquota -n 10000 /quota_bug
> 3. stop the whole ensemble (the previous operations should be in the transaction log)
> 4. start all the servers
> 5. the elected leader will shutdown with an exception (Missing stat node for count /zookeeper/quota/quota_bug/zookeeper_
> stats)
> I've debugged a bit what happening and I found the following problem:
> On startup each server loads the last snapshot and replays the last transaction log. While doing this it fills up the pTrie variable of the DataTree with the path of the nodes which have quota.
> After the leader is elected the leader servers loads the snapshot and last transaction log but it doesn't clean up the pTrie variable. This means it still contains the "/quota_bug" path. Now when the "create /quota_bug" is processed from the transaction log the DataTree already thinks that the quota nodes ("/zookeeper/quota/quota_bug/zookeeper_limits" and "/zookeeper/quota/quota_bug/zookeeper_stats") are created but those node creation actually comes later in the transaction log. This leads to the missing stat node exception.
> I think clearing the pTrie should solve this problem.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (ZOOKEEPER-1448) Node+Quota creation in
transaction log can crash leader startup
Posted by "Botond Hejj (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/ZOOKEEPER-1448?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Botond Hejj updated ZOOKEEPER-1448:
-----------------------------------
Attachment: ZOOKEEPER-1448_branch3.3.patch
I've attached a patch for 3.3 branch.
I use log4j because I want to test whether a line is written to the logs or not. I use for that a custom writeappender as I've seen in another testcase. I don't think I could do the same with slf4j.
> Node+Quota creation in transaction log can crash leader startup
> ---------------------------------------------------------------
>
> Key: ZOOKEEPER-1448
> URL: https://issues.apache.org/jira/browse/ZOOKEEPER-1448
> Project: ZooKeeper
> Issue Type: Bug
> Components: server
> Affects Versions: 3.3.5
> Reporter: Botond Hejj
> Assignee: Botond Hejj
> Priority: Critical
> Fix For: 3.3.6, 3.4.4, 3.5.0
>
> Attachments: ZOOKEEPER-1448.patch, ZOOKEEPER-1448.patch, ZOOKEEPER-1448.patch, ZOOKEEPER-1448.patch, ZOOKEEPER-1448_branch3.3.patch
>
>
> Hi,
> I've found a bug in zookeeper related to quota creation which can shutdown zookeeper leader on startup.
> Steps to reproduce:
> 1. create /quota_bug
> 2. setquota -n 10000 /quota_bug
> 3. stop the whole ensemble (the previous operations should be in the transaction log)
> 4. start all the servers
> 5. the elected leader will shutdown with an exception (Missing stat node for count /zookeeper/quota/quota_bug/zookeeper_
> stats)
> I've debugged a bit what happening and I found the following problem:
> On startup each server loads the last snapshot and replays the last transaction log. While doing this it fills up the pTrie variable of the DataTree with the path of the nodes which have quota.
> After the leader is elected the leader servers loads the snapshot and last transaction log but it doesn't clean up the pTrie variable. This means it still contains the "/quota_bug" path. Now when the "create /quota_bug" is processed from the transaction log the DataTree already thinks that the quota nodes ("/zookeeper/quota/quota_bug/zookeeper_limits" and "/zookeeper/quota/quota_bug/zookeeper_stats") are created but those node creation actually comes later in the transaction log. This leads to the missing stat node exception.
> I think clearing the pTrie should solve this problem.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (ZOOKEEPER-1448) Node+Quota creation in
transaction log can crash leader startup
Posted by "Mahadev konar (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/ZOOKEEPER-1448?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Mahadev konar updated ZOOKEEPER-1448:
-------------------------------------
Fix Version/s: (was: 3.4.4)
3.4.5
> Node+Quota creation in transaction log can crash leader startup
> ---------------------------------------------------------------
>
> Key: ZOOKEEPER-1448
> URL: https://issues.apache.org/jira/browse/ZOOKEEPER-1448
> Project: ZooKeeper
> Issue Type: Bug
> Components: server
> Affects Versions: 3.3.5
> Reporter: Botond Hejj
> Assignee: Botond Hejj
> Priority: Critical
> Fix For: 3.5.0, 3.3.7, 3.4.5
>
> Attachments: ZOOKEEPER-1448_branch3.3.patch, ZOOKEEPER-1448.patch, ZOOKEEPER-1448.patch, ZOOKEEPER-1448.patch, ZOOKEEPER-1448.patch
>
>
> Hi,
> I've found a bug in zookeeper related to quota creation which can shutdown zookeeper leader on startup.
> Steps to reproduce:
> 1. create /quota_bug
> 2. setquota -n 10000 /quota_bug
> 3. stop the whole ensemble (the previous operations should be in the transaction log)
> 4. start all the servers
> 5. the elected leader will shutdown with an exception (Missing stat node for count /zookeeper/quota/quota_bug/zookeeper_
> stats)
> I've debugged a bit what happening and I found the following problem:
> On startup each server loads the last snapshot and replays the last transaction log. While doing this it fills up the pTrie variable of the DataTree with the path of the nodes which have quota.
> After the leader is elected the leader servers loads the snapshot and last transaction log but it doesn't clean up the pTrie variable. This means it still contains the "/quota_bug" path. Now when the "create /quota_bug" is processed from the transaction log the DataTree already thinks that the quota nodes ("/zookeeper/quota/quota_bug/zookeeper_limits" and "/zookeeper/quota/quota_bug/zookeeper_stats") are created but those node creation actually comes later in the transaction log. This leads to the missing stat node exception.
> I think clearing the pTrie should solve this problem.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (ZOOKEEPER-1448) Node+Quota creation in
transaction log can crash leader startup
Posted by "Hadoop QA (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/ZOOKEEPER-1448?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13272192#comment-13272192 ]
Hadoop QA commented on ZOOKEEPER-1448:
--------------------------------------
-1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12525994/ZOOKEEPER-1448.patch
against trunk revision 1336467.
+1 @author. The patch does not contain any @author tags.
+1 tests included. The patch appears to include 3 new or modified tests.
-1 patch. The patch command could not apply the patch.
Console output: https://builds.apache.org/job/PreCommit-ZOOKEEPER-Build/1068//console
This message is automatically generated.
> Node+Quota creation in transaction log can crash leader startup
> ---------------------------------------------------------------
>
> Key: ZOOKEEPER-1448
> URL: https://issues.apache.org/jira/browse/ZOOKEEPER-1448
> Project: ZooKeeper
> Issue Type: Bug
> Components: server
> Affects Versions: 3.3.5
> Reporter: Botond Hejj
> Assignee: Botond Hejj
> Fix For: 3.3.6, 3.4.4, 3.5.0
>
> Attachments: ZOOKEEPER-1448.patch, ZOOKEEPER-1448.patch
>
>
> Hi,
> I've found a bug in zookeeper related to quota creation which can shutdown zookeeper leader on startup.
> Steps to reproduce:
> 1. create /quota_bug
> 2. setquota -n 10000 /quota_bug
> 3. stop the whole ensemble (the previous operations should be in the transaction log)
> 4. start all the servers
> 5. the elected leader will shutdown with an exception (Missing stat node for count /zookeeper/quota/quota_bug/zookeeper_
> stats)
> I've debugged a bit what happening and I found the following problem:
> On startup each server loads the last snapshot and replays the last transaction log. While doing this it fills up the pTrie variable of the DataTree with the path of the nodes which have quota.
> After the leader is elected the leader servers loads the snapshot and last transaction log but it doesn't clean up the pTrie variable. This means it still contains the "/quota_bug" path. Now when the "create /quota_bug" is processed from the transaction log the DataTree already thinks that the quota nodes ("/zookeeper/quota/quota_bug/zookeeper_limits" and "/zookeeper/quota/quota_bug/zookeeper_stats") are created but those node creation actually comes later in the transaction log. This leads to the missing stat node exception.
> I think clearing the pTrie should solve this problem.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (ZOOKEEPER-1448) Node+Quota creation in
transaction log can crash leader startup
Posted by "Patrick Hunt (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/ZOOKEEPER-1448?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Patrick Hunt updated ZOOKEEPER-1448:
------------------------------------
Fix Version/s: (was: 3.3.6)
3.3.7
> Node+Quota creation in transaction log can crash leader startup
> ---------------------------------------------------------------
>
> Key: ZOOKEEPER-1448
> URL: https://issues.apache.org/jira/browse/ZOOKEEPER-1448
> Project: ZooKeeper
> Issue Type: Bug
> Components: server
> Affects Versions: 3.3.5
> Reporter: Botond Hejj
> Assignee: Botond Hejj
> Priority: Critical
> Fix For: 3.4.4, 3.5.0, 3.3.7
>
> Attachments: ZOOKEEPER-1448.patch, ZOOKEEPER-1448.patch, ZOOKEEPER-1448.patch, ZOOKEEPER-1448.patch, ZOOKEEPER-1448_branch3.3.patch
>
>
> Hi,
> I've found a bug in zookeeper related to quota creation which can shutdown zookeeper leader on startup.
> Steps to reproduce:
> 1. create /quota_bug
> 2. setquota -n 10000 /quota_bug
> 3. stop the whole ensemble (the previous operations should be in the transaction log)
> 4. start all the servers
> 5. the elected leader will shutdown with an exception (Missing stat node for count /zookeeper/quota/quota_bug/zookeeper_
> stats)
> I've debugged a bit what happening and I found the following problem:
> On startup each server loads the last snapshot and replays the last transaction log. While doing this it fills up the pTrie variable of the DataTree with the path of the nodes which have quota.
> After the leader is elected the leader servers loads the snapshot and last transaction log but it doesn't clean up the pTrie variable. This means it still contains the "/quota_bug" path. Now when the "create /quota_bug" is processed from the transaction log the DataTree already thinks that the quota nodes ("/zookeeper/quota/quota_bug/zookeeper_limits" and "/zookeeper/quota/quota_bug/zookeeper_stats") are created but those node creation actually comes later in the transaction log. This leads to the missing stat node exception.
> I think clearing the pTrie should solve this problem.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (ZOOKEEPER-1448) Node+Quota creation in
transaction log can crash leader startup
Posted by "Hadoop QA (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/ZOOKEEPER-1448?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13272560#comment-13272560 ]
Hadoop QA commented on ZOOKEEPER-1448:
--------------------------------------
+1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12526375/ZOOKEEPER-1448.patch
against trunk revision 1336467.
+1 @author. The patch does not contain any @author tags.
+1 tests included. The patch appears to include 3 new or modified tests.
+1 javadoc. The javadoc tool did not generate any warning messages.
+1 javac. The applied patch does not increase the total number of javac compiler warnings.
+1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.
+1 release audit. The applied patch does not increase the total number of release audit warnings.
+1 core tests. The patch passed core unit tests.
+1 contrib tests. The patch passed contrib unit tests.
Test results: https://builds.apache.org/job/PreCommit-ZOOKEEPER-Build/1071//testReport/
Findbugs warnings: https://builds.apache.org/job/PreCommit-ZOOKEEPER-Build/1071//artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
Console output: https://builds.apache.org/job/PreCommit-ZOOKEEPER-Build/1071//console
This message is automatically generated.
> Node+Quota creation in transaction log can crash leader startup
> ---------------------------------------------------------------
>
> Key: ZOOKEEPER-1448
> URL: https://issues.apache.org/jira/browse/ZOOKEEPER-1448
> Project: ZooKeeper
> Issue Type: Bug
> Components: server
> Affects Versions: 3.3.5
> Reporter: Botond Hejj
> Assignee: Botond Hejj
> Fix For: 3.3.6, 3.4.4, 3.5.0
>
> Attachments: ZOOKEEPER-1448.patch, ZOOKEEPER-1448.patch, ZOOKEEPER-1448.patch, ZOOKEEPER-1448.patch
>
>
> Hi,
> I've found a bug in zookeeper related to quota creation which can shutdown zookeeper leader on startup.
> Steps to reproduce:
> 1. create /quota_bug
> 2. setquota -n 10000 /quota_bug
> 3. stop the whole ensemble (the previous operations should be in the transaction log)
> 4. start all the servers
> 5. the elected leader will shutdown with an exception (Missing stat node for count /zookeeper/quota/quota_bug/zookeeper_
> stats)
> I've debugged a bit what happening and I found the following problem:
> On startup each server loads the last snapshot and replays the last transaction log. While doing this it fills up the pTrie variable of the DataTree with the path of the nodes which have quota.
> After the leader is elected the leader servers loads the snapshot and last transaction log but it doesn't clean up the pTrie variable. This means it still contains the "/quota_bug" path. Now when the "create /quota_bug" is processed from the transaction log the DataTree already thinks that the quota nodes ("/zookeeper/quota/quota_bug/zookeeper_limits" and "/zookeeper/quota/quota_bug/zookeeper_stats") are created but those node creation actually comes later in the transaction log. This leads to the missing stat node exception.
> I think clearing the pTrie should solve this problem.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (ZOOKEEPER-1448) Node+Quota creation in
transaction log can crash leader startup
Posted by "Michi Mutsuzaki (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/ZOOKEEPER-1448?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13270217#comment-13270217 ]
Michi Mutsuzaki commented on ZOOKEEPER-1448:
--------------------------------------------
Hi Botond,
Please let us know if you need help writing a test for this. I'm not familiar with this code path myself, but I'm pretty sure some other folks will be able to help.
Thanks!
--Michi
> Node+Quota creation in transaction log can crash leader startup
> ---------------------------------------------------------------
>
> Key: ZOOKEEPER-1448
> URL: https://issues.apache.org/jira/browse/ZOOKEEPER-1448
> Project: ZooKeeper
> Issue Type: Bug
> Components: server
> Affects Versions: 3.3.5
> Reporter: Botond Hejj
> Assignee: Botond Hejj
> Fix For: 3.3.6, 3.4.4, 3.5.0
>
> Attachments: ZOOKEEPER-1448.patch
>
>
> Hi,
> I've found a bug in zookeeper related to quota creation which can shutdown zookeeper leader on startup.
> Steps to reproduce:
> 1. create /quota_bug
> 2. setquota -n 10000 /quota_bug
> 3. stop the whole ensemble (the previous operations should be in the transaction log)
> 4. start all the servers
> 5. the elected leader will shutdown with an exception (Missing stat node for count /zookeeper/quota/quota_bug/zookeeper_
> stats)
> I've debugged a bit what happening and I found the following problem:
> On startup each server loads the last snapshot and replays the last transaction log. While doing this it fills up the pTrie variable of the DataTree with the path of the nodes which have quota.
> After the leader is elected the leader servers loads the snapshot and last transaction log but it doesn't clean up the pTrie variable. This means it still contains the "/quota_bug" path. Now when the "create /quota_bug" is processed from the transaction log the DataTree already thinks that the quota nodes ("/zookeeper/quota/quota_bug/zookeeper_limits" and "/zookeeper/quota/quota_bug/zookeeper_stats") are created but those node creation actually comes later in the transaction log. This leads to the missing stat node exception.
> I think clearing the pTrie should solve this problem.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (ZOOKEEPER-1448) Node+Quota creation in
transaction log can crash leader startup
Posted by "Botond Hejj (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/ZOOKEEPER-1448?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Botond Hejj updated ZOOKEEPER-1448:
-----------------------------------
Attachment: ZOOKEEPER-1448.patch
Fixed the test. Actually not an exception was thrown just an ERROR logged in this case with a "should not happen" comment :)
> Node+Quota creation in transaction log can crash leader startup
> ---------------------------------------------------------------
>
> Key: ZOOKEEPER-1448
> URL: https://issues.apache.org/jira/browse/ZOOKEEPER-1448
> Project: ZooKeeper
> Issue Type: Bug
> Components: server
> Affects Versions: 3.3.5
> Reporter: Botond Hejj
> Assignee: Botond Hejj
> Fix For: 3.3.6, 3.4.4, 3.5.0
>
> Attachments: ZOOKEEPER-1448.patch, ZOOKEEPER-1448.patch, ZOOKEEPER-1448.patch, ZOOKEEPER-1448.patch
>
>
> Hi,
> I've found a bug in zookeeper related to quota creation which can shutdown zookeeper leader on startup.
> Steps to reproduce:
> 1. create /quota_bug
> 2. setquota -n 10000 /quota_bug
> 3. stop the whole ensemble (the previous operations should be in the transaction log)
> 4. start all the servers
> 5. the elected leader will shutdown with an exception (Missing stat node for count /zookeeper/quota/quota_bug/zookeeper_
> stats)
> I've debugged a bit what happening and I found the following problem:
> On startup each server loads the last snapshot and replays the last transaction log. While doing this it fills up the pTrie variable of the DataTree with the path of the nodes which have quota.
> After the leader is elected the leader servers loads the snapshot and last transaction log but it doesn't clean up the pTrie variable. This means it still contains the "/quota_bug" path. Now when the "create /quota_bug" is processed from the transaction log the DataTree already thinks that the quota nodes ("/zookeeper/quota/quota_bug/zookeeper_limits" and "/zookeeper/quota/quota_bug/zookeeper_stats") are created but those node creation actually comes later in the transaction log. This leads to the missing stat node exception.
> I think clearing the pTrie should solve this problem.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (ZOOKEEPER-1448) Node+Quota creation in
transaction log can crash leader startup
Posted by "Botond Hejj (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/ZOOKEEPER-1448?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Botond Hejj updated ZOOKEEPER-1448:
-----------------------------------
Attachment: ZOOKEEPER-1448.patch
Patch to clear pTrie before DataTree deserialize
> Node+Quota creation in transaction log can crash leader startup
> ---------------------------------------------------------------
>
> Key: ZOOKEEPER-1448
> URL: https://issues.apache.org/jira/browse/ZOOKEEPER-1448
> Project: ZooKeeper
> Issue Type: Bug
> Components: server
> Affects Versions: 3.3.5
> Reporter: Botond Hejj
> Fix For: 3.3.6
>
> Attachments: ZOOKEEPER-1448.patch
>
>
> Hi,
> I've found a bug in zookeeper related to quota creation which can shutdown zookeeper leader on startup.
> Steps to reproduce:
> 1. create /quota_bug
> 2. setquota -n 10000 /quota_bug
> 3. stop the whole ensemble (the previous operations should be in the transaction log)
> 4. start all the servers
> 5. the elected leader will shutdown with an exception (Missing stat node for count /zookeeper/quota/quota_bug/zookeeper_
> stats)
> I've debugged a bit what happening and I found the following problem:
> On startup each server loads the last snapshot and replays the last transaction log. While doing this it fills up the pTrie variable of the DataTree with the path of the nodes which have quota.
> After the leader is elected the leader servers loads the snapshot and last transaction log but it doesn't clean up the pTrie variable. This means it still contains the "/quota_bug" path. Now when the "create /quota_bug" is processed from the transaction log the DataTree already thinks that the quota nodes ("/zookeeper/quota/quota_bug/zookeeper_limits" and "/zookeeper/quota/quota_bug/zookeeper_stats") are created but those node creation actually comes later in the transaction log. This leads to the missing stat node exception.
> I think clearing the pTrie should solve this problem.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (ZOOKEEPER-1448) Node+Quota creation in
transaction log can crash leader startup
Posted by "Hadoop QA (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/ZOOKEEPER-1448?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13272333#comment-13272333 ]
Hadoop QA commented on ZOOKEEPER-1448:
--------------------------------------
-1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12526341/ZOOKEEPER-1448.patch
against trunk revision 1336467.
+1 @author. The patch does not contain any @author tags.
+1 tests included. The patch appears to include 3 new or modified tests.
+1 javadoc. The javadoc tool did not generate any warning messages.
-1 javac. The patch appears to cause tar ant target to fail.
+1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.
+1 release audit. The applied patch does not increase the total number of release audit warnings.
-1 core tests. The patch failed core unit tests.
+1 contrib tests. The patch passed contrib unit tests.
Test results: https://builds.apache.org/job/PreCommit-ZOOKEEPER-Build/1069//testReport/
Findbugs warnings: https://builds.apache.org/job/PreCommit-ZOOKEEPER-Build/1069//artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
Console output: https://builds.apache.org/job/PreCommit-ZOOKEEPER-Build/1069//console
This message is automatically generated.
> Node+Quota creation in transaction log can crash leader startup
> ---------------------------------------------------------------
>
> Key: ZOOKEEPER-1448
> URL: https://issues.apache.org/jira/browse/ZOOKEEPER-1448
> Project: ZooKeeper
> Issue Type: Bug
> Components: server
> Affects Versions: 3.3.5
> Reporter: Botond Hejj
> Assignee: Botond Hejj
> Fix For: 3.3.6, 3.4.4, 3.5.0
>
> Attachments: ZOOKEEPER-1448.patch, ZOOKEEPER-1448.patch, ZOOKEEPER-1448.patch
>
>
> Hi,
> I've found a bug in zookeeper related to quota creation which can shutdown zookeeper leader on startup.
> Steps to reproduce:
> 1. create /quota_bug
> 2. setquota -n 10000 /quota_bug
> 3. stop the whole ensemble (the previous operations should be in the transaction log)
> 4. start all the servers
> 5. the elected leader will shutdown with an exception (Missing stat node for count /zookeeper/quota/quota_bug/zookeeper_
> stats)
> I've debugged a bit what happening and I found the following problem:
> On startup each server loads the last snapshot and replays the last transaction log. While doing this it fills up the pTrie variable of the DataTree with the path of the nodes which have quota.
> After the leader is elected the leader servers loads the snapshot and last transaction log but it doesn't clean up the pTrie variable. This means it still contains the "/quota_bug" path. Now when the "create /quota_bug" is processed from the transaction log the DataTree already thinks that the quota nodes ("/zookeeper/quota/quota_bug/zookeeper_limits" and "/zookeeper/quota/quota_bug/zookeeper_stats") are created but those node creation actually comes later in the transaction log. This leads to the missing stat node exception.
> I think clearing the pTrie should solve this problem.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (ZOOKEEPER-1448) Node+Quota creation in
transaction log can crash leader startup
Posted by "Patrick Hunt (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/ZOOKEEPER-1448?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Patrick Hunt updated ZOOKEEPER-1448:
------------------------------------
Priority: Critical (was: Major)
> Node+Quota creation in transaction log can crash leader startup
> ---------------------------------------------------------------
>
> Key: ZOOKEEPER-1448
> URL: https://issues.apache.org/jira/browse/ZOOKEEPER-1448
> Project: ZooKeeper
> Issue Type: Bug
> Components: server
> Affects Versions: 3.3.5
> Reporter: Botond Hejj
> Assignee: Botond Hejj
> Priority: Critical
> Fix For: 3.3.6, 3.4.4, 3.5.0
>
> Attachments: ZOOKEEPER-1448.patch, ZOOKEEPER-1448.patch, ZOOKEEPER-1448.patch, ZOOKEEPER-1448.patch
>
>
> Hi,
> I've found a bug in zookeeper related to quota creation which can shutdown zookeeper leader on startup.
> Steps to reproduce:
> 1. create /quota_bug
> 2. setquota -n 10000 /quota_bug
> 3. stop the whole ensemble (the previous operations should be in the transaction log)
> 4. start all the servers
> 5. the elected leader will shutdown with an exception (Missing stat node for count /zookeeper/quota/quota_bug/zookeeper_
> stats)
> I've debugged a bit what happening and I found the following problem:
> On startup each server loads the last snapshot and replays the last transaction log. While doing this it fills up the pTrie variable of the DataTree with the path of the nodes which have quota.
> After the leader is elected the leader servers loads the snapshot and last transaction log but it doesn't clean up the pTrie variable. This means it still contains the "/quota_bug" path. Now when the "create /quota_bug" is processed from the transaction log the DataTree already thinks that the quota nodes ("/zookeeper/quota/quota_bug/zookeeper_limits" and "/zookeeper/quota/quota_bug/zookeeper_stats") are created but those node creation actually comes later in the transaction log. This leads to the missing stat node exception.
> I think clearing the pTrie should solve this problem.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (ZOOKEEPER-1448) Node+Quota creation in
transaction log can crash leader startup
Posted by "Camille Fournier (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/ZOOKEEPER-1448?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13403755#comment-13403755 ]
Camille Fournier commented on ZOOKEEPER-1448:
---------------------------------------------
I looked at this and think it's good, will do another pass on it soon and check it in if it's ready. It would be great if you could give me a patch for the 3.3.X branch though so we can get it there as well.
> Node+Quota creation in transaction log can crash leader startup
> ---------------------------------------------------------------
>
> Key: ZOOKEEPER-1448
> URL: https://issues.apache.org/jira/browse/ZOOKEEPER-1448
> Project: ZooKeeper
> Issue Type: Bug
> Components: server
> Affects Versions: 3.3.5
> Reporter: Botond Hejj
> Assignee: Botond Hejj
> Fix For: 3.3.6, 3.4.4, 3.5.0
>
> Attachments: ZOOKEEPER-1448.patch, ZOOKEEPER-1448.patch, ZOOKEEPER-1448.patch, ZOOKEEPER-1448.patch
>
>
> Hi,
> I've found a bug in zookeeper related to quota creation which can shutdown zookeeper leader on startup.
> Steps to reproduce:
> 1. create /quota_bug
> 2. setquota -n 10000 /quota_bug
> 3. stop the whole ensemble (the previous operations should be in the transaction log)
> 4. start all the servers
> 5. the elected leader will shutdown with an exception (Missing stat node for count /zookeeper/quota/quota_bug/zookeeper_
> stats)
> I've debugged a bit what happening and I found the following problem:
> On startup each server loads the last snapshot and replays the last transaction log. While doing this it fills up the pTrie variable of the DataTree with the path of the nodes which have quota.
> After the leader is elected the leader servers loads the snapshot and last transaction log but it doesn't clean up the pTrie variable. This means it still contains the "/quota_bug" path. Now when the "create /quota_bug" is processed from the transaction log the DataTree already thinks that the quota nodes ("/zookeeper/quota/quota_bug/zookeeper_limits" and "/zookeeper/quota/quota_bug/zookeeper_stats") are created but those node creation actually comes later in the transaction log. This leads to the missing stat node exception.
> I think clearing the pTrie should solve this problem.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (ZOOKEEPER-1448) Node+Quota creation in
transaction log can crash leader startup
Posted by "Patrick Hunt (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/ZOOKEEPER-1448?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Patrick Hunt updated ZOOKEEPER-1448:
------------------------------------
Fix Version/s: 3.5.0
3.4.4
> Node+Quota creation in transaction log can crash leader startup
> ---------------------------------------------------------------
>
> Key: ZOOKEEPER-1448
> URL: https://issues.apache.org/jira/browse/ZOOKEEPER-1448
> Project: ZooKeeper
> Issue Type: Bug
> Components: server
> Affects Versions: 3.3.5
> Reporter: Botond Hejj
> Assignee: Botond Hejj
> Fix For: 3.3.6, 3.4.4, 3.5.0
>
> Attachments: ZOOKEEPER-1448.patch
>
>
> Hi,
> I've found a bug in zookeeper related to quota creation which can shutdown zookeeper leader on startup.
> Steps to reproduce:
> 1. create /quota_bug
> 2. setquota -n 10000 /quota_bug
> 3. stop the whole ensemble (the previous operations should be in the transaction log)
> 4. start all the servers
> 5. the elected leader will shutdown with an exception (Missing stat node for count /zookeeper/quota/quota_bug/zookeeper_
> stats)
> I've debugged a bit what happening and I found the following problem:
> On startup each server loads the last snapshot and replays the last transaction log. While doing this it fills up the pTrie variable of the DataTree with the path of the nodes which have quota.
> After the leader is elected the leader servers loads the snapshot and last transaction log but it doesn't clean up the pTrie variable. This means it still contains the "/quota_bug" path. Now when the "create /quota_bug" is processed from the transaction log the DataTree already thinks that the quota nodes ("/zookeeper/quota/quota_bug/zookeeper_limits" and "/zookeeper/quota/quota_bug/zookeeper_stats") are created but those node creation actually comes later in the transaction log. This leads to the missing stat node exception.
> I think clearing the pTrie should solve this problem.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (ZOOKEEPER-1448) Node+Quota creation in
transaction log can crash leader startup
Posted by "Hadoop QA (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/ZOOKEEPER-1448?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13257027#comment-13257027 ]
Hadoop QA commented on ZOOKEEPER-1448:
--------------------------------------
-1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12523193/ZOOKEEPER-1448.patch
against trunk revision 1326029.
+1 @author. The patch does not contain any @author tags.
-1 tests included. The patch doesn't appear to include any new or modified tests.
Please justify why no new tests are needed for this patch.
Also please list what manual steps were performed to verify this patch.
+1 javadoc. The javadoc tool did not generate any warning messages.
+1 javac. The applied patch does not increase the total number of javac compiler warnings.
+1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.
+1 release audit. The applied patch does not increase the total number of release audit warnings.
+1 core tests. The patch passed core unit tests.
+1 contrib tests. The patch passed contrib unit tests.
Test results: https://builds.apache.org/job/PreCommit-ZOOKEEPER-Build/1041//testReport/
Findbugs warnings: https://builds.apache.org/job/PreCommit-ZOOKEEPER-Build/1041//artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
Console output: https://builds.apache.org/job/PreCommit-ZOOKEEPER-Build/1041//console
This message is automatically generated.
> Node+Quota creation in transaction log can crash leader startup
> ---------------------------------------------------------------
>
> Key: ZOOKEEPER-1448
> URL: https://issues.apache.org/jira/browse/ZOOKEEPER-1448
> Project: ZooKeeper
> Issue Type: Bug
> Components: server
> Affects Versions: 3.3.5
> Reporter: Botond Hejj
> Fix For: 3.3.6
>
> Attachments: ZOOKEEPER-1448.patch
>
>
> Hi,
> I've found a bug in zookeeper related to quota creation which can shutdown zookeeper leader on startup.
> Steps to reproduce:
> 1. create /quota_bug
> 2. setquota -n 10000 /quota_bug
> 3. stop the whole ensemble (the previous operations should be in the transaction log)
> 4. start all the servers
> 5. the elected leader will shutdown with an exception (Missing stat node for count /zookeeper/quota/quota_bug/zookeeper_
> stats)
> I've debugged a bit what happening and I found the following problem:
> On startup each server loads the last snapshot and replays the last transaction log. While doing this it fills up the pTrie variable of the DataTree with the path of the nodes which have quota.
> After the leader is elected the leader servers loads the snapshot and last transaction log but it doesn't clean up the pTrie variable. This means it still contains the "/quota_bug" path. Now when the "create /quota_bug" is processed from the transaction log the DataTree already thinks that the quota nodes ("/zookeeper/quota/quota_bug/zookeeper_limits" and "/zookeeper/quota/quota_bug/zookeeper_stats") are created but those node creation actually comes later in the transaction log. This leads to the missing stat node exception.
> I think clearing the pTrie should solve this problem.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (ZOOKEEPER-1448) Node+Quota creation in
transaction log can crash leader startup
Posted by "Botond Hejj (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/ZOOKEEPER-1448?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Botond Hejj updated ZOOKEEPER-1448:
-----------------------------------
Attachment: ZOOKEEPER-1448.patch
I've added a test to DataTreeTest.java. Tell me if that is not the right place.
> Node+Quota creation in transaction log can crash leader startup
> ---------------------------------------------------------------
>
> Key: ZOOKEEPER-1448
> URL: https://issues.apache.org/jira/browse/ZOOKEEPER-1448
> Project: ZooKeeper
> Issue Type: Bug
> Components: server
> Affects Versions: 3.3.5
> Reporter: Botond Hejj
> Assignee: Botond Hejj
> Fix For: 3.3.6, 3.4.4, 3.5.0
>
> Attachments: ZOOKEEPER-1448.patch, ZOOKEEPER-1448.patch
>
>
> Hi,
> I've found a bug in zookeeper related to quota creation which can shutdown zookeeper leader on startup.
> Steps to reproduce:
> 1. create /quota_bug
> 2. setquota -n 10000 /quota_bug
> 3. stop the whole ensemble (the previous operations should be in the transaction log)
> 4. start all the servers
> 5. the elected leader will shutdown with an exception (Missing stat node for count /zookeeper/quota/quota_bug/zookeeper_
> stats)
> I've debugged a bit what happening and I found the following problem:
> On startup each server loads the last snapshot and replays the last transaction log. While doing this it fills up the pTrie variable of the DataTree with the path of the nodes which have quota.
> After the leader is elected the leader servers loads the snapshot and last transaction log but it doesn't clean up the pTrie variable. This means it still contains the "/quota_bug" path. Now when the "create /quota_bug" is processed from the transaction log the DataTree already thinks that the quota nodes ("/zookeeper/quota/quota_bug/zookeeper_limits" and "/zookeeper/quota/quota_bug/zookeeper_stats") are created but those node creation actually comes later in the transaction log. This leads to the missing stat node exception.
> I think clearing the pTrie should solve this problem.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (ZOOKEEPER-1448) Node+Quota creation in
transaction log can crash leader startup
Posted by "Mahadev konar (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/ZOOKEEPER-1448?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Mahadev konar updated ZOOKEEPER-1448:
-------------------------------------
I looked at the patch. THe patch looks good except for the point that Pat mentioned above that its moving the test toward log4j than using slf4j. For now I am moving this out to 3.4.5 for getting it done right. Botond, if you have sometime would you please update the patch using slf4j in the testcase.
> Node+Quota creation in transaction log can crash leader startup
> ---------------------------------------------------------------
>
> Key: ZOOKEEPER-1448
> URL: https://issues.apache.org/jira/browse/ZOOKEEPER-1448
> Project: ZooKeeper
> Issue Type: Bug
> Components: server
> Affects Versions: 3.3.5
> Reporter: Botond Hejj
> Assignee: Botond Hejj
> Priority: Critical
> Fix For: 3.5.0, 3.3.7, 3.4.5
>
> Attachments: ZOOKEEPER-1448_branch3.3.patch, ZOOKEEPER-1448.patch, ZOOKEEPER-1448.patch, ZOOKEEPER-1448.patch, ZOOKEEPER-1448.patch
>
>
> Hi,
> I've found a bug in zookeeper related to quota creation which can shutdown zookeeper leader on startup.
> Steps to reproduce:
> 1. create /quota_bug
> 2. setquota -n 10000 /quota_bug
> 3. stop the whole ensemble (the previous operations should be in the transaction log)
> 4. start all the servers
> 5. the elected leader will shutdown with an exception (Missing stat node for count /zookeeper/quota/quota_bug/zookeeper_
> stats)
> I've debugged a bit what happening and I found the following problem:
> On startup each server loads the last snapshot and replays the last transaction log. While doing this it fills up the pTrie variable of the DataTree with the path of the nodes which have quota.
> After the leader is elected the leader servers loads the snapshot and last transaction log but it doesn't clean up the pTrie variable. This means it still contains the "/quota_bug" path. Now when the "create /quota_bug" is processed from the transaction log the DataTree already thinks that the quota nodes ("/zookeeper/quota/quota_bug/zookeeper_limits" and "/zookeeper/quota/quota_bug/zookeeper_stats") are created but those node creation actually comes later in the transaction log. This leads to the missing stat node exception.
> I think clearing the pTrie should solve this problem.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (ZOOKEEPER-1448) Node+Quota creation in
transaction log can crash leader startup
Posted by "Patrick Hunt (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/ZOOKEEPER-1448?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13271899#comment-13271899 ]
Patrick Hunt commented on ZOOKEEPER-1448:
-----------------------------------------
Is this ready for review? If so please click "submit patch" button. Thanks.
> Node+Quota creation in transaction log can crash leader startup
> ---------------------------------------------------------------
>
> Key: ZOOKEEPER-1448
> URL: https://issues.apache.org/jira/browse/ZOOKEEPER-1448
> Project: ZooKeeper
> Issue Type: Bug
> Components: server
> Affects Versions: 3.3.5
> Reporter: Botond Hejj
> Assignee: Botond Hejj
> Fix For: 3.3.6, 3.4.4, 3.5.0
>
> Attachments: ZOOKEEPER-1448.patch, ZOOKEEPER-1448.patch
>
>
> Hi,
> I've found a bug in zookeeper related to quota creation which can shutdown zookeeper leader on startup.
> Steps to reproduce:
> 1. create /quota_bug
> 2. setquota -n 10000 /quota_bug
> 3. stop the whole ensemble (the previous operations should be in the transaction log)
> 4. start all the servers
> 5. the elected leader will shutdown with an exception (Missing stat node for count /zookeeper/quota/quota_bug/zookeeper_
> stats)
> I've debugged a bit what happening and I found the following problem:
> On startup each server loads the last snapshot and replays the last transaction log. While doing this it fills up the pTrie variable of the DataTree with the path of the nodes which have quota.
> After the leader is elected the leader servers loads the snapshot and last transaction log but it doesn't clean up the pTrie variable. This means it still contains the "/quota_bug" path. Now when the "create /quota_bug" is processed from the transaction log the DataTree already thinks that the quota nodes ("/zookeeper/quota/quota_bug/zookeeper_limits" and "/zookeeper/quota/quota_bug/zookeeper_stats") are created but those node creation actually comes later in the transaction log. This leads to the missing stat node exception.
> I think clearing the pTrie should solve this problem.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (ZOOKEEPER-1448) Node+Quota creation in
transaction log can crash leader startup
Posted by "Botond Hejj (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/ZOOKEEPER-1448?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Botond Hejj updated ZOOKEEPER-1448:
-----------------------------------
Attachment: ZOOKEEPER-1448.patch
My last patch was for 3.3 branch. I've made one for the trunk as well
> Node+Quota creation in transaction log can crash leader startup
> ---------------------------------------------------------------
>
> Key: ZOOKEEPER-1448
> URL: https://issues.apache.org/jira/browse/ZOOKEEPER-1448
> Project: ZooKeeper
> Issue Type: Bug
> Components: server
> Affects Versions: 3.3.5
> Reporter: Botond Hejj
> Assignee: Botond Hejj
> Fix For: 3.3.6, 3.4.4, 3.5.0
>
> Attachments: ZOOKEEPER-1448.patch, ZOOKEEPER-1448.patch, ZOOKEEPER-1448.patch
>
>
> Hi,
> I've found a bug in zookeeper related to quota creation which can shutdown zookeeper leader on startup.
> Steps to reproduce:
> 1. create /quota_bug
> 2. setquota -n 10000 /quota_bug
> 3. stop the whole ensemble (the previous operations should be in the transaction log)
> 4. start all the servers
> 5. the elected leader will shutdown with an exception (Missing stat node for count /zookeeper/quota/quota_bug/zookeeper_
> stats)
> I've debugged a bit what happening and I found the following problem:
> On startup each server loads the last snapshot and replays the last transaction log. While doing this it fills up the pTrie variable of the DataTree with the path of the nodes which have quota.
> After the leader is elected the leader servers loads the snapshot and last transaction log but it doesn't clean up the pTrie variable. This means it still contains the "/quota_bug" path. Now when the "create /quota_bug" is processed from the transaction log the DataTree already thinks that the quota nodes ("/zookeeper/quota/quota_bug/zookeeper_limits" and "/zookeeper/quota/quota_bug/zookeeper_stats") are created but those node creation actually comes later in the transaction log. This leads to the missing stat node exception.
> I think clearing the pTrie should solve this problem.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira