You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@bookkeeper.apache.org by "Ivan Kelly (JIRA)" <ji...@apache.org> on 2012/10/04 19:27:50 UTC
[jira] [Created] (BOOKKEEPER-417) Hierarchical zk underreplication
manager should clean up its hierarchy when done to allow for fast
acquisition of underreplicated entries
Ivan Kelly created BOOKKEEPER-417:
-------------------------------------
Summary: Hierarchical zk underreplication manager should clean up its hierarchy when done to allow for fast acquisition of underreplicated entries
Key: BOOKKEEPER-417
URL: https://issues.apache.org/jira/browse/BOOKKEEPER-417
Project: Bookkeeper
Issue Type: Sub-task
Reporter: Ivan Kelly
Assignee: Ivan Kelly
Fix For: 4.2.0
As we traverse the hierarchy to find a underreplicated fragment, leaving the hierarchy will cause the traversal to search in many empty znodes.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (BOOKKEEPER-417) Hierarchical zk
underreplication manager should clean up its hierarchy when done to allow
for fast acquisition of underreplicated entries
Posted by "Uma Maheswara Rao G (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/BOOKKEEPER-417?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13469570#comment-13469570 ]
Uma Maheswara Rao G commented on BOOKKEEPER-417:
------------------------------------------------
Yeah, I have seen this while debugging. Thanks for filing the JIRA. I think, functionally no harm, but we have to fix this to avoid unnecessary traversals.
> Hierarchical zk underreplication manager should clean up its hierarchy when done to allow for fast acquisition of underreplicated entries
> -----------------------------------------------------------------------------------------------------------------------------------------
>
> Key: BOOKKEEPER-417
> URL: https://issues.apache.org/jira/browse/BOOKKEEPER-417
> Project: Bookkeeper
> Issue Type: Sub-task
> Components: bookkeeper-auto-recovery
> Reporter: Ivan Kelly
> Assignee: Ivan Kelly
> Fix For: 4.2.0
>
>
> As we traverse the hierarchy to find a underreplicated fragment, leaving the hierarchy will cause the traversal to search in many empty znodes.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (BOOKKEEPER-417) Hierarchical zk
underreplication manager should clean up its hierarchy when done to allow
for fast acquisition of underreplicated entries
Posted by "Hudson (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/BOOKKEEPER-417?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13477186#comment-13477186 ]
Hudson commented on BOOKKEEPER-417:
-----------------------------------
Integrated in bookkeeper-trunk #755 (See [https://builds.apache.org/job/bookkeeper-trunk/755/])
BOOKKEEPER-417: Hierarchical zk underreplication manager should clean up its hierarchy when done to allow for fast acquisition of underreplicated entries (ivank) (Revision 1398834)
Result = UNSTABLE
ivank :
Files :
* /zookeeper/bookkeeper/trunk/CHANGES.txt
* /zookeeper/bookkeeper/trunk/bookkeeper-server/pom.xml
* /zookeeper/bookkeeper/trunk/bookkeeper-server/src/main/java/org/apache/bookkeeper/meta/ZkLedgerUnderreplicationManager.java
* /zookeeper/bookkeeper/trunk/bookkeeper-server/src/test/java/org/apache/bookkeeper/replication/TestLedgerUnderreplicationManager.java
* /zookeeper/bookkeeper/trunk/bookkeeper-server/src/test/java/org/apache/bookkeeper/replication/TestReplicationWorker.java
> Hierarchical zk underreplication manager should clean up its hierarchy when done to allow for fast acquisition of underreplicated entries
> -----------------------------------------------------------------------------------------------------------------------------------------
>
> Key: BOOKKEEPER-417
> URL: https://issues.apache.org/jira/browse/BOOKKEEPER-417
> Project: Bookkeeper
> Issue Type: Sub-task
> Components: bookkeeper-auto-recovery
> Reporter: Ivan Kelly
> Assignee: Ivan Kelly
> Fix For: 4.2.0
>
> Attachments: BOOKKEEPER-417.diff, BOOKKEEPER-417.diff, BOOKKEEPER-417.diff
>
>
> As we traverse the hierarchy to find a underreplicated fragment, leaving the hierarchy will cause the traversal to search in many empty znodes.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (BOOKKEEPER-417) Hierarchical zk
underreplication manager should clean up its hierarchy when done to allow
for fast acquisition of underreplicated entries
Posted by "Ivan Kelly (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/BOOKKEEPER-417?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13477067#comment-13477067 ]
Ivan Kelly commented on BOOKKEEPER-417:
---------------------------------------
Committed as r1398834. Thanks guys.
> Hierarchical zk underreplication manager should clean up its hierarchy when done to allow for fast acquisition of underreplicated entries
> -----------------------------------------------------------------------------------------------------------------------------------------
>
> Key: BOOKKEEPER-417
> URL: https://issues.apache.org/jira/browse/BOOKKEEPER-417
> Project: Bookkeeper
> Issue Type: Sub-task
> Components: bookkeeper-auto-recovery
> Reporter: Ivan Kelly
> Assignee: Ivan Kelly
> Fix For: 4.2.0
>
> Attachments: BOOKKEEPER-417.diff, BOOKKEEPER-417.diff, BOOKKEEPER-417.diff
>
>
> As we traverse the hierarchy to find a underreplicated fragment, leaving the hierarchy will cause the traversal to search in many empty znodes.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (BOOKKEEPER-417) Hierarchical zk underreplication
manager should clean up its hierarchy when done to allow for fast
acquisition of underreplicated entries
Posted by "Ivan Kelly (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/BOOKKEEPER-417?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Ivan Kelly updated BOOKKEEPER-417:
----------------------------------
Attachment: BOOKKEEPER-417.diff
> Hierarchical zk underreplication manager should clean up its hierarchy when done to allow for fast acquisition of underreplicated entries
> -----------------------------------------------------------------------------------------------------------------------------------------
>
> Key: BOOKKEEPER-417
> URL: https://issues.apache.org/jira/browse/BOOKKEEPER-417
> Project: Bookkeeper
> Issue Type: Sub-task
> Components: bookkeeper-auto-recovery
> Reporter: Ivan Kelly
> Assignee: Ivan Kelly
> Fix For: 4.2.0
>
> Attachments: BOOKKEEPER-417.diff
>
>
> As we traverse the hierarchy to find a underreplicated fragment, leaving the hierarchy will cause the traversal to search in many empty znodes.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (BOOKKEEPER-417) Hierarchical zk underreplication
manager should clean up its hierarchy when done to allow for fast
acquisition of underreplicated entries
Posted by "Ivan Kelly (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/BOOKKEEPER-417?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Ivan Kelly updated BOOKKEEPER-417:
----------------------------------
Attachment: BOOKKEEPER-417.diff
Last patch had a race in TestReplicationWorker. Fixed in latest.
> Hierarchical zk underreplication manager should clean up its hierarchy when done to allow for fast acquisition of underreplicated entries
> -----------------------------------------------------------------------------------------------------------------------------------------
>
> Key: BOOKKEEPER-417
> URL: https://issues.apache.org/jira/browse/BOOKKEEPER-417
> Project: Bookkeeper
> Issue Type: Sub-task
> Components: bookkeeper-auto-recovery
> Reporter: Ivan Kelly
> Assignee: Ivan Kelly
> Fix For: 4.2.0
>
> Attachments: BOOKKEEPER-417.diff, BOOKKEEPER-417.diff, BOOKKEEPER-417.diff
>
>
> As we traverse the hierarchy to find a underreplicated fragment, leaving the hierarchy will cause the traversal to search in many empty znodes.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (BOOKKEEPER-417) Hierarchical zk
underreplication manager should clean up its hierarchy when done to allow
for fast acquisition of underreplicated entries
Posted by "Rakesh R (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/BOOKKEEPER-417?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13472210#comment-13472210 ]
Rakesh R commented on BOOKKEEPER-417:
-------------------------------------
I mean like following in tests and timeout could be an ideal value, so will not hang at all.
{code}
@Test(timeout = 50000)
public void testHierarchyCleanupInterference ()
{code}
> Hierarchical zk underreplication manager should clean up its hierarchy when done to allow for fast acquisition of underreplicated entries
> -----------------------------------------------------------------------------------------------------------------------------------------
>
> Key: BOOKKEEPER-417
> URL: https://issues.apache.org/jira/browse/BOOKKEEPER-417
> Project: Bookkeeper
> Issue Type: Sub-task
> Components: bookkeeper-auto-recovery
> Reporter: Ivan Kelly
> Assignee: Ivan Kelly
> Fix For: 4.2.0
>
> Attachments: BOOKKEEPER-417.diff
>
>
> As we traverse the hierarchy to find a underreplicated fragment, leaving the hierarchy will cause the traversal to search in many empty znodes.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (BOOKKEEPER-417) Hierarchical zk
underreplication manager should clean up its hierarchy when done to allow
for fast acquisition of underreplicated entries
Posted by "Ivan Kelly (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/BOOKKEEPER-417?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13470184#comment-13470184 ]
Ivan Kelly commented on BOOKKEEPER-417:
---------------------------------------
Also, I already have this implemented, I just need to break out the patch.
> Hierarchical zk underreplication manager should clean up its hierarchy when done to allow for fast acquisition of underreplicated entries
> -----------------------------------------------------------------------------------------------------------------------------------------
>
> Key: BOOKKEEPER-417
> URL: https://issues.apache.org/jira/browse/BOOKKEEPER-417
> Project: Bookkeeper
> Issue Type: Sub-task
> Components: bookkeeper-auto-recovery
> Reporter: Ivan Kelly
> Assignee: Ivan Kelly
> Fix For: 4.2.0
>
>
> As we traverse the hierarchy to find a underreplicated fragment, leaving the hierarchy will cause the traversal to search in many empty znodes.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (BOOKKEEPER-417) Hierarchical zk
underreplication manager should clean up its hierarchy when done to allow
for fast acquisition of underreplicated entries
Posted by "Uma Maheswara Rao G (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/BOOKKEEPER-417?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13473361#comment-13473361 ]
Uma Maheswara Rao G commented on BOOKKEEPER-417:
------------------------------------------------
+1 patch looks good to me. Thanks Ivan.
> Hierarchical zk underreplication manager should clean up its hierarchy when done to allow for fast acquisition of underreplicated entries
> -----------------------------------------------------------------------------------------------------------------------------------------
>
> Key: BOOKKEEPER-417
> URL: https://issues.apache.org/jira/browse/BOOKKEEPER-417
> Project: Bookkeeper
> Issue Type: Sub-task
> Components: bookkeeper-auto-recovery
> Reporter: Ivan Kelly
> Assignee: Ivan Kelly
> Fix For: 4.2.0
>
> Attachments: BOOKKEEPER-417.diff, BOOKKEEPER-417.diff
>
>
> As we traverse the hierarchy to find a underreplicated fragment, leaving the hierarchy will cause the traversal to search in many empty znodes.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (BOOKKEEPER-417) Hierarchical zk
underreplication manager should clean up its hierarchy when done to allow
for fast acquisition of underreplicated entries
Posted by "Rakesh R (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/BOOKKEEPER-417?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13473785#comment-13473785 ]
Rakesh R commented on BOOKKEEPER-417:
-------------------------------------
+1 New patch looks good. Thanks Ivan.
> Hierarchical zk underreplication manager should clean up its hierarchy when done to allow for fast acquisition of underreplicated entries
> -----------------------------------------------------------------------------------------------------------------------------------------
>
> Key: BOOKKEEPER-417
> URL: https://issues.apache.org/jira/browse/BOOKKEEPER-417
> Project: Bookkeeper
> Issue Type: Sub-task
> Components: bookkeeper-auto-recovery
> Reporter: Ivan Kelly
> Assignee: Ivan Kelly
> Fix For: 4.2.0
>
> Attachments: BOOKKEEPER-417.diff, BOOKKEEPER-417.diff
>
>
> As we traverse the hierarchy to find a underreplicated fragment, leaving the hierarchy will cause the traversal to search in many empty znodes.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (BOOKKEEPER-417) Hierarchical zk
underreplication manager should clean up its hierarchy when done to allow
for fast acquisition of underreplicated entries
Posted by "Rakesh R (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/BOOKKEEPER-417?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13471737#comment-13471737 ]
Rakesh R commented on BOOKKEEPER-417:
-------------------------------------
Thanks Ivan for introducing guava.
Patch looks nice and just one thought, its safe and good to configure timeout parameter for testHierarchyCleanupInterference as it has many loops and join.
> Hierarchical zk underreplication manager should clean up its hierarchy when done to allow for fast acquisition of underreplicated entries
> -----------------------------------------------------------------------------------------------------------------------------------------
>
> Key: BOOKKEEPER-417
> URL: https://issues.apache.org/jira/browse/BOOKKEEPER-417
> Project: Bookkeeper
> Issue Type: Sub-task
> Components: bookkeeper-auto-recovery
> Reporter: Ivan Kelly
> Assignee: Ivan Kelly
> Fix For: 4.2.0
>
> Attachments: BOOKKEEPER-417.diff
>
>
> As we traverse the hierarchy to find a underreplicated fragment, leaving the hierarchy will cause the traversal to search in many empty znodes.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (BOOKKEEPER-417) Hierarchical zk
underreplication manager should clean up its hierarchy when done to allow
for fast acquisition of underreplicated entries
Posted by "Ivan Kelly (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/BOOKKEEPER-417?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13470183#comment-13470183 ]
Ivan Kelly commented on BOOKKEEPER-417:
---------------------------------------
Yes, when I was running some benchmarks, this really slowed down recovery time if the number of ledgers being recovered was large (10,000+).
> Hierarchical zk underreplication manager should clean up its hierarchy when done to allow for fast acquisition of underreplicated entries
> -----------------------------------------------------------------------------------------------------------------------------------------
>
> Key: BOOKKEEPER-417
> URL: https://issues.apache.org/jira/browse/BOOKKEEPER-417
> Project: Bookkeeper
> Issue Type: Sub-task
> Components: bookkeeper-auto-recovery
> Reporter: Ivan Kelly
> Assignee: Ivan Kelly
> Fix For: 4.2.0
>
>
> As we traverse the hierarchy to find a underreplicated fragment, leaving the hierarchy will cause the traversal to search in many empty znodes.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (BOOKKEEPER-417) Hierarchical zk
underreplication manager should clean up its hierarchy when done to allow
for fast acquisition of underreplicated entries
Posted by "Uma Maheswara Rao G (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/BOOKKEEPER-417?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13472221#comment-13472221 ]
Uma Maheswara Rao G commented on BOOKKEEPER-417:
------------------------------------------------
Hi Ivan. Patch looks great. I have one comment adding to rakesh comment.
{code}
zkc.delete(getUrLedgerZnode(ledgerId), l.getLedgerZNodeVersion());
+ // clean up the hierarchy
+ String parts[] = getUrLedgerZnode(ledgerId).split("/");
+ for (int i = 1; i <= 4; i++) {
+ String p[] = Arrays.copyOf(parts, parts.length - i);
+ String path = Joiner.on("/").join(p);
+ Stat s = zkc.exists(path, null);
+ if (s != null) {
+ zkc.delete(path, s.getVersion());
+ }
+ }
{code}
Currently NotEmptyException handle for even leaf node.
Actually we should handle only for 'clean up the hierarchy' delete call.
When we delete leaf node, expectation is that, that will be free. But for any reason/due to some bug, if leaf node has some child node introduced, then we will silently ignore here. Actually that is not expected. It should loudly throw exception out.
So, better to handle NotEmptyException only for hierarchy cleanup delete call. What do you say?
> Hierarchical zk underreplication manager should clean up its hierarchy when done to allow for fast acquisition of underreplicated entries
> -----------------------------------------------------------------------------------------------------------------------------------------
>
> Key: BOOKKEEPER-417
> URL: https://issues.apache.org/jira/browse/BOOKKEEPER-417
> Project: Bookkeeper
> Issue Type: Sub-task
> Components: bookkeeper-auto-recovery
> Reporter: Ivan Kelly
> Assignee: Ivan Kelly
> Fix For: 4.2.0
>
> Attachments: BOOKKEEPER-417.diff
>
>
> As we traverse the hierarchy to find a underreplicated fragment, leaving the hierarchy will cause the traversal to search in many empty znodes.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (BOOKKEEPER-417) Hierarchical zk underreplication
manager should clean up its hierarchy when done to allow for fast
acquisition of underreplicated entries
Posted by "Ivan Kelly (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/BOOKKEEPER-417?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Ivan Kelly updated BOOKKEEPER-417:
----------------------------------
Attachment: BOOKKEEPER-417.diff
New patch addresses comments
> Hierarchical zk underreplication manager should clean up its hierarchy when done to allow for fast acquisition of underreplicated entries
> -----------------------------------------------------------------------------------------------------------------------------------------
>
> Key: BOOKKEEPER-417
> URL: https://issues.apache.org/jira/browse/BOOKKEEPER-417
> Project: Bookkeeper
> Issue Type: Sub-task
> Components: bookkeeper-auto-recovery
> Reporter: Ivan Kelly
> Assignee: Ivan Kelly
> Fix For: 4.2.0
>
> Attachments: BOOKKEEPER-417.diff, BOOKKEEPER-417.diff
>
>
> As we traverse the hierarchy to find a underreplicated fragment, leaving the hierarchy will cause the traversal to search in many empty znodes.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (BOOKKEEPER-417) Hierarchical zk
underreplication manager should clean up its hierarchy when done to allow
for fast acquisition of underreplicated entries
Posted by "Uma Maheswara Rao G (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/BOOKKEEPER-417?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13474165#comment-13474165 ]
Uma Maheswara Rao G commented on BOOKKEEPER-417:
------------------------------------------------
Yeah, +1 for TestReplicationWorker changes. Thanks
> Hierarchical zk underreplication manager should clean up its hierarchy when done to allow for fast acquisition of underreplicated entries
> -----------------------------------------------------------------------------------------------------------------------------------------
>
> Key: BOOKKEEPER-417
> URL: https://issues.apache.org/jira/browse/BOOKKEEPER-417
> Project: Bookkeeper
> Issue Type: Sub-task
> Components: bookkeeper-auto-recovery
> Reporter: Ivan Kelly
> Assignee: Ivan Kelly
> Fix For: 4.2.0
>
> Attachments: BOOKKEEPER-417.diff, BOOKKEEPER-417.diff, BOOKKEEPER-417.diff
>
>
> As we traverse the hierarchy to find a underreplicated fragment, leaving the hierarchy will cause the traversal to search in many empty znodes.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira