You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@hbase.apache.org by "Jimmy Xiang (JIRA)" <ji...@apache.org> on 2012/10/11 18:59:03 UTC
[jira] [Created] (HBASE-6977) Multithread processing ZK assignment
events
Jimmy Xiang created HBASE-6977:
----------------------------------
Summary: Multithread processing ZK assignment events
Key: HBASE-6977
URL: https://issues.apache.org/jira/browse/HBASE-6977
Project: HBase
Issue Type: Improvement
Components: Region Assignment
Reporter: Jimmy Xiang
Assignee: Jimmy Xiang
Related to HBASE-6976 and HBASE-6611. ZK events processing is a bottle neck for assignments, since there is only one ZK event thread. If we can use multiple threads, it should be better.
With multiple threads, the order of events could be messed up. However, if we pass all events related to one region always to the same worker thread, the order should be kept.
We need to play with it and find out how much performance imrovement we can get.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6977) Multithread processing ZK
assignment events
Posted by "Hudson (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-6977?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13484419#comment-13484419 ]
Hudson commented on HBASE-6977:
-------------------------------
Integrated in HBase-TRUNK-on-Hadoop-2.0.0 #236 (See [https://builds.apache.org/job/HBase-TRUNK-on-Hadoop-2.0.0/236/])
HBASE-6977 Multithread processing ZK assignment events (Revision 1402226)
Result = FAILURE
jxiang :
Files :
* /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/HRegionInfo.java
* /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/master/AssignmentManager.java
* /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/master/OfflineCallback.java
> Multithread processing ZK assignment events
> -------------------------------------------
>
> Key: HBASE-6977
> URL: https://issues.apache.org/jira/browse/HBASE-6977
> Project: HBase
> Issue Type: Improvement
> Components: Region Assignment
> Reporter: Jimmy Xiang
> Assignee: Jimmy Xiang
> Priority: Minor
> Fix For: 0.96.0
>
> Attachments: trunk-6977_v1.patch, trunk-6977_v2-1.patch, trunk-6977_v3.patch
>
>
> Related to HBASE-6976 and HBASE-6611. ZK events processing is a bottle neck for assignments, since there is only one ZK event thread. If we can use multiple threads, it should be better.
> With multiple threads, the order of events could be messed up. However, if we pass all events related to one region always to the same worker thread, the order should be kept.
> We need to play with it and find out how much performance imrovement we can get.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6977) Multithread processing ZK
assignment events
Posted by "Jimmy Xiang (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-6977?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13480325#comment-13480325 ]
Jimmy Xiang commented on HBASE-6977:
------------------------------------
One more change I am going to do in this issue is to enhance the nodeChildrenChanged event handler. We don't want to watch most of the children again since we are already watching them. We just need to watch those new children. In bulk assignment, there are so many children which should not be watched again. So it will avoid some efforts wasted here.
> Multithread processing ZK assignment events
> -------------------------------------------
>
> Key: HBASE-6977
> URL: https://issues.apache.org/jira/browse/HBASE-6977
> Project: HBase
> Issue Type: Improvement
> Components: Region Assignment
> Reporter: Jimmy Xiang
> Assignee: Jimmy Xiang
>
> Related to HBASE-6976 and HBASE-6611. ZK events processing is a bottle neck for assignments, since there is only one ZK event thread. If we can use multiple threads, it should be better.
> With multiple threads, the order of events could be messed up. However, if we pass all events related to one region always to the same worker thread, the order should be kept.
> We need to play with it and find out how much performance imrovement we can get.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-6977) Multithread processing ZK assignment
events
Posted by "Jimmy Xiang (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-6977?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jimmy Xiang updated HBASE-6977:
-------------------------------
Attachment: trunk-6977_v3.patch
> Multithread processing ZK assignment events
> -------------------------------------------
>
> Key: HBASE-6977
> URL: https://issues.apache.org/jira/browse/HBASE-6977
> Project: HBase
> Issue Type: Improvement
> Components: Region Assignment
> Reporter: Jimmy Xiang
> Assignee: Jimmy Xiang
> Priority: Minor
> Attachments: trunk-6977_v1.patch, trunk-6977_v2-1.patch, trunk-6977_v3.patch
>
>
> Related to HBASE-6976 and HBASE-6611. ZK events processing is a bottle neck for assignments, since there is only one ZK event thread. If we can use multiple threads, it should be better.
> With multiple threads, the order of events could be messed up. However, if we pass all events related to one region always to the same worker thread, the order should be kept.
> We need to play with it and find out how much performance imrovement we can get.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-6977) Multithread processing ZK assignment
events
Posted by "Jimmy Xiang (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-6977?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jimmy Xiang updated HBASE-6977:
-------------------------------
Attachment: trunk-6977_v2-1.patch
> Multithread processing ZK assignment events
> -------------------------------------------
>
> Key: HBASE-6977
> URL: https://issues.apache.org/jira/browse/HBASE-6977
> Project: HBase
> Issue Type: Improvement
> Components: Region Assignment
> Reporter: Jimmy Xiang
> Assignee: Jimmy Xiang
> Priority: Minor
> Attachments: trunk-6977_v1.patch, trunk-6977_v2-1.patch
>
>
> Related to HBASE-6976 and HBASE-6611. ZK events processing is a bottle neck for assignments, since there is only one ZK event thread. If we can use multiple threads, it should be better.
> With multiple threads, the order of events could be messed up. However, if we pass all events related to one region always to the same worker thread, the order should be kept.
> We need to play with it and find out how much performance imrovement we can get.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6977) Multithread processing ZK
assignment events
Posted by "Hadoop QA (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-6977?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13484299#comment-13484299 ]
Hadoop QA commented on HBASE-6977:
----------------------------------
{color:red}-1 overall{color}. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12550793/trunk-6977_v3.patch
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author tags.
{color:red}-1 tests included{color}. The patch doesn't appear to include any new or modified tests.
Please justify why no new tests are needed for this patch.
Also please list what manual steps were performed to verify this patch.
{color:green}+1 hadoop2.0{color}. The patch compiles against the hadoop 2.0 profile.
{color:red}-1 javadoc{color}. The javadoc tool appears to have generated 85 warning messages.
{color:green}+1 javac{color}. The applied patch does not increase the total number of javac compiler warnings.
{color:red}-1 findbugs{color}. The patch appears to introduce 2 new Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase the total number of release audit warnings.
{color:green}+1 core tests{color}. The patch passed unit tests in .
Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/3148//testReport/
Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/3148//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop2-compat.html
Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/3148//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html
Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/3148//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop1-compat.html
Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/3148//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/3148//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/3148//console
This message is automatically generated.
> Multithread processing ZK assignment events
> -------------------------------------------
>
> Key: HBASE-6977
> URL: https://issues.apache.org/jira/browse/HBASE-6977
> Project: HBase
> Issue Type: Improvement
> Components: Region Assignment
> Reporter: Jimmy Xiang
> Assignee: Jimmy Xiang
> Priority: Minor
> Attachments: trunk-6977_v1.patch, trunk-6977_v2-1.patch, trunk-6977_v3.patch
>
>
> Related to HBASE-6976 and HBASE-6611. ZK events processing is a bottle neck for assignments, since there is only one ZK event thread. If we can use multiple threads, it should be better.
> With multiple threads, the order of events could be messed up. However, if we pass all events related to one region always to the same worker thread, the order should be kept.
> We need to play with it and find out how much performance imrovement we can get.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-6977) Multithread processing ZK assignment
events
Posted by "Jimmy Xiang (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-6977?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jimmy Xiang updated HBASE-6977:
-------------------------------
Attachment: trunk-6977_v1.patch
> Multithread processing ZK assignment events
> -------------------------------------------
>
> Key: HBASE-6977
> URL: https://issues.apache.org/jira/browse/HBASE-6977
> Project: HBase
> Issue Type: Improvement
> Components: Region Assignment
> Reporter: Jimmy Xiang
> Assignee: Jimmy Xiang
> Priority: Minor
> Attachments: trunk-6977_v1.patch
>
>
> Related to HBASE-6976 and HBASE-6611. ZK events processing is a bottle neck for assignments, since there is only one ZK event thread. If we can use multiple threads, it should be better.
> With multiple threads, the order of events could be messed up. However, if we pass all events related to one region always to the same worker thread, the order should be kept.
> We need to play with it and find out how much performance imrovement we can get.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6977) Multithread processing ZK
assignment events
Posted by "Hadoop QA (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-6977?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13482118#comment-13482118 ]
Hadoop QA commented on HBASE-6977:
----------------------------------
{color:red}-1 overall{color}. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12550381/trunk-6977_v2-1.patch
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author tags.
{color:red}-1 tests included{color}. The patch doesn't appear to include any new or modified tests.
Please justify why no new tests are needed for this patch.
Also please list what manual steps were performed to verify this patch.
{color:green}+1 hadoop2.0{color}. The patch compiles against the hadoop 2.0 profile.
{color:red}-1 javadoc{color}. The javadoc tool appears to have generated 82 warning messages.
{color:green}+1 javac{color}. The applied patch does not increase the total number of javac compiler warnings.
{color:red}-1 findbugs{color}. The patch appears to introduce 4 new Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase the total number of release audit warnings.
{color:green}+1 core tests{color}. The patch passed unit tests in .
Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/3120//testReport/
Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/3120//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop2-compat.html
Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/3120//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop1-compat.html
Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/3120//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/3120//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html
Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/3120//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/3120//console
This message is automatically generated.
> Multithread processing ZK assignment events
> -------------------------------------------
>
> Key: HBASE-6977
> URL: https://issues.apache.org/jira/browse/HBASE-6977
> Project: HBase
> Issue Type: Improvement
> Components: Region Assignment
> Reporter: Jimmy Xiang
> Assignee: Jimmy Xiang
> Priority: Minor
> Attachments: trunk-6977_v1.patch, trunk-6977_v2-1.patch
>
>
> Related to HBASE-6976 and HBASE-6611. ZK events processing is a bottle neck for assignments, since there is only one ZK event thread. If we can use multiple threads, it should be better.
> With multiple threads, the order of events could be messed up. However, if we pass all events related to one region always to the same worker thread, the order should be kept.
> We need to play with it and find out how much performance imrovement we can get.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-6977) Multithread processing ZK assignment
events
Posted by "Jimmy Xiang (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-6977?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jimmy Xiang updated HBASE-6977:
-------------------------------
Status: Patch Available (was: Open)
> Multithread processing ZK assignment events
> -------------------------------------------
>
> Key: HBASE-6977
> URL: https://issues.apache.org/jira/browse/HBASE-6977
> Project: HBase
> Issue Type: Improvement
> Components: Region Assignment
> Reporter: Jimmy Xiang
> Assignee: Jimmy Xiang
> Priority: Minor
> Attachments: trunk-6977_v1.patch, trunk-6977_v2-1.patch, trunk-6977_v3.patch
>
>
> Related to HBASE-6976 and HBASE-6611. ZK events processing is a bottle neck for assignments, since there is only one ZK event thread. If we can use multiple threads, it should be better.
> With multiple threads, the order of events could be messed up. However, if we pass all events related to one region always to the same worker thread, the order should be kept.
> We need to play with it and find out how much performance imrovement we can get.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-6977) Multithread processing ZK assignment
events
Posted by "Jimmy Xiang (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-6977?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jimmy Xiang updated HBASE-6977:
-------------------------------
Priority: Minor (was: Major)
> Multithread processing ZK assignment events
> -------------------------------------------
>
> Key: HBASE-6977
> URL: https://issues.apache.org/jira/browse/HBASE-6977
> Project: HBase
> Issue Type: Improvement
> Components: Region Assignment
> Reporter: Jimmy Xiang
> Assignee: Jimmy Xiang
> Priority: Minor
>
> Related to HBASE-6976 and HBASE-6611. ZK events processing is a bottle neck for assignments, since there is only one ZK event thread. If we can use multiple threads, it should be better.
> With multiple threads, the order of events could be messed up. However, if we pass all events related to one region always to the same worker thread, the order should be kept.
> We need to play with it and find out how much performance imrovement we can get.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-6977) Multithread processing ZK assignment
events
Posted by "Jimmy Xiang (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-6977?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jimmy Xiang updated HBASE-6977:
-------------------------------
Status: Patch Available (was: Open)
> Multithread processing ZK assignment events
> -------------------------------------------
>
> Key: HBASE-6977
> URL: https://issues.apache.org/jira/browse/HBASE-6977
> Project: HBase
> Issue Type: Improvement
> Components: Region Assignment
> Reporter: Jimmy Xiang
> Assignee: Jimmy Xiang
> Priority: Minor
> Attachments: trunk-6977_v1.patch
>
>
> Related to HBASE-6976 and HBASE-6611. ZK events processing is a bottle neck for assignments, since there is only one ZK event thread. If we can use multiple threads, it should be better.
> With multiple threads, the order of events could be messed up. However, if we pass all events related to one region always to the same worker thread, the order should be kept.
> We need to play with it and find out how much performance imrovement we can get.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-6977) Multithread processing ZK assignment
events
Posted by "Jimmy Xiang (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-6977?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jimmy Xiang updated HBASE-6977:
-------------------------------
Resolution: Fixed
Fix Version/s: 0.96.0
Status: Resolved (was: Patch Available)
Integrated into trunk. Thanks all for reviewing it.
> Multithread processing ZK assignment events
> -------------------------------------------
>
> Key: HBASE-6977
> URL: https://issues.apache.org/jira/browse/HBASE-6977
> Project: HBase
> Issue Type: Improvement
> Components: Region Assignment
> Reporter: Jimmy Xiang
> Assignee: Jimmy Xiang
> Priority: Minor
> Fix For: 0.96.0
>
> Attachments: trunk-6977_v1.patch, trunk-6977_v2-1.patch, trunk-6977_v3.patch
>
>
> Related to HBASE-6976 and HBASE-6611. ZK events processing is a bottle neck for assignments, since there is only one ZK event thread. If we can use multiple threads, it should be better.
> With multiple threads, the order of events could be messed up. However, if we pass all events related to one region always to the same worker thread, the order should be kept.
> We need to play with it and find out how much performance imrovement we can get.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-6977) Multithread processing ZK assignment
events
Posted by "Jimmy Xiang (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-6977?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jimmy Xiang updated HBASE-6977:
-------------------------------
Hadoop Flags: Reviewed
Status: Patch Available (was: Open)
> Multithread processing ZK assignment events
> -------------------------------------------
>
> Key: HBASE-6977
> URL: https://issues.apache.org/jira/browse/HBASE-6977
> Project: HBase
> Issue Type: Improvement
> Components: Region Assignment
> Reporter: Jimmy Xiang
> Assignee: Jimmy Xiang
> Priority: Minor
> Attachments: trunk-6977_v1.patch, trunk-6977_v2-1.patch
>
>
> Related to HBASE-6976 and HBASE-6611. ZK events processing is a bottle neck for assignments, since there is only one ZK event thread. If we can use multiple threads, it should be better.
> With multiple threads, the order of events could be messed up. However, if we pass all events related to one region always to the same worker thread, the order should be kept.
> We need to play with it and find out how much performance imrovement we can get.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6977) Multithread processing ZK
assignment events
Posted by "Hudson (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-6977?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13484368#comment-13484368 ]
Hudson commented on HBASE-6977:
-------------------------------
Integrated in HBase-TRUNK #3486 (See [https://builds.apache.org/job/HBase-TRUNK/3486/])
HBASE-6977 Multithread processing ZK assignment events (Revision 1402226)
Result = FAILURE
jxiang :
Files :
* /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/HRegionInfo.java
* /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/master/AssignmentManager.java
* /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/master/OfflineCallback.java
> Multithread processing ZK assignment events
> -------------------------------------------
>
> Key: HBASE-6977
> URL: https://issues.apache.org/jira/browse/HBASE-6977
> Project: HBase
> Issue Type: Improvement
> Components: Region Assignment
> Reporter: Jimmy Xiang
> Assignee: Jimmy Xiang
> Priority: Minor
> Fix For: 0.96.0
>
> Attachments: trunk-6977_v1.patch, trunk-6977_v2-1.patch, trunk-6977_v3.patch
>
>
> Related to HBASE-6976 and HBASE-6611. ZK events processing is a bottle neck for assignments, since there is only one ZK event thread. If we can use multiple threads, it should be better.
> With multiple threads, the order of events could be messed up. However, if we pass all events related to one region always to the same worker thread, the order should be kept.
> We need to play with it and find out how much performance imrovement we can get.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6977) Multithread processing ZK
assignment events
Posted by "Jimmy Xiang (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-6977?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13481815#comment-13481815 ]
Jimmy Xiang commented on HBASE-6977:
------------------------------------
Thanks Ted for the review. I posted the second patch to RB: https://reviews.apache.org/r/7682/
> Multithread processing ZK assignment events
> -------------------------------------------
>
> Key: HBASE-6977
> URL: https://issues.apache.org/jira/browse/HBASE-6977
> Project: HBase
> Issue Type: Improvement
> Components: Region Assignment
> Reporter: Jimmy Xiang
> Assignee: Jimmy Xiang
> Priority: Minor
> Attachments: trunk-6977_v1.patch
>
>
> Related to HBASE-6976 and HBASE-6611. ZK events processing is a bottle neck for assignments, since there is only one ZK event thread. If we can use multiple threads, it should be better.
> With multiple threads, the order of events could be messed up. However, if we pass all events related to one region always to the same worker thread, the order should be kept.
> We need to play with it and find out how much performance imrovement we can get.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6977) Multithread processing ZK
assignment events
Posted by "Hadoop QA (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-6977?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13480649#comment-13480649 ]
Hadoop QA commented on HBASE-6977:
----------------------------------
{color:red}-1 overall{color}. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12550131/trunk-6977_v1.patch
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author tags.
{color:red}-1 tests included{color}. The patch doesn't appear to include any new or modified tests.
Please justify why no new tests are needed for this patch.
Also please list what manual steps were performed to verify this patch.
{color:green}+1 hadoop2.0{color}. The patch compiles against the hadoop 2.0 profile.
{color:red}-1 javadoc{color}. The javadoc tool appears to have generated 82 warning messages.
{color:green}+1 javac{color}. The applied patch does not increase the total number of javac compiler warnings.
{color:red}-1 findbugs{color}. The patch appears to introduce 5 new Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase the total number of release audit warnings.
{color:red}-1 core tests{color}. The patch failed these unit tests:
org.apache.hadoop.hbase.coprocessor.TestRegionServerCoprocessorExceptionWithAbort
Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/3100//testReport/
Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/3100//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop2-compat.html
Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/3100//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html
Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/3100//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop1-compat.html
Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/3100//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/3100//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/3100//console
This message is automatically generated.
> Multithread processing ZK assignment events
> -------------------------------------------
>
> Key: HBASE-6977
> URL: https://issues.apache.org/jira/browse/HBASE-6977
> Project: HBase
> Issue Type: Improvement
> Components: Region Assignment
> Reporter: Jimmy Xiang
> Assignee: Jimmy Xiang
> Priority: Minor
> Attachments: trunk-6977_v1.patch
>
>
> Related to HBASE-6976 and HBASE-6611. ZK events processing is a bottle neck for assignments, since there is only one ZK event thread. If we can use multiple threads, it should be better.
> With multiple threads, the order of events could be messed up. However, if we pass all events related to one region always to the same worker thread, the order should be kept.
> We need to play with it and find out how much performance imrovement we can get.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-6977) Multithread processing ZK assignment
events
Posted by "Jimmy Xiang (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-6977?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jimmy Xiang updated HBASE-6977:
-------------------------------
Status: Open (was: Patch Available)
> Multithread processing ZK assignment events
> -------------------------------------------
>
> Key: HBASE-6977
> URL: https://issues.apache.org/jira/browse/HBASE-6977
> Project: HBase
> Issue Type: Improvement
> Components: Region Assignment
> Reporter: Jimmy Xiang
> Assignee: Jimmy Xiang
> Priority: Minor
> Attachments: trunk-6977_v1.patch
>
>
> Related to HBASE-6976 and HBASE-6611. ZK events processing is a bottle neck for assignments, since there is only one ZK event thread. If we can use multiple threads, it should be better.
> With multiple threads, the order of events could be messed up. However, if we pass all events related to one region always to the same worker thread, the order should be kept.
> We need to play with it and find out how much performance imrovement we can get.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-6977) Multithread processing ZK assignment
events
Posted by "Jimmy Xiang (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-6977?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jimmy Xiang updated HBASE-6977:
-------------------------------
Status: Open (was: Patch Available)
Will address Stack's comments and upload a new patch.
> Multithread processing ZK assignment events
> -------------------------------------------
>
> Key: HBASE-6977
> URL: https://issues.apache.org/jira/browse/HBASE-6977
> Project: HBase
> Issue Type: Improvement
> Components: Region Assignment
> Reporter: Jimmy Xiang
> Assignee: Jimmy Xiang
> Priority: Minor
> Attachments: trunk-6977_v1.patch, trunk-6977_v2-1.patch
>
>
> Related to HBASE-6976 and HBASE-6611. ZK events processing is a bottle neck for assignments, since there is only one ZK event thread. If we can use multiple threads, it should be better.
> With multiple threads, the order of events could be messed up. However, if we pass all events related to one region always to the same worker thread, the order should be kept.
> We need to play with it and find out how much performance imrovement we can get.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6977) Multithread processing ZK
assignment events
Posted by "Jimmy Xiang (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-6977?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13480634#comment-13480634 ]
Jimmy Xiang commented on HBASE-6977:
------------------------------------
The patch is posted on RB: https://reviews.apache.org/r/7682/ for easy review.
In the meantime, I was thinking about HBASE-7022, using multi to batch those zk operations in bulk assigner.
I think it will improve the performance a little. I need to play with it and find out.
> Multithread processing ZK assignment events
> -------------------------------------------
>
> Key: HBASE-6977
> URL: https://issues.apache.org/jira/browse/HBASE-6977
> Project: HBase
> Issue Type: Improvement
> Components: Region Assignment
> Reporter: Jimmy Xiang
> Assignee: Jimmy Xiang
> Priority: Minor
> Attachments: trunk-6977_v1.patch
>
>
> Related to HBASE-6976 and HBASE-6611. ZK events processing is a bottle neck for assignments, since there is only one ZK event thread. If we can use multiple threads, it should be better.
> With multiple threads, the order of events could be messed up. However, if we pass all events related to one region always to the same worker thread, the order should be kept.
> We need to play with it and find out how much performance imrovement we can get.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira