You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@hbase.apache.org by "nkeywal (JIRA)" <ji...@apache.org> on 2012/05/14 12:10:48 UTC
[jira] [Created] (HBASE-5998) Bulk assignment: regionserver
optimization by using a temporary cache for table descriptors when
receveing an open regions request
nkeywal created HBASE-5998:
------------------------------
Summary: Bulk assignment: regionserver optimization by using a temporary cache for table descriptors when receveing an open regions request
Key: HBASE-5998
URL: https://issues.apache.org/jira/browse/HBASE-5998
Project: HBase
Issue Type: Improvement
Components: regionserver
Affects Versions: 0.96.0
Reporter: nkeywal
Assignee: nkeywal
Priority: Minor
During the assignment, on the regionserver, before creating the handlers we load the table description. Even if there is a cache, we check the timestamps for each region, while it's not necessary. The test below is just with one node, with more nodes the benefit will improve. By limiting the time spent in HRegion#openRegion we increase the parallelization during cluster startup, as the master is using a pool of threads to call the RS.
-- Without the fix
2012-05-14 11:40:52,501 DEBUG org.apache.hadoop.hbase.master.AssignmentManager: Bulk assigning 1193 region(s) to localhost,11003,1336988444043
2012-05-14 11:41:09,947 DEBUG org.apache.hadoop.hbase.master.AssignmentManager: Bulk assigning done for localhost,11003,1336988444043
-- With the fix
2012-05-14 11:34:40,444 DEBUG org.apache.hadoop.hbase.master.AssignmentManager: Bulk assigning 1193 region(s) to localhost,11003,1336988444043
2012-05-14 11:34:40,929 DEBUG org.apache.hadoop.hbase.master.AssignmentManager: Bulk assigning done for localhost,11003,1336988065948
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5998) Bulk assignment: regionserver
optimization by using a temporary cache for table descriptors when
receveing an open regions request
Posted by "Anoop Sam John (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5998?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13294821#comment-13294821 ]
Anoop Sam John commented on HBASE-5998:
---------------------------------------
Would it be nice to backport this to 94.1?
This seems to be a very nice improvement in bulk assign.
> Bulk assignment: regionserver optimization by using a temporary cache for table descriptors when receveing an open regions request
> ----------------------------------------------------------------------------------------------------------------------------------
>
> Key: HBASE-5998
> URL: https://issues.apache.org/jira/browse/HBASE-5998
> Project: HBase
> Issue Type: Improvement
> Components: regionserver
> Affects Versions: 0.96.0
> Reporter: nkeywal
> Assignee: nkeywal
> Priority: Minor
> Fix For: 0.96.0
>
> Attachments: 5998.v2.patch, 5998.v3.patch
>
>
> During the assignment, on the regionserver, before creating the handlers we load the table description. Even if there is a cache, we check the timestamps for each region, while it's not necessary. The test below is just with one node, with more nodes the benefit will improve. By limiting the time spent in HRegion#openRegion we increase the parallelization during cluster startup, as the master is using a pool of threads to call the RS.
> -- Without the fix
> 2012-05-14 11:40:52,501 DEBUG org.apache.hadoop.hbase.master.AssignmentManager: Bulk assigning 1193 region(s) to localhost,11003,1336988444043
> 2012-05-14 11:41:09,947 DEBUG org.apache.hadoop.hbase.master.AssignmentManager: Bulk assigning done for localhost,11003,1336988444043
> -- With the fix
> 2012-05-14 11:34:40,444 DEBUG org.apache.hadoop.hbase.master.AssignmentManager: Bulk assigning 1193 region(s) to localhost,11003,1336988444043
> 2012-05-14 11:34:40,929 DEBUG org.apache.hadoop.hbase.master.AssignmentManager: Bulk assigning done for localhost,11003,1336988065948
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5998) Bulk assignment: regionserver
optimization by using a temporary cache for table descriptors when
receveing an open regions request
Posted by "nkeywal (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5998?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
nkeywal updated HBASE-5998:
---------------------------
Resolution: Fixed
Status: Resolved (was: Patch Available)
depiste the failure mentionned by jenkins, the patch is visible in the source code.
> Bulk assignment: regionserver optimization by using a temporary cache for table descriptors when receveing an open regions request
> ----------------------------------------------------------------------------------------------------------------------------------
>
> Key: HBASE-5998
> URL: https://issues.apache.org/jira/browse/HBASE-5998
> Project: HBase
> Issue Type: Improvement
> Components: regionserver
> Affects Versions: 0.96.0
> Reporter: nkeywal
> Assignee: nkeywal
> Priority: Minor
> Fix For: 0.96.0
>
> Attachments: 5998.v2.patch, 5998.v3.patch
>
>
> During the assignment, on the regionserver, before creating the handlers we load the table description. Even if there is a cache, we check the timestamps for each region, while it's not necessary. The test below is just with one node, with more nodes the benefit will improve. By limiting the time spent in HRegion#openRegion we increase the parallelization during cluster startup, as the master is using a pool of threads to call the RS.
> -- Without the fix
> 2012-05-14 11:40:52,501 DEBUG org.apache.hadoop.hbase.master.AssignmentManager: Bulk assigning 1193 region(s) to localhost,11003,1336988444043
> 2012-05-14 11:41:09,947 DEBUG org.apache.hadoop.hbase.master.AssignmentManager: Bulk assigning done for localhost,11003,1336988444043
> -- With the fix
> 2012-05-14 11:34:40,444 DEBUG org.apache.hadoop.hbase.master.AssignmentManager: Bulk assigning 1193 region(s) to localhost,11003,1336988444043
> 2012-05-14 11:34:40,929 DEBUG org.apache.hadoop.hbase.master.AssignmentManager: Bulk assigning done for localhost,11003,1336988065948
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5998) Bulk assignment: regionserver
optimization by using a temporary cache for table descriptors when
receveing an open regions request
Posted by "nkeywal (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5998?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
nkeywal updated HBASE-5998:
---------------------------
Status: Open (was: Patch Available)
> Bulk assignment: regionserver optimization by using a temporary cache for table descriptors when receveing an open regions request
> ----------------------------------------------------------------------------------------------------------------------------------
>
> Key: HBASE-5998
> URL: https://issues.apache.org/jira/browse/HBASE-5998
> Project: HBase
> Issue Type: Improvement
> Components: regionserver
> Affects Versions: 0.96.0
> Reporter: nkeywal
> Assignee: nkeywal
> Priority: Minor
> Fix For: 0.96.0
>
> Attachments: 5998.v2.patch
>
>
> During the assignment, on the regionserver, before creating the handlers we load the table description. Even if there is a cache, we check the timestamps for each region, while it's not necessary. The test below is just with one node, with more nodes the benefit will improve. By limiting the time spent in HRegion#openRegion we increase the parallelization during cluster startup, as the master is using a pool of threads to call the RS.
> -- Without the fix
> 2012-05-14 11:40:52,501 DEBUG org.apache.hadoop.hbase.master.AssignmentManager: Bulk assigning 1193 region(s) to localhost,11003,1336988444043
> 2012-05-14 11:41:09,947 DEBUG org.apache.hadoop.hbase.master.AssignmentManager: Bulk assigning done for localhost,11003,1336988444043
> -- With the fix
> 2012-05-14 11:34:40,444 DEBUG org.apache.hadoop.hbase.master.AssignmentManager: Bulk assigning 1193 region(s) to localhost,11003,1336988444043
> 2012-05-14 11:34:40,929 DEBUG org.apache.hadoop.hbase.master.AssignmentManager: Bulk assigning done for localhost,11003,1336988065948
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5998) Bulk assignment: regionserver
optimization by using a temporary cache for table descriptors when
receveing an open regions request
Posted by "nkeywal (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5998?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13274572#comment-13274572 ]
nkeywal commented on HBASE-5998:
--------------------------------
The failure in TestDrainingServer is not directly related to my change. I will have a look and fix the flakiness in another jira.
> Bulk assignment: regionserver optimization by using a temporary cache for table descriptors when receveing an open regions request
> ----------------------------------------------------------------------------------------------------------------------------------
>
> Key: HBASE-5998
> URL: https://issues.apache.org/jira/browse/HBASE-5998
> Project: HBase
> Issue Type: Improvement
> Components: regionserver
> Affects Versions: 0.96.0
> Reporter: nkeywal
> Assignee: nkeywal
> Priority: Minor
> Fix For: 0.96.0
>
> Attachments: 5998.v2.patch, 5998.v3.patch
>
>
> During the assignment, on the regionserver, before creating the handlers we load the table description. Even if there is a cache, we check the timestamps for each region, while it's not necessary. The test below is just with one node, with more nodes the benefit will improve. By limiting the time spent in HRegion#openRegion we increase the parallelization during cluster startup, as the master is using a pool of threads to call the RS.
> -- Without the fix
> 2012-05-14 11:40:52,501 DEBUG org.apache.hadoop.hbase.master.AssignmentManager: Bulk assigning 1193 region(s) to localhost,11003,1336988444043
> 2012-05-14 11:41:09,947 DEBUG org.apache.hadoop.hbase.master.AssignmentManager: Bulk assigning done for localhost,11003,1336988444043
> -- With the fix
> 2012-05-14 11:34:40,444 DEBUG org.apache.hadoop.hbase.master.AssignmentManager: Bulk assigning 1193 region(s) to localhost,11003,1336988444043
> 2012-05-14 11:34:40,929 DEBUG org.apache.hadoop.hbase.master.AssignmentManager: Bulk assigning done for localhost,11003,1336988065948
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5998) Bulk assignment: regionserver
optimization by using a temporary cache for table descriptors when
receveing an open regions request
Posted by "Hadoop QA (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5998?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13274562#comment-13274562 ]
Hadoop QA commented on HBASE-5998:
----------------------------------
-1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12526736/5998.v3.patch
against trunk revision .
+1 @author. The patch does not contain any @author tags.
-1 tests included. The patch doesn't appear to include any new or modified tests.
Please justify why no new tests are needed for this patch.
Also please list what manual steps were performed to verify this patch.
+1 hadoop23. The patch compiles against the hadoop 0.23.x profile.
+1 javadoc. The javadoc tool did not generate any warning messages.
+1 javac. The applied patch does not increase the total number of javac compiler warnings.
-1 findbugs. The patch appears to introduce 31 new Findbugs (version 1.3.9) warnings.
+1 release audit. The applied patch does not increase the total number of release audit warnings.
-1 core tests. The patch failed these unit tests:
org.apache.hadoop.hbase.TestDrainingServer
Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/1865//testReport/
Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/1865//artifact/trunk/patchprocess/newPatchFindbugsWarnings.html
Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/1865//console
This message is automatically generated.
> Bulk assignment: regionserver optimization by using a temporary cache for table descriptors when receveing an open regions request
> ----------------------------------------------------------------------------------------------------------------------------------
>
> Key: HBASE-5998
> URL: https://issues.apache.org/jira/browse/HBASE-5998
> Project: HBase
> Issue Type: Improvement
> Components: regionserver
> Affects Versions: 0.96.0
> Reporter: nkeywal
> Assignee: nkeywal
> Priority: Minor
> Fix For: 0.96.0
>
> Attachments: 5998.v2.patch, 5998.v3.patch
>
>
> During the assignment, on the regionserver, before creating the handlers we load the table description. Even if there is a cache, we check the timestamps for each region, while it's not necessary. The test below is just with one node, with more nodes the benefit will improve. By limiting the time spent in HRegion#openRegion we increase the parallelization during cluster startup, as the master is using a pool of threads to call the RS.
> -- Without the fix
> 2012-05-14 11:40:52,501 DEBUG org.apache.hadoop.hbase.master.AssignmentManager: Bulk assigning 1193 region(s) to localhost,11003,1336988444043
> 2012-05-14 11:41:09,947 DEBUG org.apache.hadoop.hbase.master.AssignmentManager: Bulk assigning done for localhost,11003,1336988444043
> -- With the fix
> 2012-05-14 11:34:40,444 DEBUG org.apache.hadoop.hbase.master.AssignmentManager: Bulk assigning 1193 region(s) to localhost,11003,1336988444043
> 2012-05-14 11:34:40,929 DEBUG org.apache.hadoop.hbase.master.AssignmentManager: Bulk assigning done for localhost,11003,1336988065948
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5998) Bulk assignment: regionserver
optimization by using a temporary cache for table descriptors when
receveing an open regions request
Posted by "nkeywal (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5998?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
nkeywal updated HBASE-5998:
---------------------------
Attachment: 5998.v2.patch
> Bulk assignment: regionserver optimization by using a temporary cache for table descriptors when receveing an open regions request
> ----------------------------------------------------------------------------------------------------------------------------------
>
> Key: HBASE-5998
> URL: https://issues.apache.org/jira/browse/HBASE-5998
> Project: HBase
> Issue Type: Improvement
> Components: regionserver
> Affects Versions: 0.96.0
> Reporter: nkeywal
> Assignee: nkeywal
> Priority: Minor
> Attachments: 5998.v2.patch
>
>
> During the assignment, on the regionserver, before creating the handlers we load the table description. Even if there is a cache, we check the timestamps for each region, while it's not necessary. The test below is just with one node, with more nodes the benefit will improve. By limiting the time spent in HRegion#openRegion we increase the parallelization during cluster startup, as the master is using a pool of threads to call the RS.
> -- Without the fix
> 2012-05-14 11:40:52,501 DEBUG org.apache.hadoop.hbase.master.AssignmentManager: Bulk assigning 1193 region(s) to localhost,11003,1336988444043
> 2012-05-14 11:41:09,947 DEBUG org.apache.hadoop.hbase.master.AssignmentManager: Bulk assigning done for localhost,11003,1336988444043
> -- With the fix
> 2012-05-14 11:34:40,444 DEBUG org.apache.hadoop.hbase.master.AssignmentManager: Bulk assigning 1193 region(s) to localhost,11003,1336988444043
> 2012-05-14 11:34:40,929 DEBUG org.apache.hadoop.hbase.master.AssignmentManager: Bulk assigning done for localhost,11003,1336988065948
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5998) Bulk assignment: regionserver
optimization by using a temporary cache for table descriptors when
receveing an open regions request
Posted by "rajeshbabu (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5998?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
rajeshbabu updated HBASE-5998:
------------------------------
Attachment: HBASE-5998_94.patch
Patch for 94. Please review and provide comments/suggestions.
> Bulk assignment: regionserver optimization by using a temporary cache for table descriptors when receveing an open regions request
> ----------------------------------------------------------------------------------------------------------------------------------
>
> Key: HBASE-5998
> URL: https://issues.apache.org/jira/browse/HBASE-5998
> Project: HBase
> Issue Type: Improvement
> Components: regionserver
> Affects Versions: 0.96.0
> Reporter: nkeywal
> Assignee: nkeywal
> Priority: Minor
> Fix For: 0.96.0
>
> Attachments: 5998.v2.patch, 5998.v3.patch, HBASE-5998_94.patch
>
>
> During the assignment, on the regionserver, before creating the handlers we load the table description. Even if there is a cache, we check the timestamps for each region, while it's not necessary. The test below is just with one node, with more nodes the benefit will improve. By limiting the time spent in HRegion#openRegion we increase the parallelization during cluster startup, as the master is using a pool of threads to call the RS.
> -- Without the fix
> 2012-05-14 11:40:52,501 DEBUG org.apache.hadoop.hbase.master.AssignmentManager: Bulk assigning 1193 region(s) to localhost,11003,1336988444043
> 2012-05-14 11:41:09,947 DEBUG org.apache.hadoop.hbase.master.AssignmentManager: Bulk assigning done for localhost,11003,1336988444043
> -- With the fix
> 2012-05-14 11:34:40,444 DEBUG org.apache.hadoop.hbase.master.AssignmentManager: Bulk assigning 1193 region(s) to localhost,11003,1336988444043
> 2012-05-14 11:34:40,929 DEBUG org.apache.hadoop.hbase.master.AssignmentManager: Bulk assigning done for localhost,11003,1336988065948
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5998) Bulk assignment: regionserver
optimization by using a temporary cache for table descriptors when
receveing an open regions request
Posted by "Zhihong Yu (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5998?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13276213#comment-13276213 ]
Zhihong Yu commented on HBASE-5998:
-----------------------------------
Patch looks good.
> Bulk assignment: regionserver optimization by using a temporary cache for table descriptors when receveing an open regions request
> ----------------------------------------------------------------------------------------------------------------------------------
>
> Key: HBASE-5998
> URL: https://issues.apache.org/jira/browse/HBASE-5998
> Project: HBase
> Issue Type: Improvement
> Components: regionserver
> Affects Versions: 0.96.0
> Reporter: nkeywal
> Assignee: nkeywal
> Priority: Minor
> Fix For: 0.96.0
>
> Attachments: 5998.v2.patch, 5998.v3.patch
>
>
> During the assignment, on the regionserver, before creating the handlers we load the table description. Even if there is a cache, we check the timestamps for each region, while it's not necessary. The test below is just with one node, with more nodes the benefit will improve. By limiting the time spent in HRegion#openRegion we increase the parallelization during cluster startup, as the master is using a pool of threads to call the RS.
> -- Without the fix
> 2012-05-14 11:40:52,501 DEBUG org.apache.hadoop.hbase.master.AssignmentManager: Bulk assigning 1193 region(s) to localhost,11003,1336988444043
> 2012-05-14 11:41:09,947 DEBUG org.apache.hadoop.hbase.master.AssignmentManager: Bulk assigning done for localhost,11003,1336988444043
> -- With the fix
> 2012-05-14 11:34:40,444 DEBUG org.apache.hadoop.hbase.master.AssignmentManager: Bulk assigning 1193 region(s) to localhost,11003,1336988444043
> 2012-05-14 11:34:40,929 DEBUG org.apache.hadoop.hbase.master.AssignmentManager: Bulk assigning done for localhost,11003,1336988065948
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5998) Bulk assignment: regionserver
optimization by using a temporary cache for table descriptors when
receveing an open regions request
Posted by "Hadoop QA (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5998?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13274545#comment-13274545 ]
Hadoop QA commented on HBASE-5998:
----------------------------------
-1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12526734/5998.v2.patch
against trunk revision .
+1 @author. The patch does not contain any @author tags.
-1 tests included. The patch doesn't appear to include any new or modified tests.
Please justify why no new tests are needed for this patch.
Also please list what manual steps were performed to verify this patch.
-1 patch. The patch command could not apply the patch.
Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/1864//console
This message is automatically generated.
> Bulk assignment: regionserver optimization by using a temporary cache for table descriptors when receveing an open regions request
> ----------------------------------------------------------------------------------------------------------------------------------
>
> Key: HBASE-5998
> URL: https://issues.apache.org/jira/browse/HBASE-5998
> Project: HBase
> Issue Type: Improvement
> Components: regionserver
> Affects Versions: 0.96.0
> Reporter: nkeywal
> Assignee: nkeywal
> Priority: Minor
> Fix For: 0.96.0
>
> Attachments: 5998.v2.patch
>
>
> During the assignment, on the regionserver, before creating the handlers we load the table description. Even if there is a cache, we check the timestamps for each region, while it's not necessary. The test below is just with one node, with more nodes the benefit will improve. By limiting the time spent in HRegion#openRegion we increase the parallelization during cluster startup, as the master is using a pool of threads to call the RS.
> -- Without the fix
> 2012-05-14 11:40:52,501 DEBUG org.apache.hadoop.hbase.master.AssignmentManager: Bulk assigning 1193 region(s) to localhost,11003,1336988444043
> 2012-05-14 11:41:09,947 DEBUG org.apache.hadoop.hbase.master.AssignmentManager: Bulk assigning done for localhost,11003,1336988444043
> -- With the fix
> 2012-05-14 11:34:40,444 DEBUG org.apache.hadoop.hbase.master.AssignmentManager: Bulk assigning 1193 region(s) to localhost,11003,1336988444043
> 2012-05-14 11:34:40,929 DEBUG org.apache.hadoop.hbase.master.AssignmentManager: Bulk assigning done for localhost,11003,1336988065948
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5998) Bulk assignment: regionserver
optimization by using a temporary cache for table descriptors when
receveing an open regions request
Posted by "Hudson (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5998?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13276344#comment-13276344 ]
Hudson commented on HBASE-5998:
-------------------------------
Integrated in HBase-TRUNK-on-Hadoop-2.0.0 #5 (See [https://builds.apache.org/job/HBase-TRUNK-on-Hadoop-2.0.0/5/])
HBASE-5998 Bulk assignment: regionserver optimization by using a temporary cache for table descriptors when receveing an open regions request (N Keywal) (Revision 1338916)
Result = FAILURE
tedyu :
Files :
* /hbase/trunk/src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java
> Bulk assignment: regionserver optimization by using a temporary cache for table descriptors when receveing an open regions request
> ----------------------------------------------------------------------------------------------------------------------------------
>
> Key: HBASE-5998
> URL: https://issues.apache.org/jira/browse/HBASE-5998
> Project: HBase
> Issue Type: Improvement
> Components: regionserver
> Affects Versions: 0.96.0
> Reporter: nkeywal
> Assignee: nkeywal
> Priority: Minor
> Fix For: 0.96.0
>
> Attachments: 5998.v2.patch, 5998.v3.patch
>
>
> During the assignment, on the regionserver, before creating the handlers we load the table description. Even if there is a cache, we check the timestamps for each region, while it's not necessary. The test below is just with one node, with more nodes the benefit will improve. By limiting the time spent in HRegion#openRegion we increase the parallelization during cluster startup, as the master is using a pool of threads to call the RS.
> -- Without the fix
> 2012-05-14 11:40:52,501 DEBUG org.apache.hadoop.hbase.master.AssignmentManager: Bulk assigning 1193 region(s) to localhost,11003,1336988444043
> 2012-05-14 11:41:09,947 DEBUG org.apache.hadoop.hbase.master.AssignmentManager: Bulk assigning done for localhost,11003,1336988444043
> -- With the fix
> 2012-05-14 11:34:40,444 DEBUG org.apache.hadoop.hbase.master.AssignmentManager: Bulk assigning 1193 region(s) to localhost,11003,1336988444043
> 2012-05-14 11:34:40,929 DEBUG org.apache.hadoop.hbase.master.AssignmentManager: Bulk assigning done for localhost,11003,1336988065948
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5998) Bulk assignment: regionserver
optimization by using a temporary cache for table descriptors when
receveing an open regions request
Posted by "Hudson (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5998?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13400009#comment-13400009 ]
Hudson commented on HBASE-5998:
-------------------------------
Integrated in HBase-0.94-security #37 (See [https://builds.apache.org/job/HBase-0.94-security/37/])
HBASE-6214 Backport HBASE-5998 to 94.1 (Revision 1351052)
Result = SUCCESS
stack :
Files :
* /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java
> Bulk assignment: regionserver optimization by using a temporary cache for table descriptors when receveing an open regions request
> ----------------------------------------------------------------------------------------------------------------------------------
>
> Key: HBASE-5998
> URL: https://issues.apache.org/jira/browse/HBASE-5998
> Project: HBase
> Issue Type: Improvement
> Components: regionserver
> Affects Versions: 0.96.0
> Reporter: nkeywal
> Assignee: nkeywal
> Priority: Minor
> Fix For: 0.96.0
>
> Attachments: 5998.v2.patch, 5998.v3.patch, HBASE-5998_94.patch
>
>
> During the assignment, on the regionserver, before creating the handlers we load the table description. Even if there is a cache, we check the timestamps for each region, while it's not necessary. The test below is just with one node, with more nodes the benefit will improve. By limiting the time spent in HRegion#openRegion we increase the parallelization during cluster startup, as the master is using a pool of threads to call the RS.
> -- Without the fix
> 2012-05-14 11:40:52,501 DEBUG org.apache.hadoop.hbase.master.AssignmentManager: Bulk assigning 1193 region(s) to localhost,11003,1336988444043
> 2012-05-14 11:41:09,947 DEBUG org.apache.hadoop.hbase.master.AssignmentManager: Bulk assigning done for localhost,11003,1336988444043
> -- With the fix
> 2012-05-14 11:34:40,444 DEBUG org.apache.hadoop.hbase.master.AssignmentManager: Bulk assigning 1193 region(s) to localhost,11003,1336988444043
> 2012-05-14 11:34:40,929 DEBUG org.apache.hadoop.hbase.master.AssignmentManager: Bulk assigning done for localhost,11003,1336988065948
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5998) Bulk assignment: regionserver
optimization by using a temporary cache for table descriptors when
receveing an open regions request
Posted by "stack (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5998?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13295271#comment-13295271 ]
stack commented on HBASE-5998:
------------------------------
I took a look at backporting at Anoop's request. Its not a clean backport. Mind taking a look Nkeywal?
> Bulk assignment: regionserver optimization by using a temporary cache for table descriptors when receveing an open regions request
> ----------------------------------------------------------------------------------------------------------------------------------
>
> Key: HBASE-5998
> URL: https://issues.apache.org/jira/browse/HBASE-5998
> Project: HBase
> Issue Type: Improvement
> Components: regionserver
> Affects Versions: 0.96.0
> Reporter: nkeywal
> Assignee: nkeywal
> Priority: Minor
> Fix For: 0.96.0
>
> Attachments: 5998.v2.patch, 5998.v3.patch
>
>
> During the assignment, on the regionserver, before creating the handlers we load the table description. Even if there is a cache, we check the timestamps for each region, while it's not necessary. The test below is just with one node, with more nodes the benefit will improve. By limiting the time spent in HRegion#openRegion we increase the parallelization during cluster startup, as the master is using a pool of threads to call the RS.
> -- Without the fix
> 2012-05-14 11:40:52,501 DEBUG org.apache.hadoop.hbase.master.AssignmentManager: Bulk assigning 1193 region(s) to localhost,11003,1336988444043
> 2012-05-14 11:41:09,947 DEBUG org.apache.hadoop.hbase.master.AssignmentManager: Bulk assigning done for localhost,11003,1336988444043
> -- With the fix
> 2012-05-14 11:34:40,444 DEBUG org.apache.hadoop.hbase.master.AssignmentManager: Bulk assigning 1193 region(s) to localhost,11003,1336988444043
> 2012-05-14 11:34:40,929 DEBUG org.apache.hadoop.hbase.master.AssignmentManager: Bulk assigning done for localhost,11003,1336988065948
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5998) Bulk assignment: regionserver
optimization by using a temporary cache for table descriptors when
receveing an open regions request
Posted by "nkeywal (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5998?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
nkeywal updated HBASE-5998:
---------------------------
Fix Version/s: 0.96.0
Status: Patch Available (was: Open)
> Bulk assignment: regionserver optimization by using a temporary cache for table descriptors when receveing an open regions request
> ----------------------------------------------------------------------------------------------------------------------------------
>
> Key: HBASE-5998
> URL: https://issues.apache.org/jira/browse/HBASE-5998
> Project: HBase
> Issue Type: Improvement
> Components: regionserver
> Affects Versions: 0.96.0
> Reporter: nkeywal
> Assignee: nkeywal
> Priority: Minor
> Fix For: 0.96.0
>
> Attachments: 5998.v2.patch
>
>
> During the assignment, on the regionserver, before creating the handlers we load the table description. Even if there is a cache, we check the timestamps for each region, while it's not necessary. The test below is just with one node, with more nodes the benefit will improve. By limiting the time spent in HRegion#openRegion we increase the parallelization during cluster startup, as the master is using a pool of threads to call the RS.
> -- Without the fix
> 2012-05-14 11:40:52,501 DEBUG org.apache.hadoop.hbase.master.AssignmentManager: Bulk assigning 1193 region(s) to localhost,11003,1336988444043
> 2012-05-14 11:41:09,947 DEBUG org.apache.hadoop.hbase.master.AssignmentManager: Bulk assigning done for localhost,11003,1336988444043
> -- With the fix
> 2012-05-14 11:34:40,444 DEBUG org.apache.hadoop.hbase.master.AssignmentManager: Bulk assigning 1193 region(s) to localhost,11003,1336988444043
> 2012-05-14 11:34:40,929 DEBUG org.apache.hadoop.hbase.master.AssignmentManager: Bulk assigning done for localhost,11003,1336988065948
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5998) Bulk assignment: regionserver
optimization by using a temporary cache for table descriptors when
receveing an open regions request
Posted by "Hudson (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5998?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13393474#comment-13393474 ]
Hudson commented on HBASE-5998:
-------------------------------
Integrated in HBase-0.94 #259 (See [https://builds.apache.org/job/HBase-0.94/259/])
HBASE-6214 Backport HBASE-5998 to 94.1 (Revision 1351052)
Result = FAILURE
stack :
Files :
* /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java
> Bulk assignment: regionserver optimization by using a temporary cache for table descriptors when receveing an open regions request
> ----------------------------------------------------------------------------------------------------------------------------------
>
> Key: HBASE-5998
> URL: https://issues.apache.org/jira/browse/HBASE-5998
> Project: HBase
> Issue Type: Improvement
> Components: regionserver
> Affects Versions: 0.96.0
> Reporter: nkeywal
> Assignee: nkeywal
> Priority: Minor
> Fix For: 0.96.0
>
> Attachments: 5998.v2.patch, 5998.v3.patch, HBASE-5998_94.patch
>
>
> During the assignment, on the regionserver, before creating the handlers we load the table description. Even if there is a cache, we check the timestamps for each region, while it's not necessary. The test below is just with one node, with more nodes the benefit will improve. By limiting the time spent in HRegion#openRegion we increase the parallelization during cluster startup, as the master is using a pool of threads to call the RS.
> -- Without the fix
> 2012-05-14 11:40:52,501 DEBUG org.apache.hadoop.hbase.master.AssignmentManager: Bulk assigning 1193 region(s) to localhost,11003,1336988444043
> 2012-05-14 11:41:09,947 DEBUG org.apache.hadoop.hbase.master.AssignmentManager: Bulk assigning done for localhost,11003,1336988444043
> -- With the fix
> 2012-05-14 11:34:40,444 DEBUG org.apache.hadoop.hbase.master.AssignmentManager: Bulk assigning 1193 region(s) to localhost,11003,1336988444043
> 2012-05-14 11:34:40,929 DEBUG org.apache.hadoop.hbase.master.AssignmentManager: Bulk assigning done for localhost,11003,1336988065948
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5998) Bulk assignment: regionserver
optimization by using a temporary cache for table descriptors when
receveing an open regions request
Posted by "stack (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5998?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13393455#comment-13393455 ]
stack commented on HBASE-5998:
------------------------------
I committed the 0.94 backport over in hbase-6214.
> Bulk assignment: regionserver optimization by using a temporary cache for table descriptors when receveing an open regions request
> ----------------------------------------------------------------------------------------------------------------------------------
>
> Key: HBASE-5998
> URL: https://issues.apache.org/jira/browse/HBASE-5998
> Project: HBase
> Issue Type: Improvement
> Components: regionserver
> Affects Versions: 0.96.0
> Reporter: nkeywal
> Assignee: nkeywal
> Priority: Minor
> Fix For: 0.96.0
>
> Attachments: 5998.v2.patch, 5998.v3.patch, HBASE-5998_94.patch
>
>
> During the assignment, on the regionserver, before creating the handlers we load the table description. Even if there is a cache, we check the timestamps for each region, while it's not necessary. The test below is just with one node, with more nodes the benefit will improve. By limiting the time spent in HRegion#openRegion we increase the parallelization during cluster startup, as the master is using a pool of threads to call the RS.
> -- Without the fix
> 2012-05-14 11:40:52,501 DEBUG org.apache.hadoop.hbase.master.AssignmentManager: Bulk assigning 1193 region(s) to localhost,11003,1336988444043
> 2012-05-14 11:41:09,947 DEBUG org.apache.hadoop.hbase.master.AssignmentManager: Bulk assigning done for localhost,11003,1336988444043
> -- With the fix
> 2012-05-14 11:34:40,444 DEBUG org.apache.hadoop.hbase.master.AssignmentManager: Bulk assigning 1193 region(s) to localhost,11003,1336988444043
> 2012-05-14 11:34:40,929 DEBUG org.apache.hadoop.hbase.master.AssignmentManager: Bulk assigning done for localhost,11003,1336988065948
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5998) Bulk assignment: regionserver
optimization by using a temporary cache for table descriptors when
receveing an open regions request
Posted by "nkeywal (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5998?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
nkeywal updated HBASE-5998:
---------------------------
Attachment: 5998.v3.patch
> Bulk assignment: regionserver optimization by using a temporary cache for table descriptors when receveing an open regions request
> ----------------------------------------------------------------------------------------------------------------------------------
>
> Key: HBASE-5998
> URL: https://issues.apache.org/jira/browse/HBASE-5998
> Project: HBase
> Issue Type: Improvement
> Components: regionserver
> Affects Versions: 0.96.0
> Reporter: nkeywal
> Assignee: nkeywal
> Priority: Minor
> Fix For: 0.96.0
>
> Attachments: 5998.v2.patch, 5998.v3.patch
>
>
> During the assignment, on the regionserver, before creating the handlers we load the table description. Even if there is a cache, we check the timestamps for each region, while it's not necessary. The test below is just with one node, with more nodes the benefit will improve. By limiting the time spent in HRegion#openRegion we increase the parallelization during cluster startup, as the master is using a pool of threads to call the RS.
> -- Without the fix
> 2012-05-14 11:40:52,501 DEBUG org.apache.hadoop.hbase.master.AssignmentManager: Bulk assigning 1193 region(s) to localhost,11003,1336988444043
> 2012-05-14 11:41:09,947 DEBUG org.apache.hadoop.hbase.master.AssignmentManager: Bulk assigning done for localhost,11003,1336988444043
> -- With the fix
> 2012-05-14 11:34:40,444 DEBUG org.apache.hadoop.hbase.master.AssignmentManager: Bulk assigning 1193 region(s) to localhost,11003,1336988444043
> 2012-05-14 11:34:40,929 DEBUG org.apache.hadoop.hbase.master.AssignmentManager: Bulk assigning done for localhost,11003,1336988065948
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5998) Bulk assignment: regionserver
optimization by using a temporary cache for table descriptors when
receveing an open regions request
Posted by "Hudson (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5998?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13276300#comment-13276300 ]
Hudson commented on HBASE-5998:
-------------------------------
Integrated in HBase-TRUNK #2887 (See [https://builds.apache.org/job/HBase-TRUNK/2887/])
HBASE-5998 Bulk assignment: regionserver optimization by using a temporary cache for table descriptors when receveing an open regions request (N Keywal) (Revision 1338916)
Result = SUCCESS
tedyu :
Files :
* /hbase/trunk/src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java
> Bulk assignment: regionserver optimization by using a temporary cache for table descriptors when receveing an open regions request
> ----------------------------------------------------------------------------------------------------------------------------------
>
> Key: HBASE-5998
> URL: https://issues.apache.org/jira/browse/HBASE-5998
> Project: HBase
> Issue Type: Improvement
> Components: regionserver
> Affects Versions: 0.96.0
> Reporter: nkeywal
> Assignee: nkeywal
> Priority: Minor
> Fix For: 0.96.0
>
> Attachments: 5998.v2.patch, 5998.v3.patch
>
>
> During the assignment, on the regionserver, before creating the handlers we load the table description. Even if there is a cache, we check the timestamps for each region, while it's not necessary. The test below is just with one node, with more nodes the benefit will improve. By limiting the time spent in HRegion#openRegion we increase the parallelization during cluster startup, as the master is using a pool of threads to call the RS.
> -- Without the fix
> 2012-05-14 11:40:52,501 DEBUG org.apache.hadoop.hbase.master.AssignmentManager: Bulk assigning 1193 region(s) to localhost,11003,1336988444043
> 2012-05-14 11:41:09,947 DEBUG org.apache.hadoop.hbase.master.AssignmentManager: Bulk assigning done for localhost,11003,1336988444043
> -- With the fix
> 2012-05-14 11:34:40,444 DEBUG org.apache.hadoop.hbase.master.AssignmentManager: Bulk assigning 1193 region(s) to localhost,11003,1336988444043
> 2012-05-14 11:34:40,929 DEBUG org.apache.hadoop.hbase.master.AssignmentManager: Bulk assigning done for localhost,11003,1336988065948
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5998) Bulk assignment: regionserver
optimization by using a temporary cache for table descriptors when
receveing an open regions request
Posted by "nkeywal (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5998?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
nkeywal updated HBASE-5998:
---------------------------
Status: Patch Available (was: Open)
> Bulk assignment: regionserver optimization by using a temporary cache for table descriptors when receveing an open regions request
> ----------------------------------------------------------------------------------------------------------------------------------
>
> Key: HBASE-5998
> URL: https://issues.apache.org/jira/browse/HBASE-5998
> Project: HBase
> Issue Type: Improvement
> Components: regionserver
> Affects Versions: 0.96.0
> Reporter: nkeywal
> Assignee: nkeywal
> Priority: Minor
> Fix For: 0.96.0
>
> Attachments: 5998.v2.patch, 5998.v3.patch
>
>
> During the assignment, on the regionserver, before creating the handlers we load the table description. Even if there is a cache, we check the timestamps for each region, while it's not necessary. The test below is just with one node, with more nodes the benefit will improve. By limiting the time spent in HRegion#openRegion we increase the parallelization during cluster startup, as the master is using a pool of threads to call the RS.
> -- Without the fix
> 2012-05-14 11:40:52,501 DEBUG org.apache.hadoop.hbase.master.AssignmentManager: Bulk assigning 1193 region(s) to localhost,11003,1336988444043
> 2012-05-14 11:41:09,947 DEBUG org.apache.hadoop.hbase.master.AssignmentManager: Bulk assigning done for localhost,11003,1336988444043
> -- With the fix
> 2012-05-14 11:34:40,444 DEBUG org.apache.hadoop.hbase.master.AssignmentManager: Bulk assigning 1193 region(s) to localhost,11003,1336988444043
> 2012-05-14 11:34:40,929 DEBUG org.apache.hadoop.hbase.master.AssignmentManager: Bulk assigning done for localhost,11003,1336988065948
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5998) Bulk assignment: regionserver
optimization by using a temporary cache for table descriptors when
receveing an open regions request
Posted by "ramkrishna.s.vasudevan (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5998?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13295448#comment-13295448 ]
ramkrishna.s.vasudevan commented on HBASE-5998:
-----------------------------------------------
@Stack
We have a backport version. We can submit it today.
> Bulk assignment: regionserver optimization by using a temporary cache for table descriptors when receveing an open regions request
> ----------------------------------------------------------------------------------------------------------------------------------
>
> Key: HBASE-5998
> URL: https://issues.apache.org/jira/browse/HBASE-5998
> Project: HBase
> Issue Type: Improvement
> Components: regionserver
> Affects Versions: 0.96.0
> Reporter: nkeywal
> Assignee: nkeywal
> Priority: Minor
> Fix For: 0.96.0
>
> Attachments: 5998.v2.patch, 5998.v3.patch
>
>
> During the assignment, on the regionserver, before creating the handlers we load the table description. Even if there is a cache, we check the timestamps for each region, while it's not necessary. The test below is just with one node, with more nodes the benefit will improve. By limiting the time spent in HRegion#openRegion we increase the parallelization during cluster startup, as the master is using a pool of threads to call the RS.
> -- Without the fix
> 2012-05-14 11:40:52,501 DEBUG org.apache.hadoop.hbase.master.AssignmentManager: Bulk assigning 1193 region(s) to localhost,11003,1336988444043
> 2012-05-14 11:41:09,947 DEBUG org.apache.hadoop.hbase.master.AssignmentManager: Bulk assigning done for localhost,11003,1336988444043
> -- With the fix
> 2012-05-14 11:34:40,444 DEBUG org.apache.hadoop.hbase.master.AssignmentManager: Bulk assigning 1193 region(s) to localhost,11003,1336988444043
> 2012-05-14 11:34:40,929 DEBUG org.apache.hadoop.hbase.master.AssignmentManager: Bulk assigning done for localhost,11003,1336988065948
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5998) Bulk assignment: regionserver
optimization by using a temporary cache for table descriptors when
receveing an open regions request
Posted by "Zhihong Yu (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5998?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13276265#comment-13276265 ]
Zhihong Yu commented on HBASE-5998:
-----------------------------------
Integrated to trunk.
Thanks for the patch, N.
Thanks for the review, Ramkrishna.
> Bulk assignment: regionserver optimization by using a temporary cache for table descriptors when receveing an open regions request
> ----------------------------------------------------------------------------------------------------------------------------------
>
> Key: HBASE-5998
> URL: https://issues.apache.org/jira/browse/HBASE-5998
> Project: HBase
> Issue Type: Improvement
> Components: regionserver
> Affects Versions: 0.96.0
> Reporter: nkeywal
> Assignee: nkeywal
> Priority: Minor
> Fix For: 0.96.0
>
> Attachments: 5998.v2.patch, 5998.v3.patch
>
>
> During the assignment, on the regionserver, before creating the handlers we load the table description. Even if there is a cache, we check the timestamps for each region, while it's not necessary. The test below is just with one node, with more nodes the benefit will improve. By limiting the time spent in HRegion#openRegion we increase the parallelization during cluster startup, as the master is using a pool of threads to call the RS.
> -- Without the fix
> 2012-05-14 11:40:52,501 DEBUG org.apache.hadoop.hbase.master.AssignmentManager: Bulk assigning 1193 region(s) to localhost,11003,1336988444043
> 2012-05-14 11:41:09,947 DEBUG org.apache.hadoop.hbase.master.AssignmentManager: Bulk assigning done for localhost,11003,1336988444043
> -- With the fix
> 2012-05-14 11:34:40,444 DEBUG org.apache.hadoop.hbase.master.AssignmentManager: Bulk assigning 1193 region(s) to localhost,11003,1336988444043
> 2012-05-14 11:34:40,929 DEBUG org.apache.hadoop.hbase.master.AssignmentManager: Bulk assigning done for localhost,11003,1336988065948
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira