You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@hbase.apache.org by "stack (JIRA)" <ji...@apache.org> on 2010/09/21 01:57:32 UTC
[jira] Created: (HBASE-3018) Bulk assignment on startup runs
serially through the cluster servers assigning in bulk to one at a time
Bulk assignment on startup runs serially through the cluster servers assigning in bulk to one at a time
-------------------------------------------------------------------------------------------------------
Key: HBASE-3018
URL: https://issues.apache.org/jira/browse/HBASE-3018
Project: HBase
Issue Type: Improvement
Components: master
Reporter: stack
Fix For: 0.90.0
Multi-thread the bulk startup assignment of regions.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HBASE-3018) Bulk assignment on startup runs
serially through the cluster servers assigning in bulk to one at a time
Posted by "stack (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-3018?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
stack updated HBASE-3018:
-------------------------
Attachment: ba.txt
Testing this patch.
Fast assignment minimizes amount of time cluster is offline (SU backed out zk assignment because it was running too slow).
> Bulk assignment on startup runs serially through the cluster servers assigning in bulk to one at a time
> -------------------------------------------------------------------------------------------------------
>
> Key: HBASE-3018
> URL: https://issues.apache.org/jira/browse/HBASE-3018
> Project: HBase
> Issue Type: Improvement
> Components: master
> Reporter: stack
> Fix For: 0.90.0
>
> Attachments: ba.txt
>
>
> Multi-thread the bulk startup assignment of regions.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HBASE-3018) Bulk assignment on startup runs
serially through the cluster servers assigning in bulk to one at a time
Posted by "stack (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-3018?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
stack updated HBASE-3018:
-------------------------
Attachment: ba-v2.txt
Here is what I've been running on cluster. It works but its still not fast enough. Will open a new issue for doing bulk opens in single RPC (but it'll be a little tricky making it so we keep tickling zk while the RS is fielding all the opens).
{code}
M src/main/java/org/apache/hadoop/hbase/master/AssignmentManager.java
Removed reentrant lock that spanned moving region into
regionsintransition and the getting of lock on state object. It
was a bad idea. I found it actually deadlocked. It shouldn't be needed.
The call to assign will fail if the state instance is not of the right
'state' type.
Run a thread per server parcelling out region assignments so we assign
across the cluster concurrently rather than in series as we were doing.
{code}
> Bulk assignment on startup runs serially through the cluster servers assigning in bulk to one at a time
> -------------------------------------------------------------------------------------------------------
>
> Key: HBASE-3018
> URL: https://issues.apache.org/jira/browse/HBASE-3018
> Project: HBase
> Issue Type: Improvement
> Components: master
> Reporter: stack
> Fix For: 0.90.0
>
> Attachments: ba-v2.txt, ba.txt
>
>
> Multi-thread the bulk startup assignment of regions.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Resolved: (HBASE-3018) Bulk assignment on startup runs
serially through the cluster servers assigning in bulk to one at a time
Posted by "stack (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-3018?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
stack resolved HBASE-3018.
--------------------------
Assignee: stack
Resolution: Fixed
Committed this for now. Basically works. Needs to be faster still though. I made hbase-3019.
> Bulk assignment on startup runs serially through the cluster servers assigning in bulk to one at a time
> -------------------------------------------------------------------------------------------------------
>
> Key: HBASE-3018
> URL: https://issues.apache.org/jira/browse/HBASE-3018
> Project: HBase
> Issue Type: Improvement
> Components: master
> Reporter: stack
> Assignee: stack
> Fix For: 0.90.0
>
> Attachments: ba-v2.txt, ba.txt
>
>
> Multi-thread the bulk startup assignment of regions.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.