[jira] Updated: (HBASE-71) [hbase] Master should rebalance region assignments periodically

["Bryan Duxbury (JIRA)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/HBASE-71?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Bryan Duxbury updated HBASE-71:
-------------------------------

    Attachment: 71-v2.patch

This version of the patch appears to actually work. Unit tests pass locally.

A      src/test/org/apache/hadoop/hbase/TestRegionRebalancing.java
Starts up and shuts down some region servers to see if rebalancing occurs as expected.
M      src/test/org/apache/hadoop/hbase/MiniHBaseCluster.java
-startRegionServer blocks until server is online
-stopRegionServer returns the RegionServerThread so you can join externally if you want.
M      src/test/org/apache/hadoop/hbase/MultiRegionTable.java
Updated to deal with change from SortedMap to Map for HRS's onlineRegions
M      src/test/org/apache/hadoop/hbase/master/OOMEHMaster.java
changed regionServerReport method to match new interface
M      src/java/org/apache/hadoop/hbase/HServerLoad.java
added setter for numberOfRequests
M      src/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java
-added isOnline var to track when an HRS is done setting itself up
-onlineRegions is a ConcurrentHashMap instead of a TreeMap now
-getMostLoadedRegions method to determine which regions might get rebalanced
M      src/java/org/apache/hadoop/hbase/regionserver/HRegion.java
more useful logging of keys when choosing not to split
M      src/java/org/apache/hadoop/hbase/LocalHBaseCluster.java
some new methods to help synchronize your starting/stopping of regoin servers
M      src/java/org/apache/hadoop/hbase/master/ServerManager.java
-factored processing of MSG_REPORT_CLOSE into processReportClose method
-short circuit straight into unassignedRegions if we're not deleting or offlining a region instead of always managing in ProcessRegionClose todo object
-fleshed out the getAverageLoad method
M      src/java/org/apache/hadoop/hbase/master/HMaster.java
-updated regionServerReport method signature
M      src/java/org/apache/hadoop/hbase/master/ProcessRegionClose.java
-added big class comment explaining what it's for
-renamed the reassignRegion variable to offlineRegion
-changed the process() method to more clearly do what you'd expect, and took out the reassignment branch of code
M      src/java/org/apache/hadoop/hbase/master/RegionManager.java
-new state added: regionsToOffline; methods related to this added
-assignRegions will now also unassign regions if the server is over the load average
-added unassignRegions method to facilitate deciding which regions to drop
M      src/java/org/apache/hadoop/hbase/master/ChangeTableState.java
-uses new markRegionForOffline
M      src/java/org/apache/hadoop/hbase/generated/regionserver/regionserver_jsp.java
M      src/java/org/apache/hadoop/hbase/io/HbaseObjectWritable.java
includes HRegionInfo[] class 
M      src/java/org/apache/hadoop/hbase/ipc/HMasterRegionInterface.java
updated regionServerReport method
M      src/webapps/regionserver/regionserver.jsp

> [hbase] Master should rebalance region assignments periodically
> ---------------------------------------------------------------
>
>                 Key: HBASE-71
>                 URL: https://issues.apache.org/jira/browse/HBASE-71
>             Project: Hadoop HBase
>          Issue Type: Improvement
>          Components: master
>            Reporter: Bryan Duxbury
>            Assignee: Bryan Duxbury
>             Fix For: 0.2.0
>
>         Attachments: 71-v2.patch, 71.patch
>
>
> The master currently only does region assignments at startup or when there are splits or dead regionservers. This means that if you join a new regionserver to the cluster after startup, it does not get assigned a fair share of the already-served regions as you would expect. It only gets a share of new regions being served.
> The master should periodically check the balance of regions, based on whatever assignment function, instead of in reaction to the above listed events.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.