You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@hbase.apache.org by Ted Yu <yu...@gmail.com> on 2011/03/19 18:51:22 UTC
updating region server load for AssignmentManager
Hi,
I plan to utilize region server load for better balancing.
Currently the following method only calls
serverManager.regionServerReport():
public HMsg [] regionServerReport(HServerInfo serverInfo, HMsg msgs[],
HRegionInfo[] mostLoadedRegions)
This means AssignmentManager doesn't have valid server load information.
How about adding the following method to AssignmentManager ?
public void regionServerReport(HServerInfo serverInfo, HRegionInfo[]
mostLoadedRegions)
Please comment.
Re: updating region server load for AssignmentManager
Posted by Ted Yu <yu...@gmail.com>.
I have uploaded patch to HBASE-3676
I am testing a slight modification to AssignmentManager which skips regions
without any request so far:
2011-03-19 22:29:14,313 INFO
org.apache.hadoop.hbase.master.AssignmentManager: getAssignments() skipped
82 regions, minReqCount: 0
2011-03-19 22:29:14,313 INFO
org.apache.hadoop.hbase.master.AssignmentManager: getAssignments() skipped
76 regions, minReqCount: 0
2011-03-19 22:29:14,313 INFO
org.apache.hadoop.hbase.master.AssignmentManager: getAssignments() skipped
69 regions, minReqCount: 0
2011-03-19 22:29:14,314 INFO org.apache.hadoop.hbase.master.LoadBalancer:
Skipping load balancing. servers=7 regions=3 average=0.42857143
mostloaded=1 leastloaded=0
The 1 loaded server is the one hosting .META.
Please take a look.
On Sat, Mar 19, 2011 at 11:43 AM, <yu...@gmail.com> wrote:
> Thanks.
> I will create a jira and upload patch.
>
> Sent from my Verizon Wireless BlackBerry
>
> -----Original Message-----
> From: Stack <st...@duboce.net>
> Sender: saint.ack@gmail.com
> Date: Sat, 19 Mar 2011 11:37:49
> To: <de...@hbase.apache.org>
> Reply-To: dev@hbase.apache.org
> Subject: Re: updating region server load for AssignmentManager
>
> Ted:
>
> FYI, the regionServerReport is going away when we remove heartbeats:
> hbase-1502. I imagine that the master will get load info by reading
> it from zk; the assignment manager should be able to ask a Load object
> for the current state of loading (TBD).
>
> Does 1502 have to be done before you can make progress on loading? If
> so, just do the below for the moment and we'll figure how to rejigger
> it when 1502 goes in.
>
> St.Ack
>
> On Sat, Mar 19, 2011 at 10:51 AM, Ted Yu <yu...@gmail.com> wrote:
> > Hi,
> > I plan to utilize region server load for better balancing.
> > Currently the following method only calls
> > serverManager.regionServerReport():
> > public HMsg [] regionServerReport(HServerInfo serverInfo, HMsg msgs[],
> > HRegionInfo[] mostLoadedRegions)
> > This means AssignmentManager doesn't have valid server load information.
> >
> > How about adding the following method to AssignmentManager ?
> > public void regionServerReport(HServerInfo serverInfo, HRegionInfo[]
> > mostLoadedRegions)
> >
> > Please comment.
> >
>
Re: updating region server load for AssignmentManager
Posted by yu...@gmail.com.
Thanks.
I will create a jira and upload patch.
Sent from my Verizon Wireless BlackBerry
-----Original Message-----
From: Stack <st...@duboce.net>
Sender: saint.ack@gmail.com
Date: Sat, 19 Mar 2011 11:37:49
To: <de...@hbase.apache.org>
Reply-To: dev@hbase.apache.org
Subject: Re: updating region server load for AssignmentManager
Ted:
FYI, the regionServerReport is going away when we remove heartbeats:
hbase-1502. I imagine that the master will get load info by reading
it from zk; the assignment manager should be able to ask a Load object
for the current state of loading (TBD).
Does 1502 have to be done before you can make progress on loading? If
so, just do the below for the moment and we'll figure how to rejigger
it when 1502 goes in.
St.Ack
On Sat, Mar 19, 2011 at 10:51 AM, Ted Yu <yu...@gmail.com> wrote:
> Hi,
> I plan to utilize region server load for better balancing.
> Currently the following method only calls
> serverManager.regionServerReport():
> public HMsg [] regionServerReport(HServerInfo serverInfo, HMsg msgs[],
> HRegionInfo[] mostLoadedRegions)
> This means AssignmentManager doesn't have valid server load information.
>
> How about adding the following method to AssignmentManager ?
> public void regionServerReport(HServerInfo serverInfo, HRegionInfo[]
> mostLoadedRegions)
>
> Please comment.
>
Re: updating region server load for AssignmentManager
Posted by Stack <st...@duboce.net>.
Ted:
FYI, the regionServerReport is going away when we remove heartbeats:
hbase-1502. I imagine that the master will get load info by reading
it from zk; the assignment manager should be able to ask a Load object
for the current state of loading (TBD).
Does 1502 have to be done before you can make progress on loading? If
so, just do the below for the moment and we'll figure how to rejigger
it when 1502 goes in.
St.Ack
On Sat, Mar 19, 2011 at 10:51 AM, Ted Yu <yu...@gmail.com> wrote:
> Hi,
> I plan to utilize region server load for better balancing.
> Currently the following method only calls
> serverManager.regionServerReport():
> public HMsg [] regionServerReport(HServerInfo serverInfo, HMsg msgs[],
> HRegionInfo[] mostLoadedRegions)
> This means AssignmentManager doesn't have valid server load information.
>
> How about adding the following method to AssignmentManager ?
> public void regionServerReport(HServerInfo serverInfo, HRegionInfo[]
> mostLoadedRegions)
>
> Please comment.
>