Region balancing question

[Bryan Keller <br...@gmail.com>]

I have a table on a 4 node test cluster. I also have some other tables on the cluster. The table in question has a total of 12 regions. I noticed that 1 node has 6 regions, another has zero, and the remaining two nodes have the expected 3 regions. I'm a little confused how this can happen.

The node with zero regions for the table has regions for other tables, so the node seems to be functioning properly. I'm not sure how this particular table can end up having no regions on this one node. Should the balancer distribute the regions evenly across the cluster for a given table? If so, is there a way to force it to rebalance?

Re: Region balancing question

[Elliott Clark <ec...@stumbleupon.com>]

Even when balancing by table the current default balancer does not
take into account region size, request rate, or memory usage.  If you
want those things there's a new balancer in trunk (slated for 0.96)
that gives these things.  However that patch is a little bit more
involved and applying it to the 0.94 trunk would take some work.  It's
do-able just more of an undertaking.

On Thu, Aug 2, 2012 at 10:30 AM, Kevin O'dell <ke...@cloudera.com> wrote:
>
> Bryan,
>
>   https://issues.apache.org/jira/browse/HBASE-3373 did not make it into CDH4, there is not a real easy way to do this on your own.  I have attached some sample code to get your started with writing your own(a colleague of mine wrote it).
>
>
>
> On Thu, Aug 2, 2012 at 1:20 PM, Bryan Keller <br...@gmail.com> wrote:
>>
>> I'm using 0.92 (Cloudera CDH4). Yes I definitely do not want to balance all regions across all tables together, as some tables are much more active than others and thus some regions are barely being used. I was thinking this might be what the balancer was doing. The regions are balanced in terms of total regions across all tables.
>>
>> Is there a way to change this behavior in 0.92?
>>
>> On Aug 2, 2012, at 2:54 AM, Anoop Sam John <an...@huawei.com> wrote:
>>
>> > Hi
>> >  Which version you are using?
>> > From 0.94 in the balancer there are 2 ways of balancing. One is by table balancing in which balancer will make sure the regions for one table is balanced across the RSs. But in the other way of balancing in generic way it will consider all the regions across all the tables as one group only and will balance all across the RSs.  Seems the 2nd way is happening with ur case. Can u check the total number of regions (across all the tables) in every RS. Is this balanced now?
>> >
>> > "hbase.master.loadbalance.bytable" config param can select any of the 2 ways of balancing. This is added in 0.94 version. You might be using some older version?
>> >
>> > -Anoop-
>> > ________________________________________
>> > From: Bryan Keller [bryanck@gmail.com]
>> > Sent: Thursday, August 02, 2012 11:37 AM
>> > To: user@hbase.apache.org
>> > Subject: Region balancing question
>> >
>> > I have a table on a 4 node test cluster. I also have some other tables on the cluster. The table in question has a total of 12 regions. I noticed that 1 node has 6 regions, another has zero, and the remaining two nodes have the expected 3 regions. I'm a little confused how this can happen.
>> >
>> > The node with zero regions for the table has regions for other tables, so the node seems to be functioning properly. I'm not sure how this particular table can end up having no regions on this one node. Should the balancer distribute the regions evenly across the cluster for a given table? If so, is there a way to force it to rebalance?
>>
>
>
>
> --
> Kevin O'Dell
> Customer Operations Engineer, Cloudera

Re: Region balancing question

[Kevin O'dell <ke...@cloudera.com>]

Bryan,

  https://issues.apache.org/jira/browse/HBASE-3373 did not make it into
CDH4, there is not a real easy way to do this on your own.  I have attached
some sample code to get your started with writing your own(a colleague of
mine wrote it).



On Thu, Aug 2, 2012 at 1:20 PM, Bryan Keller <br...@gmail.com> wrote:

> I'm using 0.92 (Cloudera CDH4). Yes I definitely do not want to balance
> all regions across all tables together, as some tables are much more active
> than others and thus some regions are barely being used. I was thinking
> this might be what the balancer was doing. The regions are balanced in
> terms of total regions across all tables.
>
> Is there a way to change this behavior in 0.92?
>
> On Aug 2, 2012, at 2:54 AM, Anoop Sam John <an...@huawei.com> wrote:
>
> > Hi
> >  Which version you are using?
> > From 0.94 in the balancer there are 2 ways of balancing. One is by table
> balancing in which balancer will make sure the regions for one table is
> balanced across the RSs. But in the other way of balancing in generic way
> it will consider all the regions across all the tables as one group only
> and will balance all across the RSs.  Seems the 2nd way is happening with
> ur case. Can u check the total number of regions (across all the tables) in
> every RS. Is this balanced now?
> >
> > "hbase.master.loadbalance.bytable" config param can select any of the 2
> ways of balancing. This is added in 0.94 version. You might be using some
> older version?
> >
> > -Anoop-
> > ________________________________________
> > From: Bryan Keller [bryanck@gmail.com]
> > Sent: Thursday, August 02, 2012 11:37 AM
> > To: user@hbase.apache.org
> > Subject: Region balancing question
> >
> > I have a table on a 4 node test cluster. I also have some other tables
> on the cluster. The table in question has a total of 12 regions. I noticed
> that 1 node has 6 regions, another has zero, and the remaining two nodes
> have the expected 3 regions. I'm a little confused how this can happen.
> >
> > The node with zero regions for the table has regions for other tables,
> so the node seems to be functioning properly. I'm not sure how this
> particular table can end up having no regions on this one node. Should the
> balancer distribute the regions evenly across the cluster for a given
> table? If so, is there a way to force it to rebalance?
>
>


-- 
Kevin O'Dell
Customer Operations Engineer, Cloudera

Re: Region balancing question

[Bryan Keller <br...@gmail.com>]

I'm using 0.92 (Cloudera CDH4). Yes I definitely do not want to balance all regions across all tables together, as some tables are much more active than others and thus some regions are barely being used. I was thinking this might be what the balancer was doing. The regions are balanced in terms of total regions across all tables.

Is there a way to change this behavior in 0.92?

On Aug 2, 2012, at 2:54 AM, Anoop Sam John <an...@huawei.com> wrote:

> Hi
>  Which version you are using? 
> From 0.94 in the balancer there are 2 ways of balancing. One is by table balancing in which balancer will make sure the regions for one table is balanced across the RSs. But in the other way of balancing in generic way it will consider all the regions across all the tables as one group only and will balance all across the RSs.  Seems the 2nd way is happening with ur case. Can u check the total number of regions (across all the tables) in every RS. Is this balanced now?
> 
> "hbase.master.loadbalance.bytable" config param can select any of the 2 ways of balancing. This is added in 0.94 version. You might be using some older version?
> 
> -Anoop-
> ________________________________________
> From: Bryan Keller [bryanck@gmail.com]
> Sent: Thursday, August 02, 2012 11:37 AM
> To: user@hbase.apache.org
> Subject: Region balancing question
> 
> I have a table on a 4 node test cluster. I also have some other tables on the cluster. The table in question has a total of 12 regions. I noticed that 1 node has 6 regions, another has zero, and the remaining two nodes have the expected 3 regions. I'm a little confused how this can happen.
> 
> The node with zero regions for the table has regions for other tables, so the node seems to be functioning properly. I'm not sure how this particular table can end up having no regions on this one node. Should the balancer distribute the regions evenly across the cluster for a given table? If so, is there a way to force it to rebalance?

Re: Region balancing question

[Ted Yu <yu...@gmail.com>]

0.92 doesn't have this feature. The change was rolled back.

FYI

On Thu, Aug 2, 2012 at 3:01 AM, Anoop Sam John <an...@huawei.com> wrote:

> Seems this is available from 0.92 version..   See HBASE-3373
>
> -Anoop-
> ________________________________________
> From: Anoop Sam John [anoopsj@huawei.com]
> Sent: Thursday, August 02, 2012 3:24 PM
> To: user@hbase.apache.org
> Subject: RE: Region balancing question
>
> Hi
>   Which version you are using?
> From 0.94 in the balancer there are 2 ways of balancing. One is by table
> balancing in which balancer will make sure the regions for one table is
> balanced across the RSs. But in the other way of balancing in generic way
> it will consider all the regions across all the tables as one group only
> and will balance all across the RSs.  Seems the 2nd way is happening with
> ur case. Can u check the total number of regions (across all the tables) in
> every RS. Is this balanced now?
>
> "hbase.master.loadbalance.bytable" config param can select any of the 2
> ways of balancing. This is added in 0.94 version. You might be using some
> older version?
>
> -Anoop-
> ________________________________________
> From: Bryan Keller [bryanck@gmail.com]
> Sent: Thursday, August 02, 2012 11:37 AM
> To: user@hbase.apache.org
> Subject: Region balancing question
>
> I have a table on a 4 node test cluster. I also have some other tables on
> the cluster. The table in question has a total of 12 regions. I noticed
> that 1 node has 6 regions, another has zero, and the remaining two nodes
> have the expected 3 regions. I'm a little confused how this can happen.
>
> The node with zero regions for the table has regions for other tables, so
> the node seems to be functioning properly. I'm not sure how this particular
> table can end up having no regions on this one node. Should the balancer
> distribute the regions evenly across the cluster for a given table? If so,
> is there a way to force it to rebalance?
>

RE: Region balancing question

["Ramkrishna.S.Vasudevan" <ra...@huawei.com>]

hbase.master.loadbalance.bytable - By default it is true.

Regards
Ram

> -----Original Message-----
> From: Anoop Sam John [mailto:anoopsj@huawei.com]
> Sent: Thursday, August 02, 2012 3:32 PM
> To: user@hbase.apache.org
> Subject: RE: Region balancing question
> 
> Seems this is available from 0.92 version..   See HBASE-3373
> 
> -Anoop-
> ________________________________________
> From: Anoop Sam John [anoopsj@huawei.com]
> Sent: Thursday, August 02, 2012 3:24 PM
> To: user@hbase.apache.org
> Subject: RE: Region balancing question
> 
> Hi
>   Which version you are using?
> From 0.94 in the balancer there are 2 ways of balancing. One is by
> table balancing in which balancer will make sure the regions for one
> table is balanced across the RSs. But in the other way of balancing in
> generic way it will consider all the regions across all the tables as
> one group only and will balance all across the RSs.  Seems the 2nd way
> is happening with ur case. Can u check the total number of regions
> (across all the tables) in every RS. Is this balanced now?
> 
> "hbase.master.loadbalance.bytable" config param can select any of the 2
> ways of balancing. This is added in 0.94 version. You might be using
> some older version?
> 
> -Anoop-
> ________________________________________
> From: Bryan Keller [bryanck@gmail.com]
> Sent: Thursday, August 02, 2012 11:37 AM
> To: user@hbase.apache.org
> Subject: Region balancing question
> 
> I have a table on a 4 node test cluster. I also have some other tables
> on the cluster. The table in question has a total of 12 regions. I
> noticed that 1 node has 6 regions, another has zero, and the remaining
> two nodes have the expected 3 regions. I'm a little confused how this
> can happen.
> 
> The node with zero regions for the table has regions for other tables,
> so the node seems to be functioning properly. I'm not sure how this
> particular table can end up having no regions on this one node. Should
> the balancer distribute the regions evenly across the cluster for a
> given table? If so, is there a way to force it to rebalance?=

RE: Region balancing question

[Anoop Sam John <an...@huawei.com>]

Seems this is available from 0.92 version..   See HBASE-3373

-Anoop-
________________________________________
From: Anoop Sam John [anoopsj@huawei.com]
Sent: Thursday, August 02, 2012 3:24 PM
To: user@hbase.apache.org
Subject: RE: Region balancing question

Hi
  Which version you are using?
>From 0.94 in the balancer there are 2 ways of balancing. One is by table balancing in which balancer will make sure the regions for one table is balanced across the RSs. But in the other way of balancing in generic way it will consider all the regions across all the tables as one group only and will balance all across the RSs.  Seems the 2nd way is happening with ur case. Can u check the total number of regions (across all the tables) in every RS. Is this balanced now?

"hbase.master.loadbalance.bytable" config param can select any of the 2 ways of balancing. This is added in 0.94 version. You might be using some older version?

-Anoop-
________________________________________
From: Bryan Keller [bryanck@gmail.com]
Sent: Thursday, August 02, 2012 11:37 AM
To: user@hbase.apache.org
Subject: Region balancing question

I have a table on a 4 node test cluster. I also have some other tables on the cluster. The table in question has a total of 12 regions. I noticed that 1 node has 6 regions, another has zero, and the remaining two nodes have the expected 3 regions. I'm a little confused how this can happen.

The node with zero regions for the table has regions for other tables, so the node seems to be functioning properly. I'm not sure how this particular table can end up having no regions on this one node. Should the balancer distribute the regions evenly across the cluster for a given table? If so, is there a way to force it to rebalance?

RE: Region balancing question

[Anoop Sam John <an...@huawei.com>]

Hi
  Which version you are using? 
>From 0.94 in the balancer there are 2 ways of balancing. One is by table balancing in which balancer will make sure the regions for one table is balanced across the RSs. But in the other way of balancing in generic way it will consider all the regions across all the tables as one group only and will balance all across the RSs.  Seems the 2nd way is happening with ur case. Can u check the total number of regions (across all the tables) in every RS. Is this balanced now?

"hbase.master.loadbalance.bytable" config param can select any of the 2 ways of balancing. This is added in 0.94 version. You might be using some older version?

-Anoop-
________________________________________
From: Bryan Keller [bryanck@gmail.com]
Sent: Thursday, August 02, 2012 11:37 AM
To: user@hbase.apache.org
Subject: Region balancing question

I have a table on a 4 node test cluster. I also have some other tables on the cluster. The table in question has a total of 12 regions. I noticed that 1 node has 6 regions, another has zero, and the remaining two nodes have the expected 3 regions. I'm a little confused how this can happen.

The node with zero regions for the table has regions for other tables, so the node seems to be functioning properly. I'm not sure how this particular table can end up having no regions on this one node. Should the balancer distribute the regions evenly across the cluster for a given table? If so, is there a way to force it to rebalance?