You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@hbase.apache.org by Geoff Hendrey <gh...@decarta.com> on 2011/06/07 20:33:47 UTC
distribution of regions to servers
I have a table with a hundred or so regions. When I look in the hbase
web ui, I see that all the regions are on one server. Of course we have
many other tables and lots of data. Some tables seem to distribute their
regions amongst many servers.
I know there probably isn't a "pat" answer to this but: wouldn't I want
a large table with many regions to be distributed across many machines?
Just curious to understand the nuance. If I *do* want a uniform
distribution of regions to servers, how would I achieve it?
-geoff
Re: distribution of regions to servers
Posted by Ted Yu <yu...@gmail.com>.
The assumption was that regions were not evenly distributed prior to
restarting.
If they were, user wouldn't select this policy.
We can this policy effective only once - retain assignment is selected
following this new policy.
Of course the dynamic portion of load balancer needs to select the under
loaded region server (for the underlying table) when reassigning a region.
On Wed, Jun 8, 2011 at 12:56 PM, Stack <st...@duboce.net> wrote:
> On Wed, Jun 8, 2011 at 12:50 PM, Ted Yu <yu...@gmail.com> wrote:
> > I am thinking of creating a new policy for region assignment at cluster
> > startup which assigns regions from each table in round-robin fashion.
> >
>
> Don't we want to retain assignments on startup since that will ensure
> greatest locality of data? Round-robin would undo all that (We used
> to do round-robin but worked hard to make it on restart we kept the
> pre-restart assignments).
>
> St.Ack
>
RE: distribution of regions to servers
Posted by Doug Meil <do...@explorysmedical.com>.
If I understand the history correctly, round-robin was used in .89, but "retains" is the policy for .90+.
My 2-cents is that if/when region-shuffling is required, I'd rather do that with another utility and keep that out of cluster startup.
-----Original Message-----
From: saint.ack@gmail.com [mailto:saint.ack@gmail.com] On Behalf Of Stack
Sent: Wednesday, June 08, 2011 3:56 PM
To: user@hbase.apache.org
Subject: Re: distribution of regions to servers
On Wed, Jun 8, 2011 at 12:50 PM, Ted Yu <yu...@gmail.com> wrote:
> I am thinking of creating a new policy for region assignment at
> cluster startup which assigns regions from each table in round-robin fashion.
>
Don't we want to retain assignments on startup since that will ensure greatest locality of data? Round-robin would undo all that (We used to do round-robin but worked hard to make it on restart we kept the pre-restart assignments).
St.Ack
Re: distribution of regions to servers
Posted by Stack <st...@duboce.net>.
On Wed, Jun 8, 2011 at 12:50 PM, Ted Yu <yu...@gmail.com> wrote:
> I am thinking of creating a new policy for region assignment at cluster
> startup which assigns regions from each table in round-robin fashion.
>
Don't we want to retain assignments on startup since that will ensure
greatest locality of data? Round-robin would undo all that (We used
to do round-robin but worked hard to make it on restart we kept the
pre-restart assignments).
St.Ack
Re: distribution of regions to servers
Posted by Ted Yu <yu...@gmail.com>.
In trunk this behavior has been improved.
Load balancer would move the youngest region off heavily loaded region
server.
See http://zhihongyu.blogspot.com/2011/04/load-balancer-in-hbase-090.html
I am thinking of creating a new policy for region assignment at cluster
startup which assigns regions from each table in round-robin fashion.
On Wed, Jun 8, 2011 at 3:04 AM, Kleegrewe, Christian <
christian.kleegrewe@siemens.com> wrote:
> Hi geoff,
>
> Since hbase balances not at table but at cluster basis it may happen that
> all the regions for one table are located at the same region server. The
> reason for this may be the way hbase does table splits. If a region exceeds
> the configured maximum size the region is split into two, but both resulting
> regions are on the same region server. In the background there is a load
> balancer running that copies regions between region servers in case that the
> cluster gets unbalanced. The region that is copied does not necessarily
> belong to the table that you are actually creating. It seems to me that the
> regions copied for balancing in most cases do not belong to the table that
> is currently created but there is a simple workaround, if you want a
> balancing at table level you should disable all the other tables when
> generating a the new table.
>
> regards
>
> Christian
>
> ------------------8<--------------------------
>
> Siemens AG
> Corporate Technology
> Corporate Research and Technologies
> CT T DE IT3
> Otto-Hahn-Ring 6
> 81739 München, Deutschland
> Tel.: +49 (89) 636-42722
> Fax: +49 (89) 636-41423
> mailto:christian.kleegrewe@siemens.com
>
> Siemens Aktiengesellschaft: Vorsitzender des Aufsichtsrats: Gerhard Cromme;
> Vorstand: Peter Löscher, Vorsitzender; Roland Busch, Brigitte Ederer, Klaus
> Helmrich, Joe Kaeser, Barbara Kux, Hermann Requardt, Siegfried Russwurm,
> Peter Y. Solmssen, Michael Süß; Sitz der Gesellschaft: Berlin und München,
> Deutschland; Registergericht: Berlin Charlottenburg, HRB 12300, München, HRB
> 6684; WEEE-Reg.-Nr. DE 23691322
>
>
> -----Original Message-----
> From: Geoff Hendrey [mailto:ghendrey@decarta.com]
> Sent: Tuesday, June 07, 2011 8:34 PM
> To: hbase-user@hadoop.apache.org
> Subject: distribution of regions to servers
>
> I have a table with a hundred or so regions. When I look in the hbase
> web ui, I see that all the regions are on one server. Of course we have
> many other tables and lots of data. Some tables seem to distribute their
> regions amongst many servers.
>
>
>
> I know there probably isn't a "pat" answer to this but: wouldn't I want
> a large table with many regions to be distributed across many machines?
> Just curious to understand the nuance. If I *do* want a uniform
> distribution of regions to servers, how would I achieve it?
>
>
>
> -geoff
>
>
RE: distribution of regions to servers
Posted by "Kleegrewe, Christian" <ch...@siemens.com>.
Hi geoff,
Since hbase balances not at table but at cluster basis it may happen that all the regions for one table are located at the same region server. The reason for this may be the way hbase does table splits. If a region exceeds the configured maximum size the region is split into two, but both resulting regions are on the same region server. In the background there is a load balancer running that copies regions between region servers in case that the cluster gets unbalanced. The region that is copied does not necessarily belong to the table that you are actually creating. It seems to me that the regions copied for balancing in most cases do not belong to the table that is currently created but there is a simple workaround, if you want a balancing at table level you should disable all the other tables when generating a the new table.
regards
Christian
------------------8<--------------------------
Siemens AG
Corporate Technology
Corporate Research and Technologies
CT T DE IT3
Otto-Hahn-Ring 6
81739 München, Deutschland
Tel.: +49 (89) 636-42722
Fax: +49 (89) 636-41423
mailto:christian.kleegrewe@siemens.com
Siemens Aktiengesellschaft: Vorsitzender des Aufsichtsrats: Gerhard Cromme; Vorstand: Peter Löscher, Vorsitzender; Roland Busch, Brigitte Ederer, Klaus Helmrich, Joe Kaeser, Barbara Kux, Hermann Requardt, Siegfried Russwurm, Peter Y. Solmssen, Michael Süß; Sitz der Gesellschaft: Berlin und München, Deutschland; Registergericht: Berlin Charlottenburg, HRB 12300, München, HRB 6684; WEEE-Reg.-Nr. DE 23691322
-----Original Message-----
From: Geoff Hendrey [mailto:ghendrey@decarta.com]
Sent: Tuesday, June 07, 2011 8:34 PM
To: hbase-user@hadoop.apache.org
Subject: distribution of regions to servers
I have a table with a hundred or so regions. When I look in the hbase
web ui, I see that all the regions are on one server. Of course we have
many other tables and lots of data. Some tables seem to distribute their
regions amongst many servers.
I know there probably isn't a "pat" answer to this but: wouldn't I want
a large table with many regions to be distributed across many machines?
Just curious to understand the nuance. If I *do* want a uniform
distribution of regions to servers, how would I achieve it?
-geoff
Re: distribution of regions to servers
Posted by Stack <st...@duboce.net>.
On Tue, Jun 7, 2011 at 3:13 PM, Geoff Hendrey <gh...@decarta.com> wrote:
> You're right. I was thinking there was an Hbase balancer tool for spreading the table's regions around. From your response, sounds like the hbase balancer is just software that lives inside the Hbase master, as opposed to a utility that could be run. Sorry for being dense. Did I get that right?
>
Yes. We don't have much on it in our book just yet:
http://hbase.apache.org/book.html#d470e3261
RE: distribution of regions to servers
Posted by Geoff Hendrey <gh...@decarta.com>.
You're right. I was thinking there was an Hbase balancer tool for spreading the table's regions around. From your response, sounds like the hbase balancer is just software that lives inside the Hbase master, as opposed to a utility that could be run. Sorry for being dense. Did I get that right?
-geoff
-----Original Message-----
From: saint.ack@gmail.com [mailto:saint.ack@gmail.com] On Behalf Of Stack
Sent: Tuesday, June 07, 2011 3:07 PM
To: user@hbase.apache.org
Cc: hbase-user@hadoop.apache.org
Subject: Re: distribution of regions to servers
I think you are thinking of the hdfs balancer as opposed to the hbase
region balancer that I was referring too. You could run a 0.90.3
master on your 0.90.1 cluster if you wanted....the master has the
balancer code (but probably best to upgrade all to 0.90.3 anyways).
St.Ack
On Tue, Jun 7, 2011 at 3:02 PM, Geoff Hendrey <gh...@decarta.com> wrote:
> Thanks for the reply!
>
> We are on 90.1.
>
> Does this mean I could run the 90.2 balancer on my existing 90.1 cluster
> to rebalance the tables a bit?
>
> -geoff
>
> -----Original Message-----
> From: saint.ack@gmail.com [mailto:saint.ack@gmail.com] On Behalf Of
> Stack
> Sent: Tuesday, June 07, 2011 2:16 PM
> To: user@hbase.apache.org
> Cc: hbase-user@hadoop.apache.org
> Subject: Re: distribution of regions to servers
>
> Hey Geoff:
>
> Is this a 0.90.2 (or .3) hbase? It had some fixup done on the
> balancer to make it a little more random.
>
> Otherwise, yes, a table should be better spread about the cluster.
> This has come up a few times here in the past. The case has been
> well-made by now that the balancer needs to take into consideration
> the table a region is from and make an effort at distributing tables
> evenly across the cluster (There is an issue for this already).
>
> St.Ack
>
> On Tue, Jun 7, 2011 at 11:33 AM, Geoff Hendrey <gh...@decarta.com>
> wrote:
>> I have a table with a hundred or so regions. When I look in the hbase
>> web ui, I see that all the regions are on one server. Of course we
> have
>> many other tables and lots of data. Some tables seem to distribute
> their
>> regions amongst many servers.
>>
>>
>>
>> I know there probably isn't a "pat" answer to this but: wouldn't I
> want
>> a large table with many regions to be distributed across many
> machines?
>> Just curious to understand the nuance. If I *do* want a uniform
>> distribution of regions to servers, how would I achieve it?
>>
>>
>>
>> -geoff
>>
>>
>
Re: distribution of regions to servers
Posted by Stack <st...@duboce.net>.
I think you are thinking of the hdfs balancer as opposed to the hbase
region balancer that I was referring too. You could run a 0.90.3
master on your 0.90.1 cluster if you wanted....the master has the
balancer code (but probably best to upgrade all to 0.90.3 anyways).
St.Ack
On Tue, Jun 7, 2011 at 3:02 PM, Geoff Hendrey <gh...@decarta.com> wrote:
> Thanks for the reply!
>
> We are on 90.1.
>
> Does this mean I could run the 90.2 balancer on my existing 90.1 cluster
> to rebalance the tables a bit?
>
> -geoff
>
> -----Original Message-----
> From: saint.ack@gmail.com [mailto:saint.ack@gmail.com] On Behalf Of
> Stack
> Sent: Tuesday, June 07, 2011 2:16 PM
> To: user@hbase.apache.org
> Cc: hbase-user@hadoop.apache.org
> Subject: Re: distribution of regions to servers
>
> Hey Geoff:
>
> Is this a 0.90.2 (or .3) hbase? It had some fixup done on the
> balancer to make it a little more random.
>
> Otherwise, yes, a table should be better spread about the cluster.
> This has come up a few times here in the past. The case has been
> well-made by now that the balancer needs to take into consideration
> the table a region is from and make an effort at distributing tables
> evenly across the cluster (There is an issue for this already).
>
> St.Ack
>
> On Tue, Jun 7, 2011 at 11:33 AM, Geoff Hendrey <gh...@decarta.com>
> wrote:
>> I have a table with a hundred or so regions. When I look in the hbase
>> web ui, I see that all the regions are on one server. Of course we
> have
>> many other tables and lots of data. Some tables seem to distribute
> their
>> regions amongst many servers.
>>
>>
>>
>> I know there probably isn't a "pat" answer to this but: wouldn't I
> want
>> a large table with many regions to be distributed across many
> machines?
>> Just curious to understand the nuance. If I *do* want a uniform
>> distribution of regions to servers, how would I achieve it?
>>
>>
>>
>> -geoff
>>
>>
>
RE: distribution of regions to servers
Posted by Geoff Hendrey <gh...@decarta.com>.
Thanks for the reply!
We are on 90.1.
Does this mean I could run the 90.2 balancer on my existing 90.1 cluster
to rebalance the tables a bit?
-geoff
-----Original Message-----
From: saint.ack@gmail.com [mailto:saint.ack@gmail.com] On Behalf Of
Stack
Sent: Tuesday, June 07, 2011 2:16 PM
To: user@hbase.apache.org
Cc: hbase-user@hadoop.apache.org
Subject: Re: distribution of regions to servers
Hey Geoff:
Is this a 0.90.2 (or .3) hbase? It had some fixup done on the
balancer to make it a little more random.
Otherwise, yes, a table should be better spread about the cluster.
This has come up a few times here in the past. The case has been
well-made by now that the balancer needs to take into consideration
the table a region is from and make an effort at distributing tables
evenly across the cluster (There is an issue for this already).
St.Ack
On Tue, Jun 7, 2011 at 11:33 AM, Geoff Hendrey <gh...@decarta.com>
wrote:
> I have a table with a hundred or so regions. When I look in the hbase
> web ui, I see that all the regions are on one server. Of course we
have
> many other tables and lots of data. Some tables seem to distribute
their
> regions amongst many servers.
>
>
>
> I know there probably isn't a "pat" answer to this but: wouldn't I
want
> a large table with many regions to be distributed across many
machines?
> Just curious to understand the nuance. If I *do* want a uniform
> distribution of regions to servers, how would I achieve it?
>
>
>
> -geoff
>
>
Re: distribution of regions to servers
Posted by Stack <st...@duboce.net>.
Hey Geoff:
Is this a 0.90.2 (or .3) hbase? It had some fixup done on the
balancer to make it a little more random.
Otherwise, yes, a table should be better spread about the cluster.
This has come up a few times here in the past. The case has been
well-made by now that the balancer needs to take into consideration
the table a region is from and make an effort at distributing tables
evenly across the cluster (There is an issue for this already).
St.Ack
On Tue, Jun 7, 2011 at 11:33 AM, Geoff Hendrey <gh...@decarta.com> wrote:
> I have a table with a hundred or so regions. When I look in the hbase
> web ui, I see that all the regions are on one server. Of course we have
> many other tables and lots of data. Some tables seem to distribute their
> regions amongst many servers.
>
>
>
> I know there probably isn't a "pat" answer to this but: wouldn't I want
> a large table with many regions to be distributed across many machines?
> Just curious to understand the nuance. If I *do* want a uniform
> distribution of regions to servers, how would I achieve it?
>
>
>
> -geoff
>
>