question about region assignment

[Jack Levin <ma...@gmail.com>]

Hello, we have this one table what about 12 regions, that is super hot
with writes, for some reason most of the regions were assigned to a
single server, which cause it to flush and compact every 10 minutes,
causing suboptimal performance.  We do use random row_keys, so I don't
see how sorted ROWs can land on single region anyway, the issue was
that each region was assigned to the same region server, is there a
way to mitigate the issue?  I closed closing the regions manually, but
they went right back to the same server, finally I shutdown RS
process, and that table's regions went in random places around the
cluster, but still its puzzling, anyone had issues like that?

-Jack

Re: question about region assignment

[Jean-Daniel Cryans <jd...@apache.org>]

I guess you only have 1 table and you write to it sequentially such
that the regions that get moved are the ones you're not writing to?
Then yeah it's unusable but also you might be doing wrong (if that's
really your situation). You need to make sure you don't end up writing
to the latest regions, this blog post describes it best
http://ikaisays.com/2011/01/25/app-engine-datastore-tip-monotonically-increasing-values-are-bad/

In any case, the load balancer was improved quite a bit in 0.90.2 (the
RC was voted for release yesterday, should be out real soon) mainly in
this jira https://issues.apache.org/jira/browse/HBASE-3586

J-D

> hello,
>
> I am encountering the same problem with 0.90.1. All the created regions (via
> inserting puts via java api) are assigned to a single region server. What is
> interesting is that the load balancing works, since the number of regions is
> changing on all regionservers - some 'old' regions on the highly loaded
> regionserver are being re-assigned to other regionservers. I guess this is a
> bug, since the system is sort of unusable like this.
>

Re: question about region assignment

[Stan Barton <st...@gmail.com>]

Jack Levin <ma...@...> writes:

> 
> I am on 0.89-830
> 
> On Tue, Mar 29, 2011 at 10:44 AM, Ted Yu <yu...@...> wrote:
> > Are you using 0.90.1 (where regions are randomly distributed across cluser)
> > ?
> > I logged HBASE-3373 but was told it is specific to our usage.
> >
> > On Tue, Mar 29, 2011 at 10:39 AM, Jack Levin <ma...@...> wrote:
> >
> >> Hello, we have this one table what about 12 regions, that is super hot
> >> with writes, for some reason most of the regions were assigned to a
> >> single server, which cause it to flush and compact every 10 minutes,
> >> causing suboptimal performance.  We do use random row_keys, so I don't
> >> see how sorted ROWs can land on single region anyway, the issue was
> >> that each region was assigned to the same region server, is there a
> >> way to mitigate the issue?  I closed closing the regions manually, but
> >> they went right back to the same server, finally I shutdown RS
> >> process, and that table's regions went in random places around the
> >> cluster, but still its puzzling, anyone had issues like that?
> >>
> >> -Jack
> >>
> >
> 
> 

hello,

I am encountering the same problem with 0.90.1. All the created regions (via
inserting puts via java api) are assigned to a single region server. What is
interesting is that the load balancing works, since the number of regions is
changing on all regionservers - some 'old' regions on the highly loaded
regionserver are being re-assigned to other regionservers. I guess this is a
bug, since the system is sort of unusable like this.

Stan

Re: question about region assignment

[Jack Levin <ma...@gmail.com>]

I am on 0.89-830

On Tue, Mar 29, 2011 at 10:44 AM, Ted Yu <yu...@gmail.com> wrote:
> Are you using 0.90.1 (where regions are randomly distributed across cluser)
> ?
> I logged HBASE-3373 but was told it is specific to our usage.
>
> On Tue, Mar 29, 2011 at 10:39 AM, Jack Levin <ma...@gmail.com> wrote:
>
>> Hello, we have this one table what about 12 regions, that is super hot
>> with writes, for some reason most of the regions were assigned to a
>> single server, which cause it to flush and compact every 10 minutes,
>> causing suboptimal performance.  We do use random row_keys, so I don't
>> see how sorted ROWs can land on single region anyway, the issue was
>> that each region was assigned to the same region server, is there a
>> way to mitigate the issue?  I closed closing the regions manually, but
>> they went right back to the same server, finally I shutdown RS
>> process, and that table's regions went in random places around the
>> cluster, but still its puzzling, anyone had issues like that?
>>
>> -Jack
>>
>

Re: question about region assignment

[Ted Yu <yu...@gmail.com>]

Are you using 0.90.1 (where regions are randomly distributed across cluser)
?
I logged HBASE-3373 but was told it is specific to our usage.

On Tue, Mar 29, 2011 at 10:39 AM, Jack Levin <ma...@gmail.com> wrote:

> Hello, we have this one table what about 12 regions, that is super hot
> with writes, for some reason most of the regions were assigned to a
> single server, which cause it to flush and compact every 10 minutes,
> causing suboptimal performance.  We do use random row_keys, so I don't
> see how sorted ROWs can land on single region anyway, the issue was
> that each region was assigned to the same region server, is there a
> way to mitigate the issue?  I closed closing the regions manually, but
> they went right back to the same server, finally I shutdown RS
> process, and that table's regions went in random places around the
> cluster, but still its puzzling, anyone had issues like that?
>
> -Jack
>