You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@hbase.apache.org by jinhong lu <lu...@gmail.com> on 2016/06/22 04:39:24 UTC
hbase get big table problem
I got a cluster of 200 regionserver, and one of the tables is about 3T and 5 billion lines. Is it possible to get about 8000 Gets per second(about 100,000 lines)?
I found YOUNG GC occurs every several senconds, each GC cost about 1second. if I set -Xmn bigger, the GC occurs every several minutes, but each GC cost more time.
Any suggetion? thanks.
=====================
Thanks,
lujinhong
Re: hbase get big table problem
Posted by Ted Yu <yu...@gmail.com>.
Jinhong:
Please take a look at 3rd paragraph of:
http://hbase.apache.org/book.html#gcpause
Cheers
On Wed, Jun 22, 2016 at 2:09 AM, Heng Chen <he...@gmail.com> wrote:
> 8000/200 = 40, if your table balance enough, each RS will serve 40
> requests per second, that is OK for RS. Have you try set xmn smaller to
> reduce young generation?
>
> 2016-06-22 16:12 GMT+08:00 jinhong lu <lu...@gmail.com>:
>
> > 400 regions, 8000 eps for the whole table. hbase 1.0, AND heap
> > -Xmx32G,-Xms32G, -Xmn4G
> >
> > Thanks,
> > lujinhong
> >
> > > 在 2016年6月22日，15:53，Heng Chen <he...@gmail.com> 写道：
> > >
> > > How many regions do you have for the table? 8000 qps for one RS or for
> > the
> > > whole table? What's your java heap size now? and what's your hbase
> > > version?
> > >
> > >
> > > 2016-06-22 12:39 GMT+08:00 jinhong lu <lu...@gmail.com>:
> > >
> > >> I got a cluster of 200 regionserver, and one of the tables is about 3T
> > and
> > >> 5 billion lines. Is it possible to get about 8000 Gets per
> second(about
> > >> 100,000 lines)?
> > >>
> > >> I found YOUNG GC occurs every several senconds, each GC cost about
> > >> 1second. if I set -Xmn bigger, the GC occurs every several minutes,
> but
> > >> each GC cost more time.
> > >>
> > >> Any suggetion? thanks.
> > >>
> > >>
> > >>
> > >> =====================
> > >> Thanks,
> > >> lujinhong
> > >>
> > >>
> >
> >
>
Re: hbase get big table problem
Posted by Heng Chen <he...@gmail.com>.
8000/200 = 40, if your table balance enough, each RS will serve 40
requests per second, that is OK for RS. Have you try set xmn smaller to
reduce young generation?
2016-06-22 16:12 GMT+08:00 jinhong lu <lu...@gmail.com>:
> 400 regions, 8000 eps for the whole table. hbase 1.0, AND heap
> -Xmx32G,-Xms32G, -Xmn4G
>
> Thanks,
> lujinhong
>
> > 在 2016年6月22日，15:53，Heng Chen <he...@gmail.com> 写道：
> >
> > How many regions do you have for the table? 8000 qps for one RS or for
> the
> > whole table? What's your java heap size now? and what's your hbase
> > version?
> >
> >
> > 2016-06-22 12:39 GMT+08:00 jinhong lu <lu...@gmail.com>:
> >
> >> I got a cluster of 200 regionserver, and one of the tables is about 3T
> and
> >> 5 billion lines. Is it possible to get about 8000 Gets per second(about
> >> 100,000 lines)?
> >>
> >> I found YOUNG GC occurs every several senconds, each GC cost about
> >> 1second. if I set -Xmn bigger, the GC occurs every several minutes, but
> >> each GC cost more time.
> >>
> >> Any suggetion? thanks.
> >>
> >>
> >>
> >> =====================
> >> Thanks,
> >> lujinhong
> >>
> >>
>
>
Re: hbase get big table problem
Posted by jinhong lu <lu...@gmail.com>.
400 regions, 8000 eps for the whole table. hbase 1.0, AND heap -Xmx32G,-Xms32G, -Xmn4G
Thanks,
lujinhong
> 在 2016年6月22日，15:53，Heng Chen <he...@gmail.com> 写道：
>
> How many regions do you have for the table? 8000 qps for one RS or for the
> whole table? What's your java heap size now? and what's your hbase
> version?
>
>
> 2016-06-22 12:39 GMT+08:00 jinhong lu <lu...@gmail.com>:
>
>> I got a cluster of 200 regionserver, and one of the tables is about 3T and
>> 5 billion lines. Is it possible to get about 8000 Gets per second(about
>> 100,000 lines)?
>>
>> I found YOUNG GC occurs every several senconds, each GC cost about
>> 1second. if I set -Xmn bigger, the GC occurs every several minutes, but
>> each GC cost more time.
>>
>> Any suggetion? thanks.
>>
>>
>>
>> =====================
>> Thanks,
>> lujinhong
>>
>>
Re: hbase get big table problem
Posted by Heng Chen <he...@gmail.com>.
How many regions do you have for the table? 8000 qps for one RS or for the
whole table? What's your java heap size now? and what's your hbase
version?
2016-06-22 12:39 GMT+08:00 jinhong lu <lu...@gmail.com>:
> I got a cluster of 200 regionserver, and one of the tables is about 3T and
> 5 billion lines. Is it possible to get about 8000 Gets per second(about
> 100,000 lines)?
>
> I found YOUNG GC occurs every several senconds, each GC cost about
> 1second. if I set -Xmn bigger, the GC occurs every several minutes, but
> each GC cost more time.
>
> Any suggetion? thanks.
>
>
>
> =====================
> Thanks,
> lujinhong
>
>