You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@hbase.apache.org by Krishna <re...@gmail.com> on 2014/12/13 02:04:54 UTC
RS disk capacity limits
Hi,
Is there any logical/practical limit on HBase RS storage size?
Which works better for HBase - a region server with 10 disks that are each
2 TB or 2 disks that are each 10TB?
I remember, one of the recommendations is to keep each disk on RS to be
less than 6 TB - is that correct?
Thanks
Re: RS disk capacity limits
Posted by Krishna <re...@gmail.com>.
Thanks all.
Lars, in your post: "Personally I would place the maximum disk space per
machine that can be served exclusively with HBase around 6T" - what is the
reasoning behind the limit of 6TB per RS? Does having 20 TB region server
(10 disks of 2TB) cause any issues?
On Sat, Dec 13, 2014 at 8:58 PM, lars hofhansl <la...@apache.org> wrote:
>
> See also:
>
> http://hadoop-hbase.blogspot.com/2013/01/hbase-region-server-memory-sizing.html
>
> ----- Original Message -----
> From: Nick Dimiduk <nd...@gmail.com>
> To: "dev@hbase.apache.org" <de...@hbase.apache.org>
> Cc:
> Sent: Saturday, December 13, 2014 5:37 PM
> Subject: Re: RS disk capacity limits
>
> Correct, you want more spindles/TB. Furthermore, you'll need RAM to use
> that storage effectively. GB RAM/TB of disk will very depending on your
> access patterns. 20 TB raw storage is about 6.6TB after replication. Even
> with compressed BlockCache and a compression ratio of 0.3, it'll be
> difficult to keep that much data hot.
>
> On Friday, December 12, 2014, 张铎 <pa...@gmail.com> wrote:
>
> > I think 10 disks each 2TB is better.
> > A disk can only seek about 50 times per second no matter it is 10TB or
> 2TB,
> > so more disks means you can do more seeks and increase the random read
> > write performance.
> >
> > 2014-12-13 9:04 GMT+08:00 Krishna <research800@gmail.com
> <javascript:;>>:
>
> > >
> > > Hi,
> > >
> > > Is there any logical/practical limit on HBase RS storage size?
> > > Which works better for HBase - a region server with 10 disks that are
> > each
> > > 2 TB or 2 disks that are each 10TB?
> > > I remember, one of the recommendations is to keep each disk on RS to be
> > > less than 6 TB - is that correct?
> > >
> > > Thanks
> > >
> >
>
Re: RS disk capacity limits
Posted by lars hofhansl <la...@apache.org>.
See also:
http://hadoop-hbase.blogspot.com/2013/01/hbase-region-server-memory-sizing.html
----- Original Message -----
From: Nick Dimiduk <nd...@gmail.com>
To: "dev@hbase.apache.org" <de...@hbase.apache.org>
Cc:
Sent: Saturday, December 13, 2014 5:37 PM
Subject: Re: RS disk capacity limits
Correct, you want more spindles/TB. Furthermore, you'll need RAM to use
that storage effectively. GB RAM/TB of disk will very depending on your
access patterns. 20 TB raw storage is about 6.6TB after replication. Even
with compressed BlockCache and a compression ratio of 0.3, it'll be
difficult to keep that much data hot.
On Friday, December 12, 2014, 张铎 <pa...@gmail.com> wrote:
> I think 10 disks each 2TB is better.
> A disk can only seek about 50 times per second no matter it is 10TB or 2TB,
> so more disks means you can do more seeks and increase the random read
> write performance.
>
> 2014-12-13 9:04 GMT+08:00 Krishna <research800@gmail.com <javascript:;>>:
> >
> > Hi,
> >
> > Is there any logical/practical limit on HBase RS storage size?
> > Which works better for HBase - a region server with 10 disks that are
> each
> > 2 TB or 2 disks that are each 10TB?
> > I remember, one of the recommendations is to keep each disk on RS to be
> > less than 6 TB - is that correct?
> >
> > Thanks
> >
>
Re: RS disk capacity limits
Posted by Nick Dimiduk <nd...@gmail.com>.
Correct, you want more spindles/TB. Furthermore, you'll need RAM to use
that storage effectively. GB RAM/TB of disk will very depending on your
access patterns. 20 TB raw storage is about 6.6TB after replication. Even
with compressed BlockCache and a compression ratio of 0.3, it'll be
difficult to keep that much data hot.
On Friday, December 12, 2014, 张铎 <pa...@gmail.com> wrote:
> I think 10 disks each 2TB is better.
> A disk can only seek about 50 times per second no matter it is 10TB or 2TB,
> so more disks means you can do more seeks and increase the random read
> write performance.
>
> 2014-12-13 9:04 GMT+08:00 Krishna <research800@gmail.com <javascript:;>>:
> >
> > Hi,
> >
> > Is there any logical/practical limit on HBase RS storage size?
> > Which works better for HBase - a region server with 10 disks that are
> each
> > 2 TB or 2 disks that are each 10TB?
> > I remember, one of the recommendations is to keep each disk on RS to be
> > less than 6 TB - is that correct?
> >
> > Thanks
> >
>
Re: RS disk capacity limits
Posted by 张铎 <pa...@gmail.com>.
I think 10 disks each 2TB is better.
A disk can only seek about 50 times per second no matter it is 10TB or 2TB,
so more disks means you can do more seeks and increase the random read
write performance.
2014-12-13 9:04 GMT+08:00 Krishna <re...@gmail.com>:
>
> Hi,
>
> Is there any logical/practical limit on HBase RS storage size?
> Which works better for HBase - a region server with 10 disks that are each
> 2 TB or 2 disks that are each 10TB?
> I remember, one of the recommendations is to keep each disk on RS to be
> less than 6 TB - is that correct?
>
> Thanks
>