You are viewing a plain text version of this content. The canonical link for it is here.

Posted to user@hbase.apache.org by David Koch <og...@googlemail.com> on 2014/02/11 00:03:07 UTC

MAX_FILE_SIZE not respected for table

Hi,

I reduced the MAX_FILE_SIZE to 64mb on an existing HBase table to force
more regions. However, it seems that this limit is not respected by quite a
large number of regions. I find several regions of >500mb size. I checked
this by doing:

hadoop fs -ls -h /hbase/<my_table>/*/<my_col_fam>

What can be done to force re-sizing?

Thanks,

David

Re: MAX_FILE_SIZE not respected for table

Posted by David Koch <og...@googlemail.com>.

Hi Jean-Marc,

Thanks. It's ok now. I thought I had run major_compact from shell but maybe
the effect was not immediate.

/David


On Tue, Feb 11, 2014 at 12:17 AM, Jean-Marc Spaggiari <
jean-marc@spaggiari.org> wrote:

> Hi David,
>
> If I'm correct, that will take action only when something will be written
> in the region. If you are only reading from the region, even if it's 500mb,
> and even if you have setup the max size to 64MB, there will be no split.
>
> If you do a compaction of the region it will split.
>
> If you want to all spit small, just to a major_compact using HBase shell.
>
> JM
>
>
> 2014-02-10 18:07 GMT-05:00 Ted Yu <yu...@gmail.com>:
>
> > See the following usage:
> >
> > For example, to change the max size of a family to 128MB, do:
> >
> >   hbase> alter 't1', METHOD => 'table_att', *MAX_FILESIZE* => '134217728'
> >
> >
> > On Mon, Feb 10, 2014 at 3:03 PM, David Koch <og...@googlemail.com>
> wrote:
> >
> > > Hi,
> > >
> > > I reduced the MAX_FILE_SIZE to 64mb on an existing HBase table to force
> > > more regions. However, it seems that this limit is not respected by
> > quite a
> > > large number of regions. I find several regions of >500mb size. I
> checked
> > > this by doing:
> > >
> > > hadoop fs -ls -h /hbase/<my_table>/*/<my_col_fam>
> > >
> > > What can be done to force re-sizing?
> > >
> > > Thanks,
> > >
> > > David
> > >
> >
>

Re: MAX_FILE_SIZE not respected for table

Posted by Jean-Marc Spaggiari <je...@spaggiari.org>.

Hi David,

If I'm correct, that will take action only when something will be written
in the region. If you are only reading from the region, even if it's 500mb,
and even if you have setup the max size to 64MB, there will be no split.

If you do a compaction of the region it will split.

If you want to all spit small, just to a major_compact using HBase shell.

JM

2014-02-10 18:07 GMT-05:00 Ted Yu <yu...@gmail.com>:

> See the following usage:
>
> For example, to change the max size of a family to 128MB, do:
>
>   hbase> alter 't1', METHOD => 'table_att', *MAX_FILESIZE* => '134217728'
>
>
> On Mon, Feb 10, 2014 at 3:03 PM, David Koch <og...@googlemail.com> wrote:
>
> > Hi,
> >
> > I reduced the MAX_FILE_SIZE to 64mb on an existing HBase table to force
> > more regions. However, it seems that this limit is not respected by
> quite a
> > large number of regions. I find several regions of >500mb size. I checked
> > this by doing:
> >
> > hadoop fs -ls -h /hbase/<my_table>/*/<my_col_fam>
> >
> > What can be done to force re-sizing?
> >
> > Thanks,
> >
> > David
> >
>

Re: MAX_FILE_SIZE not respected for table

Posted by Ted Yu <yu...@gmail.com>.

See the following usage:

For example, to change the max size of a family to 128MB, do:

  hbase> alter 't1', METHOD => 'table_att', *MAX_FILESIZE* => '134217728'


On Mon, Feb 10, 2014 at 3:03 PM, David Koch <og...@googlemail.com> wrote:

> Hi,
>
> I reduced the MAX_FILE_SIZE to 64mb on an existing HBase table to force
> more regions. However, it seems that this limit is not respected by quite a
> large number of regions. I find several regions of >500mb size. I checked
> this by doing:
>
> hadoop fs -ls -h /hbase/<my_table>/*/<my_col_fam>
>
> What can be done to force re-sizing?
>
> Thanks,
>
> David
>