You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@hbase.apache.org by Jerry Lam <ch...@gmail.com> on 2014/08/20 16:46:13 UTC
How to change MAX_FILES_PER_REGION_PER_FAMILY in LoadIncrementalHFiles?
Hi HBase users,
I wonder if anyone knows how to make change to
the MAX_FILES_PER_REGION_PER_FAMILY in LoadIncrementalHFiles?
The default value is 32 which is quite small.
HBase Version 0.98
Thank you,
Jerry
Re: How to change MAX_FILES_PER_REGION_PER_FAMILY in LoadIncrementalHFiles?
Posted by Jerry Lam <ch...@gmail.com>.
Hi Matteo,
Thank you for addressing the issue. For now, I will just set the variable
in hbase-site.xml.
Best Regards,
Jerry
On Wed, Aug 20, 2014 at 12:33 PM, Matteo Bertozzi <th...@gmail.com>
wrote:
> yeah sorry, just looked at the code and it is not initializing the tool
> correctly to pickup the -D configuration. let me fix that, I've opened
> HBASE-11789
> as you said with the current code only the hbase-site.xml conf is used, so
> you need to set the property there.
>
> Matteo
>
>
>
> On Wed, Aug 20, 2014 at 5:24 PM, Jerry Lam <ch...@gmail.com> wrote:
>
> > Hi Matteo,
> >
> > Thank you for the info. I tried it but it doesn't seem to take any
> effect.
> > Apparently the code in the LoadIncremtnalHFiles does not take anything
> > other than variables from hbase-site.xml which is unfortunate. We have
> more
> > than 32 hfiles to bulkload. So this is really not working...
> >
> > Best Regards,
> >
> > Jerry
> >
> >
> >
> >
> >
> >
> > On Wed, Aug 20, 2014 at 10:49 AM, Matteo Bertozzi <
> theo.bertozzi@gmail.com
> > >
> > wrote:
> >
> > > you should be able to use the -D option to set the new value
> > >
> > > LoadIncrementalHFiles
> > > -Dhbase.mapreduce.bulkload.max.hfiles.perRegion.perFamily=NEW_VALUE
> > >
> > > Matteo
> > >
> > >
> > >
> > > On Wed, Aug 20, 2014 at 3:46 PM, Jerry Lam <ch...@gmail.com>
> wrote:
> > >
> > > > Hi HBase users,
> > > >
> > > > I wonder if anyone knows how to make change to
> > > > the MAX_FILES_PER_REGION_PER_FAMILY in LoadIncrementalHFiles?
> > > >
> > > > The default value is 32 which is quite small.
> > > >
> > > > HBase Version 0.98
> > > >
> > > > Thank you,
> > > >
> > > > Jerry
> > > >
> > >
> >
>
Re: How to change MAX_FILES_PER_REGION_PER_FAMILY in LoadIncrementalHFiles?
Posted by Matteo Bertozzi <th...@gmail.com>.
yeah sorry, just looked at the code and it is not initializing the tool
correctly to pickup the -D configuration. let me fix that, I've opened
HBASE-11789
as you said with the current code only the hbase-site.xml conf is used, so
you need to set the property there.
Matteo
On Wed, Aug 20, 2014 at 5:24 PM, Jerry Lam <ch...@gmail.com> wrote:
> Hi Matteo,
>
> Thank you for the info. I tried it but it doesn't seem to take any effect.
> Apparently the code in the LoadIncremtnalHFiles does not take anything
> other than variables from hbase-site.xml which is unfortunate. We have more
> than 32 hfiles to bulkload. So this is really not working...
>
> Best Regards,
>
> Jerry
>
>
>
>
>
>
> On Wed, Aug 20, 2014 at 10:49 AM, Matteo Bertozzi <theo.bertozzi@gmail.com
> >
> wrote:
>
> > you should be able to use the -D option to set the new value
> >
> > LoadIncrementalHFiles
> > -Dhbase.mapreduce.bulkload.max.hfiles.perRegion.perFamily=NEW_VALUE
> >
> > Matteo
> >
> >
> >
> > On Wed, Aug 20, 2014 at 3:46 PM, Jerry Lam <ch...@gmail.com> wrote:
> >
> > > Hi HBase users,
> > >
> > > I wonder if anyone knows how to make change to
> > > the MAX_FILES_PER_REGION_PER_FAMILY in LoadIncrementalHFiles?
> > >
> > > The default value is 32 which is quite small.
> > >
> > > HBase Version 0.98
> > >
> > > Thank you,
> > >
> > > Jerry
> > >
> >
>
Re: How to change MAX_FILES_PER_REGION_PER_FAMILY in LoadIncrementalHFiles?
Posted by Jerry Lam <ch...@gmail.com>.
Hi Matteo,
Thank you for the info. I tried it but it doesn't seem to take any effect.
Apparently the code in the LoadIncremtnalHFiles does not take anything
other than variables from hbase-site.xml which is unfortunate. We have more
than 32 hfiles to bulkload. So this is really not working...
Best Regards,
Jerry
On Wed, Aug 20, 2014 at 10:49 AM, Matteo Bertozzi <th...@gmail.com>
wrote:
> you should be able to use the -D option to set the new value
>
> LoadIncrementalHFiles
> -Dhbase.mapreduce.bulkload.max.hfiles.perRegion.perFamily=NEW_VALUE
>
> Matteo
>
>
>
> On Wed, Aug 20, 2014 at 3:46 PM, Jerry Lam <ch...@gmail.com> wrote:
>
> > Hi HBase users,
> >
> > I wonder if anyone knows how to make change to
> > the MAX_FILES_PER_REGION_PER_FAMILY in LoadIncrementalHFiles?
> >
> > The default value is 32 which is quite small.
> >
> > HBase Version 0.98
> >
> > Thank you,
> >
> > Jerry
> >
>
Re: How to change MAX_FILES_PER_REGION_PER_FAMILY in LoadIncrementalHFiles?
Posted by Matteo Bertozzi <th...@gmail.com>.
you should be able to use the -D option to set the new value
LoadIncrementalHFiles
-Dhbase.mapreduce.bulkload.max.hfiles.perRegion.perFamily=NEW_VALUE
Matteo
On Wed, Aug 20, 2014 at 3:46 PM, Jerry Lam <ch...@gmail.com> wrote:
> Hi HBase users,
>
> I wonder if anyone knows how to make change to
> the MAX_FILES_PER_REGION_PER_FAMILY in LoadIncrementalHFiles?
>
> The default value is 32 which is quite small.
>
> HBase Version 0.98
>
> Thank you,
>
> Jerry
>