You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@hbase.apache.org by Ted Yu <yu...@gmail.com> on 2011/01/22 05:09:38 UTC
hbase config for 0.90
Hi,
I copied hbase-site.xml from our staging cluster running 0.20.6 to dev
cluster which runs 0.90
Can someone go over the attached file and see if any parameter is
incorrectly set ?
Thanks
Re: hbase config for 0.90
Posted by Tatsuya Kawano <ta...@gmail.com>.
Hi Ted,
This is not a direct answer to your question, but I found the Configuration chapter of The Apache HBase Book v0.90.0 is helpful. It contains a complete list of the properties and some important/recommended settings. You might want to check it if you haven't.
http://hbase.apache.org/book.html
Thanks,
--
Tatsuya Kawano
Tokyo, Japan
On Jan 22, 2011, at 1:09 PM, Ted Yu <yu...@gmail.com> wrote:
> Hi,
> I copied hbase-site.xml from our staging cluster running 0.20.6 to dev cluster which runs 0.90
>
> Can someone go over the attached file and see if any parameter is incorrectly set ?
>
> Thanks
> <hbase-site.xml>
Re: hbase config for 0.90
Posted by Ted Yu <yu...@gmail.com>.
I encountered different kinds of failure w.r.t. hbase 0.90 running our flow
over 200GB input data.
Some values from my config were derived from processing 500GB data using
0.20.6 on our staging cluster.
I couldn't find reference to 'hfile.min.blocksize.size' in 0.90 source code.
'hbase.zookeeper.property.maxClientCnxns' is high because of zookeeper issue
I mentioned in another thread.
I will report back after further experiments.
Thanks Stack.
On Fri, Jan 21, 2011 at 9:56 PM, Stack <st...@duboce.net> wrote:
> Remove:
>
> hbase.master
> hbase.regions.slop
> hbase.regionserver.restart.on.zk.expire
> hbase.regions.percheckin
> hbase.zookeeper.property.tickTime
>
> None of the above are pertinent any more.
>
> Your 'hbase.hregion.memstore.flush.size' is very high. Why so high?
> Same for your hfile blocksize (Though IIRC, you have lots of small
> records so you are probably trying to keep down the size of your
> indices?)
>
> Your 'hbase.regionserver.msginterval' is odd... Default is 3seconds
> and you have 23seconds. Any reason?
>
> Your zk timeout is very high, higher than default 3 minutes in 0.90.
> Maybe its OK though?
>
> The 'hbase.zookeeper.property.maxClientCnxns' setting of 400 is
> abnormal. It shouldn't usually have to be this high if you reuse your
> Configuration making HTables... but leave it as is. Probably does no
> harm.
>
> Hope that helps Ted,
> St.Ack
>
>
>
>
> On Fri, Jan 21, 2011 at 8:09 PM, Ted Yu <yu...@gmail.com> wrote:
> > Hi,
> > I copied hbase-site.xml from our staging cluster running 0.20.6 to dev
> > cluster which runs 0.90
> >
> > Can someone go over the attached file and see if any parameter is
> > incorrectly set ?
> >
> > Thanks
> >
>
Re: hbase config for 0.90
Posted by Stack <st...@duboce.net>.
Oh, makes sure you don't have an hbase-default.xml from an old hbase
in your conf dir (its in the hbase-0.90.0.jar now and doc'd here
http://hbase.apache.org/configuration.html#hbase_default_configurations).
St.Ack
On Fri, Jan 21, 2011 at 9:56 PM, Stack <st...@duboce.net> wrote:
> Remove:
>
> hbase.master
> hbase.regions.slop
> hbase.regionserver.restart.on.zk.expire
> hbase.regions.percheckin
> hbase.zookeeper.property.tickTime
>
> None of the above are pertinent any more.
>
> Your 'hbase.hregion.memstore.flush.size' is very high. Why so high?
> Same for your hfile blocksize (Though IIRC, you have lots of small
> records so you are probably trying to keep down the size of your
> indices?)
>
> Your 'hbase.regionserver.msginterval' is odd... Default is 3seconds
> and you have 23seconds. Any reason?
>
> Your zk timeout is very high, higher than default 3 minutes in 0.90.
> Maybe its OK though?
>
> The 'hbase.zookeeper.property.maxClientCnxns' setting of 400 is
> abnormal. It shouldn't usually have to be this high if you reuse your
> Configuration making HTables... but leave it as is. Probably does no
> harm.
>
> Hope that helps Ted,
> St.Ack
>
>
>
>
> On Fri, Jan 21, 2011 at 8:09 PM, Ted Yu <yu...@gmail.com> wrote:
>> Hi,
>> I copied hbase-site.xml from our staging cluster running 0.20.6 to dev
>> cluster which runs 0.90
>>
>> Can someone go over the attached file and see if any parameter is
>> incorrectly set ?
>>
>> Thanks
>>
>
Re: hbase config for 0.90
Posted by Stack <st...@duboce.net>.
Remove:
hbase.master
hbase.regions.slop
hbase.regionserver.restart.on.zk.expire
hbase.regions.percheckin
hbase.zookeeper.property.tickTime
None of the above are pertinent any more.
Your 'hbase.hregion.memstore.flush.size' is very high. Why so high?
Same for your hfile blocksize (Though IIRC, you have lots of small
records so you are probably trying to keep down the size of your
indices?)
Your 'hbase.regionserver.msginterval' is odd... Default is 3seconds
and you have 23seconds. Any reason?
Your zk timeout is very high, higher than default 3 minutes in 0.90.
Maybe its OK though?
The 'hbase.zookeeper.property.maxClientCnxns' setting of 400 is
abnormal. It shouldn't usually have to be this high if you reuse your
Configuration making HTables... but leave it as is. Probably does no
harm.
Hope that helps Ted,
St.Ack
On Fri, Jan 21, 2011 at 8:09 PM, Ted Yu <yu...@gmail.com> wrote:
> Hi,
> I copied hbase-site.xml from our staging cluster running 0.20.6 to dev
> cluster which runs 0.90
>
> Can someone go over the attached file and see if any parameter is
> incorrectly set ?
>
> Thanks
>