You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@hbase.apache.org by 谢良 <xi...@xiaomi.com> on 2013/12/27 04:51:01 UTC
答复: one table flushes at much smaller sizes than other?
Do you observe some log like "Too many hlogs" before flushing a few
dozen KB log ? Just want to double-check it's weather a forcing flush issue
or not:)
Thanks,
Liang
________________________________________
发件人: Norbert Burger [norbert.burger@gmail.com]
发送时间: 2013年12月27日 5:52
收件人: user@hbase.apache.org
主题: one table flushes at much smaller sizes than other?
Hi folks, we're currently running CDH4.1.2 (HBase 0.92 + patches).
Looking through the logs, I'm seeing that some tables are flushing at much
lower size than others (a few dozen KB as opposed to the expected 128 MB).
Looking through some of great case studies like [1], [2], and [3], it's not
clear to me why this is happening. We're using the defaults for upper,
lower limits (0.35 and 0.40, respectively), and the RS status pages show
that we're only using 25% of our memstore allocation.
Our regions/RS ratio is currently at 60-70. We do have some other tables
online where column families are skewed heavily sizewise -- could this be
causing other tables in our cluster to flush more frequently than others?
Norbert
[1] http://blog.sematext.com/2012/07/16/hbase-memstore-what-you-should-know/
[2] http://www.ngdata.com/visualizing-hbase-flushes-and-compactions/
[3] http://gbif.blogspot.com/2012/07/optimizing-writes-in-hbase.html
Re: 答复: one table flushes at much smaller sizes than other?
Posted by Ted Yu <yu...@gmail.com>.
You're welcome, Norbert.
On Fri, Dec 27, 2013 at 9:44 AM, Norbert Burger <no...@gmail.com>wrote:
> Thanks Ted - this config change appears to have reduced quite a bit of the
> memstore flushes.
>
> Norbert
>
>
> On Fri, Dec 27, 2013 at 12:03 AM, Ted Yu <yu...@gmail.com> wrote:
>
> > Raising maxlogs to 64 would give you improvement.
> >
> > Cheers
> >
> > On Dec 26, 2013, at 8:35 PM, Norbert Burger <no...@gmail.com>
> > wrote:
> >
> > > Thanks Ted and Liang. I do see the "Too may hlogs" line in our logs.
> > >
> > > We're using the default values for hbase.regionserver.maxlogs = 32 and
> > > hbase.regionserver.hlog.blocksize = 64MB, so
> > >
> > > So given a heapsize = 12GB and lowerLimit = 0.35, does it make sense
> > that I
> > > should raise maxlogs to be something like 64 or 72?
> > >
> > > Norbert
> > >
> > >
> > >
> > > On Thu, Dec 26, 2013 at 11:03 PM, Ted Yu <yu...@gmail.com> wrote:
> > >
> > >> Norbert:
> > >> What's your setting for hbase.regionserver.maxlogs ?
> > >>
> > >> Cheers
> > >>
> > >> On Dec 26, 2013, at 7:51 PM, 谢良 <xi...@xiaomi.com> wrote:
> > >>
> > >>> Do you observe some log like "Too many hlogs" before flushing a few
> > >>> dozen KB log ? Just want to double-check it's weather a forcing
> flush
> > >> issue
> > >>> or not:)
> > >>>
> > >>> Thanks,
> > >>> Liang
> > >>> ________________________________________
> > >>> 发件人: Norbert Burger [norbert.burger@gmail.com]
> > >>> 发送时间: 2013年12月27日 5:52
> > >>> 收件人: user@hbase.apache.org
> > >>> 主题: one table flushes at much smaller sizes than other?
> > >>>
> > >>> Hi folks, we're currently running CDH4.1.2 (HBase 0.92 + patches).
> > >>>
> > >>> Looking through the logs, I'm seeing that some tables are flushing at
> > >> much
> > >>> lower size than others (a few dozen KB as opposed to the expected 128
> > >> MB).
> > >>>
> > >>> Looking through some of great case studies like [1], [2], and [3],
> it's
> > >> not
> > >>> clear to me why this is happening. We're using the defaults for
> upper,
> > >>> lower limits (0.35 and 0.40, respectively), and the RS status pages
> > show
> > >>> that we're only using 25% of our memstore allocation.
> > >>>
> > >>> Our regions/RS ratio is currently at 60-70. We do have some other
> > tables
> > >>> online where column families are skewed heavily sizewise -- could
> this
> > be
> > >>> causing other tables in our cluster to flush more frequently than
> > others?
> > >>>
> > >>> Norbert
> > >>>
> > >>> [1]
> > >>
> > http://blog.sematext.com/2012/07/16/hbase-memstore-what-you-should-know/
> > >>> [2] http://www.ngdata.com/visualizing-hbase-flushes-and-compactions/
> > >>> [3] http://gbif.blogspot.com/2012/07/optimizing-writes-in-hbase.html
> > >>
> >
>
Re: 答复: one table flushes at much smaller sizes than other?
Posted by Norbert Burger <no...@gmail.com>.
Thanks Ted - this config change appears to have reduced quite a bit of the
memstore flushes.
Norbert
On Fri, Dec 27, 2013 at 12:03 AM, Ted Yu <yu...@gmail.com> wrote:
> Raising maxlogs to 64 would give you improvement.
>
> Cheers
>
> On Dec 26, 2013, at 8:35 PM, Norbert Burger <no...@gmail.com>
> wrote:
>
> > Thanks Ted and Liang. I do see the "Too may hlogs" line in our logs.
> >
> > We're using the default values for hbase.regionserver.maxlogs = 32 and
> > hbase.regionserver.hlog.blocksize = 64MB, so
> >
> > So given a heapsize = 12GB and lowerLimit = 0.35, does it make sense
> that I
> > should raise maxlogs to be something like 64 or 72?
> >
> > Norbert
> >
> >
> >
> > On Thu, Dec 26, 2013 at 11:03 PM, Ted Yu <yu...@gmail.com> wrote:
> >
> >> Norbert:
> >> What's your setting for hbase.regionserver.maxlogs ?
> >>
> >> Cheers
> >>
> >> On Dec 26, 2013, at 7:51 PM, 谢良 <xi...@xiaomi.com> wrote:
> >>
> >>> Do you observe some log like "Too many hlogs" before flushing a few
> >>> dozen KB log ? Just want to double-check it's weather a forcing flush
> >> issue
> >>> or not:)
> >>>
> >>> Thanks,
> >>> Liang
> >>> ________________________________________
> >>> 发件人: Norbert Burger [norbert.burger@gmail.com]
> >>> 发送时间: 2013年12月27日 5:52
> >>> 收件人: user@hbase.apache.org
> >>> 主题: one table flushes at much smaller sizes than other?
> >>>
> >>> Hi folks, we're currently running CDH4.1.2 (HBase 0.92 + patches).
> >>>
> >>> Looking through the logs, I'm seeing that some tables are flushing at
> >> much
> >>> lower size than others (a few dozen KB as opposed to the expected 128
> >> MB).
> >>>
> >>> Looking through some of great case studies like [1], [2], and [3], it's
> >> not
> >>> clear to me why this is happening. We're using the defaults for upper,
> >>> lower limits (0.35 and 0.40, respectively), and the RS status pages
> show
> >>> that we're only using 25% of our memstore allocation.
> >>>
> >>> Our regions/RS ratio is currently at 60-70. We do have some other
> tables
> >>> online where column families are skewed heavily sizewise -- could this
> be
> >>> causing other tables in our cluster to flush more frequently than
> others?
> >>>
> >>> Norbert
> >>>
> >>> [1]
> >>
> http://blog.sematext.com/2012/07/16/hbase-memstore-what-you-should-know/
> >>> [2] http://www.ngdata.com/visualizing-hbase-flushes-and-compactions/
> >>> [3] http://gbif.blogspot.com/2012/07/optimizing-writes-in-hbase.html
> >>
>
Re: 答复: one table flushes at much smaller sizes than other?
Posted by Ted Yu <yu...@gmail.com>.
Raising maxlogs to 64 would give you improvement.
Cheers
On Dec 26, 2013, at 8:35 PM, Norbert Burger <no...@gmail.com> wrote:
> Thanks Ted and Liang. I do see the "Too may hlogs" line in our logs.
>
> We're using the default values for hbase.regionserver.maxlogs = 32 and
> hbase.regionserver.hlog.blocksize = 64MB, so
>
> So given a heapsize = 12GB and lowerLimit = 0.35, does it make sense that I
> should raise maxlogs to be something like 64 or 72?
>
> Norbert
>
>
>
> On Thu, Dec 26, 2013 at 11:03 PM, Ted Yu <yu...@gmail.com> wrote:
>
>> Norbert:
>> What's your setting for hbase.regionserver.maxlogs ?
>>
>> Cheers
>>
>> On Dec 26, 2013, at 7:51 PM, 谢良 <xi...@xiaomi.com> wrote:
>>
>>> Do you observe some log like "Too many hlogs" before flushing a few
>>> dozen KB log ? Just want to double-check it's weather a forcing flush
>> issue
>>> or not:)
>>>
>>> Thanks,
>>> Liang
>>> ________________________________________
>>> 发件人: Norbert Burger [norbert.burger@gmail.com]
>>> 发送时间: 2013年12月27日 5:52
>>> 收件人: user@hbase.apache.org
>>> 主题: one table flushes at much smaller sizes than other?
>>>
>>> Hi folks, we're currently running CDH4.1.2 (HBase 0.92 + patches).
>>>
>>> Looking through the logs, I'm seeing that some tables are flushing at
>> much
>>> lower size than others (a few dozen KB as opposed to the expected 128
>> MB).
>>>
>>> Looking through some of great case studies like [1], [2], and [3], it's
>> not
>>> clear to me why this is happening. We're using the defaults for upper,
>>> lower limits (0.35 and 0.40, respectively), and the RS status pages show
>>> that we're only using 25% of our memstore allocation.
>>>
>>> Our regions/RS ratio is currently at 60-70. We do have some other tables
>>> online where column families are skewed heavily sizewise -- could this be
>>> causing other tables in our cluster to flush more frequently than others?
>>>
>>> Norbert
>>>
>>> [1]
>> http://blog.sematext.com/2012/07/16/hbase-memstore-what-you-should-know/
>>> [2] http://www.ngdata.com/visualizing-hbase-flushes-and-compactions/
>>> [3] http://gbif.blogspot.com/2012/07/optimizing-writes-in-hbase.html
>>
Re: 答复: one table flushes at much smaller sizes than other?
Posted by Norbert Burger <no...@gmail.com>.
Thanks Ted and Liang. I do see the "Too may hlogs" line in our logs.
We're using the default values for hbase.regionserver.maxlogs = 32 and
hbase.regionserver.hlog.blocksize = 64MB, so
So given a heapsize = 12GB and lowerLimit = 0.35, does it make sense that I
should raise maxlogs to be something like 64 or 72?
Norbert
On Thu, Dec 26, 2013 at 11:03 PM, Ted Yu <yu...@gmail.com> wrote:
> Norbert:
> What's your setting for hbase.regionserver.maxlogs ?
>
> Cheers
>
> On Dec 26, 2013, at 7:51 PM, 谢良 <xi...@xiaomi.com> wrote:
>
> > Do you observe some log like "Too many hlogs" before flushing a few
> > dozen KB log ? Just want to double-check it's weather a forcing flush
> issue
> > or not:)
> >
> > Thanks,
> > Liang
> > ________________________________________
> > 发件人: Norbert Burger [norbert.burger@gmail.com]
> > 发送时间: 2013年12月27日 5:52
> > 收件人: user@hbase.apache.org
> > 主题: one table flushes at much smaller sizes than other?
> >
> > Hi folks, we're currently running CDH4.1.2 (HBase 0.92 + patches).
> >
> > Looking through the logs, I'm seeing that some tables are flushing at
> much
> > lower size than others (a few dozen KB as opposed to the expected 128
> MB).
> >
> > Looking through some of great case studies like [1], [2], and [3], it's
> not
> > clear to me why this is happening. We're using the defaults for upper,
> > lower limits (0.35 and 0.40, respectively), and the RS status pages show
> > that we're only using 25% of our memstore allocation.
> >
> > Our regions/RS ratio is currently at 60-70. We do have some other tables
> > online where column families are skewed heavily sizewise -- could this be
> > causing other tables in our cluster to flush more frequently than others?
> >
> > Norbert
> >
> > [1]
> http://blog.sematext.com/2012/07/16/hbase-memstore-what-you-should-know/
> > [2] http://www.ngdata.com/visualizing-hbase-flushes-and-compactions/
> > [3] http://gbif.blogspot.com/2012/07/optimizing-writes-in-hbase.html
>
Re: 答复: one table flushes at much smaller sizes than other?
Posted by Ted Yu <yu...@gmail.com>.
Norbert:
What's your setting for hbase.regionserver.maxlogs ?
Cheers
On Dec 26, 2013, at 7:51 PM, 谢良 <xi...@xiaomi.com> wrote:
> Do you observe some log like "Too many hlogs" before flushing a few
> dozen KB log ? Just want to double-check it's weather a forcing flush issue
> or not:)
>
> Thanks,
> Liang
> ________________________________________
> 发件人: Norbert Burger [norbert.burger@gmail.com]
> 发送时间: 2013年12月27日 5:52
> 收件人: user@hbase.apache.org
> 主题: one table flushes at much smaller sizes than other?
>
> Hi folks, we're currently running CDH4.1.2 (HBase 0.92 + patches).
>
> Looking through the logs, I'm seeing that some tables are flushing at much
> lower size than others (a few dozen KB as opposed to the expected 128 MB).
>
> Looking through some of great case studies like [1], [2], and [3], it's not
> clear to me why this is happening. We're using the defaults for upper,
> lower limits (0.35 and 0.40, respectively), and the RS status pages show
> that we're only using 25% of our memstore allocation.
>
> Our regions/RS ratio is currently at 60-70. We do have some other tables
> online where column families are skewed heavily sizewise -- could this be
> causing other tables in our cluster to flush more frequently than others?
>
> Norbert
>
> [1] http://blog.sematext.com/2012/07/16/hbase-memstore-what-you-should-know/
> [2] http://www.ngdata.com/visualizing-hbase-flushes-and-compactions/
> [3] http://gbif.blogspot.com/2012/07/optimizing-writes-in-hbase.html