You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@hbase.apache.org by Ron Echeverri <re...@maprtech.com> on 2013/09/19 01:17:08 UTC
quieting HBase metrics
In http://hbase.apache.org/book/hbase_metrics.html we see
"""
15.4.2. Warning To Ganglia Users
Warning to Ganglia Users: by default, HBase will emit a LOT of metrics per
RegionServer which may swamp your installation. Options include either
increasing Ganglia server capacity, or configuring HBase to emit fewer
metrics.
"""
However, although there is documentation to help set up Ganglia metrics for
HBase, there doesn't seem to be any that shows how to make HBase emit fewer
metrics. Could someone lend me a hand? I have increased the period in
hadoop-metrics.properties but that has proven insufficient.
thanks
rone
Re: quieting HBase metrics
Posted by Bing Jiang <ji...@gmail.com>.
hi, Ron.
Indeed, I have rewrote the implementation of
org.apache.hadoop.hbase.regionserver.metrics.RegionServerMetrics
org.apache.hadoop.hbase.regionserver.metrics.RegionServerDynamicMetrics
There are major changes as below:
1) change the context name for hbase to separate name for each other;
like hadoop-metrics.properties:
hbase-rsdm respresents DynamicMetrics
and hbase-rs represents RegionServerMetrics.
2) add a filterMapper in Metrics, when it updates to ganglia or other
metrics collector.
so in our implementation, it will fetch the filter from
hadoop-metrics.properties, and emit metrics as to settings.
3) In order to support dynamic add/remove filter in
hadoop-metrics.properties, we alternatively add a rpc to restart metrics
system in RegionServer without restarting rs.
Tks
2013/9/24 Ron Echeverri <re...@maprtech.com>
> Thanks, Bing, but i still don't understand *how* to configure that. Could
> you share your hadoop-metrics.properties file, or any other files you use
> for this?
>
> rone
>
>
> On Mon, Sep 23, 2013 at 8:29 PM, Bing Jiang <jiangbinglover@gmail.com
> >wrote:
>
> > hbase metrics contains different types:RegionServerMetrics and
> > RegionServerDynamicMetrics.
> > And there are more metrics emitted from RegionServerDynamicMetrics,
> > especially in case that there are more regions in RS.
> > RegionServerMetrics and RSDM share one metrics context, which makes the
> > control become difficult.
> >
> > In our production, we add a metrics filter in MetricsContext, which can
> be
> > configured as to our needs, which will reduce the amount of metrics, and
> > take a good effect.
> >
> >
> >
> >
> > 2013/9/24 Otis Gospodnetic <ot...@gmail.com>
> >
> >> Me, too!
> >>
> >> Otis
> >> --
> >> HBase Performance Monitoring -- http://sematext.com/spm
> >>
> >>
> >>
> >> On Mon, Sep 23, 2013 at 7:10 AM, Arati Patro <ar...@gmail.com>
> >> wrote:
> >> > Hi All,
> >> >
> >> >
> >> > I have the same query. Tried searching for a solution on the web but
> >> found
> >> > nothing very helpful. Is there some way I could disable metrics for
> >> every
> >> > table in HBase from being sent to Ganglia? Or just any information on
> >> how
> >> > to configure HBase to emit fewer metrics?
> >> >
> >> >
> >> > Thanks,
> >> > Arati Patro
> >> >
> >> >
> >> > On Thu, Sep 19, 2013 at 4:47 AM, Ron Echeverri <
> recheverri@maprtech.com
> >> >wrote:
> >> >
> >> >> In http://hbase.apache.org/book/hbase_metrics.html we see
> >> >>
> >> >> """
> >> >> 15.4.2. Warning To Ganglia Users
> >> >>
> >> >> Warning to Ganglia Users: by default, HBase will emit a LOT of
> metrics
> >> per
> >> >> RegionServer which may swamp your installation. Options include
> either
> >> >> increasing Ganglia server capacity, or configuring HBase to emit
> fewer
> >> >> metrics.
> >> >> """
> >> >>
> >> >> However, although there is documentation to help set up Ganglia
> >> metrics for
> >> >> HBase, there doesn't seem to be any that shows how to make HBase emit
> >> fewer
> >> >> metrics. Could someone lend me a hand? I have increased the period
> in
> >> >> hadoop-metrics.properties but that has proven insufficient.
> >> >>
> >> >> thanks
> >> >> rone
> >> >>
> >>
> >
> >
> >
> > --
> > Bing Jiang
> > weibo: http://weibo.com/jiangbinglover
> > BLOG: www.binospace.com
> > BLOG: http://blog.sina.com.cn/jiangbinglover
> > Focus on distributed computing, HDFS/HBase
> >
> >
> > 2013/9/24 Otis Gospodnetic <ot...@gmail.com>
> >
> >> Me, too!
> >>
> >> Otis
> >> --
> >> HBase Performance Monitoring -- http://sematext.com/spm
> >>
> >>
> >>
> >> On Mon, Sep 23, 2013 at 7:10 AM, Arati Patro <ar...@gmail.com>
> >> wrote:
> >> > Hi All,
> >> >
> >> >
> >> > I have the same query. Tried searching for a solution on the web but
> >> found
> >> > nothing very helpful. Is there some way I could disable metrics for
> >> every
> >> > table in HBase from being sent to Ganglia? Or just any information on
> >> how
> >> > to configure HBase to emit fewer metrics?
> >> >
> >> >
> >> > Thanks,
> >> > Arati Patro
> >> >
> >> >
> >> > On Thu, Sep 19, 2013 at 4:47 AM, Ron Echeverri <
> recheverri@maprtech.com
> >> >wrote:
> >> >
> >> >> In http://hbase.apache.org/book/hbase_metrics.html we see
> >> >>
> >> >> """
> >> >> 15.4.2. Warning To Ganglia Users
> >> >>
> >> >> Warning to Ganglia Users: by default, HBase will emit a LOT of
> metrics
> >> per
> >> >> RegionServer which may swamp your installation. Options include
> either
> >> >> increasing Ganglia server capacity, or configuring HBase to emit
> fewer
> >> >> metrics.
> >> >> """
> >> >>
> >> >> However, although there is documentation to help set up Ganglia
> >> metrics for
> >> >> HBase, there doesn't seem to be any that shows how to make HBase emit
> >> fewer
> >> >> metrics. Could someone lend me a hand? I have increased the period
> in
> >> >> hadoop-metrics.properties but that has proven insufficient.
> >> >>
> >> >> thanks
> >> >> rone
> >> >>
> >>
> >
> >
> >
> > --
> > Bing Jiang
> > Tel：(86)134-2619-1361
> > weibo: http://weibo.com/jiangbinglover
> > BLOG: www.binospace.com
> > BLOG: http://blog.sina.com.cn/jiangbinglover
> > Focus on distributed computing, HDFS/HBase
> >
>
--
Bing Jiang
Tel：(86)134-2619-1361
weibo: http://weibo.com/jiangbinglover
BLOG: www.binospace.com
BLOG: http://blog.sina.com.cn/jiangbinglover
Focus on distributed computing, HDFS/HBase
Re: quieting HBase metrics
Posted by Ron Echeverri <re...@maprtech.com>.
Thanks, Bing, but i still don't understand *how* to configure that. Could
you share your hadoop-metrics.properties file, or any other files you use
for this?
rone
On Mon, Sep 23, 2013 at 8:29 PM, Bing Jiang <ji...@gmail.com>wrote:
> hbase metrics contains different types:RegionServerMetrics and
> RegionServerDynamicMetrics.
> And there are more metrics emitted from RegionServerDynamicMetrics,
> especially in case that there are more regions in RS.
> RegionServerMetrics and RSDM share one metrics context, which makes the
> control become difficult.
>
> In our production, we add a metrics filter in MetricsContext, which can be
> configured as to our needs, which will reduce the amount of metrics, and
> take a good effect.
>
>
>
>
> 2013/9/24 Otis Gospodnetic <ot...@gmail.com>
>
>> Me, too!
>>
>> Otis
>> --
>> HBase Performance Monitoring -- http://sematext.com/spm
>>
>>
>>
>> On Mon, Sep 23, 2013 at 7:10 AM, Arati Patro <ar...@gmail.com>
>> wrote:
>> > Hi All,
>> >
>> >
>> > I have the same query. Tried searching for a solution on the web but
>> found
>> > nothing very helpful. Is there some way I could disable metrics for
>> every
>> > table in HBase from being sent to Ganglia? Or just any information on
>> how
>> > to configure HBase to emit fewer metrics?
>> >
>> >
>> > Thanks,
>> > Arati Patro
>> >
>> >
>> > On Thu, Sep 19, 2013 at 4:47 AM, Ron Echeverri <recheverri@maprtech.com
>> >wrote:
>> >
>> >> In http://hbase.apache.org/book/hbase_metrics.html we see
>> >>
>> >> """
>> >> 15.4.2. Warning To Ganglia Users
>> >>
>> >> Warning to Ganglia Users: by default, HBase will emit a LOT of metrics
>> per
>> >> RegionServer which may swamp your installation. Options include either
>> >> increasing Ganglia server capacity, or configuring HBase to emit fewer
>> >> metrics.
>> >> """
>> >>
>> >> However, although there is documentation to help set up Ganglia
>> metrics for
>> >> HBase, there doesn't seem to be any that shows how to make HBase emit
>> fewer
>> >> metrics. Could someone lend me a hand? I have increased the period in
>> >> hadoop-metrics.properties but that has proven insufficient.
>> >>
>> >> thanks
>> >> rone
>> >>
>>
>
>
>
> --
> Bing Jiang
> weibo: http://weibo.com/jiangbinglover
> BLOG: www.binospace.com
> BLOG: http://blog.sina.com.cn/jiangbinglover
> Focus on distributed computing, HDFS/HBase
>
>
> 2013/9/24 Otis Gospodnetic <ot...@gmail.com>
>
>> Me, too!
>>
>> Otis
>> --
>> HBase Performance Monitoring -- http://sematext.com/spm
>>
>>
>>
>> On Mon, Sep 23, 2013 at 7:10 AM, Arati Patro <ar...@gmail.com>
>> wrote:
>> > Hi All,
>> >
>> >
>> > I have the same query. Tried searching for a solution on the web but
>> found
>> > nothing very helpful. Is there some way I could disable metrics for
>> every
>> > table in HBase from being sent to Ganglia? Or just any information on
>> how
>> > to configure HBase to emit fewer metrics?
>> >
>> >
>> > Thanks,
>> > Arati Patro
>> >
>> >
>> > On Thu, Sep 19, 2013 at 4:47 AM, Ron Echeverri <recheverri@maprtech.com
>> >wrote:
>> >
>> >> In http://hbase.apache.org/book/hbase_metrics.html we see
>> >>
>> >> """
>> >> 15.4.2. Warning To Ganglia Users
>> >>
>> >> Warning to Ganglia Users: by default, HBase will emit a LOT of metrics
>> per
>> >> RegionServer which may swamp your installation. Options include either
>> >> increasing Ganglia server capacity, or configuring HBase to emit fewer
>> >> metrics.
>> >> """
>> >>
>> >> However, although there is documentation to help set up Ganglia
>> metrics for
>> >> HBase, there doesn't seem to be any that shows how to make HBase emit
>> fewer
>> >> metrics. Could someone lend me a hand? I have increased the period in
>> >> hadoop-metrics.properties but that has proven insufficient.
>> >>
>> >> thanks
>> >> rone
>> >>
>>
>
>
>
> --
> Bing Jiang
> Tel：(86)134-2619-1361
> weibo: http://weibo.com/jiangbinglover
> BLOG: www.binospace.com
> BLOG: http://blog.sina.com.cn/jiangbinglover
> Focus on distributed computing, HDFS/HBase
>
Re: quieting HBase metrics
Posted by Bing Jiang <ji...@gmail.com>.
hbase metrics contains different types:RegionServerMetrics and
RegionServerDynamicMetrics.
And there are more metrics emitted from RegionServerDynamicMetrics,
especially in case that there are more regions in RS.
RegionServerMetrics and RSDM share one metrics context, which makes the
control become difficult.
In our production, we add a metrics filter in MetricsContext, which can be
configured as to our needs, which will reduce the amount of metrics, and
take a good effect.
2013/9/24 Otis Gospodnetic <ot...@gmail.com>
> Me, too!
>
> Otis
> --
> HBase Performance Monitoring -- http://sematext.com/spm
>
>
>
> On Mon, Sep 23, 2013 at 7:10 AM, Arati Patro <ar...@gmail.com>
> wrote:
> > Hi All,
> >
> >
> > I have the same query. Tried searching for a solution on the web but
> found
> > nothing very helpful. Is there some way I could disable metrics for every
> > table in HBase from being sent to Ganglia? Or just any information on how
> > to configure HBase to emit fewer metrics?
> >
> >
> > Thanks,
> > Arati Patro
> >
> >
> > On Thu, Sep 19, 2013 at 4:47 AM, Ron Echeverri <recheverri@maprtech.com
> >wrote:
> >
> >> In http://hbase.apache.org/book/hbase_metrics.html we see
> >>
> >> """
> >> 15.4.2. Warning To Ganglia Users
> >>
> >> Warning to Ganglia Users: by default, HBase will emit a LOT of metrics
> per
> >> RegionServer which may swamp your installation. Options include either
> >> increasing Ganglia server capacity, or configuring HBase to emit fewer
> >> metrics.
> >> """
> >>
> >> However, although there is documentation to help set up Ganglia metrics
> for
> >> HBase, there doesn't seem to be any that shows how to make HBase emit
> fewer
> >> metrics. Could someone lend me a hand? I have increased the period in
> >> hadoop-metrics.properties but that has proven insufficient.
> >>
> >> thanks
> >> rone
> >>
>
--
Bing Jiang
weibo: http://weibo.com/jiangbinglover
BLOG: www.binospace.com
BLOG: http://blog.sina.com.cn/jiangbinglover
Focus on distributed computing, HDFS/HBase
2013/9/24 Otis Gospodnetic <ot...@gmail.com>
> Me, too!
>
> Otis
> --
> HBase Performance Monitoring -- http://sematext.com/spm
>
>
>
> On Mon, Sep 23, 2013 at 7:10 AM, Arati Patro <ar...@gmail.com>
> wrote:
> > Hi All,
> >
> >
> > I have the same query. Tried searching for a solution on the web but
> found
> > nothing very helpful. Is there some way I could disable metrics for every
> > table in HBase from being sent to Ganglia? Or just any information on how
> > to configure HBase to emit fewer metrics?
> >
> >
> > Thanks,
> > Arati Patro
> >
> >
> > On Thu, Sep 19, 2013 at 4:47 AM, Ron Echeverri <recheverri@maprtech.com
> >wrote:
> >
> >> In http://hbase.apache.org/book/hbase_metrics.html we see
> >>
> >> """
> >> 15.4.2. Warning To Ganglia Users
> >>
> >> Warning to Ganglia Users: by default, HBase will emit a LOT of metrics
> per
> >> RegionServer which may swamp your installation. Options include either
> >> increasing Ganglia server capacity, or configuring HBase to emit fewer
> >> metrics.
> >> """
> >>
> >> However, although there is documentation to help set up Ganglia metrics
> for
> >> HBase, there doesn't seem to be any that shows how to make HBase emit
> fewer
> >> metrics. Could someone lend me a hand? I have increased the period in
> >> hadoop-metrics.properties but that has proven insufficient.
> >>
> >> thanks
> >> rone
> >>
>
--
Bing Jiang
Tel：(86)134-2619-1361
weibo: http://weibo.com/jiangbinglover
BLOG: www.binospace.com
BLOG: http://blog.sina.com.cn/jiangbinglover
Focus on distributed computing, HDFS/HBase
Re: quieting HBase metrics
Posted by Otis Gospodnetic <ot...@gmail.com>.
Me, too!
Otis
--
HBase Performance Monitoring -- http://sematext.com/spm
On Mon, Sep 23, 2013 at 7:10 AM, Arati Patro <ar...@gmail.com> wrote:
> Hi All,
>
>
> I have the same query. Tried searching for a solution on the web but found
> nothing very helpful. Is there some way I could disable metrics for every
> table in HBase from being sent to Ganglia? Or just any information on how
> to configure HBase to emit fewer metrics?
>
>
> Thanks,
> Arati Patro
>
>
> On Thu, Sep 19, 2013 at 4:47 AM, Ron Echeverri <re...@maprtech.com>wrote:
>
>> In http://hbase.apache.org/book/hbase_metrics.html we see
>>
>> """
>> 15.4.2. Warning To Ganglia Users
>>
>> Warning to Ganglia Users: by default, HBase will emit a LOT of metrics per
>> RegionServer which may swamp your installation. Options include either
>> increasing Ganglia server capacity, or configuring HBase to emit fewer
>> metrics.
>> """
>>
>> However, although there is documentation to help set up Ganglia metrics for
>> HBase, there doesn't seem to be any that shows how to make HBase emit fewer
>> metrics. Could someone lend me a hand? I have increased the period in
>> hadoop-metrics.properties but that has proven insufficient.
>>
>> thanks
>> rone
>>
Re: quieting HBase metrics
Posted by Arati Patro <ar...@gmail.com>.
Hi All,
I have the same query. Tried searching for a solution on the web but found
nothing very helpful. Is there some way I could disable metrics for every
table in HBase from being sent to Ganglia? Or just any information on how
to configure HBase to emit fewer metrics?
Thanks,
Arati Patro
On Thu, Sep 19, 2013 at 4:47 AM, Ron Echeverri <re...@maprtech.com>wrote:
> In http://hbase.apache.org/book/hbase_metrics.html we see
>
> """
> 15.4.2. Warning To Ganglia Users
>
> Warning to Ganglia Users: by default, HBase will emit a LOT of metrics per
> RegionServer which may swamp your installation. Options include either
> increasing Ganglia server capacity, or configuring HBase to emit fewer
> metrics.
> """
>
> However, although there is documentation to help set up Ganglia metrics for
> HBase, there doesn't seem to be any that shows how to make HBase emit fewer
> metrics. Could someone lend me a hand? I have increased the period in
> hadoop-metrics.properties but that has proven insufficient.
>
> thanks
> rone
>