You are viewing a plain text version of this content. The canonical link for it is here.
Posted to hdfs-dev@hadoop.apache.org by lei liu <li...@gmail.com> on 2013/11/21 04:34:19 UTC
Metrics2 code
I use cdh-4.3.1 version. I am reading the code about metrics2.
There are COUNTER and GAUGE metric type in metrics v2. What is the difference
between the two?
There is @Metric MutableCounterLong bytesWritten attribute in
DataNodeMetrics, which is used to statistics written bytes per second on
DataNode.So I think the value of MutableCounterLong should be divided
by 10and be reseted to zero per ten seconds in
MutableCounterLong.snapshot
method, is that right? But MutableCounterLong.snapshot method don't do
that. I miss anything please tell me.
Thanks,
LiuLei
Re: Metrics2 code
Posted by Andrew Wang <an...@cloudera.com>.
I'm not sure how Ganglia works, but I suspect that it's polling the metric
every 10 seconds and taking the difference to give you the bytes written in
the last 10 seconds, and divide that by 10 to do bytes/second.
Best,
Andrew
On Thu, Nov 21, 2013 at 1:57 AM, lei liu <li...@gmail.com> wrote:
> Hi Andrew, thanks for your reply.
>
> The metrics2 configuration is :
> *.sink.ganglia.class=org.apache.hadoop.metrics2.sink.ganglia.GangliaSink31
> *.sink.ganglia.period=10
> *.sink.ganglia.supportsparse=true
> namenode.sink.ganglia.servers=10.232.130.6:8649
> datanodenode.sink.ganglia.servers=10.232.130.6:8649
> journalnode.sink.ganglia.servers=10.232.130.6:8649
>
> There is @Metric MutableCounterLong bytesWritten attribute in
> DataNodeMetrics, which is used to statistics written bytes per second on
> one DataNode, is that right? If the metric is not divided by 10 and is not
> reseted to zero per ten seconds, how does the metric statistics written
> bytes per second?
>
>
> Thanks,
> LiuLei
>
>
>
>
>
>
>
> 2013/11/21 Andrew Wang <an...@cloudera.com>
>
> > Hey LiuLei,
> >
> > Gauges can go up and down, counters only go up. Snapshot doesn't actually
> > reset anything, it's just a way for the metrics system to get an updated
> > value. There aren't any time-based rolling metrics to my knowledge
> besides
> > MutableQuantiles.
> >
> > Best,
> > Andrew
> >
> >
> > On Wed, Nov 20, 2013 at 7:34 PM, lei liu <li...@gmail.com> wrote:
> >
> > > I use cdh-4.3.1 version. I am reading the code about metrics2.
> > >
> > > There are COUNTER and GAUGE metric type in metrics v2. What is the
> > > difference
> > > between the two?
> > >
> > >
> > > There is @Metric MutableCounterLong bytesWritten attribute in
> > > DataNodeMetrics, which is used to statistics written bytes per second
> on
> > > DataNode.So I think the value of MutableCounterLong should be divided
> > > by 10and be reseted to zero per ten seconds in
> > > MutableCounterLong.snapshot
> > > method, is that right? But MutableCounterLong.snapshot method don't do
> > > that. I miss anything please tell me.
> > >
> > > Thanks,
> > >
> > > LiuLei
> > >
> >
>
Re: Metrics2 code
Posted by lei liu <li...@gmail.com>.
Hi Andrew, thanks for your reply.
The metrics2 configuration is :
*.sink.ganglia.class=org.apache.hadoop.metrics2.sink.ganglia.GangliaSink31
*.sink.ganglia.period=10
*.sink.ganglia.supportsparse=true
namenode.sink.ganglia.servers=10.232.130.6:8649
datanodenode.sink.ganglia.servers=10.232.130.6:8649
journalnode.sink.ganglia.servers=10.232.130.6:8649
There is @Metric MutableCounterLong bytesWritten attribute in
DataNodeMetrics, which is used to statistics written bytes per second on
one DataNode, is that right? If the metric is not divided by 10 and is not
reseted to zero per ten seconds, how does the metric statistics written
bytes per second?
Thanks,
LiuLei
2013/11/21 Andrew Wang <an...@cloudera.com>
> Hey LiuLei,
>
> Gauges can go up and down, counters only go up. Snapshot doesn't actually
> reset anything, it's just a way for the metrics system to get an updated
> value. There aren't any time-based rolling metrics to my knowledge besides
> MutableQuantiles.
>
> Best,
> Andrew
>
>
> On Wed, Nov 20, 2013 at 7:34 PM, lei liu <li...@gmail.com> wrote:
>
> > I use cdh-4.3.1 version. I am reading the code about metrics2.
> >
> > There are COUNTER and GAUGE metric type in metrics v2. What is the
> > difference
> > between the two?
> >
> >
> > There is @Metric MutableCounterLong bytesWritten attribute in
> > DataNodeMetrics, which is used to statistics written bytes per second on
> > DataNode.So I think the value of MutableCounterLong should be divided
> > by 10and be reseted to zero per ten seconds in
> > MutableCounterLong.snapshot
> > method, is that right? But MutableCounterLong.snapshot method don't do
> > that. I miss anything please tell me.
> >
> > Thanks,
> >
> > LiuLei
> >
>
Re: Metrics2 code
Posted by Andrew Wang <an...@cloudera.com>.
Hey LiuLei,
Gauges can go up and down, counters only go up. Snapshot doesn't actually
reset anything, it's just a way for the metrics system to get an updated
value. There aren't any time-based rolling metrics to my knowledge besides
MutableQuantiles.
Best,
Andrew
On Wed, Nov 20, 2013 at 7:34 PM, lei liu <li...@gmail.com> wrote:
> I use cdh-4.3.1 version. I am reading the code about metrics2.
>
> There are COUNTER and GAUGE metric type in metrics v2. What is the
> difference
> between the two?
>
>
> There is @Metric MutableCounterLong bytesWritten attribute in
> DataNodeMetrics, which is used to statistics written bytes per second on
> DataNode.So I think the value of MutableCounterLong should be divided
> by 10and be reseted to zero per ten seconds in
> MutableCounterLong.snapshot
> method, is that right? But MutableCounterLong.snapshot method don't do
> that. I miss anything please tell me.
>
> Thanks,
>
> LiuLei
>