You are viewing a plain text version of this content. The canonical link for it is here.
Posted to hdfs-user@hadoop.apache.org by lei liu <li...@gmail.com> on 2013/08/07 12:57:16 UTC
MutableCounterLong metrics display in ganglia
I use hadoop-2.0.5 and config hadoop-metrics2.properties file with below
content.
*.sink.ganglia.class=org.
apache.hadoop.metrics2.sink.ganglia.GangliaSink31
*.sink.ganglia.period=10
*.sink.ganglia.supportsparse=true
namenode.sink.ganglia.servers=10.232.98.74:8649
datanode.sink.ganglia.servers=10.232.98.74:8649
I write one programme that call FSDataOutputStream.hsync() method once per
second.
There is "@Metric MutableCounterLong fsyncCount" metrics in
DataNodeMetrics, when FSDataOutputStream.hsync() method is called, the
value of fsyncCount is increased, dataNode send the value of fsyncCount
to ganglia every ten seconds, so I think the value of fsyncCount in
ganglia should be 10, 20 ,30, 40 and so on . but the ganglia display
1,1,1,1,1 ...... , so the value is
the value of fsyncCount is set to zero every ten seconds and
”fsyncCount.value/10“ .
Is the the value of MutableCounterLong class set to zero every ten
seconds and MutableCounterLong .value/10?
Thanks,
LiuLei
Re: MutableCounterLong metrics display in ganglia
Posted by lei liu <li...@gmail.com>.
Thanks Harsh for your reply.
What are difference MutableCounterLong and MutableGaugeLong class ?
I find the MutableCounterLong is used to calculate throughput, the value be
reset per ten seconds, and MutableGaugeLong is up-count and no reset.
I am newer for hadoop-2.0.5, please tell me if there is an error.
Thanks,
LiuLei
2013/8/9 Harsh J <ha...@cloudera.com>
> The counter, being num-ops, should up-count and not reset. Note that
> your test may be at fault though - calling hsync may not always call
> NN#fsync(…) unless you are passing the proper flags to make it always
> do so.
>
> On Wed, Aug 7, 2013 at 4:27 PM, lei liu <li...@gmail.com> wrote:
> > I use hadoop-2.0.5 and config hadoop-metrics2.properties file with below
> > content.
> > *.sink.ganglia.class=org.
> > apache.hadoop.metrics2.sink.ganglia.GangliaSink31
> > *.sink.ganglia.period=10
> > *.sink.ganglia.supportsparse=true
> > namenode.sink.ganglia.servers=10.232.98.74:8649
> > datanode.sink.ganglia.servers=10.232.98.74:8649
> >
> > I write one programme that call FSDataOutputStream.hsync() method once
> per
> > second.
> >
> > There is "@Metric MutableCounterLong fsyncCount" metrics in
> DataNodeMetrics,
> > when FSDataOutputStream.hsync() method is called, the value of
> fsyncCount
> > is increased, dataNode send the value of fsyncCount to ganglia every ten
> > seconds, so I think the value of fsyncCount in ganglia should be 10, 20
> > ,30, 40 and so on . but the ganglia display 1,1,1,1,1 ...... , so the
> value
> > is
> > the value of fsyncCount is set to zero every ten seconds and
> > ”fsyncCount.value/10“ .
> >
> >
> > Is the the value of MutableCounterLong class set to zero every ten
> seconds
> > and MutableCounterLong .value/10?
> >
> > Thanks,
> >
> > LiuLei
> >
> >
>
>
>
> --
> Harsh J
>
Re: MutableCounterLong metrics display in ganglia
Posted by lei liu <li...@gmail.com>.
Thanks Harsh for your reply.
What are difference MutableCounterLong and MutableGaugeLong class ?
I find the MutableCounterLong is used to calculate throughput, the value be
reset per ten seconds, and MutableGaugeLong is up-count and no reset.
I am newer for hadoop-2.0.5, please tell me if there is an error.
Thanks,
LiuLei
2013/8/9 Harsh J <ha...@cloudera.com>
> The counter, being num-ops, should up-count and not reset. Note that
> your test may be at fault though - calling hsync may not always call
> NN#fsync(…) unless you are passing the proper flags to make it always
> do so.
>
> On Wed, Aug 7, 2013 at 4:27 PM, lei liu <li...@gmail.com> wrote:
> > I use hadoop-2.0.5 and config hadoop-metrics2.properties file with below
> > content.
> > *.sink.ganglia.class=org.
> > apache.hadoop.metrics2.sink.ganglia.GangliaSink31
> > *.sink.ganglia.period=10
> > *.sink.ganglia.supportsparse=true
> > namenode.sink.ganglia.servers=10.232.98.74:8649
> > datanode.sink.ganglia.servers=10.232.98.74:8649
> >
> > I write one programme that call FSDataOutputStream.hsync() method once
> per
> > second.
> >
> > There is "@Metric MutableCounterLong fsyncCount" metrics in
> DataNodeMetrics,
> > when FSDataOutputStream.hsync() method is called, the value of
> fsyncCount
> > is increased, dataNode send the value of fsyncCount to ganglia every ten
> > seconds, so I think the value of fsyncCount in ganglia should be 10, 20
> > ,30, 40 and so on . but the ganglia display 1,1,1,1,1 ...... , so the
> value
> > is
> > the value of fsyncCount is set to zero every ten seconds and
> > ”fsyncCount.value/10“ .
> >
> >
> > Is the the value of MutableCounterLong class set to zero every ten
> seconds
> > and MutableCounterLong .value/10?
> >
> > Thanks,
> >
> > LiuLei
> >
> >
>
>
>
> --
> Harsh J
>
Re: MutableCounterLong metrics display in ganglia
Posted by lei liu <li...@gmail.com>.
Thanks Harsh for your reply.
What are difference MutableCounterLong and MutableGaugeLong class ?
I find the MutableCounterLong is used to calculate throughput, the value be
reset per ten seconds, and MutableGaugeLong is up-count and no reset.
I am newer for hadoop-2.0.5, please tell me if there is an error.
Thanks,
LiuLei
2013/8/9 Harsh J <ha...@cloudera.com>
> The counter, being num-ops, should up-count and not reset. Note that
> your test may be at fault though - calling hsync may not always call
> NN#fsync(…) unless you are passing the proper flags to make it always
> do so.
>
> On Wed, Aug 7, 2013 at 4:27 PM, lei liu <li...@gmail.com> wrote:
> > I use hadoop-2.0.5 and config hadoop-metrics2.properties file with below
> > content.
> > *.sink.ganglia.class=org.
> > apache.hadoop.metrics2.sink.ganglia.GangliaSink31
> > *.sink.ganglia.period=10
> > *.sink.ganglia.supportsparse=true
> > namenode.sink.ganglia.servers=10.232.98.74:8649
> > datanode.sink.ganglia.servers=10.232.98.74:8649
> >
> > I write one programme that call FSDataOutputStream.hsync() method once
> per
> > second.
> >
> > There is "@Metric MutableCounterLong fsyncCount" metrics in
> DataNodeMetrics,
> > when FSDataOutputStream.hsync() method is called, the value of
> fsyncCount
> > is increased, dataNode send the value of fsyncCount to ganglia every ten
> > seconds, so I think the value of fsyncCount in ganglia should be 10, 20
> > ,30, 40 and so on . but the ganglia display 1,1,1,1,1 ...... , so the
> value
> > is
> > the value of fsyncCount is set to zero every ten seconds and
> > ”fsyncCount.value/10“ .
> >
> >
> > Is the the value of MutableCounterLong class set to zero every ten
> seconds
> > and MutableCounterLong .value/10?
> >
> > Thanks,
> >
> > LiuLei
> >
> >
>
>
>
> --
> Harsh J
>
Re: MutableCounterLong metrics display in ganglia
Posted by lei liu <li...@gmail.com>.
Thanks Harsh for your reply.
What are difference MutableCounterLong and MutableGaugeLong class ?
I find the MutableCounterLong is used to calculate throughput, the value be
reset per ten seconds, and MutableGaugeLong is up-count and no reset.
I am newer for hadoop-2.0.5, please tell me if there is an error.
Thanks,
LiuLei
2013/8/9 Harsh J <ha...@cloudera.com>
> The counter, being num-ops, should up-count and not reset. Note that
> your test may be at fault though - calling hsync may not always call
> NN#fsync(…) unless you are passing the proper flags to make it always
> do so.
>
> On Wed, Aug 7, 2013 at 4:27 PM, lei liu <li...@gmail.com> wrote:
> > I use hadoop-2.0.5 and config hadoop-metrics2.properties file with below
> > content.
> > *.sink.ganglia.class=org.
> > apache.hadoop.metrics2.sink.ganglia.GangliaSink31
> > *.sink.ganglia.period=10
> > *.sink.ganglia.supportsparse=true
> > namenode.sink.ganglia.servers=10.232.98.74:8649
> > datanode.sink.ganglia.servers=10.232.98.74:8649
> >
> > I write one programme that call FSDataOutputStream.hsync() method once
> per
> > second.
> >
> > There is "@Metric MutableCounterLong fsyncCount" metrics in
> DataNodeMetrics,
> > when FSDataOutputStream.hsync() method is called, the value of
> fsyncCount
> > is increased, dataNode send the value of fsyncCount to ganglia every ten
> > seconds, so I think the value of fsyncCount in ganglia should be 10, 20
> > ,30, 40 and so on . but the ganglia display 1,1,1,1,1 ...... , so the
> value
> > is
> > the value of fsyncCount is set to zero every ten seconds and
> > ”fsyncCount.value/10“ .
> >
> >
> > Is the the value of MutableCounterLong class set to zero every ten
> seconds
> > and MutableCounterLong .value/10?
> >
> > Thanks,
> >
> > LiuLei
> >
> >
>
>
>
> --
> Harsh J
>
Re: MutableCounterLong metrics display in ganglia
Posted by Harsh J <ha...@cloudera.com>.
The counter, being num-ops, should up-count and not reset. Note that
your test may be at fault though - calling hsync may not always call
NN#fsync(…) unless you are passing the proper flags to make it always
do so.
On Wed, Aug 7, 2013 at 4:27 PM, lei liu <li...@gmail.com> wrote:
> I use hadoop-2.0.5 and config hadoop-metrics2.properties file with below
> content.
> *.sink.ganglia.class=org.
> apache.hadoop.metrics2.sink.ganglia.GangliaSink31
> *.sink.ganglia.period=10
> *.sink.ganglia.supportsparse=true
> namenode.sink.ganglia.servers=10.232.98.74:8649
> datanode.sink.ganglia.servers=10.232.98.74:8649
>
> I write one programme that call FSDataOutputStream.hsync() method once per
> second.
>
> There is "@Metric MutableCounterLong fsyncCount" metrics in DataNodeMetrics,
> when FSDataOutputStream.hsync() method is called, the value of fsyncCount
> is increased, dataNode send the value of fsyncCount to ganglia every ten
> seconds, so I think the value of fsyncCount in ganglia should be 10, 20
> ,30, 40 and so on . but the ganglia display 1,1,1,1,1 ...... , so the value
> is
> the value of fsyncCount is set to zero every ten seconds and
> ”fsyncCount.value/10“ .
>
>
> Is the the value of MutableCounterLong class set to zero every ten seconds
> and MutableCounterLong .value/10?
>
> Thanks,
>
> LiuLei
>
>
--
Harsh J
Re: MutableCounterLong metrics display in ganglia
Posted by Harsh J <ha...@cloudera.com>.
The counter, being num-ops, should up-count and not reset. Note that
your test may be at fault though - calling hsync may not always call
NN#fsync(…) unless you are passing the proper flags to make it always
do so.
On Wed, Aug 7, 2013 at 4:27 PM, lei liu <li...@gmail.com> wrote:
> I use hadoop-2.0.5 and config hadoop-metrics2.properties file with below
> content.
> *.sink.ganglia.class=org.
> apache.hadoop.metrics2.sink.ganglia.GangliaSink31
> *.sink.ganglia.period=10
> *.sink.ganglia.supportsparse=true
> namenode.sink.ganglia.servers=10.232.98.74:8649
> datanode.sink.ganglia.servers=10.232.98.74:8649
>
> I write one programme that call FSDataOutputStream.hsync() method once per
> second.
>
> There is "@Metric MutableCounterLong fsyncCount" metrics in DataNodeMetrics,
> when FSDataOutputStream.hsync() method is called, the value of fsyncCount
> is increased, dataNode send the value of fsyncCount to ganglia every ten
> seconds, so I think the value of fsyncCount in ganglia should be 10, 20
> ,30, 40 and so on . but the ganglia display 1,1,1,1,1 ...... , so the value
> is
> the value of fsyncCount is set to zero every ten seconds and
> ”fsyncCount.value/10“ .
>
>
> Is the the value of MutableCounterLong class set to zero every ten seconds
> and MutableCounterLong .value/10?
>
> Thanks,
>
> LiuLei
>
>
--
Harsh J
Re: MutableCounterLong metrics display in ganglia
Posted by Harsh J <ha...@cloudera.com>.
The counter, being num-ops, should up-count and not reset. Note that
your test may be at fault though - calling hsync may not always call
NN#fsync(…) unless you are passing the proper flags to make it always
do so.
On Wed, Aug 7, 2013 at 4:27 PM, lei liu <li...@gmail.com> wrote:
> I use hadoop-2.0.5 and config hadoop-metrics2.properties file with below
> content.
> *.sink.ganglia.class=org.
> apache.hadoop.metrics2.sink.ganglia.GangliaSink31
> *.sink.ganglia.period=10
> *.sink.ganglia.supportsparse=true
> namenode.sink.ganglia.servers=10.232.98.74:8649
> datanode.sink.ganglia.servers=10.232.98.74:8649
>
> I write one programme that call FSDataOutputStream.hsync() method once per
> second.
>
> There is "@Metric MutableCounterLong fsyncCount" metrics in DataNodeMetrics,
> when FSDataOutputStream.hsync() method is called, the value of fsyncCount
> is increased, dataNode send the value of fsyncCount to ganglia every ten
> seconds, so I think the value of fsyncCount in ganglia should be 10, 20
> ,30, 40 and so on . but the ganglia display 1,1,1,1,1 ...... , so the value
> is
> the value of fsyncCount is set to zero every ten seconds and
> ”fsyncCount.value/10“ .
>
>
> Is the the value of MutableCounterLong class set to zero every ten seconds
> and MutableCounterLong .value/10?
>
> Thanks,
>
> LiuLei
>
>
--
Harsh J
Re: MutableCounterLong metrics display in ganglia
Posted by Harsh J <ha...@cloudera.com>.
The counter, being num-ops, should up-count and not reset. Note that
your test may be at fault though - calling hsync may not always call
NN#fsync(…) unless you are passing the proper flags to make it always
do so.
On Wed, Aug 7, 2013 at 4:27 PM, lei liu <li...@gmail.com> wrote:
> I use hadoop-2.0.5 and config hadoop-metrics2.properties file with below
> content.
> *.sink.ganglia.class=org.
> apache.hadoop.metrics2.sink.ganglia.GangliaSink31
> *.sink.ganglia.period=10
> *.sink.ganglia.supportsparse=true
> namenode.sink.ganglia.servers=10.232.98.74:8649
> datanode.sink.ganglia.servers=10.232.98.74:8649
>
> I write one programme that call FSDataOutputStream.hsync() method once per
> second.
>
> There is "@Metric MutableCounterLong fsyncCount" metrics in DataNodeMetrics,
> when FSDataOutputStream.hsync() method is called, the value of fsyncCount
> is increased, dataNode send the value of fsyncCount to ganglia every ten
> seconds, so I think the value of fsyncCount in ganglia should be 10, 20
> ,30, 40 and so on . but the ganglia display 1,1,1,1,1 ...... , so the value
> is
> the value of fsyncCount is set to zero every ten seconds and
> ”fsyncCount.value/10“ .
>
>
> Is the the value of MutableCounterLong class set to zero every ten seconds
> and MutableCounterLong .value/10?
>
> Thanks,
>
> LiuLei
>
>
--
Harsh J