Representing hadoop metrics on ganglia web interface

[Nishant Verma <ni...@gmail.com>]

Hello

We are supposed to collect hadoop metrics and see the cluster health and
performance.  I was going through below link from apache which lists
different metrics exposed by hadoop.

https://hadoop.apache.org/docs/r2.7.3/hadoop-project-dist/hadoop-common/Metrics.html

I also read that we can collect and display these metrics on ganglia graphs.

So, I did ganglia setup which included gmetad service, gmond services,etc.

I am able to start the web front end of ganglia. I then opened
hadoop-metrics.properties and hadoop-metrics2.properties and literally
uncommented every line like namenode.sink.ganglia.servers=localhost:8649,
etc. And left nodemanager.sink.file_jvm.class lines commented because they
pointed to file sink which I guess is not needed here.

I started my ganglia related daemons and opened the front end.

But I can only see node or server level parameters graphically present
like:-
load last hour
memory last hour
CPU last hour
network last hour
CPU idle
Disk Space Available

Why did the metrics from that list not come? How to bring them on ganglia
graphs? Can they be brought at all like this? If yes, how? If no, then how
to monitor and leverage those metrics?

Hadoop Version - 2.7.3 Open Source cluster setup on AWS EC2 instances.

Thanks

Re: Representing hadoop metrics on ganglia web interface

[Akira Ajisaka <aa...@apache.org>]

Hi Nishant,

Multicast is used to communicate between Ganglia daemons by default and it is banned in AWS EC2.
Would you try unicast setting?

Regards,
Akira

On 2017/08/04 12:37, Nishant Verma wrote:
> Hello
>
> We are supposed to collect hadoop metrics and see the cluster health and performance.  I was going through below link from apache which lists different metrics exposed by hadoop.
>
> https://hadoop.apache.org/docs/r2.7.3/hadoop-project-dist/hadoop-common/Metrics.html
>
> I also read that we can collect and display these metrics on ganglia graphs.
>
> So, I did ganglia setup which included gmetad service, gmond services,etc.
>
> I am able to start the web front end of ganglia. I then opened hadoop-metrics.properties and hadoop-metrics2.properties and literally uncommented every line like namenode.sink.ganglia.servers=localhost:8649, etc. And left nodemanager.sink.file_jvm.class lines commented because they pointed to file sink which I guess is not needed here.
>
> I started my ganglia related daemons and opened the front end.
>
> But I can only see node or server level parameters graphically present like:-
> load last hour
> memory last hour
> CPU last hour
> network last hour
> CPU idle
> Disk Space Available
>
> Why did the metrics from that list not come? How to bring them on ganglia graphs? Can they be brought at all like this? If yes, how? If no, then how to monitor and leverage those metrics?
>
> Hadoop Version - 2.7.3 Open Source cluster setup on AWS EC2 instances.
>
> Thanks
>

---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscribe@hadoop.apache.org
For additional commands, e-mail: user-help@hadoop.apache.org