You are viewing a plain text version of this content. The canonical link for it is here.

Posted to hdfs-user@hadoop.apache.org by "Marco A. S. Netto" <ma...@gmail.com> on 2014/06/13 00:49:06 UTC

monitor mapper running execution times with hadoop 2.2.0

Hi,

I would like to monitor the average execution time of mappers and reducers
or something better to check the hadoop throughput.

I configured the hadoop metrics2 as follows:
*.sink.ganglia.period=10
*.sink.ganglia.supportsparse=true
*.sink.ganglia.servers=GANGLIA_SERVER_IP:8649

mrappmaster.sink.ganglia.class=org.apache.hadoop.metrics2.sink.ganglia.GangliaSink31
resourcemanager.sink.ganglia.class=org.apache.hadoop.metrics2.sink.ganglia.GangliaSink31
mapred.sink.ganglia.class=org.apache.hadoop.metrics2.sink.ganglia.GangliaSink31
namenode.sink.ganglia.class=org.apache.hadoop.metrics2.sink.ganglia.GangliaSink31
datanode.sink.ganglia.class=org.apache.hadoop.metrics2.sink.ganglia.GangliaSink31
nodemanager.sink.ganglia.class=org.apache.hadoop.metrics2.sink.ganglia.GangliaSink31

These two lines seem to be ignored:
maptask.sink.ganglia.class=org.apache.hadoop.metrics2.sink.ganglia.GangliaSink31
reducetask.sink.ganglia.class=org.apache.hadoop.metrics2.sink.ganglia.GangliaSink31


Is there any way to monitor the progress of a hadoop app? Even without
Ganglia?

Kind regards,
Marco