You are viewing a plain text version of this content. The canonical link for it is here.

Posted to common-user@hadoop.apache.org by Kris Jirapinyo <kj...@biz360.com> on 2009/01/30 03:21:30 UTC

Counters in Hadoop

Hi all,
    I am using counters in Hadoop via the reporter.  I can see this custom
counter fine after I run my job.  However, if somehow I restart the cluster,
then when I look into the Hadoop Job History, I can't seem to find the
information of my previous counter values anywhere.  Where is it stored (or
is it not)?  Also, I need to be able to write this counter value to either a
local file or even to a file in HDFS...is there a good way to do that?

Re: Counters in Hadoop

Posted by Amareshwari Sriramadasu <am...@yahoo-inc.com>.

Kris Jirapinyo wrote:
> Hi all,
>     I am using counters in Hadoop via the reporter.  I can see this custom
> counter fine after I run my job.  However, if somehow I restart the cluster,
> then when I look into the Hadoop Job History, I can't seem to find the
> information of my previous counter values anywhere.  Where is it stored (or
> is it not)?  Also, I need to be able to write this counter value to either a
> local file or even to a file in HDFS...is there a good way to do that?
>
>   
Counters are available in JobHistory, but yes they are not shown on the 
web ui.
There is an open jira for the same: 
https://issues.apache.org/jira/browse/HADOOP-3200.
For now, you can open the history log file and see the counters.


Thanks
Amareshwari