You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-dev@hadoop.apache.org by "Doug Cutting (JIRA)" <ji...@apache.org> on 2006/09/05 20:29:23 UTC
[jira] Commented: (HADOOP-481) Hadoop mapred metrics should include
per job input/output statistics rather than per-task statistics
[ http://issues.apache.org/jira/browse/HADOOP-481?page=comments#action_12432647 ]
Doug Cutting commented on HADOOP-481:
-------------------------------------
Shouldn't we use our existing metrics API for stuff like this? As with HADOOP-492, it seems like the TaskTracker and JobTracker should implement the MetricsContext API, providing a MetricsRecord factory. These can be used by the MapReduce kernel code for the metrics desired here, and in supplied to user code the uses in HADOOP-492.
We might even need to write a multiplexing MetricsContext, that can send metrics to both the JobTracker and to, e.g., Ganglia. But we should not be adding new metrics APIs when we already have one. If the current metrics API is somehow inappropriate, let's fix that instead of create another.
> Hadoop mapred metrics should include per job input/output statistics rather than per-task statistics
> ----------------------------------------------------------------------------------------------------
>
> Key: HADOOP-481
> URL: http://issues.apache.org/jira/browse/HADOOP-481
> Project: Hadoop
> Issue Type: Improvement
> Components: mapred, metrics
> Affects Versions: 0.6.0
> Reporter: Milind Bhandarkar
> Assigned To: Milind Bhandarkar
> Priority: Minor
> Fix For: 0.6.0
>
> Attachments: reports.patch
>
>
> Currently hadoop reports metrics such as input bytes, input records, etc on per-task basis. Accurate aggregation of these metrics is required at the job-level and reporting should be done on a per-job basis.
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators: http://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira