[jira] [Commented] (MAPREDUCE-901) Move Framework Counters into a TaskMetric structure

["Luke Lu (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/MAPREDUCE-901?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13083846#comment-13083846 ] 

Luke Lu commented on MAPREDUCE-901:
-----------------------------------

The port, which includes smarter counter limits (exclude framework counters for limits) lgtm. Defer +1 to Jenkins. There are some improvements can be made to improve binary compatibility (for most existing job jars) that we did/does not promise, deferring to a separate JIRA if binary compatibility becomes an issue.

> Move Framework Counters into a TaskMetric structure
> ---------------------------------------------------
>
>                 Key: MAPREDUCE-901
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-901
>             Project: Hadoop Map/Reduce
>          Issue Type: Improvement
>          Components: task
>    Affects Versions: 0.21.0
>            Reporter: Owen O'Malley
>            Assignee: Luke Lu
>             Fix For: 0.23.0
>
>         Attachments: 901_1.patch, 901_1.patch, FrameworkCounterGroup.java, MAPREDUCE-901.patch, MAPREDUCE-901.patch, MAPREDUCE-901.patch, mr-901-trunk-v1.patch
>
>
> I think we should move all of the Counters that the framework updates into a single class called TaskMetrics. TaskMetrics would have specific fields for each of the metrics like input records, input bytes, output records, etc.
> It would both reduce the serialized size of the heartbeats (by shrinking the Counters down to just the user's counters) and decrease the latency for updates to the JobTracker (since Counters are sent at most 1/minute instead of 1/heartbeat).

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira