[jira] Updated: (MAPREDUCE-1304) Add counters for task time spent in GC

["Chris Douglas (JIRA)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/MAPREDUCE-1304?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Chris Douglas updated MAPREDUCE-1304:
-------------------------------------

    Status: Open  (was: Patch Available)

Great idea. A few comments on the impl:
* This has no effect:
{noformat}
+    job.getConfiguration().set("io.sort.record.pct", "0.50");
{noformat}
* Moving the update of the GC counter to {{Task::updateCounters}} makes sense. To capture any {{FileSystem}} activity in the committer, the call to {{updateCounters}} in {{Task::done}} should be after the commit, anyway. This is a bug in the current code.
* Adding {{java.lang.management}} components shouldn't be widening the {{TaskReporter}} API and adding fields. There should be an {{\*Updater}} object tracking the particular statistic being tracked (as in the {{FileSystem}} counters) rather than letting the {{TaskReporter}} keep this state.
* Updating the GC counters must be synchronized or it risks reporting incorrect results (moving the update to {{Task::updateCounters}} would be sufficient).

> Add counters for task time spent in GC
> --------------------------------------
>
>                 Key: MAPREDUCE-1304
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-1304
>             Project: Hadoop Map/Reduce
>          Issue Type: New Feature
>          Components: task
>            Reporter: Todd Lipcon
>            Assignee: Aaron Kimball
>         Attachments: MAPREDUCE-1304.2.patch, MAPREDUCE-1304.patch
>
>
> It's easy to grab the number of millis spent in GC (see JvmMetrics for example). Exposing these as task counters would be handy - occasionally I've seen user jobs where long GC pauses cause big "unexplainable" performance problems, and a large counter would make it obvious to the user what's going on.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.