[jira] Updated: (HADOOP-2367) Get representative hprof information from tasks

["Owen O'Malley (JIRA)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/HADOOP-2367?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Owen O'Malley updated HADOOP-2367:
----------------------------------

    Attachment: profile.patch

This patch adds a config variable to enable profiling of the tasks. It also adds a range variable that controls which maps and reduces are profiled, which defaults to 0-2. The job client pulls down the profiles to the launching machine and they are available on the web/ui.

I also had to fix the web/ui to not put tasklogs in a table, because the browser silently truncates the table. I also fixed the task log servlet to quote html characters when being sent to the browser.

There is one problem with the current patch in that sometimes the first task is reported to the job client before the jvm is done writing the information, so the first debug log on the client side is only partial. If the user goes to the web/ui, the entire file is available.

> Get representative hprof information from tasks
> -----------------------------------------------
>
>                 Key: HADOOP-2367
>                 URL: https://issues.apache.org/jira/browse/HADOOP-2367
>             Project: Hadoop
>          Issue Type: New Feature
>          Components: mapred
>            Reporter: Owen O'Malley
>            Assignee: Owen O'Malley
>             Fix For: 0.16.0
>
>         Attachments: profile.patch
>
>
> It would be great to get a representative (2 or 3) sample of builtin java profiler for a sample of maps and reduces. I'd store the information in the userlog directory and make it available in the local file system of the submitting application.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.