[jira] Commented: (HADOOP-3708) Provide accounting functionality for Hadoop resource manager

["Matei Zaharia (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/HADOOP-3708?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12640635#action_12640635 ] 

Matei Zaharia commented on HADOOP-3708:
---------------------------------------

A lot of this data is available in the job history logs written by the job tracker. At Facebook, we have a set of scripts that parse these logs and put the data in a MySQL database, where it can be queried. Would you be interested in something like that? Or do you want something more real-time, integrated into the Hadoop web UI?

> Provide accounting functionality for Hadoop resource manager
> ------------------------------------------------------------
>
>                 Key: HADOOP-3708
>                 URL: https://issues.apache.org/jira/browse/HADOOP-3708
>             Project: Hadoop Core
>          Issue Type: New Feature
>          Components: mapred
>            Reporter: Hemanth Yamijala
>            Assignee: Hemanth Yamijala
>
> HADOOP-3421 describes requirements for a new resource manager in Hadoop to schedule Map/Reduce jobs. In production systems, it would be useful to produce accounting information related to the scheduling - such as job start and run times, resources used, etc. This information can be consumed by other systems to build accounting for shared resources. This JIRA is for tracking the requirements, approach and implementation for producing accounting information.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.