[jira] Created: (MAPREDUCE-751) Rumen: a tool to extract job characterization data from job tracker logs

["Dick King (JIRA)" <ji...@apache.org>]

Rumen: a tool to extract job characterization data from job tracker logs
------------------------------------------------------------------------

                 Key: MAPREDUCE-751
                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-751
             Project: Hadoop Map/Reduce
          Issue Type: Bug
            Reporter: Dick King


 We propose a new map/reduce component, rumen, which can be used to process job history logs to produce any or all of the following:

      * Retrospective info describing the statistical behavior of the
amount of time it would have taken to launch a job into a certain
percentage of the number of mapper slots in the log's cluster, given the
load over the period covered by the log

      * Statistical info as to the runtimes and shuffle times, etc. of
the tasks and jobs covered by the log

      * files describing detailed job trace information, and the
network topology as inferred from the host locations and rack IDs that
arise in the job tracker log.  In addition to this facility, rumen
includes readers for this information to return job and detailed task
information to other tools.

        These other tools include a more advanced version of gridmix, and also includes mumak: see blocked issues.




-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.