[jira] [Updated] (MAPREDUCE-279) Map-Reduce 2.0

["Greg Roelofs (JIRA)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/MAPREDUCE-279?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Greg Roelofs updated MAPREDUCE-279:
-----------------------------------

    Attachment: yarn-state-machine.task.png
                yarn-state-machine.task-attempt.png
                yarn-state-machine.job.png
                yarn-state-machine.task.dot
                yarn-state-machine.task-attempt.dot
                yarn-state-machine.job.dot

dot(1) files for the Job, Task, and TaskAttempt state machines in MRv2, at least as of late March.  I found the graphs very useful while learning and modifying the MRv2 code for MAPREDUCE-2405.

These can be converted to PostScript or PNG or whatnot with dot, which is part of the Graphviz distribution (graphviz.org, I think).  Here's a sample command for PostScript:

{{dot -Tps yarn-state-machine.task-attempt.dot > yarn-state-machine.task-attempt.ps}}

Ultimately a version of these should be produced natively in some StateMachine method ({{toDot()}}?), and I think Chris Douglas may take that up eventually.  However, some of the desirable info (e.g., which states send events to or receive them from other state machines) can't really be discovered automatically, so there will continue to be a place for hand-rolled graphs.

> Map-Reduce 2.0
> --------------
>
>                 Key: MAPREDUCE-279
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-279
>             Project: Hadoop Map/Reduce
>          Issue Type: Improvement
>          Components: jobtracker, tasktracker
>            Reporter: Arun C Murthy
>            Assignee: Arun C Murthy
>             Fix For: 0.23.0
>
>         Attachments: MR-279.patch, MR-279.patch, MR-279.sh, MR-279_MR_files_to_move.txt, capacity-scheduler-dark-theme.png, multi-column-stable-sort-default-theme.png, yarn-state-machine.job.dot, yarn-state-machine.job.png, yarn-state-machine.task-attempt.dot, yarn-state-machine.task-attempt.png, yarn-state-machine.task.dot, yarn-state-machine.task.png
>
>
> Re-factor MapReduce into a generic resource scheduler and a per-job, user-defined component that manages the application execution. 

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira