[jira] [Updated] (GIRAPH-13) Port Giraph to YARN

["Eli Reisman (JIRA)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/GIRAPH-13?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Eli Reisman updated GIRAPH-13:
------------------------------

    Attachment: GIRAPH-13-1.patch

Just a placeholder and request for criticism as I launch into this endeavor, and to stoke the fire so folks will start opining on the approach where it affects (or offends) them. Thanks!

This patch will (hopefully) include the following as time goes on:

1. POM adjustments to add a munge flag and to exclude org.apache.giraph.yarn package from the build when Hadoop MRv1 profiles are compiled.

2. Minimal munge surgery to inject my job running code into GiraphRunner. Alternately, I will attempt a GiraphJob conf setting and factory of some sort to handle this w/o munging, if I can.

3. GiraphJob for YARN (prototype included here)

4. Application Master for Giraph (prototype mostly finished and included here)

5. Unit tests for the YARN job launching classes (for everything that happens before the BSP work starts, we have tests for that)

                
> Port Giraph to YARN
> -------------------
>
>                 Key: GIRAPH-13
>                 URL: https://issues.apache.org/jira/browse/GIRAPH-13
>             Project: Giraph
>          Issue Type: New Feature
>            Reporter: Jakob Homan
>            Assignee: Eli Reisman
>         Attachments: GIRAPH-13-1.patch
>
>
> Now that YARN (aka MR2 aka MAPREDUCE-279) has been merged into the Hadoop trunk, we should think about what it would take to separate out the graph processing bits of Giraph from the MR1-specific code so as to take advantage of the less-MR centric aspects of YARN, while still supporting both over the medium term.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira