[jira] [Commented] (MAPREDUCE-2911) Hamster: Hadoop And Mpi on the same cluSTER

["Hamza Zafar (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/MAPREDUCE-2911?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14519945#comment-14519945 ] 

Hamza Zafar commented on MAPREDUCE-2911:
----------------------------------------

Hi Guys,
We have written a Hadoop YARN runtime for MPJ Express [1]. MPJ Express is an open source java based MPI primarily maintained at NUST HPC lab [2]. The code can be downloaded from the MPJ Express website [3].  Currently the MPI processes use the MPJ Express's Ethernet based device driver.

Thanks
--Hamza

[1] http://mpjexpress.org/
[2] http://hpc.seecs.nust.edu.pk/
[3] http://mpjexpress.org/news_5.html

> Hamster: Hadoop And Mpi on the same cluSTER
> -------------------------------------------
>
>                 Key: MAPREDUCE-2911
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-2911
>             Project: Hadoop Map/Reduce
>          Issue Type: New Feature
>          Components: mrv2
>    Affects Versions: 0.23.0
>         Environment: All Unix-Environments
>            Reporter: Milind Bhandarkar
>   Original Estimate: 336h
>  Remaining Estimate: 336h
>
> MPI is commonly used for many machine-learning applications. OpenMPI (http://www.open-mpi.org/) is a popular BSD-licensed version of MPI. In the past, running MPI application on a Hadoop cluster was achieved using Hadoop Streaming (http://videolectures.net/nipsworkshops2010_ye_gbd/), but it was kludgy. After the resource-manager separation from JobTracker in Hadoop, we have all the tools needed to make MPI a first-class citizen on a Hadoop cluster. I am currently working on the patch to make MPI an application-master. Initial version of this patch will be available soon (hopefully before September 10.) This jira will track the development of Hamster: The application master for MPI.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)