Re: questions about hadoop map reduce and compute intensive related applications

[gm...@cs.ucf.edu]

 
> Integrating MPI with map-reduce is currently difficult and/or very ugly, 
> however. Not impossible and there are hackish ways to do the job, but 
> they are hacks.

There is an project out of Sandia National Lab that puts MR and MPI 
together in a library if you're interested --> 
http://www.sandia.gov/~sjplimp/mapreduce.html

The project isn't mature yet, and I haven't actually used it myself.

-- 
--
Grant Mackey
PhD student Computer Engineering
University of Central Florida

Re: questions about hadoop map reduce and compute intensive related applications

[elton sky <el...@gmail.com>]

thanks gmackey,

There is an project out of Sandia National Lab that puts MR and MPI together
> in a library if you're interested -->
> http://www.sandia.gov/~sjplimp/mapreduce.html


That is a implementation of MR using MPI. I saw that as well but haven't
tried it out.

I am actually looking at programming model level integration. You know, for
some applications, you can take advantage of high through put from MR and
message passing from MPI.

-Elton


On Tue, May 3, 2011 at 12:55 AM, <gm...@cs.ucf.edu> wrote:

>
>  Integrating MPI with map-reduce is currently difficult and/or very ugly,
>> however. Not impossible and there are hackish ways to do the job, but they
>> are hacks.
>>
>
> There is an project out of Sandia National Lab that puts MR and MPI
> together in a library if you're interested -->
> http://www.sandia.gov/~sjplimp/mapreduce.html
>
> The project isn't mature yet, and I haven't actually used it myself.
>
> --
> --
> Grant Mackey
> PhD student Computer Engineering
> University of Central Florida
>