FlumeJava

[Jake Mannix <ja...@gmail.com>]

Google paper <http://portal.acm.org/citation.cfm?id=1806596.1806638> (requires
ACM membership) on their internal, higher-level java map/reduce api.

Also of note, Jeff Hammerbacher seems to agree with me that this would be
*awesome* for Hadoop-land:

https://issues.apache.org/jira/browse/MAPREDUCE-1849

Thoughts?

I may just try to dredge up my initial pass at this kind of thing
(essentially a generified version of DistributedRowMatrix - DataSet<K,V>,
which then accepts Map<K,V,KOUT,VOUT> mappers and various reducers etc...),
and see if I can help with this.

  -jake

Re: FlumeJava

[Ted Dunning <te...@gmail.com>]

This would be great, especially if there are in-memory and delicate
map-reduce modes (see twister's task model
http://www.iterativemapreduce.org/) to allow performance gains for small
problems.

On Wed, Jun 9, 2010 at 5:03 PM, Jake Mannix <ja...@gmail.com> wrote:

> Google paper <http://portal.acm.org/citation.cfm?id=1806596.1806638>
> (requires
> ACM membership) on their internal, higher-level java map/reduce api.
>
> Also of note, Jeff Hammerbacher seems to agree with me that this would be
> *awesome* for Hadoop-land:
>
> https://issues.apache.org/jira/browse/MAPREDUCE-1849
>
> Thoughts?
>
> I may just try to dredge up my initial pass at this kind of thing
> (essentially a generified version of DistributedRowMatrix - DataSet<K,V>,
> which then accepts Map<K,V,KOUT,VOUT> mappers and various reducers etc...),
> and see if I can help with this.
>
>  -jake
>