MapReduce using the CLR (.NET/Mono)

[Fredrik Hedberg <fr...@gmail.com>]

Hi,

I've started a very basic effort to facilitate MapReduce development
using the Common Language Runtime (CLR/.NET/Mono). This would allow
writing MapReduce applications using any CLR supported language (C#,
VB.NET, F#, IronPython, IronRuby etc). Using Mono [1], this would be
supported on most UNIX platforms that Hadoop runs on.

The C# code uses the Pipes interface and is currently modeled after
the Pipes C++ API, passing around byte arrays. The idea is to evolve
the API to use all the fancy features of the CLR such as generics,
lambda functions etc. and eventually LINQ.

I've uploaded the initial code to Google Code [1], in order to
facilitate rapid collaborative development before submitting the code
for consideration to the main Hadoop repository.

The current version is NOT ready for end-users, and thus; this is a
call to arms for developers that would like to use MapReduce and
Hadoop via CLR applications and are interested in contributing to the
effort.


Fredrik

[1] http://code.google.com/p/hadoop-sharp/