You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@tuscany.apache.org by Chris Trezzo <ct...@gmail.com> on 2008/08/08 11:08:17 UTC

Map-Reduce Project Update

Hi All,

A brief update:

I have supplied a patch[1] that provides the initial code in my effort  
to represent the Map-Reduce programing model in SCA using java  
components.

Tomorrow I hope to get this first step fully working.

The goal is to have the java components correctly count the words in  
an input file and write the output to a file, where both of the files  
are specified as SCA properties.

In the Java composite, there are several components. They are as  
follows:

JobConf - This contains the run() method that executes the job. It is  
in charge of reading data from the input file, as well as writing data  
to the output file.
Mapper - This contains the map() method.
IntermediateCollector - This component collects the intermediate <Key,  
Value> pairs generated by the map function. It also sorts these pairs  
by the key, and submits each key with all of its values to the reduce  
function.
Reducer - This contains the reduce() method.
OutputCollector - This component collects the final <Key, Value> pairs  
generated by the reduce function.

Thanks,
Chris Trezzo

[1] https://issues.apache.org/jira/browse/TUSCANY-2540