Output of MAP Class only

["Rajen Bhatt (RBEI/EST1)" <Ra...@in.bosch.com>]

Dear Experts:
We are scaling down an image to few partitions and processing each partition in the MAP class. We want to save the output of MAP class only to demonstrate and present how this output sub images are looking like. Can somebody give us a hint on how to only save the output of MAP class (before we send them to the REDUCER class) ? We want to demonstrate to the students the process of MAP REDUCE flow on simple image processing operation. We are doing the attached images operation using MAP REDUCE. We are sending the original image and getting as an output the edge detected image. First one is the input Image, we are splitting it, performing the edge detection on each split, and then merging in the reducer.




Thanks and Regards,

~~
Dr. Rajen Bhatt
(Corporate Research @ Robert Bosch, India)
Off: +91-80-4191-2025
Mob: +91-9901241005

AW: Output of MAP Class only

[Christoph Schmitz <Ch...@1und1.de>]

Hi Rajen,

you can write stuff to the task attempt directory and it will be included in the output of your MapReduce job.

You can get the directory from the Mapper context:

FileOutputFormat.getWorkOutputPath(context)

In that path, you can just open new files via the FileSystem methods.

Hope that helps,
Christoph

-----Ursprüngliche Nachricht-----
Von: Rajen Bhatt (RBEI/EST1) [mailto:Rajen.Bhatt@in.bosch.com] 
Gesendet: Freitag, 30. September 2011 12:47
An: mapreduce-user@hadoop.apache.org
Betreff: Output of MAP Class only

Dear Experts:
We are scaling down an image to few partitions and processing each partition in the MAP class. We want to save the output of MAP class only to demonstrate and present how this output sub images are looking like. Can somebody give us a hint on how to only save the output of MAP class (before we send them to the REDUCER class) ? We want to demonstrate to the students the process of MAP REDUCE flow on simple image processing operation. We are doing the attached images operation using MAP REDUCE. We are sending the original image and getting as an output the edge detected image. First one is the input Image, we are splitting it, performing the edge detection on each split, and then merging in the reducer. 
 
 
 
 
Thanks and Regards,
 
~~
Dr. Rajen Bhatt 
(Corporate Research @ Robert Bosch, India)
Off: +91-80-4191-2025
Mob: +91-9901241005
 
 
 
 

Re: Output of MAP Class only

[Kamesh <ka...@imaginea.com>]

On Friday 30 September 2011 04:17 PM, Rajen Bhatt (RBEI/EST1) wrote:
> Dear Experts:
> We are scaling down an image to few partitions and processing each 
> partition in the MAP class. We want to save the output of MAP class 
> only to demonstrate and present how this output sub images are looking 
> like. Can somebody give us a hint on how to only save the output of 
> MAP class (before we send them to the REDUCER class) ? We want to 
> demonstrate to the students the process of MAP REDUCE flow on simple 
> image processing operation. We are doing the attached images operation 
> using MAP REDUCE. We are sending the original image and getting as an 
> output the edge detected image. First one is the input Image, we are 
> splitting it, performing the edge detection on each split, and then 
> merging in the reducer.
> Thanks and Regards,
> ~~
> *Dr. Rajen Bhatt *
> *(*/*Corporate Research*/**/*@ Robert Bosch, India*/*)*
> Off: +91-80-4191-2025
> Mob: +91-9901241005

I have the following approach

At the reducer side, first write the map output to a file and then 
perform the reduce logic. After completion of the reduce logic again 
write the output to another file. Something like below

reduce(key,value, context) {
  mo.write(key,value,namedoutputfile1)
  reduce Logic
  mo.write(key,value after reduce logic, namedoutputfile2)
}

mo : an instance of Multiple Outputs

So here, namedoutputfile1 has mapper output and nameoutputfile2 has 
reducer output.

PS: The above approach works only when there is only one mapper from 
which reducer gets its input.


-- 
/Thanks&Regards,/
/Bh.V.S.Kamesh/