You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-user@hadoop.apache.org by Usman Waheed <us...@opera.com> on 2009/09/17 13:30:15 UTC
Specify output file names when using hadoop streaming
Hi,
I am a newbie to hadoop. I have a simple MAP only job but would like to
output the results to different file names instead of the files
part-0000x(s).
I would like to name these output files based on the log file that were
processed.
As an example if I processed log1 and log2 i would like the map keys
output from log1 to go to part-log1 and map keys output from log2 to
part-log2 and so forth.
My mapper is in Perl and am using hadoop streaming.
From the documentation I have read so far looks like i can do this in
Java but was wondering if this is doable via hadoop streaming also.
Any clues would help.
Thanks,
Usman