You are viewing a plain text version of this content. The canonical link for it is here.
Posted to mapreduce-user@hadoop.apache.org by ranjith k <ra...@gmail.com> on 2011/04/01 17:05:07 UTC
i can't get the file name in map program
hello.
I am new to hadoop map reduce programming. I need to write a map reduce
program. I have a input folder, it contain a 10 number of documents in text
format. My aim is to write a map reduce program that read each text file and
create the word count of each text file separately. My input split is each
line. The map function is called for each line of text. But i need my file
name in map function. How can i get the file name to my map function.
Similarly i need to write the output of each file separately, is it
possible?
My hadoop version is Hadoop 0.20.2.
please help me .
Advanced thanks.
Re: i can't get the file name in map program
Posted by ranjith k <ra...@gmail.com>.
Thank you..
On Sat, Apr 2, 2011 at 1:22 AM, Harsh J <qw...@gmail.com> wrote:
> Hello,
>
> (Inline reply.)
>
> On Fri, Apr 1, 2011 at 8:35 PM, ranjith k <ra...@gmail.com> wrote:
> > hello.
> > I am new to hadoop map reduce programming. I need to write a map reduce
> > program. I have a input folder, it contain a 10 number of documents in
> text
> > format. My aim is to write a map reduce program that read each text file
> and
> > create the word count of each text file separately. My input split is
> each
> > line. The map function is called for each line of text. But i need my
> file
> > name in map function. How can i get the file name to my map function.
>
> This is covered in the docs as part of the Map/Reduce Tutorial itself.
> Have a look at the table right below this para-link:
>
> http://hadoop.apache.org/common/docs/r0.20.0/mapred_tutorial.html#Task+JVM+Reuse
>
> > Similarly i need to write the output of each file separately, is it
> > possible?
>
> You can achieve some levels of output file-naming using the
> MultipleOutputs class.
>
> > My hadoop version is Hadoop 0.20.2.
>
> --
> Harsh J
> http://harshj.com
>
--
Ranjith k
+918129419842
Re: i can't get the file name in map program
Posted by Harsh J <qw...@gmail.com>.
Hello,
(Inline reply.)
On Fri, Apr 1, 2011 at 8:35 PM, ranjith k <ra...@gmail.com> wrote:
> hello.
> I am new to hadoop map reduce programming. I need to write a map reduce
> program. I have a input folder, it contain a 10 number of documents in text
> format. My aim is to write a map reduce program that read each text file and
> create the word count of each text file separately. My input split is each
> line. The map function is called for each line of text. But i need my file
> name in map function. How can i get the file name to my map function.
This is covered in the docs as part of the Map/Reduce Tutorial itself.
Have a look at the table right below this para-link:
http://hadoop.apache.org/common/docs/r0.20.0/mapred_tutorial.html#Task+JVM+Reuse
> Similarly i need to write the output of each file separately, is it
> possible?
You can achieve some levels of output file-naming using the
MultipleOutputs class.
> My hadoop version is Hadoop 0.20.2.
--
Harsh J
http://harshj.com