Hardoop Environment Setup

[radhika sridhar <ra...@gmail.com>]

Hi All,

I am a Graduate student. I am working on Hadoop for a college project and
have a few question on the hardoop set up.

> I am running hadoop on windows OS with cygin installed.
> In eclipse when i open the org.apache.hadoop.example.WordCount example
> file, i see a void main written for this class.
> so i am trying to run this program from the standalone mode, by passing
> the command line arguments.
>
> 1) Will i be able to run the program this way, because i am running this
> trough the windows system. If not
> how can i do the local set up so that i can make the changes to the file
> and run them on my system to test it?
>
> 2) if i make the changes to the file ( just have added few
> System.Out.println statements) and did a jar file from the examples
> package
> and tried to run it from cygwin, again it failed telling its not able to
> fine the main class. I haven't done any other changes to this apart from
> the print statements
>
> 3) In the example program, during the configuation state, we set the input
> path for the program. Will we be able to set two or more different paths
> this way?
> That is, Suppose i have two different files to be read, say one file is
> already in memory, will i be able to set the configurations such that the
> input for the Map-Reduce is
> to read one file from the disk and read the other from the memory in the
> same Map-Reduce iteration.

Please advise as to how i can proceed from here.
-- 
Thanks,
Radhika Sridhar

Re: Hardoop Environment Setup

[Sandy <sn...@gmail.com>]

On Mon, Sep 22, 2008 at 7:40 PM, radhika sridhar
<ra...@gmail.com>wrote:

> Hi All,
>
> I am a Graduate student. I am working on Hadoop for a college project and
> have a few question on the hardoop set up.
>
> > I am running hadoop on windows OS with cygin installed.
> > In eclipse when i open the org.apache.hadoop.example.WordCount example
> > file, i see a void main written for this class.
> > so i am trying to run this program from the standalone mode, by passing
> > the command line arguments.
>

> >
> > 1) Will i be able to run the program this way, because i am running this
> > trough the windows system. If not
> > how can i do the local set up so that i can make the changes to the file
> > and run them on my system to test it?


> >
> > 2) if i make the changes to the file ( just have added few
> > System.Out.println statements) and did a jar file from the examples
> > package
> > and tried to run it from cygwin, again it failed telling its not able to
> > fine the main class. I haven't done any other changes to this apart from
> > the print statements


Does your main look like the main in this example?
http://wiki.apache.org/hadoop/WordCount
If not, copy it in.

Also, if you make some changes to the WordCount.java located in the examples
file, you'll need to use the jar located in the build/ folder.


> >
> > 3) In the example program, during the configuation state, we set the
> input
> > path for the program. Will we be able to set two or more different paths
> > this way?
> > That is, Suppose i have two different files to be read, say one file is
> > already in memory, will i be able to set the configurations such that the
> > input for the Map-Reduce is
> > to read one file from the disk and read the other from the memory in the
> > same Map-Reduce iteration.
>
> Please advise as to how i can proceed from here.
> --
> Thanks,
> Radhika Sridhar
>