You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-user@hadoop.apache.org by da...@uni-koblenz.de on 2011/03/23 16:12:07 UTC
Setup Hadoop for Eclipse
Hi list,
I am having problems to setup Hadoop for Eclipse. I followed the
instructions under:
http://wiki.apache.org/hadoop/EclipseEnvironment
Since I want to debug MapReduce jobs in distributed mode, I decided to
check out 'common', 'hdfs' and 'mapreduce' from SVN. This leaves me with
three different projects:
-hadoop-trunk
-hdfs-trunk
-mapreduce-trunk
Building them as described in the above tutorial worked fine. I am also
able to start the filesystem calling 'bin/start-dfs.sh' in the hdfs-trunk
project.
In order to start mapreduce, I now navigate to the mapreduce-trunk
project, and call 'bin/start-mapred.sh'.However this causes a
java.lang.ClassNotFoundException:
org.apache.hadoop.hdfs.DistributedFileSystem.
What am I missing?
Is it correct, to check-out the three components as different projects?
Help would really be appreciated!
David
RE: Setup Hadoop for Eclipse
Posted by Evert Lammerts <Ev...@sara.nl>.
The VM's (and config) you can find here have not been used for developing and debugging MR jobs in semi-distributed mode, but the setup might help you:
https://grid.sara.nl/wiki/index.php/Hadoop/MapReduce_Development_Environment
Cheers,
Evert Lammerts
> -----Original Message-----
> From: David Saile [mailto:david@uni-koblenz.de]
> Sent: vrijdag 25 maart 2011 13:36
> To: common-user@hadoop.apache.org
> Subject: Re: Setup Hadoop for Eclipse
>
> Thank you very much for the quick reply!
>
> Unfortunately I don't see how this page can help me. I am not using
> Windows, so no need for cygwin.
>
> Are there any other necessary build-path modifications? I tried adding
> the hdfs project to the build-path of the hadoop-common project, but
> that didn't help.
>
> David
>
>
> Am 25.03.2011 um 13:15 schrieb baran cakici:
>
> > I don't know that it helps, but this page is very useful for Work
> with
> > Eclipse.
> >
> > http://ebiquity.umbc.edu/Tutorials/Hadoop/04.5%20env-vars.html
> >
> > regards
> >
> > Baran
> >
> > 2011/3/25 David Saile <da...@uni-koblenz.de>
> >
> >> Does anyone have a working configuration in Eclipse for hadoop
> checked out
> >> as separate projects from SVN (as described under
> >> http://wiki.apache.org/hadoop/EclipseEnvironment)?
> >>
> >> I'm probably missing something simple here, so knowing that others
> made it
> >> work would help!
> >>
> >> Thanks
> >> David
> >>
> >>
> >> Am 23.03.2011 um 16:12 schrieb david@uni-koblenz.de:
> >>
> >>> Hi list,
> >>>
> >>> I am having problems to setup Hadoop for Eclipse. I followed the
> >>> instructions under:
> >>> http://wiki.apache.org/hadoop/EclipseEnvironment
> >>>
> >>> Since I want to debug MapReduce jobs in distributed mode, I decided
> to
> >>> check out 'common', 'hdfs' and 'mapreduce' from SVN. This leaves me
> with
> >>> three different projects:
> >>> -hadoop-trunk
> >>> -hdfs-trunk
> >>> -mapreduce-trunk
> >>>
> >>> Building them as described in the above tutorial worked fine. I am
> also
> >>> able to start the filesystem calling 'bin/start-dfs.sh' in the
> hdfs-trunk
> >>> project.
> >>>
> >>> In order to start mapreduce, I now navigate to the mapreduce-trunk
> >>> project, and call 'bin/start-mapred.sh'.However this causes a
> >>> java.lang.ClassNotFoundException:
> >>> org.apache.hadoop.hdfs.DistributedFileSystem.
> >>>
> >>>
> >>> What am I missing?
> >>> Is it correct, to check-out the three components as different
> projects?
> >>>
> >>> Help would really be appreciated!
> >>>
> >>> David
> >>>
> >>
> >>
Re: Setup Hadoop for Eclipse
Posted by baran cakici <ba...@gmail.com>.
I'm working Hadoop with Eclipse on Windows, I did not make any build-path
modifications. As far as I know , you don't need this.You can try this page.
It's not about Eclipse but it is useful for other configurations.Reason of
your problem can be about your config...
http://www.michael-noll.com/tutorials/running-hadoop-on-ubuntu-linux-single-node-cluster/
unfortunately I have not any suggetions..
Baran
2011/3/25 David Saile <da...@uni-koblenz.de>
> Thank you very much for the quick reply!
>
> Unfortunately I don't see how this page can help me. I am not using
> Windows, so no need for cygwin.
>
> Are there any other necessary build-path modifications? I tried adding the
> hdfs project to the build-path of the hadoop-common project, but that didn't
> help.
>
> David
>
>
> Am 25.03.2011 um 13:15 schrieb baran cakici:
>
> > I don't know that it helps, but this page is very useful for Work with
> > Eclipse.
> >
> > http://ebiquity.umbc.edu/Tutorials/Hadoop/04.5%20env-vars.html
> >
> > regards
> >
> > Baran
> >
> > 2011/3/25 David Saile <da...@uni-koblenz.de>
> >
> >> Does anyone have a working configuration in Eclipse for hadoop checked
> out
> >> as separate projects from SVN (as described under
> >> http://wiki.apache.org/hadoop/EclipseEnvironment)?
> >>
> >> I'm probably missing something simple here, so knowing that others made
> it
> >> work would help!
> >>
> >> Thanks
> >> David
> >>
> >>
> >> Am 23.03.2011 um 16:12 schrieb david@uni-koblenz.de:
> >>
> >>> Hi list,
> >>>
> >>> I am having problems to setup Hadoop for Eclipse. I followed the
> >>> instructions under:
> >>> http://wiki.apache.org/hadoop/EclipseEnvironment
> >>>
> >>> Since I want to debug MapReduce jobs in distributed mode, I decided to
> >>> check out 'common', 'hdfs' and 'mapreduce' from SVN. This leaves me
> with
> >>> three different projects:
> >>> -hadoop-trunk
> >>> -hdfs-trunk
> >>> -mapreduce-trunk
> >>>
> >>> Building them as described in the above tutorial worked fine. I am also
> >>> able to start the filesystem calling 'bin/start-dfs.sh' in the
> hdfs-trunk
> >>> project.
> >>>
> >>> In order to start mapreduce, I now navigate to the mapreduce-trunk
> >>> project, and call 'bin/start-mapred.sh'.However this causes a
> >>> java.lang.ClassNotFoundException:
> >>> org.apache.hadoop.hdfs.DistributedFileSystem.
> >>>
> >>>
> >>> What am I missing?
> >>> Is it correct, to check-out the three components as different projects?
> >>>
> >>> Help would really be appreciated!
> >>>
> >>> David
> >>>
> >>
> >>
>
>
Re: Setup Hadoop for Eclipse
Posted by David Saile <da...@uni-koblenz.de>.
Thank you very much for the quick reply!
Unfortunately I don't see how this page can help me. I am not using Windows, so no need for cygwin.
Are there any other necessary build-path modifications? I tried adding the hdfs project to the build-path of the hadoop-common project, but that didn't help.
David
Am 25.03.2011 um 13:15 schrieb baran cakici:
> I don't know that it helps, but this page is very useful for Work with
> Eclipse.
>
> http://ebiquity.umbc.edu/Tutorials/Hadoop/04.5%20env-vars.html
>
> regards
>
> Baran
>
> 2011/3/25 David Saile <da...@uni-koblenz.de>
>
>> Does anyone have a working configuration in Eclipse for hadoop checked out
>> as separate projects from SVN (as described under
>> http://wiki.apache.org/hadoop/EclipseEnvironment)?
>>
>> I'm probably missing something simple here, so knowing that others made it
>> work would help!
>>
>> Thanks
>> David
>>
>>
>> Am 23.03.2011 um 16:12 schrieb david@uni-koblenz.de:
>>
>>> Hi list,
>>>
>>> I am having problems to setup Hadoop for Eclipse. I followed the
>>> instructions under:
>>> http://wiki.apache.org/hadoop/EclipseEnvironment
>>>
>>> Since I want to debug MapReduce jobs in distributed mode, I decided to
>>> check out 'common', 'hdfs' and 'mapreduce' from SVN. This leaves me with
>>> three different projects:
>>> -hadoop-trunk
>>> -hdfs-trunk
>>> -mapreduce-trunk
>>>
>>> Building them as described in the above tutorial worked fine. I am also
>>> able to start the filesystem calling 'bin/start-dfs.sh' in the hdfs-trunk
>>> project.
>>>
>>> In order to start mapreduce, I now navigate to the mapreduce-trunk
>>> project, and call 'bin/start-mapred.sh'.However this causes a
>>> java.lang.ClassNotFoundException:
>>> org.apache.hadoop.hdfs.DistributedFileSystem.
>>>
>>>
>>> What am I missing?
>>> Is it correct, to check-out the three components as different projects?
>>>
>>> Help would really be appreciated!
>>>
>>> David
>>>
>>
>>
Re: Setup Hadoop for Eclipse
Posted by baran cakici <ba...@gmail.com>.
I don't know that it helps, but this page is very useful for Work with
Eclipse.
http://ebiquity.umbc.edu/Tutorials/Hadoop/04.5%20env-vars.html
regards
Baran
2011/3/25 David Saile <da...@uni-koblenz.de>
> Does anyone have a working configuration in Eclipse for hadoop checked out
> as separate projects from SVN (as described under
> http://wiki.apache.org/hadoop/EclipseEnvironment)?
>
> I'm probably missing something simple here, so knowing that others made it
> work would help!
>
> Thanks
> David
>
>
> Am 23.03.2011 um 16:12 schrieb david@uni-koblenz.de:
>
> > Hi list,
> >
> > I am having problems to setup Hadoop for Eclipse. I followed the
> > instructions under:
> > http://wiki.apache.org/hadoop/EclipseEnvironment
> >
> > Since I want to debug MapReduce jobs in distributed mode, I decided to
> > check out 'common', 'hdfs' and 'mapreduce' from SVN. This leaves me with
> > three different projects:
> > -hadoop-trunk
> > -hdfs-trunk
> > -mapreduce-trunk
> >
> > Building them as described in the above tutorial worked fine. I am also
> > able to start the filesystem calling 'bin/start-dfs.sh' in the hdfs-trunk
> > project.
> >
> > In order to start mapreduce, I now navigate to the mapreduce-trunk
> > project, and call 'bin/start-mapred.sh'.However this causes a
> > java.lang.ClassNotFoundException:
> > org.apache.hadoop.hdfs.DistributedFileSystem.
> >
> >
> > What am I missing?
> > Is it correct, to check-out the three components as different projects?
> >
> > Help would really be appreciated!
> >
> > David
> >
>
>
Re: Setup Hadoop for Eclipse
Posted by David Saile <da...@uni-koblenz.de>.
Does anyone have a working configuration in Eclipse for hadoop checked out as separate projects from SVN (as described under http://wiki.apache.org/hadoop/EclipseEnvironment)?
I'm probably missing something simple here, so knowing that others made it work would help!
Thanks
David
Am 23.03.2011 um 16:12 schrieb david@uni-koblenz.de:
> Hi list,
>
> I am having problems to setup Hadoop for Eclipse. I followed the
> instructions under:
> http://wiki.apache.org/hadoop/EclipseEnvironment
>
> Since I want to debug MapReduce jobs in distributed mode, I decided to
> check out 'common', 'hdfs' and 'mapreduce' from SVN. This leaves me with
> three different projects:
> -hadoop-trunk
> -hdfs-trunk
> -mapreduce-trunk
>
> Building them as described in the above tutorial worked fine. I am also
> able to start the filesystem calling 'bin/start-dfs.sh' in the hdfs-trunk
> project.
>
> In order to start mapreduce, I now navigate to the mapreduce-trunk
> project, and call 'bin/start-mapred.sh'.However this causes a
> java.lang.ClassNotFoundException:
> org.apache.hadoop.hdfs.DistributedFileSystem.
>
>
> What am I missing?
> Is it correct, to check-out the three components as different projects?
>
> Help would really be appreciated!
>
> David
>