You are viewing a plain text version of this content. The canonical link for it is here.
Posted to general@hadoop.apache.org by psdc1978 <ps...@gmail.com> on 2009/11/23 23:10:16 UTC
How compile hadoop-0.20.1
Hi all,
1 - I'm trying to compile hadoop-0.20.1 but I'm facing several errors.
Hadoop includes several subprojects, but these subprojects are all compiled
in one file called hadoop-0.20.1-core.jar. If I just want to update the code
in the MapReduce component, do I have to compile all the project?
2 - Why Hadoop only have one jar for all the project? For example, MapReduce
subcomponent could be inside mapreduce.jar, Common subcomponent could be
inside common.jar, HDFS subcomponent could be inside hdfs.jar, etc.
3 - Is there any tutorial that helps how to compile Hadoop?
4 - Should I try to compile hadoop in windows with Cygwin, or should I
compile inside Ubuntu?
Thanks,
--
Pedro
Re: How compile hadoop-0.20.1
Posted by Rekha Joshi <re...@yahoo-inc.com>.
You got a reply/inline replies on all that..maybe you missed..Attached. Thanks!
On 11/24/09 3:47 PM, "psdc1978" <ps...@gmail.com> wrote:
Hi all,
1 - I'm trying to compile hadoop-0.20.1 but I'm facing several errors.
Hadoop includes several subprojects, but these subprojects are all
compiled in one file called hadoop-0.20.1-core.jar. If I just want to
update the code in the MapReduce component, do I have to compile all
the project?
2 - Why Hadoop only have one jar (hadoop-0.20.1-core.jar) for all the
project? For example, MapReduce subcomponent could be inside
mapreduce.jar, Common subcomponent could be inside common.jar, HDFS
subcomponent could be inside hdfs.jar, etc.
3 - Is there any tutorial that helps how to compile Hadoop, especially
only the MapReduce component?
4 - Should I try to compile MapReduce in windows with Cygwin, or
should I compile inside Ubuntu?
Thanks,
--
Pedro
Re: How compile hadoop-0.20.1
Posted by psdc1978 <ps...@gmail.com>.
Hi all,
1 - I'm trying to compile hadoop-0.20.1 but I'm facing several errors.
Hadoop includes several subprojects, but these subprojects are all
compiled in one file called hadoop-0.20.1-core.jar. If I just want to
update the code in the MapReduce component, do I have to compile all
the project?
2 - Why Hadoop only have one jar (hadoop-0.20.1-core.jar) for all the
project? For example, MapReduce subcomponent could be inside
mapreduce.jar, Common subcomponent could be inside common.jar, HDFS
subcomponent could be inside hdfs.jar, etc.
3 - Is there any tutorial that helps how to compile Hadoop, especially
only the MapReduce component?
4 - Should I try to compile MapReduce in windows with Cygwin, or
should I compile inside Ubuntu?
Thanks,
--
Pedro
Re: How compile hadoop-0.20.1
Posted by Todd Lipcon <to...@cloudera.com>.
Hi Pedro,
If you're trying to compile 0.20.1, that version is pre-project-split.
There's only one jar because it predates the point at which Hadoop was split
into common/hdfs/MR. I've answered the specific questions below:
On Mon, Nov 23, 2009 at 2:39 PM, psdc1978 <ps...@gmail.com> wrote:
> Hi all,
>
>
> 1 - I'm trying to compile hadoop-0.20.1 but I'm facing several
> errors. Hadoop includes several subprojects, but these subprojects
> are all compiled in one file called hadoop-0.20.1-core.jar. If I just
> want to update the code in the MapReduce component, do I have to
> compile all the project?
>
In 0.20.1, yes. But it compiles quickly - only a couple minutes.
>
> 2 - Why Hadoop only have one jar for all the project? For example,
> MapReduce subcomponent could be inside mapreduce.jar, Common
> subcomponent could be inside common.jar, HDFS subcomponent could be
> inside hdfs.jar, etc.
>
>
As noted above, 0.20.1 is pre-split. In 0.21 and above, it will be separate
jars as you expected.
> 3 - Is there any tutorial that helps how to compile Hadoop,
> especially only the MapReduce component?
>
> In 0.20.1, you can't do just one component, really. "ant bin-package" is a
good way to compile that won't require a lot of other junk on your system.
> 4 - Should I try to compile MapReduce in windows with Cygwin, or
> should I compile inside Ubuntu?
>
> Ubuntu should be easier, though I'm told it's possible to do so in Cygwin.
-Todd
Re: How compile hadoop-0.20.1
Posted by psdc1978 <ps...@gmail.com>.
Hi all,
1 - I'm trying to compile hadoop-0.20.1 but I'm facing several
errors. Hadoop includes several subprojects, but these subprojects
are all compiled in one file called hadoop-0.20.1-core.jar. If I just
want to update the code in the MapReduce component, do I have to
compile all the project?
2 - Why Hadoop only have one jar for all the project? For example,
MapReduce subcomponent could be inside mapreduce.jar, Common
subcomponent could be inside common.jar, HDFS subcomponent could be
inside hdfs.jar, etc.
3 - Is there any tutorial that helps how to compile Hadoop,
especially only the MapReduce component?
4 - Should I try to compile MapReduce in windows with Cygwin, or
should I compile inside Ubuntu?
Thanks,
--
Pedro
Re: How compile hadoop-0.20.1
Posted by Owen O'Malley <om...@apache.org>.
On Nov 23, 2009, at 2:10 PM, psdc1978 wrote:
> Hi all,
>
> 1 - I'm trying to compile hadoop-0.20.1 but I'm facing several errors.
> Hadoop includes several subprojects, but these subprojects are all
> compiled
> in one file called hadoop-0.20.1-core.jar. If I just want to update
> the code
> in the MapReduce component, do I have to compile all the project?
The project split happened after the 0.20 branch. Common, HDFS, and
MapReduce were a single project named Core.
> 3 - Is there any tutorial that helps how to compile Hadoop?
http://wiki.apache.org/hadoop/HowToRelease
> 4 - Should I try to compile hadoop in windows with Cygwin, or should I
> compile inside Ubuntu?
I'd strongly suggest Ubuntu. Very few organizations other than
Microsoft run large Windows clusters. *smile* Of course compiling the
jars can happen on any operating system, but the native parts (if you
want them) need to be compiled on the deployment platform.
-- Owen
Re: How compile hadoop-0.20.1
Posted by Steve Loughran <st...@apache.org>.
psdc1978 wrote:
> Hi all,
>
> 1 - I'm trying to compile hadoop-0.20.1 but I'm facing several errors.
> Hadoop includes several subprojects, but these subprojects are all compiled
> in one file called hadoop-0.20.1-core.jar. If I just want to update the code
> in the MapReduce component, do I have to compile all the project?
yes
>
> 2 - Why Hadoop only have one jar for all the project? For example, MapReduce
> subcomponent could be inside mapreduce.jar, Common subcomponent could be
> inside common.jar, HDFS subcomponent could be inside hdfs.jar, etc.
in SVN_HEAD Hadoop it is split up. That will give you a version
synchronisation problem that didn't exist before, of course
> 3 - Is there any tutorial that helps how to compile Hadoop?
Check out SVN, run "ant -p" to list the targets
>
> 4 - Should I try to compile hadoop in windows with Cygwin, or should I
> compile inside Ubuntu?
If you have linux, I would go for that. It does build under cygwin, and
on OS/X, but there's a lot to be said for working in Linux on your
desktop so that the problems you encounter are the same as you'd see in
the cluster