You are viewing a plain text version of this content. The canonical link for it is here.
Posted to mapreduce-dev@hadoop.apache.org by HU Wenjing A <We...@alcatel-sbell.com.cn> on 2012/06/08 02:53:24 UTC
hadoop source code build failed
Hi,
I am sorry if this question was asked a lot of times before but I am unable to solve my problem. I check out the hadoop 0.22.0 source code and want to build it using ant&ivy,but I encountered some problems.As far as my understanding goes, I need to build common, hdfs, and mapred. It seems that common could be build successfully, but hdfs and mapred failed the build. Here is the error I receive for those two part:
hdfs:
Buildfile: /root/hadoop-0.22.0/hadoop_compile/hdfs/build.xml
clover.setup:
clover.info:
[echo]
[echo] Clover not found. Code coverage reports disabled.
[echo]
clover:
ivy-download:
[get] Getting: http://repo2.maven.org/maven2/org/apache/ivy/ivy/2.1.0/ivy-2.1.0.jar
[get] To: /root/hadoop-0.22.0/hadoop_compile/hdfs/ivy/ivy-2.1.0.jar
[get] Not modified - so not downloaded
ivy-init-dirs:
ivy-probe-antlib:
ivy-init-antlib:
ivy-init:
[ivy:configure] :: Apache Ivy 2.3.0-rc1 - 20120416000235 :: http://ant.apache.org/ivy/ ::
[ivy:configure] :: loading settings :: file = /root/hadoop-0.22.0/hadoop_compile/hdfs/ivy/ivysettings.xml
ivy-resolve-common:
[ivy:resolve]
[ivy:resolve] :: problems summary ::
[ivy:resolve] :::: WARNINGS
[ivy:resolve] module not found: org.apache.hadoop#hadoop-common;0.22.0-SNAPSHOT
[ivy:resolve] ==== apache-snapshot: tried
[ivy:resolve] https://repository.apache.org/content/repositories/snapshots/org/apache/hadoop/hadoop-common/0.22.0-SNAPSHOT/hadoop-common-0.22.0-SNAPSHOT.pom
[ivy:resolve] -- artifact org.apache.hadoop#hadoop-common;0.22.0-SNAPSHOT!hadoop-common.jar:
[ivy:resolve] https://repository.apache.org/content/repositories/snapshots/org/apache/hadoop/hadoop-common/0.22.0-SNAPSHOT/hadoop-common-0.22.0-SNAPSHOT.jar
[ivy:resolve] ==== maven2: tried
[ivy:resolve] http://repo1.maven.org/maven2/org/apache/hadoop/hadoop-common/0.22.0-SNAPSHOT/hadoop-common-0.22.0-SNAPSHOT.pom
[ivy:resolve] -- artifact org.apache.hadoop#hadoop-common;0.22.0-SNAPSHOT!hadoop-common.jar:
[ivy:resolve] http://repo1.maven.org/maven2/org/apache/hadoop/hadoop-common/0.22.0-SNAPSHOT/hadoop-common-0.22.0-SNAPSHOT.jar
[ivy:resolve] ::::::::::::::::::::::::::::::::::::::::::::::
[ivy:resolve] :: UNRESOLVED DEPENDENCIES ::
[ivy:resolve] ::::::::::::::::::::::::::::::::::::::::::::::
[ivy:resolve] :: org.apache.hadoop#hadoop-common;0.22.0-SNAPSHOT: not found
[ivy:resolve] ::::::::::::::::::::::::::::::::::::::::::::::
[ivy:resolve]
[ivy:resolve] :: USE VERBOSE OR DEBUG MESSAGE LEVEL FOR MORE DETAILS
BUILD FAILED
/root/hadoop-0.22.0/hadoop_compile/hdfs/build.xml:1814: impossible to resolve dependencies:
resolve failed - see output for details
Total time: 52 seconds
mapreduce:
Buildfile: /root/hadoop-0.22.0/hadoop_compile/mapreduce/build.xml
clover.setup:
clover.info:
[echo]
[echo] Clover not found. Code coverage reports disabled.
[echo]
clover:
ivy-download:
[get] Getting: http://repo2.maven.org/maven2/org/apache/ivy/ivy/2.1.0/ivy-2.1.0.jar
[get] To: /root/hadoop-0.22.0/hadoop_compile/mapreduce/ivy/ivy-2.1.0.jar
[get] Not modified - so not downloaded
ivy-init-dirs:
ivy-probe-antlib:
ivy-init-antlib:
ivy-init:
[ivy:configure] :: Apache Ivy 2.3.0-rc1 - 20120416000235 :: http://ant.apache.org/ivy/ ::
[ivy:configure] :: loading settings :: file = /root/hadoop-0.22.0/hadoop_compile/mapreduce/ivy/ivysettings.xml
ivy-resolve-common:
[ivy:resolve]
[ivy:resolve] :: problems summary ::
[ivy:resolve] :::: WARNINGS
[ivy:resolve] module not found: org.apache.hadoop#hadoop-common;0.22.0-SNAPSHOT
[ivy:resolve] ==== apache-snapshot: tried
[ivy:resolve] https://repository.apache.org/content/repositories/snapshots/org/apache/hadoop/hadoop-common/0.22.0-SNAPSHOT/hadoop-common-0.22.0-SNAPSHOT.pom
[ivy:resolve] -- artifact org.apache.hadoop#hadoop-common;0.22.0-SNAPSHOT!hadoop-common.jar:
[ivy:resolve] https://repository.apache.org/content/repositories/snapshots/org/apache/hadoop/hadoop-common/0.22.0-SNAPSHOT/hadoop-common-0.22.0-SNAPSHOT.jar
[ivy:resolve] ==== maven2: tried
[ivy:resolve] http://repo1.maven.org/maven2/org/apache/hadoop/hadoop-common/0.22.0-SNAPSHOT/hadoop-common-0.22.0-SNAPSHOT.pom
[ivy:resolve] -- artifact org.apache.hadoop#hadoop-common;0.22.0-SNAPSHOT!hadoop-common.jar:
[ivy:resolve] http://repo1.maven.org/maven2/org/apache/hadoop/hadoop-common/0.22.0-SNAPSHOT/hadoop-common-0.22.0-SNAPSHOT.jar
[ivy:resolve] module not found: org.apache.hadoop#hadoop-common-test;0.22.0-SNAPSHOT
[ivy:resolve] ==== apache-snapshot: tried
[ivy:resolve] https://repository.apache.org/content/repositories/snapshots/org/apache/hadoop/hadoop-common-test/0.22.0-SNAPSHOT/hadoop-common-test-0.22.0-SNAPSHOT.pom
[ivy:resolve] -- artifact org.apache.hadoop#hadoop-common-test;0.22.0-SNAPSHOT!hadoop-common-test.jar:
[ivy:resolve] https://repository.apache.org/content/repositories/snapshots/org/apache/hadoop/hadoop-common-test/0.22.0-SNAPSHOT/hadoop-common-test-0.22.0-SNAPSHOT.jar
[ivy:resolve] ==== maven2: tried
[ivy:resolve] http://repo1.maven.org/maven2/org/apache/hadoop/hadoop-common-test/0.22.0-SNAPSHOT/hadoop-common-test-0.22.0-SNAPSHOT.pom
[ivy:resolve] -- artifact org.apache.hadoop#hadoop-common-test;0.22.0-SNAPSHOT!hadoop-common-test.jar:
[ivy:resolve] http://repo1.maven.org/maven2/org/apache/hadoop/hadoop-common-test/0.22.0-SNAPSHOT/hadoop-common-test-0.22.0-SNAPSHOT.jar
[ivy:resolve] module not found: org.apache.hadoop#hadoop-hdfs;0.22.0-SNAPSHOT
[ivy:resolve] ==== apache-snapshot: tried
[ivy:resolve] https://repository.apache.org/content/repositories/snapshots/org/apache/hadoop/hadoop-hdfs/0.22.0-SNAPSHOT/hadoop-hdfs-0.22.0-SNAPSHOT.pom
[ivy:resolve] -- artifact org.apache.hadoop#hadoop-hdfs;0.22.0-SNAPSHOT!hadoop-hdfs.jar:
[ivy:resolve] https://repository.apache.org/content/repositories/snapshots/org/apache/hadoop/hadoop-hdfs/0.22.0-SNAPSHOT/hadoop-hdfs-0.22.0-SNAPSHOT.jar
[ivy:resolve] ==== maven2: tried
[ivy:resolve] http://repo1.maven.org/maven2/org/apache/hadoop/hadoop-hdfs/0.22.0-SNAPSHOT/hadoop-hdfs-0.22.0-SNAPSHOT.pom
[ivy:resolve] -- artifact org.apache.hadoop#hadoop-hdfs;0.22.0-SNAPSHOT!hadoop-hdfs.jar:
[ivy:resolve] http://repo1.maven.org/maven2/org/apache/hadoop/hadoop-hdfs/0.22.0-SNAPSHOT/hadoop-hdfs-0.22.0-SNAPSHOT.jar
[ivy:resolve] ::::::::::::::::::::::::::::::::::::::::::::::
[ivy:resolve] :: UNRESOLVED DEPENDENCIES ::
[ivy:resolve] ::::::::::::::::::::::::::::::::::::::::::::::
[ivy:resolve] :: org.apache.hadoop#hadoop-common;0.22.0-SNAPSHOT: not found
[ivy:resolve] :: org.apache.hadoop#hadoop-common-test;0.22.0-SNAPSHOT: not found
[ivy:resolve] :: org.apache.hadoop#hadoop-hdfs;0.22.0-SNAPSHOT: not found
[ivy:resolve] ::::::::::::::::::::::::::::::::::::::::::::::
[ivy:resolve]
[ivy:resolve] :: USE VERBOSE OR DEBUG MESSAGE LEVEL FOR MORE DETAILS
BUILD FAILED
/root/hadoop-0.22.0/hadoop_compile/mapreduce/build.xml:2117: impossible to resolve dependencies:
resolve failed - see output for details
Total time: 39 seconds
I searched the internet and see someone says this is because of the ivy configuration and give solution like below：
for hdfs part ，modify the ivy.xml as following：（---- means delete，+++ means add）
hdfs/ivy.xml
<dependency org="org.apache.hadoop" name="hadoop-common"
----- rev="${hadoop-common.version}"
+++ rev="${hadoop-common.version}-SNAPSHOT"
conf="common->default"/>
hdfs/src/contrib/hdfsproxy/ivy.xml
<dependency org="org.apache.hadoop"
name="hadoop-common"
---- rev="${hadoop-common.version}"
+++ rev="${hadoop-common.version}-SNAPSHOT"
conf="common->default"/>
<dependency org="org.apache.hadoop"
name="hadoop-common-test"
--- rev="${hadoop-common.version}"
+++ rev="${hadoop-common.version}-SNAPSHOT"
conf="common->default"/>
hdfs/src/contrib/thriftfs/ivy.xml
<dependency org="org.apache.hadoop"
name="hadoop-common"
--- rev="${hadoop-common.version}"
+++ rev="${hadoop-common.version}-SNAPSHOT"
conf="common->default"/>
for mapreduce part：
first modify all the ivy.xml in project mapreduce:
rev="${hadoop-common.version}" replaced by rev="${hadoop-common.version}-SNAPSHOT"
rev="${hadoop-hdfs.version}" replaced by rev="${hadoop-hdfs.version}-SNAPSHOT"
then modify the file buil.xml
<unzip
---- src="${common.ivy.lib.dir}/hadoop-hdfs-${hadoop-hdfs.version}.jar"
+++ src="${common.ivy.lib.dir}/hadoop-hdfs-${hadoop-hdfs.version}-SNAPSHOT.jar" dest="${build.dir}">
Bur after the modification ，the build still failed .So could you give me some suggestions about the source code build especially for the 0.22.0 version
Thanks & Best regards
wenjing
Re: hadoop source code build failed
Posted by 此案彼岸 <sw...@126.com>.
于 2012/6/8 10:15, Nishchay P 写道:
> Hi Wenjing,
>
> It looks like the website
> https://repository.apache.org/content/repositories/snapshots/org/apache/had
> oop/hadoop-common/ doesn't have 0.22.0 may be if you can checkout
> 0.22.1-SNAPSHOT the build might work.
>
> Thanks,
> Nishchay
>
>
>
>
> On 08/06/12 6:23 AM, "HU Wenjing A" <We...@alcatel-sbell.com.cn>
> wrote:
>
>> Hi,
>>
>> I am sorry if this question was asked a lot of times before but I am
>> unable to solve my problem. I check out the hadoop 0.22.0 source code and
>> want to build it using ant&ivy,but I encountered some problems.As far as
>> my understanding goes, I need to build common, hdfs, and mapred. It seems
>> that common could be build successfully, but hdfs and mapred failed the
>> build. Here is the error I receive for those two part:
>>
>> hdfs:
>>
>> Buildfile: /root/hadoop-0.22.0/hadoop_compile/hdfs/build.xml
>>
>> clover.setup:
>>
>> clover.info:
>> [echo]
>> [echo] Clover not found. Code coverage reports disabled.
>> [echo]
>>
>> clover:
>>
>> ivy-download:
>> [get] Getting:
>> http://repo2.maven.org/maven2/org/apache/ivy/ivy/2.1.0/ivy-2.1.0.jar
>> [get] To: /root/hadoop-0.22.0/hadoop_compile/hdfs/ivy/ivy-2.1.0.jar
>> [get] Not modified - so not downloaded
>>
>> ivy-init-dirs:
>>
>> ivy-probe-antlib:
>>
>> ivy-init-antlib:
>>
>> ivy-init:
>> [ivy:configure] :: Apache Ivy 2.3.0-rc1 - 20120416000235 ::
>> http://ant.apache.org/ivy/ ::
>> [ivy:configure] :: loading settings :: file =
>> /root/hadoop-0.22.0/hadoop_compile/hdfs/ivy/ivysettings.xml
>>
>> ivy-resolve-common:
>> [ivy:resolve]
>> [ivy:resolve] :: problems summary ::
>> [ivy:resolve] :::: WARNINGS
>> [ivy:resolve] module not found:
>> org.apache.hadoop#hadoop-common;0.22.0-SNAPSHOT
>> [ivy:resolve] ==== apache-snapshot: tried
>> [ivy:resolve]
>> https://repository.apache.org/content/repositories/snapshots/org/apache/ha
>> doop/hadoop-common/0.22.0-SNAPSHOT/hadoop-common-0.22.0-SNAPSHOT.pom
>> [ivy:resolve] -- artifact
>> org.apache.hadoop#hadoop-common;0.22.0-SNAPSHOT!hadoop-common.jar:
>> [ivy:resolve]
>> https://repository.apache.org/content/repositories/snapshots/org/apache/ha
>> doop/hadoop-common/0.22.0-SNAPSHOT/hadoop-common-0.22.0-SNAPSHOT.jar
>> [ivy:resolve] ==== maven2: tried
>> [ivy:resolve]
>> http://repo1.maven.org/maven2/org/apache/hadoop/hadoop-common/0.22.0-SNAPS
>> HOT/hadoop-common-0.22.0-SNAPSHOT.pom
>> [ivy:resolve] -- artifact
>> org.apache.hadoop#hadoop-common;0.22.0-SNAPSHOT!hadoop-common.jar:
>> [ivy:resolve]
>> http://repo1.maven.org/maven2/org/apache/hadoop/hadoop-common/0.22.0-SNAPS
>> HOT/hadoop-common-0.22.0-SNAPSHOT.jar
>> [ivy:resolve] ::::::::::::::::::::::::::::::::::::::::::::::
>> [ivy:resolve] :: UNRESOLVED DEPENDENCIES ::
>> [ivy:resolve] ::::::::::::::::::::::::::::::::::::::::::::::
>> [ivy:resolve] ::
>> org.apache.hadoop#hadoop-common;0.22.0-SNAPSHOT: not found
>> [ivy:resolve] ::::::::::::::::::::::::::::::::::::::::::::::
>> [ivy:resolve]
>> [ivy:resolve] :: USE VERBOSE OR DEBUG MESSAGE LEVEL FOR MORE DETAILS
>>
>> BUILD FAILED
>> /root/hadoop-0.22.0/hadoop_compile/hdfs/build.xml:1814: impossible to
>> resolve dependencies:
>> resolve failed - see output for details
>>
>> Total time: 52 seconds
>>
>>
>> mapreduce:
>>
>> Buildfile: /root/hadoop-0.22.0/hadoop_compile/mapreduce/build.xml
>>
>> clover.setup:
>>
>> clover.info:
>> [echo]
>> [echo] Clover not found. Code coverage reports disabled.
>> [echo]
>>
>> clover:
>>
>> ivy-download:
>> [get] Getting:
>> http://repo2.maven.org/maven2/org/apache/ivy/ivy/2.1.0/ivy-2.1.0.jar
>> [get] To:
>> /root/hadoop-0.22.0/hadoop_compile/mapreduce/ivy/ivy-2.1.0.jar
>> [get] Not modified - so not downloaded
>>
>> ivy-init-dirs:
>>
>> ivy-probe-antlib:
>>
>> ivy-init-antlib:
>>
>> ivy-init:
>> [ivy:configure] :: Apache Ivy 2.3.0-rc1 - 20120416000235 ::
>> http://ant.apache.org/ivy/ ::
>> [ivy:configure] :: loading settings :: file =
>> /root/hadoop-0.22.0/hadoop_compile/mapreduce/ivy/ivysettings.xml
>>
>> ivy-resolve-common:
>> [ivy:resolve]
>> [ivy:resolve] :: problems summary ::
>> [ivy:resolve] :::: WARNINGS
>> [ivy:resolve] module not found:
>> org.apache.hadoop#hadoop-common;0.22.0-SNAPSHOT
>> [ivy:resolve] ==== apache-snapshot: tried
>> [ivy:resolve]
>> https://repository.apache.org/content/repositories/snapshots/org/apache/ha
>> doop/hadoop-common/0.22.0-SNAPSHOT/hadoop-common-0.22.0-SNAPSHOT.pom
>> [ivy:resolve] -- artifact
>> org.apache.hadoop#hadoop-common;0.22.0-SNAPSHOT!hadoop-common.jar:
>> [ivy:resolve]
>> https://repository.apache.org/content/repositories/snapshots/org/apache/ha
>> doop/hadoop-common/0.22.0-SNAPSHOT/hadoop-common-0.22.0-SNAPSHOT.jar
>> [ivy:resolve] ==== maven2: tried
>> [ivy:resolve]
>> http://repo1.maven.org/maven2/org/apache/hadoop/hadoop-common/0.22.0-SNAPS
>> HOT/hadoop-common-0.22.0-SNAPSHOT.pom
>> [ivy:resolve] -- artifact
>> org.apache.hadoop#hadoop-common;0.22.0-SNAPSHOT!hadoop-common.jar:
>> [ivy:resolve]
>> http://repo1.maven.org/maven2/org/apache/hadoop/hadoop-common/0.22.0-SNAPS
>> HOT/hadoop-common-0.22.0-SNAPSHOT.jar
>> [ivy:resolve] module not found:
>> org.apache.hadoop#hadoop-common-test;0.22.0-SNAPSHOT
>> [ivy:resolve] ==== apache-snapshot: tried
>> [ivy:resolve]
>> https://repository.apache.org/content/repositories/snapshots/org/apache/ha
>> doop/hadoop-common-test/0.22.0-SNAPSHOT/hadoop-common-test-0.22.0-SNAPSHOT
>> .pom
>> [ivy:resolve] -- artifact
>> org.apache.hadoop#hadoop-common-test;0.22.0-SNAPSHOT!hadoop-common-test.ja
>> r:
>> [ivy:resolve]
>> https://repository.apache.org/content/repositories/snapshots/org/apache/ha
>> doop/hadoop-common-test/0.22.0-SNAPSHOT/hadoop-common-test-0.22.0-SNAPSHOT
>> .jar
>> [ivy:resolve] ==== maven2: tried
>> [ivy:resolve]
>> http://repo1.maven.org/maven2/org/apache/hadoop/hadoop-common-test/0.22.0-
>> SNAPSHOT/hadoop-common-test-0.22.0-SNAPSHOT.pom
>> [ivy:resolve] -- artifact
>> org.apache.hadoop#hadoop-common-test;0.22.0-SNAPSHOT!hadoop-common-test.ja
>> r:
>> [ivy:resolve]
>> http://repo1.maven.org/maven2/org/apache/hadoop/hadoop-common-test/0.22.0-
>> SNAPSHOT/hadoop-common-test-0.22.0-SNAPSHOT.jar
>> [ivy:resolve] module not found:
>> org.apache.hadoop#hadoop-hdfs;0.22.0-SNAPSHOT
>> [ivy:resolve] ==== apache-snapshot: tried
>> [ivy:resolve]
>> https://repository.apache.org/content/repositories/snapshots/org/apache/ha
>> doop/hadoop-hdfs/0.22.0-SNAPSHOT/hadoop-hdfs-0.22.0-SNAPSHOT.pom
>> [ivy:resolve] -- artifact
>> org.apache.hadoop#hadoop-hdfs;0.22.0-SNAPSHOT!hadoop-hdfs.jar:
>> [ivy:resolve]
>> https://repository.apache.org/content/repositories/snapshots/org/apache/ha
>> doop/hadoop-hdfs/0.22.0-SNAPSHOT/hadoop-hdfs-0.22.0-SNAPSHOT.jar
>> [ivy:resolve] ==== maven2: tried
>> [ivy:resolve]
>> http://repo1.maven.org/maven2/org/apache/hadoop/hadoop-hdfs/0.22.0-SNAPSHO
>> T/hadoop-hdfs-0.22.0-SNAPSHOT.pom
>> [ivy:resolve] -- artifact
>> org.apache.hadoop#hadoop-hdfs;0.22.0-SNAPSHOT!hadoop-hdfs.jar:
>> [ivy:resolve]
>> http://repo1.maven.org/maven2/org/apache/hadoop/hadoop-hdfs/0.22.0-SNAPSHO
>> T/hadoop-hdfs-0.22.0-SNAPSHOT.jar
>> [ivy:resolve] ::::::::::::::::::::::::::::::::::::::::::::::
>> [ivy:resolve] :: UNRESOLVED DEPENDENCIES ::
>> [ivy:resolve] ::::::::::::::::::::::::::::::::::::::::::::::
>> [ivy:resolve] ::
>> org.apache.hadoop#hadoop-common;0.22.0-SNAPSHOT: not found
>> [ivy:resolve] ::
>> org.apache.hadoop#hadoop-common-test;0.22.0-SNAPSHOT: not found
>> [ivy:resolve] :: org.apache.hadoop#hadoop-hdfs;0.22.0-SNAPSHOT:
>> not found
>> [ivy:resolve] ::::::::::::::::::::::::::::::::::::::::::::::
>> [ivy:resolve]
>> [ivy:resolve] :: USE VERBOSE OR DEBUG MESSAGE LEVEL FOR MORE DETAILS
>>
>> BUILD FAILED
>> /root/hadoop-0.22.0/hadoop_compile/mapreduce/build.xml:2117: impossible
>> to resolve dependencies:
>> resolve failed - see output for details
>>
>> Total time: 39 seconds
>>
>> I searched the internet and see someone says this is because of the ivy
>> configuration and give solution like below：
>>
>> for hdfs part ，modify the ivy.xml as following：（---- means delete，+++
>> means add）
>>
>> hdfs/ivy.xml
>>
>> <dependency org="org.apache.hadoop" name="hadoop-common"
>>
>> ----- rev="${hadoop-common.version}"
>>
>> +++ rev="${hadoop-common.version}-SNAPSHOT"
>>
>> conf="common->default"/>
>>
>> hdfs/src/contrib/hdfsproxy/ivy.xml
>>
>> <dependency org="org.apache.hadoop"
>>
>> name="hadoop-common"
>>
>> ---- rev="${hadoop-common.version}"
>>
>> +++ rev="${hadoop-common.version}-SNAPSHOT"
>>
>> conf="common->default"/>
>>
>> <dependency org="org.apache.hadoop"
>>
>> name="hadoop-common-test"
>>
>> --- rev="${hadoop-common.version}"
>>
>> +++ rev="${hadoop-common.version}-SNAPSHOT"
>>
>> conf="common->default"/>
>>
>> hdfs/src/contrib/thriftfs/ivy.xml
>>
>> <dependency org="org.apache.hadoop"
>>
>> name="hadoop-common"
>>
>> --- rev="${hadoop-common.version}"
>>
>> +++ rev="${hadoop-common.version}-SNAPSHOT"
>>
>> conf="common->default"/>
>>
>>
>> for mapreduce part：
>> first modify all the ivy.xml in project mapreduce:
>>
>> rev="${hadoop-common.version}" replaced by
>> rev="${hadoop-common.version}-SNAPSHOT"
>>
>> rev="${hadoop-hdfs.version}" replaced by
>> rev="${hadoop-hdfs.version}-SNAPSHOT"
>> then modify the file buil.xml
>>
>> <unzip
>>
>> ----
>> src="${common.ivy.lib.dir}/hadoop-hdfs-${hadoop-hdfs.version}.jar"
>>
>> +++
>> src="${common.ivy.lib.dir}/hadoop-hdfs-${hadoop-hdfs.version}-SNAPSHOT.jar
>> " dest="${build.dir}">
>>
>>
>> Bur after the modification ，the build still failed .So could you give me
>> some suggestions about the source code build especially for the 0.22.0
>> version
>>
>> Thanks & Best regards
>> wenjing
>>
>>
>>
>
> ________________________________
>
> The information contained in this communication is intended solely for the use of the individual or entity to whom it is addressed and others authorized to receive it. It may contain confidential or legally privileged information. If you are not the intended recipient you are hereby notified that any disclosure, copying, distribution or taking any action in reliance on the contents of this information is strictly prohibited and may be unlawful. If you have received this communication in error, please notify us immediately by responding to this email and then delete it from your system. The firm is neither liable for the proper and complete transmission of the information contained in this communication nor for any delay in its receipt.
>
Hi Nishchay,
I am Wenjing just using my personal mailbox. First thank you so much for
response. I just saw this mail and according to your response I checked
the url and found it's true there are some files missing for the version
0.22.0. Actually I have changed back to version 0.21.0 to do the souce
code build and all of the three main parts of the code can be build
successfully. ：)
Thanks again. O(∩_∩)O~
Thanks & best regards,
Wenjing
Re: hadoop source code build failed
Posted by Nishchay P <Ni...@inmobi.com>.
Hi Wenjing,
It looks like the website
https://repository.apache.org/content/repositories/snapshots/org/apache/had
oop/hadoop-common/ doesn't have 0.22.0 may be if you can checkout
0.22.1-SNAPSHOT the build might work.
Thanks,
Nishchay
On 08/06/12 6:23 AM, "HU Wenjing A" <We...@alcatel-sbell.com.cn>
wrote:
>
>Hi,
>
> I am sorry if this question was asked a lot of times before but I am
>unable to solve my problem. I check out the hadoop 0.22.0 source code and
>want to build it using ant&ivy,but I encountered some problems.As far as
>my understanding goes, I need to build common, hdfs, and mapred. It seems
>that common could be build successfully, but hdfs and mapred failed the
>build. Here is the error I receive for those two part:
>
>hdfs:
>
>Buildfile: /root/hadoop-0.22.0/hadoop_compile/hdfs/build.xml
>
>clover.setup:
>
>clover.info:
> [echo]
> [echo] Clover not found. Code coverage reports disabled.
> [echo]
>
>clover:
>
>ivy-download:
> [get] Getting:
>http://repo2.maven.org/maven2/org/apache/ivy/ivy/2.1.0/ivy-2.1.0.jar
> [get] To: /root/hadoop-0.22.0/hadoop_compile/hdfs/ivy/ivy-2.1.0.jar
> [get] Not modified - so not downloaded
>
>ivy-init-dirs:
>
>ivy-probe-antlib:
>
>ivy-init-antlib:
>
>ivy-init:
>[ivy:configure] :: Apache Ivy 2.3.0-rc1 - 20120416000235 ::
>http://ant.apache.org/ivy/ ::
>[ivy:configure] :: loading settings :: file =
>/root/hadoop-0.22.0/hadoop_compile/hdfs/ivy/ivysettings.xml
>
>ivy-resolve-common:
>[ivy:resolve]
>[ivy:resolve] :: problems summary ::
>[ivy:resolve] :::: WARNINGS
>[ivy:resolve] module not found:
>org.apache.hadoop#hadoop-common;0.22.0-SNAPSHOT
>[ivy:resolve] ==== apache-snapshot: tried
>[ivy:resolve]
>https://repository.apache.org/content/repositories/snapshots/org/apache/ha
>doop/hadoop-common/0.22.0-SNAPSHOT/hadoop-common-0.22.0-SNAPSHOT.pom
>[ivy:resolve] -- artifact
>org.apache.hadoop#hadoop-common;0.22.0-SNAPSHOT!hadoop-common.jar:
>[ivy:resolve]
>https://repository.apache.org/content/repositories/snapshots/org/apache/ha
>doop/hadoop-common/0.22.0-SNAPSHOT/hadoop-common-0.22.0-SNAPSHOT.jar
>[ivy:resolve] ==== maven2: tried
>[ivy:resolve]
>http://repo1.maven.org/maven2/org/apache/hadoop/hadoop-common/0.22.0-SNAPS
>HOT/hadoop-common-0.22.0-SNAPSHOT.pom
>[ivy:resolve] -- artifact
>org.apache.hadoop#hadoop-common;0.22.0-SNAPSHOT!hadoop-common.jar:
>[ivy:resolve]
>http://repo1.maven.org/maven2/org/apache/hadoop/hadoop-common/0.22.0-SNAPS
>HOT/hadoop-common-0.22.0-SNAPSHOT.jar
>[ivy:resolve] ::::::::::::::::::::::::::::::::::::::::::::::
>[ivy:resolve] :: UNRESOLVED DEPENDENCIES ::
>[ivy:resolve] ::::::::::::::::::::::::::::::::::::::::::::::
>[ivy:resolve] ::
>org.apache.hadoop#hadoop-common;0.22.0-SNAPSHOT: not found
>[ivy:resolve] ::::::::::::::::::::::::::::::::::::::::::::::
>[ivy:resolve]
>[ivy:resolve] :: USE VERBOSE OR DEBUG MESSAGE LEVEL FOR MORE DETAILS
>
>BUILD FAILED
>/root/hadoop-0.22.0/hadoop_compile/hdfs/build.xml:1814: impossible to
>resolve dependencies:
> resolve failed - see output for details
>
>Total time: 52 seconds
>
>
>mapreduce:
>
>Buildfile: /root/hadoop-0.22.0/hadoop_compile/mapreduce/build.xml
>
>clover.setup:
>
>clover.info:
> [echo]
> [echo] Clover not found. Code coverage reports disabled.
> [echo]
>
>clover:
>
>ivy-download:
> [get] Getting:
>http://repo2.maven.org/maven2/org/apache/ivy/ivy/2.1.0/ivy-2.1.0.jar
> [get] To:
>/root/hadoop-0.22.0/hadoop_compile/mapreduce/ivy/ivy-2.1.0.jar
> [get] Not modified - so not downloaded
>
>ivy-init-dirs:
>
>ivy-probe-antlib:
>
>ivy-init-antlib:
>
>ivy-init:
>[ivy:configure] :: Apache Ivy 2.3.0-rc1 - 20120416000235 ::
>http://ant.apache.org/ivy/ ::
>[ivy:configure] :: loading settings :: file =
>/root/hadoop-0.22.0/hadoop_compile/mapreduce/ivy/ivysettings.xml
>
>ivy-resolve-common:
>[ivy:resolve]
>[ivy:resolve] :: problems summary ::
>[ivy:resolve] :::: WARNINGS
>[ivy:resolve] module not found:
>org.apache.hadoop#hadoop-common;0.22.0-SNAPSHOT
>[ivy:resolve] ==== apache-snapshot: tried
>[ivy:resolve]
>https://repository.apache.org/content/repositories/snapshots/org/apache/ha
>doop/hadoop-common/0.22.0-SNAPSHOT/hadoop-common-0.22.0-SNAPSHOT.pom
>[ivy:resolve] -- artifact
>org.apache.hadoop#hadoop-common;0.22.0-SNAPSHOT!hadoop-common.jar:
>[ivy:resolve]
>https://repository.apache.org/content/repositories/snapshots/org/apache/ha
>doop/hadoop-common/0.22.0-SNAPSHOT/hadoop-common-0.22.0-SNAPSHOT.jar
>[ivy:resolve] ==== maven2: tried
>[ivy:resolve]
>http://repo1.maven.org/maven2/org/apache/hadoop/hadoop-common/0.22.0-SNAPS
>HOT/hadoop-common-0.22.0-SNAPSHOT.pom
>[ivy:resolve] -- artifact
>org.apache.hadoop#hadoop-common;0.22.0-SNAPSHOT!hadoop-common.jar:
>[ivy:resolve]
>http://repo1.maven.org/maven2/org/apache/hadoop/hadoop-common/0.22.0-SNAPS
>HOT/hadoop-common-0.22.0-SNAPSHOT.jar
>[ivy:resolve] module not found:
>org.apache.hadoop#hadoop-common-test;0.22.0-SNAPSHOT
>[ivy:resolve] ==== apache-snapshot: tried
>[ivy:resolve]
>https://repository.apache.org/content/repositories/snapshots/org/apache/ha
>doop/hadoop-common-test/0.22.0-SNAPSHOT/hadoop-common-test-0.22.0-SNAPSHOT
>.pom
>[ivy:resolve] -- artifact
>org.apache.hadoop#hadoop-common-test;0.22.0-SNAPSHOT!hadoop-common-test.ja
>r:
>[ivy:resolve]
>https://repository.apache.org/content/repositories/snapshots/org/apache/ha
>doop/hadoop-common-test/0.22.0-SNAPSHOT/hadoop-common-test-0.22.0-SNAPSHOT
>.jar
>[ivy:resolve] ==== maven2: tried
>[ivy:resolve]
>http://repo1.maven.org/maven2/org/apache/hadoop/hadoop-common-test/0.22.0-
>SNAPSHOT/hadoop-common-test-0.22.0-SNAPSHOT.pom
>[ivy:resolve] -- artifact
>org.apache.hadoop#hadoop-common-test;0.22.0-SNAPSHOT!hadoop-common-test.ja
>r:
>[ivy:resolve]
>http://repo1.maven.org/maven2/org/apache/hadoop/hadoop-common-test/0.22.0-
>SNAPSHOT/hadoop-common-test-0.22.0-SNAPSHOT.jar
>[ivy:resolve] module not found:
>org.apache.hadoop#hadoop-hdfs;0.22.0-SNAPSHOT
>[ivy:resolve] ==== apache-snapshot: tried
>[ivy:resolve]
>https://repository.apache.org/content/repositories/snapshots/org/apache/ha
>doop/hadoop-hdfs/0.22.0-SNAPSHOT/hadoop-hdfs-0.22.0-SNAPSHOT.pom
>[ivy:resolve] -- artifact
>org.apache.hadoop#hadoop-hdfs;0.22.0-SNAPSHOT!hadoop-hdfs.jar:
>[ivy:resolve]
>https://repository.apache.org/content/repositories/snapshots/org/apache/ha
>doop/hadoop-hdfs/0.22.0-SNAPSHOT/hadoop-hdfs-0.22.0-SNAPSHOT.jar
>[ivy:resolve] ==== maven2: tried
>[ivy:resolve]
>http://repo1.maven.org/maven2/org/apache/hadoop/hadoop-hdfs/0.22.0-SNAPSHO
>T/hadoop-hdfs-0.22.0-SNAPSHOT.pom
>[ivy:resolve] -- artifact
>org.apache.hadoop#hadoop-hdfs;0.22.0-SNAPSHOT!hadoop-hdfs.jar:
>[ivy:resolve]
>http://repo1.maven.org/maven2/org/apache/hadoop/hadoop-hdfs/0.22.0-SNAPSHO
>T/hadoop-hdfs-0.22.0-SNAPSHOT.jar
>[ivy:resolve] ::::::::::::::::::::::::::::::::::::::::::::::
>[ivy:resolve] :: UNRESOLVED DEPENDENCIES ::
>[ivy:resolve] ::::::::::::::::::::::::::::::::::::::::::::::
>[ivy:resolve] ::
>org.apache.hadoop#hadoop-common;0.22.0-SNAPSHOT: not found
>[ivy:resolve] ::
>org.apache.hadoop#hadoop-common-test;0.22.0-SNAPSHOT: not found
>[ivy:resolve] :: org.apache.hadoop#hadoop-hdfs;0.22.0-SNAPSHOT:
>not found
>[ivy:resolve] ::::::::::::::::::::::::::::::::::::::::::::::
>[ivy:resolve]
>[ivy:resolve] :: USE VERBOSE OR DEBUG MESSAGE LEVEL FOR MORE DETAILS
>
>BUILD FAILED
>/root/hadoop-0.22.0/hadoop_compile/mapreduce/build.xml:2117: impossible
>to resolve dependencies:
> resolve failed - see output for details
>
>Total time: 39 seconds
>
>I searched the internet and see someone says this is because of the ivy
>configuration and give solution like below：
>
>for hdfs part ，modify the ivy.xml as following：（---- means delete，+++
>means add）
>
>hdfs/ivy.xml
>
><dependency org="org.apache.hadoop" name="hadoop-common"
>
>----- rev="${hadoop-common.version}"
>
>+++ rev="${hadoop-common.version}-SNAPSHOT"
>
> conf="common->default"/>
>
>hdfs/src/contrib/hdfsproxy/ivy.xml
>
> <dependency org="org.apache.hadoop"
>
> name="hadoop-common"
>
>---- rev="${hadoop-common.version}"
>
>+++ rev="${hadoop-common.version}-SNAPSHOT"
>
> conf="common->default"/>
>
> <dependency org="org.apache.hadoop"
>
> name="hadoop-common-test"
>
>--- rev="${hadoop-common.version}"
>
>+++ rev="${hadoop-common.version}-SNAPSHOT"
>
> conf="common->default"/>
>
>hdfs/src/contrib/thriftfs/ivy.xml
>
> <dependency org="org.apache.hadoop"
>
> name="hadoop-common"
>
>--- rev="${hadoop-common.version}"
>
>+++ rev="${hadoop-common.version}-SNAPSHOT"
>
> conf="common->default"/>
>
>
>for mapreduce part：
>first modify all the ivy.xml in project mapreduce:
>
> rev="${hadoop-common.version}" replaced by
>rev="${hadoop-common.version}-SNAPSHOT"
>
> rev="${hadoop-hdfs.version}" replaced by
>rev="${hadoop-hdfs.version}-SNAPSHOT"
>then modify the file buil.xml
>
> <unzip
>
> ----
>src="${common.ivy.lib.dir}/hadoop-hdfs-${hadoop-hdfs.version}.jar"
>
> +++
>src="${common.ivy.lib.dir}/hadoop-hdfs-${hadoop-hdfs.version}-SNAPSHOT.jar
>" dest="${build.dir}">
>
>
>Bur after the modification ，the build still failed .So could you give me
>some suggestions about the source code build especially for the 0.22.0
>version
>
>Thanks & Best regards
>wenjing
>
>
>
________________________________
The information contained in this communication is intended solely for the use of the individual or entity to whom it is addressed and others authorized to receive it. It may contain confidential or legally privileged information. If you are not the intended recipient you are hereby notified that any disclosure, copying, distribution or taking any action in reliance on the contents of this information is strictly prohibited and may be unlawful. If you have received this communication in error, please notify us immediately by responding to this email and then delete it from your system. The firm is neither liable for the proper and complete transmission of the information contained in this communication nor for any delay in its receipt.