You are viewing a plain text version of this content. The canonical link for it is here.
Posted to mapreduce-user@hadoop.apache.org by Pankaj Misra <pa...@impetus.co.in> on 2012/08/09 12:05:39 UTC
Apache Hadoop 0.23.1 Source Build Failing
Dear All,
I am building hadoop 0.23.1 release from source with native support. I have already built/installed the following pre-requisites for native support
1. gcc-c++ 4.7.1
2. protoc 2.4.1
3. autotools chain
4. JDK 1.6.0_33
5. zlib 1.2.5-6
6. lzo 2.06-2
I have also set the following variables and exported them
export LD_LIBRARY_PATH=/usr/local/lib
Other variables are also set as given below.
export LD_LIBRARY_PATH=/usr/local/lib
export JAVA_HOME=/usr/java/jdk1.6.0_33
export ANT_HOME=/home/fedora/apache-ant-1.8.4
export MAVEN_HOME=/home/fedora/apache-maven-3.0.4
export PATH=$JAVA_HOME/bin:$ANT_HOME/bin:$MAVEN_HOME/bin:$PATH
export HADOOP_COMMON_HOME=/home/fedora/hadoop/bin/release-0.23.1/hadoop-0.23.1
export HADOOP_HDFS_HOME=$HADOOP_COMMON_HOME
export YARN_HOME=$HADOOP_COMMON_HOME
export HADOOP_MAPRED_HOME=$HADOOP_COMMON_HOME
export HADOOP_CONF_DIR=$HADOOP_COMMON_HOME/etc/hadoop
I am using the following to build the source with native support(using root user access).
mvn clean install -Pdist -Pnative -DskipTests=true
However, after building a number of sub-projects, the build fails at the nodemanager with the following error
[INFO] Compiling 129 source files to /home/fedora/hadoop/src/release-0.23.1/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/target/classes
[INFO]
[INFO] --- make-maven-plugin:1.0-beta-1:autoreconf (compile) @ hadoop-yarn-server-nodemanager ---
[INFO]
[INFO] --- make-maven-plugin:1.0-beta-1:configure (compile) @ hadoop-yarn-server-nodemanager ---
[INFO] checking for gcc... gcc
[INFO] checking whether the C compiler works... no
[INFO] configure: error: in `/home/fedora/hadoop/src/release-0.23.1/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/target/native/container-executor':
[INFO] configure: error: C compiler cannot create executables
[INFO] See `config.log' for more details
The config.log mentions the following issues.
configure:2562: checking whether the C compiler works
configure:2584: gcc -DHADOOP_CONF_DIR=/etc/hadoop -m32 conftest.c >&5
/bin/ld: cannot find crt1.o: No such file or directory
/bin/ld: cannot find crti.o: No such file or directory
/bin/ld: skipping incompatible /usr/lib/gcc/x86_64-redhat-linux/4.7.0/libgcc_s.so when searching for -lgcc_s
/bin/ld: cannot find -lgcc_s
/bin/ld: skipping incompatible /usr/lib64/libc.so when searching for -lc
/bin/ld: cannot find -lc
/bin/ld: skipping incompatible /usr/lib/gcc/x86_64-redhat-linux/4.7.0/libgcc_s.so when searching for -lgcc_s
/bin/ld: cannot find -lgcc_s
/bin/ld: cannot find crtn.o: No such file or directory
collect2: error: ld returned 1 exit status
configure:2588: $? = 1
configure:2626: result: no
configure: failed program was:
| /* confdefs.h */
| #define PACKAGE_NAME "linux-container-executor"
| #define PACKAGE_TARNAME "linux-container-executor"
| #define PACKAGE_VERSION "1.0.0"
| #define PACKAGE_STRING "linux-container-executor 1.0.0"
| #define PACKAGE_BUGREPORT "mapreduce-dev@hadoop.apache.org"
| #define PACKAGE_URL ""
| /* end confdefs.h. */
|
| int
| main ()
| {
|
| ;
| return 0;
| }
configure:2631: error: in `/home/fedora/hadoop/src/release-0.23.1/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/target/native/container-executor':
configure:2633: error: C compiler cannot create executables
Overall build summary is as given below
[INFO] Reactor Summary:
[INFO]
[INFO] Apache Hadoop Main ................................ SUCCESS [7.920s]
[INFO] Apache Hadoop Project POM ......................... SUCCESS [1.405s]
[INFO] Apache Hadoop Annotations ......................... SUCCESS [6.452s]
[INFO] Apache Hadoop Assemblies .......................... SUCCESS [0.637s]
[INFO] Apache Hadoop Project Dist POM .................... SUCCESS [3.236s]
[INFO] Apache Hadoop Auth ................................ SUCCESS [8.805s]
[INFO] Apache Hadoop Auth Examples ....................... SUCCESS [5.105s]
[INFO] Apache Hadoop Common .............................. SUCCESS [1:55.833s]
[INFO] Apache Hadoop Common Project ...................... SUCCESS [0.210s]
[INFO] Apache Hadoop HDFS ................................ SUCCESS [1:24.218s]
[INFO] Apache Hadoop HttpFS .............................. SUCCESS [20.992s]
[INFO] Apache Hadoop HDFS Project ........................ SUCCESS [0.127s]
[INFO] hadoop-yarn ....................................... SUCCESS [0.598s]
[INFO] hadoop-yarn-api ................................... SUCCESS [57.281s]
[INFO] hadoop-yarn-common ................................ SUCCESS [1:01.264s]
[INFO] hadoop-yarn-server ................................ SUCCESS [0.190s]
[INFO] hadoop-yarn-server-common ......................... SUCCESS [19.280s]
[INFO] hadoop-yarn-server-nodemanager .................... FAILURE [17.016s]
[INFO] hadoop-yarn-server-web-proxy ...................... SKIPPED
[INFO] hadoop-yarn-server-resourcemanager ................ SKIPPED
[INFO] hadoop-yarn-server-tests .......................... SKIPPED
[INFO] hadoop-mapreduce-client ........................... SKIPPED
[INFO] hadoop-mapreduce-client-core ...................... SKIPPED
[INFO] hadoop-yarn-applications .......................... SKIPPED
[INFO] hadoop-yarn-applications-distributedshell ......... SKIPPED
[INFO] hadoop-yarn-site .................................. SKIPPED
[INFO] hadoop-mapreduce-client-common .................... SKIPPED
[INFO] hadoop-mapreduce-client-shuffle ................... SKIPPED
[INFO] hadoop-mapreduce-client-app ....................... SKIPPED
[INFO] hadoop-mapreduce-client-hs ........................ SKIPPED
[INFO] hadoop-mapreduce-client-jobclient ................. SKIPPED
[INFO] Apache Hadoop MapReduce Examples .................. SKIPPED
[INFO] hadoop-mapreduce .................................. SKIPPED
[INFO] Apache Hadoop MapReduce Streaming ................. SKIPPED
[INFO] Apache Hadoop Distributed Copy .................... SKIPPED
[INFO] Apache Hadoop Archives ............................ SKIPPED
[INFO] Apache Hadoop Rumen ............................... SKIPPED
[INFO] Apache Hadoop Extras .............................. SKIPPED
[INFO] Apache Hadoop Tools Dist .......................... SKIPPED
[INFO] Apache Hadoop Tools ............................... SKIPPED
[INFO] Apache Hadoop Distribution ........................ SKIPPED
[INFO] Apache Hadoop Client .............................. SKIPPED
[INFO] Apache Hadoop Mini-Cluster ........................ SKIPPED
[INFO] ----------------------------------------------------------------------
My doubts here are that is the if C compiler was not able to create the executables, the other projects which were build before node manager should have also failed, since they are also using the auto tools chain. I am able to build the entire source code if I build without native support, but I do not want to skip the native support build and I am trying to build the entire source code with native libraries support.
My current OS is Fedora 17 64-bit and I have also got the same issue on CentOS 6.2 64-bit. I dont know if I am missing something for successful build with native libraries, and would request for the community's help for the same.
Thanks and Regards
Pankaj Misra
________________________________
Impetus webinar: Designing a Test Automation Framework for Interoperable Systems; July 25 (10:00am PT). http://lf1.me/0E/
Follow us on www.twitter.com/impetustech
NOTE: This message may contain information that is confidential, proprietary, privileged or otherwise protected by law. The message is intended solely for the named addressee. If received in error, please destroy and notify the sender. Any use of this email is prohibited when received in error. Impetus does not represent, warrant and/or guarantee, that the integrity of this communication has been maintained nor that the communication is free of errors, virus, interception or interference.
Re: Apache Hadoop 0.23.1 Source Build Failing
Posted by Ioan Eugen Stan <st...@gmail.com>.
Hello Pankaj,
Sorry, I pressed send too fast. The linker can't find some libraries.
Check to see if you have them installed (locate, find, rpm). If not
install them and try again.
If you have them installed and ld can't find them, add the paths to
the proper environment variable (I think LD_LIBRARY_PATH). Check the
ld man page ( $ man ld) for details.
e.g.
$ export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/path/to/lib
http://linux.about.com/library/cmd/blcmdl1_ld.htm
I hope this helps,
On Thu, Aug 9, 2012 at 1:36 PM, Ioan Eugen Stan <st...@gmail.com> wrote:
> It seems that /bin/ld dos not exist so the compiler cannot perform
> linking. Looking at the Fedora docs it seems that ld is located in
> /usr/bin/ld so you may have to create a symlink to it:
>
> $ ln -s /usr/bin/ld /bin/ld
>
> First check that you have ld installed with: $ which ld
>
> The scripts should also use `which ld` to find the proper path to ld.
> So maybe you could raise an issue on JIRA with this.
>
> http://docs.fedoraproject.org/en-US/Fedora/13/html/Release_Notes/sect-Release_Notes-The_GCC_Compiler_Collection.html
>
> Cheers,
>
> On Thu, Aug 9, 2012 at 1:05 PM, Pankaj Misra <pa...@impetus.co.in> wrote:
>> Dear All,
>>
>> I am building hadoop 0.23.1 release from source with native support. I have already built/installed the following pre-requisites for native support
>> 1. gcc-c++ 4.7.1
>> 2. protoc 2.4.1
>> 3. autotools chain
>> 4. JDK 1.6.0_33
>> 5. zlib 1.2.5-6
>> 6. lzo 2.06-2
>>
>> I have also set the following variables and exported them
>> export LD_LIBRARY_PATH=/usr/local/lib
>>
>> Other variables are also set as given below.
>> export LD_LIBRARY_PATH=/usr/local/lib
>> export JAVA_HOME=/usr/java/jdk1.6.0_33
>> export ANT_HOME=/home/fedora/apache-ant-1.8.4
>> export MAVEN_HOME=/home/fedora/apache-maven-3.0.4
>> export PATH=$JAVA_HOME/bin:$ANT_HOME/bin:$MAVEN_HOME/bin:$PATH
>> export HADOOP_COMMON_HOME=/home/fedora/hadoop/bin/release-0.23.1/hadoop-0.23.1
>> export HADOOP_HDFS_HOME=$HADOOP_COMMON_HOME
>> export YARN_HOME=$HADOOP_COMMON_HOME
>> export HADOOP_MAPRED_HOME=$HADOOP_COMMON_HOME
>> export HADOOP_CONF_DIR=$HADOOP_COMMON_HOME/etc/hadoop
>>
>>
>> I am using the following to build the source with native support(using root user access).
>> mvn clean install -Pdist -Pnative -DskipTests=true
>>
>> However, after building a number of sub-projects, the build fails at the nodemanager with the following error
>> [INFO] Compiling 129 source files to /home/fedora/hadoop/src/release-0.23.1/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/target/classes
>> [INFO]
>> [INFO] --- make-maven-plugin:1.0-beta-1:autoreconf (compile) @ hadoop-yarn-server-nodemanager ---
>> [INFO]
>> [INFO] --- make-maven-plugin:1.0-beta-1:configure (compile) @ hadoop-yarn-server-nodemanager ---
>> [INFO] checking for gcc... gcc
>> [INFO] checking whether the C compiler works... no
>> [INFO] configure: error: in `/home/fedora/hadoop/src/release-0.23.1/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/target/native/container-executor':
>> [INFO] configure: error: C compiler cannot create executables
>> [INFO] See `config.log' for more details
>>
>>
>> The config.log mentions the following issues.
>>
>> configure:2562: checking whether the C compiler works
>> configure:2584: gcc -DHADOOP_CONF_DIR=/etc/hadoop -m32 conftest.c >&5
>> /bin/ld: cannot find crt1.o: No such file or directory
>> /bin/ld: cannot find crti.o: No such file or directory
>> /bin/ld: skipping incompatible /usr/lib/gcc/x86_64-redhat-linux/4.7.0/libgcc_s.so when searching for -lgcc_s
>> /bin/ld: cannot find -lgcc_s
>> /bin/ld: skipping incompatible /usr/lib64/libc.so when searching for -lc
>> /bin/ld: cannot find -lc
>> /bin/ld: skipping incompatible /usr/lib/gcc/x86_64-redhat-linux/4.7.0/libgcc_s.so when searching for -lgcc_s
>> /bin/ld: cannot find -lgcc_s
>> /bin/ld: cannot find crtn.o: No such file or directory
>> collect2: error: ld returned 1 exit status
>> configure:2588: $? = 1
>> configure:2626: result: no
>> configure: failed program was:
>> | /* confdefs.h */
>> | #define PACKAGE_NAME "linux-container-executor"
>> | #define PACKAGE_TARNAME "linux-container-executor"
>> | #define PACKAGE_VERSION "1.0.0"
>> | #define PACKAGE_STRING "linux-container-executor 1.0.0"
>> | #define PACKAGE_BUGREPORT "mapreduce-dev@hadoop.apache.org"
>> | #define PACKAGE_URL ""
>> | /* end confdefs.h. */
>> |
>> | int
>> | main ()
>> | {
>> |
>> | ;
>> | return 0;
>> | }
>> configure:2631: error: in `/home/fedora/hadoop/src/release-0.23.1/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/target/native/container-executor':
>> configure:2633: error: C compiler cannot create executables
>>
>> Overall build summary is as given below
>> [INFO] Reactor Summary:
>> [INFO]
>> [INFO] Apache Hadoop Main ................................ SUCCESS [7.920s]
>> [INFO] Apache Hadoop Project POM ......................... SUCCESS [1.405s]
>> [INFO] Apache Hadoop Annotations ......................... SUCCESS [6.452s]
>> [INFO] Apache Hadoop Assemblies .......................... SUCCESS [0.637s]
>> [INFO] Apache Hadoop Project Dist POM .................... SUCCESS [3.236s]
>> [INFO] Apache Hadoop Auth ................................ SUCCESS [8.805s]
>> [INFO] Apache Hadoop Auth Examples ....................... SUCCESS [5.105s]
>> [INFO] Apache Hadoop Common .............................. SUCCESS [1:55.833s]
>> [INFO] Apache Hadoop Common Project ...................... SUCCESS [0.210s]
>> [INFO] Apache Hadoop HDFS ................................ SUCCESS [1:24.218s]
>> [INFO] Apache Hadoop HttpFS .............................. SUCCESS [20.992s]
>> [INFO] Apache Hadoop HDFS Project ........................ SUCCESS [0.127s]
>> [INFO] hadoop-yarn ....................................... SUCCESS [0.598s]
>> [INFO] hadoop-yarn-api ................................... SUCCESS [57.281s]
>> [INFO] hadoop-yarn-common ................................ SUCCESS [1:01.264s]
>> [INFO] hadoop-yarn-server ................................ SUCCESS [0.190s]
>> [INFO] hadoop-yarn-server-common ......................... SUCCESS [19.280s]
>> [INFO] hadoop-yarn-server-nodemanager .................... FAILURE [17.016s]
>> [INFO] hadoop-yarn-server-web-proxy ...................... SKIPPED
>> [INFO] hadoop-yarn-server-resourcemanager ................ SKIPPED
>> [INFO] hadoop-yarn-server-tests .......................... SKIPPED
>> [INFO] hadoop-mapreduce-client ........................... SKIPPED
>> [INFO] hadoop-mapreduce-client-core ...................... SKIPPED
>> [INFO] hadoop-yarn-applications .......................... SKIPPED
>> [INFO] hadoop-yarn-applications-distributedshell ......... SKIPPED
>> [INFO] hadoop-yarn-site .................................. SKIPPED
>> [INFO] hadoop-mapreduce-client-common .................... SKIPPED
>> [INFO] hadoop-mapreduce-client-shuffle ................... SKIPPED
>> [INFO] hadoop-mapreduce-client-app ....................... SKIPPED
>> [INFO] hadoop-mapreduce-client-hs ........................ SKIPPED
>> [INFO] hadoop-mapreduce-client-jobclient ................. SKIPPED
>> [INFO] Apache Hadoop MapReduce Examples .................. SKIPPED
>> [INFO] hadoop-mapreduce .................................. SKIPPED
>> [INFO] Apache Hadoop MapReduce Streaming ................. SKIPPED
>> [INFO] Apache Hadoop Distributed Copy .................... SKIPPED
>> [INFO] Apache Hadoop Archives ............................ SKIPPED
>> [INFO] Apache Hadoop Rumen ............................... SKIPPED
>> [INFO] Apache Hadoop Extras .............................. SKIPPED
>> [INFO] Apache Hadoop Tools Dist .......................... SKIPPED
>> [INFO] Apache Hadoop Tools ............................... SKIPPED
>> [INFO] Apache Hadoop Distribution ........................ SKIPPED
>> [INFO] Apache Hadoop Client .............................. SKIPPED
>> [INFO] Apache Hadoop Mini-Cluster ........................ SKIPPED
>> [INFO] ----------------------------------------------------------------------
>>
>> My doubts here are that is the if C compiler was not able to create the executables, the other projects which were build before node manager should have also failed, since they are also using the auto tools chain. I am able to build the entire source code if I build without native support, but I do not want to skip the native support build and I am trying to build the entire source code with native libraries support.
>>
>> My current OS is Fedora 17 64-bit and I have also got the same issue on CentOS 6.2 64-bit. I dont know if I am missing something for successful build with native libraries, and would request for the community's help for the same.
>>
>>
>>
>> Thanks and Regards
>> Pankaj Misra
>>
>> ________________________________
>>
>> Impetus webinar: Designing a Test Automation Framework for Interoperable Systems; July 25 (10:00am PT). http://lf1.me/0E/
>>
>> Follow us on www.twitter.com/impetustech
>>
>>
>> NOTE: This message may contain information that is confidential, proprietary, privileged or otherwise protected by law. The message is intended solely for the named addressee. If received in error, please destroy and notify the sender. Any use of this email is prohibited when received in error. Impetus does not represent, warrant and/or guarantee, that the integrity of this communication has been maintained nor that the communication is free of errors, virus, interception or interference.
>
>
>
> --
> Ioan Eugen Stan / CTO / http://axemblr.com
--
Ioan Eugen Stan / CTO / http://axemblr.com
Re: Apache Hadoop 0.23.1 Source Build Failing
Posted by Steve Loughran <st...@hortonworks.com>.
On 9 August 2012 03:46, Pankaj Misra <pa...@impetus.co.in> wrote:
> Thanks Ioan for the help and sharing the link, appreciate it.
>
> The symlink as specified below already exists, and the the response of "$
> which ld" is
>
> [root@fedora-0 container-executor]# which ld
> /bin/ld
>
> Yes, I will surely raise a JIRA for this issue if it does not get
> resolved, and once I am sure that I am not missing out anything.
>
>
'ld' is there -it's returning an error about linkage problems
configure:2584: gcc -DHADOOP_CONF_DIR=/etc/hadoop -m32 conftest.c >&5
/bin/ld: cannot find crt1.o: No such file or directory
/bin/ld: cannot find crti.o: No such file or directory
/bin/ld: skipping incompatible
/usr/lib/gcc/x86_64-redhat-linux/4.7.0/libgcc_s.so
when searching for -lgcc_s
/bin/ld: cannot find -lgcc_s
/bin/ld: skipping incompatible /usr/lib64/libc.so when searching for -lc
/bin/ld: cannot find -lc
/bin/ld: skipping incompatible
/usr/lib/gcc/x86_64-redhat-linux/4.7.0/libgcc_s.so
when searching for -lgcc_s
/bin/ld: cannot find -lgcc_s
/bin/ld: cannot find crtn.o: No such file or directory
"-m32" means 32 bit binaries -unless you can turn that bit of the build
off, you'll need the 32 bit libraries on your system as well as 64 bit ones
Re: Apache Hadoop 0.23.1 Source Build Failing
Posted by Steve Loughran <st...@hortonworks.com>.
On 9 August 2012 03:46, Pankaj Misra <pa...@impetus.co.in> wrote:
> Thanks Ioan for the help and sharing the link, appreciate it.
>
> The symlink as specified below already exists, and the the response of "$
> which ld" is
>
> [root@fedora-0 container-executor]# which ld
> /bin/ld
>
> Yes, I will surely raise a JIRA for this issue if it does not get
> resolved, and once I am sure that I am not missing out anything.
>
>
'ld' is there -it's returning an error about linkage problems
configure:2584: gcc -DHADOOP_CONF_DIR=/etc/hadoop -m32 conftest.c >&5
/bin/ld: cannot find crt1.o: No such file or directory
/bin/ld: cannot find crti.o: No such file or directory
/bin/ld: skipping incompatible
/usr/lib/gcc/x86_64-redhat-linux/4.7.0/libgcc_s.so
when searching for -lgcc_s
/bin/ld: cannot find -lgcc_s
/bin/ld: skipping incompatible /usr/lib64/libc.so when searching for -lc
/bin/ld: cannot find -lc
/bin/ld: skipping incompatible
/usr/lib/gcc/x86_64-redhat-linux/4.7.0/libgcc_s.so
when searching for -lgcc_s
/bin/ld: cannot find -lgcc_s
/bin/ld: cannot find crtn.o: No such file or directory
"-m32" means 32 bit binaries -unless you can turn that bit of the build
off, you'll need the 32 bit libraries on your system as well as 64 bit ones
Re: Apache Hadoop 0.23.1 Source Build Failing
Posted by Steve Loughran <st...@hortonworks.com>.
On 9 August 2012 03:46, Pankaj Misra <pa...@impetus.co.in> wrote:
> Thanks Ioan for the help and sharing the link, appreciate it.
>
> The symlink as specified below already exists, and the the response of "$
> which ld" is
>
> [root@fedora-0 container-executor]# which ld
> /bin/ld
>
> Yes, I will surely raise a JIRA for this issue if it does not get
> resolved, and once I am sure that I am not missing out anything.
>
>
'ld' is there -it's returning an error about linkage problems
configure:2584: gcc -DHADOOP_CONF_DIR=/etc/hadoop -m32 conftest.c >&5
/bin/ld: cannot find crt1.o: No such file or directory
/bin/ld: cannot find crti.o: No such file or directory
/bin/ld: skipping incompatible
/usr/lib/gcc/x86_64-redhat-linux/4.7.0/libgcc_s.so
when searching for -lgcc_s
/bin/ld: cannot find -lgcc_s
/bin/ld: skipping incompatible /usr/lib64/libc.so when searching for -lc
/bin/ld: cannot find -lc
/bin/ld: skipping incompatible
/usr/lib/gcc/x86_64-redhat-linux/4.7.0/libgcc_s.so
when searching for -lgcc_s
/bin/ld: cannot find -lgcc_s
/bin/ld: cannot find crtn.o: No such file or directory
"-m32" means 32 bit binaries -unless you can turn that bit of the build
off, you'll need the 32 bit libraries on your system as well as 64 bit ones
unsubscribe
Posted by Andrei Krichevskiy <an...@googlemail.com>.
Sent from my iPhone
On Aug 9, 2012, at 12:46, Pankaj Misra <pa...@impetus.co.in>
wrote:
> Thanks Ioan for the help and sharing the link, appreciate it.
>
> The symlink as specified below already exists, and the the response
> of "$ which ld" is
>
> [root@fedora-0 container-executor]# which ld
> /bin/ld
>
> Yes, I will surely raise a JIRA for this issue if it does not get
> resolved, and once I am sure that I am not missing out anything.
>
> Thanks and Regards
> Pankaj Misra
>
>
> ________________________________________
> From: Ioan Eugen Stan [stan.ieugen@gmail.com]
> Sent: Thursday, August 09, 2012 4:06 PM
> To: user@hadoop.apache.org
> Subject: Re: Apache Hadoop 0.23.1 Source Build Failing
>
> It seems that /bin/ld dos not exist so the compiler cannot perform
> linking. Looking at the Fedora docs it seems that ld is located in
> /usr/bin/ld so you may have to create a symlink to it:
>
> $ ln -s /usr/bin/ld /bin/ld
>
> First check that you have ld installed with: $ which ld
>
> The scripts should also use `which ld` to find the proper path to ld.
> So maybe you could raise an issue on JIRA with this.
>
> http://docs.fedoraproject.org/en-US/Fedora/13/html/Release_Notes/sect-Release_Notes-The_GCC_Compiler_Collection.html
>
> Cheers,
>
> On Thu, Aug 9, 2012 at 1:05 PM, Pankaj Misra <pankaj.misra@impetus.co.in
> > wrote:
>> Dear All,
>>
>> I am building hadoop 0.23.1 release from source with native
>> support. I have already built/installed the following pre-
>> requisites for native support
>> 1. gcc-c++ 4.7.1
>> 2. protoc 2.4.1
>> 3. autotools chain
>> 4. JDK 1.6.0_33
>> 5. zlib 1.2.5-6
>> 6. lzo 2.06-2
>>
>> I have also set the following variables and exported them
>> export LD_LIBRARY_PATH=/usr/local/lib
>>
>> Other variables are also set as given below.
>> export LD_LIBRARY_PATH=/usr/local/lib
>> export JAVA_HOME=/usr/java/jdk1.6.0_33
>> export ANT_HOME=/home/fedora/apache-ant-1.8.4
>> export MAVEN_HOME=/home/fedora/apache-maven-3.0.4
>> export PATH=$JAVA_HOME/bin:$ANT_HOME/bin:$MAVEN_HOME/bin:$PATH
>> export HADOOP_COMMON_HOME=/home/fedora/hadoop/bin/release-0.23.1/
>> hadoop-0.23.1
>> export HADOOP_HDFS_HOME=$HADOOP_COMMON_HOME
>> export YARN_HOME=$HADOOP_COMMON_HOME
>> export HADOOP_MAPRED_HOME=$HADOOP_COMMON_HOME
>> export HADOOP_CONF_DIR=$HADOOP_COMMON_HOME/etc/hadoop
>>
>>
>> I am using the following to build the source with native support
>> (using root user access).
>> mvn clean install -Pdist -Pnative -DskipTests=true
>>
>> However, after building a number of sub-projects, the build fails
>> at the nodemanager with the following error
>> [INFO] Compiling 129 source files to /home/fedora/hadoop/src/
>> release-0.23.1/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-
>> server/hadoop-yarn-server-nodemanager/target/classes
>> [INFO]
>> [INFO] --- make-maven-plugin:1.0-beta-1:autoreconf (compile) @
>> hadoop-yarn-server-nodemanager ---
>> [INFO]
>> [INFO] --- make-maven-plugin:1.0-beta-1:configure (compile) @
>> hadoop-yarn-server-nodemanager ---
>> [INFO] checking for gcc... gcc
>> [INFO] checking whether the C compiler works... no
>> [INFO] configure: error: in `/home/fedora/hadoop/src/release-0.23.1/
>> hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-
>> server-nodemanager/target/native/container-executor':
>> [INFO] configure: error: C compiler cannot create executables
>> [INFO] See `config.log' for more details
>>
>>
>> The config.log mentions the following issues.
>>
>> configure:2562: checking whether the C compiler works
>> configure:2584: gcc -DHADOOP_CONF_DIR=/etc/hadoop -m32
>> conftest.c >&5
>> /bin/ld: cannot find crt1.o: No such file or directory
>> /bin/ld: cannot find crti.o: No such file or directory
>> /bin/ld: skipping incompatible /usr/lib/gcc/x86_64-redhat-linux/
>> 4.7.0/libgcc_s.so when searching for -lgcc_s
>> /bin/ld: cannot find -lgcc_s
>> /bin/ld: skipping incompatible /usr/lib64/libc.so when searching
>> for -lc
>> /bin/ld: cannot find -lc
>> /bin/ld: skipping incompatible /usr/lib/gcc/x86_64-redhat-linux/
>> 4.7.0/libgcc_s.so when searching for -lgcc_s
>> /bin/ld: cannot find -lgcc_s
>> /bin/ld: cannot find crtn.o: No such file or directory
>> collect2: error: ld returned 1 exit status
>> configure:2588: $? = 1
>> configure:2626: result: no
>> configure: failed program was:
>> | /* confdefs.h */
>> | #define PACKAGE_NAME "linux-container-executor"
>> | #define PACKAGE_TARNAME "linux-container-executor"
>> | #define PACKAGE_VERSION "1.0.0"
>> | #define PACKAGE_STRING "linux-container-executor 1.0.0"
>> | #define PACKAGE_BUGREPORT "mapreduce-dev@hadoop.apache.org"
>> | #define PACKAGE_URL ""
>> | /* end confdefs.h. */
>> |
>> | int
>> | main ()
>> | {
>> |
>> | ;
>> | return 0;
>> | }
>> configure:2631: error: in `/home/fedora/hadoop/src/release-0.23.1/
>> hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-
>> server-nodemanager/target/native/container-executor':
>> configure:2633: error: C compiler cannot create executables
>>
>> Overall build summary is as given below
>> [INFO] Reactor Summary:
>> [INFO]
>> [INFO] Apache Hadoop Main ................................ SUCCESS
>> [7.920s]
>> [INFO] Apache Hadoop Project POM ......................... SUCCESS
>> [1.405s]
>> [INFO] Apache Hadoop Annotations ......................... SUCCESS
>> [6.452s]
>> [INFO] Apache Hadoop Assemblies .......................... SUCCESS
>> [0.637s]
>> [INFO] Apache Hadoop Project Dist POM .................... SUCCESS
>> [3.236s]
>> [INFO] Apache Hadoop Auth ................................ SUCCESS
>> [8.805s]
>> [INFO] Apache Hadoop Auth Examples ....................... SUCCESS
>> [5.105s]
>> [INFO] Apache Hadoop Common .............................. SUCCESS
>> [1:55.833s]
>> [INFO] Apache Hadoop Common Project ...................... SUCCESS
>> [0.210s]
>> [INFO] Apache Hadoop HDFS ................................ SUCCESS
>> [1:24.218s]
>> [INFO] Apache Hadoop HttpFS .............................. SUCCESS
>> [20.992s]
>> [INFO] Apache Hadoop HDFS Project ........................ SUCCESS
>> [0.127s]
>> [INFO] hadoop-yarn ....................................... SUCCESS
>> [0.598s]
>> [INFO] hadoop-yarn-api ................................... SUCCESS
>> [57.281s]
>> [INFO] hadoop-yarn-common ................................ SUCCESS
>> [1:01.264s]
>> [INFO] hadoop-yarn-server ................................ SUCCESS
>> [0.190s]
>> [INFO] hadoop-yarn-server-common ......................... SUCCESS
>> [19.280s]
>> [INFO] hadoop-yarn-server-nodemanager .................... FAILURE
>> [17.016s]
>> [INFO] hadoop-yarn-server-web-proxy ...................... SKIPPED
>> [INFO] hadoop-yarn-server-resourcemanager ................ SKIPPED
>> [INFO] hadoop-yarn-server-tests .......................... SKIPPED
>> [INFO] hadoop-mapreduce-client ........................... SKIPPED
>> [INFO] hadoop-mapreduce-client-core ...................... SKIPPED
>> [INFO] hadoop-yarn-applications .......................... SKIPPED
>> [INFO] hadoop-yarn-applications-distributedshell ......... SKIPPED
>> [INFO] hadoop-yarn-site .................................. SKIPPED
>> [INFO] hadoop-mapreduce-client-common .................... SKIPPED
>> [INFO] hadoop-mapreduce-client-shuffle ................... SKIPPED
>> [INFO] hadoop-mapreduce-client-app ....................... SKIPPED
>> [INFO] hadoop-mapreduce-client-hs ........................ SKIPPED
>> [INFO] hadoop-mapreduce-client-jobclient ................. SKIPPED
>> [INFO] Apache Hadoop MapReduce Examples .................. SKIPPED
>> [INFO] hadoop-mapreduce .................................. SKIPPED
>> [INFO] Apache Hadoop MapReduce Streaming ................. SKIPPED
>> [INFO] Apache Hadoop Distributed Copy .................... SKIPPED
>> [INFO] Apache Hadoop Archives ............................ SKIPPED
>> [INFO] Apache Hadoop Rumen ............................... SKIPPED
>> [INFO] Apache Hadoop Extras .............................. SKIPPED
>> [INFO] Apache Hadoop Tools Dist .......................... SKIPPED
>> [INFO] Apache Hadoop Tools ............................... SKIPPED
>> [INFO] Apache Hadoop Distribution ........................ SKIPPED
>> [INFO] Apache Hadoop Client .............................. SKIPPED
>> [INFO] Apache Hadoop Mini-Cluster ........................ SKIPPED
>> [INFO]
>> ---
>> -------------------------------------------------------------------
>>
>> My doubts here are that is the if C compiler was not able to create
>> the executables, the other projects which were build before node
>> manager should have also failed, since they are also using the auto
>> tools chain. I am able to build the entire source code if I build
>> without native support, but I do not want to skip the native
>> support build and I am trying to build the entire source code with
>> native libraries support.
>>
>> My current OS is Fedora 17 64-bit and I have also got the same
>> issue on CentOS 6.2 64-bit. I dont know if I am missing something
>> for successful build with native libraries, and would request for
>> the community's help for the same.
>>
>>
>>
>> Thanks and Regards
>> Pankaj Misra
>>
>> ________________________________
>>
>> Impetus webinar: Designing a Test Automation Framework for
>> Interoperable Systems; July 25 (10:00am PT). http://lf1.me/0E/
>>
>> Follow us on www.twitter.com/impetustech
>>
>>
>> NOTE: This message may contain information that is confidential,
>> proprietary, privileged or otherwise protected by law. The message
>> is intended solely for the named addressee. If received in error,
>> please destroy and notify the sender. Any use of this email is
>> prohibited when received in error. Impetus does not represent,
>> warrant and/or guarantee, that the integrity of this communication
>> has been maintained nor that the communication is free of errors,
>> virus, interception or interference.
>
>
>
> --
> Ioan Eugen Stan / CTO / http://axemblr.com
>
> ________________________________
>
> Impetus webinar: Designing a Test Automation Framework for
> Interoperable Systems; July 25 (10:00am PT). http://lf1.me/0E/
>
> Follow us on www.twitter.com/impetustech
>
>
> NOTE: This message may contain information that is confidential,
> proprietary, privileged or otherwise protected by law. The message
> is intended solely for the named addressee. If received in error,
> please destroy and notify the sender. Any use of this email is
> prohibited when received in error. Impetus does not represent,
> warrant and/or guarantee, that the integrity of this communication
> has been maintained nor that the communication is free of errors,
> virus, interception or interference.
Re: Apache Hadoop 0.23.1 Source Build Failing
Posted by Steve Loughran <st...@hortonworks.com>.
On 9 August 2012 03:46, Pankaj Misra <pa...@impetus.co.in> wrote:
> Thanks Ioan for the help and sharing the link, appreciate it.
>
> The symlink as specified below already exists, and the the response of "$
> which ld" is
>
> [root@fedora-0 container-executor]# which ld
> /bin/ld
>
> Yes, I will surely raise a JIRA for this issue if it does not get
> resolved, and once I am sure that I am not missing out anything.
>
>
'ld' is there -it's returning an error about linkage problems
configure:2584: gcc -DHADOOP_CONF_DIR=/etc/hadoop -m32 conftest.c >&5
/bin/ld: cannot find crt1.o: No such file or directory
/bin/ld: cannot find crti.o: No such file or directory
/bin/ld: skipping incompatible
/usr/lib/gcc/x86_64-redhat-linux/4.7.0/libgcc_s.so
when searching for -lgcc_s
/bin/ld: cannot find -lgcc_s
/bin/ld: skipping incompatible /usr/lib64/libc.so when searching for -lc
/bin/ld: cannot find -lc
/bin/ld: skipping incompatible
/usr/lib/gcc/x86_64-redhat-linux/4.7.0/libgcc_s.so
when searching for -lgcc_s
/bin/ld: cannot find -lgcc_s
/bin/ld: cannot find crtn.o: No such file or directory
"-m32" means 32 bit binaries -unless you can turn that bit of the build
off, you'll need the 32 bit libraries on your system as well as 64 bit ones
unsubscribe
Posted by Andrei Krichevskiy <an...@googlemail.com>.
Sent from my iPhone
On Aug 9, 2012, at 12:46, Pankaj Misra <pa...@impetus.co.in>
wrote:
> Thanks Ioan for the help and sharing the link, appreciate it.
>
> The symlink as specified below already exists, and the the response
> of "$ which ld" is
>
> [root@fedora-0 container-executor]# which ld
> /bin/ld
>
> Yes, I will surely raise a JIRA for this issue if it does not get
> resolved, and once I am sure that I am not missing out anything.
>
> Thanks and Regards
> Pankaj Misra
>
>
> ________________________________________
> From: Ioan Eugen Stan [stan.ieugen@gmail.com]
> Sent: Thursday, August 09, 2012 4:06 PM
> To: user@hadoop.apache.org
> Subject: Re: Apache Hadoop 0.23.1 Source Build Failing
>
> It seems that /bin/ld dos not exist so the compiler cannot perform
> linking. Looking at the Fedora docs it seems that ld is located in
> /usr/bin/ld so you may have to create a symlink to it:
>
> $ ln -s /usr/bin/ld /bin/ld
>
> First check that you have ld installed with: $ which ld
>
> The scripts should also use `which ld` to find the proper path to ld.
> So maybe you could raise an issue on JIRA with this.
>
> http://docs.fedoraproject.org/en-US/Fedora/13/html/Release_Notes/sect-Release_Notes-The_GCC_Compiler_Collection.html
>
> Cheers,
>
> On Thu, Aug 9, 2012 at 1:05 PM, Pankaj Misra <pankaj.misra@impetus.co.in
> > wrote:
>> Dear All,
>>
>> I am building hadoop 0.23.1 release from source with native
>> support. I have already built/installed the following pre-
>> requisites for native support
>> 1. gcc-c++ 4.7.1
>> 2. protoc 2.4.1
>> 3. autotools chain
>> 4. JDK 1.6.0_33
>> 5. zlib 1.2.5-6
>> 6. lzo 2.06-2
>>
>> I have also set the following variables and exported them
>> export LD_LIBRARY_PATH=/usr/local/lib
>>
>> Other variables are also set as given below.
>> export LD_LIBRARY_PATH=/usr/local/lib
>> export JAVA_HOME=/usr/java/jdk1.6.0_33
>> export ANT_HOME=/home/fedora/apache-ant-1.8.4
>> export MAVEN_HOME=/home/fedora/apache-maven-3.0.4
>> export PATH=$JAVA_HOME/bin:$ANT_HOME/bin:$MAVEN_HOME/bin:$PATH
>> export HADOOP_COMMON_HOME=/home/fedora/hadoop/bin/release-0.23.1/
>> hadoop-0.23.1
>> export HADOOP_HDFS_HOME=$HADOOP_COMMON_HOME
>> export YARN_HOME=$HADOOP_COMMON_HOME
>> export HADOOP_MAPRED_HOME=$HADOOP_COMMON_HOME
>> export HADOOP_CONF_DIR=$HADOOP_COMMON_HOME/etc/hadoop
>>
>>
>> I am using the following to build the source with native support
>> (using root user access).
>> mvn clean install -Pdist -Pnative -DskipTests=true
>>
>> However, after building a number of sub-projects, the build fails
>> at the nodemanager with the following error
>> [INFO] Compiling 129 source files to /home/fedora/hadoop/src/
>> release-0.23.1/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-
>> server/hadoop-yarn-server-nodemanager/target/classes
>> [INFO]
>> [INFO] --- make-maven-plugin:1.0-beta-1:autoreconf (compile) @
>> hadoop-yarn-server-nodemanager ---
>> [INFO]
>> [INFO] --- make-maven-plugin:1.0-beta-1:configure (compile) @
>> hadoop-yarn-server-nodemanager ---
>> [INFO] checking for gcc... gcc
>> [INFO] checking whether the C compiler works... no
>> [INFO] configure: error: in `/home/fedora/hadoop/src/release-0.23.1/
>> hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-
>> server-nodemanager/target/native/container-executor':
>> [INFO] configure: error: C compiler cannot create executables
>> [INFO] See `config.log' for more details
>>
>>
>> The config.log mentions the following issues.
>>
>> configure:2562: checking whether the C compiler works
>> configure:2584: gcc -DHADOOP_CONF_DIR=/etc/hadoop -m32
>> conftest.c >&5
>> /bin/ld: cannot find crt1.o: No such file or directory
>> /bin/ld: cannot find crti.o: No such file or directory
>> /bin/ld: skipping incompatible /usr/lib/gcc/x86_64-redhat-linux/
>> 4.7.0/libgcc_s.so when searching for -lgcc_s
>> /bin/ld: cannot find -lgcc_s
>> /bin/ld: skipping incompatible /usr/lib64/libc.so when searching
>> for -lc
>> /bin/ld: cannot find -lc
>> /bin/ld: skipping incompatible /usr/lib/gcc/x86_64-redhat-linux/
>> 4.7.0/libgcc_s.so when searching for -lgcc_s
>> /bin/ld: cannot find -lgcc_s
>> /bin/ld: cannot find crtn.o: No such file or directory
>> collect2: error: ld returned 1 exit status
>> configure:2588: $? = 1
>> configure:2626: result: no
>> configure: failed program was:
>> | /* confdefs.h */
>> | #define PACKAGE_NAME "linux-container-executor"
>> | #define PACKAGE_TARNAME "linux-container-executor"
>> | #define PACKAGE_VERSION "1.0.0"
>> | #define PACKAGE_STRING "linux-container-executor 1.0.0"
>> | #define PACKAGE_BUGREPORT "mapreduce-dev@hadoop.apache.org"
>> | #define PACKAGE_URL ""
>> | /* end confdefs.h. */
>> |
>> | int
>> | main ()
>> | {
>> |
>> | ;
>> | return 0;
>> | }
>> configure:2631: error: in `/home/fedora/hadoop/src/release-0.23.1/
>> hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-
>> server-nodemanager/target/native/container-executor':
>> configure:2633: error: C compiler cannot create executables
>>
>> Overall build summary is as given below
>> [INFO] Reactor Summary:
>> [INFO]
>> [INFO] Apache Hadoop Main ................................ SUCCESS
>> [7.920s]
>> [INFO] Apache Hadoop Project POM ......................... SUCCESS
>> [1.405s]
>> [INFO] Apache Hadoop Annotations ......................... SUCCESS
>> [6.452s]
>> [INFO] Apache Hadoop Assemblies .......................... SUCCESS
>> [0.637s]
>> [INFO] Apache Hadoop Project Dist POM .................... SUCCESS
>> [3.236s]
>> [INFO] Apache Hadoop Auth ................................ SUCCESS
>> [8.805s]
>> [INFO] Apache Hadoop Auth Examples ....................... SUCCESS
>> [5.105s]
>> [INFO] Apache Hadoop Common .............................. SUCCESS
>> [1:55.833s]
>> [INFO] Apache Hadoop Common Project ...................... SUCCESS
>> [0.210s]
>> [INFO] Apache Hadoop HDFS ................................ SUCCESS
>> [1:24.218s]
>> [INFO] Apache Hadoop HttpFS .............................. SUCCESS
>> [20.992s]
>> [INFO] Apache Hadoop HDFS Project ........................ SUCCESS
>> [0.127s]
>> [INFO] hadoop-yarn ....................................... SUCCESS
>> [0.598s]
>> [INFO] hadoop-yarn-api ................................... SUCCESS
>> [57.281s]
>> [INFO] hadoop-yarn-common ................................ SUCCESS
>> [1:01.264s]
>> [INFO] hadoop-yarn-server ................................ SUCCESS
>> [0.190s]
>> [INFO] hadoop-yarn-server-common ......................... SUCCESS
>> [19.280s]
>> [INFO] hadoop-yarn-server-nodemanager .................... FAILURE
>> [17.016s]
>> [INFO] hadoop-yarn-server-web-proxy ...................... SKIPPED
>> [INFO] hadoop-yarn-server-resourcemanager ................ SKIPPED
>> [INFO] hadoop-yarn-server-tests .......................... SKIPPED
>> [INFO] hadoop-mapreduce-client ........................... SKIPPED
>> [INFO] hadoop-mapreduce-client-core ...................... SKIPPED
>> [INFO] hadoop-yarn-applications .......................... SKIPPED
>> [INFO] hadoop-yarn-applications-distributedshell ......... SKIPPED
>> [INFO] hadoop-yarn-site .................................. SKIPPED
>> [INFO] hadoop-mapreduce-client-common .................... SKIPPED
>> [INFO] hadoop-mapreduce-client-shuffle ................... SKIPPED
>> [INFO] hadoop-mapreduce-client-app ....................... SKIPPED
>> [INFO] hadoop-mapreduce-client-hs ........................ SKIPPED
>> [INFO] hadoop-mapreduce-client-jobclient ................. SKIPPED
>> [INFO] Apache Hadoop MapReduce Examples .................. SKIPPED
>> [INFO] hadoop-mapreduce .................................. SKIPPED
>> [INFO] Apache Hadoop MapReduce Streaming ................. SKIPPED
>> [INFO] Apache Hadoop Distributed Copy .................... SKIPPED
>> [INFO] Apache Hadoop Archives ............................ SKIPPED
>> [INFO] Apache Hadoop Rumen ............................... SKIPPED
>> [INFO] Apache Hadoop Extras .............................. SKIPPED
>> [INFO] Apache Hadoop Tools Dist .......................... SKIPPED
>> [INFO] Apache Hadoop Tools ............................... SKIPPED
>> [INFO] Apache Hadoop Distribution ........................ SKIPPED
>> [INFO] Apache Hadoop Client .............................. SKIPPED
>> [INFO] Apache Hadoop Mini-Cluster ........................ SKIPPED
>> [INFO]
>> ---
>> -------------------------------------------------------------------
>>
>> My doubts here are that is the if C compiler was not able to create
>> the executables, the other projects which were build before node
>> manager should have also failed, since they are also using the auto
>> tools chain. I am able to build the entire source code if I build
>> without native support, but I do not want to skip the native
>> support build and I am trying to build the entire source code with
>> native libraries support.
>>
>> My current OS is Fedora 17 64-bit and I have also got the same
>> issue on CentOS 6.2 64-bit. I dont know if I am missing something
>> for successful build with native libraries, and would request for
>> the community's help for the same.
>>
>>
>>
>> Thanks and Regards
>> Pankaj Misra
>>
>> ________________________________
>>
>> Impetus webinar: Designing a Test Automation Framework for
>> Interoperable Systems; July 25 (10:00am PT). http://lf1.me/0E/
>>
>> Follow us on www.twitter.com/impetustech
>>
>>
>> NOTE: This message may contain information that is confidential,
>> proprietary, privileged or otherwise protected by law. The message
>> is intended solely for the named addressee. If received in error,
>> please destroy and notify the sender. Any use of this email is
>> prohibited when received in error. Impetus does not represent,
>> warrant and/or guarantee, that the integrity of this communication
>> has been maintained nor that the communication is free of errors,
>> virus, interception or interference.
>
>
>
> --
> Ioan Eugen Stan / CTO / http://axemblr.com
>
> ________________________________
>
> Impetus webinar: Designing a Test Automation Framework for
> Interoperable Systems; July 25 (10:00am PT). http://lf1.me/0E/
>
> Follow us on www.twitter.com/impetustech
>
>
> NOTE: This message may contain information that is confidential,
> proprietary, privileged or otherwise protected by law. The message
> is intended solely for the named addressee. If received in error,
> please destroy and notify the sender. Any use of this email is
> prohibited when received in error. Impetus does not represent,
> warrant and/or guarantee, that the integrity of this communication
> has been maintained nor that the communication is free of errors,
> virus, interception or interference.
unsubscribe
Posted by Andrei Krichevskiy <an...@googlemail.com>.
Sent from my iPhone
On Aug 9, 2012, at 12:46, Pankaj Misra <pa...@impetus.co.in>
wrote:
> Thanks Ioan for the help and sharing the link, appreciate it.
>
> The symlink as specified below already exists, and the the response
> of "$ which ld" is
>
> [root@fedora-0 container-executor]# which ld
> /bin/ld
>
> Yes, I will surely raise a JIRA for this issue if it does not get
> resolved, and once I am sure that I am not missing out anything.
>
> Thanks and Regards
> Pankaj Misra
>
>
> ________________________________________
> From: Ioan Eugen Stan [stan.ieugen@gmail.com]
> Sent: Thursday, August 09, 2012 4:06 PM
> To: user@hadoop.apache.org
> Subject: Re: Apache Hadoop 0.23.1 Source Build Failing
>
> It seems that /bin/ld dos not exist so the compiler cannot perform
> linking. Looking at the Fedora docs it seems that ld is located in
> /usr/bin/ld so you may have to create a symlink to it:
>
> $ ln -s /usr/bin/ld /bin/ld
>
> First check that you have ld installed with: $ which ld
>
> The scripts should also use `which ld` to find the proper path to ld.
> So maybe you could raise an issue on JIRA with this.
>
> http://docs.fedoraproject.org/en-US/Fedora/13/html/Release_Notes/sect-Release_Notes-The_GCC_Compiler_Collection.html
>
> Cheers,
>
> On Thu, Aug 9, 2012 at 1:05 PM, Pankaj Misra <pankaj.misra@impetus.co.in
> > wrote:
>> Dear All,
>>
>> I am building hadoop 0.23.1 release from source with native
>> support. I have already built/installed the following pre-
>> requisites for native support
>> 1. gcc-c++ 4.7.1
>> 2. protoc 2.4.1
>> 3. autotools chain
>> 4. JDK 1.6.0_33
>> 5. zlib 1.2.5-6
>> 6. lzo 2.06-2
>>
>> I have also set the following variables and exported them
>> export LD_LIBRARY_PATH=/usr/local/lib
>>
>> Other variables are also set as given below.
>> export LD_LIBRARY_PATH=/usr/local/lib
>> export JAVA_HOME=/usr/java/jdk1.6.0_33
>> export ANT_HOME=/home/fedora/apache-ant-1.8.4
>> export MAVEN_HOME=/home/fedora/apache-maven-3.0.4
>> export PATH=$JAVA_HOME/bin:$ANT_HOME/bin:$MAVEN_HOME/bin:$PATH
>> export HADOOP_COMMON_HOME=/home/fedora/hadoop/bin/release-0.23.1/
>> hadoop-0.23.1
>> export HADOOP_HDFS_HOME=$HADOOP_COMMON_HOME
>> export YARN_HOME=$HADOOP_COMMON_HOME
>> export HADOOP_MAPRED_HOME=$HADOOP_COMMON_HOME
>> export HADOOP_CONF_DIR=$HADOOP_COMMON_HOME/etc/hadoop
>>
>>
>> I am using the following to build the source with native support
>> (using root user access).
>> mvn clean install -Pdist -Pnative -DskipTests=true
>>
>> However, after building a number of sub-projects, the build fails
>> at the nodemanager with the following error
>> [INFO] Compiling 129 source files to /home/fedora/hadoop/src/
>> release-0.23.1/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-
>> server/hadoop-yarn-server-nodemanager/target/classes
>> [INFO]
>> [INFO] --- make-maven-plugin:1.0-beta-1:autoreconf (compile) @
>> hadoop-yarn-server-nodemanager ---
>> [INFO]
>> [INFO] --- make-maven-plugin:1.0-beta-1:configure (compile) @
>> hadoop-yarn-server-nodemanager ---
>> [INFO] checking for gcc... gcc
>> [INFO] checking whether the C compiler works... no
>> [INFO] configure: error: in `/home/fedora/hadoop/src/release-0.23.1/
>> hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-
>> server-nodemanager/target/native/container-executor':
>> [INFO] configure: error: C compiler cannot create executables
>> [INFO] See `config.log' for more details
>>
>>
>> The config.log mentions the following issues.
>>
>> configure:2562: checking whether the C compiler works
>> configure:2584: gcc -DHADOOP_CONF_DIR=/etc/hadoop -m32
>> conftest.c >&5
>> /bin/ld: cannot find crt1.o: No such file or directory
>> /bin/ld: cannot find crti.o: No such file or directory
>> /bin/ld: skipping incompatible /usr/lib/gcc/x86_64-redhat-linux/
>> 4.7.0/libgcc_s.so when searching for -lgcc_s
>> /bin/ld: cannot find -lgcc_s
>> /bin/ld: skipping incompatible /usr/lib64/libc.so when searching
>> for -lc
>> /bin/ld: cannot find -lc
>> /bin/ld: skipping incompatible /usr/lib/gcc/x86_64-redhat-linux/
>> 4.7.0/libgcc_s.so when searching for -lgcc_s
>> /bin/ld: cannot find -lgcc_s
>> /bin/ld: cannot find crtn.o: No such file or directory
>> collect2: error: ld returned 1 exit status
>> configure:2588: $? = 1
>> configure:2626: result: no
>> configure: failed program was:
>> | /* confdefs.h */
>> | #define PACKAGE_NAME "linux-container-executor"
>> | #define PACKAGE_TARNAME "linux-container-executor"
>> | #define PACKAGE_VERSION "1.0.0"
>> | #define PACKAGE_STRING "linux-container-executor 1.0.0"
>> | #define PACKAGE_BUGREPORT "mapreduce-dev@hadoop.apache.org"
>> | #define PACKAGE_URL ""
>> | /* end confdefs.h. */
>> |
>> | int
>> | main ()
>> | {
>> |
>> | ;
>> | return 0;
>> | }
>> configure:2631: error: in `/home/fedora/hadoop/src/release-0.23.1/
>> hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-
>> server-nodemanager/target/native/container-executor':
>> configure:2633: error: C compiler cannot create executables
>>
>> Overall build summary is as given below
>> [INFO] Reactor Summary:
>> [INFO]
>> [INFO] Apache Hadoop Main ................................ SUCCESS
>> [7.920s]
>> [INFO] Apache Hadoop Project POM ......................... SUCCESS
>> [1.405s]
>> [INFO] Apache Hadoop Annotations ......................... SUCCESS
>> [6.452s]
>> [INFO] Apache Hadoop Assemblies .......................... SUCCESS
>> [0.637s]
>> [INFO] Apache Hadoop Project Dist POM .................... SUCCESS
>> [3.236s]
>> [INFO] Apache Hadoop Auth ................................ SUCCESS
>> [8.805s]
>> [INFO] Apache Hadoop Auth Examples ....................... SUCCESS
>> [5.105s]
>> [INFO] Apache Hadoop Common .............................. SUCCESS
>> [1:55.833s]
>> [INFO] Apache Hadoop Common Project ...................... SUCCESS
>> [0.210s]
>> [INFO] Apache Hadoop HDFS ................................ SUCCESS
>> [1:24.218s]
>> [INFO] Apache Hadoop HttpFS .............................. SUCCESS
>> [20.992s]
>> [INFO] Apache Hadoop HDFS Project ........................ SUCCESS
>> [0.127s]
>> [INFO] hadoop-yarn ....................................... SUCCESS
>> [0.598s]
>> [INFO] hadoop-yarn-api ................................... SUCCESS
>> [57.281s]
>> [INFO] hadoop-yarn-common ................................ SUCCESS
>> [1:01.264s]
>> [INFO] hadoop-yarn-server ................................ SUCCESS
>> [0.190s]
>> [INFO] hadoop-yarn-server-common ......................... SUCCESS
>> [19.280s]
>> [INFO] hadoop-yarn-server-nodemanager .................... FAILURE
>> [17.016s]
>> [INFO] hadoop-yarn-server-web-proxy ...................... SKIPPED
>> [INFO] hadoop-yarn-server-resourcemanager ................ SKIPPED
>> [INFO] hadoop-yarn-server-tests .......................... SKIPPED
>> [INFO] hadoop-mapreduce-client ........................... SKIPPED
>> [INFO] hadoop-mapreduce-client-core ...................... SKIPPED
>> [INFO] hadoop-yarn-applications .......................... SKIPPED
>> [INFO] hadoop-yarn-applications-distributedshell ......... SKIPPED
>> [INFO] hadoop-yarn-site .................................. SKIPPED
>> [INFO] hadoop-mapreduce-client-common .................... SKIPPED
>> [INFO] hadoop-mapreduce-client-shuffle ................... SKIPPED
>> [INFO] hadoop-mapreduce-client-app ....................... SKIPPED
>> [INFO] hadoop-mapreduce-client-hs ........................ SKIPPED
>> [INFO] hadoop-mapreduce-client-jobclient ................. SKIPPED
>> [INFO] Apache Hadoop MapReduce Examples .................. SKIPPED
>> [INFO] hadoop-mapreduce .................................. SKIPPED
>> [INFO] Apache Hadoop MapReduce Streaming ................. SKIPPED
>> [INFO] Apache Hadoop Distributed Copy .................... SKIPPED
>> [INFO] Apache Hadoop Archives ............................ SKIPPED
>> [INFO] Apache Hadoop Rumen ............................... SKIPPED
>> [INFO] Apache Hadoop Extras .............................. SKIPPED
>> [INFO] Apache Hadoop Tools Dist .......................... SKIPPED
>> [INFO] Apache Hadoop Tools ............................... SKIPPED
>> [INFO] Apache Hadoop Distribution ........................ SKIPPED
>> [INFO] Apache Hadoop Client .............................. SKIPPED
>> [INFO] Apache Hadoop Mini-Cluster ........................ SKIPPED
>> [INFO]
>> ---
>> -------------------------------------------------------------------
>>
>> My doubts here are that is the if C compiler was not able to create
>> the executables, the other projects which were build before node
>> manager should have also failed, since they are also using the auto
>> tools chain. I am able to build the entire source code if I build
>> without native support, but I do not want to skip the native
>> support build and I am trying to build the entire source code with
>> native libraries support.
>>
>> My current OS is Fedora 17 64-bit and I have also got the same
>> issue on CentOS 6.2 64-bit. I dont know if I am missing something
>> for successful build with native libraries, and would request for
>> the community's help for the same.
>>
>>
>>
>> Thanks and Regards
>> Pankaj Misra
>>
>> ________________________________
>>
>> Impetus webinar: Designing a Test Automation Framework for
>> Interoperable Systems; July 25 (10:00am PT). http://lf1.me/0E/
>>
>> Follow us on www.twitter.com/impetustech
>>
>>
>> NOTE: This message may contain information that is confidential,
>> proprietary, privileged or otherwise protected by law. The message
>> is intended solely for the named addressee. If received in error,
>> please destroy and notify the sender. Any use of this email is
>> prohibited when received in error. Impetus does not represent,
>> warrant and/or guarantee, that the integrity of this communication
>> has been maintained nor that the communication is free of errors,
>> virus, interception or interference.
>
>
>
> --
> Ioan Eugen Stan / CTO / http://axemblr.com
>
> ________________________________
>
> Impetus webinar: Designing a Test Automation Framework for
> Interoperable Systems; July 25 (10:00am PT). http://lf1.me/0E/
>
> Follow us on www.twitter.com/impetustech
>
>
> NOTE: This message may contain information that is confidential,
> proprietary, privileged or otherwise protected by law. The message
> is intended solely for the named addressee. If received in error,
> please destroy and notify the sender. Any use of this email is
> prohibited when received in error. Impetus does not represent,
> warrant and/or guarantee, that the integrity of this communication
> has been maintained nor that the communication is free of errors,
> virus, interception or interference.
unsubscribe
Posted by Andrei Krichevskiy <an...@googlemail.com>.
Sent from my iPhone
On Aug 9, 2012, at 12:46, Pankaj Misra <pa...@impetus.co.in>
wrote:
> Thanks Ioan for the help and sharing the link, appreciate it.
>
> The symlink as specified below already exists, and the the response
> of "$ which ld" is
>
> [root@fedora-0 container-executor]# which ld
> /bin/ld
>
> Yes, I will surely raise a JIRA for this issue if it does not get
> resolved, and once I am sure that I am not missing out anything.
>
> Thanks and Regards
> Pankaj Misra
>
>
> ________________________________________
> From: Ioan Eugen Stan [stan.ieugen@gmail.com]
> Sent: Thursday, August 09, 2012 4:06 PM
> To: user@hadoop.apache.org
> Subject: Re: Apache Hadoop 0.23.1 Source Build Failing
>
> It seems that /bin/ld dos not exist so the compiler cannot perform
> linking. Looking at the Fedora docs it seems that ld is located in
> /usr/bin/ld so you may have to create a symlink to it:
>
> $ ln -s /usr/bin/ld /bin/ld
>
> First check that you have ld installed with: $ which ld
>
> The scripts should also use `which ld` to find the proper path to ld.
> So maybe you could raise an issue on JIRA with this.
>
> http://docs.fedoraproject.org/en-US/Fedora/13/html/Release_Notes/sect-Release_Notes-The_GCC_Compiler_Collection.html
>
> Cheers,
>
> On Thu, Aug 9, 2012 at 1:05 PM, Pankaj Misra <pankaj.misra@impetus.co.in
> > wrote:
>> Dear All,
>>
>> I am building hadoop 0.23.1 release from source with native
>> support. I have already built/installed the following pre-
>> requisites for native support
>> 1. gcc-c++ 4.7.1
>> 2. protoc 2.4.1
>> 3. autotools chain
>> 4. JDK 1.6.0_33
>> 5. zlib 1.2.5-6
>> 6. lzo 2.06-2
>>
>> I have also set the following variables and exported them
>> export LD_LIBRARY_PATH=/usr/local/lib
>>
>> Other variables are also set as given below.
>> export LD_LIBRARY_PATH=/usr/local/lib
>> export JAVA_HOME=/usr/java/jdk1.6.0_33
>> export ANT_HOME=/home/fedora/apache-ant-1.8.4
>> export MAVEN_HOME=/home/fedora/apache-maven-3.0.4
>> export PATH=$JAVA_HOME/bin:$ANT_HOME/bin:$MAVEN_HOME/bin:$PATH
>> export HADOOP_COMMON_HOME=/home/fedora/hadoop/bin/release-0.23.1/
>> hadoop-0.23.1
>> export HADOOP_HDFS_HOME=$HADOOP_COMMON_HOME
>> export YARN_HOME=$HADOOP_COMMON_HOME
>> export HADOOP_MAPRED_HOME=$HADOOP_COMMON_HOME
>> export HADOOP_CONF_DIR=$HADOOP_COMMON_HOME/etc/hadoop
>>
>>
>> I am using the following to build the source with native support
>> (using root user access).
>> mvn clean install -Pdist -Pnative -DskipTests=true
>>
>> However, after building a number of sub-projects, the build fails
>> at the nodemanager with the following error
>> [INFO] Compiling 129 source files to /home/fedora/hadoop/src/
>> release-0.23.1/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-
>> server/hadoop-yarn-server-nodemanager/target/classes
>> [INFO]
>> [INFO] --- make-maven-plugin:1.0-beta-1:autoreconf (compile) @
>> hadoop-yarn-server-nodemanager ---
>> [INFO]
>> [INFO] --- make-maven-plugin:1.0-beta-1:configure (compile) @
>> hadoop-yarn-server-nodemanager ---
>> [INFO] checking for gcc... gcc
>> [INFO] checking whether the C compiler works... no
>> [INFO] configure: error: in `/home/fedora/hadoop/src/release-0.23.1/
>> hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-
>> server-nodemanager/target/native/container-executor':
>> [INFO] configure: error: C compiler cannot create executables
>> [INFO] See `config.log' for more details
>>
>>
>> The config.log mentions the following issues.
>>
>> configure:2562: checking whether the C compiler works
>> configure:2584: gcc -DHADOOP_CONF_DIR=/etc/hadoop -m32
>> conftest.c >&5
>> /bin/ld: cannot find crt1.o: No such file or directory
>> /bin/ld: cannot find crti.o: No such file or directory
>> /bin/ld: skipping incompatible /usr/lib/gcc/x86_64-redhat-linux/
>> 4.7.0/libgcc_s.so when searching for -lgcc_s
>> /bin/ld: cannot find -lgcc_s
>> /bin/ld: skipping incompatible /usr/lib64/libc.so when searching
>> for -lc
>> /bin/ld: cannot find -lc
>> /bin/ld: skipping incompatible /usr/lib/gcc/x86_64-redhat-linux/
>> 4.7.0/libgcc_s.so when searching for -lgcc_s
>> /bin/ld: cannot find -lgcc_s
>> /bin/ld: cannot find crtn.o: No such file or directory
>> collect2: error: ld returned 1 exit status
>> configure:2588: $? = 1
>> configure:2626: result: no
>> configure: failed program was:
>> | /* confdefs.h */
>> | #define PACKAGE_NAME "linux-container-executor"
>> | #define PACKAGE_TARNAME "linux-container-executor"
>> | #define PACKAGE_VERSION "1.0.0"
>> | #define PACKAGE_STRING "linux-container-executor 1.0.0"
>> | #define PACKAGE_BUGREPORT "mapreduce-dev@hadoop.apache.org"
>> | #define PACKAGE_URL ""
>> | /* end confdefs.h. */
>> |
>> | int
>> | main ()
>> | {
>> |
>> | ;
>> | return 0;
>> | }
>> configure:2631: error: in `/home/fedora/hadoop/src/release-0.23.1/
>> hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-
>> server-nodemanager/target/native/container-executor':
>> configure:2633: error: C compiler cannot create executables
>>
>> Overall build summary is as given below
>> [INFO] Reactor Summary:
>> [INFO]
>> [INFO] Apache Hadoop Main ................................ SUCCESS
>> [7.920s]
>> [INFO] Apache Hadoop Project POM ......................... SUCCESS
>> [1.405s]
>> [INFO] Apache Hadoop Annotations ......................... SUCCESS
>> [6.452s]
>> [INFO] Apache Hadoop Assemblies .......................... SUCCESS
>> [0.637s]
>> [INFO] Apache Hadoop Project Dist POM .................... SUCCESS
>> [3.236s]
>> [INFO] Apache Hadoop Auth ................................ SUCCESS
>> [8.805s]
>> [INFO] Apache Hadoop Auth Examples ....................... SUCCESS
>> [5.105s]
>> [INFO] Apache Hadoop Common .............................. SUCCESS
>> [1:55.833s]
>> [INFO] Apache Hadoop Common Project ...................... SUCCESS
>> [0.210s]
>> [INFO] Apache Hadoop HDFS ................................ SUCCESS
>> [1:24.218s]
>> [INFO] Apache Hadoop HttpFS .............................. SUCCESS
>> [20.992s]
>> [INFO] Apache Hadoop HDFS Project ........................ SUCCESS
>> [0.127s]
>> [INFO] hadoop-yarn ....................................... SUCCESS
>> [0.598s]
>> [INFO] hadoop-yarn-api ................................... SUCCESS
>> [57.281s]
>> [INFO] hadoop-yarn-common ................................ SUCCESS
>> [1:01.264s]
>> [INFO] hadoop-yarn-server ................................ SUCCESS
>> [0.190s]
>> [INFO] hadoop-yarn-server-common ......................... SUCCESS
>> [19.280s]
>> [INFO] hadoop-yarn-server-nodemanager .................... FAILURE
>> [17.016s]
>> [INFO] hadoop-yarn-server-web-proxy ...................... SKIPPED
>> [INFO] hadoop-yarn-server-resourcemanager ................ SKIPPED
>> [INFO] hadoop-yarn-server-tests .......................... SKIPPED
>> [INFO] hadoop-mapreduce-client ........................... SKIPPED
>> [INFO] hadoop-mapreduce-client-core ...................... SKIPPED
>> [INFO] hadoop-yarn-applications .......................... SKIPPED
>> [INFO] hadoop-yarn-applications-distributedshell ......... SKIPPED
>> [INFO] hadoop-yarn-site .................................. SKIPPED
>> [INFO] hadoop-mapreduce-client-common .................... SKIPPED
>> [INFO] hadoop-mapreduce-client-shuffle ................... SKIPPED
>> [INFO] hadoop-mapreduce-client-app ....................... SKIPPED
>> [INFO] hadoop-mapreduce-client-hs ........................ SKIPPED
>> [INFO] hadoop-mapreduce-client-jobclient ................. SKIPPED
>> [INFO] Apache Hadoop MapReduce Examples .................. SKIPPED
>> [INFO] hadoop-mapreduce .................................. SKIPPED
>> [INFO] Apache Hadoop MapReduce Streaming ................. SKIPPED
>> [INFO] Apache Hadoop Distributed Copy .................... SKIPPED
>> [INFO] Apache Hadoop Archives ............................ SKIPPED
>> [INFO] Apache Hadoop Rumen ............................... SKIPPED
>> [INFO] Apache Hadoop Extras .............................. SKIPPED
>> [INFO] Apache Hadoop Tools Dist .......................... SKIPPED
>> [INFO] Apache Hadoop Tools ............................... SKIPPED
>> [INFO] Apache Hadoop Distribution ........................ SKIPPED
>> [INFO] Apache Hadoop Client .............................. SKIPPED
>> [INFO] Apache Hadoop Mini-Cluster ........................ SKIPPED
>> [INFO]
>> ---
>> -------------------------------------------------------------------
>>
>> My doubts here are that is the if C compiler was not able to create
>> the executables, the other projects which were build before node
>> manager should have also failed, since they are also using the auto
>> tools chain. I am able to build the entire source code if I build
>> without native support, but I do not want to skip the native
>> support build and I am trying to build the entire source code with
>> native libraries support.
>>
>> My current OS is Fedora 17 64-bit and I have also got the same
>> issue on CentOS 6.2 64-bit. I dont know if I am missing something
>> for successful build with native libraries, and would request for
>> the community's help for the same.
>>
>>
>>
>> Thanks and Regards
>> Pankaj Misra
>>
>> ________________________________
>>
>> Impetus webinar: Designing a Test Automation Framework for
>> Interoperable Systems; July 25 (10:00am PT). http://lf1.me/0E/
>>
>> Follow us on www.twitter.com/impetustech
>>
>>
>> NOTE: This message may contain information that is confidential,
>> proprietary, privileged or otherwise protected by law. The message
>> is intended solely for the named addressee. If received in error,
>> please destroy and notify the sender. Any use of this email is
>> prohibited when received in error. Impetus does not represent,
>> warrant and/or guarantee, that the integrity of this communication
>> has been maintained nor that the communication is free of errors,
>> virus, interception or interference.
>
>
>
> --
> Ioan Eugen Stan / CTO / http://axemblr.com
>
> ________________________________
>
> Impetus webinar: Designing a Test Automation Framework for
> Interoperable Systems; July 25 (10:00am PT). http://lf1.me/0E/
>
> Follow us on www.twitter.com/impetustech
>
>
> NOTE: This message may contain information that is confidential,
> proprietary, privileged or otherwise protected by law. The message
> is intended solely for the named addressee. If received in error,
> please destroy and notify the sender. Any use of this email is
> prohibited when received in error. Impetus does not represent,
> warrant and/or guarantee, that the integrity of this communication
> has been maintained nor that the communication is free of errors,
> virus, interception or interference.
RE: Apache Hadoop 0.23.1 Source Build Failing
Posted by Pankaj Misra <pa...@impetus.co.in>.
Thanks Ioan for the help and sharing the link, appreciate it.
The symlink as specified below already exists, and the the response of "$ which ld" is
[root@fedora-0 container-executor]# which ld
/bin/ld
Yes, I will surely raise a JIRA for this issue if it does not get resolved, and once I am sure that I am not missing out anything.
Thanks and Regards
Pankaj Misra
________________________________________
From: Ioan Eugen Stan [stan.ieugen@gmail.com]
Sent: Thursday, August 09, 2012 4:06 PM
To: user@hadoop.apache.org
Subject: Re: Apache Hadoop 0.23.1 Source Build Failing
It seems that /bin/ld dos not exist so the compiler cannot perform
linking. Looking at the Fedora docs it seems that ld is located in
/usr/bin/ld so you may have to create a symlink to it:
$ ln -s /usr/bin/ld /bin/ld
First check that you have ld installed with: $ which ld
The scripts should also use `which ld` to find the proper path to ld.
So maybe you could raise an issue on JIRA with this.
http://docs.fedoraproject.org/en-US/Fedora/13/html/Release_Notes/sect-Release_Notes-The_GCC_Compiler_Collection.html
Cheers,
On Thu, Aug 9, 2012 at 1:05 PM, Pankaj Misra <pa...@impetus.co.in> wrote:
> Dear All,
>
> I am building hadoop 0.23.1 release from source with native support. I have already built/installed the following pre-requisites for native support
> 1. gcc-c++ 4.7.1
> 2. protoc 2.4.1
> 3. autotools chain
> 4. JDK 1.6.0_33
> 5. zlib 1.2.5-6
> 6. lzo 2.06-2
>
> I have also set the following variables and exported them
> export LD_LIBRARY_PATH=/usr/local/lib
>
> Other variables are also set as given below.
> export LD_LIBRARY_PATH=/usr/local/lib
> export JAVA_HOME=/usr/java/jdk1.6.0_33
> export ANT_HOME=/home/fedora/apache-ant-1.8.4
> export MAVEN_HOME=/home/fedora/apache-maven-3.0.4
> export PATH=$JAVA_HOME/bin:$ANT_HOME/bin:$MAVEN_HOME/bin:$PATH
> export HADOOP_COMMON_HOME=/home/fedora/hadoop/bin/release-0.23.1/hadoop-0.23.1
> export HADOOP_HDFS_HOME=$HADOOP_COMMON_HOME
> export YARN_HOME=$HADOOP_COMMON_HOME
> export HADOOP_MAPRED_HOME=$HADOOP_COMMON_HOME
> export HADOOP_CONF_DIR=$HADOOP_COMMON_HOME/etc/hadoop
>
>
> I am using the following to build the source with native support(using root user access).
> mvn clean install -Pdist -Pnative -DskipTests=true
>
> However, after building a number of sub-projects, the build fails at the nodemanager with the following error
> [INFO] Compiling 129 source files to /home/fedora/hadoop/src/release-0.23.1/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/target/classes
> [INFO]
> [INFO] --- make-maven-plugin:1.0-beta-1:autoreconf (compile) @ hadoop-yarn-server-nodemanager ---
> [INFO]
> [INFO] --- make-maven-plugin:1.0-beta-1:configure (compile) @ hadoop-yarn-server-nodemanager ---
> [INFO] checking for gcc... gcc
> [INFO] checking whether the C compiler works... no
> [INFO] configure: error: in `/home/fedora/hadoop/src/release-0.23.1/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/target/native/container-executor':
> [INFO] configure: error: C compiler cannot create executables
> [INFO] See `config.log' for more details
>
>
> The config.log mentions the following issues.
>
> configure:2562: checking whether the C compiler works
> configure:2584: gcc -DHADOOP_CONF_DIR=/etc/hadoop -m32 conftest.c >&5
> /bin/ld: cannot find crt1.o: No such file or directory
> /bin/ld: cannot find crti.o: No such file or directory
> /bin/ld: skipping incompatible /usr/lib/gcc/x86_64-redhat-linux/4.7.0/libgcc_s.so when searching for -lgcc_s
> /bin/ld: cannot find -lgcc_s
> /bin/ld: skipping incompatible /usr/lib64/libc.so when searching for -lc
> /bin/ld: cannot find -lc
> /bin/ld: skipping incompatible /usr/lib/gcc/x86_64-redhat-linux/4.7.0/libgcc_s.so when searching for -lgcc_s
> /bin/ld: cannot find -lgcc_s
> /bin/ld: cannot find crtn.o: No such file or directory
> collect2: error: ld returned 1 exit status
> configure:2588: $? = 1
> configure:2626: result: no
> configure: failed program was:
> | /* confdefs.h */
> | #define PACKAGE_NAME "linux-container-executor"
> | #define PACKAGE_TARNAME "linux-container-executor"
> | #define PACKAGE_VERSION "1.0.0"
> | #define PACKAGE_STRING "linux-container-executor 1.0.0"
> | #define PACKAGE_BUGREPORT "mapreduce-dev@hadoop.apache.org"
> | #define PACKAGE_URL ""
> | /* end confdefs.h. */
> |
> | int
> | main ()
> | {
> |
> | ;
> | return 0;
> | }
> configure:2631: error: in `/home/fedora/hadoop/src/release-0.23.1/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/target/native/container-executor':
> configure:2633: error: C compiler cannot create executables
>
> Overall build summary is as given below
> [INFO] Reactor Summary:
> [INFO]
> [INFO] Apache Hadoop Main ................................ SUCCESS [7.920s]
> [INFO] Apache Hadoop Project POM ......................... SUCCESS [1.405s]
> [INFO] Apache Hadoop Annotations ......................... SUCCESS [6.452s]
> [INFO] Apache Hadoop Assemblies .......................... SUCCESS [0.637s]
> [INFO] Apache Hadoop Project Dist POM .................... SUCCESS [3.236s]
> [INFO] Apache Hadoop Auth ................................ SUCCESS [8.805s]
> [INFO] Apache Hadoop Auth Examples ....................... SUCCESS [5.105s]
> [INFO] Apache Hadoop Common .............................. SUCCESS [1:55.833s]
> [INFO] Apache Hadoop Common Project ...................... SUCCESS [0.210s]
> [INFO] Apache Hadoop HDFS ................................ SUCCESS [1:24.218s]
> [INFO] Apache Hadoop HttpFS .............................. SUCCESS [20.992s]
> [INFO] Apache Hadoop HDFS Project ........................ SUCCESS [0.127s]
> [INFO] hadoop-yarn ....................................... SUCCESS [0.598s]
> [INFO] hadoop-yarn-api ................................... SUCCESS [57.281s]
> [INFO] hadoop-yarn-common ................................ SUCCESS [1:01.264s]
> [INFO] hadoop-yarn-server ................................ SUCCESS [0.190s]
> [INFO] hadoop-yarn-server-common ......................... SUCCESS [19.280s]
> [INFO] hadoop-yarn-server-nodemanager .................... FAILURE [17.016s]
> [INFO] hadoop-yarn-server-web-proxy ...................... SKIPPED
> [INFO] hadoop-yarn-server-resourcemanager ................ SKIPPED
> [INFO] hadoop-yarn-server-tests .......................... SKIPPED
> [INFO] hadoop-mapreduce-client ........................... SKIPPED
> [INFO] hadoop-mapreduce-client-core ...................... SKIPPED
> [INFO] hadoop-yarn-applications .......................... SKIPPED
> [INFO] hadoop-yarn-applications-distributedshell ......... SKIPPED
> [INFO] hadoop-yarn-site .................................. SKIPPED
> [INFO] hadoop-mapreduce-client-common .................... SKIPPED
> [INFO] hadoop-mapreduce-client-shuffle ................... SKIPPED
> [INFO] hadoop-mapreduce-client-app ....................... SKIPPED
> [INFO] hadoop-mapreduce-client-hs ........................ SKIPPED
> [INFO] hadoop-mapreduce-client-jobclient ................. SKIPPED
> [INFO] Apache Hadoop MapReduce Examples .................. SKIPPED
> [INFO] hadoop-mapreduce .................................. SKIPPED
> [INFO] Apache Hadoop MapReduce Streaming ................. SKIPPED
> [INFO] Apache Hadoop Distributed Copy .................... SKIPPED
> [INFO] Apache Hadoop Archives ............................ SKIPPED
> [INFO] Apache Hadoop Rumen ............................... SKIPPED
> [INFO] Apache Hadoop Extras .............................. SKIPPED
> [INFO] Apache Hadoop Tools Dist .......................... SKIPPED
> [INFO] Apache Hadoop Tools ............................... SKIPPED
> [INFO] Apache Hadoop Distribution ........................ SKIPPED
> [INFO] Apache Hadoop Client .............................. SKIPPED
> [INFO] Apache Hadoop Mini-Cluster ........................ SKIPPED
> [INFO] ----------------------------------------------------------------------
>
> My doubts here are that is the if C compiler was not able to create the executables, the other projects which were build before node manager should have also failed, since they are also using the auto tools chain. I am able to build the entire source code if I build without native support, but I do not want to skip the native support build and I am trying to build the entire source code with native libraries support.
>
> My current OS is Fedora 17 64-bit and I have also got the same issue on CentOS 6.2 64-bit. I dont know if I am missing something for successful build with native libraries, and would request for the community's help for the same.
>
>
>
> Thanks and Regards
> Pankaj Misra
>
> ________________________________
>
> Impetus webinar: Designing a Test Automation Framework for Interoperable Systems; July 25 (10:00am PT). http://lf1.me/0E/
>
> Follow us on www.twitter.com/impetustech
>
>
> NOTE: This message may contain information that is confidential, proprietary, privileged or otherwise protected by law. The message is intended solely for the named addressee. If received in error, please destroy and notify the sender. Any use of this email is prohibited when received in error. Impetus does not represent, warrant and/or guarantee, that the integrity of this communication has been maintained nor that the communication is free of errors, virus, interception or interference.
--
Ioan Eugen Stan / CTO / http://axemblr.com
________________________________
Impetus webinar: Designing a Test Automation Framework for Interoperable Systems; July 25 (10:00am PT). http://lf1.me/0E/
Follow us on www.twitter.com/impetustech
NOTE: This message may contain information that is confidential, proprietary, privileged or otherwise protected by law. The message is intended solely for the named addressee. If received in error, please destroy and notify the sender. Any use of this email is prohibited when received in error. Impetus does not represent, warrant and/or guarantee, that the integrity of this communication has been maintained nor that the communication is free of errors, virus, interception or interference.
Re: Apache Hadoop 0.23.1 Source Build Failing
Posted by Ioan Eugen Stan <st...@gmail.com>.
Hello Pankaj,
Sorry, I pressed send too fast. The linker can't find some libraries.
Check to see if you have them installed (locate, find, rpm). If not
install them and try again.
If you have them installed and ld can't find them, add the paths to
the proper environment variable (I think LD_LIBRARY_PATH). Check the
ld man page ( $ man ld) for details.
e.g.
$ export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/path/to/lib
http://linux.about.com/library/cmd/blcmdl1_ld.htm
I hope this helps,
On Thu, Aug 9, 2012 at 1:36 PM, Ioan Eugen Stan <st...@gmail.com> wrote:
> It seems that /bin/ld dos not exist so the compiler cannot perform
> linking. Looking at the Fedora docs it seems that ld is located in
> /usr/bin/ld so you may have to create a symlink to it:
>
> $ ln -s /usr/bin/ld /bin/ld
>
> First check that you have ld installed with: $ which ld
>
> The scripts should also use `which ld` to find the proper path to ld.
> So maybe you could raise an issue on JIRA with this.
>
> http://docs.fedoraproject.org/en-US/Fedora/13/html/Release_Notes/sect-Release_Notes-The_GCC_Compiler_Collection.html
>
> Cheers,
>
> On Thu, Aug 9, 2012 at 1:05 PM, Pankaj Misra <pa...@impetus.co.in> wrote:
>> Dear All,
>>
>> I am building hadoop 0.23.1 release from source with native support. I have already built/installed the following pre-requisites for native support
>> 1. gcc-c++ 4.7.1
>> 2. protoc 2.4.1
>> 3. autotools chain
>> 4. JDK 1.6.0_33
>> 5. zlib 1.2.5-6
>> 6. lzo 2.06-2
>>
>> I have also set the following variables and exported them
>> export LD_LIBRARY_PATH=/usr/local/lib
>>
>> Other variables are also set as given below.
>> export LD_LIBRARY_PATH=/usr/local/lib
>> export JAVA_HOME=/usr/java/jdk1.6.0_33
>> export ANT_HOME=/home/fedora/apache-ant-1.8.4
>> export MAVEN_HOME=/home/fedora/apache-maven-3.0.4
>> export PATH=$JAVA_HOME/bin:$ANT_HOME/bin:$MAVEN_HOME/bin:$PATH
>> export HADOOP_COMMON_HOME=/home/fedora/hadoop/bin/release-0.23.1/hadoop-0.23.1
>> export HADOOP_HDFS_HOME=$HADOOP_COMMON_HOME
>> export YARN_HOME=$HADOOP_COMMON_HOME
>> export HADOOP_MAPRED_HOME=$HADOOP_COMMON_HOME
>> export HADOOP_CONF_DIR=$HADOOP_COMMON_HOME/etc/hadoop
>>
>>
>> I am using the following to build the source with native support(using root user access).
>> mvn clean install -Pdist -Pnative -DskipTests=true
>>
>> However, after building a number of sub-projects, the build fails at the nodemanager with the following error
>> [INFO] Compiling 129 source files to /home/fedora/hadoop/src/release-0.23.1/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/target/classes
>> [INFO]
>> [INFO] --- make-maven-plugin:1.0-beta-1:autoreconf (compile) @ hadoop-yarn-server-nodemanager ---
>> [INFO]
>> [INFO] --- make-maven-plugin:1.0-beta-1:configure (compile) @ hadoop-yarn-server-nodemanager ---
>> [INFO] checking for gcc... gcc
>> [INFO] checking whether the C compiler works... no
>> [INFO] configure: error: in `/home/fedora/hadoop/src/release-0.23.1/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/target/native/container-executor':
>> [INFO] configure: error: C compiler cannot create executables
>> [INFO] See `config.log' for more details
>>
>>
>> The config.log mentions the following issues.
>>
>> configure:2562: checking whether the C compiler works
>> configure:2584: gcc -DHADOOP_CONF_DIR=/etc/hadoop -m32 conftest.c >&5
>> /bin/ld: cannot find crt1.o: No such file or directory
>> /bin/ld: cannot find crti.o: No such file or directory
>> /bin/ld: skipping incompatible /usr/lib/gcc/x86_64-redhat-linux/4.7.0/libgcc_s.so when searching for -lgcc_s
>> /bin/ld: cannot find -lgcc_s
>> /bin/ld: skipping incompatible /usr/lib64/libc.so when searching for -lc
>> /bin/ld: cannot find -lc
>> /bin/ld: skipping incompatible /usr/lib/gcc/x86_64-redhat-linux/4.7.0/libgcc_s.so when searching for -lgcc_s
>> /bin/ld: cannot find -lgcc_s
>> /bin/ld: cannot find crtn.o: No such file or directory
>> collect2: error: ld returned 1 exit status
>> configure:2588: $? = 1
>> configure:2626: result: no
>> configure: failed program was:
>> | /* confdefs.h */
>> | #define PACKAGE_NAME "linux-container-executor"
>> | #define PACKAGE_TARNAME "linux-container-executor"
>> | #define PACKAGE_VERSION "1.0.0"
>> | #define PACKAGE_STRING "linux-container-executor 1.0.0"
>> | #define PACKAGE_BUGREPORT "mapreduce-dev@hadoop.apache.org"
>> | #define PACKAGE_URL ""
>> | /* end confdefs.h. */
>> |
>> | int
>> | main ()
>> | {
>> |
>> | ;
>> | return 0;
>> | }
>> configure:2631: error: in `/home/fedora/hadoop/src/release-0.23.1/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/target/native/container-executor':
>> configure:2633: error: C compiler cannot create executables
>>
>> Overall build summary is as given below
>> [INFO] Reactor Summary:
>> [INFO]
>> [INFO] Apache Hadoop Main ................................ SUCCESS [7.920s]
>> [INFO] Apache Hadoop Project POM ......................... SUCCESS [1.405s]
>> [INFO] Apache Hadoop Annotations ......................... SUCCESS [6.452s]
>> [INFO] Apache Hadoop Assemblies .......................... SUCCESS [0.637s]
>> [INFO] Apache Hadoop Project Dist POM .................... SUCCESS [3.236s]
>> [INFO] Apache Hadoop Auth ................................ SUCCESS [8.805s]
>> [INFO] Apache Hadoop Auth Examples ....................... SUCCESS [5.105s]
>> [INFO] Apache Hadoop Common .............................. SUCCESS [1:55.833s]
>> [INFO] Apache Hadoop Common Project ...................... SUCCESS [0.210s]
>> [INFO] Apache Hadoop HDFS ................................ SUCCESS [1:24.218s]
>> [INFO] Apache Hadoop HttpFS .............................. SUCCESS [20.992s]
>> [INFO] Apache Hadoop HDFS Project ........................ SUCCESS [0.127s]
>> [INFO] hadoop-yarn ....................................... SUCCESS [0.598s]
>> [INFO] hadoop-yarn-api ................................... SUCCESS [57.281s]
>> [INFO] hadoop-yarn-common ................................ SUCCESS [1:01.264s]
>> [INFO] hadoop-yarn-server ................................ SUCCESS [0.190s]
>> [INFO] hadoop-yarn-server-common ......................... SUCCESS [19.280s]
>> [INFO] hadoop-yarn-server-nodemanager .................... FAILURE [17.016s]
>> [INFO] hadoop-yarn-server-web-proxy ...................... SKIPPED
>> [INFO] hadoop-yarn-server-resourcemanager ................ SKIPPED
>> [INFO] hadoop-yarn-server-tests .......................... SKIPPED
>> [INFO] hadoop-mapreduce-client ........................... SKIPPED
>> [INFO] hadoop-mapreduce-client-core ...................... SKIPPED
>> [INFO] hadoop-yarn-applications .......................... SKIPPED
>> [INFO] hadoop-yarn-applications-distributedshell ......... SKIPPED
>> [INFO] hadoop-yarn-site .................................. SKIPPED
>> [INFO] hadoop-mapreduce-client-common .................... SKIPPED
>> [INFO] hadoop-mapreduce-client-shuffle ................... SKIPPED
>> [INFO] hadoop-mapreduce-client-app ....................... SKIPPED
>> [INFO] hadoop-mapreduce-client-hs ........................ SKIPPED
>> [INFO] hadoop-mapreduce-client-jobclient ................. SKIPPED
>> [INFO] Apache Hadoop MapReduce Examples .................. SKIPPED
>> [INFO] hadoop-mapreduce .................................. SKIPPED
>> [INFO] Apache Hadoop MapReduce Streaming ................. SKIPPED
>> [INFO] Apache Hadoop Distributed Copy .................... SKIPPED
>> [INFO] Apache Hadoop Archives ............................ SKIPPED
>> [INFO] Apache Hadoop Rumen ............................... SKIPPED
>> [INFO] Apache Hadoop Extras .............................. SKIPPED
>> [INFO] Apache Hadoop Tools Dist .......................... SKIPPED
>> [INFO] Apache Hadoop Tools ............................... SKIPPED
>> [INFO] Apache Hadoop Distribution ........................ SKIPPED
>> [INFO] Apache Hadoop Client .............................. SKIPPED
>> [INFO] Apache Hadoop Mini-Cluster ........................ SKIPPED
>> [INFO] ----------------------------------------------------------------------
>>
>> My doubts here are that is the if C compiler was not able to create the executables, the other projects which were build before node manager should have also failed, since they are also using the auto tools chain. I am able to build the entire source code if I build without native support, but I do not want to skip the native support build and I am trying to build the entire source code with native libraries support.
>>
>> My current OS is Fedora 17 64-bit and I have also got the same issue on CentOS 6.2 64-bit. I dont know if I am missing something for successful build with native libraries, and would request for the community's help for the same.
>>
>>
>>
>> Thanks and Regards
>> Pankaj Misra
>>
>> ________________________________
>>
>> Impetus webinar: Designing a Test Automation Framework for Interoperable Systems; July 25 (10:00am PT). http://lf1.me/0E/
>>
>> Follow us on www.twitter.com/impetustech
>>
>>
>> NOTE: This message may contain information that is confidential, proprietary, privileged or otherwise protected by law. The message is intended solely for the named addressee. If received in error, please destroy and notify the sender. Any use of this email is prohibited when received in error. Impetus does not represent, warrant and/or guarantee, that the integrity of this communication has been maintained nor that the communication is free of errors, virus, interception or interference.
>
>
>
> --
> Ioan Eugen Stan / CTO / http://axemblr.com
--
Ioan Eugen Stan / CTO / http://axemblr.com
RE: Apache Hadoop 0.23.1 Source Build Failing
Posted by Pankaj Misra <pa...@impetus.co.in>.
Thanks Ioan for the help and sharing the link, appreciate it.
The symlink as specified below already exists, and the the response of "$ which ld" is
[root@fedora-0 container-executor]# which ld
/bin/ld
Yes, I will surely raise a JIRA for this issue if it does not get resolved, and once I am sure that I am not missing out anything.
Thanks and Regards
Pankaj Misra
________________________________________
From: Ioan Eugen Stan [stan.ieugen@gmail.com]
Sent: Thursday, August 09, 2012 4:06 PM
To: user@hadoop.apache.org
Subject: Re: Apache Hadoop 0.23.1 Source Build Failing
It seems that /bin/ld dos not exist so the compiler cannot perform
linking. Looking at the Fedora docs it seems that ld is located in
/usr/bin/ld so you may have to create a symlink to it:
$ ln -s /usr/bin/ld /bin/ld
First check that you have ld installed with: $ which ld
The scripts should also use `which ld` to find the proper path to ld.
So maybe you could raise an issue on JIRA with this.
http://docs.fedoraproject.org/en-US/Fedora/13/html/Release_Notes/sect-Release_Notes-The_GCC_Compiler_Collection.html
Cheers,
On Thu, Aug 9, 2012 at 1:05 PM, Pankaj Misra <pa...@impetus.co.in> wrote:
> Dear All,
>
> I am building hadoop 0.23.1 release from source with native support. I have already built/installed the following pre-requisites for native support
> 1. gcc-c++ 4.7.1
> 2. protoc 2.4.1
> 3. autotools chain
> 4. JDK 1.6.0_33
> 5. zlib 1.2.5-6
> 6. lzo 2.06-2
>
> I have also set the following variables and exported them
> export LD_LIBRARY_PATH=/usr/local/lib
>
> Other variables are also set as given below.
> export LD_LIBRARY_PATH=/usr/local/lib
> export JAVA_HOME=/usr/java/jdk1.6.0_33
> export ANT_HOME=/home/fedora/apache-ant-1.8.4
> export MAVEN_HOME=/home/fedora/apache-maven-3.0.4
> export PATH=$JAVA_HOME/bin:$ANT_HOME/bin:$MAVEN_HOME/bin:$PATH
> export HADOOP_COMMON_HOME=/home/fedora/hadoop/bin/release-0.23.1/hadoop-0.23.1
> export HADOOP_HDFS_HOME=$HADOOP_COMMON_HOME
> export YARN_HOME=$HADOOP_COMMON_HOME
> export HADOOP_MAPRED_HOME=$HADOOP_COMMON_HOME
> export HADOOP_CONF_DIR=$HADOOP_COMMON_HOME/etc/hadoop
>
>
> I am using the following to build the source with native support(using root user access).
> mvn clean install -Pdist -Pnative -DskipTests=true
>
> However, after building a number of sub-projects, the build fails at the nodemanager with the following error
> [INFO] Compiling 129 source files to /home/fedora/hadoop/src/release-0.23.1/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/target/classes
> [INFO]
> [INFO] --- make-maven-plugin:1.0-beta-1:autoreconf (compile) @ hadoop-yarn-server-nodemanager ---
> [INFO]
> [INFO] --- make-maven-plugin:1.0-beta-1:configure (compile) @ hadoop-yarn-server-nodemanager ---
> [INFO] checking for gcc... gcc
> [INFO] checking whether the C compiler works... no
> [INFO] configure: error: in `/home/fedora/hadoop/src/release-0.23.1/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/target/native/container-executor':
> [INFO] configure: error: C compiler cannot create executables
> [INFO] See `config.log' for more details
>
>
> The config.log mentions the following issues.
>
> configure:2562: checking whether the C compiler works
> configure:2584: gcc -DHADOOP_CONF_DIR=/etc/hadoop -m32 conftest.c >&5
> /bin/ld: cannot find crt1.o: No such file or directory
> /bin/ld: cannot find crti.o: No such file or directory
> /bin/ld: skipping incompatible /usr/lib/gcc/x86_64-redhat-linux/4.7.0/libgcc_s.so when searching for -lgcc_s
> /bin/ld: cannot find -lgcc_s
> /bin/ld: skipping incompatible /usr/lib64/libc.so when searching for -lc
> /bin/ld: cannot find -lc
> /bin/ld: skipping incompatible /usr/lib/gcc/x86_64-redhat-linux/4.7.0/libgcc_s.so when searching for -lgcc_s
> /bin/ld: cannot find -lgcc_s
> /bin/ld: cannot find crtn.o: No such file or directory
> collect2: error: ld returned 1 exit status
> configure:2588: $? = 1
> configure:2626: result: no
> configure: failed program was:
> | /* confdefs.h */
> | #define PACKAGE_NAME "linux-container-executor"
> | #define PACKAGE_TARNAME "linux-container-executor"
> | #define PACKAGE_VERSION "1.0.0"
> | #define PACKAGE_STRING "linux-container-executor 1.0.0"
> | #define PACKAGE_BUGREPORT "mapreduce-dev@hadoop.apache.org"
> | #define PACKAGE_URL ""
> | /* end confdefs.h. */
> |
> | int
> | main ()
> | {
> |
> | ;
> | return 0;
> | }
> configure:2631: error: in `/home/fedora/hadoop/src/release-0.23.1/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/target/native/container-executor':
> configure:2633: error: C compiler cannot create executables
>
> Overall build summary is as given below
> [INFO] Reactor Summary:
> [INFO]
> [INFO] Apache Hadoop Main ................................ SUCCESS [7.920s]
> [INFO] Apache Hadoop Project POM ......................... SUCCESS [1.405s]
> [INFO] Apache Hadoop Annotations ......................... SUCCESS [6.452s]
> [INFO] Apache Hadoop Assemblies .......................... SUCCESS [0.637s]
> [INFO] Apache Hadoop Project Dist POM .................... SUCCESS [3.236s]
> [INFO] Apache Hadoop Auth ................................ SUCCESS [8.805s]
> [INFO] Apache Hadoop Auth Examples ....................... SUCCESS [5.105s]
> [INFO] Apache Hadoop Common .............................. SUCCESS [1:55.833s]
> [INFO] Apache Hadoop Common Project ...................... SUCCESS [0.210s]
> [INFO] Apache Hadoop HDFS ................................ SUCCESS [1:24.218s]
> [INFO] Apache Hadoop HttpFS .............................. SUCCESS [20.992s]
> [INFO] Apache Hadoop HDFS Project ........................ SUCCESS [0.127s]
> [INFO] hadoop-yarn ....................................... SUCCESS [0.598s]
> [INFO] hadoop-yarn-api ................................... SUCCESS [57.281s]
> [INFO] hadoop-yarn-common ................................ SUCCESS [1:01.264s]
> [INFO] hadoop-yarn-server ................................ SUCCESS [0.190s]
> [INFO] hadoop-yarn-server-common ......................... SUCCESS [19.280s]
> [INFO] hadoop-yarn-server-nodemanager .................... FAILURE [17.016s]
> [INFO] hadoop-yarn-server-web-proxy ...................... SKIPPED
> [INFO] hadoop-yarn-server-resourcemanager ................ SKIPPED
> [INFO] hadoop-yarn-server-tests .......................... SKIPPED
> [INFO] hadoop-mapreduce-client ........................... SKIPPED
> [INFO] hadoop-mapreduce-client-core ...................... SKIPPED
> [INFO] hadoop-yarn-applications .......................... SKIPPED
> [INFO] hadoop-yarn-applications-distributedshell ......... SKIPPED
> [INFO] hadoop-yarn-site .................................. SKIPPED
> [INFO] hadoop-mapreduce-client-common .................... SKIPPED
> [INFO] hadoop-mapreduce-client-shuffle ................... SKIPPED
> [INFO] hadoop-mapreduce-client-app ....................... SKIPPED
> [INFO] hadoop-mapreduce-client-hs ........................ SKIPPED
> [INFO] hadoop-mapreduce-client-jobclient ................. SKIPPED
> [INFO] Apache Hadoop MapReduce Examples .................. SKIPPED
> [INFO] hadoop-mapreduce .................................. SKIPPED
> [INFO] Apache Hadoop MapReduce Streaming ................. SKIPPED
> [INFO] Apache Hadoop Distributed Copy .................... SKIPPED
> [INFO] Apache Hadoop Archives ............................ SKIPPED
> [INFO] Apache Hadoop Rumen ............................... SKIPPED
> [INFO] Apache Hadoop Extras .............................. SKIPPED
> [INFO] Apache Hadoop Tools Dist .......................... SKIPPED
> [INFO] Apache Hadoop Tools ............................... SKIPPED
> [INFO] Apache Hadoop Distribution ........................ SKIPPED
> [INFO] Apache Hadoop Client .............................. SKIPPED
> [INFO] Apache Hadoop Mini-Cluster ........................ SKIPPED
> [INFO] ----------------------------------------------------------------------
>
> My doubts here are that is the if C compiler was not able to create the executables, the other projects which were build before node manager should have also failed, since they are also using the auto tools chain. I am able to build the entire source code if I build without native support, but I do not want to skip the native support build and I am trying to build the entire source code with native libraries support.
>
> My current OS is Fedora 17 64-bit and I have also got the same issue on CentOS 6.2 64-bit. I dont know if I am missing something for successful build with native libraries, and would request for the community's help for the same.
>
>
>
> Thanks and Regards
> Pankaj Misra
>
> ________________________________
>
> Impetus webinar: Designing a Test Automation Framework for Interoperable Systems; July 25 (10:00am PT). http://lf1.me/0E/
>
> Follow us on www.twitter.com/impetustech
>
>
> NOTE: This message may contain information that is confidential, proprietary, privileged or otherwise protected by law. The message is intended solely for the named addressee. If received in error, please destroy and notify the sender. Any use of this email is prohibited when received in error. Impetus does not represent, warrant and/or guarantee, that the integrity of this communication has been maintained nor that the communication is free of errors, virus, interception or interference.
--
Ioan Eugen Stan / CTO / http://axemblr.com
________________________________
Impetus webinar: Designing a Test Automation Framework for Interoperable Systems; July 25 (10:00am PT). http://lf1.me/0E/
Follow us on www.twitter.com/impetustech
NOTE: This message may contain information that is confidential, proprietary, privileged or otherwise protected by law. The message is intended solely for the named addressee. If received in error, please destroy and notify the sender. Any use of this email is prohibited when received in error. Impetus does not represent, warrant and/or guarantee, that the integrity of this communication has been maintained nor that the communication is free of errors, virus, interception or interference.
Re: Apache Hadoop 0.23.1 Source Build Failing
Posted by Ioan Eugen Stan <st...@gmail.com>.
Hello Pankaj,
Sorry, I pressed send too fast. The linker can't find some libraries.
Check to see if you have them installed (locate, find, rpm). If not
install them and try again.
If you have them installed and ld can't find them, add the paths to
the proper environment variable (I think LD_LIBRARY_PATH). Check the
ld man page ( $ man ld) for details.
e.g.
$ export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/path/to/lib
http://linux.about.com/library/cmd/blcmdl1_ld.htm
I hope this helps,
On Thu, Aug 9, 2012 at 1:36 PM, Ioan Eugen Stan <st...@gmail.com> wrote:
> It seems that /bin/ld dos not exist so the compiler cannot perform
> linking. Looking at the Fedora docs it seems that ld is located in
> /usr/bin/ld so you may have to create a symlink to it:
>
> $ ln -s /usr/bin/ld /bin/ld
>
> First check that you have ld installed with: $ which ld
>
> The scripts should also use `which ld` to find the proper path to ld.
> So maybe you could raise an issue on JIRA with this.
>
> http://docs.fedoraproject.org/en-US/Fedora/13/html/Release_Notes/sect-Release_Notes-The_GCC_Compiler_Collection.html
>
> Cheers,
>
> On Thu, Aug 9, 2012 at 1:05 PM, Pankaj Misra <pa...@impetus.co.in> wrote:
>> Dear All,
>>
>> I am building hadoop 0.23.1 release from source with native support. I have already built/installed the following pre-requisites for native support
>> 1. gcc-c++ 4.7.1
>> 2. protoc 2.4.1
>> 3. autotools chain
>> 4. JDK 1.6.0_33
>> 5. zlib 1.2.5-6
>> 6. lzo 2.06-2
>>
>> I have also set the following variables and exported them
>> export LD_LIBRARY_PATH=/usr/local/lib
>>
>> Other variables are also set as given below.
>> export LD_LIBRARY_PATH=/usr/local/lib
>> export JAVA_HOME=/usr/java/jdk1.6.0_33
>> export ANT_HOME=/home/fedora/apache-ant-1.8.4
>> export MAVEN_HOME=/home/fedora/apache-maven-3.0.4
>> export PATH=$JAVA_HOME/bin:$ANT_HOME/bin:$MAVEN_HOME/bin:$PATH
>> export HADOOP_COMMON_HOME=/home/fedora/hadoop/bin/release-0.23.1/hadoop-0.23.1
>> export HADOOP_HDFS_HOME=$HADOOP_COMMON_HOME
>> export YARN_HOME=$HADOOP_COMMON_HOME
>> export HADOOP_MAPRED_HOME=$HADOOP_COMMON_HOME
>> export HADOOP_CONF_DIR=$HADOOP_COMMON_HOME/etc/hadoop
>>
>>
>> I am using the following to build the source with native support(using root user access).
>> mvn clean install -Pdist -Pnative -DskipTests=true
>>
>> However, after building a number of sub-projects, the build fails at the nodemanager with the following error
>> [INFO] Compiling 129 source files to /home/fedora/hadoop/src/release-0.23.1/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/target/classes
>> [INFO]
>> [INFO] --- make-maven-plugin:1.0-beta-1:autoreconf (compile) @ hadoop-yarn-server-nodemanager ---
>> [INFO]
>> [INFO] --- make-maven-plugin:1.0-beta-1:configure (compile) @ hadoop-yarn-server-nodemanager ---
>> [INFO] checking for gcc... gcc
>> [INFO] checking whether the C compiler works... no
>> [INFO] configure: error: in `/home/fedora/hadoop/src/release-0.23.1/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/target/native/container-executor':
>> [INFO] configure: error: C compiler cannot create executables
>> [INFO] See `config.log' for more details
>>
>>
>> The config.log mentions the following issues.
>>
>> configure:2562: checking whether the C compiler works
>> configure:2584: gcc -DHADOOP_CONF_DIR=/etc/hadoop -m32 conftest.c >&5
>> /bin/ld: cannot find crt1.o: No such file or directory
>> /bin/ld: cannot find crti.o: No such file or directory
>> /bin/ld: skipping incompatible /usr/lib/gcc/x86_64-redhat-linux/4.7.0/libgcc_s.so when searching for -lgcc_s
>> /bin/ld: cannot find -lgcc_s
>> /bin/ld: skipping incompatible /usr/lib64/libc.so when searching for -lc
>> /bin/ld: cannot find -lc
>> /bin/ld: skipping incompatible /usr/lib/gcc/x86_64-redhat-linux/4.7.0/libgcc_s.so when searching for -lgcc_s
>> /bin/ld: cannot find -lgcc_s
>> /bin/ld: cannot find crtn.o: No such file or directory
>> collect2: error: ld returned 1 exit status
>> configure:2588: $? = 1
>> configure:2626: result: no
>> configure: failed program was:
>> | /* confdefs.h */
>> | #define PACKAGE_NAME "linux-container-executor"
>> | #define PACKAGE_TARNAME "linux-container-executor"
>> | #define PACKAGE_VERSION "1.0.0"
>> | #define PACKAGE_STRING "linux-container-executor 1.0.0"
>> | #define PACKAGE_BUGREPORT "mapreduce-dev@hadoop.apache.org"
>> | #define PACKAGE_URL ""
>> | /* end confdefs.h. */
>> |
>> | int
>> | main ()
>> | {
>> |
>> | ;
>> | return 0;
>> | }
>> configure:2631: error: in `/home/fedora/hadoop/src/release-0.23.1/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/target/native/container-executor':
>> configure:2633: error: C compiler cannot create executables
>>
>> Overall build summary is as given below
>> [INFO] Reactor Summary:
>> [INFO]
>> [INFO] Apache Hadoop Main ................................ SUCCESS [7.920s]
>> [INFO] Apache Hadoop Project POM ......................... SUCCESS [1.405s]
>> [INFO] Apache Hadoop Annotations ......................... SUCCESS [6.452s]
>> [INFO] Apache Hadoop Assemblies .......................... SUCCESS [0.637s]
>> [INFO] Apache Hadoop Project Dist POM .................... SUCCESS [3.236s]
>> [INFO] Apache Hadoop Auth ................................ SUCCESS [8.805s]
>> [INFO] Apache Hadoop Auth Examples ....................... SUCCESS [5.105s]
>> [INFO] Apache Hadoop Common .............................. SUCCESS [1:55.833s]
>> [INFO] Apache Hadoop Common Project ...................... SUCCESS [0.210s]
>> [INFO] Apache Hadoop HDFS ................................ SUCCESS [1:24.218s]
>> [INFO] Apache Hadoop HttpFS .............................. SUCCESS [20.992s]
>> [INFO] Apache Hadoop HDFS Project ........................ SUCCESS [0.127s]
>> [INFO] hadoop-yarn ....................................... SUCCESS [0.598s]
>> [INFO] hadoop-yarn-api ................................... SUCCESS [57.281s]
>> [INFO] hadoop-yarn-common ................................ SUCCESS [1:01.264s]
>> [INFO] hadoop-yarn-server ................................ SUCCESS [0.190s]
>> [INFO] hadoop-yarn-server-common ......................... SUCCESS [19.280s]
>> [INFO] hadoop-yarn-server-nodemanager .................... FAILURE [17.016s]
>> [INFO] hadoop-yarn-server-web-proxy ...................... SKIPPED
>> [INFO] hadoop-yarn-server-resourcemanager ................ SKIPPED
>> [INFO] hadoop-yarn-server-tests .......................... SKIPPED
>> [INFO] hadoop-mapreduce-client ........................... SKIPPED
>> [INFO] hadoop-mapreduce-client-core ...................... SKIPPED
>> [INFO] hadoop-yarn-applications .......................... SKIPPED
>> [INFO] hadoop-yarn-applications-distributedshell ......... SKIPPED
>> [INFO] hadoop-yarn-site .................................. SKIPPED
>> [INFO] hadoop-mapreduce-client-common .................... SKIPPED
>> [INFO] hadoop-mapreduce-client-shuffle ................... SKIPPED
>> [INFO] hadoop-mapreduce-client-app ....................... SKIPPED
>> [INFO] hadoop-mapreduce-client-hs ........................ SKIPPED
>> [INFO] hadoop-mapreduce-client-jobclient ................. SKIPPED
>> [INFO] Apache Hadoop MapReduce Examples .................. SKIPPED
>> [INFO] hadoop-mapreduce .................................. SKIPPED
>> [INFO] Apache Hadoop MapReduce Streaming ................. SKIPPED
>> [INFO] Apache Hadoop Distributed Copy .................... SKIPPED
>> [INFO] Apache Hadoop Archives ............................ SKIPPED
>> [INFO] Apache Hadoop Rumen ............................... SKIPPED
>> [INFO] Apache Hadoop Extras .............................. SKIPPED
>> [INFO] Apache Hadoop Tools Dist .......................... SKIPPED
>> [INFO] Apache Hadoop Tools ............................... SKIPPED
>> [INFO] Apache Hadoop Distribution ........................ SKIPPED
>> [INFO] Apache Hadoop Client .............................. SKIPPED
>> [INFO] Apache Hadoop Mini-Cluster ........................ SKIPPED
>> [INFO] ----------------------------------------------------------------------
>>
>> My doubts here are that is the if C compiler was not able to create the executables, the other projects which were build before node manager should have also failed, since they are also using the auto tools chain. I am able to build the entire source code if I build without native support, but I do not want to skip the native support build and I am trying to build the entire source code with native libraries support.
>>
>> My current OS is Fedora 17 64-bit and I have also got the same issue on CentOS 6.2 64-bit. I dont know if I am missing something for successful build with native libraries, and would request for the community's help for the same.
>>
>>
>>
>> Thanks and Regards
>> Pankaj Misra
>>
>> ________________________________
>>
>> Impetus webinar: Designing a Test Automation Framework for Interoperable Systems; July 25 (10:00am PT). http://lf1.me/0E/
>>
>> Follow us on www.twitter.com/impetustech
>>
>>
>> NOTE: This message may contain information that is confidential, proprietary, privileged or otherwise protected by law. The message is intended solely for the named addressee. If received in error, please destroy and notify the sender. Any use of this email is prohibited when received in error. Impetus does not represent, warrant and/or guarantee, that the integrity of this communication has been maintained nor that the communication is free of errors, virus, interception or interference.
>
>
>
> --
> Ioan Eugen Stan / CTO / http://axemblr.com
--
Ioan Eugen Stan / CTO / http://axemblr.com
RE: Apache Hadoop 0.23.1 Source Build Failing
Posted by Pankaj Misra <pa...@impetus.co.in>.
Thanks Ioan for the help and sharing the link, appreciate it.
The symlink as specified below already exists, and the the response of "$ which ld" is
[root@fedora-0 container-executor]# which ld
/bin/ld
Yes, I will surely raise a JIRA for this issue if it does not get resolved, and once I am sure that I am not missing out anything.
Thanks and Regards
Pankaj Misra
________________________________________
From: Ioan Eugen Stan [stan.ieugen@gmail.com]
Sent: Thursday, August 09, 2012 4:06 PM
To: user@hadoop.apache.org
Subject: Re: Apache Hadoop 0.23.1 Source Build Failing
It seems that /bin/ld dos not exist so the compiler cannot perform
linking. Looking at the Fedora docs it seems that ld is located in
/usr/bin/ld so you may have to create a symlink to it:
$ ln -s /usr/bin/ld /bin/ld
First check that you have ld installed with: $ which ld
The scripts should also use `which ld` to find the proper path to ld.
So maybe you could raise an issue on JIRA with this.
http://docs.fedoraproject.org/en-US/Fedora/13/html/Release_Notes/sect-Release_Notes-The_GCC_Compiler_Collection.html
Cheers,
On Thu, Aug 9, 2012 at 1:05 PM, Pankaj Misra <pa...@impetus.co.in> wrote:
> Dear All,
>
> I am building hadoop 0.23.1 release from source with native support. I have already built/installed the following pre-requisites for native support
> 1. gcc-c++ 4.7.1
> 2. protoc 2.4.1
> 3. autotools chain
> 4. JDK 1.6.0_33
> 5. zlib 1.2.5-6
> 6. lzo 2.06-2
>
> I have also set the following variables and exported them
> export LD_LIBRARY_PATH=/usr/local/lib
>
> Other variables are also set as given below.
> export LD_LIBRARY_PATH=/usr/local/lib
> export JAVA_HOME=/usr/java/jdk1.6.0_33
> export ANT_HOME=/home/fedora/apache-ant-1.8.4
> export MAVEN_HOME=/home/fedora/apache-maven-3.0.4
> export PATH=$JAVA_HOME/bin:$ANT_HOME/bin:$MAVEN_HOME/bin:$PATH
> export HADOOP_COMMON_HOME=/home/fedora/hadoop/bin/release-0.23.1/hadoop-0.23.1
> export HADOOP_HDFS_HOME=$HADOOP_COMMON_HOME
> export YARN_HOME=$HADOOP_COMMON_HOME
> export HADOOP_MAPRED_HOME=$HADOOP_COMMON_HOME
> export HADOOP_CONF_DIR=$HADOOP_COMMON_HOME/etc/hadoop
>
>
> I am using the following to build the source with native support(using root user access).
> mvn clean install -Pdist -Pnative -DskipTests=true
>
> However, after building a number of sub-projects, the build fails at the nodemanager with the following error
> [INFO] Compiling 129 source files to /home/fedora/hadoop/src/release-0.23.1/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/target/classes
> [INFO]
> [INFO] --- make-maven-plugin:1.0-beta-1:autoreconf (compile) @ hadoop-yarn-server-nodemanager ---
> [INFO]
> [INFO] --- make-maven-plugin:1.0-beta-1:configure (compile) @ hadoop-yarn-server-nodemanager ---
> [INFO] checking for gcc... gcc
> [INFO] checking whether the C compiler works... no
> [INFO] configure: error: in `/home/fedora/hadoop/src/release-0.23.1/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/target/native/container-executor':
> [INFO] configure: error: C compiler cannot create executables
> [INFO] See `config.log' for more details
>
>
> The config.log mentions the following issues.
>
> configure:2562: checking whether the C compiler works
> configure:2584: gcc -DHADOOP_CONF_DIR=/etc/hadoop -m32 conftest.c >&5
> /bin/ld: cannot find crt1.o: No such file or directory
> /bin/ld: cannot find crti.o: No such file or directory
> /bin/ld: skipping incompatible /usr/lib/gcc/x86_64-redhat-linux/4.7.0/libgcc_s.so when searching for -lgcc_s
> /bin/ld: cannot find -lgcc_s
> /bin/ld: skipping incompatible /usr/lib64/libc.so when searching for -lc
> /bin/ld: cannot find -lc
> /bin/ld: skipping incompatible /usr/lib/gcc/x86_64-redhat-linux/4.7.0/libgcc_s.so when searching for -lgcc_s
> /bin/ld: cannot find -lgcc_s
> /bin/ld: cannot find crtn.o: No such file or directory
> collect2: error: ld returned 1 exit status
> configure:2588: $? = 1
> configure:2626: result: no
> configure: failed program was:
> | /* confdefs.h */
> | #define PACKAGE_NAME "linux-container-executor"
> | #define PACKAGE_TARNAME "linux-container-executor"
> | #define PACKAGE_VERSION "1.0.0"
> | #define PACKAGE_STRING "linux-container-executor 1.0.0"
> | #define PACKAGE_BUGREPORT "mapreduce-dev@hadoop.apache.org"
> | #define PACKAGE_URL ""
> | /* end confdefs.h. */
> |
> | int
> | main ()
> | {
> |
> | ;
> | return 0;
> | }
> configure:2631: error: in `/home/fedora/hadoop/src/release-0.23.1/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/target/native/container-executor':
> configure:2633: error: C compiler cannot create executables
>
> Overall build summary is as given below
> [INFO] Reactor Summary:
> [INFO]
> [INFO] Apache Hadoop Main ................................ SUCCESS [7.920s]
> [INFO] Apache Hadoop Project POM ......................... SUCCESS [1.405s]
> [INFO] Apache Hadoop Annotations ......................... SUCCESS [6.452s]
> [INFO] Apache Hadoop Assemblies .......................... SUCCESS [0.637s]
> [INFO] Apache Hadoop Project Dist POM .................... SUCCESS [3.236s]
> [INFO] Apache Hadoop Auth ................................ SUCCESS [8.805s]
> [INFO] Apache Hadoop Auth Examples ....................... SUCCESS [5.105s]
> [INFO] Apache Hadoop Common .............................. SUCCESS [1:55.833s]
> [INFO] Apache Hadoop Common Project ...................... SUCCESS [0.210s]
> [INFO] Apache Hadoop HDFS ................................ SUCCESS [1:24.218s]
> [INFO] Apache Hadoop HttpFS .............................. SUCCESS [20.992s]
> [INFO] Apache Hadoop HDFS Project ........................ SUCCESS [0.127s]
> [INFO] hadoop-yarn ....................................... SUCCESS [0.598s]
> [INFO] hadoop-yarn-api ................................... SUCCESS [57.281s]
> [INFO] hadoop-yarn-common ................................ SUCCESS [1:01.264s]
> [INFO] hadoop-yarn-server ................................ SUCCESS [0.190s]
> [INFO] hadoop-yarn-server-common ......................... SUCCESS [19.280s]
> [INFO] hadoop-yarn-server-nodemanager .................... FAILURE [17.016s]
> [INFO] hadoop-yarn-server-web-proxy ...................... SKIPPED
> [INFO] hadoop-yarn-server-resourcemanager ................ SKIPPED
> [INFO] hadoop-yarn-server-tests .......................... SKIPPED
> [INFO] hadoop-mapreduce-client ........................... SKIPPED
> [INFO] hadoop-mapreduce-client-core ...................... SKIPPED
> [INFO] hadoop-yarn-applications .......................... SKIPPED
> [INFO] hadoop-yarn-applications-distributedshell ......... SKIPPED
> [INFO] hadoop-yarn-site .................................. SKIPPED
> [INFO] hadoop-mapreduce-client-common .................... SKIPPED
> [INFO] hadoop-mapreduce-client-shuffle ................... SKIPPED
> [INFO] hadoop-mapreduce-client-app ....................... SKIPPED
> [INFO] hadoop-mapreduce-client-hs ........................ SKIPPED
> [INFO] hadoop-mapreduce-client-jobclient ................. SKIPPED
> [INFO] Apache Hadoop MapReduce Examples .................. SKIPPED
> [INFO] hadoop-mapreduce .................................. SKIPPED
> [INFO] Apache Hadoop MapReduce Streaming ................. SKIPPED
> [INFO] Apache Hadoop Distributed Copy .................... SKIPPED
> [INFO] Apache Hadoop Archives ............................ SKIPPED
> [INFO] Apache Hadoop Rumen ............................... SKIPPED
> [INFO] Apache Hadoop Extras .............................. SKIPPED
> [INFO] Apache Hadoop Tools Dist .......................... SKIPPED
> [INFO] Apache Hadoop Tools ............................... SKIPPED
> [INFO] Apache Hadoop Distribution ........................ SKIPPED
> [INFO] Apache Hadoop Client .............................. SKIPPED
> [INFO] Apache Hadoop Mini-Cluster ........................ SKIPPED
> [INFO] ----------------------------------------------------------------------
>
> My doubts here are that is the if C compiler was not able to create the executables, the other projects which were build before node manager should have also failed, since they are also using the auto tools chain. I am able to build the entire source code if I build without native support, but I do not want to skip the native support build and I am trying to build the entire source code with native libraries support.
>
> My current OS is Fedora 17 64-bit and I have also got the same issue on CentOS 6.2 64-bit. I dont know if I am missing something for successful build with native libraries, and would request for the community's help for the same.
>
>
>
> Thanks and Regards
> Pankaj Misra
>
> ________________________________
>
> Impetus webinar: Designing a Test Automation Framework for Interoperable Systems; July 25 (10:00am PT). http://lf1.me/0E/
>
> Follow us on www.twitter.com/impetustech
>
>
> NOTE: This message may contain information that is confidential, proprietary, privileged or otherwise protected by law. The message is intended solely for the named addressee. If received in error, please destroy and notify the sender. Any use of this email is prohibited when received in error. Impetus does not represent, warrant and/or guarantee, that the integrity of this communication has been maintained nor that the communication is free of errors, virus, interception or interference.
--
Ioan Eugen Stan / CTO / http://axemblr.com
________________________________
Impetus webinar: Designing a Test Automation Framework for Interoperable Systems; July 25 (10:00am PT). http://lf1.me/0E/
Follow us on www.twitter.com/impetustech
NOTE: This message may contain information that is confidential, proprietary, privileged or otherwise protected by law. The message is intended solely for the named addressee. If received in error, please destroy and notify the sender. Any use of this email is prohibited when received in error. Impetus does not represent, warrant and/or guarantee, that the integrity of this communication has been maintained nor that the communication is free of errors, virus, interception or interference.
RE: Apache Hadoop 0.23.1 Source Build Failing
Posted by Pankaj Misra <pa...@impetus.co.in>.
Thanks Ioan for the help and sharing the link, appreciate it.
The symlink as specified below already exists, and the the response of "$ which ld" is
[root@fedora-0 container-executor]# which ld
/bin/ld
Yes, I will surely raise a JIRA for this issue if it does not get resolved, and once I am sure that I am not missing out anything.
Thanks and Regards
Pankaj Misra
________________________________________
From: Ioan Eugen Stan [stan.ieugen@gmail.com]
Sent: Thursday, August 09, 2012 4:06 PM
To: user@hadoop.apache.org
Subject: Re: Apache Hadoop 0.23.1 Source Build Failing
It seems that /bin/ld dos not exist so the compiler cannot perform
linking. Looking at the Fedora docs it seems that ld is located in
/usr/bin/ld so you may have to create a symlink to it:
$ ln -s /usr/bin/ld /bin/ld
First check that you have ld installed with: $ which ld
The scripts should also use `which ld` to find the proper path to ld.
So maybe you could raise an issue on JIRA with this.
http://docs.fedoraproject.org/en-US/Fedora/13/html/Release_Notes/sect-Release_Notes-The_GCC_Compiler_Collection.html
Cheers,
On Thu, Aug 9, 2012 at 1:05 PM, Pankaj Misra <pa...@impetus.co.in> wrote:
> Dear All,
>
> I am building hadoop 0.23.1 release from source with native support. I have already built/installed the following pre-requisites for native support
> 1. gcc-c++ 4.7.1
> 2. protoc 2.4.1
> 3. autotools chain
> 4. JDK 1.6.0_33
> 5. zlib 1.2.5-6
> 6. lzo 2.06-2
>
> I have also set the following variables and exported them
> export LD_LIBRARY_PATH=/usr/local/lib
>
> Other variables are also set as given below.
> export LD_LIBRARY_PATH=/usr/local/lib
> export JAVA_HOME=/usr/java/jdk1.6.0_33
> export ANT_HOME=/home/fedora/apache-ant-1.8.4
> export MAVEN_HOME=/home/fedora/apache-maven-3.0.4
> export PATH=$JAVA_HOME/bin:$ANT_HOME/bin:$MAVEN_HOME/bin:$PATH
> export HADOOP_COMMON_HOME=/home/fedora/hadoop/bin/release-0.23.1/hadoop-0.23.1
> export HADOOP_HDFS_HOME=$HADOOP_COMMON_HOME
> export YARN_HOME=$HADOOP_COMMON_HOME
> export HADOOP_MAPRED_HOME=$HADOOP_COMMON_HOME
> export HADOOP_CONF_DIR=$HADOOP_COMMON_HOME/etc/hadoop
>
>
> I am using the following to build the source with native support(using root user access).
> mvn clean install -Pdist -Pnative -DskipTests=true
>
> However, after building a number of sub-projects, the build fails at the nodemanager with the following error
> [INFO] Compiling 129 source files to /home/fedora/hadoop/src/release-0.23.1/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/target/classes
> [INFO]
> [INFO] --- make-maven-plugin:1.0-beta-1:autoreconf (compile) @ hadoop-yarn-server-nodemanager ---
> [INFO]
> [INFO] --- make-maven-plugin:1.0-beta-1:configure (compile) @ hadoop-yarn-server-nodemanager ---
> [INFO] checking for gcc... gcc
> [INFO] checking whether the C compiler works... no
> [INFO] configure: error: in `/home/fedora/hadoop/src/release-0.23.1/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/target/native/container-executor':
> [INFO] configure: error: C compiler cannot create executables
> [INFO] See `config.log' for more details
>
>
> The config.log mentions the following issues.
>
> configure:2562: checking whether the C compiler works
> configure:2584: gcc -DHADOOP_CONF_DIR=/etc/hadoop -m32 conftest.c >&5
> /bin/ld: cannot find crt1.o: No such file or directory
> /bin/ld: cannot find crti.o: No such file or directory
> /bin/ld: skipping incompatible /usr/lib/gcc/x86_64-redhat-linux/4.7.0/libgcc_s.so when searching for -lgcc_s
> /bin/ld: cannot find -lgcc_s
> /bin/ld: skipping incompatible /usr/lib64/libc.so when searching for -lc
> /bin/ld: cannot find -lc
> /bin/ld: skipping incompatible /usr/lib/gcc/x86_64-redhat-linux/4.7.0/libgcc_s.so when searching for -lgcc_s
> /bin/ld: cannot find -lgcc_s
> /bin/ld: cannot find crtn.o: No such file or directory
> collect2: error: ld returned 1 exit status
> configure:2588: $? = 1
> configure:2626: result: no
> configure: failed program was:
> | /* confdefs.h */
> | #define PACKAGE_NAME "linux-container-executor"
> | #define PACKAGE_TARNAME "linux-container-executor"
> | #define PACKAGE_VERSION "1.0.0"
> | #define PACKAGE_STRING "linux-container-executor 1.0.0"
> | #define PACKAGE_BUGREPORT "mapreduce-dev@hadoop.apache.org"
> | #define PACKAGE_URL ""
> | /* end confdefs.h. */
> |
> | int
> | main ()
> | {
> |
> | ;
> | return 0;
> | }
> configure:2631: error: in `/home/fedora/hadoop/src/release-0.23.1/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/target/native/container-executor':
> configure:2633: error: C compiler cannot create executables
>
> Overall build summary is as given below
> [INFO] Reactor Summary:
> [INFO]
> [INFO] Apache Hadoop Main ................................ SUCCESS [7.920s]
> [INFO] Apache Hadoop Project POM ......................... SUCCESS [1.405s]
> [INFO] Apache Hadoop Annotations ......................... SUCCESS [6.452s]
> [INFO] Apache Hadoop Assemblies .......................... SUCCESS [0.637s]
> [INFO] Apache Hadoop Project Dist POM .................... SUCCESS [3.236s]
> [INFO] Apache Hadoop Auth ................................ SUCCESS [8.805s]
> [INFO] Apache Hadoop Auth Examples ....................... SUCCESS [5.105s]
> [INFO] Apache Hadoop Common .............................. SUCCESS [1:55.833s]
> [INFO] Apache Hadoop Common Project ...................... SUCCESS [0.210s]
> [INFO] Apache Hadoop HDFS ................................ SUCCESS [1:24.218s]
> [INFO] Apache Hadoop HttpFS .............................. SUCCESS [20.992s]
> [INFO] Apache Hadoop HDFS Project ........................ SUCCESS [0.127s]
> [INFO] hadoop-yarn ....................................... SUCCESS [0.598s]
> [INFO] hadoop-yarn-api ................................... SUCCESS [57.281s]
> [INFO] hadoop-yarn-common ................................ SUCCESS [1:01.264s]
> [INFO] hadoop-yarn-server ................................ SUCCESS [0.190s]
> [INFO] hadoop-yarn-server-common ......................... SUCCESS [19.280s]
> [INFO] hadoop-yarn-server-nodemanager .................... FAILURE [17.016s]
> [INFO] hadoop-yarn-server-web-proxy ...................... SKIPPED
> [INFO] hadoop-yarn-server-resourcemanager ................ SKIPPED
> [INFO] hadoop-yarn-server-tests .......................... SKIPPED
> [INFO] hadoop-mapreduce-client ........................... SKIPPED
> [INFO] hadoop-mapreduce-client-core ...................... SKIPPED
> [INFO] hadoop-yarn-applications .......................... SKIPPED
> [INFO] hadoop-yarn-applications-distributedshell ......... SKIPPED
> [INFO] hadoop-yarn-site .................................. SKIPPED
> [INFO] hadoop-mapreduce-client-common .................... SKIPPED
> [INFO] hadoop-mapreduce-client-shuffle ................... SKIPPED
> [INFO] hadoop-mapreduce-client-app ....................... SKIPPED
> [INFO] hadoop-mapreduce-client-hs ........................ SKIPPED
> [INFO] hadoop-mapreduce-client-jobclient ................. SKIPPED
> [INFO] Apache Hadoop MapReduce Examples .................. SKIPPED
> [INFO] hadoop-mapreduce .................................. SKIPPED
> [INFO] Apache Hadoop MapReduce Streaming ................. SKIPPED
> [INFO] Apache Hadoop Distributed Copy .................... SKIPPED
> [INFO] Apache Hadoop Archives ............................ SKIPPED
> [INFO] Apache Hadoop Rumen ............................... SKIPPED
> [INFO] Apache Hadoop Extras .............................. SKIPPED
> [INFO] Apache Hadoop Tools Dist .......................... SKIPPED
> [INFO] Apache Hadoop Tools ............................... SKIPPED
> [INFO] Apache Hadoop Distribution ........................ SKIPPED
> [INFO] Apache Hadoop Client .............................. SKIPPED
> [INFO] Apache Hadoop Mini-Cluster ........................ SKIPPED
> [INFO] ----------------------------------------------------------------------
>
> My doubts here are that is the if C compiler was not able to create the executables, the other projects which were build before node manager should have also failed, since they are also using the auto tools chain. I am able to build the entire source code if I build without native support, but I do not want to skip the native support build and I am trying to build the entire source code with native libraries support.
>
> My current OS is Fedora 17 64-bit and I have also got the same issue on CentOS 6.2 64-bit. I dont know if I am missing something for successful build with native libraries, and would request for the community's help for the same.
>
>
>
> Thanks and Regards
> Pankaj Misra
>
> ________________________________
>
> Impetus webinar: Designing a Test Automation Framework for Interoperable Systems; July 25 (10:00am PT). http://lf1.me/0E/
>
> Follow us on www.twitter.com/impetustech
>
>
> NOTE: This message may contain information that is confidential, proprietary, privileged or otherwise protected by law. The message is intended solely for the named addressee. If received in error, please destroy and notify the sender. Any use of this email is prohibited when received in error. Impetus does not represent, warrant and/or guarantee, that the integrity of this communication has been maintained nor that the communication is free of errors, virus, interception or interference.
--
Ioan Eugen Stan / CTO / http://axemblr.com
________________________________
Impetus webinar: Designing a Test Automation Framework for Interoperable Systems; July 25 (10:00am PT). http://lf1.me/0E/
Follow us on www.twitter.com/impetustech
NOTE: This message may contain information that is confidential, proprietary, privileged or otherwise protected by law. The message is intended solely for the named addressee. If received in error, please destroy and notify the sender. Any use of this email is prohibited when received in error. Impetus does not represent, warrant and/or guarantee, that the integrity of this communication has been maintained nor that the communication is free of errors, virus, interception or interference.
Re: Apache Hadoop 0.23.1 Source Build Failing
Posted by Ioan Eugen Stan <st...@gmail.com>.
Hello Pankaj,
Sorry, I pressed send too fast. The linker can't find some libraries.
Check to see if you have them installed (locate, find, rpm). If not
install them and try again.
If you have them installed and ld can't find them, add the paths to
the proper environment variable (I think LD_LIBRARY_PATH). Check the
ld man page ( $ man ld) for details.
e.g.
$ export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/path/to/lib
http://linux.about.com/library/cmd/blcmdl1_ld.htm
I hope this helps,
On Thu, Aug 9, 2012 at 1:36 PM, Ioan Eugen Stan <st...@gmail.com> wrote:
> It seems that /bin/ld dos not exist so the compiler cannot perform
> linking. Looking at the Fedora docs it seems that ld is located in
> /usr/bin/ld so you may have to create a symlink to it:
>
> $ ln -s /usr/bin/ld /bin/ld
>
> First check that you have ld installed with: $ which ld
>
> The scripts should also use `which ld` to find the proper path to ld.
> So maybe you could raise an issue on JIRA with this.
>
> http://docs.fedoraproject.org/en-US/Fedora/13/html/Release_Notes/sect-Release_Notes-The_GCC_Compiler_Collection.html
>
> Cheers,
>
> On Thu, Aug 9, 2012 at 1:05 PM, Pankaj Misra <pa...@impetus.co.in> wrote:
>> Dear All,
>>
>> I am building hadoop 0.23.1 release from source with native support. I have already built/installed the following pre-requisites for native support
>> 1. gcc-c++ 4.7.1
>> 2. protoc 2.4.1
>> 3. autotools chain
>> 4. JDK 1.6.0_33
>> 5. zlib 1.2.5-6
>> 6. lzo 2.06-2
>>
>> I have also set the following variables and exported them
>> export LD_LIBRARY_PATH=/usr/local/lib
>>
>> Other variables are also set as given below.
>> export LD_LIBRARY_PATH=/usr/local/lib
>> export JAVA_HOME=/usr/java/jdk1.6.0_33
>> export ANT_HOME=/home/fedora/apache-ant-1.8.4
>> export MAVEN_HOME=/home/fedora/apache-maven-3.0.4
>> export PATH=$JAVA_HOME/bin:$ANT_HOME/bin:$MAVEN_HOME/bin:$PATH
>> export HADOOP_COMMON_HOME=/home/fedora/hadoop/bin/release-0.23.1/hadoop-0.23.1
>> export HADOOP_HDFS_HOME=$HADOOP_COMMON_HOME
>> export YARN_HOME=$HADOOP_COMMON_HOME
>> export HADOOP_MAPRED_HOME=$HADOOP_COMMON_HOME
>> export HADOOP_CONF_DIR=$HADOOP_COMMON_HOME/etc/hadoop
>>
>>
>> I am using the following to build the source with native support(using root user access).
>> mvn clean install -Pdist -Pnative -DskipTests=true
>>
>> However, after building a number of sub-projects, the build fails at the nodemanager with the following error
>> [INFO] Compiling 129 source files to /home/fedora/hadoop/src/release-0.23.1/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/target/classes
>> [INFO]
>> [INFO] --- make-maven-plugin:1.0-beta-1:autoreconf (compile) @ hadoop-yarn-server-nodemanager ---
>> [INFO]
>> [INFO] --- make-maven-plugin:1.0-beta-1:configure (compile) @ hadoop-yarn-server-nodemanager ---
>> [INFO] checking for gcc... gcc
>> [INFO] checking whether the C compiler works... no
>> [INFO] configure: error: in `/home/fedora/hadoop/src/release-0.23.1/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/target/native/container-executor':
>> [INFO] configure: error: C compiler cannot create executables
>> [INFO] See `config.log' for more details
>>
>>
>> The config.log mentions the following issues.
>>
>> configure:2562: checking whether the C compiler works
>> configure:2584: gcc -DHADOOP_CONF_DIR=/etc/hadoop -m32 conftest.c >&5
>> /bin/ld: cannot find crt1.o: No such file or directory
>> /bin/ld: cannot find crti.o: No such file or directory
>> /bin/ld: skipping incompatible /usr/lib/gcc/x86_64-redhat-linux/4.7.0/libgcc_s.so when searching for -lgcc_s
>> /bin/ld: cannot find -lgcc_s
>> /bin/ld: skipping incompatible /usr/lib64/libc.so when searching for -lc
>> /bin/ld: cannot find -lc
>> /bin/ld: skipping incompatible /usr/lib/gcc/x86_64-redhat-linux/4.7.0/libgcc_s.so when searching for -lgcc_s
>> /bin/ld: cannot find -lgcc_s
>> /bin/ld: cannot find crtn.o: No such file or directory
>> collect2: error: ld returned 1 exit status
>> configure:2588: $? = 1
>> configure:2626: result: no
>> configure: failed program was:
>> | /* confdefs.h */
>> | #define PACKAGE_NAME "linux-container-executor"
>> | #define PACKAGE_TARNAME "linux-container-executor"
>> | #define PACKAGE_VERSION "1.0.0"
>> | #define PACKAGE_STRING "linux-container-executor 1.0.0"
>> | #define PACKAGE_BUGREPORT "mapreduce-dev@hadoop.apache.org"
>> | #define PACKAGE_URL ""
>> | /* end confdefs.h. */
>> |
>> | int
>> | main ()
>> | {
>> |
>> | ;
>> | return 0;
>> | }
>> configure:2631: error: in `/home/fedora/hadoop/src/release-0.23.1/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/target/native/container-executor':
>> configure:2633: error: C compiler cannot create executables
>>
>> Overall build summary is as given below
>> [INFO] Reactor Summary:
>> [INFO]
>> [INFO] Apache Hadoop Main ................................ SUCCESS [7.920s]
>> [INFO] Apache Hadoop Project POM ......................... SUCCESS [1.405s]
>> [INFO] Apache Hadoop Annotations ......................... SUCCESS [6.452s]
>> [INFO] Apache Hadoop Assemblies .......................... SUCCESS [0.637s]
>> [INFO] Apache Hadoop Project Dist POM .................... SUCCESS [3.236s]
>> [INFO] Apache Hadoop Auth ................................ SUCCESS [8.805s]
>> [INFO] Apache Hadoop Auth Examples ....................... SUCCESS [5.105s]
>> [INFO] Apache Hadoop Common .............................. SUCCESS [1:55.833s]
>> [INFO] Apache Hadoop Common Project ...................... SUCCESS [0.210s]
>> [INFO] Apache Hadoop HDFS ................................ SUCCESS [1:24.218s]
>> [INFO] Apache Hadoop HttpFS .............................. SUCCESS [20.992s]
>> [INFO] Apache Hadoop HDFS Project ........................ SUCCESS [0.127s]
>> [INFO] hadoop-yarn ....................................... SUCCESS [0.598s]
>> [INFO] hadoop-yarn-api ................................... SUCCESS [57.281s]
>> [INFO] hadoop-yarn-common ................................ SUCCESS [1:01.264s]
>> [INFO] hadoop-yarn-server ................................ SUCCESS [0.190s]
>> [INFO] hadoop-yarn-server-common ......................... SUCCESS [19.280s]
>> [INFO] hadoop-yarn-server-nodemanager .................... FAILURE [17.016s]
>> [INFO] hadoop-yarn-server-web-proxy ...................... SKIPPED
>> [INFO] hadoop-yarn-server-resourcemanager ................ SKIPPED
>> [INFO] hadoop-yarn-server-tests .......................... SKIPPED
>> [INFO] hadoop-mapreduce-client ........................... SKIPPED
>> [INFO] hadoop-mapreduce-client-core ...................... SKIPPED
>> [INFO] hadoop-yarn-applications .......................... SKIPPED
>> [INFO] hadoop-yarn-applications-distributedshell ......... SKIPPED
>> [INFO] hadoop-yarn-site .................................. SKIPPED
>> [INFO] hadoop-mapreduce-client-common .................... SKIPPED
>> [INFO] hadoop-mapreduce-client-shuffle ................... SKIPPED
>> [INFO] hadoop-mapreduce-client-app ....................... SKIPPED
>> [INFO] hadoop-mapreduce-client-hs ........................ SKIPPED
>> [INFO] hadoop-mapreduce-client-jobclient ................. SKIPPED
>> [INFO] Apache Hadoop MapReduce Examples .................. SKIPPED
>> [INFO] hadoop-mapreduce .................................. SKIPPED
>> [INFO] Apache Hadoop MapReduce Streaming ................. SKIPPED
>> [INFO] Apache Hadoop Distributed Copy .................... SKIPPED
>> [INFO] Apache Hadoop Archives ............................ SKIPPED
>> [INFO] Apache Hadoop Rumen ............................... SKIPPED
>> [INFO] Apache Hadoop Extras .............................. SKIPPED
>> [INFO] Apache Hadoop Tools Dist .......................... SKIPPED
>> [INFO] Apache Hadoop Tools ............................... SKIPPED
>> [INFO] Apache Hadoop Distribution ........................ SKIPPED
>> [INFO] Apache Hadoop Client .............................. SKIPPED
>> [INFO] Apache Hadoop Mini-Cluster ........................ SKIPPED
>> [INFO] ----------------------------------------------------------------------
>>
>> My doubts here are that is the if C compiler was not able to create the executables, the other projects which were build before node manager should have also failed, since they are also using the auto tools chain. I am able to build the entire source code if I build without native support, but I do not want to skip the native support build and I am trying to build the entire source code with native libraries support.
>>
>> My current OS is Fedora 17 64-bit and I have also got the same issue on CentOS 6.2 64-bit. I dont know if I am missing something for successful build with native libraries, and would request for the community's help for the same.
>>
>>
>>
>> Thanks and Regards
>> Pankaj Misra
>>
>> ________________________________
>>
>> Impetus webinar: Designing a Test Automation Framework for Interoperable Systems; July 25 (10:00am PT). http://lf1.me/0E/
>>
>> Follow us on www.twitter.com/impetustech
>>
>>
>> NOTE: This message may contain information that is confidential, proprietary, privileged or otherwise protected by law. The message is intended solely for the named addressee. If received in error, please destroy and notify the sender. Any use of this email is prohibited when received in error. Impetus does not represent, warrant and/or guarantee, that the integrity of this communication has been maintained nor that the communication is free of errors, virus, interception or interference.
>
>
>
> --
> Ioan Eugen Stan / CTO / http://axemblr.com
--
Ioan Eugen Stan / CTO / http://axemblr.com
Re: Apache Hadoop 0.23.1 Source Build Failing
Posted by Ioan Eugen Stan <st...@gmail.com>.
It seems that /bin/ld dos not exist so the compiler cannot perform
linking. Looking at the Fedora docs it seems that ld is located in
/usr/bin/ld so you may have to create a symlink to it:
$ ln -s /usr/bin/ld /bin/ld
First check that you have ld installed with: $ which ld
The scripts should also use `which ld` to find the proper path to ld.
So maybe you could raise an issue on JIRA with this.
http://docs.fedoraproject.org/en-US/Fedora/13/html/Release_Notes/sect-Release_Notes-The_GCC_Compiler_Collection.html
Cheers,
On Thu, Aug 9, 2012 at 1:05 PM, Pankaj Misra <pa...@impetus.co.in> wrote:
> Dear All,
>
> I am building hadoop 0.23.1 release from source with native support. I have already built/installed the following pre-requisites for native support
> 1. gcc-c++ 4.7.1
> 2. protoc 2.4.1
> 3. autotools chain
> 4. JDK 1.6.0_33
> 5. zlib 1.2.5-6
> 6. lzo 2.06-2
>
> I have also set the following variables and exported them
> export LD_LIBRARY_PATH=/usr/local/lib
>
> Other variables are also set as given below.
> export LD_LIBRARY_PATH=/usr/local/lib
> export JAVA_HOME=/usr/java/jdk1.6.0_33
> export ANT_HOME=/home/fedora/apache-ant-1.8.4
> export MAVEN_HOME=/home/fedora/apache-maven-3.0.4
> export PATH=$JAVA_HOME/bin:$ANT_HOME/bin:$MAVEN_HOME/bin:$PATH
> export HADOOP_COMMON_HOME=/home/fedora/hadoop/bin/release-0.23.1/hadoop-0.23.1
> export HADOOP_HDFS_HOME=$HADOOP_COMMON_HOME
> export YARN_HOME=$HADOOP_COMMON_HOME
> export HADOOP_MAPRED_HOME=$HADOOP_COMMON_HOME
> export HADOOP_CONF_DIR=$HADOOP_COMMON_HOME/etc/hadoop
>
>
> I am using the following to build the source with native support(using root user access).
> mvn clean install -Pdist -Pnative -DskipTests=true
>
> However, after building a number of sub-projects, the build fails at the nodemanager with the following error
> [INFO] Compiling 129 source files to /home/fedora/hadoop/src/release-0.23.1/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/target/classes
> [INFO]
> [INFO] --- make-maven-plugin:1.0-beta-1:autoreconf (compile) @ hadoop-yarn-server-nodemanager ---
> [INFO]
> [INFO] --- make-maven-plugin:1.0-beta-1:configure (compile) @ hadoop-yarn-server-nodemanager ---
> [INFO] checking for gcc... gcc
> [INFO] checking whether the C compiler works... no
> [INFO] configure: error: in `/home/fedora/hadoop/src/release-0.23.1/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/target/native/container-executor':
> [INFO] configure: error: C compiler cannot create executables
> [INFO] See `config.log' for more details
>
>
> The config.log mentions the following issues.
>
> configure:2562: checking whether the C compiler works
> configure:2584: gcc -DHADOOP_CONF_DIR=/etc/hadoop -m32 conftest.c >&5
> /bin/ld: cannot find crt1.o: No such file or directory
> /bin/ld: cannot find crti.o: No such file or directory
> /bin/ld: skipping incompatible /usr/lib/gcc/x86_64-redhat-linux/4.7.0/libgcc_s.so when searching for -lgcc_s
> /bin/ld: cannot find -lgcc_s
> /bin/ld: skipping incompatible /usr/lib64/libc.so when searching for -lc
> /bin/ld: cannot find -lc
> /bin/ld: skipping incompatible /usr/lib/gcc/x86_64-redhat-linux/4.7.0/libgcc_s.so when searching for -lgcc_s
> /bin/ld: cannot find -lgcc_s
> /bin/ld: cannot find crtn.o: No such file or directory
> collect2: error: ld returned 1 exit status
> configure:2588: $? = 1
> configure:2626: result: no
> configure: failed program was:
> | /* confdefs.h */
> | #define PACKAGE_NAME "linux-container-executor"
> | #define PACKAGE_TARNAME "linux-container-executor"
> | #define PACKAGE_VERSION "1.0.0"
> | #define PACKAGE_STRING "linux-container-executor 1.0.0"
> | #define PACKAGE_BUGREPORT "mapreduce-dev@hadoop.apache.org"
> | #define PACKAGE_URL ""
> | /* end confdefs.h. */
> |
> | int
> | main ()
> | {
> |
> | ;
> | return 0;
> | }
> configure:2631: error: in `/home/fedora/hadoop/src/release-0.23.1/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/target/native/container-executor':
> configure:2633: error: C compiler cannot create executables
>
> Overall build summary is as given below
> [INFO] Reactor Summary:
> [INFO]
> [INFO] Apache Hadoop Main ................................ SUCCESS [7.920s]
> [INFO] Apache Hadoop Project POM ......................... SUCCESS [1.405s]
> [INFO] Apache Hadoop Annotations ......................... SUCCESS [6.452s]
> [INFO] Apache Hadoop Assemblies .......................... SUCCESS [0.637s]
> [INFO] Apache Hadoop Project Dist POM .................... SUCCESS [3.236s]
> [INFO] Apache Hadoop Auth ................................ SUCCESS [8.805s]
> [INFO] Apache Hadoop Auth Examples ....................... SUCCESS [5.105s]
> [INFO] Apache Hadoop Common .............................. SUCCESS [1:55.833s]
> [INFO] Apache Hadoop Common Project ...................... SUCCESS [0.210s]
> [INFO] Apache Hadoop HDFS ................................ SUCCESS [1:24.218s]
> [INFO] Apache Hadoop HttpFS .............................. SUCCESS [20.992s]
> [INFO] Apache Hadoop HDFS Project ........................ SUCCESS [0.127s]
> [INFO] hadoop-yarn ....................................... SUCCESS [0.598s]
> [INFO] hadoop-yarn-api ................................... SUCCESS [57.281s]
> [INFO] hadoop-yarn-common ................................ SUCCESS [1:01.264s]
> [INFO] hadoop-yarn-server ................................ SUCCESS [0.190s]
> [INFO] hadoop-yarn-server-common ......................... SUCCESS [19.280s]
> [INFO] hadoop-yarn-server-nodemanager .................... FAILURE [17.016s]
> [INFO] hadoop-yarn-server-web-proxy ...................... SKIPPED
> [INFO] hadoop-yarn-server-resourcemanager ................ SKIPPED
> [INFO] hadoop-yarn-server-tests .......................... SKIPPED
> [INFO] hadoop-mapreduce-client ........................... SKIPPED
> [INFO] hadoop-mapreduce-client-core ...................... SKIPPED
> [INFO] hadoop-yarn-applications .......................... SKIPPED
> [INFO] hadoop-yarn-applications-distributedshell ......... SKIPPED
> [INFO] hadoop-yarn-site .................................. SKIPPED
> [INFO] hadoop-mapreduce-client-common .................... SKIPPED
> [INFO] hadoop-mapreduce-client-shuffle ................... SKIPPED
> [INFO] hadoop-mapreduce-client-app ....................... SKIPPED
> [INFO] hadoop-mapreduce-client-hs ........................ SKIPPED
> [INFO] hadoop-mapreduce-client-jobclient ................. SKIPPED
> [INFO] Apache Hadoop MapReduce Examples .................. SKIPPED
> [INFO] hadoop-mapreduce .................................. SKIPPED
> [INFO] Apache Hadoop MapReduce Streaming ................. SKIPPED
> [INFO] Apache Hadoop Distributed Copy .................... SKIPPED
> [INFO] Apache Hadoop Archives ............................ SKIPPED
> [INFO] Apache Hadoop Rumen ............................... SKIPPED
> [INFO] Apache Hadoop Extras .............................. SKIPPED
> [INFO] Apache Hadoop Tools Dist .......................... SKIPPED
> [INFO] Apache Hadoop Tools ............................... SKIPPED
> [INFO] Apache Hadoop Distribution ........................ SKIPPED
> [INFO] Apache Hadoop Client .............................. SKIPPED
> [INFO] Apache Hadoop Mini-Cluster ........................ SKIPPED
> [INFO] ----------------------------------------------------------------------
>
> My doubts here are that is the if C compiler was not able to create the executables, the other projects which were build before node manager should have also failed, since they are also using the auto tools chain. I am able to build the entire source code if I build without native support, but I do not want to skip the native support build and I am trying to build the entire source code with native libraries support.
>
> My current OS is Fedora 17 64-bit and I have also got the same issue on CentOS 6.2 64-bit. I dont know if I am missing something for successful build with native libraries, and would request for the community's help for the same.
>
>
>
> Thanks and Regards
> Pankaj Misra
>
> ________________________________
>
> Impetus webinar: Designing a Test Automation Framework for Interoperable Systems; July 25 (10:00am PT). http://lf1.me/0E/
>
> Follow us on www.twitter.com/impetustech
>
>
> NOTE: This message may contain information that is confidential, proprietary, privileged or otherwise protected by law. The message is intended solely for the named addressee. If received in error, please destroy and notify the sender. Any use of this email is prohibited when received in error. Impetus does not represent, warrant and/or guarantee, that the integrity of this communication has been maintained nor that the communication is free of errors, virus, interception or interference.
--
Ioan Eugen Stan / CTO / http://axemblr.com
Re: Apache Hadoop 0.23.1 Source Build Failing
Posted by Ioan Eugen Stan <st...@gmail.com>.
It seems that /bin/ld dos not exist so the compiler cannot perform
linking. Looking at the Fedora docs it seems that ld is located in
/usr/bin/ld so you may have to create a symlink to it:
$ ln -s /usr/bin/ld /bin/ld
First check that you have ld installed with: $ which ld
The scripts should also use `which ld` to find the proper path to ld.
So maybe you could raise an issue on JIRA with this.
http://docs.fedoraproject.org/en-US/Fedora/13/html/Release_Notes/sect-Release_Notes-The_GCC_Compiler_Collection.html
Cheers,
On Thu, Aug 9, 2012 at 1:05 PM, Pankaj Misra <pa...@impetus.co.in> wrote:
> Dear All,
>
> I am building hadoop 0.23.1 release from source with native support. I have already built/installed the following pre-requisites for native support
> 1. gcc-c++ 4.7.1
> 2. protoc 2.4.1
> 3. autotools chain
> 4. JDK 1.6.0_33
> 5. zlib 1.2.5-6
> 6. lzo 2.06-2
>
> I have also set the following variables and exported them
> export LD_LIBRARY_PATH=/usr/local/lib
>
> Other variables are also set as given below.
> export LD_LIBRARY_PATH=/usr/local/lib
> export JAVA_HOME=/usr/java/jdk1.6.0_33
> export ANT_HOME=/home/fedora/apache-ant-1.8.4
> export MAVEN_HOME=/home/fedora/apache-maven-3.0.4
> export PATH=$JAVA_HOME/bin:$ANT_HOME/bin:$MAVEN_HOME/bin:$PATH
> export HADOOP_COMMON_HOME=/home/fedora/hadoop/bin/release-0.23.1/hadoop-0.23.1
> export HADOOP_HDFS_HOME=$HADOOP_COMMON_HOME
> export YARN_HOME=$HADOOP_COMMON_HOME
> export HADOOP_MAPRED_HOME=$HADOOP_COMMON_HOME
> export HADOOP_CONF_DIR=$HADOOP_COMMON_HOME/etc/hadoop
>
>
> I am using the following to build the source with native support(using root user access).
> mvn clean install -Pdist -Pnative -DskipTests=true
>
> However, after building a number of sub-projects, the build fails at the nodemanager with the following error
> [INFO] Compiling 129 source files to /home/fedora/hadoop/src/release-0.23.1/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/target/classes
> [INFO]
> [INFO] --- make-maven-plugin:1.0-beta-1:autoreconf (compile) @ hadoop-yarn-server-nodemanager ---
> [INFO]
> [INFO] --- make-maven-plugin:1.0-beta-1:configure (compile) @ hadoop-yarn-server-nodemanager ---
> [INFO] checking for gcc... gcc
> [INFO] checking whether the C compiler works... no
> [INFO] configure: error: in `/home/fedora/hadoop/src/release-0.23.1/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/target/native/container-executor':
> [INFO] configure: error: C compiler cannot create executables
> [INFO] See `config.log' for more details
>
>
> The config.log mentions the following issues.
>
> configure:2562: checking whether the C compiler works
> configure:2584: gcc -DHADOOP_CONF_DIR=/etc/hadoop -m32 conftest.c >&5
> /bin/ld: cannot find crt1.o: No such file or directory
> /bin/ld: cannot find crti.o: No such file or directory
> /bin/ld: skipping incompatible /usr/lib/gcc/x86_64-redhat-linux/4.7.0/libgcc_s.so when searching for -lgcc_s
> /bin/ld: cannot find -lgcc_s
> /bin/ld: skipping incompatible /usr/lib64/libc.so when searching for -lc
> /bin/ld: cannot find -lc
> /bin/ld: skipping incompatible /usr/lib/gcc/x86_64-redhat-linux/4.7.0/libgcc_s.so when searching for -lgcc_s
> /bin/ld: cannot find -lgcc_s
> /bin/ld: cannot find crtn.o: No such file or directory
> collect2: error: ld returned 1 exit status
> configure:2588: $? = 1
> configure:2626: result: no
> configure: failed program was:
> | /* confdefs.h */
> | #define PACKAGE_NAME "linux-container-executor"
> | #define PACKAGE_TARNAME "linux-container-executor"
> | #define PACKAGE_VERSION "1.0.0"
> | #define PACKAGE_STRING "linux-container-executor 1.0.0"
> | #define PACKAGE_BUGREPORT "mapreduce-dev@hadoop.apache.org"
> | #define PACKAGE_URL ""
> | /* end confdefs.h. */
> |
> | int
> | main ()
> | {
> |
> | ;
> | return 0;
> | }
> configure:2631: error: in `/home/fedora/hadoop/src/release-0.23.1/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/target/native/container-executor':
> configure:2633: error: C compiler cannot create executables
>
> Overall build summary is as given below
> [INFO] Reactor Summary:
> [INFO]
> [INFO] Apache Hadoop Main ................................ SUCCESS [7.920s]
> [INFO] Apache Hadoop Project POM ......................... SUCCESS [1.405s]
> [INFO] Apache Hadoop Annotations ......................... SUCCESS [6.452s]
> [INFO] Apache Hadoop Assemblies .......................... SUCCESS [0.637s]
> [INFO] Apache Hadoop Project Dist POM .................... SUCCESS [3.236s]
> [INFO] Apache Hadoop Auth ................................ SUCCESS [8.805s]
> [INFO] Apache Hadoop Auth Examples ....................... SUCCESS [5.105s]
> [INFO] Apache Hadoop Common .............................. SUCCESS [1:55.833s]
> [INFO] Apache Hadoop Common Project ...................... SUCCESS [0.210s]
> [INFO] Apache Hadoop HDFS ................................ SUCCESS [1:24.218s]
> [INFO] Apache Hadoop HttpFS .............................. SUCCESS [20.992s]
> [INFO] Apache Hadoop HDFS Project ........................ SUCCESS [0.127s]
> [INFO] hadoop-yarn ....................................... SUCCESS [0.598s]
> [INFO] hadoop-yarn-api ................................... SUCCESS [57.281s]
> [INFO] hadoop-yarn-common ................................ SUCCESS [1:01.264s]
> [INFO] hadoop-yarn-server ................................ SUCCESS [0.190s]
> [INFO] hadoop-yarn-server-common ......................... SUCCESS [19.280s]
> [INFO] hadoop-yarn-server-nodemanager .................... FAILURE [17.016s]
> [INFO] hadoop-yarn-server-web-proxy ...................... SKIPPED
> [INFO] hadoop-yarn-server-resourcemanager ................ SKIPPED
> [INFO] hadoop-yarn-server-tests .......................... SKIPPED
> [INFO] hadoop-mapreduce-client ........................... SKIPPED
> [INFO] hadoop-mapreduce-client-core ...................... SKIPPED
> [INFO] hadoop-yarn-applications .......................... SKIPPED
> [INFO] hadoop-yarn-applications-distributedshell ......... SKIPPED
> [INFO] hadoop-yarn-site .................................. SKIPPED
> [INFO] hadoop-mapreduce-client-common .................... SKIPPED
> [INFO] hadoop-mapreduce-client-shuffle ................... SKIPPED
> [INFO] hadoop-mapreduce-client-app ....................... SKIPPED
> [INFO] hadoop-mapreduce-client-hs ........................ SKIPPED
> [INFO] hadoop-mapreduce-client-jobclient ................. SKIPPED
> [INFO] Apache Hadoop MapReduce Examples .................. SKIPPED
> [INFO] hadoop-mapreduce .................................. SKIPPED
> [INFO] Apache Hadoop MapReduce Streaming ................. SKIPPED
> [INFO] Apache Hadoop Distributed Copy .................... SKIPPED
> [INFO] Apache Hadoop Archives ............................ SKIPPED
> [INFO] Apache Hadoop Rumen ............................... SKIPPED
> [INFO] Apache Hadoop Extras .............................. SKIPPED
> [INFO] Apache Hadoop Tools Dist .......................... SKIPPED
> [INFO] Apache Hadoop Tools ............................... SKIPPED
> [INFO] Apache Hadoop Distribution ........................ SKIPPED
> [INFO] Apache Hadoop Client .............................. SKIPPED
> [INFO] Apache Hadoop Mini-Cluster ........................ SKIPPED
> [INFO] ----------------------------------------------------------------------
>
> My doubts here are that is the if C compiler was not able to create the executables, the other projects which were build before node manager should have also failed, since they are also using the auto tools chain. I am able to build the entire source code if I build without native support, but I do not want to skip the native support build and I am trying to build the entire source code with native libraries support.
>
> My current OS is Fedora 17 64-bit and I have also got the same issue on CentOS 6.2 64-bit. I dont know if I am missing something for successful build with native libraries, and would request for the community's help for the same.
>
>
>
> Thanks and Regards
> Pankaj Misra
>
> ________________________________
>
> Impetus webinar: Designing a Test Automation Framework for Interoperable Systems; July 25 (10:00am PT). http://lf1.me/0E/
>
> Follow us on www.twitter.com/impetustech
>
>
> NOTE: This message may contain information that is confidential, proprietary, privileged or otherwise protected by law. The message is intended solely for the named addressee. If received in error, please destroy and notify the sender. Any use of this email is prohibited when received in error. Impetus does not represent, warrant and/or guarantee, that the integrity of this communication has been maintained nor that the communication is free of errors, virus, interception or interference.
--
Ioan Eugen Stan / CTO / http://axemblr.com
Re: Apache Hadoop 0.23.1 Source Build Failing
Posted by Ioan Eugen Stan <st...@gmail.com>.
It seems that /bin/ld dos not exist so the compiler cannot perform
linking. Looking at the Fedora docs it seems that ld is located in
/usr/bin/ld so you may have to create a symlink to it:
$ ln -s /usr/bin/ld /bin/ld
First check that you have ld installed with: $ which ld
The scripts should also use `which ld` to find the proper path to ld.
So maybe you could raise an issue on JIRA with this.
http://docs.fedoraproject.org/en-US/Fedora/13/html/Release_Notes/sect-Release_Notes-The_GCC_Compiler_Collection.html
Cheers,
On Thu, Aug 9, 2012 at 1:05 PM, Pankaj Misra <pa...@impetus.co.in> wrote:
> Dear All,
>
> I am building hadoop 0.23.1 release from source with native support. I have already built/installed the following pre-requisites for native support
> 1. gcc-c++ 4.7.1
> 2. protoc 2.4.1
> 3. autotools chain
> 4. JDK 1.6.0_33
> 5. zlib 1.2.5-6
> 6. lzo 2.06-2
>
> I have also set the following variables and exported them
> export LD_LIBRARY_PATH=/usr/local/lib
>
> Other variables are also set as given below.
> export LD_LIBRARY_PATH=/usr/local/lib
> export JAVA_HOME=/usr/java/jdk1.6.0_33
> export ANT_HOME=/home/fedora/apache-ant-1.8.4
> export MAVEN_HOME=/home/fedora/apache-maven-3.0.4
> export PATH=$JAVA_HOME/bin:$ANT_HOME/bin:$MAVEN_HOME/bin:$PATH
> export HADOOP_COMMON_HOME=/home/fedora/hadoop/bin/release-0.23.1/hadoop-0.23.1
> export HADOOP_HDFS_HOME=$HADOOP_COMMON_HOME
> export YARN_HOME=$HADOOP_COMMON_HOME
> export HADOOP_MAPRED_HOME=$HADOOP_COMMON_HOME
> export HADOOP_CONF_DIR=$HADOOP_COMMON_HOME/etc/hadoop
>
>
> I am using the following to build the source with native support(using root user access).
> mvn clean install -Pdist -Pnative -DskipTests=true
>
> However, after building a number of sub-projects, the build fails at the nodemanager with the following error
> [INFO] Compiling 129 source files to /home/fedora/hadoop/src/release-0.23.1/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/target/classes
> [INFO]
> [INFO] --- make-maven-plugin:1.0-beta-1:autoreconf (compile) @ hadoop-yarn-server-nodemanager ---
> [INFO]
> [INFO] --- make-maven-plugin:1.0-beta-1:configure (compile) @ hadoop-yarn-server-nodemanager ---
> [INFO] checking for gcc... gcc
> [INFO] checking whether the C compiler works... no
> [INFO] configure: error: in `/home/fedora/hadoop/src/release-0.23.1/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/target/native/container-executor':
> [INFO] configure: error: C compiler cannot create executables
> [INFO] See `config.log' for more details
>
>
> The config.log mentions the following issues.
>
> configure:2562: checking whether the C compiler works
> configure:2584: gcc -DHADOOP_CONF_DIR=/etc/hadoop -m32 conftest.c >&5
> /bin/ld: cannot find crt1.o: No such file or directory
> /bin/ld: cannot find crti.o: No such file or directory
> /bin/ld: skipping incompatible /usr/lib/gcc/x86_64-redhat-linux/4.7.0/libgcc_s.so when searching for -lgcc_s
> /bin/ld: cannot find -lgcc_s
> /bin/ld: skipping incompatible /usr/lib64/libc.so when searching for -lc
> /bin/ld: cannot find -lc
> /bin/ld: skipping incompatible /usr/lib/gcc/x86_64-redhat-linux/4.7.0/libgcc_s.so when searching for -lgcc_s
> /bin/ld: cannot find -lgcc_s
> /bin/ld: cannot find crtn.o: No such file or directory
> collect2: error: ld returned 1 exit status
> configure:2588: $? = 1
> configure:2626: result: no
> configure: failed program was:
> | /* confdefs.h */
> | #define PACKAGE_NAME "linux-container-executor"
> | #define PACKAGE_TARNAME "linux-container-executor"
> | #define PACKAGE_VERSION "1.0.0"
> | #define PACKAGE_STRING "linux-container-executor 1.0.0"
> | #define PACKAGE_BUGREPORT "mapreduce-dev@hadoop.apache.org"
> | #define PACKAGE_URL ""
> | /* end confdefs.h. */
> |
> | int
> | main ()
> | {
> |
> | ;
> | return 0;
> | }
> configure:2631: error: in `/home/fedora/hadoop/src/release-0.23.1/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/target/native/container-executor':
> configure:2633: error: C compiler cannot create executables
>
> Overall build summary is as given below
> [INFO] Reactor Summary:
> [INFO]
> [INFO] Apache Hadoop Main ................................ SUCCESS [7.920s]
> [INFO] Apache Hadoop Project POM ......................... SUCCESS [1.405s]
> [INFO] Apache Hadoop Annotations ......................... SUCCESS [6.452s]
> [INFO] Apache Hadoop Assemblies .......................... SUCCESS [0.637s]
> [INFO] Apache Hadoop Project Dist POM .................... SUCCESS [3.236s]
> [INFO] Apache Hadoop Auth ................................ SUCCESS [8.805s]
> [INFO] Apache Hadoop Auth Examples ....................... SUCCESS [5.105s]
> [INFO] Apache Hadoop Common .............................. SUCCESS [1:55.833s]
> [INFO] Apache Hadoop Common Project ...................... SUCCESS [0.210s]
> [INFO] Apache Hadoop HDFS ................................ SUCCESS [1:24.218s]
> [INFO] Apache Hadoop HttpFS .............................. SUCCESS [20.992s]
> [INFO] Apache Hadoop HDFS Project ........................ SUCCESS [0.127s]
> [INFO] hadoop-yarn ....................................... SUCCESS [0.598s]
> [INFO] hadoop-yarn-api ................................... SUCCESS [57.281s]
> [INFO] hadoop-yarn-common ................................ SUCCESS [1:01.264s]
> [INFO] hadoop-yarn-server ................................ SUCCESS [0.190s]
> [INFO] hadoop-yarn-server-common ......................... SUCCESS [19.280s]
> [INFO] hadoop-yarn-server-nodemanager .................... FAILURE [17.016s]
> [INFO] hadoop-yarn-server-web-proxy ...................... SKIPPED
> [INFO] hadoop-yarn-server-resourcemanager ................ SKIPPED
> [INFO] hadoop-yarn-server-tests .......................... SKIPPED
> [INFO] hadoop-mapreduce-client ........................... SKIPPED
> [INFO] hadoop-mapreduce-client-core ...................... SKIPPED
> [INFO] hadoop-yarn-applications .......................... SKIPPED
> [INFO] hadoop-yarn-applications-distributedshell ......... SKIPPED
> [INFO] hadoop-yarn-site .................................. SKIPPED
> [INFO] hadoop-mapreduce-client-common .................... SKIPPED
> [INFO] hadoop-mapreduce-client-shuffle ................... SKIPPED
> [INFO] hadoop-mapreduce-client-app ....................... SKIPPED
> [INFO] hadoop-mapreduce-client-hs ........................ SKIPPED
> [INFO] hadoop-mapreduce-client-jobclient ................. SKIPPED
> [INFO] Apache Hadoop MapReduce Examples .................. SKIPPED
> [INFO] hadoop-mapreduce .................................. SKIPPED
> [INFO] Apache Hadoop MapReduce Streaming ................. SKIPPED
> [INFO] Apache Hadoop Distributed Copy .................... SKIPPED
> [INFO] Apache Hadoop Archives ............................ SKIPPED
> [INFO] Apache Hadoop Rumen ............................... SKIPPED
> [INFO] Apache Hadoop Extras .............................. SKIPPED
> [INFO] Apache Hadoop Tools Dist .......................... SKIPPED
> [INFO] Apache Hadoop Tools ............................... SKIPPED
> [INFO] Apache Hadoop Distribution ........................ SKIPPED
> [INFO] Apache Hadoop Client .............................. SKIPPED
> [INFO] Apache Hadoop Mini-Cluster ........................ SKIPPED
> [INFO] ----------------------------------------------------------------------
>
> My doubts here are that is the if C compiler was not able to create the executables, the other projects which were build before node manager should have also failed, since they are also using the auto tools chain. I am able to build the entire source code if I build without native support, but I do not want to skip the native support build and I am trying to build the entire source code with native libraries support.
>
> My current OS is Fedora 17 64-bit and I have also got the same issue on CentOS 6.2 64-bit. I dont know if I am missing something for successful build with native libraries, and would request for the community's help for the same.
>
>
>
> Thanks and Regards
> Pankaj Misra
>
> ________________________________
>
> Impetus webinar: Designing a Test Automation Framework for Interoperable Systems; July 25 (10:00am PT). http://lf1.me/0E/
>
> Follow us on www.twitter.com/impetustech
>
>
> NOTE: This message may contain information that is confidential, proprietary, privileged or otherwise protected by law. The message is intended solely for the named addressee. If received in error, please destroy and notify the sender. Any use of this email is prohibited when received in error. Impetus does not represent, warrant and/or guarantee, that the integrity of this communication has been maintained nor that the communication is free of errors, virus, interception or interference.
--
Ioan Eugen Stan / CTO / http://axemblr.com
Re: Apache Hadoop 0.23.1 Source Build Failing
Posted by Ioan Eugen Stan <st...@gmail.com>.
It seems that /bin/ld dos not exist so the compiler cannot perform
linking. Looking at the Fedora docs it seems that ld is located in
/usr/bin/ld so you may have to create a symlink to it:
$ ln -s /usr/bin/ld /bin/ld
First check that you have ld installed with: $ which ld
The scripts should also use `which ld` to find the proper path to ld.
So maybe you could raise an issue on JIRA with this.
http://docs.fedoraproject.org/en-US/Fedora/13/html/Release_Notes/sect-Release_Notes-The_GCC_Compiler_Collection.html
Cheers,
On Thu, Aug 9, 2012 at 1:05 PM, Pankaj Misra <pa...@impetus.co.in> wrote:
> Dear All,
>
> I am building hadoop 0.23.1 release from source with native support. I have already built/installed the following pre-requisites for native support
> 1. gcc-c++ 4.7.1
> 2. protoc 2.4.1
> 3. autotools chain
> 4. JDK 1.6.0_33
> 5. zlib 1.2.5-6
> 6. lzo 2.06-2
>
> I have also set the following variables and exported them
> export LD_LIBRARY_PATH=/usr/local/lib
>
> Other variables are also set as given below.
> export LD_LIBRARY_PATH=/usr/local/lib
> export JAVA_HOME=/usr/java/jdk1.6.0_33
> export ANT_HOME=/home/fedora/apache-ant-1.8.4
> export MAVEN_HOME=/home/fedora/apache-maven-3.0.4
> export PATH=$JAVA_HOME/bin:$ANT_HOME/bin:$MAVEN_HOME/bin:$PATH
> export HADOOP_COMMON_HOME=/home/fedora/hadoop/bin/release-0.23.1/hadoop-0.23.1
> export HADOOP_HDFS_HOME=$HADOOP_COMMON_HOME
> export YARN_HOME=$HADOOP_COMMON_HOME
> export HADOOP_MAPRED_HOME=$HADOOP_COMMON_HOME
> export HADOOP_CONF_DIR=$HADOOP_COMMON_HOME/etc/hadoop
>
>
> I am using the following to build the source with native support(using root user access).
> mvn clean install -Pdist -Pnative -DskipTests=true
>
> However, after building a number of sub-projects, the build fails at the nodemanager with the following error
> [INFO] Compiling 129 source files to /home/fedora/hadoop/src/release-0.23.1/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/target/classes
> [INFO]
> [INFO] --- make-maven-plugin:1.0-beta-1:autoreconf (compile) @ hadoop-yarn-server-nodemanager ---
> [INFO]
> [INFO] --- make-maven-plugin:1.0-beta-1:configure (compile) @ hadoop-yarn-server-nodemanager ---
> [INFO] checking for gcc... gcc
> [INFO] checking whether the C compiler works... no
> [INFO] configure: error: in `/home/fedora/hadoop/src/release-0.23.1/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/target/native/container-executor':
> [INFO] configure: error: C compiler cannot create executables
> [INFO] See `config.log' for more details
>
>
> The config.log mentions the following issues.
>
> configure:2562: checking whether the C compiler works
> configure:2584: gcc -DHADOOP_CONF_DIR=/etc/hadoop -m32 conftest.c >&5
> /bin/ld: cannot find crt1.o: No such file or directory
> /bin/ld: cannot find crti.o: No such file or directory
> /bin/ld: skipping incompatible /usr/lib/gcc/x86_64-redhat-linux/4.7.0/libgcc_s.so when searching for -lgcc_s
> /bin/ld: cannot find -lgcc_s
> /bin/ld: skipping incompatible /usr/lib64/libc.so when searching for -lc
> /bin/ld: cannot find -lc
> /bin/ld: skipping incompatible /usr/lib/gcc/x86_64-redhat-linux/4.7.0/libgcc_s.so when searching for -lgcc_s
> /bin/ld: cannot find -lgcc_s
> /bin/ld: cannot find crtn.o: No such file or directory
> collect2: error: ld returned 1 exit status
> configure:2588: $? = 1
> configure:2626: result: no
> configure: failed program was:
> | /* confdefs.h */
> | #define PACKAGE_NAME "linux-container-executor"
> | #define PACKAGE_TARNAME "linux-container-executor"
> | #define PACKAGE_VERSION "1.0.0"
> | #define PACKAGE_STRING "linux-container-executor 1.0.0"
> | #define PACKAGE_BUGREPORT "mapreduce-dev@hadoop.apache.org"
> | #define PACKAGE_URL ""
> | /* end confdefs.h. */
> |
> | int
> | main ()
> | {
> |
> | ;
> | return 0;
> | }
> configure:2631: error: in `/home/fedora/hadoop/src/release-0.23.1/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/target/native/container-executor':
> configure:2633: error: C compiler cannot create executables
>
> Overall build summary is as given below
> [INFO] Reactor Summary:
> [INFO]
> [INFO] Apache Hadoop Main ................................ SUCCESS [7.920s]
> [INFO] Apache Hadoop Project POM ......................... SUCCESS [1.405s]
> [INFO] Apache Hadoop Annotations ......................... SUCCESS [6.452s]
> [INFO] Apache Hadoop Assemblies .......................... SUCCESS [0.637s]
> [INFO] Apache Hadoop Project Dist POM .................... SUCCESS [3.236s]
> [INFO] Apache Hadoop Auth ................................ SUCCESS [8.805s]
> [INFO] Apache Hadoop Auth Examples ....................... SUCCESS [5.105s]
> [INFO] Apache Hadoop Common .............................. SUCCESS [1:55.833s]
> [INFO] Apache Hadoop Common Project ...................... SUCCESS [0.210s]
> [INFO] Apache Hadoop HDFS ................................ SUCCESS [1:24.218s]
> [INFO] Apache Hadoop HttpFS .............................. SUCCESS [20.992s]
> [INFO] Apache Hadoop HDFS Project ........................ SUCCESS [0.127s]
> [INFO] hadoop-yarn ....................................... SUCCESS [0.598s]
> [INFO] hadoop-yarn-api ................................... SUCCESS [57.281s]
> [INFO] hadoop-yarn-common ................................ SUCCESS [1:01.264s]
> [INFO] hadoop-yarn-server ................................ SUCCESS [0.190s]
> [INFO] hadoop-yarn-server-common ......................... SUCCESS [19.280s]
> [INFO] hadoop-yarn-server-nodemanager .................... FAILURE [17.016s]
> [INFO] hadoop-yarn-server-web-proxy ...................... SKIPPED
> [INFO] hadoop-yarn-server-resourcemanager ................ SKIPPED
> [INFO] hadoop-yarn-server-tests .......................... SKIPPED
> [INFO] hadoop-mapreduce-client ........................... SKIPPED
> [INFO] hadoop-mapreduce-client-core ...................... SKIPPED
> [INFO] hadoop-yarn-applications .......................... SKIPPED
> [INFO] hadoop-yarn-applications-distributedshell ......... SKIPPED
> [INFO] hadoop-yarn-site .................................. SKIPPED
> [INFO] hadoop-mapreduce-client-common .................... SKIPPED
> [INFO] hadoop-mapreduce-client-shuffle ................... SKIPPED
> [INFO] hadoop-mapreduce-client-app ....................... SKIPPED
> [INFO] hadoop-mapreduce-client-hs ........................ SKIPPED
> [INFO] hadoop-mapreduce-client-jobclient ................. SKIPPED
> [INFO] Apache Hadoop MapReduce Examples .................. SKIPPED
> [INFO] hadoop-mapreduce .................................. SKIPPED
> [INFO] Apache Hadoop MapReduce Streaming ................. SKIPPED
> [INFO] Apache Hadoop Distributed Copy .................... SKIPPED
> [INFO] Apache Hadoop Archives ............................ SKIPPED
> [INFO] Apache Hadoop Rumen ............................... SKIPPED
> [INFO] Apache Hadoop Extras .............................. SKIPPED
> [INFO] Apache Hadoop Tools Dist .......................... SKIPPED
> [INFO] Apache Hadoop Tools ............................... SKIPPED
> [INFO] Apache Hadoop Distribution ........................ SKIPPED
> [INFO] Apache Hadoop Client .............................. SKIPPED
> [INFO] Apache Hadoop Mini-Cluster ........................ SKIPPED
> [INFO] ----------------------------------------------------------------------
>
> My doubts here are that is the if C compiler was not able to create the executables, the other projects which were build before node manager should have also failed, since they are also using the auto tools chain. I am able to build the entire source code if I build without native support, but I do not want to skip the native support build and I am trying to build the entire source code with native libraries support.
>
> My current OS is Fedora 17 64-bit and I have also got the same issue on CentOS 6.2 64-bit. I dont know if I am missing something for successful build with native libraries, and would request for the community's help for the same.
>
>
>
> Thanks and Regards
> Pankaj Misra
>
> ________________________________
>
> Impetus webinar: Designing a Test Automation Framework for Interoperable Systems; July 25 (10:00am PT). http://lf1.me/0E/
>
> Follow us on www.twitter.com/impetustech
>
>
> NOTE: This message may contain information that is confidential, proprietary, privileged or otherwise protected by law. The message is intended solely for the named addressee. If received in error, please destroy and notify the sender. Any use of this email is prohibited when received in error. Impetus does not represent, warrant and/or guarantee, that the integrity of this communication has been maintained nor that the communication is free of errors, virus, interception or interference.
--
Ioan Eugen Stan / CTO / http://axemblr.com