You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@hadoop.apache.org by Salman Toor <sa...@it.uu.se> on 2013/11/05 12:58:08 UTC
Re: C++ example for hadoop-2.2.0
Hi,
I have solved the problem! Have downloaded the source, compiled it on the machine and now can successfully link to the library.
Thank you everyone for you quick responses.
Regards..
Salman.
Salman Toor, PhD
salman.toor@it.uu.se
On Nov 4, 2013, at 3:21 PM, Andre Kelpe wrote:
> No, because I was trying to set up a cluster automatically with the
> tarballs from apache.org.
>
> - André
>
> On Mon, Nov 4, 2013 at 3:05 PM, Salman Toor <sa...@it.uu.se> wrote:
>> Hi,
>>
>> Did you tried to compile with source?
>>
>> /Salman.
>>
>>
>> Salman Toor, PhD
>> salman.toor@it.uu.se
>>
>>
>>
>> On Nov 4, 2013, at 2:55 PM, Andre Kelpe wrote:
>>
>> I reported the 32bit/64bit problem a few weeks ago. There hasn't been
>> much activity around it though:
>> https://issues.apache.org/jira/browse/HADOOP-9911
>>
>> - André
>>
>> On Mon, Nov 4, 2013 at 2:20 PM, Salman Toor <sa...@it.uu.se> wrote:
>>
>> Hi,
>>
>>
>> Ok so 2.x is not a new version its another branch. Good to know! Actually
>>
>> 32bit will be difficult as the code I got have already some dependencies on
>>
>> 64 bit.
>>
>>
>> Otherwise I will continue with 1.x version. Can you suggest some version
>>
>> with 1.x series which is stable and work on the cluster environment?
>>
>> especially with C++ ...
>>
>>
>> Regards..
>>
>> Salman.
>>
>>
>> Salman Toor, PhD
>>
>> salman.toor@it.uu.se
>>
>>
>>
>>
>> On Nov 4, 2013, at 1:54 PM, Amr Shahin wrote:
>>
>>
>> Well, the 2 series isn't exactly the "next version". It's a continuation of
>>
>> branch .2.
>>
>> Also, the error message from the gcc indicates that the library you're
>>
>> trying to link to isn't compatible which made me suspicious. check the
>>
>> documentation to see if hadoop has 64 libraries, or otherwise compile
>>
>> against the 32 ones
>>
>>
>>
>> On Mon, Nov 4, 2013 at 4:51 PM, Salman Toor <sa...@it.uu.se> wrote:
>>
>>
>> Hi,
>>
>>
>> Thanks for your answer!
>>
>>
>> But are you sure about it? Actually Hadoop version 1.2 have both 32 and 64
>>
>> bit libraries so I believe the the next version should have both... But I am
>>
>> not sure just a random guess :-(
>>
>>
>> Regards..
>>
>> Salman.
>>
>>
>> Salman Toor, PhD
>>
>> salman.toor@it.uu.se
>>
>>
>>
>>
>> On Nov 4, 2013, at 1:38 PM, Amr Shahin wrote:
>>
>>
>> I believe hadoop isn't compatible with 64 architecture. Try installing the
>>
>> 32 libraries and compile against them.
>>
>> This error (skipping incompatible
>>
>> /home/sztoor/hadoop-2.2.0/lib/native/libhadooppipes.a when searching
>>
>> -lhadooppipes) indicates so
>>
>>
>>
>> On Mon, Nov 4, 2013 at 2:44 PM, Salman Toor <sa...@it.uu.se> wrote:
>>
>>
>> Hi,
>>
>>
>> Can someone give a pointer?
>>
>>
>>
>> Thanks in advance.
>>
>>
>> Regards..
>>
>> Salman.
>>
>>
>>
>> Salman Toor, PhD
>>
>> salman.toor@it.uu.se
>>
>>
>>
>>
>> On Nov 3, 2013, at 11:31 PM, Salman Toor wrote:
>>
>>
>> Hi,
>>
>>
>> I am quite new to the Hadoop world, previously was running hadoop-1.2.0
>>
>> stable version on my small cluster and encountered some strange problems
>>
>> like the local path to the mapper file didn't copy to the hdfs .... It
>>
>> works fine on the single node setup but on multiple node simple word-count
>>
>> python example didn't work... I read on the blog that it might be the
>>
>> problem in the version I am using. So I thought to change the version and
>>
>> downloaded the Hadoop 2.2.0. This version has yarn together with many new
>>
>> features that I hope I will learn in the future. Now simple wordcount
>>
>> example works without any problem on the multi-node setup. I am using simple
>>
>> python example.
>>
>>
>> Now I would like to compile my C++ code. Since the directory structure
>>
>> together with other things have been changed. I have started to get the
>>
>> following error:
>>
>>
>> ----------------
>>
>> /urs/bin/ld: skipping incompatible
>>
>> /home/sztoor/hadoop-2.2.0/lib/native/libhadooputils.a when searching
>>
>> -lhadooputils
>>
>> cannot find -lhadooputils
>>
>>
>> /urs/bin/ld: skipping incompatible
>>
>> /home/sztoor/hadoop-2.2.0/lib/native/libhadooppipes.a when searching
>>
>> -lhadooppipes
>>
>> cannot find -lhadooppipes
>>
>> ------------------
>>
>>
>> I have managed to run the c++ example successfully with the 1.2.0 version
>>
>> on single node setup.
>>
>>
>> I am having 64bit Ubuntu machine. previously I was using Linux-amd64-64
>>
>>
>> Now in new version, "lib" and "include" directories are in the
>>
>> hadoop-2.2.0 directory. No build.xml is available...
>>
>>
>> Can someone please give me an example of a makefile based on the version
>>
>> 2.2.0? Or suggest me which version I should go for? Or if there are some
>>
>> prerequisites that I should do before compiling my code?
>>
>>
>> Thanks in advance.
>>
>>
>> Regards..
>>
>> Salman.
>>
>>
>>
>>
>> Salman Toor, PhD
>>
>> salman.toor@it.uu.se
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>> --
>> André Kelpe
>> andre@concurrentinc.com
>> http://concurrentinc.com
>>
>>
>
>
>
> --
> André Kelpe
> andre@concurrentinc.com
> http://concurrentinc.com
Re: C++ example for hadoop-2.2.0
Posted by Salman Toor <sa...@it.uu.se>.
Hi,
Sorry for the late response.
As I have mentioned earlier I am quite new to the system so have very primitive knowledge.
Basically I have followed the following blog:
http://jugnu-life.blogspot.com/2013/09/build-and-compile-hadoop-from-source.html
except the following line in the end
# mvn package -Pdist -DskipTests -Dtar
I have used following command
# mvn package -Pdist,native,docs -DskipTests -Dtar
And than in "hadoop-dist/target" directory you can find "hadoop-2.2.0" which will contain the right native libraries.
Regards..
Salman.
Salman Toor, PhD
salman.toor@it.uu.se
On Nov 5, 2013, at 9:42 PM, jerome wrote:
> Hi Salman
>
> I've trying to compile too te obtain the good library. But i've fac some compile issue. Could you please, send me how you compile the sources?
>
> Reagards
>
> On Tue, 5 Nov 2013 12:58:08 +0100, Salman Toor <sa...@it.uu.se> wrote:
>> Hi,
>>
>> I have solved the problem! Have downloaded the source, compiled it on
>> the machine and now can successfully link to the library.
>>
>> Thank you everyone for you quick responses.
>>
>> Regards..
>> Salman.
>>
>> Salman Toor, PhD
>> salman.toor@it.uu.se [1]
>>
>> On Nov 4, 2013, at 3:21 PM, Andre Kelpe wrote:
>>
>> No, because I was trying to set up a cluster automatically with the
>> tarballs from apache.org [2].
>>
>> - André
>>
>> On Mon, Nov 4, 2013 at 3:05 PM, Salman Toor wrote:
>> Hi,
>>
>> Did you tried to compile with source?
>>
>> /Salman.
>>
>> Salman Toor, PhD
>> salman.toor@it.uu.se [4]
>>
>> On Nov 4, 2013, at 2:55 PM, Andre Kelpe wrote:
>>
>> I reported the 32bit/64bit problem a few weeks ago. There hasn't been
>> much activity around it though:
>> https://issues.apache.org/jira/browse/HADOOP-9911 [5]
>>
>> - André
>>
>> On Mon, Nov 4, 2013 at 2:20 PM, Salman Toor wrote:
>>
>> Hi,
>>
>> Ok so 2.x is not a new version its another branch. Good to know!
>> Actually
>>
>> 32bit will be difficult as the code I got have already some
>> dependencies on
>>
>> 64 bit.
>>
>> Otherwise I will continue with 1.x version. Can you suggest some
>> version
>>
>> with 1.x series which is stable and work on the cluster environment?
>>
>> especially with C++ ...
>>
>> Regards..
>>
>> Salman.
>>
>> Salman Toor, PhD
>>
>> salman.toor@it.uu.se [7]
>>
>> On Nov 4, 2013, at 1:54 PM, Amr Shahin wrote:
>>
>> Well, the 2 series isn't exactly the "next version". It's a
>> continuation of
>>
>> branch .2.
>>
>> Also, the error message from the gcc indicates that the library
>> you're
>>
>> trying to link to isn't compatible which made me suspicious. check
>> the
>>
>> documentation to see if hadoop has 64 libraries, or otherwise compile
>>
>> against the 32 ones
>>
>> On Mon, Nov 4, 2013 at 4:51 PM, Salman Toor wrote:
>>
>> Hi,
>>
>> Thanks for your answer!
>>
>> But are you sure about it? Actually Hadoop version 1.2 have both 32
>> and 64
>>
>> bit libraries so I believe the the next version should have both...
>> But I am
>>
>> not sure just a random guess :-(
>>
>> Regards..
>>
>> Salman.
>>
>> Salman Toor, PhD
>>
>> salman.toor@it.uu.se [9]
>>
>> On Nov 4, 2013, at 1:38 PM, Amr Shahin wrote:
>>
>> I believe hadoop isn't compatible with 64 architecture. Try
>> installing the
>>
>> 32 libraries and compile against them.
>>
>> This error (skipping incompatible
>>
>> /home/sztoor/hadoop-2.2.0/lib/native/libhadooppipes.a when searching
>>
>> -lhadooppipes) indicates so
>>
>> On Mon, Nov 4, 2013 at 2:44 PM, Salman Toor wrote:
>>
>> Hi,
>>
>> Can someone give a pointer?
>>
>> Thanks in advance.
>>
>> Regards..
>>
>> Salman.
>>
>> Salman Toor, PhD
>>
>> salman.toor@it.uu.se [11]
>>
>> On Nov 3, 2013, at 11:31 PM, Salman Toor wrote:
>>
>> Hi,
>>
>> I am quite new to the Hadoop world, previously was running
>> hadoop-1.2.0
>>
>> stable version on my small cluster and encountered some strange
>> problems
>>
>> like the local path to the mapper file didn't copy to the hdfs ....
>> It
>>
>> works fine on the single node setup but on multiple node simple
>> word-count
>>
>> python example didn't work... I read on the blog that it might be the
>>
>> problem in the version I am using. So I thought to change the version
>> and
>>
>> downloaded the Hadoop 2.2.0. This version has yarn together with many
>> new
>>
>> features that I hope I will learn in the future. Now simple wordcount
>>
>> example works without any problem on the multi-node setup. I am using
>> simple
>>
>> python example.
>>
>> Now I would like to compile my C++ code. Since the directory
>> structure
>>
>> together with other things have been changed. I have started to get
>> the
>>
>> following error:
>>
>> ----------------
>>
>> /urs/bin/ld: skipping incompatible
>>
>> /home/sztoor/hadoop-2.2.0/lib/native/libhadooputils.a when searching
>>
>> -lhadooputils
>>
>> cannot find -lhadooputils
>>
>> /urs/bin/ld: skipping incompatible
>>
>> /home/sztoor/hadoop-2.2.0/lib/native/libhadooppipes.a when searching
>>
>> -lhadooppipes
>>
>> cannot find -lhadooppipes
>>
>> ------------------
>>
>> I have managed to run the c++ example successfully with the 1.2.0
>> version
>>
>> on single node setup.
>>
>> I am having 64bit Ubuntu machine. previously I was using
>> Linux-amd64-64
>>
>> Now in new version, "lib" and "include" directories are in the
>>
>> hadoop-2.2.0 directory. No build.xml is available...
>>
>> Can someone please give me an example of a makefile based on the
>> version
>>
>> 2.2.0? Or suggest me which version I should go for? Or if there are
>> some
>>
>> prerequisites that I should do before compiling my code?
>>
>> Thanks in advance.
>>
>> Regards..
>>
>> Salman.
>>
>> Salman Toor, PhD
>>
>> salman.toor@it.uu.se [12]
>>
>> --
>> André Kelpe
>> andre@concurrentinc.com [13]
>> http://concurrentinc.com [14]
>>
>> --
>> André Kelpe
>> andre@concurrentinc.com [15]
>> http://concurrentinc.com
>>
>>
>>
>> Links:
>> ------
>> [1] mailto:salman.toor@it.uu.se
>> [2] http://apache.org
>> [3] mailto:salman.toor@it.uu.se
>> [4] mailto:salman.toor@it.uu.se
>> [5] https://issues.apache.org/jira/browse/HADOOP-9911
>> [6] mailto:salman.toor@it.uu.se
>> [7] mailto:salman.toor@it.uu.se
>> [8] mailto:salman.toor@it.uu.se
>> [9] mailto:salman.toor@it.uu.se
>> [10] mailto:salman.toor@it.uu.se
>> [11] mailto:salman.toor@it.uu.se
>> [12] mailto:salman.toor@it.uu.se
>> [13] mailto:andre@concurrentinc.com
>> [14] http://concurrentinc.com
>> [15] mailto:andre@concurrentinc.com
>
Re: C++ example for hadoop-2.2.0
Posted by Salman Toor <sa...@it.uu.se>.
Hi,
Sorry for the late response.
As I have mentioned earlier I am quite new to the system so have very primitive knowledge.
Basically I have followed the following blog:
http://jugnu-life.blogspot.com/2013/09/build-and-compile-hadoop-from-source.html
except the following line in the end
# mvn package -Pdist -DskipTests -Dtar
I have used following command
# mvn package -Pdist,native,docs -DskipTests -Dtar
And than in "hadoop-dist/target" directory you can find "hadoop-2.2.0" which will contain the right native libraries.
Regards..
Salman.
Salman Toor, PhD
salman.toor@it.uu.se
On Nov 5, 2013, at 9:42 PM, jerome wrote:
> Hi Salman
>
> I've trying to compile too te obtain the good library. But i've fac some compile issue. Could you please, send me how you compile the sources?
>
> Reagards
>
> On Tue, 5 Nov 2013 12:58:08 +0100, Salman Toor <sa...@it.uu.se> wrote:
>> Hi,
>>
>> I have solved the problem! Have downloaded the source, compiled it on
>> the machine and now can successfully link to the library.
>>
>> Thank you everyone for you quick responses.
>>
>> Regards..
>> Salman.
>>
>> Salman Toor, PhD
>> salman.toor@it.uu.se [1]
>>
>> On Nov 4, 2013, at 3:21 PM, Andre Kelpe wrote:
>>
>> No, because I was trying to set up a cluster automatically with the
>> tarballs from apache.org [2].
>>
>> - André
>>
>> On Mon, Nov 4, 2013 at 3:05 PM, Salman Toor wrote:
>> Hi,
>>
>> Did you tried to compile with source?
>>
>> /Salman.
>>
>> Salman Toor, PhD
>> salman.toor@it.uu.se [4]
>>
>> On Nov 4, 2013, at 2:55 PM, Andre Kelpe wrote:
>>
>> I reported the 32bit/64bit problem a few weeks ago. There hasn't been
>> much activity around it though:
>> https://issues.apache.org/jira/browse/HADOOP-9911 [5]
>>
>> - André
>>
>> On Mon, Nov 4, 2013 at 2:20 PM, Salman Toor wrote:
>>
>> Hi,
>>
>> Ok so 2.x is not a new version its another branch. Good to know!
>> Actually
>>
>> 32bit will be difficult as the code I got have already some
>> dependencies on
>>
>> 64 bit.
>>
>> Otherwise I will continue with 1.x version. Can you suggest some
>> version
>>
>> with 1.x series which is stable and work on the cluster environment?
>>
>> especially with C++ ...
>>
>> Regards..
>>
>> Salman.
>>
>> Salman Toor, PhD
>>
>> salman.toor@it.uu.se [7]
>>
>> On Nov 4, 2013, at 1:54 PM, Amr Shahin wrote:
>>
>> Well, the 2 series isn't exactly the "next version". It's a
>> continuation of
>>
>> branch .2.
>>
>> Also, the error message from the gcc indicates that the library
>> you're
>>
>> trying to link to isn't compatible which made me suspicious. check
>> the
>>
>> documentation to see if hadoop has 64 libraries, or otherwise compile
>>
>> against the 32 ones
>>
>> On Mon, Nov 4, 2013 at 4:51 PM, Salman Toor wrote:
>>
>> Hi,
>>
>> Thanks for your answer!
>>
>> But are you sure about it? Actually Hadoop version 1.2 have both 32
>> and 64
>>
>> bit libraries so I believe the the next version should have both...
>> But I am
>>
>> not sure just a random guess :-(
>>
>> Regards..
>>
>> Salman.
>>
>> Salman Toor, PhD
>>
>> salman.toor@it.uu.se [9]
>>
>> On Nov 4, 2013, at 1:38 PM, Amr Shahin wrote:
>>
>> I believe hadoop isn't compatible with 64 architecture. Try
>> installing the
>>
>> 32 libraries and compile against them.
>>
>> This error (skipping incompatible
>>
>> /home/sztoor/hadoop-2.2.0/lib/native/libhadooppipes.a when searching
>>
>> -lhadooppipes) indicates so
>>
>> On Mon, Nov 4, 2013 at 2:44 PM, Salman Toor wrote:
>>
>> Hi,
>>
>> Can someone give a pointer?
>>
>> Thanks in advance.
>>
>> Regards..
>>
>> Salman.
>>
>> Salman Toor, PhD
>>
>> salman.toor@it.uu.se [11]
>>
>> On Nov 3, 2013, at 11:31 PM, Salman Toor wrote:
>>
>> Hi,
>>
>> I am quite new to the Hadoop world, previously was running
>> hadoop-1.2.0
>>
>> stable version on my small cluster and encountered some strange
>> problems
>>
>> like the local path to the mapper file didn't copy to the hdfs ....
>> It
>>
>> works fine on the single node setup but on multiple node simple
>> word-count
>>
>> python example didn't work... I read on the blog that it might be the
>>
>> problem in the version I am using. So I thought to change the version
>> and
>>
>> downloaded the Hadoop 2.2.0. This version has yarn together with many
>> new
>>
>> features that I hope I will learn in the future. Now simple wordcount
>>
>> example works without any problem on the multi-node setup. I am using
>> simple
>>
>> python example.
>>
>> Now I would like to compile my C++ code. Since the directory
>> structure
>>
>> together with other things have been changed. I have started to get
>> the
>>
>> following error:
>>
>> ----------------
>>
>> /urs/bin/ld: skipping incompatible
>>
>> /home/sztoor/hadoop-2.2.0/lib/native/libhadooputils.a when searching
>>
>> -lhadooputils
>>
>> cannot find -lhadooputils
>>
>> /urs/bin/ld: skipping incompatible
>>
>> /home/sztoor/hadoop-2.2.0/lib/native/libhadooppipes.a when searching
>>
>> -lhadooppipes
>>
>> cannot find -lhadooppipes
>>
>> ------------------
>>
>> I have managed to run the c++ example successfully with the 1.2.0
>> version
>>
>> on single node setup.
>>
>> I am having 64bit Ubuntu machine. previously I was using
>> Linux-amd64-64
>>
>> Now in new version, "lib" and "include" directories are in the
>>
>> hadoop-2.2.0 directory. No build.xml is available...
>>
>> Can someone please give me an example of a makefile based on the
>> version
>>
>> 2.2.0? Or suggest me which version I should go for? Or if there are
>> some
>>
>> prerequisites that I should do before compiling my code?
>>
>> Thanks in advance.
>>
>> Regards..
>>
>> Salman.
>>
>> Salman Toor, PhD
>>
>> salman.toor@it.uu.se [12]
>>
>> --
>> André Kelpe
>> andre@concurrentinc.com [13]
>> http://concurrentinc.com [14]
>>
>> --
>> André Kelpe
>> andre@concurrentinc.com [15]
>> http://concurrentinc.com
>>
>>
>>
>> Links:
>> ------
>> [1] mailto:salman.toor@it.uu.se
>> [2] http://apache.org
>> [3] mailto:salman.toor@it.uu.se
>> [4] mailto:salman.toor@it.uu.se
>> [5] https://issues.apache.org/jira/browse/HADOOP-9911
>> [6] mailto:salman.toor@it.uu.se
>> [7] mailto:salman.toor@it.uu.se
>> [8] mailto:salman.toor@it.uu.se
>> [9] mailto:salman.toor@it.uu.se
>> [10] mailto:salman.toor@it.uu.se
>> [11] mailto:salman.toor@it.uu.se
>> [12] mailto:salman.toor@it.uu.se
>> [13] mailto:andre@concurrentinc.com
>> [14] http://concurrentinc.com
>> [15] mailto:andre@concurrentinc.com
>
Re: C++ example for hadoop-2.2.0
Posted by Salman Toor <sa...@it.uu.se>.
Hi,
Sorry for the late response.
As I have mentioned earlier I am quite new to the system so have very primitive knowledge.
Basically I have followed the following blog:
http://jugnu-life.blogspot.com/2013/09/build-and-compile-hadoop-from-source.html
except the following line in the end
# mvn package -Pdist -DskipTests -Dtar
I have used following command
# mvn package -Pdist,native,docs -DskipTests -Dtar
And than in "hadoop-dist/target" directory you can find "hadoop-2.2.0" which will contain the right native libraries.
Regards..
Salman.
Salman Toor, PhD
salman.toor@it.uu.se
On Nov 5, 2013, at 9:42 PM, jerome wrote:
> Hi Salman
>
> I've trying to compile too te obtain the good library. But i've fac some compile issue. Could you please, send me how you compile the sources?
>
> Reagards
>
> On Tue, 5 Nov 2013 12:58:08 +0100, Salman Toor <sa...@it.uu.se> wrote:
>> Hi,
>>
>> I have solved the problem! Have downloaded the source, compiled it on
>> the machine and now can successfully link to the library.
>>
>> Thank you everyone for you quick responses.
>>
>> Regards..
>> Salman.
>>
>> Salman Toor, PhD
>> salman.toor@it.uu.se [1]
>>
>> On Nov 4, 2013, at 3:21 PM, Andre Kelpe wrote:
>>
>> No, because I was trying to set up a cluster automatically with the
>> tarballs from apache.org [2].
>>
>> - André
>>
>> On Mon, Nov 4, 2013 at 3:05 PM, Salman Toor wrote:
>> Hi,
>>
>> Did you tried to compile with source?
>>
>> /Salman.
>>
>> Salman Toor, PhD
>> salman.toor@it.uu.se [4]
>>
>> On Nov 4, 2013, at 2:55 PM, Andre Kelpe wrote:
>>
>> I reported the 32bit/64bit problem a few weeks ago. There hasn't been
>> much activity around it though:
>> https://issues.apache.org/jira/browse/HADOOP-9911 [5]
>>
>> - André
>>
>> On Mon, Nov 4, 2013 at 2:20 PM, Salman Toor wrote:
>>
>> Hi,
>>
>> Ok so 2.x is not a new version its another branch. Good to know!
>> Actually
>>
>> 32bit will be difficult as the code I got have already some
>> dependencies on
>>
>> 64 bit.
>>
>> Otherwise I will continue with 1.x version. Can you suggest some
>> version
>>
>> with 1.x series which is stable and work on the cluster environment?
>>
>> especially with C++ ...
>>
>> Regards..
>>
>> Salman.
>>
>> Salman Toor, PhD
>>
>> salman.toor@it.uu.se [7]
>>
>> On Nov 4, 2013, at 1:54 PM, Amr Shahin wrote:
>>
>> Well, the 2 series isn't exactly the "next version". It's a
>> continuation of
>>
>> branch .2.
>>
>> Also, the error message from the gcc indicates that the library
>> you're
>>
>> trying to link to isn't compatible which made me suspicious. check
>> the
>>
>> documentation to see if hadoop has 64 libraries, or otherwise compile
>>
>> against the 32 ones
>>
>> On Mon, Nov 4, 2013 at 4:51 PM, Salman Toor wrote:
>>
>> Hi,
>>
>> Thanks for your answer!
>>
>> But are you sure about it? Actually Hadoop version 1.2 have both 32
>> and 64
>>
>> bit libraries so I believe the the next version should have both...
>> But I am
>>
>> not sure just a random guess :-(
>>
>> Regards..
>>
>> Salman.
>>
>> Salman Toor, PhD
>>
>> salman.toor@it.uu.se [9]
>>
>> On Nov 4, 2013, at 1:38 PM, Amr Shahin wrote:
>>
>> I believe hadoop isn't compatible with 64 architecture. Try
>> installing the
>>
>> 32 libraries and compile against them.
>>
>> This error (skipping incompatible
>>
>> /home/sztoor/hadoop-2.2.0/lib/native/libhadooppipes.a when searching
>>
>> -lhadooppipes) indicates so
>>
>> On Mon, Nov 4, 2013 at 2:44 PM, Salman Toor wrote:
>>
>> Hi,
>>
>> Can someone give a pointer?
>>
>> Thanks in advance.
>>
>> Regards..
>>
>> Salman.
>>
>> Salman Toor, PhD
>>
>> salman.toor@it.uu.se [11]
>>
>> On Nov 3, 2013, at 11:31 PM, Salman Toor wrote:
>>
>> Hi,
>>
>> I am quite new to the Hadoop world, previously was running
>> hadoop-1.2.0
>>
>> stable version on my small cluster and encountered some strange
>> problems
>>
>> like the local path to the mapper file didn't copy to the hdfs ....
>> It
>>
>> works fine on the single node setup but on multiple node simple
>> word-count
>>
>> python example didn't work... I read on the blog that it might be the
>>
>> problem in the version I am using. So I thought to change the version
>> and
>>
>> downloaded the Hadoop 2.2.0. This version has yarn together with many
>> new
>>
>> features that I hope I will learn in the future. Now simple wordcount
>>
>> example works without any problem on the multi-node setup. I am using
>> simple
>>
>> python example.
>>
>> Now I would like to compile my C++ code. Since the directory
>> structure
>>
>> together with other things have been changed. I have started to get
>> the
>>
>> following error:
>>
>> ----------------
>>
>> /urs/bin/ld: skipping incompatible
>>
>> /home/sztoor/hadoop-2.2.0/lib/native/libhadooputils.a when searching
>>
>> -lhadooputils
>>
>> cannot find -lhadooputils
>>
>> /urs/bin/ld: skipping incompatible
>>
>> /home/sztoor/hadoop-2.2.0/lib/native/libhadooppipes.a when searching
>>
>> -lhadooppipes
>>
>> cannot find -lhadooppipes
>>
>> ------------------
>>
>> I have managed to run the c++ example successfully with the 1.2.0
>> version
>>
>> on single node setup.
>>
>> I am having 64bit Ubuntu machine. previously I was using
>> Linux-amd64-64
>>
>> Now in new version, "lib" and "include" directories are in the
>>
>> hadoop-2.2.0 directory. No build.xml is available...
>>
>> Can someone please give me an example of a makefile based on the
>> version
>>
>> 2.2.0? Or suggest me which version I should go for? Or if there are
>> some
>>
>> prerequisites that I should do before compiling my code?
>>
>> Thanks in advance.
>>
>> Regards..
>>
>> Salman.
>>
>> Salman Toor, PhD
>>
>> salman.toor@it.uu.se [12]
>>
>> --
>> André Kelpe
>> andre@concurrentinc.com [13]
>> http://concurrentinc.com [14]
>>
>> --
>> André Kelpe
>> andre@concurrentinc.com [15]
>> http://concurrentinc.com
>>
>>
>>
>> Links:
>> ------
>> [1] mailto:salman.toor@it.uu.se
>> [2] http://apache.org
>> [3] mailto:salman.toor@it.uu.se
>> [4] mailto:salman.toor@it.uu.se
>> [5] https://issues.apache.org/jira/browse/HADOOP-9911
>> [6] mailto:salman.toor@it.uu.se
>> [7] mailto:salman.toor@it.uu.se
>> [8] mailto:salman.toor@it.uu.se
>> [9] mailto:salman.toor@it.uu.se
>> [10] mailto:salman.toor@it.uu.se
>> [11] mailto:salman.toor@it.uu.se
>> [12] mailto:salman.toor@it.uu.se
>> [13] mailto:andre@concurrentinc.com
>> [14] http://concurrentinc.com
>> [15] mailto:andre@concurrentinc.com
>
Re: C++ example for hadoop-2.2.0
Posted by Salman Toor <sa...@it.uu.se>.
Hi,
Sorry for the late response.
As I have mentioned earlier I am quite new to the system so have very primitive knowledge.
Basically I have followed the following blog:
http://jugnu-life.blogspot.com/2013/09/build-and-compile-hadoop-from-source.html
except the following line in the end
# mvn package -Pdist -DskipTests -Dtar
I have used following command
# mvn package -Pdist,native,docs -DskipTests -Dtar
And than in "hadoop-dist/target" directory you can find "hadoop-2.2.0" which will contain the right native libraries.
Regards..
Salman.
Salman Toor, PhD
salman.toor@it.uu.se
On Nov 5, 2013, at 9:42 PM, jerome wrote:
> Hi Salman
>
> I've trying to compile too te obtain the good library. But i've fac some compile issue. Could you please, send me how you compile the sources?
>
> Reagards
>
> On Tue, 5 Nov 2013 12:58:08 +0100, Salman Toor <sa...@it.uu.se> wrote:
>> Hi,
>>
>> I have solved the problem! Have downloaded the source, compiled it on
>> the machine and now can successfully link to the library.
>>
>> Thank you everyone for you quick responses.
>>
>> Regards..
>> Salman.
>>
>> Salman Toor, PhD
>> salman.toor@it.uu.se [1]
>>
>> On Nov 4, 2013, at 3:21 PM, Andre Kelpe wrote:
>>
>> No, because I was trying to set up a cluster automatically with the
>> tarballs from apache.org [2].
>>
>> - André
>>
>> On Mon, Nov 4, 2013 at 3:05 PM, Salman Toor wrote:
>> Hi,
>>
>> Did you tried to compile with source?
>>
>> /Salman.
>>
>> Salman Toor, PhD
>> salman.toor@it.uu.se [4]
>>
>> On Nov 4, 2013, at 2:55 PM, Andre Kelpe wrote:
>>
>> I reported the 32bit/64bit problem a few weeks ago. There hasn't been
>> much activity around it though:
>> https://issues.apache.org/jira/browse/HADOOP-9911 [5]
>>
>> - André
>>
>> On Mon, Nov 4, 2013 at 2:20 PM, Salman Toor wrote:
>>
>> Hi,
>>
>> Ok so 2.x is not a new version its another branch. Good to know!
>> Actually
>>
>> 32bit will be difficult as the code I got have already some
>> dependencies on
>>
>> 64 bit.
>>
>> Otherwise I will continue with 1.x version. Can you suggest some
>> version
>>
>> with 1.x series which is stable and work on the cluster environment?
>>
>> especially with C++ ...
>>
>> Regards..
>>
>> Salman.
>>
>> Salman Toor, PhD
>>
>> salman.toor@it.uu.se [7]
>>
>> On Nov 4, 2013, at 1:54 PM, Amr Shahin wrote:
>>
>> Well, the 2 series isn't exactly the "next version". It's a
>> continuation of
>>
>> branch .2.
>>
>> Also, the error message from the gcc indicates that the library
>> you're
>>
>> trying to link to isn't compatible which made me suspicious. check
>> the
>>
>> documentation to see if hadoop has 64 libraries, or otherwise compile
>>
>> against the 32 ones
>>
>> On Mon, Nov 4, 2013 at 4:51 PM, Salman Toor wrote:
>>
>> Hi,
>>
>> Thanks for your answer!
>>
>> But are you sure about it? Actually Hadoop version 1.2 have both 32
>> and 64
>>
>> bit libraries so I believe the the next version should have both...
>> But I am
>>
>> not sure just a random guess :-(
>>
>> Regards..
>>
>> Salman.
>>
>> Salman Toor, PhD
>>
>> salman.toor@it.uu.se [9]
>>
>> On Nov 4, 2013, at 1:38 PM, Amr Shahin wrote:
>>
>> I believe hadoop isn't compatible with 64 architecture. Try
>> installing the
>>
>> 32 libraries and compile against them.
>>
>> This error (skipping incompatible
>>
>> /home/sztoor/hadoop-2.2.0/lib/native/libhadooppipes.a when searching
>>
>> -lhadooppipes) indicates so
>>
>> On Mon, Nov 4, 2013 at 2:44 PM, Salman Toor wrote:
>>
>> Hi,
>>
>> Can someone give a pointer?
>>
>> Thanks in advance.
>>
>> Regards..
>>
>> Salman.
>>
>> Salman Toor, PhD
>>
>> salman.toor@it.uu.se [11]
>>
>> On Nov 3, 2013, at 11:31 PM, Salman Toor wrote:
>>
>> Hi,
>>
>> I am quite new to the Hadoop world, previously was running
>> hadoop-1.2.0
>>
>> stable version on my small cluster and encountered some strange
>> problems
>>
>> like the local path to the mapper file didn't copy to the hdfs ....
>> It
>>
>> works fine on the single node setup but on multiple node simple
>> word-count
>>
>> python example didn't work... I read on the blog that it might be the
>>
>> problem in the version I am using. So I thought to change the version
>> and
>>
>> downloaded the Hadoop 2.2.0. This version has yarn together with many
>> new
>>
>> features that I hope I will learn in the future. Now simple wordcount
>>
>> example works without any problem on the multi-node setup. I am using
>> simple
>>
>> python example.
>>
>> Now I would like to compile my C++ code. Since the directory
>> structure
>>
>> together with other things have been changed. I have started to get
>> the
>>
>> following error:
>>
>> ----------------
>>
>> /urs/bin/ld: skipping incompatible
>>
>> /home/sztoor/hadoop-2.2.0/lib/native/libhadooputils.a when searching
>>
>> -lhadooputils
>>
>> cannot find -lhadooputils
>>
>> /urs/bin/ld: skipping incompatible
>>
>> /home/sztoor/hadoop-2.2.0/lib/native/libhadooppipes.a when searching
>>
>> -lhadooppipes
>>
>> cannot find -lhadooppipes
>>
>> ------------------
>>
>> I have managed to run the c++ example successfully with the 1.2.0
>> version
>>
>> on single node setup.
>>
>> I am having 64bit Ubuntu machine. previously I was using
>> Linux-amd64-64
>>
>> Now in new version, "lib" and "include" directories are in the
>>
>> hadoop-2.2.0 directory. No build.xml is available...
>>
>> Can someone please give me an example of a makefile based on the
>> version
>>
>> 2.2.0? Or suggest me which version I should go for? Or if there are
>> some
>>
>> prerequisites that I should do before compiling my code?
>>
>> Thanks in advance.
>>
>> Regards..
>>
>> Salman.
>>
>> Salman Toor, PhD
>>
>> salman.toor@it.uu.se [12]
>>
>> --
>> André Kelpe
>> andre@concurrentinc.com [13]
>> http://concurrentinc.com [14]
>>
>> --
>> André Kelpe
>> andre@concurrentinc.com [15]
>> http://concurrentinc.com
>>
>>
>>
>> Links:
>> ------
>> [1] mailto:salman.toor@it.uu.se
>> [2] http://apache.org
>> [3] mailto:salman.toor@it.uu.se
>> [4] mailto:salman.toor@it.uu.se
>> [5] https://issues.apache.org/jira/browse/HADOOP-9911
>> [6] mailto:salman.toor@it.uu.se
>> [7] mailto:salman.toor@it.uu.se
>> [8] mailto:salman.toor@it.uu.se
>> [9] mailto:salman.toor@it.uu.se
>> [10] mailto:salman.toor@it.uu.se
>> [11] mailto:salman.toor@it.uu.se
>> [12] mailto:salman.toor@it.uu.se
>> [13] mailto:andre@concurrentinc.com
>> [14] http://concurrentinc.com
>> [15] mailto:andre@concurrentinc.com
>
Re: C++ example for hadoop-2.2.0
Posted by jerome <je...@ibt.unam.mx>.
Hi Salman
I've trying to compile too te obtain the good library. But i've fac
some compile issue. Could you please, send me how you compile the
sources?
Reagards
On Tue, 5 Nov 2013 12:58:08 +0100, Salman Toor <sa...@it.uu.se>
wrote:
> Hi,
>
> I have solved the problem! Have downloaded the source, compiled it on
> the machine and now can successfully link to the library.
>
> Thank you everyone for you quick responses.
>
> Regards..
> Salman.
>
> Salman Toor, PhD
> salman.toor@it.uu.se [1]
>
> On Nov 4, 2013, at 3:21 PM, Andre Kelpe wrote:
>
> No, because I was trying to set up a cluster automatically with the
> tarballs from apache.org [2].
>
> - André
>
> On Mon, Nov 4, 2013 at 3:05 PM, Salman Toor wrote:
> Hi,
>
> Did you tried to compile with source?
>
> /Salman.
>
> Salman Toor, PhD
> salman.toor@it.uu.se [4]
>
> On Nov 4, 2013, at 2:55 PM, Andre Kelpe wrote:
>
> I reported the 32bit/64bit problem a few weeks ago. There hasn't been
> much activity around it though:
> https://issues.apache.org/jira/browse/HADOOP-9911 [5]
>
> - André
>
> On Mon, Nov 4, 2013 at 2:20 PM, Salman Toor wrote:
>
> Hi,
>
> Ok so 2.x is not a new version its another branch. Good to know!
> Actually
>
> 32bit will be difficult as the code I got have already some
> dependencies on
>
> 64 bit.
>
> Otherwise I will continue with 1.x version. Can you suggest some
> version
>
> with 1.x series which is stable and work on the cluster environment?
>
> especially with C++ ...
>
> Regards..
>
> Salman.
>
> Salman Toor, PhD
>
> salman.toor@it.uu.se [7]
>
> On Nov 4, 2013, at 1:54 PM, Amr Shahin wrote:
>
> Well, the 2 series isn't exactly the "next version". It's a
> continuation of
>
> branch .2.
>
> Also, the error message from the gcc indicates that the library
> you're
>
> trying to link to isn't compatible which made me suspicious. check
> the
>
> documentation to see if hadoop has 64 libraries, or otherwise compile
>
> against the 32 ones
>
> On Mon, Nov 4, 2013 at 4:51 PM, Salman Toor wrote:
>
> Hi,
>
> Thanks for your answer!
>
> But are you sure about it? Actually Hadoop version 1.2 have both 32
> and 64
>
> bit libraries so I believe the the next version should have both...
> But I am
>
> not sure just a random guess :-(
>
> Regards..
>
> Salman.
>
> Salman Toor, PhD
>
> salman.toor@it.uu.se [9]
>
> On Nov 4, 2013, at 1:38 PM, Amr Shahin wrote:
>
> I believe hadoop isn't compatible with 64 architecture. Try
> installing the
>
> 32 libraries and compile against them.
>
> This error (skipping incompatible
>
> /home/sztoor/hadoop-2.2.0/lib/native/libhadooppipes.a when searching
>
> -lhadooppipes) indicates so
>
> On Mon, Nov 4, 2013 at 2:44 PM, Salman Toor wrote:
>
> Hi,
>
> Can someone give a pointer?
>
> Thanks in advance.
>
> Regards..
>
> Salman.
>
> Salman Toor, PhD
>
> salman.toor@it.uu.se [11]
>
> On Nov 3, 2013, at 11:31 PM, Salman Toor wrote:
>
> Hi,
>
> I am quite new to the Hadoop world, previously was running
> hadoop-1.2.0
>
> stable version on my small cluster and encountered some strange
> problems
>
> like the local path to the mapper file didn't copy to the hdfs ....
> It
>
> works fine on the single node setup but on multiple node simple
> word-count
>
> python example didn't work... I read on the blog that it might be the
>
> problem in the version I am using. So I thought to change the version
> and
>
> downloaded the Hadoop 2.2.0. This version has yarn together with many
> new
>
> features that I hope I will learn in the future. Now simple wordcount
>
> example works without any problem on the multi-node setup. I am using
> simple
>
> python example.
>
> Now I would like to compile my C++ code. Since the directory
> structure
>
> together with other things have been changed. I have started to get
> the
>
> following error:
>
> ----------------
>
> /urs/bin/ld: skipping incompatible
>
> /home/sztoor/hadoop-2.2.0/lib/native/libhadooputils.a when searching
>
> -lhadooputils
>
> cannot find -lhadooputils
>
> /urs/bin/ld: skipping incompatible
>
> /home/sztoor/hadoop-2.2.0/lib/native/libhadooppipes.a when searching
>
> -lhadooppipes
>
> cannot find -lhadooppipes
>
> ------------------
>
> I have managed to run the c++ example successfully with the 1.2.0
> version
>
> on single node setup.
>
> I am having 64bit Ubuntu machine. previously I was using
> Linux-amd64-64
>
> Now in new version, "lib" and "include" directories are in the
>
> hadoop-2.2.0 directory. No build.xml is available...
>
> Can someone please give me an example of a makefile based on the
> version
>
> 2.2.0? Or suggest me which version I should go for? Or if there are
> some
>
> prerequisites that I should do before compiling my code?
>
> Thanks in advance.
>
> Regards..
>
> Salman.
>
> Salman Toor, PhD
>
> salman.toor@it.uu.se [12]
>
> --
> André Kelpe
> andre@concurrentinc.com [13]
> http://concurrentinc.com [14]
>
> --
> André Kelpe
> andre@concurrentinc.com [15]
> http://concurrentinc.com
>
>
>
> Links:
> ------
> [1] mailto:salman.toor@it.uu.se
> [2] http://apache.org
> [3] mailto:salman.toor@it.uu.se
> [4] mailto:salman.toor@it.uu.se
> [5] https://issues.apache.org/jira/browse/HADOOP-9911
> [6] mailto:salman.toor@it.uu.se
> [7] mailto:salman.toor@it.uu.se
> [8] mailto:salman.toor@it.uu.se
> [9] mailto:salman.toor@it.uu.se
> [10] mailto:salman.toor@it.uu.se
> [11] mailto:salman.toor@it.uu.se
> [12] mailto:salman.toor@it.uu.se
> [13] mailto:andre@concurrentinc.com
> [14] http://concurrentinc.com
> [15] mailto:andre@concurrentinc.com
Re: C++ example for hadoop-2.2.0
Posted by jerome <je...@ibt.unam.mx>.
Hi Salman
I've trying to compile too te obtain the good library. But i've fac
some compile issue. Could you please, send me how you compile the
sources?
Reagards
On Tue, 5 Nov 2013 12:58:08 +0100, Salman Toor <sa...@it.uu.se>
wrote:
> Hi,
>
> I have solved the problem! Have downloaded the source, compiled it on
> the machine and now can successfully link to the library.
>
> Thank you everyone for you quick responses.
>
> Regards..
> Salman.
>
> Salman Toor, PhD
> salman.toor@it.uu.se [1]
>
> On Nov 4, 2013, at 3:21 PM, Andre Kelpe wrote:
>
> No, because I was trying to set up a cluster automatically with the
> tarballs from apache.org [2].
>
> - André
>
> On Mon, Nov 4, 2013 at 3:05 PM, Salman Toor wrote:
> Hi,
>
> Did you tried to compile with source?
>
> /Salman.
>
> Salman Toor, PhD
> salman.toor@it.uu.se [4]
>
> On Nov 4, 2013, at 2:55 PM, Andre Kelpe wrote:
>
> I reported the 32bit/64bit problem a few weeks ago. There hasn't been
> much activity around it though:
> https://issues.apache.org/jira/browse/HADOOP-9911 [5]
>
> - André
>
> On Mon, Nov 4, 2013 at 2:20 PM, Salman Toor wrote:
>
> Hi,
>
> Ok so 2.x is not a new version its another branch. Good to know!
> Actually
>
> 32bit will be difficult as the code I got have already some
> dependencies on
>
> 64 bit.
>
> Otherwise I will continue with 1.x version. Can you suggest some
> version
>
> with 1.x series which is stable and work on the cluster environment?
>
> especially with C++ ...
>
> Regards..
>
> Salman.
>
> Salman Toor, PhD
>
> salman.toor@it.uu.se [7]
>
> On Nov 4, 2013, at 1:54 PM, Amr Shahin wrote:
>
> Well, the 2 series isn't exactly the "next version". It's a
> continuation of
>
> branch .2.
>
> Also, the error message from the gcc indicates that the library
> you're
>
> trying to link to isn't compatible which made me suspicious. check
> the
>
> documentation to see if hadoop has 64 libraries, or otherwise compile
>
> against the 32 ones
>
> On Mon, Nov 4, 2013 at 4:51 PM, Salman Toor wrote:
>
> Hi,
>
> Thanks for your answer!
>
> But are you sure about it? Actually Hadoop version 1.2 have both 32
> and 64
>
> bit libraries so I believe the the next version should have both...
> But I am
>
> not sure just a random guess :-(
>
> Regards..
>
> Salman.
>
> Salman Toor, PhD
>
> salman.toor@it.uu.se [9]
>
> On Nov 4, 2013, at 1:38 PM, Amr Shahin wrote:
>
> I believe hadoop isn't compatible with 64 architecture. Try
> installing the
>
> 32 libraries and compile against them.
>
> This error (skipping incompatible
>
> /home/sztoor/hadoop-2.2.0/lib/native/libhadooppipes.a when searching
>
> -lhadooppipes) indicates so
>
> On Mon, Nov 4, 2013 at 2:44 PM, Salman Toor wrote:
>
> Hi,
>
> Can someone give a pointer?
>
> Thanks in advance.
>
> Regards..
>
> Salman.
>
> Salman Toor, PhD
>
> salman.toor@it.uu.se [11]
>
> On Nov 3, 2013, at 11:31 PM, Salman Toor wrote:
>
> Hi,
>
> I am quite new to the Hadoop world, previously was running
> hadoop-1.2.0
>
> stable version on my small cluster and encountered some strange
> problems
>
> like the local path to the mapper file didn't copy to the hdfs ....
> It
>
> works fine on the single node setup but on multiple node simple
> word-count
>
> python example didn't work... I read on the blog that it might be the
>
> problem in the version I am using. So I thought to change the version
> and
>
> downloaded the Hadoop 2.2.0. This version has yarn together with many
> new
>
> features that I hope I will learn in the future. Now simple wordcount
>
> example works without any problem on the multi-node setup. I am using
> simple
>
> python example.
>
> Now I would like to compile my C++ code. Since the directory
> structure
>
> together with other things have been changed. I have started to get
> the
>
> following error:
>
> ----------------
>
> /urs/bin/ld: skipping incompatible
>
> /home/sztoor/hadoop-2.2.0/lib/native/libhadooputils.a when searching
>
> -lhadooputils
>
> cannot find -lhadooputils
>
> /urs/bin/ld: skipping incompatible
>
> /home/sztoor/hadoop-2.2.0/lib/native/libhadooppipes.a when searching
>
> -lhadooppipes
>
> cannot find -lhadooppipes
>
> ------------------
>
> I have managed to run the c++ example successfully with the 1.2.0
> version
>
> on single node setup.
>
> I am having 64bit Ubuntu machine. previously I was using
> Linux-amd64-64
>
> Now in new version, "lib" and "include" directories are in the
>
> hadoop-2.2.0 directory. No build.xml is available...
>
> Can someone please give me an example of a makefile based on the
> version
>
> 2.2.0? Or suggest me which version I should go for? Or if there are
> some
>
> prerequisites that I should do before compiling my code?
>
> Thanks in advance.
>
> Regards..
>
> Salman.
>
> Salman Toor, PhD
>
> salman.toor@it.uu.se [12]
>
> --
> André Kelpe
> andre@concurrentinc.com [13]
> http://concurrentinc.com [14]
>
> --
> André Kelpe
> andre@concurrentinc.com [15]
> http://concurrentinc.com
>
>
>
> Links:
> ------
> [1] mailto:salman.toor@it.uu.se
> [2] http://apache.org
> [3] mailto:salman.toor@it.uu.se
> [4] mailto:salman.toor@it.uu.se
> [5] https://issues.apache.org/jira/browse/HADOOP-9911
> [6] mailto:salman.toor@it.uu.se
> [7] mailto:salman.toor@it.uu.se
> [8] mailto:salman.toor@it.uu.se
> [9] mailto:salman.toor@it.uu.se
> [10] mailto:salman.toor@it.uu.se
> [11] mailto:salman.toor@it.uu.se
> [12] mailto:salman.toor@it.uu.se
> [13] mailto:andre@concurrentinc.com
> [14] http://concurrentinc.com
> [15] mailto:andre@concurrentinc.com
Re: C++ example for hadoop-2.2.0
Posted by jerome <je...@ibt.unam.mx>.
Hi Salman
I've trying to compile too te obtain the good library. But i've fac
some compile issue. Could you please, send me how you compile the
sources?
Reagards
On Tue, 5 Nov 2013 12:58:08 +0100, Salman Toor <sa...@it.uu.se>
wrote:
> Hi,
>
> I have solved the problem! Have downloaded the source, compiled it on
> the machine and now can successfully link to the library.
>
> Thank you everyone for you quick responses.
>
> Regards..
> Salman.
>
> Salman Toor, PhD
> salman.toor@it.uu.se [1]
>
> On Nov 4, 2013, at 3:21 PM, Andre Kelpe wrote:
>
> No, because I was trying to set up a cluster automatically with the
> tarballs from apache.org [2].
>
> - André
>
> On Mon, Nov 4, 2013 at 3:05 PM, Salman Toor wrote:
> Hi,
>
> Did you tried to compile with source?
>
> /Salman.
>
> Salman Toor, PhD
> salman.toor@it.uu.se [4]
>
> On Nov 4, 2013, at 2:55 PM, Andre Kelpe wrote:
>
> I reported the 32bit/64bit problem a few weeks ago. There hasn't been
> much activity around it though:
> https://issues.apache.org/jira/browse/HADOOP-9911 [5]
>
> - André
>
> On Mon, Nov 4, 2013 at 2:20 PM, Salman Toor wrote:
>
> Hi,
>
> Ok so 2.x is not a new version its another branch. Good to know!
> Actually
>
> 32bit will be difficult as the code I got have already some
> dependencies on
>
> 64 bit.
>
> Otherwise I will continue with 1.x version. Can you suggest some
> version
>
> with 1.x series which is stable and work on the cluster environment?
>
> especially with C++ ...
>
> Regards..
>
> Salman.
>
> Salman Toor, PhD
>
> salman.toor@it.uu.se [7]
>
> On Nov 4, 2013, at 1:54 PM, Amr Shahin wrote:
>
> Well, the 2 series isn't exactly the "next version". It's a
> continuation of
>
> branch .2.
>
> Also, the error message from the gcc indicates that the library
> you're
>
> trying to link to isn't compatible which made me suspicious. check
> the
>
> documentation to see if hadoop has 64 libraries, or otherwise compile
>
> against the 32 ones
>
> On Mon, Nov 4, 2013 at 4:51 PM, Salman Toor wrote:
>
> Hi,
>
> Thanks for your answer!
>
> But are you sure about it? Actually Hadoop version 1.2 have both 32
> and 64
>
> bit libraries so I believe the the next version should have both...
> But I am
>
> not sure just a random guess :-(
>
> Regards..
>
> Salman.
>
> Salman Toor, PhD
>
> salman.toor@it.uu.se [9]
>
> On Nov 4, 2013, at 1:38 PM, Amr Shahin wrote:
>
> I believe hadoop isn't compatible with 64 architecture. Try
> installing the
>
> 32 libraries and compile against them.
>
> This error (skipping incompatible
>
> /home/sztoor/hadoop-2.2.0/lib/native/libhadooppipes.a when searching
>
> -lhadooppipes) indicates so
>
> On Mon, Nov 4, 2013 at 2:44 PM, Salman Toor wrote:
>
> Hi,
>
> Can someone give a pointer?
>
> Thanks in advance.
>
> Regards..
>
> Salman.
>
> Salman Toor, PhD
>
> salman.toor@it.uu.se [11]
>
> On Nov 3, 2013, at 11:31 PM, Salman Toor wrote:
>
> Hi,
>
> I am quite new to the Hadoop world, previously was running
> hadoop-1.2.0
>
> stable version on my small cluster and encountered some strange
> problems
>
> like the local path to the mapper file didn't copy to the hdfs ....
> It
>
> works fine on the single node setup but on multiple node simple
> word-count
>
> python example didn't work... I read on the blog that it might be the
>
> problem in the version I am using. So I thought to change the version
> and
>
> downloaded the Hadoop 2.2.0. This version has yarn together with many
> new
>
> features that I hope I will learn in the future. Now simple wordcount
>
> example works without any problem on the multi-node setup. I am using
> simple
>
> python example.
>
> Now I would like to compile my C++ code. Since the directory
> structure
>
> together with other things have been changed. I have started to get
> the
>
> following error:
>
> ----------------
>
> /urs/bin/ld: skipping incompatible
>
> /home/sztoor/hadoop-2.2.0/lib/native/libhadooputils.a when searching
>
> -lhadooputils
>
> cannot find -lhadooputils
>
> /urs/bin/ld: skipping incompatible
>
> /home/sztoor/hadoop-2.2.0/lib/native/libhadooppipes.a when searching
>
> -lhadooppipes
>
> cannot find -lhadooppipes
>
> ------------------
>
> I have managed to run the c++ example successfully with the 1.2.0
> version
>
> on single node setup.
>
> I am having 64bit Ubuntu machine. previously I was using
> Linux-amd64-64
>
> Now in new version, "lib" and "include" directories are in the
>
> hadoop-2.2.0 directory. No build.xml is available...
>
> Can someone please give me an example of a makefile based on the
> version
>
> 2.2.0? Or suggest me which version I should go for? Or if there are
> some
>
> prerequisites that I should do before compiling my code?
>
> Thanks in advance.
>
> Regards..
>
> Salman.
>
> Salman Toor, PhD
>
> salman.toor@it.uu.se [12]
>
> --
> André Kelpe
> andre@concurrentinc.com [13]
> http://concurrentinc.com [14]
>
> --
> André Kelpe
> andre@concurrentinc.com [15]
> http://concurrentinc.com
>
>
>
> Links:
> ------
> [1] mailto:salman.toor@it.uu.se
> [2] http://apache.org
> [3] mailto:salman.toor@it.uu.se
> [4] mailto:salman.toor@it.uu.se
> [5] https://issues.apache.org/jira/browse/HADOOP-9911
> [6] mailto:salman.toor@it.uu.se
> [7] mailto:salman.toor@it.uu.se
> [8] mailto:salman.toor@it.uu.se
> [9] mailto:salman.toor@it.uu.se
> [10] mailto:salman.toor@it.uu.se
> [11] mailto:salman.toor@it.uu.se
> [12] mailto:salman.toor@it.uu.se
> [13] mailto:andre@concurrentinc.com
> [14] http://concurrentinc.com
> [15] mailto:andre@concurrentinc.com
Re: C++ example for hadoop-2.2.0
Posted by jerome <je...@ibt.unam.mx>.
Hi Salman
I've trying to compile too te obtain the good library. But i've fac
some compile issue. Could you please, send me how you compile the
sources?
Reagards
On Tue, 5 Nov 2013 12:58:08 +0100, Salman Toor <sa...@it.uu.se>
wrote:
> Hi,
>
> I have solved the problem! Have downloaded the source, compiled it on
> the machine and now can successfully link to the library.
>
> Thank you everyone for you quick responses.
>
> Regards..
> Salman.
>
> Salman Toor, PhD
> salman.toor@it.uu.se [1]
>
> On Nov 4, 2013, at 3:21 PM, Andre Kelpe wrote:
>
> No, because I was trying to set up a cluster automatically with the
> tarballs from apache.org [2].
>
> - André
>
> On Mon, Nov 4, 2013 at 3:05 PM, Salman Toor wrote:
> Hi,
>
> Did you tried to compile with source?
>
> /Salman.
>
> Salman Toor, PhD
> salman.toor@it.uu.se [4]
>
> On Nov 4, 2013, at 2:55 PM, Andre Kelpe wrote:
>
> I reported the 32bit/64bit problem a few weeks ago. There hasn't been
> much activity around it though:
> https://issues.apache.org/jira/browse/HADOOP-9911 [5]
>
> - André
>
> On Mon, Nov 4, 2013 at 2:20 PM, Salman Toor wrote:
>
> Hi,
>
> Ok so 2.x is not a new version its another branch. Good to know!
> Actually
>
> 32bit will be difficult as the code I got have already some
> dependencies on
>
> 64 bit.
>
> Otherwise I will continue with 1.x version. Can you suggest some
> version
>
> with 1.x series which is stable and work on the cluster environment?
>
> especially with C++ ...
>
> Regards..
>
> Salman.
>
> Salman Toor, PhD
>
> salman.toor@it.uu.se [7]
>
> On Nov 4, 2013, at 1:54 PM, Amr Shahin wrote:
>
> Well, the 2 series isn't exactly the "next version". It's a
> continuation of
>
> branch .2.
>
> Also, the error message from the gcc indicates that the library
> you're
>
> trying to link to isn't compatible which made me suspicious. check
> the
>
> documentation to see if hadoop has 64 libraries, or otherwise compile
>
> against the 32 ones
>
> On Mon, Nov 4, 2013 at 4:51 PM, Salman Toor wrote:
>
> Hi,
>
> Thanks for your answer!
>
> But are you sure about it? Actually Hadoop version 1.2 have both 32
> and 64
>
> bit libraries so I believe the the next version should have both...
> But I am
>
> not sure just a random guess :-(
>
> Regards..
>
> Salman.
>
> Salman Toor, PhD
>
> salman.toor@it.uu.se [9]
>
> On Nov 4, 2013, at 1:38 PM, Amr Shahin wrote:
>
> I believe hadoop isn't compatible with 64 architecture. Try
> installing the
>
> 32 libraries and compile against them.
>
> This error (skipping incompatible
>
> /home/sztoor/hadoop-2.2.0/lib/native/libhadooppipes.a when searching
>
> -lhadooppipes) indicates so
>
> On Mon, Nov 4, 2013 at 2:44 PM, Salman Toor wrote:
>
> Hi,
>
> Can someone give a pointer?
>
> Thanks in advance.
>
> Regards..
>
> Salman.
>
> Salman Toor, PhD
>
> salman.toor@it.uu.se [11]
>
> On Nov 3, 2013, at 11:31 PM, Salman Toor wrote:
>
> Hi,
>
> I am quite new to the Hadoop world, previously was running
> hadoop-1.2.0
>
> stable version on my small cluster and encountered some strange
> problems
>
> like the local path to the mapper file didn't copy to the hdfs ....
> It
>
> works fine on the single node setup but on multiple node simple
> word-count
>
> python example didn't work... I read on the blog that it might be the
>
> problem in the version I am using. So I thought to change the version
> and
>
> downloaded the Hadoop 2.2.0. This version has yarn together with many
> new
>
> features that I hope I will learn in the future. Now simple wordcount
>
> example works without any problem on the multi-node setup. I am using
> simple
>
> python example.
>
> Now I would like to compile my C++ code. Since the directory
> structure
>
> together with other things have been changed. I have started to get
> the
>
> following error:
>
> ----------------
>
> /urs/bin/ld: skipping incompatible
>
> /home/sztoor/hadoop-2.2.0/lib/native/libhadooputils.a when searching
>
> -lhadooputils
>
> cannot find -lhadooputils
>
> /urs/bin/ld: skipping incompatible
>
> /home/sztoor/hadoop-2.2.0/lib/native/libhadooppipes.a when searching
>
> -lhadooppipes
>
> cannot find -lhadooppipes
>
> ------------------
>
> I have managed to run the c++ example successfully with the 1.2.0
> version
>
> on single node setup.
>
> I am having 64bit Ubuntu machine. previously I was using
> Linux-amd64-64
>
> Now in new version, "lib" and "include" directories are in the
>
> hadoop-2.2.0 directory. No build.xml is available...
>
> Can someone please give me an example of a makefile based on the
> version
>
> 2.2.0? Or suggest me which version I should go for? Or if there are
> some
>
> prerequisites that I should do before compiling my code?
>
> Thanks in advance.
>
> Regards..
>
> Salman.
>
> Salman Toor, PhD
>
> salman.toor@it.uu.se [12]
>
> --
> André Kelpe
> andre@concurrentinc.com [13]
> http://concurrentinc.com [14]
>
> --
> André Kelpe
> andre@concurrentinc.com [15]
> http://concurrentinc.com
>
>
>
> Links:
> ------
> [1] mailto:salman.toor@it.uu.se
> [2] http://apache.org
> [3] mailto:salman.toor@it.uu.se
> [4] mailto:salman.toor@it.uu.se
> [5] https://issues.apache.org/jira/browse/HADOOP-9911
> [6] mailto:salman.toor@it.uu.se
> [7] mailto:salman.toor@it.uu.se
> [8] mailto:salman.toor@it.uu.se
> [9] mailto:salman.toor@it.uu.se
> [10] mailto:salman.toor@it.uu.se
> [11] mailto:salman.toor@it.uu.se
> [12] mailto:salman.toor@it.uu.se
> [13] mailto:andre@concurrentinc.com
> [14] http://concurrentinc.com
> [15] mailto:andre@concurrentinc.com