You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-user@hadoop.apache.org by Wilson Wayne - wwilso <Wa...@acxiom.com> on 2012/05/08 00:10:17 UTC
Hadoop in Eclipse
I'm new to Hadoop and am trying to get it setup in Eclipse. I'm following the "Working with Hadoop under Eclipse" wiki to do this. First let me make sure this will do what I am hoping it will do. I'm not so much interested in modifying the Hadoop code itself (although I may in the not too distant future). I'm more looking to be able to code and test mappers and reducers in Eclipse and leverage the debugging and other tools in Eclipse in that process. If that's not what the wiki is describing than let me know that and ignore my issue. If there is a way to do this, would you please point me to where I can find the instructions. If this is what the wiki is describing than I have an issue I need help with.
I've followed the directions on the Wiki and even worked through a couple of issues by searching the archive. However, now I've hit an error that while I did find it in the archive, the fix that was presented is not fixing my issue. After importing all the projects, I get a build error: "Project 'MapReduceTools' is missing required library: 'lib/hadoop-core.jar'". I've added the M2_REPO variable as someone suggested in the archive where I found this error once. That hasn't fixed or even changed the error. I've looked in my maven repository (c:\Users\wwilso\.m2\repository) and there is not a "Hadoop-core.jar" anywhere in the repository that windows explorer can find. In fact, windows explorer can't find a "Hadoop-core.jar" anywhere on my c:\ drive.
I did my build using the directions from the wiki (got source using git in cygwin and then executing the two maven commands in cywgin). I did use the patch for the "scripts/saveVersion.sh" not found error to resolve that issue. That's the only thing I have changed from the wiki or code. Using Maven-3.0.4, ant-1.8.3, and protobuf-2.4.1. Anyone have any ideas? Thanks in advance.
WAYNE WILSON
Product Technical Consultant
Acxiom Corporation
EML wayne.wilson@acxiom.com<ma...@acxiom.com>
TEL 501.342.2870
MBL 972.984.0268
5213 Rolling Rock Dr, McKinney, TX, 75070, USA
WWW.ACXIOM.COM<http://WWW.ACXIOM.COM>
[Friend Us on Facebook]<http://www.facebook.com/acxiomcorp> [Link Us on LinkedIn] <http://www.linkedin.com/groupRegistration?gid=2901735> [Follow Us on Twitter] <http://twitter.com/acxiom>
[cid:image004.png@01CB84F1.26214350]
________________________________
The information contained in this communication is confidential, is intended only for the use of the recipient named above, and may be legally privileged. If the reader of this message is not the intended recipient, you are hereby notified that any dissemination, distribution or copying of this communication is strictly prohibited. If you have received this communication in error, please resend this communication to the sender and delete the original message or any copy of it from your computer system. Thank you.
Re: Hadoop in Eclipse
Posted by Keith Wiley <kw...@keithwiley.com>.
Seconded. In Eclipse, I create a new empty Java project, then import as an external jar the core hadoop jar. That's it. It'll then offer code completion and online code checking as you type. To build my code I use ant on the command line and a build.xml file that I derived from the hadoop build.xml, but vastly simplified. I run the driver jar locally which then runs M/R either locally or on a cluster...or I can upload the jar to Azure...or presumably AWS although I haven't worked with that cloud yet.
The Hadoop core jar should get you going.
Cheers!
________________________________________
Sent from my phone, please excuse my brevity.
Keith Wiley, kwiley@keithwiley.com, http://keithwiley.com
Jagat <ja...@gmail.com> wrote:
Hello,
If you are looking to setup eclipse just to write mappers and reducers for
your code then
Create a eclipse java project
Download Apache bundle of Hadoop from Apache hadoop website
If you are using Hadoop 0.23 then go to share folder and add all those jars
e.g mapreduce , hdfs etc into project build path
If you are using Hadoop 1.x or 0,20 series then jars are present right
inside the home folder so you can directly add them to build path
Just run the main class which is your driver for your mapper and reducer
and remember to use job.setJarByClass method
Hope this helps.
On Tue, May 8, 2012 at 3:40 AM, Wilson Wayne - wwilso <
Wayne.Wilson@acxiom.com> wrote:
> I’m new to Hadoop and am trying to get it setup in Eclipse. I’m
> following the “Working with Hadoop under Eclipse” wiki to do this. First
> let me make sure this will do what I am hoping it will do. I’m not so much
> interested in modifying the Hadoop code itself (although I may in the not
> too distant future). I’m more looking to be able to code and test mappers
> and reducers in Eclipse and leverage the debugging and other tools in
> Eclipse in that process. If that’s not what the wiki is describing than
> let me know that and ignore my issue. If there is a way to do this, would
> you please point me to where I can find the instructions. If this is what
> the wiki is describing than I have an issue I need help with.****
>
> ** **
>
> I’ve followed the directions on the Wiki and even worked through a couple
> of issues by searching the archive. However, now I’ve hit an error that
> while I did find it in the archive, the fix that was presented is not
> fixing my issue. After importing all the projects, I get a build error:
> “Project ‘MapReduceTools’ is missing required library:
> ‘lib/hadoop-core.jar’”. I’ve added the M2_REPO variable as someone
> suggested in the archive where I found this error once. That hasn’t fixed
> or even changed the error. I’ve looked in my maven repository
> (c:\Users\wwilso\.m2\repository) and there is not a “Hadoop-core.jar”
> anywhere in the repository that windows explorer can find. In fact,
> windows explorer can’t find a “Hadoop-core.jar” anywhere on my c:\ drive.
> ****
>
> ** **
>
> I did my build using the directions from the wiki (got source using git in
> cygwin and then executing the two maven commands in cywgin). I did use the
> patch for the “scripts/saveVersion.sh” not found error to resolve that
> issue. That’s the only thing I have changed from the wiki or code. Using
> Maven-3.0.4, ant-1.8.3, and protobuf-2.4.1. Anyone have any ideas? Thanks
> in advance.****
>
> ** **
>
> ** **
>
> WAYNE WILSON
> Product Technical Consultant
> Acxiom Corporation
> EML wayne.wilson@acxiom.com <fi...@acxiom.com>
> TEL 501.342.2870
> MBL 972.984.0268
> 5213 Rolling Rock Dr, McKinney, TX, 75070, USA
> WWW.ACXIOM.COM
>
> ****
>
> [image: Friend Us on Facebook] <http://www.facebook.com/acxiomcorp>; [image:
> Link Us on LinkedIn]<http://www.linkedin.com/groupRegistration?gid=2901735>;
> [image: Follow Us on Twitter] <http://twitter.com/acxiom>;****
>
> ** **
>
> [image: cid:image004.png@01CB84F1.26214350]****
>_____________________________________________
>
> The information contained in this communication is confidential, is
> intended only for the use of the recipient named above, and may be legally
> privileged. If the reader of this message is not the intended recipient,
> you are hereby notified that any dissemination, distribution or copying of
> this communication is strictly prohibited. If you have received this
> communication in error, please resend this communication to the sender and
> delete the original message or any copy of it from your computer system.
> Thank you.****
>
> ** **
>
Re: Hadoop in Eclipse
Posted by Jagat <ja...@gmail.com>.
Hello,
If you are looking to setup eclipse just to write mappers and reducers for
your code then
Create a eclipse java project
Download Apache bundle of Hadoop from Apache hadoop website
If you are using Hadoop 0.23 then go to share folder and add all those jars
e.g mapreduce , hdfs etc into project build path
If you are using Hadoop 1.x or 0,20 series then jars are present right
inside the home folder so you can directly add them to build path
Just run the main class which is your driver for your mapper and reducer
and remember to use job.setJarByClass method
Hope this helps.
On Tue, May 8, 2012 at 3:40 AM, Wilson Wayne - wwilso <
Wayne.Wilson@acxiom.com> wrote:
> I’m new to Hadoop and am trying to get it setup in Eclipse. I’m
> following the “Working with Hadoop under Eclipse” wiki to do this. First
> let me make sure this will do what I am hoping it will do. I’m not so much
> interested in modifying the Hadoop code itself (although I may in the not
> too distant future). I’m more looking to be able to code and test mappers
> and reducers in Eclipse and leverage the debugging and other tools in
> Eclipse in that process. If that’s not what the wiki is describing than
> let me know that and ignore my issue. If there is a way to do this, would
> you please point me to where I can find the instructions. If this is what
> the wiki is describing than I have an issue I need help with.****
>
> ** **
>
> I’ve followed the directions on the Wiki and even worked through a couple
> of issues by searching the archive. However, now I’ve hit an error that
> while I did find it in the archive, the fix that was presented is not
> fixing my issue. After importing all the projects, I get a build error:
> “Project ‘MapReduceTools’ is missing required library:
> ‘lib/hadoop-core.jar’”. I’ve added the M2_REPO variable as someone
> suggested in the archive where I found this error once. That hasn’t fixed
> or even changed the error. I’ve looked in my maven repository
> (c:\Users\wwilso\.m2\repository) and there is not a “Hadoop-core.jar”
> anywhere in the repository that windows explorer can find. In fact,
> windows explorer can’t find a “Hadoop-core.jar” anywhere on my c:\ drive.
> ****
>
> ** **
>
> I did my build using the directions from the wiki (got source using git in
> cygwin and then executing the two maven commands in cywgin). I did use the
> patch for the “scripts/saveVersion.sh” not found error to resolve that
> issue. That’s the only thing I have changed from the wiki or code. Using
> Maven-3.0.4, ant-1.8.3, and protobuf-2.4.1. Anyone have any ideas? Thanks
> in advance.****
>
> ** **
>
> ** **
>
> WAYNE WILSON
> Product Technical Consultant
> Acxiom Corporation
> EML wayne.wilson@acxiom.com <fi...@acxiom.com>
> TEL 501.342.2870
> MBL 972.984.0268
> 5213 Rolling Rock Dr, McKinney, TX, 75070, USA
> WWW.ACXIOM.COM
>
> ****
>
> [image: Friend Us on Facebook] <http://www.facebook.com/acxiomcorp> [image:
> Link Us on LinkedIn]<http://www.linkedin.com/groupRegistration?gid=2901735>
> [image: Follow Us on Twitter] <http://twitter.com/acxiom>****
>
> ** **
>
> [image: cid:image004.png@01CB84F1.26214350]****
> ------------------------------
>
> The information contained in this communication is confidential, is
> intended only for the use of the recipient named above, and may be legally
> privileged. If the reader of this message is not the intended recipient,
> you are hereby notified that any dissemination, distribution or copying of
> this communication is strictly prohibited. If you have received this
> communication in error, please resend this communication to the sender and
> delete the original message or any copy of it from your computer system.
> Thank you.****
>
> ** **
>
Re: Hadoop in Eclipse
Posted by Serge Blazhiyevskyy <Se...@nice.com>.
Hi,
Oracle removed java packages from partners web-site.
Try to follow these steps:
http://blog.flexion.org/2012/01/16/install-sun-java-6-jre-jdk-from-deb-pack
ages/
Let me know if you run into problems
Regards,
Serge
On 5/8/12 8:13 AM, "Wilson Wayne - wwilso" <Wa...@acxiom.com> wrote:
>Hey Serge, I'm trying to follow your steps on your blog. I've downloaded
>the Oracle Virtual box (VirtualBox-4.1.14-77440-Win.exe) and Ubuntu
>(ubuntu-10.04.4-desktop-i386.iso). I've gotten to step 5 and ran into an
>issue with the second command, "sudo apt-get update". It appears to be
>just a data check issue with the GPG key, so I went ahead and tried the
>third command. That command also has an error, so maybe it's more than a
>data check error. Here's what I got:
>
>wwilso@wwilso-laptop:~$ sudo add-apt-repository "deb
>http://archive.canonical.com/ lucid partner"
>wwilso@wwilso-laptop:~$ sudo apt-get update
>Get:1 http://archive.cloudera.com lucid-cdh3 Release.gpg [197B]
>Ign http://archive.cloudera.com/debian/ lucid-cdh3/contrib
>Translation-en_US
>Get:2 http://security.ubuntu.com lucid-security Release.gpg [198B]
>
>Ign http://security.ubuntu.com/ubuntu/ lucid-security/main
>Translation-en_US
>Ign http://security.ubuntu.com/ubuntu/ lucid-security/restricted
>Translation-en_US
>Ign http://security.ubuntu.com/ubuntu/ lucid-security/universe
>Translation-en_US
>Ign http://security.ubuntu.com/ubuntu/ lucid-security/multiverse
>Translation-en_US
>Get:3 http://archive.canonical.com lucid Release.gpg [198B]
>
>Ign http://archive.canonical.com/ lucid/partner Translation-en_US
>Get:4 http://archive.cloudera.com lucid-cdh3 Release [2,419B]
>Get:5 http://us.archive.ubuntu.com lucid Release.gpg [189B]
>
>Ign http://us.archive.ubuntu.com/ubuntu/ lucid/main Translation-en_US
>
>Ign http://us.archive.ubuntu.com/ubuntu/ lucid/restricted
>Translation-en_US
>Ign http://us.archive.ubuntu.com/ubuntu/ lucid/universe Translation-en_US
>
>Ign http://us.archive.ubuntu.com/ubuntu/ lucid/multiverse
>Translation-en_US
>Get:6 http://us.archive.ubuntu.com lucid-updates Release.gpg [198B]
>Ign http://archive.cloudera.com lucid-cdh3 Release
>
>Get:7 http://security.ubuntu.com lucid-security Release [57.3kB]
>Get:8 http://archive.canonical.com lucid Release [8,215B]
>
>Ign http://archive.cloudera.com lucid-cdh3/contrib Packages
>
>Ign http://us.archive.ubuntu.com/ubuntu/ lucid-updates/main
>Translation-en_US
>Ign http://us.archive.ubuntu.com/ubuntu/ lucid-updates/restricted
>Translation-en_US
>Ign http://us.archive.ubuntu.com/ubuntu/ lucid-updates/universe
>Translation-en_US
>Ign http://us.archive.ubuntu.com/ubuntu/ lucid-updates/multiverse
>Translation-en_US
>Get:9 http://us.archive.ubuntu.com lucid Release [57.2kB]
>
>Ign http://archive.cloudera.com lucid-cdh3/contrib Sources
>
>Ign http://archive.cloudera.com lucid-cdh3/contrib Packages
>
>Ign http://archive.cloudera.com lucid-cdh3/contrib Sources
>
>Get:10 http://archive.canonical.com lucid/partner Packages [13.6kB]
>
>Get:11 http://archive.cloudera.com lucid-cdh3/contrib Packages [13.0kB]
>
>Get:12 http://us.archive.ubuntu.com lucid-updates Release [57.3kB]
>
>Get:13 http://security.ubuntu.com lucid-security/main Packages [412kB]
>
>Get:14 http://archive.cloudera.com lucid-cdh3/contrib Sources [4,597B]
>
>Get:15 http://us.archive.ubuntu.com lucid/main Packages [1,386kB]
>
>Get:16 http://security.ubuntu.com lucid-security/restricted Packages
>[2,855B]
>Get:17 http://security.ubuntu.com lucid-security/main Sources [122kB]
>
>Get:18 http://security.ubuntu.com lucid-security/restricted Sources
>[1,259B]
>Get:19 http://security.ubuntu.com lucid-security/universe Packages
>[129kB]
>Get:20 http://security.ubuntu.com lucid-security/universe Sources
>[39.3kB]
>Get:21 http://security.ubuntu.com lucid-security/multiverse Packages
>[5,368B]
>Get:22 http://security.ubuntu.com lucid-security/multiverse Sources
>[2,323B]
>Get:23 http://us.archive.ubuntu.com lucid/restricted Packages [6,208B]
>
>Get:24 http://us.archive.ubuntu.com lucid/main Sources [659kB]
>
>Get:25 http://us.archive.ubuntu.com lucid/restricted Sources [3,775B]
>
>Get:26 http://us.archive.ubuntu.com lucid/universe Packages [5,448kB]
>
>Get:27 http://us.archive.ubuntu.com lucid/universe Sources [3,165kB]
>
>Get:28 http://us.archive.ubuntu.com lucid/multiverse Packages [180kB]
>
>Get:29 http://us.archive.ubuntu.com lucid/multiverse Sources [119kB]
>
>Get:30 http://us.archive.ubuntu.com lucid-updates/main Packages [613kB]
>
>Get:31 http://us.archive.ubuntu.com lucid-updates/restricted Packages
>[4,617B]
>Get:32 http://us.archive.ubuntu.com lucid-updates/main Sources [221kB]
>
>Get:33 http://us.archive.ubuntu.com lucid-updates/restricted Sources
>[2,194B]
>Get:34 http://us.archive.ubuntu.com lucid-updates/universe Packages
>[268kB]
>Get:35 http://us.archive.ubuntu.com lucid-updates/universe Sources
>[98.4kB]
>Get:36 http://us.archive.ubuntu.com lucid-updates/multiverse Packages
>[11.3kB]
>Get:37 http://us.archive.ubuntu.com lucid-updates/multiverse Sources
>[5,537B]
>Fetched 13.1MB in 22s (590kB/s)
>
>Reading package lists... Done
>W: GPG error: http://archive.cloudera.com lucid-cdh3 Release: The
>following signatures couldn't be verified because the public key is not
>available: NO_PUBKEY 327574EE02A818DD
>wwilso@wwilso-laptop:~$ sudo apt-get install sun-java6-jdk
>Reading package lists... Done
>Building dependency tree
>Reading state information... Done
>Package sun-java6-jdk is not available, but is referred to by another
>package.
>This may mean that the package is missing, has been obsoleted, or
>is only available from another source
>E: Package sun-java6-jdk has no installation candidate
>
>
>Any ideas?
>
>
>-----Original Message-----
>From: Serge Blazhiyevskyy [mailto:Serge.Blazhiyevskyy@nice.com]
>Sent: Monday, May 07, 2012 5:45 PM
>To: common-user@hadoop.apache.org
>Subject: Re: Hadoop in Eclipse
>
>Here is the link to cloudera binaries
>
>
>https://ccp.cloudera.com/display/SUPPORT/CDH3+Downloadable+Tarballs
>
>
>Download the file that starts with hadoop
>
>
>Also, I have some basic instructions on my blog:
>
>Hadoopway.blogspot.com
>
>Good luck!
>
>
>
>On 5/7/12 3:40 PM, "Wilson Wayne - wwilso" <Wa...@acxiom.com>
>wrote:
>
>>Thanks for the quick response. To show just how new I am to all this,
>>where would I be able to download those libraries or find them if the
>>maven build created them? Also, would I only need the MapReduce ones or
>>all of them. Would I need the Hadoop-core.jar that doesn't appear to be
>>on my system anywhere?
>>
>>Sorry for the very basic, newbie questions.
>>
>>-----Original Message-----
>>From: Serge Blazhiyevskyy [mailto:Serge.Blazhiyevskyy@nice.com]
>>Sent: Monday, May 07, 2012 5:15 PM
>>To: common-user@hadoop.apache.org; core-user@hadoop.apache.org
>>Subject: Re: Hadoop in Eclipse
>>
>>If you want to test your mapper and reducers in eclipse, you can just
>>link with hadoop libraries in project settings.
>>
>>Regards,
>>Serge
>>
>>From: Wilson Wayne - wwilso
>><Wa...@acxiom.com>>
>>Reply-To:
>>"common-user@hadoop.apache.org<ma...@hadoop.apache.org>"
>><co...@hadoop.apache.org>>
>>Date: Mon, 7 May 2012 16:10:17 -0600
>>To: "core-user@hadoop.apache.org<ma...@hadoop.apache.org>"
>><co...@hadoop.apache.org>>
>>Subject: Hadoop in Eclipse
>>
>>I'm new to Hadoop and am trying to get it setup in Eclipse. I'm
>>following the "Working with Hadoop under Eclipse" wiki to do this. First
>>let me make sure this will do what I am hoping it will do. I'm not so
>>much interested in modifying the Hadoop code itself (although I may in
>>the not too distant future). I'm more looking to be able to code and
>>test mappers and reducers in Eclipse and leverage the debugging and other
>>tools in Eclipse in that process. If that's not what the wiki is
>>describing than let me know that and ignore my issue. If there is a way
>>to do this, would you please point me to where I can find the
>>instructions. If this is what the wiki is describing than I have an
>>issue I need help with.
>>
>>I've followed the directions on the Wiki and even worked through a couple
>>of issues by searching the archive. However, now I've hit an error that
>>while I did find it in the archive, the fix that was presented is not
>>fixing my issue. After importing all the projects, I get a build error:
>>"Project 'MapReduceTools' is missing required library:
>>'lib/hadoop-core.jar'". I've added the M2_REPO variable as someone
>>suggested in the archive where I found this error once. That hasn't
>>fixed or even changed the error. I've looked in my maven repository
>>(c:\Users\wwilso\.m2\repository) and there is not a "Hadoop-core.jar"
>>anywhere in the repository that windows explorer can find. In fact,
>>windows explorer can't find a "Hadoop-core.jar" anywhere on my c:\ drive.
>>
>>I did my build using the directions from the wiki (got source using git
>>in cygwin and then executing the two maven commands in cywgin). I did
>>use the patch for the "scripts/saveVersion.sh" not found error to resolve
>>that issue. That's the only thing I have changed from the wiki or code.
>>Using Maven-3.0.4, ant-1.8.3, and protobuf-2.4.1. Anyone have any ideas?
>> Thanks in advance.
>>
>>
>>WAYNE WILSON
>>Product Technical Consultant
>>Acxiom Corporation
>>EML wayne.wilson@acxiom.com<ma...@acxiom.com>
>>TEL 501.342.2870
>>MBL 972.984.0268
>>5213 Rolling Rock Dr, McKinney, TX, 75070, USA
>>WWW.ACXIOM.COM<http://WWW.ACXIOM.COM>
>>
>>
>>[cid:image001.gif@01CD2C6F.7A0BD440]<http://www.facebook.com/acxiomcorp>
>>[cid:image002.gif@01CD2C6F.7A0BD440]
>><http://www.linkedin.com/groupRegistration?gid=2901735>
>>[cid:image003.gif@01CD2C6F.7A0BD440] <http://twitter.com/acxiom>
>>
>>[cid:image004.png@01CD2C6F.7A0BD440]
>>________________________________
>>The information contained in this communication is confidential, is
>>intended only for the use of the recipient named above, and may be
>>legally privileged. If the reader of this message is not the intended
>>recipient, you are hereby notified that any dissemination, distribution
>>or copying of this communication is strictly prohibited. If you have
>>received this communication in error, please resend this communication to
>>the sender and delete the original message or any copy of it from your
>>computer system. Thank you.
>>
>
>**************************************************************************
>*
>The information contained in this communication is confidential, is
>intended only for the use of the recipient named above, and may be legally
>privileged.
>
>If the reader of this message is not the intended recipient, you are
>hereby notified that any dissemination, distribution or copying of this
>communication is strictly prohibited.
>
>If you have received this communication in error, please resend this
>communication to the sender and delete the original message or any copy
>of it from your computer system.
>
>Thank You.
>**************************************************************************
>**
>
RE: Hadoop in Eclipse
Posted by Wilson Wayne - wwilso <Wa...@acxiom.com>.
Hey Serge, I'm trying to follow your steps on your blog. I've downloaded the Oracle Virtual box (VirtualBox-4.1.14-77440-Win.exe) and Ubuntu (ubuntu-10.04.4-desktop-i386.iso). I've gotten to step 5 and ran into an issue with the second command, "sudo apt-get update". It appears to be just a data check issue with the GPG key, so I went ahead and tried the third command. That command also has an error, so maybe it's more than a data check error. Here's what I got:
wwilso@wwilso-laptop:~$ sudo add-apt-repository "deb http://archive.canonical.com/ lucid partner"
wwilso@wwilso-laptop:~$ sudo apt-get update
Get:1 http://archive.cloudera.com lucid-cdh3 Release.gpg [197B]
Ign http://archive.cloudera.com/debian/ lucid-cdh3/contrib Translation-en_US
Get:2 http://security.ubuntu.com lucid-security Release.gpg [198B]
Ign http://security.ubuntu.com/ubuntu/ lucid-security/main Translation-en_US
Ign http://security.ubuntu.com/ubuntu/ lucid-security/restricted Translation-en_US
Ign http://security.ubuntu.com/ubuntu/ lucid-security/universe Translation-en_US
Ign http://security.ubuntu.com/ubuntu/ lucid-security/multiverse Translation-en_US
Get:3 http://archive.canonical.com lucid Release.gpg [198B]
Ign http://archive.canonical.com/ lucid/partner Translation-en_US
Get:4 http://archive.cloudera.com lucid-cdh3 Release [2,419B]
Get:5 http://us.archive.ubuntu.com lucid Release.gpg [189B]
Ign http://us.archive.ubuntu.com/ubuntu/ lucid/main Translation-en_US
Ign http://us.archive.ubuntu.com/ubuntu/ lucid/restricted Translation-en_US
Ign http://us.archive.ubuntu.com/ubuntu/ lucid/universe Translation-en_US
Ign http://us.archive.ubuntu.com/ubuntu/ lucid/multiverse Translation-en_US
Get:6 http://us.archive.ubuntu.com lucid-updates Release.gpg [198B]
Ign http://archive.cloudera.com lucid-cdh3 Release
Get:7 http://security.ubuntu.com lucid-security Release [57.3kB]
Get:8 http://archive.canonical.com lucid Release [8,215B]
Ign http://archive.cloudera.com lucid-cdh3/contrib Packages
Ign http://us.archive.ubuntu.com/ubuntu/ lucid-updates/main Translation-en_US
Ign http://us.archive.ubuntu.com/ubuntu/ lucid-updates/restricted Translation-en_US
Ign http://us.archive.ubuntu.com/ubuntu/ lucid-updates/universe Translation-en_US
Ign http://us.archive.ubuntu.com/ubuntu/ lucid-updates/multiverse Translation-en_US
Get:9 http://us.archive.ubuntu.com lucid Release [57.2kB]
Ign http://archive.cloudera.com lucid-cdh3/contrib Sources
Ign http://archive.cloudera.com lucid-cdh3/contrib Packages
Ign http://archive.cloudera.com lucid-cdh3/contrib Sources
Get:10 http://archive.canonical.com lucid/partner Packages [13.6kB]
Get:11 http://archive.cloudera.com lucid-cdh3/contrib Packages [13.0kB]
Get:12 http://us.archive.ubuntu.com lucid-updates Release [57.3kB]
Get:13 http://security.ubuntu.com lucid-security/main Packages [412kB]
Get:14 http://archive.cloudera.com lucid-cdh3/contrib Sources [4,597B]
Get:15 http://us.archive.ubuntu.com lucid/main Packages [1,386kB]
Get:16 http://security.ubuntu.com lucid-security/restricted Packages [2,855B]
Get:17 http://security.ubuntu.com lucid-security/main Sources [122kB]
Get:18 http://security.ubuntu.com lucid-security/restricted Sources [1,259B]
Get:19 http://security.ubuntu.com lucid-security/universe Packages [129kB]
Get:20 http://security.ubuntu.com lucid-security/universe Sources [39.3kB]
Get:21 http://security.ubuntu.com lucid-security/multiverse Packages [5,368B]
Get:22 http://security.ubuntu.com lucid-security/multiverse Sources [2,323B]
Get:23 http://us.archive.ubuntu.com lucid/restricted Packages [6,208B]
Get:24 http://us.archive.ubuntu.com lucid/main Sources [659kB]
Get:25 http://us.archive.ubuntu.com lucid/restricted Sources [3,775B]
Get:26 http://us.archive.ubuntu.com lucid/universe Packages [5,448kB]
Get:27 http://us.archive.ubuntu.com lucid/universe Sources [3,165kB]
Get:28 http://us.archive.ubuntu.com lucid/multiverse Packages [180kB]
Get:29 http://us.archive.ubuntu.com lucid/multiverse Sources [119kB]
Get:30 http://us.archive.ubuntu.com lucid-updates/main Packages [613kB]
Get:31 http://us.archive.ubuntu.com lucid-updates/restricted Packages [4,617B]
Get:32 http://us.archive.ubuntu.com lucid-updates/main Sources [221kB]
Get:33 http://us.archive.ubuntu.com lucid-updates/restricted Sources [2,194B]
Get:34 http://us.archive.ubuntu.com lucid-updates/universe Packages [268kB]
Get:35 http://us.archive.ubuntu.com lucid-updates/universe Sources [98.4kB]
Get:36 http://us.archive.ubuntu.com lucid-updates/multiverse Packages [11.3kB]
Get:37 http://us.archive.ubuntu.com lucid-updates/multiverse Sources [5,537B]
Fetched 13.1MB in 22s (590kB/s)
Reading package lists... Done
W: GPG error: http://archive.cloudera.com lucid-cdh3 Release: The following signatures couldn't be verified because the public key is not available: NO_PUBKEY 327574EE02A818DD
wwilso@wwilso-laptop:~$ sudo apt-get install sun-java6-jdk
Reading package lists... Done
Building dependency tree
Reading state information... Done
Package sun-java6-jdk is not available, but is referred to by another package.
This may mean that the package is missing, has been obsoleted, or
is only available from another source
E: Package sun-java6-jdk has no installation candidate
Any ideas?
-----Original Message-----
From: Serge Blazhiyevskyy [mailto:Serge.Blazhiyevskyy@nice.com]
Sent: Monday, May 07, 2012 5:45 PM
To: common-user@hadoop.apache.org
Subject: Re: Hadoop in Eclipse
Here is the link to cloudera binaries
https://ccp.cloudera.com/display/SUPPORT/CDH3+Downloadable+Tarballs
Download the file that starts with hadoop
Also, I have some basic instructions on my blog:
Hadoopway.blogspot.com
Good luck!
On 5/7/12 3:40 PM, "Wilson Wayne - wwilso" <Wa...@acxiom.com> wrote:
>Thanks for the quick response. To show just how new I am to all this,
>where would I be able to download those libraries or find them if the
>maven build created them? Also, would I only need the MapReduce ones or
>all of them. Would I need the Hadoop-core.jar that doesn't appear to be
>on my system anywhere?
>
>Sorry for the very basic, newbie questions.
>
>-----Original Message-----
>From: Serge Blazhiyevskyy [mailto:Serge.Blazhiyevskyy@nice.com]
>Sent: Monday, May 07, 2012 5:15 PM
>To: common-user@hadoop.apache.org; core-user@hadoop.apache.org
>Subject: Re: Hadoop in Eclipse
>
>If you want to test your mapper and reducers in eclipse, you can just
>link with hadoop libraries in project settings.
>
>Regards,
>Serge
>
>From: Wilson Wayne - wwilso
><Wa...@acxiom.com>>
>Reply-To:
>"common-user@hadoop.apache.org<ma...@hadoop.apache.org>"
><co...@hadoop.apache.org>>
>Date: Mon, 7 May 2012 16:10:17 -0600
>To: "core-user@hadoop.apache.org<ma...@hadoop.apache.org>"
><co...@hadoop.apache.org>>
>Subject: Hadoop in Eclipse
>
>I'm new to Hadoop and am trying to get it setup in Eclipse. I'm
>following the "Working with Hadoop under Eclipse" wiki to do this. First
>let me make sure this will do what I am hoping it will do. I'm not so
>much interested in modifying the Hadoop code itself (although I may in
>the not too distant future). I'm more looking to be able to code and
>test mappers and reducers in Eclipse and leverage the debugging and other
>tools in Eclipse in that process. If that's not what the wiki is
>describing than let me know that and ignore my issue. If there is a way
>to do this, would you please point me to where I can find the
>instructions. If this is what the wiki is describing than I have an
>issue I need help with.
>
>I've followed the directions on the Wiki and even worked through a couple
>of issues by searching the archive. However, now I've hit an error that
>while I did find it in the archive, the fix that was presented is not
>fixing my issue. After importing all the projects, I get a build error:
>"Project 'MapReduceTools' is missing required library:
>'lib/hadoop-core.jar'". I've added the M2_REPO variable as someone
>suggested in the archive where I found this error once. That hasn't
>fixed or even changed the error. I've looked in my maven repository
>(c:\Users\wwilso\.m2\repository) and there is not a "Hadoop-core.jar"
>anywhere in the repository that windows explorer can find. In fact,
>windows explorer can't find a "Hadoop-core.jar" anywhere on my c:\ drive.
>
>I did my build using the directions from the wiki (got source using git
>in cygwin and then executing the two maven commands in cywgin). I did
>use the patch for the "scripts/saveVersion.sh" not found error to resolve
>that issue. That's the only thing I have changed from the wiki or code.
>Using Maven-3.0.4, ant-1.8.3, and protobuf-2.4.1. Anyone have any ideas?
> Thanks in advance.
>
>
>WAYNE WILSON
>Product Technical Consultant
>Acxiom Corporation
>EML wayne.wilson@acxiom.com<ma...@acxiom.com>
>TEL 501.342.2870
>MBL 972.984.0268
>5213 Rolling Rock Dr, McKinney, TX, 75070, USA
>WWW.ACXIOM.COM<http://WWW.ACXIOM.COM>
>
>
>[cid:image001.gif@01CD2C6F.7A0BD440]<http://www.facebook.com/acxiomcorp>
>[cid:image002.gif@01CD2C6F.7A0BD440]
><http://www.linkedin.com/groupRegistration?gid=2901735>
>[cid:image003.gif@01CD2C6F.7A0BD440] <http://twitter.com/acxiom>
>
>[cid:image004.png@01CD2C6F.7A0BD440]
>________________________________
>The information contained in this communication is confidential, is
>intended only for the use of the recipient named above, and may be
>legally privileged. If the reader of this message is not the intended
>recipient, you are hereby notified that any dissemination, distribution
>or copying of this communication is strictly prohibited. If you have
>received this communication in error, please resend this communication to
>the sender and delete the original message or any copy of it from your
>computer system. Thank you.
>
***************************************************************************
The information contained in this communication is confidential, is
intended only for the use of the recipient named above, and may be legally
privileged.
If the reader of this message is not the intended recipient, you are
hereby notified that any dissemination, distribution or copying of this
communication is strictly prohibited.
If you have received this communication in error, please resend this
communication to the sender and delete the original message or any copy
of it from your computer system.
Thank You.
****************************************************************************
RE: Hadoop in Eclipse
Posted by Wilson Wayne - wwilso <Wa...@acxiom.com>.
Great! Thanks for the help. I'll give it a try.
-----Original Message-----
From: Serge Blazhiyevskyy [mailto:Serge.Blazhiyevskyy@nice.com]
Sent: Monday, May 07, 2012 5:45 PM
To: common-user@hadoop.apache.org
Subject: Re: Hadoop in Eclipse
Here is the link to cloudera binaries
https://ccp.cloudera.com/display/SUPPORT/CDH3+Downloadable+Tarballs
Download the file that starts with hadoop
Also, I have some basic instructions on my blog:
Hadoopway.blogspot.com
Good luck!
On 5/7/12 3:40 PM, "Wilson Wayne - wwilso" <Wa...@acxiom.com> wrote:
>Thanks for the quick response. To show just how new I am to all this,
>where would I be able to download those libraries or find them if the
>maven build created them? Also, would I only need the MapReduce ones or
>all of them. Would I need the Hadoop-core.jar that doesn't appear to be
>on my system anywhere?
>
>Sorry for the very basic, newbie questions.
>
>-----Original Message-----
>From: Serge Blazhiyevskyy [mailto:Serge.Blazhiyevskyy@nice.com]
>Sent: Monday, May 07, 2012 5:15 PM
>To: common-user@hadoop.apache.org; core-user@hadoop.apache.org
>Subject: Re: Hadoop in Eclipse
>
>If you want to test your mapper and reducers in eclipse, you can just
>link with hadoop libraries in project settings.
>
>Regards,
>Serge
>
>From: Wilson Wayne - wwilso
><Wa...@acxiom.com>>
>Reply-To:
>"common-user@hadoop.apache.org<ma...@hadoop.apache.org>"
><co...@hadoop.apache.org>>
>Date: Mon, 7 May 2012 16:10:17 -0600
>To: "core-user@hadoop.apache.org<ma...@hadoop.apache.org>"
><co...@hadoop.apache.org>>
>Subject: Hadoop in Eclipse
>
>I'm new to Hadoop and am trying to get it setup in Eclipse. I'm
>following the "Working with Hadoop under Eclipse" wiki to do this. First
>let me make sure this will do what I am hoping it will do. I'm not so
>much interested in modifying the Hadoop code itself (although I may in
>the not too distant future). I'm more looking to be able to code and
>test mappers and reducers in Eclipse and leverage the debugging and other
>tools in Eclipse in that process. If that's not what the wiki is
>describing than let me know that and ignore my issue. If there is a way
>to do this, would you please point me to where I can find the
>instructions. If this is what the wiki is describing than I have an
>issue I need help with.
>
>I've followed the directions on the Wiki and even worked through a couple
>of issues by searching the archive. However, now I've hit an error that
>while I did find it in the archive, the fix that was presented is not
>fixing my issue. After importing all the projects, I get a build error:
>"Project 'MapReduceTools' is missing required library:
>'lib/hadoop-core.jar'". I've added the M2_REPO variable as someone
>suggested in the archive where I found this error once. That hasn't
>fixed or even changed the error. I've looked in my maven repository
>(c:\Users\wwilso\.m2\repository) and there is not a "Hadoop-core.jar"
>anywhere in the repository that windows explorer can find. In fact,
>windows explorer can't find a "Hadoop-core.jar" anywhere on my c:\ drive.
>
>I did my build using the directions from the wiki (got source using git
>in cygwin and then executing the two maven commands in cywgin). I did
>use the patch for the "scripts/saveVersion.sh" not found error to resolve
>that issue. That's the only thing I have changed from the wiki or code.
>Using Maven-3.0.4, ant-1.8.3, and protobuf-2.4.1. Anyone have any ideas?
> Thanks in advance.
>
>
>WAYNE WILSON
>Product Technical Consultant
>Acxiom Corporation
>EML wayne.wilson@acxiom.com<ma...@acxiom.com>
>TEL 501.342.2870
>MBL 972.984.0268
>5213 Rolling Rock Dr, McKinney, TX, 75070, USA
>WWW.ACXIOM.COM<http://WWW.ACXIOM.COM>
>
>
>[cid:image001.gif@01CD2C6F.7A0BD440]<http://www.facebook.com/acxiomcorp>
>[cid:image002.gif@01CD2C6F.7A0BD440]
><http://www.linkedin.com/groupRegistration?gid=2901735>
>[cid:image003.gif@01CD2C6F.7A0BD440] <http://twitter.com/acxiom>
>
>[cid:image004.png@01CD2C6F.7A0BD440]
>________________________________
>The information contained in this communication is confidential, is
>intended only for the use of the recipient named above, and may be
>legally privileged. If the reader of this message is not the intended
>recipient, you are hereby notified that any dissemination, distribution
>or copying of this communication is strictly prohibited. If you have
>received this communication in error, please resend this communication to
>the sender and delete the original message or any copy of it from your
>computer system. Thank you.
>
***************************************************************************
The information contained in this communication is confidential, is
intended only for the use of the recipient named above, and may be legally
privileged.
If the reader of this message is not the intended recipient, you are
hereby notified that any dissemination, distribution or copying of this
communication is strictly prohibited.
If you have received this communication in error, please resend this
communication to the sender and delete the original message or any copy
of it from your computer system.
Thank You.
****************************************************************************
Re: Hadoop in Eclipse
Posted by Serge Blazhiyevskyy <Se...@nice.com>.
Here is the link to cloudera binaries
https://ccp.cloudera.com/display/SUPPORT/CDH3+Downloadable+Tarballs
Download the file that starts with hadoop
Also, I have some basic instructions on my blog:
Hadoopway.blogspot.com
Good luck!
On 5/7/12 3:40 PM, "Wilson Wayne - wwilso" <Wa...@acxiom.com> wrote:
>Thanks for the quick response. To show just how new I am to all this,
>where would I be able to download those libraries or find them if the
>maven build created them? Also, would I only need the MapReduce ones or
>all of them. Would I need the Hadoop-core.jar that doesn't appear to be
>on my system anywhere?
>
>Sorry for the very basic, newbie questions.
>
>-----Original Message-----
>From: Serge Blazhiyevskyy [mailto:Serge.Blazhiyevskyy@nice.com]
>Sent: Monday, May 07, 2012 5:15 PM
>To: common-user@hadoop.apache.org; core-user@hadoop.apache.org
>Subject: Re: Hadoop in Eclipse
>
>If you want to test your mapper and reducers in eclipse, you can just
>link with hadoop libraries in project settings.
>
>Regards,
>Serge
>
>From: Wilson Wayne - wwilso
><Wa...@acxiom.com>>
>Reply-To:
>"common-user@hadoop.apache.org<ma...@hadoop.apache.org>"
><co...@hadoop.apache.org>>
>Date: Mon, 7 May 2012 16:10:17 -0600
>To: "core-user@hadoop.apache.org<ma...@hadoop.apache.org>"
><co...@hadoop.apache.org>>
>Subject: Hadoop in Eclipse
>
>I'm new to Hadoop and am trying to get it setup in Eclipse. I'm
>following the "Working with Hadoop under Eclipse" wiki to do this. First
>let me make sure this will do what I am hoping it will do. I'm not so
>much interested in modifying the Hadoop code itself (although I may in
>the not too distant future). I'm more looking to be able to code and
>test mappers and reducers in Eclipse and leverage the debugging and other
>tools in Eclipse in that process. If that's not what the wiki is
>describing than let me know that and ignore my issue. If there is a way
>to do this, would you please point me to where I can find the
>instructions. If this is what the wiki is describing than I have an
>issue I need help with.
>
>I've followed the directions on the Wiki and even worked through a couple
>of issues by searching the archive. However, now I've hit an error that
>while I did find it in the archive, the fix that was presented is not
>fixing my issue. After importing all the projects, I get a build error:
>"Project 'MapReduceTools' is missing required library:
>'lib/hadoop-core.jar'". I've added the M2_REPO variable as someone
>suggested in the archive where I found this error once. That hasn't
>fixed or even changed the error. I've looked in my maven repository
>(c:\Users\wwilso\.m2\repository) and there is not a "Hadoop-core.jar"
>anywhere in the repository that windows explorer can find. In fact,
>windows explorer can't find a "Hadoop-core.jar" anywhere on my c:\ drive.
>
>I did my build using the directions from the wiki (got source using git
>in cygwin and then executing the two maven commands in cywgin). I did
>use the patch for the "scripts/saveVersion.sh" not found error to resolve
>that issue. That's the only thing I have changed from the wiki or code.
>Using Maven-3.0.4, ant-1.8.3, and protobuf-2.4.1. Anyone have any ideas?
> Thanks in advance.
>
>
>WAYNE WILSON
>Product Technical Consultant
>Acxiom Corporation
>EML wayne.wilson@acxiom.com<ma...@acxiom.com>
>TEL 501.342.2870
>MBL 972.984.0268
>5213 Rolling Rock Dr, McKinney, TX, 75070, USA
>WWW.ACXIOM.COM<http://WWW.ACXIOM.COM>
>
>
>[cid:image001.gif@01CD2C6F.7A0BD440]<http://www.facebook.com/acxiomcorp>
>[cid:image002.gif@01CD2C6F.7A0BD440]
><http://www.linkedin.com/groupRegistration?gid=2901735>
>[cid:image003.gif@01CD2C6F.7A0BD440] <http://twitter.com/acxiom>
>
>[cid:image004.png@01CD2C6F.7A0BD440]
>________________________________
>The information contained in this communication is confidential, is
>intended only for the use of the recipient named above, and may be
>legally privileged. If the reader of this message is not the intended
>recipient, you are hereby notified that any dissemination, distribution
>or copying of this communication is strictly prohibited. If you have
>received this communication in error, please resend this communication to
>the sender and delete the original message or any copy of it from your
>computer system. Thank you.
>
RE: Hadoop in Eclipse
Posted by Wilson Wayne - wwilso <Wa...@acxiom.com>.
Thanks for the quick response. To show just how new I am to all this, where would I be able to download those libraries or find them if the maven build created them? Also, would I only need the MapReduce ones or all of them. Would I need the Hadoop-core.jar that doesn't appear to be on my system anywhere?
Sorry for the very basic, newbie questions.
-----Original Message-----
From: Serge Blazhiyevskyy [mailto:Serge.Blazhiyevskyy@nice.com]
Sent: Monday, May 07, 2012 5:15 PM
To: common-user@hadoop.apache.org; core-user@hadoop.apache.org
Subject: Re: Hadoop in Eclipse
If you want to test your mapper and reducers in eclipse, you can just link with hadoop libraries in project settings.
Regards,
Serge
From: Wilson Wayne - wwilso <Wa...@acxiom.com>>
Reply-To: "common-user@hadoop.apache.org<ma...@hadoop.apache.org>" <co...@hadoop.apache.org>>
Date: Mon, 7 May 2012 16:10:17 -0600
To: "core-user@hadoop.apache.org<ma...@hadoop.apache.org>" <co...@hadoop.apache.org>>
Subject: Hadoop in Eclipse
I'm new to Hadoop and am trying to get it setup in Eclipse. I'm following the "Working with Hadoop under Eclipse" wiki to do this. First let me make sure this will do what I am hoping it will do. I'm not so much interested in modifying the Hadoop code itself (although I may in the not too distant future). I'm more looking to be able to code and test mappers and reducers in Eclipse and leverage the debugging and other tools in Eclipse in that process. If that's not what the wiki is describing than let me know that and ignore my issue. If there is a way to do this, would you please point me to where I can find the instructions. If this is what the wiki is describing than I have an issue I need help with.
I've followed the directions on the Wiki and even worked through a couple of issues by searching the archive. However, now I've hit an error that while I did find it in the archive, the fix that was presented is not fixing my issue. After importing all the projects, I get a build error: "Project 'MapReduceTools' is missing required library: 'lib/hadoop-core.jar'". I've added the M2_REPO variable as someone suggested in the archive where I found this error once. That hasn't fixed or even changed the error. I've looked in my maven repository (c:\Users\wwilso\.m2\repository) and there is not a "Hadoop-core.jar" anywhere in the repository that windows explorer can find. In fact, windows explorer can't find a "Hadoop-core.jar" anywhere on my c:\ drive.
I did my build using the directions from the wiki (got source using git in cygwin and then executing the two maven commands in cywgin). I did use the patch for the "scripts/saveVersion.sh" not found error to resolve that issue. That's the only thing I have changed from the wiki or code. Using Maven-3.0.4, ant-1.8.3, and protobuf-2.4.1. Anyone have any ideas? Thanks in advance.
WAYNE WILSON
Product Technical Consultant
Acxiom Corporation
EML wayne.wilson@acxiom.com<ma...@acxiom.com>
TEL 501.342.2870
MBL 972.984.0268
5213 Rolling Rock Dr, McKinney, TX, 75070, USA
WWW.ACXIOM.COM<http://WWW.ACXIOM.COM>
[cid:image001.gif@01CD2C6F.7A0BD440]<http://www.facebook.com/acxiomcorp> [cid:image002.gif@01CD2C6F.7A0BD440] <http://www.linkedin.com/groupRegistration?gid=2901735> [cid:image003.gif@01CD2C6F.7A0BD440] <http://twitter.com/acxiom>
[cid:image004.png@01CD2C6F.7A0BD440]
________________________________
The information contained in this communication is confidential, is intended only for the use of the recipient named above, and may be legally privileged. If the reader of this message is not the intended recipient, you are hereby notified that any dissemination, distribution or copying of this communication is strictly prohibited. If you have received this communication in error, please resend this communication to the sender and delete the original message or any copy of it from your computer system. Thank you.
Re: Hadoop in Eclipse
Posted by Denny Lee <De...@microsoft.com>.
Hi Wayne,
For the error message you're receiving – you will need to copy the Hadoop jar files from the Hadoop cluster you're trying to connect to. A couple good references on that call out include:
Configuring Eclipse for Hadoop Development (a screencast …<http://www.google.com/url?sa=t&rct=j&q=setup%20eclipse%20for%20hadoop&source=web&cd=1&ved=0CFoQFjAA&url=http%3A%2F%2Fwww.cloudera.com%2Fblog%2F2009%2F04%2Fconfiguring-eclipse-for-hadoop-development-a-screencast%2F&ei=rP6oT_uJK6OriAKL1bCyAg&usg=AFQjCNFSn9WUJRda4w6assu3NBdaZXOf3g&sig2=5q0pujY-WQyh7V77qJUsnQ>
Hadoop Tutorial – YDN<http://www.google.com/url?sa=t&rct=j&q=setup%20eclipse%20for%20hadoop&source=web&cd=2&ved=0CFsQFjAB&url=http%3A%2F%2Fdeveloper.yahoo.com%2Fhadoop%2Ftutorial%2Fmodule3.html&ei=rP6oT_uJK6OriAKL1bCyAg&usg=AFQjCNH4LIKdAY3IxTnqV5uguF0HyYFZmg&sig2=ywIgtV6bIKiRxnWfhKMVxA>
HTH!
--
denny.lee@microsoft.com<ma...@microsoft.com> | about.me/dennylee<http://about.me/dennylee>
From: Wilson Wayne - wwilso <Wa...@acxiom.com>>
Reply-To: "common-user@hadoop.apache.org<ma...@hadoop.apache.org>" <co...@hadoop.apache.org>>
Date: Monday, May 7, 2012 3:10 PM
To: "core-user@hadoop.apache.org<ma...@hadoop.apache.org>" <co...@hadoop.apache.org>>
Subject: Hadoop in Eclipse
I’m new to Hadoop and am trying to get it setup in Eclipse. I’m following the “Working with Hadoop under Eclipse” wiki to do this. First let me make sure this will do what I am hoping it will do. I’m not so much interested in modifying the Hadoop code itself (although I may in the not too distant future). I’m more looking to be able to code and test mappers and reducers in Eclipse and leverage the debugging and other tools in Eclipse in that process. If that’s not what the wiki is describing than let me know that and ignore my issue. If there is a way to do this, would you please point me to where I can find the instructions. If this is what the wiki is describing than I have an issue I need help with.
I’ve followed the directions on the Wiki and even worked through a couple of issues by searching the archive. However, now I’ve hit an error that while I did find it in the archive, the fix that was presented is not fixing my issue. After importing all the projects, I get a build error: “Project ‘MapReduceTools’ is missing required library: ‘lib/hadoop-core.jar’”. I’ve added the M2_REPO variable as someone suggested in the archive where I found this error once. That hasn’t fixed or even changed the error. I’ve looked in my maven repository (c:\Users\wwilso\.m2\repository) and there is not a “Hadoop-core.jar” anywhere in the repository that windows explorer can find. In fact, windows explorer can’t find a “Hadoop-core.jar” anywhere on my c:\ drive.
I did my build using the directions from the wiki (got source using git in cygwin and then executing the two maven commands in cywgin). I did use the patch for the “scripts/saveVersion.sh” not found error to resolve that issue. That’s the only thing I have changed from the wiki or code. Using Maven-3.0.4, ant-1.8.3, and protobuf-2.4.1. Anyone have any ideas? Thanks in advance.
WAYNE WILSON
Product Technical Consultant
Acxiom Corporation
EML wayne.wilson@acxiom.com<ma...@acxiom.com>
TEL 501.342.2870
MBL 972.984.0268
5213 Rolling Rock Dr, McKinney, TX, 75070, USA
WWW.ACXIOM.COM<http://WWW.ACXIOM.COM>
[Friend Us on Facebook]<http://www.facebook.com/acxiomcorp> [Link Us on LinkedIn] <http://www.linkedin.com/groupRegistration?gid=2901735> [Follow Us on Twitter] <http://twitter.com/acxiom>
[cid:image004.png@01CB84F1.26214350]
________________________________
The information contained in this communication is confidential, is intended only for the use of the recipient named above, and may be legally privileged. If the reader of this message is not the intended recipient, you are hereby notified that any dissemination, distribution or copying of this communication is strictly prohibited. If you have received this communication in error, please resend this communication to the sender and delete the original message or any copy of it from your computer system. Thank you.
Re: Hadoop in Eclipse
Posted by Serge Blazhiyevskyy <Se...@nice.com>.
If you want to test your mapper and reducers in eclipse, you can just link with hadoop libraries in project settings.
Regards,
Serge
From: Wilson Wayne - wwilso <Wa...@acxiom.com>>
Reply-To: "common-user@hadoop.apache.org<ma...@hadoop.apache.org>" <co...@hadoop.apache.org>>
Date: Mon, 7 May 2012 16:10:17 -0600
To: "core-user@hadoop.apache.org<ma...@hadoop.apache.org>" <co...@hadoop.apache.org>>
Subject: Hadoop in Eclipse
I’m new to Hadoop and am trying to get it setup in Eclipse. I’m following the “Working with Hadoop under Eclipse” wiki to do this. First let me make sure this will do what I am hoping it will do. I’m not so much interested in modifying the Hadoop code itself (although I may in the not too distant future). I’m more looking to be able to code and test mappers and reducers in Eclipse and leverage the debugging and other tools in Eclipse in that process. If that’s not what the wiki is describing than let me know that and ignore my issue. If there is a way to do this, would you please point me to where I can find the instructions. If this is what the wiki is describing than I have an issue I need help with.
I’ve followed the directions on the Wiki and even worked through a couple of issues by searching the archive. However, now I’ve hit an error that while I did find it in the archive, the fix that was presented is not fixing my issue. After importing all the projects, I get a build error: “Project ‘MapReduceTools’ is missing required library: ‘lib/hadoop-core.jar’”. I’ve added the M2_REPO variable as someone suggested in the archive where I found this error once. That hasn’t fixed or even changed the error. I’ve looked in my maven repository (c:\Users\wwilso\.m2\repository) and there is not a “Hadoop-core.jar” anywhere in the repository that windows explorer can find. In fact, windows explorer can’t find a “Hadoop-core.jar” anywhere on my c:\ drive.
I did my build using the directions from the wiki (got source using git in cygwin and then executing the two maven commands in cywgin). I did use the patch for the “scripts/saveVersion.sh” not found error to resolve that issue. That’s the only thing I have changed from the wiki or code. Using Maven-3.0.4, ant-1.8.3, and protobuf-2.4.1. Anyone have any ideas? Thanks in advance.
WAYNE WILSON
Product Technical Consultant
Acxiom Corporation
EML wayne.wilson@acxiom.com<ma...@acxiom.com>
TEL 501.342.2870
MBL 972.984.0268
5213 Rolling Rock Dr, McKinney, TX, 75070, USA
WWW.ACXIOM.COM<http://WWW.ACXIOM.COM>
[cid:image001.gif@01CD2C6F.7A0BD440]<http://www.facebook.com/acxiomcorp> [cid:image002.gif@01CD2C6F.7A0BD440] <http://www.linkedin.com/groupRegistration?gid=2901735> [cid:image003.gif@01CD2C6F.7A0BD440] <http://twitter.com/acxiom>
[cid:image004.png@01CD2C6F.7A0BD440]
________________________________
The information contained in this communication is confidential, is intended only for the use of the recipient named above, and may be legally privileged. If the reader of this message is not the intended recipient, you are hereby notified that any dissemination, distribution or copying of this communication is strictly prohibited. If you have received this communication in error, please resend this communication to the sender and delete the original message or any copy of it from your computer system. Thank you.