You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-user@hadoop.apache.org by Evert Lammerts <Ev...@sara.nl> on 2010/11/24 11:20:49 UTC

eclipse plugin

Hi list,

I've installed the eclipse plugin on Eclipse Helios. In order to get it to
work I had to replace hadoop-core.jar in the plugin by the one shipped in
CDH3, and now I can browse the filesystem. But MR is still complaining.
Eclipse tells me this:

An internal error occurred during: "Map/Reduce location status updater".
org/codehaus/jackson/map/JsonMappingException

And ~/workspace/.metadata/.log sais:

!ENTRY org.eclipse.core.jobs 4 2 2010-11-24 11:13:52.468
!MESSAGE An internal error occurred during: "Map/Reduce location status
updater".
!STACK 0
java.lang.NoClassDefFoundError:
org/codehaus/jackson/map/JsonMappingException
        at
org.apache.hadoop.eclipse.server.HadoopServer.getJobClient(HadoopServer.java
:473)
        at
org.apache.hadoop.eclipse.server.HadoopServer$LocationStatusUpdater.run(Hado
opServer.java:102)
        at org.eclipse.core.internal.jobs.Worker.run(Worker.java:54)
Caused by: java.lang.ClassNotFoundException:
org.codehaus.jackson.map.JsonMappingException
        at
org.eclipse.osgi.internal.loader.BundleLoader.findClassInternal(BundleLoader
.java:506)
        at
org.eclipse.osgi.internal.loader.BundleLoader.findClass(BundleLoader.java:42
2)
        at
org.eclipse.osgi.internal.loader.BundleLoader.findClass(BundleLoader.java:41
0)
        at
org.eclipse.osgi.internal.baseadaptor.DefaultClassLoader.loadClass(DefaultCl
assLoader.java:107)
        at java.lang.ClassLoader.loadClass(ClassLoader.java:248)
        ... 3 more


I guess this means that it cannot find the Jackson libraries, so they're
probably not on the plugin's classpath. So I tried including the Jackson
libraries in the plugin, without any luck. (I have of course pointed the
plugin to my Hadoop installation directory as well, at /usr/lib/hadoop, and,
hoping it included all libraries in this path, I've also pointed it to
hadoop's lib subdir).

Has anybody been successful with this plugin in Eclipse Helios?

Cheers,

Evert Lammerts
Consultant eScience Support
SARA Computing & Network Services 
High Performance Computing & Visualization

Phone: +31 20 888 4101
Email: evert.lammerts@sara.nl
http://www.sara.nl

Re: eclipse plugin

Posted by Steve Loughran <st...@apache.org>.
On 24/11/10 20:54, Evert Lammerts wrote:
> Hi all,
>
> I dug a bit deeper, and I'm guessing that the use of this plugin is not recommended anymore in the 0.20 API. Is that right? It still tells me to extend MapReduceBase, for example.
>
> How do people develop their MR applications here? Is it possible to easily run Mappers and Reducers from an IDE like Eclipse?
>
> Kind regards,
> Evert
>

1. we code in our IDE of choice, be it eclipse, intellij IDEa, netbeans 
or something else

2. we bring up in VM hadoop clusters and run work in it using the 
MiniMRCluster class

3. we use MRUnit to do more structured testing

4. we submit jobs to real clusters with test datasets and look at the logs

RE: eclipse plugin

Posted by Evert Lammerts <Ev...@sara.nl>.
Hi all,

I dug a bit deeper, and I'm guessing that the use of this plugin is not recommended anymore in the 0.20 API. Is that right? It still tells me to extend MapReduceBase, for example.

How do people develop their MR applications here? Is it possible to easily run Mappers and Reducers from an IDE like Eclipse?

Kind regards,
Evert

________________________________________
From: Evert Lammerts [Evert.Lammerts@sara.nl]
Sent: Wednesday, November 24, 2010 11:20 AM
To: 'cdh-user@cloudera.org'; 'common-user@hadoop.apache.org'
Subject: eclipse plugin

Hi list,

I've installed the eclipse plugin on Eclipse Helios. In order to get it to
work I had to replace hadoop-core.jar in the plugin by the one shipped in
CDH3, and now I can browse the filesystem. But MR is still complaining.
Eclipse tells me this:

An internal error occurred during: "Map/Reduce location status updater".
org/codehaus/jackson/map/JsonMappingException

And ~/workspace/.metadata/.log sais:

!ENTRY org.eclipse.core.jobs 4 2 2010-11-24 11:13:52.468
!MESSAGE An internal error occurred during: "Map/Reduce location status
updater".
!STACK 0
java.lang.NoClassDefFoundError:
org/codehaus/jackson/map/JsonMappingException
        at
org.apache.hadoop.eclipse.server.HadoopServer.getJobClient(HadoopServer.java
:473)
        at
org.apache.hadoop.eclipse.server.HadoopServer$LocationStatusUpdater.run(Hado
opServer.java:102)
        at org.eclipse.core.internal.jobs.Worker.run(Worker.java:54)
Caused by: java.lang.ClassNotFoundException:
org.codehaus.jackson.map.JsonMappingException
        at
org.eclipse.osgi.internal.loader.BundleLoader.findClassInternal(BundleLoader
.java:506)
        at
org.eclipse.osgi.internal.loader.BundleLoader.findClass(BundleLoader.java:42
2)
        at
org.eclipse.osgi.internal.loader.BundleLoader.findClass(BundleLoader.java:41
0)
        at
org.eclipse.osgi.internal.baseadaptor.DefaultClassLoader.loadClass(DefaultCl
assLoader.java:107)
        at java.lang.ClassLoader.loadClass(ClassLoader.java:248)
        ... 3 more


I guess this means that it cannot find the Jackson libraries, so they're
probably not on the plugin's classpath. So I tried including the Jackson
libraries in the plugin, without any luck. (I have of course pointed the
plugin to my Hadoop installation directory as well, at /usr/lib/hadoop, and,
hoping it included all libraries in this path, I've also pointed it to
hadoop's lib subdir).

Has anybody been successful with this plugin in Eclipse Helios?

Cheers,

Evert Lammerts
Consultant eScience Support
SARA Computing & Network Services
High Performance Computing & Visualization

Phone: +31 20 888 4101
Email: evert.lammerts@sara.nl
http://www.sara.nl