You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@pig.apache.org by Weiping Qu <qu...@informatik.uni-kl.de> on 2013/06/10 22:58:08 UTC

running pig from eclipse on hadoop cluster

Hi,

I am currently running pig from eclipse on hadoop cluster.
I added the hadoop conf location to the runtime configuration.
But the mapreduce jobs failed as the built class files of pig cannot be 
called by hadoop.
I added class file location to the classpath, but it did not work.
Any hints?

java.lang.RuntimeException: java.lang.ClassNotFoundException: org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigOutputFormat
	at org.apache.hadoop.conf.Configuration.getClass(Configuration.java:866)
	at org.apache.hadoop.mapreduce.JobContext.getOutputFormatClass(JobContext.java:231)
	at org.apache.hadoop.mapred.Task.initialize(Task.java:488)
	at org.apache.hadoop.mapred.MapTask.run(MapTask.java:352)
	at org.apache.hadoop.mapred.Child$4.run(Child.java:259)
	at java.security.AccessController.doPrivileged(Native Method)
	at javax.security.auth.Subject.doAs(Subject.java:415)
	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1059)
	at org.apache.hadoop.mapred.Child.main(Child.java:253)
Caused by: java.lang.ClassNotFoundException: org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigOutputFormat
	at java.net.URLClassLoader$1.run(URLClassLoader.java:366)
	at java.net.URLClassLoader$1.run(URLClassLoader.java:355)
	at java.security.AccessController.doPrivileged(Native Method)
	at java.net.URLClassLoader.findClass(URLClassLoader.java:354)
	at java.lang.ClassLoader.loadClass(ClassLoader.java:423)
	at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308)
	at java.lang.ClassLoader.loadClass(ClassLoader.java:356)
	at java.lang.Class.forName0(Native Method)
	at java.lang.Class.forName(Class.java:264)
	at org.apache.hadoop.conf.Configuration.getClassByName(Configuration.java:819)
	at org.apache.hadoop.conf.Configuration.getClass(Configuration.java:864)
	... 8 more


Best,
Weiping

Re: running pig from eclipse on hadoop cluster

Posted by Weiping Qu <qu...@informatik.uni-kl.de>.
Hi,

Forget the question raised before. It's solved.

> Hi,
>
> I am currently running pig from eclipse on hadoop cluster.
> I added the hadoop conf location to the runtime configuration.
> But the mapreduce jobs failed as the built class files of pig cannot 
> be called by hadoop.
> I added class file location to the classpath, but it did not work.
> Any hints?
>
> java.lang.RuntimeException: java.lang.ClassNotFoundException: 
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigOutputFormat
>     at 
> org.apache.hadoop.conf.Configuration.getClass(Configuration.java:866)
>     at 
> org.apache.hadoop.mapreduce.JobContext.getOutputFormatClass(JobContext.java:231)
>     at org.apache.hadoop.mapred.Task.initialize(Task.java:488)
>     at org.apache.hadoop.mapred.MapTask.run(MapTask.java:352)
>     at org.apache.hadoop.mapred.Child$4.run(Child.java:259)
>     at java.security.AccessController.doPrivileged(Native Method)
>     at javax.security.auth.Subject.doAs(Subject.java:415)
>     at 
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1059)
>     at org.apache.hadoop.mapred.Child.main(Child.java:253)
> Caused by: java.lang.ClassNotFoundException: 
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigOutputFormat
>     at java.net.URLClassLoader$1.run(URLClassLoader.java:366)
>     at java.net.URLClassLoader$1.run(URLClassLoader.java:355)
>     at java.security.AccessController.doPrivileged(Native Method)
>     at java.net.URLClassLoader.findClass(URLClassLoader.java:354)
>     at java.lang.ClassLoader.loadClass(ClassLoader.java:423)
>     at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308)
>     at java.lang.ClassLoader.loadClass(ClassLoader.java:356)
>     at java.lang.Class.forName0(Native Method)
>     at java.lang.Class.forName(Class.java:264)
>     at 
> org.apache.hadoop.conf.Configuration.getClassByName(Configuration.java:819)
>     at 
> org.apache.hadoop.conf.Configuration.getClass(Configuration.java:864)
>     ... 8 more
>
>
> Best,
> Weiping
>
>
>