You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@pig.apache.org by "j.barrett Strausser" <j....@gmail.com> on 2013/04/10 16:05:57 UTC
Pigunit Failure 10.1 running locally
Greetings all,
I am trying to run Pigunit and receiving an error. I had this previously
working, but had to rebuild my local workstation and didn't have everything
I should have had checked in. This is too deep into pig/hadoop for me to
effectively debug.
I'm using scala so my versions/dependencies look like :
libraryDependencies += "org.apache.pig" % "pig" % "0.10.1"
libraryDependencies += "org.apache.hadoop" % "hadoop-common" % "2.0.3-alpha"
libraryDependencies += "org.apache.hadoop" % "hadoop-core" % "1.1.2"
I built the pigunit.jar file by hand as instructed in the docs.
Any thoughts?
The error is :
java.lang.VerifyError: (class:
org/apache/hadoop/mapred/JobTrackerInstrumentation, method: create
signature:
(Lorg/apache/hadoop/mapred/JobTracker;Lorg/apache/hadoop/mapred/JobConf;)Lorg/apache/hadoop/mapred/JobTrackerInstrumentation;)
Incompatible argument to function
The stack trace is :
[info] at
org.apache.hadoop.mapred.LocalJobRunner.<init>(LocalJobRunner.java:420)
[info] at org.apache.hadoop.mapred.JobClient.init(JobClient.java:488)
[info] at org.apache.hadoop.mapred.JobClient.<init>(JobClient.java:473)
[info] at
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher.launchPig(MapReduceLauncher.java:156)
[info] at org.apache.pig.PigServer.launchPlan(PigServer.java:1279)
[info] at
org.apache.pig.PigServer.executeCompiledLogicalPlan(PigServer.java:1264)
[info] at org.apache.pig.PigServer.storeEx(PigServer.java:961)
[info] at org.apache.pig.PigServer.store(PigServer.java:928)
[info] at org.apache.pig.PigServer.openIterator(PigServer.java:841)
[info] at org.apache.pig.pigunit.PigTest.getAlias(PigTest.java:183)
[info] ...
--
https://github.com/bearrito
@barrettsmash
Re: Pigunit Failure 10.1 running locally
Posted by "j.barrett Strausser" <j....@gmail.com>.
Yeah like I said the versions in the got commit I posted work.
We can mark this as closed
On Apr 10, 2013 1:20 PM, "MARCOS MEDRADO RUBINELLI" <
marcosm@buscapecompany.com> wrote:
> That VerifyError is probably a result of mixing Haddop 1.x and 2.x
> libraries. Check what version your pig was built against and update your
> dependencies to match.
>
> Regards,
> Marcos
>
> On 10-04-2013 11:05, j.barrett Strausser wrote:
> > Greetings all,
> >
> > I am trying to run Pigunit and receiving an error. I had this previously
> > working, but had to rebuild my local workstation and didn't have
> everything
> > I should have had checked in. This is too deep into pig/hadoop for me to
> > effectively debug.
> >
> > I'm using scala so my versions/dependencies look like :
> >
> > libraryDependencies += "org.apache.pig" % "pig" % "0.10.1"
> >
> > libraryDependencies += "org.apache.hadoop" % "hadoop-common" %
> "2.0.3-alpha"
> >
> > libraryDependencies += "org.apache.hadoop" % "hadoop-core" % "1.1.2"
> >
> > I built the pigunit.jar file by hand as instructed in the docs.
> >
> > Any thoughts?
> >
> > The error is :
> >
> > java.lang.VerifyError: (class:
> > org/apache/hadoop/mapred/JobTrackerInstrumentation, method: create
> > signature:
> >
> (Lorg/apache/hadoop/mapred/JobTracker;Lorg/apache/hadoop/mapred/JobConf;)Lorg/apache/hadoop/mapred/JobTrackerInstrumentation;)
> > Incompatible argument to function
>
Re: Pigunit Failure 10.1 running locally
Posted by MARCOS MEDRADO RUBINELLI <ma...@buscapecompany.com>.
That VerifyError is probably a result of mixing Haddop 1.x and 2.x
libraries. Check what version your pig was built against and update your
dependencies to match.
Regards,
Marcos
On 10-04-2013 11:05, j.barrett Strausser wrote:
> Greetings all,
>
> I am trying to run Pigunit and receiving an error. I had this previously
> working, but had to rebuild my local workstation and didn't have everything
> I should have had checked in. This is too deep into pig/hadoop for me to
> effectively debug.
>
> I'm using scala so my versions/dependencies look like :
>
> libraryDependencies += "org.apache.pig" % "pig" % "0.10.1"
>
> libraryDependencies += "org.apache.hadoop" % "hadoop-common" % "2.0.3-alpha"
>
> libraryDependencies += "org.apache.hadoop" % "hadoop-core" % "1.1.2"
>
> I built the pigunit.jar file by hand as instructed in the docs.
>
> Any thoughts?
>
> The error is :
>
> java.lang.VerifyError: (class:
> org/apache/hadoop/mapred/JobTrackerInstrumentation, method: create
> signature:
> (Lorg/apache/hadoop/mapred/JobTracker;Lorg/apache/hadoop/mapred/JobConf;)Lorg/apache/hadoop/mapred/JobTrackerInstrumentation;)
> Incompatible argument to function
Re: Pigunit Failure 10.1 running locally
Posted by "j.barrett Strausser" <j....@gmail.com>.
Yeah, I think for some reason, the pig version I was using had a dependency
on hadoop-core that had not been baked into hadoop common, but I'm not
really sure as I was piecing together some backups to get back to working.
I just sorted out all the dependencies issues a moment ago.
Below is my build.sbt file. If you aren't using scala it might not make
sense, but its easy enough to read off. I can confirm that all the versions
listed there are compatible with pigunit built from the 0.10.1 release.
https://github.com/bearrito/scala-pig/commit/4d287a535ac9d395a72f2fc8f53d9b66737d010c
All this headache is what I get for not pushing my work upstream.
Thanks,
barrett
On Wed, Apr 10, 2013 at 11:37 AM, Prashant Kommireddi
<pr...@gmail.com>wrote:
> What Hadoop version are you trying to use? Looks like you have 2
> different versions (1.X and 2.X) listed as dependencies.
>
> Sent from my iPhone
>
> On Apr 10, 2013, at 7:06 AM, "j.barrett Strausser"
> <j....@gmail.com> wrote:
>
> > Greetings all,
> >
> > I am trying to run Pigunit and receiving an error. I had this previously
> > working, but had to rebuild my local workstation and didn't have
> everything
> > I should have had checked in. This is too deep into pig/hadoop for me to
> > effectively debug.
> >
> > I'm using scala so my versions/dependencies look like :
> >
> > libraryDependencies += "org.apache.pig" % "pig" % "0.10.1"
> >
> > libraryDependencies += "org.apache.hadoop" % "hadoop-common" %
> "2.0.3-alpha"
> >
> > libraryDependencies += "org.apache.hadoop" % "hadoop-core" % "1.1.2"
> >
> > I built the pigunit.jar file by hand as instructed in the docs.
> >
> > Any thoughts?
> >
> > The error is :
> >
> > java.lang.VerifyError: (class:
> > org/apache/hadoop/mapred/JobTrackerInstrumentation, method: create
> > signature:
> >
> (Lorg/apache/hadoop/mapred/JobTracker;Lorg/apache/hadoop/mapred/JobConf;)Lorg/apache/hadoop/mapred/JobTrackerInstrumentation;)
> > Incompatible argument to function
> >
> >
> > The stack trace is :
> >
> > [info] at
> > org.apache.hadoop.mapred.LocalJobRunner.<init>(LocalJobRunner.java:420)
> > [info] at org.apache.hadoop.mapred.JobClient.init(JobClient.java:488)
> > [info] at org.apache.hadoop.mapred.JobClient.<init>(JobClient.java:473)
> > [info] at
> >
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher.launchPig(MapReduceLauncher.java:156)
> > [info] at org.apache.pig.PigServer.launchPlan(PigServer.java:1279)
> > [info] at
> > org.apache.pig.PigServer.executeCompiledLogicalPlan(PigServer.java:1264)
> > [info] at org.apache.pig.PigServer.storeEx(PigServer.java:961)
> > [info] at org.apache.pig.PigServer.store(PigServer.java:928)
> > [info] at org.apache.pig.PigServer.openIterator(PigServer.java:841)
> > [info] at org.apache.pig.pigunit.PigTest.getAlias(PigTest.java:183)
> > [info] ...
> >
> >
> >
> >
> >
> > --
> >
> >
> > https://github.com/bearrito
> > @barrettsmash
>
--
https://github.com/bearrito
@barrettsmash
Re: Pigunit Failure 10.1 running locally
Posted by Prashant Kommireddi <pr...@gmail.com>.
What Hadoop version are you trying to use? Looks like you have 2
different versions (1.X and 2.X) listed as dependencies.
Sent from my iPhone
On Apr 10, 2013, at 7:06 AM, "j.barrett Strausser"
<j....@gmail.com> wrote:
> Greetings all,
>
> I am trying to run Pigunit and receiving an error. I had this previously
> working, but had to rebuild my local workstation and didn't have everything
> I should have had checked in. This is too deep into pig/hadoop for me to
> effectively debug.
>
> I'm using scala so my versions/dependencies look like :
>
> libraryDependencies += "org.apache.pig" % "pig" % "0.10.1"
>
> libraryDependencies += "org.apache.hadoop" % "hadoop-common" % "2.0.3-alpha"
>
> libraryDependencies += "org.apache.hadoop" % "hadoop-core" % "1.1.2"
>
> I built the pigunit.jar file by hand as instructed in the docs.
>
> Any thoughts?
>
> The error is :
>
> java.lang.VerifyError: (class:
> org/apache/hadoop/mapred/JobTrackerInstrumentation, method: create
> signature:
> (Lorg/apache/hadoop/mapred/JobTracker;Lorg/apache/hadoop/mapred/JobConf;)Lorg/apache/hadoop/mapred/JobTrackerInstrumentation;)
> Incompatible argument to function
>
>
> The stack trace is :
>
> [info] at
> org.apache.hadoop.mapred.LocalJobRunner.<init>(LocalJobRunner.java:420)
> [info] at org.apache.hadoop.mapred.JobClient.init(JobClient.java:488)
> [info] at org.apache.hadoop.mapred.JobClient.<init>(JobClient.java:473)
> [info] at
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher.launchPig(MapReduceLauncher.java:156)
> [info] at org.apache.pig.PigServer.launchPlan(PigServer.java:1279)
> [info] at
> org.apache.pig.PigServer.executeCompiledLogicalPlan(PigServer.java:1264)
> [info] at org.apache.pig.PigServer.storeEx(PigServer.java:961)
> [info] at org.apache.pig.PigServer.store(PigServer.java:928)
> [info] at org.apache.pig.PigServer.openIterator(PigServer.java:841)
> [info] at org.apache.pig.pigunit.PigTest.getAlias(PigTest.java:183)
> [info] ...
>
>
>
>
>
> --
>
>
> https://github.com/bearrito
> @barrettsmash