You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@pig.apache.org by Matthew Hayes <mh...@linkedin.com> on 2012/06/08 21:07:59 UTC
NoClassDefFoundError after upgrading to pig 0.10.0 from 0.9.0
I'm using ivy to download dependencies for pig, but after updating the version to 0.10.0 I am getting errors in my unit tests:
[testng] java.lang.NoClassDefFoundError: org/codehaus/jackson/map/util/LRUMap
[testng] at org.apache.pig.builtin.JsonMetadata.<init>(JsonMetadata.java:75)
[testng] at org.apache.pig.builtin.PigStorage.getSchema(PigStorage.java:466)
[testng] at org.apache.pig.newplan.logical.relational.LOLoad.getSchemaFromMetaData(LOLoad.java:151)
[testng] at org.apache.pig.newplan.logical.relational.LOLoad.getSchema(LOLoad.java:110)
...etc.
It looks like the dependency on jackson-mapper-asl is coming from avro. Looking in jackson-mapper-asl-1.7.3.jar I see the LRUMap class. However using ivy to pull down dependencies I get jackson-mapper-asl-1.4.2.jar, not jackson-mapper-asl-1.7.3.jar. When I build pig 0.10.0 from source it uses 1.7.3.
Is there an inconsistency between the avro version pig is compiled against and the avro version specified in the pom?
Thanks,
Matt
RE: NoClassDefFoundError after upgrading to pig 0.10.0 from 0.9.0
Posted by Matthew Hayes <mh...@linkedin.com>.
https://issues.apache.org/jira/browse/PIG-2789
________________________________________
From: Matthew Hayes [mhayes@linkedin.com]
Sent: Monday, July 02, 2012 11:22 PM
To: user@pig.apache.org
Subject: RE: NoClassDefFoundError after upgrading to pig 0.10.0 from 0.9.0
Sure thanks for confirming, will do :)
-Matt
________________________________________
From: Gianmarco De Francisci Morales [gdfm@apache.org]
Sent: Monday, July 02, 2012 6:36 AM
To: user@pig.apache.org
Subject: Re: NoClassDefFoundError after upgrading to pig 0.10.0 from 0.9.0
We can simply generate the pom dynamically as we already do with the
ivy.xml file.
Cheers,
--
Gianmarco
On Mon, Jul 2, 2012 at 3:58 AM, Dmitriy Ryaboy <dv...@gmail.com> wrote:
> Yep, you are right, the pom is not generated, but checked in
> statically; looks like it's out of date. One more reason to mavenize
> Pig.. can you open a Jira for this? We'll fix.
>
> D
>
> On Mon, Jun 11, 2012 at 10:15 AM, Matthew Hayes <mh...@linkedin.com>
> wrote:
> > What I mean is: are the versions in the pom for pig 0.10.0 inconsistent
> with the versions specified in the ivy file used to build pig? To be
> clear, I am building a separate project, and I am getting pig and its
> dependencies using ivy.
> >
> > Looking in ivy.xml in the pig 0.10.0 release:
> >
> > <dependency org="org.apache.avro" name="avro" rev="${avro.version}"
> > conf="compile->default;checkstyle->master"/>
> >
> > ...
> >
> > <dependency org="org.codehaus.jackson" name="jackson-mapper-asl"
> rev="${jackson.version}"
> > conf="compile->master"/>
> > <dependency org="org.codehaus.jackson" name="jackson-core-asl"
> rev="${jackson.version}"
> > conf="compile->master"/>
> >
> > Where avro.version is avro.version=1.5.3 and jackson.version=1.7.3.
> >
> > However, in the pom.xml for pig 0.10.0:
> >
> > <groupId>org.apache.hadoop</groupId>
> > <artifactId>avro</artifactId>
> > <version>1.3.2</version>
> >
> > And when I look up the pom for org.apache.hadoop's avro 1.3.2 in the
> central repository, I see a version of jackson inconsistent with what pig
> was compiled with:
> >
> > <dependency>
> > <groupId>org.codehaus.jackson</groupId>
> > <artifactId>jackson-mapper-asl</artifactId>
> > <version>1.4.2</version>
> > <scope>compile</scope>
> > </dependency>
> >
> > It's 1.4.2, not 1.7.3.
> >
> > Am I doing something wrong here? Below is my ivy.xml. It's the same as
> what I used for 0.9.0 but I changed the pig version to 0.10.0.
> >
> > <ivy-module version="2.0">
> > <info organisation="datafu" module="datafu"/>
> > <dependencies>
> > <dependency org="org.apache.pig" name="pig" rev="0.10.0"/>
> > <dependency org="it.unimi.dsi" name="fastutil" rev="6.3"/>
> > <dependency org="joda-time" name="joda-time" rev="1.6"/>
> > <dependency org="org.apache.commons" name="commons-math"
> rev="2.1"/>
> > <dependency org="commons-io" name="commons-io" rev="1.4"/>
> > <dependency org="org.apache.hadoop" name="hadoop-core"
> rev="0.20.2"/>
> > <dependency org="org.testng" name="testng" rev="6.2"/>
> > <dependency org="com.google.guava" name="guava" rev="r06" />
> >
> > <!-- needed for pigunit to work -->
> > <dependency org="log4j" name="log4j" rev="1.2.14" />
> > <dependency org="jline" name="jline" rev="0.9.94" />
> > <dependency org="org.antlr" name="antlr" rev="3.2" />
> > </dependencies>
> > </ivy-module>
> >
> > Thanks,
> > Matt
> > ________________________________________
> > From: Dmitriy Ryaboy [dvryaboy@gmail.com]
> > Sent: Sunday, June 10, 2012 5:05 PM
> > To: user@pig.apache.org
> > Subject: Re: NoClassDefFoundError after upgrading to pig 0.10.0 from
> 0.9.0
> >
> > I am not sure what you mean -- ivy pulls down 1.4.2 but pig builds from
> 1.7.3? Pig gets its jars from ivy, so it must be building from whatever ivy
> pulls down. Sounds like there is some packaging problem, but I am not sure
> how to interpret what you said.
> >
> > On Jun 8, 2012, at 12:07 PM, Matthew Hayes <mh...@linkedin.com> wrote:
> >
> >> I'm using ivy to download dependencies for pig, but after updating the
> version to 0.10.0 I am getting errors in my unit tests:
> >>
> >> [testng] java.lang.NoClassDefFoundError:
> org/codehaus/jackson/map/util/LRUMap
> >> [testng] at
> org.apache.pig.builtin.JsonMetadata.<init>(JsonMetadata.java:75)
> >> [testng] at
> org.apache.pig.builtin.PigStorage.getSchema(PigStorage.java:466)
> >> [testng] at
> org.apache.pig.newplan.logical.relational.LOLoad.getSchemaFromMetaData(LOLoad.java:151)
> >> [testng] at
> org.apache.pig.newplan.logical.relational.LOLoad.getSchema(LOLoad.java:110)
> >> ...etc.
> >>
> >> It looks like the dependency on jackson-mapper-asl is coming from avro.
> Looking in jackson-mapper-asl-1.7.3.jar I see the LRUMap class. However
> using ivy to pull down dependencies I get jackson-mapper-asl-1.4.2.jar, not
> jackson-mapper-asl-1.7.3.jar. When I build pig 0.10.0 from source it uses
> 1.7.3.
> >>
> >> Is there an inconsistency between the avro version pig is compiled
> against and the avro version specified in the pom?
> >>
> >> Thanks,
> >> Matt
>
RE: NoClassDefFoundError after upgrading to pig 0.10.0 from 0.9.0
Posted by Matthew Hayes <mh...@linkedin.com>.
Sure thanks for confirming, will do :)
-Matt
________________________________________
From: Gianmarco De Francisci Morales [gdfm@apache.org]
Sent: Monday, July 02, 2012 6:36 AM
To: user@pig.apache.org
Subject: Re: NoClassDefFoundError after upgrading to pig 0.10.0 from 0.9.0
We can simply generate the pom dynamically as we already do with the
ivy.xml file.
Cheers,
--
Gianmarco
On Mon, Jul 2, 2012 at 3:58 AM, Dmitriy Ryaboy <dv...@gmail.com> wrote:
> Yep, you are right, the pom is not generated, but checked in
> statically; looks like it's out of date. One more reason to mavenize
> Pig.. can you open a Jira for this? We'll fix.
>
> D
>
> On Mon, Jun 11, 2012 at 10:15 AM, Matthew Hayes <mh...@linkedin.com>
> wrote:
> > What I mean is: are the versions in the pom for pig 0.10.0 inconsistent
> with the versions specified in the ivy file used to build pig? To be
> clear, I am building a separate project, and I am getting pig and its
> dependencies using ivy.
> >
> > Looking in ivy.xml in the pig 0.10.0 release:
> >
> > <dependency org="org.apache.avro" name="avro" rev="${avro.version}"
> > conf="compile->default;checkstyle->master"/>
> >
> > ...
> >
> > <dependency org="org.codehaus.jackson" name="jackson-mapper-asl"
> rev="${jackson.version}"
> > conf="compile->master"/>
> > <dependency org="org.codehaus.jackson" name="jackson-core-asl"
> rev="${jackson.version}"
> > conf="compile->master"/>
> >
> > Where avro.version is avro.version=1.5.3 and jackson.version=1.7.3.
> >
> > However, in the pom.xml for pig 0.10.0:
> >
> > <groupId>org.apache.hadoop</groupId>
> > <artifactId>avro</artifactId>
> > <version>1.3.2</version>
> >
> > And when I look up the pom for org.apache.hadoop's avro 1.3.2 in the
> central repository, I see a version of jackson inconsistent with what pig
> was compiled with:
> >
> > <dependency>
> > <groupId>org.codehaus.jackson</groupId>
> > <artifactId>jackson-mapper-asl</artifactId>
> > <version>1.4.2</version>
> > <scope>compile</scope>
> > </dependency>
> >
> > It's 1.4.2, not 1.7.3.
> >
> > Am I doing something wrong here? Below is my ivy.xml. It's the same as
> what I used for 0.9.0 but I changed the pig version to 0.10.0.
> >
> > <ivy-module version="2.0">
> > <info organisation="datafu" module="datafu"/>
> > <dependencies>
> > <dependency org="org.apache.pig" name="pig" rev="0.10.0"/>
> > <dependency org="it.unimi.dsi" name="fastutil" rev="6.3"/>
> > <dependency org="joda-time" name="joda-time" rev="1.6"/>
> > <dependency org="org.apache.commons" name="commons-math"
> rev="2.1"/>
> > <dependency org="commons-io" name="commons-io" rev="1.4"/>
> > <dependency org="org.apache.hadoop" name="hadoop-core"
> rev="0.20.2"/>
> > <dependency org="org.testng" name="testng" rev="6.2"/>
> > <dependency org="com.google.guava" name="guava" rev="r06" />
> >
> > <!-- needed for pigunit to work -->
> > <dependency org="log4j" name="log4j" rev="1.2.14" />
> > <dependency org="jline" name="jline" rev="0.9.94" />
> > <dependency org="org.antlr" name="antlr" rev="3.2" />
> > </dependencies>
> > </ivy-module>
> >
> > Thanks,
> > Matt
> > ________________________________________
> > From: Dmitriy Ryaboy [dvryaboy@gmail.com]
> > Sent: Sunday, June 10, 2012 5:05 PM
> > To: user@pig.apache.org
> > Subject: Re: NoClassDefFoundError after upgrading to pig 0.10.0 from
> 0.9.0
> >
> > I am not sure what you mean -- ivy pulls down 1.4.2 but pig builds from
> 1.7.3? Pig gets its jars from ivy, so it must be building from whatever ivy
> pulls down. Sounds like there is some packaging problem, but I am not sure
> how to interpret what you said.
> >
> > On Jun 8, 2012, at 12:07 PM, Matthew Hayes <mh...@linkedin.com> wrote:
> >
> >> I'm using ivy to download dependencies for pig, but after updating the
> version to 0.10.0 I am getting errors in my unit tests:
> >>
> >> [testng] java.lang.NoClassDefFoundError:
> org/codehaus/jackson/map/util/LRUMap
> >> [testng] at
> org.apache.pig.builtin.JsonMetadata.<init>(JsonMetadata.java:75)
> >> [testng] at
> org.apache.pig.builtin.PigStorage.getSchema(PigStorage.java:466)
> >> [testng] at
> org.apache.pig.newplan.logical.relational.LOLoad.getSchemaFromMetaData(LOLoad.java:151)
> >> [testng] at
> org.apache.pig.newplan.logical.relational.LOLoad.getSchema(LOLoad.java:110)
> >> ...etc.
> >>
> >> It looks like the dependency on jackson-mapper-asl is coming from avro.
> Looking in jackson-mapper-asl-1.7.3.jar I see the LRUMap class. However
> using ivy to pull down dependencies I get jackson-mapper-asl-1.4.2.jar, not
> jackson-mapper-asl-1.7.3.jar. When I build pig 0.10.0 from source it uses
> 1.7.3.
> >>
> >> Is there an inconsistency between the avro version pig is compiled
> against and the avro version specified in the pom?
> >>
> >> Thanks,
> >> Matt
>
Re: NoClassDefFoundError after upgrading to pig 0.10.0 from 0.9.0
Posted by Gianmarco De Francisci Morales <gd...@apache.org>.
We can simply generate the pom dynamically as we already do with the
ivy.xml file.
Cheers,
--
Gianmarco
On Mon, Jul 2, 2012 at 3:58 AM, Dmitriy Ryaboy <dv...@gmail.com> wrote:
> Yep, you are right, the pom is not generated, but checked in
> statically; looks like it's out of date. One more reason to mavenize
> Pig.. can you open a Jira for this? We'll fix.
>
> D
>
> On Mon, Jun 11, 2012 at 10:15 AM, Matthew Hayes <mh...@linkedin.com>
> wrote:
> > What I mean is: are the versions in the pom for pig 0.10.0 inconsistent
> with the versions specified in the ivy file used to build pig? To be
> clear, I am building a separate project, and I am getting pig and its
> dependencies using ivy.
> >
> > Looking in ivy.xml in the pig 0.10.0 release:
> >
> > <dependency org="org.apache.avro" name="avro" rev="${avro.version}"
> > conf="compile->default;checkstyle->master"/>
> >
> > ...
> >
> > <dependency org="org.codehaus.jackson" name="jackson-mapper-asl"
> rev="${jackson.version}"
> > conf="compile->master"/>
> > <dependency org="org.codehaus.jackson" name="jackson-core-asl"
> rev="${jackson.version}"
> > conf="compile->master"/>
> >
> > Where avro.version is avro.version=1.5.3 and jackson.version=1.7.3.
> >
> > However, in the pom.xml for pig 0.10.0:
> >
> > <groupId>org.apache.hadoop</groupId>
> > <artifactId>avro</artifactId>
> > <version>1.3.2</version>
> >
> > And when I look up the pom for org.apache.hadoop's avro 1.3.2 in the
> central repository, I see a version of jackson inconsistent with what pig
> was compiled with:
> >
> > <dependency>
> > <groupId>org.codehaus.jackson</groupId>
> > <artifactId>jackson-mapper-asl</artifactId>
> > <version>1.4.2</version>
> > <scope>compile</scope>
> > </dependency>
> >
> > It's 1.4.2, not 1.7.3.
> >
> > Am I doing something wrong here? Below is my ivy.xml. It's the same as
> what I used for 0.9.0 but I changed the pig version to 0.10.0.
> >
> > <ivy-module version="2.0">
> > <info organisation="datafu" module="datafu"/>
> > <dependencies>
> > <dependency org="org.apache.pig" name="pig" rev="0.10.0"/>
> > <dependency org="it.unimi.dsi" name="fastutil" rev="6.3"/>
> > <dependency org="joda-time" name="joda-time" rev="1.6"/>
> > <dependency org="org.apache.commons" name="commons-math"
> rev="2.1"/>
> > <dependency org="commons-io" name="commons-io" rev="1.4"/>
> > <dependency org="org.apache.hadoop" name="hadoop-core"
> rev="0.20.2"/>
> > <dependency org="org.testng" name="testng" rev="6.2"/>
> > <dependency org="com.google.guava" name="guava" rev="r06" />
> >
> > <!-- needed for pigunit to work -->
> > <dependency org="log4j" name="log4j" rev="1.2.14" />
> > <dependency org="jline" name="jline" rev="0.9.94" />
> > <dependency org="org.antlr" name="antlr" rev="3.2" />
> > </dependencies>
> > </ivy-module>
> >
> > Thanks,
> > Matt
> > ________________________________________
> > From: Dmitriy Ryaboy [dvryaboy@gmail.com]
> > Sent: Sunday, June 10, 2012 5:05 PM
> > To: user@pig.apache.org
> > Subject: Re: NoClassDefFoundError after upgrading to pig 0.10.0 from
> 0.9.0
> >
> > I am not sure what you mean -- ivy pulls down 1.4.2 but pig builds from
> 1.7.3? Pig gets its jars from ivy, so it must be building from whatever ivy
> pulls down. Sounds like there is some packaging problem, but I am not sure
> how to interpret what you said.
> >
> > On Jun 8, 2012, at 12:07 PM, Matthew Hayes <mh...@linkedin.com> wrote:
> >
> >> I'm using ivy to download dependencies for pig, but after updating the
> version to 0.10.0 I am getting errors in my unit tests:
> >>
> >> [testng] java.lang.NoClassDefFoundError:
> org/codehaus/jackson/map/util/LRUMap
> >> [testng] at
> org.apache.pig.builtin.JsonMetadata.<init>(JsonMetadata.java:75)
> >> [testng] at
> org.apache.pig.builtin.PigStorage.getSchema(PigStorage.java:466)
> >> [testng] at
> org.apache.pig.newplan.logical.relational.LOLoad.getSchemaFromMetaData(LOLoad.java:151)
> >> [testng] at
> org.apache.pig.newplan.logical.relational.LOLoad.getSchema(LOLoad.java:110)
> >> ...etc.
> >>
> >> It looks like the dependency on jackson-mapper-asl is coming from avro.
> Looking in jackson-mapper-asl-1.7.3.jar I see the LRUMap class. However
> using ivy to pull down dependencies I get jackson-mapper-asl-1.4.2.jar, not
> jackson-mapper-asl-1.7.3.jar. When I build pig 0.10.0 from source it uses
> 1.7.3.
> >>
> >> Is there an inconsistency between the avro version pig is compiled
> against and the avro version specified in the pom?
> >>
> >> Thanks,
> >> Matt
>
Re: NoClassDefFoundError after upgrading to pig 0.10.0 from 0.9.0
Posted by Dmitriy Ryaboy <dv...@gmail.com>.
Yep, you are right, the pom is not generated, but checked in
statically; looks like it's out of date. One more reason to mavenize
Pig.. can you open a Jira for this? We'll fix.
D
On Mon, Jun 11, 2012 at 10:15 AM, Matthew Hayes <mh...@linkedin.com> wrote:
> What I mean is: are the versions in the pom for pig 0.10.0 inconsistent with the versions specified in the ivy file used to build pig? To be clear, I am building a separate project, and I am getting pig and its dependencies using ivy.
>
> Looking in ivy.xml in the pig 0.10.0 release:
>
> <dependency org="org.apache.avro" name="avro" rev="${avro.version}"
> conf="compile->default;checkstyle->master"/>
>
> ...
>
> <dependency org="org.codehaus.jackson" name="jackson-mapper-asl" rev="${jackson.version}"
> conf="compile->master"/>
> <dependency org="org.codehaus.jackson" name="jackson-core-asl" rev="${jackson.version}"
> conf="compile->master"/>
>
> Where avro.version is avro.version=1.5.3 and jackson.version=1.7.3.
>
> However, in the pom.xml for pig 0.10.0:
>
> <groupId>org.apache.hadoop</groupId>
> <artifactId>avro</artifactId>
> <version>1.3.2</version>
>
> And when I look up the pom for org.apache.hadoop's avro 1.3.2 in the central repository, I see a version of jackson inconsistent with what pig was compiled with:
>
> <dependency>
> <groupId>org.codehaus.jackson</groupId>
> <artifactId>jackson-mapper-asl</artifactId>
> <version>1.4.2</version>
> <scope>compile</scope>
> </dependency>
>
> It's 1.4.2, not 1.7.3.
>
> Am I doing something wrong here? Below is my ivy.xml. It's the same as what I used for 0.9.0 but I changed the pig version to 0.10.0.
>
> <ivy-module version="2.0">
> <info organisation="datafu" module="datafu"/>
> <dependencies>
> <dependency org="org.apache.pig" name="pig" rev="0.10.0"/>
> <dependency org="it.unimi.dsi" name="fastutil" rev="6.3"/>
> <dependency org="joda-time" name="joda-time" rev="1.6"/>
> <dependency org="org.apache.commons" name="commons-math" rev="2.1"/>
> <dependency org="commons-io" name="commons-io" rev="1.4"/>
> <dependency org="org.apache.hadoop" name="hadoop-core" rev="0.20.2"/>
> <dependency org="org.testng" name="testng" rev="6.2"/>
> <dependency org="com.google.guava" name="guava" rev="r06" />
>
> <!-- needed for pigunit to work -->
> <dependency org="log4j" name="log4j" rev="1.2.14" />
> <dependency org="jline" name="jline" rev="0.9.94" />
> <dependency org="org.antlr" name="antlr" rev="3.2" />
> </dependencies>
> </ivy-module>
>
> Thanks,
> Matt
> ________________________________________
> From: Dmitriy Ryaboy [dvryaboy@gmail.com]
> Sent: Sunday, June 10, 2012 5:05 PM
> To: user@pig.apache.org
> Subject: Re: NoClassDefFoundError after upgrading to pig 0.10.0 from 0.9.0
>
> I am not sure what you mean -- ivy pulls down 1.4.2 but pig builds from 1.7.3? Pig gets its jars from ivy, so it must be building from whatever ivy pulls down. Sounds like there is some packaging problem, but I am not sure how to interpret what you said.
>
> On Jun 8, 2012, at 12:07 PM, Matthew Hayes <mh...@linkedin.com> wrote:
>
>> I'm using ivy to download dependencies for pig, but after updating the version to 0.10.0 I am getting errors in my unit tests:
>>
>> [testng] java.lang.NoClassDefFoundError: org/codehaus/jackson/map/util/LRUMap
>> [testng] at org.apache.pig.builtin.JsonMetadata.<init>(JsonMetadata.java:75)
>> [testng] at org.apache.pig.builtin.PigStorage.getSchema(PigStorage.java:466)
>> [testng] at org.apache.pig.newplan.logical.relational.LOLoad.getSchemaFromMetaData(LOLoad.java:151)
>> [testng] at org.apache.pig.newplan.logical.relational.LOLoad.getSchema(LOLoad.java:110)
>> ...etc.
>>
>> It looks like the dependency on jackson-mapper-asl is coming from avro. Looking in jackson-mapper-asl-1.7.3.jar I see the LRUMap class. However using ivy to pull down dependencies I get jackson-mapper-asl-1.4.2.jar, not jackson-mapper-asl-1.7.3.jar. When I build pig 0.10.0 from source it uses 1.7.3.
>>
>> Is there an inconsistency between the avro version pig is compiled against and the avro version specified in the pom?
>>
>> Thanks,
>> Matt
RE: NoClassDefFoundError after upgrading to pig 0.10.0 from 0.9.0
Posted by Matthew Hayes <mh...@linkedin.com>.
Does anyone have any insight into the problem here? Am I doing something wrong?
________________________________________
From: Matthew Hayes [mhayes@linkedin.com]
Sent: Monday, June 11, 2012 10:15 AM
To: user@pig.apache.org
Subject: RE: NoClassDefFoundError after upgrading to pig 0.10.0 from 0.9.0
What I mean is: are the versions in the pom for pig 0.10.0 inconsistent with the versions specified in the ivy file used to build pig? To be clear, I am building a separate project, and I am getting pig and its dependencies using ivy.
Looking in ivy.xml in the pig 0.10.0 release:
<dependency org="org.apache.avro" name="avro" rev="${avro.version}"
conf="compile->default;checkstyle->master"/>
...
<dependency org="org.codehaus.jackson" name="jackson-mapper-asl" rev="${jackson.version}"
conf="compile->master"/>
<dependency org="org.codehaus.jackson" name="jackson-core-asl" rev="${jackson.version}"
conf="compile->master"/>
Where avro.version is avro.version=1.5.3 and jackson.version=1.7.3.
However, in the pom.xml for pig 0.10.0:
<groupId>org.apache.hadoop</groupId>
<artifactId>avro</artifactId>
<version>1.3.2</version>
And when I look up the pom for org.apache.hadoop's avro 1.3.2 in the central repository, I see a version of jackson inconsistent with what pig was compiled with:
<dependency>
<groupId>org.codehaus.jackson</groupId>
<artifactId>jackson-mapper-asl</artifactId>
<version>1.4.2</version>
<scope>compile</scope>
</dependency>
It's 1.4.2, not 1.7.3.
Am I doing something wrong here? Below is my ivy.xml. It's the same as what I used for 0.9.0 but I changed the pig version to 0.10.0.
<ivy-module version="2.0">
<info organisation="datafu" module="datafu"/>
<dependencies>
<dependency org="org.apache.pig" name="pig" rev="0.10.0"/>
<dependency org="it.unimi.dsi" name="fastutil" rev="6.3"/>
<dependency org="joda-time" name="joda-time" rev="1.6"/>
<dependency org="org.apache.commons" name="commons-math" rev="2.1"/>
<dependency org="commons-io" name="commons-io" rev="1.4"/>
<dependency org="org.apache.hadoop" name="hadoop-core" rev="0.20.2"/>
<dependency org="org.testng" name="testng" rev="6.2"/>
<dependency org="com.google.guava" name="guava" rev="r06" />
<!-- needed for pigunit to work -->
<dependency org="log4j" name="log4j" rev="1.2.14" />
<dependency org="jline" name="jline" rev="0.9.94" />
<dependency org="org.antlr" name="antlr" rev="3.2" />
</dependencies>
</ivy-module>
Thanks,
Matt
________________________________________
From: Dmitriy Ryaboy [dvryaboy@gmail.com]
Sent: Sunday, June 10, 2012 5:05 PM
To: user@pig.apache.org
Subject: Re: NoClassDefFoundError after upgrading to pig 0.10.0 from 0.9.0
I am not sure what you mean -- ivy pulls down 1.4.2 but pig builds from 1.7.3? Pig gets its jars from ivy, so it must be building from whatever ivy pulls down. Sounds like there is some packaging problem, but I am not sure how to interpret what you said.
On Jun 8, 2012, at 12:07 PM, Matthew Hayes <mh...@linkedin.com> wrote:
> I'm using ivy to download dependencies for pig, but after updating the version to 0.10.0 I am getting errors in my unit tests:
>
> [testng] java.lang.NoClassDefFoundError: org/codehaus/jackson/map/util/LRUMap
> [testng] at org.apache.pig.builtin.JsonMetadata.<init>(JsonMetadata.java:75)
> [testng] at org.apache.pig.builtin.PigStorage.getSchema(PigStorage.java:466)
> [testng] at org.apache.pig.newplan.logical.relational.LOLoad.getSchemaFromMetaData(LOLoad.java:151)
> [testng] at org.apache.pig.newplan.logical.relational.LOLoad.getSchema(LOLoad.java:110)
> ...etc.
>
> It looks like the dependency on jackson-mapper-asl is coming from avro. Looking in jackson-mapper-asl-1.7.3.jar I see the LRUMap class. However using ivy to pull down dependencies I get jackson-mapper-asl-1.4.2.jar, not jackson-mapper-asl-1.7.3.jar. When I build pig 0.10.0 from source it uses 1.7.3.
>
> Is there an inconsistency between the avro version pig is compiled against and the avro version specified in the pom?
>
> Thanks,
> Matt
RE: NoClassDefFoundError after upgrading to pig 0.10.0 from 0.9.0
Posted by Matthew Hayes <mh...@linkedin.com>.
What I mean is: are the versions in the pom for pig 0.10.0 inconsistent with the versions specified in the ivy file used to build pig? To be clear, I am building a separate project, and I am getting pig and its dependencies using ivy.
Looking in ivy.xml in the pig 0.10.0 release:
<dependency org="org.apache.avro" name="avro" rev="${avro.version}"
conf="compile->default;checkstyle->master"/>
...
<dependency org="org.codehaus.jackson" name="jackson-mapper-asl" rev="${jackson.version}"
conf="compile->master"/>
<dependency org="org.codehaus.jackson" name="jackson-core-asl" rev="${jackson.version}"
conf="compile->master"/>
Where avro.version is avro.version=1.5.3 and jackson.version=1.7.3.
However, in the pom.xml for pig 0.10.0:
<groupId>org.apache.hadoop</groupId>
<artifactId>avro</artifactId>
<version>1.3.2</version>
And when I look up the pom for org.apache.hadoop's avro 1.3.2 in the central repository, I see a version of jackson inconsistent with what pig was compiled with:
<dependency>
<groupId>org.codehaus.jackson</groupId>
<artifactId>jackson-mapper-asl</artifactId>
<version>1.4.2</version>
<scope>compile</scope>
</dependency>
It's 1.4.2, not 1.7.3.
Am I doing something wrong here? Below is my ivy.xml. It's the same as what I used for 0.9.0 but I changed the pig version to 0.10.0.
<ivy-module version="2.0">
<info organisation="datafu" module="datafu"/>
<dependencies>
<dependency org="org.apache.pig" name="pig" rev="0.10.0"/>
<dependency org="it.unimi.dsi" name="fastutil" rev="6.3"/>
<dependency org="joda-time" name="joda-time" rev="1.6"/>
<dependency org="org.apache.commons" name="commons-math" rev="2.1"/>
<dependency org="commons-io" name="commons-io" rev="1.4"/>
<dependency org="org.apache.hadoop" name="hadoop-core" rev="0.20.2"/>
<dependency org="org.testng" name="testng" rev="6.2"/>
<dependency org="com.google.guava" name="guava" rev="r06" />
<!-- needed for pigunit to work -->
<dependency org="log4j" name="log4j" rev="1.2.14" />
<dependency org="jline" name="jline" rev="0.9.94" />
<dependency org="org.antlr" name="antlr" rev="3.2" />
</dependencies>
</ivy-module>
Thanks,
Matt
________________________________________
From: Dmitriy Ryaboy [dvryaboy@gmail.com]
Sent: Sunday, June 10, 2012 5:05 PM
To: user@pig.apache.org
Subject: Re: NoClassDefFoundError after upgrading to pig 0.10.0 from 0.9.0
I am not sure what you mean -- ivy pulls down 1.4.2 but pig builds from 1.7.3? Pig gets its jars from ivy, so it must be building from whatever ivy pulls down. Sounds like there is some packaging problem, but I am not sure how to interpret what you said.
On Jun 8, 2012, at 12:07 PM, Matthew Hayes <mh...@linkedin.com> wrote:
> I'm using ivy to download dependencies for pig, but after updating the version to 0.10.0 I am getting errors in my unit tests:
>
> [testng] java.lang.NoClassDefFoundError: org/codehaus/jackson/map/util/LRUMap
> [testng] at org.apache.pig.builtin.JsonMetadata.<init>(JsonMetadata.java:75)
> [testng] at org.apache.pig.builtin.PigStorage.getSchema(PigStorage.java:466)
> [testng] at org.apache.pig.newplan.logical.relational.LOLoad.getSchemaFromMetaData(LOLoad.java:151)
> [testng] at org.apache.pig.newplan.logical.relational.LOLoad.getSchema(LOLoad.java:110)
> ...etc.
>
> It looks like the dependency on jackson-mapper-asl is coming from avro. Looking in jackson-mapper-asl-1.7.3.jar I see the LRUMap class. However using ivy to pull down dependencies I get jackson-mapper-asl-1.4.2.jar, not jackson-mapper-asl-1.7.3.jar. When I build pig 0.10.0 from source it uses 1.7.3.
>
> Is there an inconsistency between the avro version pig is compiled against and the avro version specified in the pom?
>
> Thanks,
> Matt
Re: NoClassDefFoundError after upgrading to pig 0.10.0 from 0.9.0
Posted by Dmitriy Ryaboy <dv...@gmail.com>.
I am not sure what you mean -- ivy pulls down 1.4.2 but pig builds from 1.7.3? Pig gets its jars from ivy, so it must be building from whatever ivy pulls down. Sounds like there is some packaging problem, but I am not sure how to interpret what you said.
On Jun 8, 2012, at 12:07 PM, Matthew Hayes <mh...@linkedin.com> wrote:
> I'm using ivy to download dependencies for pig, but after updating the version to 0.10.0 I am getting errors in my unit tests:
>
> [testng] java.lang.NoClassDefFoundError: org/codehaus/jackson/map/util/LRUMap
> [testng] at org.apache.pig.builtin.JsonMetadata.<init>(JsonMetadata.java:75)
> [testng] at org.apache.pig.builtin.PigStorage.getSchema(PigStorage.java:466)
> [testng] at org.apache.pig.newplan.logical.relational.LOLoad.getSchemaFromMetaData(LOLoad.java:151)
> [testng] at org.apache.pig.newplan.logical.relational.LOLoad.getSchema(LOLoad.java:110)
> ...etc.
>
> It looks like the dependency on jackson-mapper-asl is coming from avro. Looking in jackson-mapper-asl-1.7.3.jar I see the LRUMap class. However using ivy to pull down dependencies I get jackson-mapper-asl-1.4.2.jar, not jackson-mapper-asl-1.7.3.jar. When I build pig 0.10.0 from source it uses 1.7.3.
>
> Is there an inconsistency between the avro version pig is compiled against and the avro version specified in the pom?
>
> Thanks,
> Matt