You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@spark.apache.org by ey-chih chow <ey...@hotmail.com> on 2015/03/05 19:31:58 UTC
IncompatibleClassChangeError
Hi,
I am using CDH5.3.2 now for a Spark project. I got the following exception:
java.lang.IncompatibleClassChangeError: Found interface
org.apache.hadoop.mapreduce.TaskAttemptContext, but class was expected
I used all the CDH5.3.2 jar files in my pom file to generate the application
jar file. What else I should do to fix the problem?
Ey-Chih Chow
--
View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/IncompatibleClassChangeError-tp21934.html
Sent from the Apache Spark User List mailing list archive at Nabble.com.
---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscribe@spark.apache.org
For additional commands, e-mail: user-help@spark.apache.org
Re: IncompatibleClassChangeError
Posted by "M. Dale" <me...@yahoo.com.INVALID>.
In Hadoop 1.x TaskAttemptContext is a class (for example,
https://hadoop.apache.org/docs/r1.2.1/api/org/apache/hadoop/mapred/TaskAttemptContext.html)
In Hadoop 2.x TaskAttemptContext is an interface
(https://hadoop.apache.org/docs/r2.4.0/api/org/apache/hadoop/mapreduce/TaskAttemptContext.html)
>From
http://www.cloudera.com/content/cloudera/en/documentation/core/latest/topics/cdh_vd_cdh_package_tarball.html
it looks like
CDH 5.3.2 uses Hadoop 2.5.
Are you using any third party libraries that come in hadoop1 (default) vs.
hadoop2 versions like avro-mapred (see
https://issues.apache.org/jira/browse/SPARK-3039)?
If so make sure you include:
<dependency>
...
<classifier>hadoop2</classifier>
</dependency>
What version of Spark are you using? Are you using Avro? If so
SPARK-3039 is fixed in Spark 1.3.
Markus
On 03/05/2015 01:31 PM, ey-chih chow wrote:
> Hi,
>
> I am using CDH5.3.2 now for a Spark project. I got the following exception:
>
> java.lang.IncompatibleClassChangeError: Found interface
> org.apache.hadoop.mapreduce.TaskAttemptContext, but class was expected
>
> I used all the CDH5.3.2 jar files in my pom file to generate the application
> jar file. What else I should do to fix the problem?
>
> Ey-Chih Chow
>
>
>
> --
> View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/IncompatibleClassChangeError-tp21934.html
> Sent from the Apache Spark User List mailing list archive at Nabble.com.
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: user-unsubscribe@spark.apache.org
> For additional commands, e-mail: user-help@spark.apache.org
>
---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscribe@spark.apache.org
For additional commands, e-mail: user-help@spark.apache.org