You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@kylin.apache.org by liam <li...@gmail.com> on 2015/07/31 10:40:05 UTC
Re: Found interface org.apache.hadoop.mapreduce.JobContext, but class
was expected
Hi,
Can anyone help this?
Thx!
2015-07-30 18:11 GMT+08:00 liam <li...@gmail.com>:
> Hi,all
> I failed in building the sample cube “kylin_sales_cube”,I found the error
> message from the logs(showed bellow).
> Did anyone get the same issue before? Any help would be highly
> appreciated.
> I use : hadoop(2.6.0) , hive(1.0.1) , kylin(0.7.2) , hbase(1.0.1.1)
>
> -
> -
>
> ——Here :ERROR Message-----
>
> - [pool-7-thread-2]:[2015-07-30
> 17:48:20,435][ERROR][org.apache.kylin.job.impl.threadpool.DefaultScheduler$JobRunner.run(DefaultScheduler.java:137)]
> - ExecuteException job:2da67e50-02f7-4a11-aa67-2a6a5dff5f36
> - org.apache.kylin.job.exception.ExecuteException:
> org.apache.kylin.job.exception.ExecuteException:
> java.lang.IncompatibleClassChangeError: Found interface
> org.apache.hadoop.mapreduce.JobContext, but class was expected
> - at
> org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:110)
> - at
> org.apache.kylin.job.impl.threadpool.DefaultScheduler$JobRunner.run(DefaultScheduler.java:133)
> - at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> - at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> - at java.lang.Thread.run(Thread.java:745)
> - Caused by: org.apache.kylin.job.exception.ExecuteException:
> java.lang.IncompatibleClassChangeError: Found interface
> org.apache.hadoop.mapreduce.JobContext, but class was expected
> - at
> org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:110)
> - at
> org.apache.kylin.job.execution.DefaultChainedExecutable.doWork(DefaultChainedExecutable.java:50)
> - at
> org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:106)
> - ... 4 more
> - Caused by: java.lang.IncompatibleClassChangeError: Found interface
> org.apache.hadoop.mapreduce.JobContext, but class was expected
> - at
> org.apache.hive.hcatalog.mapreduce.HCatBaseInputFormat.getSplits(HCatBaseInputFormat.java:102)
> - at
> org.apache.hadoop.mapreduce.JobSubmitter.writeNewSplits(JobSubmitter.java:493)
> - at
> org.apache.hadoop.mapreduce.JobSubmitter.writeSplits(JobSubmitter.java:510)
> - at
> org.apache.hadoop.mapreduce.JobSubmitter.submitJobInternal(JobSubmitter.java:394)
> - at org.apache.hadoop.mapreduce.Job$10.run(Job.java:1285)
> - at org.apache.hadoop.mapreduce.Job$10.run(Job.java:1282)
> - at java.security.AccessController.doPrivileged(Native Method)
> - at javax.security.auth.Subject.doAs(Subject.java:415)
> - at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1614)
> - at org.apache.hadoop.mapreduce.Job.submit(Job.java:1282)
> - at
> org.apache.kylin.job.hadoop.AbstractHadoopJob.waitForCompletion(AbstractHadoopJob.java:123)
> - at
> org.apache.kylin.job.hadoop.cube.FactDistinctColumnsJob.run(FactDistinctColumnsJob.java:80)
> - at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
> - at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:84)
> - at
> org.apache.kylin.job.common.MapReduceExecutable.doWork(MapReduceExecutable.java:112)
> - at
> org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:106)
> - ... 6 more
>
>
>
Re: Found interface org.apache.hadoop.mapreduce.JobContext, but class
was expected
Posted by hongbin ma <ma...@apache.org>.
Thank your for your information, Yerui!
On Fri, Jul 31, 2015 at 6:25 PM, Yerui Sun <su...@gmail.com> wrote:
> Hi，liam
> I just met your same problem a few days ago and fixed it. The difference
> is hive-0.13.1 in our enviroment, but I think the reason is same.
> The reason is that org.apache.hadoop.mapreduce.JobContext is class in
> hadoop-1 but changed to interface in hadoop-2, and your
> hive-hcatalog-core.jar must be complied on hadoop-1.
> As I know, hive-hcatalog-core-0.13.1.jar in maven.apache.org is
> complied on hadoop-1, but not clear about hive 1.0.1. I think you should
> have a try, to re-complie your hive-hcatalog-core jar on hadoop-2.
>
> Good luck and looking forward your good news -:)
>
>
> Best Regards,
> Yerui Sun
> sunyerui@gmail.com
>
>
>
> > 在 2015年7月31日，16:40，liam <li...@gmail.com> 写道：
> >
> > Hi,
> > Can anyone help this?
> > Thx!
> >
> > 2015-07-30 18:11 GMT+08:00 liam <li...@gmail.com>:
> >
> >> Hi,all
> >> I failed in building the sample cube “kylin_sales_cube”,I found the
> error
> >> message from the logs(showed bellow).
> >> Did anyone get the same issue before? Any help would be highly
> >> appreciated.
> >> I use : hadoop(2.6.0) , hive(1.0.1) , kylin(0.7.2) , hbase(1.0.1.1)
> >>
> >> -
> >> -
> >>
> >> ——Here :ERROR Message-----
> >>
> >> - [pool-7-thread-2]:[2015-07-30
> >>
> 17:48:20,435][ERROR][org.apache.kylin.job.impl.threadpool.DefaultScheduler$JobRunner.run(DefaultScheduler.java:137)]
> >> - ExecuteException job:2da67e50-02f7-4a11-aa67-2a6a5dff5f36
> >> - org.apache.kylin.job.exception.ExecuteException:
> >> org.apache.kylin.job.exception.ExecuteException:
> >> java.lang.IncompatibleClassChangeError: Found interface
> >> org.apache.hadoop.mapreduce.JobContext, but class was expected
> >> - at
> >>
> org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:110)
> >> - at
> >>
> org.apache.kylin.job.impl.threadpool.DefaultScheduler$JobRunner.run(DefaultScheduler.java:133)
> >> - at
> >>
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> >> - at
> >>
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> >> - at java.lang.Thread.run(Thread.java:745)
> >> - Caused by: org.apache.kylin.job.exception.ExecuteException:
> >> java.lang.IncompatibleClassChangeError: Found interface
> >> org.apache.hadoop.mapreduce.JobContext, but class was expected
> >> - at
> >>
> org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:110)
> >> - at
> >>
> org.apache.kylin.job.execution.DefaultChainedExecutable.doWork(DefaultChainedExecutable.java:50)
> >> - at
> >>
> org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:106)
> >> - ... 4 more
> >> - Caused by: java.lang.IncompatibleClassChangeError: Found interface
> >> org.apache.hadoop.mapreduce.JobContext, but class was expected
> >> - at
> >>
> org.apache.hive.hcatalog.mapreduce.HCatBaseInputFormat.getSplits(HCatBaseInputFormat.java:102)
> >> - at
> >>
> org.apache.hadoop.mapreduce.JobSubmitter.writeNewSplits(JobSubmitter.java:493)
> >> - at
> >>
> org.apache.hadoop.mapreduce.JobSubmitter.writeSplits(JobSubmitter.java:510)
> >> - at
> >>
> org.apache.hadoop.mapreduce.JobSubmitter.submitJobInternal(JobSubmitter.java:394)
> >> - at org.apache.hadoop.mapreduce.Job$10.run(Job.java:1285)
> >> - at org.apache.hadoop.mapreduce.Job$10.run(Job.java:1282)
> >> - at java.security.AccessController.doPrivileged(Native
> Method)
> >> - at javax.security.auth.Subject.doAs(Subject.java:415)
> >> - at
> >>
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1614)
> >> - at org.apache.hadoop.mapreduce.Job.submit(Job.java:1282)
> >> - at
> >>
> org.apache.kylin.job.hadoop.AbstractHadoopJob.waitForCompletion(AbstractHadoopJob.java:123)
> >> - at
> >>
> org.apache.kylin.job.hadoop.cube.FactDistinctColumnsJob.run(FactDistinctColumnsJob.java:80)
> >> - at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
> >> - at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:84)
> >> - at
> >>
> org.apache.kylin.job.common.MapReduceExecutable.doWork(MapReduceExecutable.java:112)
> >> - at
> >>
> org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:106)
> >> - ... 6 more
> >>
> >>
> >>
>
>
--
Regards,
*Bin Mahone | 马洪宾*
Apache Kylin: http://kylin.io
Github: https://github.com/binmahone
Re: Found interface org.apache.hadoop.mapreduce.JobContext, but class
was expected
Posted by liam <li...@gmail.com>.
Hi,Yerui Sun
Your message is very helpful,I'll try it.
Thank you !
2015-07-31 18:25 GMT+08:00 Yerui Sun <su...@gmail.com>:
> Hi，liam
> I just met your same problem a few days ago and fixed it. The difference
> is hive-0.13.1 in our enviroment, but I think the reason is same.
> The reason is that org.apache.hadoop.mapreduce.JobContext is class in
> hadoop-1 but changed to interface in hadoop-2, and your
> hive-hcatalog-core.jar must be complied on hadoop-1.
> As I know, hive-hcatalog-core-0.13.1.jar in maven.apache.org is
> complied on hadoop-1, but not clear about hive 1.0.1. I think you should
> have a try, to re-complie your hive-hcatalog-core jar on hadoop-2.
>
> Good luck and looking forward your good news -:)
>
>
> Best Regards,
> Yerui Sun
> sunyerui@gmail.com
>
>
>
> > 在 2015年7月31日，16:40，liam <li...@gmail.com> 写道：
> >
> > Hi,
> > Can anyone help this?
> > Thx!
> >
> > 2015-07-30 18:11 GMT+08:00 liam <li...@gmail.com>:
> >
> >> Hi,all
> >> I failed in building the sample cube “kylin_sales_cube”,I found the
> error
> >> message from the logs(showed bellow).
> >> Did anyone get the same issue before? Any help would be highly
> >> appreciated.
> >> I use : hadoop(2.6.0) , hive(1.0.1) , kylin(0.7.2) , hbase(1.0.1.1)
> >>
> >> -
> >> -
> >>
> >> ——Here :ERROR Message-----
> >>
> >> - [pool-7-thread-2]:[2015-07-30
> >>
> 17:48:20,435][ERROR][org.apache.kylin.job.impl.threadpool.DefaultScheduler$JobRunner.run(DefaultScheduler.java:137)]
> >> - ExecuteException job:2da67e50-02f7-4a11-aa67-2a6a5dff5f36
> >> - org.apache.kylin.job.exception.ExecuteException:
> >> org.apache.kylin.job.exception.ExecuteException:
> >> java.lang.IncompatibleClassChangeError: Found interface
> >> org.apache.hadoop.mapreduce.JobContext, but class was expected
> >> - at
> >>
> org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:110)
> >> - at
> >>
> org.apache.kylin.job.impl.threadpool.DefaultScheduler$JobRunner.run(DefaultScheduler.java:133)
> >> - at
> >>
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> >> - at
> >>
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> >> - at java.lang.Thread.run(Thread.java:745)
> >> - Caused by: org.apache.kylin.job.exception.ExecuteException:
> >> java.lang.IncompatibleClassChangeError: Found interface
> >> org.apache.hadoop.mapreduce.JobContext, but class was expected
> >> - at
> >>
> org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:110)
> >> - at
> >>
> org.apache.kylin.job.execution.DefaultChainedExecutable.doWork(DefaultChainedExecutable.java:50)
> >> - at
> >>
> org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:106)
> >> - ... 4 more
> >> - Caused by: java.lang.IncompatibleClassChangeError: Found interface
> >> org.apache.hadoop.mapreduce.JobContext, but class was expected
> >> - at
> >>
> org.apache.hive.hcatalog.mapreduce.HCatBaseInputFormat.getSplits(HCatBaseInputFormat.java:102)
> >> - at
> >>
> org.apache.hadoop.mapreduce.JobSubmitter.writeNewSplits(JobSubmitter.java:493)
> >> - at
> >>
> org.apache.hadoop.mapreduce.JobSubmitter.writeSplits(JobSubmitter.java:510)
> >> - at
> >>
> org.apache.hadoop.mapreduce.JobSubmitter.submitJobInternal(JobSubmitter.java:394)
> >> - at org.apache.hadoop.mapreduce.Job$10.run(Job.java:1285)
> >> - at org.apache.hadoop.mapreduce.Job$10.run(Job.java:1282)
> >> - at java.security.AccessController.doPrivileged(Native
> Method)
> >> - at javax.security.auth.Subject.doAs(Subject.java:415)
> >> - at
> >>
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1614)
> >> - at org.apache.hadoop.mapreduce.Job.submit(Job.java:1282)
> >> - at
> >>
> org.apache.kylin.job.hadoop.AbstractHadoopJob.waitForCompletion(AbstractHadoopJob.java:123)
> >> - at
> >>
> org.apache.kylin.job.hadoop.cube.FactDistinctColumnsJob.run(FactDistinctColumnsJob.java:80)
> >> - at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
> >> - at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:84)
> >> - at
> >>
> org.apache.kylin.job.common.MapReduceExecutable.doWork(MapReduceExecutable.java:112)
> >> - at
> >>
> org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:106)
> >> - ... 6 more
> >>
> >>
> >>
>
>
Re: Found interface org.apache.hadoop.mapreduce.JobContext, but class was expected
Posted by Yerui Sun <su...@gmail.com>.
Hi，liam
I just met your same problem a few days ago and fixed it. The difference is hive-0.13.1 in our enviroment, but I think the reason is same.
The reason is that org.apache.hadoop.mapreduce.JobContext is class in hadoop-1 but changed to interface in hadoop-2, and your hive-hcatalog-core.jar must be complied on hadoop-1.
As I know, hive-hcatalog-core-0.13.1.jar in maven.apache.org is complied on hadoop-1, but not clear about hive 1.0.1. I think you should have a try, to re-complie your hive-hcatalog-core jar on hadoop-2.
Good luck and looking forward your good news -:)
Best Regards,
Yerui Sun
sunyerui@gmail.com
> 在 2015年7月31日，16:40，liam <li...@gmail.com> 写道：
>
> Hi,
> Can anyone help this?
> Thx!
>
> 2015-07-30 18:11 GMT+08:00 liam <li...@gmail.com>:
>
>> Hi,all
>> I failed in building the sample cube “kylin_sales_cube”,I found the error
>> message from the logs(showed bellow).
>> Did anyone get the same issue before? Any help would be highly
>> appreciated.
>> I use : hadoop(2.6.0) , hive(1.0.1) , kylin(0.7.2) , hbase(1.0.1.1)
>>
>> -
>> -
>>
>> ——Here :ERROR Message-----
>>
>> - [pool-7-thread-2]:[2015-07-30
>> 17:48:20,435][ERROR][org.apache.kylin.job.impl.threadpool.DefaultScheduler$JobRunner.run(DefaultScheduler.java:137)]
>> - ExecuteException job:2da67e50-02f7-4a11-aa67-2a6a5dff5f36
>> - org.apache.kylin.job.exception.ExecuteException:
>> org.apache.kylin.job.exception.ExecuteException:
>> java.lang.IncompatibleClassChangeError: Found interface
>> org.apache.hadoop.mapreduce.JobContext, but class was expected
>> - at
>> org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:110)
>> - at
>> org.apache.kylin.job.impl.threadpool.DefaultScheduler$JobRunner.run(DefaultScheduler.java:133)
>> - at
>> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
>> - at
>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
>> - at java.lang.Thread.run(Thread.java:745)
>> - Caused by: org.apache.kylin.job.exception.ExecuteException:
>> java.lang.IncompatibleClassChangeError: Found interface
>> org.apache.hadoop.mapreduce.JobContext, but class was expected
>> - at
>> org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:110)
>> - at
>> org.apache.kylin.job.execution.DefaultChainedExecutable.doWork(DefaultChainedExecutable.java:50)
>> - at
>> org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:106)
>> - ... 4 more
>> - Caused by: java.lang.IncompatibleClassChangeError: Found interface
>> org.apache.hadoop.mapreduce.JobContext, but class was expected
>> - at
>> org.apache.hive.hcatalog.mapreduce.HCatBaseInputFormat.getSplits(HCatBaseInputFormat.java:102)
>> - at
>> org.apache.hadoop.mapreduce.JobSubmitter.writeNewSplits(JobSubmitter.java:493)
>> - at
>> org.apache.hadoop.mapreduce.JobSubmitter.writeSplits(JobSubmitter.java:510)
>> - at
>> org.apache.hadoop.mapreduce.JobSubmitter.submitJobInternal(JobSubmitter.java:394)
>> - at org.apache.hadoop.mapreduce.Job$10.run(Job.java:1285)
>> - at org.apache.hadoop.mapreduce.Job$10.run(Job.java:1282)
>> - at java.security.AccessController.doPrivileged(Native Method)
>> - at javax.security.auth.Subject.doAs(Subject.java:415)
>> - at
>> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1614)
>> - at org.apache.hadoop.mapreduce.Job.submit(Job.java:1282)
>> - at
>> org.apache.kylin.job.hadoop.AbstractHadoopJob.waitForCompletion(AbstractHadoopJob.java:123)
>> - at
>> org.apache.kylin.job.hadoop.cube.FactDistinctColumnsJob.run(FactDistinctColumnsJob.java:80)
>> - at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
>> - at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:84)
>> - at
>> org.apache.kylin.job.common.MapReduceExecutable.doWork(MapReduceExecutable.java:112)
>> - at
>> org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:106)
>> - ... 6 more
>>
>>
>>
Re: Found interface org.apache.hadoop.mapreduce.JobContext, but class
was expected
Posted by hongbin ma <ma...@apache.org>.
Just opened a ticket on this:
https://issues.apache.org/jira/browse/KYLIN-920
please do NOT expect a very quick fix on it.
--
Regards,
*Bin Mahone | 马洪宾*
Apache Kylin: http://kylin.io
Github: https://github.com/binmahone
Re: Found interface org.apache.hadoop.mapreduce.JobContext, but class
was expected
Posted by hongbin ma <ma...@apache.org>.
hi,
we may have not tested against hive 1.0
most of our dev/production environments are aligned with hdp 2.2, please
refer to
http://docs.hortonworks.com/HDPDocuments/HDP2/HDP-2.2.4/HDP_RelNotes_v224/index.html
for appropriate hive/hadoop/hbase versions.
Still we'll work on supporting newer versions like it in your environment.
--
Regards,
*Bin Mahone | 马洪宾*
Apache Kylin: http://kylin.io
Github: https://github.com/binmahone