You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@kylin.apache.org by liao ke <li...@gmail.com> on 2018/09/14 02:23:23 UTC

kylin 2.3.1 upgrade to 2.4.1 error happen

dear friends:
 After kylin 2.3.1 upgrade to 2.4.1 ,some problem had happened.
(we took the the version of kylin binary :
apache-kylin-2.4.1-bin-hbase1x.tar.gz
<http://www.apache.org/dyn/closer.cgi/kylin/apache-kylin-2.4.1/apache-kylin-2.4.1-bin-hbase1x.tar.gz>
)
During the cube buiding flow,at the 3 step,error happend.

#3 Step Name: Extract Fact Table Distinct Columns

error like below:



Error: java.io.IOException: wrong value class:
org.apache.hadoop.io.ArrayPrimitiveWritable is not class
org.apache.hadoop.io.BytesWritable at
org.apache.hadoop.io.SequenceFile$Writer.append(SequenceFile.java:1311) at
org.apache.hadoop.mapreduce.lib.output.SequenceFileOutputFormat$1.write(SequenceFileOutputFormat.java:83)
at
org.apache.hadoop.mapreduce.lib.output.MultipleOutputs.write(MultipleOutputs.java:432)
at
org.apache.kylin.engine.mr.steps.FactDistinctColumnsReducer.outputDict(FactDistinctColumnsReducer.java:243)
at
org.apache.kylin.engine.mr.steps.FactDistinctColumnsReducer.doCleanup(FactDistinctColumnsReducer.java:217)
at org.apache.kylin.engine.mr.KylinReducer.cleanup(KylinReducer.java:103)
at org.apache.hadoop.mapreduce.Reducer.run(Reducer.java:179) at
org.apache.hadoop.mapred.ReduceTask.runNewReducer(ReduceTask.java:627) at
org.apache.hadoop.mapred.ReduceTask.run(ReduceTask.java:389) at
org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:168) at
java.security.AccessController.doPrivileged(Native Method) at
javax.security.auth.Subject.doAs(Subject.java:422) at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1709)
at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:162)

Re: kylin 2.3.1 upgrade to 2.4.1 error happen

Posted by ShaoFeng Shi <sh...@apache.org>.
Hi Liao Ke,

What's going on with this issue?

I checked the code, it is related with the change in KYLIN-3324, but all
places using the value class has been updated. Unless this job is created
by Kylin 2.3 but executed by Kylin 2.4; If so, try to discard and then
resubmit the job with Kylin 2.4.



liao ke <li...@gmail.com> 于2018年9月14日周五 上午10:23写道：

> dear friends:
>  After kylin 2.3.1 upgrade to 2.4.1 ,some problem had happened.
> (we took the the version of kylin binary :
> apache-kylin-2.4.1-bin-hbase1x.tar.gz
> <http://www.apache.org/dyn/closer.cgi/kylin/apache-kylin-2.4.1/apache-kylin-2.4.1-bin-hbase1x.tar.gz>
> )
> During the cube buiding flow,at the 3 step,error happend.
>
> #3 Step Name: Extract Fact Table Distinct Columns
>
> error like below:
>
>
>
> Error: java.io.IOException: wrong value class:
> org.apache.hadoop.io.ArrayPrimitiveWritable is not class
> org.apache.hadoop.io.BytesWritable at
> org.apache.hadoop.io.SequenceFile$Writer.append(SequenceFile.java:1311) at
> org.apache.hadoop.mapreduce.lib.output.SequenceFileOutputFormat$1.write(SequenceFileOutputFormat.java:83)
> at
> org.apache.hadoop.mapreduce.lib.output.MultipleOutputs.write(MultipleOutputs.java:432)
> at
> org.apache.kylin.engine.mr.steps.FactDistinctColumnsReducer.outputDict(FactDistinctColumnsReducer.java:243)
> at
> org.apache.kylin.engine.mr.steps.FactDistinctColumnsReducer.doCleanup(FactDistinctColumnsReducer.java:217)
> at org.apache.kylin.engine.mr.KylinReducer.cleanup(KylinReducer.java:103)
> at org.apache.hadoop.mapreduce.Reducer.run(Reducer.java:179) at
> org.apache.hadoop.mapred.ReduceTask.runNewReducer(ReduceTask.java:627) at
> org.apache.hadoop.mapred.ReduceTask.run(ReduceTask.java:389) at
> org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:168) at
> java.security.AccessController.doPrivileged(Native Method) at
> javax.security.auth.Subject.doAs(Subject.java:422) at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1709)
> at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:162)
>


-- 
Best regards,

Shaofeng Shi 史少锋