You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@kylin.apache.org by "mzz (Jira)" <ji...@apache.org> on 2021/10/21 11:50:00 UTC
[jira] [Commented] (KYLIN-5107) when excuting the cube job at "Convert Cuboid Data to HFile" step：Error running child
[ https://issues.apache.org/jira/browse/KYLIN-5107?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17432411#comment-17432411 ]
mzz commented on KYLIN-5107:
----------------------------
[~zhangyaqian]help ,thx.
> when excuting the cube job at "Convert Cuboid Data to HFile" step：Error running child
> --------------------------------------------------------------------------------------
>
> Key: KYLIN-5107
> URL: https://issues.apache.org/jira/browse/KYLIN-5107
> Project: Kylin
> Issue Type: Bug
> Components: Job Engine
> Affects Versions: v3.1.2
> Environment: hadoop3.2.1
> habse2.3.4
> Reporter: mzz
> Priority: Blocker
>
> As title, when excuting the cube job at "Convert Cuboid Data to HFile" step,throw an exception:
> {code:java}
> 2021-10-21 10:15:38,928 ERROR [main] org.apache.kylin.engine.mr.KylinReducer:
> java.lang.NoSuchMethodError: org.apache.hadoop.hbase.util.FSUtils.setStoragePolicy(Lorg/apache/hadoop/fs/FileSystem;Lorg/apache/hadoop/fs/Path;Ljava/lang/String;)V
> at org.apache.kylin.storage.hbase.steps.HFileOutputFormat3.configureStoragePolicy(HFileOutputFormat3.java:468)
> at org.apache.kylin.storage.hbase.steps.HFileOutputFormat3$1.write(HFileOutputFormat3.java:287)
> at org.apache.kylin.storage.hbase.steps.HFileOutputFormat3$1.write(HFileOutputFormat3.java:243)
> at org.apache.hadoop.mapred.ReduceTask$NewTrackingRecordWriter.write(ReduceTask.java:559)
> at org.apache.hadoop.mapreduce.task.TaskInputOutputContextImpl.write(TaskInputOutputContextImpl.java:89)
> at org.apache.hadoop.mapreduce.lib.reduce.WrappedReducer$Context.write(WrappedReducer.java:105)
> at org.apache.kylin.storage.hbase.steps.CubeHFileJob$KeyValueReducer.doReduce(CubeHFileJob.java:141)
> at org.apache.kylin.storage.hbase.steps.CubeHFileJob$KeyValueReducer.doReduce(CubeHFileJob.java:135)
> at org.apache.kylin.engine.mr.KylinReducer.reduce(KylinReducer.java:77)
> at org.apache.hadoop.mapreduce.Reducer.run(Reducer.java:171)
> at org.apache.hadoop.mapred.ReduceTask.runNewReducer(ReduceTask.java:628)
> at org.apache.hadoop.mapred.ReduceTask.run(ReduceTask.java:390)
> at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:174)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:422)
> at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1732)
> at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:168)
> 2021-10-21 10:15:38,929 INFO [main] org.apache.kylin.engine.mr.KylinReducer: Do cleanup, available memory: 2773m
> 2021-10-21 10:15:38,929 INFO [main] org.apache.kylin.engine.mr.KylinReducer: Total rows: 1
> 2021-10-21 10:15:38,929 ERROR [main] org.apache.hadoop.mapred.YarnChild: Error running child : java.lang.NoSuchMethodError: org.apache.hadoop.hbase.util.FSUtils.setStoragePolicy(Lorg/apache/hadoop/fs/FileSystem;Lorg/apache/hadoop/fs/Path;Ljava/lang/String;)V
> at org.apache.kylin.storage.hbase.steps.HFileOutputFormat3.configureStoragePolicy(HFileOutputFormat3.java:468)
> at org.apache.kylin.storage.hbase.steps.HFileOutputFormat3$1.write(HFileOutputFormat3.java:287)
> at org.apache.kylin.storage.hbase.steps.HFileOutputFormat3$1.write(HFileOutputFormat3.java:243)
> at org.apache.hadoop.mapred.ReduceTask$NewTrackingRecordWriter.write(ReduceTask.java:559)
> at org.apache.hadoop.mapreduce.task.TaskInputOutputContextImpl.write(TaskInputOutputContextImpl.java:89)
> at org.apache.hadoop.mapreduce.lib.reduce.WrappedReducer$Context.write(WrappedReducer.java:105)
> at org.apache.kylin.storage.hbase.steps.CubeHFileJob$KeyValueReducer.doReduce(CubeHFileJob.java:141)
> at org.apache.kylin.storage.hbase.steps.CubeHFileJob$KeyValueReducer.doReduce(CubeHFileJob.java:135)
> at org.apache.kylin.engine.mr.KylinReducer.reduce(KylinReducer.java:77)
> at org.apache.hadoop.mapreduce.Reducer.run(Reducer.java:171)
> at org.apache.hadoop.mapred.ReduceTask.runNewReducer(ReduceTask.java:628)
> at org.apache.hadoop.mapred.ReduceTask.run(ReduceTask.java:390)
> at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:174)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:422)
> at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1732)
> at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:168)
> {code}
> I try move all of hbase libs to {KYLIN_HOME}/lib ,but this problem not solved.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)