You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@kylin.apache.org by "mzz (Jira)" <ji...@apache.org> on 2021/10/21 05:29:00 UTC
[jira] [Created] (KYLIN-5107) when excuting the cube job at "Convert Cuboid Data to HFile" step：Error running child
mzz created KYLIN-5107:
--------------------------
Summary: when excuting the cube job at "Convert Cuboid Data to HFile" step：Error running child
Key: KYLIN-5107
URL: https://issues.apache.org/jira/browse/KYLIN-5107
Project: Kylin
Issue Type: Bug
Components: Job Engine
Affects Versions: v3.1.2
Environment: hadoop3.2.1
habse2.3.4
Reporter: mzz
As title, when excuting the cube job at "Convert Cuboid Data to HFile" step,throw an exception:
{code:java}
2021-10-21 10:15:38,928 ERROR [main] org.apache.kylin.engine.mr.KylinReducer:
java.lang.NoSuchMethodError: org.apache.hadoop.hbase.util.FSUtils.setStoragePolicy(Lorg/apache/hadoop/fs/FileSystem;Lorg/apache/hadoop/fs/Path;Ljava/lang/String;)V
at org.apache.kylin.storage.hbase.steps.HFileOutputFormat3.configureStoragePolicy(HFileOutputFormat3.java:468)
at org.apache.kylin.storage.hbase.steps.HFileOutputFormat3$1.write(HFileOutputFormat3.java:287)
at org.apache.kylin.storage.hbase.steps.HFileOutputFormat3$1.write(HFileOutputFormat3.java:243)
at org.apache.hadoop.mapred.ReduceTask$NewTrackingRecordWriter.write(ReduceTask.java:559)
at org.apache.hadoop.mapreduce.task.TaskInputOutputContextImpl.write(TaskInputOutputContextImpl.java:89)
at org.apache.hadoop.mapreduce.lib.reduce.WrappedReducer$Context.write(WrappedReducer.java:105)
at org.apache.kylin.storage.hbase.steps.CubeHFileJob$KeyValueReducer.doReduce(CubeHFileJob.java:141)
at org.apache.kylin.storage.hbase.steps.CubeHFileJob$KeyValueReducer.doReduce(CubeHFileJob.java:135)
at org.apache.kylin.engine.mr.KylinReducer.reduce(KylinReducer.java:77)
at org.apache.hadoop.mapreduce.Reducer.run(Reducer.java:171)
at org.apache.hadoop.mapred.ReduceTask.runNewReducer(ReduceTask.java:628)
at org.apache.hadoop.mapred.ReduceTask.run(ReduceTask.java:390)
at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:174)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:422)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1732)
at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:168)
2021-10-21 10:15:38,929 INFO [main] org.apache.kylin.engine.mr.KylinReducer: Do cleanup, available memory: 2773m
2021-10-21 10:15:38,929 INFO [main] org.apache.kylin.engine.mr.KylinReducer: Total rows: 1
2021-10-21 10:15:38,929 ERROR [main] org.apache.hadoop.mapred.YarnChild: Error running child : java.lang.NoSuchMethodError: org.apache.hadoop.hbase.util.FSUtils.setStoragePolicy(Lorg/apache/hadoop/fs/FileSystem;Lorg/apache/hadoop/fs/Path;Ljava/lang/String;)V
at org.apache.kylin.storage.hbase.steps.HFileOutputFormat3.configureStoragePolicy(HFileOutputFormat3.java:468)
at org.apache.kylin.storage.hbase.steps.HFileOutputFormat3$1.write(HFileOutputFormat3.java:287)
at org.apache.kylin.storage.hbase.steps.HFileOutputFormat3$1.write(HFileOutputFormat3.java:243)
at org.apache.hadoop.mapred.ReduceTask$NewTrackingRecordWriter.write(ReduceTask.java:559)
at org.apache.hadoop.mapreduce.task.TaskInputOutputContextImpl.write(TaskInputOutputContextImpl.java:89)
at org.apache.hadoop.mapreduce.lib.reduce.WrappedReducer$Context.write(WrappedReducer.java:105)
at org.apache.kylin.storage.hbase.steps.CubeHFileJob$KeyValueReducer.doReduce(CubeHFileJob.java:141)
at org.apache.kylin.storage.hbase.steps.CubeHFileJob$KeyValueReducer.doReduce(CubeHFileJob.java:135)
at org.apache.kylin.engine.mr.KylinReducer.reduce(KylinReducer.java:77)
at org.apache.hadoop.mapreduce.Reducer.run(Reducer.java:171)
at org.apache.hadoop.mapred.ReduceTask.runNewReducer(ReduceTask.java:628)
at org.apache.hadoop.mapred.ReduceTask.run(ReduceTask.java:390)
at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:174)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:422)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1732)
at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:168)
{code}
I try move all of hbase lib has on {KYLIN_HOME}/lib ,but this problem not solved.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)