You are viewing a plain text version of this content. The canonical link for it is here.

Posted to user@kylin.apache.org by lufeng <am...@163.com> on 2017/05/02 03:54:17 UTC

Cube build error on Step 7 Build Base Cuboid Data

Hi All 

The cube build failed on step 7 with following error message. Any help please 

Hadoop 2.7 HBase 1.1.2 Kylin 1.6.0

<<<
2017-05-01 01:10:38,430 INFO  [pool-9-thread-10] common.AbstractHadoopJob:499 : HDFS meta dir is: file:///opt/kylin/bin/../tomcat/temp/kylin_job_meta8686959176405233518/meta
2017-05-01 01:10:38,430 INFO  [pool-9-thread-10] common.AbstractHadoopJob:372 : Job 'tmpfiles' updated -- file:///opt/kylin/bin/../tomcat/temp/kylin_job_meta8686959176405233518/meta
2017-05-01 01:10:38,433 INFO  [pool-9-thread-10] mapred.FileInputFormat:249 : Total input paths to process : 0
2017-05-01 01:10:38,433 INFO  [pool-9-thread-10] common.AbstractHadoopJob:506 : tempMetaFileString is : file:///opt/kylin/bin/../tomcat/temp/kylin_job_meta8686959176405233518/meta
2017-05-01 01:10:38,440 ERROR [pool-9-thread-10] common.MapReduceExecutable:127 : error execute MapReduceExecutable{id=37f887a1-1f2a-40d0-a4dc-9eca7be28ab7-06, name=Build Base Cuboid Data, state=RUNNING}
java.lang.IllegalArgumentException: Map input splits are 0 bytes, something is wrong!
        at org.apache.kylin.engine.mr.common.AbstractHadoopJob.getTotalMapInputMB(AbstractHadoopJob.java:555)
        at org.apache.kylin.engine.mr.steps.CuboidJob.setReduceTaskNum(CuboidJob.java:175)
        at org.apache.kylin.engine.mr.steps.CuboidJob.run(CuboidJob.java:138)
        at org.apache.kylin.engine.mr.MRUtil.runMRJob(MRUtil.java:92)
        at org.apache.kylin.engine.mr.common.MapReduceExecutable.doWork(MapReduceExecutable.java:120)
        at org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:113)
        at org.apache.kylin.job.execution.DefaultChainedExecutable.doWork(DefaultChainedExecutable.java:57)
        at org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:113)
        at org.apache.kylin.job.impl.threadpool.DefaultScheduler$JobRunner.run(DefaultScheduler.java:136)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
        at java.lang.Thread.run(Thread.java:745)
>>>>

I found the same error message in Jira (https://issues.apache.org/jira/browse/KYLIN-1912 <https://issues.apache.org/jira/browse/KYLIN-1912>) but I am not sure they are same issues. 

Thanks.

Re: Cube build error on Step 7 Build Base Cuboid Data

Posted by lufeng <am...@163.com>.

Hi ShaoFeng

I think it’s not an empty segment problem because Kylin has also consider this situation. Maybe is an none file issue that Kylin is not checked. 

I have open a issue on Kylin JIRA. Please check it https://issues.apache.org/jira/browse/KYLIN-2590 <https://issues.apache.org/jira/browse/KYLIN-2590>

> 在 2017年5月4日，上午9:28，ShaoFeng Shi <sh...@apache.org> 写道：
> 
> Lufeng,
> 
> Thanks for the update. I also received reporting about this error recently, but no finding. Is it because the intermediate table be empty (no data in the selected time range)? As Kylin allows empty segment, this should not block the build, please feel free to open a JIRA to us.

Re: Cube build error on Step 7 Build Base Cuboid Data

Posted by lufeng <am...@163.com>.

Hi ShaoFeng

I think it’s not an empty segment problem because Kylin has also consider this situation. Maybe is an none file issue that Kylin is not checked. 

I have open a issue on Kylin JIRA. Please check it https://issues.apache.org/jira/browse/KYLIN-2590 <https://issues.apache.org/jira/browse/KYLIN-2590>

> 在 2017年5月4日，上午9:28，ShaoFeng Shi <sh...@apache.org> 写道：
> 
> Lufeng,
> 
> Thanks for the update. I also received reporting about this error recently, but no finding. Is it because the intermediate table be empty (no data in the selected time range)? As Kylin allows empty segment, this should not block the build, please feel free to open a JIRA to us.

Re: Cube build error on Step 7 Build Base Cuboid Data

Posted by ShaoFeng Shi <sh...@apache.org>.

Lufeng,

Thanks for the update. I also received reporting about this error recently,
but no finding. Is it because the intermediate table be empty (no data in
the selected time range)? As Kylin allows empty segment, this should not
block the build, please feel free to open a JIRA to us.

Re: Cube build error on Step 7 Build Base Cuboid Data

Posted by ShaoFeng Shi <sh...@apache.org>.

Lufeng,

Thanks for the update. I also received reporting about this error recently,
but no finding. Is it because the intermediate table be empty (no data in
the selected time range)? As Kylin allows empty segment, this should not
block the build, please feel free to open a JIRA to us.

Re: Cube build error on Step 7 Build Base Cuboid Data

Posted by lufeng <am...@163.com>.

I sovled this issue by changing execution engine from Tez to MR。 Because Tez engine has zero file issue . https://issues.apache.org/jira/browse/HIVE-13988 <https://issues.apache.org/jira/browse/HIVE-13988>

Thanks all.

> 在 2017年5月2日，上午11:54，lufeng <am...@163.com> 写道：
> 
> Hi All 
> 
> The cube build failed on step 7 with following error message. Any help please 
> 
> Hadoop 2.7 HBase 1.1.2 Kylin 1.6.0
> 
> <<<
> 2017-05-01 01:10:38,430 INFO  [pool-9-thread-10] common.AbstractHadoopJob:499 : HDFS meta dir is: file:///opt/kylin/bin/../tomcat/temp/kylin_job_meta8686959176405233518/meta <file:///opt/kylin/tomcat/temp/kylin_job_meta8686959176405233518/meta>
> 2017-05-01 01:10:38,430 INFO  [pool-9-thread-10] common.AbstractHadoopJob:372 : Job 'tmpfiles' updated -- file:///opt/kylin/bin/../tomcat/temp/kylin_job_meta8686959176405233518/meta <file:///opt/kylin/tomcat/temp/kylin_job_meta8686959176405233518/meta>
> 2017-05-01 01:10:38,433 INFO  [pool-9-thread-10] mapred.FileInputFormat:249 : Total input paths to process : 0
> 2017-05-01 01:10:38,433 INFO  [pool-9-thread-10] common.AbstractHadoopJob:506 : tempMetaFileString is : file:///opt/kylin/bin/../tomcat/temp/kylin_job_meta8686959176405233518/meta <file:///opt/kylin/tomcat/temp/kylin_job_meta8686959176405233518/meta>
> 2017-05-01 01:10:38,440 ERROR [pool-9-thread-10] common.MapReduceExecutable:127 : error execute MapReduceExecutable{id=37f887a1-1f2a-40d0-a4dc-9eca7be28ab7-06, name=Build Base Cuboid Data, state=RUNNING}
> java.lang.IllegalArgumentException: Map input splits are 0 bytes, something is wrong!
>         at org.apache.kylin.engine.mr.common.AbstractHadoopJob.getTotalMapInputMB(AbstractHadoopJob.java:555)
>         at org.apache.kylin.engine.mr.steps.CuboidJob.setReduceTaskNum(CuboidJob.java:175)
>         at org.apache.kylin.engine.mr.steps.CuboidJob.run(CuboidJob.java:138)
>         at org.apache.kylin.engine.mr.MRUtil.runMRJob(MRUtil.java:92)
>         at org.apache.kylin.engine.mr.common.MapReduceExecutable.doWork(MapReduceExecutable.java:120)
>         at org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:113)
>         at org.apache.kylin.job.execution.DefaultChainedExecutable.doWork(DefaultChainedExecutable.java:57)
>         at org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:113)
>         at org.apache.kylin.job.impl.threadpool.DefaultScheduler$JobRunner.run(DefaultScheduler.java:136)
>         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
>         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
>         at java.lang.Thread.run(Thread.java:745)
> >>>>
> 
> I found the same error message in Jira (https://issues.apache.org/jira/browse/KYLIN-1912 <https://issues.apache.org/jira/browse/KYLIN-1912>) but I am not sure they are same issues. 
> 
> Thanks.