You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@kylin.apache.org by "LIU Ze (刘则)" <li...@wanda.cn> on 2015/11/05 11:44:33 UTC

Error in " Load HFile to HBase Table"

hello,

________________________________
hive and hbase in two cluster.

I set "kylin.hbase.cluster.fs=hdfs://10.209.21.11:10010" in kylin.properties ,and hive cluster is "hdfs://wanda"

in the step of " Load HFile to HBase Table"  ,it maked a error.

in the class of "org.apache.hadoop.fs.FileSystem" ,the method "checkPath"  will check hdfs path,and it find two path is different.
but in the before,the" org.apache.kylin.job.cube.CubingJobBuilder.createBulkLoadStep" had get the hbase cluster path,Why are they different?

the error:
[pool-5-thread-10]:[2015-11-04 20:25:32,706][ERROR][org.apache.kylin.job.common.HadoopShellExecutable.doWork(HadoopShellExecutable.java:64)] - error execute HadoopShellExecutable{id=1a90fef6-89b1-49b0-80f2-274d3022d76a-10, name=Load HFil
e to HBase Table, state=RUNNING}
java.io.IOException: BulkLoad encountered an unrecoverable problem
        at org.apache.hadoop.hbase.mapreduce.LoadIncrementalHFiles.bulkLoadPhase(LoadIncrementalHFiles.java:443)
        at org.apache.hadoop.hbase.mapreduce.LoadIncrementalHFiles.doBulkLoad(LoadIncrementalHFiles.java:375)
        at org.apache.hadoop.hbase.mapreduce.LoadIncrementalHFiles.run(LoadIncrementalHFiles.java:951)
        at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
        at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:84)
        at org.apache.kylin.job.hadoop.hbase.BulkLoadJob.run(BulkLoadJob.java:87)
        at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
        at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:84)
        at org.apache.kylin.job.common.HadoopShellExecutable.doWork(HadoopShellExecutable.java:62)
        at org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:107)
        at org.apache.kylin.job.execution.DefaultChainedExecutable.doWork(DefaultChainedExecutable.java:51)
        at org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:107)
        at org.apache.kylin.job.impl.threadpool.DefaultScheduler$JobRunner.run(DefaultScheduler.java:130)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
        at java.lang.Thread.run(Thread.java:745)
Caused by: org.apache.hadoop.hbase.client.RetriesExhaustedException: Failed after attempts=35, exceptions:
Wed Nov 04 04:16:21 GMT-08:00 2015, org.apache.hadoop.hbase.client.RpcRetryingCaller@409926b7, java.io.IOException: java.io.IOException: Wrong FS: hdfs://10.209.21.11:10010/tmp/kylin/kylin_metadata/kylin-1a90fef6-89b1-49b0-80f2-274d3022d
76a/app/hfile/F1/cff10314d34d45bd8540e23f0df4e2a2, expected: hdfs://wanda
        at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2132)
        at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:101)
        at org.apache.hadoop.hbase.ipc.RpcExecutor.consumerLoop(RpcExecutor.java:130)
        at org.apache.hadoop.hbase.ipc.RpcExecutor$1.run(RpcExecutor.java:107)
        at java.lang.Thread.run(Thread.java:745)