You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@kylin.apache.org by Billy Liu <bi...@apache.org> on 2016/12/19 10:46:54 UTC
Re: Error when #2 Step: Redistribute Flat Hive Table - File does not exist
There may be something wrong in the step 1. Please check the log carefully.
One quick solution, discard the job and submit the build again.
2016-12-19 18:35 GMT+08:00 雨日听风 <49...@qq.com>:
> When I build a cube in kylin1.6, I get error in step #2: Redistribute Flat
> Hive Table
>
> Please help! Thank you very much!
>
> env: kylin1.6 is in a independent server, and have 2 other server cluster:
> clusterA(hive only) and clusterB(hbase only).
> Error is:
>
> 2016-12-19 10:28:00,641 INFO [pool-8-thread-7]
> execution.AbstractExecutable:36 : Compute row count of flat hive table,
> cmd:
> 2016-12-19 10:28:00,642 INFO [pool-8-thread-7]
> execution.AbstractExecutable:36 : hive -e "USE boco;
> SET dfs.replication=2;
> SET hive.exec.compress.output=true;
> SET hive.auto.convert.join.noconditionaltask=true;
> SET hive.auto.convert.join.noconditionaltask.size=100000000;
> SET mapreduce.output.fileoutputformat.compress.type=BLOCK;
> SET mapreduce.job.split.metainfo.maxsize=-1;
> SET mapreduce.job.queuename=young;
> SET tez.queue.name=young;
>
> set hive.exec.compress.output=false;
>
> set hive.exec.compress.output=false;
> INSERT OVERWRITE DIRECTORY '/young/kylin_test/kylin_
> metadata_test/kylin-678266c0-ba0e-48b4-bdb5-6e578320375a/row_count'
> SELECT count(*) FROM kylin_intermediate_hbase_in_
> testCluster_CUBE_f9468805_eabf_4b54_bf2b_182e4c86214a;
>
> "
> 2016-12-19 10:28:03,277 INFO [pool-8-thread-7]
> execution.AbstractExecutable:36 : WARNING: Use "yarn jar" to launch YARN
> applications.
> 2016-12-19 10:28:04,444 INFO [pool-8-thread-7]
> execution.AbstractExecutable:36 :
> 2016-12-19 10:28:04,445 INFO [pool-8-thread-7]
> execution.AbstractExecutable:36 : Logging initialized using configuration
> in file:/etc/hive/conf/hive-log4j.properties
> 2016-12-19 10:28:14,700 INFO [pool-8-thread-7]
> execution.AbstractExecutable:36 : OK
> 2016-12-19 10:28:14,703 INFO [pool-8-thread-7]
> execution.AbstractExecutable:36 : Time taken: 0.935 seconds
> 2016-12-19 10:28:15,559 INFO [pool-8-thread-7]
> execution.AbstractExecutable:36 : Query ID =
> young_20161219102814_a7104fd4-ba83-47fc-ac0b-0c9bef4e1969
> 2016-12-19 10:28:15,560 INFO [pool-8-thread-7]
> execution.AbstractExecutable:36 : Total jobs = 1
> 2016-12-19 10:28:15,575 INFO [pool-8-thread-7]
> execution.AbstractExecutable:36 : Launching Job 1 out of 1
> 2016-12-19 10:28:22,842 INFO [pool-8-thread-7]
> execution.AbstractExecutable:36 :
> 2016-12-19 10:28:22,842 INFO [pool-8-thread-7]
> execution.AbstractExecutable:36 :
> 2016-12-19 10:28:23,104 INFO [pool-8-thread-7]
> execution.AbstractExecutable:36 : Status: Running (Executing on YARN
> cluster with App id application_1473415773736_1063281)
> 2016-12-19 10:28:23,104 INFO [pool-8-thread-7]
> execution.AbstractExecutable:36 :
> 2016-12-19 10:28:23,104 INFO [pool-8-thread-7]
> execution.AbstractExecutable:36 : Map 1: -/- Reducer 2: 0/1
> 2016-12-19 10:28:23,307 INFO [pool-8-thread-7]
> execution.AbstractExecutable:36 : Map 1: 0/2 Reducer 2: 0/1
> 2016-12-19 10:28:26,363 INFO [pool-8-thread-7]
> execution.AbstractExecutable:36 : Map 1: 0/2 Reducer 2: 0/1
> 2016-12-19 10:28:26,567 INFO [pool-8-thread-7]
> execution.AbstractExecutable:36 : Map 1: 0(+1)/2 Reducer 2: 0/1
> 2016-12-19 10:28:26,596 INFO [pool-7-thread-1]
> threadpool.DefaultScheduler:118 : Job Fetcher: 1 should running, 1 actual
> running, 0 ready, 0 already succeed, 3 error, 1 discarded, 0 others
> 2016-12-19 10:28:26,769 INFO [pool-8-thread-7]
> execution.AbstractExecutable:36 : Map 1: 0(+2)/2 Reducer 2: 0/1
> 2016-12-19 10:28:29,810 INFO [pool-8-thread-7]
> execution.AbstractExecutable:36 : Map 1: 0(+2)/2 Reducer 2: 0/1
> 2016-12-19 10:28:30,217 INFO [pool-8-thread-7]
> execution.AbstractExecutable:36 : Map 1: 1(+1)/2 Reducer 2: 0(+1)/1
> 2016-12-19 10:28:30,826 INFO [pool-8-thread-7]
> execution.AbstractExecutable:36 : Map 1: 2/2 Reducer 2: 0(+1)/1
> 2016-12-19 10:28:31,232 INFO [pool-8-thread-7]
> execution.AbstractExecutable:36 : Map 1: 2/2 Reducer 2: 1/1
> 2016-12-19 10:28:31,319 INFO [pool-8-thread-7]
> execution.AbstractExecutable:36 : Moving data to: /young/kylin_test/kylin_
> metadata_test/kylin-678266c0-ba0e-48b4-bdb5-6e578320375a/row_count
> 2016-12-19 10:28:31,406 INFO [pool-8-thread-7]
> execution.AbstractExecutable:36 : OK
> 2016-12-19 10:28:31,454 INFO [pool-8-thread-7]
> execution.AbstractExecutable:36 : Time taken: 16.701 seconds
> 2016-12-19 10:28:35,074 ERROR [pool-8-thread-7]
> execution.AbstractExecutable:357 : job:678266c0-ba0e-48b4-bdb5-6e578320375a-01
> execute finished with exception
> java.io.FileNotFoundException: File does not exist:
> /young/kylin_test/kylin_metadata_test/kylin-678266c0-
> ba0e-48b4-bdb5-6e578320375a/row_count/000000_0
> at org.apache.hadoop.hdfs.server.namenode.INodeFile.valueOf(
> INodeFile.java:71)
> at org.apache.hadoop.hdfs.server.namenode.INodeFile.valueOf(
> INodeFile.java:61)
> at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.
> getBlockLocationsInt(FSNamesystem.java:1820)
> at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getBlockLocations(
> FSNamesystem.java:1791)
> at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getBlockLocations(
> FSNamesystem.java:1704)
> at org.apache.hadoop.hdfs.server.namenode.NameNodeRpcServer.
> getBlockLocations(NameNodeRpcServer.java:587)
> at org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolServerSi
> deTranslatorPB.getBlockLocations(ClientNamenodeProtocolServerSi
> deTranslatorPB.java:365)
> at org.apache.hadoop.hdfs.protocol.proto.ClientNamenodeProtocolProtos$
> ClientNamenodeProtocol$2.callBlockingMethod(ClientNamenodeProtocolProtos.
> java)
> at org.apache.hadoop.ipc.ProtobufRpcEngine$Server$
> ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:616)
> at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:969)
> at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2081)
> at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2077)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:415)
> at org.apache.hadoop.security.UserGroupInformation.doAs(
> UserGroupInformation.java:1657)
> at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2075)
>
> at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
> at sun.reflect.NativeConstructorAccessorImpl.newInstance(
> NativeConstructorAccessorImpl.java:57)
> at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(
> DelegatingConstructorAccessorImpl.java:45)
> at java.lang.reflect.Constructor.newInstance(Constructor.java:526)
> at org.apache.hadoop.ipc.RemoteException.instantiateException(
> RemoteException.java:106)
> at org.apache.hadoop.ipc.RemoteException.unwrapRemoteException(
> RemoteException.java:73)
> at org.apache.hadoop.hdfs.DFSClient.callGetBlockLocations(
> DFSClient.java:1242)
> at org.apache.hadoop.hdfs.DFSClient.getLocatedBlocks(DFSClient.java:1227)
> at org.apache.hadoop.hdfs.DFSClient.getLocatedBlocks(DFSClient.java:1215)
> at org.apache.hadoop.hdfs.DFSInputStream.fetchLocatedBlocksAndGetLastBl
> ockLength(DFSInputStream.java:303)
> at org.apache.hadoop.hdfs.DFSInputStream.openInfo(
> DFSInputStream.java:269)
> at org.apache.hadoop.hdfs.DFSInputStream.<init>(DFSInputStream.java:261)
> at org.apache.hadoop.hdfs.DFSClient.open(DFSClient.java:1540)
> at org.apache.hadoop.hdfs.DistributedFileSystem$3.
> doCall(DistributedFileSystem.java:303)
> at org.apache.hadoop.hdfs.DistributedFileSystem$3.
> doCall(DistributedFileSystem.java:299)
> at org.apache.hadoop.fs.FileSystemLinkResolver.resolve(
> FileSystemLinkResolver.java:81)
> at org.apache.hadoop.hdfs.DistributedFileSystem.open(
> DistributedFileSystem.java:299)
> at org.apache.hadoop.fs.FileSystem.open(FileSystem.java:767)
> at org.apache.kylin.source.hive.HiveMRInput$
> RedistributeFlatHiveTableStep.readRowCountFromFile(HiveMRInput.java:294)
> at org.apache.kylin.source.hive.HiveMRInput$
> RedistributeFlatHiveTableStep.doWork(HiveMRInput.java:337)
> at org.apache.kylin.job.execution.AbstractExecutable.
> execute(AbstractExecutable.java:113)
> at org.apache.kylin.job.execution.DefaultChainedExecutable.doWork(
> DefaultChainedExecutable.java:57)
> at org.apache.kylin.job.execution.AbstractExecutable.
> execute(AbstractExecutable.java:113)
> at org.apache.kylin.job.impl.threadpool.DefaultScheduler$JobRunner.run(
> DefaultScheduler.java:136)
> at java.util.concurrent.ThreadPoolExecutor.runWorker(
> ThreadPoolExecutor.java:1145)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(
> ThreadPoolExecutor.java:615)
> at java.lang.Thread.run(Thread.java:745)
> Caused by: org.apache.hadoop.ipc.RemoteException(java.io.FileNotFoundException):
> File does not exist: /young/kylin_test/kylin_metadata_test/kylin-678266c0-
> ba0e-48b4-bdb5-6e578320375a/row_count/000000_0
> at org.apache.hadoop.hdfs.server.namenode.INodeFile.valueOf(
> INodeFile.java:71)
> at org.apache.hadoop.hdfs.server.namenode.INodeFile.valueOf(
> INodeFile.java:61)
> at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.
> getBlockLocationsInt(FSNamesystem.java:1820)
> at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getBlockLocations(
> FSNamesystem.java:1791)
> at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getBlockLocations(
> FSNamesystem.java:1704)
> at org.apache.hadoop.hdfs.server.namenode.NameNodeRpcServer.
> getBlockLocations(NameNodeRpcServer.java:587)
> at org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolServerSi
> deTranslatorPB.getBlockLocations(ClientNamenodeProtocolServerSi
> deTranslatorPB.java:365)
> at org.apache.hadoop.hdfs.protocol.proto.ClientNamenodeProtocolProtos$
> ClientNamenodeProtocol$2.callBlockingMethod(ClientNamenodeProtocolProtos.
> java)
> at org.apache.hadoop.ipc.ProtobufRpcEngine$Server$
> ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:616)
> at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:969)
> at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2081)
> at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2077)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:415)
> at org.apache.hadoop.security.UserGroupInformation.doAs(
> UserGroupInformation.java:1657)
> at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2075)
>
> at org.apache.hadoop.ipc.Client.call(Client.java:1427)
> at org.apache.hadoop.ipc.Client.call(Client.java:1358)
> at org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.
> invoke(ProtobufRpcEngine.java:229)
> at com.sun.proxy.$Proxy25.getBlockLocations(Unknown Source)
> at org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolTranslat
> orPB.getBlockLocations(ClientNamenodeProtocolTranslatorPB.java:255)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at sun.reflect.NativeMethodAccessorImpl.invoke(
> NativeMethodAccessorImpl.java:57)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(
> DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:606)
> at org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(
> RetryInvocationHandler.java:252)
> at org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(
> RetryInvocationHandler.java:104)
> at com.sun.proxy.$Proxy26.getBlockLocations(Unknown Source)
> at org.apache.hadoop.hdfs.DFSClient.callGetBlockLocations(
> DFSClient.java:1240)
> ... 20 more
> 2016-12-19 10:28:35,078 DEBUG [pool-8-thread-7] dao.ExecutableDao:210 :
> updating job output, id: 678266c0-ba0e-48b4-bdb5-6e578320375a-01
> 2016-12-19 10:28:35,082 DEBUG [pool-8-thread-7]
> hbase.HBaseResourceStore:262 : Update row /execute_output/678266c0-ba0e-48b4-bdb5-6e578320375a-01
> from oldTs: 1482114480638, to newTs: 1482114515078, operation result: true
> 2016-12-19 10:28:35,085 DEBUG [pool-8-thread-7] dao.ExecutableDao:210 :
> updating job output, id: 678266c0-ba0e-48b4-bdb5-6e578320375a-01
>