You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@kylin.apache.org by "Shaofeng SHI (JIRA)" <ji...@apache.org> on 2017/05/31 01:00:27 UTC

[jira] [Commented] (KYLIN-2651) Couldn't find global dict index file

    [ https://issues.apache.org/jira/browse/KYLIN-2651?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16030451#comment-16030451 ] 

Shaofeng SHI commented on KYLIN-2651:
-------------------------------------

[~kangkaisen], kaisen, do you have idea with this error? I'm running with latest master branch.

> Couldn't find global dict index file
> ------------------------------------
>
>                 Key: KYLIN-2651
>                 URL: https://issues.apache.org/jira/browse/KYLIN-2651
>             Project: Kylin
>          Issue Type: Bug
>            Reporter: Shaofeng SHI
>
> Today I got an error when run a tes cube with Spark engine:
> {code}
> 17/05/31 08:50:28 WARN TaskSetManager: Lost task 0.0 in stage 0.0 (TID 0, sandbox): java.lang.RuntimeException: java.io.FileNotFoundException: File does not exist: /kylin/kylin_metadata/resources/GlobalDict/dict/DEFAULT.TEST_KYLIN_FACT/TEST_COUNT_DISTINCT_BITMAP/version_1496191754004/.index
> 	at org.apache.hadoop.hdfs.server.namenode.INodeFile.valueOf(INodeFile.java:71)
> 	at org.apache.hadoop.hdfs.server.namenode.INodeFile.valueOf(INodeFile.java:61)
> 	at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getBlockLocationsInt(FSNamesystem.java:1828)
> 	at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getBlockLocations(FSNamesystem.java:1799)
> 	at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getBlockLocations(FSNamesystem.java:1712)
> 	at org.apache.hadoop.hdfs.server.namenode.NameNodeRpcServer.getBlockLocations(NameNodeRpcServer.java:652)
> 	at org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolServerSideTranslatorPB.getBlockLocations(ClientNamenodeProtocolServerSideTranslatorPB.java:365)
> 	at org.apache.hadoop.hdfs.protocol.proto.ClientNamenodeProtocolProtos$ClientNamenodeProtocol$2.callBlockingMethod(ClientNamenodeProtocolProtos.java)
> 	at org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:616)
> 	at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:969)
> 	at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2151)
> 	at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2147)
> 	at java.security.AccessController.doPrivileged(Native Method)
> 	at javax.security.auth.Subject.doAs(Subject.java:415)
> 	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1657)
> 	at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2145)
> 	at org.apache.kylin.dict.DictionaryManager.getDictionaryInfo(DictionaryManager.java:129)
> 	at org.apache.kylin.cube.CubeManager.getDictionary(CubeManager.java:264)
> 	at org.apache.kylin.cube.CubeSegment.getDictionary(CubeSegment.java:329)
> 	at org.apache.kylin.cube.CubeSegment.buildDictionaryMap(CubeSegment.java:321)
> 	at org.apache.kylin.engine.spark.SparkCubingByLayer$2.call(SparkCubingByLayer.java:205)
> 	at org.apache.kylin.engine.spark.SparkCubingByLayer$2.call(SparkCubingByLayer.java:193)
> 	at org.apache.spark.api.java.JavaPairRDD$$anonfun$pairFunToScalaFun$1.apply(JavaPairRDD.scala:1018)
> 	at org.apache.spark.api.java.JavaPairRDD$$anonfun$pairFunToScalaFun$1.apply(JavaPairRDD.scala:1018)
> 	at scala.collection.Iterator$$anon$11.next(Iterator.scala:328)
> 	at org.apache.spark.util.collection.ExternalSorter.insertAll(ExternalSorter.scala:191)
> 	at org.apache.spark.shuffle.sort.SortShuffleWriter.write(SortShuffleWriter.scala:64)
> 	at org.apache.spark.scheduler.ShuffleMapTask.runTask(ShuffleMapTask.scala:73)
> 	at org.apache.spark.scheduler.ShuffleMapTask.runTask(ShuffleMapTask.scala:41)
> 	at org.apache.spark.scheduler.Task.run(Task.scala:89)
> 	at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:227)
> 	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> 	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> 	at java.lang.Thread.run(Thread.java:745)
> {code}
> Then I checked the file system, there is only the ".index_v2" file:
> {code}
> [root@sandbox ~]# hadoop fs -ls /kylin/kylin_metadata/resources/GlobalDict/dict/DEFAULT.TEST_KYLIN_FACT/TEST_COUNT_DISTINCT_BITMAP/version_1496191754004/
> Found 2 items
> -rw-r--r--   3 shishaofeng hdfs         93 2017-05-31 08:49 /kylin/kylin_metadata/resources/GlobalDict/dict/DEFAULT.TEST_KYLIN_FACT/TEST_COUNT_DISTINCT_BITMAP/version_1496191754004/.index_v2
> -rw-r--r--   3 shishaofeng hdfs       8036 2017-05-31 08:49 /kylin/kylin_metadata/resources/GlobalDict/dict/DEFAULT.TEST_KYLIN_FACT/TEST_COUNT_DISTINCT_BITMAP/version_1496191754004/cached_1496191753923_1
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)