You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@kylin.apache.org by "weibin0516 (Jira)" <ji...@apache.org> on 2020/01/14 06:54:00 UTC
[jira] [Closed] (KYLIN-4339) Extract Fact Table Distinct Columns
fail due to no kylin installed on worker node
[ https://issues.apache.org/jira/browse/KYLIN-4339?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
weibin0516 closed KYLIN-4339.
-----------------------------
Resolution: Duplicate
> Extract Fact Table Distinct Columns fail due to no kylin installed on worker node
> ---------------------------------------------------------------------------------
>
> Key: KYLIN-4339
> URL: https://issues.apache.org/jira/browse/KYLIN-4339
> Project: Kylin
> Issue Type: Bug
> Affects Versions: v3.0.0
> Reporter: weibin0516
> Assignee: weibin0516
> Priority: Major
>
> After set kylin.engine.spark-fact-distinct to true, using spark engine to build cube will fail, error message as follow
> {code:java}
> 2020-01-13 22:19:23 INFO BlockManagerMaster:54 - BlockManagerMaster stopped
> 2020-01-13 22:19:23 INFO OutputCommitCoordinator$OutputCommitCoordinatorEndpoint:54 - OutputCommitCoordinator stopped!
> 2020-01-13 22:19:23 INFO SparkContext:54 - Successfully stopped SparkContext
> Exception in thread "main" java.lang.RuntimeException: error execute org.apache.kylin.engine.spark.SparkFactDistinct. Root cause: Job aborted due to stage failure: Task 9 in stage 1.0 failed 4 times, most recent failure: Lost task 9.3 in stage 1.0 (TID 32, sql-gateway-eu95-17.gz00c.test.alipay.net, executor 7): org.apache.kylin.common.KylinConfigCannotInitException: Didn't find KYLIN_CONF or KYLIN_HOME, please set one of them
> at org.apache.kylin.common.KylinConfig.getSitePropertiesFile(KylinConfig.java:336)
> at org.apache.kylin.common.KylinConfig.buildSiteOrderedProps(KylinConfig.java:378)
> at org.apache.kylin.common.KylinConfig.buildSiteProperties(KylinConfig.java:358)
> at org.apache.kylin.common.KylinConfig.getInstanceFromEnv(KylinConfig.java:137)
> at org.apache.kylin.dict.CacheDictionary.enableCache(CacheDictionary.java:105)
> at org.apache.kylin.dict.TrieDictionaryForest.initForestCache(TrieDictionaryForest.java:394)
> at org.apache.kylin.dict.TrieDictionaryForest.init(TrieDictionaryForest.java:77)
> at org.apache.kylin.dict.TrieDictionaryForest.<init>(TrieDictionaryForest.java:67)
> at org.apache.kylin.dict.TrieDictionaryForestBuilder.build(TrieDictionaryForestBuilder.java:114)
> at org.apache.kylin.dict.DictionaryGenerator$NumberTrieDictForestBuilder.build(DictionaryGenerator.java:312)
> at org.apache.kylin.engine.spark.SparkFactDistinct$MultiOutputFunction.call(SparkFactDistinct.java:774)
> at org.apache.kylin.engine.spark.SparkFactDistinct$MultiOutputFunction.call(SparkFactDistinct.java:650)
> at org.apache.spark.api.java.JavaRDDLike$$anonfun$fn$7$1.apply(JavaRDDLike.scala:186)
> at org.apache.spark.api.java.JavaRDDLike$$anonfun$fn$7$1.apply(JavaRDDLike.scala:186)
> at org.apache.spark.rdd.RDD$$anonfun$mapPartitions$1$$anonfun$apply$23.apply(RDD.scala:800)
> at org.apache.spark.rdd.RDD$$anonfun$mapPartitions$1$$anonfun$apply$23.apply(RDD.scala:800)
> at org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:38)
> at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:324)
> at org.apache.spark.rdd.RDD.iterator(RDD.scala:288)
> at org.apache.spark.scheduler.ResultTask.runTask(ResultTask.scala:87)
> at org.apache.spark.scheduler.Task.run(Task.scala:109)
> at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:345)
> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> at java.lang.Thread.run(Thread.java:756)
> {code}
> we should put kylin.properties in the execution environment of spark application (via --files) to fix this problem
--
This message was sent by Atlassian Jira
(v8.3.4#803005)