You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@kylin.apache.org by "weibin0516 (Jira)" <ji...@apache.org> on 2020/01/13 14:53:00 UTC
[jira] [Created] (KYLIN-4339) Extract Fact Table Distinct Columns
fail due to no kylin installed on worker node
weibin0516 created KYLIN-4339:
---------------------------------
Summary: Extract Fact Table Distinct Columns fail due to no kylin installed on worker node
Key: KYLIN-4339
URL: https://issues.apache.org/jira/browse/KYLIN-4339
Project: Kylin
Issue Type: Bug
Affects Versions: v3.0.0
Reporter: weibin0516
Assignee: weibin0516
After set kylin.engine.spark-fact-distinct to true, using spark engine to build cube will fail, error message as follow
{code:java}
2020-01-13 22:19:23 INFO BlockManagerMaster:54 - BlockManagerMaster stopped
2020-01-13 22:19:23 INFO OutputCommitCoordinator$OutputCommitCoordinatorEndpoint:54 - OutputCommitCoordinator stopped!
2020-01-13 22:19:23 INFO SparkContext:54 - Successfully stopped SparkContext
Exception in thread "main" java.lang.RuntimeException: error execute org.apache.kylin.engine.spark.SparkFactDistinct. Root cause: Job aborted due to stage failure: Task 9 in stage 1.0 failed 4 times, most recent failure: Lost task 9.3 in stage 1.0 (TID 32, sql-gateway-eu95-17.gz00c.test.alipay.net, executor 7): org.apache.kylin.common.KylinConfigCannotInitException: Didn't find KYLIN_CONF or KYLIN_HOME, please set one of them
at org.apache.kylin.common.KylinConfig.getSitePropertiesFile(KylinConfig.java:336)
at org.apache.kylin.common.KylinConfig.buildSiteOrderedProps(KylinConfig.java:378)
at org.apache.kylin.common.KylinConfig.buildSiteProperties(KylinConfig.java:358)
at org.apache.kylin.common.KylinConfig.getInstanceFromEnv(KylinConfig.java:137)
at org.apache.kylin.dict.CacheDictionary.enableCache(CacheDictionary.java:105)
at org.apache.kylin.dict.TrieDictionaryForest.initForestCache(TrieDictionaryForest.java:394)
at org.apache.kylin.dict.TrieDictionaryForest.init(TrieDictionaryForest.java:77)
at org.apache.kylin.dict.TrieDictionaryForest.<init>(TrieDictionaryForest.java:67)
at org.apache.kylin.dict.TrieDictionaryForestBuilder.build(TrieDictionaryForestBuilder.java:114)
at org.apache.kylin.dict.DictionaryGenerator$NumberTrieDictForestBuilder.build(DictionaryGenerator.java:312)
at org.apache.kylin.engine.spark.SparkFactDistinct$MultiOutputFunction.call(SparkFactDistinct.java:774)
at org.apache.kylin.engine.spark.SparkFactDistinct$MultiOutputFunction.call(SparkFactDistinct.java:650)
at org.apache.spark.api.java.JavaRDDLike$$anonfun$fn$7$1.apply(JavaRDDLike.scala:186)
at org.apache.spark.api.java.JavaRDDLike$$anonfun$fn$7$1.apply(JavaRDDLike.scala:186)
at org.apache.spark.rdd.RDD$$anonfun$mapPartitions$1$$anonfun$apply$23.apply(RDD.scala:800)
at org.apache.spark.rdd.RDD$$anonfun$mapPartitions$1$$anonfun$apply$23.apply(RDD.scala:800)
at org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:38)
at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:324)
at org.apache.spark.rdd.RDD.iterator(RDD.scala:288)
at org.apache.spark.scheduler.ResultTask.runTask(ResultTask.scala:87)
at org.apache.spark.scheduler.Task.run(Task.scala:109)
at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:345)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:756)
{code}
we should put kylin.properties in the execution environment of spark application (via --files) to fix this problem
--
This message was sent by Atlassian Jira
(v8.3.4#803005)