You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@kylin.apache.org by "ASF subversion and git services (JIRA)" <ji...@apache.org> on 2018/12/23 01:45:00 UTC
[jira] [Commented] (KYLIN-3726) KylinSession should load spark
properties from spark-defaults.conf
[ https://issues.apache.org/jira/browse/KYLIN-3726?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16727773#comment-16727773 ]
ASF subversion and git services commented on KYLIN-3726:
--------------------------------------------------------
Commit 011ff0887d4a83b3a3373487ad66ce4b652bd7ff in kylin's branch refs/heads/kylin-on-parquet from Liu Shaohui
[ https://gitbox.apache.org/repos/asf?p=kylin.git;h=011ff08 ]
KYLIN-3726: KylinSession should load spark properties from spark-defaults.conf
> KylinSession should load spark properties from spark-defaults.conf
> ------------------------------------------------------------------
>
> Key: KYLIN-3726
> URL: https://issues.apache.org/jira/browse/KYLIN-3726
> Project: Kylin
> Issue Type: Sub-task
> Components: Storage - Parquet
> Reporter: Liu Shaohui
> Assignee: Liu Shaohui
> Priority: Major
>
> When testing parquet storage, the spark session job failed to be submit for no JAVA_HOME in executor env. This config is set in the spark default property file: spark-defaults.conf.
>
> {code}
> 2018-12-18,15:13:15,466 ERROR org.apache.spark.deploy.yarn.YarnAllocator: Failed to launch executor 6 on container container_e823_1541646991414_1025309_01_000007 java.util.NoSuchElementException: key not found: JAVA_HOME at scala.collection.MapLike$class.default(MapLike.scala:228) at scala.collection.AbstractMap.default(Map.scala:59) at scala.collection.mutable.HashMap.apply(HashMap.scala:65) at org.apache.spark.deploy.yarn.ExecutorRunnable$$anonfun$prepareEnvironment$3$$anonfun$apply$3.apply(ExecutorRunnable.scala:286) at org.apache.spark.deploy.yarn.ExecutorRunnable$$anonfun$prepareEnvironment$3$$anonfun$apply$3.apply(ExecutorRunnable.scala:275) at scala.Option.foreach(Option.scala:257) at org.apache.spark.deploy.yarn.ExecutorRunnable$$anonfun$prepareEnvironment$3.apply(ExecutorRunnable.scala:275) at org.apache.spark.deploy.yarn.ExecutorRunnable$$anonfun$prepareEnvironment$3.apply(ExecutorRunnable.scala:274) at scala.Option.foreach(Option.scala:257) at org.apache.spark.deploy.yarn.ExecutorRunnable.prepareEnvironment(ExecutorRunnable.scala:274) at org.apache.spark.deploy.yarn.ExecutorRunnable.startContainer(ExecutorRunnable.scala:92) at org.apache.spark.deploy.yarn.ExecutorRunnable.run(ExecutorRunnable.scala:69) at org.apache.spark.deploy.yarn.YarnAllocator$$anonfun$runAllocatedContainers$1$$anon$1.run(YarnAllocator.scala:556) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) at java.lang.Thread.run(Thread.java:745)
> {code}
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)