You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@kylin.apache.org by Na Zhai <na...@kyligence.io> on 2019/01/31 04:52:39 UTC
答复: Error when activating spark interactive on v2.6
Hi, hubert stefani.
It seems the conflict of the jar, maybe you can try to put the jersey-client jar in the yarn service into the jars directory of the Spark.
发送自 Windows 10 版邮件<https://go.microsoft.com/fwlink/?LinkId=550986>应用
________________________________
发件人: hubert stefani <hu...@yahoo.fr>
发送时间: Monday, January 28, 2019 10:24:58 PM
收件人: user@kylin.apache.org
主题: Error when activating spark interactive on v2.6
Hi,
It seems that when we activate the property hereunder , the spark build job fails. Have you noticed such behaviour ? Should I open an issue on Jira ?
Context :
Kylin : v2.6
AWS : EMR 5.18
Property :
kylin.engine.spark-conf.spark.hadoop.yarn.timeline-service.enabled
Error Log :
2019-01-28 10:50:36,384 INFO [pool-13-thread-1] spark.SparkExecutable:38 : 2019-01-28 10:50:36 WARN SparkConf:66 - The configuration key 'spark.yarn.executor.memoryOverhead' has been deprecated as of Spark 2.3 and may be removed in the future. Please use the new key 'spark.executor.memoryOverhead' instead.
2019-01-28 10:50:37,732 INFO [pool-13-thread-1] spark.SparkExecutable:38 : Exception in thread "main" java.lang.NoClassDefFoundError: com/sun/jersey/api/client/config/ClientConfig
2019-01-28 10:50:37,733 INFO [pool-13-thread-1] spark.SparkExecutable:38 : at org.apache.hadoop.yarn.client.api.TimelineClient.createTimelineClient(TimelineClient.java:55)
2019-01-28 10:50:37,733 INFO [pool-13-thread-1] spark.SparkExecutable:38 : at org.apache.hadoop.yarn.client.api.impl.YarnClientImpl.createTimelineClient(YarnClientImpl.java:181)
2019-01-28 10:50:37,733 INFO [pool-13-thread-1] spark.SparkExecutable:38 : at org.apache.hadoop.yarn.client.api.impl.YarnClientImpl.serviceInit(YarnClientImpl.java:168)
2019-01-28 10:50:37,733 INFO [pool-13-thread-1] spark.SparkExecutable:38 : at org.apache.hadoop.service.AbstractService.init(AbstractService.java:163)
2019-01-28 10:50:37,733 INFO [pool-13-thread-1] spark.SparkExecutable:38 : at org.apache.spark.deploy.yarn.Client.submitApplication(Client.scala:151)
2019-01-28 10:50:37,734 INFO [pool-13-thread-1] spark.SparkExecutable:38 : at org.apache.spark.deploy.yarn.Client.run(Client.scala:1152)
2019-01-28 10:50:37,734 INFO [pool-13-thread-1] spark.SparkExecutable:38 : at org.apache.spark.deploy.yarn.YarnClusterApplication.start(Client.scala:1520)
2019-01-28 10:50:37,734 INFO [pool-13-thread-1] spark.SparkExecutable:38 : at org.apache.spark.deploy.SparkSubmit$.org$apache$spark$deploy$SparkSubmit$$runMain(SparkSubmit.scala:894)
2019-01-28 10:50:37,734 INFO [pool-13-thread-1] spark.SparkExecutable:38 : at org.apache.spark.deploy.SparkSubmit$.doRunMain$1(SparkSubmit.scala:198)
2019-01-28 10:50:37,734 INFO [pool-13-thread-1] spark.SparkExecutable:38 : at org.apache.spark.deploy.SparkSubmit$.submit(SparkSubmit.scala:228)
2019-01-28 10:50:37,734 INFO [pool-13-thread-1] spark.SparkExecutable:38 : at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:137)
2019-01-28 10:50:37,734 INFO [pool-13-thread-1] spark.SparkExecutable:38 : at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala)
2019-01-28 10:50:37,735 INFO [pool-13-thread-1] spark.SparkExecutable:38 : Caused by: java.lang.ClassNotFoundException: com.sun.jersey.api.client.config.ClientConfig
2019-01-28 10:50:37,735 INFO [pool-13-thread-1] spark.SparkExecutable:38 : at java.net.URLClassLoader.findClass(URLClassLoader.java:382)
2019-01-28 10:50:37,735 INFO [pool-13-thread-1] spark.SparkExecutable:38 : at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
2019-01-28 10:50:37,735 INFO [pool-13-thread-1] spark.SparkExecutable:38 : at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:349)
2019-01-28 10:50:37,735 INFO [pool-13-thread-1] spark.SparkExecutable:38 : at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
2019-01-28 10:50:37,735 INFO [pool-13-thread-1] spark.SparkExecutable:38 : ... 12 more
2019-01-28 10:50:37,737 INFO [pool-13-thread-1] spark.SparkExecutable:38 : 2019-01-28 10:50:37 INFO ShutdownHookManager:54 - Shutdown hook called
Hubert