You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@spark.apache.org by "Marcelo Vanzin (JIRA)" <ji...@apache.org> on 2014/11/21 01:28:33 UTC
[jira] [Comment Edited] (SPARK-4492) Exception when following
SimpleApp tutorial java.lang.ClassNotFoundException:
org.apache.spark.deploy.yarn.YarnSparkHadoopUtil
[ https://issues.apache.org/jira/browse/SPARK-4492?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14220287#comment-14220287 ]
Marcelo Vanzin edited comment on SPARK-4492 at 11/21/14 12:28 AM:
------------------------------------------------------------------
It's kinda hard to embed Spark. Especially in sam's case, where he's trying to use yarn: spark-yarn artifacts aren't even published to maven central (at least since 1.1), so you have no way of adding them to yours easily. (The final assembly isn't published either, so you can't even use that to embed the "Spark + kitchen sink" jar.)
was (Author: vanzin):
It's kinda hard to embed Spark. Especially in sam's case, where he's trying to use yarn: spark-yarn project's aren't even published to maven central (at least since 1.1), so you have no way of adding them to yours easily. (The final assembly isn't published either, so you can't even use that to embed the "Spark + kitchen sink" jar.)
> Exception when following SimpleApp tutorial java.lang.ClassNotFoundException: org.apache.spark.deploy.yarn.YarnSparkHadoopUtil
> ------------------------------------------------------------------------------------------------------------------------------
>
> Key: SPARK-4492
> URL: https://issues.apache.org/jira/browse/SPARK-4492
> Project: Spark
> Issue Type: Bug
> Reporter: sam
>
> When I follow the example here https://spark.apache.org/docs/1.0.2/quick-start.html and run with "java -cp my.jar my.main.Class" with master set to "yarn-client" I get the below exception.
> Exception in thread "main" java.lang.ExceptionInInitializerError
> at org.apache.spark.SparkContext.<init>(SparkContext.scala:228)
> at com.barclays.SimpleApp$.main(SimpleApp.scala:11)
> at com.barclays.SimpleApp.main(SimpleApp.scala)
> Caused by: org.apache.spark.SparkException: Unable to load YARN support
> at org.apache.spark.deploy.SparkHadoopUtil$.liftedTree1$1(SparkHadoopUtil.scala:106)
> at org.apache.spark.deploy.SparkHadoopUtil$.<init>(SparkHadoopUtil.scala:101)
> at org.apache.spark.deploy.SparkHadoopUtil$.<clinit>(SparkHadoopUtil.scala)
> ... 3 more
> Caused by: java.lang.ClassNotFoundException: org.apache.spark.deploy.yarn.YarnSparkHadoopUtil
> at java.net.URLClassLoader$1.run(URLClassLoader.java:202)
> at java.security.AccessController.doPrivileged(Native Method)
> at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:306)
> at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:247)
> at java.lang.Class.forName0(Native Method)
> at java.lang.Class.forName(Class.java:169)
> at org.apache.spark.deploy.SparkHadoopUtil$.liftedTree1$1(SparkHadoopUtil.scala:102)
> ... 5 more
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org