You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@spark.apache.org by "Jeff Zhang (JIRA)" <ji...@apache.org> on 2016/03/17 10:29:33 UTC
[jira] [Updated] (SPARK-13955) Spark in yarn mode fails
[ https://issues.apache.org/jira/browse/SPARK-13955?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jeff Zhang updated SPARK-13955:
-------------------------------
Description:
I ran spark-shell in yarn client, but from the logs seems the spark assembly jar is not uploaded to HDFS. This may be known issue in the process of SPARK-11157, create this ticket to track this issue. [~vanzin]
{noformat}
16/03/17 11:58:59 WARN Client: Neither spark.yarn.jars nor spark.yarn.archive is set, falling back to uploading libraries under SPARK_HOME.
16/03/17 11:58:59 INFO Client: Uploading resource file:/Users/jzhang/github/spark/lib/apache-rat-0.10.jar -> hdfs://localhost:9000/user/jzhang/.sparkStaging/application_1458187008455_0004/apache-rat-0.10.jar
16/03/17 11:58:59 INFO Client: Uploading resource file:/Users/jzhang/github/spark/lib/apache-rat-0.11.jar -> hdfs://localhost:9000/user/jzhang/.sparkStaging/application_1458187008455_0004/apache-rat-0.11.jar
16/03/17 11:59:00 INFO Client: Uploading resource file:/private/var/folders/dp/hmchg5dd3vbcvds26q91spdw0000gp/T/spark-36cacbad-ca5b-482b-8ca8-607499acaaba/__spark_conf__4427292248554277597.zip -> hdfs://localhost:9000/user/jzhang/.sparkStaging/application_1458187008455_0004/__spark_conf__4427292248554277597.zip
{noformat}
message in AM container
{noformat}
Error: Could not find or load main class org.apache.spark.deploy.yarn.ExecutorLauncher
{noformat}
was:
Seems the spark assembly jar is not uploaded to AM. This may be known issue in the process of SPARK-11157, create this ticket to track this issue. [~vanzin]
{noformat}
16/03/17 11:58:59 WARN Client: Neither spark.yarn.jars nor spark.yarn.archive is set, falling back to uploading libraries under SPARK_HOME.
16/03/17 11:58:59 INFO Client: Uploading resource file:/Users/jzhang/github/spark/lib/apache-rat-0.10.jar -> hdfs://localhost:9000/user/jzhang/.sparkStaging/application_1458187008455_0004/apache-rat-0.10.jar
16/03/17 11:58:59 INFO Client: Uploading resource file:/Users/jzhang/github/spark/lib/apache-rat-0.11.jar -> hdfs://localhost:9000/user/jzhang/.sparkStaging/application_1458187008455_0004/apache-rat-0.11.jar
16/03/17 11:59:00 INFO Client: Uploading resource file:/private/var/folders/dp/hmchg5dd3vbcvds26q91spdw0000gp/T/spark-36cacbad-ca5b-482b-8ca8-607499acaaba/__spark_conf__4427292248554277597.zip -> hdfs://localhost:9000/user/jzhang/.sparkStaging/application_1458187008455_0004/__spark_conf__4427292248554277597.zip
{noformat}
message in AM container
{noformat}
Error: Could not find or load main class org.apache.spark.deploy.yarn.ExecutorLauncher
{noformat}
> Spark in yarn mode fails
> ------------------------
>
> Key: SPARK-13955
> URL: https://issues.apache.org/jira/browse/SPARK-13955
> Project: Spark
> Issue Type: Bug
> Affects Versions: 2.0.0
> Reporter: Jeff Zhang
>
> I ran spark-shell in yarn client, but from the logs seems the spark assembly jar is not uploaded to HDFS. This may be known issue in the process of SPARK-11157, create this ticket to track this issue. [~vanzin]
> {noformat}
> 16/03/17 11:58:59 WARN Client: Neither spark.yarn.jars nor spark.yarn.archive is set, falling back to uploading libraries under SPARK_HOME.
> 16/03/17 11:58:59 INFO Client: Uploading resource file:/Users/jzhang/github/spark/lib/apache-rat-0.10.jar -> hdfs://localhost:9000/user/jzhang/.sparkStaging/application_1458187008455_0004/apache-rat-0.10.jar
> 16/03/17 11:58:59 INFO Client: Uploading resource file:/Users/jzhang/github/spark/lib/apache-rat-0.11.jar -> hdfs://localhost:9000/user/jzhang/.sparkStaging/application_1458187008455_0004/apache-rat-0.11.jar
> 16/03/17 11:59:00 INFO Client: Uploading resource file:/private/var/folders/dp/hmchg5dd3vbcvds26q91spdw0000gp/T/spark-36cacbad-ca5b-482b-8ca8-607499acaaba/__spark_conf__4427292248554277597.zip -> hdfs://localhost:9000/user/jzhang/.sparkStaging/application_1458187008455_0004/__spark_conf__4427292248554277597.zip
> {noformat}
> message in AM container
> {noformat}
> Error: Could not find or load main class org.apache.spark.deploy.yarn.ExecutorLauncher
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org