You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@spark.apache.org by "Andrew Or (JIRA)" <ji...@apache.org> on 2014/05/11 00:08:33 UTC
[jira] [Created] (SPARK-1755) Spark-submit --name does not resolve
to application name on YARN
Andrew Or created SPARK-1755:
--------------------------------
Summary: Spark-submit --name does not resolve to application name on YARN
Key: SPARK-1755
URL: https://issues.apache.org/jira/browse/SPARK-1755
Project: Spark
Issue Type: Bug
Affects Versions: 1.0.0
Reporter: Andrew Or
Fix For: 1.0.1
In YARN client mode, --name is ignored because the deploy mode is client, and the name is for some reason a cluster config. (See https://github.com/apache/spark/blob/master/core/src/main/scala/org/apache/spark/deploy/SparkSubmit.scala#L170)
In YARN cluster mode, --name is passed to the org.apache.spark.deploy.yarn.Client as a command line argument. The Client class, however, uses this name only as the app name for the RM, but not for Spark. In other words, when SparkConf attempts to load default configs, application name is not set.
In both cases, passing --name to SparkSubmit does not actually cause Spark to adopt it as its application name, despite what the usage promises.
--
This message was sent by Atlassian JIRA
(v6.2#6252)