[jira] [Commented] (SPARK-1755) Spark-submit --name does not resolve to application name on YARN

["Andrew Or (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/SPARK-1755?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13993196#comment-13993196 ] 

Andrew Or commented on SPARK-1755:
----------------------------------

You are correct. However, it also doesn't work for yarn-cluster.

> Spark-submit --name does not resolve to application name on YARN
> ----------------------------------------------------------------
>
>                 Key: SPARK-1755
>                 URL: https://issues.apache.org/jira/browse/SPARK-1755
>             Project: Spark
>          Issue Type: Bug
>    Affects Versions: 0.9.1
>            Reporter: Andrew Or
>            Assignee: Andrew Or
>            Priority: Blocker
>             Fix For: 1.0.0
>
>
> In YARN client mode, --name is ignored because the deploy mode is client, and the name is for some reason a [cluster config|https://github.com/apache/spark/blob/master/core/src/main/scala/org/apache/spark/deploy/SparkSubmit.scala#L170)].
> In YARN cluster mode, --name is passed to the org.apache.spark.deploy.yarn.Client as a command line argument. The Client class, however, uses this name only as the [app name for the RM|https://github.com/apache/spark/blob/master/yarn/stable/src/main/scala/org/apache/spark/deploy/yarn/Client.scala#L80], but not for Spark. In other words, when SparkConf attempts to load default configs, application name is not set.
> In both cases, passing --name to SparkSubmit does not actually cause Spark to adopt it as its application name, despite what the usage promises.



--
This message was sent by Atlassian JIRA
(v6.2#6252)