You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@spark.apache.org by "Andrew Or (JIRA)" <ji...@apache.org> on 2015/07/02 02:31:04 UTC
[jira] [Commented] (SPARK-5222) YARN client and cluster modes have
different app name behaviors
[ https://issues.apache.org/jira/browse/SPARK-5222?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14611241#comment-14611241 ]
Andrew Or commented on SPARK-5222:
----------------------------------
I'm inclined to close this as a "Won't Fix" because the user can set the app name in his/her main method, in which case there's nothing we can do because cluster mode will not have access to the name until later. I'll let this sit for a few more days to see if others have any objections to this resolution.
> YARN client and cluster modes have different app name behaviors
> ---------------------------------------------------------------
>
> Key: SPARK-5222
> URL: https://issues.apache.org/jira/browse/SPARK-5222
> Project: Spark
> Issue Type: Bug
> Components: YARN
> Affects Versions: 1.0.0
> Reporter: Andrew Or
> Assignee: Tao Wang
> Priority: Minor
>
> The behavior is summarized in a table produced by [~WangTaoTheTonic] here: https://github.com/apache/spark/pull/3557
> SPARK_YARN_APP_NAME is respected only in client mode but not in cluster mode. This results in the strange behavior where the app name changes if the user runs the same application but uses a different deploy mode from before. We should make sure the app name behavior is consistent across deploy modes regardless of what variable or config is set.
> Additionally, it should be noted that because "spark.app.name" is required of all applications, the setting of "SPARK_YARN_APP_NAME" will not take effect unless we handle it preemptively in Spark submit.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org