[jira] [Commented] (SPARK-13063) Make the SPARK YARN STAGING DIR as configurable

["Devaraj K (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/SPARK-13063?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15121239#comment-15121239 ] 

Devaraj K commented on SPARK-13063:
-----------------------------------

If this default location is already used by another apps(like Mapreduce), users may need to change it to some other directory. Presently we don’t have provision to change the value, we can make this as configurable and have the current staging dir as default value.

> Make the SPARK YARN STAGING DIR as configurable
> -----------------------------------------------
>
>                 Key: SPARK-13063
>                 URL: https://issues.apache.org/jira/browse/SPARK-13063
>             Project: Spark
>          Issue Type: Improvement
>          Components: YARN
>            Reporter: Devaraj K
>            Priority: Minor
>
> SPARK YARN STAGING DIR is based on the file system home directory. If the user wants to change this staging directory due to the same used by any other applications, there is no provision for the user to specify a different directory for staging dir.
> {code:xml}
>  val stagingDirPath = new Path(remoteFs.getHomeDirectory, stagingDir)
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org