[jira] [Commented] (SPARK-13063) Make the SPARK YARN STAGING DIR as configurable

["Thomas Graves (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/SPARK-13063?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15214419#comment-15214419 ] 

Thomas Graves commented on SPARK-13063:
---------------------------------------

it seems harmless to allow this to be configurable if people have usecases for it.  Lots of different hadoop configurations   Just change it to allow the /user part to be configurable and document the config in the YARN docs.

> Make the SPARK YARN STAGING DIR as configurable
> -----------------------------------------------
>
>                 Key: SPARK-13063
>                 URL: https://issues.apache.org/jira/browse/SPARK-13063
>             Project: Spark
>          Issue Type: Improvement
>          Components: YARN
>            Reporter: Devaraj K
>            Priority: Minor
>
> SPARK YARN STAGING DIR is based on the file system home directory. If the user wants to change this staging directory due to the same used by any other applications, there is no provision for the user to specify a different directory for staging dir.
> {code:xml}
>  val stagingDirPath = new Path(remoteFs.getHomeDirectory, stagingDir)
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org