[jira] [Resolved] (SPARK-24053) Support add subdirectory named as user name on staging directory

["Marcelo Vanzin (JIRA)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/SPARK-24053?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Marcelo Vanzin resolved SPARK-24053.
------------------------------------
    Resolution: Not A Bug

Copy & paste of PR comments:

{quote}
> When we have multiple users on the same cluster

Then the staging directory would be created under the respective user already (it's created under the user's home directory). I have no idea why you need this.

Moreover, you can set spark.yarn.stagingDir to whatever you want, and may even reference env variables or system properties. e.g.

spark.yarn.stagingDir=/tmp/${system:user.name}

And the staging directory will be created under that location. So really you don't need to write any code for this, even if you have a weird deployment where users don't have their own home directory.
{quote}

>  Support add subdirectory named as user name on staging directory
> -----------------------------------------------------------------
>
>                 Key: SPARK-24053
>                 URL: https://issues.apache.org/jira/browse/SPARK-24053
>             Project: Spark
>          Issue Type: Improvement
>          Components: Spark Core
>    Affects Versions: 2.3.0
>            Reporter: zhoukang
>            Priority: Minor
>
> When we have multiple users on the same cluster,we can support add subdirectory which named as login user



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org