You are viewing a plain text version of this content. The canonical link for it is here.

Posted to issues@flink.apache.org by "Xintong Song (Jira)" <ji...@apache.org> on 2021/10/20 04:49:00 UTC

[jira] [Commented] (FLINK-23169) Support user-level app staging directory when yarn.staging-directory is specified

    [ https://issues.apache.org/jira/browse/FLINK-23169?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17430878#comment-17430878 ] 

Xintong Song commented on FLINK-23169:
--------------------------------------

hi [~hackergin],

I'm trying to understand:
# Why is it a problem for you to have staging files of all users in the same directory?
# Can this be workaround by configuring different staging directories for users?

> Support user-level app staging directory when yarn.staging-directory is specified
> ---------------------------------------------------------------------------------
>
>                 Key: FLINK-23169
>                 URL: https://issues.apache.org/jira/browse/FLINK-23169
>             Project: Flink
>          Issue Type: Improvement
>          Components: Deployment / YARN
>            Reporter: jinfeng
>            Priority: Major
>
> When yarn.staging-directory is specified,  different users will use the same directory as the staging directory.   It may not friendly for a job platform to submit job for different users.  I propose to use the user-level directory by default when yarn.staging-directory is specified.  We only need to make small changes  for `getStagingDir` function in 
> YarnClusterDescriptor 



--
This message was sent by Atlassian Jira
(v8.3.4#803005)