You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@flink.apache.org by "zhangzhiyuan (Jira)" <ji...@apache.org> on 2020/04/27 10:32:00 UTC
[jira] [Updated] (FLINK-17412) Make the FLINK_YARN_FILES value
configurable
[ https://issues.apache.org/jira/browse/FLINK-17412?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
zhangzhiyuan updated FLINK-17412:
---------------------------------
Description:
When the application is deployed to YARN, the file of the application will be uploaded to the directory returned by
{code:java}
AbstractYarnClusterDescriptor#getYarnFilesDir(ApplicationId) {code}
method. When the same user name is used to submit applications to different clusters, it may cause directory name conflicts(because different YARN clusters may generate the same applicationId). In addition, when the application is directly killed by the user, the corresponding directory will continue to exist and occupy disk space, and the judgment conditions for cleaning such invalid directories are complex. Therefore, it is hoped that a configuration option can be added to support each application to dynamically specify the directory where its files are located during deployment.
was:
When the application is deployed to YARN, the file of the application will be uploaded to the directory returned by
{code:java}AbstractYarnClusterDescriptor#getYarnFilesDir(ApplicationId) {code}
method. When the same user name is used to submit applications to different clusters, it may cause directory name conflicts. In addition, when the application is directly killed by the user, the corresponding directory will continue to exist and occupy disk space, and the judgment conditions for cleaning such invalid directories are complex. Therefore, it is hoped that a configuration option can be added to support each application to dynamically specify the directory where its files are located during deployment.
> Make the FLINK_YARN_FILES value configurable
> --------------------------------------------
>
> Key: FLINK-17412
> URL: https://issues.apache.org/jira/browse/FLINK-17412
> Project: Flink
> Issue Type: Wish
> Components: Deployment / YARN
> Affects Versions: 1.7.3, 1.8.0
> Reporter: zhangzhiyuan
> Priority: Major
>
> When the application is deployed to YARN, the file of the application will be uploaded to the directory returned by
> {code:java}
> AbstractYarnClusterDescriptor#getYarnFilesDir(ApplicationId) {code}
> method. When the same user name is used to submit applications to different clusters, it may cause directory name conflicts(because different YARN clusters may generate the same applicationId). In addition, when the application is directly killed by the user, the corresponding directory will continue to exist and occupy disk space, and the judgment conditions for cleaning such invalid directories are complex. Therefore, it is hoped that a configuration option can be added to support each application to dynamically specify the directory where its files are located during deployment.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)