[jira] [Commented] (FLINK-4873) Add config option to specify "home directory" for YARN client resource sharing

["Gyula Fora (Jira)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/FLINK-4873?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16926473#comment-16926473 ] 

Gyula Fora commented on FLINK-4873:
-----------------------------------

Hm, looks like I had the change commit on my master branch which I have overwritten recently... but we can definitely add this feature. Should be simple enough :)

> Add config option to specify "home directory" for YARN client resource sharing
> ------------------------------------------------------------------------------
>
>                 Key: FLINK-4873
>                 URL: https://issues.apache.org/jira/browse/FLINK-4873
>             Project: Flink
>          Issue Type: Improvement
>          Components: Deployment / YARN
>    Affects Versions: 1.1.3, 1.2.0
>            Reporter: Gyula Fora
>            Priority: Major
>              Labels: pull-request-available
>          Time Spent: 10m
>  Remaining Estimate: 0h
>
> The YARN client currently uses FileSystem.getHomeDirectory() to store the jar files that needed to be shared on the cluster. This pretty much forces users to run HDFS or something compatible with the Hadoop FS api on the cluster.
> In some production environments file systems (distributed or simply shared) are simply mounted under the same path and do not require the use of the hadoop api for convenience. If we want to run Flink on YARN in these cases we would need to be able to define the "home directory" where Flink should copy the files for sharing.
> It could be something like:
> yarn.resource.upload.dir in the flink-conf.yaml



--
This message was sent by Atlassian Jira
(v8.3.2#803003)