[jira] [Commented] (FLINK-5668) passing taskmanager configuration through taskManagerEnv instead of file

["Robert Metzger (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/FLINK-5668?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15887703#comment-15887703 ] 

Robert Metzger commented on FLINK-5668:
---------------------------------------

[~wheat9] I still did not understand how you can "inject" a custom path for some of the resources deployed by Flink on YARN.
All the paths are programatically generated and there are no configuration parameters for passing custom paths (correct me if I'm wrong).

Are you planning to basically fork Flink and create a custom YARN client / Application Master implementation that allows using custom paths?

> regarding this issue, do you have an idea why the current implementation writes the configuration into a file on default.FS?

I think we didn't have your use case in mind when implementing the code. We assumed that one file system will be used for distributing all required files. Also, this approach works nicely will all the Hadoop vendor's versions.

> What do you think if passing the configuration through the ``taskManagerEnv``?

I can only assess this approach once I've understood what you are trying to achieve.

> passing taskmanager configuration through taskManagerEnv instead of file
> ------------------------------------------------------------------------
>
>                 Key: FLINK-5668
>                 URL: https://issues.apache.org/jira/browse/FLINK-5668
>             Project: Flink
>          Issue Type: Improvement
>          Components: YARN
>            Reporter: Bill Liu
>   Original Estimate: 48h
>  Remaining Estimate: 48h
>
> When create a Flink cluster on Yarn,  JobManager depends on  HDFS to share  taskmanager-conf.yaml  with TaskManager.
> It's better to share the taskmanager-conf.yaml  on JobManager Web server instead of HDFS, which could reduce the HDFS dependency  at job startup.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)