[jira] [Closed] (FLINK-15097) flink can not use user specified hdfs conf when submitting app in client node

["Zili Chen (Jira)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/FLINK-15097?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Zili Chen closed FLINK-15097.
-----------------------------
    Resolution: Duplicate

> flink can not use user specified hdfs conf when submitting app in client node
> -----------------------------------------------------------------------------
>
>                 Key: FLINK-15097
>                 URL: https://issues.apache.org/jira/browse/FLINK-15097
>             Project: Flink
>          Issue Type: Bug
>          Components: Client / Job Submission
>    Affects Versions: 1.9.1
>            Reporter: qian wang
>            Priority: Major
>         Attachments: 0001-adjust-read-hdfs-conf-order.patch
>
>
> now if cluster node had set env HADOOP_CONF_DIR，flink would force use the hdfs-site.xml in the corresponding dir, then user who submitted app in the client node couldn't use custom specified hdfs-site.xml/hdfs-default through setting fs.hdfs.hdfssite or fs.hdfs.hdfsdefault so as to set custom blocksize or replication num. For example Using yarnship to upload my hdfs conf dir and set fs.hdfs.hdfssite direct to \{conf dir}/hdfs-site.xml is useless
> Deep in code it is due to the order of choosing conf in HadoopUtils.java，the conf in HADOOP_CONF_DIR will override user's uploaded conf, i think the way is  not sensible, so i reverse the order which flink read hdfs conf in order to let user custom conf uploaded override HADOOP_CONF_DIR



--
This message was sent by Atlassian Jira
(v8.3.4#803005)