You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@flink.apache.org by "Peter Vary (Jira)" <ji...@apache.org> on 2022/10/25 08:22:00 UTC
[jira] [Created] (FLINK-29754) HadoopConfigLoader should consider Hadoop configuration files
Peter Vary created FLINK-29754:
----------------------------------
Summary: HadoopConfigLoader should consider Hadoop configuration files
Key: FLINK-29754
URL: https://issues.apache.org/jira/browse/FLINK-29754
Project: Flink
Issue Type: Bug
Components: FileSystems
Reporter: Peter Vary
Currently [HadoopConfigLoader|https://github.com/apache/flink/blob/master/flink-filesystems/flink-hadoop-fs/src/main/java/org/apache/flink/runtime/util/HadoopConfigLoader.java] considers Hadoop configurations on the classpath, but does not consider Hadoop configuration files which are set in another way.
So if the Hadoop configuration is set through the {{HADOOP_CONF_DIR}} environment variable, then the configuration loaded by the HadoopConfigLoader will not contain the values set there.
This can cause unexpected behaviour when setting checkpoint / savepoint dirs on S3, and the specific S3 configurations are set in the Hadoop configuration files
--
This message was sent by Atlassian Jira
(v8.20.10#820010)