You are viewing a plain text version of this content. The canonical link for it is here.

Posted to user@flink.apache.org by "Foster, Craig" <fo...@amazon.com> on 2017/03/23 16:43:50 UTC

Alternatives to FLINK_CONF_DIR, HADOOP_CONF_DIR, YARN_CONF_DIR

Can I set these in the configuration file? This would be ideal vs. environment variables for me but I’m not seeing it in the documentation.

Thanks,
Craig

Re: Alternatives to FLINK_CONF_DIR, HADOOP_CONF_DIR, YARN_CONF_DIR

Posted by "Foster, Craig" <fo...@amazon.com>.

Thanks! I looked in the config.sh file that sort of works with both the configuration file and these environment variables. After inspection, it doesn’t make sense to set FLINK_CONF_DIR in that config file since that location determines where we would look for that file.

However, I thought that I could add a patch to this shell script to at least allow someone to set the Hadoop and Yarn configuration directories in the config file.

From: "Bajaj, Abhinav" <ab...@here.com>
Date: Thursday, March 23, 2017 at 10:42 AM
To: "Foster, Craig" <fo...@amazon.com>
Cc: "user@flink.apache.org" <us...@flink.apache.org>
Subject: Re: Alternatives to FLINK_CONF_DIR, HADOOP_CONF_DIR, YARN_CONF_DIR

I think the FLINK_CONF_DIR points to the conf directory. This is the place where the Flink CLI looks for the flink-conf.yaml file.

I think there is an alternate option for HADOOP_CONF_DIR, YARN_CONF_DIR but I am not sure.
Check this https://ci.apache.org/projects/flink/flink-docs-release-1.3/setup/config.html#common-options
Config option - “fs.hdfs.hadoopconf:“

~ Abhi

From: "Foster, Craig" <fo...@amazon.com>
Reply-To: "user@flink.apache.org" <us...@flink.apache.org>
Date: Thursday, March 23, 2017 at 9:43 AM
To: "user@flink.apache.org" <us...@flink.apache.org>
Subject: Alternatives to FLINK_CONF_DIR, HADOOP_CONF_DIR, YARN_CONF_DIR

Can I set these in the configuration file? This would be ideal vs. environment variables for me but I’m not seeing it in the documentation.

Thanks,
Craig

Re: Alternatives to FLINK_CONF_DIR, HADOOP_CONF_DIR, YARN_CONF_DIR

Posted by "Bajaj, Abhinav" <ab...@here.com>.

I think the FLINK_CONF_DIR points to the conf directory. This is the place where the Flink CLI looks for the flink-conf.yaml file.

I think there is an alternate option for HADOOP_CONF_DIR, YARN_CONF_DIR but I am not sure.
Check this https://ci.apache.org/projects/flink/flink-docs-release-1.3/setup/config.html#common-options
Config option - “fs.hdfs.hadoopconf:“

~ Abhi


From: "Foster, Craig" <fo...@amazon.com>
Reply-To: "user@flink.apache.org" <us...@flink.apache.org>
Date: Thursday, March 23, 2017 at 9:43 AM
To: "user@flink.apache.org" <us...@flink.apache.org>
Subject: Alternatives to FLINK_CONF_DIR, HADOOP_CONF_DIR, YARN_CONF_DIR

Can I set these in the configuration file? This would be ideal vs. environment variables for me but I’m not seeing it in the documentation.

Thanks,
Craig