You are viewing a plain text version of this content. The canonical link for it is here.
Posted to mapreduce-user@hadoop.apache.org by Henning Blohm <he...@zfabrik.de> on 2016/04/21 21:15:02 UTC
Best way to pass configuration properties to MRv2 jobs
Hi,
in our Hadoop 2.6.0 cluster, we need to pass some properties to all
Hadoop processes so they can be referenced using ${...} syntax in
configuration files. This works reasonably well using
HADOOP_NAMENODE_OPTS and the like.
For Map/Reduce jobs however, we need to speficy not only
mapred.child.java.opts
to pass system properties, in addition we need to set
yarn.app.mapreduce.am.command-opts
for anything that is referenced in Hadoop configuration files.
In the end however almost all the properties passed are available as
environment variables as well.
Hence my question:
* Is it possible to use reference environment variables in configuration
files directly?
* Does anybody know of a simpler way to make sure some system properties
are _always_ set for all Yarn processes?
Thanks,
Henning
---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscribe@hadoop.apache.org
For additional commands, e-mail: user-help@hadoop.apache.org