[jira] [Created] (YARN-4727) Unable to override the $HADOOP_CONF_DIR env variable for container

["Terence Yim (JIRA)" <ji...@apache.org>]

Terence Yim created YARN-4727:
---------------------------------

             Summary: Unable to override the $HADOOP_CONF_DIR env variable for container
                 Key: YARN-4727
                 URL: https://issues.apache.org/jira/browse/YARN-4727
             Project: Hadoop YARN
          Issue Type: Bug
          Components: nodemanager
    Affects Versions: 2.6.4, 2.7.2, 2.5.2, 2.4.1
            Reporter: Terence Yim


Given the default config of "yarn.nodemanager.env-whitelist", application should be able to set the env variable $HADOOP_CONF_DIR to value other than the one in the NodeManager system environment. However, I believe due to a bug in the {{org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch}} class, it is not possible so.

>From the {{sanitizeEnv()}} method in the ContainerLaunch class (https://github.com/apache/hadoop/blob/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/launcher/ContainerLaunch.java#L991)

{noformat}
String[] whitelist = conf.get(YarnConfiguration.NM_ENV_WHITELIST, YarnConfiguration.DEFAULT_NM_ENV_WHITELIST).split(",");
    
for(String whitelistEnvVariable : whitelist) {
  putEnvIfAbsent(environment, whitelistEnvVariable.trim());
}

...

private static void putEnvIfAbsent(
    Map<String, String> environment, String variable) {
  if (environment.get(variable) == null) {
    putEnvIfNotNull(environment, variable, System.getenv(variable));
  }
}
{noformat}
The {{putEnvIfAbsent}} call simply use the {{whitelistEnvVariable}} to call {{System.getenv(whitelistEnvVariable)}} and set it to the environment map. It should be using the one from the {{launchContext}} instead.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)