[jira] [Updated] (YARN-7226) Whitelisted variables do not support delayed variable expansion

["Jason Lowe (JIRA)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/YARN-7226?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Jason Lowe updated YARN-7226:
-----------------------------
    Attachment: YARN-7226.001.patch

Attaching a patch that takes the approach described above.  ContainerLaunch#sanitizeEnv is already performing the above two steps, so all we need to do is remove the whitelist variable codepaths from ContainerExecutor#writeLaunchEnv so these variables can come through like normal variables when they are set by the container.


> Whitelisted variables do not support delayed variable expansion
> ---------------------------------------------------------------
>
>                 Key: YARN-7226
>                 URL: https://issues.apache.org/jira/browse/YARN-7226
>             Project: Hadoop YARN
>          Issue Type: Bug
>          Components: nodemanager
>    Affects Versions: 2.9.0, 2.8.1, 3.0.0-alpha4
>            Reporter: Jason Lowe
>            Assignee: Jason Lowe
>         Attachments: YARN-7226.001.patch
>
>
> The nodemanager supports a configurable list of environment variables, via yarn.nodemanager.env-whitelist, that will be propagated to the container's environment unless those variables were specified in the container launch context.  Unfortunately the handling of these whitelisted variables prevents using delayed variable expansion.  For example, if a user shipped their own version of hadoop with their job via the distributed cache and specified:
> {noformat}
> HADOOP_COMMON_HOME={{PWD}}/my-private-hadoop/
> {noformat}
>  as part of their job, the variable will be set as the *literal* string:
> {noformat}
> $PWD/my-private-hadoop/
> {noformat}
> rather than having $PWD expand to the container's current directory as it does for any other, non-whitelisted variable being set to the same value.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-issues-help@hadoop.apache.org