[jira] [Commented] (YARN-8071) Provide Spark-like API for setting Environment Variables to enable vars with commas

["Jim Brennan (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/YARN-8071?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16412063#comment-16412063 ] 

Jim Brennan commented on YARN-8071:
-----------------------------------

Alternate solutions to this problem (with patches) are provided in the following Jiras:

[YARN-8029] - addresses the specific issue for docker yarn mount paths by changing the delimiter for those environment variables from comma to semicolon.

[YARN-6830] - Modifies the regex used by Apps.setEnvFromInputString() to allow commas in a quoted string.  This requires that users of those variables remove the surrounding strings before using them.

See discussion in [YARN-6830] about the alternatives.

> Provide Spark-like API for setting Environment Variables to enable vars with commas
> -----------------------------------------------------------------------------------
>
>                 Key: YARN-8071
>                 URL: https://issues.apache.org/jira/browse/YARN-8071
>             Project: Hadoop YARN
>          Issue Type: Bug
>          Components: yarn
>    Affects Versions: 3.0.0
>            Reporter: Jim Brennan
>            Assignee: Jim Brennan
>            Priority: Major
>
> YARN-6830 describes a problem where environment variables that contain commas cannot be specified via {{-Dmapreduce.map.env}}.
> For example:
> {{-Dmapreduce.map.env="MODE=bar,IMAGE_NAME=foo,MOUNTS=/tmp/foo,/tmp/bar"}}
> will set {{MOUNTS}} to {{/tmp/foo}}
> In that Jira, [~aw] suggested that we change the API to provide a way to specify environment variables individually, the same way that Spark does.
> {quote}Rather than fight with a regex why not redefine the API instead?
>  
> -Dmapreduce.map.env.MODE=bar
>  -Dmapreduce.map.env.IMAGE_NAME=foo
>  -Dmapreduce.map.env.MOUNTS=/tmp/foo,/tmp/bar
> ...
> e.g, mapreduce.map.env.[foo]=bar gets turned into foo=bar
> This greatly simplifies the input validation needed and makes it clear what is actually being defined.
> {quote}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-issues-help@hadoop.apache.org