[jira] [Comment Edited] (TEZ-4254) Don't unset the tez config if both mr and tez config have same value.

["Attila Magyar (Jira)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/TEZ-4254?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17258763#comment-17258763 ] 

Attila Magyar edited comment on TEZ-4254 at 1/5/21, 9:00 AM:
-------------------------------------------------------------

Hi [~Rajkumar Singh],

I'm trying to understand the reason behind the patch by an example. 

Let's say we have these 2 configs (before the patch)

 
{code:java}
mapreduce.shuffle.ssl.enabled=true
tez.runtime.shuffle.ssl.enable=true 

{code}
 

First we unset the deprecated _mapreduce.shuffle.ssl.enabled_, and since we prefer TEZ we don't overwrite to _tez.runtime.shuffle.ssl.enable_ with _mapreduce.shuffle.ssl.enabled_.

 

But after the patch, we still unset _mapreduce.shuffle.ssl.enabled_ then we overwrite _tez.runtime.shuffle.ssl.enable_ with _mapreduce.shuffle.ssl.enabled_ (true to true). How does this make any difference? Did I miss something?

 


was (Author: amagyar):
Hi [~Rajkumar Singh],

I'm trying to understand the reason behind the patch by an example. 

Let's say we have these 2 configs (before the patch)

 
{code:java}
mapreduce.shuffle.ssl.enabled=true
tez.runtime.shuffle.ssl.enable=true 

{code}
 

First we unset the deprecated _mapreduce.shuffle.ssl.enabled_, and since we prefer TEZ we don't overwrite to _tez.runtime.shuffle.ssl.enable_ with _mapreduce.shuffle.ssl.enabled_.

 

But after the patch, we still unset _mapreduce.shuffle.ssl.enabled_ then we overwrite _tez.runtime.shuffle.ssl.enable_ with _tez.runtime.shuffle.ssl.enable_ (true to true). How does this make any difference? Did I miss something?

 

> Don't unset the tez config if both mr and tez config have same value.
> ---------------------------------------------------------------------
>
>                 Key: TEZ-4254
>                 URL: https://issues.apache.org/jira/browse/TEZ-4254
>             Project: Apache Tez
>          Issue Type: Bug
>    Affects Versions: 0.9.1
>            Reporter: Rajkumar Singh
>            Priority: Major
>         Attachments: TEZ-4254.patch
>
>
> while translated mr config to the tez config we unset config here.
> https://github.com/apache/tez/blob/master/tez-mapreduce/src/main/java/org/apache/tez/mapreduce/hadoop/MRHelpers.java#L187
> There is no need to unset the config if both mr and tez have same value.
> we observed if equivalent of tez config for mapreduce.shuffle.ssl.enabled (tez.runtime.shuffle.ssl.enable) set to true, we unset the config, this propagate config tez.runtime.shuffle.ssl.enable={mapreduce.shuffle.ssl.enabled} at runtime and variable substitution does not work correctly in the absence of mrconfig runtime.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)