You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-issues@hadoop.apache.org by "Masatake Iwasaki (JIRA)" <ji...@apache.org> on 2014/11/03 21:15:34 UTC
[jira] [Commented] (HADOOP-7056) NN unintentionally reads
mapred-site.xml
[ https://issues.apache.org/jira/browse/HADOOP-7056?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14195018#comment-14195018 ]
Masatake Iwasaki commented on HADOOP-7056:
------------------------------------------
just for noting. o.a.h.mapreduce.util.ConfigUtil#loadResources which is called in JobConf loads not only mapred-\{default,site\}.xml but also yarn-\{default,site\}.xml.
{code}
public static void loadResources() {
addDeprecatedKeys();
Configuration.addDefaultResource("mapred-default.xml");
Configuration.addDefaultResource("mapred-site.xml");
Configuration.addDefaultResource("yarn-default.xml");
Configuration.addDefaultResource("yarn-site.xml");
}
{code}
> NN unintentionally reads mapred-site.xml
> ----------------------------------------
>
> Key: HADOOP-7056
> URL: https://issues.apache.org/jira/browse/HADOOP-7056
> Project: Hadoop Common
> Issue Type: Bug
> Affects Versions: 0.22.0
> Reporter: Aaron T. Myers
> Priority: Critical
>
> It seems that {{mapred-site.xml}} is now unintentionally being read by the NN, and probably all the daemons.
> This is because the {{Groups}} constructor calls {{ReflectionUtils.newInstance(...)}}, which in turn calls {{ReflectionUtils.setConf(...)}}, which calls {{ReflectionUtils.setJobConf(...)}}, which creates an instance of {{o.a.h.mapred.JobConf}}, which calls {{o.a.h.mapreduce.util.ConfigUtil.loadResources(...)}}, which reads in {{mapred-site.xml}} and {{mapred-default.xml}}. Phew!
> I suspect the appropriate solution is for {{ReflectionUtils.setConf(...)}} to be changed to not call (the mapred-specific) {{ReflectionUtils.setJobConf(...)}}, but there's a comment in there which says it's being left in place for backward compatibility. Can somebody with more knowledge of that situation please comment?
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)