[jira] [Commented] (HADOOP-8115) configuration entry in core-site.xml gets silently ignored

["Robert Joseph Evans (Commented) (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/HADOOP-8115?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13217270#comment-13217270 ] 

Robert Joseph Evans commented on HADOOP-8115:
---------------------------------------------

Unfortunately the order of loading is not always consistent.  Just look at the bug I just did HDFS-3012.  Because we us static blocks to load the resources those static blocks are executed when the class file that contains them is loaded.  The order that class files are loaded is based off of when java decided to load the file, and because java currently does lazy loading it tends to only load them when code inside that class file is actually going to be executed for the first time.  Because core-default.xml and core-site.xml are part of the Configuration class they are always going to be the first to load.  But there is no guarantee the order that others will come in, or even if they will be loaded, which is the problem that showed up in HDFS-3012.

Because the ordering cannot really be controlled when using static blocks to load the configuration the only way to really do (1) is to give each file some sort of a globally universal priority.  Who wins when a configuration value is in hdfs-site.xml and also in mapred-site.xml?  We have that problem now, but because we tend to support more of (2) by convention instead of having the enforcement it is not really as much of a concern as if we officially support (1).  I personally would prefer to see option (2) as it is closer to what we do now, but I am not sure what the proper way is to implement it.
                
> configuration entry in core-site.xml gets silently ignored
> ----------------------------------------------------------
>
>                 Key: HADOOP-8115
>                 URL: https://issues.apache.org/jira/browse/HADOOP-8115
>             Project: Hadoop Common
>          Issue Type: Bug
>         Environment: v 0.20.203.0
> Standard tar release (i.e. not Cloudera or anything)
> Ubuntu
>            Reporter: Marc Harris
>
> The order of loading configuration files (and thus the order of priority from least to most) seems to be as follows:
> core-default.xml, core-site.xml, hdfs-default.xml, hdfs-site.xml.
> This means that a configuration parameter that is set in hdfs-default.xml will override that value set in core-site.xml. 
> Either
> (1) Parameters should be able to go in any site.xml file, and override any default.xml, even if they don't "match", or
> (2) Putting a parameter in the "wrong" site.xml file should be considered an error, and result in at the very least a warning.
> What in fact happens is that the parameter is silently ignored, which is the worst combination.
> I my opinion, it is counter-intuitive that a value in a site.xml file should be overridden by a value in a default.xml file, so I would choose option (1).
> The particular example here was dfs.http.address, by the way.
> I marked this as major rather than minor since it was not at all obvious what the problem (and therefore the workaround was) and eventually required attaching to a running production service with a debugger to find out why the parameter I was setting was being ignored.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira