You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-user@hadoop.apache.org by Arun C Murthy <ar...@yahoo-inc.com> on 2007/09/15 06:23:35 UTC

IMPORTANT: Changes to hadoop's configuration

Please note: 

If, and only if, you're running Hadoop from straight from the svn *trunk*, as of Sept 12 Hadoop no longer uses the *mapred-default.xml* file.

Please move your mapred-default.xml configuration properties to your *hadoop-site.xml*. Hadoop's configuration also supports the notion of *final* parameters which, once specified, cannot be overridden by subsequently loaded configuration files. These changes will roll-out in the hadoop-0.15.0 release only.

Currently, the only documentation on how the new configuration works is at http://issues.apache.org/jira/browse/HADOOP-785. Wiki (or other) documentation, is forthcoming (http://issues.apache.org/jira/browse/HADOOP-1881).

thanks,
Arun
  

Re: IMPORTANT: Changes to hadoop's configuration

Posted by Doug Cutting <cu...@apache.org>.
Arun C Murthy wrote:
> The patch did update the javadoc in question. In fact, it ensured that 
> we updated all of hadoop's API javadocs.

This was also noted in the INCOMPATIBLE section of CHANGES.txt, so folks 
are forewarned.  Still, we do in general need better configuration 
documentation, and a warning when the now-ignored hadoop-default.xml is 
detected would also help.

Doug

Re: IMPORTANT: Changes to hadoop's configuration

Posted by Arun C Murthy <ar...@yahoo-inc.com>.
Eric Baldeschwieler wrote:
> Why did we accept a patch that fundamentally changes the user  
> experience without requiring that the patch included updated  
> documentation in forest of javadoc?
> 

The patch did update the javadoc in question. In fact, it ensured that 
we updated all of hadoop's API javadocs.

This mail was intended more as a heads-up for people who might not be 
aware of this incompatible change i.e. just using older configuration files.

There is also HADOOP-1881 to mainly add a new page to the hadoop website 
(via forrest) and to:
a) update the HowToConfigure wiki so as to not lead people astray.
b) further embellish the javadoc with more examples

> I propose that committers should reject feature patches that do not  
> included documentation.
> 
> Thoughts?
> 
> ---
> 
> On a related note, does the release check for mapred-default.xml and  
> warn if it exists?  Seems like a good idea.
> 

Good idea. I'll file a jira for this and propose we keep this warning 
for atleast 0.15.0.

Arun

> On Sep 14, 2007, at 9:23 PM, Arun C Murthy wrote:
> 
>> Please note:
>>
>> If, and only if, you're running Hadoop from straight from the svn  
>> *trunk*, as of Sept 12 Hadoop no longer uses the *mapred- default.xml* 
>> file.
>>
>> Please move your mapred-default.xml configuration properties to  your 
>> *hadoop-site.xml*. Hadoop's configuration also supports the  notion of 
>> *final* parameters which, once specified, cannot be  overridden by 
>> subsequently loaded configuration files. These  changes will roll-out 
>> in the hadoop-0.15.0 release only.
>>
>> Currently, the only documentation on how the new configuration  works 
>> is at http://issues.apache.org/jira/browse/HADOOP-785. Wiki  (or 
>> other) documentation, is forthcoming (http://issues.apache.org/ 
>> jira/browse/HADOOP-1881).
>>
>> thanks,
>> Arun
>>
>>
> 


Re: IMPORTANT: Changes to hadoop's configuration

Posted by Eric Baldeschwieler <er...@yahoo-inc.com>.
Why did we accept a patch that fundamentally changes the user  
experience without requiring that the patch included updated  
documentation in forest of javadoc?

I propose that committers should reject feature patches that do not  
included documentation.

Thoughts?

---

On a related note, does the release check for mapred-default.xml and  
warn if it exists?  Seems like a good idea.

On Sep 14, 2007, at 9:23 PM, Arun C Murthy wrote:

> Please note:
>
> If, and only if, you're running Hadoop from straight from the svn  
> *trunk*, as of Sept 12 Hadoop no longer uses the *mapred- 
> default.xml* file.
>
> Please move your mapred-default.xml configuration properties to  
> your *hadoop-site.xml*. Hadoop's configuration also supports the  
> notion of *final* parameters which, once specified, cannot be  
> overridden by subsequently loaded configuration files. These  
> changes will roll-out in the hadoop-0.15.0 release only.
>
> Currently, the only documentation on how the new configuration  
> works is at http://issues.apache.org/jira/browse/HADOOP-785. Wiki  
> (or other) documentation, is forthcoming (http://issues.apache.org/ 
> jira/browse/HADOOP-1881).
>
> thanks,
> Arun
>
>