[JIRA] (COLL-117) seeing warnings "attempt to override final parameter: dfs.data.dir" in tasktracker logs

[Bikash Singhal <bi...@cybage.com>]

Hi Hadoopers,

I have received WARN in the hadoop cluster. Has anybody seen this . Any solution?


> 2010-06-06 01:45:04,079 WARN org.apache.hadoop.conf.Configuration: /var/lib/hadoop-0.20/cache/hadoop/mapred/local/taskTracker/jobcache/job_201006060025_0003/job.xml:a attempt to override final parameter: dfs.data.dir;  Ignoring.
> ....
> 2010-06-06 01:49:41,501 WARN org.apache.hadoop.conf.Configuration: /var/lib/hadoop-0.20/cache/hadoop/mapred/local/taskTracker/jobcache/job_201006060025_0004/job.xml:a attempt to override final parameter: dfs.data.dir;  Ignoring.


"Legal Disclaimer: This electronic message and all contents contain information from Cybage Software Private Limited which may be privileged, confidential, or otherwise protected from disclosure. The information is intended to be for the addressee(s) only. If you are not an addressee, any disclosure, copy, distribution, or use of the contents of this message is strictly prohibited. If you have received this electronic message in error please notify the sender by reply e-mail to and destroy the original message and all copies. Cybage has taken every reasonable precaution to minimize the risk of malicious content in the mail, but is not liable for any damage you may sustain as a result of any malicious content in this e-mail. You should carry out your own malicious content checks before opening the e-mail or attachment." 
www.cybage.com

Re: [JIRA] (COLL-117) seeing warnings "attempt to override final parameter: dfs.data.dir" in tasktracker logs

[Allen Wittenauer <aw...@linkedin.com>]

Again, removing a bunch of CC:'es.

On Jun 21, 2010, at 2:26 AM, Bikash Singhal wrote:

> 
> Hi Hadoopers,
> 
> I have received WARN in the hadoop cluster. Has anybody seen this . Any solution?
> 
> 
>> 2010-06-06 01:45:04,079 WARN org.apache.hadoop.conf.Configuration: /var/lib/hadoop-0.20/cache/hadoop/mapred/local/taskTracker/jobcache/job_201006060025_0003/job.xml:a attempt to override final parameter: dfs.data.dir;  Ignoring.
>> ....
>> 2010-06-06 01:49:41,501 WARN org.apache.hadoop.conf.Configuration: /var/lib/hadoop-0.20/cache/hadoop/mapred/local/taskTracker/jobcache/job_201006060025_0004/job.xml:a attempt to override final parameter: dfs.data.dir;  Ignoring.


It means your client has 'dfs.data.dir' in its hadoop config.  On the server side, it is marked as final, which means the client can't override the value.  This message is completely harmless for this particular parameter.

Re: [JIRA] (COLL-117) seeing warnings "attempt to override final parameter: dfs.data.dir" in tasktracker logs

[Edward Capriolo <ed...@gmail.com>]

On Mon, Jun 21, 2010 at 5:26 AM, Bikash Singhal <bi...@cybage.com> wrote:

>
> Hi Hadoopers,
>
> I have received WARN in the hadoop cluster. Has anybody seen this . Any
> solution?
>
>
> > 2010-06-06 01:45:04,079 WARN org.apache.hadoop.conf.Configuration:
> /var/lib/hadoop-0.20/cache/hadoop/mapred/local/taskTracker/jobcache/job_201006060025_0003/job.xml:a
> attempt to override final parameter: dfs.data.dir;  Ignoring.
> > ....
> > 2010-06-06 01:49:41,501 WARN org.apache.hadoop.conf.Configuration:
> /var/lib/hadoop-0.20/cache/hadoop/mapred/local/taskTracker/jobcache/job_201006060025_0004/job.xml:a
> attempt to override final parameter: dfs.data.dir;  Ignoring.
>
>
> "Legal Disclaimer: This electronic message and all contents contain
> information from Cybage Software Private Limited which may be privileged,
> confidential, or otherwise protected from disclosure. The information is
> intended to be for the addressee(s) only. If you are not an addressee, any
> disclosure, copy, distribution, or use of the contents of this message is
> strictly prohibited. If you have received this electronic message in error
> please notify the sender by reply e-mail to and destroy the original message
> and all copies. Cybage has taken every reasonable precaution to minimize the
> risk of malicious content in the mail, but is not liable for any damage you
> may sustain as a result of any malicious content in this e-mail. You should
> carry out your own malicious content checks before opening the e-mail or
> attachment."
> www.cybage.com
>
>
This is a WARNING. It happens when you launch jobs using the same
configuration directory as you DataNodes, TaskTrackers. What it means is
that your jobs was launched and the JobConf which read in your configuration
has a setting for dfs.data.dir. Firstly dfs.data.dir can not be changed on a
job per job basis as it is read once on start up. If it could be changed on
a job by job basis, your initial configuration marks it as final, even if
you try to set to the same value as you might be doing you will still see
this warning.

Also please do not use a "shotgun" attempt to find the proper mailing list.
Each list has its own purpose.

Regards,
Edward

Re: [JIRA] (COLL-117) seeing warnings "attempt to override final parameter: dfs.data.dir" in tasktracker logs

[Allen Wittenauer <aw...@linkedin.com>]

Again, removing a bunch of CC:'es.

On Jun 21, 2010, at 2:26 AM, Bikash Singhal wrote:

> 
> Hi Hadoopers,
> 
> I have received WARN in the hadoop cluster. Has anybody seen this . Any solution?
> 
> 
>> 2010-06-06 01:45:04,079 WARN org.apache.hadoop.conf.Configuration: /var/lib/hadoop-0.20/cache/hadoop/mapred/local/taskTracker/jobcache/job_201006060025_0003/job.xml:a attempt to override final parameter: dfs.data.dir;  Ignoring.
>> ....
>> 2010-06-06 01:49:41,501 WARN org.apache.hadoop.conf.Configuration: /var/lib/hadoop-0.20/cache/hadoop/mapred/local/taskTracker/jobcache/job_201006060025_0004/job.xml:a attempt to override final parameter: dfs.data.dir;  Ignoring.


It means your client has 'dfs.data.dir' in its hadoop config.  On the server side, it is marked as final, which means the client can't override the value.  This message is completely harmless for this particular parameter.