You are viewing a plain text version of this content. The canonical link for it is here.
Posted to hdfs-issues@hadoop.apache.org by "Rajit Saha (Created) (JIRA)" <ji...@apache.org> on 2011/10/07 23:28:29 UTC
[jira] [Created] (HDFS-2417) Warnings about attempt to override
final parameter while getting delegation token
Warnings about attempt to override final parameter while getting delegation token
---------------------------------------------------------------------------------
Key: HDFS-2417
URL: https://issues.apache.org/jira/browse/HDFS-2417
Project: Hadoop HDFS
Issue Type: Bug
Components: name-node
Affects Versions: 0.20.205.0
Reporter: Rajit Saha
I am seeing whenever I run any Mapreduce job and its trying to acquire delegation from NN, In JT log following warnings coming about "a attempt to override final parameter:"
The log snippet in JT log
2011-10-07 20:29:19,096 WARN org.apache.hadoop.conf.Configuration:
/tmp/mapred-local/jobTracker/job_201110072015_0005.xml:a attempt to override final parameter:
mapred.job.reuse.jvm.num.tasks; Ignoring.
2011-10-07 20:29:19,096 WARN org.apache.hadoop.conf.Configuration:
/tmp/mapred-local/jobTracker/job_201110072015_0005.xml:a attempt to override final parameter: mapred.system.dir;
Ignoring.
2011-10-07 20:29:19,096 WARN org.apache.hadoop.conf.Configuration:
/tmp/mapred-local/jobTracker/job_201110072015_0005.xml:a attempt to override final parameter:
hadoop.job.history.user.location; Ignoring.
2011-10-07 20:29:19,096 WARN org.apache.hadoop.conf.Configuration:
/tmp/mapred-local/jobTracker/job_201110072015_0005.xml:a attempt to override final parameter: mapred.local.dir;
Ignoring.
2011-10-07 20:29:19,096 WARN org.apache.hadoop.conf.Configuration:
/tmp/mapred-local/jobTracker/job_201110072015_0005.xml:a attempt to override final parameter: m
apred.job.tracker.http.address; Ignoring.
2011-10-07 20:29:19,096 WARN org.apache.hadoop.conf.Configuration:
/tmp/mapred-local/jobTracker/job_201110072015_0005.xml:a attempt to override final parameter: d
fs.data.dir; Ignoring.
2011-10-07 20:29:19,096 WARN org.apache.hadoop.conf.Configuration:
/tmp/mapred-local/jobTracker/job_201110072015_0005.xml:a attempt to override final parameter: d
fs.http.address; Ignoring.
2011-10-07 20:29:19,096 WARN org.apache.hadoop.conf.Configuration:
/tmp/mapred-local/jobTracker/job_201110072015_0005.xml:a attempt to override final parameter: m
apreduce.admin.map.child.java.opts; Ignoring.
2011-10-07 20:29:19,096 WARN org.apache.hadoop.conf.Configuration:
/tmp/mapred-local/jobTracker/job_201110072015_0005.xml:a attempt to override final parameter:
mapreduce.history.server.http.address; Ignoring.
2011-10-07 20:29:19,096 WARN org.apache.hadoop.conf.Configuration:
/tmp/mapred-local/jobTracker/job_201110072015_0005.xml:a attempt to override final parameter: m
apreduce.history.server.embedded; Ignoring.
2011-10-07 20:29:19,096 WARN org.apache.hadoop.conf.Configuration:
/tmp/mapred-local/jobTracker/job_201110072015_0005.xml:a attempt to override final parameter: m
apreduce.jobtracker.split.metainfo.maxsize; Ignoring.2011-10-07 20:29:19,096 WARN
org.apache.hadoop.conf.Configuration: /tmp/mapred-local/jobTracker/job_201110072015_0005.xml:a attempt to
override final parameter: m
apreduce.admin.reduce.child.java.opts; Ignoring.2011-10-07 20:29:19,096 WARN org.apache.hadoop.conf.Configuration:
/tmp/mapred-local/jobTracker/job_201110072015_0005.xml:a attempt to override final parameter: h
adoop.tmp.dir; Ignoring.
2011-10-07 20:29:19,096 WARN org.apache.hadoop.conf.Configuration:
/tmp/mapred-local/jobTracker/job_201110072015_0005.xml:a attempt to override final parameter:
mapred.jobtracker.maxtasks.per.job; Ignoring.
2011-10-07 20:29:19,096 WARN org.apache.hadoop.conf.Configuration:
/tmp/mapred-local/jobTracker/job_201110072015_0005.xml:a attempt to override final parameter:
mapred.job.tracker; Ignoring.
2011-10-07 20:29:19,096 WARN org.apache.hadoop.conf.Configuration:
/tmp/mapred-local/jobTracker/job_201110072015_0005.xml:a attempt to override final parameter: dfs.name.dir;
Ignoring.
2011-10-07 20:29:19,096 WARN org.apache.hadoop.conf.Configuration:
/tmp/mapred-local/jobTracker/job_201110072015_0005.xml:a attempt to override final parameter: m
apred.temp.dir; Ignoring.2011-10-07 20:29:19,103 INFO
org.apache.hadoop.mapreduce.security.token.DelegationTokenRenewal: registering token for renewal for service
=<NN IP>:50470 and jobID
= job_201110072015_0005
2011-10-07 20:29:19,103 INFO org.apache.hadoop.mapreduce.security.token.DelegationTokenRenewal: registering token for
renewal for service =<NN IP>:8020 and jobID = job_201110072015_0005
The STDOUT of distcp job when these warnings logged into JT log
----------------------------------------------------------------
$hadoop distcp hftp://<NN>:50070/tmp/inp out
11/10/07 20:29:17 INFO tools.DistCp:
srcPaths=[hftp://<NN>:50070/tmp/inp]
11/10/07 20:29:17 INFO tools.DistCp: destPath=out
11/10/07 20:29:18 INFO security.TokenCache: Got dt for
hftp://<NN>:50070/tmp/inp;uri=<NN IP>:50470;t.service=<NN IP>:50470
11/10/07 20:29:18 INFO hdfs.DFSClient: Created HDFS_DELEGATION_TOKEN token 7 for <user> on <NN IP>:8020
11/10/07 20:29:18 INFO security.TokenCache: Got dt for
1318019341/as;uri=<NN IP>:8020;t.service=<NN IP>:8020
..
..
If this is not critical, it may be unnecessary noise in log
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-2417) Warnings about attempt to override
final parameter while getting delegation token
Posted by "Ravi Prakash (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HDFS-2417?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13126366#comment-13126366 ]
Ravi Prakash commented on HDFS-2417:
------------------------------------
Hadoop QA is trying to apply the patch on trunk. But this goes only on 0.20-security. Patch for trunk is already committed on HADOOP-7664.
No tests are included because this is a trivial change in the logging of messages. Please commit.
> Warnings about attempt to override final parameter while getting delegation token
> ---------------------------------------------------------------------------------
>
> Key: HDFS-2417
> URL: https://issues.apache.org/jira/browse/HDFS-2417
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: name-node
> Affects Versions: 0.20.205.0
> Reporter: Rajit Saha
> Attachments: HDFS-2417.patch
>
>
> I am seeing whenever I run any Mapreduce job and its trying to acquire delegation from NN, In JT log following warnings coming about "a attempt to override final parameter:"
> The log snippet in JT log
> 2011-10-07 20:29:19,096 WARN org.apache.hadoop.conf.Configuration:
> /tmp/mapred-local/jobTracker/job_201110072015_0005.xml:a attempt to override final parameter:
> mapred.job.reuse.jvm.num.tasks; Ignoring.
> 2011-10-07 20:29:19,096 WARN org.apache.hadoop.conf.Configuration:
> /tmp/mapred-local/jobTracker/job_201110072015_0005.xml:a attempt to override final parameter: mapred.system.dir;
> Ignoring.
> 2011-10-07 20:29:19,096 WARN org.apache.hadoop.conf.Configuration:
> /tmp/mapred-local/jobTracker/job_201110072015_0005.xml:a attempt to override final parameter:
> hadoop.job.history.user.location; Ignoring.
> 2011-10-07 20:29:19,096 WARN org.apache.hadoop.conf.Configuration:
> /tmp/mapred-local/jobTracker/job_201110072015_0005.xml:a attempt to override final parameter: mapred.local.dir;
> Ignoring.
> 2011-10-07 20:29:19,096 WARN org.apache.hadoop.conf.Configuration:
> /tmp/mapred-local/jobTracker/job_201110072015_0005.xml:a attempt to override final parameter: m
> apred.job.tracker.http.address; Ignoring.
> 2011-10-07 20:29:19,096 WARN org.apache.hadoop.conf.Configuration:
> /tmp/mapred-local/jobTracker/job_201110072015_0005.xml:a attempt to override final parameter: d
> fs.data.dir; Ignoring.
> 2011-10-07 20:29:19,096 WARN org.apache.hadoop.conf.Configuration:
> /tmp/mapred-local/jobTracker/job_201110072015_0005.xml:a attempt to override final parameter: d
> fs.http.address; Ignoring.
> 2011-10-07 20:29:19,096 WARN org.apache.hadoop.conf.Configuration:
> /tmp/mapred-local/jobTracker/job_201110072015_0005.xml:a attempt to override final parameter: m
> apreduce.admin.map.child.java.opts; Ignoring.
> 2011-10-07 20:29:19,096 WARN org.apache.hadoop.conf.Configuration:
> /tmp/mapred-local/jobTracker/job_201110072015_0005.xml:a attempt to override final parameter:
> mapreduce.history.server.http.address; Ignoring.
> 2011-10-07 20:29:19,096 WARN org.apache.hadoop.conf.Configuration:
> /tmp/mapred-local/jobTracker/job_201110072015_0005.xml:a attempt to override final parameter: m
> apreduce.history.server.embedded; Ignoring.
> 2011-10-07 20:29:19,096 WARN org.apache.hadoop.conf.Configuration:
> /tmp/mapred-local/jobTracker/job_201110072015_0005.xml:a attempt to override final parameter: m
> apreduce.jobtracker.split.metainfo.maxsize; Ignoring.2011-10-07 20:29:19,096 WARN
> org.apache.hadoop.conf.Configuration: /tmp/mapred-local/jobTracker/job_201110072015_0005.xml:a attempt to
> override final parameter: m
> apreduce.admin.reduce.child.java.opts; Ignoring.2011-10-07 20:29:19,096 WARN org.apache.hadoop.conf.Configuration:
> /tmp/mapred-local/jobTracker/job_201110072015_0005.xml:a attempt to override final parameter: h
> adoop.tmp.dir; Ignoring.
> 2011-10-07 20:29:19,096 WARN org.apache.hadoop.conf.Configuration:
> /tmp/mapred-local/jobTracker/job_201110072015_0005.xml:a attempt to override final parameter:
> mapred.jobtracker.maxtasks.per.job; Ignoring.
> 2011-10-07 20:29:19,096 WARN org.apache.hadoop.conf.Configuration:
> /tmp/mapred-local/jobTracker/job_201110072015_0005.xml:a attempt to override final parameter:
> mapred.job.tracker; Ignoring.
> 2011-10-07 20:29:19,096 WARN org.apache.hadoop.conf.Configuration:
> /tmp/mapred-local/jobTracker/job_201110072015_0005.xml:a attempt to override final parameter: dfs.name.dir;
> Ignoring.
> 2011-10-07 20:29:19,096 WARN org.apache.hadoop.conf.Configuration:
> /tmp/mapred-local/jobTracker/job_201110072015_0005.xml:a attempt to override final parameter: m
> apred.temp.dir; Ignoring.2011-10-07 20:29:19,103 INFO
> org.apache.hadoop.mapreduce.security.token.DelegationTokenRenewal: registering token for renewal for service
> =<NN IP>:50470 and jobID
> = job_201110072015_0005
> 2011-10-07 20:29:19,103 INFO org.apache.hadoop.mapreduce.security.token.DelegationTokenRenewal: registering token for
> renewal for service =<NN IP>:8020 and jobID = job_201110072015_0005
> The STDOUT of distcp job when these warnings logged into JT log
> ----------------------------------------------------------------
> $hadoop distcp hftp://<NN>:50070/tmp/inp out
> 11/10/07 20:29:17 INFO tools.DistCp:
> srcPaths=[hftp://<NN>:50070/tmp/inp]
> 11/10/07 20:29:17 INFO tools.DistCp: destPath=out
> 11/10/07 20:29:18 INFO security.TokenCache: Got dt for
> hftp://<NN>:50070/tmp/inp;uri=<NN IP>:50470;t.service=<NN IP>:50470
> 11/10/07 20:29:18 INFO hdfs.DFSClient: Created HDFS_DELEGATION_TOKEN token 7 for <user> on <NN IP>:8020
> 11/10/07 20:29:18 INFO security.TokenCache: Got dt for
> 1318019341/as;uri=<NN IP>:8020;t.service=<NN IP>:8020
> ..
> ..
> If this is not critical, it may be unnecessary noise in log
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-2417) Warnings about attempt to override
final parameter while getting delegation token
Posted by "Tsz Wo (Nicholas), SZE (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HDFS-2417?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13146582#comment-13146582 ]
Tsz Wo (Nicholas), SZE commented on HDFS-2417:
----------------------------------------------
Is the patch the same as the one in HADOOP-7664 except that it is for 0.20? If yes, we should post the patch in HADOOP-7664.
> Warnings about attempt to override final parameter while getting delegation token
> ---------------------------------------------------------------------------------
>
> Key: HDFS-2417
> URL: https://issues.apache.org/jira/browse/HDFS-2417
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: name-node
> Affects Versions: 0.20.205.0
> Reporter: Rajit Saha
> Assignee: Ravi Prakash
> Attachments: HDFS-2417.patch
>
>
> I am seeing whenever I run any Mapreduce job and its trying to acquire delegation from NN, In JT log following warnings coming about "a attempt to override final parameter:"
> The log snippet in JT log
> 2011-10-07 20:29:19,096 WARN org.apache.hadoop.conf.Configuration:
> /tmp/mapred-local/jobTracker/job_201110072015_0005.xml:a attempt to override final parameter:
> mapred.job.reuse.jvm.num.tasks; Ignoring.
> 2011-10-07 20:29:19,096 WARN org.apache.hadoop.conf.Configuration:
> /tmp/mapred-local/jobTracker/job_201110072015_0005.xml:a attempt to override final parameter: mapred.system.dir;
> Ignoring.
> 2011-10-07 20:29:19,096 WARN org.apache.hadoop.conf.Configuration:
> /tmp/mapred-local/jobTracker/job_201110072015_0005.xml:a attempt to override final parameter:
> hadoop.job.history.user.location; Ignoring.
> 2011-10-07 20:29:19,096 WARN org.apache.hadoop.conf.Configuration:
> /tmp/mapred-local/jobTracker/job_201110072015_0005.xml:a attempt to override final parameter: mapred.local.dir;
> Ignoring.
> 2011-10-07 20:29:19,096 WARN org.apache.hadoop.conf.Configuration:
> /tmp/mapred-local/jobTracker/job_201110072015_0005.xml:a attempt to override final parameter: m
> apred.job.tracker.http.address; Ignoring.
> 2011-10-07 20:29:19,096 WARN org.apache.hadoop.conf.Configuration:
> /tmp/mapred-local/jobTracker/job_201110072015_0005.xml:a attempt to override final parameter: d
> fs.data.dir; Ignoring.
> 2011-10-07 20:29:19,096 WARN org.apache.hadoop.conf.Configuration:
> /tmp/mapred-local/jobTracker/job_201110072015_0005.xml:a attempt to override final parameter: d
> fs.http.address; Ignoring.
> 2011-10-07 20:29:19,096 WARN org.apache.hadoop.conf.Configuration:
> /tmp/mapred-local/jobTracker/job_201110072015_0005.xml:a attempt to override final parameter: m
> apreduce.admin.map.child.java.opts; Ignoring.
> 2011-10-07 20:29:19,096 WARN org.apache.hadoop.conf.Configuration:
> /tmp/mapred-local/jobTracker/job_201110072015_0005.xml:a attempt to override final parameter:
> mapreduce.history.server.http.address; Ignoring.
> 2011-10-07 20:29:19,096 WARN org.apache.hadoop.conf.Configuration:
> /tmp/mapred-local/jobTracker/job_201110072015_0005.xml:a attempt to override final parameter: m
> apreduce.history.server.embedded; Ignoring.
> 2011-10-07 20:29:19,096 WARN org.apache.hadoop.conf.Configuration:
> /tmp/mapred-local/jobTracker/job_201110072015_0005.xml:a attempt to override final parameter: m
> apreduce.jobtracker.split.metainfo.maxsize; Ignoring.2011-10-07 20:29:19,096 WARN
> org.apache.hadoop.conf.Configuration: /tmp/mapred-local/jobTracker/job_201110072015_0005.xml:a attempt to
> override final parameter: m
> apreduce.admin.reduce.child.java.opts; Ignoring.2011-10-07 20:29:19,096 WARN org.apache.hadoop.conf.Configuration:
> /tmp/mapred-local/jobTracker/job_201110072015_0005.xml:a attempt to override final parameter: h
> adoop.tmp.dir; Ignoring.
> 2011-10-07 20:29:19,096 WARN org.apache.hadoop.conf.Configuration:
> /tmp/mapred-local/jobTracker/job_201110072015_0005.xml:a attempt to override final parameter:
> mapred.jobtracker.maxtasks.per.job; Ignoring.
> 2011-10-07 20:29:19,096 WARN org.apache.hadoop.conf.Configuration:
> /tmp/mapred-local/jobTracker/job_201110072015_0005.xml:a attempt to override final parameter:
> mapred.job.tracker; Ignoring.
> 2011-10-07 20:29:19,096 WARN org.apache.hadoop.conf.Configuration:
> /tmp/mapred-local/jobTracker/job_201110072015_0005.xml:a attempt to override final parameter: dfs.name.dir;
> Ignoring.
> 2011-10-07 20:29:19,096 WARN org.apache.hadoop.conf.Configuration:
> /tmp/mapred-local/jobTracker/job_201110072015_0005.xml:a attempt to override final parameter: m
> apred.temp.dir; Ignoring.2011-10-07 20:29:19,103 INFO
> org.apache.hadoop.mapreduce.security.token.DelegationTokenRenewal: registering token for renewal for service
> =<NN IP>:50470 and jobID
> = job_201110072015_0005
> 2011-10-07 20:29:19,103 INFO org.apache.hadoop.mapreduce.security.token.DelegationTokenRenewal: registering token for
> renewal for service =<NN IP>:8020 and jobID = job_201110072015_0005
> The STDOUT of distcp job when these warnings logged into JT log
> ----------------------------------------------------------------
> $hadoop distcp hftp://<NN>:50070/tmp/inp out
> 11/10/07 20:29:17 INFO tools.DistCp:
> srcPaths=[hftp://<NN>:50070/tmp/inp]
> 11/10/07 20:29:17 INFO tools.DistCp: destPath=out
> 11/10/07 20:29:18 INFO security.TokenCache: Got dt for
> hftp://<NN>:50070/tmp/inp;uri=<NN IP>:50470;t.service=<NN IP>:50470
> 11/10/07 20:29:18 INFO hdfs.DFSClient: Created HDFS_DELEGATION_TOKEN token 7 for <user> on <NN IP>:8020
> 11/10/07 20:29:18 INFO security.TokenCache: Got dt for
> 1318019341/as;uri=<NN IP>:8020;t.service=<NN IP>:8020
> ..
> ..
> If this is not critical, it may be unnecessary noise in log
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-2417) Warnings about attempt to override
final parameter while getting delegation token
Posted by "Ravi Prakash (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HDFS-2417?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13126359#comment-13126359 ]
Ravi Prakash commented on HDFS-2417:
------------------------------------
HADOOP-7664 just went into trunk and 0.23. Can someone please commit the patch I provided here for 0.20?
The patch should fix the problem if its spurious.
> Warnings about attempt to override final parameter while getting delegation token
> ---------------------------------------------------------------------------------
>
> Key: HDFS-2417
> URL: https://issues.apache.org/jira/browse/HDFS-2417
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: name-node
> Affects Versions: 0.20.205.0
> Reporter: Rajit Saha
> Attachments: HDFS-2417.patch
>
>
> I am seeing whenever I run any Mapreduce job and its trying to acquire delegation from NN, In JT log following warnings coming about "a attempt to override final parameter:"
> The log snippet in JT log
> 2011-10-07 20:29:19,096 WARN org.apache.hadoop.conf.Configuration:
> /tmp/mapred-local/jobTracker/job_201110072015_0005.xml:a attempt to override final parameter:
> mapred.job.reuse.jvm.num.tasks; Ignoring.
> 2011-10-07 20:29:19,096 WARN org.apache.hadoop.conf.Configuration:
> /tmp/mapred-local/jobTracker/job_201110072015_0005.xml:a attempt to override final parameter: mapred.system.dir;
> Ignoring.
> 2011-10-07 20:29:19,096 WARN org.apache.hadoop.conf.Configuration:
> /tmp/mapred-local/jobTracker/job_201110072015_0005.xml:a attempt to override final parameter:
> hadoop.job.history.user.location; Ignoring.
> 2011-10-07 20:29:19,096 WARN org.apache.hadoop.conf.Configuration:
> /tmp/mapred-local/jobTracker/job_201110072015_0005.xml:a attempt to override final parameter: mapred.local.dir;
> Ignoring.
> 2011-10-07 20:29:19,096 WARN org.apache.hadoop.conf.Configuration:
> /tmp/mapred-local/jobTracker/job_201110072015_0005.xml:a attempt to override final parameter: m
> apred.job.tracker.http.address; Ignoring.
> 2011-10-07 20:29:19,096 WARN org.apache.hadoop.conf.Configuration:
> /tmp/mapred-local/jobTracker/job_201110072015_0005.xml:a attempt to override final parameter: d
> fs.data.dir; Ignoring.
> 2011-10-07 20:29:19,096 WARN org.apache.hadoop.conf.Configuration:
> /tmp/mapred-local/jobTracker/job_201110072015_0005.xml:a attempt to override final parameter: d
> fs.http.address; Ignoring.
> 2011-10-07 20:29:19,096 WARN org.apache.hadoop.conf.Configuration:
> /tmp/mapred-local/jobTracker/job_201110072015_0005.xml:a attempt to override final parameter: m
> apreduce.admin.map.child.java.opts; Ignoring.
> 2011-10-07 20:29:19,096 WARN org.apache.hadoop.conf.Configuration:
> /tmp/mapred-local/jobTracker/job_201110072015_0005.xml:a attempt to override final parameter:
> mapreduce.history.server.http.address; Ignoring.
> 2011-10-07 20:29:19,096 WARN org.apache.hadoop.conf.Configuration:
> /tmp/mapred-local/jobTracker/job_201110072015_0005.xml:a attempt to override final parameter: m
> apreduce.history.server.embedded; Ignoring.
> 2011-10-07 20:29:19,096 WARN org.apache.hadoop.conf.Configuration:
> /tmp/mapred-local/jobTracker/job_201110072015_0005.xml:a attempt to override final parameter: m
> apreduce.jobtracker.split.metainfo.maxsize; Ignoring.2011-10-07 20:29:19,096 WARN
> org.apache.hadoop.conf.Configuration: /tmp/mapred-local/jobTracker/job_201110072015_0005.xml:a attempt to
> override final parameter: m
> apreduce.admin.reduce.child.java.opts; Ignoring.2011-10-07 20:29:19,096 WARN org.apache.hadoop.conf.Configuration:
> /tmp/mapred-local/jobTracker/job_201110072015_0005.xml:a attempt to override final parameter: h
> adoop.tmp.dir; Ignoring.
> 2011-10-07 20:29:19,096 WARN org.apache.hadoop.conf.Configuration:
> /tmp/mapred-local/jobTracker/job_201110072015_0005.xml:a attempt to override final parameter:
> mapred.jobtracker.maxtasks.per.job; Ignoring.
> 2011-10-07 20:29:19,096 WARN org.apache.hadoop.conf.Configuration:
> /tmp/mapred-local/jobTracker/job_201110072015_0005.xml:a attempt to override final parameter:
> mapred.job.tracker; Ignoring.
> 2011-10-07 20:29:19,096 WARN org.apache.hadoop.conf.Configuration:
> /tmp/mapred-local/jobTracker/job_201110072015_0005.xml:a attempt to override final parameter: dfs.name.dir;
> Ignoring.
> 2011-10-07 20:29:19,096 WARN org.apache.hadoop.conf.Configuration:
> /tmp/mapred-local/jobTracker/job_201110072015_0005.xml:a attempt to override final parameter: m
> apred.temp.dir; Ignoring.2011-10-07 20:29:19,103 INFO
> org.apache.hadoop.mapreduce.security.token.DelegationTokenRenewal: registering token for renewal for service
> =<NN IP>:50470 and jobID
> = job_201110072015_0005
> 2011-10-07 20:29:19,103 INFO org.apache.hadoop.mapreduce.security.token.DelegationTokenRenewal: registering token for
> renewal for service =<NN IP>:8020 and jobID = job_201110072015_0005
> The STDOUT of distcp job when these warnings logged into JT log
> ----------------------------------------------------------------
> $hadoop distcp hftp://<NN>:50070/tmp/inp out
> 11/10/07 20:29:17 INFO tools.DistCp:
> srcPaths=[hftp://<NN>:50070/tmp/inp]
> 11/10/07 20:29:17 INFO tools.DistCp: destPath=out
> 11/10/07 20:29:18 INFO security.TokenCache: Got dt for
> hftp://<NN>:50070/tmp/inp;uri=<NN IP>:50470;t.service=<NN IP>:50470
> 11/10/07 20:29:18 INFO hdfs.DFSClient: Created HDFS_DELEGATION_TOKEN token 7 for <user> on <NN IP>:8020
> 11/10/07 20:29:18 INFO security.TokenCache: Got dt for
> 1318019341/as;uri=<NN IP>:8020;t.service=<NN IP>:8020
> ..
> ..
> If this is not critical, it may be unnecessary noise in log
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-2417) Warnings about attempt to override
final parameter while getting delegation token
Posted by "Ravi Prakash (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HDFS-2417?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Ravi Prakash updated HDFS-2417:
-------------------------------
Attachment: HDFS-2417.patch
Rebased HADOOP-7664 to branch-0.20-security. The same patch also applies to branch-0.20-security-205.
> Warnings about attempt to override final parameter while getting delegation token
> ---------------------------------------------------------------------------------
>
> Key: HDFS-2417
> URL: https://issues.apache.org/jira/browse/HDFS-2417
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: name-node
> Affects Versions: 0.20.205.0
> Reporter: Rajit Saha
> Attachments: HDFS-2417.patch
>
>
> I am seeing whenever I run any Mapreduce job and its trying to acquire delegation from NN, In JT log following warnings coming about "a attempt to override final parameter:"
> The log snippet in JT log
> 2011-10-07 20:29:19,096 WARN org.apache.hadoop.conf.Configuration:
> /tmp/mapred-local/jobTracker/job_201110072015_0005.xml:a attempt to override final parameter:
> mapred.job.reuse.jvm.num.tasks; Ignoring.
> 2011-10-07 20:29:19,096 WARN org.apache.hadoop.conf.Configuration:
> /tmp/mapred-local/jobTracker/job_201110072015_0005.xml:a attempt to override final parameter: mapred.system.dir;
> Ignoring.
> 2011-10-07 20:29:19,096 WARN org.apache.hadoop.conf.Configuration:
> /tmp/mapred-local/jobTracker/job_201110072015_0005.xml:a attempt to override final parameter:
> hadoop.job.history.user.location; Ignoring.
> 2011-10-07 20:29:19,096 WARN org.apache.hadoop.conf.Configuration:
> /tmp/mapred-local/jobTracker/job_201110072015_0005.xml:a attempt to override final parameter: mapred.local.dir;
> Ignoring.
> 2011-10-07 20:29:19,096 WARN org.apache.hadoop.conf.Configuration:
> /tmp/mapred-local/jobTracker/job_201110072015_0005.xml:a attempt to override final parameter: m
> apred.job.tracker.http.address; Ignoring.
> 2011-10-07 20:29:19,096 WARN org.apache.hadoop.conf.Configuration:
> /tmp/mapred-local/jobTracker/job_201110072015_0005.xml:a attempt to override final parameter: d
> fs.data.dir; Ignoring.
> 2011-10-07 20:29:19,096 WARN org.apache.hadoop.conf.Configuration:
> /tmp/mapred-local/jobTracker/job_201110072015_0005.xml:a attempt to override final parameter: d
> fs.http.address; Ignoring.
> 2011-10-07 20:29:19,096 WARN org.apache.hadoop.conf.Configuration:
> /tmp/mapred-local/jobTracker/job_201110072015_0005.xml:a attempt to override final parameter: m
> apreduce.admin.map.child.java.opts; Ignoring.
> 2011-10-07 20:29:19,096 WARN org.apache.hadoop.conf.Configuration:
> /tmp/mapred-local/jobTracker/job_201110072015_0005.xml:a attempt to override final parameter:
> mapreduce.history.server.http.address; Ignoring.
> 2011-10-07 20:29:19,096 WARN org.apache.hadoop.conf.Configuration:
> /tmp/mapred-local/jobTracker/job_201110072015_0005.xml:a attempt to override final parameter: m
> apreduce.history.server.embedded; Ignoring.
> 2011-10-07 20:29:19,096 WARN org.apache.hadoop.conf.Configuration:
> /tmp/mapred-local/jobTracker/job_201110072015_0005.xml:a attempt to override final parameter: m
> apreduce.jobtracker.split.metainfo.maxsize; Ignoring.2011-10-07 20:29:19,096 WARN
> org.apache.hadoop.conf.Configuration: /tmp/mapred-local/jobTracker/job_201110072015_0005.xml:a attempt to
> override final parameter: m
> apreduce.admin.reduce.child.java.opts; Ignoring.2011-10-07 20:29:19,096 WARN org.apache.hadoop.conf.Configuration:
> /tmp/mapred-local/jobTracker/job_201110072015_0005.xml:a attempt to override final parameter: h
> adoop.tmp.dir; Ignoring.
> 2011-10-07 20:29:19,096 WARN org.apache.hadoop.conf.Configuration:
> /tmp/mapred-local/jobTracker/job_201110072015_0005.xml:a attempt to override final parameter:
> mapred.jobtracker.maxtasks.per.job; Ignoring.
> 2011-10-07 20:29:19,096 WARN org.apache.hadoop.conf.Configuration:
> /tmp/mapred-local/jobTracker/job_201110072015_0005.xml:a attempt to override final parameter:
> mapred.job.tracker; Ignoring.
> 2011-10-07 20:29:19,096 WARN org.apache.hadoop.conf.Configuration:
> /tmp/mapred-local/jobTracker/job_201110072015_0005.xml:a attempt to override final parameter: dfs.name.dir;
> Ignoring.
> 2011-10-07 20:29:19,096 WARN org.apache.hadoop.conf.Configuration:
> /tmp/mapred-local/jobTracker/job_201110072015_0005.xml:a attempt to override final parameter: m
> apred.temp.dir; Ignoring.2011-10-07 20:29:19,103 INFO
> org.apache.hadoop.mapreduce.security.token.DelegationTokenRenewal: registering token for renewal for service
> =<NN IP>:50470 and jobID
> = job_201110072015_0005
> 2011-10-07 20:29:19,103 INFO org.apache.hadoop.mapreduce.security.token.DelegationTokenRenewal: registering token for
> renewal for service =<NN IP>:8020 and jobID = job_201110072015_0005
> The STDOUT of distcp job when these warnings logged into JT log
> ----------------------------------------------------------------
> $hadoop distcp hftp://<NN>:50070/tmp/inp out
> 11/10/07 20:29:17 INFO tools.DistCp:
> srcPaths=[hftp://<NN>:50070/tmp/inp]
> 11/10/07 20:29:17 INFO tools.DistCp: destPath=out
> 11/10/07 20:29:18 INFO security.TokenCache: Got dt for
> hftp://<NN>:50070/tmp/inp;uri=<NN IP>:50470;t.service=<NN IP>:50470
> 11/10/07 20:29:18 INFO hdfs.DFSClient: Created HDFS_DELEGATION_TOKEN token 7 for <user> on <NN IP>:8020
> 11/10/07 20:29:18 INFO security.TokenCache: Got dt for
> 1318019341/as;uri=<NN IP>:8020;t.service=<NN IP>:8020
> ..
> ..
> If this is not critical, it may be unnecessary noise in log
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-2417) Warnings about attempt to override
final parameter while getting delegation token
Posted by "Ravi Prakash (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HDFS-2417?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Ravi Prakash updated HDFS-2417:
-------------------------------
Resolution: Duplicate
Status: Resolved (was: Patch Available)
Attached the patch for branch-0.20-security to HADOOP-7664
> Warnings about attempt to override final parameter while getting delegation token
> ---------------------------------------------------------------------------------
>
> Key: HDFS-2417
> URL: https://issues.apache.org/jira/browse/HDFS-2417
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: name-node
> Affects Versions: 0.20.205.0
> Reporter: Rajit Saha
> Assignee: Ravi Prakash
> Attachments: HDFS-2417.patch
>
>
> I am seeing whenever I run any Mapreduce job and its trying to acquire delegation from NN, In JT log following warnings coming about "a attempt to override final parameter:"
> The log snippet in JT log
> 2011-10-07 20:29:19,096 WARN org.apache.hadoop.conf.Configuration:
> /tmp/mapred-local/jobTracker/job_201110072015_0005.xml:a attempt to override final parameter:
> mapred.job.reuse.jvm.num.tasks; Ignoring.
> 2011-10-07 20:29:19,096 WARN org.apache.hadoop.conf.Configuration:
> /tmp/mapred-local/jobTracker/job_201110072015_0005.xml:a attempt to override final parameter: mapred.system.dir;
> Ignoring.
> 2011-10-07 20:29:19,096 WARN org.apache.hadoop.conf.Configuration:
> /tmp/mapred-local/jobTracker/job_201110072015_0005.xml:a attempt to override final parameter:
> hadoop.job.history.user.location; Ignoring.
> 2011-10-07 20:29:19,096 WARN org.apache.hadoop.conf.Configuration:
> /tmp/mapred-local/jobTracker/job_201110072015_0005.xml:a attempt to override final parameter: mapred.local.dir;
> Ignoring.
> 2011-10-07 20:29:19,096 WARN org.apache.hadoop.conf.Configuration:
> /tmp/mapred-local/jobTracker/job_201110072015_0005.xml:a attempt to override final parameter: m
> apred.job.tracker.http.address; Ignoring.
> 2011-10-07 20:29:19,096 WARN org.apache.hadoop.conf.Configuration:
> /tmp/mapred-local/jobTracker/job_201110072015_0005.xml:a attempt to override final parameter: d
> fs.data.dir; Ignoring.
> 2011-10-07 20:29:19,096 WARN org.apache.hadoop.conf.Configuration:
> /tmp/mapred-local/jobTracker/job_201110072015_0005.xml:a attempt to override final parameter: d
> fs.http.address; Ignoring.
> 2011-10-07 20:29:19,096 WARN org.apache.hadoop.conf.Configuration:
> /tmp/mapred-local/jobTracker/job_201110072015_0005.xml:a attempt to override final parameter: m
> apreduce.admin.map.child.java.opts; Ignoring.
> 2011-10-07 20:29:19,096 WARN org.apache.hadoop.conf.Configuration:
> /tmp/mapred-local/jobTracker/job_201110072015_0005.xml:a attempt to override final parameter:
> mapreduce.history.server.http.address; Ignoring.
> 2011-10-07 20:29:19,096 WARN org.apache.hadoop.conf.Configuration:
> /tmp/mapred-local/jobTracker/job_201110072015_0005.xml:a attempt to override final parameter: m
> apreduce.history.server.embedded; Ignoring.
> 2011-10-07 20:29:19,096 WARN org.apache.hadoop.conf.Configuration:
> /tmp/mapred-local/jobTracker/job_201110072015_0005.xml:a attempt to override final parameter: m
> apreduce.jobtracker.split.metainfo.maxsize; Ignoring.2011-10-07 20:29:19,096 WARN
> org.apache.hadoop.conf.Configuration: /tmp/mapred-local/jobTracker/job_201110072015_0005.xml:a attempt to
> override final parameter: m
> apreduce.admin.reduce.child.java.opts; Ignoring.2011-10-07 20:29:19,096 WARN org.apache.hadoop.conf.Configuration:
> /tmp/mapred-local/jobTracker/job_201110072015_0005.xml:a attempt to override final parameter: h
> adoop.tmp.dir; Ignoring.
> 2011-10-07 20:29:19,096 WARN org.apache.hadoop.conf.Configuration:
> /tmp/mapred-local/jobTracker/job_201110072015_0005.xml:a attempt to override final parameter:
> mapred.jobtracker.maxtasks.per.job; Ignoring.
> 2011-10-07 20:29:19,096 WARN org.apache.hadoop.conf.Configuration:
> /tmp/mapred-local/jobTracker/job_201110072015_0005.xml:a attempt to override final parameter:
> mapred.job.tracker; Ignoring.
> 2011-10-07 20:29:19,096 WARN org.apache.hadoop.conf.Configuration:
> /tmp/mapred-local/jobTracker/job_201110072015_0005.xml:a attempt to override final parameter: dfs.name.dir;
> Ignoring.
> 2011-10-07 20:29:19,096 WARN org.apache.hadoop.conf.Configuration:
> /tmp/mapred-local/jobTracker/job_201110072015_0005.xml:a attempt to override final parameter: m
> apred.temp.dir; Ignoring.2011-10-07 20:29:19,103 INFO
> org.apache.hadoop.mapreduce.security.token.DelegationTokenRenewal: registering token for renewal for service
> =<NN IP>:50470 and jobID
> = job_201110072015_0005
> 2011-10-07 20:29:19,103 INFO org.apache.hadoop.mapreduce.security.token.DelegationTokenRenewal: registering token for
> renewal for service =<NN IP>:8020 and jobID = job_201110072015_0005
> The STDOUT of distcp job when these warnings logged into JT log
> ----------------------------------------------------------------
> $hadoop distcp hftp://<NN>:50070/tmp/inp out
> 11/10/07 20:29:17 INFO tools.DistCp:
> srcPaths=[hftp://<NN>:50070/tmp/inp]
> 11/10/07 20:29:17 INFO tools.DistCp: destPath=out
> 11/10/07 20:29:18 INFO security.TokenCache: Got dt for
> hftp://<NN>:50070/tmp/inp;uri=<NN IP>:50470;t.service=<NN IP>:50470
> 11/10/07 20:29:18 INFO hdfs.DFSClient: Created HDFS_DELEGATION_TOKEN token 7 for <user> on <NN IP>:8020
> 11/10/07 20:29:18 INFO security.TokenCache: Got dt for
> 1318019341/as;uri=<NN IP>:8020;t.service=<NN IP>:8020
> ..
> ..
> If this is not critical, it may be unnecessary noise in log
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Assigned] (HDFS-2417) Warnings about attempt to override
final parameter while getting delegation token
Posted by "Ravi Prakash (Assigned) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HDFS-2417?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Ravi Prakash reassigned HDFS-2417:
----------------------------------
Assignee: Ravi Prakash
> Warnings about attempt to override final parameter while getting delegation token
> ---------------------------------------------------------------------------------
>
> Key: HDFS-2417
> URL: https://issues.apache.org/jira/browse/HDFS-2417
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: name-node
> Affects Versions: 0.20.205.0
> Reporter: Rajit Saha
> Assignee: Ravi Prakash
> Attachments: HDFS-2417.patch
>
>
> I am seeing whenever I run any Mapreduce job and its trying to acquire delegation from NN, In JT log following warnings coming about "a attempt to override final parameter:"
> The log snippet in JT log
> 2011-10-07 20:29:19,096 WARN org.apache.hadoop.conf.Configuration:
> /tmp/mapred-local/jobTracker/job_201110072015_0005.xml:a attempt to override final parameter:
> mapred.job.reuse.jvm.num.tasks; Ignoring.
> 2011-10-07 20:29:19,096 WARN org.apache.hadoop.conf.Configuration:
> /tmp/mapred-local/jobTracker/job_201110072015_0005.xml:a attempt to override final parameter: mapred.system.dir;
> Ignoring.
> 2011-10-07 20:29:19,096 WARN org.apache.hadoop.conf.Configuration:
> /tmp/mapred-local/jobTracker/job_201110072015_0005.xml:a attempt to override final parameter:
> hadoop.job.history.user.location; Ignoring.
> 2011-10-07 20:29:19,096 WARN org.apache.hadoop.conf.Configuration:
> /tmp/mapred-local/jobTracker/job_201110072015_0005.xml:a attempt to override final parameter: mapred.local.dir;
> Ignoring.
> 2011-10-07 20:29:19,096 WARN org.apache.hadoop.conf.Configuration:
> /tmp/mapred-local/jobTracker/job_201110072015_0005.xml:a attempt to override final parameter: m
> apred.job.tracker.http.address; Ignoring.
> 2011-10-07 20:29:19,096 WARN org.apache.hadoop.conf.Configuration:
> /tmp/mapred-local/jobTracker/job_201110072015_0005.xml:a attempt to override final parameter: d
> fs.data.dir; Ignoring.
> 2011-10-07 20:29:19,096 WARN org.apache.hadoop.conf.Configuration:
> /tmp/mapred-local/jobTracker/job_201110072015_0005.xml:a attempt to override final parameter: d
> fs.http.address; Ignoring.
> 2011-10-07 20:29:19,096 WARN org.apache.hadoop.conf.Configuration:
> /tmp/mapred-local/jobTracker/job_201110072015_0005.xml:a attempt to override final parameter: m
> apreduce.admin.map.child.java.opts; Ignoring.
> 2011-10-07 20:29:19,096 WARN org.apache.hadoop.conf.Configuration:
> /tmp/mapred-local/jobTracker/job_201110072015_0005.xml:a attempt to override final parameter:
> mapreduce.history.server.http.address; Ignoring.
> 2011-10-07 20:29:19,096 WARN org.apache.hadoop.conf.Configuration:
> /tmp/mapred-local/jobTracker/job_201110072015_0005.xml:a attempt to override final parameter: m
> apreduce.history.server.embedded; Ignoring.
> 2011-10-07 20:29:19,096 WARN org.apache.hadoop.conf.Configuration:
> /tmp/mapred-local/jobTracker/job_201110072015_0005.xml:a attempt to override final parameter: m
> apreduce.jobtracker.split.metainfo.maxsize; Ignoring.2011-10-07 20:29:19,096 WARN
> org.apache.hadoop.conf.Configuration: /tmp/mapred-local/jobTracker/job_201110072015_0005.xml:a attempt to
> override final parameter: m
> apreduce.admin.reduce.child.java.opts; Ignoring.2011-10-07 20:29:19,096 WARN org.apache.hadoop.conf.Configuration:
> /tmp/mapred-local/jobTracker/job_201110072015_0005.xml:a attempt to override final parameter: h
> adoop.tmp.dir; Ignoring.
> 2011-10-07 20:29:19,096 WARN org.apache.hadoop.conf.Configuration:
> /tmp/mapred-local/jobTracker/job_201110072015_0005.xml:a attempt to override final parameter:
> mapred.jobtracker.maxtasks.per.job; Ignoring.
> 2011-10-07 20:29:19,096 WARN org.apache.hadoop.conf.Configuration:
> /tmp/mapred-local/jobTracker/job_201110072015_0005.xml:a attempt to override final parameter:
> mapred.job.tracker; Ignoring.
> 2011-10-07 20:29:19,096 WARN org.apache.hadoop.conf.Configuration:
> /tmp/mapred-local/jobTracker/job_201110072015_0005.xml:a attempt to override final parameter: dfs.name.dir;
> Ignoring.
> 2011-10-07 20:29:19,096 WARN org.apache.hadoop.conf.Configuration:
> /tmp/mapred-local/jobTracker/job_201110072015_0005.xml:a attempt to override final parameter: m
> apred.temp.dir; Ignoring.2011-10-07 20:29:19,103 INFO
> org.apache.hadoop.mapreduce.security.token.DelegationTokenRenewal: registering token for renewal for service
> =<NN IP>:50470 and jobID
> = job_201110072015_0005
> 2011-10-07 20:29:19,103 INFO org.apache.hadoop.mapreduce.security.token.DelegationTokenRenewal: registering token for
> renewal for service =<NN IP>:8020 and jobID = job_201110072015_0005
> The STDOUT of distcp job when these warnings logged into JT log
> ----------------------------------------------------------------
> $hadoop distcp hftp://<NN>:50070/tmp/inp out
> 11/10/07 20:29:17 INFO tools.DistCp:
> srcPaths=[hftp://<NN>:50070/tmp/inp]
> 11/10/07 20:29:17 INFO tools.DistCp: destPath=out
> 11/10/07 20:29:18 INFO security.TokenCache: Got dt for
> hftp://<NN>:50070/tmp/inp;uri=<NN IP>:50470;t.service=<NN IP>:50470
> 11/10/07 20:29:18 INFO hdfs.DFSClient: Created HDFS_DELEGATION_TOKEN token 7 for <user> on <NN IP>:8020
> 11/10/07 20:29:18 INFO security.TokenCache: Got dt for
> 1318019341/as;uri=<NN IP>:8020;t.service=<NN IP>:8020
> ..
> ..
> If this is not critical, it may be unnecessary noise in log
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-2417) Warnings about attempt to override
final parameter while getting delegation token
Posted by "Hadoop QA (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HDFS-2417?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13126361#comment-13126361 ]
Hadoop QA commented on HDFS-2417:
---------------------------------
-1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12498331/HDFS-2417.patch
against trunk revision .
+1 @author. The patch does not contain any @author tags.
-1 tests included. The patch doesn't appear to include any new or modified tests.
Please justify why no new tests are needed for this patch.
Also please list what manual steps were performed to verify this patch.
-1 patch. The patch command could not apply the patch.
Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/1363//console
This message is automatically generated.
> Warnings about attempt to override final parameter while getting delegation token
> ---------------------------------------------------------------------------------
>
> Key: HDFS-2417
> URL: https://issues.apache.org/jira/browse/HDFS-2417
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: name-node
> Affects Versions: 0.20.205.0
> Reporter: Rajit Saha
> Attachments: HDFS-2417.patch
>
>
> I am seeing whenever I run any Mapreduce job and its trying to acquire delegation from NN, In JT log following warnings coming about "a attempt to override final parameter:"
> The log snippet in JT log
> 2011-10-07 20:29:19,096 WARN org.apache.hadoop.conf.Configuration:
> /tmp/mapred-local/jobTracker/job_201110072015_0005.xml:a attempt to override final parameter:
> mapred.job.reuse.jvm.num.tasks; Ignoring.
> 2011-10-07 20:29:19,096 WARN org.apache.hadoop.conf.Configuration:
> /tmp/mapred-local/jobTracker/job_201110072015_0005.xml:a attempt to override final parameter: mapred.system.dir;
> Ignoring.
> 2011-10-07 20:29:19,096 WARN org.apache.hadoop.conf.Configuration:
> /tmp/mapred-local/jobTracker/job_201110072015_0005.xml:a attempt to override final parameter:
> hadoop.job.history.user.location; Ignoring.
> 2011-10-07 20:29:19,096 WARN org.apache.hadoop.conf.Configuration:
> /tmp/mapred-local/jobTracker/job_201110072015_0005.xml:a attempt to override final parameter: mapred.local.dir;
> Ignoring.
> 2011-10-07 20:29:19,096 WARN org.apache.hadoop.conf.Configuration:
> /tmp/mapred-local/jobTracker/job_201110072015_0005.xml:a attempt to override final parameter: m
> apred.job.tracker.http.address; Ignoring.
> 2011-10-07 20:29:19,096 WARN org.apache.hadoop.conf.Configuration:
> /tmp/mapred-local/jobTracker/job_201110072015_0005.xml:a attempt to override final parameter: d
> fs.data.dir; Ignoring.
> 2011-10-07 20:29:19,096 WARN org.apache.hadoop.conf.Configuration:
> /tmp/mapred-local/jobTracker/job_201110072015_0005.xml:a attempt to override final parameter: d
> fs.http.address; Ignoring.
> 2011-10-07 20:29:19,096 WARN org.apache.hadoop.conf.Configuration:
> /tmp/mapred-local/jobTracker/job_201110072015_0005.xml:a attempt to override final parameter: m
> apreduce.admin.map.child.java.opts; Ignoring.
> 2011-10-07 20:29:19,096 WARN org.apache.hadoop.conf.Configuration:
> /tmp/mapred-local/jobTracker/job_201110072015_0005.xml:a attempt to override final parameter:
> mapreduce.history.server.http.address; Ignoring.
> 2011-10-07 20:29:19,096 WARN org.apache.hadoop.conf.Configuration:
> /tmp/mapred-local/jobTracker/job_201110072015_0005.xml:a attempt to override final parameter: m
> apreduce.history.server.embedded; Ignoring.
> 2011-10-07 20:29:19,096 WARN org.apache.hadoop.conf.Configuration:
> /tmp/mapred-local/jobTracker/job_201110072015_0005.xml:a attempt to override final parameter: m
> apreduce.jobtracker.split.metainfo.maxsize; Ignoring.2011-10-07 20:29:19,096 WARN
> org.apache.hadoop.conf.Configuration: /tmp/mapred-local/jobTracker/job_201110072015_0005.xml:a attempt to
> override final parameter: m
> apreduce.admin.reduce.child.java.opts; Ignoring.2011-10-07 20:29:19,096 WARN org.apache.hadoop.conf.Configuration:
> /tmp/mapred-local/jobTracker/job_201110072015_0005.xml:a attempt to override final parameter: h
> adoop.tmp.dir; Ignoring.
> 2011-10-07 20:29:19,096 WARN org.apache.hadoop.conf.Configuration:
> /tmp/mapred-local/jobTracker/job_201110072015_0005.xml:a attempt to override final parameter:
> mapred.jobtracker.maxtasks.per.job; Ignoring.
> 2011-10-07 20:29:19,096 WARN org.apache.hadoop.conf.Configuration:
> /tmp/mapred-local/jobTracker/job_201110072015_0005.xml:a attempt to override final parameter:
> mapred.job.tracker; Ignoring.
> 2011-10-07 20:29:19,096 WARN org.apache.hadoop.conf.Configuration:
> /tmp/mapred-local/jobTracker/job_201110072015_0005.xml:a attempt to override final parameter: dfs.name.dir;
> Ignoring.
> 2011-10-07 20:29:19,096 WARN org.apache.hadoop.conf.Configuration:
> /tmp/mapred-local/jobTracker/job_201110072015_0005.xml:a attempt to override final parameter: m
> apred.temp.dir; Ignoring.2011-10-07 20:29:19,103 INFO
> org.apache.hadoop.mapreduce.security.token.DelegationTokenRenewal: registering token for renewal for service
> =<NN IP>:50470 and jobID
> = job_201110072015_0005
> 2011-10-07 20:29:19,103 INFO org.apache.hadoop.mapreduce.security.token.DelegationTokenRenewal: registering token for
> renewal for service =<NN IP>:8020 and jobID = job_201110072015_0005
> The STDOUT of distcp job when these warnings logged into JT log
> ----------------------------------------------------------------
> $hadoop distcp hftp://<NN>:50070/tmp/inp out
> 11/10/07 20:29:17 INFO tools.DistCp:
> srcPaths=[hftp://<NN>:50070/tmp/inp]
> 11/10/07 20:29:17 INFO tools.DistCp: destPath=out
> 11/10/07 20:29:18 INFO security.TokenCache: Got dt for
> hftp://<NN>:50070/tmp/inp;uri=<NN IP>:50470;t.service=<NN IP>:50470
> 11/10/07 20:29:18 INFO hdfs.DFSClient: Created HDFS_DELEGATION_TOKEN token 7 for <user> on <NN IP>:8020
> 11/10/07 20:29:18 INFO security.TokenCache: Got dt for
> 1318019341/as;uri=<NN IP>:8020;t.service=<NN IP>:8020
> ..
> ..
> If this is not critical, it may be unnecessary noise in log
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-2417) Warnings about attempt to override
final parameter while getting delegation token
Posted by "Ravi Prakash (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HDFS-2417?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13142456#comment-13142456 ]
Ravi Prakash commented on HDFS-2417:
------------------------------------
Nicholas / Suresh! Could you please commit this patch?
> Warnings about attempt to override final parameter while getting delegation token
> ---------------------------------------------------------------------------------
>
> Key: HDFS-2417
> URL: https://issues.apache.org/jira/browse/HDFS-2417
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: name-node
> Affects Versions: 0.20.205.0
> Reporter: Rajit Saha
> Assignee: Ravi Prakash
> Attachments: HDFS-2417.patch
>
>
> I am seeing whenever I run any Mapreduce job and its trying to acquire delegation from NN, In JT log following warnings coming about "a attempt to override final parameter:"
> The log snippet in JT log
> 2011-10-07 20:29:19,096 WARN org.apache.hadoop.conf.Configuration:
> /tmp/mapred-local/jobTracker/job_201110072015_0005.xml:a attempt to override final parameter:
> mapred.job.reuse.jvm.num.tasks; Ignoring.
> 2011-10-07 20:29:19,096 WARN org.apache.hadoop.conf.Configuration:
> /tmp/mapred-local/jobTracker/job_201110072015_0005.xml:a attempt to override final parameter: mapred.system.dir;
> Ignoring.
> 2011-10-07 20:29:19,096 WARN org.apache.hadoop.conf.Configuration:
> /tmp/mapred-local/jobTracker/job_201110072015_0005.xml:a attempt to override final parameter:
> hadoop.job.history.user.location; Ignoring.
> 2011-10-07 20:29:19,096 WARN org.apache.hadoop.conf.Configuration:
> /tmp/mapred-local/jobTracker/job_201110072015_0005.xml:a attempt to override final parameter: mapred.local.dir;
> Ignoring.
> 2011-10-07 20:29:19,096 WARN org.apache.hadoop.conf.Configuration:
> /tmp/mapred-local/jobTracker/job_201110072015_0005.xml:a attempt to override final parameter: m
> apred.job.tracker.http.address; Ignoring.
> 2011-10-07 20:29:19,096 WARN org.apache.hadoop.conf.Configuration:
> /tmp/mapred-local/jobTracker/job_201110072015_0005.xml:a attempt to override final parameter: d
> fs.data.dir; Ignoring.
> 2011-10-07 20:29:19,096 WARN org.apache.hadoop.conf.Configuration:
> /tmp/mapred-local/jobTracker/job_201110072015_0005.xml:a attempt to override final parameter: d
> fs.http.address; Ignoring.
> 2011-10-07 20:29:19,096 WARN org.apache.hadoop.conf.Configuration:
> /tmp/mapred-local/jobTracker/job_201110072015_0005.xml:a attempt to override final parameter: m
> apreduce.admin.map.child.java.opts; Ignoring.
> 2011-10-07 20:29:19,096 WARN org.apache.hadoop.conf.Configuration:
> /tmp/mapred-local/jobTracker/job_201110072015_0005.xml:a attempt to override final parameter:
> mapreduce.history.server.http.address; Ignoring.
> 2011-10-07 20:29:19,096 WARN org.apache.hadoop.conf.Configuration:
> /tmp/mapred-local/jobTracker/job_201110072015_0005.xml:a attempt to override final parameter: m
> apreduce.history.server.embedded; Ignoring.
> 2011-10-07 20:29:19,096 WARN org.apache.hadoop.conf.Configuration:
> /tmp/mapred-local/jobTracker/job_201110072015_0005.xml:a attempt to override final parameter: m
> apreduce.jobtracker.split.metainfo.maxsize; Ignoring.2011-10-07 20:29:19,096 WARN
> org.apache.hadoop.conf.Configuration: /tmp/mapred-local/jobTracker/job_201110072015_0005.xml:a attempt to
> override final parameter: m
> apreduce.admin.reduce.child.java.opts; Ignoring.2011-10-07 20:29:19,096 WARN org.apache.hadoop.conf.Configuration:
> /tmp/mapred-local/jobTracker/job_201110072015_0005.xml:a attempt to override final parameter: h
> adoop.tmp.dir; Ignoring.
> 2011-10-07 20:29:19,096 WARN org.apache.hadoop.conf.Configuration:
> /tmp/mapred-local/jobTracker/job_201110072015_0005.xml:a attempt to override final parameter:
> mapred.jobtracker.maxtasks.per.job; Ignoring.
> 2011-10-07 20:29:19,096 WARN org.apache.hadoop.conf.Configuration:
> /tmp/mapred-local/jobTracker/job_201110072015_0005.xml:a attempt to override final parameter:
> mapred.job.tracker; Ignoring.
> 2011-10-07 20:29:19,096 WARN org.apache.hadoop.conf.Configuration:
> /tmp/mapred-local/jobTracker/job_201110072015_0005.xml:a attempt to override final parameter: dfs.name.dir;
> Ignoring.
> 2011-10-07 20:29:19,096 WARN org.apache.hadoop.conf.Configuration:
> /tmp/mapred-local/jobTracker/job_201110072015_0005.xml:a attempt to override final parameter: m
> apred.temp.dir; Ignoring.2011-10-07 20:29:19,103 INFO
> org.apache.hadoop.mapreduce.security.token.DelegationTokenRenewal: registering token for renewal for service
> =<NN IP>:50470 and jobID
> = job_201110072015_0005
> 2011-10-07 20:29:19,103 INFO org.apache.hadoop.mapreduce.security.token.DelegationTokenRenewal: registering token for
> renewal for service =<NN IP>:8020 and jobID = job_201110072015_0005
> The STDOUT of distcp job when these warnings logged into JT log
> ----------------------------------------------------------------
> $hadoop distcp hftp://<NN>:50070/tmp/inp out
> 11/10/07 20:29:17 INFO tools.DistCp:
> srcPaths=[hftp://<NN>:50070/tmp/inp]
> 11/10/07 20:29:17 INFO tools.DistCp: destPath=out
> 11/10/07 20:29:18 INFO security.TokenCache: Got dt for
> hftp://<NN>:50070/tmp/inp;uri=<NN IP>:50470;t.service=<NN IP>:50470
> 11/10/07 20:29:18 INFO hdfs.DFSClient: Created HDFS_DELEGATION_TOKEN token 7 for <user> on <NN IP>:8020
> 11/10/07 20:29:18 INFO security.TokenCache: Got dt for
> 1318019341/as;uri=<NN IP>:8020;t.service=<NN IP>:8020
> ..
> ..
> If this is not critical, it may be unnecessary noise in log
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-2417) Warnings about attempt to override
final parameter while getting delegation token
Posted by "Ravi Prakash (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HDFS-2417?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Ravi Prakash updated HDFS-2417:
-------------------------------
Status: Patch Available (was: Open)
> Warnings about attempt to override final parameter while getting delegation token
> ---------------------------------------------------------------------------------
>
> Key: HDFS-2417
> URL: https://issues.apache.org/jira/browse/HDFS-2417
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: name-node
> Affects Versions: 0.20.205.0
> Reporter: Rajit Saha
> Attachments: HDFS-2417.patch
>
>
> I am seeing whenever I run any Mapreduce job and its trying to acquire delegation from NN, In JT log following warnings coming about "a attempt to override final parameter:"
> The log snippet in JT log
> 2011-10-07 20:29:19,096 WARN org.apache.hadoop.conf.Configuration:
> /tmp/mapred-local/jobTracker/job_201110072015_0005.xml:a attempt to override final parameter:
> mapred.job.reuse.jvm.num.tasks; Ignoring.
> 2011-10-07 20:29:19,096 WARN org.apache.hadoop.conf.Configuration:
> /tmp/mapred-local/jobTracker/job_201110072015_0005.xml:a attempt to override final parameter: mapred.system.dir;
> Ignoring.
> 2011-10-07 20:29:19,096 WARN org.apache.hadoop.conf.Configuration:
> /tmp/mapred-local/jobTracker/job_201110072015_0005.xml:a attempt to override final parameter:
> hadoop.job.history.user.location; Ignoring.
> 2011-10-07 20:29:19,096 WARN org.apache.hadoop.conf.Configuration:
> /tmp/mapred-local/jobTracker/job_201110072015_0005.xml:a attempt to override final parameter: mapred.local.dir;
> Ignoring.
> 2011-10-07 20:29:19,096 WARN org.apache.hadoop.conf.Configuration:
> /tmp/mapred-local/jobTracker/job_201110072015_0005.xml:a attempt to override final parameter: m
> apred.job.tracker.http.address; Ignoring.
> 2011-10-07 20:29:19,096 WARN org.apache.hadoop.conf.Configuration:
> /tmp/mapred-local/jobTracker/job_201110072015_0005.xml:a attempt to override final parameter: d
> fs.data.dir; Ignoring.
> 2011-10-07 20:29:19,096 WARN org.apache.hadoop.conf.Configuration:
> /tmp/mapred-local/jobTracker/job_201110072015_0005.xml:a attempt to override final parameter: d
> fs.http.address; Ignoring.
> 2011-10-07 20:29:19,096 WARN org.apache.hadoop.conf.Configuration:
> /tmp/mapred-local/jobTracker/job_201110072015_0005.xml:a attempt to override final parameter: m
> apreduce.admin.map.child.java.opts; Ignoring.
> 2011-10-07 20:29:19,096 WARN org.apache.hadoop.conf.Configuration:
> /tmp/mapred-local/jobTracker/job_201110072015_0005.xml:a attempt to override final parameter:
> mapreduce.history.server.http.address; Ignoring.
> 2011-10-07 20:29:19,096 WARN org.apache.hadoop.conf.Configuration:
> /tmp/mapred-local/jobTracker/job_201110072015_0005.xml:a attempt to override final parameter: m
> apreduce.history.server.embedded; Ignoring.
> 2011-10-07 20:29:19,096 WARN org.apache.hadoop.conf.Configuration:
> /tmp/mapred-local/jobTracker/job_201110072015_0005.xml:a attempt to override final parameter: m
> apreduce.jobtracker.split.metainfo.maxsize; Ignoring.2011-10-07 20:29:19,096 WARN
> org.apache.hadoop.conf.Configuration: /tmp/mapred-local/jobTracker/job_201110072015_0005.xml:a attempt to
> override final parameter: m
> apreduce.admin.reduce.child.java.opts; Ignoring.2011-10-07 20:29:19,096 WARN org.apache.hadoop.conf.Configuration:
> /tmp/mapred-local/jobTracker/job_201110072015_0005.xml:a attempt to override final parameter: h
> adoop.tmp.dir; Ignoring.
> 2011-10-07 20:29:19,096 WARN org.apache.hadoop.conf.Configuration:
> /tmp/mapred-local/jobTracker/job_201110072015_0005.xml:a attempt to override final parameter:
> mapred.jobtracker.maxtasks.per.job; Ignoring.
> 2011-10-07 20:29:19,096 WARN org.apache.hadoop.conf.Configuration:
> /tmp/mapred-local/jobTracker/job_201110072015_0005.xml:a attempt to override final parameter:
> mapred.job.tracker; Ignoring.
> 2011-10-07 20:29:19,096 WARN org.apache.hadoop.conf.Configuration:
> /tmp/mapred-local/jobTracker/job_201110072015_0005.xml:a attempt to override final parameter: dfs.name.dir;
> Ignoring.
> 2011-10-07 20:29:19,096 WARN org.apache.hadoop.conf.Configuration:
> /tmp/mapred-local/jobTracker/job_201110072015_0005.xml:a attempt to override final parameter: m
> apred.temp.dir; Ignoring.2011-10-07 20:29:19,103 INFO
> org.apache.hadoop.mapreduce.security.token.DelegationTokenRenewal: registering token for renewal for service
> =<NN IP>:50470 and jobID
> = job_201110072015_0005
> 2011-10-07 20:29:19,103 INFO org.apache.hadoop.mapreduce.security.token.DelegationTokenRenewal: registering token for
> renewal for service =<NN IP>:8020 and jobID = job_201110072015_0005
> The STDOUT of distcp job when these warnings logged into JT log
> ----------------------------------------------------------------
> $hadoop distcp hftp://<NN>:50070/tmp/inp out
> 11/10/07 20:29:17 INFO tools.DistCp:
> srcPaths=[hftp://<NN>:50070/tmp/inp]
> 11/10/07 20:29:17 INFO tools.DistCp: destPath=out
> 11/10/07 20:29:18 INFO security.TokenCache: Got dt for
> hftp://<NN>:50070/tmp/inp;uri=<NN IP>:50470;t.service=<NN IP>:50470
> 11/10/07 20:29:18 INFO hdfs.DFSClient: Created HDFS_DELEGATION_TOKEN token 7 for <user> on <NN IP>:8020
> 11/10/07 20:29:18 INFO security.TokenCache: Got dt for
> 1318019341/as;uri=<NN IP>:8020;t.service=<NN IP>:8020
> ..
> ..
> If this is not critical, it may be unnecessary noise in log
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-2417) Warnings about attempt to override
final parameter while getting delegation token
Posted by "Aaron T. Myers (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HDFS-2417?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13123238#comment-13123238 ]
Aaron T. Myers commented on HDFS-2417:
--------------------------------------
Hi Rajit, is this a duplicate of HADOOP-7664?
> Warnings about attempt to override final parameter while getting delegation token
> ---------------------------------------------------------------------------------
>
> Key: HDFS-2417
> URL: https://issues.apache.org/jira/browse/HDFS-2417
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: name-node
> Affects Versions: 0.20.205.0
> Reporter: Rajit Saha
>
> I am seeing whenever I run any Mapreduce job and its trying to acquire delegation from NN, In JT log following warnings coming about "a attempt to override final parameter:"
> The log snippet in JT log
> 2011-10-07 20:29:19,096 WARN org.apache.hadoop.conf.Configuration:
> /tmp/mapred-local/jobTracker/job_201110072015_0005.xml:a attempt to override final parameter:
> mapred.job.reuse.jvm.num.tasks; Ignoring.
> 2011-10-07 20:29:19,096 WARN org.apache.hadoop.conf.Configuration:
> /tmp/mapred-local/jobTracker/job_201110072015_0005.xml:a attempt to override final parameter: mapred.system.dir;
> Ignoring.
> 2011-10-07 20:29:19,096 WARN org.apache.hadoop.conf.Configuration:
> /tmp/mapred-local/jobTracker/job_201110072015_0005.xml:a attempt to override final parameter:
> hadoop.job.history.user.location; Ignoring.
> 2011-10-07 20:29:19,096 WARN org.apache.hadoop.conf.Configuration:
> /tmp/mapred-local/jobTracker/job_201110072015_0005.xml:a attempt to override final parameter: mapred.local.dir;
> Ignoring.
> 2011-10-07 20:29:19,096 WARN org.apache.hadoop.conf.Configuration:
> /tmp/mapred-local/jobTracker/job_201110072015_0005.xml:a attempt to override final parameter: m
> apred.job.tracker.http.address; Ignoring.
> 2011-10-07 20:29:19,096 WARN org.apache.hadoop.conf.Configuration:
> /tmp/mapred-local/jobTracker/job_201110072015_0005.xml:a attempt to override final parameter: d
> fs.data.dir; Ignoring.
> 2011-10-07 20:29:19,096 WARN org.apache.hadoop.conf.Configuration:
> /tmp/mapred-local/jobTracker/job_201110072015_0005.xml:a attempt to override final parameter: d
> fs.http.address; Ignoring.
> 2011-10-07 20:29:19,096 WARN org.apache.hadoop.conf.Configuration:
> /tmp/mapred-local/jobTracker/job_201110072015_0005.xml:a attempt to override final parameter: m
> apreduce.admin.map.child.java.opts; Ignoring.
> 2011-10-07 20:29:19,096 WARN org.apache.hadoop.conf.Configuration:
> /tmp/mapred-local/jobTracker/job_201110072015_0005.xml:a attempt to override final parameter:
> mapreduce.history.server.http.address; Ignoring.
> 2011-10-07 20:29:19,096 WARN org.apache.hadoop.conf.Configuration:
> /tmp/mapred-local/jobTracker/job_201110072015_0005.xml:a attempt to override final parameter: m
> apreduce.history.server.embedded; Ignoring.
> 2011-10-07 20:29:19,096 WARN org.apache.hadoop.conf.Configuration:
> /tmp/mapred-local/jobTracker/job_201110072015_0005.xml:a attempt to override final parameter: m
> apreduce.jobtracker.split.metainfo.maxsize; Ignoring.2011-10-07 20:29:19,096 WARN
> org.apache.hadoop.conf.Configuration: /tmp/mapred-local/jobTracker/job_201110072015_0005.xml:a attempt to
> override final parameter: m
> apreduce.admin.reduce.child.java.opts; Ignoring.2011-10-07 20:29:19,096 WARN org.apache.hadoop.conf.Configuration:
> /tmp/mapred-local/jobTracker/job_201110072015_0005.xml:a attempt to override final parameter: h
> adoop.tmp.dir; Ignoring.
> 2011-10-07 20:29:19,096 WARN org.apache.hadoop.conf.Configuration:
> /tmp/mapred-local/jobTracker/job_201110072015_0005.xml:a attempt to override final parameter:
> mapred.jobtracker.maxtasks.per.job; Ignoring.
> 2011-10-07 20:29:19,096 WARN org.apache.hadoop.conf.Configuration:
> /tmp/mapred-local/jobTracker/job_201110072015_0005.xml:a attempt to override final parameter:
> mapred.job.tracker; Ignoring.
> 2011-10-07 20:29:19,096 WARN org.apache.hadoop.conf.Configuration:
> /tmp/mapred-local/jobTracker/job_201110072015_0005.xml:a attempt to override final parameter: dfs.name.dir;
> Ignoring.
> 2011-10-07 20:29:19,096 WARN org.apache.hadoop.conf.Configuration:
> /tmp/mapred-local/jobTracker/job_201110072015_0005.xml:a attempt to override final parameter: m
> apred.temp.dir; Ignoring.2011-10-07 20:29:19,103 INFO
> org.apache.hadoop.mapreduce.security.token.DelegationTokenRenewal: registering token for renewal for service
> =<NN IP>:50470 and jobID
> = job_201110072015_0005
> 2011-10-07 20:29:19,103 INFO org.apache.hadoop.mapreduce.security.token.DelegationTokenRenewal: registering token for
> renewal for service =<NN IP>:8020 and jobID = job_201110072015_0005
> The STDOUT of distcp job when these warnings logged into JT log
> ----------------------------------------------------------------
> $hadoop distcp hftp://<NN>:50070/tmp/inp out
> 11/10/07 20:29:17 INFO tools.DistCp:
> srcPaths=[hftp://<NN>:50070/tmp/inp]
> 11/10/07 20:29:17 INFO tools.DistCp: destPath=out
> 11/10/07 20:29:18 INFO security.TokenCache: Got dt for
> hftp://<NN>:50070/tmp/inp;uri=<NN IP>:50470;t.service=<NN IP>:50470
> 11/10/07 20:29:18 INFO hdfs.DFSClient: Created HDFS_DELEGATION_TOKEN token 7 for <user> on <NN IP>:8020
> 11/10/07 20:29:18 INFO security.TokenCache: Got dt for
> 1318019341/as;uri=<NN IP>:8020;t.service=<NN IP>:8020
> ..
> ..
> If this is not critical, it may be unnecessary noise in log
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-2417) Warnings about attempt to override
final parameter while getting delegation token
Posted by "Rajit Saha (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HDFS-2417?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13123259#comment-13123259 ]
Rajit Saha commented on HDFS-2417:
----------------------------------
Thanks Aaron, we have started seeing this in .20.205 as well , we found HADOOP-7664 in .23, are they coming from same source
> Warnings about attempt to override final parameter while getting delegation token
> ---------------------------------------------------------------------------------
>
> Key: HDFS-2417
> URL: https://issues.apache.org/jira/browse/HDFS-2417
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: name-node
> Affects Versions: 0.20.205.0
> Reporter: Rajit Saha
>
> I am seeing whenever I run any Mapreduce job and its trying to acquire delegation from NN, In JT log following warnings coming about "a attempt to override final parameter:"
> The log snippet in JT log
> 2011-10-07 20:29:19,096 WARN org.apache.hadoop.conf.Configuration:
> /tmp/mapred-local/jobTracker/job_201110072015_0005.xml:a attempt to override final parameter:
> mapred.job.reuse.jvm.num.tasks; Ignoring.
> 2011-10-07 20:29:19,096 WARN org.apache.hadoop.conf.Configuration:
> /tmp/mapred-local/jobTracker/job_201110072015_0005.xml:a attempt to override final parameter: mapred.system.dir;
> Ignoring.
> 2011-10-07 20:29:19,096 WARN org.apache.hadoop.conf.Configuration:
> /tmp/mapred-local/jobTracker/job_201110072015_0005.xml:a attempt to override final parameter:
> hadoop.job.history.user.location; Ignoring.
> 2011-10-07 20:29:19,096 WARN org.apache.hadoop.conf.Configuration:
> /tmp/mapred-local/jobTracker/job_201110072015_0005.xml:a attempt to override final parameter: mapred.local.dir;
> Ignoring.
> 2011-10-07 20:29:19,096 WARN org.apache.hadoop.conf.Configuration:
> /tmp/mapred-local/jobTracker/job_201110072015_0005.xml:a attempt to override final parameter: m
> apred.job.tracker.http.address; Ignoring.
> 2011-10-07 20:29:19,096 WARN org.apache.hadoop.conf.Configuration:
> /tmp/mapred-local/jobTracker/job_201110072015_0005.xml:a attempt to override final parameter: d
> fs.data.dir; Ignoring.
> 2011-10-07 20:29:19,096 WARN org.apache.hadoop.conf.Configuration:
> /tmp/mapred-local/jobTracker/job_201110072015_0005.xml:a attempt to override final parameter: d
> fs.http.address; Ignoring.
> 2011-10-07 20:29:19,096 WARN org.apache.hadoop.conf.Configuration:
> /tmp/mapred-local/jobTracker/job_201110072015_0005.xml:a attempt to override final parameter: m
> apreduce.admin.map.child.java.opts; Ignoring.
> 2011-10-07 20:29:19,096 WARN org.apache.hadoop.conf.Configuration:
> /tmp/mapred-local/jobTracker/job_201110072015_0005.xml:a attempt to override final parameter:
> mapreduce.history.server.http.address; Ignoring.
> 2011-10-07 20:29:19,096 WARN org.apache.hadoop.conf.Configuration:
> /tmp/mapred-local/jobTracker/job_201110072015_0005.xml:a attempt to override final parameter: m
> apreduce.history.server.embedded; Ignoring.
> 2011-10-07 20:29:19,096 WARN org.apache.hadoop.conf.Configuration:
> /tmp/mapred-local/jobTracker/job_201110072015_0005.xml:a attempt to override final parameter: m
> apreduce.jobtracker.split.metainfo.maxsize; Ignoring.2011-10-07 20:29:19,096 WARN
> org.apache.hadoop.conf.Configuration: /tmp/mapred-local/jobTracker/job_201110072015_0005.xml:a attempt to
> override final parameter: m
> apreduce.admin.reduce.child.java.opts; Ignoring.2011-10-07 20:29:19,096 WARN org.apache.hadoop.conf.Configuration:
> /tmp/mapred-local/jobTracker/job_201110072015_0005.xml:a attempt to override final parameter: h
> adoop.tmp.dir; Ignoring.
> 2011-10-07 20:29:19,096 WARN org.apache.hadoop.conf.Configuration:
> /tmp/mapred-local/jobTracker/job_201110072015_0005.xml:a attempt to override final parameter:
> mapred.jobtracker.maxtasks.per.job; Ignoring.
> 2011-10-07 20:29:19,096 WARN org.apache.hadoop.conf.Configuration:
> /tmp/mapred-local/jobTracker/job_201110072015_0005.xml:a attempt to override final parameter:
> mapred.job.tracker; Ignoring.
> 2011-10-07 20:29:19,096 WARN org.apache.hadoop.conf.Configuration:
> /tmp/mapred-local/jobTracker/job_201110072015_0005.xml:a attempt to override final parameter: dfs.name.dir;
> Ignoring.
> 2011-10-07 20:29:19,096 WARN org.apache.hadoop.conf.Configuration:
> /tmp/mapred-local/jobTracker/job_201110072015_0005.xml:a attempt to override final parameter: m
> apred.temp.dir; Ignoring.2011-10-07 20:29:19,103 INFO
> org.apache.hadoop.mapreduce.security.token.DelegationTokenRenewal: registering token for renewal for service
> =<NN IP>:50470 and jobID
> = job_201110072015_0005
> 2011-10-07 20:29:19,103 INFO org.apache.hadoop.mapreduce.security.token.DelegationTokenRenewal: registering token for
> renewal for service =<NN IP>:8020 and jobID = job_201110072015_0005
> The STDOUT of distcp job when these warnings logged into JT log
> ----------------------------------------------------------------
> $hadoop distcp hftp://<NN>:50070/tmp/inp out
> 11/10/07 20:29:17 INFO tools.DistCp:
> srcPaths=[hftp://<NN>:50070/tmp/inp]
> 11/10/07 20:29:17 INFO tools.DistCp: destPath=out
> 11/10/07 20:29:18 INFO security.TokenCache: Got dt for
> hftp://<NN>:50070/tmp/inp;uri=<NN IP>:50470;t.service=<NN IP>:50470
> 11/10/07 20:29:18 INFO hdfs.DFSClient: Created HDFS_DELEGATION_TOKEN token 7 for <user> on <NN IP>:8020
> 11/10/07 20:29:18 INFO security.TokenCache: Got dt for
> 1318019341/as;uri=<NN IP>:8020;t.service=<NN IP>:8020
> ..
> ..
> If this is not critical, it may be unnecessary noise in log
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira