You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-issues@hadoop.apache.org by "Todd Lipcon (Assigned) (JIRA)" <ji...@apache.org> on 2012/03/08 02:18:57 UTC
[jira] [Assigned] (HADOOP-8153) Fail to submit mapred job on a
secured-HA-HDFS: logic URI cannot be picked up by job submission.
[ https://issues.apache.org/jira/browse/HADOOP-8153?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Todd Lipcon reassigned HADOOP-8153:
-----------------------------------
Assignee: Todd Lipcon
> Fail to submit mapred job on a secured-HA-HDFS: logic URI cannot be picked up by job submission.
> ------------------------------------------------------------------------------------------------
>
> Key: HADOOP-8153
> URL: https://issues.apache.org/jira/browse/HADOOP-8153
> Project: Hadoop Common
> Issue Type: Bug
> Components: ha, security
> Affects Versions: 0.24.0
> Reporter: Mingjie Lai
> Assignee: Todd Lipcon
> Fix For: 0.24.0
>
>
> When testing the combination of NN HA + security + yarn, I found that the mapred job submission cannot pick up the logic URI of a nameservice.
> I have logic URI configured in core-site.xml
> {code}
> <property>
> <name>fs.defaultFS</name>
> <value>hdfs://ns1</value>
> </property>
> {code}
> HDFS client can work with the HA deployment/configs:
> {code}
> [root@nn1 hadoop]# hdfs dfs -ls /
> Found 6 items
> drwxr-xr-x - hbase hadoop 0 2012-03-07 20:42 /hbase
> drwxrwxrwx - yarn hadoop 0 2012-03-07 20:42 /logs
> drwxr-xr-x - mapred hadoop 0 2012-03-07 20:42 /mapred
> drwxr-xr-x - mapred hadoop 0 2012-03-07 20:42 /mr-history
> drwxrwxrwt - hdfs hadoop 0 2012-03-07 21:57 /tmp
> drwxr-xr-x - hdfs hadoop 0 2012-03-07 20:42 /user
> {code}
> but cannot submit a mapred job with security turned on
> {code}
> [root@nn1 hadoop]# /usr/lib/hadoop/bin/yarn --config ./conf jar share/hadoop/mapreduce/hadoop-mapreduce-examples-0.24.0-SNAPSHOT.jar randomwriter out
> Running 0 maps.
> Job started: Wed Mar 07 23:28:23 UTC 2012
> java.lang.IllegalArgumentException: java.net.UnknownHostException: ns1
> at org.apache.hadoop.security.SecurityUtil.buildTokenService(SecurityUtil.java:431)
> at org.apache.hadoop.security.SecurityUtil.buildDTServiceName(SecurityUtil.java:312)
> at org.apache.hadoop.fs.FileSystem.getCanonicalServiceName(FileSystem.java:217)
> at org.apache.hadoop.mapreduce.security.TokenCache.obtainTokensForNamenodesInternal(TokenCache.java:119)
> at org.apache.hadoop.mapreduce.security.TokenCache.obtainTokensForNamenodesInternal(TokenCache.java:97)
> at org.apache.hadoop.mapreduce.security.TokenCache.obtainTokensForNamenodes(TokenCache.java:80)
> at org.apache.hadoop.mapreduce.lib.output.FileOutputFormat.checkOutputSpecs(FileOutputFormat.java:137)
> at org.apache.hadoop.mapreduce.JobSubmitter.checkSpecs(JobSubmitter.java:411)
> at org.apache.hadoop.mapreduce.JobSubmitter.submitJobInternal(JobSubmitter.java:326)
> at org.apache.hadoop.mapreduce.Job$11.run(Job.java:1221)
> at org.apache.hadoop.mapreduce.Job$11.run(Job.java:1218)
> ....
> {code}0.24
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira