You are viewing a plain text version of this content. The canonical link for it is here.
Posted to mapreduce-user@hadoop.apache.org by Nishanth S <ch...@gmail.com> on 2015/05/05 21:09:32 UTC
Help with Kerberos
Hello ,
I am getting the below exception when trying to run a simple map reduce
from a machine outside the hadoop cluster.This is the kerberos code I am
using.Please help
org.apache.hadoop.conf.Configuration conf = new
org.apache.hadoop.conf.Configuration();
conf.set("hadoop.security.authentication", "Kerberos");
UserGroupInformation.setConfiguration(conf);
try{
UserGroupInformation.loginUserFromKeytab("abc@domain.com",
"C:\\work\\documents\\devkeytab\\abc.keytab");
Exception in thread "main" java.io.IOException: Can't get Master Kerberos
principal for use as renewer
at
org.apache.hadoop.mapreduce.security.TokenCache.obtainTokensForNamenodesInternal(TokenCache.java:116)
at
org.apache.hadoop.mapreduce.security.TokenCache.obtainTokensForNamenodesInternal(TokenCache.java:100)
at
org.apache.hadoop.mapreduce.security.TokenCache.obtainTokensForNamenodes(TokenCache.java:80)
at
org.apache.hadoop.mapreduce.lib.output.FileOutputFormat.checkOutputSpecs(FileOutputFormat.java:142)
at
org.apache.hadoop.mapreduce.JobSubmitter.checkSpecs(JobSubmitter.java:562)
at
org.apache.hadoop.mapreduce.JobSubmitter.submitJobInternal(JobSubmitter.java:432)
at org.apache.hadoop.mapreduce.Job$10.run(Job.java:1296)
at org.apache.hadoop.mapreduce.Job$10.run(Job.java:1293)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Unknown Source)
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1628)
at org.apache.hadoop.mapreduce.Job.submit(Job.java:1293)
at org.apache.hadoop.mapreduce.Job.waitForCompletion(Job.java:1314)
Re: Help with Kerberos
Posted by Nishanth S <ch...@gmail.com>.
I am getting the below error now after adding all configuration
files(hdfs-site.xml, yarn files,etc) to the classpath of my client
project.The login is succesful but when I try to access hdfs the below is
error is encountered.I see many posts having the same issue but did not se
any resolution.Any help is appreciated.
Exception in thread "main" java.io.IOException: Failed on local exception:
java.io.IOException: java.lang.IllegalArgumentException:
Server has invalid Kerberos principal: hdfs/abc.domain.com@DOMAIN.COM; Host
Details : local host is: "xyz/10.a.b.c"; destination host is: "
bc.domain.com.domain.com":8020;
at org.apache.hadoop.net.NetUtils.wrapException(NetUtils.java:772)
at org.apache.hadoop.ipc.Client.call(Client.java:1472)
at org.apache.hadoop.ipc.Client.call(Client.java:1399)
at
org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.invoke(ProtobufRpcEngine.java:232)
at com.sun.proxy.$Proxy9.getDelegationToken(Unknown Source)
at
org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolTranslatorPB.getDelegationToken(ClientNamenodeProtocolTranslatorPB.java:909)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
On Tue, May 5, 2015 at 5:59 PM, Birender Saini <bs...@hortonworks.com>
wrote:
> Are those principals in KDC?
>
> Try to get kerberos ticket for that principal using the same key tab
> that's in config.
>
>
>
> -------- Original message --------
> From: Nishanth S <ch...@gmail.com>
> Date:05/05/2015 6:33 PM (GMT-05:00)
> To: user@hadoop.apache.org
> Subject: Re: Help with Kerberos
>
> Thanks Chris.I checked the configuration files and see that these
> properties are in place.I am trying to connect to hadoop cluster in linux
> from a windows machine.
>
>
>
>
> On Tue, May 5, 2015 at 1:25 PM, Chris Nauroth <cn...@hortonworks.com>
> wrote:
>
>> Hello Nishanth,
>>
>> The job submission was unable to identify the principal to use for renewal
>> of HDFS delegation tokens. Delegation tokens are subject to a soft
>> expiration, which can be extended by a renewer requesting an extension
>> from the NameNode periodically. In the context of MapReduce running on
>> YARN, the ResourceManager acts as the delegation token renewer.
>>
>> Specifically, this exception likely indicates that your configuration is
>> missing either (or both) of the properties yarn.resourcemanager.address
>> and yarn.resourcemanager.principal.
>>
>> <property>
>> <description>The address of the applications manager interface in the
>> RM.</description>
>> <name>yarn.resourcemanager.address</name>
>> <value>${yarn.resourcemanager.hostname}:8032</value>
>> </property>
>>
>> <property>
>> <description>The Kerberos principal for the resource
>> manager.</description>
>> <name>yarn.resourcemanager.principal</name>
>> </property>
>>
>> You might want to review the secure mode documentation. It has more
>> details on this and other relevant configuration properties.
>>
>>
>> http://hadoop.apache.org/docs/r2.7.0/hadoop-project-dist/hadoop-common/Secu
>> reMode.html
>> <http://hadoop.apache.org/docs/r2.7.0/hadoop-project-dist/hadoop-common/SecureMode.html>
>>
>> I hope this helps.
>>
>> --Chris Nauroth
>>
>>
>> From: Nishanth S <ch...@gmail.com>
>> Reply-To: "user@hadoop.apache.org" <us...@hadoop.apache.org>
>> Date: Tuesday, May 5, 2015 at 12:09 PM
>> To: "user@hadoop.apache.org" <us...@hadoop.apache.org>
>> Subject: Help with Kerberos
>>
>>
>> Hello ,
>>
>> I am getting the below exception when trying to run a simple map reduce
>> from a machine outside the hadoop cluster.This is the kerberos code I am
>> using.Please help
>>
>>
>> org.apache.hadoop.conf.Configuration conf = new
>> org.apache.hadoop.conf.Configuration();
>> conf.set("hadoop.security.authentication", "Kerberos");
>> UserGroupInformation.setConfiguration(conf);
>> try{
>> UserGroupInformation.loginUserFromKeytab("abc@domain.com",
>> "C:\\work\\documents\\devkeytab\\abc.keytab");
>>
>>
>>
>> Exception in thread "main" java.io.IOException: Can't get Master Kerberos
>> principal for use as renewer
>> at
>>
>> org.apache.hadoop.mapreduce.security.TokenCache.obtainTokensForNamenodesInt
>> ernal(TokenCache.java:116)
>> at
>>
>> org.apache.hadoop.mapreduce.security.TokenCache.obtainTokensForNamenodesInt
>> ernal(TokenCache.java:100)
>> at
>>
>> org.apache.hadoop.mapreduce.security.TokenCache.obtainTokensForNamenodes(To
>> kenCache.java:80)
>> at
>>
>> org.apache.hadoop.mapreduce.lib.output.FileOutputFormat.checkOutputSpecs(Fi
>> leOutputFormat.java:142)
>> at
>> org.apache.hadoop.mapreduce.JobSubmitter.checkSpecs(JobSubmitter.java:562)
>> at
>>
>> org.apache.hadoop.mapreduce.JobSubmitter.submitJobInternal(JobSubmitter.jav
>> a:432)
>> at org.apache.hadoop.mapreduce.Job$10.run(Job.java:1296)
>> at org.apache.hadoop.mapreduce.Job$10.run(Job.java:1293)
>> at java.security.AccessController.doPrivileged(Native Method)
>> at javax.security.auth.Subject.doAs(Unknown Source)
>> at
>>
>> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.j
>> ava:1628)
>> at org.apache.hadoop.mapreduce.Job.submit(Job.java:1293)
>> at org.apache.hadoop.mapreduce.Job.waitForCompletion(Job.java:1314)
>>
>>
>
Re: Help with Kerberos
Posted by Nishanth S <ch...@gmail.com>.
I am getting the below error now after adding all configuration
files(hdfs-site.xml, yarn files,etc) to the classpath of my client
project.The login is succesful but when I try to access hdfs the below is
error is encountered.I see many posts having the same issue but did not se
any resolution.Any help is appreciated.
Exception in thread "main" java.io.IOException: Failed on local exception:
java.io.IOException: java.lang.IllegalArgumentException:
Server has invalid Kerberos principal: hdfs/abc.domain.com@DOMAIN.COM; Host
Details : local host is: "xyz/10.a.b.c"; destination host is: "
bc.domain.com.domain.com":8020;
at org.apache.hadoop.net.NetUtils.wrapException(NetUtils.java:772)
at org.apache.hadoop.ipc.Client.call(Client.java:1472)
at org.apache.hadoop.ipc.Client.call(Client.java:1399)
at
org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.invoke(ProtobufRpcEngine.java:232)
at com.sun.proxy.$Proxy9.getDelegationToken(Unknown Source)
at
org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolTranslatorPB.getDelegationToken(ClientNamenodeProtocolTranslatorPB.java:909)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
On Tue, May 5, 2015 at 5:59 PM, Birender Saini <bs...@hortonworks.com>
wrote:
> Are those principals in KDC?
>
> Try to get kerberos ticket for that principal using the same key tab
> that's in config.
>
>
>
> -------- Original message --------
> From: Nishanth S <ch...@gmail.com>
> Date:05/05/2015 6:33 PM (GMT-05:00)
> To: user@hadoop.apache.org
> Subject: Re: Help with Kerberos
>
> Thanks Chris.I checked the configuration files and see that these
> properties are in place.I am trying to connect to hadoop cluster in linux
> from a windows machine.
>
>
>
>
> On Tue, May 5, 2015 at 1:25 PM, Chris Nauroth <cn...@hortonworks.com>
> wrote:
>
>> Hello Nishanth,
>>
>> The job submission was unable to identify the principal to use for renewal
>> of HDFS delegation tokens. Delegation tokens are subject to a soft
>> expiration, which can be extended by a renewer requesting an extension
>> from the NameNode periodically. In the context of MapReduce running on
>> YARN, the ResourceManager acts as the delegation token renewer.
>>
>> Specifically, this exception likely indicates that your configuration is
>> missing either (or both) of the properties yarn.resourcemanager.address
>> and yarn.resourcemanager.principal.
>>
>> <property>
>> <description>The address of the applications manager interface in the
>> RM.</description>
>> <name>yarn.resourcemanager.address</name>
>> <value>${yarn.resourcemanager.hostname}:8032</value>
>> </property>
>>
>> <property>
>> <description>The Kerberos principal for the resource
>> manager.</description>
>> <name>yarn.resourcemanager.principal</name>
>> </property>
>>
>> You might want to review the secure mode documentation. It has more
>> details on this and other relevant configuration properties.
>>
>>
>> http://hadoop.apache.org/docs/r2.7.0/hadoop-project-dist/hadoop-common/Secu
>> reMode.html
>> <http://hadoop.apache.org/docs/r2.7.0/hadoop-project-dist/hadoop-common/SecureMode.html>
>>
>> I hope this helps.
>>
>> --Chris Nauroth
>>
>>
>> From: Nishanth S <ch...@gmail.com>
>> Reply-To: "user@hadoop.apache.org" <us...@hadoop.apache.org>
>> Date: Tuesday, May 5, 2015 at 12:09 PM
>> To: "user@hadoop.apache.org" <us...@hadoop.apache.org>
>> Subject: Help with Kerberos
>>
>>
>> Hello ,
>>
>> I am getting the below exception when trying to run a simple map reduce
>> from a machine outside the hadoop cluster.This is the kerberos code I am
>> using.Please help
>>
>>
>> org.apache.hadoop.conf.Configuration conf = new
>> org.apache.hadoop.conf.Configuration();
>> conf.set("hadoop.security.authentication", "Kerberos");
>> UserGroupInformation.setConfiguration(conf);
>> try{
>> UserGroupInformation.loginUserFromKeytab("abc@domain.com",
>> "C:\\work\\documents\\devkeytab\\abc.keytab");
>>
>>
>>
>> Exception in thread "main" java.io.IOException: Can't get Master Kerberos
>> principal for use as renewer
>> at
>>
>> org.apache.hadoop.mapreduce.security.TokenCache.obtainTokensForNamenodesInt
>> ernal(TokenCache.java:116)
>> at
>>
>> org.apache.hadoop.mapreduce.security.TokenCache.obtainTokensForNamenodesInt
>> ernal(TokenCache.java:100)
>> at
>>
>> org.apache.hadoop.mapreduce.security.TokenCache.obtainTokensForNamenodes(To
>> kenCache.java:80)
>> at
>>
>> org.apache.hadoop.mapreduce.lib.output.FileOutputFormat.checkOutputSpecs(Fi
>> leOutputFormat.java:142)
>> at
>> org.apache.hadoop.mapreduce.JobSubmitter.checkSpecs(JobSubmitter.java:562)
>> at
>>
>> org.apache.hadoop.mapreduce.JobSubmitter.submitJobInternal(JobSubmitter.jav
>> a:432)
>> at org.apache.hadoop.mapreduce.Job$10.run(Job.java:1296)
>> at org.apache.hadoop.mapreduce.Job$10.run(Job.java:1293)
>> at java.security.AccessController.doPrivileged(Native Method)
>> at javax.security.auth.Subject.doAs(Unknown Source)
>> at
>>
>> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.j
>> ava:1628)
>> at org.apache.hadoop.mapreduce.Job.submit(Job.java:1293)
>> at org.apache.hadoop.mapreduce.Job.waitForCompletion(Job.java:1314)
>>
>>
>
Re: Help with Kerberos
Posted by Nishanth S <ch...@gmail.com>.
I am getting the below error now after adding all configuration
files(hdfs-site.xml, yarn files,etc) to the classpath of my client
project.The login is succesful but when I try to access hdfs the below is
error is encountered.I see many posts having the same issue but did not se
any resolution.Any help is appreciated.
Exception in thread "main" java.io.IOException: Failed on local exception:
java.io.IOException: java.lang.IllegalArgumentException:
Server has invalid Kerberos principal: hdfs/abc.domain.com@DOMAIN.COM; Host
Details : local host is: "xyz/10.a.b.c"; destination host is: "
bc.domain.com.domain.com":8020;
at org.apache.hadoop.net.NetUtils.wrapException(NetUtils.java:772)
at org.apache.hadoop.ipc.Client.call(Client.java:1472)
at org.apache.hadoop.ipc.Client.call(Client.java:1399)
at
org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.invoke(ProtobufRpcEngine.java:232)
at com.sun.proxy.$Proxy9.getDelegationToken(Unknown Source)
at
org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolTranslatorPB.getDelegationToken(ClientNamenodeProtocolTranslatorPB.java:909)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
On Tue, May 5, 2015 at 5:59 PM, Birender Saini <bs...@hortonworks.com>
wrote:
> Are those principals in KDC?
>
> Try to get kerberos ticket for that principal using the same key tab
> that's in config.
>
>
>
> -------- Original message --------
> From: Nishanth S <ch...@gmail.com>
> Date:05/05/2015 6:33 PM (GMT-05:00)
> To: user@hadoop.apache.org
> Subject: Re: Help with Kerberos
>
> Thanks Chris.I checked the configuration files and see that these
> properties are in place.I am trying to connect to hadoop cluster in linux
> from a windows machine.
>
>
>
>
> On Tue, May 5, 2015 at 1:25 PM, Chris Nauroth <cn...@hortonworks.com>
> wrote:
>
>> Hello Nishanth,
>>
>> The job submission was unable to identify the principal to use for renewal
>> of HDFS delegation tokens. Delegation tokens are subject to a soft
>> expiration, which can be extended by a renewer requesting an extension
>> from the NameNode periodically. In the context of MapReduce running on
>> YARN, the ResourceManager acts as the delegation token renewer.
>>
>> Specifically, this exception likely indicates that your configuration is
>> missing either (or both) of the properties yarn.resourcemanager.address
>> and yarn.resourcemanager.principal.
>>
>> <property>
>> <description>The address of the applications manager interface in the
>> RM.</description>
>> <name>yarn.resourcemanager.address</name>
>> <value>${yarn.resourcemanager.hostname}:8032</value>
>> </property>
>>
>> <property>
>> <description>The Kerberos principal for the resource
>> manager.</description>
>> <name>yarn.resourcemanager.principal</name>
>> </property>
>>
>> You might want to review the secure mode documentation. It has more
>> details on this and other relevant configuration properties.
>>
>>
>> http://hadoop.apache.org/docs/r2.7.0/hadoop-project-dist/hadoop-common/Secu
>> reMode.html
>> <http://hadoop.apache.org/docs/r2.7.0/hadoop-project-dist/hadoop-common/SecureMode.html>
>>
>> I hope this helps.
>>
>> --Chris Nauroth
>>
>>
>> From: Nishanth S <ch...@gmail.com>
>> Reply-To: "user@hadoop.apache.org" <us...@hadoop.apache.org>
>> Date: Tuesday, May 5, 2015 at 12:09 PM
>> To: "user@hadoop.apache.org" <us...@hadoop.apache.org>
>> Subject: Help with Kerberos
>>
>>
>> Hello ,
>>
>> I am getting the below exception when trying to run a simple map reduce
>> from a machine outside the hadoop cluster.This is the kerberos code I am
>> using.Please help
>>
>>
>> org.apache.hadoop.conf.Configuration conf = new
>> org.apache.hadoop.conf.Configuration();
>> conf.set("hadoop.security.authentication", "Kerberos");
>> UserGroupInformation.setConfiguration(conf);
>> try{
>> UserGroupInformation.loginUserFromKeytab("abc@domain.com",
>> "C:\\work\\documents\\devkeytab\\abc.keytab");
>>
>>
>>
>> Exception in thread "main" java.io.IOException: Can't get Master Kerberos
>> principal for use as renewer
>> at
>>
>> org.apache.hadoop.mapreduce.security.TokenCache.obtainTokensForNamenodesInt
>> ernal(TokenCache.java:116)
>> at
>>
>> org.apache.hadoop.mapreduce.security.TokenCache.obtainTokensForNamenodesInt
>> ernal(TokenCache.java:100)
>> at
>>
>> org.apache.hadoop.mapreduce.security.TokenCache.obtainTokensForNamenodes(To
>> kenCache.java:80)
>> at
>>
>> org.apache.hadoop.mapreduce.lib.output.FileOutputFormat.checkOutputSpecs(Fi
>> leOutputFormat.java:142)
>> at
>> org.apache.hadoop.mapreduce.JobSubmitter.checkSpecs(JobSubmitter.java:562)
>> at
>>
>> org.apache.hadoop.mapreduce.JobSubmitter.submitJobInternal(JobSubmitter.jav
>> a:432)
>> at org.apache.hadoop.mapreduce.Job$10.run(Job.java:1296)
>> at org.apache.hadoop.mapreduce.Job$10.run(Job.java:1293)
>> at java.security.AccessController.doPrivileged(Native Method)
>> at javax.security.auth.Subject.doAs(Unknown Source)
>> at
>>
>> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.j
>> ava:1628)
>> at org.apache.hadoop.mapreduce.Job.submit(Job.java:1293)
>> at org.apache.hadoop.mapreduce.Job.waitForCompletion(Job.java:1314)
>>
>>
>
Re: Help with Kerberos
Posted by Nishanth S <ch...@gmail.com>.
I am getting the below error now after adding all configuration
files(hdfs-site.xml, yarn files,etc) to the classpath of my client
project.The login is succesful but when I try to access hdfs the below is
error is encountered.I see many posts having the same issue but did not se
any resolution.Any help is appreciated.
Exception in thread "main" java.io.IOException: Failed on local exception:
java.io.IOException: java.lang.IllegalArgumentException:
Server has invalid Kerberos principal: hdfs/abc.domain.com@DOMAIN.COM; Host
Details : local host is: "xyz/10.a.b.c"; destination host is: "
bc.domain.com.domain.com":8020;
at org.apache.hadoop.net.NetUtils.wrapException(NetUtils.java:772)
at org.apache.hadoop.ipc.Client.call(Client.java:1472)
at org.apache.hadoop.ipc.Client.call(Client.java:1399)
at
org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.invoke(ProtobufRpcEngine.java:232)
at com.sun.proxy.$Proxy9.getDelegationToken(Unknown Source)
at
org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolTranslatorPB.getDelegationToken(ClientNamenodeProtocolTranslatorPB.java:909)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
On Tue, May 5, 2015 at 5:59 PM, Birender Saini <bs...@hortonworks.com>
wrote:
> Are those principals in KDC?
>
> Try to get kerberos ticket for that principal using the same key tab
> that's in config.
>
>
>
> -------- Original message --------
> From: Nishanth S <ch...@gmail.com>
> Date:05/05/2015 6:33 PM (GMT-05:00)
> To: user@hadoop.apache.org
> Subject: Re: Help with Kerberos
>
> Thanks Chris.I checked the configuration files and see that these
> properties are in place.I am trying to connect to hadoop cluster in linux
> from a windows machine.
>
>
>
>
> On Tue, May 5, 2015 at 1:25 PM, Chris Nauroth <cn...@hortonworks.com>
> wrote:
>
>> Hello Nishanth,
>>
>> The job submission was unable to identify the principal to use for renewal
>> of HDFS delegation tokens. Delegation tokens are subject to a soft
>> expiration, which can be extended by a renewer requesting an extension
>> from the NameNode periodically. In the context of MapReduce running on
>> YARN, the ResourceManager acts as the delegation token renewer.
>>
>> Specifically, this exception likely indicates that your configuration is
>> missing either (or both) of the properties yarn.resourcemanager.address
>> and yarn.resourcemanager.principal.
>>
>> <property>
>> <description>The address of the applications manager interface in the
>> RM.</description>
>> <name>yarn.resourcemanager.address</name>
>> <value>${yarn.resourcemanager.hostname}:8032</value>
>> </property>
>>
>> <property>
>> <description>The Kerberos principal for the resource
>> manager.</description>
>> <name>yarn.resourcemanager.principal</name>
>> </property>
>>
>> You might want to review the secure mode documentation. It has more
>> details on this and other relevant configuration properties.
>>
>>
>> http://hadoop.apache.org/docs/r2.7.0/hadoop-project-dist/hadoop-common/Secu
>> reMode.html
>> <http://hadoop.apache.org/docs/r2.7.0/hadoop-project-dist/hadoop-common/SecureMode.html>
>>
>> I hope this helps.
>>
>> --Chris Nauroth
>>
>>
>> From: Nishanth S <ch...@gmail.com>
>> Reply-To: "user@hadoop.apache.org" <us...@hadoop.apache.org>
>> Date: Tuesday, May 5, 2015 at 12:09 PM
>> To: "user@hadoop.apache.org" <us...@hadoop.apache.org>
>> Subject: Help with Kerberos
>>
>>
>> Hello ,
>>
>> I am getting the below exception when trying to run a simple map reduce
>> from a machine outside the hadoop cluster.This is the kerberos code I am
>> using.Please help
>>
>>
>> org.apache.hadoop.conf.Configuration conf = new
>> org.apache.hadoop.conf.Configuration();
>> conf.set("hadoop.security.authentication", "Kerberos");
>> UserGroupInformation.setConfiguration(conf);
>> try{
>> UserGroupInformation.loginUserFromKeytab("abc@domain.com",
>> "C:\\work\\documents\\devkeytab\\abc.keytab");
>>
>>
>>
>> Exception in thread "main" java.io.IOException: Can't get Master Kerberos
>> principal for use as renewer
>> at
>>
>> org.apache.hadoop.mapreduce.security.TokenCache.obtainTokensForNamenodesInt
>> ernal(TokenCache.java:116)
>> at
>>
>> org.apache.hadoop.mapreduce.security.TokenCache.obtainTokensForNamenodesInt
>> ernal(TokenCache.java:100)
>> at
>>
>> org.apache.hadoop.mapreduce.security.TokenCache.obtainTokensForNamenodes(To
>> kenCache.java:80)
>> at
>>
>> org.apache.hadoop.mapreduce.lib.output.FileOutputFormat.checkOutputSpecs(Fi
>> leOutputFormat.java:142)
>> at
>> org.apache.hadoop.mapreduce.JobSubmitter.checkSpecs(JobSubmitter.java:562)
>> at
>>
>> org.apache.hadoop.mapreduce.JobSubmitter.submitJobInternal(JobSubmitter.jav
>> a:432)
>> at org.apache.hadoop.mapreduce.Job$10.run(Job.java:1296)
>> at org.apache.hadoop.mapreduce.Job$10.run(Job.java:1293)
>> at java.security.AccessController.doPrivileged(Native Method)
>> at javax.security.auth.Subject.doAs(Unknown Source)
>> at
>>
>> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.j
>> ava:1628)
>> at org.apache.hadoop.mapreduce.Job.submit(Job.java:1293)
>> at org.apache.hadoop.mapreduce.Job.waitForCompletion(Job.java:1314)
>>
>>
>
RE: Help with Kerberos
Posted by Birender Saini <bs...@hortonworks.com>.
Are those principals in KDC?
Try to get kerberos ticket for that principal using the same key tab that's in config.
-------- Original message --------
From: Nishanth S <ch...@gmail.com>
Date:05/05/2015 6:33 PM (GMT-05:00)
To: user@hadoop.apache.org
Subject: Re: Help with Kerberos
Thanks Chris.I checked the configuration files and see that these properties are in place.I am trying to connect to hadoop cluster in linux from a windows machine.
On Tue, May 5, 2015 at 1:25 PM, Chris Nauroth <cn...@hortonworks.com>> wrote:
Hello Nishanth,
The job submission was unable to identify the principal to use for renewal
of HDFS delegation tokens. Delegation tokens are subject to a soft
expiration, which can be extended by a renewer requesting an extension
from the NameNode periodically. In the context of MapReduce running on
YARN, the ResourceManager acts as the delegation token renewer.
Specifically, this exception likely indicates that your configuration is
missing either (or both) of the properties yarn.resourcemanager.address
and yarn.resourcemanager.principal.
<property>
<description>The address of the applications manager interface in the
RM.</description>
<name>yarn.resourcemanager.address</name>
<value>${yarn.resourcemanager.hostname}:8032</value>
</property>
<property>
<description>The Kerberos principal for the resource
manager.</description>
<name>yarn.resourcemanager.principal</name>
</property>
You might want to review the secure mode documentation. It has more
details on this and other relevant configuration properties.
http://hadoop.apache.org/docs/r2.7.0/hadoop-project-dist/hadoop-common/Secu
reMode.html
I hope this helps.
--Chris Nauroth
From: Nishanth S <ch...@gmail.com>>
Reply-To: "user@hadoop.apache.org<ma...@hadoop.apache.org>" <us...@hadoop.apache.org>>
Date: Tuesday, May 5, 2015 at 12:09 PM
To: "user@hadoop.apache.org<ma...@hadoop.apache.org>" <us...@hadoop.apache.org>>
Subject: Help with Kerberos
Hello ,
I am getting the below exception when trying to run a simple map reduce
from a machine outside the hadoop cluster.This is the kerberos code I am
using.Please help
org.apache.hadoop.conf.Configuration conf = new
org.apache.hadoop.conf.Configuration();
conf.set("hadoop.security.authentication", "Kerberos");
UserGroupInformation.setConfiguration(conf);
try{
UserGroupInformation.loginUserFromKeytab("abc@domain.com<ma...@domain.com>",
"C:\\work\\documents\\devkeytab\\abc.keytab");
Exception in thread "main" java.io.IOException: Can't get Master Kerberos
principal for use as renewer
at
org.apache.hadoop.mapreduce.security.TokenCache.obtainTokensForNamenodesInt
ernal(TokenCache.java:116)
at
org.apache.hadoop.mapreduce.security.TokenCache.obtainTokensForNamenodesInt
ernal(TokenCache.java:100)
at
org.apache.hadoop.mapreduce.security.TokenCache.obtainTokensForNamenodes(To
kenCache.java:80)
at
org.apache.hadoop.mapreduce.lib.output.FileOutputFormat.checkOutputSpecs(Fi
leOutputFormat.java:142)
at
org.apache.hadoop.mapreduce.JobSubmitter.checkSpecs(JobSubmitter.java:562)
at
org.apache.hadoop.mapreduce.JobSubmitter.submitJobInternal(JobSubmitter.jav
a:432)
at org.apache.hadoop.mapreduce.Job$10.run(Job.java:1296)
at org.apache.hadoop.mapreduce.Job$10.run(Job.java:1293)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Unknown Source)
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.j
ava:1628)
at org.apache.hadoop.mapreduce.Job.submit(Job.java:1293)
at org.apache.hadoop.mapreduce.Job.waitForCompletion(Job.java:1314)
RE: Help with Kerberos
Posted by Birender Saini <bs...@hortonworks.com>.
Are those principals in KDC?
Try to get kerberos ticket for that principal using the same key tab that's in config.
-------- Original message --------
From: Nishanth S <ch...@gmail.com>
Date:05/05/2015 6:33 PM (GMT-05:00)
To: user@hadoop.apache.org
Subject: Re: Help with Kerberos
Thanks Chris.I checked the configuration files and see that these properties are in place.I am trying to connect to hadoop cluster in linux from a windows machine.
On Tue, May 5, 2015 at 1:25 PM, Chris Nauroth <cn...@hortonworks.com>> wrote:
Hello Nishanth,
The job submission was unable to identify the principal to use for renewal
of HDFS delegation tokens. Delegation tokens are subject to a soft
expiration, which can be extended by a renewer requesting an extension
from the NameNode periodically. In the context of MapReduce running on
YARN, the ResourceManager acts as the delegation token renewer.
Specifically, this exception likely indicates that your configuration is
missing either (or both) of the properties yarn.resourcemanager.address
and yarn.resourcemanager.principal.
<property>
<description>The address of the applications manager interface in the
RM.</description>
<name>yarn.resourcemanager.address</name>
<value>${yarn.resourcemanager.hostname}:8032</value>
</property>
<property>
<description>The Kerberos principal for the resource
manager.</description>
<name>yarn.resourcemanager.principal</name>
</property>
You might want to review the secure mode documentation. It has more
details on this and other relevant configuration properties.
http://hadoop.apache.org/docs/r2.7.0/hadoop-project-dist/hadoop-common/Secu
reMode.html
I hope this helps.
--Chris Nauroth
From: Nishanth S <ch...@gmail.com>>
Reply-To: "user@hadoop.apache.org<ma...@hadoop.apache.org>" <us...@hadoop.apache.org>>
Date: Tuesday, May 5, 2015 at 12:09 PM
To: "user@hadoop.apache.org<ma...@hadoop.apache.org>" <us...@hadoop.apache.org>>
Subject: Help with Kerberos
Hello ,
I am getting the below exception when trying to run a simple map reduce
from a machine outside the hadoop cluster.This is the kerberos code I am
using.Please help
org.apache.hadoop.conf.Configuration conf = new
org.apache.hadoop.conf.Configuration();
conf.set("hadoop.security.authentication", "Kerberos");
UserGroupInformation.setConfiguration(conf);
try{
UserGroupInformation.loginUserFromKeytab("abc@domain.com<ma...@domain.com>",
"C:\\work\\documents\\devkeytab\\abc.keytab");
Exception in thread "main" java.io.IOException: Can't get Master Kerberos
principal for use as renewer
at
org.apache.hadoop.mapreduce.security.TokenCache.obtainTokensForNamenodesInt
ernal(TokenCache.java:116)
at
org.apache.hadoop.mapreduce.security.TokenCache.obtainTokensForNamenodesInt
ernal(TokenCache.java:100)
at
org.apache.hadoop.mapreduce.security.TokenCache.obtainTokensForNamenodes(To
kenCache.java:80)
at
org.apache.hadoop.mapreduce.lib.output.FileOutputFormat.checkOutputSpecs(Fi
leOutputFormat.java:142)
at
org.apache.hadoop.mapreduce.JobSubmitter.checkSpecs(JobSubmitter.java:562)
at
org.apache.hadoop.mapreduce.JobSubmitter.submitJobInternal(JobSubmitter.jav
a:432)
at org.apache.hadoop.mapreduce.Job$10.run(Job.java:1296)
at org.apache.hadoop.mapreduce.Job$10.run(Job.java:1293)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Unknown Source)
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.j
ava:1628)
at org.apache.hadoop.mapreduce.Job.submit(Job.java:1293)
at org.apache.hadoop.mapreduce.Job.waitForCompletion(Job.java:1314)
RE: Help with Kerberos
Posted by Birender Saini <bs...@hortonworks.com>.
Are those principals in KDC?
Try to get kerberos ticket for that principal using the same key tab that's in config.
-------- Original message --------
From: Nishanth S <ch...@gmail.com>
Date:05/05/2015 6:33 PM (GMT-05:00)
To: user@hadoop.apache.org
Subject: Re: Help with Kerberos
Thanks Chris.I checked the configuration files and see that these properties are in place.I am trying to connect to hadoop cluster in linux from a windows machine.
On Tue, May 5, 2015 at 1:25 PM, Chris Nauroth <cn...@hortonworks.com>> wrote:
Hello Nishanth,
The job submission was unable to identify the principal to use for renewal
of HDFS delegation tokens. Delegation tokens are subject to a soft
expiration, which can be extended by a renewer requesting an extension
from the NameNode periodically. In the context of MapReduce running on
YARN, the ResourceManager acts as the delegation token renewer.
Specifically, this exception likely indicates that your configuration is
missing either (or both) of the properties yarn.resourcemanager.address
and yarn.resourcemanager.principal.
<property>
<description>The address of the applications manager interface in the
RM.</description>
<name>yarn.resourcemanager.address</name>
<value>${yarn.resourcemanager.hostname}:8032</value>
</property>
<property>
<description>The Kerberos principal for the resource
manager.</description>
<name>yarn.resourcemanager.principal</name>
</property>
You might want to review the secure mode documentation. It has more
details on this and other relevant configuration properties.
http://hadoop.apache.org/docs/r2.7.0/hadoop-project-dist/hadoop-common/Secu
reMode.html
I hope this helps.
--Chris Nauroth
From: Nishanth S <ch...@gmail.com>>
Reply-To: "user@hadoop.apache.org<ma...@hadoop.apache.org>" <us...@hadoop.apache.org>>
Date: Tuesday, May 5, 2015 at 12:09 PM
To: "user@hadoop.apache.org<ma...@hadoop.apache.org>" <us...@hadoop.apache.org>>
Subject: Help with Kerberos
Hello ,
I am getting the below exception when trying to run a simple map reduce
from a machine outside the hadoop cluster.This is the kerberos code I am
using.Please help
org.apache.hadoop.conf.Configuration conf = new
org.apache.hadoop.conf.Configuration();
conf.set("hadoop.security.authentication", "Kerberos");
UserGroupInformation.setConfiguration(conf);
try{
UserGroupInformation.loginUserFromKeytab("abc@domain.com<ma...@domain.com>",
"C:\\work\\documents\\devkeytab\\abc.keytab");
Exception in thread "main" java.io.IOException: Can't get Master Kerberos
principal for use as renewer
at
org.apache.hadoop.mapreduce.security.TokenCache.obtainTokensForNamenodesInt
ernal(TokenCache.java:116)
at
org.apache.hadoop.mapreduce.security.TokenCache.obtainTokensForNamenodesInt
ernal(TokenCache.java:100)
at
org.apache.hadoop.mapreduce.security.TokenCache.obtainTokensForNamenodes(To
kenCache.java:80)
at
org.apache.hadoop.mapreduce.lib.output.FileOutputFormat.checkOutputSpecs(Fi
leOutputFormat.java:142)
at
org.apache.hadoop.mapreduce.JobSubmitter.checkSpecs(JobSubmitter.java:562)
at
org.apache.hadoop.mapreduce.JobSubmitter.submitJobInternal(JobSubmitter.jav
a:432)
at org.apache.hadoop.mapreduce.Job$10.run(Job.java:1296)
at org.apache.hadoop.mapreduce.Job$10.run(Job.java:1293)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Unknown Source)
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.j
ava:1628)
at org.apache.hadoop.mapreduce.Job.submit(Job.java:1293)
at org.apache.hadoop.mapreduce.Job.waitForCompletion(Job.java:1314)
RE: Help with Kerberos
Posted by Birender Saini <bs...@hortonworks.com>.
Are those principals in KDC?
Try to get kerberos ticket for that principal using the same key tab that's in config.
-------- Original message --------
From: Nishanth S <ch...@gmail.com>
Date:05/05/2015 6:33 PM (GMT-05:00)
To: user@hadoop.apache.org
Subject: Re: Help with Kerberos
Thanks Chris.I checked the configuration files and see that these properties are in place.I am trying to connect to hadoop cluster in linux from a windows machine.
On Tue, May 5, 2015 at 1:25 PM, Chris Nauroth <cn...@hortonworks.com>> wrote:
Hello Nishanth,
The job submission was unable to identify the principal to use for renewal
of HDFS delegation tokens. Delegation tokens are subject to a soft
expiration, which can be extended by a renewer requesting an extension
from the NameNode periodically. In the context of MapReduce running on
YARN, the ResourceManager acts as the delegation token renewer.
Specifically, this exception likely indicates that your configuration is
missing either (or both) of the properties yarn.resourcemanager.address
and yarn.resourcemanager.principal.
<property>
<description>The address of the applications manager interface in the
RM.</description>
<name>yarn.resourcemanager.address</name>
<value>${yarn.resourcemanager.hostname}:8032</value>
</property>
<property>
<description>The Kerberos principal for the resource
manager.</description>
<name>yarn.resourcemanager.principal</name>
</property>
You might want to review the secure mode documentation. It has more
details on this and other relevant configuration properties.
http://hadoop.apache.org/docs/r2.7.0/hadoop-project-dist/hadoop-common/Secu
reMode.html
I hope this helps.
--Chris Nauroth
From: Nishanth S <ch...@gmail.com>>
Reply-To: "user@hadoop.apache.org<ma...@hadoop.apache.org>" <us...@hadoop.apache.org>>
Date: Tuesday, May 5, 2015 at 12:09 PM
To: "user@hadoop.apache.org<ma...@hadoop.apache.org>" <us...@hadoop.apache.org>>
Subject: Help with Kerberos
Hello ,
I am getting the below exception when trying to run a simple map reduce
from a machine outside the hadoop cluster.This is the kerberos code I am
using.Please help
org.apache.hadoop.conf.Configuration conf = new
org.apache.hadoop.conf.Configuration();
conf.set("hadoop.security.authentication", "Kerberos");
UserGroupInformation.setConfiguration(conf);
try{
UserGroupInformation.loginUserFromKeytab("abc@domain.com<ma...@domain.com>",
"C:\\work\\documents\\devkeytab\\abc.keytab");
Exception in thread "main" java.io.IOException: Can't get Master Kerberos
principal for use as renewer
at
org.apache.hadoop.mapreduce.security.TokenCache.obtainTokensForNamenodesInt
ernal(TokenCache.java:116)
at
org.apache.hadoop.mapreduce.security.TokenCache.obtainTokensForNamenodesInt
ernal(TokenCache.java:100)
at
org.apache.hadoop.mapreduce.security.TokenCache.obtainTokensForNamenodes(To
kenCache.java:80)
at
org.apache.hadoop.mapreduce.lib.output.FileOutputFormat.checkOutputSpecs(Fi
leOutputFormat.java:142)
at
org.apache.hadoop.mapreduce.JobSubmitter.checkSpecs(JobSubmitter.java:562)
at
org.apache.hadoop.mapreduce.JobSubmitter.submitJobInternal(JobSubmitter.jav
a:432)
at org.apache.hadoop.mapreduce.Job$10.run(Job.java:1296)
at org.apache.hadoop.mapreduce.Job$10.run(Job.java:1293)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Unknown Source)
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.j
ava:1628)
at org.apache.hadoop.mapreduce.Job.submit(Job.java:1293)
at org.apache.hadoop.mapreduce.Job.waitForCompletion(Job.java:1314)
Re: Help with Kerberos
Posted by Nishanth S <ch...@gmail.com>.
Thanks Chris.I checked the configuration files and see that these
properties are in place.I am trying to connect to hadoop cluster in linux
from a windows machine.
On Tue, May 5, 2015 at 1:25 PM, Chris Nauroth <cn...@hortonworks.com>
wrote:
> Hello Nishanth,
>
> The job submission was unable to identify the principal to use for renewal
> of HDFS delegation tokens. Delegation tokens are subject to a soft
> expiration, which can be extended by a renewer requesting an extension
> from the NameNode periodically. In the context of MapReduce running on
> YARN, the ResourceManager acts as the delegation token renewer.
>
> Specifically, this exception likely indicates that your configuration is
> missing either (or both) of the properties yarn.resourcemanager.address
> and yarn.resourcemanager.principal.
>
> <property>
> <description>The address of the applications manager interface in the
> RM.</description>
> <name>yarn.resourcemanager.address</name>
> <value>${yarn.resourcemanager.hostname}:8032</value>
> </property>
>
> <property>
> <description>The Kerberos principal for the resource
> manager.</description>
> <name>yarn.resourcemanager.principal</name>
> </property>
>
> You might want to review the secure mode documentation. It has more
> details on this and other relevant configuration properties.
>
> http://hadoop.apache.org/docs/r2.7.0/hadoop-project-dist/hadoop-common/Secu
> reMode.html
>
> I hope this helps.
>
> --Chris Nauroth
>
>
> From: Nishanth S <ch...@gmail.com>
> Reply-To: "user@hadoop.apache.org" <us...@hadoop.apache.org>
> Date: Tuesday, May 5, 2015 at 12:09 PM
> To: "user@hadoop.apache.org" <us...@hadoop.apache.org>
> Subject: Help with Kerberos
>
>
> Hello ,
>
> I am getting the below exception when trying to run a simple map reduce
> from a machine outside the hadoop cluster.This is the kerberos code I am
> using.Please help
>
>
> org.apache.hadoop.conf.Configuration conf = new
> org.apache.hadoop.conf.Configuration();
> conf.set("hadoop.security.authentication", "Kerberos");
> UserGroupInformation.setConfiguration(conf);
> try{
> UserGroupInformation.loginUserFromKeytab("abc@domain.com",
> "C:\\work\\documents\\devkeytab\\abc.keytab");
>
>
>
> Exception in thread "main" java.io.IOException: Can't get Master Kerberos
> principal for use as renewer
> at
> org.apache.hadoop.mapreduce.security.TokenCache.obtainTokensForNamenodesInt
> ernal(TokenCache.java:116)
> at
> org.apache.hadoop.mapreduce.security.TokenCache.obtainTokensForNamenodesInt
> ernal(TokenCache.java:100)
> at
> org.apache.hadoop.mapreduce.security.TokenCache.obtainTokensForNamenodes(To
> kenCache.java:80)
> at
> org.apache.hadoop.mapreduce.lib.output.FileOutputFormat.checkOutputSpecs(Fi
> leOutputFormat.java:142)
> at
> org.apache.hadoop.mapreduce.JobSubmitter.checkSpecs(JobSubmitter.java:562)
> at
> org.apache.hadoop.mapreduce.JobSubmitter.submitJobInternal(JobSubmitter.jav
> a:432)
> at org.apache.hadoop.mapreduce.Job$10.run(Job.java:1296)
> at org.apache.hadoop.mapreduce.Job$10.run(Job.java:1293)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Unknown Source)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.j
> ava:1628)
> at org.apache.hadoop.mapreduce.Job.submit(Job.java:1293)
> at org.apache.hadoop.mapreduce.Job.waitForCompletion(Job.java:1314)
>
>
Re: Help with Kerberos
Posted by Nishanth S <ch...@gmail.com>.
Thanks Chris.I checked the configuration files and see that these
properties are in place.I am trying to connect to hadoop cluster in linux
from a windows machine.
On Tue, May 5, 2015 at 1:25 PM, Chris Nauroth <cn...@hortonworks.com>
wrote:
> Hello Nishanth,
>
> The job submission was unable to identify the principal to use for renewal
> of HDFS delegation tokens. Delegation tokens are subject to a soft
> expiration, which can be extended by a renewer requesting an extension
> from the NameNode periodically. In the context of MapReduce running on
> YARN, the ResourceManager acts as the delegation token renewer.
>
> Specifically, this exception likely indicates that your configuration is
> missing either (or both) of the properties yarn.resourcemanager.address
> and yarn.resourcemanager.principal.
>
> <property>
> <description>The address of the applications manager interface in the
> RM.</description>
> <name>yarn.resourcemanager.address</name>
> <value>${yarn.resourcemanager.hostname}:8032</value>
> </property>
>
> <property>
> <description>The Kerberos principal for the resource
> manager.</description>
> <name>yarn.resourcemanager.principal</name>
> </property>
>
> You might want to review the secure mode documentation. It has more
> details on this and other relevant configuration properties.
>
> http://hadoop.apache.org/docs/r2.7.0/hadoop-project-dist/hadoop-common/Secu
> reMode.html
>
> I hope this helps.
>
> --Chris Nauroth
>
>
> From: Nishanth S <ch...@gmail.com>
> Reply-To: "user@hadoop.apache.org" <us...@hadoop.apache.org>
> Date: Tuesday, May 5, 2015 at 12:09 PM
> To: "user@hadoop.apache.org" <us...@hadoop.apache.org>
> Subject: Help with Kerberos
>
>
> Hello ,
>
> I am getting the below exception when trying to run a simple map reduce
> from a machine outside the hadoop cluster.This is the kerberos code I am
> using.Please help
>
>
> org.apache.hadoop.conf.Configuration conf = new
> org.apache.hadoop.conf.Configuration();
> conf.set("hadoop.security.authentication", "Kerberos");
> UserGroupInformation.setConfiguration(conf);
> try{
> UserGroupInformation.loginUserFromKeytab("abc@domain.com",
> "C:\\work\\documents\\devkeytab\\abc.keytab");
>
>
>
> Exception in thread "main" java.io.IOException: Can't get Master Kerberos
> principal for use as renewer
> at
> org.apache.hadoop.mapreduce.security.TokenCache.obtainTokensForNamenodesInt
> ernal(TokenCache.java:116)
> at
> org.apache.hadoop.mapreduce.security.TokenCache.obtainTokensForNamenodesInt
> ernal(TokenCache.java:100)
> at
> org.apache.hadoop.mapreduce.security.TokenCache.obtainTokensForNamenodes(To
> kenCache.java:80)
> at
> org.apache.hadoop.mapreduce.lib.output.FileOutputFormat.checkOutputSpecs(Fi
> leOutputFormat.java:142)
> at
> org.apache.hadoop.mapreduce.JobSubmitter.checkSpecs(JobSubmitter.java:562)
> at
> org.apache.hadoop.mapreduce.JobSubmitter.submitJobInternal(JobSubmitter.jav
> a:432)
> at org.apache.hadoop.mapreduce.Job$10.run(Job.java:1296)
> at org.apache.hadoop.mapreduce.Job$10.run(Job.java:1293)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Unknown Source)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.j
> ava:1628)
> at org.apache.hadoop.mapreduce.Job.submit(Job.java:1293)
> at org.apache.hadoop.mapreduce.Job.waitForCompletion(Job.java:1314)
>
>
Re: Help with Kerberos
Posted by Nishanth S <ch...@gmail.com>.
Thanks Chris.I checked the configuration files and see that these
properties are in place.I am trying to connect to hadoop cluster in linux
from a windows machine.
On Tue, May 5, 2015 at 1:25 PM, Chris Nauroth <cn...@hortonworks.com>
wrote:
> Hello Nishanth,
>
> The job submission was unable to identify the principal to use for renewal
> of HDFS delegation tokens. Delegation tokens are subject to a soft
> expiration, which can be extended by a renewer requesting an extension
> from the NameNode periodically. In the context of MapReduce running on
> YARN, the ResourceManager acts as the delegation token renewer.
>
> Specifically, this exception likely indicates that your configuration is
> missing either (or both) of the properties yarn.resourcemanager.address
> and yarn.resourcemanager.principal.
>
> <property>
> <description>The address of the applications manager interface in the
> RM.</description>
> <name>yarn.resourcemanager.address</name>
> <value>${yarn.resourcemanager.hostname}:8032</value>
> </property>
>
> <property>
> <description>The Kerberos principal for the resource
> manager.</description>
> <name>yarn.resourcemanager.principal</name>
> </property>
>
> You might want to review the secure mode documentation. It has more
> details on this and other relevant configuration properties.
>
> http://hadoop.apache.org/docs/r2.7.0/hadoop-project-dist/hadoop-common/Secu
> reMode.html
>
> I hope this helps.
>
> --Chris Nauroth
>
>
> From: Nishanth S <ch...@gmail.com>
> Reply-To: "user@hadoop.apache.org" <us...@hadoop.apache.org>
> Date: Tuesday, May 5, 2015 at 12:09 PM
> To: "user@hadoop.apache.org" <us...@hadoop.apache.org>
> Subject: Help with Kerberos
>
>
> Hello ,
>
> I am getting the below exception when trying to run a simple map reduce
> from a machine outside the hadoop cluster.This is the kerberos code I am
> using.Please help
>
>
> org.apache.hadoop.conf.Configuration conf = new
> org.apache.hadoop.conf.Configuration();
> conf.set("hadoop.security.authentication", "Kerberos");
> UserGroupInformation.setConfiguration(conf);
> try{
> UserGroupInformation.loginUserFromKeytab("abc@domain.com",
> "C:\\work\\documents\\devkeytab\\abc.keytab");
>
>
>
> Exception in thread "main" java.io.IOException: Can't get Master Kerberos
> principal for use as renewer
> at
> org.apache.hadoop.mapreduce.security.TokenCache.obtainTokensForNamenodesInt
> ernal(TokenCache.java:116)
> at
> org.apache.hadoop.mapreduce.security.TokenCache.obtainTokensForNamenodesInt
> ernal(TokenCache.java:100)
> at
> org.apache.hadoop.mapreduce.security.TokenCache.obtainTokensForNamenodes(To
> kenCache.java:80)
> at
> org.apache.hadoop.mapreduce.lib.output.FileOutputFormat.checkOutputSpecs(Fi
> leOutputFormat.java:142)
> at
> org.apache.hadoop.mapreduce.JobSubmitter.checkSpecs(JobSubmitter.java:562)
> at
> org.apache.hadoop.mapreduce.JobSubmitter.submitJobInternal(JobSubmitter.jav
> a:432)
> at org.apache.hadoop.mapreduce.Job$10.run(Job.java:1296)
> at org.apache.hadoop.mapreduce.Job$10.run(Job.java:1293)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Unknown Source)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.j
> ava:1628)
> at org.apache.hadoop.mapreduce.Job.submit(Job.java:1293)
> at org.apache.hadoop.mapreduce.Job.waitForCompletion(Job.java:1314)
>
>
Re: Help with Kerberos
Posted by Nishanth S <ch...@gmail.com>.
Thanks Chris.I checked the configuration files and see that these
properties are in place.I am trying to connect to hadoop cluster in linux
from a windows machine.
On Tue, May 5, 2015 at 1:25 PM, Chris Nauroth <cn...@hortonworks.com>
wrote:
> Hello Nishanth,
>
> The job submission was unable to identify the principal to use for renewal
> of HDFS delegation tokens. Delegation tokens are subject to a soft
> expiration, which can be extended by a renewer requesting an extension
> from the NameNode periodically. In the context of MapReduce running on
> YARN, the ResourceManager acts as the delegation token renewer.
>
> Specifically, this exception likely indicates that your configuration is
> missing either (or both) of the properties yarn.resourcemanager.address
> and yarn.resourcemanager.principal.
>
> <property>
> <description>The address of the applications manager interface in the
> RM.</description>
> <name>yarn.resourcemanager.address</name>
> <value>${yarn.resourcemanager.hostname}:8032</value>
> </property>
>
> <property>
> <description>The Kerberos principal for the resource
> manager.</description>
> <name>yarn.resourcemanager.principal</name>
> </property>
>
> You might want to review the secure mode documentation. It has more
> details on this and other relevant configuration properties.
>
> http://hadoop.apache.org/docs/r2.7.0/hadoop-project-dist/hadoop-common/Secu
> reMode.html
>
> I hope this helps.
>
> --Chris Nauroth
>
>
> From: Nishanth S <ch...@gmail.com>
> Reply-To: "user@hadoop.apache.org" <us...@hadoop.apache.org>
> Date: Tuesday, May 5, 2015 at 12:09 PM
> To: "user@hadoop.apache.org" <us...@hadoop.apache.org>
> Subject: Help with Kerberos
>
>
> Hello ,
>
> I am getting the below exception when trying to run a simple map reduce
> from a machine outside the hadoop cluster.This is the kerberos code I am
> using.Please help
>
>
> org.apache.hadoop.conf.Configuration conf = new
> org.apache.hadoop.conf.Configuration();
> conf.set("hadoop.security.authentication", "Kerberos");
> UserGroupInformation.setConfiguration(conf);
> try{
> UserGroupInformation.loginUserFromKeytab("abc@domain.com",
> "C:\\work\\documents\\devkeytab\\abc.keytab");
>
>
>
> Exception in thread "main" java.io.IOException: Can't get Master Kerberos
> principal for use as renewer
> at
> org.apache.hadoop.mapreduce.security.TokenCache.obtainTokensForNamenodesInt
> ernal(TokenCache.java:116)
> at
> org.apache.hadoop.mapreduce.security.TokenCache.obtainTokensForNamenodesInt
> ernal(TokenCache.java:100)
> at
> org.apache.hadoop.mapreduce.security.TokenCache.obtainTokensForNamenodes(To
> kenCache.java:80)
> at
> org.apache.hadoop.mapreduce.lib.output.FileOutputFormat.checkOutputSpecs(Fi
> leOutputFormat.java:142)
> at
> org.apache.hadoop.mapreduce.JobSubmitter.checkSpecs(JobSubmitter.java:562)
> at
> org.apache.hadoop.mapreduce.JobSubmitter.submitJobInternal(JobSubmitter.jav
> a:432)
> at org.apache.hadoop.mapreduce.Job$10.run(Job.java:1296)
> at org.apache.hadoop.mapreduce.Job$10.run(Job.java:1293)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Unknown Source)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.j
> ava:1628)
> at org.apache.hadoop.mapreduce.Job.submit(Job.java:1293)
> at org.apache.hadoop.mapreduce.Job.waitForCompletion(Job.java:1314)
>
>
Re: Help with Kerberos
Posted by Chris Nauroth <cn...@hortonworks.com>.
Hello Nishanth,
The job submission was unable to identify the principal to use for renewal
of HDFS delegation tokens. Delegation tokens are subject to a soft
expiration, which can be extended by a renewer requesting an extension
from the NameNode periodically. In the context of MapReduce running on
YARN, the ResourceManager acts as the delegation token renewer.
Specifically, this exception likely indicates that your configuration is
missing either (or both) of the properties yarn.resourcemanager.address
and yarn.resourcemanager.principal.
<property>
<description>The address of the applications manager interface in the
RM.</description>
<name>yarn.resourcemanager.address</name>
<value>${yarn.resourcemanager.hostname}:8032</value>
</property>
<property>
<description>The Kerberos principal for the resource
manager.</description>
<name>yarn.resourcemanager.principal</name>
</property>
You might want to review the secure mode documentation. It has more
details on this and other relevant configuration properties.
http://hadoop.apache.org/docs/r2.7.0/hadoop-project-dist/hadoop-common/Secu
reMode.html
I hope this helps.
--Chris Nauroth
From: Nishanth S <ch...@gmail.com>
Reply-To: "user@hadoop.apache.org" <us...@hadoop.apache.org>
Date: Tuesday, May 5, 2015 at 12:09 PM
To: "user@hadoop.apache.org" <us...@hadoop.apache.org>
Subject: Help with Kerberos
Hello ,
I am getting the below exception when trying to run a simple map reduce
from a machine outside the hadoop cluster.This is the kerberos code I am
using.Please help
org.apache.hadoop.conf.Configuration conf = new
org.apache.hadoop.conf.Configuration();
conf.set("hadoop.security.authentication", "Kerberos");
UserGroupInformation.setConfiguration(conf);
try{
UserGroupInformation.loginUserFromKeytab("abc@domain.com",
"C:\\work\\documents\\devkeytab\\abc.keytab");
Exception in thread "main" java.io.IOException: Can't get Master Kerberos
principal for use as renewer
at
org.apache.hadoop.mapreduce.security.TokenCache.obtainTokensForNamenodesInt
ernal(TokenCache.java:116)
at
org.apache.hadoop.mapreduce.security.TokenCache.obtainTokensForNamenodesInt
ernal(TokenCache.java:100)
at
org.apache.hadoop.mapreduce.security.TokenCache.obtainTokensForNamenodes(To
kenCache.java:80)
at
org.apache.hadoop.mapreduce.lib.output.FileOutputFormat.checkOutputSpecs(Fi
leOutputFormat.java:142)
at
org.apache.hadoop.mapreduce.JobSubmitter.checkSpecs(JobSubmitter.java:562)
at
org.apache.hadoop.mapreduce.JobSubmitter.submitJobInternal(JobSubmitter.jav
a:432)
at org.apache.hadoop.mapreduce.Job$10.run(Job.java:1296)
at org.apache.hadoop.mapreduce.Job$10.run(Job.java:1293)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Unknown Source)
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.j
ava:1628)
at org.apache.hadoop.mapreduce.Job.submit(Job.java:1293)
at org.apache.hadoop.mapreduce.Job.waitForCompletion(Job.java:1314)
Re: Help with Kerberos
Posted by Chris Nauroth <cn...@hortonworks.com>.
Hello Nishanth,
The job submission was unable to identify the principal to use for renewal
of HDFS delegation tokens. Delegation tokens are subject to a soft
expiration, which can be extended by a renewer requesting an extension
from the NameNode periodically. In the context of MapReduce running on
YARN, the ResourceManager acts as the delegation token renewer.
Specifically, this exception likely indicates that your configuration is
missing either (or both) of the properties yarn.resourcemanager.address
and yarn.resourcemanager.principal.
<property>
<description>The address of the applications manager interface in the
RM.</description>
<name>yarn.resourcemanager.address</name>
<value>${yarn.resourcemanager.hostname}:8032</value>
</property>
<property>
<description>The Kerberos principal for the resource
manager.</description>
<name>yarn.resourcemanager.principal</name>
</property>
You might want to review the secure mode documentation. It has more
details on this and other relevant configuration properties.
http://hadoop.apache.org/docs/r2.7.0/hadoop-project-dist/hadoop-common/Secu
reMode.html
I hope this helps.
--Chris Nauroth
From: Nishanth S <ch...@gmail.com>
Reply-To: "user@hadoop.apache.org" <us...@hadoop.apache.org>
Date: Tuesday, May 5, 2015 at 12:09 PM
To: "user@hadoop.apache.org" <us...@hadoop.apache.org>
Subject: Help with Kerberos
Hello ,
I am getting the below exception when trying to run a simple map reduce
from a machine outside the hadoop cluster.This is the kerberos code I am
using.Please help
org.apache.hadoop.conf.Configuration conf = new
org.apache.hadoop.conf.Configuration();
conf.set("hadoop.security.authentication", "Kerberos");
UserGroupInformation.setConfiguration(conf);
try{
UserGroupInformation.loginUserFromKeytab("abc@domain.com",
"C:\\work\\documents\\devkeytab\\abc.keytab");
Exception in thread "main" java.io.IOException: Can't get Master Kerberos
principal for use as renewer
at
org.apache.hadoop.mapreduce.security.TokenCache.obtainTokensForNamenodesInt
ernal(TokenCache.java:116)
at
org.apache.hadoop.mapreduce.security.TokenCache.obtainTokensForNamenodesInt
ernal(TokenCache.java:100)
at
org.apache.hadoop.mapreduce.security.TokenCache.obtainTokensForNamenodes(To
kenCache.java:80)
at
org.apache.hadoop.mapreduce.lib.output.FileOutputFormat.checkOutputSpecs(Fi
leOutputFormat.java:142)
at
org.apache.hadoop.mapreduce.JobSubmitter.checkSpecs(JobSubmitter.java:562)
at
org.apache.hadoop.mapreduce.JobSubmitter.submitJobInternal(JobSubmitter.jav
a:432)
at org.apache.hadoop.mapreduce.Job$10.run(Job.java:1296)
at org.apache.hadoop.mapreduce.Job$10.run(Job.java:1293)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Unknown Source)
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.j
ava:1628)
at org.apache.hadoop.mapreduce.Job.submit(Job.java:1293)
at org.apache.hadoop.mapreduce.Job.waitForCompletion(Job.java:1314)
Re: Help with Kerberos
Posted by Chris Nauroth <cn...@hortonworks.com>.
Hello Nishanth,
The job submission was unable to identify the principal to use for renewal
of HDFS delegation tokens. Delegation tokens are subject to a soft
expiration, which can be extended by a renewer requesting an extension
from the NameNode periodically. In the context of MapReduce running on
YARN, the ResourceManager acts as the delegation token renewer.
Specifically, this exception likely indicates that your configuration is
missing either (or both) of the properties yarn.resourcemanager.address
and yarn.resourcemanager.principal.
<property>
<description>The address of the applications manager interface in the
RM.</description>
<name>yarn.resourcemanager.address</name>
<value>${yarn.resourcemanager.hostname}:8032</value>
</property>
<property>
<description>The Kerberos principal for the resource
manager.</description>
<name>yarn.resourcemanager.principal</name>
</property>
You might want to review the secure mode documentation. It has more
details on this and other relevant configuration properties.
http://hadoop.apache.org/docs/r2.7.0/hadoop-project-dist/hadoop-common/Secu
reMode.html
I hope this helps.
--Chris Nauroth
From: Nishanth S <ch...@gmail.com>
Reply-To: "user@hadoop.apache.org" <us...@hadoop.apache.org>
Date: Tuesday, May 5, 2015 at 12:09 PM
To: "user@hadoop.apache.org" <us...@hadoop.apache.org>
Subject: Help with Kerberos
Hello ,
I am getting the below exception when trying to run a simple map reduce
from a machine outside the hadoop cluster.This is the kerberos code I am
using.Please help
org.apache.hadoop.conf.Configuration conf = new
org.apache.hadoop.conf.Configuration();
conf.set("hadoop.security.authentication", "Kerberos");
UserGroupInformation.setConfiguration(conf);
try{
UserGroupInformation.loginUserFromKeytab("abc@domain.com",
"C:\\work\\documents\\devkeytab\\abc.keytab");
Exception in thread "main" java.io.IOException: Can't get Master Kerberos
principal for use as renewer
at
org.apache.hadoop.mapreduce.security.TokenCache.obtainTokensForNamenodesInt
ernal(TokenCache.java:116)
at
org.apache.hadoop.mapreduce.security.TokenCache.obtainTokensForNamenodesInt
ernal(TokenCache.java:100)
at
org.apache.hadoop.mapreduce.security.TokenCache.obtainTokensForNamenodes(To
kenCache.java:80)
at
org.apache.hadoop.mapreduce.lib.output.FileOutputFormat.checkOutputSpecs(Fi
leOutputFormat.java:142)
at
org.apache.hadoop.mapreduce.JobSubmitter.checkSpecs(JobSubmitter.java:562)
at
org.apache.hadoop.mapreduce.JobSubmitter.submitJobInternal(JobSubmitter.jav
a:432)
at org.apache.hadoop.mapreduce.Job$10.run(Job.java:1296)
at org.apache.hadoop.mapreduce.Job$10.run(Job.java:1293)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Unknown Source)
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.j
ava:1628)
at org.apache.hadoop.mapreduce.Job.submit(Job.java:1293)
at org.apache.hadoop.mapreduce.Job.waitForCompletion(Job.java:1314)
Re: Help with Kerberos
Posted by Chris Nauroth <cn...@hortonworks.com>.
Hello Nishanth,
The job submission was unable to identify the principal to use for renewal
of HDFS delegation tokens. Delegation tokens are subject to a soft
expiration, which can be extended by a renewer requesting an extension
from the NameNode periodically. In the context of MapReduce running on
YARN, the ResourceManager acts as the delegation token renewer.
Specifically, this exception likely indicates that your configuration is
missing either (or both) of the properties yarn.resourcemanager.address
and yarn.resourcemanager.principal.
<property>
<description>The address of the applications manager interface in the
RM.</description>
<name>yarn.resourcemanager.address</name>
<value>${yarn.resourcemanager.hostname}:8032</value>
</property>
<property>
<description>The Kerberos principal for the resource
manager.</description>
<name>yarn.resourcemanager.principal</name>
</property>
You might want to review the secure mode documentation. It has more
details on this and other relevant configuration properties.
http://hadoop.apache.org/docs/r2.7.0/hadoop-project-dist/hadoop-common/Secu
reMode.html
I hope this helps.
--Chris Nauroth
From: Nishanth S <ch...@gmail.com>
Reply-To: "user@hadoop.apache.org" <us...@hadoop.apache.org>
Date: Tuesday, May 5, 2015 at 12:09 PM
To: "user@hadoop.apache.org" <us...@hadoop.apache.org>
Subject: Help with Kerberos
Hello ,
I am getting the below exception when trying to run a simple map reduce
from a machine outside the hadoop cluster.This is the kerberos code I am
using.Please help
org.apache.hadoop.conf.Configuration conf = new
org.apache.hadoop.conf.Configuration();
conf.set("hadoop.security.authentication", "Kerberos");
UserGroupInformation.setConfiguration(conf);
try{
UserGroupInformation.loginUserFromKeytab("abc@domain.com",
"C:\\work\\documents\\devkeytab\\abc.keytab");
Exception in thread "main" java.io.IOException: Can't get Master Kerberos
principal for use as renewer
at
org.apache.hadoop.mapreduce.security.TokenCache.obtainTokensForNamenodesInt
ernal(TokenCache.java:116)
at
org.apache.hadoop.mapreduce.security.TokenCache.obtainTokensForNamenodesInt
ernal(TokenCache.java:100)
at
org.apache.hadoop.mapreduce.security.TokenCache.obtainTokensForNamenodes(To
kenCache.java:80)
at
org.apache.hadoop.mapreduce.lib.output.FileOutputFormat.checkOutputSpecs(Fi
leOutputFormat.java:142)
at
org.apache.hadoop.mapreduce.JobSubmitter.checkSpecs(JobSubmitter.java:562)
at
org.apache.hadoop.mapreduce.JobSubmitter.submitJobInternal(JobSubmitter.jav
a:432)
at org.apache.hadoop.mapreduce.Job$10.run(Job.java:1296)
at org.apache.hadoop.mapreduce.Job$10.run(Job.java:1293)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Unknown Source)
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.j
ava:1628)
at org.apache.hadoop.mapreduce.Job.submit(Job.java:1293)
at org.apache.hadoop.mapreduce.Job.waitForCompletion(Job.java:1314)