You are viewing a plain text version of this content. The canonical link for it is here.
Posted to mapreduce-user@hadoop.apache.org by "Vishwakarma, Chhaya" <Ch...@Thinkbiganalytics.com> on 2015/11/02 11:45:37 UTC
Authenticating to Kerberos enabled Hadoop cluster using Java
I have Kerberos enabled Hadoop cluster, I need to perform HDFS operations using JAVA code.
I have keytab file and username can someone please suggest how can I autheticate to Kerberos using JAVA code?
Regards,
Chhaya
RE: Authenticating to Kerberos enabled Hadoop cluster using Java
Posted by Daniel Schulz <da...@hotmail.com>.
Hi Chhaya,
Thank you for asking straight away. At my latest project (HDP with Kerberos) we used authenticated users; as they submitted Java jobs MR or Spark reached through their user name and looked for a valid Kerberos Principal. Upon file access, HDFS does so. So your user needs to have a valid Kerberos Ticket from `kinit` or a keytab file.
Of course: you could issue `kinit` via JNI; but this looks rather like a hack. If a service needs credentials, I'd rather use Keystores for that. On Hadoop with Kerberos, your user's Kerberos Principal is the default way to go.
Alternatively, you may use this parameter chain before start up:
SET KINIT=c:\Program Files\Java\jdk1.6.0_20\bin\kinit
SET KEYTAB=d:\webapps\app1\WEB-INF\serverhostname.keytab
SET SPN=HTTP/app1.intranet.company.com@REALM002.COMPANY.COM
SET KRB5INI=d:\webapps\app1\WEB-INF\krb5.ini
"%KINIT%" -k -t "%KEYTAB%" %SPN% -J-Dsun.security.krb5.debug=true "-J-Djava.security.krb5.conf=%KRB5INI%"source: michael-behrendt.net/blog/2011/01/kerberos-keytab-uberprufen-mit-java-boardmitteln (German)
Kind regards, Daniel.
From: Chhaya.Vishwakarma@Thinkbiganalytics.comTo: user@hadoop.apache.org
Subject: Authenticating to Kerberos enabled Hadoop cluster using Java
Date: Mon, 2 Nov 2015 10:45:37 +0000
I have Kerberos enabled Hadoop cluster, I need to perform HDFS operations using JAVA code.
I have keytab file and username can someone please suggest how can I autheticate to Kerberos using JAVA code?
Regards,
Chhaya
RE: Authenticating to Kerberos enabled Hadoop cluster using Java
Posted by Daniel Schulz <da...@hotmail.com>.
Hi Chhaya,
Thank you for asking straight away. At my latest project (HDP with Kerberos) we used authenticated users; as they submitted Java jobs MR or Spark reached through their user name and looked for a valid Kerberos Principal. Upon file access, HDFS does so. So your user needs to have a valid Kerberos Ticket from `kinit` or a keytab file.
Of course: you could issue `kinit` via JNI; but this looks rather like a hack. If a service needs credentials, I'd rather use Keystores for that. On Hadoop with Kerberos, your user's Kerberos Principal is the default way to go.
Alternatively, you may use this parameter chain before start up:
SET KINIT=c:\Program Files\Java\jdk1.6.0_20\bin\kinit
SET KEYTAB=d:\webapps\app1\WEB-INF\serverhostname.keytab
SET SPN=HTTP/app1.intranet.company.com@REALM002.COMPANY.COM
SET KRB5INI=d:\webapps\app1\WEB-INF\krb5.ini
"%KINIT%" -k -t "%KEYTAB%" %SPN% -J-Dsun.security.krb5.debug=true "-J-Djava.security.krb5.conf=%KRB5INI%"source: michael-behrendt.net/blog/2011/01/kerberos-keytab-uberprufen-mit-java-boardmitteln (German)
Kind regards, Daniel.
From: Chhaya.Vishwakarma@Thinkbiganalytics.comTo: user@hadoop.apache.org
Subject: Authenticating to Kerberos enabled Hadoop cluster using Java
Date: Mon, 2 Nov 2015 10:45:37 +0000
I have Kerberos enabled Hadoop cluster, I need to perform HDFS operations using JAVA code.
I have keytab file and username can someone please suggest how can I autheticate to Kerberos using JAVA code?
Regards,
Chhaya
RE: Authenticating to Kerberos enabled Hadoop cluster using Java
Posted by "Vishwakarma, Chhaya" <Ch...@Thinkbiganalytics.com>.
Thanks to all I got it working ☺
From: Vinayakumar B [mailto:vinayakumarb.apache@gmail.com]
Sent: 02 November 2015 20:12
To: user@hadoop.apache.org
Subject: RE: Authenticating to Kerberos enabled Hadoop cluster using Java
For simplicity You just can copy HADOOP_CONF_DIR from one of the cluster's machine. And place it in class path of the client program.
Principal you are using to login is the client principal. It can be different from server principal.
-Vinay
On Nov 2, 2015 22:37, "Vishwakarma, Chhaya" <Ch...@thinkbiganalytics.com>> wrote:
Thanks Subroto
dfs.namenode.kerberos.principal=hdfs/hadoop@HADOOP.COM<ma...@HADOOP.COM> this is principal specified here “UserGroupInformation.loginUserFromKeytab(String, String) “ or it is something different ?
From: Subroto Sanyal [mailto:ssanyal@datameer.com<ma...@datameer.com>]
Sent: 02 November 2015 19:54
To: user@hadoop.apache.org<ma...@hadoop.apache.org>
Subject: Re: Authenticating to Kerberos enabled Hadoop cluster using Java
Hi Chhaya,
Few configuration you need to set:
hadoop.security.authentication=kerberos
hadoop.security.authorization=true
dfs.namenode.kerberos.principal=hdfs/hadoop@HADOOP.COM<ma...@HADOOP.COM>
fs.defaultFS=hdfs://host:port
Further you need to use org.apache.hadoop.security.UserGroupInformation.loginUserFromKeytab(String, String) as suggested in one of the trailing mail.
Cheers,
Subroto Sanyal
On Mon, Nov 2, 2015 at 3:13 PM, Vishwakarma, Chhaya <Ch...@thinkbiganalytics.com>> wrote:
Code is successfully authenticating to Kerberos but when I try to run any hdfs command I get error as "Failed to specify server's kerberos principal name"
Can somebody please assist me on this?
Sent from my android device.
-----Original Message-----
From: andreina j <an...@huawei.com>>
To: "user@hadoop.apache.org<ma...@hadoop.apache.org>" <us...@hadoop.apache.org>>
Sent: Mon, 02 Nov 2015 4:57 pm
Subject: RE: Authenticating to Kerberos enabled Hadoop cluster using Java
Hi Chhaya,
Please find below sample code .
System.setProperty("java.security.krb5.conf", "D:\\data\\Desktop\\cluster-test\\krb5.conf");
// Login using keytab if have access to keytab. else
UserGroupInformation.loginUserFromKeytab("hdfs/hadoop@HADOOP.COM<ma...@HADOOP.COM>",
" D:\\data\\Desktop\\cluster-test\\conf\\hdfs.keytab");
Note: Above 2 lines should be at the beginning in your application.
Regards
Andreina J
From: Vishwakarma, Chhaya [mailto:Chhaya.Vishwakarma@Thinkbiganalytics.com<ma...@Thinkbiganalytics.com>]
Sent: 02 November 2015 PM 04:20
To: user@hadoop.apache.org<ma...@hadoop.apache.org>
Subject: RE: Authenticating to Kerberos enabled Hadoop cluster using Java
Thanks Niranjan It would be great if you can share a sample code if any?
From: Niranjan Subramanian [mailto:niranjan@webaction.com]
Sent: 02 November 2015 16:18
To: user@hadoop.apache.org<ma...@hadoop.apache.org>
Subject: Re: Authenticating to Kerberos enabled Hadoop cluster using Java
Hi Chhaya,
You can use the UserGroupInformation class from org.apache.hadoop.security package.
Specifically following 2 methods of that class
UserGroupInformation.setConfiguration(hdfsConfiguration);
UserGroupInformation.loginUserFromKeytab(principal, keytabPath);
Regards,
Niranjan
On 02-Nov-2015, at 4:15 pm, Vishwakarma, Chhaya <Ch...@Thinkbiganalytics.com>> wrote:
I have Kerberos enabled Hadoop cluster, I need to perform HDFS operations using JAVA code.
I have keytab file and username can someone please suggest how can I autheticate to Kerberos using JAVA code?
Regards,
Chhaya
RE: Authenticating to Kerberos enabled Hadoop cluster using Java
Posted by "Vishwakarma, Chhaya" <Ch...@Thinkbiganalytics.com>.
Thanks to all I got it working ☺
From: Vinayakumar B [mailto:vinayakumarb.apache@gmail.com]
Sent: 02 November 2015 20:12
To: user@hadoop.apache.org
Subject: RE: Authenticating to Kerberos enabled Hadoop cluster using Java
For simplicity You just can copy HADOOP_CONF_DIR from one of the cluster's machine. And place it in class path of the client program.
Principal you are using to login is the client principal. It can be different from server principal.
-Vinay
On Nov 2, 2015 22:37, "Vishwakarma, Chhaya" <Ch...@thinkbiganalytics.com>> wrote:
Thanks Subroto
dfs.namenode.kerberos.principal=hdfs/hadoop@HADOOP.COM<ma...@HADOOP.COM> this is principal specified here “UserGroupInformation.loginUserFromKeytab(String, String) “ or it is something different ?
From: Subroto Sanyal [mailto:ssanyal@datameer.com<ma...@datameer.com>]
Sent: 02 November 2015 19:54
To: user@hadoop.apache.org<ma...@hadoop.apache.org>
Subject: Re: Authenticating to Kerberos enabled Hadoop cluster using Java
Hi Chhaya,
Few configuration you need to set:
hadoop.security.authentication=kerberos
hadoop.security.authorization=true
dfs.namenode.kerberos.principal=hdfs/hadoop@HADOOP.COM<ma...@HADOOP.COM>
fs.defaultFS=hdfs://host:port
Further you need to use org.apache.hadoop.security.UserGroupInformation.loginUserFromKeytab(String, String) as suggested in one of the trailing mail.
Cheers,
Subroto Sanyal
On Mon, Nov 2, 2015 at 3:13 PM, Vishwakarma, Chhaya <Ch...@thinkbiganalytics.com>> wrote:
Code is successfully authenticating to Kerberos but when I try to run any hdfs command I get error as "Failed to specify server's kerberos principal name"
Can somebody please assist me on this?
Sent from my android device.
-----Original Message-----
From: andreina j <an...@huawei.com>>
To: "user@hadoop.apache.org<ma...@hadoop.apache.org>" <us...@hadoop.apache.org>>
Sent: Mon, 02 Nov 2015 4:57 pm
Subject: RE: Authenticating to Kerberos enabled Hadoop cluster using Java
Hi Chhaya,
Please find below sample code .
System.setProperty("java.security.krb5.conf", "D:\\data\\Desktop\\cluster-test\\krb5.conf");
// Login using keytab if have access to keytab. else
UserGroupInformation.loginUserFromKeytab("hdfs/hadoop@HADOOP.COM<ma...@HADOOP.COM>",
" D:\\data\\Desktop\\cluster-test\\conf\\hdfs.keytab");
Note: Above 2 lines should be at the beginning in your application.
Regards
Andreina J
From: Vishwakarma, Chhaya [mailto:Chhaya.Vishwakarma@Thinkbiganalytics.com<ma...@Thinkbiganalytics.com>]
Sent: 02 November 2015 PM 04:20
To: user@hadoop.apache.org<ma...@hadoop.apache.org>
Subject: RE: Authenticating to Kerberos enabled Hadoop cluster using Java
Thanks Niranjan It would be great if you can share a sample code if any?
From: Niranjan Subramanian [mailto:niranjan@webaction.com]
Sent: 02 November 2015 16:18
To: user@hadoop.apache.org<ma...@hadoop.apache.org>
Subject: Re: Authenticating to Kerberos enabled Hadoop cluster using Java
Hi Chhaya,
You can use the UserGroupInformation class from org.apache.hadoop.security package.
Specifically following 2 methods of that class
UserGroupInformation.setConfiguration(hdfsConfiguration);
UserGroupInformation.loginUserFromKeytab(principal, keytabPath);
Regards,
Niranjan
On 02-Nov-2015, at 4:15 pm, Vishwakarma, Chhaya <Ch...@Thinkbiganalytics.com>> wrote:
I have Kerberos enabled Hadoop cluster, I need to perform HDFS operations using JAVA code.
I have keytab file and username can someone please suggest how can I autheticate to Kerberos using JAVA code?
Regards,
Chhaya
RE: Authenticating to Kerberos enabled Hadoop cluster using Java
Posted by "Vishwakarma, Chhaya" <Ch...@Thinkbiganalytics.com>.
Thanks to all I got it working ☺
From: Vinayakumar B [mailto:vinayakumarb.apache@gmail.com]
Sent: 02 November 2015 20:12
To: user@hadoop.apache.org
Subject: RE: Authenticating to Kerberos enabled Hadoop cluster using Java
For simplicity You just can copy HADOOP_CONF_DIR from one of the cluster's machine. And place it in class path of the client program.
Principal you are using to login is the client principal. It can be different from server principal.
-Vinay
On Nov 2, 2015 22:37, "Vishwakarma, Chhaya" <Ch...@thinkbiganalytics.com>> wrote:
Thanks Subroto
dfs.namenode.kerberos.principal=hdfs/hadoop@HADOOP.COM<ma...@HADOOP.COM> this is principal specified here “UserGroupInformation.loginUserFromKeytab(String, String) “ or it is something different ?
From: Subroto Sanyal [mailto:ssanyal@datameer.com<ma...@datameer.com>]
Sent: 02 November 2015 19:54
To: user@hadoop.apache.org<ma...@hadoop.apache.org>
Subject: Re: Authenticating to Kerberos enabled Hadoop cluster using Java
Hi Chhaya,
Few configuration you need to set:
hadoop.security.authentication=kerberos
hadoop.security.authorization=true
dfs.namenode.kerberos.principal=hdfs/hadoop@HADOOP.COM<ma...@HADOOP.COM>
fs.defaultFS=hdfs://host:port
Further you need to use org.apache.hadoop.security.UserGroupInformation.loginUserFromKeytab(String, String) as suggested in one of the trailing mail.
Cheers,
Subroto Sanyal
On Mon, Nov 2, 2015 at 3:13 PM, Vishwakarma, Chhaya <Ch...@thinkbiganalytics.com>> wrote:
Code is successfully authenticating to Kerberos but when I try to run any hdfs command I get error as "Failed to specify server's kerberos principal name"
Can somebody please assist me on this?
Sent from my android device.
-----Original Message-----
From: andreina j <an...@huawei.com>>
To: "user@hadoop.apache.org<ma...@hadoop.apache.org>" <us...@hadoop.apache.org>>
Sent: Mon, 02 Nov 2015 4:57 pm
Subject: RE: Authenticating to Kerberos enabled Hadoop cluster using Java
Hi Chhaya,
Please find below sample code .
System.setProperty("java.security.krb5.conf", "D:\\data\\Desktop\\cluster-test\\krb5.conf");
// Login using keytab if have access to keytab. else
UserGroupInformation.loginUserFromKeytab("hdfs/hadoop@HADOOP.COM<ma...@HADOOP.COM>",
" D:\\data\\Desktop\\cluster-test\\conf\\hdfs.keytab");
Note: Above 2 lines should be at the beginning in your application.
Regards
Andreina J
From: Vishwakarma, Chhaya [mailto:Chhaya.Vishwakarma@Thinkbiganalytics.com<ma...@Thinkbiganalytics.com>]
Sent: 02 November 2015 PM 04:20
To: user@hadoop.apache.org<ma...@hadoop.apache.org>
Subject: RE: Authenticating to Kerberos enabled Hadoop cluster using Java
Thanks Niranjan It would be great if you can share a sample code if any?
From: Niranjan Subramanian [mailto:niranjan@webaction.com]
Sent: 02 November 2015 16:18
To: user@hadoop.apache.org<ma...@hadoop.apache.org>
Subject: Re: Authenticating to Kerberos enabled Hadoop cluster using Java
Hi Chhaya,
You can use the UserGroupInformation class from org.apache.hadoop.security package.
Specifically following 2 methods of that class
UserGroupInformation.setConfiguration(hdfsConfiguration);
UserGroupInformation.loginUserFromKeytab(principal, keytabPath);
Regards,
Niranjan
On 02-Nov-2015, at 4:15 pm, Vishwakarma, Chhaya <Ch...@Thinkbiganalytics.com>> wrote:
I have Kerberos enabled Hadoop cluster, I need to perform HDFS operations using JAVA code.
I have keytab file and username can someone please suggest how can I autheticate to Kerberos using JAVA code?
Regards,
Chhaya
RE: Authenticating to Kerberos enabled Hadoop cluster using Java
Posted by "Vishwakarma, Chhaya" <Ch...@Thinkbiganalytics.com>.
Thanks to all I got it working ☺
From: Vinayakumar B [mailto:vinayakumarb.apache@gmail.com]
Sent: 02 November 2015 20:12
To: user@hadoop.apache.org
Subject: RE: Authenticating to Kerberos enabled Hadoop cluster using Java
For simplicity You just can copy HADOOP_CONF_DIR from one of the cluster's machine. And place it in class path of the client program.
Principal you are using to login is the client principal. It can be different from server principal.
-Vinay
On Nov 2, 2015 22:37, "Vishwakarma, Chhaya" <Ch...@thinkbiganalytics.com>> wrote:
Thanks Subroto
dfs.namenode.kerberos.principal=hdfs/hadoop@HADOOP.COM<ma...@HADOOP.COM> this is principal specified here “UserGroupInformation.loginUserFromKeytab(String, String) “ or it is something different ?
From: Subroto Sanyal [mailto:ssanyal@datameer.com<ma...@datameer.com>]
Sent: 02 November 2015 19:54
To: user@hadoop.apache.org<ma...@hadoop.apache.org>
Subject: Re: Authenticating to Kerberos enabled Hadoop cluster using Java
Hi Chhaya,
Few configuration you need to set:
hadoop.security.authentication=kerberos
hadoop.security.authorization=true
dfs.namenode.kerberos.principal=hdfs/hadoop@HADOOP.COM<ma...@HADOOP.COM>
fs.defaultFS=hdfs://host:port
Further you need to use org.apache.hadoop.security.UserGroupInformation.loginUserFromKeytab(String, String) as suggested in one of the trailing mail.
Cheers,
Subroto Sanyal
On Mon, Nov 2, 2015 at 3:13 PM, Vishwakarma, Chhaya <Ch...@thinkbiganalytics.com>> wrote:
Code is successfully authenticating to Kerberos but when I try to run any hdfs command I get error as "Failed to specify server's kerberos principal name"
Can somebody please assist me on this?
Sent from my android device.
-----Original Message-----
From: andreina j <an...@huawei.com>>
To: "user@hadoop.apache.org<ma...@hadoop.apache.org>" <us...@hadoop.apache.org>>
Sent: Mon, 02 Nov 2015 4:57 pm
Subject: RE: Authenticating to Kerberos enabled Hadoop cluster using Java
Hi Chhaya,
Please find below sample code .
System.setProperty("java.security.krb5.conf", "D:\\data\\Desktop\\cluster-test\\krb5.conf");
// Login using keytab if have access to keytab. else
UserGroupInformation.loginUserFromKeytab("hdfs/hadoop@HADOOP.COM<ma...@HADOOP.COM>",
" D:\\data\\Desktop\\cluster-test\\conf\\hdfs.keytab");
Note: Above 2 lines should be at the beginning in your application.
Regards
Andreina J
From: Vishwakarma, Chhaya [mailto:Chhaya.Vishwakarma@Thinkbiganalytics.com<ma...@Thinkbiganalytics.com>]
Sent: 02 November 2015 PM 04:20
To: user@hadoop.apache.org<ma...@hadoop.apache.org>
Subject: RE: Authenticating to Kerberos enabled Hadoop cluster using Java
Thanks Niranjan It would be great if you can share a sample code if any?
From: Niranjan Subramanian [mailto:niranjan@webaction.com]
Sent: 02 November 2015 16:18
To: user@hadoop.apache.org<ma...@hadoop.apache.org>
Subject: Re: Authenticating to Kerberos enabled Hadoop cluster using Java
Hi Chhaya,
You can use the UserGroupInformation class from org.apache.hadoop.security package.
Specifically following 2 methods of that class
UserGroupInformation.setConfiguration(hdfsConfiguration);
UserGroupInformation.loginUserFromKeytab(principal, keytabPath);
Regards,
Niranjan
On 02-Nov-2015, at 4:15 pm, Vishwakarma, Chhaya <Ch...@Thinkbiganalytics.com>> wrote:
I have Kerberos enabled Hadoop cluster, I need to perform HDFS operations using JAVA code.
I have keytab file and username can someone please suggest how can I autheticate to Kerberos using JAVA code?
Regards,
Chhaya
RE: Authenticating to Kerberos enabled Hadoop cluster using Java
Posted by Vinayakumar B <vi...@gmail.com>.
For simplicity You just can copy HADOOP_CONF_DIR from one of the cluster's
machine. And place it in class path of the client program.
Principal you are using to login is the client principal. It can be
different from server principal.
-Vinay
On Nov 2, 2015 22:37, "Vishwakarma, Chhaya" <
Chhaya.Vishwakarma@thinkbiganalytics.com> wrote:
> Thanks Subroto
>
>
>
> dfs.namenode.kerberos.principal=hdfs/hadoop@HADOOP.COM this is principal
> specified here “UserGroupInformation.loginUserFromKeytab(String, String) “
> or it is something different ?
>
>
>
> *From:* Subroto Sanyal [mailto:ssanyal@datameer.com]
> *Sent:* 02 November 2015 19:54
> *To:* user@hadoop.apache.org
> *Subject:* Re: Authenticating to Kerberos enabled Hadoop cluster using
> Java
>
>
>
> Hi Chhaya,
>
>
>
> Few configuration you need to set:
>
> hadoop.security.authentication=kerberos
>
> hadoop.security.authorization=true
>
> dfs.namenode.kerberos.principal=hdfs/hadoop@HADOOP.COM
>
> fs.defaultFS=hdfs://host:port
>
> Further you need to
> use org.apache.hadoop.security.UserGroupInformation.loginUserFromKeytab(String,
> String) as suggested in one of the trailing mail.
>
>
>
> Cheers,
>
> Subroto Sanyal
>
>
>
> On Mon, Nov 2, 2015 at 3:13 PM, Vishwakarma, Chhaya <
> Chhaya.Vishwakarma@thinkbiganalytics.com> wrote:
>
> Code is successfully authenticating to Kerberos but when I try to run any
> hdfs command I get error as "Failed to specify server's kerberos principal
> name"
>
> Can somebody please assist me on this?
>
> Sent from my android device.
>
>
>
> -----Original Message-----
> From: andreina j <an...@huawei.com>
> To: "user@hadoop.apache.org" <us...@hadoop.apache.org>
> Sent: Mon, 02 Nov 2015 4:57 pm
> Subject: RE: Authenticating to Kerberos enabled Hadoop cluster using Java
>
> Hi Chhaya,
>
>
>
> Please find below sample code .
>
>
>
> System.setProperty("java.security.krb5.conf",
> "D:\\data\\Desktop\\cluster-test\\krb5.conf");
>
> // Login using keytab if have access to keytab. else
> UserGroupInformation.loginUserFromKeytab("hdfs/hadoop@HADOOP.COM",
> " D:\\data\\Desktop\\cluster-test\\conf\\hdfs.keytab");
>
>
>
> Note: Above 2 lines should be at the beginning in your application.
>
>
>
> Regards
>
> Andreina J
>
>
>
> *From:* Vishwakarma, Chhaya [mailto:
> Chhaya.Vishwakarma@Thinkbiganalytics.com]
> *Sent:* 02 November 2015 PM 04:20
> *To:* user@hadoop.apache.org
> *Subject:* RE: Authenticating to Kerberos enabled Hadoop cluster using
> Java
>
>
>
> Thanks Niranjan It would be great if you can share a sample code if any?
>
>
>
> *From:* Niranjan Subramanian [mailto:niranjan@webaction.com
> <ni...@webaction.com>]
> *Sent:* 02 November 2015 16:18
> *To:* user@hadoop.apache.org
> *Subject:* Re: Authenticating to Kerberos enabled Hadoop cluster using
> Java
>
>
>
> Hi Chhaya,
>
>
>
> You can use the UserGroupInformation class from org.apache.hadoop.security
> package.
>
>
>
> Specifically following 2 methods of that class
>
>
>
> UserGroupInformation.setConfiguration(hdfsConfiguration);
>
> UserGroupInformation.loginUserFromKeytab(principal, keytabPath);
>
>
>
> Regards,
>
> Niranjan
>
>
>
> On 02-Nov-2015, at 4:15 pm, Vishwakarma, Chhaya <
> Chhaya.Vishwakarma@Thinkbiganalytics.com> wrote:
>
>
>
> I have Kerberos enabled Hadoop cluster, I need to perform HDFS operations
> using JAVA code.
>
> I have keytab file and username can someone please suggest how can I
> autheticate to Kerberos using JAVA code?
>
> Regards,
>
> Chhaya
>
>
>
>
>
RE: Authenticating to Kerberos enabled Hadoop cluster using Java
Posted by Vinayakumar B <vi...@gmail.com>.
For simplicity You just can copy HADOOP_CONF_DIR from one of the cluster's
machine. And place it in class path of the client program.
Principal you are using to login is the client principal. It can be
different from server principal.
-Vinay
On Nov 2, 2015 22:37, "Vishwakarma, Chhaya" <
Chhaya.Vishwakarma@thinkbiganalytics.com> wrote:
> Thanks Subroto
>
>
>
> dfs.namenode.kerberos.principal=hdfs/hadoop@HADOOP.COM this is principal
> specified here “UserGroupInformation.loginUserFromKeytab(String, String) “
> or it is something different ?
>
>
>
> *From:* Subroto Sanyal [mailto:ssanyal@datameer.com]
> *Sent:* 02 November 2015 19:54
> *To:* user@hadoop.apache.org
> *Subject:* Re: Authenticating to Kerberos enabled Hadoop cluster using
> Java
>
>
>
> Hi Chhaya,
>
>
>
> Few configuration you need to set:
>
> hadoop.security.authentication=kerberos
>
> hadoop.security.authorization=true
>
> dfs.namenode.kerberos.principal=hdfs/hadoop@HADOOP.COM
>
> fs.defaultFS=hdfs://host:port
>
> Further you need to
> use org.apache.hadoop.security.UserGroupInformation.loginUserFromKeytab(String,
> String) as suggested in one of the trailing mail.
>
>
>
> Cheers,
>
> Subroto Sanyal
>
>
>
> On Mon, Nov 2, 2015 at 3:13 PM, Vishwakarma, Chhaya <
> Chhaya.Vishwakarma@thinkbiganalytics.com> wrote:
>
> Code is successfully authenticating to Kerberos but when I try to run any
> hdfs command I get error as "Failed to specify server's kerberos principal
> name"
>
> Can somebody please assist me on this?
>
> Sent from my android device.
>
>
>
> -----Original Message-----
> From: andreina j <an...@huawei.com>
> To: "user@hadoop.apache.org" <us...@hadoop.apache.org>
> Sent: Mon, 02 Nov 2015 4:57 pm
> Subject: RE: Authenticating to Kerberos enabled Hadoop cluster using Java
>
> Hi Chhaya,
>
>
>
> Please find below sample code .
>
>
>
> System.setProperty("java.security.krb5.conf",
> "D:\\data\\Desktop\\cluster-test\\krb5.conf");
>
> // Login using keytab if have access to keytab. else
> UserGroupInformation.loginUserFromKeytab("hdfs/hadoop@HADOOP.COM",
> " D:\\data\\Desktop\\cluster-test\\conf\\hdfs.keytab");
>
>
>
> Note: Above 2 lines should be at the beginning in your application.
>
>
>
> Regards
>
> Andreina J
>
>
>
> *From:* Vishwakarma, Chhaya [mailto:
> Chhaya.Vishwakarma@Thinkbiganalytics.com]
> *Sent:* 02 November 2015 PM 04:20
> *To:* user@hadoop.apache.org
> *Subject:* RE: Authenticating to Kerberos enabled Hadoop cluster using
> Java
>
>
>
> Thanks Niranjan It would be great if you can share a sample code if any?
>
>
>
> *From:* Niranjan Subramanian [mailto:niranjan@webaction.com
> <ni...@webaction.com>]
> *Sent:* 02 November 2015 16:18
> *To:* user@hadoop.apache.org
> *Subject:* Re: Authenticating to Kerberos enabled Hadoop cluster using
> Java
>
>
>
> Hi Chhaya,
>
>
>
> You can use the UserGroupInformation class from org.apache.hadoop.security
> package.
>
>
>
> Specifically following 2 methods of that class
>
>
>
> UserGroupInformation.setConfiguration(hdfsConfiguration);
>
> UserGroupInformation.loginUserFromKeytab(principal, keytabPath);
>
>
>
> Regards,
>
> Niranjan
>
>
>
> On 02-Nov-2015, at 4:15 pm, Vishwakarma, Chhaya <
> Chhaya.Vishwakarma@Thinkbiganalytics.com> wrote:
>
>
>
> I have Kerberos enabled Hadoop cluster, I need to perform HDFS operations
> using JAVA code.
>
> I have keytab file and username can someone please suggest how can I
> autheticate to Kerberos using JAVA code?
>
> Regards,
>
> Chhaya
>
>
>
>
>
RE: Authenticating to Kerberos enabled Hadoop cluster using Java
Posted by Vinayakumar B <vi...@gmail.com>.
For simplicity You just can copy HADOOP_CONF_DIR from one of the cluster's
machine. And place it in class path of the client program.
Principal you are using to login is the client principal. It can be
different from server principal.
-Vinay
On Nov 2, 2015 22:37, "Vishwakarma, Chhaya" <
Chhaya.Vishwakarma@thinkbiganalytics.com> wrote:
> Thanks Subroto
>
>
>
> dfs.namenode.kerberos.principal=hdfs/hadoop@HADOOP.COM this is principal
> specified here “UserGroupInformation.loginUserFromKeytab(String, String) “
> or it is something different ?
>
>
>
> *From:* Subroto Sanyal [mailto:ssanyal@datameer.com]
> *Sent:* 02 November 2015 19:54
> *To:* user@hadoop.apache.org
> *Subject:* Re: Authenticating to Kerberos enabled Hadoop cluster using
> Java
>
>
>
> Hi Chhaya,
>
>
>
> Few configuration you need to set:
>
> hadoop.security.authentication=kerberos
>
> hadoop.security.authorization=true
>
> dfs.namenode.kerberos.principal=hdfs/hadoop@HADOOP.COM
>
> fs.defaultFS=hdfs://host:port
>
> Further you need to
> use org.apache.hadoop.security.UserGroupInformation.loginUserFromKeytab(String,
> String) as suggested in one of the trailing mail.
>
>
>
> Cheers,
>
> Subroto Sanyal
>
>
>
> On Mon, Nov 2, 2015 at 3:13 PM, Vishwakarma, Chhaya <
> Chhaya.Vishwakarma@thinkbiganalytics.com> wrote:
>
> Code is successfully authenticating to Kerberos but when I try to run any
> hdfs command I get error as "Failed to specify server's kerberos principal
> name"
>
> Can somebody please assist me on this?
>
> Sent from my android device.
>
>
>
> -----Original Message-----
> From: andreina j <an...@huawei.com>
> To: "user@hadoop.apache.org" <us...@hadoop.apache.org>
> Sent: Mon, 02 Nov 2015 4:57 pm
> Subject: RE: Authenticating to Kerberos enabled Hadoop cluster using Java
>
> Hi Chhaya,
>
>
>
> Please find below sample code .
>
>
>
> System.setProperty("java.security.krb5.conf",
> "D:\\data\\Desktop\\cluster-test\\krb5.conf");
>
> // Login using keytab if have access to keytab. else
> UserGroupInformation.loginUserFromKeytab("hdfs/hadoop@HADOOP.COM",
> " D:\\data\\Desktop\\cluster-test\\conf\\hdfs.keytab");
>
>
>
> Note: Above 2 lines should be at the beginning in your application.
>
>
>
> Regards
>
> Andreina J
>
>
>
> *From:* Vishwakarma, Chhaya [mailto:
> Chhaya.Vishwakarma@Thinkbiganalytics.com]
> *Sent:* 02 November 2015 PM 04:20
> *To:* user@hadoop.apache.org
> *Subject:* RE: Authenticating to Kerberos enabled Hadoop cluster using
> Java
>
>
>
> Thanks Niranjan It would be great if you can share a sample code if any?
>
>
>
> *From:* Niranjan Subramanian [mailto:niranjan@webaction.com
> <ni...@webaction.com>]
> *Sent:* 02 November 2015 16:18
> *To:* user@hadoop.apache.org
> *Subject:* Re: Authenticating to Kerberos enabled Hadoop cluster using
> Java
>
>
>
> Hi Chhaya,
>
>
>
> You can use the UserGroupInformation class from org.apache.hadoop.security
> package.
>
>
>
> Specifically following 2 methods of that class
>
>
>
> UserGroupInformation.setConfiguration(hdfsConfiguration);
>
> UserGroupInformation.loginUserFromKeytab(principal, keytabPath);
>
>
>
> Regards,
>
> Niranjan
>
>
>
> On 02-Nov-2015, at 4:15 pm, Vishwakarma, Chhaya <
> Chhaya.Vishwakarma@Thinkbiganalytics.com> wrote:
>
>
>
> I have Kerberos enabled Hadoop cluster, I need to perform HDFS operations
> using JAVA code.
>
> I have keytab file and username can someone please suggest how can I
> autheticate to Kerberos using JAVA code?
>
> Regards,
>
> Chhaya
>
>
>
>
>
RE: Authenticating to Kerberos enabled Hadoop cluster using Java
Posted by Vinayakumar B <vi...@gmail.com>.
For simplicity You just can copy HADOOP_CONF_DIR from one of the cluster's
machine. And place it in class path of the client program.
Principal you are using to login is the client principal. It can be
different from server principal.
-Vinay
On Nov 2, 2015 22:37, "Vishwakarma, Chhaya" <
Chhaya.Vishwakarma@thinkbiganalytics.com> wrote:
> Thanks Subroto
>
>
>
> dfs.namenode.kerberos.principal=hdfs/hadoop@HADOOP.COM this is principal
> specified here “UserGroupInformation.loginUserFromKeytab(String, String) “
> or it is something different ?
>
>
>
> *From:* Subroto Sanyal [mailto:ssanyal@datameer.com]
> *Sent:* 02 November 2015 19:54
> *To:* user@hadoop.apache.org
> *Subject:* Re: Authenticating to Kerberos enabled Hadoop cluster using
> Java
>
>
>
> Hi Chhaya,
>
>
>
> Few configuration you need to set:
>
> hadoop.security.authentication=kerberos
>
> hadoop.security.authorization=true
>
> dfs.namenode.kerberos.principal=hdfs/hadoop@HADOOP.COM
>
> fs.defaultFS=hdfs://host:port
>
> Further you need to
> use org.apache.hadoop.security.UserGroupInformation.loginUserFromKeytab(String,
> String) as suggested in one of the trailing mail.
>
>
>
> Cheers,
>
> Subroto Sanyal
>
>
>
> On Mon, Nov 2, 2015 at 3:13 PM, Vishwakarma, Chhaya <
> Chhaya.Vishwakarma@thinkbiganalytics.com> wrote:
>
> Code is successfully authenticating to Kerberos but when I try to run any
> hdfs command I get error as "Failed to specify server's kerberos principal
> name"
>
> Can somebody please assist me on this?
>
> Sent from my android device.
>
>
>
> -----Original Message-----
> From: andreina j <an...@huawei.com>
> To: "user@hadoop.apache.org" <us...@hadoop.apache.org>
> Sent: Mon, 02 Nov 2015 4:57 pm
> Subject: RE: Authenticating to Kerberos enabled Hadoop cluster using Java
>
> Hi Chhaya,
>
>
>
> Please find below sample code .
>
>
>
> System.setProperty("java.security.krb5.conf",
> "D:\\data\\Desktop\\cluster-test\\krb5.conf");
>
> // Login using keytab if have access to keytab. else
> UserGroupInformation.loginUserFromKeytab("hdfs/hadoop@HADOOP.COM",
> " D:\\data\\Desktop\\cluster-test\\conf\\hdfs.keytab");
>
>
>
> Note: Above 2 lines should be at the beginning in your application.
>
>
>
> Regards
>
> Andreina J
>
>
>
> *From:* Vishwakarma, Chhaya [mailto:
> Chhaya.Vishwakarma@Thinkbiganalytics.com]
> *Sent:* 02 November 2015 PM 04:20
> *To:* user@hadoop.apache.org
> *Subject:* RE: Authenticating to Kerberos enabled Hadoop cluster using
> Java
>
>
>
> Thanks Niranjan It would be great if you can share a sample code if any?
>
>
>
> *From:* Niranjan Subramanian [mailto:niranjan@webaction.com
> <ni...@webaction.com>]
> *Sent:* 02 November 2015 16:18
> *To:* user@hadoop.apache.org
> *Subject:* Re: Authenticating to Kerberos enabled Hadoop cluster using
> Java
>
>
>
> Hi Chhaya,
>
>
>
> You can use the UserGroupInformation class from org.apache.hadoop.security
> package.
>
>
>
> Specifically following 2 methods of that class
>
>
>
> UserGroupInformation.setConfiguration(hdfsConfiguration);
>
> UserGroupInformation.loginUserFromKeytab(principal, keytabPath);
>
>
>
> Regards,
>
> Niranjan
>
>
>
> On 02-Nov-2015, at 4:15 pm, Vishwakarma, Chhaya <
> Chhaya.Vishwakarma@Thinkbiganalytics.com> wrote:
>
>
>
> I have Kerberos enabled Hadoop cluster, I need to perform HDFS operations
> using JAVA code.
>
> I have keytab file and username can someone please suggest how can I
> autheticate to Kerberos using JAVA code?
>
> Regards,
>
> Chhaya
>
>
>
>
>
RE: Authenticating to Kerberos enabled Hadoop cluster using Java
Posted by "Vishwakarma, Chhaya" <Ch...@Thinkbiganalytics.com>.
Thanks Subroto
dfs.namenode.kerberos.principal=hdfs/hadoop@HADOOP.COM<ma...@HADOOP.COM> this is principal specified here “UserGroupInformation.loginUserFromKeytab(String, String) “ or it is something different ?
From: Subroto Sanyal [mailto:ssanyal@datameer.com]
Sent: 02 November 2015 19:54
To: user@hadoop.apache.org
Subject: Re: Authenticating to Kerberos enabled Hadoop cluster using Java
Hi Chhaya,
Few configuration you need to set:
hadoop.security.authentication=kerberos
hadoop.security.authorization=true
dfs.namenode.kerberos.principal=hdfs/hadoop@HADOOP.COM<ma...@HADOOP.COM>
fs.defaultFS=hdfs://host:port
Further you need to use org.apache.hadoop.security.UserGroupInformation.loginUserFromKeytab(String, String) as suggested in one of the trailing mail.
Cheers,
Subroto Sanyal
On Mon, Nov 2, 2015 at 3:13 PM, Vishwakarma, Chhaya <Ch...@thinkbiganalytics.com>> wrote:
Code is successfully authenticating to Kerberos but when I try to run any hdfs command I get error as "Failed to specify server's kerberos principal name"
Can somebody please assist me on this?
Sent from my android device.
-----Original Message-----
From: andreina j <an...@huawei.com>>
To: "user@hadoop.apache.org<ma...@hadoop.apache.org>" <us...@hadoop.apache.org>>
Sent: Mon, 02 Nov 2015 4:57 pm
Subject: RE: Authenticating to Kerberos enabled Hadoop cluster using Java
Hi Chhaya,
Please find below sample code .
System.setProperty("java.security.krb5.conf", "D:\\data\\Desktop\\cluster-test\\krb5.conf");
// Login using keytab if have access to keytab. else
UserGroupInformation.loginUserFromKeytab("hdfs/hadoop@HADOOP.COM<ma...@HADOOP.COM>",
" D:\\data\\Desktop\\cluster-test\\conf\\hdfs.keytab");
Note: Above 2 lines should be at the beginning in your application.
Regards
Andreina J
From: Vishwakarma, Chhaya [mailto:Chhaya.Vishwakarma@Thinkbiganalytics.com<ma...@Thinkbiganalytics.com>]
Sent: 02 November 2015 PM 04:20
To: user@hadoop.apache.org<ma...@hadoop.apache.org>
Subject: RE: Authenticating to Kerberos enabled Hadoop cluster using Java
Thanks Niranjan It would be great if you can share a sample code if any?
From: Niranjan Subramanian [mailto:niranjan@webaction.com]
Sent: 02 November 2015 16:18
To: user@hadoop.apache.org<ma...@hadoop.apache.org>
Subject: Re: Authenticating to Kerberos enabled Hadoop cluster using Java
Hi Chhaya,
You can use the UserGroupInformation class from org.apache.hadoop.security package.
Specifically following 2 methods of that class
UserGroupInformation.setConfiguration(hdfsConfiguration);
UserGroupInformation.loginUserFromKeytab(principal, keytabPath);
Regards,
Niranjan
On 02-Nov-2015, at 4:15 pm, Vishwakarma, Chhaya <Ch...@Thinkbiganalytics.com>> wrote:
I have Kerberos enabled Hadoop cluster, I need to perform HDFS operations using JAVA code.
I have keytab file and username can someone please suggest how can I autheticate to Kerberos using JAVA code?
Regards,
Chhaya
RE: Authenticating to Kerberos enabled Hadoop cluster using Java
Posted by "Vishwakarma, Chhaya" <Ch...@Thinkbiganalytics.com>.
Thanks Subroto
dfs.namenode.kerberos.principal=hdfs/hadoop@HADOOP.COM<ma...@HADOOP.COM> this is principal specified here “UserGroupInformation.loginUserFromKeytab(String, String) “ or it is something different ?
From: Subroto Sanyal [mailto:ssanyal@datameer.com]
Sent: 02 November 2015 19:54
To: user@hadoop.apache.org
Subject: Re: Authenticating to Kerberos enabled Hadoop cluster using Java
Hi Chhaya,
Few configuration you need to set:
hadoop.security.authentication=kerberos
hadoop.security.authorization=true
dfs.namenode.kerberos.principal=hdfs/hadoop@HADOOP.COM<ma...@HADOOP.COM>
fs.defaultFS=hdfs://host:port
Further you need to use org.apache.hadoop.security.UserGroupInformation.loginUserFromKeytab(String, String) as suggested in one of the trailing mail.
Cheers,
Subroto Sanyal
On Mon, Nov 2, 2015 at 3:13 PM, Vishwakarma, Chhaya <Ch...@thinkbiganalytics.com>> wrote:
Code is successfully authenticating to Kerberos but when I try to run any hdfs command I get error as "Failed to specify server's kerberos principal name"
Can somebody please assist me on this?
Sent from my android device.
-----Original Message-----
From: andreina j <an...@huawei.com>>
To: "user@hadoop.apache.org<ma...@hadoop.apache.org>" <us...@hadoop.apache.org>>
Sent: Mon, 02 Nov 2015 4:57 pm
Subject: RE: Authenticating to Kerberos enabled Hadoop cluster using Java
Hi Chhaya,
Please find below sample code .
System.setProperty("java.security.krb5.conf", "D:\\data\\Desktop\\cluster-test\\krb5.conf");
// Login using keytab if have access to keytab. else
UserGroupInformation.loginUserFromKeytab("hdfs/hadoop@HADOOP.COM<ma...@HADOOP.COM>",
" D:\\data\\Desktop\\cluster-test\\conf\\hdfs.keytab");
Note: Above 2 lines should be at the beginning in your application.
Regards
Andreina J
From: Vishwakarma, Chhaya [mailto:Chhaya.Vishwakarma@Thinkbiganalytics.com<ma...@Thinkbiganalytics.com>]
Sent: 02 November 2015 PM 04:20
To: user@hadoop.apache.org<ma...@hadoop.apache.org>
Subject: RE: Authenticating to Kerberos enabled Hadoop cluster using Java
Thanks Niranjan It would be great if you can share a sample code if any?
From: Niranjan Subramanian [mailto:niranjan@webaction.com]
Sent: 02 November 2015 16:18
To: user@hadoop.apache.org<ma...@hadoop.apache.org>
Subject: Re: Authenticating to Kerberos enabled Hadoop cluster using Java
Hi Chhaya,
You can use the UserGroupInformation class from org.apache.hadoop.security package.
Specifically following 2 methods of that class
UserGroupInformation.setConfiguration(hdfsConfiguration);
UserGroupInformation.loginUserFromKeytab(principal, keytabPath);
Regards,
Niranjan
On 02-Nov-2015, at 4:15 pm, Vishwakarma, Chhaya <Ch...@Thinkbiganalytics.com>> wrote:
I have Kerberos enabled Hadoop cluster, I need to perform HDFS operations using JAVA code.
I have keytab file and username can someone please suggest how can I autheticate to Kerberos using JAVA code?
Regards,
Chhaya
RE: Authenticating to Kerberos enabled Hadoop cluster using Java
Posted by "Vishwakarma, Chhaya" <Ch...@Thinkbiganalytics.com>.
Thanks Subroto
dfs.namenode.kerberos.principal=hdfs/hadoop@HADOOP.COM<ma...@HADOOP.COM> this is principal specified here “UserGroupInformation.loginUserFromKeytab(String, String) “ or it is something different ?
From: Subroto Sanyal [mailto:ssanyal@datameer.com]
Sent: 02 November 2015 19:54
To: user@hadoop.apache.org
Subject: Re: Authenticating to Kerberos enabled Hadoop cluster using Java
Hi Chhaya,
Few configuration you need to set:
hadoop.security.authentication=kerberos
hadoop.security.authorization=true
dfs.namenode.kerberos.principal=hdfs/hadoop@HADOOP.COM<ma...@HADOOP.COM>
fs.defaultFS=hdfs://host:port
Further you need to use org.apache.hadoop.security.UserGroupInformation.loginUserFromKeytab(String, String) as suggested in one of the trailing mail.
Cheers,
Subroto Sanyal
On Mon, Nov 2, 2015 at 3:13 PM, Vishwakarma, Chhaya <Ch...@thinkbiganalytics.com>> wrote:
Code is successfully authenticating to Kerberos but when I try to run any hdfs command I get error as "Failed to specify server's kerberos principal name"
Can somebody please assist me on this?
Sent from my android device.
-----Original Message-----
From: andreina j <an...@huawei.com>>
To: "user@hadoop.apache.org<ma...@hadoop.apache.org>" <us...@hadoop.apache.org>>
Sent: Mon, 02 Nov 2015 4:57 pm
Subject: RE: Authenticating to Kerberos enabled Hadoop cluster using Java
Hi Chhaya,
Please find below sample code .
System.setProperty("java.security.krb5.conf", "D:\\data\\Desktop\\cluster-test\\krb5.conf");
// Login using keytab if have access to keytab. else
UserGroupInformation.loginUserFromKeytab("hdfs/hadoop@HADOOP.COM<ma...@HADOOP.COM>",
" D:\\data\\Desktop\\cluster-test\\conf\\hdfs.keytab");
Note: Above 2 lines should be at the beginning in your application.
Regards
Andreina J
From: Vishwakarma, Chhaya [mailto:Chhaya.Vishwakarma@Thinkbiganalytics.com<ma...@Thinkbiganalytics.com>]
Sent: 02 November 2015 PM 04:20
To: user@hadoop.apache.org<ma...@hadoop.apache.org>
Subject: RE: Authenticating to Kerberos enabled Hadoop cluster using Java
Thanks Niranjan It would be great if you can share a sample code if any?
From: Niranjan Subramanian [mailto:niranjan@webaction.com]
Sent: 02 November 2015 16:18
To: user@hadoop.apache.org<ma...@hadoop.apache.org>
Subject: Re: Authenticating to Kerberos enabled Hadoop cluster using Java
Hi Chhaya,
You can use the UserGroupInformation class from org.apache.hadoop.security package.
Specifically following 2 methods of that class
UserGroupInformation.setConfiguration(hdfsConfiguration);
UserGroupInformation.loginUserFromKeytab(principal, keytabPath);
Regards,
Niranjan
On 02-Nov-2015, at 4:15 pm, Vishwakarma, Chhaya <Ch...@Thinkbiganalytics.com>> wrote:
I have Kerberos enabled Hadoop cluster, I need to perform HDFS operations using JAVA code.
I have keytab file and username can someone please suggest how can I autheticate to Kerberos using JAVA code?
Regards,
Chhaya
RE: Authenticating to Kerberos enabled Hadoop cluster using Java
Posted by "Vishwakarma, Chhaya" <Ch...@Thinkbiganalytics.com>.
Thanks Subroto
dfs.namenode.kerberos.principal=hdfs/hadoop@HADOOP.COM<ma...@HADOOP.COM> this is principal specified here “UserGroupInformation.loginUserFromKeytab(String, String) “ or it is something different ?
From: Subroto Sanyal [mailto:ssanyal@datameer.com]
Sent: 02 November 2015 19:54
To: user@hadoop.apache.org
Subject: Re: Authenticating to Kerberos enabled Hadoop cluster using Java
Hi Chhaya,
Few configuration you need to set:
hadoop.security.authentication=kerberos
hadoop.security.authorization=true
dfs.namenode.kerberos.principal=hdfs/hadoop@HADOOP.COM<ma...@HADOOP.COM>
fs.defaultFS=hdfs://host:port
Further you need to use org.apache.hadoop.security.UserGroupInformation.loginUserFromKeytab(String, String) as suggested in one of the trailing mail.
Cheers,
Subroto Sanyal
On Mon, Nov 2, 2015 at 3:13 PM, Vishwakarma, Chhaya <Ch...@thinkbiganalytics.com>> wrote:
Code is successfully authenticating to Kerberos but when I try to run any hdfs command I get error as "Failed to specify server's kerberos principal name"
Can somebody please assist me on this?
Sent from my android device.
-----Original Message-----
From: andreina j <an...@huawei.com>>
To: "user@hadoop.apache.org<ma...@hadoop.apache.org>" <us...@hadoop.apache.org>>
Sent: Mon, 02 Nov 2015 4:57 pm
Subject: RE: Authenticating to Kerberos enabled Hadoop cluster using Java
Hi Chhaya,
Please find below sample code .
System.setProperty("java.security.krb5.conf", "D:\\data\\Desktop\\cluster-test\\krb5.conf");
// Login using keytab if have access to keytab. else
UserGroupInformation.loginUserFromKeytab("hdfs/hadoop@HADOOP.COM<ma...@HADOOP.COM>",
" D:\\data\\Desktop\\cluster-test\\conf\\hdfs.keytab");
Note: Above 2 lines should be at the beginning in your application.
Regards
Andreina J
From: Vishwakarma, Chhaya [mailto:Chhaya.Vishwakarma@Thinkbiganalytics.com<ma...@Thinkbiganalytics.com>]
Sent: 02 November 2015 PM 04:20
To: user@hadoop.apache.org<ma...@hadoop.apache.org>
Subject: RE: Authenticating to Kerberos enabled Hadoop cluster using Java
Thanks Niranjan It would be great if you can share a sample code if any?
From: Niranjan Subramanian [mailto:niranjan@webaction.com]
Sent: 02 November 2015 16:18
To: user@hadoop.apache.org<ma...@hadoop.apache.org>
Subject: Re: Authenticating to Kerberos enabled Hadoop cluster using Java
Hi Chhaya,
You can use the UserGroupInformation class from org.apache.hadoop.security package.
Specifically following 2 methods of that class
UserGroupInformation.setConfiguration(hdfsConfiguration);
UserGroupInformation.loginUserFromKeytab(principal, keytabPath);
Regards,
Niranjan
On 02-Nov-2015, at 4:15 pm, Vishwakarma, Chhaya <Ch...@Thinkbiganalytics.com>> wrote:
I have Kerberos enabled Hadoop cluster, I need to perform HDFS operations using JAVA code.
I have keytab file and username can someone please suggest how can I autheticate to Kerberos using JAVA code?
Regards,
Chhaya
Re: Authenticating to Kerberos enabled Hadoop cluster using Java
Posted by Subroto Sanyal <ss...@datameer.com>.
Hi Chhaya,
Few configuration you need to set:
hadoop.security.authentication=kerberos
hadoop.security.authorization=true
dfs.namenode.kerberos.principal=hdfs/hadoop@HADOOP.COM
fs.defaultFS=hdfs://host:port
Further you need to
use org.apache.hadoop.security.UserGroupInformation.loginUserFromKeytab(String,
String) as suggested in one of the trailing mail.
Cheers,
Subroto Sanyal
On Mon, Nov 2, 2015 at 3:13 PM, Vishwakarma, Chhaya <
Chhaya.Vishwakarma@thinkbiganalytics.com> wrote:
> Code is successfully authenticating to Kerberos but when I try to run any
> hdfs command I get error as "Failed to specify server's kerberos principal
> name"
>
> Can somebody please assist me on this?
>
> Sent from my android device.
>
>
> -----Original Message-----
> From: andreina j <an...@huawei.com>
> To: "user@hadoop.apache.org" <us...@hadoop.apache.org>
> Sent: Mon, 02 Nov 2015 4:57 pm
> Subject: RE: Authenticating to Kerberos enabled Hadoop cluster using Java
>
> Hi Chhaya,
>
>
>
> Please find below sample code .
>
>
>
> System.setProperty("java.security.krb5.conf", "D:
> \\data\\Desktop\\cluster-test\\krb5.conf");
>
> // Login using keytab if have access to keytab. else
> UserGroupInformation.loginUserFromKeytab("hdfs/hadoop@HADOOP.COM",
> " D:\\data\\Desktop\\cluster-test\\conf\\hdfs.keytab");
>
>
>
> Note: Above 2 lines should be at the beginning in your application.
>
>
>
> Regards
>
> Andreina J
>
>
>
> *From:* Vishwakarma, Chhaya [mailto:
> Chhaya.Vishwakarma@Thinkbiganalytics.com]
> *Sent:* 02 November 2015 PM 04:20
> *To:* user@hadoop.apache.org
> *Subject:* RE: Authenticating to Kerberos enabled Hadoop cluster using
> Java
>
>
>
> Thanks Niranjan It would be great if you can share a sample code if any?
>
>
>
> *From:* Niranjan Subramanian [mailto:niranjan@webaction.com
> <ni...@webaction.com>]
> *Sent:* 02 November 2015 16:18
> *To:* user@hadoop.apache.org
> *Subject:* Re: Authenticating to Kerberos enabled Hadoop cluster using
> Java
>
>
>
> Hi Chhaya,
>
>
>
> You can use the UserGroupInformation class from org.apache.hadoop.security
> package.
>
>
>
> Specifically following 2 methods of that class
>
>
>
> UserGroupInformation.setConfiguration(hdfsConfiguration);
>
> UserGroupInformation.loginUserFromKeytab(principal, keytabPath);
>
>
>
> Regards,
>
> Niranjan
>
>
>
> On 02-Nov-2015, at 4:15 pm, Vishwakarma, Chhaya <
> Chhaya.Vishwakarma@Thinkbiganalytics.com> wrote:
>
>
>
> I have Kerberos enabled Hadoop cluster, I need to perform HDFS operations
> using JAVA code.
>
> I have keytab file and username can someone please suggest how can I
> autheticate to Kerberos using JAVA code?
>
> Regards,
>
> Chhaya
>
>
>
Re: Authenticating to Kerberos enabled Hadoop cluster using Java
Posted by Subroto Sanyal <ss...@datameer.com>.
Hi Chhaya,
Few configuration you need to set:
hadoop.security.authentication=kerberos
hadoop.security.authorization=true
dfs.namenode.kerberos.principal=hdfs/hadoop@HADOOP.COM
fs.defaultFS=hdfs://host:port
Further you need to
use org.apache.hadoop.security.UserGroupInformation.loginUserFromKeytab(String,
String) as suggested in one of the trailing mail.
Cheers,
Subroto Sanyal
On Mon, Nov 2, 2015 at 3:13 PM, Vishwakarma, Chhaya <
Chhaya.Vishwakarma@thinkbiganalytics.com> wrote:
> Code is successfully authenticating to Kerberos but when I try to run any
> hdfs command I get error as "Failed to specify server's kerberos principal
> name"
>
> Can somebody please assist me on this?
>
> Sent from my android device.
>
>
> -----Original Message-----
> From: andreina j <an...@huawei.com>
> To: "user@hadoop.apache.org" <us...@hadoop.apache.org>
> Sent: Mon, 02 Nov 2015 4:57 pm
> Subject: RE: Authenticating to Kerberos enabled Hadoop cluster using Java
>
> Hi Chhaya,
>
>
>
> Please find below sample code .
>
>
>
> System.setProperty("java.security.krb5.conf", "D:
> \\data\\Desktop\\cluster-test\\krb5.conf");
>
> // Login using keytab if have access to keytab. else
> UserGroupInformation.loginUserFromKeytab("hdfs/hadoop@HADOOP.COM",
> " D:\\data\\Desktop\\cluster-test\\conf\\hdfs.keytab");
>
>
>
> Note: Above 2 lines should be at the beginning in your application.
>
>
>
> Regards
>
> Andreina J
>
>
>
> *From:* Vishwakarma, Chhaya [mailto:
> Chhaya.Vishwakarma@Thinkbiganalytics.com]
> *Sent:* 02 November 2015 PM 04:20
> *To:* user@hadoop.apache.org
> *Subject:* RE: Authenticating to Kerberos enabled Hadoop cluster using
> Java
>
>
>
> Thanks Niranjan It would be great if you can share a sample code if any?
>
>
>
> *From:* Niranjan Subramanian [mailto:niranjan@webaction.com
> <ni...@webaction.com>]
> *Sent:* 02 November 2015 16:18
> *To:* user@hadoop.apache.org
> *Subject:* Re: Authenticating to Kerberos enabled Hadoop cluster using
> Java
>
>
>
> Hi Chhaya,
>
>
>
> You can use the UserGroupInformation class from org.apache.hadoop.security
> package.
>
>
>
> Specifically following 2 methods of that class
>
>
>
> UserGroupInformation.setConfiguration(hdfsConfiguration);
>
> UserGroupInformation.loginUserFromKeytab(principal, keytabPath);
>
>
>
> Regards,
>
> Niranjan
>
>
>
> On 02-Nov-2015, at 4:15 pm, Vishwakarma, Chhaya <
> Chhaya.Vishwakarma@Thinkbiganalytics.com> wrote:
>
>
>
> I have Kerberos enabled Hadoop cluster, I need to perform HDFS operations
> using JAVA code.
>
> I have keytab file and username can someone please suggest how can I
> autheticate to Kerberos using JAVA code?
>
> Regards,
>
> Chhaya
>
>
>
Re: Authenticating to Kerberos enabled Hadoop cluster using Java
Posted by Subroto Sanyal <ss...@datameer.com>.
Hi Chhaya,
Few configuration you need to set:
hadoop.security.authentication=kerberos
hadoop.security.authorization=true
dfs.namenode.kerberos.principal=hdfs/hadoop@HADOOP.COM
fs.defaultFS=hdfs://host:port
Further you need to
use org.apache.hadoop.security.UserGroupInformation.loginUserFromKeytab(String,
String) as suggested in one of the trailing mail.
Cheers,
Subroto Sanyal
On Mon, Nov 2, 2015 at 3:13 PM, Vishwakarma, Chhaya <
Chhaya.Vishwakarma@thinkbiganalytics.com> wrote:
> Code is successfully authenticating to Kerberos but when I try to run any
> hdfs command I get error as "Failed to specify server's kerberos principal
> name"
>
> Can somebody please assist me on this?
>
> Sent from my android device.
>
>
> -----Original Message-----
> From: andreina j <an...@huawei.com>
> To: "user@hadoop.apache.org" <us...@hadoop.apache.org>
> Sent: Mon, 02 Nov 2015 4:57 pm
> Subject: RE: Authenticating to Kerberos enabled Hadoop cluster using Java
>
> Hi Chhaya,
>
>
>
> Please find below sample code .
>
>
>
> System.setProperty("java.security.krb5.conf", "D:
> \\data\\Desktop\\cluster-test\\krb5.conf");
>
> // Login using keytab if have access to keytab. else
> UserGroupInformation.loginUserFromKeytab("hdfs/hadoop@HADOOP.COM",
> " D:\\data\\Desktop\\cluster-test\\conf\\hdfs.keytab");
>
>
>
> Note: Above 2 lines should be at the beginning in your application.
>
>
>
> Regards
>
> Andreina J
>
>
>
> *From:* Vishwakarma, Chhaya [mailto:
> Chhaya.Vishwakarma@Thinkbiganalytics.com]
> *Sent:* 02 November 2015 PM 04:20
> *To:* user@hadoop.apache.org
> *Subject:* RE: Authenticating to Kerberos enabled Hadoop cluster using
> Java
>
>
>
> Thanks Niranjan It would be great if you can share a sample code if any?
>
>
>
> *From:* Niranjan Subramanian [mailto:niranjan@webaction.com
> <ni...@webaction.com>]
> *Sent:* 02 November 2015 16:18
> *To:* user@hadoop.apache.org
> *Subject:* Re: Authenticating to Kerberos enabled Hadoop cluster using
> Java
>
>
>
> Hi Chhaya,
>
>
>
> You can use the UserGroupInformation class from org.apache.hadoop.security
> package.
>
>
>
> Specifically following 2 methods of that class
>
>
>
> UserGroupInformation.setConfiguration(hdfsConfiguration);
>
> UserGroupInformation.loginUserFromKeytab(principal, keytabPath);
>
>
>
> Regards,
>
> Niranjan
>
>
>
> On 02-Nov-2015, at 4:15 pm, Vishwakarma, Chhaya <
> Chhaya.Vishwakarma@Thinkbiganalytics.com> wrote:
>
>
>
> I have Kerberos enabled Hadoop cluster, I need to perform HDFS operations
> using JAVA code.
>
> I have keytab file and username can someone please suggest how can I
> autheticate to Kerberos using JAVA code?
>
> Regards,
>
> Chhaya
>
>
>
Re: Authenticating to Kerberos enabled Hadoop cluster using Java
Posted by Subroto Sanyal <ss...@datameer.com>.
Hi Chhaya,
Few configuration you need to set:
hadoop.security.authentication=kerberos
hadoop.security.authorization=true
dfs.namenode.kerberos.principal=hdfs/hadoop@HADOOP.COM
fs.defaultFS=hdfs://host:port
Further you need to
use org.apache.hadoop.security.UserGroupInformation.loginUserFromKeytab(String,
String) as suggested in one of the trailing mail.
Cheers,
Subroto Sanyal
On Mon, Nov 2, 2015 at 3:13 PM, Vishwakarma, Chhaya <
Chhaya.Vishwakarma@thinkbiganalytics.com> wrote:
> Code is successfully authenticating to Kerberos but when I try to run any
> hdfs command I get error as "Failed to specify server's kerberos principal
> name"
>
> Can somebody please assist me on this?
>
> Sent from my android device.
>
>
> -----Original Message-----
> From: andreina j <an...@huawei.com>
> To: "user@hadoop.apache.org" <us...@hadoop.apache.org>
> Sent: Mon, 02 Nov 2015 4:57 pm
> Subject: RE: Authenticating to Kerberos enabled Hadoop cluster using Java
>
> Hi Chhaya,
>
>
>
> Please find below sample code .
>
>
>
> System.setProperty("java.security.krb5.conf", "D:
> \\data\\Desktop\\cluster-test\\krb5.conf");
>
> // Login using keytab if have access to keytab. else
> UserGroupInformation.loginUserFromKeytab("hdfs/hadoop@HADOOP.COM",
> " D:\\data\\Desktop\\cluster-test\\conf\\hdfs.keytab");
>
>
>
> Note: Above 2 lines should be at the beginning in your application.
>
>
>
> Regards
>
> Andreina J
>
>
>
> *From:* Vishwakarma, Chhaya [mailto:
> Chhaya.Vishwakarma@Thinkbiganalytics.com]
> *Sent:* 02 November 2015 PM 04:20
> *To:* user@hadoop.apache.org
> *Subject:* RE: Authenticating to Kerberos enabled Hadoop cluster using
> Java
>
>
>
> Thanks Niranjan It would be great if you can share a sample code if any?
>
>
>
> *From:* Niranjan Subramanian [mailto:niranjan@webaction.com
> <ni...@webaction.com>]
> *Sent:* 02 November 2015 16:18
> *To:* user@hadoop.apache.org
> *Subject:* Re: Authenticating to Kerberos enabled Hadoop cluster using
> Java
>
>
>
> Hi Chhaya,
>
>
>
> You can use the UserGroupInformation class from org.apache.hadoop.security
> package.
>
>
>
> Specifically following 2 methods of that class
>
>
>
> UserGroupInformation.setConfiguration(hdfsConfiguration);
>
> UserGroupInformation.loginUserFromKeytab(principal, keytabPath);
>
>
>
> Regards,
>
> Niranjan
>
>
>
> On 02-Nov-2015, at 4:15 pm, Vishwakarma, Chhaya <
> Chhaya.Vishwakarma@Thinkbiganalytics.com> wrote:
>
>
>
> I have Kerberos enabled Hadoop cluster, I need to perform HDFS operations
> using JAVA code.
>
> I have keytab file and username can someone please suggest how can I
> autheticate to Kerberos using JAVA code?
>
> Regards,
>
> Chhaya
>
>
>
RE: Authenticating to Kerberos enabled Hadoop cluster using Java
Posted by "Vishwakarma, Chhaya" <Ch...@Thinkbiganalytics.com>.
Code is successfully authenticating to Kerberos but when I try to run any hdfs command I get error as "Failed to specify server's kerberos principal name"
Can somebody please assist me on this?
Sent from my android device.
-----Original Message-----
From: andreina j <an...@huawei.com>
To: "user@hadoop.apache.org" <us...@hadoop.apache.org>
Sent: Mon, 02 Nov 2015 4:57 pm
Subject: RE: Authenticating to Kerberos enabled Hadoop cluster using Java
Hi Chhaya,
Please find below sample code .
System.setProperty("java.security.krb5.conf", "D:\\data\\Desktop\\cluster-test\\krb5.conf<file:///\\data\Desktop\cluster-test\krb5.conf>");
// Login using keytab if have access to keytab. else
UserGroupInformation.loginUserFromKeytab("hdfs/hadoop@HADOOP.COM<ma...@HADOOP.COM>",
" D:\\data\\Desktop\\cluster-test\\conf\\hdfs.keytab<file:///\\data\Desktop\cluster-test\conf\hdfs.keytab>");
Note: Above 2 lines should be at the beginning in your application.
Regards
Andreina J
From: Vishwakarma, Chhaya [mailto:Chhaya.Vishwakarma@Thinkbiganalytics.com]
Sent: 02 November 2015 PM 04:20
To: user@hadoop.apache.org
Subject: RE: Authenticating to Kerberos enabled Hadoop cluster using Java
Thanks Niranjan It would be great if you can share a sample code if any?
From: Niranjan Subramanian [mailto:niranjan@webaction.com]
Sent: 02 November 2015 16:18
To: user@hadoop.apache.org<ma...@hadoop.apache.org>
Subject: Re: Authenticating to Kerberos enabled Hadoop cluster using Java
Hi Chhaya,
You can use the UserGroupInformation class from org.apache.hadoop.security package.
Specifically following 2 methods of that class
UserGroupInformation.setConfiguration(hdfsConfiguration);
UserGroupInformation.loginUserFromKeytab(principal, keytabPath);
Regards,
Niranjan
On 02-Nov-2015, at 4:15 pm, Vishwakarma, Chhaya <Ch...@Thinkbiganalytics.com>> wrote:
I have Kerberos enabled Hadoop cluster, I need to perform HDFS operations using JAVA code.
I have keytab file and username can someone please suggest how can I autheticate to Kerberos using JAVA code?
Regards,
Chhaya
RE: Authenticating to Kerberos enabled Hadoop cluster using Java
Posted by "Vishwakarma, Chhaya" <Ch...@Thinkbiganalytics.com>.
Code is successfully authenticating to Kerberos but when I try to run any hdfs command I get error as "Failed to specify server's kerberos principal name"
Can somebody please assist me on this?
Sent from my android device.
-----Original Message-----
From: andreina j <an...@huawei.com>
To: "user@hadoop.apache.org" <us...@hadoop.apache.org>
Sent: Mon, 02 Nov 2015 4:57 pm
Subject: RE: Authenticating to Kerberos enabled Hadoop cluster using Java
Hi Chhaya,
Please find below sample code .
System.setProperty("java.security.krb5.conf", "D:\\data\\Desktop\\cluster-test\\krb5.conf<file:///\\data\Desktop\cluster-test\krb5.conf>");
// Login using keytab if have access to keytab. else
UserGroupInformation.loginUserFromKeytab("hdfs/hadoop@HADOOP.COM<ma...@HADOOP.COM>",
" D:\\data\\Desktop\\cluster-test\\conf\\hdfs.keytab<file:///\\data\Desktop\cluster-test\conf\hdfs.keytab>");
Note: Above 2 lines should be at the beginning in your application.
Regards
Andreina J
From: Vishwakarma, Chhaya [mailto:Chhaya.Vishwakarma@Thinkbiganalytics.com]
Sent: 02 November 2015 PM 04:20
To: user@hadoop.apache.org
Subject: RE: Authenticating to Kerberos enabled Hadoop cluster using Java
Thanks Niranjan It would be great if you can share a sample code if any?
From: Niranjan Subramanian [mailto:niranjan@webaction.com]
Sent: 02 November 2015 16:18
To: user@hadoop.apache.org<ma...@hadoop.apache.org>
Subject: Re: Authenticating to Kerberos enabled Hadoop cluster using Java
Hi Chhaya,
You can use the UserGroupInformation class from org.apache.hadoop.security package.
Specifically following 2 methods of that class
UserGroupInformation.setConfiguration(hdfsConfiguration);
UserGroupInformation.loginUserFromKeytab(principal, keytabPath);
Regards,
Niranjan
On 02-Nov-2015, at 4:15 pm, Vishwakarma, Chhaya <Ch...@Thinkbiganalytics.com>> wrote:
I have Kerberos enabled Hadoop cluster, I need to perform HDFS operations using JAVA code.
I have keytab file and username can someone please suggest how can I autheticate to Kerberos using JAVA code?
Regards,
Chhaya
RE: Authenticating to Kerberos enabled Hadoop cluster using Java
Posted by "Vishwakarma, Chhaya" <Ch...@Thinkbiganalytics.com>.
Code is successfully authenticating to Kerberos but when I try to run any hdfs command I get error as "Failed to specify server's kerberos principal name"
Can somebody please assist me on this?
Sent from my android device.
-----Original Message-----
From: andreina j <an...@huawei.com>
To: "user@hadoop.apache.org" <us...@hadoop.apache.org>
Sent: Mon, 02 Nov 2015 4:57 pm
Subject: RE: Authenticating to Kerberos enabled Hadoop cluster using Java
Hi Chhaya,
Please find below sample code .
System.setProperty("java.security.krb5.conf", "D:\\data\\Desktop\\cluster-test\\krb5.conf<file:///\\data\Desktop\cluster-test\krb5.conf>");
// Login using keytab if have access to keytab. else
UserGroupInformation.loginUserFromKeytab("hdfs/hadoop@HADOOP.COM<ma...@HADOOP.COM>",
" D:\\data\\Desktop\\cluster-test\\conf\\hdfs.keytab<file:///\\data\Desktop\cluster-test\conf\hdfs.keytab>");
Note: Above 2 lines should be at the beginning in your application.
Regards
Andreina J
From: Vishwakarma, Chhaya [mailto:Chhaya.Vishwakarma@Thinkbiganalytics.com]
Sent: 02 November 2015 PM 04:20
To: user@hadoop.apache.org
Subject: RE: Authenticating to Kerberos enabled Hadoop cluster using Java
Thanks Niranjan It would be great if you can share a sample code if any?
From: Niranjan Subramanian [mailto:niranjan@webaction.com]
Sent: 02 November 2015 16:18
To: user@hadoop.apache.org<ma...@hadoop.apache.org>
Subject: Re: Authenticating to Kerberos enabled Hadoop cluster using Java
Hi Chhaya,
You can use the UserGroupInformation class from org.apache.hadoop.security package.
Specifically following 2 methods of that class
UserGroupInformation.setConfiguration(hdfsConfiguration);
UserGroupInformation.loginUserFromKeytab(principal, keytabPath);
Regards,
Niranjan
On 02-Nov-2015, at 4:15 pm, Vishwakarma, Chhaya <Ch...@Thinkbiganalytics.com>> wrote:
I have Kerberos enabled Hadoop cluster, I need to perform HDFS operations using JAVA code.
I have keytab file and username can someone please suggest how can I autheticate to Kerberos using JAVA code?
Regards,
Chhaya
RE: Authenticating to Kerberos enabled Hadoop cluster using Java
Posted by "Vishwakarma, Chhaya" <Ch...@Thinkbiganalytics.com>.
Code is successfully authenticating to Kerberos but when I try to run any hdfs command I get error as "Failed to specify server's kerberos principal name"
Can somebody please assist me on this?
Sent from my android device.
-----Original Message-----
From: andreina j <an...@huawei.com>
To: "user@hadoop.apache.org" <us...@hadoop.apache.org>
Sent: Mon, 02 Nov 2015 4:57 pm
Subject: RE: Authenticating to Kerberos enabled Hadoop cluster using Java
Hi Chhaya,
Please find below sample code .
System.setProperty("java.security.krb5.conf", "D:\\data\\Desktop\\cluster-test\\krb5.conf<file:///\\data\Desktop\cluster-test\krb5.conf>");
// Login using keytab if have access to keytab. else
UserGroupInformation.loginUserFromKeytab("hdfs/hadoop@HADOOP.COM<ma...@HADOOP.COM>",
" D:\\data\\Desktop\\cluster-test\\conf\\hdfs.keytab<file:///\\data\Desktop\cluster-test\conf\hdfs.keytab>");
Note: Above 2 lines should be at the beginning in your application.
Regards
Andreina J
From: Vishwakarma, Chhaya [mailto:Chhaya.Vishwakarma@Thinkbiganalytics.com]
Sent: 02 November 2015 PM 04:20
To: user@hadoop.apache.org
Subject: RE: Authenticating to Kerberos enabled Hadoop cluster using Java
Thanks Niranjan It would be great if you can share a sample code if any?
From: Niranjan Subramanian [mailto:niranjan@webaction.com]
Sent: 02 November 2015 16:18
To: user@hadoop.apache.org<ma...@hadoop.apache.org>
Subject: Re: Authenticating to Kerberos enabled Hadoop cluster using Java
Hi Chhaya,
You can use the UserGroupInformation class from org.apache.hadoop.security package.
Specifically following 2 methods of that class
UserGroupInformation.setConfiguration(hdfsConfiguration);
UserGroupInformation.loginUserFromKeytab(principal, keytabPath);
Regards,
Niranjan
On 02-Nov-2015, at 4:15 pm, Vishwakarma, Chhaya <Ch...@Thinkbiganalytics.com>> wrote:
I have Kerberos enabled Hadoop cluster, I need to perform HDFS operations using JAVA code.
I have keytab file and username can someone please suggest how can I autheticate to Kerberos using JAVA code?
Regards,
Chhaya
RE: Authenticating to Kerberos enabled Hadoop cluster using Java
Posted by andreina j <an...@huawei.com>.
Hi Chhaya,
Please find below sample code .
System.setProperty("java.security.krb5.conf", "D:\\data\\Desktop\\cluster-test\\krb5.conf<file:///\\data\Desktop\cluster-test\krb5.conf>");
// Login using keytab if have access to keytab. else
UserGroupInformation.loginUserFromKeytab("hdfs/hadoop@HADOOP.COM<ma...@HADOOP.COM>",
" D:\\data\\Desktop\\cluster-test\\conf\\hdfs.keytab<file:///\\data\Desktop\cluster-test\conf\hdfs.keytab>");
Note: Above 2 lines should be at the beginning in your application.
Regards
Andreina J
From: Vishwakarma, Chhaya [mailto:Chhaya.Vishwakarma@Thinkbiganalytics.com]
Sent: 02 November 2015 PM 04:20
To: user@hadoop.apache.org
Subject: RE: Authenticating to Kerberos enabled Hadoop cluster using Java
Thanks Niranjan It would be great if you can share a sample code if any?
From: Niranjan Subramanian [mailto:niranjan@webaction.com]
Sent: 02 November 2015 16:18
To: user@hadoop.apache.org<ma...@hadoop.apache.org>
Subject: Re: Authenticating to Kerberos enabled Hadoop cluster using Java
Hi Chhaya,
You can use the UserGroupInformation class from org.apache.hadoop.security package.
Specifically following 2 methods of that class
UserGroupInformation.setConfiguration(hdfsConfiguration);
UserGroupInformation.loginUserFromKeytab(principal, keytabPath);
Regards,
Niranjan
On 02-Nov-2015, at 4:15 pm, Vishwakarma, Chhaya <Ch...@Thinkbiganalytics.com>> wrote:
I have Kerberos enabled Hadoop cluster, I need to perform HDFS operations using JAVA code.
I have keytab file and username can someone please suggest how can I autheticate to Kerberos using JAVA code?
Regards,
Chhaya
RE: Authenticating to Kerberos enabled Hadoop cluster using Java
Posted by andreina j <an...@huawei.com>.
Hi Chhaya,
Please find below sample code .
System.setProperty("java.security.krb5.conf", "D:\\data\\Desktop\\cluster-test\\krb5.conf<file:///\\data\Desktop\cluster-test\krb5.conf>");
// Login using keytab if have access to keytab. else
UserGroupInformation.loginUserFromKeytab("hdfs/hadoop@HADOOP.COM<ma...@HADOOP.COM>",
" D:\\data\\Desktop\\cluster-test\\conf\\hdfs.keytab<file:///\\data\Desktop\cluster-test\conf\hdfs.keytab>");
Note: Above 2 lines should be at the beginning in your application.
Regards
Andreina J
From: Vishwakarma, Chhaya [mailto:Chhaya.Vishwakarma@Thinkbiganalytics.com]
Sent: 02 November 2015 PM 04:20
To: user@hadoop.apache.org
Subject: RE: Authenticating to Kerberos enabled Hadoop cluster using Java
Thanks Niranjan It would be great if you can share a sample code if any?
From: Niranjan Subramanian [mailto:niranjan@webaction.com]
Sent: 02 November 2015 16:18
To: user@hadoop.apache.org<ma...@hadoop.apache.org>
Subject: Re: Authenticating to Kerberos enabled Hadoop cluster using Java
Hi Chhaya,
You can use the UserGroupInformation class from org.apache.hadoop.security package.
Specifically following 2 methods of that class
UserGroupInformation.setConfiguration(hdfsConfiguration);
UserGroupInformation.loginUserFromKeytab(principal, keytabPath);
Regards,
Niranjan
On 02-Nov-2015, at 4:15 pm, Vishwakarma, Chhaya <Ch...@Thinkbiganalytics.com>> wrote:
I have Kerberos enabled Hadoop cluster, I need to perform HDFS operations using JAVA code.
I have keytab file and username can someone please suggest how can I autheticate to Kerberos using JAVA code?
Regards,
Chhaya
RE: Authenticating to Kerberos enabled Hadoop cluster using Java
Posted by andreina j <an...@huawei.com>.
Hi Chhaya,
Please find below sample code .
System.setProperty("java.security.krb5.conf", "D:\\data\\Desktop\\cluster-test\\krb5.conf<file:///\\data\Desktop\cluster-test\krb5.conf>");
// Login using keytab if have access to keytab. else
UserGroupInformation.loginUserFromKeytab("hdfs/hadoop@HADOOP.COM<ma...@HADOOP.COM>",
" D:\\data\\Desktop\\cluster-test\\conf\\hdfs.keytab<file:///\\data\Desktop\cluster-test\conf\hdfs.keytab>");
Note: Above 2 lines should be at the beginning in your application.
Regards
Andreina J
From: Vishwakarma, Chhaya [mailto:Chhaya.Vishwakarma@Thinkbiganalytics.com]
Sent: 02 November 2015 PM 04:20
To: user@hadoop.apache.org
Subject: RE: Authenticating to Kerberos enabled Hadoop cluster using Java
Thanks Niranjan It would be great if you can share a sample code if any?
From: Niranjan Subramanian [mailto:niranjan@webaction.com]
Sent: 02 November 2015 16:18
To: user@hadoop.apache.org<ma...@hadoop.apache.org>
Subject: Re: Authenticating to Kerberos enabled Hadoop cluster using Java
Hi Chhaya,
You can use the UserGroupInformation class from org.apache.hadoop.security package.
Specifically following 2 methods of that class
UserGroupInformation.setConfiguration(hdfsConfiguration);
UserGroupInformation.loginUserFromKeytab(principal, keytabPath);
Regards,
Niranjan
On 02-Nov-2015, at 4:15 pm, Vishwakarma, Chhaya <Ch...@Thinkbiganalytics.com>> wrote:
I have Kerberos enabled Hadoop cluster, I need to perform HDFS operations using JAVA code.
I have keytab file and username can someone please suggest how can I autheticate to Kerberos using JAVA code?
Regards,
Chhaya
RE: Authenticating to Kerberos enabled Hadoop cluster using Java
Posted by andreina j <an...@huawei.com>.
Hi Chhaya,
Please find below sample code .
System.setProperty("java.security.krb5.conf", "D:\\data\\Desktop\\cluster-test\\krb5.conf<file:///\\data\Desktop\cluster-test\krb5.conf>");
// Login using keytab if have access to keytab. else
UserGroupInformation.loginUserFromKeytab("hdfs/hadoop@HADOOP.COM<ma...@HADOOP.COM>",
" D:\\data\\Desktop\\cluster-test\\conf\\hdfs.keytab<file:///\\data\Desktop\cluster-test\conf\hdfs.keytab>");
Note: Above 2 lines should be at the beginning in your application.
Regards
Andreina J
From: Vishwakarma, Chhaya [mailto:Chhaya.Vishwakarma@Thinkbiganalytics.com]
Sent: 02 November 2015 PM 04:20
To: user@hadoop.apache.org
Subject: RE: Authenticating to Kerberos enabled Hadoop cluster using Java
Thanks Niranjan It would be great if you can share a sample code if any?
From: Niranjan Subramanian [mailto:niranjan@webaction.com]
Sent: 02 November 2015 16:18
To: user@hadoop.apache.org<ma...@hadoop.apache.org>
Subject: Re: Authenticating to Kerberos enabled Hadoop cluster using Java
Hi Chhaya,
You can use the UserGroupInformation class from org.apache.hadoop.security package.
Specifically following 2 methods of that class
UserGroupInformation.setConfiguration(hdfsConfiguration);
UserGroupInformation.loginUserFromKeytab(principal, keytabPath);
Regards,
Niranjan
On 02-Nov-2015, at 4:15 pm, Vishwakarma, Chhaya <Ch...@Thinkbiganalytics.com>> wrote:
I have Kerberos enabled Hadoop cluster, I need to perform HDFS operations using JAVA code.
I have keytab file and username can someone please suggest how can I autheticate to Kerberos using JAVA code?
Regards,
Chhaya
RE: Authenticating to Kerberos enabled Hadoop cluster using Java
Posted by "Vishwakarma, Chhaya" <Ch...@Thinkbiganalytics.com>.
Thanks Niranjan It would be great if you can share a sample code if any?
From: Niranjan Subramanian [mailto:niranjan@webaction.com]
Sent: 02 November 2015 16:18
To: user@hadoop.apache.org
Subject: Re: Authenticating to Kerberos enabled Hadoop cluster using Java
Hi Chhaya,
You can use the UserGroupInformation class from org.apache.hadoop.security package.
Specifically following 2 methods of that class
UserGroupInformation.setConfiguration(hdfsConfiguration);
UserGroupInformation.loginUserFromKeytab(principal, keytabPath);
Regards,
Niranjan
On 02-Nov-2015, at 4:15 pm, Vishwakarma, Chhaya <Ch...@Thinkbiganalytics.com>> wrote:
I have Kerberos enabled Hadoop cluster, I need to perform HDFS operations using JAVA code.
I have keytab file and username can someone please suggest how can I autheticate to Kerberos using JAVA code?
Regards,
Chhaya
RE: Authenticating to Kerberos enabled Hadoop cluster using Java
Posted by "Vishwakarma, Chhaya" <Ch...@Thinkbiganalytics.com>.
Thanks Niranjan It would be great if you can share a sample code if any?
From: Niranjan Subramanian [mailto:niranjan@webaction.com]
Sent: 02 November 2015 16:18
To: user@hadoop.apache.org
Subject: Re: Authenticating to Kerberos enabled Hadoop cluster using Java
Hi Chhaya,
You can use the UserGroupInformation class from org.apache.hadoop.security package.
Specifically following 2 methods of that class
UserGroupInformation.setConfiguration(hdfsConfiguration);
UserGroupInformation.loginUserFromKeytab(principal, keytabPath);
Regards,
Niranjan
On 02-Nov-2015, at 4:15 pm, Vishwakarma, Chhaya <Ch...@Thinkbiganalytics.com>> wrote:
I have Kerberos enabled Hadoop cluster, I need to perform HDFS operations using JAVA code.
I have keytab file and username can someone please suggest how can I autheticate to Kerberos using JAVA code?
Regards,
Chhaya
RE: Authenticating to Kerberos enabled Hadoop cluster using Java
Posted by "Vishwakarma, Chhaya" <Ch...@Thinkbiganalytics.com>.
Thanks Niranjan It would be great if you can share a sample code if any?
From: Niranjan Subramanian [mailto:niranjan@webaction.com]
Sent: 02 November 2015 16:18
To: user@hadoop.apache.org
Subject: Re: Authenticating to Kerberos enabled Hadoop cluster using Java
Hi Chhaya,
You can use the UserGroupInformation class from org.apache.hadoop.security package.
Specifically following 2 methods of that class
UserGroupInformation.setConfiguration(hdfsConfiguration);
UserGroupInformation.loginUserFromKeytab(principal, keytabPath);
Regards,
Niranjan
On 02-Nov-2015, at 4:15 pm, Vishwakarma, Chhaya <Ch...@Thinkbiganalytics.com>> wrote:
I have Kerberos enabled Hadoop cluster, I need to perform HDFS operations using JAVA code.
I have keytab file and username can someone please suggest how can I autheticate to Kerberos using JAVA code?
Regards,
Chhaya
RE: Authenticating to Kerberos enabled Hadoop cluster using Java
Posted by "Vishwakarma, Chhaya" <Ch...@Thinkbiganalytics.com>.
Thanks Niranjan It would be great if you can share a sample code if any?
From: Niranjan Subramanian [mailto:niranjan@webaction.com]
Sent: 02 November 2015 16:18
To: user@hadoop.apache.org
Subject: Re: Authenticating to Kerberos enabled Hadoop cluster using Java
Hi Chhaya,
You can use the UserGroupInformation class from org.apache.hadoop.security package.
Specifically following 2 methods of that class
UserGroupInformation.setConfiguration(hdfsConfiguration);
UserGroupInformation.loginUserFromKeytab(principal, keytabPath);
Regards,
Niranjan
On 02-Nov-2015, at 4:15 pm, Vishwakarma, Chhaya <Ch...@Thinkbiganalytics.com>> wrote:
I have Kerberos enabled Hadoop cluster, I need to perform HDFS operations using JAVA code.
I have keytab file and username can someone please suggest how can I autheticate to Kerberos using JAVA code?
Regards,
Chhaya
Re: Authenticating to Kerberos enabled Hadoop cluster using Java
Posted by Niranjan Subramanian <ni...@webaction.com>.
Hi Chhaya,
You can use the UserGroupInformation class from org.apache.hadoop.security package.
Specifically following 2 methods of that class
UserGroupInformation.setConfiguration(hdfsConfiguration);
UserGroupInformation.loginUserFromKeytab(principal, keytabPath);
Regards,
Niranjan
On 02-Nov-2015, at 4:15 pm, Vishwakarma, Chhaya <Ch...@Thinkbiganalytics.com> wrote:
> I have Kerberos enabled Hadoop cluster, I need to perform HDFS operations using JAVA code.
>
> I have keytab file and username can someone please suggest how can I autheticate to Kerberos using JAVA code?
>
> Regards,
> Chhaya
Re: Authenticating to Kerberos enabled Hadoop cluster using Java
Posted by Niranjan Subramanian <ni...@webaction.com>.
Hi Chhaya,
You can use the UserGroupInformation class from org.apache.hadoop.security package.
Specifically following 2 methods of that class
UserGroupInformation.setConfiguration(hdfsConfiguration);
UserGroupInformation.loginUserFromKeytab(principal, keytabPath);
Regards,
Niranjan
On 02-Nov-2015, at 4:15 pm, Vishwakarma, Chhaya <Ch...@Thinkbiganalytics.com> wrote:
> I have Kerberos enabled Hadoop cluster, I need to perform HDFS operations using JAVA code.
>
> I have keytab file and username can someone please suggest how can I autheticate to Kerberos using JAVA code?
>
> Regards,
> Chhaya
Re: Authenticating to Kerberos enabled Hadoop cluster using Java
Posted by Niranjan Subramanian <ni...@webaction.com>.
Hi Chhaya,
You can use the UserGroupInformation class from org.apache.hadoop.security package.
Specifically following 2 methods of that class
UserGroupInformation.setConfiguration(hdfsConfiguration);
UserGroupInformation.loginUserFromKeytab(principal, keytabPath);
Regards,
Niranjan
On 02-Nov-2015, at 4:15 pm, Vishwakarma, Chhaya <Ch...@Thinkbiganalytics.com> wrote:
> I have Kerberos enabled Hadoop cluster, I need to perform HDFS operations using JAVA code.
>
> I have keytab file and username can someone please suggest how can I autheticate to Kerberos using JAVA code?
>
> Regards,
> Chhaya
Re: Authenticating to Kerberos enabled Hadoop cluster using Java
Posted by Niranjan Subramanian <ni...@webaction.com>.
Hi Chhaya,
You can use the UserGroupInformation class from org.apache.hadoop.security package.
Specifically following 2 methods of that class
UserGroupInformation.setConfiguration(hdfsConfiguration);
UserGroupInformation.loginUserFromKeytab(principal, keytabPath);
Regards,
Niranjan
On 02-Nov-2015, at 4:15 pm, Vishwakarma, Chhaya <Ch...@Thinkbiganalytics.com> wrote:
> I have Kerberos enabled Hadoop cluster, I need to perform HDFS operations using JAVA code.
>
> I have keytab file and username can someone please suggest how can I autheticate to Kerberos using JAVA code?
>
> Regards,
> Chhaya
RE: Authenticating to Kerberos enabled Hadoop cluster using Java
Posted by Daniel Schulz <da...@hotmail.com>.
Hi Chhaya,
Thank you for asking straight away. At my latest project (HDP with Kerberos) we used authenticated users; as they submitted Java jobs MR or Spark reached through their user name and looked for a valid Kerberos Principal. Upon file access, HDFS does so. So your user needs to have a valid Kerberos Ticket from `kinit` or a keytab file.
Of course: you could issue `kinit` via JNI; but this looks rather like a hack. If a service needs credentials, I'd rather use Keystores for that. On Hadoop with Kerberos, your user's Kerberos Principal is the default way to go.
Alternatively, you may use this parameter chain before start up:
SET KINIT=c:\Program Files\Java\jdk1.6.0_20\bin\kinit
SET KEYTAB=d:\webapps\app1\WEB-INF\serverhostname.keytab
SET SPN=HTTP/app1.intranet.company.com@REALM002.COMPANY.COM
SET KRB5INI=d:\webapps\app1\WEB-INF\krb5.ini
"%KINIT%" -k -t "%KEYTAB%" %SPN% -J-Dsun.security.krb5.debug=true "-J-Djava.security.krb5.conf=%KRB5INI%"source: michael-behrendt.net/blog/2011/01/kerberos-keytab-uberprufen-mit-java-boardmitteln (German)
Kind regards, Daniel.
From: Chhaya.Vishwakarma@Thinkbiganalytics.comTo: user@hadoop.apache.org
Subject: Authenticating to Kerberos enabled Hadoop cluster using Java
Date: Mon, 2 Nov 2015 10:45:37 +0000
I have Kerberos enabled Hadoop cluster, I need to perform HDFS operations using JAVA code.
I have keytab file and username can someone please suggest how can I autheticate to Kerberos using JAVA code?
Regards,
Chhaya
RE: Authenticating to Kerberos enabled Hadoop cluster using Java
Posted by Daniel Schulz <da...@hotmail.com>.
Hi Chhaya,
Thank you for asking straight away. At my latest project (HDP with Kerberos) we used authenticated users; as they submitted Java jobs MR or Spark reached through their user name and looked for a valid Kerberos Principal. Upon file access, HDFS does so. So your user needs to have a valid Kerberos Ticket from `kinit` or a keytab file.
Of course: you could issue `kinit` via JNI; but this looks rather like a hack. If a service needs credentials, I'd rather use Keystores for that. On Hadoop with Kerberos, your user's Kerberos Principal is the default way to go.
Alternatively, you may use this parameter chain before start up:
SET KINIT=c:\Program Files\Java\jdk1.6.0_20\bin\kinit
SET KEYTAB=d:\webapps\app1\WEB-INF\serverhostname.keytab
SET SPN=HTTP/app1.intranet.company.com@REALM002.COMPANY.COM
SET KRB5INI=d:\webapps\app1\WEB-INF\krb5.ini
"%KINIT%" -k -t "%KEYTAB%" %SPN% -J-Dsun.security.krb5.debug=true "-J-Djava.security.krb5.conf=%KRB5INI%"source: michael-behrendt.net/blog/2011/01/kerberos-keytab-uberprufen-mit-java-boardmitteln (German)
Kind regards, Daniel.
From: Chhaya.Vishwakarma@Thinkbiganalytics.comTo: user@hadoop.apache.org
Subject: Authenticating to Kerberos enabled Hadoop cluster using Java
Date: Mon, 2 Nov 2015 10:45:37 +0000
I have Kerberos enabled Hadoop cluster, I need to perform HDFS operations using JAVA code.
I have keytab file and username can someone please suggest how can I autheticate to Kerberos using JAVA code?
Regards,
Chhaya