You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-user@hadoop.apache.org by Jeff Zhang <zj...@gmail.com> on 2013/04/25 07:36:12 UTC
How to connect to hadoop through ssh tunnel and kerberos authentication
Hi all,
I could connect to hadoop cluster by ssh tunnel before when there's no
kerberos authentication. Now our cluster need to upgrade to kerberos
authentication. I try to connect to it by ssh tunnel again. But failed.
Could anyone guide me to do that ? Is there any tutorial for this ?
Here's what I did.
1. create a forwardable ticket in my client machine.
2.
edit ~/.ssh/config file
GSSAPIAuthentication yes
GSSAPIDelegateCredentials yes
3.
execute command "ssh -N -D 3600 gateway_host " to create a ssh
connection to my gateway host
4. config my core-site.xml file for ssh tunnel connection
<property>
<name>hadoophack.tunnel.port</name>
<value>3600</value></property>
<property>
<description>If users connect through a SOCKS proxy, we don't
want their SocketFactory settings interfering with the socket
factory associated with the actual daemons.</description>
<name>hadoop.rpc.socket.factory.class.default</name>
<value>org.apache.hadoop.net.SocksSocketFactory</value>
<final>true</final></property>
And there's the error message when I run "hadoop fs -ls /"
13/04/24 22:31:13 ERROR security.UserGroupInformation:
PriviledgedActionException
as:jianfezhang@CORP.EBAY.COMcause:javax.security.sasl.SaslException:
GSS initiate failed [Caused by
GSSException: No valid credentials provided (Mechanism level: Cannot get
kdc for realm CORP.EBAY.COM)]
13/04/24 22:31:13 INFO security.UserGroupInformation: Initiating logout for
jianfezhang@CORP.EBAY.COM
13/04/24 22:31:13 INFO security.UserGroupInformation: Initiating re-login
for jianfezhang@CORP.EBAY.COM
13/04/24 22:31:17 ERROR security.UserGroupInformation:
PriviledgedActionException
as:jianfezhang@CORP.EBAY.COMcause:javax.security.sasl.SaslException:
GSS initiate failed [Caused by
GSSException: No valid credentials provided (Mechanism level: Cannot get
kdc for realm CORP.EBAY.COM)]
13/04/24 22:31:17 WARN security.UserGroupInformation: Not attempting to
re-login since the last re-login was attempted less than 600 seconds before.
13/04/24 22:31:21 ERROR security.UserGroupInformation:
PriviledgedActionException
as:jianfezhang@CORP.EBAY.COMcause:javax.security.sasl.SaslException:
GSS initiate failed [Caused by
GSSException: No valid credentials provided (Mechanism level: Cannot get
kdc for realm CORP.EBAY.COM)]
--
Best Regards
Jeff Zhang
Re: How to connect to hadoop through ssh tunnel and kerberos authentication
Posted by Jeff Zhang <zj...@gmail.com>.
Yes, I have the entry for CORP.EBAY.COM
here's krb5.conf
[libdefaults]
noaddresses = true
default_realm = CORP.EBAY.COM
ticket_lifetime = 36000
renew_lifetime = 604800
default_tgs_enctypes = aes256-cts aes128-cts arcfour-hmac-md5 des-cbc-md5
des-cbc-crc
default_tkt_enctypes = aes256-cts aes128-cts arcfour-hmac-md5 des-cbc-md5
des-cbc-crc
permitted_enctypes = aes256-cts aes128-cts arcfour-hmac-md5 des-cbc-md5
des-cbc-crc
dns_lookup_realm = true
dns_lookup_kdc = true
passwd_check_s_address = false
udp_preference_limit = 1
ccache_type = 3
kdc_timesync = 0
[domain_realm]
dvd-entdc-002.corp.ebay.com = CORP.EBAY.COM
dvd-entdc-001.corp.ebay.com = CORP.EBAY.COM
rhv-dmzdc-002.corp.ebay.com = CORP.EBAY.COM
.corp.ebay.com = CORP.EBAY.COM
.phx.ebay.com = CORP.EBAY.COM
corp.ebay.com = CORP.EBAY.COM
phx.ebay.com = CORP.EBAY.COM
phxaishdc9en09.corp.ebay.com = CORP.EBAY.COM
rhv-dmzdc-001.corp.ebay.com = CORP.EBAY.COM
rhv-dmzdc-003.corp.ebay.com = CORP.EBAY.COM
[realms]
CORP.EBAY.COM = {
kdc = dvd-entdc-001.corp.ebay.com:88
master_kdc = dvd-entdc-001.corp.ebay.com:88
kpasswd = dvd-entdc-001.corp.ebay.com:464
kpasswd_server = dvd-entdc-001.corp.ebay.com:464
kdc = dvd-entdc-002.corp.ebay.com:88
master_kdc = dvd-entdc-002.corp.ebay.com:88
kpasswd = dvd-entdc-002.corp.ebay.com:464
kpasswd_server = dvd-entdc-002.corp.ebay.com:464
kdc = rhv-dmzdc-001.corp.ebay.com:88
master_kdc = rhv-dmzdc-001.corp.ebay.com:88
kpasswd = rhv-dmzdc-001.corp.ebay.com:464
kpasswd_server = rhv-dmzdc-001.corp.ebay.com:464
kdc = rhv-dmzdc-002.corp.ebay.com:88
master_kdc = rhv-dmzdc-002.corp.ebay.com:88
kpasswd = rhv-dmzdc-002.corp.ebay.com:464
kpasswd_server = rhv-dmzdc-002.corp.ebay.com:464
kdc = rhv-dmzdc-003.corp.ebay.com:88
master_kdc = rhv-dmzdc-003.corp.ebay.com:88
kpasswd = rhv-dmzdc-003.corp.ebay.com:464
kpasswd_server = rhv-dmzdc-003.corp.ebay.com:464
}
On Fri, Apr 26, 2013 at 3:34 AM, Daryn Sharp <da...@yahoo-inc.com> wrote:
> The important part of the error is "Cannot get kdc for realm
> CORP.EBAY.COM". Check if the gateway's /etc/krb5.conf has an entry for
> CORP.EBAY.COM in the [realms] section. Or if you actually have
> appropriate dns service records for kerberos, you can use "dns_lookup_kdc =
> true".
>
> Daryn
>
>
> On Apr 25, 2013, at 12:36 AM, Jeff Zhang wrote:
>
> Hi all,
>
>
> I could connect to hadoop cluster by ssh tunnel before when there's no
> kerberos authentication. Now our cluster need to upgrade to kerberos
> authentication. I try to connect to it by ssh tunnel again. But failed.
>
> Could anyone guide me to do that ? Is there any tutorial for this ?
>
> Here's what I did.
>
> 1. create a forwardable ticket in my client machine.
> 2.
>
> edit ~/.ssh/config file
>
> GSSAPIAuthentication yes
>
> GSSAPIDelegateCredentials yes
> 3.
>
> execute command "ssh -N -D 3600 gateway_host " to create a ssh
> connection to my gateway host
> 4. config my core-site.xml file for ssh tunnel connection
>
> <property>
> <name>hadoophack.tunnel.port</name>
> <value>3600</value></property>
> <property>
> <description>If users connect through a SOCKS proxy, we don't
> want their SocketFactory settings interfering with the socket
> factory associated with the actual daemons.</description>
> <name>hadoop.rpc.socket.factory.class.default</name>
> <value>org.apache.hadoop.net.SocksSocketFactory</value>
> <final>true</final></property>
>
> And there's the error message when I run "hadoop fs -ls /"
> 13/04/24 22:31:13 ERROR security.UserGroupInformation:
> PriviledgedActionException as:jianfezhang@CORP.EBAY.COMcause:javax.security.sasl.SaslException: GSS initiate failed [Caused by
> GSSException: No valid credentials provided (Mechanism level: Cannot get
> kdc for realm CORP.EBAY.COM)]
> 13/04/24 22:31:13 INFO security.UserGroupInformation: Initiating logout
> for jianfezhang@CORP.EBAY.COM
> 13/04/24 22:31:13 INFO security.UserGroupInformation: Initiating re-login
> for jianfezhang@CORP.EBAY.COM
> 13/04/24 22:31:17 ERROR security.UserGroupInformation:
> PriviledgedActionException as:jianfezhang@CORP.EBAY.COMcause:javax.security.sasl.SaslException: GSS initiate failed [Caused by
> GSSException: No valid credentials provided (Mechanism level: Cannot get
> kdc for realm CORP.EBAY.COM)]
> 13/04/24 22:31:17 WARN security.UserGroupInformation: Not attempting to
> re-login since the last re-login was attempted less than 600 seconds before.
> 13/04/24 22:31:21 ERROR security.UserGroupInformation:
> PriviledgedActionException as:jianfezhang@CORP.EBAY.COMcause:javax.security.sasl.SaslException: GSS initiate failed [Caused by
> GSSException: No valid credentials provided (Mechanism level: Cannot get
> kdc for realm CORP.EBAY.COM)]
>
> --
> Best Regards
>
> Jeff Zhang
>
>
>
--
Best Regards
Jeff Zhang
Re: How to connect to hadoop through ssh tunnel and kerberos authentication
Posted by Jeff Zhang <zj...@gmail.com>.
Yes, I have the entry for CORP.EBAY.COM
here's krb5.conf
[libdefaults]
noaddresses = true
default_realm = CORP.EBAY.COM
ticket_lifetime = 36000
renew_lifetime = 604800
default_tgs_enctypes = aes256-cts aes128-cts arcfour-hmac-md5 des-cbc-md5
des-cbc-crc
default_tkt_enctypes = aes256-cts aes128-cts arcfour-hmac-md5 des-cbc-md5
des-cbc-crc
permitted_enctypes = aes256-cts aes128-cts arcfour-hmac-md5 des-cbc-md5
des-cbc-crc
dns_lookup_realm = true
dns_lookup_kdc = true
passwd_check_s_address = false
udp_preference_limit = 1
ccache_type = 3
kdc_timesync = 0
[domain_realm]
dvd-entdc-002.corp.ebay.com = CORP.EBAY.COM
dvd-entdc-001.corp.ebay.com = CORP.EBAY.COM
rhv-dmzdc-002.corp.ebay.com = CORP.EBAY.COM
.corp.ebay.com = CORP.EBAY.COM
.phx.ebay.com = CORP.EBAY.COM
corp.ebay.com = CORP.EBAY.COM
phx.ebay.com = CORP.EBAY.COM
phxaishdc9en09.corp.ebay.com = CORP.EBAY.COM
rhv-dmzdc-001.corp.ebay.com = CORP.EBAY.COM
rhv-dmzdc-003.corp.ebay.com = CORP.EBAY.COM
[realms]
CORP.EBAY.COM = {
kdc = dvd-entdc-001.corp.ebay.com:88
master_kdc = dvd-entdc-001.corp.ebay.com:88
kpasswd = dvd-entdc-001.corp.ebay.com:464
kpasswd_server = dvd-entdc-001.corp.ebay.com:464
kdc = dvd-entdc-002.corp.ebay.com:88
master_kdc = dvd-entdc-002.corp.ebay.com:88
kpasswd = dvd-entdc-002.corp.ebay.com:464
kpasswd_server = dvd-entdc-002.corp.ebay.com:464
kdc = rhv-dmzdc-001.corp.ebay.com:88
master_kdc = rhv-dmzdc-001.corp.ebay.com:88
kpasswd = rhv-dmzdc-001.corp.ebay.com:464
kpasswd_server = rhv-dmzdc-001.corp.ebay.com:464
kdc = rhv-dmzdc-002.corp.ebay.com:88
master_kdc = rhv-dmzdc-002.corp.ebay.com:88
kpasswd = rhv-dmzdc-002.corp.ebay.com:464
kpasswd_server = rhv-dmzdc-002.corp.ebay.com:464
kdc = rhv-dmzdc-003.corp.ebay.com:88
master_kdc = rhv-dmzdc-003.corp.ebay.com:88
kpasswd = rhv-dmzdc-003.corp.ebay.com:464
kpasswd_server = rhv-dmzdc-003.corp.ebay.com:464
}
On Fri, Apr 26, 2013 at 3:34 AM, Daryn Sharp <da...@yahoo-inc.com> wrote:
> The important part of the error is "Cannot get kdc for realm
> CORP.EBAY.COM". Check if the gateway's /etc/krb5.conf has an entry for
> CORP.EBAY.COM in the [realms] section. Or if you actually have
> appropriate dns service records for kerberos, you can use "dns_lookup_kdc =
> true".
>
> Daryn
>
>
> On Apr 25, 2013, at 12:36 AM, Jeff Zhang wrote:
>
> Hi all,
>
>
> I could connect to hadoop cluster by ssh tunnel before when there's no
> kerberos authentication. Now our cluster need to upgrade to kerberos
> authentication. I try to connect to it by ssh tunnel again. But failed.
>
> Could anyone guide me to do that ? Is there any tutorial for this ?
>
> Here's what I did.
>
> 1. create a forwardable ticket in my client machine.
> 2.
>
> edit ~/.ssh/config file
>
> GSSAPIAuthentication yes
>
> GSSAPIDelegateCredentials yes
> 3.
>
> execute command "ssh -N -D 3600 gateway_host " to create a ssh
> connection to my gateway host
> 4. config my core-site.xml file for ssh tunnel connection
>
> <property>
> <name>hadoophack.tunnel.port</name>
> <value>3600</value></property>
> <property>
> <description>If users connect through a SOCKS proxy, we don't
> want their SocketFactory settings interfering with the socket
> factory associated with the actual daemons.</description>
> <name>hadoop.rpc.socket.factory.class.default</name>
> <value>org.apache.hadoop.net.SocksSocketFactory</value>
> <final>true</final></property>
>
> And there's the error message when I run "hadoop fs -ls /"
> 13/04/24 22:31:13 ERROR security.UserGroupInformation:
> PriviledgedActionException as:jianfezhang@CORP.EBAY.COMcause:javax.security.sasl.SaslException: GSS initiate failed [Caused by
> GSSException: No valid credentials provided (Mechanism level: Cannot get
> kdc for realm CORP.EBAY.COM)]
> 13/04/24 22:31:13 INFO security.UserGroupInformation: Initiating logout
> for jianfezhang@CORP.EBAY.COM
> 13/04/24 22:31:13 INFO security.UserGroupInformation: Initiating re-login
> for jianfezhang@CORP.EBAY.COM
> 13/04/24 22:31:17 ERROR security.UserGroupInformation:
> PriviledgedActionException as:jianfezhang@CORP.EBAY.COMcause:javax.security.sasl.SaslException: GSS initiate failed [Caused by
> GSSException: No valid credentials provided (Mechanism level: Cannot get
> kdc for realm CORP.EBAY.COM)]
> 13/04/24 22:31:17 WARN security.UserGroupInformation: Not attempting to
> re-login since the last re-login was attempted less than 600 seconds before.
> 13/04/24 22:31:21 ERROR security.UserGroupInformation:
> PriviledgedActionException as:jianfezhang@CORP.EBAY.COMcause:javax.security.sasl.SaslException: GSS initiate failed [Caused by
> GSSException: No valid credentials provided (Mechanism level: Cannot get
> kdc for realm CORP.EBAY.COM)]
>
> --
> Best Regards
>
> Jeff Zhang
>
>
>
--
Best Regards
Jeff Zhang
Re: How to connect to hadoop through ssh tunnel and kerberos authentication
Posted by Jeff Zhang <zj...@gmail.com>.
Yes, I have the entry for CORP.EBAY.COM
here's krb5.conf
[libdefaults]
noaddresses = true
default_realm = CORP.EBAY.COM
ticket_lifetime = 36000
renew_lifetime = 604800
default_tgs_enctypes = aes256-cts aes128-cts arcfour-hmac-md5 des-cbc-md5
des-cbc-crc
default_tkt_enctypes = aes256-cts aes128-cts arcfour-hmac-md5 des-cbc-md5
des-cbc-crc
permitted_enctypes = aes256-cts aes128-cts arcfour-hmac-md5 des-cbc-md5
des-cbc-crc
dns_lookup_realm = true
dns_lookup_kdc = true
passwd_check_s_address = false
udp_preference_limit = 1
ccache_type = 3
kdc_timesync = 0
[domain_realm]
dvd-entdc-002.corp.ebay.com = CORP.EBAY.COM
dvd-entdc-001.corp.ebay.com = CORP.EBAY.COM
rhv-dmzdc-002.corp.ebay.com = CORP.EBAY.COM
.corp.ebay.com = CORP.EBAY.COM
.phx.ebay.com = CORP.EBAY.COM
corp.ebay.com = CORP.EBAY.COM
phx.ebay.com = CORP.EBAY.COM
phxaishdc9en09.corp.ebay.com = CORP.EBAY.COM
rhv-dmzdc-001.corp.ebay.com = CORP.EBAY.COM
rhv-dmzdc-003.corp.ebay.com = CORP.EBAY.COM
[realms]
CORP.EBAY.COM = {
kdc = dvd-entdc-001.corp.ebay.com:88
master_kdc = dvd-entdc-001.corp.ebay.com:88
kpasswd = dvd-entdc-001.corp.ebay.com:464
kpasswd_server = dvd-entdc-001.corp.ebay.com:464
kdc = dvd-entdc-002.corp.ebay.com:88
master_kdc = dvd-entdc-002.corp.ebay.com:88
kpasswd = dvd-entdc-002.corp.ebay.com:464
kpasswd_server = dvd-entdc-002.corp.ebay.com:464
kdc = rhv-dmzdc-001.corp.ebay.com:88
master_kdc = rhv-dmzdc-001.corp.ebay.com:88
kpasswd = rhv-dmzdc-001.corp.ebay.com:464
kpasswd_server = rhv-dmzdc-001.corp.ebay.com:464
kdc = rhv-dmzdc-002.corp.ebay.com:88
master_kdc = rhv-dmzdc-002.corp.ebay.com:88
kpasswd = rhv-dmzdc-002.corp.ebay.com:464
kpasswd_server = rhv-dmzdc-002.corp.ebay.com:464
kdc = rhv-dmzdc-003.corp.ebay.com:88
master_kdc = rhv-dmzdc-003.corp.ebay.com:88
kpasswd = rhv-dmzdc-003.corp.ebay.com:464
kpasswd_server = rhv-dmzdc-003.corp.ebay.com:464
}
On Fri, Apr 26, 2013 at 3:34 AM, Daryn Sharp <da...@yahoo-inc.com> wrote:
> The important part of the error is "Cannot get kdc for realm
> CORP.EBAY.COM". Check if the gateway's /etc/krb5.conf has an entry for
> CORP.EBAY.COM in the [realms] section. Or if you actually have
> appropriate dns service records for kerberos, you can use "dns_lookup_kdc =
> true".
>
> Daryn
>
>
> On Apr 25, 2013, at 12:36 AM, Jeff Zhang wrote:
>
> Hi all,
>
>
> I could connect to hadoop cluster by ssh tunnel before when there's no
> kerberos authentication. Now our cluster need to upgrade to kerberos
> authentication. I try to connect to it by ssh tunnel again. But failed.
>
> Could anyone guide me to do that ? Is there any tutorial for this ?
>
> Here's what I did.
>
> 1. create a forwardable ticket in my client machine.
> 2.
>
> edit ~/.ssh/config file
>
> GSSAPIAuthentication yes
>
> GSSAPIDelegateCredentials yes
> 3.
>
> execute command "ssh -N -D 3600 gateway_host " to create a ssh
> connection to my gateway host
> 4. config my core-site.xml file for ssh tunnel connection
>
> <property>
> <name>hadoophack.tunnel.port</name>
> <value>3600</value></property>
> <property>
> <description>If users connect through a SOCKS proxy, we don't
> want their SocketFactory settings interfering with the socket
> factory associated with the actual daemons.</description>
> <name>hadoop.rpc.socket.factory.class.default</name>
> <value>org.apache.hadoop.net.SocksSocketFactory</value>
> <final>true</final></property>
>
> And there's the error message when I run "hadoop fs -ls /"
> 13/04/24 22:31:13 ERROR security.UserGroupInformation:
> PriviledgedActionException as:jianfezhang@CORP.EBAY.COMcause:javax.security.sasl.SaslException: GSS initiate failed [Caused by
> GSSException: No valid credentials provided (Mechanism level: Cannot get
> kdc for realm CORP.EBAY.COM)]
> 13/04/24 22:31:13 INFO security.UserGroupInformation: Initiating logout
> for jianfezhang@CORP.EBAY.COM
> 13/04/24 22:31:13 INFO security.UserGroupInformation: Initiating re-login
> for jianfezhang@CORP.EBAY.COM
> 13/04/24 22:31:17 ERROR security.UserGroupInformation:
> PriviledgedActionException as:jianfezhang@CORP.EBAY.COMcause:javax.security.sasl.SaslException: GSS initiate failed [Caused by
> GSSException: No valid credentials provided (Mechanism level: Cannot get
> kdc for realm CORP.EBAY.COM)]
> 13/04/24 22:31:17 WARN security.UserGroupInformation: Not attempting to
> re-login since the last re-login was attempted less than 600 seconds before.
> 13/04/24 22:31:21 ERROR security.UserGroupInformation:
> PriviledgedActionException as:jianfezhang@CORP.EBAY.COMcause:javax.security.sasl.SaslException: GSS initiate failed [Caused by
> GSSException: No valid credentials provided (Mechanism level: Cannot get
> kdc for realm CORP.EBAY.COM)]
>
> --
> Best Regards
>
> Jeff Zhang
>
>
>
--
Best Regards
Jeff Zhang
Re: How to connect to hadoop through ssh tunnel and kerberos authentication
Posted by Jeff Zhang <zj...@gmail.com>.
Yes, I have the entry for CORP.EBAY.COM
here's krb5.conf
[libdefaults]
noaddresses = true
default_realm = CORP.EBAY.COM
ticket_lifetime = 36000
renew_lifetime = 604800
default_tgs_enctypes = aes256-cts aes128-cts arcfour-hmac-md5 des-cbc-md5
des-cbc-crc
default_tkt_enctypes = aes256-cts aes128-cts arcfour-hmac-md5 des-cbc-md5
des-cbc-crc
permitted_enctypes = aes256-cts aes128-cts arcfour-hmac-md5 des-cbc-md5
des-cbc-crc
dns_lookup_realm = true
dns_lookup_kdc = true
passwd_check_s_address = false
udp_preference_limit = 1
ccache_type = 3
kdc_timesync = 0
[domain_realm]
dvd-entdc-002.corp.ebay.com = CORP.EBAY.COM
dvd-entdc-001.corp.ebay.com = CORP.EBAY.COM
rhv-dmzdc-002.corp.ebay.com = CORP.EBAY.COM
.corp.ebay.com = CORP.EBAY.COM
.phx.ebay.com = CORP.EBAY.COM
corp.ebay.com = CORP.EBAY.COM
phx.ebay.com = CORP.EBAY.COM
phxaishdc9en09.corp.ebay.com = CORP.EBAY.COM
rhv-dmzdc-001.corp.ebay.com = CORP.EBAY.COM
rhv-dmzdc-003.corp.ebay.com = CORP.EBAY.COM
[realms]
CORP.EBAY.COM = {
kdc = dvd-entdc-001.corp.ebay.com:88
master_kdc = dvd-entdc-001.corp.ebay.com:88
kpasswd = dvd-entdc-001.corp.ebay.com:464
kpasswd_server = dvd-entdc-001.corp.ebay.com:464
kdc = dvd-entdc-002.corp.ebay.com:88
master_kdc = dvd-entdc-002.corp.ebay.com:88
kpasswd = dvd-entdc-002.corp.ebay.com:464
kpasswd_server = dvd-entdc-002.corp.ebay.com:464
kdc = rhv-dmzdc-001.corp.ebay.com:88
master_kdc = rhv-dmzdc-001.corp.ebay.com:88
kpasswd = rhv-dmzdc-001.corp.ebay.com:464
kpasswd_server = rhv-dmzdc-001.corp.ebay.com:464
kdc = rhv-dmzdc-002.corp.ebay.com:88
master_kdc = rhv-dmzdc-002.corp.ebay.com:88
kpasswd = rhv-dmzdc-002.corp.ebay.com:464
kpasswd_server = rhv-dmzdc-002.corp.ebay.com:464
kdc = rhv-dmzdc-003.corp.ebay.com:88
master_kdc = rhv-dmzdc-003.corp.ebay.com:88
kpasswd = rhv-dmzdc-003.corp.ebay.com:464
kpasswd_server = rhv-dmzdc-003.corp.ebay.com:464
}
On Fri, Apr 26, 2013 at 3:34 AM, Daryn Sharp <da...@yahoo-inc.com> wrote:
> The important part of the error is "Cannot get kdc for realm
> CORP.EBAY.COM". Check if the gateway's /etc/krb5.conf has an entry for
> CORP.EBAY.COM in the [realms] section. Or if you actually have
> appropriate dns service records for kerberos, you can use "dns_lookup_kdc =
> true".
>
> Daryn
>
>
> On Apr 25, 2013, at 12:36 AM, Jeff Zhang wrote:
>
> Hi all,
>
>
> I could connect to hadoop cluster by ssh tunnel before when there's no
> kerberos authentication. Now our cluster need to upgrade to kerberos
> authentication. I try to connect to it by ssh tunnel again. But failed.
>
> Could anyone guide me to do that ? Is there any tutorial for this ?
>
> Here's what I did.
>
> 1. create a forwardable ticket in my client machine.
> 2.
>
> edit ~/.ssh/config file
>
> GSSAPIAuthentication yes
>
> GSSAPIDelegateCredentials yes
> 3.
>
> execute command "ssh -N -D 3600 gateway_host " to create a ssh
> connection to my gateway host
> 4. config my core-site.xml file for ssh tunnel connection
>
> <property>
> <name>hadoophack.tunnel.port</name>
> <value>3600</value></property>
> <property>
> <description>If users connect through a SOCKS proxy, we don't
> want their SocketFactory settings interfering with the socket
> factory associated with the actual daemons.</description>
> <name>hadoop.rpc.socket.factory.class.default</name>
> <value>org.apache.hadoop.net.SocksSocketFactory</value>
> <final>true</final></property>
>
> And there's the error message when I run "hadoop fs -ls /"
> 13/04/24 22:31:13 ERROR security.UserGroupInformation:
> PriviledgedActionException as:jianfezhang@CORP.EBAY.COMcause:javax.security.sasl.SaslException: GSS initiate failed [Caused by
> GSSException: No valid credentials provided (Mechanism level: Cannot get
> kdc for realm CORP.EBAY.COM)]
> 13/04/24 22:31:13 INFO security.UserGroupInformation: Initiating logout
> for jianfezhang@CORP.EBAY.COM
> 13/04/24 22:31:13 INFO security.UserGroupInformation: Initiating re-login
> for jianfezhang@CORP.EBAY.COM
> 13/04/24 22:31:17 ERROR security.UserGroupInformation:
> PriviledgedActionException as:jianfezhang@CORP.EBAY.COMcause:javax.security.sasl.SaslException: GSS initiate failed [Caused by
> GSSException: No valid credentials provided (Mechanism level: Cannot get
> kdc for realm CORP.EBAY.COM)]
> 13/04/24 22:31:17 WARN security.UserGroupInformation: Not attempting to
> re-login since the last re-login was attempted less than 600 seconds before.
> 13/04/24 22:31:21 ERROR security.UserGroupInformation:
> PriviledgedActionException as:jianfezhang@CORP.EBAY.COMcause:javax.security.sasl.SaslException: GSS initiate failed [Caused by
> GSSException: No valid credentials provided (Mechanism level: Cannot get
> kdc for realm CORP.EBAY.COM)]
>
> --
> Best Regards
>
> Jeff Zhang
>
>
>
--
Best Regards
Jeff Zhang
Re: How to connect to hadoop through ssh tunnel and kerberos
authentication
Posted by Daryn Sharp <da...@yahoo-inc.com>.
The important part of the error is "Cannot get kdc for realm CORP.EBAY.COM<http://CORP.EBAY.COM>". Check if the gateway's /etc/krb5.conf has an entry for CORP.EBAY.COM<http://CORP.EBAY.COM> in the [realms] section. Or if you actually have appropriate dns service records for kerberos, you can use "dns_lookup_kdc = true".
Daryn
On Apr 25, 2013, at 12:36 AM, Jeff Zhang wrote:
Hi all,
I could connect to hadoop cluster by ssh tunnel before when there's no kerberos authentication. Now our cluster need to upgrade to kerberos authentication. I try to connect to it by ssh tunnel again. But failed.
Could anyone guide me to do that ? Is there any tutorial for this ?
Here's what I did.
1. create a forwardable ticket in my client machine.
2. edit ~/.ssh/config file
GSSAPIAuthentication yes
GSSAPIDelegateCredentials yes
3. execute command "ssh -N -D 3600 gateway_host " to create a ssh connection to my gateway host
4. config my core-site.xml file for ssh tunnel connection
<property>
<name>hadoophack.tunnel.port</name>
<value>3600</value>
</property>
<property>
<description>If users connect through a SOCKS proxy, we don't
want their SocketFactory settings interfering with the socket
factory associated with the actual daemons.</description>
<name>hadoop.rpc.socket.factory.class.default</name>
<value>org.apache.hadoop.net.SocksSocketFactory</value>
<final>true</final>
</property>
And there's the error message when I run "hadoop fs -ls /"
13/04/24 22:31:13 ERROR security.UserGroupInformation: PriviledgedActionException as:jianfezhang@CORP.EBAY.COM<ma...@CORP.EBAY.COM> cause:javax.security.sasl.SaslException: GSS initiate failed [Caused by GSSException: No valid credentials provided (Mechanism level: Cannot get kdc for realm CORP.EBAY.COM<http://CORP.EBAY.COM/>)]
13/04/24 22:31:13 INFO security.UserGroupInformation: Initiating logout for jianfezhang@CORP.EBAY.COM<ma...@CORP.EBAY.COM>
13/04/24 22:31:13 INFO security.UserGroupInformation: Initiating re-login for jianfezhang@CORP.EBAY.COM<ma...@CORP.EBAY.COM>
13/04/24 22:31:17 ERROR security.UserGroupInformation: PriviledgedActionException as:jianfezhang@CORP.EBAY.COM<ma...@CORP.EBAY.COM> cause:javax.security.sasl.SaslException: GSS initiate failed [Caused by GSSException: No valid credentials provided (Mechanism level: Cannot get kdc for realm CORP.EBAY.COM<http://CORP.EBAY.COM/>)]
13/04/24 22:31:17 WARN security.UserGroupInformation: Not attempting to re-login since the last re-login was attempted less than 600 seconds before.
13/04/24 22:31:21 ERROR security.UserGroupInformation: PriviledgedActionException as:jianfezhang@CORP.EBAY.COM<ma...@CORP.EBAY.COM> cause:javax.security.sasl.SaslException: GSS initiate failed [Caused by GSSException: No valid credentials provided (Mechanism level: Cannot get kdc for realm CORP.EBAY.COM<http://CORP.EBAY.COM/>)]
--
Best Regards
Jeff Zhang
Re: How to connect to hadoop through ssh tunnel and kerberos
authentication
Posted by Daryn Sharp <da...@yahoo-inc.com>.
The important part of the error is "Cannot get kdc for realm CORP.EBAY.COM<http://CORP.EBAY.COM>". Check if the gateway's /etc/krb5.conf has an entry for CORP.EBAY.COM<http://CORP.EBAY.COM> in the [realms] section. Or if you actually have appropriate dns service records for kerberos, you can use "dns_lookup_kdc = true".
Daryn
On Apr 25, 2013, at 12:36 AM, Jeff Zhang wrote:
Hi all,
I could connect to hadoop cluster by ssh tunnel before when there's no kerberos authentication. Now our cluster need to upgrade to kerberos authentication. I try to connect to it by ssh tunnel again. But failed.
Could anyone guide me to do that ? Is there any tutorial for this ?
Here's what I did.
1. create a forwardable ticket in my client machine.
2. edit ~/.ssh/config file
GSSAPIAuthentication yes
GSSAPIDelegateCredentials yes
3. execute command "ssh -N -D 3600 gateway_host " to create a ssh connection to my gateway host
4. config my core-site.xml file for ssh tunnel connection
<property>
<name>hadoophack.tunnel.port</name>
<value>3600</value>
</property>
<property>
<description>If users connect through a SOCKS proxy, we don't
want their SocketFactory settings interfering with the socket
factory associated with the actual daemons.</description>
<name>hadoop.rpc.socket.factory.class.default</name>
<value>org.apache.hadoop.net.SocksSocketFactory</value>
<final>true</final>
</property>
And there's the error message when I run "hadoop fs -ls /"
13/04/24 22:31:13 ERROR security.UserGroupInformation: PriviledgedActionException as:jianfezhang@CORP.EBAY.COM<ma...@CORP.EBAY.COM> cause:javax.security.sasl.SaslException: GSS initiate failed [Caused by GSSException: No valid credentials provided (Mechanism level: Cannot get kdc for realm CORP.EBAY.COM<http://CORP.EBAY.COM/>)]
13/04/24 22:31:13 INFO security.UserGroupInformation: Initiating logout for jianfezhang@CORP.EBAY.COM<ma...@CORP.EBAY.COM>
13/04/24 22:31:13 INFO security.UserGroupInformation: Initiating re-login for jianfezhang@CORP.EBAY.COM<ma...@CORP.EBAY.COM>
13/04/24 22:31:17 ERROR security.UserGroupInformation: PriviledgedActionException as:jianfezhang@CORP.EBAY.COM<ma...@CORP.EBAY.COM> cause:javax.security.sasl.SaslException: GSS initiate failed [Caused by GSSException: No valid credentials provided (Mechanism level: Cannot get kdc for realm CORP.EBAY.COM<http://CORP.EBAY.COM/>)]
13/04/24 22:31:17 WARN security.UserGroupInformation: Not attempting to re-login since the last re-login was attempted less than 600 seconds before.
13/04/24 22:31:21 ERROR security.UserGroupInformation: PriviledgedActionException as:jianfezhang@CORP.EBAY.COM<ma...@CORP.EBAY.COM> cause:javax.security.sasl.SaslException: GSS initiate failed [Caused by GSSException: No valid credentials provided (Mechanism level: Cannot get kdc for realm CORP.EBAY.COM<http://CORP.EBAY.COM/>)]
--
Best Regards
Jeff Zhang
Re: How to connect to hadoop through ssh tunnel and kerberos
authentication
Posted by Daryn Sharp <da...@yahoo-inc.com>.
The important part of the error is "Cannot get kdc for realm CORP.EBAY.COM<http://CORP.EBAY.COM>". Check if the gateway's /etc/krb5.conf has an entry for CORP.EBAY.COM<http://CORP.EBAY.COM> in the [realms] section. Or if you actually have appropriate dns service records for kerberos, you can use "dns_lookup_kdc = true".
Daryn
On Apr 25, 2013, at 12:36 AM, Jeff Zhang wrote:
Hi all,
I could connect to hadoop cluster by ssh tunnel before when there's no kerberos authentication. Now our cluster need to upgrade to kerberos authentication. I try to connect to it by ssh tunnel again. But failed.
Could anyone guide me to do that ? Is there any tutorial for this ?
Here's what I did.
1. create a forwardable ticket in my client machine.
2. edit ~/.ssh/config file
GSSAPIAuthentication yes
GSSAPIDelegateCredentials yes
3. execute command "ssh -N -D 3600 gateway_host " to create a ssh connection to my gateway host
4. config my core-site.xml file for ssh tunnel connection
<property>
<name>hadoophack.tunnel.port</name>
<value>3600</value>
</property>
<property>
<description>If users connect through a SOCKS proxy, we don't
want their SocketFactory settings interfering with the socket
factory associated with the actual daemons.</description>
<name>hadoop.rpc.socket.factory.class.default</name>
<value>org.apache.hadoop.net.SocksSocketFactory</value>
<final>true</final>
</property>
And there's the error message when I run "hadoop fs -ls /"
13/04/24 22:31:13 ERROR security.UserGroupInformation: PriviledgedActionException as:jianfezhang@CORP.EBAY.COM<ma...@CORP.EBAY.COM> cause:javax.security.sasl.SaslException: GSS initiate failed [Caused by GSSException: No valid credentials provided (Mechanism level: Cannot get kdc for realm CORP.EBAY.COM<http://CORP.EBAY.COM/>)]
13/04/24 22:31:13 INFO security.UserGroupInformation: Initiating logout for jianfezhang@CORP.EBAY.COM<ma...@CORP.EBAY.COM>
13/04/24 22:31:13 INFO security.UserGroupInformation: Initiating re-login for jianfezhang@CORP.EBAY.COM<ma...@CORP.EBAY.COM>
13/04/24 22:31:17 ERROR security.UserGroupInformation: PriviledgedActionException as:jianfezhang@CORP.EBAY.COM<ma...@CORP.EBAY.COM> cause:javax.security.sasl.SaslException: GSS initiate failed [Caused by GSSException: No valid credentials provided (Mechanism level: Cannot get kdc for realm CORP.EBAY.COM<http://CORP.EBAY.COM/>)]
13/04/24 22:31:17 WARN security.UserGroupInformation: Not attempting to re-login since the last re-login was attempted less than 600 seconds before.
13/04/24 22:31:21 ERROR security.UserGroupInformation: PriviledgedActionException as:jianfezhang@CORP.EBAY.COM<ma...@CORP.EBAY.COM> cause:javax.security.sasl.SaslException: GSS initiate failed [Caused by GSSException: No valid credentials provided (Mechanism level: Cannot get kdc for realm CORP.EBAY.COM<http://CORP.EBAY.COM/>)]
--
Best Regards
Jeff Zhang
Re: How to connect to hadoop through ssh tunnel and kerberos
authentication
Posted by Daryn Sharp <da...@yahoo-inc.com>.
The important part of the error is "Cannot get kdc for realm CORP.EBAY.COM<http://CORP.EBAY.COM>". Check if the gateway's /etc/krb5.conf has an entry for CORP.EBAY.COM<http://CORP.EBAY.COM> in the [realms] section. Or if you actually have appropriate dns service records for kerberos, you can use "dns_lookup_kdc = true".
Daryn
On Apr 25, 2013, at 12:36 AM, Jeff Zhang wrote:
Hi all,
I could connect to hadoop cluster by ssh tunnel before when there's no kerberos authentication. Now our cluster need to upgrade to kerberos authentication. I try to connect to it by ssh tunnel again. But failed.
Could anyone guide me to do that ? Is there any tutorial for this ?
Here's what I did.
1. create a forwardable ticket in my client machine.
2. edit ~/.ssh/config file
GSSAPIAuthentication yes
GSSAPIDelegateCredentials yes
3. execute command "ssh -N -D 3600 gateway_host " to create a ssh connection to my gateway host
4. config my core-site.xml file for ssh tunnel connection
<property>
<name>hadoophack.tunnel.port</name>
<value>3600</value>
</property>
<property>
<description>If users connect through a SOCKS proxy, we don't
want their SocketFactory settings interfering with the socket
factory associated with the actual daemons.</description>
<name>hadoop.rpc.socket.factory.class.default</name>
<value>org.apache.hadoop.net.SocksSocketFactory</value>
<final>true</final>
</property>
And there's the error message when I run "hadoop fs -ls /"
13/04/24 22:31:13 ERROR security.UserGroupInformation: PriviledgedActionException as:jianfezhang@CORP.EBAY.COM<ma...@CORP.EBAY.COM> cause:javax.security.sasl.SaslException: GSS initiate failed [Caused by GSSException: No valid credentials provided (Mechanism level: Cannot get kdc for realm CORP.EBAY.COM<http://CORP.EBAY.COM/>)]
13/04/24 22:31:13 INFO security.UserGroupInformation: Initiating logout for jianfezhang@CORP.EBAY.COM<ma...@CORP.EBAY.COM>
13/04/24 22:31:13 INFO security.UserGroupInformation: Initiating re-login for jianfezhang@CORP.EBAY.COM<ma...@CORP.EBAY.COM>
13/04/24 22:31:17 ERROR security.UserGroupInformation: PriviledgedActionException as:jianfezhang@CORP.EBAY.COM<ma...@CORP.EBAY.COM> cause:javax.security.sasl.SaslException: GSS initiate failed [Caused by GSSException: No valid credentials provided (Mechanism level: Cannot get kdc for realm CORP.EBAY.COM<http://CORP.EBAY.COM/>)]
13/04/24 22:31:17 WARN security.UserGroupInformation: Not attempting to re-login since the last re-login was attempted less than 600 seconds before.
13/04/24 22:31:21 ERROR security.UserGroupInformation: PriviledgedActionException as:jianfezhang@CORP.EBAY.COM<ma...@CORP.EBAY.COM> cause:javax.security.sasl.SaslException: GSS initiate failed [Caused by GSSException: No valid credentials provided (Mechanism level: Cannot get kdc for realm CORP.EBAY.COM<http://CORP.EBAY.COM/>)]
--
Best Regards
Jeff Zhang