You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-issues@hadoop.apache.org by "Vinay (JIRA)" <ji...@apache.org> on 2012/11/08 09:50:12 UTC
[jira] [Created] (HADOOP-9019)
KerberosAuthenticator.doSpnegoSequence(..) should create a HTTP principal
with hostname everytime
Vinay created HADOOP-9019:
-----------------------------
Summary: KerberosAuthenticator.doSpnegoSequence(..) should create a HTTP principal with hostname everytime
Key: HADOOP-9019
URL: https://issues.apache.org/jira/browse/HADOOP-9019
Project: Hadoop Common
Issue Type: Bug
Reporter: Vinay
in KerberosAuthenticator.doSpnegoSequence(..) following line of code will just create a principal of the form "HTTP/<host>",
{code} String servicePrincipal = KerberosUtil.getServicePrincipal("HTTP",
KerberosAuthenticator.this.url.getHost());{code}
but uri.getHost() is not sure of always getting hostname. If uri contains IP, then it just returns IP.
For SPNEGO authentication principal should always be created with <hostname>.
This code should be something like this, which will look /etc/hosts to get hostname
{code} String hostname = InetAddress.getByName(
KerberosAuthenticator.this.url.getHost()).getHostName();
String servicePrincipal = KerberosUtil.getServicePrincipal("HTTP",
hostname);{code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-9019)
KerberosAuthenticator.doSpnegoSequence(..) should create a HTTP principal
with hostname everytime
Posted by "Aaron T. Myers (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-9019?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13499159#comment-13499159 ]
Aaron T. Myers commented on HADOOP-9019:
----------------------------------------
bq. In a private cluster where machines dont have any DNS server and also mapping of all ip/hostnames may not be present in /etc/hosts. In that case we may need to use IP instead of hostname in configs.
But how will this proposal help in such a situation? In order for this to work, the cluster will have to be able to do reverse name lookups on the IP address, so properly configured DNS of some sort (either a DNS server or populated /etc/hosts) will have to be present.
> KerberosAuthenticator.doSpnegoSequence(..) should create a HTTP principal with hostname everytime
> --------------------------------------------------------------------------------------------------
>
> Key: HADOOP-9019
> URL: https://issues.apache.org/jira/browse/HADOOP-9019
> Project: Hadoop Common
> Issue Type: Bug
> Reporter: Vinay
>
> in KerberosAuthenticator.doSpnegoSequence(..) following line of code will just create a principal of the form "HTTP/<host>",
> {code} String servicePrincipal = KerberosUtil.getServicePrincipal("HTTP",
> KerberosAuthenticator.this.url.getHost());{code}
> but uri.getHost() is not sure of always getting hostname. If uri contains IP, then it just returns IP.
> For SPNEGO authentication principal should always be created with <hostname>.
> This code should be something like this, which will look /etc/hosts to get hostname
> {code} String hostname = InetAddress.getByName(
> KerberosAuthenticator.this.url.getHost()).getHostName();
> String servicePrincipal = KerberosUtil.getServicePrincipal("HTTP",
> hostname);{code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-9019)
KerberosAuthenticator.doSpnegoSequence(..) should create a HTTP principal
with hostname everytime
Posted by "Allen Wittenauer (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-9019?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13494690#comment-13494690 ]
Allen Wittenauer commented on HADOOP-9019:
------------------------------------------
I seem to recall that using IP addresses in principals was a big no-no since many clients will do a reverse lookup as part of the validation sequence. (This is why one of the most effective ways to break Kerberos is via DNS MITM attacks.) In other words, using FQDN here is more of a Kerberos thing than a Hadoop thing.
> KerberosAuthenticator.doSpnegoSequence(..) should create a HTTP principal with hostname everytime
> --------------------------------------------------------------------------------------------------
>
> Key: HADOOP-9019
> URL: https://issues.apache.org/jira/browse/HADOOP-9019
> Project: Hadoop Common
> Issue Type: Bug
> Reporter: Vinay
>
> in KerberosAuthenticator.doSpnegoSequence(..) following line of code will just create a principal of the form "HTTP/<host>",
> {code} String servicePrincipal = KerberosUtil.getServicePrincipal("HTTP",
> KerberosAuthenticator.this.url.getHost());{code}
> but uri.getHost() is not sure of always getting hostname. If uri contains IP, then it just returns IP.
> For SPNEGO authentication principal should always be created with <hostname>.
> This code should be something like this, which will look /etc/hosts to get hostname
> {code} String hostname = InetAddress.getByName(
> KerberosAuthenticator.this.url.getHost()).getHostName();
> String servicePrincipal = KerberosUtil.getServicePrincipal("HTTP",
> hostname);{code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-9019)
KerberosAuthenticator.doSpnegoSequence(..) should create a HTTP principal
with hostname everytime
Posted by "Aaron T. Myers (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-9019?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13498245#comment-13498245 ]
Aaron T. Myers commented on HADOOP-9019:
----------------------------------------
He can use the _HOST macro, but that's not the issue. In some places in the code _HOST is replaced with the literal address specified in some config property. My understanding is Vinay wants to do something like this:
{code}
<property>
<name>fs.defaultFS</name>
<value>hdfs://192.168.0.2</value>
</property>
<property>
<name>dfs.namenode.kerberos.principal</name>
<value>hdfs/_HOST@EXAMPLE.COM</value>
</property>
{code}
However, this will result in Hadoop concluding the appropriate principal name is "hdfs/192.168.0.2@EXAMPLE.COM". What Vinay is proposing is that Hadoop do a reverse lookup on 192.168.0.2 before forming the principal name, so that the expected principal is determined to be "hdfs/host1.example.com@EXAMPLE.COM".
It's still not clear to me why putting the IP address in the configs is ever desirable, though. In the example above, why not just set {{fs.defaultFS}} to "{{hdfs://host1.example.com}}"?
> KerberosAuthenticator.doSpnegoSequence(..) should create a HTTP principal with hostname everytime
> --------------------------------------------------------------------------------------------------
>
> Key: HADOOP-9019
> URL: https://issues.apache.org/jira/browse/HADOOP-9019
> Project: Hadoop Common
> Issue Type: Bug
> Reporter: Vinay
>
> in KerberosAuthenticator.doSpnegoSequence(..) following line of code will just create a principal of the form "HTTP/<host>",
> {code} String servicePrincipal = KerberosUtil.getServicePrincipal("HTTP",
> KerberosAuthenticator.this.url.getHost());{code}
> but uri.getHost() is not sure of always getting hostname. If uri contains IP, then it just returns IP.
> For SPNEGO authentication principal should always be created with <hostname>.
> This code should be something like this, which will look /etc/hosts to get hostname
> {code} String hostname = InetAddress.getByName(
> KerberosAuthenticator.this.url.getHost()).getHostName();
> String servicePrincipal = KerberosUtil.getServicePrincipal("HTTP",
> hostname);{code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-9019)
KerberosAuthenticator.doSpnegoSequence(..) should create a HTTP principal
with hostname everytime
Posted by "Aaron T. Myers (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-9019?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13495596#comment-13495596 ]
Aaron T. Myers commented on HADOOP-9019:
----------------------------------------
I agree with you, Allen, but I think the thing that Vinay is trying to do is use principals that contain FQDNs, but put IP addresses in the config files for the hosts. My understanding is that Vinay is proposing that we make Hadoop perform the reverse DNS lookup you're referring to.
Vinay: please correct me if I misunderstand your issue or proposal.
> KerberosAuthenticator.doSpnegoSequence(..) should create a HTTP principal with hostname everytime
> --------------------------------------------------------------------------------------------------
>
> Key: HADOOP-9019
> URL: https://issues.apache.org/jira/browse/HADOOP-9019
> Project: Hadoop Common
> Issue Type: Bug
> Reporter: Vinay
>
> in KerberosAuthenticator.doSpnegoSequence(..) following line of code will just create a principal of the form "HTTP/<host>",
> {code} String servicePrincipal = KerberosUtil.getServicePrincipal("HTTP",
> KerberosAuthenticator.this.url.getHost());{code}
> but uri.getHost() is not sure of always getting hostname. If uri contains IP, then it just returns IP.
> For SPNEGO authentication principal should always be created with <hostname>.
> This code should be something like this, which will look /etc/hosts to get hostname
> {code} String hostname = InetAddress.getByName(
> KerberosAuthenticator.this.url.getHost()).getHostName();
> String servicePrincipal = KerberosUtil.getServicePrincipal("HTTP",
> hostname);{code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-9019)
KerberosAuthenticator.doSpnegoSequence(..) should create a HTTP principal
with hostname everytime
Posted by "Vinay (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-9019?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13497763#comment-13497763 ]
Vinay commented on HADOOP-9019:
-------------------------------
Yes Aaron, your understanding is right. Since SPNEGO always use hostname in HTTP principal and HTTP principal is hardcoded in {{KerberosAuthenticator}} , lets support both type of configurations (ip and hostnames) in config files instead of assuming customer configures only hostnames.
Any thoughts..?
> KerberosAuthenticator.doSpnegoSequence(..) should create a HTTP principal with hostname everytime
> --------------------------------------------------------------------------------------------------
>
> Key: HADOOP-9019
> URL: https://issues.apache.org/jira/browse/HADOOP-9019
> Project: Hadoop Common
> Issue Type: Bug
> Reporter: Vinay
>
> in KerberosAuthenticator.doSpnegoSequence(..) following line of code will just create a principal of the form "HTTP/<host>",
> {code} String servicePrincipal = KerberosUtil.getServicePrincipal("HTTP",
> KerberosAuthenticator.this.url.getHost());{code}
> but uri.getHost() is not sure of always getting hostname. If uri contains IP, then it just returns IP.
> For SPNEGO authentication principal should always be created with <hostname>.
> This code should be something like this, which will look /etc/hosts to get hostname
> {code} String hostname = InetAddress.getByName(
> KerberosAuthenticator.this.url.getHost()).getHostName();
> String servicePrincipal = KerberosUtil.getServicePrincipal("HTTP",
> hostname);{code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-9019)
KerberosAuthenticator.doSpnegoSequence(..) should create a HTTP principal
with hostname everytime
Posted by "Vinay (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-9019?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13493758#comment-13493758 ]
Vinay commented on HADOOP-9019:
-------------------------------
Hi Aaron,
I understand, we can use hostnames in configuration to make this work.
But I feel that will be a work-around only. We should not restrict user to configure hostnames unless it is unavoidable. It should work even when Ip is configured.
> KerberosAuthenticator.doSpnegoSequence(..) should create a HTTP principal with hostname everytime
> --------------------------------------------------------------------------------------------------
>
> Key: HADOOP-9019
> URL: https://issues.apache.org/jira/browse/HADOOP-9019
> Project: Hadoop Common
> Issue Type: Bug
> Reporter: Vinay
>
> in KerberosAuthenticator.doSpnegoSequence(..) following line of code will just create a principal of the form "HTTP/<host>",
> {code} String servicePrincipal = KerberosUtil.getServicePrincipal("HTTP",
> KerberosAuthenticator.this.url.getHost());{code}
> but uri.getHost() is not sure of always getting hostname. If uri contains IP, then it just returns IP.
> For SPNEGO authentication principal should always be created with <hostname>.
> This code should be something like this, which will look /etc/hosts to get hostname
> {code} String hostname = InetAddress.getByName(
> KerberosAuthenticator.this.url.getHost()).getHostName();
> String servicePrincipal = KerberosUtil.getServicePrincipal("HTTP",
> hostname);{code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-9019)
KerberosAuthenticator.doSpnegoSequence(..) should create a HTTP principal
with hostname everytime
Posted by "Aaron T. Myers (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-9019?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13493472#comment-13493472 ]
Aaron T. Myers commented on HADOOP-9019:
----------------------------------------
Hi Vinay, is the issue here that you're using IP addresses in the config instead of hostnames? If so, why not just use hostnames in your configuration?
The change you suggest here is probably fine, but I'd like to understand what the motivation is.
> KerberosAuthenticator.doSpnegoSequence(..) should create a HTTP principal with hostname everytime
> --------------------------------------------------------------------------------------------------
>
> Key: HADOOP-9019
> URL: https://issues.apache.org/jira/browse/HADOOP-9019
> Project: Hadoop Common
> Issue Type: Bug
> Reporter: Vinay
>
> in KerberosAuthenticator.doSpnegoSequence(..) following line of code will just create a principal of the form "HTTP/<host>",
> {code} String servicePrincipal = KerberosUtil.getServicePrincipal("HTTP",
> KerberosAuthenticator.this.url.getHost());{code}
> but uri.getHost() is not sure of always getting hostname. If uri contains IP, then it just returns IP.
> For SPNEGO authentication principal should always be created with <hostname>.
> This code should be something like this, which will look /etc/hosts to get hostname
> {code} String hostname = InetAddress.getByName(
> KerberosAuthenticator.this.url.getHost()).getHostName();
> String servicePrincipal = KerberosUtil.getServicePrincipal("HTTP",
> hostname);{code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-9019)
KerberosAuthenticator.doSpnegoSequence(..) should create a HTTP principal
with hostname everytime
Posted by "Vinay (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-9019?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13498691#comment-13498691 ]
Vinay commented on HADOOP-9019:
-------------------------------
Hi Aaron,
Your explanation is correct. Anyway I am not worried about hdfs principal, for that any alternative principal name instead of "_HOST" such as 'hdfs/hadoop' will work. But especially for SPNEGO, we need 'HTTP' principal in the form of 'HTTP/<hostname>' only.
In a private cluster where machines dont have any DNS server and also mapping of all ip/hostnames may not be present in /etc/hosts. In that case we may need to use IP instead of hostname in configs.
So in that case, we need to resolve the above problem.
> KerberosAuthenticator.doSpnegoSequence(..) should create a HTTP principal with hostname everytime
> --------------------------------------------------------------------------------------------------
>
> Key: HADOOP-9019
> URL: https://issues.apache.org/jira/browse/HADOOP-9019
> Project: Hadoop Common
> Issue Type: Bug
> Reporter: Vinay
>
> in KerberosAuthenticator.doSpnegoSequence(..) following line of code will just create a principal of the form "HTTP/<host>",
> {code} String servicePrincipal = KerberosUtil.getServicePrincipal("HTTP",
> KerberosAuthenticator.this.url.getHost());{code}
> but uri.getHost() is not sure of always getting hostname. If uri contains IP, then it just returns IP.
> For SPNEGO authentication principal should always be created with <hostname>.
> This code should be something like this, which will look /etc/hosts to get hostname
> {code} String hostname = InetAddress.getByName(
> KerberosAuthenticator.this.url.getHost()).getHostName();
> String servicePrincipal = KerberosUtil.getServicePrincipal("HTTP",
> hostname);{code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-9019)
KerberosAuthenticator.doSpnegoSequence(..) should create a HTTP principal
with hostname everytime
Posted by "Aaron T. Myers (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-9019?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13494482#comment-13494482 ]
Aaron T. Myers commented on HADOOP-9019:
----------------------------------------
Got it. Thanks for the explanation.
I'm not opposed to this change, but it does seem like a bit of an odd use case. These machines have to have hostnames (with properly configured reverse DNS, no less) so I don't understand why folks would want to put IP addresses in their configs.
I won't object to the change if folks want to make it, though.
> KerberosAuthenticator.doSpnegoSequence(..) should create a HTTP principal with hostname everytime
> --------------------------------------------------------------------------------------------------
>
> Key: HADOOP-9019
> URL: https://issues.apache.org/jira/browse/HADOOP-9019
> Project: Hadoop Common
> Issue Type: Bug
> Reporter: Vinay
>
> in KerberosAuthenticator.doSpnegoSequence(..) following line of code will just create a principal of the form "HTTP/<host>",
> {code} String servicePrincipal = KerberosUtil.getServicePrincipal("HTTP",
> KerberosAuthenticator.this.url.getHost());{code}
> but uri.getHost() is not sure of always getting hostname. If uri contains IP, then it just returns IP.
> For SPNEGO authentication principal should always be created with <hostname>.
> This code should be something like this, which will look /etc/hosts to get hostname
> {code} String hostname = InetAddress.getByName(
> KerberosAuthenticator.this.url.getHost()).getHostName();
> String servicePrincipal = KerberosUtil.getServicePrincipal("HTTP",
> hostname);{code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-9019)
KerberosAuthenticator.doSpnegoSequence(..) should create a HTTP principal
with hostname everytime
Posted by "Vinay (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-9019?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13500017#comment-13500017 ]
Vinay commented on HADOOP-9019:
-------------------------------
oh!! Yes, You are right.
Only my point is we should SPNEGO should work with both type of configurations. Ip/hostname.
As I already told, I agree that with hostname configured, this will work. But also we should consider configuring IP.
> KerberosAuthenticator.doSpnegoSequence(..) should create a HTTP principal with hostname everytime
> --------------------------------------------------------------------------------------------------
>
> Key: HADOOP-9019
> URL: https://issues.apache.org/jira/browse/HADOOP-9019
> Project: Hadoop Common
> Issue Type: Bug
> Reporter: Vinay
>
> in KerberosAuthenticator.doSpnegoSequence(..) following line of code will just create a principal of the form "HTTP/<host>",
> {code} String servicePrincipal = KerberosUtil.getServicePrincipal("HTTP",
> KerberosAuthenticator.this.url.getHost());{code}
> but uri.getHost() is not sure of always getting hostname. If uri contains IP, then it just returns IP.
> For SPNEGO authentication principal should always be created with <hostname>.
> This code should be something like this, which will look /etc/hosts to get hostname
> {code} String hostname = InetAddress.getByName(
> KerberosAuthenticator.this.url.getHost()).getHostName();
> String servicePrincipal = KerberosUtil.getServicePrincipal("HTTP",
> hostname);{code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-9019)
KerberosAuthenticator.doSpnegoSequence(..) should create a HTTP principal
with hostname everytime
Posted by "Allen Wittenauer (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-9019?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13498206#comment-13498206 ]
Allen Wittenauer commented on HADOOP-9019:
------------------------------------------
Can't you just use the _HOST macro instead? (or is that only in my branch? I can't remember what is supposed by the Apache version anymore...)
> KerberosAuthenticator.doSpnegoSequence(..) should create a HTTP principal with hostname everytime
> --------------------------------------------------------------------------------------------------
>
> Key: HADOOP-9019
> URL: https://issues.apache.org/jira/browse/HADOOP-9019
> Project: Hadoop Common
> Issue Type: Bug
> Reporter: Vinay
>
> in KerberosAuthenticator.doSpnegoSequence(..) following line of code will just create a principal of the form "HTTP/<host>",
> {code} String servicePrincipal = KerberosUtil.getServicePrincipal("HTTP",
> KerberosAuthenticator.this.url.getHost());{code}
> but uri.getHost() is not sure of always getting hostname. If uri contains IP, then it just returns IP.
> For SPNEGO authentication principal should always be created with <hostname>.
> This code should be something like this, which will look /etc/hosts to get hostname
> {code} String hostname = InetAddress.getByName(
> KerberosAuthenticator.this.url.getHost()).getHostName();
> String servicePrincipal = KerberosUtil.getServicePrincipal("HTTP",
> hostname);{code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira