You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-issues@hadoop.apache.org by "Kan Zhang (JIRA)" <ji...@apache.org> on 2012/09/01 21:43:08 UTC
[jira] [Created] (HADOOP-8758) Support for pluggable token
implementations
Kan Zhang created HADOOP-8758:
---------------------------------
Summary: Support for pluggable token implementations
Key: HADOOP-8758
URL: https://issues.apache.org/jira/browse/HADOOP-8758
Project: Hadoop Common
Issue Type: Improvement
Components: ipc, security
Reporter: Kan Zhang
Assignee: Kan Zhang
Variants of the delegation token mechanism have been employed by different Hadoop services (NN, JT, RM, etc) to re-authenticate a previously Kerberos-authenticated client. While existing delegation token mechanism compliments Kerberos well, it doesn't necessarily have to be coupled with Kerberos. In principle, delegation tokens can be coupled with any authentication mechanism that bootstraps security. In particular, it can be coupled with other token implementations that use the same DIGEST-MD5 auth method. For example, a token can be pre-generated in an out-of-band manner and configured as a shared secret key between NN and JT to allow JT to make initial authentication to NN. This simple example doesn't deal with token renewal etc, but it helps to illustrate the point that if we can support multiple pluggable token implementations, it opens up the possibility for different users to plug in the token implementation of their choice to bootstrap security. Such token based mechanism has advantages over Kerberos in that 1) it doesn't require Kerberos infrastructure, 2) it leverages existing SASL DIGEST-MD5 auth method and doesn't require adding a new RPC auth method.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-8758) Support for pluggable token
implementations
Posted by "Owen O'Malley (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-8758?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13447751#comment-13447751 ]
Owen O'Malley commented on HADOOP-8758:
---------------------------------------
This is reasonable, although it isn't pluggable token implementations as much as adding new authentication mechanisms. (In other words, you shouldn't change the token mechanisms like the secret manager, but add an alternative to Kerberos and delegation tokens.)
I really wish we had a better SASL shared secret implementation than one that depends on MD5, whose strength is already pretty questionable. Clearly that is a related, but different issue.
I think a fruitful direction would be to figure out how to authenticate the HTTP connection that fetches delegation tokens. The HTTP authentication is already pluggable using servlet filters. Further, we've already done proprietary and spnego security filters, so I suspect that it is flexible enough for most purposes. It doesn't require any sasl, rpc, or client changes and doesn't break backwards compatibility. Would that work for your use case?
> Support for pluggable token implementations
> -------------------------------------------
>
> Key: HADOOP-8758
> URL: https://issues.apache.org/jira/browse/HADOOP-8758
> Project: Hadoop Common
> Issue Type: Improvement
> Components: ipc, security
> Reporter: Kan Zhang
> Assignee: Kan Zhang
>
> Variants of the delegation token mechanism have been employed by different Hadoop services (NN, JT, RM, etc) to re-authenticate a previously Kerberos-authenticated client. While existing delegation token mechanism compliments Kerberos well, it doesn't necessarily have to be coupled with Kerberos. In principle, delegation tokens can be coupled with any authentication mechanism that bootstraps security. In particular, it can be coupled with other token implementations that use the same DIGEST-MD5 auth method. For example, a token can be pre-generated in an out-of-band manner and configured as a shared secret key between NN and JT to allow JT to make initial authentication to NN. This simple example doesn't deal with token renewal etc, but it helps to illustrate the point that if we can support multiple pluggable token implementations, it opens up the possibility for different users to plug in the token implementation of their choice to bootstrap security. Such token based mechanism has advantages over Kerberos in that 1) it doesn't require Kerberos infrastructure, 2) it leverages existing SASL DIGEST-MD5 auth method and doesn't require adding a new RPC auth method.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-8758) Support for pluggable token
implementations
Posted by "Daryn Sharp (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-8758?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13447777#comment-13447777 ]
Daryn Sharp commented on HADOOP-8758:
-------------------------------------
As Bobby mentioned, and I've mentioned to Owen offline, I've really been wanting to configure the login context to use an auth other than kerberos, yet still use a secret manager to generate and validate tokens. Many of the tough issues I've had to debug may be caught by the test suites if simple auth used tokens too.
One problem to solve is that kerberos has crept up above the login context's abstraction, ex. the UGI. Although it's technically possible to use a different auth, there's a lot of places where checks for kerberos is hardcoded.
> Support for pluggable token implementations
> -------------------------------------------
>
> Key: HADOOP-8758
> URL: https://issues.apache.org/jira/browse/HADOOP-8758
> Project: Hadoop Common
> Issue Type: Improvement
> Components: ipc, security
> Reporter: Kan Zhang
> Assignee: Kan Zhang
>
> Variants of the delegation token mechanism have been employed by different Hadoop services (NN, JT, RM, etc) to re-authenticate a previously Kerberos-authenticated client. While existing delegation token mechanism compliments Kerberos well, it doesn't necessarily have to be coupled with Kerberos. In principle, delegation tokens can be coupled with any authentication mechanism that bootstraps security. In particular, it can be coupled with other token implementations that use the same DIGEST-MD5 auth method. For example, a token can be pre-generated in an out-of-band manner and configured as a shared secret key between NN and JT to allow JT to make initial authentication to NN. This simple example doesn't deal with token renewal etc, but it helps to illustrate the point that if we can support multiple pluggable token implementations, it opens up the possibility for different users to plug in the token implementation of their choice to bootstrap security. Such token based mechanism has advantages over Kerberos in that 1) it doesn't require Kerberos infrastructure, 2) it leverages existing SASL DIGEST-MD5 auth method and doesn't require adding a new RPC auth method.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-8758) Support for pluggable token
implementations
Posted by "Kan Zhang (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-8758?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13451461#comment-13451461 ]
Kan Zhang commented on HADOOP-8758:
-----------------------------------
PS. In the above comment, when I said SecretManagers, I was referring to delegation token SecretManagers.
> Support for pluggable token implementations
> -------------------------------------------
>
> Key: HADOOP-8758
> URL: https://issues.apache.org/jira/browse/HADOOP-8758
> Project: Hadoop Common
> Issue Type: Improvement
> Components: ipc, security
> Reporter: Kan Zhang
> Assignee: Kan Zhang
>
> Variants of the delegation token mechanism have been employed by different Hadoop services (NN, JT, RM, etc) to re-authenticate a previously Kerberos-authenticated client. While existing delegation token mechanism compliments Kerberos well, it doesn't necessarily have to be coupled with Kerberos. In principle, delegation tokens can be coupled with any authentication mechanism that bootstraps security. In particular, it can be coupled with other token implementations that use the same DIGEST-MD5 auth method. For example, a token can be pre-generated in an out-of-band manner and configured as a shared secret key between NN and JT to allow JT to make initial authentication to NN. This simple example doesn't deal with token renewal etc, but it helps to illustrate the point that if we can support multiple pluggable token implementations, it opens up the possibility for different users to plug in the token implementation of their choice to bootstrap security. Such token based mechanism has advantages over Kerberos in that 1) it doesn't require Kerberos infrastructure, 2) it leverages existing SASL DIGEST-MD5 auth method and doesn't require adding a new RPC auth method.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-8758) Support for pluggable token
implementations
Posted by "Kan Zhang (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-8758?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13448554#comment-13448554 ]
Kan Zhang commented on HADOOP-8758:
-----------------------------------
[~owen.omalley], on second thought, I think you do have a point on HTTP option. We'll explore more in that direction.
> Support for pluggable token implementations
> -------------------------------------------
>
> Key: HADOOP-8758
> URL: https://issues.apache.org/jira/browse/HADOOP-8758
> Project: Hadoop Common
> Issue Type: Improvement
> Components: ipc, security
> Reporter: Kan Zhang
> Assignee: Kan Zhang
>
> Variants of the delegation token mechanism have been employed by different Hadoop services (NN, JT, RM, etc) to re-authenticate a previously Kerberos-authenticated client. While existing delegation token mechanism compliments Kerberos well, it doesn't necessarily have to be coupled with Kerberos. In principle, delegation tokens can be coupled with any authentication mechanism that bootstraps security. In particular, it can be coupled with other token implementations that use the same DIGEST-MD5 auth method. For example, a token can be pre-generated in an out-of-band manner and configured as a shared secret key between NN and JT to allow JT to make initial authentication to NN. This simple example doesn't deal with token renewal etc, but it helps to illustrate the point that if we can support multiple pluggable token implementations, it opens up the possibility for different users to plug in the token implementation of their choice to bootstrap security. Such token based mechanism has advantages over Kerberos in that 1) it doesn't require Kerberos infrastructure, 2) it leverages existing SASL DIGEST-MD5 auth method and doesn't require adding a new RPC auth method.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-8758) Support for pluggable token
implementations
Posted by "Kan Zhang (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-8758?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13451640#comment-13451640 ]
Kan Zhang commented on HADOOP-8758:
-----------------------------------
> We could consider supporting multiple internal auth methods, but currently I don't see a need.
By enabling DIGEST-MD5 to be used as both internal and external auth methods and pluggable token implementation, it's possible to configure multiple token implementations for both internal and external authentication, although there is only one auth method (DIGEST-MD5). I can see some use cases for external auth, but currently no use case for internal auth to use multiple token implementations. For example, one may need to set up pair-wise keys for NN/JT and Oozie/JT so that they can make initial authenticated connections. Those pair-wise keys may be implemented by different SecretManagers. On JT, both SecretManagers need to be configured for external auth using DIGEST-MD5.
> Support for pluggable token implementations
> -------------------------------------------
>
> Key: HADOOP-8758
> URL: https://issues.apache.org/jira/browse/HADOOP-8758
> Project: Hadoop Common
> Issue Type: Improvement
> Components: ipc, security
> Reporter: Kan Zhang
> Assignee: Kan Zhang
>
> Variants of the delegation token mechanism have been employed by different Hadoop services (NN, JT, RM, etc) to re-authenticate a previously Kerberos-authenticated client. While existing delegation token mechanism compliments Kerberos well, it doesn't necessarily have to be coupled with Kerberos. In principle, delegation tokens can be coupled with any authentication mechanism that bootstraps security. In particular, it can be coupled with other token implementations that use the same DIGEST-MD5 auth method. For example, a token can be pre-generated in an out-of-band manner and configured as a shared secret key between NN and JT to allow JT to make initial authentication to NN. This simple example doesn't deal with token renewal etc, but it helps to illustrate the point that if we can support multiple pluggable token implementations, it opens up the possibility for different users to plug in the token implementation of their choice to bootstrap security. Such token based mechanism has advantages over Kerberos in that 1) it doesn't require Kerberos infrastructure, 2) it leverages existing SASL DIGEST-MD5 auth method and doesn't require adding a new RPC auth method.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-8758) Support for pluggable token
implementations
Posted by "Daryn Sharp (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-8758?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13457026#comment-13457026 ]
Daryn Sharp commented on HADOOP-8758:
-------------------------------------
bq. Kerberos and tokens are coupled in the sense that today delegation tokens can only be fetched if authenticated over Kerberos
HADOOP-8779 and its subtasks will allow tokens to be used with basic auth. It's the first step in the more generalized support for tokens with any auth. There are a lot of conditionals that assume secure auth means kerberos which can be addressed by this jira.
bq. Secret managers were created to serve SASL DIGEST-MD5 callbacks. They are credential managers for DIGEST-MD5 mechanism. If you are talking about adding auth methods other than DIGEST-MD5, the right place is adding a SASL mechanism, not overloading DIGEST-MD5.
I'm not sure a new mechanism per auth method is needed since DIGEST-MD5 is a generic auth mechanism supported by other auth services. For instance, LDAP directly supports the DIGEST-MD5 mechanism and it's a common practice to provide the backend auth via PAM. LoginContext is java's PAM-like equivalent, so it seems a natural for the SASL callbacks to delegate to a LoginContext which can be configured to validate the credentials via the secret manager (tokens) or other auth modules such as LDAP. These additional auth methods would be supported by a simple change to the LoginContext config and no other code changes!
> Support for pluggable token implementations
> -------------------------------------------
>
> Key: HADOOP-8758
> URL: https://issues.apache.org/jira/browse/HADOOP-8758
> Project: Hadoop Common
> Issue Type: Improvement
> Components: ipc, security
> Reporter: Kan Zhang
> Assignee: Kan Zhang
>
> Variants of the delegation token mechanism have been employed by different Hadoop services (NN, JT, RM, etc) to re-authenticate a previously Kerberos-authenticated client. While existing delegation token mechanism compliments Kerberos well, it doesn't necessarily have to be coupled with Kerberos. In principle, delegation tokens can be coupled with any authentication mechanism that bootstraps security. In particular, it can be coupled with other token implementations that use the same DIGEST-MD5 auth method. For example, a token can be pre-generated in an out-of-band manner and configured as a shared secret key between NN and JT to allow JT to make initial authentication to NN. This simple example doesn't deal with token renewal etc, but it helps to illustrate the point that if we can support multiple pluggable token implementations, it opens up the possibility for different users to plug in the token implementation of their choice to bootstrap security. Such token based mechanism has advantages over Kerberos in that 1) it doesn't require Kerberos infrastructure, 2) it leverages existing SASL DIGEST-MD5 auth method and doesn't require adding a new RPC auth method.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-8758) Support for pluggable token
implementations
Posted by "Kan Zhang (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-8758?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13448518#comment-13448518 ]
Kan Zhang commented on HADOOP-8758:
-----------------------------------
Thanks all for your comments.
[~eric14], one use case is exactly the Gateway model you mentioned. Suppose a cluster can be accessed only through a gateway machine and the gateway is configured to authenticate its external clients using some other method (e.g., LDAP). We want to turn on Hadoop security within the cluster to support multi-tenancy, but we don't want to invest in Kerberos. One thing we can do is pre-generate and configure a shared key between Gateway and NN during install and use it to set up a secure connection between Gateway and NN in place of Kerberos. With pluggable interface to support multiple types of delegation tokens, such shared keys can be implemented in the form of a special kind of tokens and only connections authenticated using these special tokens can be used to fetch delegation tokens. In other words, these special tokens are used in place of Kerberos tickets to bootstrap security. The difference is Kerberos mechanism is more general; it can be used by any Kerberized clients without install changes. Whereas here we need to pre-configure our special tokens in a pair-wise manner for any pair of entities that we need to enable authenticated connections. Hence, its usefulness is limited to pre-known services, not arbitrary clients.
[~revans2] and [~daryn], I agree we should allow "SIMPLE" auth to be coupled with tokens.
[~owen.omalley], actually I'm proposing we use the same authentication mechanism as existing delegation tokens use (yes, we may need to consider upgrading the mechanism at some point and it depends on what's available in standard libs). What's new here is a new type of credentials, hence a new type of SecretManager to deal with it. Since the SecretManagers will be stacked together and deal with different types of credentials, they are orthogonal and can evolve independently. HTTP could be used for fetching tokens, but the root issue here is which auth method to use for intra-cluster authentication (ex. between Gateway and NN) and preferably we don't want to introduce an external dependency (ex, LDAP or Kerberos KDC). We could use a shared key based mechanism on HTTP, but that doesn't save us any trouble in configuring the shared keys. We might as well use RPC with its existing token framework.
[~tucu00] Agree with decoupling intra-cluster authentication from user-facing authentication. This JIRA is a step in that direction. Regarding consolidating different token implementations, I see the benefit and have discussed it with you and others. Let's leave it to another topic.
> Support for pluggable token implementations
> -------------------------------------------
>
> Key: HADOOP-8758
> URL: https://issues.apache.org/jira/browse/HADOOP-8758
> Project: Hadoop Common
> Issue Type: Improvement
> Components: ipc, security
> Reporter: Kan Zhang
> Assignee: Kan Zhang
>
> Variants of the delegation token mechanism have been employed by different Hadoop services (NN, JT, RM, etc) to re-authenticate a previously Kerberos-authenticated client. While existing delegation token mechanism compliments Kerberos well, it doesn't necessarily have to be coupled with Kerberos. In principle, delegation tokens can be coupled with any authentication mechanism that bootstraps security. In particular, it can be coupled with other token implementations that use the same DIGEST-MD5 auth method. For example, a token can be pre-generated in an out-of-band manner and configured as a shared secret key between NN and JT to allow JT to make initial authentication to NN. This simple example doesn't deal with token renewal etc, but it helps to illustrate the point that if we can support multiple pluggable token implementations, it opens up the possibility for different users to plug in the token implementation of their choice to bootstrap security. Such token based mechanism has advantages over Kerberos in that 1) it doesn't require Kerberos infrastructure, 2) it leverages existing SASL DIGEST-MD5 auth method and doesn't require adding a new RPC auth method.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-8758) Support for pluggable token
implementations
Posted by "Alejandro Abdelnur (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-8758?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13448081#comment-13448081 ]
Alejandro Abdelnur commented on HADOOP-8758:
--------------------------------------------
A recurrent pattern I've seen in secure deployments is that in many cases they don't want to expose Kerberos outside of the cluster/gateway machines. Specially on the browsers. Due to internal IT rules, they don't want to kerberize enduser computers (many times laptops). Given this, I think we should consider decoupling cluster/inter-services authentication (based on Kerberos) from user facing authentication (based on something more palatable like LDAP or AD).
Regarding the current token mechanism, there are several classes and sub-classses for different components, I wonder why we need all of them instead using a single implementation.
Finally, on Eric's comment, I think most services already provide proxyuser support for propagating caller credentials. JT/NN/Oozie/HttpFS do. May not the ideal mechanism but it works fine.
> Support for pluggable token implementations
> -------------------------------------------
>
> Key: HADOOP-8758
> URL: https://issues.apache.org/jira/browse/HADOOP-8758
> Project: Hadoop Common
> Issue Type: Improvement
> Components: ipc, security
> Reporter: Kan Zhang
> Assignee: Kan Zhang
>
> Variants of the delegation token mechanism have been employed by different Hadoop services (NN, JT, RM, etc) to re-authenticate a previously Kerberos-authenticated client. While existing delegation token mechanism compliments Kerberos well, it doesn't necessarily have to be coupled with Kerberos. In principle, delegation tokens can be coupled with any authentication mechanism that bootstraps security. In particular, it can be coupled with other token implementations that use the same DIGEST-MD5 auth method. For example, a token can be pre-generated in an out-of-band manner and configured as a shared secret key between NN and JT to allow JT to make initial authentication to NN. This simple example doesn't deal with token renewal etc, but it helps to illustrate the point that if we can support multiple pluggable token implementations, it opens up the possibility for different users to plug in the token implementation of their choice to bootstrap security. Such token based mechanism has advantages over Kerberos in that 1) it doesn't require Kerberos infrastructure, 2) it leverages existing SASL DIGEST-MD5 auth method and doesn't require adding a new RPC auth method.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-8758) Support for pluggable token
implementations
Posted by "Daryn Sharp (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-8758?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13448717#comment-13448717 ]
Daryn Sharp commented on HADOOP-8758:
-------------------------------------
Kan, this is a great discusson. Would you clarify the references to needing multiple secret managers? I think we should be able to support multiple auth types with the existing secret managers. The managers are just an authz mechanism that is or should be decoupled from auth. That said, what additional functionality would these other secret managers provide?
> Support for pluggable token implementations
> -------------------------------------------
>
> Key: HADOOP-8758
> URL: https://issues.apache.org/jira/browse/HADOOP-8758
> Project: Hadoop Common
> Issue Type: Improvement
> Components: ipc, security
> Reporter: Kan Zhang
> Assignee: Kan Zhang
>
> Variants of the delegation token mechanism have been employed by different Hadoop services (NN, JT, RM, etc) to re-authenticate a previously Kerberos-authenticated client. While existing delegation token mechanism compliments Kerberos well, it doesn't necessarily have to be coupled with Kerberos. In principle, delegation tokens can be coupled with any authentication mechanism that bootstraps security. In particular, it can be coupled with other token implementations that use the same DIGEST-MD5 auth method. For example, a token can be pre-generated in an out-of-band manner and configured as a shared secret key between NN and JT to allow JT to make initial authentication to NN. This simple example doesn't deal with token renewal etc, but it helps to illustrate the point that if we can support multiple pluggable token implementations, it opens up the possibility for different users to plug in the token implementation of their choice to bootstrap security. Such token based mechanism has advantages over Kerberos in that 1) it doesn't require Kerberos infrastructure, 2) it leverages existing SASL DIGEST-MD5 auth method and doesn't require adding a new RPC auth method.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-8758) Support for pluggable token
implementations
Posted by "eric baldeschwieler (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-8758?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13450776#comment-13450776 ]
eric baldeschwieler commented on HADOOP-8758:
---------------------------------------------
cool!
> Support for pluggable token implementations
> -------------------------------------------
>
> Key: HADOOP-8758
> URL: https://issues.apache.org/jira/browse/HADOOP-8758
> Project: Hadoop Common
> Issue Type: Improvement
> Components: ipc, security
> Reporter: Kan Zhang
> Assignee: Kan Zhang
>
> Variants of the delegation token mechanism have been employed by different Hadoop services (NN, JT, RM, etc) to re-authenticate a previously Kerberos-authenticated client. While existing delegation token mechanism compliments Kerberos well, it doesn't necessarily have to be coupled with Kerberos. In principle, delegation tokens can be coupled with any authentication mechanism that bootstraps security. In particular, it can be coupled with other token implementations that use the same DIGEST-MD5 auth method. For example, a token can be pre-generated in an out-of-band manner and configured as a shared secret key between NN and JT to allow JT to make initial authentication to NN. This simple example doesn't deal with token renewal etc, but it helps to illustrate the point that if we can support multiple pluggable token implementations, it opens up the possibility for different users to plug in the token implementation of their choice to bootstrap security. Such token based mechanism has advantages over Kerberos in that 1) it doesn't require Kerberos infrastructure, 2) it leverages existing SASL DIGEST-MD5 auth method and doesn't require adding a new RPC auth method.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-8758) Support for pluggable token
implementations
Posted by "Daryn Sharp (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-8758?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13448161#comment-13448161 ]
Daryn Sharp commented on HADOOP-8758:
-------------------------------------
@Eric: Today, hadoop's rpc security as all predicated on kerberos so the gateway daemon would require kerberos auth (keytab) to communicate with other hadoop daemons. The NN and other daemons issuing tokens need to be configured to trust the gateway daemon as a quasi-root user so it can pretend to be other users. This is the "proxy token" support currently used by oozie. The oozie server is kerberos auth-ed and the other daemons allow oozie to request tokens to masquerade as other users.
Proxy tokens provide hadoop with rudimentary support for non-kerberos auth, albeit in a questionably secure way. If I could successfully exploit an oozie server or gateway and gain access to its keytab, then I could access almost anyone's data. It would be more secure if the hadoop daemons directly support other auth methods to avoid having quasi-root users configured within the system.
bq. I really like the idea of code path elimination!
Me too, especially in this case! Less security code paths by always using tokens for authz would be a dream come true. The code would be streamlined, and devs w/o access to a secure cluster would know if they broke tokens, regardless of whether kerberos, ldap, etc, or no security is enabled.
> Support for pluggable token implementations
> -------------------------------------------
>
> Key: HADOOP-8758
> URL: https://issues.apache.org/jira/browse/HADOOP-8758
> Project: Hadoop Common
> Issue Type: Improvement
> Components: ipc, security
> Reporter: Kan Zhang
> Assignee: Kan Zhang
>
> Variants of the delegation token mechanism have been employed by different Hadoop services (NN, JT, RM, etc) to re-authenticate a previously Kerberos-authenticated client. While existing delegation token mechanism compliments Kerberos well, it doesn't necessarily have to be coupled with Kerberos. In principle, delegation tokens can be coupled with any authentication mechanism that bootstraps security. In particular, it can be coupled with other token implementations that use the same DIGEST-MD5 auth method. For example, a token can be pre-generated in an out-of-band manner and configured as a shared secret key between NN and JT to allow JT to make initial authentication to NN. This simple example doesn't deal with token renewal etc, but it helps to illustrate the point that if we can support multiple pluggable token implementations, it opens up the possibility for different users to plug in the token implementation of their choice to bootstrap security. Such token based mechanism has advantages over Kerberos in that 1) it doesn't require Kerberos infrastructure, 2) it leverages existing SASL DIGEST-MD5 auth method and doesn't require adding a new RPC auth method.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-8758) Support for pluggable token
implementations
Posted by "Daryn Sharp (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-8758?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13452039#comment-13452039 ]
Daryn Sharp commented on HADOOP-8758:
-------------------------------------
bq. Today, in terms of RPC, we have one external auth method (Kerberos) and one internal auth method (DIGEST-MD5). And they are tightly coupled. By breaking that tight coupling, we can make external auth methods (SIMPLE auth can be one, LDAP can be another) configurable.
Kerberos and tokens are _not_ coupled in any form. The secret managers know nothing about the authentication method. Multiple auth methods, such as LDAP, can currently be added with no change to the secret managers.
The blocking issue is that hadoop's SASL DIGEST-MD5 callbacks directly call the token secret managers. What it should do is call into a {{LoginContext}} which can be configured for multiple auth methods.
> Support for pluggable token implementations
> -------------------------------------------
>
> Key: HADOOP-8758
> URL: https://issues.apache.org/jira/browse/HADOOP-8758
> Project: Hadoop Common
> Issue Type: Improvement
> Components: ipc, security
> Reporter: Kan Zhang
> Assignee: Kan Zhang
>
> Variants of the delegation token mechanism have been employed by different Hadoop services (NN, JT, RM, etc) to re-authenticate a previously Kerberos-authenticated client. While existing delegation token mechanism compliments Kerberos well, it doesn't necessarily have to be coupled with Kerberos. In principle, delegation tokens can be coupled with any authentication mechanism that bootstraps security. In particular, it can be coupled with other token implementations that use the same DIGEST-MD5 auth method. For example, a token can be pre-generated in an out-of-band manner and configured as a shared secret key between NN and JT to allow JT to make initial authentication to NN. This simple example doesn't deal with token renewal etc, but it helps to illustrate the point that if we can support multiple pluggable token implementations, it opens up the possibility for different users to plug in the token implementation of their choice to bootstrap security. Such token based mechanism has advantages over Kerberos in that 1) it doesn't require Kerberos infrastructure, 2) it leverages existing SASL DIGEST-MD5 auth method and doesn't require adding a new RPC auth method.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-8758) Support for pluggable token
implementations
Posted by "Kan Zhang (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-8758?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13451646#comment-13451646 ]
Kan Zhang commented on HADOOP-8758:
-----------------------------------
> For example, one may need to set up pair-wise keys for NN/JT and Oozie/JT so that they can make initial authenticated connections. Those pair-wise keys may be implemented by different SecretManagers. On JT, both SecretManagers need to be configured for external auth using DIGEST-MD5.
Sorry, NN doesn't initiate connections to JT. I should have used JT/NN and Oozie/NN as examples. In this case, both SecretManagers need to be configured on NN for JT and Oozie to make initial authenticated connections to NN.
> Support for pluggable token implementations
> -------------------------------------------
>
> Key: HADOOP-8758
> URL: https://issues.apache.org/jira/browse/HADOOP-8758
> Project: Hadoop Common
> Issue Type: Improvement
> Components: ipc, security
> Reporter: Kan Zhang
> Assignee: Kan Zhang
>
> Variants of the delegation token mechanism have been employed by different Hadoop services (NN, JT, RM, etc) to re-authenticate a previously Kerberos-authenticated client. While existing delegation token mechanism compliments Kerberos well, it doesn't necessarily have to be coupled with Kerberos. In principle, delegation tokens can be coupled with any authentication mechanism that bootstraps security. In particular, it can be coupled with other token implementations that use the same DIGEST-MD5 auth method. For example, a token can be pre-generated in an out-of-band manner and configured as a shared secret key between NN and JT to allow JT to make initial authentication to NN. This simple example doesn't deal with token renewal etc, but it helps to illustrate the point that if we can support multiple pluggable token implementations, it opens up the possibility for different users to plug in the token implementation of their choice to bootstrap security. Such token based mechanism has advantages over Kerberos in that 1) it doesn't require Kerberos infrastructure, 2) it leverages existing SASL DIGEST-MD5 auth method and doesn't require adding a new RPC auth method.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-8758) Support for pluggable token
implementations
Posted by "Kan Zhang (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-8758?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13456451#comment-13456451 ]
Kan Zhang commented on HADOOP-8758:
-----------------------------------
bq. Kerberos and tokens are not coupled in any form. The secret managers know nothing about the authentication method. Multiple auth methods, such as LDAP, can currently be added with no change to the secret managers.
Kerberos and tokens are coupled in the sense that today delegation tokens can only be fetched if authenticated over Kerberos. This JIRA is to break that coupling such that connections authenticated over a special type of tokens (depending on configuration) are also allowed to fetch delegation tokens.
bq. The blocking issue is that hadoop's SASL DIGEST-MD5 callbacks directly call the token secret managers. What it should do is call into a LoginContext which can be configured for multiple auth methods.
Secret managers were created to serve SASL DIGEST-MD5 callbacks. They are credential managers for DIGEST-MD5 mechanism. If you are talking about adding auth methods other than DIGEST-MD5, the right place is adding a SASL mechanism, not overloading DIGEST-MD5.
> Support for pluggable token implementations
> -------------------------------------------
>
> Key: HADOOP-8758
> URL: https://issues.apache.org/jira/browse/HADOOP-8758
> Project: Hadoop Common
> Issue Type: Improvement
> Components: ipc, security
> Reporter: Kan Zhang
> Assignee: Kan Zhang
>
> Variants of the delegation token mechanism have been employed by different Hadoop services (NN, JT, RM, etc) to re-authenticate a previously Kerberos-authenticated client. While existing delegation token mechanism compliments Kerberos well, it doesn't necessarily have to be coupled with Kerberos. In principle, delegation tokens can be coupled with any authentication mechanism that bootstraps security. In particular, it can be coupled with other token implementations that use the same DIGEST-MD5 auth method. For example, a token can be pre-generated in an out-of-band manner and configured as a shared secret key between NN and JT to allow JT to make initial authentication to NN. This simple example doesn't deal with token renewal etc, but it helps to illustrate the point that if we can support multiple pluggable token implementations, it opens up the possibility for different users to plug in the token implementation of their choice to bootstrap security. Such token based mechanism has advantages over Kerberos in that 1) it doesn't require Kerberos infrastructure, 2) it leverages existing SASL DIGEST-MD5 auth method and doesn't require adding a new RPC auth method.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-8758) Support for pluggable token
implementations
Posted by "Daryn Sharp (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-8758?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13450662#comment-13450662 ]
Daryn Sharp commented on HADOOP-8758:
-------------------------------------
I've been experimenting with allowing simple auth to issue and allow tokens, but not require them for backwards compat. I am very excited to find it isn't very hard to modify the NN/DFS and prove it works via {{FsShell}}. However a lot of {{UGI.isSecurityEnabled()}} throughout hadoop need to be visited.
I'll file another jira, probably with sub-tasks to keep it simple -- no pun intended. This work will require the removal hardcoded assumptions of kerberos which should make pluggable authentication much easier.
> Support for pluggable token implementations
> -------------------------------------------
>
> Key: HADOOP-8758
> URL: https://issues.apache.org/jira/browse/HADOOP-8758
> Project: Hadoop Common
> Issue Type: Improvement
> Components: ipc, security
> Reporter: Kan Zhang
> Assignee: Kan Zhang
>
> Variants of the delegation token mechanism have been employed by different Hadoop services (NN, JT, RM, etc) to re-authenticate a previously Kerberos-authenticated client. While existing delegation token mechanism compliments Kerberos well, it doesn't necessarily have to be coupled with Kerberos. In principle, delegation tokens can be coupled with any authentication mechanism that bootstraps security. In particular, it can be coupled with other token implementations that use the same DIGEST-MD5 auth method. For example, a token can be pre-generated in an out-of-band manner and configured as a shared secret key between NN and JT to allow JT to make initial authentication to NN. This simple example doesn't deal with token renewal etc, but it helps to illustrate the point that if we can support multiple pluggable token implementations, it opens up the possibility for different users to plug in the token implementation of their choice to bootstrap security. Such token based mechanism has advantages over Kerberos in that 1) it doesn't require Kerberos infrastructure, 2) it leverages existing SASL DIGEST-MD5 auth method and doesn't require adding a new RPC auth method.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-8758) Support for pluggable token
implementations
Posted by "Kan Zhang (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-8758?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13451452#comment-13451452 ]
Kan Zhang commented on HADOOP-8758:
-----------------------------------
Daryn, I agree we should support multiple external authentication methods and we should make external auth methods configurable by the user. By "external", I meant the auth methods that are allowed to fetch delegation tokens. These external auth methods are typically used when clients make initial connections to a Hadoop cluster. After that, delegation tokens may be fetched and future connections may be set up using an internal auth method (with those fetched delegation tokens). Today, in terms of RPC, we have one external auth method (Kerberos) and one internal auth method (DIGEST-MD5). And they are tightly coupled. By breaking that tight coupling, we can make external auth methods (SIMPLE auth can be one, LDAP can be another) configurable. I should point out SecretManagers are not authorization methods, nor are they authentication methods. They are just different ways to set up credentials (shared keys) to be used with DIGEST-MD5 auth method. We could consider supporting multiple internal auth methods, but currently I don't see a need. The purpose of this JIRA is to allow DIGEST-MD5 to be used also as an external auth method, with a different way to set up credentials (tokens are pre-generated and pre-configured). Did I answer your question?
> Support for pluggable token implementations
> -------------------------------------------
>
> Key: HADOOP-8758
> URL: https://issues.apache.org/jira/browse/HADOOP-8758
> Project: Hadoop Common
> Issue Type: Improvement
> Components: ipc, security
> Reporter: Kan Zhang
> Assignee: Kan Zhang
>
> Variants of the delegation token mechanism have been employed by different Hadoop services (NN, JT, RM, etc) to re-authenticate a previously Kerberos-authenticated client. While existing delegation token mechanism compliments Kerberos well, it doesn't necessarily have to be coupled with Kerberos. In principle, delegation tokens can be coupled with any authentication mechanism that bootstraps security. In particular, it can be coupled with other token implementations that use the same DIGEST-MD5 auth method. For example, a token can be pre-generated in an out-of-band manner and configured as a shared secret key between NN and JT to allow JT to make initial authentication to NN. This simple example doesn't deal with token renewal etc, but it helps to illustrate the point that if we can support multiple pluggable token implementations, it opens up the possibility for different users to plug in the token implementation of their choice to bootstrap security. Such token based mechanism has advantages over Kerberos in that 1) it doesn't require Kerberos infrastructure, 2) it leverages existing SASL DIGEST-MD5 auth method and doesn't require adding a new RPC auth method.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-8758) Support for pluggable token
implementations
Posted by "Kan Zhang (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-8758?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13446789#comment-13446789 ]
Kan Zhang commented on HADOOP-8758:
-----------------------------------
Upon initial look at the code, it seems the changes required to the RPC layer is small and low risk. The main changes seem to be passing in a list of SecretManagers, instead of just one, to the constructor of SaslDigestCallbackHandler. And the callback handler's private method getPassorword() needs to consult all SecretManagers in turn and return the first password found. No changes to the flow of connection setup. Changes are localized to how we return the required token password when given a token identifier.
> Support for pluggable token implementations
> -------------------------------------------
>
> Key: HADOOP-8758
> URL: https://issues.apache.org/jira/browse/HADOOP-8758
> Project: Hadoop Common
> Issue Type: Improvement
> Components: ipc, security
> Reporter: Kan Zhang
> Assignee: Kan Zhang
>
> Variants of the delegation token mechanism have been employed by different Hadoop services (NN, JT, RM, etc) to re-authenticate a previously Kerberos-authenticated client. While existing delegation token mechanism compliments Kerberos well, it doesn't necessarily have to be coupled with Kerberos. In principle, delegation tokens can be coupled with any authentication mechanism that bootstraps security. In particular, it can be coupled with other token implementations that use the same DIGEST-MD5 auth method. For example, a token can be pre-generated in an out-of-band manner and configured as a shared secret key between NN and JT to allow JT to make initial authentication to NN. This simple example doesn't deal with token renewal etc, but it helps to illustrate the point that if we can support multiple pluggable token implementations, it opens up the possibility for different users to plug in the token implementation of their choice to bootstrap security. Such token based mechanism has advantages over Kerberos in that 1) it doesn't require Kerberos infrastructure, 2) it leverages existing SASL DIGEST-MD5 auth method and doesn't require adding a new RPC auth method.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-8758) Support for pluggable token
implementations
Posted by "Kan Zhang (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-8758?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13461308#comment-13461308 ]
Kan Zhang commented on HADOOP-8758:
-----------------------------------
Daryn, as I explained before, this JIRA is to remove the requirement on Kerberos as the only external auth method and allow for other auth methods like SIMPLE and, in particular, DIGEST-MD5 itself to be used as external auth methods. If you want to do the removing in HADOOP-8779 and enable SIMPLE auth to be used as an alternative to Kerberos, that's fine. Once HADOOP-8779 is done, this JIRA will be trivial.
You might have some misunderstanding of DIGEST-MD5 and how it's used in Hadoop RPC. You can't delegate SASL DIGEST-MD5 callbacks to a LoginContext which validates client credentials on LDAP, since in DIGEST-MD5, clients will not send their credentials to server. DIGEST-MD5 is a challenge-response protocol. Server needs to know client credentials (token passwords) to perform authentication and hence the need for callbacks. Secret managers are there to serve SASL DIGEST-MD5 callbacks.
> Support for pluggable token implementations
> -------------------------------------------
>
> Key: HADOOP-8758
> URL: https://issues.apache.org/jira/browse/HADOOP-8758
> Project: Hadoop Common
> Issue Type: Improvement
> Components: ipc, security
> Reporter: Kan Zhang
> Assignee: Kan Zhang
>
> Variants of the delegation token mechanism have been employed by different Hadoop services (NN, JT, RM, etc) to re-authenticate a previously Kerberos-authenticated client. While existing delegation token mechanism compliments Kerberos well, it doesn't necessarily have to be coupled with Kerberos. In principle, delegation tokens can be coupled with any authentication mechanism that bootstraps security. In particular, it can be coupled with other token implementations that use the same DIGEST-MD5 auth method. For example, a token can be pre-generated in an out-of-band manner and configured as a shared secret key between NN and JT to allow JT to make initial authentication to NN. This simple example doesn't deal with token renewal etc, but it helps to illustrate the point that if we can support multiple pluggable token implementations, it opens up the possibility for different users to plug in the token implementation of their choice to bootstrap security. Such token based mechanism has advantages over Kerberos in that 1) it doesn't require Kerberos infrastructure, 2) it leverages existing SASL DIGEST-MD5 auth method and doesn't require adding a new RPC auth method.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-8758) Support for pluggable token
implementations
Posted by "Kan Zhang (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-8758?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13446795#comment-13446795 ]
Kan Zhang commented on HADOOP-8758:
-----------------------------------
Existing SecretManager is always included in the list and is consulted first. Others can be made configurable by the users. This way there is no change to existing behavior.
> Support for pluggable token implementations
> -------------------------------------------
>
> Key: HADOOP-8758
> URL: https://issues.apache.org/jira/browse/HADOOP-8758
> Project: Hadoop Common
> Issue Type: Improvement
> Components: ipc, security
> Reporter: Kan Zhang
> Assignee: Kan Zhang
>
> Variants of the delegation token mechanism have been employed by different Hadoop services (NN, JT, RM, etc) to re-authenticate a previously Kerberos-authenticated client. While existing delegation token mechanism compliments Kerberos well, it doesn't necessarily have to be coupled with Kerberos. In principle, delegation tokens can be coupled with any authentication mechanism that bootstraps security. In particular, it can be coupled with other token implementations that use the same DIGEST-MD5 auth method. For example, a token can be pre-generated in an out-of-band manner and configured as a shared secret key between NN and JT to allow JT to make initial authentication to NN. This simple example doesn't deal with token renewal etc, but it helps to illustrate the point that if we can support multiple pluggable token implementations, it opens up the possibility for different users to plug in the token implementation of their choice to bootstrap security. Such token based mechanism has advantages over Kerberos in that 1) it doesn't require Kerberos infrastructure, 2) it leverages existing SASL DIGEST-MD5 auth method and doesn't require adding a new RPC auth method.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-8758) Support for pluggable token
implementations
Posted by "eric baldeschwieler (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-8758?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13447480#comment-13447480 ]
eric baldeschwieler commented on HADOOP-8758:
---------------------------------------------
Sounds useful.
Can anyone expand on possible use cases? If we can sort out how to make simple things simpler and more complicated things possible, than this could be a good direction to go.
> Support for pluggable token implementations
> -------------------------------------------
>
> Key: HADOOP-8758
> URL: https://issues.apache.org/jira/browse/HADOOP-8758
> Project: Hadoop Common
> Issue Type: Improvement
> Components: ipc, security
> Reporter: Kan Zhang
> Assignee: Kan Zhang
>
> Variants of the delegation token mechanism have been employed by different Hadoop services (NN, JT, RM, etc) to re-authenticate a previously Kerberos-authenticated client. While existing delegation token mechanism compliments Kerberos well, it doesn't necessarily have to be coupled with Kerberos. In principle, delegation tokens can be coupled with any authentication mechanism that bootstraps security. In particular, it can be coupled with other token implementations that use the same DIGEST-MD5 auth method. For example, a token can be pre-generated in an out-of-band manner and configured as a shared secret key between NN and JT to allow JT to make initial authentication to NN. This simple example doesn't deal with token renewal etc, but it helps to illustrate the point that if we can support multiple pluggable token implementations, it opens up the possibility for different users to plug in the token implementation of their choice to bootstrap security. Such token based mechanism has advantages over Kerberos in that 1) it doesn't require Kerberos infrastructure, 2) it leverages existing SASL DIGEST-MD5 auth method and doesn't require adding a new RPC auth method.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-8758) Support for pluggable token
implementations
Posted by "Robert Joseph Evans (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-8758?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13447676#comment-13447676 ]
Robert Joseph Evans commented on HADOOP-8758:
---------------------------------------------
One really nice use case that Daryn Sharp has talked a lot about is testing. If you could easily replace Kerberos with a mock all of the unit tests can be run with security enabled. There are a lot of security bugs that creep in because we cannot test with security on all the time.
It also would allow us to use tokens when "security" is off. This would reduce the number of code paths dramatically which in turn is will improve the coverage of the unit tests. I don't know of many people that really will want to replace Kerberos entirely in their grids some might, but at a minimum from a testing and stability perspective it is very interesting.
> Support for pluggable token implementations
> -------------------------------------------
>
> Key: HADOOP-8758
> URL: https://issues.apache.org/jira/browse/HADOOP-8758
> Project: Hadoop Common
> Issue Type: Improvement
> Components: ipc, security
> Reporter: Kan Zhang
> Assignee: Kan Zhang
>
> Variants of the delegation token mechanism have been employed by different Hadoop services (NN, JT, RM, etc) to re-authenticate a previously Kerberos-authenticated client. While existing delegation token mechanism compliments Kerberos well, it doesn't necessarily have to be coupled with Kerberos. In principle, delegation tokens can be coupled with any authentication mechanism that bootstraps security. In particular, it can be coupled with other token implementations that use the same DIGEST-MD5 auth method. For example, a token can be pre-generated in an out-of-band manner and configured as a shared secret key between NN and JT to allow JT to make initial authentication to NN. This simple example doesn't deal with token renewal etc, but it helps to illustrate the point that if we can support multiple pluggable token implementations, it opens up the possibility for different users to plug in the token implementation of their choice to bootstrap security. Such token based mechanism has advantages over Kerberos in that 1) it doesn't require Kerberos infrastructure, 2) it leverages existing SASL DIGEST-MD5 auth method and doesn't require adding a new RPC auth method.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-8758) Support for pluggable token
implementations
Posted by "eric baldeschwieler (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-8758?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13448057#comment-13448057 ]
eric baldeschwieler commented on HADOOP-8758:
---------------------------------------------
I really like the idea of code path elimination!
What about the use case of a proxy, where all external work comes via a gateway that already has authenticated the user via LDAP or some other mechanism? If all user interaction comes from this central point, can we hook that up to the existing token mechanism and maintain security?
- Gateway -> HDFS
- Gateway -> MR -> HDFS
- Gateway -> MR -> HBase -> HDFS
- Gateway -> Oozie -> MR -> Hive -> (HCat & MR) -> HDFS
> Support for pluggable token implementations
> -------------------------------------------
>
> Key: HADOOP-8758
> URL: https://issues.apache.org/jira/browse/HADOOP-8758
> Project: Hadoop Common
> Issue Type: Improvement
> Components: ipc, security
> Reporter: Kan Zhang
> Assignee: Kan Zhang
>
> Variants of the delegation token mechanism have been employed by different Hadoop services (NN, JT, RM, etc) to re-authenticate a previously Kerberos-authenticated client. While existing delegation token mechanism compliments Kerberos well, it doesn't necessarily have to be coupled with Kerberos. In principle, delegation tokens can be coupled with any authentication mechanism that bootstraps security. In particular, it can be coupled with other token implementations that use the same DIGEST-MD5 auth method. For example, a token can be pre-generated in an out-of-band manner and configured as a shared secret key between NN and JT to allow JT to make initial authentication to NN. This simple example doesn't deal with token renewal etc, but it helps to illustrate the point that if we can support multiple pluggable token implementations, it opens up the possibility for different users to plug in the token implementation of their choice to bootstrap security. Such token based mechanism has advantages over Kerberos in that 1) it doesn't require Kerberos infrastructure, 2) it leverages existing SASL DIGEST-MD5 auth method and doesn't require adding a new RPC auth method.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira