You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@knox.apache.org by Andrew Bumstead <an...@bigdatapartnership.com> on 2015/11/24 16:56:10 UTC
Issue setting up Knox to use SSO provider
Hello,
I am looking to create a set up using an SSO provider (such as ForgeRocks
OpenAM) to issue JWT authentication tokens that allow access to cluster
services through Knox. So far having followed the available documentation I
haven't been able to get SSO working in Knox.
My set up consists of Knox running in a sandbox with the rest of HDP 2.3,
and OpenAM running as a tomcat webapp on my local machine.
I have added the following to my topology:
<gateway>
<provider>
<role>federation</role>
<name>HeaderPreAuth</name>
<enabled>true</enabled>
<param>
<name>preauth.validation.method</name>
<value>preauth.ip.validation</value>
</param>
<param>
<name>preauth.ip.addresses</name>
<value>127.100.0.1,127.0.0.1,10.0.2.2</value>
</param>
<param>
<name>preauth.custom.header</name>
<value>USER</value>
</param>
</provider>
Using this configuration and executing the following command returns the
listing of HDFS user folders, however performs no validation on the USER
header 'root'. The logs suggest OpenAM isn't being used to validate it.
curl -ik --header "USER:root" '
https://127.0.0.1:8443/gateway/default/webhdfs/v1/user?op=LISTSTATUS'
It is not clear to me how Knox should be configured to use the correct SSO
provider, such as the URL, port and protocol. Would you be able to provide
any information as to how this can currently be done on the Knox platform?
Many Thanks,
Andrew Bumstead
--
*NOTICE AND DISCLAIMER*
This email (including attachments) is confidential. If you are not the
intended recipient, notify the sender immediately, delete this email from
your system and do not disclose or use for any purpose.
Business Address: Eagle House, 163 City Road, London, EC1V 1NR. United
Kingdom
Registered Office: Finsgate, 5-7 Cranwood Street, London, EC1V 9EE. United
Kingdom
Big Data Partnership Limited is a company registered in England & Wales
with Company No 7904824
Re: Issue setting up Knox to use SSO provider
Posted by larry mccay <la...@gmail.com>.
Glad that cleared it up for you.
And...yes, the functionality that you are looking for isn't quite there yet.
The hard part is already there which is the JWT parsing and validation.
What's missing is the signature verification for tokens that come from an
issuer other than Knox itself.
On Wed, Nov 25, 2015 at 4:26 AM, Andrew Bumstead <
andrew.bumstead@bigdatapartnership.com> wrote:
> Hi Larry,
>
> Thanks for the clarification, that's very helpful. I did misunderstand the
> SSO capability in the documentation. I'll do some more research into using
> JWT authentication, but from what you've said it does sound like the
> functionality I'm looking for isn't built in.
>
> Thanks,
>
> Andrew
>
> On 24 November 2015 at 16:19, larry mccay <la...@gmail.com> wrote:
>
>> Hi Andrew -
>>
>> HeaderPreAuth is not intended for tokens that need validation.
>> Preauthentication implementations, such as this, assume that access to
>> the endpoint is gated by a proxying authentication proxy and that there is
>> sufficient trust between the proxy and the recipient of a HTTP Header that
>> it can be blindly trusted.
>>
>> For instance, SiteMinder (and solutions like it) allow you to configure
>> the propagation of certain aspects of the authentication event to be done
>> through HTTP Headers to the downstream app. The downstream app assumes that
>> there is no way to access it other than through SiteMinder and can trust
>> the SM_USER header value.
>>
>> The following covers it pretty well but if you have read it and not
>> understood it this way then it should be made more clear.
>>
>> http://knox.apache.org/books/knox-0-6-0/user-guide.html#Preauthenticated+SSO+Provider
>>
>> What you are looking for is direct support for JWT tokens being delivered
>> as a header.
>>
>> We don't currently have an implementation that can accept JWT tokens that
>> are issued by arbitrary authorities. We do have the ability to issue a
>> token to a requesting client and subsequently accept that token as a bearer
>> token. Note that this is a specific type of header and not just a named
>> header.
>>
>> I think that there would be great value in extending our implementation
>> to be able to verify the signature of some other issuer and accepting the
>> principal from the incoming token.
>>
>> You can find the AccessTokenFederationFilter which would accept JWT
>> bearer tokens as access tokens in:
>>
>>
>> https://github.com/apache/knox/blob/d6f6f6efc10b19a487c9101f2f0261f978f19d5e/gateway-provider-security-jwt/src/main/java/org/apache/hadoop/gateway/provider/federation/jwt/filter/AccessTokenFederationFilter.java
>>
>> As I said though, the verification of the token signature would need to
>> be changed to look up the public key for the issuer of the token - this
>> would need to be done in:
>>
>>
>> https://github.com/apache/knox/blob/58ffaf2136bd065b5a6d74444611919cf15dad8a/gateway-server/src/main/java/org/apache/hadoop/gateway/services/token/impl/DefaultTokenAuthorityService.java
>>
>> If this is of particular interest to you then I would suggest filing a
>> JIRA for it.
>> If you are interested in contributing to the required enhancement - we
>> would be happy to help you along and grateful for the hand. :)
>>
>> thanks,
>>
>> --larry
>>
>>
>> On Tue, Nov 24, 2015 at 10:56 AM, Andrew Bumstead <
>> andrew.bumstead@bigdatapartnership.com> wrote:
>>
>>> Hello,
>>>
>>> I am looking to create a set up using an SSO provider (such as
>>> ForgeRocks OpenAM) to issue JWT authentication tokens that allow access to
>>> cluster services through Knox. So far having followed the available
>>> documentation I haven't been able to get SSO working in Knox.
>>>
>>> My set up consists of Knox running in a sandbox with the rest of HDP
>>> 2.3, and OpenAM running as a tomcat webapp on my local machine.
>>>
>>> I have added the following to my topology:
>>>
>>> <gateway>
>>> <provider>
>>> <role>federation</role>
>>> <name>HeaderPreAuth</name>
>>> <enabled>true</enabled>
>>> <param>
>>> <name>preauth.validation.method</name>
>>> <value>preauth.ip.validation</value>
>>> </param>
>>> <param>
>>> <name>preauth.ip.addresses</name>
>>> <value>127.100.0.1,127.0.0.1,10.0.2.2</value>
>>> </param>
>>> <param>
>>> <name>preauth.custom.header</name>
>>> <value>USER</value>
>>> </param>
>>> </provider>
>>>
>>> Using this configuration and executing the following command returns the
>>> listing of HDFS user folders, however performs no validation on the USER
>>> header 'root'. The logs suggest OpenAM isn't being used to validate it.
>>>
>>> curl -ik --header "USER:root" '
>>> https://127.0.0.1:8443/gateway/default/webhdfs/v1/user?op=LISTSTATUS'
>>>
>>> It is not clear to me how Knox should be configured to use the correct
>>> SSO provider, such as the URL, port and protocol. Would you be able to
>>> provide any information as to how this can currently be done on the Knox
>>> platform?
>>>
>>> Many Thanks,
>>>
>>> Andrew Bumstead
>>>
>>> *NOTICE AND DISCLAIMER*
>>>
>>> This email (including attachments) is confidential. If you are not the
>>> intended recipient, notify the sender immediately, delete this email from
>>> your system and do not disclose or use for any purpose.
>>>
>>> Business Address: Eagle House, 163 City Road, London, EC1V 1NR. United
>>> Kingdom
>>> Registered Office: Finsgate, 5-7 Cranwood Street, London, EC1V 9EE.
>>> United Kingdom
>>> Big Data Partnership Limited is a company registered in England & Wales
>>> with Company No 7904824
>>>
>>
>>
>
> *NOTICE AND DISCLAIMER*
>
> This email (including attachments) is confidential. If you are not the
> intended recipient, notify the sender immediately, delete this email from
> your system and do not disclose or use for any purpose.
>
> Business Address: Eagle House, 163 City Road, London, EC1V 1NR. United
> Kingdom
> Registered Office: Finsgate, 5-7 Cranwood Street, London, EC1V 9EE. United
> Kingdom
> Big Data Partnership Limited is a company registered in England & Wales
> with Company No 7904824
>
Re: Issue setting up Knox to use SSO provider
Posted by Andrew Bumstead <an...@bigdatapartnership.com>.
Hi Larry,
Thanks for the clarification, that's very helpful. I did misunderstand the
SSO capability in the documentation. I'll do some more research into using
JWT authentication, but from what you've said it does sound like the
functionality I'm looking for isn't built in.
Thanks,
Andrew
On 24 November 2015 at 16:19, larry mccay <la...@gmail.com> wrote:
> Hi Andrew -
>
> HeaderPreAuth is not intended for tokens that need validation.
> Preauthentication implementations, such as this, assume that access to the
> endpoint is gated by a proxying authentication proxy and that there is
> sufficient trust between the proxy and the recipient of a HTTP Header that
> it can be blindly trusted.
>
> For instance, SiteMinder (and solutions like it) allow you to configure
> the propagation of certain aspects of the authentication event to be done
> through HTTP Headers to the downstream app. The downstream app assumes that
> there is no way to access it other than through SiteMinder and can trust
> the SM_USER header value.
>
> The following covers it pretty well but if you have read it and not
> understood it this way then it should be made more clear.
>
> http://knox.apache.org/books/knox-0-6-0/user-guide.html#Preauthenticated+SSO+Provider
>
> What you are looking for is direct support for JWT tokens being delivered
> as a header.
>
> We don't currently have an implementation that can accept JWT tokens that
> are issued by arbitrary authorities. We do have the ability to issue a
> token to a requesting client and subsequently accept that token as a bearer
> token. Note that this is a specific type of header and not just a named
> header.
>
> I think that there would be great value in extending our implementation to
> be able to verify the signature of some other issuer and accepting the
> principal from the incoming token.
>
> You can find the AccessTokenFederationFilter which would accept JWT
> bearer tokens as access tokens in:
>
>
> https://github.com/apache/knox/blob/d6f6f6efc10b19a487c9101f2f0261f978f19d5e/gateway-provider-security-jwt/src/main/java/org/apache/hadoop/gateway/provider/federation/jwt/filter/AccessTokenFederationFilter.java
>
> As I said though, the verification of the token signature would need to be
> changed to look up the public key for the issuer of the token - this would
> need to be done in:
>
>
> https://github.com/apache/knox/blob/58ffaf2136bd065b5a6d74444611919cf15dad8a/gateway-server/src/main/java/org/apache/hadoop/gateway/services/token/impl/DefaultTokenAuthorityService.java
>
> If this is of particular interest to you then I would suggest filing a
> JIRA for it.
> If you are interested in contributing to the required enhancement - we
> would be happy to help you along and grateful for the hand. :)
>
> thanks,
>
> --larry
>
>
> On Tue, Nov 24, 2015 at 10:56 AM, Andrew Bumstead <
> andrew.bumstead@bigdatapartnership.com> wrote:
>
>> Hello,
>>
>> I am looking to create a set up using an SSO provider (such as ForgeRocks
>> OpenAM) to issue JWT authentication tokens that allow access to cluster
>> services through Knox. So far having followed the available documentation I
>> haven't been able to get SSO working in Knox.
>>
>> My set up consists of Knox running in a sandbox with the rest of HDP 2.3,
>> and OpenAM running as a tomcat webapp on my local machine.
>>
>> I have added the following to my topology:
>>
>> <gateway>
>> <provider>
>> <role>federation</role>
>> <name>HeaderPreAuth</name>
>> <enabled>true</enabled>
>> <param>
>> <name>preauth.validation.method</name>
>> <value>preauth.ip.validation</value>
>> </param>
>> <param>
>> <name>preauth.ip.addresses</name>
>> <value>127.100.0.1,127.0.0.1,10.0.2.2</value>
>> </param>
>> <param>
>> <name>preauth.custom.header</name>
>> <value>USER</value>
>> </param>
>> </provider>
>>
>> Using this configuration and executing the following command returns the
>> listing of HDFS user folders, however performs no validation on the USER
>> header 'root'. The logs suggest OpenAM isn't being used to validate it.
>>
>> curl -ik --header "USER:root" '
>> https://127.0.0.1:8443/gateway/default/webhdfs/v1/user?op=LISTSTATUS'
>>
>> It is not clear to me how Knox should be configured to use the correct
>> SSO provider, such as the URL, port and protocol. Would you be able to
>> provide any information as to how this can currently be done on the Knox
>> platform?
>>
>> Many Thanks,
>>
>> Andrew Bumstead
>>
>> *NOTICE AND DISCLAIMER*
>>
>> This email (including attachments) is confidential. If you are not the
>> intended recipient, notify the sender immediately, delete this email from
>> your system and do not disclose or use for any purpose.
>>
>> Business Address: Eagle House, 163 City Road, London, EC1V 1NR. United
>> Kingdom
>> Registered Office: Finsgate, 5-7 Cranwood Street, London, EC1V 9EE.
>> United Kingdom
>> Big Data Partnership Limited is a company registered in England & Wales
>> with Company No 7904824
>>
>
>
--
*NOTICE AND DISCLAIMER*
This email (including attachments) is confidential. If you are not the
intended recipient, notify the sender immediately, delete this email from
your system and do not disclose or use for any purpose.
Business Address: Eagle House, 163 City Road, London, EC1V 1NR. United
Kingdom
Registered Office: Finsgate, 5-7 Cranwood Street, London, EC1V 9EE. United
Kingdom
Big Data Partnership Limited is a company registered in England & Wales
with Company No 7904824
Re: Issue setting up Knox to use SSO provider
Posted by larry mccay <la...@gmail.com>.
Hi Andrew -
HeaderPreAuth is not intended for tokens that need validation.
Preauthentication implementations, such as this, assume that access to the
endpoint is gated by a proxying authentication proxy and that there is
sufficient trust between the proxy and the recipient of a HTTP Header that
it can be blindly trusted.
For instance, SiteMinder (and solutions like it) allow you to configure the
propagation of certain aspects of the authentication event to be done
through HTTP Headers to the downstream app. The downstream app assumes that
there is no way to access it other than through SiteMinder and can trust
the SM_USER header value.
The following covers it pretty well but if you have read it and not
understood it this way then it should be made more clear.
http://knox.apache.org/books/knox-0-6-0/user-guide.html#Preauthenticated+SSO+Provider
What you are looking for is direct support for JWT tokens being delivered
as a header.
We don't currently have an implementation that can accept JWT tokens that
are issued by arbitrary authorities. We do have the ability to issue a
token to a requesting client and subsequently accept that token as a bearer
token. Note that this is a specific type of header and not just a named
header.
I think that there would be great value in extending our implementation to
be able to verify the signature of some other issuer and accepting the
principal from the incoming token.
You can find the AccessTokenFederationFilter which would accept JWT bearer
tokens as access tokens in:
https://github.com/apache/knox/blob/d6f6f6efc10b19a487c9101f2f0261f978f19d5e/gateway-provider-security-jwt/src/main/java/org/apache/hadoop/gateway/provider/federation/jwt/filter/AccessTokenFederationFilter.java
As I said though, the verification of the token signature would need to be
changed to look up the public key for the issuer of the token - this would
need to be done in:
https://github.com/apache/knox/blob/58ffaf2136bd065b5a6d74444611919cf15dad8a/gateway-server/src/main/java/org/apache/hadoop/gateway/services/token/impl/DefaultTokenAuthorityService.java
If this is of particular interest to you then I would suggest filing a JIRA
for it.
If you are interested in contributing to the required enhancement - we
would be happy to help you along and grateful for the hand. :)
thanks,
--larry
On Tue, Nov 24, 2015 at 10:56 AM, Andrew Bumstead <
andrew.bumstead@bigdatapartnership.com> wrote:
> Hello,
>
> I am looking to create a set up using an SSO provider (such as ForgeRocks
> OpenAM) to issue JWT authentication tokens that allow access to cluster
> services through Knox. So far having followed the available documentation I
> haven't been able to get SSO working in Knox.
>
> My set up consists of Knox running in a sandbox with the rest of HDP 2.3,
> and OpenAM running as a tomcat webapp on my local machine.
>
> I have added the following to my topology:
>
> <gateway>
> <provider>
> <role>federation</role>
> <name>HeaderPreAuth</name>
> <enabled>true</enabled>
> <param>
> <name>preauth.validation.method</name>
> <value>preauth.ip.validation</value>
> </param>
> <param>
> <name>preauth.ip.addresses</name>
> <value>127.100.0.1,127.0.0.1,10.0.2.2</value>
> </param>
> <param>
> <name>preauth.custom.header</name>
> <value>USER</value>
> </param>
> </provider>
>
> Using this configuration and executing the following command returns the
> listing of HDFS user folders, however performs no validation on the USER
> header 'root'. The logs suggest OpenAM isn't being used to validate it.
>
> curl -ik --header "USER:root" '
> https://127.0.0.1:8443/gateway/default/webhdfs/v1/user?op=LISTSTATUS'
>
> It is not clear to me how Knox should be configured to use the correct SSO
> provider, such as the URL, port and protocol. Would you be able to provide
> any information as to how this can currently be done on the Knox platform?
>
> Many Thanks,
>
> Andrew Bumstead
>
> *NOTICE AND DISCLAIMER*
>
> This email (including attachments) is confidential. If you are not the
> intended recipient, notify the sender immediately, delete this email from
> your system and do not disclose or use for any purpose.
>
> Business Address: Eagle House, 163 City Road, London, EC1V 1NR. United
> Kingdom
> Registered Office: Finsgate, 5-7 Cranwood Street, London, EC1V 9EE. United
> Kingdom
> Big Data Partnership Limited is a company registered in England & Wales
> with Company No 7904824
>