You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@knox.apache.org by Aneela Saleem <an...@platalytics.com> on 2015/07/04 16:56:55 UTC
Apache Knox Web API
Hi Everyone,
I'm going to start development for Hadoop security through Apache Knox. Can
anyone please suggest me some good API for Knox.
So far i have found following:
https://cwiki.apache.org/confluence/display/KNOX/Client+Usage
https://knox.apache.org/books/knox-0-5-0/dev-guide.html#Gateway+Services
Re: Apache Knox Web API
Posted by Aneela Saleem <an...@platalytics.com>.
yes it's listening on 389
On Mon, Jul 27, 2015 at 7:22 PM, Kevin Minder <ke...@hortonworks.com>
wrote:
> I believe the default LDAP port is 389. Is your OpenLDAP server
> listening on 389?
>
> Otherwise would it be possible for you to build and use the master
> branch version of Knox. We have recently added several LDAP diagnostics
> that might help us here.
>
> From: Aneela Saleem
> Reply-To: "user@knox.apache.org"
> Date: Monday, July 27, 2015 at 10:14 AM
> To: "user@knox.apache.org"
> Subject: Re: Apache Knox Web API
>
> Hi Kevin,
>
> I'm using OpenLDAP
>
> On Mon, Jul 27, 2015 at 6:59 PM, Kevin Minder <
> kevin.minder@hortonworks.com> wrote:
>
>> I’m suspecting this
>> <param>
>> <name>main.ldapRealm.contextFactory.url</name>
>> <value>ldap://localhost</value>
>> </param>
>> What LDAP server are you using?
>>
>> From: Aneela Saleem
>> Reply-To: "user@knox.apache.org"
>> Date: Sunday, July 26, 2015 at 2:53 PM
>> To: "user@knox.apache.org"
>> Subject: Re: Apache Knox Web API
>>
>> <param>
>> <name>main.ldapRealm.contextFactory.url</name>
>> <value>ldap://localhost</value>
>> </param>
>>
>
>
Re: Apache Knox Web API
Posted by Aneela Saleem <an...@platalytics.com>.
Hi Kevin,
I reverted LDAP to simple authentication from startTLS. Still knox not
working. I can't figure out the issue. What should i do now?
On Mon, Jul 27, 2015 at 10:43 PM, Kevin Minder <kevin.minder@hortonworks.com
> wrote:
> Yes. As is typical with SSL setup this can be complex but it is
> covered in the User’s Guide.
> http://knox.apache.org/books/knox-0-6-0/user-guide.html#Authentication
>
> From: Aneela Saleem
> Reply-To: "user@knox.apache.org"
> Date: Monday, July 27, 2015 at 1:36 PM
>
> To: "user@knox.apache.org"
> Subject: Re: Apache Knox Web API
>
> Ok what if a forget startTLS thing and start with LDAPS, is it
> supported in Shiro LDAP Realm?
>
> On Mon, Jul 27, 2015 at 8:46 PM, Kevin Minder <
> kevin.minder@hortonworks.com> wrote:
>
>> Ok did a bit more digging and it looks like the Shiro LDAP Realm we are
>> using does not implement StartTLS. It seems as though other Shiro Realm
>> implementations do as evidence here
>>
>> http://jmchung.github.io/blog/2014/10/03/integrating-shiro-with-cas-authentication-via-ldap/
>> But I see no evidence that the JndiLdapRealm upon which the
>> KnoxLdapRealm is based has the code described here for StartTLS support.
>> https://docs.oracle.com/javase/jndi/tutorial/ldap/ext/starttls.html
>> This would be a valuable are for you to contribute to either Knox or
>> Shiro if this capability is important for your use case.
>> Also note that LDAPS should provide equivalent security.
>>
>> From: Aneela Saleem
>> Reply-To: "user@knox.apache.org"
>> Date: Monday, July 27, 2015 at 11:07 AM
>>
>> To: "user@knox.apache.org"
>> Subject: Re: Apache Knox Web API
>>
>> I just tried to enable startTLS for LDAP. I just followed this link:
>>
>>
>> https://www.digitalocean.com/community/tutorials/how-to-encrypt-openldap-connections-using-starttls
>>
>> On Mon, Jul 27, 2015 at 8:02 PM, Kevin Minder <
>> kevin.minder@hortonworks.com> wrote:
>>
>>> Well what have you changed since it last worked?
>>>
>>> From: Aneela Saleem
>>> Reply-To: "user@knox.apache.org"
>>> Date: Monday, July 27, 2015 at 11:01 AM
>>>
>>> To: "user@knox.apache.org"
>>> Subject: Re: Apache Knox Web API
>>>
>>> But what could be the issue as it was working fine before
>>>
>>> On Mon, Jul 27, 2015 at 7:35 PM, Kevin Minder <
>>> kevin.minder@hortonworks.com> wrote:
>>>
>>>> In the development branch (called master) we have added several
>>>> features to help diagnose LDAP issues. However to take advantage of these
>>>> you will need to build Knox from source as these features are not yet
>>>> included in an official release.
>>>>
>>>> From: Aneela Saleem
>>>> Reply-To: "user@knox.apache.org"
>>>> Date: Monday, July 27, 2015 at 10:26 AM
>>>>
>>>> To: "user@knox.apache.org"
>>>> Subject: Re: Apache Knox Web API
>>>>
>>>> But i did not get your point
>>>>
>>>> On Mon, Jul 27, 2015 at 7:22 PM, Kevin Minder <
>>>> kevin.minder@hortonworks.com> wrote:
>>>>
>>>>> I believe the default LDAP port is 389. Is your OpenLDAP server
>>>>> listening on 389?
>>>>>
>>>>> Otherwise would it be possible for you to build and use the master
>>>>> branch version of Knox. We have recently added several LDAP diagnostics
>>>>> that might help us here.
>>>>>
>>>>> From: Aneela Saleem
>>>>> Reply-To: "user@knox.apache.org"
>>>>> Date: Monday, July 27, 2015 at 10:14 AM
>>>>> To: "user@knox.apache.org"
>>>>> Subject: Re: Apache Knox Web API
>>>>>
>>>>> Hi Kevin,
>>>>>
>>>>> I'm using OpenLDAP
>>>>>
>>>>> On Mon, Jul 27, 2015 at 6:59 PM, Kevin Minder <
>>>>> kevin.minder@hortonworks.com> wrote:
>>>>>
>>>>>> I’m suspecting this
>>>>>> <param>
>>>>>> <name>main.ldapRealm.contextFactory.url</name>
>>>>>> <value>ldap://localhost</value>
>>>>>> </param>
>>>>>> What LDAP server are you using?
>>>>>>
>>>>>> From: Aneela Saleem
>>>>>> Reply-To: "user@knox.apache.org"
>>>>>> Date: Sunday, July 26, 2015 at 2:53 PM
>>>>>> To: "user@knox.apache.org"
>>>>>> Subject: Re: Apache Knox Web API
>>>>>>
>>>>>> <param>
>>>>>> <name>main.ldapRealm.contextFactory.url</name>
>>>>>> <value>ldap://localhost</value>
>>>>>> </param>
>>>>>>
>>>>>
>>>>>
>>>>
>>>
>>
>
Re: Apache Knox Web API
Posted by Kevin Minder <ke...@hortonworks.com>.
Yes. As is typical with SSL setup this can be complex but it is covered in the User’s Guide.
http://knox.apache.org/books/knox-0-6-0/user-guide.html#Authentication
From: Aneela Saleem
Reply-To: "user@knox.apache.org<ma...@knox.apache.org>"
Date: Monday, July 27, 2015 at 1:36 PM
To: "user@knox.apache.org<ma...@knox.apache.org>"
Subject: Re: Apache Knox Web API
Ok what if a forget startTLS thing and start with LDAPS, is it supported in Shiro LDAP Realm?
On Mon, Jul 27, 2015 at 8:46 PM, Kevin Minder <ke...@hortonworks.com>> wrote:
Ok did a bit more digging and it looks like the Shiro LDAP Realm we are using does not implement StartTLS. It seems as though other Shiro Realm implementations do as evidence here
http://jmchung.github.io/blog/2014/10/03/integrating-shiro-with-cas-authentication-via-ldap/
But I see no evidence that the JndiLdapRealm upon which the KnoxLdapRealm is based has the code described here for StartTLS support.
https://docs.oracle.com/javase/jndi/tutorial/ldap/ext/starttls.html
This would be a valuable are for you to contribute to either Knox or Shiro if this capability is important for your use case.
Also note that LDAPS should provide equivalent security.
From: Aneela Saleem
Reply-To: "user@knox.apache.org<ma...@knox.apache.org>"
Date: Monday, July 27, 2015 at 11:07 AM
To: "user@knox.apache.org<ma...@knox.apache.org>"
Subject: Re: Apache Knox Web API
I just tried to enable startTLS for LDAP. I just followed this link:
https://www.digitalocean.com/community/tutorials/how-to-encrypt-openldap-connections-using-starttls
On Mon, Jul 27, 2015 at 8:02 PM, Kevin Minder <ke...@hortonworks.com>> wrote:
Well what have you changed since it last worked?
From: Aneela Saleem
Reply-To: "user@knox.apache.org<ma...@knox.apache.org>"
Date: Monday, July 27, 2015 at 11:01 AM
To: "user@knox.apache.org<ma...@knox.apache.org>"
Subject: Re: Apache Knox Web API
But what could be the issue as it was working fine before
On Mon, Jul 27, 2015 at 7:35 PM, Kevin Minder <ke...@hortonworks.com>> wrote:
In the development branch (called master) we have added several features to help diagnose LDAP issues. However to take advantage of these you will need to build Knox from source as these features are not yet included in an official release.
From: Aneela Saleem
Reply-To: "user@knox.apache.org<ma...@knox.apache.org>"
Date: Monday, July 27, 2015 at 10:26 AM
To: "user@knox.apache.org<ma...@knox.apache.org>"
Subject: Re: Apache Knox Web API
But i did not get your point
On Mon, Jul 27, 2015 at 7:22 PM, Kevin Minder <ke...@hortonworks.com>> wrote:
I believe the default LDAP port is 389. Is your OpenLDAP server listening on 389?
Otherwise would it be possible for you to build and use the master branch version of Knox. We have recently added several LDAP diagnostics that might help us here.
From: Aneela Saleem
Reply-To: "user@knox.apache.org<ma...@knox.apache.org>"
Date: Monday, July 27, 2015 at 10:14 AM
To: "user@knox.apache.org<ma...@knox.apache.org>"
Subject: Re: Apache Knox Web API
Hi Kevin,
I'm using OpenLDAP
On Mon, Jul 27, 2015 at 6:59 PM, Kevin Minder <ke...@hortonworks.com>> wrote:
I’m suspecting this
<param>
<name>main.ldapRealm.contextFactory.url</name>
<value>ldap://localhost</value>
</param>
What LDAP server are you using?
From: Aneela Saleem
Reply-To: "user@knox.apache.org<ma...@knox.apache.org>"
Date: Sunday, July 26, 2015 at 2:53 PM
To: "user@knox.apache.org<ma...@knox.apache.org>"
Subject: Re: Apache Knox Web API
<param>
<name>main.ldapRealm.contextFactory.url</name>
<value>ldap://localhost</value>
</param>
Re: Apache Knox Web API
Posted by Aneela Saleem <an...@platalytics.com>.
Ok what if a forget startTLS thing and start with LDAPS, is it supported in
Shiro LDAP Realm?
On Mon, Jul 27, 2015 at 8:46 PM, Kevin Minder <ke...@hortonworks.com>
wrote:
> Ok did a bit more digging and it looks like the Shiro LDAP Realm we are
> using does not implement StartTLS. It seems as though other Shiro Realm
> implementations do as evidence here
>
> http://jmchung.github.io/blog/2014/10/03/integrating-shiro-with-cas-authentication-via-ldap/
> But I see no evidence that the JndiLdapRealm upon which the KnoxLdapRealm
> is based has the code described here for StartTLS support.
> https://docs.oracle.com/javase/jndi/tutorial/ldap/ext/starttls.html
> This would be a valuable are for you to contribute to either Knox or Shiro
> if this capability is important for your use case.
> Also note that LDAPS should provide equivalent security.
>
> From: Aneela Saleem
> Reply-To: "user@knox.apache.org"
> Date: Monday, July 27, 2015 at 11:07 AM
>
> To: "user@knox.apache.org"
> Subject: Re: Apache Knox Web API
>
> I just tried to enable startTLS for LDAP. I just followed this link:
>
>
> https://www.digitalocean.com/community/tutorials/how-to-encrypt-openldap-connections-using-starttls
>
> On Mon, Jul 27, 2015 at 8:02 PM, Kevin Minder <
> kevin.minder@hortonworks.com> wrote:
>
>> Well what have you changed since it last worked?
>>
>> From: Aneela Saleem
>> Reply-To: "user@knox.apache.org"
>> Date: Monday, July 27, 2015 at 11:01 AM
>>
>> To: "user@knox.apache.org"
>> Subject: Re: Apache Knox Web API
>>
>> But what could be the issue as it was working fine before
>>
>> On Mon, Jul 27, 2015 at 7:35 PM, Kevin Minder <
>> kevin.minder@hortonworks.com> wrote:
>>
>>> In the development branch (called master) we have added several
>>> features to help diagnose LDAP issues. However to take advantage of these
>>> you will need to build Knox from source as these features are not yet
>>> included in an official release.
>>>
>>> From: Aneela Saleem
>>> Reply-To: "user@knox.apache.org"
>>> Date: Monday, July 27, 2015 at 10:26 AM
>>>
>>> To: "user@knox.apache.org"
>>> Subject: Re: Apache Knox Web API
>>>
>>> But i did not get your point
>>>
>>> On Mon, Jul 27, 2015 at 7:22 PM, Kevin Minder <
>>> kevin.minder@hortonworks.com> wrote:
>>>
>>>> I believe the default LDAP port is 389. Is your OpenLDAP server
>>>> listening on 389?
>>>>
>>>> Otherwise would it be possible for you to build and use the master
>>>> branch version of Knox. We have recently added several LDAP diagnostics
>>>> that might help us here.
>>>>
>>>> From: Aneela Saleem
>>>> Reply-To: "user@knox.apache.org"
>>>> Date: Monday, July 27, 2015 at 10:14 AM
>>>> To: "user@knox.apache.org"
>>>> Subject: Re: Apache Knox Web API
>>>>
>>>> Hi Kevin,
>>>>
>>>> I'm using OpenLDAP
>>>>
>>>> On Mon, Jul 27, 2015 at 6:59 PM, Kevin Minder <
>>>> kevin.minder@hortonworks.com> wrote:
>>>>
>>>>> I’m suspecting this
>>>>> <param>
>>>>> <name>main.ldapRealm.contextFactory.url</name>
>>>>> <value>ldap://localhost</value>
>>>>> </param>
>>>>> What LDAP server are you using?
>>>>>
>>>>> From: Aneela Saleem
>>>>> Reply-To: "user@knox.apache.org"
>>>>> Date: Sunday, July 26, 2015 at 2:53 PM
>>>>> To: "user@knox.apache.org"
>>>>> Subject: Re: Apache Knox Web API
>>>>>
>>>>> <param>
>>>>> <name>main.ldapRealm.contextFactory.url</name>
>>>>> <value>ldap://localhost</value>
>>>>> </param>
>>>>>
>>>>
>>>>
>>>
>>
>
Re: Apache Knox Web API
Posted by Kevin Minder <ke...@hortonworks.com>.
Ok did a bit more digging and it looks like the Shiro LDAP Realm we are using does not implement StartTLS. It seems as though other Shiro Realm implementations do as evidence here
http://jmchung.github.io/blog/2014/10/03/integrating-shiro-with-cas-authentication-via-ldap/
But I see no evidence that the JndiLdapRealm upon which the KnoxLdapRealm is based has the code described here for StartTLS support.
https://docs.oracle.com/javase/jndi/tutorial/ldap/ext/starttls.html
This would be a valuable are for you to contribute to either Knox or Shiro if this capability is important for your use case.
Also note that LDAPS should provide equivalent security.
From: Aneela Saleem
Reply-To: "user@knox.apache.org<ma...@knox.apache.org>"
Date: Monday, July 27, 2015 at 11:07 AM
To: "user@knox.apache.org<ma...@knox.apache.org>"
Subject: Re: Apache Knox Web API
I just tried to enable startTLS for LDAP. I just followed this link:
https://www.digitalocean.com/community/tutorials/how-to-encrypt-openldap-connections-using-starttls
On Mon, Jul 27, 2015 at 8:02 PM, Kevin Minder <ke...@hortonworks.com>> wrote:
Well what have you changed since it last worked?
From: Aneela Saleem
Reply-To: "user@knox.apache.org<ma...@knox.apache.org>"
Date: Monday, July 27, 2015 at 11:01 AM
To: "user@knox.apache.org<ma...@knox.apache.org>"
Subject: Re: Apache Knox Web API
But what could be the issue as it was working fine before
On Mon, Jul 27, 2015 at 7:35 PM, Kevin Minder <ke...@hortonworks.com>> wrote:
In the development branch (called master) we have added several features to help diagnose LDAP issues. However to take advantage of these you will need to build Knox from source as these features are not yet included in an official release.
From: Aneela Saleem
Reply-To: "user@knox.apache.org<ma...@knox.apache.org>"
Date: Monday, July 27, 2015 at 10:26 AM
To: "user@knox.apache.org<ma...@knox.apache.org>"
Subject: Re: Apache Knox Web API
But i did not get your point
On Mon, Jul 27, 2015 at 7:22 PM, Kevin Minder <ke...@hortonworks.com>> wrote:
I believe the default LDAP port is 389. Is your OpenLDAP server listening on 389?
Otherwise would it be possible for you to build and use the master branch version of Knox. We have recently added several LDAP diagnostics that might help us here.
From: Aneela Saleem
Reply-To: "user@knox.apache.org<ma...@knox.apache.org>"
Date: Monday, July 27, 2015 at 10:14 AM
To: "user@knox.apache.org<ma...@knox.apache.org>"
Subject: Re: Apache Knox Web API
Hi Kevin,
I'm using OpenLDAP
On Mon, Jul 27, 2015 at 6:59 PM, Kevin Minder <ke...@hortonworks.com>> wrote:
I’m suspecting this
<param>
<name>main.ldapRealm.contextFactory.url</name>
<value>ldap://localhost</value>
</param>
What LDAP server are you using?
From: Aneela Saleem
Reply-To: "user@knox.apache.org<ma...@knox.apache.org>"
Date: Sunday, July 26, 2015 at 2:53 PM
To: "user@knox.apache.org<ma...@knox.apache.org>"
Subject: Re: Apache Knox Web API
<param>
<name>main.ldapRealm.contextFactory.url</name>
<value>ldap://localhost</value>
</param>
Re: Apache Knox Web API
Posted by Aneela Saleem <an...@platalytics.com>.
I just tried to enable startTLS for LDAP. I just followed this link:
https://www.digitalocean.com/community/tutorials/how-to-encrypt-openldap-connections-using-starttls
On Mon, Jul 27, 2015 at 8:02 PM, Kevin Minder <ke...@hortonworks.com>
wrote:
> Well what have you changed since it last worked?
>
> From: Aneela Saleem
> Reply-To: "user@knox.apache.org"
> Date: Monday, July 27, 2015 at 11:01 AM
>
> To: "user@knox.apache.org"
> Subject: Re: Apache Knox Web API
>
> But what could be the issue as it was working fine before
>
> On Mon, Jul 27, 2015 at 7:35 PM, Kevin Minder <
> kevin.minder@hortonworks.com> wrote:
>
>> In the development branch (called master) we have added several
>> features to help diagnose LDAP issues. However to take advantage of these
>> you will need to build Knox from source as these features are not yet
>> included in an official release.
>>
>> From: Aneela Saleem
>> Reply-To: "user@knox.apache.org"
>> Date: Monday, July 27, 2015 at 10:26 AM
>>
>> To: "user@knox.apache.org"
>> Subject: Re: Apache Knox Web API
>>
>> But i did not get your point
>>
>> On Mon, Jul 27, 2015 at 7:22 PM, Kevin Minder <
>> kevin.minder@hortonworks.com> wrote:
>>
>>> I believe the default LDAP port is 389. Is your OpenLDAP server
>>> listening on 389?
>>>
>>> Otherwise would it be possible for you to build and use the master
>>> branch version of Knox. We have recently added several LDAP diagnostics
>>> that might help us here.
>>>
>>> From: Aneela Saleem
>>> Reply-To: "user@knox.apache.org"
>>> Date: Monday, July 27, 2015 at 10:14 AM
>>> To: "user@knox.apache.org"
>>> Subject: Re: Apache Knox Web API
>>>
>>> Hi Kevin,
>>>
>>> I'm using OpenLDAP
>>>
>>> On Mon, Jul 27, 2015 at 6:59 PM, Kevin Minder <
>>> kevin.minder@hortonworks.com> wrote:
>>>
>>>> I’m suspecting this
>>>> <param>
>>>> <name>main.ldapRealm.contextFactory.url</name>
>>>> <value>ldap://localhost</value>
>>>> </param>
>>>> What LDAP server are you using?
>>>>
>>>> From: Aneela Saleem
>>>> Reply-To: "user@knox.apache.org"
>>>> Date: Sunday, July 26, 2015 at 2:53 PM
>>>> To: "user@knox.apache.org"
>>>> Subject: Re: Apache Knox Web API
>>>>
>>>> <param>
>>>> <name>main.ldapRealm.contextFactory.url</name>
>>>> <value>ldap://localhost</value>
>>>> </param>
>>>>
>>>
>>>
>>
>
Re: Apache Knox Web API
Posted by Kevin Minder <ke...@hortonworks.com>.
Well what have you changed since it last worked?
From: Aneela Saleem
Reply-To: "user@knox.apache.org<ma...@knox.apache.org>"
Date: Monday, July 27, 2015 at 11:01 AM
To: "user@knox.apache.org<ma...@knox.apache.org>"
Subject: Re: Apache Knox Web API
But what could be the issue as it was working fine before
On Mon, Jul 27, 2015 at 7:35 PM, Kevin Minder <ke...@hortonworks.com>> wrote:
In the development branch (called master) we have added several features to help diagnose LDAP issues. However to take advantage of these you will need to build Knox from source as these features are not yet included in an official release.
From: Aneela Saleem
Reply-To: "user@knox.apache.org<ma...@knox.apache.org>"
Date: Monday, July 27, 2015 at 10:26 AM
To: "user@knox.apache.org<ma...@knox.apache.org>"
Subject: Re: Apache Knox Web API
But i did not get your point
On Mon, Jul 27, 2015 at 7:22 PM, Kevin Minder <ke...@hortonworks.com>> wrote:
I believe the default LDAP port is 389. Is your OpenLDAP server listening on 389?
Otherwise would it be possible for you to build and use the master branch version of Knox. We have recently added several LDAP diagnostics that might help us here.
From: Aneela Saleem
Reply-To: "user@knox.apache.org<ma...@knox.apache.org>"
Date: Monday, July 27, 2015 at 10:14 AM
To: "user@knox.apache.org<ma...@knox.apache.org>"
Subject: Re: Apache Knox Web API
Hi Kevin,
I'm using OpenLDAP
On Mon, Jul 27, 2015 at 6:59 PM, Kevin Minder <ke...@hortonworks.com>> wrote:
I’m suspecting this
<param>
<name>main.ldapRealm.contextFactory.url</name>
<value>ldap://localhost</value>
</param>
What LDAP server are you using?
From: Aneela Saleem
Reply-To: "user@knox.apache.org<ma...@knox.apache.org>"
Date: Sunday, July 26, 2015 at 2:53 PM
To: "user@knox.apache.org<ma...@knox.apache.org>"
Subject: Re: Apache Knox Web API
<param>
<name>main.ldapRealm.contextFactory.url</name>
<value>ldap://localhost</value>
</param>
Re: Apache Knox Web API
Posted by Aneela Saleem <an...@platalytics.com>.
But what could be the issue as it was working fine before
On Mon, Jul 27, 2015 at 7:35 PM, Kevin Minder <ke...@hortonworks.com>
wrote:
> In the development branch (called master) we have added several features
> to help diagnose LDAP issues. However to take advantage of these you will
> need to build Knox from source as these features are not yet included in an
> official release.
>
> From: Aneela Saleem
> Reply-To: "user@knox.apache.org"
> Date: Monday, July 27, 2015 at 10:26 AM
>
> To: "user@knox.apache.org"
> Subject: Re: Apache Knox Web API
>
> But i did not get your point
>
> On Mon, Jul 27, 2015 at 7:22 PM, Kevin Minder <
> kevin.minder@hortonworks.com> wrote:
>
>> I believe the default LDAP port is 389. Is your OpenLDAP server
>> listening on 389?
>>
>> Otherwise would it be possible for you to build and use the master
>> branch version of Knox. We have recently added several LDAP diagnostics
>> that might help us here.
>>
>> From: Aneela Saleem
>> Reply-To: "user@knox.apache.org"
>> Date: Monday, July 27, 2015 at 10:14 AM
>> To: "user@knox.apache.org"
>> Subject: Re: Apache Knox Web API
>>
>> Hi Kevin,
>>
>> I'm using OpenLDAP
>>
>> On Mon, Jul 27, 2015 at 6:59 PM, Kevin Minder <
>> kevin.minder@hortonworks.com> wrote:
>>
>>> I’m suspecting this
>>> <param>
>>> <name>main.ldapRealm.contextFactory.url</name>
>>> <value>ldap://localhost</value>
>>> </param>
>>> What LDAP server are you using?
>>>
>>> From: Aneela Saleem
>>> Reply-To: "user@knox.apache.org"
>>> Date: Sunday, July 26, 2015 at 2:53 PM
>>> To: "user@knox.apache.org"
>>> Subject: Re: Apache Knox Web API
>>>
>>> <param>
>>> <name>main.ldapRealm.contextFactory.url</name>
>>> <value>ldap://localhost</value>
>>> </param>
>>>
>>
>>
>
Re: Apache Knox Web API
Posted by Kevin Minder <ke...@hortonworks.com>.
In the development branch (called master) we have added several features to help diagnose LDAP issues. However to take advantage of these you will need to build Knox from source as these features are not yet included in an official release.
From: Aneela Saleem
Reply-To: "user@knox.apache.org<ma...@knox.apache.org>"
Date: Monday, July 27, 2015 at 10:26 AM
To: "user@knox.apache.org<ma...@knox.apache.org>"
Subject: Re: Apache Knox Web API
But i did not get your point
On Mon, Jul 27, 2015 at 7:22 PM, Kevin Minder <ke...@hortonworks.com>> wrote:
I believe the default LDAP port is 389. Is your OpenLDAP server listening on 389?
Otherwise would it be possible for you to build and use the master branch version of Knox. We have recently added several LDAP diagnostics that might help us here.
From: Aneela Saleem
Reply-To: "user@knox.apache.org<ma...@knox.apache.org>"
Date: Monday, July 27, 2015 at 10:14 AM
To: "user@knox.apache.org<ma...@knox.apache.org>"
Subject: Re: Apache Knox Web API
Hi Kevin,
I'm using OpenLDAP
On Mon, Jul 27, 2015 at 6:59 PM, Kevin Minder <ke...@hortonworks.com>> wrote:
I’m suspecting this
<param>
<name>main.ldapRealm.contextFactory.url</name>
<value>ldap://localhost</value>
</param>
What LDAP server are you using?
From: Aneela Saleem
Reply-To: "user@knox.apache.org<ma...@knox.apache.org>"
Date: Sunday, July 26, 2015 at 2:53 PM
To: "user@knox.apache.org<ma...@knox.apache.org>"
Subject: Re: Apache Knox Web API
<param>
<name>main.ldapRealm.contextFactory.url</name>
<value>ldap://localhost</value>
</param>
Re: Apache Knox Web API
Posted by Aneela Saleem <an...@platalytics.com>.
But i did not get your point
On Mon, Jul 27, 2015 at 7:22 PM, Kevin Minder <ke...@hortonworks.com>
wrote:
> I believe the default LDAP port is 389. Is your OpenLDAP server
> listening on 389?
>
> Otherwise would it be possible for you to build and use the master
> branch version of Knox. We have recently added several LDAP diagnostics
> that might help us here.
>
> From: Aneela Saleem
> Reply-To: "user@knox.apache.org"
> Date: Monday, July 27, 2015 at 10:14 AM
> To: "user@knox.apache.org"
> Subject: Re: Apache Knox Web API
>
> Hi Kevin,
>
> I'm using OpenLDAP
>
> On Mon, Jul 27, 2015 at 6:59 PM, Kevin Minder <
> kevin.minder@hortonworks.com> wrote:
>
>> I’m suspecting this
>> <param>
>> <name>main.ldapRealm.contextFactory.url</name>
>> <value>ldap://localhost</value>
>> </param>
>> What LDAP server are you using?
>>
>> From: Aneela Saleem
>> Reply-To: "user@knox.apache.org"
>> Date: Sunday, July 26, 2015 at 2:53 PM
>> To: "user@knox.apache.org"
>> Subject: Re: Apache Knox Web API
>>
>> <param>
>> <name>main.ldapRealm.contextFactory.url</name>
>> <value>ldap://localhost</value>
>> </param>
>>
>
>
Re: Apache Knox Web API
Posted by Kevin Minder <ke...@hortonworks.com>.
I believe the default LDAP port is 389. Is your OpenLDAP server listening on 389?
Otherwise would it be possible for you to build and use the master branch version of Knox. We have recently added several LDAP diagnostics that might help us here.
From: Aneela Saleem
Reply-To: "user@knox.apache.org<ma...@knox.apache.org>"
Date: Monday, July 27, 2015 at 10:14 AM
To: "user@knox.apache.org<ma...@knox.apache.org>"
Subject: Re: Apache Knox Web API
Hi Kevin,
I'm using OpenLDAP
On Mon, Jul 27, 2015 at 6:59 PM, Kevin Minder <ke...@hortonworks.com>> wrote:
I’m suspecting this
<param>
<name>main.ldapRealm.contextFactory.url</name>
<value>ldap://localhost</value>
</param>
What LDAP server are you using?
From: Aneela Saleem
Reply-To: "user@knox.apache.org<ma...@knox.apache.org>"
Date: Sunday, July 26, 2015 at 2:53 PM
To: "user@knox.apache.org<ma...@knox.apache.org>"
Subject: Re: Apache Knox Web API
<param>
<name>main.ldapRealm.contextFactory.url</name>
<value>ldap://localhost</value>
</param>
Re: Apache Knox Web API
Posted by Aneela Saleem <an...@platalytics.com>.
Hi Kevin,
I'm using OpenLDAP
On Mon, Jul 27, 2015 at 6:59 PM, Kevin Minder <ke...@hortonworks.com>
wrote:
> I’m suspecting this
> <param>
> <name>main.ldapRealm.contextFactory.url</name>
> <value>ldap://localhost</value>
> </param>
> What LDAP server are you using?
>
> From: Aneela Saleem
> Reply-To: "user@knox.apache.org"
> Date: Sunday, July 26, 2015 at 2:53 PM
> To: "user@knox.apache.org"
> Subject: Re: Apache Knox Web API
>
> <param>
> <name>main.ldapRealm.contextFactory.url</name>
> <value>ldap://localhost</value>
> </param>
>
Re: Apache Knox Web API
Posted by Kevin Minder <ke...@hortonworks.com>.
I’m suspecting this
<param>
<name>main.ldapRealm.contextFactory.url</name>
<value>ldap://localhost</value>
</param>
What LDAP server are you using?
From: Aneela Saleem
Reply-To: "user@knox.apache.org<ma...@knox.apache.org>"
Date: Sunday, July 26, 2015 at 2:53 PM
To: "user@knox.apache.org<ma...@knox.apache.org>"
Subject: Re: Apache Knox Web API
<param>
<name>main.ldapRealm.contextFactory.url</name>
<value>ldap://localhost</value>
</param>
Re: Apache Knox Web API
Posted by Aneela Saleem <an...@platalytics.com>.
Hi Kevin,
I tried your Latter Code and continuously getting "HTTP/1.1 401
Unauthorized"
Following is my Topology file:
<topology>
<gateway>
<provider>
<role>authentication</role>
<name>ShiroProvider</name>
<enabled>true</enabled>
<param>
<name>sessionTimeout</name>
<value>30</value>
</param>
<param>
<name>main.ldapRealm</name>
<value>org.apache.hadoop.gateway.shirorealm.KnoxLdapRealm</value>
</param>
<param>
<name>main.ldapContextFactory</name>
<value>org.apache.hadoop.gateway.shirorealm.KnoxLdapContextFactory</value>
</param>
<param>
<name>main.ldapRealm.contextFactory</name>
<value>$ldapContextFactory</value>
</param>
<param>
<name>main.ldapRealm.userDnTemplate</name>
<value>cn={0},dc=platalytics,dc=com</value>
</param>
<param>
<name>main.ldapRealm.contextFactory.url</name>
<value>ldap://localhost</value>
</param>
<param>
<name>main.ldapRealm.contextFactory.authenticationMechanism</name>
<value>simple</value>
</param>
<param>
<name>urls./**</name>
<value>authcBasic</value>
</param>
</provider>
<provider>
<role>authorization</role>
<name>AclsAuthz</name>
<enabled>true</enabled>
<param>
<name>knox.acl</name>
<value>admin;*;*</value>
</param>
</provider>
<provider>
<role>identity-assertion</role>
<name>Default</name>
<enabled>true</enabled>
</provider>
<provider>
<role>hostmap</role>
<name>static</name>
<enabled>true</enabled>
<param><name>localhost</name><value>127.0.0.1</value></param>
</provider>
</gateway>
<service>
<role>NAMENODE</role>
<url>hdfs://localhost:8020</url>
</service>
<service>
<role>JOBTRACKER</role>
<url>rpc://localhost:8050</url>
</service>
<service>
<role>RESOURCEMANAGER</role>
<url>http://red3:8088/ws</url>
</service>
<service>
<role>WEBHDFS</role>
<url>http://localhost:50070/webhdfs</url>
</service>
<service>
<role>WEBHCAT</role>
<url>http://webcat-host:50111/templeton</url>
</service>
<service>
<role>WEBHBASE</role>
<url>http://webhbase-host :60080</url>
</service>
<service>
<role>HIVE</role>
<url>http://hive-host :10001/cliservice</url>
</service>
</topology>
Can you please see what i'm missing?
On Wed, Jul 8, 2015 at 6:52 PM, Kevin Minder <ke...@hortonworks.com>
wrote:
> Ok, if you want the simplest thing that will work try the code below.
> This time I’ve tested it. However, keep in mind this code removes much of
> the benefit of SSL between the client and Knox due to the use
> of TrustSelfSignedStrategy and NoopHostnameVerifier. If you are using this
> in production there are a few different routes to go. For example if you
> are using CA signed certs much of the SSL setup code below isn’t even
> required. Can you provide more context about what you are actually trying
> to accomplish?
>
>
> import org.apache.http.HttpEntity;
> import org.apache.http.auth.AuthScope;
> import org.apache.http.auth.UsernamePasswordCredentials;
> import org.apache.http.client.CredentialsProvider;
> import org.apache.http.client.methods.CloseableHttpResponse;
> import org.apache.http.client.methods.HttpGet;
> import org.apache.http.client.protocol.HttpClientContext;
> import org.apache.http.conn.ssl.NoopHostnameVerifier;
> import org.apache.http.conn.ssl.SSLConnectionSocketFactory;
> import org.apache.http.conn.ssl.TrustSelfSignedStrategy;
> import org.apache.http.impl.client.BasicCredentialsProvider;
> import org.apache.http.impl.client.CloseableHttpClient;
> import org.apache.http.impl.client.HttpClients;
> import org.apache.http.ssl.SSLContextBuilder;
> import org.apache.http.util.EntityUtils;
>
> import javax.net.ssl.SSLContext;
>
> public class HttpClientSslNoVerifySslSample {
>
> public static void main( String[] args ) throws Exception {
>
> SSLContext sslContext = SSLContextBuilder.create()
> .loadTrustMaterial( new TrustSelfSignedStrategy() ) // *** Trust
> self signed certs. ***
> .build();
> SSLConnectionSocketFactory sslFactory = new
> SSLConnectionSocketFactory( sslContext );
>
> CloseableHttpClient client = HttpClients.custom()
> .setSSLSocketFactory( sslFactory )
> .setSSLHostnameVerifier( new NoopHostnameVerifier() ) // *** Allow
> all host names. ***
> .build();
>
> HttpClientContext cliContext = HttpClientContext.create();
> CredentialsProvider credentialsProvider = new
> BasicCredentialsProvider();
> credentialsProvider.setCredentials(
> new AuthScope( AuthScope.ANY_HOST, AuthScope.ANY_PORT ),
> new UsernamePasswordCredentials( "guest", "guest-password" ) );
> cliContext.setCredentialsProvider( credentialsProvider );
>
> HttpGet method = new HttpGet( "
> https://localhost:8443/gateway/sandbox/webhdfs/v1?op=GETHOMEDIRECTORY" );
> CloseableHttpResponse response = client.execute( method, cliContext );
> HttpEntity entity = response.getEntity();
> System.out.println( EntityUtils.toString( entity ) );
>
> response.close();
> client.close();
> }
>
> }
>
>
> From: Hafiz Mujadid <ha...@gmail.com>
> Reply-To: "user@knox.apache.org" <us...@knox.apache.org>
> Date: Wednesday, July 8, 2015 at 4:53 AM
>
> To: "user@knox.apache.org" <us...@knox.apache.org>
> Subject: Re: Apache Knox Web API
>
> Hi Kevin!
>
> I tried this code and got following exception
>
> Error: keytool error: java.io.IOException: Keystore was tampered with, or
> password was incorrect
> java.io.IOException: Keystore was tampered with, or password was incorrect
> at sun.security.provider.JavaKeyStore.engineLoad(JavaKeyStore.java:772)
> at sun.security.provider.JavaKeyStore$JKS.engineLoad(JavaKeyStore.java:55)
>
>
> I regenerated password for key store and replaced
> *trustStore.load(stream, "wrong".toCharArray())*
> to
> *trustStore.load(stream, "changeit".toCharArray())*
>
> but still it's not working.
>
> On Wed, Jul 8, 2015 at 1:46 AM, Kevin Minder <kevin.minder@hortonworks.com
> > wrote:
>
>> Take a look at this below. This is a bit of a mod of an existing
>> sample I had laying around so don’t take it as tested.
>>
>> import org.apache.http.HttpEntity;
>> import org.apache.http.auth.AuthScope;
>> import org.apache.http.auth.UsernamePasswordCredentials;
>> import org.apache.http.client.CredentialsProvider;
>> import org.apache.http.client.methods.CloseableHttpResponse;
>> import org.apache.http.client.methods.HttpGet;
>> import org.apache.http.client.protocol.HttpClientContext;
>> import org.apache.http.conn.ssl.AllowAllHostnameVerifier;
>> import org.apache.http.conn.ssl.SSLConnectionSocketFactory;
>> import org.apache.http.conn.ssl.SSLContexts;
>> import org.apache.http.conn.ssl.TrustSelfSignedStrategy;
>> import org.apache.http.impl.client.BasicCredentialsProvider;
>> import org.apache.http.impl.client.CloseableHttpClient;
>> import org.apache.http.impl.client.HttpClients;
>> import org.apache.http.util.EntityUtils;
>>
>> import javax.net.ssl.SSLContext;
>> import java.io.File;
>> import java.io.FileInputStream;
>> import java.security.KeyStore;
>>
>> public class HttpClientSslTest {
>>
>> public static void main( String[] args ) throws Exception {
>>
>> KeyStore trustStore = KeyStore.getInstance( KeyStore.getDefaultType() );
>> FileInputStream stream = new FileInputStream( new File( "gateway.jks" ) );
>> trustStore.load( stream, "wrong".toCharArray() );
>> stream.close();
>>
>> SSLContext sslContext = SSLContexts.custom()
>> .loadTrustMaterial( trustStore, *new TrustSelfSignedStrategy()* ) // *** Trust self signed certs. ***
>> .build();
>> SSLConnectionSocketFactory sslFactory = new SSLConnectionSocketFactory( sslContext );
>>
>> CloseableHttpClient client = HttpClients.custom()
>> .setSSLSocketFactory( sslFactory )
>> .setHostnameVerifier( *new AllowAllHostnameVerifier()* ) // *** Trust all host names. ***
>> .build();
>>
>> HttpClientContext cliContext = HttpClientContext.create();
>> CredentialsProvider credentialsProvider = new BasicCredentialsProvider();
>> credentialsProvider.setCredentials(
>> new AuthScope( AuthScope.ANY_HOST, AuthScope.ANY_PORT ),
>> new UsernamePasswordCredentials( "guest", "guest-password" ) );
>> cliContext.setCredentialsProvider( credentialsProvider );
>>
>> HttpGet method = new HttpGet( "https://localhost:8443/gateway/sandbox/webhdfs/v1?op=GETHOMEDIRECTORY" );
>> CloseableHttpResponse response = client.execute( method, cliContext );
>> HttpEntity entity = response.getEntity();
>> System.out.println( EntityUtils.toString( entity ) );
>>
>> response.close();
>> client.close();
>> }
>>
>> }
>>
>>
>> From: Hafiz Mujadid <ha...@gmail.com>
>> Reply-To: "user@knox.apache.org" <us...@knox.apache.org>
>> Date: Tuesday, July 7, 2015 at 4:05 PM
>> To: "user@knox.apache.org" <us...@knox.apache.org>
>> Subject: Re: Apache Knox Web API
>>
>> Hi larry!
>>
>> As suggested by you, I tried to use knox rest api using Apache
>> HttpClient
>>
>> here is my code
>>
>> val provider = new BasicCredentialsProvider()
>> val credentials = new UsernamePasswordCredentials("admin", "12345")
>> provider.setCredentials(AuthScope.ANY, credentials)
>> val client =
>> HttpClientBuilder.create().setDefaultCredentialsProvider(provider) .build()
>> val response = client.execute(new HttpGet("
>> https://localhost:8443/gateway/sample/webhdfs/v1?op=LISTSTATUS"))
>> val statusCode = response.getStatusLine.getStatusCode
>> val input = response.getEntity().getContent()
>> if (statusCode == HttpStatus.SC_OK)
>> println("ok")
>>
>>
>> but I am getting following SSL related exception.
>>
>>
>> Exception in thread "main" javax.net.ssl.SSLHandshakeException:
>> sun.security.validator.ValidatorException: PKIX path building failed:
>> sun.security.provider.certpath.SunCertPathBuilderException: unable to find
>> valid certification path to requested target
>> at sun.security.ssl.Alerts.getSSLException(Alerts.java:192)
>> at sun.security.ssl.SSLSocketImpl.fatal(SSLSocketImpl.java:1904)
>>
>> Any suggestion?
>>
>>
>> On Mon, Jul 6, 2015 at 10:34 PM, Hafiz Mujadid <ha...@gmail.com>
>> wrote:
>>
>>> thanks for your help .:)
>>>
>>> On Mon, Jul 6, 2015 at 10:05 PM, larry mccay <la...@gmail.com>
>>> wrote:
>>>
>>>> As I mentioned, you can dig into the source of the gateway-shell
>>>> classes - which are used when scripting with groovy.
>>>> Here is a link to an hdfs Get request:
>>>> https://github.com/apache/knox/blob/master/gateway-shell/src/main/java/org/apache/hadoop/gateway/shell/hdfs/Get.java#L32
>>>>
>>>> Going to the HttpClient level is like going to bare metal - it
>>>> provides you greatest level of control but you will need to build
>>>> abstractions around its use in order to avoid lots of redundant boilerplate
>>>> code. Which is why we have provided such classes for the scripting.
>>>>
>>>> You can also look at the DefaultDispatch code as an example - it is a
>>>> bit more complicated since it covers more general usecases but you may
>>>> glean some insights from it.
>>>>
>>>> Otherwise, google for examples of "Apache HttpClient REST basic
>>>> authentication" and see what you find.
>>>>
>>>> Hope this is useful for you!
>>>>
>>>>
>>>> On Sun, Jul 5, 2015 at 11:40 AM, Hafiz Mujadid <
>>>> hafizmujadid00@gmail.com> wrote:
>>>>
>>>>> Hi Larry!
>>>>>
>>>>> Can you provide the link to samples using httclient on github etc.?
>>>>>
>>>>> Thanks
>>>>>
>>>>> On Sat, Jul 4, 2015 at 9:40 PM, larry mccay <la...@gmail.com>
>>>>> wrote:
>>>>>
>>>>>> Then you will want to consider the Client library from the first
>>>>>> link.
>>>>>> You can look in the {GATEWAY_HOME}/samples directory for examples of
>>>>>> it's use.
>>>>>> The groovy scripts are a great way to do it or you can use the
>>>>>> underlying java classes that groovy uses.
>>>>>> The latter will require you to dig into the source a bit more to see
>>>>>> how to use them.
>>>>>>
>>>>>> You can also use Apache HttpClient and there are samples of that as
>>>>>> well.
>>>>>>
>>>>>> On Sat, Jul 4, 2015 at 12:04 PM, Aneela Saleem <
>>>>>> aneela@platalytics.com> wrote:
>>>>>>
>>>>>>> Thanks Larry.
>>>>>>>
>>>>>>> Actually I need some client API like java so that I authenticate
>>>>>>> / authorize my users programmatically through Knox.
>>>>>>>
>>>>>>> On Sat, Jul 4, 2015 at 8:50 PM, larry mccay <la...@gmail.com>
>>>>>>> wrote:
>>>>>>>
>>>>>>>> Hi Aneela -
>>>>>>>>
>>>>>>>> I assume that you mean that you would like to add support for a
>>>>>>>> Hadoop API that Knox currently lacks.
>>>>>>>> My suggestion is that you find one that your organization or your
>>>>>>>> personal interests require.
>>>>>>>>
>>>>>>>> There are lots of Jira's filed for bug fixes and other
>>>>>>>> features/enhancements as well.
>>>>>>>>
>>>>>>>> Feel free to start a discussion regarding any contribution that
>>>>>>>> you would like to make.
>>>>>>>>
>>>>>>>> As far as the links that you referenced:
>>>>>>>>
>>>>>>>> 1. The first is a client library for scripting interactions with
>>>>>>>> Hadoop services through Knox - there are some really interesting and
>>>>>>>> powerful capabilities there.
>>>>>>>> 2. The second is actually pointing to a section the dev guide that
>>>>>>>> needs to be completed. We have what we call Gateway Services in the kernel
>>>>>>>> of the Knox server that provide implementations for core server interfaces
>>>>>>>> - crypto, SSL, credential aliasing, etc. I don't think that you want to
>>>>>>>> work in that space. If you want to work on adding new API support for
>>>>>>>> services then you should refer to the Services section -
>>>>>>>> https://knox.apache.org/books/knox-0-6-0/dev-guide.html#Services.
>>>>>>>>
>>>>>>>> Note that the link that I provided above is for the 0.6.0 dev
>>>>>>>> guide. There is a new configuration driven way to add API support to Knox
>>>>>>>> that was added in the 0.6.0 release.
>>>>>>>>
>>>>>>>> Thanks for your interest in contributing to Apache Knox!
>>>>>>>>
>>>>>>>> --larry
>>>>>>>>
>>>>>>>>
>>>>>>>> On Sat, Jul 4, 2015 at 10:56 AM, Aneela Saleem <
>>>>>>>> aneela@platalytics.com> wrote:
>>>>>>>>
>>>>>>>>> Hi Everyone,
>>>>>>>>>
>>>>>>>>> I'm going to start development for Hadoop security through
>>>>>>>>> Apache Knox. Can anyone please suggest me some good API for Knox.
>>>>>>>>>
>>>>>>>>> So far i have found following:
>>>>>>>>>
>>>>>>>>> https://cwiki.apache.org/confluence/display/KNOX/Client+Usage
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> https://knox.apache.org/books/knox-0-5-0/dev-guide.html#Gateway+Services
>>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>>
>>>>>
>>>>>
>>>>> --
>>>>> Regards: HAFIZ MUJADID
>>>>>
>>>>
>>>>
>>>
>>>
>>> --
>>> Regards: HAFIZ MUJADID
>>>
>>
>>
>>
>> --
>> Regards: HAFIZ MUJADID
>>
>
>
>
> --
> Regards: HAFIZ MUJADID
>
Re: Apache Knox Web API
Posted by Hafiz Mujadid <ha...@gmail.com>.
Hi Kevin!
Fist of all thanks for your response :)
I am not using in production yet but planning to use it. For the time being
I am just trying to authenticate ldap users with apache knox. I don't know
whether I am at right direction or not. But at this time I am using http
client to authenticate users with in different groups.
On Wed, Jul 8, 2015 at 6:52 PM, Kevin Minder <ke...@hortonworks.com>
wrote:
> Ok, if you want the simplest thing that will work try the code below.
> This time I’ve tested it. However, keep in mind this code removes much of
> the benefit of SSL between the client and Knox due to the use
> of TrustSelfSignedStrategy and NoopHostnameVerifier. If you are using this
> in production there are a few different routes to go. For example if you
> are using CA signed certs much of the SSL setup code below isn’t even
> required. Can you provide more context about what you are actually trying
> to accomplish?
>
>
> import org.apache.http.HttpEntity;
> import org.apache.http.auth.AuthScope;
> import org.apache.http.auth.UsernamePasswordCredentials;
> import org.apache.http.client.CredentialsProvider;
> import org.apache.http.client.methods.CloseableHttpResponse;
> import org.apache.http.client.methods.HttpGet;
> import org.apache.http.client.protocol.HttpClientContext;
> import org.apache.http.conn.ssl.NoopHostnameVerifier;
> import org.apache.http.conn.ssl.SSLConnectionSocketFactory;
> import org.apache.http.conn.ssl.TrustSelfSignedStrategy;
> import org.apache.http.impl.client.BasicCredentialsProvider;
> import org.apache.http.impl.client.CloseableHttpClient;
> import org.apache.http.impl.client.HttpClients;
> import org.apache.http.ssl.SSLContextBuilder;
> import org.apache.http.util.EntityUtils;
>
> import javax.net.ssl.SSLContext;
>
> public class HttpClientSslNoVerifySslSample {
>
> public static void main( String[] args ) throws Exception {
>
> SSLContext sslContext = SSLContextBuilder.create()
> .loadTrustMaterial( new TrustSelfSignedStrategy() ) // *** Trust
> self signed certs. ***
> .build();
> SSLConnectionSocketFactory sslFactory = new
> SSLConnectionSocketFactory( sslContext );
>
> CloseableHttpClient client = HttpClients.custom()
> .setSSLSocketFactory( sslFactory )
> .setSSLHostnameVerifier( new NoopHostnameVerifier() ) // *** Allow
> all host names. ***
> .build();
>
> HttpClientContext cliContext = HttpClientContext.create();
> CredentialsProvider credentialsProvider = new
> BasicCredentialsProvider();
> credentialsProvider.setCredentials(
> new AuthScope( AuthScope.ANY_HOST, AuthScope.ANY_PORT ),
> new UsernamePasswordCredentials( "guest", "guest-password" ) );
> cliContext.setCredentialsProvider( credentialsProvider );
>
> HttpGet method = new HttpGet( "
> https://localhost:8443/gateway/sandbox/webhdfs/v1?op=GETHOMEDIRECTORY" );
> CloseableHttpResponse response = client.execute( method, cliContext );
> HttpEntity entity = response.getEntity();
> System.out.println( EntityUtils.toString( entity ) );
>
> response.close();
> client.close();
> }
>
> }
>
>
> From: Hafiz Mujadid <ha...@gmail.com>
> Reply-To: "user@knox.apache.org" <us...@knox.apache.org>
> Date: Wednesday, July 8, 2015 at 4:53 AM
>
> To: "user@knox.apache.org" <us...@knox.apache.org>
> Subject: Re: Apache Knox Web API
>
> Hi Kevin!
>
> I tried this code and got following exception
>
> Error: keytool error: java.io.IOException: Keystore was tampered with, or
> password was incorrect
> java.io.IOException: Keystore was tampered with, or password was incorrect
> at sun.security.provider.JavaKeyStore.engineLoad(JavaKeyStore.java:772)
> at sun.security.provider.JavaKeyStore$JKS.engineLoad(JavaKeyStore.java:55)
>
>
> I regenerated password for key store and replaced
> *trustStore.load(stream, "wrong".toCharArray())*
> to
> *trustStore.load(stream, "changeit".toCharArray())*
>
> but still it's not working.
>
> On Wed, Jul 8, 2015 at 1:46 AM, Kevin Minder <kevin.minder@hortonworks.com
> > wrote:
>
>> Take a look at this below. This is a bit of a mod of an existing
>> sample I had laying around so don’t take it as tested.
>>
>> import org.apache.http.HttpEntity;
>> import org.apache.http.auth.AuthScope;
>> import org.apache.http.auth.UsernamePasswordCredentials;
>> import org.apache.http.client.CredentialsProvider;
>> import org.apache.http.client.methods.CloseableHttpResponse;
>> import org.apache.http.client.methods.HttpGet;
>> import org.apache.http.client.protocol.HttpClientContext;
>> import org.apache.http.conn.ssl.AllowAllHostnameVerifier;
>> import org.apache.http.conn.ssl.SSLConnectionSocketFactory;
>> import org.apache.http.conn.ssl.SSLContexts;
>> import org.apache.http.conn.ssl.TrustSelfSignedStrategy;
>> import org.apache.http.impl.client.BasicCredentialsProvider;
>> import org.apache.http.impl.client.CloseableHttpClient;
>> import org.apache.http.impl.client.HttpClients;
>> import org.apache.http.util.EntityUtils;
>>
>> import javax.net.ssl.SSLContext;
>> import java.io.File;
>> import java.io.FileInputStream;
>> import java.security.KeyStore;
>>
>> public class HttpClientSslTest {
>>
>> public static void main( String[] args ) throws Exception {
>>
>> KeyStore trustStore = KeyStore.getInstance( KeyStore.getDefaultType() );
>> FileInputStream stream = new FileInputStream( new File( "gateway.jks" ) );
>> trustStore.load( stream, "wrong".toCharArray() );
>> stream.close();
>>
>> SSLContext sslContext = SSLContexts.custom()
>> .loadTrustMaterial( trustStore, *new TrustSelfSignedStrategy()* ) // *** Trust self signed certs. ***
>> .build();
>> SSLConnectionSocketFactory sslFactory = new SSLConnectionSocketFactory( sslContext );
>>
>> CloseableHttpClient client = HttpClients.custom()
>> .setSSLSocketFactory( sslFactory )
>> .setHostnameVerifier( *new AllowAllHostnameVerifier()* ) // *** Trust all host names. ***
>> .build();
>>
>> HttpClientContext cliContext = HttpClientContext.create();
>> CredentialsProvider credentialsProvider = new BasicCredentialsProvider();
>> credentialsProvider.setCredentials(
>> new AuthScope( AuthScope.ANY_HOST, AuthScope.ANY_PORT ),
>> new UsernamePasswordCredentials( "guest", "guest-password" ) );
>> cliContext.setCredentialsProvider( credentialsProvider );
>>
>> HttpGet method = new HttpGet( "https://localhost:8443/gateway/sandbox/webhdfs/v1?op=GETHOMEDIRECTORY" );
>> CloseableHttpResponse response = client.execute( method, cliContext );
>> HttpEntity entity = response.getEntity();
>> System.out.println( EntityUtils.toString( entity ) );
>>
>> response.close();
>> client.close();
>> }
>>
>> }
>>
>>
>> From: Hafiz Mujadid <ha...@gmail.com>
>> Reply-To: "user@knox.apache.org" <us...@knox.apache.org>
>> Date: Tuesday, July 7, 2015 at 4:05 PM
>> To: "user@knox.apache.org" <us...@knox.apache.org>
>> Subject: Re: Apache Knox Web API
>>
>> Hi larry!
>>
>> As suggested by you, I tried to use knox rest api using Apache
>> HttpClient
>>
>> here is my code
>>
>> val provider = new BasicCredentialsProvider()
>> val credentials = new UsernamePasswordCredentials("admin", "12345")
>> provider.setCredentials(AuthScope.ANY, credentials)
>> val client =
>> HttpClientBuilder.create().setDefaultCredentialsProvider(provider) .build()
>> val response = client.execute(new HttpGet("
>> https://localhost:8443/gateway/sample/webhdfs/v1?op=LISTSTATUS"))
>> val statusCode = response.getStatusLine.getStatusCode
>> val input = response.getEntity().getContent()
>> if (statusCode == HttpStatus.SC_OK)
>> println("ok")
>>
>>
>> but I am getting following SSL related exception.
>>
>>
>> Exception in thread "main" javax.net.ssl.SSLHandshakeException:
>> sun.security.validator.ValidatorException: PKIX path building failed:
>> sun.security.provider.certpath.SunCertPathBuilderException: unable to find
>> valid certification path to requested target
>> at sun.security.ssl.Alerts.getSSLException(Alerts.java:192)
>> at sun.security.ssl.SSLSocketImpl.fatal(SSLSocketImpl.java:1904)
>>
>> Any suggestion?
>>
>>
>> On Mon, Jul 6, 2015 at 10:34 PM, Hafiz Mujadid <ha...@gmail.com>
>> wrote:
>>
>>> thanks for your help .:)
>>>
>>> On Mon, Jul 6, 2015 at 10:05 PM, larry mccay <la...@gmail.com>
>>> wrote:
>>>
>>>> As I mentioned, you can dig into the source of the gateway-shell
>>>> classes - which are used when scripting with groovy.
>>>> Here is a link to an hdfs Get request:
>>>> https://github.com/apache/knox/blob/master/gateway-shell/src/main/java/org/apache/hadoop/gateway/shell/hdfs/Get.java#L32
>>>>
>>>> Going to the HttpClient level is like going to bare metal - it
>>>> provides you greatest level of control but you will need to build
>>>> abstractions around its use in order to avoid lots of redundant boilerplate
>>>> code. Which is why we have provided such classes for the scripting.
>>>>
>>>> You can also look at the DefaultDispatch code as an example - it is a
>>>> bit more complicated since it covers more general usecases but you may
>>>> glean some insights from it.
>>>>
>>>> Otherwise, google for examples of "Apache HttpClient REST basic
>>>> authentication" and see what you find.
>>>>
>>>> Hope this is useful for you!
>>>>
>>>>
>>>> On Sun, Jul 5, 2015 at 11:40 AM, Hafiz Mujadid <
>>>> hafizmujadid00@gmail.com> wrote:
>>>>
>>>>> Hi Larry!
>>>>>
>>>>> Can you provide the link to samples using httclient on github etc.?
>>>>>
>>>>> Thanks
>>>>>
>>>>> On Sat, Jul 4, 2015 at 9:40 PM, larry mccay <la...@gmail.com>
>>>>> wrote:
>>>>>
>>>>>> Then you will want to consider the Client library from the first
>>>>>> link.
>>>>>> You can look in the {GATEWAY_HOME}/samples directory for examples of
>>>>>> it's use.
>>>>>> The groovy scripts are a great way to do it or you can use the
>>>>>> underlying java classes that groovy uses.
>>>>>> The latter will require you to dig into the source a bit more to see
>>>>>> how to use them.
>>>>>>
>>>>>> You can also use Apache HttpClient and there are samples of that as
>>>>>> well.
>>>>>>
>>>>>> On Sat, Jul 4, 2015 at 12:04 PM, Aneela Saleem <
>>>>>> aneela@platalytics.com> wrote:
>>>>>>
>>>>>>> Thanks Larry.
>>>>>>>
>>>>>>> Actually I need some client API like java so that I authenticate
>>>>>>> / authorize my users programmatically through Knox.
>>>>>>>
>>>>>>> On Sat, Jul 4, 2015 at 8:50 PM, larry mccay <la...@gmail.com>
>>>>>>> wrote:
>>>>>>>
>>>>>>>> Hi Aneela -
>>>>>>>>
>>>>>>>> I assume that you mean that you would like to add support for a
>>>>>>>> Hadoop API that Knox currently lacks.
>>>>>>>> My suggestion is that you find one that your organization or your
>>>>>>>> personal interests require.
>>>>>>>>
>>>>>>>> There are lots of Jira's filed for bug fixes and other
>>>>>>>> features/enhancements as well.
>>>>>>>>
>>>>>>>> Feel free to start a discussion regarding any contribution that
>>>>>>>> you would like to make.
>>>>>>>>
>>>>>>>> As far as the links that you referenced:
>>>>>>>>
>>>>>>>> 1. The first is a client library for scripting interactions with
>>>>>>>> Hadoop services through Knox - there are some really interesting and
>>>>>>>> powerful capabilities there.
>>>>>>>> 2. The second is actually pointing to a section the dev guide that
>>>>>>>> needs to be completed. We have what we call Gateway Services in the kernel
>>>>>>>> of the Knox server that provide implementations for core server interfaces
>>>>>>>> - crypto, SSL, credential aliasing, etc. I don't think that you want to
>>>>>>>> work in that space. If you want to work on adding new API support for
>>>>>>>> services then you should refer to the Services section -
>>>>>>>> https://knox.apache.org/books/knox-0-6-0/dev-guide.html#Services.
>>>>>>>>
>>>>>>>> Note that the link that I provided above is for the 0.6.0 dev
>>>>>>>> guide. There is a new configuration driven way to add API support to Knox
>>>>>>>> that was added in the 0.6.0 release.
>>>>>>>>
>>>>>>>> Thanks for your interest in contributing to Apache Knox!
>>>>>>>>
>>>>>>>> --larry
>>>>>>>>
>>>>>>>>
>>>>>>>> On Sat, Jul 4, 2015 at 10:56 AM, Aneela Saleem <
>>>>>>>> aneela@platalytics.com> wrote:
>>>>>>>>
>>>>>>>>> Hi Everyone,
>>>>>>>>>
>>>>>>>>> I'm going to start development for Hadoop security through
>>>>>>>>> Apache Knox. Can anyone please suggest me some good API for Knox.
>>>>>>>>>
>>>>>>>>> So far i have found following:
>>>>>>>>>
>>>>>>>>> https://cwiki.apache.org/confluence/display/KNOX/Client+Usage
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> https://knox.apache.org/books/knox-0-5-0/dev-guide.html#Gateway+Services
>>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>>
>>>>>
>>>>>
>>>>> --
>>>>> Regards: HAFIZ MUJADID
>>>>>
>>>>
>>>>
>>>
>>>
>>> --
>>> Regards: HAFIZ MUJADID
>>>
>>
>>
>>
>> --
>> Regards: HAFIZ MUJADID
>>
>
>
>
> --
> Regards: HAFIZ MUJADID
>
--
Regards: HAFIZ MUJADID
Re: Apache Knox Web API
Posted by Kevin Minder <ke...@hortonworks.com>.
Ok, if you want the simplest thing that will work try the code below. This time I’ve tested it. However, keep in mind this code removes much of the benefit of SSL between the client and Knox due to the use of TrustSelfSignedStrategy and NoopHostnameVerifier. If you are using this in production there are a few different routes to go. For example if you are using CA signed certs much of the SSL setup code below isn’t even required. Can you provide more context about what you are actually trying to accomplish?
import org.apache.http.HttpEntity;
import org.apache.http.auth.AuthScope;
import org.apache.http.auth.UsernamePasswordCredentials;
import org.apache.http.client.CredentialsProvider;
import org.apache.http.client.methods.CloseableHttpResponse;
import org.apache.http.client.methods.HttpGet;
import org.apache.http.client.protocol.HttpClientContext;
import org.apache.http.conn.ssl.NoopHostnameVerifier;
import org.apache.http.conn.ssl.SSLConnectionSocketFactory;
import org.apache.http.conn.ssl.TrustSelfSignedStrategy;
import org.apache.http.impl.client.BasicCredentialsProvider;
import org.apache.http.impl.client.CloseableHttpClient;
import org.apache.http.impl.client.HttpClients;
import org.apache.http.ssl.SSLContextBuilder;
import org.apache.http.util.EntityUtils;
import javax.net.ssl.SSLContext;
public class HttpClientSslNoVerifySslSample {
public static void main( String[] args ) throws Exception {
SSLContext sslContext = SSLContextBuilder.create()
.loadTrustMaterial( new TrustSelfSignedStrategy() ) // *** Trust self signed certs. ***
.build();
SSLConnectionSocketFactory sslFactory = new SSLConnectionSocketFactory( sslContext );
CloseableHttpClient client = HttpClients.custom()
.setSSLSocketFactory( sslFactory )
.setSSLHostnameVerifier( new NoopHostnameVerifier() ) // *** Allow all host names. ***
.build();
HttpClientContext cliContext = HttpClientContext.create();
CredentialsProvider credentialsProvider = new BasicCredentialsProvider();
credentialsProvider.setCredentials(
new AuthScope( AuthScope.ANY_HOST, AuthScope.ANY_PORT ),
new UsernamePasswordCredentials( "guest", "guest-password" ) );
cliContext.setCredentialsProvider( credentialsProvider );
HttpGet method = new HttpGet( "https://localhost:8443/gateway/sandbox/webhdfs/v1?op=GETHOMEDIRECTORY" );
CloseableHttpResponse response = client.execute( method, cliContext );
HttpEntity entity = response.getEntity();
System.out.println( EntityUtils.toString( entity ) );
response.close();
client.close();
}
}
From: Hafiz Mujadid <ha...@gmail.com>>
Reply-To: "user@knox.apache.org<ma...@knox.apache.org>" <us...@knox.apache.org>>
Date: Wednesday, July 8, 2015 at 4:53 AM
To: "user@knox.apache.org<ma...@knox.apache.org>" <us...@knox.apache.org>>
Subject: Re: Apache Knox Web API
Hi Kevin!
I tried this code and got following exception
Error: keytool error: java.io.IOException: Keystore was tampered with, or password was incorrect
java.io.IOException: Keystore was tampered with, or password was incorrect
at sun.security.provider.JavaKeyStore.engineLoad(JavaKeyStore.java:772)
at sun.security.provider.JavaKeyStore$JKS.engineLoad(JavaKeyStore.java:55)
I regenerated password for key store and replaced
trustStore.load(stream, "wrong".toCharArray())
to
trustStore.load(stream, "changeit".toCharArray())
but still it's not working.
On Wed, Jul 8, 2015 at 1:46 AM, Kevin Minder <ke...@hortonworks.com>> wrote:
Take a look at this below. This is a bit of a mod of an existing sample I had laying around so don’t take it as tested.
import org.apache.http.HttpEntity;
import org.apache.http.auth.AuthScope;
import org.apache.http.auth.UsernamePasswordCredentials;
import org.apache.http.client.CredentialsProvider;
import org.apache.http.client.methods.CloseableHttpResponse;
import org.apache.http.client.methods.HttpGet;
import org.apache.http.client.protocol.HttpClientContext;
import org.apache.http.conn.ssl.AllowAllHostnameVerifier;
import org.apache.http.conn.ssl.SSLConnectionSocketFactory;
import org.apache.http.conn.ssl.SSLContexts;
import org.apache.http.conn.ssl.TrustSelfSignedStrategy;
import org.apache.http.impl.client.BasicCredentialsProvider;
import org.apache.http.impl.client.CloseableHttpClient;
import org.apache.http.impl.client.HttpClients;
import org.apache.http.util.EntityUtils;
import javax.net.ssl.SSLContext;
import java.io.File;
import java.io.FileInputStream;
import java.security.KeyStore;
public class HttpClientSslTest {
public static void main( String[] args ) throws Exception {
KeyStore trustStore = KeyStore.getInstance( KeyStore.getDefaultType() );
FileInputStream stream = new FileInputStream( new File( "gateway.jks" ) );
trustStore.load( stream, "wrong".toCharArray() );
stream.close();
SSLContext sslContext = SSLContexts.custom()
.loadTrustMaterial( trustStore, new TrustSelfSignedStrategy() ) // *** Trust self signed certs. ***
.build();
SSLConnectionSocketFactory sslFactory = new SSLConnectionSocketFactory( sslContext );
CloseableHttpClient client = HttpClients.custom()
.setSSLSocketFactory( sslFactory )
.setHostnameVerifier( new AllowAllHostnameVerifier() ) // *** Trust all host names. ***
.build();
HttpClientContext cliContext = HttpClientContext.create();
CredentialsProvider credentialsProvider = new BasicCredentialsProvider();
credentialsProvider.setCredentials(
new AuthScope( AuthScope.ANY_HOST, AuthScope.ANY_PORT ),
new UsernamePasswordCredentials( "guest", "guest-password" ) );
cliContext.setCredentialsProvider( credentialsProvider );
HttpGet method = new HttpGet( "https://localhost:8443/gateway/sandbox/webhdfs/v1?op=GETHOMEDIRECTORY" );
CloseableHttpResponse response = client.execute( method, cliContext );
HttpEntity entity = response.getEntity();
System.out.println( EntityUtils.toString( entity ) );
response.close();
client.close();
}
}
From: Hafiz Mujadid <ha...@gmail.com>>
Reply-To: "user@knox.apache.org<ma...@knox.apache.org>" <us...@knox.apache.org>>
Date: Tuesday, July 7, 2015 at 4:05 PM
To: "user@knox.apache.org<ma...@knox.apache.org>" <us...@knox.apache.org>>
Subject: Re: Apache Knox Web API
Hi larry!
As suggested by you, I tried to use knox rest api using Apache HttpClient
here is my code
val provider = new BasicCredentialsProvider()
val credentials = new UsernamePasswordCredentials("admin", "12345")
provider.setCredentials(AuthScope.ANY, credentials)
val client = HttpClientBuilder.create().setDefaultCredentialsProvider(provider) .build()
val response = client.execute(new HttpGet("https://localhost:8443/gateway/sample/webhdfs/v1?op=LISTSTATUS"))
val statusCode = response.getStatusLine.getStatusCode
val input = response.getEntity().getContent()
if (statusCode == HttpStatus.SC_OK)
println("ok")
but I am getting following SSL related exception.
Exception in thread "main" javax.net.ssl.SSLHandshakeException: sun.security.validator.ValidatorException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target
at sun.security.ssl.Alerts.getSSLException(Alerts.java:192)
at sun.security.ssl.SSLSocketImpl.fatal(SSLSocketImpl.java:1904)
Any suggestion?
On Mon, Jul 6, 2015 at 10:34 PM, Hafiz Mujadid <ha...@gmail.com>> wrote:
thanks for your help .:)
On Mon, Jul 6, 2015 at 10:05 PM, larry mccay <la...@gmail.com>> wrote:
As I mentioned, you can dig into the source of the gateway-shell classes - which are used when scripting with groovy.
Here is a link to an hdfs Get request: https://github.com/apache/knox/blob/master/gateway-shell/src/main/java/org/apache/hadoop/gateway/shell/hdfs/Get.java#L32
Going to the HttpClient level is like going to bare metal - it provides you greatest level of control but you will need to build abstractions around its use in order to avoid lots of redundant boilerplate code. Which is why we have provided such classes for the scripting.
You can also look at the DefaultDispatch code as an example - it is a bit more complicated since it covers more general usecases but you may glean some insights from it.
Otherwise, google for examples of "Apache HttpClient REST basic authentication" and see what you find.
Hope this is useful for you!
On Sun, Jul 5, 2015 at 11:40 AM, Hafiz Mujadid <ha...@gmail.com>> wrote:
Hi Larry!
Can you provide the link to samples using httclient on github etc.?
Thanks
On Sat, Jul 4, 2015 at 9:40 PM, larry mccay <la...@gmail.com>> wrote:
Then you will want to consider the Client library from the first link.
You can look in the {GATEWAY_HOME}/samples directory for examples of it's use.
The groovy scripts are a great way to do it or you can use the underlying java classes that groovy uses.
The latter will require you to dig into the source a bit more to see how to use them.
You can also use Apache HttpClient and there are samples of that as well.
On Sat, Jul 4, 2015 at 12:04 PM, Aneela Saleem <an...@platalytics.com>> wrote:
Thanks Larry.
Actually I need some client API like java so that I authenticate / authorize my users programmatically through Knox.
On Sat, Jul 4, 2015 at 8:50 PM, larry mccay <la...@gmail.com>> wrote:
Hi Aneela -
I assume that you mean that you would like to add support for a Hadoop API that Knox currently lacks.
My suggestion is that you find one that your organization or your personal interests require.
There are lots of Jira's filed for bug fixes and other features/enhancements as well.
Feel free to start a discussion regarding any contribution that you would like to make.
As far as the links that you referenced:
1. The first is a client library for scripting interactions with Hadoop services through Knox - there are some really interesting and powerful capabilities there.
2. The second is actually pointing to a section the dev guide that needs to be completed. We have what we call Gateway Services in the kernel of the Knox server that provide implementations for core server interfaces - crypto, SSL, credential aliasing, etc. I don't think that you want to work in that space. If you want to work on adding new API support for services then you should refer to the Services section - https://knox.apache.org/books/knox-0-6-0/dev-guide.html#Services.
Note that the link that I provided above is for the 0.6.0 dev guide. There is a new configuration driven way to add API support to Knox that was added in the 0.6.0 release.
Thanks for your interest in contributing to Apache Knox!
--larry
On Sat, Jul 4, 2015 at 10:56 AM, Aneela Saleem <an...@platalytics.com>> wrote:
Hi Everyone,
I'm going to start development for Hadoop security through Apache Knox. Can anyone please suggest me some good API for Knox.
So far i have found following:
https://cwiki.apache.org/confluence/display/KNOX/Client+Usage
https://knox.apache.org/books/knox-0-5-0/dev-guide.html#Gateway+Services
--
Regards: HAFIZ MUJADID
--
Regards: HAFIZ MUJADID
--
Regards: HAFIZ MUJADID
--
Regards: HAFIZ MUJADID
Re: Apache Knox Web API
Posted by Hafiz Mujadid <ha...@gmail.com>.
Hi Kevin!
I tried this code and got following exception
Error: keytool error: java.io.IOException: Keystore was tampered with, or
password was incorrect
java.io.IOException: Keystore was tampered with, or password was incorrect
at sun.security.provider.JavaKeyStore.engineLoad(JavaKeyStore.java:772)
at sun.security.provider.JavaKeyStore$JKS.engineLoad(JavaKeyStore.java:55)
I regenerated password for key store and replaced
*trustStore.load(stream, "wrong".toCharArray())*
to
*trustStore.load(stream, "changeit".toCharArray())*
but still it's not working.
On Wed, Jul 8, 2015 at 1:46 AM, Kevin Minder <ke...@hortonworks.com>
wrote:
> Take a look at this below. This is a bit of a mod of an existing sample
> I had laying around so don’t take it as tested.
>
> import org.apache.http.HttpEntity;
> import org.apache.http.auth.AuthScope;
> import org.apache.http.auth.UsernamePasswordCredentials;
> import org.apache.http.client.CredentialsProvider;
> import org.apache.http.client.methods.CloseableHttpResponse;
> import org.apache.http.client.methods.HttpGet;
> import org.apache.http.client.protocol.HttpClientContext;
> import org.apache.http.conn.ssl.AllowAllHostnameVerifier;
> import org.apache.http.conn.ssl.SSLConnectionSocketFactory;
> import org.apache.http.conn.ssl.SSLContexts;
> import org.apache.http.conn.ssl.TrustSelfSignedStrategy;
> import org.apache.http.impl.client.BasicCredentialsProvider;
> import org.apache.http.impl.client.CloseableHttpClient;
> import org.apache.http.impl.client.HttpClients;
> import org.apache.http.util.EntityUtils;
>
> import javax.net.ssl.SSLContext;
> import java.io.File;
> import java.io.FileInputStream;
> import java.security.KeyStore;
>
> public class HttpClientSslTest {
>
> public static void main( String[] args ) throws Exception {
>
> KeyStore trustStore = KeyStore.getInstance( KeyStore.getDefaultType() );
> FileInputStream stream = new FileInputStream( new File( "gateway.jks" ) );
> trustStore.load( stream, "wrong".toCharArray() );
> stream.close();
>
> SSLContext sslContext = SSLContexts.custom()
> .loadTrustMaterial( trustStore, *new TrustSelfSignedStrategy()* ) // *** Trust self signed certs. ***
> .build();
> SSLConnectionSocketFactory sslFactory = new SSLConnectionSocketFactory( sslContext );
>
> CloseableHttpClient client = HttpClients.custom()
> .setSSLSocketFactory( sslFactory )
> .setHostnameVerifier( *new AllowAllHostnameVerifier()* ) // *** Trust all host names. ***
> .build();
>
> HttpClientContext cliContext = HttpClientContext.create();
> CredentialsProvider credentialsProvider = new BasicCredentialsProvider();
> credentialsProvider.setCredentials(
> new AuthScope( AuthScope.ANY_HOST, AuthScope.ANY_PORT ),
> new UsernamePasswordCredentials( "guest", "guest-password" ) );
> cliContext.setCredentialsProvider( credentialsProvider );
>
> HttpGet method = new HttpGet( "https://localhost:8443/gateway/sandbox/webhdfs/v1?op=GETHOMEDIRECTORY" );
> CloseableHttpResponse response = client.execute( method, cliContext );
> HttpEntity entity = response.getEntity();
> System.out.println( EntityUtils.toString( entity ) );
>
> response.close();
> client.close();
> }
>
> }
>
>
> From: Hafiz Mujadid <ha...@gmail.com>
> Reply-To: "user@knox.apache.org" <us...@knox.apache.org>
> Date: Tuesday, July 7, 2015 at 4:05 PM
> To: "user@knox.apache.org" <us...@knox.apache.org>
> Subject: Re: Apache Knox Web API
>
> Hi larry!
>
> As suggested by you, I tried to use knox rest api using Apache HttpClient
>
> here is my code
>
> val provider = new BasicCredentialsProvider()
> val credentials = new UsernamePasswordCredentials("admin", "12345")
> provider.setCredentials(AuthScope.ANY, credentials)
> val client =
> HttpClientBuilder.create().setDefaultCredentialsProvider(provider) .build()
> val response = client.execute(new HttpGet("
> https://localhost:8443/gateway/sample/webhdfs/v1?op=LISTSTATUS"))
> val statusCode = response.getStatusLine.getStatusCode
> val input = response.getEntity().getContent()
> if (statusCode == HttpStatus.SC_OK)
> println("ok")
>
>
> but I am getting following SSL related exception.
>
>
> Exception in thread "main" javax.net.ssl.SSLHandshakeException:
> sun.security.validator.ValidatorException: PKIX path building failed:
> sun.security.provider.certpath.SunCertPathBuilderException: unable to find
> valid certification path to requested target
> at sun.security.ssl.Alerts.getSSLException(Alerts.java:192)
> at sun.security.ssl.SSLSocketImpl.fatal(SSLSocketImpl.java:1904)
>
> Any suggestion?
>
>
> On Mon, Jul 6, 2015 at 10:34 PM, Hafiz Mujadid <ha...@gmail.com>
> wrote:
>
>> thanks for your help .:)
>>
>> On Mon, Jul 6, 2015 at 10:05 PM, larry mccay <la...@gmail.com>
>> wrote:
>>
>>> As I mentioned, you can dig into the source of the gateway-shell classes
>>> - which are used when scripting with groovy.
>>> Here is a link to an hdfs Get request:
>>> https://github.com/apache/knox/blob/master/gateway-shell/src/main/java/org/apache/hadoop/gateway/shell/hdfs/Get.java#L32
>>>
>>> Going to the HttpClient level is like going to bare metal - it
>>> provides you greatest level of control but you will need to build
>>> abstractions around its use in order to avoid lots of redundant boilerplate
>>> code. Which is why we have provided such classes for the scripting.
>>>
>>> You can also look at the DefaultDispatch code as an example - it is a
>>> bit more complicated since it covers more general usecases but you may
>>> glean some insights from it.
>>>
>>> Otherwise, google for examples of "Apache HttpClient REST basic
>>> authentication" and see what you find.
>>>
>>> Hope this is useful for you!
>>>
>>>
>>> On Sun, Jul 5, 2015 at 11:40 AM, Hafiz Mujadid <hafizmujadid00@gmail.com
>>> > wrote:
>>>
>>>> Hi Larry!
>>>>
>>>> Can you provide the link to samples using httclient on github etc.?
>>>>
>>>> Thanks
>>>>
>>>> On Sat, Jul 4, 2015 at 9:40 PM, larry mccay <la...@gmail.com>
>>>> wrote:
>>>>
>>>>> Then you will want to consider the Client library from the first link.
>>>>> You can look in the {GATEWAY_HOME}/samples directory for examples of
>>>>> it's use.
>>>>> The groovy scripts are a great way to do it or you can use the
>>>>> underlying java classes that groovy uses.
>>>>> The latter will require you to dig into the source a bit more to see
>>>>> how to use them.
>>>>>
>>>>> You can also use Apache HttpClient and there are samples of that as
>>>>> well.
>>>>>
>>>>> On Sat, Jul 4, 2015 at 12:04 PM, Aneela Saleem <aneela@platalytics.com
>>>>> > wrote:
>>>>>
>>>>>> Thanks Larry.
>>>>>>
>>>>>> Actually I need some client API like java so that I authenticate /
>>>>>> authorize my users programmatically through Knox.
>>>>>>
>>>>>> On Sat, Jul 4, 2015 at 8:50 PM, larry mccay <la...@gmail.com>
>>>>>> wrote:
>>>>>>
>>>>>>> Hi Aneela -
>>>>>>>
>>>>>>> I assume that you mean that you would like to add support for a
>>>>>>> Hadoop API that Knox currently lacks.
>>>>>>> My suggestion is that you find one that your organization or your
>>>>>>> personal interests require.
>>>>>>>
>>>>>>> There are lots of Jira's filed for bug fixes and other
>>>>>>> features/enhancements as well.
>>>>>>>
>>>>>>> Feel free to start a discussion regarding any contribution that
>>>>>>> you would like to make.
>>>>>>>
>>>>>>> As far as the links that you referenced:
>>>>>>>
>>>>>>> 1. The first is a client library for scripting interactions with
>>>>>>> Hadoop services through Knox - there are some really interesting and
>>>>>>> powerful capabilities there.
>>>>>>> 2. The second is actually pointing to a section the dev guide that
>>>>>>> needs to be completed. We have what we call Gateway Services in the kernel
>>>>>>> of the Knox server that provide implementations for core server interfaces
>>>>>>> - crypto, SSL, credential aliasing, etc. I don't think that you want to
>>>>>>> work in that space. If you want to work on adding new API support for
>>>>>>> services then you should refer to the Services section -
>>>>>>> https://knox.apache.org/books/knox-0-6-0/dev-guide.html#Services.
>>>>>>>
>>>>>>> Note that the link that I provided above is for the 0.6.0 dev
>>>>>>> guide. There is a new configuration driven way to add API support to Knox
>>>>>>> that was added in the 0.6.0 release.
>>>>>>>
>>>>>>> Thanks for your interest in contributing to Apache Knox!
>>>>>>>
>>>>>>> --larry
>>>>>>>
>>>>>>>
>>>>>>> On Sat, Jul 4, 2015 at 10:56 AM, Aneela Saleem <
>>>>>>> aneela@platalytics.com> wrote:
>>>>>>>
>>>>>>>> Hi Everyone,
>>>>>>>>
>>>>>>>> I'm going to start development for Hadoop security through Apache
>>>>>>>> Knox. Can anyone please suggest me some good API for Knox.
>>>>>>>>
>>>>>>>> So far i have found following:
>>>>>>>>
>>>>>>>> https://cwiki.apache.org/confluence/display/KNOX/Client+Usage
>>>>>>>>
>>>>>>>>
>>>>>>>> https://knox.apache.org/books/knox-0-5-0/dev-guide.html#Gateway+Services
>>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>
>>>>>
>>>>
>>>>
>>>> --
>>>> Regards: HAFIZ MUJADID
>>>>
>>>
>>>
>>
>>
>> --
>> Regards: HAFIZ MUJADID
>>
>
>
>
> --
> Regards: HAFIZ MUJADID
>
--
Regards: HAFIZ MUJADID
Re: Apache Knox Web API
Posted by Kevin Minder <ke...@hortonworks.com>.
Take a look at this below. This is a bit of a mod of an existing sample I had laying around so don’t take it as tested.
import org.apache.http.HttpEntity;
import org.apache.http.auth.AuthScope;
import org.apache.http.auth.UsernamePasswordCredentials;
import org.apache.http.client.CredentialsProvider;
import org.apache.http.client.methods.CloseableHttpResponse;
import org.apache.http.client.methods.HttpGet;
import org.apache.http.client.protocol.HttpClientContext;
import org.apache.http.conn.ssl.AllowAllHostnameVerifier;
import org.apache.http.conn.ssl.SSLConnectionSocketFactory;
import org.apache.http.conn.ssl.SSLContexts;
import org.apache.http.conn.ssl.TrustSelfSignedStrategy;
import org.apache.http.impl.client.BasicCredentialsProvider;
import org.apache.http.impl.client.CloseableHttpClient;
import org.apache.http.impl.client.HttpClients;
import org.apache.http.util.EntityUtils;
import javax.net.ssl.SSLContext;
import java.io.File;
import java.io.FileInputStream;
import java.security.KeyStore;
public class HttpClientSslTest {
public static void main( String[] args ) throws Exception {
KeyStore trustStore = KeyStore.getInstance( KeyStore.getDefaultType() );
FileInputStream stream = new FileInputStream( new File( "gateway.jks" ) );
trustStore.load( stream, "wrong".toCharArray() );
stream.close();
SSLContext sslContext = SSLContexts.custom()
.loadTrustMaterial( trustStore, new TrustSelfSignedStrategy() ) // *** Trust self signed certs. ***
.build();
SSLConnectionSocketFactory sslFactory = new SSLConnectionSocketFactory( sslContext );
CloseableHttpClient client = HttpClients.custom()
.setSSLSocketFactory( sslFactory )
.setHostnameVerifier( new AllowAllHostnameVerifier() ) // *** Trust all host names. ***
.build();
HttpClientContext cliContext = HttpClientContext.create();
CredentialsProvider credentialsProvider = new BasicCredentialsProvider();
credentialsProvider.setCredentials(
new AuthScope( AuthScope.ANY_HOST, AuthScope.ANY_PORT ),
new UsernamePasswordCredentials( "guest", "guest-password" ) );
cliContext.setCredentialsProvider( credentialsProvider );
HttpGet method = new HttpGet( "https://localhost:8443/gateway/sandbox/webhdfs/v1?op=GETHOMEDIRECTORY" );
CloseableHttpResponse response = client.execute( method, cliContext );
HttpEntity entity = response.getEntity();
System.out.println( EntityUtils.toString( entity ) );
response.close();
client.close();
}
}
From: Hafiz Mujadid <ha...@gmail.com>>
Reply-To: "user@knox.apache.org<ma...@knox.apache.org>" <us...@knox.apache.org>>
Date: Tuesday, July 7, 2015 at 4:05 PM
To: "user@knox.apache.org<ma...@knox.apache.org>" <us...@knox.apache.org>>
Subject: Re: Apache Knox Web API
Hi larry!
As suggested by you, I tried to use knox rest api using Apache HttpClient
here is my code
val provider = new BasicCredentialsProvider()
val credentials = new UsernamePasswordCredentials("admin", "12345")
provider.setCredentials(AuthScope.ANY, credentials)
val client = HttpClientBuilder.create().setDefaultCredentialsProvider(provider) .build()
val response = client.execute(new HttpGet("https://localhost:8443/gateway/sample/webhdfs/v1?op=LISTSTATUS"))
val statusCode = response.getStatusLine.getStatusCode
val input = response.getEntity().getContent()
if (statusCode == HttpStatus.SC_OK)
println("ok")
but I am getting following SSL related exception.
Exception in thread "main" javax.net.ssl.SSLHandshakeException: sun.security.validator.ValidatorException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target
at sun.security.ssl.Alerts.getSSLException(Alerts.java:192)
at sun.security.ssl.SSLSocketImpl.fatal(SSLSocketImpl.java:1904)
Any suggestion?
On Mon, Jul 6, 2015 at 10:34 PM, Hafiz Mujadid <ha...@gmail.com>> wrote:
thanks for your help .:)
On Mon, Jul 6, 2015 at 10:05 PM, larry mccay <la...@gmail.com>> wrote:
As I mentioned, you can dig into the source of the gateway-shell classes - which are used when scripting with groovy.
Here is a link to an hdfs Get request: https://github.com/apache/knox/blob/master/gateway-shell/src/main/java/org/apache/hadoop/gateway/shell/hdfs/Get.java#L32
Going to the HttpClient level is like going to bare metal - it provides you greatest level of control but you will need to build abstractions around its use in order to avoid lots of redundant boilerplate code. Which is why we have provided such classes for the scripting.
You can also look at the DefaultDispatch code as an example - it is a bit more complicated since it covers more general usecases but you may glean some insights from it.
Otherwise, google for examples of "Apache HttpClient REST basic authentication" and see what you find.
Hope this is useful for you!
On Sun, Jul 5, 2015 at 11:40 AM, Hafiz Mujadid <ha...@gmail.com>> wrote:
Hi Larry!
Can you provide the link to samples using httclient on github etc.?
Thanks
On Sat, Jul 4, 2015 at 9:40 PM, larry mccay <la...@gmail.com>> wrote:
Then you will want to consider the Client library from the first link.
You can look in the {GATEWAY_HOME}/samples directory for examples of it's use.
The groovy scripts are a great way to do it or you can use the underlying java classes that groovy uses.
The latter will require you to dig into the source a bit more to see how to use them.
You can also use Apache HttpClient and there are samples of that as well.
On Sat, Jul 4, 2015 at 12:04 PM, Aneela Saleem <an...@platalytics.com>> wrote:
Thanks Larry.
Actually I need some client API like java so that I authenticate / authorize my users programmatically through Knox.
On Sat, Jul 4, 2015 at 8:50 PM, larry mccay <la...@gmail.com>> wrote:
Hi Aneela -
I assume that you mean that you would like to add support for a Hadoop API that Knox currently lacks.
My suggestion is that you find one that your organization or your personal interests require.
There are lots of Jira's filed for bug fixes and other features/enhancements as well.
Feel free to start a discussion regarding any contribution that you would like to make.
As far as the links that you referenced:
1. The first is a client library for scripting interactions with Hadoop services through Knox - there are some really interesting and powerful capabilities there.
2. The second is actually pointing to a section the dev guide that needs to be completed. We have what we call Gateway Services in the kernel of the Knox server that provide implementations for core server interfaces - crypto, SSL, credential aliasing, etc. I don't think that you want to work in that space. If you want to work on adding new API support for services then you should refer to the Services section - https://knox.apache.org/books/knox-0-6-0/dev-guide.html#Services.
Note that the link that I provided above is for the 0.6.0 dev guide. There is a new configuration driven way to add API support to Knox that was added in the 0.6.0 release.
Thanks for your interest in contributing to Apache Knox!
--larry
On Sat, Jul 4, 2015 at 10:56 AM, Aneela Saleem <an...@platalytics.com>> wrote:
Hi Everyone,
I'm going to start development for Hadoop security through Apache Knox. Can anyone please suggest me some good API for Knox.
So far i have found following:
https://cwiki.apache.org/confluence/display/KNOX/Client+Usage
https://knox.apache.org/books/knox-0-5-0/dev-guide.html#Gateway+Services
--
Regards: HAFIZ MUJADID
--
Regards: HAFIZ MUJADID
--
Regards: HAFIZ MUJADID
Re: Apache Knox Web API
Posted by Hafiz Mujadid <ha...@gmail.com>.
Hi larry!
As suggested by you, I tried to use knox rest api using Apache HttpClient
here is my code
val provider = new BasicCredentialsProvider()
val credentials = new UsernamePasswordCredentials("admin", "12345")
provider.setCredentials(AuthScope.ANY, credentials)
val client =
HttpClientBuilder.create().setDefaultCredentialsProvider(provider) .build()
val response = client.execute(new HttpGet("
https://localhost:8443/gateway/sample/webhdfs/v1?op=LISTSTATUS"))
val statusCode = response.getStatusLine.getStatusCode
val input = response.getEntity().getContent()
if (statusCode == HttpStatus.SC_OK)
println("ok")
but I am getting following SSL related exception.
Exception in thread "main" javax.net.ssl.SSLHandshakeException:
sun.security.validator.ValidatorException: PKIX path building failed:
sun.security.provider.certpath.SunCertPathBuilderException: unable to find
valid certification path to requested target
at sun.security.ssl.Alerts.getSSLException(Alerts.java:192)
at sun.security.ssl.SSLSocketImpl.fatal(SSLSocketImpl.java:1904)
Any suggestion?
On Mon, Jul 6, 2015 at 10:34 PM, Hafiz Mujadid <ha...@gmail.com>
wrote:
> thanks for your help .:)
>
> On Mon, Jul 6, 2015 at 10:05 PM, larry mccay <la...@gmail.com>
> wrote:
>
>> As I mentioned, you can dig into the source of the gateway-shell classes
>> - which are used when scripting with groovy.
>> Here is a link to an hdfs Get request:
>> https://github.com/apache/knox/blob/master/gateway-shell/src/main/java/org/apache/hadoop/gateway/shell/hdfs/Get.java#L32
>>
>> Going to the HttpClient level is like going to bare metal - it provides
>> you greatest level of control but you will need to build abstractions
>> around its use in order to avoid lots of redundant boilerplate code. Which
>> is why we have provided such classes for the scripting.
>>
>> You can also look at the DefaultDispatch code as an example - it is a bit
>> more complicated since it covers more general usecases but you may glean
>> some insights from it.
>>
>> Otherwise, google for examples of "Apache HttpClient REST basic
>> authentication" and see what you find.
>>
>> Hope this is useful for you!
>>
>>
>> On Sun, Jul 5, 2015 at 11:40 AM, Hafiz Mujadid <ha...@gmail.com>
>> wrote:
>>
>>> Hi Larry!
>>>
>>> Can you provide the link to samples using httclient on github etc.?
>>>
>>> Thanks
>>>
>>> On Sat, Jul 4, 2015 at 9:40 PM, larry mccay <la...@gmail.com>
>>> wrote:
>>>
>>>> Then you will want to consider the Client library from the first link.
>>>> You can look in the {GATEWAY_HOME}/samples directory for examples of
>>>> it's use.
>>>> The groovy scripts are a great way to do it or you can use the
>>>> underlying java classes that groovy uses.
>>>> The latter will require you to dig into the source a bit more to see
>>>> how to use them.
>>>>
>>>> You can also use Apache HttpClient and there are samples of that as
>>>> well.
>>>>
>>>> On Sat, Jul 4, 2015 at 12:04 PM, Aneela Saleem <an...@platalytics.com>
>>>> wrote:
>>>>
>>>>> Thanks Larry.
>>>>>
>>>>> Actually I need some client API like java so that I authenticate /
>>>>> authorize my users programmatically through Knox.
>>>>>
>>>>> On Sat, Jul 4, 2015 at 8:50 PM, larry mccay <la...@gmail.com>
>>>>> wrote:
>>>>>
>>>>>> Hi Aneela -
>>>>>>
>>>>>> I assume that you mean that you would like to add support for a
>>>>>> Hadoop API that Knox currently lacks.
>>>>>> My suggestion is that you find one that your organization or your
>>>>>> personal interests require.
>>>>>>
>>>>>> There are lots of Jira's filed for bug fixes and other
>>>>>> features/enhancements as well.
>>>>>>
>>>>>> Feel free to start a discussion regarding any contribution that you
>>>>>> would like to make.
>>>>>>
>>>>>> As far as the links that you referenced:
>>>>>>
>>>>>> 1. The first is a client library for scripting interactions with
>>>>>> Hadoop services through Knox - there are some really interesting and
>>>>>> powerful capabilities there.
>>>>>> 2. The second is actually pointing to a section the dev guide that
>>>>>> needs to be completed. We have what we call Gateway Services in the kernel
>>>>>> of the Knox server that provide implementations for core server interfaces
>>>>>> - crypto, SSL, credential aliasing, etc. I don't think that you want to
>>>>>> work in that space. If you want to work on adding new API support for
>>>>>> services then you should refer to the Services section -
>>>>>> https://knox.apache.org/books/knox-0-6-0/dev-guide.html#Services.
>>>>>>
>>>>>> Note that the link that I provided above is for the 0.6.0 dev guide.
>>>>>> There is a new configuration driven way to add API support to Knox that was
>>>>>> added in the 0.6.0 release.
>>>>>>
>>>>>> Thanks for your interest in contributing to Apache Knox!
>>>>>>
>>>>>> --larry
>>>>>>
>>>>>>
>>>>>> On Sat, Jul 4, 2015 at 10:56 AM, Aneela Saleem <
>>>>>> aneela@platalytics.com> wrote:
>>>>>>
>>>>>>> Hi Everyone,
>>>>>>>
>>>>>>> I'm going to start development for Hadoop security through Apache
>>>>>>> Knox. Can anyone please suggest me some good API for Knox.
>>>>>>>
>>>>>>> So far i have found following:
>>>>>>>
>>>>>>> https://cwiki.apache.org/confluence/display/KNOX/Client+Usage
>>>>>>>
>>>>>>>
>>>>>>> https://knox.apache.org/books/knox-0-5-0/dev-guide.html#Gateway+Services
>>>>>>>
>>>>>>
>>>>>>
>>>>>
>>>>
>>>
>>>
>>> --
>>> Regards: HAFIZ MUJADID
>>>
>>
>>
>
>
> --
> Regards: HAFIZ MUJADID
>
--
Regards: HAFIZ MUJADID
Re: Apache Knox Web API
Posted by Hafiz Mujadid <ha...@gmail.com>.
thanks for your help .:)
On Mon, Jul 6, 2015 at 10:05 PM, larry mccay <la...@gmail.com> wrote:
> As I mentioned, you can dig into the source of the gateway-shell classes -
> which are used when scripting with groovy.
> Here is a link to an hdfs Get request:
> https://github.com/apache/knox/blob/master/gateway-shell/src/main/java/org/apache/hadoop/gateway/shell/hdfs/Get.java#L32
>
> Going to the HttpClient level is like going to bare metal - it provides
> you greatest level of control but you will need to build abstractions
> around its use in order to avoid lots of redundant boilerplate code. Which
> is why we have provided such classes for the scripting.
>
> You can also look at the DefaultDispatch code as an example - it is a bit
> more complicated since it covers more general usecases but you may glean
> some insights from it.
>
> Otherwise, google for examples of "Apache HttpClient REST basic
> authentication" and see what you find.
>
> Hope this is useful for you!
>
>
> On Sun, Jul 5, 2015 at 11:40 AM, Hafiz Mujadid <ha...@gmail.com>
> wrote:
>
>> Hi Larry!
>>
>> Can you provide the link to samples using httclient on github etc.?
>>
>> Thanks
>>
>> On Sat, Jul 4, 2015 at 9:40 PM, larry mccay <la...@gmail.com>
>> wrote:
>>
>>> Then you will want to consider the Client library from the first link.
>>> You can look in the {GATEWAY_HOME}/samples directory for examples of
>>> it's use.
>>> The groovy scripts are a great way to do it or you can use the
>>> underlying java classes that groovy uses.
>>> The latter will require you to dig into the source a bit more to see how
>>> to use them.
>>>
>>> You can also use Apache HttpClient and there are samples of that as well.
>>>
>>> On Sat, Jul 4, 2015 at 12:04 PM, Aneela Saleem <an...@platalytics.com>
>>> wrote:
>>>
>>>> Thanks Larry.
>>>>
>>>> Actually I need some client API like java so that I authenticate /
>>>> authorize my users programmatically through Knox.
>>>>
>>>> On Sat, Jul 4, 2015 at 8:50 PM, larry mccay <la...@gmail.com>
>>>> wrote:
>>>>
>>>>> Hi Aneela -
>>>>>
>>>>> I assume that you mean that you would like to add support for a Hadoop
>>>>> API that Knox currently lacks.
>>>>> My suggestion is that you find one that your organization or your
>>>>> personal interests require.
>>>>>
>>>>> There are lots of Jira's filed for bug fixes and other
>>>>> features/enhancements as well.
>>>>>
>>>>> Feel free to start a discussion regarding any contribution that you
>>>>> would like to make.
>>>>>
>>>>> As far as the links that you referenced:
>>>>>
>>>>> 1. The first is a client library for scripting interactions with
>>>>> Hadoop services through Knox - there are some really interesting and
>>>>> powerful capabilities there.
>>>>> 2. The second is actually pointing to a section the dev guide that
>>>>> needs to be completed. We have what we call Gateway Services in the kernel
>>>>> of the Knox server that provide implementations for core server interfaces
>>>>> - crypto, SSL, credential aliasing, etc. I don't think that you want to
>>>>> work in that space. If you want to work on adding new API support for
>>>>> services then you should refer to the Services section -
>>>>> https://knox.apache.org/books/knox-0-6-0/dev-guide.html#Services.
>>>>>
>>>>> Note that the link that I provided above is for the 0.6.0 dev guide.
>>>>> There is a new configuration driven way to add API support to Knox that was
>>>>> added in the 0.6.0 release.
>>>>>
>>>>> Thanks for your interest in contributing to Apache Knox!
>>>>>
>>>>> --larry
>>>>>
>>>>>
>>>>> On Sat, Jul 4, 2015 at 10:56 AM, Aneela Saleem <aneela@platalytics.com
>>>>> > wrote:
>>>>>
>>>>>> Hi Everyone,
>>>>>>
>>>>>> I'm going to start development for Hadoop security through Apache
>>>>>> Knox. Can anyone please suggest me some good API for Knox.
>>>>>>
>>>>>> So far i have found following:
>>>>>>
>>>>>> https://cwiki.apache.org/confluence/display/KNOX/Client+Usage
>>>>>>
>>>>>>
>>>>>> https://knox.apache.org/books/knox-0-5-0/dev-guide.html#Gateway+Services
>>>>>>
>>>>>
>>>>>
>>>>
>>>
>>
>>
>> --
>> Regards: HAFIZ MUJADID
>>
>
>
--
Regards: HAFIZ MUJADID
Re: Apache Knox Web API
Posted by larry mccay <la...@gmail.com>.
As I mentioned, you can dig into the source of the gateway-shell classes -
which are used when scripting with groovy.
Here is a link to an hdfs Get request:
https://github.com/apache/knox/blob/master/gateway-shell/src/main/java/org/apache/hadoop/gateway/shell/hdfs/Get.java#L32
Going to the HttpClient level is like going to bare metal - it provides you
greatest level of control but you will need to build abstractions around
its use in order to avoid lots of redundant boilerplate code. Which is why
we have provided such classes for the scripting.
You can also look at the DefaultDispatch code as an example - it is a bit
more complicated since it covers more general usecases but you may glean
some insights from it.
Otherwise, google for examples of "Apache HttpClient REST basic
authentication" and see what you find.
Hope this is useful for you!
On Sun, Jul 5, 2015 at 11:40 AM, Hafiz Mujadid <ha...@gmail.com>
wrote:
> Hi Larry!
>
> Can you provide the link to samples using httclient on github etc.?
>
> Thanks
>
> On Sat, Jul 4, 2015 at 9:40 PM, larry mccay <la...@gmail.com> wrote:
>
>> Then you will want to consider the Client library from the first link.
>> You can look in the {GATEWAY_HOME}/samples directory for examples of it's
>> use.
>> The groovy scripts are a great way to do it or you can use the underlying
>> java classes that groovy uses.
>> The latter will require you to dig into the source a bit more to see how
>> to use them.
>>
>> You can also use Apache HttpClient and there are samples of that as well.
>>
>> On Sat, Jul 4, 2015 at 12:04 PM, Aneela Saleem <an...@platalytics.com>
>> wrote:
>>
>>> Thanks Larry.
>>>
>>> Actually I need some client API like java so that I authenticate /
>>> authorize my users programmatically through Knox.
>>>
>>> On Sat, Jul 4, 2015 at 8:50 PM, larry mccay <la...@gmail.com>
>>> wrote:
>>>
>>>> Hi Aneela -
>>>>
>>>> I assume that you mean that you would like to add support for a Hadoop
>>>> API that Knox currently lacks.
>>>> My suggestion is that you find one that your organization or your
>>>> personal interests require.
>>>>
>>>> There are lots of Jira's filed for bug fixes and other
>>>> features/enhancements as well.
>>>>
>>>> Feel free to start a discussion regarding any contribution that you
>>>> would like to make.
>>>>
>>>> As far as the links that you referenced:
>>>>
>>>> 1. The first is a client library for scripting interactions with Hadoop
>>>> services through Knox - there are some really interesting and powerful
>>>> capabilities there.
>>>> 2. The second is actually pointing to a section the dev guide that
>>>> needs to be completed. We have what we call Gateway Services in the kernel
>>>> of the Knox server that provide implementations for core server interfaces
>>>> - crypto, SSL, credential aliasing, etc. I don't think that you want to
>>>> work in that space. If you want to work on adding new API support for
>>>> services then you should refer to the Services section -
>>>> https://knox.apache.org/books/knox-0-6-0/dev-guide.html#Services.
>>>>
>>>> Note that the link that I provided above is for the 0.6.0 dev guide.
>>>> There is a new configuration driven way to add API support to Knox that was
>>>> added in the 0.6.0 release.
>>>>
>>>> Thanks for your interest in contributing to Apache Knox!
>>>>
>>>> --larry
>>>>
>>>>
>>>> On Sat, Jul 4, 2015 at 10:56 AM, Aneela Saleem <an...@platalytics.com>
>>>> wrote:
>>>>
>>>>> Hi Everyone,
>>>>>
>>>>> I'm going to start development for Hadoop security through Apache
>>>>> Knox. Can anyone please suggest me some good API for Knox.
>>>>>
>>>>> So far i have found following:
>>>>>
>>>>> https://cwiki.apache.org/confluence/display/KNOX/Client+Usage
>>>>>
>>>>>
>>>>> https://knox.apache.org/books/knox-0-5-0/dev-guide.html#Gateway+Services
>>>>>
>>>>
>>>>
>>>
>>
>
>
> --
> Regards: HAFIZ MUJADID
>
Re: Apache Knox Web API
Posted by Hafiz Mujadid <ha...@gmail.com>.
Hi Larry!
Can you provide the link to samples using httclient on github etc.?
Thanks
On Sat, Jul 4, 2015 at 9:40 PM, larry mccay <la...@gmail.com> wrote:
> Then you will want to consider the Client library from the first link.
> You can look in the {GATEWAY_HOME}/samples directory for examples of it's
> use.
> The groovy scripts are a great way to do it or you can use the underlying
> java classes that groovy uses.
> The latter will require you to dig into the source a bit more to see how
> to use them.
>
> You can also use Apache HttpClient and there are samples of that as well.
>
> On Sat, Jul 4, 2015 at 12:04 PM, Aneela Saleem <an...@platalytics.com>
> wrote:
>
>> Thanks Larry.
>>
>> Actually I need some client API like java so that I authenticate /
>> authorize my users programmatically through Knox.
>>
>> On Sat, Jul 4, 2015 at 8:50 PM, larry mccay <la...@gmail.com>
>> wrote:
>>
>>> Hi Aneela -
>>>
>>> I assume that you mean that you would like to add support for a Hadoop
>>> API that Knox currently lacks.
>>> My suggestion is that you find one that your organization or your
>>> personal interests require.
>>>
>>> There are lots of Jira's filed for bug fixes and other
>>> features/enhancements as well.
>>>
>>> Feel free to start a discussion regarding any contribution that you
>>> would like to make.
>>>
>>> As far as the links that you referenced:
>>>
>>> 1. The first is a client library for scripting interactions with Hadoop
>>> services through Knox - there are some really interesting and powerful
>>> capabilities there.
>>> 2. The second is actually pointing to a section the dev guide that needs
>>> to be completed. We have what we call Gateway Services in the kernel of the
>>> Knox server that provide implementations for core server interfaces -
>>> crypto, SSL, credential aliasing, etc. I don't think that you want to work
>>> in that space. If you want to work on adding new API support for services
>>> then you should refer to the Services section -
>>> https://knox.apache.org/books/knox-0-6-0/dev-guide.html#Services.
>>>
>>> Note that the link that I provided above is for the 0.6.0 dev guide.
>>> There is a new configuration driven way to add API support to Knox that was
>>> added in the 0.6.0 release.
>>>
>>> Thanks for your interest in contributing to Apache Knox!
>>>
>>> --larry
>>>
>>>
>>> On Sat, Jul 4, 2015 at 10:56 AM, Aneela Saleem <an...@platalytics.com>
>>> wrote:
>>>
>>>> Hi Everyone,
>>>>
>>>> I'm going to start development for Hadoop security through Apache Knox.
>>>> Can anyone please suggest me some good API for Knox.
>>>>
>>>> So far i have found following:
>>>>
>>>> https://cwiki.apache.org/confluence/display/KNOX/Client+Usage
>>>>
>>>> https://knox.apache.org/books/knox-0-5-0/dev-guide.html#Gateway+Services
>>>>
>>>
>>>
>>
>
--
Regards: HAFIZ MUJADID
Re: Apache Knox Web API
Posted by larry mccay <la...@gmail.com>.
Then you will want to consider the Client library from the first link.
You can look in the {GATEWAY_HOME}/samples directory for examples of it's
use.
The groovy scripts are a great way to do it or you can use the underlying
java classes that groovy uses.
The latter will require you to dig into the source a bit more to see how to
use them.
You can also use Apache HttpClient and there are samples of that as well.
On Sat, Jul 4, 2015 at 12:04 PM, Aneela Saleem <an...@platalytics.com>
wrote:
> Thanks Larry.
>
> Actually I need some client API like java so that I authenticate /
> authorize my users programmatically through Knox.
>
> On Sat, Jul 4, 2015 at 8:50 PM, larry mccay <la...@gmail.com> wrote:
>
>> Hi Aneela -
>>
>> I assume that you mean that you would like to add support for a Hadoop
>> API that Knox currently lacks.
>> My suggestion is that you find one that your organization or your
>> personal interests require.
>>
>> There are lots of Jira's filed for bug fixes and other
>> features/enhancements as well.
>>
>> Feel free to start a discussion regarding any contribution that you would
>> like to make.
>>
>> As far as the links that you referenced:
>>
>> 1. The first is a client library for scripting interactions with Hadoop
>> services through Knox - there are some really interesting and powerful
>> capabilities there.
>> 2. The second is actually pointing to a section the dev guide that needs
>> to be completed. We have what we call Gateway Services in the kernel of the
>> Knox server that provide implementations for core server interfaces -
>> crypto, SSL, credential aliasing, etc. I don't think that you want to work
>> in that space. If you want to work on adding new API support for services
>> then you should refer to the Services section -
>> https://knox.apache.org/books/knox-0-6-0/dev-guide.html#Services.
>>
>> Note that the link that I provided above is for the 0.6.0 dev guide.
>> There is a new configuration driven way to add API support to Knox that was
>> added in the 0.6.0 release.
>>
>> Thanks for your interest in contributing to Apache Knox!
>>
>> --larry
>>
>>
>> On Sat, Jul 4, 2015 at 10:56 AM, Aneela Saleem <an...@platalytics.com>
>> wrote:
>>
>>> Hi Everyone,
>>>
>>> I'm going to start development for Hadoop security through Apache Knox.
>>> Can anyone please suggest me some good API for Knox.
>>>
>>> So far i have found following:
>>>
>>> https://cwiki.apache.org/confluence/display/KNOX/Client+Usage
>>>
>>> https://knox.apache.org/books/knox-0-5-0/dev-guide.html#Gateway+Services
>>>
>>
>>
>
Re: Apache Knox Web API
Posted by Aneela Saleem <an...@platalytics.com>.
Thanks Larry.
Actually I need some client API like java so that I authenticate /
authorize my users programmatically through Knox.
On Sat, Jul 4, 2015 at 8:50 PM, larry mccay <la...@gmail.com> wrote:
> Hi Aneela -
>
> I assume that you mean that you would like to add support for a Hadoop API
> that Knox currently lacks.
> My suggestion is that you find one that your organization or your personal
> interests require.
>
> There are lots of Jira's filed for bug fixes and other
> features/enhancements as well.
>
> Feel free to start a discussion regarding any contribution that you would
> like to make.
>
> As far as the links that you referenced:
>
> 1. The first is a client library for scripting interactions with Hadoop
> services through Knox - there are some really interesting and powerful
> capabilities there.
> 2. The second is actually pointing to a section the dev guide that needs
> to be completed. We have what we call Gateway Services in the kernel of the
> Knox server that provide implementations for core server interfaces -
> crypto, SSL, credential aliasing, etc. I don't think that you want to work
> in that space. If you want to work on adding new API support for services
> then you should refer to the Services section -
> https://knox.apache.org/books/knox-0-6-0/dev-guide.html#Services.
>
> Note that the link that I provided above is for the 0.6.0 dev guide. There
> is a new configuration driven way to add API support to Knox that was added
> in the 0.6.0 release.
>
> Thanks for your interest in contributing to Apache Knox!
>
> --larry
>
>
> On Sat, Jul 4, 2015 at 10:56 AM, Aneela Saleem <an...@platalytics.com>
> wrote:
>
>> Hi Everyone,
>>
>> I'm going to start development for Hadoop security through Apache Knox.
>> Can anyone please suggest me some good API for Knox.
>>
>> So far i have found following:
>>
>> https://cwiki.apache.org/confluence/display/KNOX/Client+Usage
>>
>> https://knox.apache.org/books/knox-0-5-0/dev-guide.html#Gateway+Services
>>
>
>
Re: Apache Knox Web API
Posted by larry mccay <la...@gmail.com>.
Hi Aneela -
I assume that you mean that you would like to add support for a Hadoop API
that Knox currently lacks.
My suggestion is that you find one that your organization or your personal
interests require.
There are lots of Jira's filed for bug fixes and other
features/enhancements as well.
Feel free to start a discussion regarding any contribution that you would
like to make.
As far as the links that you referenced:
1. The first is a client library for scripting interactions with Hadoop
services through Knox - there are some really interesting and powerful
capabilities there.
2. The second is actually pointing to a section the dev guide that needs to
be completed. We have what we call Gateway Services in the kernel of the
Knox server that provide implementations for core server interfaces -
crypto, SSL, credential aliasing, etc. I don't think that you want to work
in that space. If you want to work on adding new API support for services
then you should refer to the Services section -
https://knox.apache.org/books/knox-0-6-0/dev-guide.html#Services.
Note that the link that I provided above is for the 0.6.0 dev guide. There
is a new configuration driven way to add API support to Knox that was added
in the 0.6.0 release.
Thanks for your interest in contributing to Apache Knox!
--larry
On Sat, Jul 4, 2015 at 10:56 AM, Aneela Saleem <an...@platalytics.com>
wrote:
> Hi Everyone,
>
> I'm going to start development for Hadoop security through Apache Knox.
> Can anyone please suggest me some good API for Knox.
>
> So far i have found following:
>
> https://cwiki.apache.org/confluence/display/KNOX/Client+Usage
>
> https://knox.apache.org/books/knox-0-5-0/dev-guide.html#Gateway+Services
>