You are viewing a plain text version of this content. The canonical link for it is here.
Posted to mapreduce-user@hadoop.apache.org by Manoj Babu <ma...@gmail.com> on 2014/05/04 14:27:21 UTC
Reg HttpFS
Hi,
How to accesss files in hdfs using HttpFS that is protected by kerberos?
Kerberos authentication works only where is is configured ex: edge node.
If i am triggering request from other system then how do i authenticate?
Kindly advise.
Cheers!
Manoj.
Fwd: Reg HttpFS
Posted by larry mccay <la...@gmail.com>.
Let's try from this email address....
---------- Forwarded message ----------
From: Larry McCay <lm...@hortonworks.com>
Date: Fri, May 16, 2014 at 3:18 PM
Subject: Fwd: Reg HttpFS
To: larry mccay <la...@gmail.com>
---------- Forwarded message ----------
From: Larry McCay <lm...@hortonworks.com>
Date: Fri, May 16, 2014 at 3:06 PM
Subject: Fwd: Reg HttpFS
To: dev@knox.apache.org
This is an email from someone looking for Knox...
---------- Forwarded message ----------
From: Larry McCay <lm...@hortonworks.com>
Date: Fri, May 16, 2014 at 12:53 PM
Subject: Re: Reg HttpFS
To: user@hadoop.apache.org
Hi Manoj -
This is often done by going through a gateway or intermediary that is
configured as a trusted proxy to the cluster. That is, the intermediary can
authenticate to the target services as itself with kerberos and dispatch
the REST request with a doas parameter that indicates the identity of the
user to issue the request on behalf of.
This is precisely what Apache Knox does for such deployments. You may want
to take a look there.
http://knox.apache.org
Currently, out of the box, Knox has an authentication provider to
authentication HTTP Basic credentials against an LDAP server.
There is an ApacheDS LDAP server as part of the Knox distribution as well -
for quickly testing your deployment.
Feel free to engage the Knox user/dev lists.
HTH,
--larry
On Thu, May 15, 2014 at 4:44 AM, Manoj Babu <ma...@gmail.com> wrote:
> Hi Alejandro,
>
> Thanks for your response. Right now i am following this approach from
> edge-node where kerberos is configured. I am not able to understand the hit
> provided can you provide a sample to trigger request from other external
> machine to authenticate where kerberos not configured in client where
> request is to be triggered?
>
> $ kinit
>
> After entering pwd
>
> $ curl --negotiate -u : -b ~/cookiejar.txt -c ~/cookiejar.txt
> http://localhost:14000/webhdfs/v1/?op=liststatus
>
> Cheers!
> Manoj.
>
>
> On Fri, May 9, 2014 at 12:13 PM, Alejandro Abdelnur <tu...@cloudera.com>wrote:
>
>> Manoj,
>>
>> Please look at
>> http://hadoop.apache.org/docs/r2.4.0/hadoop-hdfs-httpfs/httpfs-default.htmllook at the 'httpfs.authentication.*' properties.
>>
>> Thanks.
>>
>>
>> On Sun, May 4, 2014 at 5:27 AM, Manoj Babu <ma...@gmail.com> wrote:
>>
>>> Hi,
>>>
>>> How to accesss files in hdfs using HttpFS that is protected by kerberos?
>>> Kerberos authentication works only where is is configured ex: edge node.
>>> If i am triggering request from other system then how do i authenticate?
>>>
>>> Kindly advise.
>>>
>>> Cheers!
>>> Manoj.
>>>
>>
>>
>>
>> --
>> Alejandro
>>
>
>
CONFIDENTIALITY NOTICE
NOTICE: This message is intended for the use of the individual or entity to
which it is addressed and may contain information that is confidential,
privileged and exempt from disclosure under applicable law. If the reader
of this message is not the intended recipient, you are hereby notified that
any printing, copying, dissemination, distribution, disclosure or
forwarding of this communication is strictly prohibited. If you have
received this communication in error, please contact the sender immediately
and delete it from your system. Thank You.
Re: Reg HttpFS
Posted by Larry McCay <lm...@hortonworks.com>.
Hi Manoj -
This is often done by going through a gateway or intermediary that is
configured as a trusted proxy to the cluster. That is, the intermediary can
authenticate to the target services as itself with kerberos and dispatch
the REST request with a doas parameter that indicates the identity of the
user to issue the request on behalf of.
This is precisely what Apache Knox does for such deployments. You may want
to take a look there.
http://knox.apache.org
Currently, out of the box, Knox has an authentication provider to
authentication HTTP Basic credentials against an LDAP server.
There is an ApacheDS LDAP server as part of the Knox distribution as well -
for quickly testing your deployment.
Feel free to engage the Knox user/dev lists.
HTH,
--larry
On Thu, May 15, 2014 at 4:44 AM, Manoj Babu <ma...@gmail.com> wrote:
> Hi Alejandro,
>
> Thanks for your response. Right now i am following this approach from
> edge-node where kerberos is configured. I am not able to understand the hit
> provided can you provide a sample to trigger request from other external
> machine to authenticate where kerberos not configured in client where
> request is to be triggered?
>
> $ kinit
>
> After entering pwd
>
> $ curl --negotiate -u : -b ~/cookiejar.txt -c ~/cookiejar.txt
> http://localhost:14000/webhdfs/v1/?op=liststatus
>
> Cheers!
> Manoj.
>
>
> On Fri, May 9, 2014 at 12:13 PM, Alejandro Abdelnur <tu...@cloudera.com>wrote:
>
>> Manoj,
>>
>> Please look at
>> http://hadoop.apache.org/docs/r2.4.0/hadoop-hdfs-httpfs/httpfs-default.htmllook at the 'httpfs.authentication.*' properties.
>>
>> Thanks.
>>
>>
>> On Sun, May 4, 2014 at 5:27 AM, Manoj Babu <ma...@gmail.com> wrote:
>>
>>> Hi,
>>>
>>> How to accesss files in hdfs using HttpFS that is protected by kerberos?
>>> Kerberos authentication works only where is is configured ex: edge node.
>>> If i am triggering request from other system then how do i authenticate?
>>>
>>> Kindly advise.
>>>
>>> Cheers!
>>> Manoj.
>>>
>>
>>
>>
>> --
>> Alejandro
>>
>
>
--
CONFIDENTIALITY NOTICE
NOTICE: This message is intended for the use of the individual or entity to
which it is addressed and may contain information that is confidential,
privileged and exempt from disclosure under applicable law. If the reader
of this message is not the intended recipient, you are hereby notified that
any printing, copying, dissemination, distribution, disclosure or
forwarding of this communication is strictly prohibited. If you have
received this communication in error, please contact the sender immediately
and delete it from your system. Thank You.
Re: Reg HttpFS
Posted by Larry McCay <lm...@hortonworks.com>.
Hi Manoj -
This is often done by going through a gateway or intermediary that is
configured as a trusted proxy to the cluster. That is, the intermediary can
authenticate to the target services as itself with kerberos and dispatch
the REST request with a doas parameter that indicates the identity of the
user to issue the request on behalf of.
This is precisely what Apache Knox does for such deployments. You may want
to take a look there.
http://knox.apache.org
Currently, out of the box, Knox has an authentication provider to
authentication HTTP Basic credentials against an LDAP server.
There is an ApacheDS LDAP server as part of the Knox distribution as well -
for quickly testing your deployment.
Feel free to engage the Knox user/dev lists.
HTH,
--larry
On Thu, May 15, 2014 at 4:44 AM, Manoj Babu <ma...@gmail.com> wrote:
> Hi Alejandro,
>
> Thanks for your response. Right now i am following this approach from
> edge-node where kerberos is configured. I am not able to understand the hit
> provided can you provide a sample to trigger request from other external
> machine to authenticate where kerberos not configured in client where
> request is to be triggered?
>
> $ kinit
>
> After entering pwd
>
> $ curl --negotiate -u : -b ~/cookiejar.txt -c ~/cookiejar.txt
> http://localhost:14000/webhdfs/v1/?op=liststatus
>
> Cheers!
> Manoj.
>
>
> On Fri, May 9, 2014 at 12:13 PM, Alejandro Abdelnur <tu...@cloudera.com>wrote:
>
>> Manoj,
>>
>> Please look at
>> http://hadoop.apache.org/docs/r2.4.0/hadoop-hdfs-httpfs/httpfs-default.htmllook at the 'httpfs.authentication.*' properties.
>>
>> Thanks.
>>
>>
>> On Sun, May 4, 2014 at 5:27 AM, Manoj Babu <ma...@gmail.com> wrote:
>>
>>> Hi,
>>>
>>> How to accesss files in hdfs using HttpFS that is protected by kerberos?
>>> Kerberos authentication works only where is is configured ex: edge node.
>>> If i am triggering request from other system then how do i authenticate?
>>>
>>> Kindly advise.
>>>
>>> Cheers!
>>> Manoj.
>>>
>>
>>
>>
>> --
>> Alejandro
>>
>
>
--
CONFIDENTIALITY NOTICE
NOTICE: This message is intended for the use of the individual or entity to
which it is addressed and may contain information that is confidential,
privileged and exempt from disclosure under applicable law. If the reader
of this message is not the intended recipient, you are hereby notified that
any printing, copying, dissemination, distribution, disclosure or
forwarding of this communication is strictly prohibited. If you have
received this communication in error, please contact the sender immediately
and delete it from your system. Thank You.
Re: Reg HttpFS
Posted by Larry McCay <lm...@hortonworks.com>.
Hi Manoj -
This is often done by going through a gateway or intermediary that is
configured as a trusted proxy to the cluster. That is, the intermediary can
authenticate to the target services as itself with kerberos and dispatch
the REST request with a doas parameter that indicates the identity of the
user to issue the request on behalf of.
This is precisely what Apache Knox does for such deployments. You may want
to take a look there.
http://knox.apache.org
Currently, out of the box, Knox has an authentication provider to
authentication HTTP Basic credentials against an LDAP server.
There is an ApacheDS LDAP server as part of the Knox distribution as well -
for quickly testing your deployment.
Feel free to engage the Knox user/dev lists.
HTH,
--larry
On Thu, May 15, 2014 at 4:44 AM, Manoj Babu <ma...@gmail.com> wrote:
> Hi Alejandro,
>
> Thanks for your response. Right now i am following this approach from
> edge-node where kerberos is configured. I am not able to understand the hit
> provided can you provide a sample to trigger request from other external
> machine to authenticate where kerberos not configured in client where
> request is to be triggered?
>
> $ kinit
>
> After entering pwd
>
> $ curl --negotiate -u : -b ~/cookiejar.txt -c ~/cookiejar.txt
> http://localhost:14000/webhdfs/v1/?op=liststatus
>
> Cheers!
> Manoj.
>
>
> On Fri, May 9, 2014 at 12:13 PM, Alejandro Abdelnur <tu...@cloudera.com>wrote:
>
>> Manoj,
>>
>> Please look at
>> http://hadoop.apache.org/docs/r2.4.0/hadoop-hdfs-httpfs/httpfs-default.htmllook at the 'httpfs.authentication.*' properties.
>>
>> Thanks.
>>
>>
>> On Sun, May 4, 2014 at 5:27 AM, Manoj Babu <ma...@gmail.com> wrote:
>>
>>> Hi,
>>>
>>> How to accesss files in hdfs using HttpFS that is protected by kerberos?
>>> Kerberos authentication works only where is is configured ex: edge node.
>>> If i am triggering request from other system then how do i authenticate?
>>>
>>> Kindly advise.
>>>
>>> Cheers!
>>> Manoj.
>>>
>>
>>
>>
>> --
>> Alejandro
>>
>
>
--
CONFIDENTIALITY NOTICE
NOTICE: This message is intended for the use of the individual or entity to
which it is addressed and may contain information that is confidential,
privileged and exempt from disclosure under applicable law. If the reader
of this message is not the intended recipient, you are hereby notified that
any printing, copying, dissemination, distribution, disclosure or
forwarding of this communication is strictly prohibited. If you have
received this communication in error, please contact the sender immediately
and delete it from your system. Thank You.
Re: Reg HttpFS
Posted by Larry McCay <lm...@hortonworks.com>.
Hi Manoj -
This is often done by going through a gateway or intermediary that is
configured as a trusted proxy to the cluster. That is, the intermediary can
authenticate to the target services as itself with kerberos and dispatch
the REST request with a doas parameter that indicates the identity of the
user to issue the request on behalf of.
This is precisely what Apache Knox does for such deployments. You may want
to take a look there.
http://knox.apache.org
Currently, out of the box, Knox has an authentication provider to
authentication HTTP Basic credentials against an LDAP server.
There is an ApacheDS LDAP server as part of the Knox distribution as well -
for quickly testing your deployment.
Feel free to engage the Knox user/dev lists.
HTH,
--larry
On Thu, May 15, 2014 at 4:44 AM, Manoj Babu <ma...@gmail.com> wrote:
> Hi Alejandro,
>
> Thanks for your response. Right now i am following this approach from
> edge-node where kerberos is configured. I am not able to understand the hit
> provided can you provide a sample to trigger request from other external
> machine to authenticate where kerberos not configured in client where
> request is to be triggered?
>
> $ kinit
>
> After entering pwd
>
> $ curl --negotiate -u : -b ~/cookiejar.txt -c ~/cookiejar.txt
> http://localhost:14000/webhdfs/v1/?op=liststatus
>
> Cheers!
> Manoj.
>
>
> On Fri, May 9, 2014 at 12:13 PM, Alejandro Abdelnur <tu...@cloudera.com>wrote:
>
>> Manoj,
>>
>> Please look at
>> http://hadoop.apache.org/docs/r2.4.0/hadoop-hdfs-httpfs/httpfs-default.htmllook at the 'httpfs.authentication.*' properties.
>>
>> Thanks.
>>
>>
>> On Sun, May 4, 2014 at 5:27 AM, Manoj Babu <ma...@gmail.com> wrote:
>>
>>> Hi,
>>>
>>> How to accesss files in hdfs using HttpFS that is protected by kerberos?
>>> Kerberos authentication works only where is is configured ex: edge node.
>>> If i am triggering request from other system then how do i authenticate?
>>>
>>> Kindly advise.
>>>
>>> Cheers!
>>> Manoj.
>>>
>>
>>
>>
>> --
>> Alejandro
>>
>
>
--
CONFIDENTIALITY NOTICE
NOTICE: This message is intended for the use of the individual or entity to
which it is addressed and may contain information that is confidential,
privileged and exempt from disclosure under applicable law. If the reader
of this message is not the intended recipient, you are hereby notified that
any printing, copying, dissemination, distribution, disclosure or
forwarding of this communication is strictly prohibited. If you have
received this communication in error, please contact the sender immediately
and delete it from your system. Thank You.
Re: Reg HttpFS
Posted by Manoj Babu <ma...@gmail.com>.
Hi Alejandro,
Thanks for your response. Right now i am following this approach from
edge-node where kerberos is configured. I am not able to understand the hit
provided can you provide a sample to trigger request from other external
machine to authenticate where kerberos not configured in client where
request is to be triggered?
$ kinit
After entering pwd
$ curl --negotiate -u : -b ~/cookiejar.txt -c ~/cookiejar.txt
http://localhost:14000/webhdfs/v1/?op=liststatus
Cheers!
Manoj.
On Fri, May 9, 2014 at 12:13 PM, Alejandro Abdelnur <tu...@cloudera.com>wrote:
> Manoj,
>
> Please look at
> http://hadoop.apache.org/docs/r2.4.0/hadoop-hdfs-httpfs/httpfs-default.htmllook at the 'httpfs.authentication.*' properties.
>
> Thanks.
>
>
> On Sun, May 4, 2014 at 5:27 AM, Manoj Babu <ma...@gmail.com> wrote:
>
>> Hi,
>>
>> How to accesss files in hdfs using HttpFS that is protected by kerberos?
>> Kerberos authentication works only where is is configured ex: edge node.
>> If i am triggering request from other system then how do i authenticate?
>>
>> Kindly advise.
>>
>> Cheers!
>> Manoj.
>>
>
>
>
> --
> Alejandro
>
Re: Reg HttpFS
Posted by Manoj Babu <ma...@gmail.com>.
Hi Alejandro,
Thanks for your response. Right now i am following this approach from
edge-node where kerberos is configured. I am not able to understand the hit
provided can you provide a sample to trigger request from other external
machine to authenticate where kerberos not configured in client where
request is to be triggered?
$ kinit
After entering pwd
$ curl --negotiate -u : -b ~/cookiejar.txt -c ~/cookiejar.txt
http://localhost:14000/webhdfs/v1/?op=liststatus
Cheers!
Manoj.
On Fri, May 9, 2014 at 12:13 PM, Alejandro Abdelnur <tu...@cloudera.com>wrote:
> Manoj,
>
> Please look at
> http://hadoop.apache.org/docs/r2.4.0/hadoop-hdfs-httpfs/httpfs-default.htmllook at the 'httpfs.authentication.*' properties.
>
> Thanks.
>
>
> On Sun, May 4, 2014 at 5:27 AM, Manoj Babu <ma...@gmail.com> wrote:
>
>> Hi,
>>
>> How to accesss files in hdfs using HttpFS that is protected by kerberos?
>> Kerberos authentication works only where is is configured ex: edge node.
>> If i am triggering request from other system then how do i authenticate?
>>
>> Kindly advise.
>>
>> Cheers!
>> Manoj.
>>
>
>
>
> --
> Alejandro
>
Re: Reg HttpFS
Posted by Manoj Babu <ma...@gmail.com>.
Hi Alejandro,
Thanks for your response. Right now i am following this approach from
edge-node where kerberos is configured. I am not able to understand the hit
provided can you provide a sample to trigger request from other external
machine to authenticate where kerberos not configured in client where
request is to be triggered?
$ kinit
After entering pwd
$ curl --negotiate -u : -b ~/cookiejar.txt -c ~/cookiejar.txt
http://localhost:14000/webhdfs/v1/?op=liststatus
Cheers!
Manoj.
On Fri, May 9, 2014 at 12:13 PM, Alejandro Abdelnur <tu...@cloudera.com>wrote:
> Manoj,
>
> Please look at
> http://hadoop.apache.org/docs/r2.4.0/hadoop-hdfs-httpfs/httpfs-default.htmllook at the 'httpfs.authentication.*' properties.
>
> Thanks.
>
>
> On Sun, May 4, 2014 at 5:27 AM, Manoj Babu <ma...@gmail.com> wrote:
>
>> Hi,
>>
>> How to accesss files in hdfs using HttpFS that is protected by kerberos?
>> Kerberos authentication works only where is is configured ex: edge node.
>> If i am triggering request from other system then how do i authenticate?
>>
>> Kindly advise.
>>
>> Cheers!
>> Manoj.
>>
>
>
>
> --
> Alejandro
>
Re: Reg HttpFS
Posted by Manoj Babu <ma...@gmail.com>.
Hi Alejandro,
Thanks for your response. Right now i am following this approach from
edge-node where kerberos is configured. I am not able to understand the hit
provided can you provide a sample to trigger request from other external
machine to authenticate where kerberos not configured in client where
request is to be triggered?
$ kinit
After entering pwd
$ curl --negotiate -u : -b ~/cookiejar.txt -c ~/cookiejar.txt
http://localhost:14000/webhdfs/v1/?op=liststatus
Cheers!
Manoj.
On Fri, May 9, 2014 at 12:13 PM, Alejandro Abdelnur <tu...@cloudera.com>wrote:
> Manoj,
>
> Please look at
> http://hadoop.apache.org/docs/r2.4.0/hadoop-hdfs-httpfs/httpfs-default.htmllook at the 'httpfs.authentication.*' properties.
>
> Thanks.
>
>
> On Sun, May 4, 2014 at 5:27 AM, Manoj Babu <ma...@gmail.com> wrote:
>
>> Hi,
>>
>> How to accesss files in hdfs using HttpFS that is protected by kerberos?
>> Kerberos authentication works only where is is configured ex: edge node.
>> If i am triggering request from other system then how do i authenticate?
>>
>> Kindly advise.
>>
>> Cheers!
>> Manoj.
>>
>
>
>
> --
> Alejandro
>
Re: Reg HttpFS
Posted by Alejandro Abdelnur <tu...@cloudera.com>.
Manoj,
Please look at
http://hadoop.apache.org/docs/r2.4.0/hadoop-hdfs-httpfs/httpfs-default.htmllook
at the 'httpfs.authentication.*' properties.
Thanks.
On Sun, May 4, 2014 at 5:27 AM, Manoj Babu <ma...@gmail.com> wrote:
> Hi,
>
> How to accesss files in hdfs using HttpFS that is protected by kerberos?
> Kerberos authentication works only where is is configured ex: edge node.
> If i am triggering request from other system then how do i authenticate?
>
> Kindly advise.
>
> Cheers!
> Manoj.
>
--
Alejandro
Re: Reg HttpFS
Posted by Alejandro Abdelnur <tu...@cloudera.com>.
Manoj,
Please look at
http://hadoop.apache.org/docs/r2.4.0/hadoop-hdfs-httpfs/httpfs-default.htmllook
at the 'httpfs.authentication.*' properties.
Thanks.
On Sun, May 4, 2014 at 5:27 AM, Manoj Babu <ma...@gmail.com> wrote:
> Hi,
>
> How to accesss files in hdfs using HttpFS that is protected by kerberos?
> Kerberos authentication works only where is is configured ex: edge node.
> If i am triggering request from other system then how do i authenticate?
>
> Kindly advise.
>
> Cheers!
> Manoj.
>
--
Alejandro
Re: Reg HttpFS
Posted by Alejandro Abdelnur <tu...@cloudera.com>.
Manoj,
Please look at
http://hadoop.apache.org/docs/r2.4.0/hadoop-hdfs-httpfs/httpfs-default.htmllook
at the 'httpfs.authentication.*' properties.
Thanks.
On Sun, May 4, 2014 at 5:27 AM, Manoj Babu <ma...@gmail.com> wrote:
> Hi,
>
> How to accesss files in hdfs using HttpFS that is protected by kerberos?
> Kerberos authentication works only where is is configured ex: edge node.
> If i am triggering request from other system then how do i authenticate?
>
> Kindly advise.
>
> Cheers!
> Manoj.
>
--
Alejandro
Re: Reg HttpFS
Posted by Alejandro Abdelnur <tu...@cloudera.com>.
Manoj,
Please look at
http://hadoop.apache.org/docs/r2.4.0/hadoop-hdfs-httpfs/httpfs-default.htmllook
at the 'httpfs.authentication.*' properties.
Thanks.
On Sun, May 4, 2014 at 5:27 AM, Manoj Babu <ma...@gmail.com> wrote:
> Hi,
>
> How to accesss files in hdfs using HttpFS that is protected by kerberos?
> Kerberos authentication works only where is is configured ex: edge node.
> If i am triggering request from other system then how do i authenticate?
>
> Kindly advise.
>
> Cheers!
> Manoj.
>
--
Alejandro