You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-user@hadoop.apache.org by Carmen Manzulli <ca...@gmail.com> on 2015/06/02 14:26:36 UTC
WebHdfs API
Hi,
I would like to know hot to use WebHDFS methods of RESTful Web Services to
open and read a file from a web application....instead of using curl!
For example, i would know ho to create a client and submit a Get HTTP
request to the namenode.
Thanks in advance,
Carmen.
Re: WebHdfs API
Posted by Larry McCay <lm...@hortonworks.com>.
To build on what Manoj has described, there are few aspects to your usecase and cluster interaction from outside the cluster that you need to be aware of…
* If the cluster is secured - you will need to use SPNEGO - HttpClient does support this too
* if you need to invoke the REST APIs as each authenticated enduser rather than as a single identity representing the application then you will need your application to be configured as a trusted proxy to HDFS
* if you have the cluster firewalled off from the rest of your network then you will need to punch holes in the firewall for the host:ports that you need access to
You can simplify the above by use a reverse proxy like Apache Knox to be an API Gateway for the REST APIs that you need access to.
See http://knox.apache.org/ for more details.
On Jun 2, 2015, at 9:56 AM, Manoj Babu <ma...@gmail.com>> wrote:
Hi - You can invoke the REST service using HTTP request with help of any HTTP clients. Example if it is an Java web application you can use Apache commons HTTP client.
Thanks.
On Tuesday, June 2, 2015, Carmen Manzulli <ca...@gmail.com>> wrote:
Hi,
I would like to know hot to use WebHDFS methods of RESTful Web Services to open and read a file from a web application....instead of using curl!
For example, i would know ho to create a client and submit a Get HTTP request to the namenode.
Thanks in advance,
Carmen.
--
Cheers!
Manoj.
Re: WebHdfs API
Posted by Larry McCay <lm...@hortonworks.com>.
To build on what Manoj has described, there are few aspects to your usecase and cluster interaction from outside the cluster that you need to be aware of…
* If the cluster is secured - you will need to use SPNEGO - HttpClient does support this too
* if you need to invoke the REST APIs as each authenticated enduser rather than as a single identity representing the application then you will need your application to be configured as a trusted proxy to HDFS
* if you have the cluster firewalled off from the rest of your network then you will need to punch holes in the firewall for the host:ports that you need access to
You can simplify the above by use a reverse proxy like Apache Knox to be an API Gateway for the REST APIs that you need access to.
See http://knox.apache.org/ for more details.
On Jun 2, 2015, at 9:56 AM, Manoj Babu <ma...@gmail.com>> wrote:
Hi - You can invoke the REST service using HTTP request with help of any HTTP clients. Example if it is an Java web application you can use Apache commons HTTP client.
Thanks.
On Tuesday, June 2, 2015, Carmen Manzulli <ca...@gmail.com>> wrote:
Hi,
I would like to know hot to use WebHDFS methods of RESTful Web Services to open and read a file from a web application....instead of using curl!
For example, i would know ho to create a client and submit a Get HTTP request to the namenode.
Thanks in advance,
Carmen.
--
Cheers!
Manoj.
Re: WebHdfs API
Posted by Larry McCay <lm...@hortonworks.com>.
To build on what Manoj has described, there are few aspects to your usecase and cluster interaction from outside the cluster that you need to be aware of…
* If the cluster is secured - you will need to use SPNEGO - HttpClient does support this too
* if you need to invoke the REST APIs as each authenticated enduser rather than as a single identity representing the application then you will need your application to be configured as a trusted proxy to HDFS
* if you have the cluster firewalled off from the rest of your network then you will need to punch holes in the firewall for the host:ports that you need access to
You can simplify the above by use a reverse proxy like Apache Knox to be an API Gateway for the REST APIs that you need access to.
See http://knox.apache.org/ for more details.
On Jun 2, 2015, at 9:56 AM, Manoj Babu <ma...@gmail.com>> wrote:
Hi - You can invoke the REST service using HTTP request with help of any HTTP clients. Example if it is an Java web application you can use Apache commons HTTP client.
Thanks.
On Tuesday, June 2, 2015, Carmen Manzulli <ca...@gmail.com>> wrote:
Hi,
I would like to know hot to use WebHDFS methods of RESTful Web Services to open and read a file from a web application....instead of using curl!
For example, i would know ho to create a client and submit a Get HTTP request to the namenode.
Thanks in advance,
Carmen.
--
Cheers!
Manoj.
Re: WebHdfs API
Posted by Larry McCay <lm...@hortonworks.com>.
To build on what Manoj has described, there are few aspects to your usecase and cluster interaction from outside the cluster that you need to be aware of…
* If the cluster is secured - you will need to use SPNEGO - HttpClient does support this too
* if you need to invoke the REST APIs as each authenticated enduser rather than as a single identity representing the application then you will need your application to be configured as a trusted proxy to HDFS
* if you have the cluster firewalled off from the rest of your network then you will need to punch holes in the firewall for the host:ports that you need access to
You can simplify the above by use a reverse proxy like Apache Knox to be an API Gateway for the REST APIs that you need access to.
See http://knox.apache.org/ for more details.
On Jun 2, 2015, at 9:56 AM, Manoj Babu <ma...@gmail.com>> wrote:
Hi - You can invoke the REST service using HTTP request with help of any HTTP clients. Example if it is an Java web application you can use Apache commons HTTP client.
Thanks.
On Tuesday, June 2, 2015, Carmen Manzulli <ca...@gmail.com>> wrote:
Hi,
I would like to know hot to use WebHDFS methods of RESTful Web Services to open and read a file from a web application....instead of using curl!
For example, i would know ho to create a client and submit a Get HTTP request to the namenode.
Thanks in advance,
Carmen.
--
Cheers!
Manoj.
Re: WebHdfs API
Posted by Manoj Babu <ma...@gmail.com>.
Hi - You can invoke the REST service using HTTP request with help of any
HTTP clients. Example if it is an Java web application you can use Apache
commons HTTP client.
Thanks.
On Tuesday, June 2, 2015, Carmen Manzulli <ca...@gmail.com> wrote:
> Hi,
> I would like to know hot to use WebHDFS methods of RESTful Web Services to
> open and read a file from a web application....instead of using curl!
>
> For example, i would know ho to create a client and submit a Get HTTP
> request to the namenode.
>
> Thanks in advance,
>
> Carmen.
>
--
Cheers!
Manoj.
Re: WebHdfs API
Posted by Manoj Babu <ma...@gmail.com>.
Hi - You can invoke the REST service using HTTP request with help of any
HTTP clients. Example if it is an Java web application you can use Apache
commons HTTP client.
Thanks.
On Tuesday, June 2, 2015, Carmen Manzulli <ca...@gmail.com> wrote:
> Hi,
> I would like to know hot to use WebHDFS methods of RESTful Web Services to
> open and read a file from a web application....instead of using curl!
>
> For example, i would know ho to create a client and submit a Get HTTP
> request to the namenode.
>
> Thanks in advance,
>
> Carmen.
>
--
Cheers!
Manoj.
Re: WebHdfs API
Posted by Manoj Babu <ma...@gmail.com>.
Hi - You can invoke the REST service using HTTP request with help of any
HTTP clients. Example if it is an Java web application you can use Apache
commons HTTP client.
Thanks.
On Tuesday, June 2, 2015, Carmen Manzulli <ca...@gmail.com> wrote:
> Hi,
> I would like to know hot to use WebHDFS methods of RESTful Web Services to
> open and read a file from a web application....instead of using curl!
>
> For example, i would know ho to create a client and submit a Get HTTP
> request to the namenode.
>
> Thanks in advance,
>
> Carmen.
>
--
Cheers!
Manoj.
Re: WebHdfs API
Posted by Manoj Babu <ma...@gmail.com>.
Hi - You can invoke the REST service using HTTP request with help of any
HTTP clients. Example if it is an Java web application you can use Apache
commons HTTP client.
Thanks.
On Tuesday, June 2, 2015, Carmen Manzulli <ca...@gmail.com> wrote:
> Hi,
> I would like to know hot to use WebHDFS methods of RESTful Web Services to
> open and read a file from a web application....instead of using curl!
>
> For example, i would know ho to create a client and submit a Get HTTP
> request to the namenode.
>
> Thanks in advance,
>
> Carmen.
>
--
Cheers!
Manoj.